2024-05-15T15:31:10.027 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-15T15:31:10.028 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175 --owner scheduled_gabrioux@teuthology --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --name gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi --verbose -- /home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175/orig.config.yaml 2024-05-15T15:31:10.083 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-15T15:31:10.180 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175 branch: wip-65480-squid description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: gabrioux@ibm.com first_in_suite: false job_id: '7707175' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/reef ' name: gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-65480-squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-65480-squid sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 6602 sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-65480-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD1Jby171EFSq6R2aEagIDF938YcGcCIHBPugkkRAnO3W5v+tFD5wjFlceBGa/wwrAlmbHxq8ovHYAIRFzTKwvE= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGihwTLvMGdJFQVEb+CPSk7GxKcvqtZ1DZVuYytq9C2M8LYigFYPnrK7MeJ+fMNeSI5htt1nt9b4/PNaQp7EmEQ= tasks: - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef roleless: true - print: '**** done end installing reef cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-15_15:01:21 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-05-15T15:31:10.180 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa; will attempt to use it 2024-05-15T15:31:10.181 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks 2024-05-15T15:31:10.181 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-15T15:31:10.183 INFO:teuthology.task.internal:Checking packages... 2024-05-15T15:31:10.210 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'de5210d58560a4684fdb5b238ef3e6e0285a1de8' 2024-05-15T15:31:10.210 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-15T15:31:10.211 INFO:teuthology.packaging:ref: None 2024-05-15T15:31:10.211 INFO:teuthology.packaging:tag: None 2024-05-15T15:31:10.211 INFO:teuthology.packaging:branch: wip-65480-squid 2024-05-15T15:31:10.211 INFO:teuthology.packaging:sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:31:10.211 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-65480-squid 2024-05-15T15:31:10.357 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2576.gde5210d5 2024-05-15T15:31:10.359 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-15T15:31:10.366 INFO:teuthology.task.internal:no buildpackages task found 2024-05-15T15:31:10.366 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-15T15:31:10.379 INFO:teuthology.task.internal:Saving configuration 2024-05-15T15:31:10.400 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-15T15:31:10.406 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-15T15:31:10.431 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 15:25:28.011305', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD1Jby171EFSq6R2aEagIDF938YcGcCIHBPugkkRAnO3W5v+tFD5wjFlceBGa/wwrAlmbHxq8ovHYAIRFzTKwvE='} 2024-05-15T15:31:10.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 15:25:28.009512', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGihwTLvMGdJFQVEb+CPSk7GxKcvqtZ1DZVuYytq9C2M8LYigFYPnrK7MeJ+fMNeSI5htt1nt9b4/PNaQp7EmEQ='} 2024-05-15T15:31:10.458 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-15T15:31:10.463 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-05-15T15:31:10.464 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-05-15T15:31:10.464 INFO:teuthology.run_tasks:Running task console_log... 2024-05-15T15:31:10.535 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7572e480d0>, signals=[15]) 2024-05-15T15:31:10.535 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-15T15:31:10.542 INFO:teuthology.task.internal:Opening connections... 2024-05-15T15:31:10.542 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:31:10.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:31:10.634 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-05-15T15:31:10.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:31:10.713 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-15T15:31:10.719 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-05-15T15:31:10.738 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-05-15T15:31:10.739 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-05-15T15:31:10.797 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-05-15T15:31:10.797 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-05-15T15:31:10.798 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-15T15:31:10.799 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-15T15:31:10.799 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-15T15:31:10.799 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-05-15T15:31:10.823 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-05-15T15:31:10.841 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-05-15T15:31:10.841 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-15T15:31:10.899 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-05-15T15:31:10.900 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-15T15:31:10.900 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-15T15:31:10.900 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-15T15:31:10.900 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-05-15T15:31:10.923 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-15T15:31:10.935 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-15T15:31:10.941 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-15T15:31:10.942 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-05-15T15:31:10.944 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-05-15T15:31:10.960 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-15T15:31:10.966 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-15T15:31:10.967 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-05-15T15:31:11.002 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-05-15T15:31:11.106 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-15T15:31:11.166 INFO:teuthology.run_tasks:Running task kernel... 2024-05-15T15:31:11.181 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-15T15:31:11.182 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-15T15:31:11.182 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-15T15:31:11.182 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-15T15:31:11.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T15:31:11.199 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-05-15T15:31:11.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T15:31:11.200 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-05-15T15:31:11.256 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-437.el9.x86_64 2024-05-15T15:31:11.256 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.14.0-437.el9.x86_64 2024-05-15T15:31:11.256 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2024-05-15T15:31:11.258 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-437.el9.x86_64 2024-05-15T15:31:11.258 INFO:teuthology.task.kernel:Running kernel on smithi049: 5.14.0-437.el9.x86_64 2024-05-15T15:31:11.258 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2024-05-15T15:31:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-15T15:31:14.516 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.1 MB 00:01 2024-05-15T15:31:16.242 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-05-15T15:31:20.542 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.2 MB 00:00 2024-05-15T15:31:20.615 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 19 MB 00:05 2024-05-15T15:31:22.060 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 16 kB 00:00 2024-05-15T15:31:24.776 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-15T15:31:26.218 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 16 kB 00:00 2024-05-15T15:31:28.103 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-15T15:31:33.411 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-05-15T15:31:35.467 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-15T15:31:35.468 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-15T15:31:35.552 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:31:35.566 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:31:35.566 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-05-15T15:31:35.566 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:31:35.566 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:31:35.566 INFO:teuthology.orchestra.run.smithi049.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-15T15:31:35.566 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-15T15:31:35.567 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-15T15:31:35.567 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-15T15:31:35.567 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-15T15:31:35.567 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:31:35.567 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:31:35.567 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:31:35.567 INFO:teuthology.orchestra.run.smithi049.stdout:Install 4 Packages 2024-05-15T15:31:35.567 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:31:35.568 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 78 M 2024-05-15T15:31:35.568 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 123 M 2024-05-15T15:31:35.568 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:31:35.854 INFO:teuthology.orchestra.run.smithi049.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 3.2 MB/s | 550 kB 00:00 2024-05-15T15:31:37.371 INFO:teuthology.orchestra.run.smithi049.stdout:(2/4): kernel-modules-core-5.14.0-446.el9.x86_6 18 MB/s | 28 MB 00:01 2024-05-15T15:31:38.246 INFO:teuthology.orchestra.run.smithi049.stdout:(3/4): kernel-core-5.14.0-446.el9.x86_64.rpm 6.1 MB/s | 15 MB 00:02 2024-05-15T15:31:39.746 INFO:teuthology.orchestra.run.smithi049.stdout:(4/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 8.4 MB/s | 34 MB 00:04 2024-05-15T15:31:39.746 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:31:39.746 INFO:teuthology.orchestra.run.smithi049.stdout:Total 19 MB/s | 78 MB 00:04 2024-05-15T15:31:40.458 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:31:40.865 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:31:40.865 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:31:41.813 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:31:41.814 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:31:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:31:45.078 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-15T15:31:45.163 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-15T15:31:47.956 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-15T15:31:48.086 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-15T15:31:53.164 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-15T15:31:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:31:58.297 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:32:29.144 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:32:29.184 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:32:29.532 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:32:29.533 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-15T15:32:29.533 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-15T15:32:29.533 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-15T15:32:30.357 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:32:30.357 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:32:30.357 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:32:30.357 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-15T15:32:30.357 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-15T15:32:30.357 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-15T15:32:30.357 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-15T15:32:30.357 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:32:30.357 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:32:30.701 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-05-15T15:32:30.780 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-15T15:32:30.780 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-15T15:32:30.780 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-15T15:32:30.781 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-15T15:32:30.781 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-15T15:32:30.781 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-05-15T15:32:30.781 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=None, version=distro) 2024-05-15T15:32:30.781 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-05-15T15:32:30.833 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-15T15:32:30.833 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-15T15:32:30.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2024-05-15T15:32:30.865 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2024-05-15T15:32:30.865 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2024-05-15T15:32:30.865 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.06 2024-05-15T15:32:30.865 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 68.el9 2024-05-15T15:32:30.865 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2024-05-15T15:32:30.865 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Group : Unspecified 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 8264254 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : builder@centos.org 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : CentOS 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2024-05-15T15:32:30.866 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:32:30.867 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-15T15:32:30.867 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-15T15:32:30.867 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-15T15:32:30.867 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2024-05-15T15:32:30.867 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:32:30.867 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2024-05-15T15:32:30.868 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-15T15:32:30.868 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 5.14.0-446.el9.x86_64 2024-05-15T15:32:30.868 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-15T15:32:31.490 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2024-05-15T15:32:32.963 INFO:teuthology.orchestra.run.smithi049.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-15T15:32:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:done 2024-05-15T15:32:32.984 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-05-15T15:32:33.006 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.nK38CKGt9k 2024-05-15T15:32:33.006 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nK38CKGt9k 2024-05-15T15:32:33.072 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.nK38CKGt9k 2024-05-15T15:32:33.216 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.nK38CKGt9k is 6KB 2024-05-15T15:32:33.264 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.nK38CKGt9k 2024-05-15T15:32:33.279 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2024-05-15T15:32:33.341 INFO:teuthology.orchestra.run.smithi049.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-15T15:32:33.341 INFO:teuthology.orchestra.run.smithi049.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-15T15:32:33.341 INFO:teuthology.orchestra.run.smithi049.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-15T15:32:33.342 INFO:teuthology.orchestra.run.smithi049.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-15T15:32:33.343 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-15T15:32:33.493 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2024-05-15T15:33:03.496 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-05-15T15:33:03.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-15T15:33:03.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:33:10.373 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 200 kB/s | 21 MB 01:48 2024-05-15T15:33:15.553 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-15T15:33:17.584 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-15T15:33:17.584 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-15T15:33:17.685 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:33:17.697 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:33:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-05-15T15:33:17.697 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:33:17.697 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:33:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-15T15:33:17.698 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-15T15:33:17.698 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-15T15:33:17.698 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-15T15:33:17.698 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-15T15:33:17.698 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:33:17.698 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:33:17.698 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:33:17.698 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-05-15T15:33:17.698 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:33:17.699 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 78 M 2024-05-15T15:33:17.699 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 123 M 2024-05-15T15:33:17.699 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:33:17.996 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 5.9 MB/s | 550 kB 00:00 2024-05-15T15:33:18.580 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-05-15T15:33:18.930 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 33 MB/s | 34 MB 00:01 2024-05-15T15:33:19.488 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 19 MB/s | 28 MB 00:01 2024-05-15T15:33:19.488 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:33:19.488 INFO:teuthology.orchestra.run.smithi144.stdout:Total 44 MB/s | 78 MB 00:01 2024-05-15T15:33:20.177 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:33:20.506 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:33:20.506 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:33:21.427 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:33:21.427 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:33:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:33:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-15T15:33:24.564 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-15T15:33:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-15T15:33:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-15T15:33:32.098 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-15T15:33:32.221 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:33:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:33:37.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-05-15T15:33:46.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-15T15:33:46.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:33:50.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-05-15T15:34:02.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-15T15:34:02.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:34:06.112 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:34:06.149 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:34:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:34:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-15T15:34:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-15T15:34:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-15T15:34:07.244 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-15T15:34:07.245 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:34:07.245 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:34:07.245 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-15T15:34:07.245 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-15T15:34:07.245 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-15T15:34:07.245 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-15T15:34:07.245 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:34:07.245 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:34:07.609 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-05-15T15:34:07.687 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-15T15:34:07.688 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-15T15:34:07.688 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-15T15:34:07.688 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-15T15:34:07.688 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-15T15:34:07.688 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-05-15T15:34:07.688 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-05-15T15:34:07.688 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-05-15T15:34:07.738 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-15T15:34:07.738 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-15T15:34:07.739 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 68.el9 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8264254 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-15T15:34:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:34:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2024-05-15T15:34:07.771 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-15T15:34:07.771 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 5.14.0-446.el9.x86_64 2024-05-15T15:34:07.772 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-15T15:34:08.454 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-05-15T15:34:09.857 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-15T15:34:09.886 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-05-15T15:34:09.889 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-05-15T15:34:09.903 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.3xUgjF9Y4E 2024-05-15T15:34:09.904 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3xUgjF9Y4E 2024-05-15T15:34:09.980 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.3xUgjF9Y4E 2024-05-15T15:34:10.120 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.3xUgjF9Y4E is 6KB 2024-05-15T15:34:10.168 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.3xUgjF9Y4E 2024-05-15T15:34:10.182 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2024-05-15T15:34:10.243 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-15T15:34:10.243 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-15T15:34:10.244 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-15T15:34:10.244 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-15T15:34:10.245 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-15T15:34:10.409 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-05-15T15:34:40.441 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-05-15T15:34:40.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-15T15:34:40.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:35:02.039 DEBUG:teuthology.orchestra.remote:timed out 2024-05-15T15:35:17.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-15T15:35:17.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:35:17.422 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-05-15T15:35:17.777 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-15T15:35:17.777 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-446.el9.x86_64"... 2024-05-15T15:35:17.777 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-05-15T15:35:17.795 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-446.el9.x86_64 2024-05-15T15:35:17.795 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-15T15:35:17.795 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-15T15:35:17.795 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-15T15:35:18.796 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-05-15T15:35:18.796 DEBUG:teuthology.orchestra.run.smithi049:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-15T15:35:18.867 INFO:teuthology.orchestra.run.smithi049.stdout:ttyS1 2024-05-15T15:35:18.893 DEBUG:teuthology.parallel:result is None 2024-05-15T15:35:40.445 DEBUG:teuthology.orchestra.remote:timed out 2024-05-15T15:35:49.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-15T15:35:49.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:35:50.006 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-05-15T15:35:50.414 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-15T15:35:50.414 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-446.el9.x86_64"... 2024-05-15T15:35:50.415 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-05-15T15:35:50.429 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-446.el9.x86_64 2024-05-15T15:35:50.429 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-15T15:35:50.429 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-15T15:35:50.429 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-15T15:35:51.430 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-05-15T15:35:51.431 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-15T15:35:51.500 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-05-15T15:35:51.535 DEBUG:teuthology.parallel:result is None 2024-05-15T15:35:51.536 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-15T15:35:51.544 INFO:teuthology.task.internal:Creating test directory... 2024-05-15T15:35:51.544 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-15T15:35:51.547 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-15T15:35:51.564 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-15T15:35:51.572 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-15T15:35:51.583 INFO:teuthology.task.internal:Creating archive directory... 2024-05-15T15:35:51.584 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-15T15:35:51.607 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-15T15:35:51.645 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-15T15:35:51.656 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-15T15:35:51.657 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-15T15:35:51.681 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-15T15:35:51.720 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T15:35:51.721 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T15:35:51.731 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T15:35:51.732 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T15:35:51.734 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-15T15:35:51.748 INFO:teuthology.task.internal:Configuring sudo... 2024-05-15T15:35:51.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-15T15:35:51.778 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-15T15:35:51.804 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-15T15:35:51.817 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-15T15:35:51.817 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-15T15:35:51.846 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-15T15:35:51.861 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T15:35:51.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T15:35:52.005 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T15:35:52.047 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T15:35:52.089 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:35:52.089 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-15T15:35:52.156 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T15:35:52.199 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T15:35:52.242 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T15:35:52.322 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T15:35:52.366 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:35:52.366 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-15T15:35:52.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-05-15T15:35:52.440 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-05-15T15:35:52.477 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T15:35:52.523 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T15:35:52.992 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-15T15:35:53.004 INFO:teuthology.task.internal:Starting timer... 2024-05-15T15:35:53.004 INFO:teuthology.run_tasks:Running task pcp... 2024-05-15T15:35:53.022 INFO:teuthology.run_tasks:Running task selinux... 2024-05-15T15:35:53.035 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-15T15:35:53.036 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-05-15T15:35:53.079 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-05-15T15:35:53.080 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-05-15T15:35:53.133 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-05-15T15:35:53.135 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-15T15:35:53.135 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-05-15T15:35:53.175 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-05-15T15:35:53.175 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-05-15T15:35:53.198 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-05-15T15:35:53.199 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2024-05-15T15:35:53.199 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T15:35:53.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T15:35:53.244 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T15:35:53.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T15:35:53.268 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-15T15:35:53.269 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-05-15T15:35:53.312 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-05-15T15:35:53.339 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-15T15:35:53.352 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-15T15:35:53.372 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-15T15:35:53.374 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-15T15:40:18.968 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-05-15T15:40:18.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-15T15:40:18.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:40:19.056 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-05-15T15:40:19.143 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-15T15:40:19.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-15T15:40:19.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T15:40:19.227 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-05-15T15:40:19.305 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-15T15:40:19.305 INFO:teuthology.run_tasks:Running task clock... 2024-05-15T15:40:19.316 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-15T15:40:19.317 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-15T15:40:19.317 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T15:40:19.320 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-15T15:40:19.320 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T15:40:19.350 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-15T15:40:19.365 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-15T15:40:19.380 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-15T15:40:19.391 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-05-15T15:40:19.395 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-15T15:40:19.403 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-05-15T15:40:19.418 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-15T15:40:19.421 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-05-15T15:40:19.432 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-15T15:40:19.432 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-05-15T15:40:19.448 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-15T15:40:19.462 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-15T15:40:19.488 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-05-15T15:40:19.491 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T15:40:19.491 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-05-15T15:40:19.491 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T15:40:19.491 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T15:40:19.491 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T15:40:19.491 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T15:40:19.518 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-05-15T15:40:19.521 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T15:40:19.521 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-05-15T15:40:19.521 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T15:40:19.521 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T15:40:19.521 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T15:40:19.521 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T15:40:19.526 INFO:teuthology.run_tasks:Running task install... 2024-05-15T15:40:19.535 DEBUG:teuthology.task.install:project ceph 2024-05-15T15:40:19.535 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'de5210d58560a4684fdb5b238ef3e6e0285a1de8'}} 2024-05-15T15:40:19.535 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'de5210d58560a4684fdb5b238ef3e6e0285a1de8'} 2024-05-15T15:40:19.535 INFO:teuthology.task.install:Using flavor: default 2024-05-15T15:40:19.542 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-15T15:40:19.542 INFO:teuthology.task.install:extra packages: [] 2024-05-15T15:40:19.543 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de5210d58560a4684fdb5b238ef3e6e0285a1de8', 'tag': None, 'wait_for_package': False} 2024-05-15T15:40:19.543 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-15T15:40:19.543 INFO:teuthology.packaging:ref: None 2024-05-15T15:40:19.543 INFO:teuthology.packaging:tag: None 2024-05-15T15:40:19.544 INFO:teuthology.packaging:branch: reef 2024-05-15T15:40:19.544 INFO:teuthology.packaging:sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:40:19.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-05-15T15:40:19.546 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de5210d58560a4684fdb5b238ef3e6e0285a1de8', 'tag': None, 'wait_for_package': False} 2024-05-15T15:40:19.547 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-15T15:40:19.547 INFO:teuthology.packaging:ref: None 2024-05-15T15:40:19.547 INFO:teuthology.packaging:tag: None 2024-05-15T15:40:19.547 INFO:teuthology.packaging:branch: reef 2024-05-15T15:40:19.547 INFO:teuthology.packaging:sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:40:19.547 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-05-15T15:40:19.883 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/flavors/default/ 2024-05-15T15:40:19.884 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1343.g2965081f 2024-05-15T15:40:19.896 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/flavors/default/ 2024-05-15T15:40:19.897 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1343.g2965081f 2024-05-15T15:40:20.005 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-15T15:40:20.006 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:40:20.006 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-15T15:40:20.032 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-15T15:40:20.033 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-15T15:40:20.033 INFO:teuthology.packaging:ref: None 2024-05-15T15:40:20.033 INFO:teuthology.packaging:tag: None 2024-05-15T15:40:20.033 INFO:teuthology.packaging:branch: reef 2024-05-15T15:40:20.033 INFO:teuthology.packaging:sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:40:20.033 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-15T15:40:20.067 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-15T15:40:20.068 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:40:20.068 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-15T15:40:20.096 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-15T15:40:20.097 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-15T15:40:20.097 INFO:teuthology.packaging:ref: None 2024-05-15T15:40:20.097 INFO:teuthology.packaging:tag: None 2024-05-15T15:40:20.097 INFO:teuthology.packaging:branch: reef 2024-05-15T15:40:20.097 INFO:teuthology.packaging:sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:40:20.097 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-15T15:40:20.098 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-15T15:40:20.166 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-15T15:40:20.173 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-15T15:40:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-05-15T15:40:20.239 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-05-15T15:40:20.246 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-15T15:40:20.314 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-05-15T15:40:20.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-05-15T15:40:20.468 INFO:teuthology.orchestra.run.smithi144.stdout:57 files removed 2024-05-15T15:40:20.496 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw 2024-05-15T15:40:20.566 INFO:teuthology.orchestra.run.smithi049.stdout:57 files removed 2024-05-15T15:40:20.594 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw 2024-05-15T15:40:21.246 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 163 kB/s | 77 kB 00:00 2024-05-15T15:40:21.360 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 162 kB/s | 77 kB 00:00 2024-05-15T15:40:21.694 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 41 kB/s | 11 kB 00:00 2024-05-15T15:40:21.713 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-05-15T15:40:22.061 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-05-15T15:40:22.072 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-05-15T15:40:22.545 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.1 MB 00:00 2024-05-15T15:40:22.955 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.1 MB 00:00 2024-05-15T15:40:24.421 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-15T15:40:25.006 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-05-15T15:40:29.073 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-15T15:40:29.963 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-15T15:40:30.841 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-05-15T15:40:31.430 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-05-15T15:40:31.593 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 34 MB/s | 21 MB 00:00 2024-05-15T15:40:32.760 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-15T15:40:36.801 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-15T15:40:38.049 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-05-15T15:40:39.150 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:40:39.161 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:40:39.161 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:18.2.2-1343.g2965081f.el9 ceph 7.7 M 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 3.3 M 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 3.0 M 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:18.2.2-1343.g2965081f.el9 ceph 5.1 M 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:18.2.2-1343.g2965081f.el9 ceph 18 M 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:18.2.2-1343.g2965081f.el9 ceph 25 k 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-15T15:40:39.162 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 709 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 474 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 4.5 M 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:18.2.2-1343.g2965081f.el9 ceph 45 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:18.2.2-1343.g2965081f.el9 ceph 129 k 2024-05-15T15:40:39.163 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:18.2.2-1343.g2965081f.el9 ceph 162 k 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:18.2.2-1343.g2965081f.el9 ceph 320 k 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:18.2.2-1343.g2965081f.el9 ceph 298 k 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:18.2.2-1343.g2965081f.el9 ceph 99 k 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout:Install 28 Packages 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 2 Packages 2024-05-15T15:40:39.164 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:40:39.165 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 53 M 2024-05-15T15:40:39.165 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:40:40.501 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:40:40.517 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:40:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-15T15:40:40.517 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:40:40.517 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:18.2.2-1343.g2965081f.el9 ceph 7.7 M 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 3.3 M 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 3.0 M 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:18.2.2-1343.g2965081f.el9 ceph 5.1 M 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:18.2.2-1343.g2965081f.el9 ceph 18 M 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:18.2.2-1343.g2965081f.el9 ceph 25 k 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-15T15:40:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 709 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 474 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:18.2.2-1343.g2965081f.el9 ceph 4.5 M 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:18.2.2-1343.g2965081f.el9 ceph 45 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:18.2.2-1343.g2965081f.el9 ceph 129 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:18.2.2-1343.g2965081f.el9 ceph 162 k 2024-05-15T15:40:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:18.2.2-1343.g2965081f.el9 ceph 320 k 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:18.2.2-1343.g2965081f.el9 ceph 298 k 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:18.2.2-1343.g2965081f.el9 ceph 99 k 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout:Install 28 Packages 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 2 Packages 2024-05-15T15:40:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:40:40.521 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 53 M 2024-05-15T15:40:40.521 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:40:40.535 INFO:teuthology.orchestra.run.smithi144.stdout:(1/30): ceph-base-18.2.2-1343.g2965081f.el9.x86 4.5 MB/s | 5.1 MB 00:01 2024-05-15T15:40:40.619 INFO:teuthology.orchestra.run.smithi144.stdout:(2/30): ceph-selinux-18.2.2-1343.g2965081f.el9. 302 kB/s | 25 kB 00:00 2024-05-15T15:40:40.761 INFO:teuthology.orchestra.run.smithi144.stdout:(3/30): libcephfs2-18.2.2-1343.g2965081f.el9.x8 4.9 MB/s | 709 kB 00:00 2024-05-15T15:40:40.886 INFO:teuthology.orchestra.run.smithi144.stdout:(4/30): libradosstriper1-18.2.2-1343.g2965081f. 3.7 MB/s | 474 kB 00:00 2024-05-15T15:40:41.470 INFO:teuthology.orchestra.run.smithi144.stdout:(5/30): librgw2-18.2.2-1343.g2965081f.el9.x86_6 7.6 MB/s | 4.5 MB 00:00 2024-05-15T15:40:41.578 INFO:teuthology.orchestra.run.smithi144.stdout:(6/30): python3-ceph-argparse-18.2.2-1343.g2965 415 kB/s | 45 kB 00:00 2024-05-15T15:40:41.662 INFO:teuthology.orchestra.run.smithi144.stdout:(7/30): python3-ceph-common-18.2.2-1343.g296508 1.5 MB/s | 129 kB 00:00 2024-05-15T15:40:41.717 INFO:teuthology.orchestra.run.smithi049.stdout:(1/30): ceph-base-18.2.2-1343.g2965081f.el9.x86 5.2 MB/s | 5.1 MB 00:00 2024-05-15T15:40:41.754 INFO:teuthology.orchestra.run.smithi144.stdout:(8/30): python3-cephfs-18.2.2-1343.g2965081f.el 1.7 MB/s | 162 kB 00:00 2024-05-15T15:40:41.758 INFO:teuthology.orchestra.run.smithi049.stdout:(2/30): ceph-selinux-18.2.2-1343.g2965081f.el9. 608 kB/s | 25 kB 00:00 2024-05-15T15:40:41.859 INFO:teuthology.orchestra.run.smithi049.stdout:(3/30): libcephfs2-18.2.2-1343.g2965081f.el9.x8 6.9 MB/s | 709 kB 00:00 2024-05-15T15:40:41.943 INFO:teuthology.orchestra.run.smithi049.stdout:(4/30): libradosstriper1-18.2.2-1343.g2965081f. 5.6 MB/s | 474 kB 00:00 2024-05-15T15:40:41.971 INFO:teuthology.orchestra.run.smithi144.stdout:(9/30): ceph-common-18.2.2-1343.g2965081f.el9.x 7.1 MB/s | 18 MB 00:02 2024-05-15T15:40:42.046 INFO:teuthology.orchestra.run.smithi144.stdout:(10/30): python3-rbd-18.2.2-1343.g2965081f.el9. 3.9 MB/s | 298 kB 00:00 2024-05-15T15:40:42.060 INFO:teuthology.orchestra.run.smithi049.stdout:(5/30): ceph-radosgw-18.2.2-1343.g2965081f.el9. 5.9 MB/s | 7.7 MB 00:01 2024-05-15T15:40:42.101 INFO:teuthology.orchestra.run.smithi049.stdout:(6/30): python3-ceph-argparse-18.2.2-1343.g2965 1.1 MB/s | 45 kB 00:00 2024-05-15T15:40:42.144 INFO:teuthology.orchestra.run.smithi049.stdout:(7/30): python3-ceph-common-18.2.2-1343.g296508 3.0 MB/s | 129 kB 00:00 2024-05-15T15:40:42.194 INFO:teuthology.orchestra.run.smithi049.stdout:(8/30): python3-cephfs-18.2.2-1343.g2965081f.el 3.2 MB/s | 162 kB 00:00 2024-05-15T15:40:42.252 INFO:teuthology.orchestra.run.smithi049.stdout:(9/30): python3-rados-18.2.2-1343.g2965081f.el9 5.4 MB/s | 320 kB 00:00 2024-05-15T15:40:42.271 INFO:teuthology.orchestra.run.smithi144.stdout:(11/30): python3-rgw-18.2.2-1343.g2965081f.el9. 442 kB/s | 99 kB 00:00 2024-05-15T15:40:42.305 INFO:teuthology.orchestra.run.smithi144.stdout:(12/30): python3-rados-18.2.2-1343.g2965081f.el 581 kB/s | 320 kB 00:00 2024-05-15T15:40:42.339 INFO:teuthology.orchestra.run.smithi049.stdout:(10/30): librgw2-18.2.2-1343.g2965081f.el9.x86_ 11 MB/s | 4.5 MB 00:00 2024-05-15T15:40:42.362 INFO:teuthology.orchestra.run.smithi049.stdout:(11/30): python3-rbd-18.2.2-1343.g2965081f.el9. 2.7 MB/s | 298 kB 00:00 2024-05-15T15:40:42.387 INFO:teuthology.orchestra.run.smithi049.stdout:(12/30): python3-rgw-18.2.2-1343.g2965081f.el9. 2.1 MB/s | 99 kB 00:00 2024-05-15T15:40:42.480 INFO:teuthology.orchestra.run.smithi144.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 512 kB/s | 107 kB 00:00 2024-05-15T15:40:42.495 INFO:teuthology.orchestra.run.smithi049.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 801 kB/s | 107 kB 00:00 2024-05-15T15:40:42.505 INFO:teuthology.orchestra.run.smithi144.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 799 kB/s | 160 kB 00:00 2024-05-15T15:40:42.521 INFO:teuthology.orchestra.run.smithi049.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-15T15:40:42.531 INFO:teuthology.orchestra.run.smithi144.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-05-15T15:40:42.596 INFO:teuthology.orchestra.run.smithi049.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-05-15T15:40:42.631 INFO:teuthology.orchestra.run.smithi144.stdout:(16/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-05-15T15:40:42.657 INFO:teuthology.orchestra.run.smithi144.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-15T15:40:42.688 INFO:teuthology.orchestra.run.smithi049.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-05-15T15:40:42.740 INFO:teuthology.orchestra.run.smithi144.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 716 kB/s | 42 kB 00:00 2024-05-15T15:40:42.741 INFO:teuthology.orchestra.run.smithi049.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-05-15T15:40:42.764 INFO:teuthology.orchestra.run.smithi049.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 561 kB/s | 42 kB 00:00 2024-05-15T15:40:42.858 INFO:teuthology.orchestra.run.smithi144.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 552 kB/s | 111 kB 00:00 2024-05-15T15:40:42.933 INFO:teuthology.orchestra.run.smithi144.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-05-15T15:40:43.016 INFO:teuthology.orchestra.run.smithi144.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-05-15T15:40:43.066 INFO:teuthology.orchestra.run.smithi144.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 971 kB/s | 49 kB 00:00 2024-05-15T15:40:43.150 INFO:teuthology.orchestra.run.smithi144.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 809 kB/s | 67 kB 00:00 2024-05-15T15:40:43.247 INFO:teuthology.orchestra.run.smithi049.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 218 kB/s | 111 kB 00:00 2024-05-15T15:40:43.381 INFO:teuthology.orchestra.run.smithi049.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 501 kB/s | 309 kB 00:00 2024-05-15T15:40:43.481 INFO:teuthology.orchestra.run.smithi049.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-05-15T15:40:43.581 INFO:teuthology.orchestra.run.smithi049.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-05-15T15:40:43.675 INFO:teuthology.orchestra.run.smithi144.stdout:(24/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.6 MB/s | 839 kB 00:00 2024-05-15T15:40:43.690 INFO:teuthology.orchestra.run.smithi049.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-05-15T15:40:43.799 INFO:teuthology.orchestra.run.smithi049.stdout:(24/30): libarrow-9.0.0-11.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-05-15T15:40:43.851 INFO:teuthology.orchestra.run.smithi144.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-05-15T15:40:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-15T15:40:43.933 INFO:teuthology.orchestra.run.smithi049.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-05-15T15:40:44.033 INFO:teuthology.orchestra.run.smithi049.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-15T15:40:44.267 INFO:teuthology.orchestra.run.smithi049.stdout:(28/30): librados2-18.2.2-1343.g2965081f.el9.x8 9.9 MB/s | 3.3 MB 00:00 2024-05-15T15:40:44.334 INFO:teuthology.orchestra.run.smithi144.stdout:(26/30): libarrow-9.0.0-11.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2024-05-15T15:40:44.409 INFO:teuthology.orchestra.run.smithi049.stdout:(29/30): librbd1-18.2.2-1343.g2965081f.el9.x86_ 8.0 MB/s | 3.0 MB 00:00 2024-05-15T15:40:44.851 INFO:teuthology.orchestra.run.smithi144.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-05-15T15:40:45.176 INFO:teuthology.orchestra.run.smithi144.stdout:(28/30): librados2-18.2.2-1343.g2965081f.el9.x8 3.9 MB/s | 3.3 MB 00:00 2024-05-15T15:40:45.343 INFO:teuthology.orchestra.run.smithi144.stdout:(29/30): librbd1-18.2.2-1343.g2965081f.el9.x86_ 6.1 MB/s | 3.0 MB 00:00 2024-05-15T15:40:48.085 INFO:teuthology.orchestra.run.smithi144.stdout:(30/30): ceph-radosgw-18.2.2-1343.g2965081f.el9 910 kB/s | 7.7 MB 00:08 2024-05-15T15:40:48.088 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:40:48.088 INFO:teuthology.orchestra.run.smithi144.stdout:Total 5.9 MB/s | 53 MB 00:08 2024-05-15T15:40:48.163 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:40:48.198 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:40:48.198 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:40:48.747 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:40:48.747 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:40:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:40:50.164 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-05-15T15:40:50.198 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-05-15T15:40:50.266 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-05-15T15:40:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-05-15T15:40:50.714 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:18.2.2-1343.g2965081f.el9.x86_64 5/32 2024-05-15T15:40:50.813 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:18.2.2-1343.g2965081f.el9.x86_64 5/32 2024-05-15T15:40:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:18.2.2-1343.g2965081f.el9.x86_64 6/32 2024-05-15T15:40:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 7/32 2024-05-15T15:40:50.987 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 7/32 2024-05-15T15:40:51.015 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-05-15T15:40:51.059 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:18.2.2-1343.g2965081f.el9. 9/32 2024-05-15T15:40:51.124 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:18.2.2-1343.g2965081f.el9.x86_64 10/32 2024-05-15T15:40:51.151 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:18.2.2-1343.g2965081f.el9.x86_6 11/32 2024-05-15T15:40:51.218 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1343.g2965081f.el9.x86_6 11/32 2024-05-15T15:40:51.323 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-05-15T15:40:51.351 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-05-15T15:40:51.391 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-05-15T15:40:51.423 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-05-15T15:40:51.451 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-05-15T15:40:51.486 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-05-15T15:40:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-05-15T15:40:51.819 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-05-15T15:40:51.858 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 20/32 2024-05-15T15:40:51.947 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 20/32 2024-05-15T15:40:51.979 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:18.2.2-1343.g2965081f.el9.x86_64 21/32 2024-05-15T15:40:52.015 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-05-15T15:40:52.417 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-05-15T15:40:52.443 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:18.2.2-1343.g2965081f.el9.x86_64 24/32 2024-05-15T15:40:52.506 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:18.2.2-1343.g2965081f.el9.x86_64 24/32 2024-05-15T15:40:52.574 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:18.2.2-1343.g2965081f.el9.x86_64 25/32 2024-05-15T15:40:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:18.2.2-1343.g2965081f.el9.x8 26/32 2024-05-15T15:40:52.700 INFO:teuthology.orchestra.run.smithi049.stdout:(30/30): ceph-common-18.2.2-1343.g2965081f.el9. 1.5 MB/s | 18 MB 00:11 2024-05-15T15:40:52.705 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:40:52.705 INFO:teuthology.orchestra.run.smithi049.stdout:Total 4.3 MB/s | 53 MB 00:12 2024-05-15T15:40:52.779 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:40:52.820 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:40:52.820 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:40:53.407 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:40:53.408 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:40:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 27/32 2024-05-15T15:40:54.827 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 27/32 2024-05-15T15:40:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:40:54.959 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-05-15T15:40:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-05-15T15:40:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-05-15T15:40:55.363 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 27/32 2024-05-15T15:40:55.384 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-05-15T15:40:55.405 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 28/32 2024-05-15T15:40:55.487 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:18.2.2-1343.g2965081f.el9.x86_64 5/32 2024-05-15T15:40:55.559 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 28/32 2024-05-15T15:40:55.559 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-15T15:40:55.559 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-15T15:40:55.560 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:40:55.588 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:18.2.2-1343.g2965081f.el9.x86_64 5/32 2024-05-15T15:40:55.590 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:18.2.2-1343.g2965081f.el9.x86_64 29/32 2024-05-15T15:40:55.675 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:18.2.2-1343.g2965081f.el9.x86_64 6/32 2024-05-15T15:40:55.697 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 7/32 2024-05-15T15:40:55.744 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 7/32 2024-05-15T15:40:55.780 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-05-15T15:40:55.817 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:18.2.2-1343.g2965081f.el9. 9/32 2024-05-15T15:40:55.891 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:18.2.2-1343.g2965081f.el9.x86_64 10/32 2024-05-15T15:40:55.916 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:18.2.2-1343.g2965081f.el9.x86_6 11/32 2024-05-15T15:40:55.982 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1343.g2965081f.el9.x86_6 11/32 2024-05-15T15:40:56.091 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-05-15T15:40:56.126 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-05-15T15:40:56.164 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-05-15T15:40:56.197 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-05-15T15:40:56.224 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-05-15T15:40:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-05-15T15:40:56.293 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-05-15T15:40:56.616 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-05-15T15:40:56.664 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 20/32 2024-05-15T15:40:56.755 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 20/32 2024-05-15T15:40:56.786 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:18.2.2-1343.g2965081f.el9.x86_64 21/32 2024-05-15T15:40:56.821 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-05-15T15:40:57.243 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-05-15T15:40:57.267 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:18.2.2-1343.g2965081f.el9.x86_64 24/32 2024-05-15T15:40:57.338 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:18.2.2-1343.g2965081f.el9.x86_64 24/32 2024-05-15T15:40:57.417 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:18.2.2-1343.g2965081f.el9.x86_64 25/32 2024-05-15T15:40:57.461 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:18.2.2-1343.g2965081f.el9.x8 26/32 2024-05-15T15:40:59.601 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 27/32 2024-05-15T15:40:59.633 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 27/32 2024-05-15T15:41:00.203 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 27/32 2024-05-15T15:41:00.260 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 28/32 2024-05-15T15:41:00.753 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 28/32 2024-05-15T15:41:00.754 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-15T15:41:00.754 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-15T15:41:00.754 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:00.789 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:18.2.2-1343.g2965081f.el9.x86_64 29/32 2024-05-15T15:41:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1343.g2965081f.el9.x86_64 29/32 2024-05-15T15:41:04.960 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-05-15T15:41:04.960 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-05-15T15:41:04.960 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-05-15T15:41:04.960 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-05-15T15:41:04.960 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-05-15T15:41:04.961 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-05-15T15:41:04.961 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-05-15T15:41:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:05.735 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:18.2.2-1343.g2965081f.el9.x86_64 30/32 2024-05-15T15:41:05.761 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1343.g2965081f.el9.x86_64 30/32 2024-05-15T15:41:05.761 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:05.761 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-15T15:41:05.761 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T15:41:05.761 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T15:41:05.761 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:05.763 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-15T15:41:05.846 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-15T15:41:05.846 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-15T15:41:06.972 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-15T15:41:06.972 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 1/32 2024-05-15T15:41:06.972 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 2/32 2024-05-15T15:41:06.972 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:18.2.2-1343.g2965081f.el9.x86_64 3/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:18.2.2-1343.g2965081f.el9.x86_64 4/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 5/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:18.2.2-1343.g2965081f.el9.x86_6 6/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:18.2.2-1343.g2965081f.el9.x86_64 7/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1343.g2965081f.el9. 8/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:18.2.2-1343.g2965081f.el9.x8 9/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:18.2.2-1343.g2965081f.el9.x86_64 10/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:18.2.2-1343.g2965081f.el9.x86_64 11/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:18.2.2-1343.g2965081f.el9.x86_64 12/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:18.2.2-1343.g2965081f.el9.x86_64 13/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-05-15T15:41:06.973 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:18.2.2-1343.g2965081f.el9.x86_64 29/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-05-15T15:41:06.974 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 31/32 2024-05-15T15:41:07.722 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-05-15T15:41:07.722 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:07.722 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-15T15:41:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-15T15:41:07.724 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-15T15:41:07.724 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-15T15:41:07.724 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.724 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-15T15:41:07.724 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-15T15:41:07.724 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-15T15:41:07.724 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:07.725 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-05-15T15:41:07.726 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-15T15:41:07.726 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:07.726 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:41:08.122 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-test 2024-05-15T15:41:08.707 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:32 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:41:08.828 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:41:08.829 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:08.829 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-15T15:41:08.829 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:08.829 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:41:08.829 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:18.2.2-1343.g2965081f.el9 ceph 40 M 2024-05-15T15:41:08.829 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-15T15:41:08.830 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:18.2.2-1343.g2965081f.el9 ceph 166 k 2024-05-15T15:41:08.830 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-15T15:41:08.830 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-15T15:41:08.830 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:08.830 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:41:08.830 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:08.830 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-05-15T15:41:08.830 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:08.831 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 40 M 2024-05-15T15:41:08.831 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 163 M 2024-05-15T15:41:08.831 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:41:09.268 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 814 kB/s | 305 kB 00:00 2024-05-15T15:41:09.335 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-05-15T15:41:09.444 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): libcephsqlite-18.2.2-1343.g2965081f.el9. 302 kB/s | 166 kB 00:00 2024-05-15T15:41:10.508 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1343.g2965081f.el9.x86_64 29/32 2024-05-15T15:41:10.508 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-05-15T15:41:10.508 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-05-15T15:41:10.508 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-05-15T15:41:10.508 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-05-15T15:41:10.508 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-05-15T15:41:10.508 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-05-15T15:41:10.508 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-05-15T15:41:10.508 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:11.283 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:18.2.2-1343.g2965081f.el9.x86_64 30/32 2024-05-15T15:41:11.309 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1343.g2965081f.el9.x86_64 30/32 2024-05-15T15:41:11.309 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:11.309 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-15T15:41:11.310 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T15:41:11.310 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T15:41:11.310 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:11.311 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-15T15:41:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-15T15:41:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-15T15:41:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 1/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 2/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:18.2.2-1343.g2965081f.el9.x86_64 3/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:18.2.2-1343.g2965081f.el9.x86_64 4/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 5/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:18.2.2-1343.g2965081f.el9.x86_6 6/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:18.2.2-1343.g2965081f.el9.x86_64 7/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1343.g2965081f.el9. 8/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:18.2.2-1343.g2965081f.el9.x8 9/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:18.2.2-1343.g2965081f.el9.x86_64 10/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:18.2.2-1343.g2965081f.el9.x86_64 11/32 2024-05-15T15:41:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:18.2.2-1343.g2965081f.el9.x86_64 12/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:18.2.2-1343.g2965081f.el9.x86_64 13/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-05-15T15:41:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-05-15T15:41:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-05-15T15:41:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-05-15T15:41:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-05-15T15:41:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-05-15T15:41:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-05-15T15:41:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:18.2.2-1343.g2965081f.el9.x86_64 29/32 2024-05-15T15:41:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-05-15T15:41:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 31/32 2024-05-15T15:41:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-15T15:41:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-15T15:41:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-3.el9.x86_64 2024-05-15T15:41:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-15T15:41:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:13.231 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:41:13.569 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-test 2024-05-15T15:41:14.168 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:36 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:41:14.294 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:41:14.294 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:18.2.2-1343.g2965081f.el9 ceph 40 M 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:18.2.2-1343.g2965081f.el9 ceph 166 k 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:14.295 INFO:teuthology.orchestra.run.smithi049.stdout:Install 4 Packages 2024-05-15T15:41:14.296 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:14.296 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 40 M 2024-05-15T15:41:14.296 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 163 M 2024-05-15T15:41:14.296 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:41:14.842 INFO:teuthology.orchestra.run.smithi049.stdout:(1/4): libcephsqlite-18.2.2-1343.g2965081f.el9. 724 kB/s | 166 kB 00:00 2024-05-15T15:41:14.942 INFO:teuthology.orchestra.run.smithi049.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 926 kB/s | 305 kB 00:00 2024-05-15T15:41:15.001 INFO:teuthology.orchestra.run.smithi049.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 402 kB/s | 64 kB 00:00 2024-05-15T15:41:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:(4/4): ceph-test-18.2.2-1343.g2965081f.el9.x86_ 16 MB/s | 40 MB 00:02 2024-05-15T15:41:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:Total 14 MB/s | 40 MB 00:02 2024-05-15T15:41:17.156 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:41:17.179 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:41:17.179 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:41:17.512 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:41:17.512 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:41:18.008 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:41:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-15T15:41:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-15T15:41:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:18.2.2-1343.g2965081f.el9.x86_64 3/4 2024-05-15T15:41:22.103 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1343.g2965081f.el9.x86_64 3/4 2024-05-15T15:41:22.179 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:18.2.2-1343.g2965081f.el9.x86_64 4/4 2024-05-15T15:41:24.179 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-test-2:18.2.2-1343.g2965081f.el9.x86_64 4/4 2024-05-15T15:41:24.179 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:18.2.2-1343.g2965081f.el9.x86_64 1/4 2024-05-15T15:41:24.179 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:18.2.2-1343.g2965081f.el9.x86_64 2/4 2024-05-15T15:41:24.179 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-15T15:41:24.443 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-15T15:41:24.443 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:24.443 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:41:24.443 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:24.443 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:24.443 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-15T15:41:24.443 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-15T15:41:24.443 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:24.443 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:41:24.745 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph 2024-05-15T15:41:25.343 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:47 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:41:25.473 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:41:25.475 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:18.2.2-1343.g2965081f.el9 ceph 6.5 k 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:18.2.2-1343.g2965081f.el9 ceph 2.1 M 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:18.2.2-1343.g2965081f.el9 ceph 1.5 M 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 246 k 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:18.2.2-1343.g2965081f.el9 ceph 4.7 M 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:18.2.2-1343.g2965081f.el9 ceph 17 M 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-15T15:41:25.476 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-15T15:41:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-15T15:41:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-15T15:41:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-15T15:41:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-15T15:41:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:25.479 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:41:25.479 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:25.479 INFO:teuthology.orchestra.run.smithi049.stdout:Install 33 Packages 2024-05-15T15:41:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:25.480 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 29 M 2024-05-15T15:41:25.480 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 104 M 2024-05-15T15:41:25.480 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:41:26.067 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): ceph-test-18.2.2-1343.g2965081f.el9.x86_ 2.3 MB/s | 40 MB 00:17 2024-05-15T15:41:26.069 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:26.069 INFO:teuthology.orchestra.run.smithi144.stdout:Total 2.3 MB/s | 40 MB 00:17 2024-05-15T15:41:26.095 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:41:26.114 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:41:26.114 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:41:26.244 INFO:teuthology.orchestra.run.smithi049.stdout:(1/33): ceph-18.2.2-1343.g2965081f.el9.x86_64.r 37 kB/s | 6.5 kB 00:00 2024-05-15T15:41:26.428 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:41:26.429 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:41:26.478 INFO:teuthology.orchestra.run.smithi049.stdout:(2/33): ceph-mgr-18.2.2-1343.g2965081f.el9.x86_ 3.5 MB/s | 1.5 MB 00:00 2024-05-15T15:41:26.528 INFO:teuthology.orchestra.run.smithi049.stdout:(3/33): ceph-mds-18.2.2-1343.g2965081f.el9.x86_ 4.5 MB/s | 2.1 MB 00:00 2024-05-15T15:41:26.629 INFO:teuthology.orchestra.run.smithi049.stdout:(4/33): ceph-mgr-modules-core-18.2.2-1343.g2965 2.4 MB/s | 246 kB 00:00 2024-05-15T15:41:26.737 INFO:teuthology.orchestra.run.smithi049.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-05-15T15:41:26.896 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:41:26.982 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-15T15:41:27.033 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-15T15:41:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:18.2.2-1343.g2965081f.el9.x86_64 3/4 2024-05-15T15:41:27.096 INFO:teuthology.orchestra.run.smithi049.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 3.4 MB/s | 1.2 MB 00:00 2024-05-15T15:41:27.179 INFO:teuthology.orchestra.run.smithi049.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-05-15T15:41:27.396 INFO:teuthology.orchestra.run.smithi049.stdout:(8/33): ceph-osd-18.2.2-1343.g2965081f.el9.x86_ 18 MB/s | 17 MB 00:00 2024-05-15T15:41:27.421 INFO:teuthology.orchestra.run.smithi049.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 559 kB/s | 135 kB 00:00 2024-05-15T15:41:27.555 INFO:teuthology.orchestra.run.smithi049.stdout:(10/33): ceph-mon-18.2.2-1343.g2965081f.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-05-15T15:41:27.580 INFO:teuthology.orchestra.run.smithi049.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 680 kB/s | 125 kB 00:00 2024-05-15T15:41:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 756 kB/s | 215 kB 00:00 2024-05-15T15:41:27.781 INFO:teuthology.orchestra.run.smithi049.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 173 kB/s | 35 kB 00:00 2024-05-15T15:41:27.856 INFO:teuthology.orchestra.run.smithi049.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 556 kB/s | 42 kB 00:00 2024-05-15T15:41:27.923 INFO:teuthology.orchestra.run.smithi049.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 468 kB/s | 172 kB 00:00 2024-05-15T15:41:27.965 INFO:teuthology.orchestra.run.smithi049.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 197 kB/s | 51 kB 00:00 2024-05-15T15:41:28.207 INFO:teuthology.orchestra.run.smithi049.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 124 kB/s | 43 kB 00:00 2024-05-15T15:41:28.282 INFO:teuthology.orchestra.run.smithi049.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 143 kB/s | 11 kB 00:00 2024-05-15T15:41:28.332 INFO:teuthology.orchestra.run.smithi049.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 451 kB/s | 172 kB 00:00 2024-05-15T15:41:28.358 INFO:teuthology.orchestra.run.smithi049.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 234 kB/s | 18 kB 00:00 2024-05-15T15:41:28.517 INFO:teuthology.orchestra.run.smithi049.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 126 kB/s | 23 kB 00:00 2024-05-15T15:41:28.542 INFO:teuthology.orchestra.run.smithi049.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 106 kB/s | 19 kB 00:00 2024-05-15T15:41:28.567 INFO:teuthology.orchestra.run.smithi049.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 596 kB/s | 358 kB 00:00 2024-05-15T15:41:28.593 INFO:teuthology.orchestra.run.smithi049.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-05-15T15:41:28.651 INFO:teuthology.orchestra.run.smithi049.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 944 kB/s | 79 kB 00:00 2024-05-15T15:41:28.676 INFO:teuthology.orchestra.run.smithi049.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 345 kB/s | 46 kB 00:00 2024-05-15T15:41:28.727 INFO:teuthology.orchestra.run.smithi049.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-05-15T15:41:28.752 INFO:teuthology.orchestra.run.smithi049.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 164 kB/s | 16 kB 00:00 2024-05-15T15:41:28.777 INFO:teuthology.orchestra.run.smithi049.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 897 kB/s | 90 kB 00:00 2024-05-15T15:41:28.802 INFO:teuthology.orchestra.run.smithi049.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-05-15T15:41:28.869 INFO:teuthology.orchestra.run.smithi049.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 300 kB/s | 20 kB 00:00 2024-05-15T15:41:28.895 INFO:teuthology.orchestra.run.smithi049.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-05-15T15:41:28.936 INFO:teuthology.orchestra.run.smithi049.stdout:(33/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-05-15T15:41:28.941 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:28.941 INFO:teuthology.orchestra.run.smithi049.stdout:Total 8.4 MB/s | 29 MB 00:03 2024-05-15T15:41:29.069 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:41:29.110 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:41:29.110 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:41:29.500 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:41:29.501 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:41:30.675 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:41:30.748 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-15T15:41:30.780 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-15T15:41:30.797 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1343.g2965081f.el9.x86_64 3/4 2024-05-15T15:41:30.813 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-15T15:41:30.875 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:18.2.2-1343.g2965081f.el9.x86_64 4/4 2024-05-15T15:41:30.910 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-15T15:41:30.995 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-15T15:41:31.042 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-15T15:41:31.076 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-15T15:41:31.114 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-15T15:41:31.149 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-15T15:41:31.182 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-15T15:41:31.223 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-15T15:41:31.267 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-15T15:41:31.300 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-15T15:41:31.339 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-15T15:41:31.416 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-15T15:41:31.577 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-15T15:41:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-15T15:41:31.727 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-15T15:41:31.765 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-15T15:41:31.823 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-15T15:41:31.894 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-15T15:41:32.092 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-15T15:41:32.135 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-15T15:41:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-15T15:41:32.306 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-15T15:41:32.350 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-15T15:41:32.472 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-15T15:41:32.632 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1343.g2965081f.el9. 28/33 2024-05-15T15:41:32.661 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 29/33 2024-05-15T15:41:32.687 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 29/33 2024-05-15T15:41:32.688 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:32.688 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-15T15:41:32.688 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T15:41:32.688 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T15:41:32.688 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:33.018 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-test-2:18.2.2-1343.g2965081f.el9.x86_64 4/4 2024-05-15T15:41:33.018 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:18.2.2-1343.g2965081f.el9.x86_64 1/4 2024-05-15T15:41:33.018 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:18.2.2-1343.g2965081f.el9.x86_64 2/4 2024-05-15T15:41:33.018 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-15T15:41:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-15T15:41:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:33.286 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:41:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-15T15:41:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-15T15:41:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:33.286 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:41:33.591 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph 2024-05-15T15:41:34.169 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:18.2.2-1343.g2965081f.el9.x86_64 30/33 2024-05-15T15:41:34.177 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:58 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:41:34.199 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:18.2.2-1343.g2965081f.el9.x86_64 30/33 2024-05-15T15:41:34.199 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:34.199 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-15T15:41:34.199 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T15:41:34.199 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T15:41:34.199 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:34.303 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:41:34.305 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:34.305 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-15T15:41:34.305 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:34.305 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:41:34.305 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:18.2.2-1343.g2965081f.el9 ceph 6.5 k 2024-05-15T15:41:34.305 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-15T15:41:34.305 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:18.2.2-1343.g2965081f.el9 ceph 2.1 M 2024-05-15T15:41:34.305 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:18.2.2-1343.g2965081f.el9 ceph 1.5 M 2024-05-15T15:41:34.305 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 246 k 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:18.2.2-1343.g2965081f.el9 ceph 4.7 M 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:18.2.2-1343.g2965081f.el9 ceph 17 M 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-15T15:41:34.306 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-15T15:41:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-15T15:41:34.308 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-15T15:41:34.308 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-15T15:41:34.308 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:34.308 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:41:34.308 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:34.308 INFO:teuthology.orchestra.run.smithi144.stdout:Install 33 Packages 2024-05-15T15:41:34.308 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:34.308 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 29 M 2024-05-15T15:41:34.309 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 104 M 2024-05-15T15:41:34.309 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:41:34.662 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:18.2.2-1343.g2965081f.el9.x86_64 31/33 2024-05-15T15:41:34.688 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:18.2.2-1343.g2965081f.el9.x86_64 31/33 2024-05-15T15:41:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-15T15:41:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T15:41:34.688 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T15:41:34.688 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:34.904 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:18.2.2-1343.g2965081f.el9.x86_64 32/33 2024-05-15T15:41:34.930 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:18.2.2-1343.g2965081f.el9.x86_64 32/33 2024-05-15T15:41:34.930 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:34.930 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-15T15:41:34.930 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T15:41:34.931 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T15:41:34.931 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:35.004 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:18.2.2-1343.g2965081f.el9.x86_64 33/33 2024-05-15T15:41:35.354 INFO:teuthology.orchestra.run.smithi144.stdout:(1/33): ceph-18.2.2-1343.g2965081f.el9.x86_64.r 21 kB/s | 6.5 kB 00:00 2024-05-15T15:41:35.821 INFO:teuthology.orchestra.run.smithi144.stdout:(2/33): ceph-mgr-18.2.2-1343.g2965081f.el9.x86_ 1.9 MB/s | 1.5 MB 00:00 2024-05-15T15:41:36.471 INFO:teuthology.orchestra.run.smithi144.stdout:(3/33): ceph-mds-18.2.2-1343.g2965081f.el9.x86_ 1.5 MB/s | 2.1 MB 00:01 2024-05-15T15:41:36.621 INFO:teuthology.orchestra.run.smithi144.stdout:(4/33): ceph-mgr-modules-core-18.2.2-1343.g2965 1.6 MB/s | 246 kB 00:00 2024-05-15T15:41:36.736 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-2:18.2.2-1343.g2965081f.el9.x86_64 33/33 2024-05-15T15:41:36.736 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:18.2.2-1343.g2965081f.el9.x86_64 1/33 2024-05-15T15:41:36.736 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:18.2.2-1343.g2965081f.el9.x86_64 2/33 2024-05-15T15:41:36.737 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 3/33 2024-05-15T15:41:36.737 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:18.2.2-1343.g2965081f.el9.x86_64 4/33 2024-05-15T15:41:36.737 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:18.2.2-1343.g2965081f.el9.x86_64 5/33 2024-05-15T15:41:36.737 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1343.g2965081f.el9. 6/33 2024-05-15T15:41:36.738 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-15T15:41:36.738 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-15T15:41:36.738 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-15T15:41:36.738 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-15T15:41:36.738 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-15T15:41:36.738 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-15T15:41:36.739 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-15T15:41:36.739 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-15T15:41:36.739 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-15T15:41:36.739 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-15T15:41:36.739 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-15T15:41:36.740 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-15T15:41:36.740 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-15T15:41:36.740 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-15T15:41:36.740 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-15T15:41:36.740 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-15T15:41:36.740 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-15T15:41:36.741 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-15T15:41:36.741 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-15T15:41:36.741 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-15T15:41:36.741 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-15T15:41:36.741 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-15T15:41:36.741 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-15T15:41:36.742 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-15T15:41:36.742 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-15T15:41:36.742 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-15T15:41:36.855 INFO:teuthology.orchestra.run.smithi144.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-15T15:41:37.072 INFO:teuthology.orchestra.run.smithi144.stdout:(6/33): ceph-mon-18.2.2-1343.g2965081f.el9.x86_ 2.7 MB/s | 4.7 MB 00:01 2024-05-15T15:41:37.231 INFO:teuthology.orchestra.run.smithi144.stdout:(7/33): ceph-osd-18.2.2-1343.g2965081f.el9.x86_ 12 MB/s | 17 MB 00:01 2024-05-15T15:41:37.273 INFO:teuthology.orchestra.run.smithi144.stdout:(8/33): python3-cryptography-36.0.1-4.el9.x86_6 2.9 MB/s | 1.2 MB 00:00 2024-05-15T15:41:37.323 INFO:teuthology.orchestra.run.smithi144.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.5 MB/s | 125 kB 00:00 2024-05-15T15:41:37.348 INFO:teuthology.orchestra.run.smithi144.stdout:(10/33): python3-ply-3.11-14.el9.noarch.rpm 386 kB/s | 106 kB 00:00 2024-05-15T15:41:37.390 INFO:teuthology.orchestra.run.smithi144.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 851 kB/s | 135 kB 00:00 2024-05-15T15:41:37.415 INFO:teuthology.orchestra.run.smithi144.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-05-15T15:41:37.457 INFO:teuthology.orchestra.run.smithi144.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-05-15T15:41:37.482 INFO:teuthology.orchestra.run.smithi144.stdout:(14/33): python3-markupsafe-1.1.1-12.el9.x86_64 376 kB/s | 35 kB 00:00 2024-05-15T15:41:37.533 INFO:teuthology.orchestra.run.smithi144.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-15T15:41:37.651 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-15T15:41:37.652 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-15T15:41:37.653 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-15T15:41:37.654 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:37.654 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:41:37.666 INFO:teuthology.orchestra.run.smithi144.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 203 kB/s | 51 kB 00:00 2024-05-15T15:41:37.792 INFO:teuthology.orchestra.run.smithi144.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 141 kB/s | 43 kB 00:00 2024-05-15T15:41:37.884 INFO:teuthology.orchestra.run.smithi144.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 117 kB/s | 11 kB 00:00 2024-05-15T15:41:37.950 INFO:teuthology.orchestra.run.smithi144.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 411 kB/s | 172 kB 00:00 2024-05-15T15:41:37.975 INFO:teuthology.orchestra.run.smithi144.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 193 kB/s | 18 kB 00:00 2024-05-15T15:41:37.995 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-base 2024-05-15T15:41:38.026 INFO:teuthology.orchestra.run.smithi144.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 308 kB/s | 23 kB 00:00 2024-05-15T15:41:38.060 INFO:teuthology.orchestra.run.smithi144.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 232 kB/s | 19 kB 00:00 2024-05-15T15:41:38.101 INFO:teuthology.orchestra.run.smithi144.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-05-15T15:41:38.127 INFO:teuthology.orchestra.run.smithi144.stdout:(24/33): python3-cherrypy-18.6.1-2.el9.noarch.r 779 kB/s | 358 kB 00:00 2024-05-15T15:41:38.194 INFO:teuthology.orchestra.run.smithi144.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 346 kB/s | 46 kB 00:00 2024-05-15T15:41:38.227 INFO:teuthology.orchestra.run.smithi144.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 626 kB/s | 79 kB 00:00 2024-05-15T15:41:38.261 INFO:teuthology.orchestra.run.smithi144.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-05-15T15:41:38.303 INFO:teuthology.orchestra.run.smithi144.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 150 kB/s | 16 kB 00:00 2024-05-15T15:41:38.370 INFO:teuthology.orchestra.run.smithi144.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 330 kB/s | 36 kB 00:00 2024-05-15T15:41:38.395 INFO:teuthology.orchestra.run.smithi144.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 536 kB/s | 90 kB 00:00 2024-05-15T15:41:38.487 INFO:teuthology.orchestra.run.smithi144.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 218 kB/s | 20 kB 00:00 2024-05-15T15:41:38.512 INFO:teuthology.orchestra.run.smithi144.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-05-15T15:41:38.538 INFO:teuthology.orchestra.run.smithi144.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2024-05-15T15:41:38.542 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:38.543 INFO:teuthology.orchestra.run.smithi144.stdout:Total 6.9 MB/s | 29 MB 00:04 2024-05-15T15:41:38.594 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:00 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:41:38.660 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:41:38.677 INFO:teuthology.orchestra.run.smithi049.stdout:Package ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:41:38.697 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:41:38.697 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:41:38.720 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:41:38.721 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-15T15:41:38.722 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:41:38.784 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install cephadm 2024-05-15T15:41:39.073 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:41:39.073 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:41:39.382 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:01 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:41:39.510 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:41:39.511 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-15T15:41:39.511 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:39.511 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:41:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 224 k 2024-05-15T15:41:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:39.512 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:41:39.512 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:39.512 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-05-15T15:41:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:39.512 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 224 k 2024-05-15T15:41:39.512 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 216 k 2024-05-15T15:41:39.512 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:41:40.223 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:41:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-15T15:41:40.330 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-15T15:41:40.364 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-15T15:41:40.454 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-15T15:41:40.525 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-15T15:41:40.566 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-15T15:41:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-15T15:41:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-15T15:41:40.647 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm-18.2.2-1343.g2965081f.el9.noarch.rpm 197 kB/s | 224 kB 00:01 2024-05-15T15:41:40.648 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:40.648 INFO:teuthology.orchestra.run.smithi049.stdout:Total 197 kB/s | 224 kB 00:01 2024-05-15T15:41:40.648 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:41:40.655 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:41:40.655 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:41:40.658 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:41:40.659 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:41:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-15T15:41:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-15T15:41:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:41:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-15T15:41:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-15T15:41:40.810 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-15T15:41:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-15T15:41:40.920 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-15T15:41:41.065 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-15T15:41:41.146 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-15T15:41:41.191 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/1 2024-05-15T15:41:41.199 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-15T15:41:41.238 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-15T15:41:41.287 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-15T15:41:41.287 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/1 2024-05-15T15:41:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-15T15:41:41.537 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-15T15:41:41.583 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-15T15:41:41.635 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/1 2024-05-15T15:41:41.652 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-15T15:41:41.742 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-15T15:41:41.774 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-15T15:41:41.890 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-15T15:41:41.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/1 2024-05-15T15:41:41.910 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:41.910 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:41:41.910 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:41:41.910 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:41.910 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:41:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1343.g2965081f.el9. 28/33 2024-05-15T15:41:42.077 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 29/33 2024-05-15T15:41:42.108 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 29/33 2024-05-15T15:41:42.108 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:42.108 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-15T15:41:42.108 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T15:41:42.108 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T15:41:42.109 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:42.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-immutable-object-cache 2024-05-15T15:41:42.723 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:04 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:41:42.854 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:41:42.855 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:42.858 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-05-15T15:41:42.858 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:42.858 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:41:42.858 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1343.g2965081f.el9 ceph 143 k 2024-05-15T15:41:42.858 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:42.858 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:41:42.859 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:42.859 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-05-15T15:41:42.859 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:42.859 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 143 k 2024-05-15T15:41:42.859 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 395 k 2024-05-15T15:41:42.859 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:41:43.519 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:18.2.2-1343.g2965081f.el9.x86_64 30/33 2024-05-15T15:41:43.547 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:18.2.2-1343.g2965081f.el9.x86_64 30/33 2024-05-15T15:41:43.547 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:43.547 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-15T15:41:43.547 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T15:41:43.547 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T15:41:43.547 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:43.561 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-immutable-object-cache-18.2.2-1343.g296508 202 kB/s | 143 kB 00:00 2024-05-15T15:41:43.562 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:43.562 INFO:teuthology.orchestra.run.smithi049.stdout:Total 202 kB/s | 143 kB 00:00 2024-05-15T15:41:43.562 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:41:43.571 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:41:43.571 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:41:43.631 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:41:43.632 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:41:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:41:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1343.g2965081f. 1/1 2024-05-15T15:41:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1343.g2965081f. 1/1 2024-05-15T15:41:43.873 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:43.873 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-15T15:41:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:43.987 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:18.2.2-1343.g2965081f.el9.x86_64 31/33 2024-05-15T15:41:44.010 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:18.2.2-1343.g2965081f.el9.x86_64 31/33 2024-05-15T15:41:44.011 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:44.011 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-15T15:41:44.011 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T15:41:44.011 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T15:41:44.011 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:44.212 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:18.2.2-1343.g2965081f.el9.x86_64 32/33 2024-05-15T15:41:44.236 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:18.2.2-1343.g2965081f.el9.x86_64 32/33 2024-05-15T15:41:44.236 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:44.236 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-15T15:41:44.236 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T15:41:44.236 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T15:41:44.236 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:44.301 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:18.2.2-1343.g2965081f.el9.x86_64 33/33 2024-05-15T15:41:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1343.g2965081f. 1/1 2024-05-15T15:41:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:44.750 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:41:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:44.751 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:41:44.984 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr 2024-05-15T15:41:45.584 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:07 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:41:45.668 INFO:teuthology.orchestra.run.smithi049.stdout:Package ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:41:45.711 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:41:45.712 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-15T15:41:45.712 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:41:45.775 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-dashboard 2024-05-15T15:41:46.062 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-2:18.2.2-1343.g2965081f.el9.x86_64 33/33 2024-05-15T15:41:46.062 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:18.2.2-1343.g2965081f.el9.x86_64 1/33 2024-05-15T15:41:46.062 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:18.2.2-1343.g2965081f.el9.x86_64 2/33 2024-05-15T15:41:46.062 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 3/33 2024-05-15T15:41:46.062 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:18.2.2-1343.g2965081f.el9.x86_64 4/33 2024-05-15T15:41:46.062 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:18.2.2-1343.g2965081f.el9.x86_64 5/33 2024-05-15T15:41:46.062 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1343.g2965081f.el9. 6/33 2024-05-15T15:41:46.062 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-15T15:41:46.063 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-15T15:41:46.063 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-15T15:41:46.063 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-15T15:41:46.063 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-15T15:41:46.063 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-15T15:41:46.063 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-15T15:41:46.063 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-15T15:41:46.063 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-15T15:41:46.064 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-15T15:41:46.065 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-15T15:41:46.065 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-15T15:41:46.065 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-15T15:41:46.066 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-15T15:41:46.066 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-15T15:41:46.371 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:08 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:41:46.499 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:41:46.500 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:46.500 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-15T15:41:46.500 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:46.500 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:41:46.500 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 3.5 M 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 24 k 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 16 k 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout:Install 5 Packages 2024-05-15T15:41:46.501 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:46.502 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 3.8 M 2024-05-15T15:41:46.502 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 87 M 2024-05-15T15:41:46.502 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-15T15:41:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-15T15:41:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:47.008 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:41:47.073 INFO:teuthology.orchestra.run.smithi049.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1343.g2965 45 kB/s | 16 kB 00:00 2024-05-15T15:41:47.248 INFO:teuthology.orchestra.run.smithi049.stdout:(2/5): ceph-grafana-dashboards-18.2.2-1343.g296 45 kB/s | 24 kB 00:00 2024-05-15T15:41:47.393 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-base 2024-05-15T15:41:47.482 INFO:teuthology.orchestra.run.smithi049.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 75 kB/s | 31 kB 00:00 2024-05-15T15:41:47.724 INFO:teuthology.orchestra.run.smithi049.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 397 kB/s | 188 kB 00:00 2024-05-15T15:41:47.969 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:11 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:41:48.049 INFO:teuthology.orchestra.run.smithi144.stdout:Package ceph-base-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:41:48.090 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:41:48.090 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-05-15T15:41:48.091 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:41:48.159 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install cephadm 2024-05-15T15:41:48.734 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:12 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:41:48.857 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:41:48.858 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-15T15:41:48.858 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:48.858 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:41:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 224 k 2024-05-15T15:41:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:48.859 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:41:48.859 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:48.859 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-05-15T15:41:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:48.860 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 224 k 2024-05-15T15:41:48.860 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 216 k 2024-05-15T15:41:48.860 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:41:49.216 INFO:teuthology.orchestra.run.smithi049.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1343.g2965081f 1.4 MB/s | 3.5 MB 00:02 2024-05-15T15:41:49.277 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:49.277 INFO:teuthology.orchestra.run.smithi049.stdout:Total 1.4 MB/s | 3.8 MB 00:02 2024-05-15T15:41:49.277 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:41:49.277 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:41:49.277 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:41:49.300 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:41:49.300 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:41:49.594 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:41:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-05-15T15:41:49.703 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-05-15T15:41:49.733 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1343.g2965081f.el9.n 3/5 2024-05-15T15:41:49.796 INFO:teuthology.orchestra.run.smithi144.stdout:cephadm-18.2.2-1343.g2965081f.el9.noarch.rpm 239 kB/s | 224 kB 00:00 2024-05-15T15:41:49.796 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:49.797 INFO:teuthology.orchestra.run.smithi144.stdout:Total 239 kB/s | 224 kB 00:00 2024-05-15T15:41:49.797 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:41:49.803 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:41:49.803 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:41:49.806 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:41:49.807 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:41:49.884 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:41:50.371 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/1 2024-05-15T15:41:50.460 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/1 2024-05-15T15:41:50.807 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/1 2024-05-15T15:41:50.848 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1343.g2965081f.el9. 4/5 2024-05-15T15:41:50.886 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1343.g2965081f.el9.noarc 5/5 2024-05-15T15:41:51.034 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/1 2024-05-15T15:41:51.034 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:51.034 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:41:51.034 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:41:51.034 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:51.035 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:41:51.254 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-immutable-object-cache 2024-05-15T15:41:51.842 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:15 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:41:51.916 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1343.g2965081f.el9.noarc 5/5 2024-05-15T15:41:51.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1343.g2965081f.el9. 1/5 2024-05-15T15:41:51.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1343.g2965081f.el9.noarc 2/5 2024-05-15T15:41:51.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1343.g2965081f.el9.n 3/5 2024-05-15T15:41:51.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-05-15T15:41:51.970 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1343.g2965081f.el9 ceph 143 k 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 143 k 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 395 k 2024-05-15T15:41:51.971 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:41:52.174 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-05-15T15:41:52.174 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:52.174 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:41:52.174 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:41:52.175 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:41:52.175 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:41:52.175 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-15T15:41:52.175 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-15T15:41:52.175 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:52.175 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:41:52.372 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-15T15:41:52.486 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-immutable-object-cache-18.2.2-1343.g296508 278 kB/s | 143 kB 00:00 2024-05-15T15:41:52.486 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:52.486 INFO:teuthology.orchestra.run.smithi144.stdout:Total 277 kB/s | 143 kB 00:00 2024-05-15T15:41:52.486 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:41:52.494 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:41:52.494 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:41:52.551 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:41:52.552 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:41:52.703 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:41:52.753 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1343.g2965081f. 1/1 2024-05-15T15:41:52.779 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1343.g2965081f. 1/1 2024-05-15T15:41:52.779 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:41:52.779 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-15T15:41:52.779 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:52.970 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:14 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:41:53.100 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 7.4 M 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-15T15:41:53.101 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout:Install 12 Packages 2024-05-15T15:41:53.102 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:41:53.103 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 38 M 2024-05-15T15:41:53.103 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 209 M 2024-05-15T15:41:53.103 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:41:53.473 INFO:teuthology.orchestra.run.smithi049.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-05-15T15:41:53.557 INFO:teuthology.orchestra.run.smithi049.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2024-05-15T15:41:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-05-15T15:41:53.632 INFO:teuthology.orchestra.run.smithi049.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-05-15T15:41:53.691 INFO:teuthology.orchestra.run.smithi049.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-05-15T15:41:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1343.g2965081f. 1/1 2024-05-15T15:41:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:53.699 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:41:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:41:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:53.699 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:41:53.975 INFO:teuthology.orchestra.run.smithi049.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.2 MB/s | 3.0 MB 00:00 2024-05-15T15:41:53.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr 2024-05-15T15:41:54.075 INFO:teuthology.orchestra.run.smithi049.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-15T15:41:54.326 INFO:teuthology.orchestra.run.smithi049.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-05-15T15:41:54.392 INFO:teuthology.orchestra.run.smithi049.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 6.8 MB/s | 4.7 MB 00:00 2024-05-15T15:41:54.493 INFO:teuthology.orchestra.run.smithi049.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.5 MB/s | 250 kB 00:00 2024-05-15T15:41:54.566 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:18 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:41:54.652 INFO:teuthology.orchestra.run.smithi144.stdout:Package ceph-mgr-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:41:54.694 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:41:54.695 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-05-15T15:41:54.695 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:41:54.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-dashboard 2024-05-15T15:41:55.342 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:19 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:41:55.467 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:41:55.467 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 3.5 M 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 24 k 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 16 k 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:55.468 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:41:55.469 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:41:55.469 INFO:teuthology.orchestra.run.smithi144.stdout:Install 5 Packages 2024-05-15T15:41:55.469 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:41:55.469 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 3.8 M 2024-05-15T15:41:55.469 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 87 M 2024-05-15T15:41:55.469 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:41:55.831 INFO:teuthology.orchestra.run.smithi144.stdout:(1/5): ceph-grafana-dashboards-18.2.2-1343.g296 152 kB/s | 24 kB 00:00 2024-05-15T15:41:55.856 INFO:teuthology.orchestra.run.smithi144.stdout:(2/5): ceph-prometheus-alerts-18.2.2-1343.g2965 89 kB/s | 16 kB 00:00 2024-05-15T15:41:55.931 INFO:teuthology.orchestra.run.smithi144.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 309 kB/s | 31 kB 00:00 2024-05-15T15:41:55.968 INFO:teuthology.orchestra.run.smithi049.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-15T15:41:56.006 INFO:teuthology.orchestra.run.smithi144.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2024-05-15T15:41:56.393 INFO:teuthology.orchestra.run.smithi049.stdout:(12/12): ceph-mgr-diskprediction-local-18.2.2-1 2.5 MB/s | 7.4 MB 00:02 2024-05-15T15:41:56.394 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:56.394 INFO:teuthology.orchestra.run.smithi049.stdout:Total 12 MB/s | 38 MB 00:03 2024-05-15T15:41:56.767 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:41:56.795 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:41:56.795 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:41:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:41:57.328 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:41:57.790 INFO:teuthology.orchestra.run.smithi144.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1343.g2965081f 1.7 MB/s | 3.5 MB 00:02 2024-05-15T15:41:57.792 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:41:57.792 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.6 MB/s | 3.8 MB 00:02 2024-05-15T15:41:57.793 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:41:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:41:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:41:57.867 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:41:57.867 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:41:58.127 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:41:58.163 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:41:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-05-15T15:41:58.236 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-15T15:41:58.245 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-05-15T15:41:58.273 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1343.g2965081f.el9.n 3/5 2024-05-15T15:41:58.283 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-15T15:41:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-15T15:41:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-15T15:41:58.622 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-15T15:41:58.658 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-15T15:41:58.750 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-15T15:41:59.319 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1343.g2965081f.el9. 4/5 2024-05-15T15:41:59.352 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1343.g2965081f.el9.noarc 5/5 2024-05-15T15:41:59.474 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-15T15:41:59.601 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-15T15:42:00.371 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1343.g2965081f.el9.noarc 5/5 2024-05-15T15:42:00.371 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1343.g2965081f.el9. 1/5 2024-05-15T15:42:00.371 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1343.g2965081f.el9.noarc 2/5 2024-05-15T15:42:00.371 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1343.g2965081f.el9.n 3/5 2024-05-15T15:42:00.371 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-05-15T15:42:00.616 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-05-15T15:42:00.616 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:00.616 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:00.617 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:42:00.617 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:42:00.617 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:42:00.617 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-15T15:42:00.617 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-15T15:42:00.617 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:00.617 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:00.852 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-15T15:42:00.941 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-15T15:42:01.435 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:25 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:01.563 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:01.565 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-05-15T15:42:01.565 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-15T15:42:01.565 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-05-15T15:42:01.565 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:42:01.565 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 7.4 M 2024-05-15T15:42:01.565 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-15T15:42:01.566 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-15T15:42:01.567 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:01.567 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:42:01.567 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-05-15T15:42:01.567 INFO:teuthology.orchestra.run.smithi144.stdout:Install 12 Packages 2024-05-15T15:42:01.567 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:01.568 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 38 M 2024-05-15T15:42:01.568 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 209 M 2024-05-15T15:42:01.568 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:42:02.035 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-15T15:42:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1343.g29650 12/12 2024-05-15T15:42:02.298 INFO:teuthology.orchestra.run.smithi144.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 394 kB/s | 195 kB 00:00 2024-05-15T15:42:02.381 INFO:teuthology.orchestra.run.smithi144.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 394 kB/s | 33 kB 00:00 2024-05-15T15:42:02.407 INFO:teuthology.orchestra.run.smithi144.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-05-15T15:42:02.491 INFO:teuthology.orchestra.run.smithi144.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 219 kB/s | 18 kB 00:00 2024-05-15T15:42:02.549 INFO:teuthology.orchestra.run.smithi144.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 607 kB/s | 35 kB 00:00 2024-05-15T15:42:02.599 INFO:teuthology.orchestra.run.smithi144.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-05-15T15:42:02.641 INFO:teuthology.orchestra.run.smithi144.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-15T15:42:02.750 INFO:teuthology.orchestra.run.smithi144.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2024-05-15T15:42:02.817 INFO:teuthology.orchestra.run.smithi144.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-05-15T15:42:02.842 INFO:teuthology.orchestra.run.smithi144.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-05-15T15:42:03.075 INFO:teuthology.orchestra.run.smithi144.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 63 MB/s | 16 MB 00:00 2024-05-15T15:42:03.801 INFO:teuthology.orchestra.run.smithi144.stdout:(12/12): ceph-mgr-diskprediction-local-18.2.2-1 3.7 MB/s | 7.4 MB 00:01 2024-05-15T15:42:03.803 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:03.803 INFO:teuthology.orchestra.run.smithi144.stdout:Total 17 MB/s | 38 MB 00:02 2024-05-15T15:42:04.160 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:42:04.186 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:42:04.186 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:42:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1343.g29650 12/12 2024-05-15T15:42:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1343.g29650 1/12 2024-05-15T15:42:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-15T15:42:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-15T15:42:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-15T15:42:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-15T15:42:04.340 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-15T15:42:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-15T15:42:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-15T15:42:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-15T15:42:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-15T15:42:04.341 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-15T15:42:04.691 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:42:04.692 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:42:04.790 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-15T15:42:04.791 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-15T15:42:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-15T15:42:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-15T15:42:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-15T15:42:04.792 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:04.792 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:05.125 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-rook 2024-05-15T15:42:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:42:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-15T15:42:05.615 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-15T15:42:05.715 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-15T15:42:05.730 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:27 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:05.750 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-15T15:42:05.857 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:05.861 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-15T15:42:05.861 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:05.861 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:42:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 49 k 2024-05-15T15:42:05.861 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-15T15:42:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-15T15:42:05.861 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:42:05.862 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:05.863 INFO:teuthology.orchestra.run.smithi049.stdout:Install 13 Packages 2024-05-15T15:42:05.863 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:05.863 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 2.2 M 2024-05-15T15:42:05.863 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 26 M 2024-05-15T15:42:05.863 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:42:05.952 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-15T15:42:05.990 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-15T15:42:06.073 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-15T15:42:06.383 INFO:teuthology.orchestra.run.smithi049.stdout:(1/13): ceph-mgr-rook-18.2.2-1343.g2965081f.el9 258 kB/s | 49 kB 00:00 2024-05-15T15:42:06.466 INFO:teuthology.orchestra.run.smithi049.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 93 kB/s | 26 kB 00:00 2024-05-15T15:42:06.491 INFO:teuthology.orchestra.run.smithi049.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 64 kB/s | 19 kB 00:00 2024-05-15T15:42:06.764 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-15T15:42:06.809 INFO:teuthology.orchestra.run.smithi049.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 464 kB/s | 159 kB 00:00 2024-05-15T15:42:06.879 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-15T15:42:06.901 INFO:teuthology.orchestra.run.smithi049.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 429 kB/s | 222 kB 00:00 2024-05-15T15:42:06.926 INFO:teuthology.orchestra.run.smithi049.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 642 kB/s | 279 kB 00:00 2024-05-15T15:42:06.951 INFO:teuthology.orchestra.run.smithi049.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 377 kB/s | 54 kB 00:00 2024-05-15T15:42:07.185 INFO:teuthology.orchestra.run.smithi049.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 58 kB/s | 15 kB 00:00 2024-05-15T15:42:07.210 INFO:teuthology.orchestra.run.smithi049.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 104 kB/s | 32 kB 00:00 2024-05-15T15:42:07.419 INFO:teuthology.orchestra.run.smithi049.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 440 kB/s | 206 kB 00:00 2024-05-15T15:42:07.443 INFO:teuthology.orchestra.run.smithi049.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 253 kB/s | 59 kB 00:00 2024-05-15T15:42:07.619 INFO:teuthology.orchestra.run.smithi049.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 448 kB/s | 90 kB 00:00 2024-05-15T15:42:07.727 INFO:teuthology.orchestra.run.smithi049.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2024-05-15T15:42:07.728 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:07.728 INFO:teuthology.orchestra.run.smithi049.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-05-15T15:42:07.803 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:42:07.820 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:42:07.820 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:42:07.918 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:42:07.918 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:42:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-15T15:42:08.432 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:42:08.615 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-15T15:42:08.672 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-15T15:42:08.736 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-15T15:42:08.772 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-15T15:42:08.813 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-15T15:42:08.910 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-15T15:42:09.030 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-15T15:42:09.090 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-15T15:42:09.171 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-15T15:42:09.198 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1343.g29650 12/12 2024-05-15T15:42:09.847 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-15T15:42:09.918 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-15T15:42:09.951 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-15T15:42:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-15T15:42:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:18.2.2-1343.g2965081f.el9.noarch 13/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1343.g2965081f.el9.noarch 13/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1343.g2965081f.el9.noarch 1/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-15T15:42:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-15T15:42:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-15T15:42:11.106 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-15T15:42:11.106 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:11.106 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:42:11.106 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:42:11.106 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:11.107 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:11.348 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-cephadm 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1343.g29650 12/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1343.g29650 1/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-15T15:42:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-15T15:42:11.948 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:33 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:12.074 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:12.074 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:12.074 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 139 k 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:12.075 INFO:teuthology.orchestra.run.smithi049.stdout:Install 6 Packages 2024-05-15T15:42:12.076 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:12.076 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 7.0 M 2024-05-15T15:42:12.076 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 33 M 2024-05-15T15:42:12.076 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:42:12.122 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-15T15:42:12.123 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-15T15:42:12.124 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-15T15:42:12.124 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-15T15:42:12.124 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-15T15:42:12.124 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-15T15:42:12.124 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:12.124 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:12.485 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-rook 2024-05-15T15:42:12.926 INFO:teuthology.orchestra.run.smithi049.stdout:(1/6): ceph-mgr-cephadm-18.2.2-1343.g2965081f.e 600 kB/s | 139 kB 00:00 2024-05-15T15:42:12.993 INFO:teuthology.orchestra.run.smithi049.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 832 kB/s | 248 kB 00:00 2024-05-15T15:42:13.065 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:37 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:13.118 INFO:teuthology.orchestra.run.smithi049.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.8 MB/s | 548 kB 00:00 2024-05-15T15:42:13.152 INFO:teuthology.orchestra.run.smithi049.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 366 kB/s | 58 kB 00:00 2024-05-15T15:42:13.185 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:13.186 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:13.186 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-15T15:42:13.186 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:13.186 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 49 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-15T15:42:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-15T15:42:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-15T15:42:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:13.188 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:42:13.188 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:13.188 INFO:teuthology.orchestra.run.smithi144.stdout:Install 13 Packages 2024-05-15T15:42:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:13.188 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 2.2 M 2024-05-15T15:42:13.188 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 26 M 2024-05-15T15:42:13.189 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:42:13.244 INFO:teuthology.orchestra.run.smithi049.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 409 kB/s | 51 kB 00:00 2024-05-15T15:42:13.507 INFO:teuthology.orchestra.run.smithi144.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 200 kB/s | 26 kB 00:00 2024-05-15T15:42:13.532 INFO:teuthology.orchestra.run.smithi144.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 125 kB/s | 19 kB 00:00 2024-05-15T15:42:13.674 INFO:teuthology.orchestra.run.smithi144.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-05-15T15:42:13.699 INFO:teuthology.orchestra.run.smithi144.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 950 kB/s | 159 kB 00:00 2024-05-15T15:42:13.724 INFO:teuthology.orchestra.run.smithi144.stdout:(5/13): ceph-mgr-rook-18.2.2-1343.g2965081f.el9 142 kB/s | 49 kB 00:00 2024-05-15T15:42:13.758 INFO:teuthology.orchestra.run.smithi144.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 911 kB/s | 54 kB 00:00 2024-05-15T15:42:13.900 INFO:teuthology.orchestra.run.smithi144.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 183 kB/s | 32 kB 00:00 2024-05-15T15:42:13.926 INFO:teuthology.orchestra.run.smithi144.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-05-15T15:42:13.959 INFO:teuthology.orchestra.run.smithi144.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 85 kB/s | 15 kB 00:00 2024-05-15T15:42:14.026 INFO:teuthology.orchestra.run.smithi144.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 1.6 MB/s | 206 kB 00:00 2024-05-15T15:42:14.060 INFO:teuthology.orchestra.run.smithi144.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 769 kB/s | 59 kB 00:00 2024-05-15T15:42:14.085 INFO:teuthology.orchestra.run.smithi144.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-05-15T15:42:14.085 INFO:teuthology.orchestra.run.smithi049.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.3 MB/s | 6.0 MB 00:01 2024-05-15T15:42:14.087 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:14.087 INFO:teuthology.orchestra.run.smithi049.stdout:Total 3.5 MB/s | 7.0 MB 00:02 2024-05-15T15:42:14.164 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:42:14.177 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:42:14.177 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:42:14.282 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:42:14.282 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:42:14.493 INFO:teuthology.orchestra.run.smithi144.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.8 MB/s | 1.0 MB 00:00 2024-05-15T15:42:14.496 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:14.496 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-05-15T15:42:14.560 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:42:14.574 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:42:14.574 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:42:14.592 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:42:14.658 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:42:14.659 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:42:14.768 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-15T15:42:14.822 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-15T15:42:15.168 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:42:15.330 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-15T15:42:15.376 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-15T15:42:15.385 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-15T15:42:15.441 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-15T15:42:15.469 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-15T15:42:15.478 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-15T15:42:15.520 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-15T15:42:15.541 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-15T15:42:15.566 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1343.g2965081f.el9.noarch 6/6 2024-05-15T15:42:15.612 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-15T15:42:15.723 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-15T15:42:15.780 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-15T15:42:16.344 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1343.g2965081f.el9.noarch 6/6 2024-05-15T15:42:16.344 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/6 2024-05-15T15:42:16.344 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-15T15:42:16.344 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-15T15:42:16.344 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-15T15:42:16.344 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-15T15:42:16.488 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-15T15:42:16.538 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-15T15:42:16.572 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-15T15:42:16.612 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-15T15:42:16.612 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:42:16.612 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:42:16.612 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-15T15:42:16.612 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-15T15:42:16.612 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-15T15:42:16.612 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-15T15:42:16.613 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-15T15:42:16.613 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:16.613 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:16.619 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-15T15:42:16.642 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:18.2.2-1343.g2965081f.el9.noarch 13/13 2024-05-15T15:42:16.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-fuse 2024-05-15T15:42:17.372 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1343.g2965081f.el9.noarch 13/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1343.g2965081f.el9.noarch 1/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-15T15:42:17.373 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-15T15:42:17.464 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:39 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:17.593 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:17.593 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:17.593 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-15T15:42:17.593 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:17.594 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:42:17.594 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:18.2.2-1343.g2965081f.el9 ceph 849 k 2024-05-15T15:42:17.594 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-15T15:42:17.594 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-15T15:42:17.594 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:17.594 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:42:17.594 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:17.594 INFO:teuthology.orchestra.run.smithi049.stdout:Install 2 Packages 2024-05-15T15:42:17.594 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:17.595 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 929 k 2024-05-15T15:42:17.595 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 2.7 M 2024-05-15T15:42:17.595 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:42:17.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-15T15:42:17.696 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:17.696 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:17.696 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:42:17.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-15T15:42:17.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-15T15:42:17.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-15T15:42:17.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-15T15:42:17.696 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-15T15:42:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-15T15:42:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-15T15:42:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-15T15:42:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-15T15:42:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-15T15:42:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-15T15:42:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-15T15:42:17.697 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:17.697 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:17.821 INFO:teuthology.orchestra.run.smithi049.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 681 kB/s | 80 kB 00:00 2024-05-15T15:42:17.945 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-cephadm 2024-05-15T15:42:18.021 INFO:teuthology.orchestra.run.smithi049.stdout:(2/2): ceph-fuse-18.2.2-1343.g2965081f.el9.x86_ 2.6 MB/s | 849 kB 00:00 2024-05-15T15:42:18.021 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:18.022 INFO:teuthology.orchestra.run.smithi049.stdout:Total 2.1 MB/s | 929 kB 00:00 2024-05-15T15:42:18.034 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:42:18.057 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:42:18.057 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:42:18.139 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:42:18.139 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:42:18.340 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:42:18.482 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-15T15:42:18.521 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:42 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:18.557 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 2/2 2024-05-15T15:42:18.641 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:18.641 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:18.641 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1343.g2965081f.el9 ceph-noarch 139 k 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout:Install 6 Packages 2024-05-15T15:42:18.642 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:18.643 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 7.0 M 2024-05-15T15:42:18.643 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 33 M 2024-05-15T15:42:18.643 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:42:19.031 INFO:teuthology.orchestra.run.smithi144.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-05-15T15:42:19.106 INFO:teuthology.orchestra.run.smithi144.stdout:(2/6): ceph-mgr-cephadm-18.2.2-1343.g2965081f.e 507 kB/s | 139 kB 00:00 2024-05-15T15:42:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 2/2 2024-05-15T15:42:19.212 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 1/2 2024-05-15T15:42:19.273 INFO:teuthology.orchestra.run.smithi144.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.2 MB/s | 548 kB 00:00 2024-05-15T15:42:19.298 INFO:teuthology.orchestra.run.smithi144.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 302 kB/s | 58 kB 00:00 2024-05-15T15:42:19.324 INFO:teuthology.orchestra.run.smithi144.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 1.0 MB/s | 51 kB 00:00 2024-05-15T15:42:19.416 INFO:teuthology.orchestra.run.smithi144.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-05-15T15:42:19.417 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:19.417 INFO:teuthology.orchestra.run.smithi144.stdout:Total 9.0 MB/s | 7.0 MB 00:00 2024-05-15T15:42:19.487 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:42:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-15T15:42:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:19.488 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:42:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-15T15:42:19.488 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:19.488 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:19.498 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:42:19.498 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:42:19.593 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:42:19.594 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:42:19.793 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install librados-devel 2024-05-15T15:42:19.896 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:42:20.068 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-15T15:42:20.118 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-15T15:42:20.402 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:42 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:20.530 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:18.2.2-1343.g2965081f.el9 ceph 127 k 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:20.531 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 127 k 2024-05-15T15:42:20.532 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 456 k 2024-05-15T15:42:20.532 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:42:20.649 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-15T15:42:20.738 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-15T15:42:20.799 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-15T15:42:20.830 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1343.g2965081f.el9.noarch 6/6 2024-05-15T15:42:21.142 INFO:teuthology.orchestra.run.smithi049.stdout:librados-devel-18.2.2-1343.g2965081f.el9.x86_64 207 kB/s | 127 kB 00:00 2024-05-15T15:42:21.143 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:21.143 INFO:teuthology.orchestra.run.smithi049.stdout:Total 207 kB/s | 127 kB 00:00 2024-05-15T15:42:21.143 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:42:21.150 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:42:21.150 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:42:21.216 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:42:21.217 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:42:21.374 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:42:21.484 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1343.g2965081f.el9.noarch 6/6 2024-05-15T15:42:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1343.g2965081f.el9.noarch 1/6 2024-05-15T15:42:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-15T15:42:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-15T15:42:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-15T15:42:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:18.2.2-1343.g2965081f.el9.noarch 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-15T15:42:21.845 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:21.846 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:22.031 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:22.113 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-fuse 2024-05-15T15:42:22.295 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:22.296 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:42:22.296 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:22.296 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:22.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install libcephfs2 2024-05-15T15:42:22.691 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:46 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:22.813 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:22.814 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:22.814 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-15T15:42:22.814 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:22.814 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:42:22.814 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:18.2.2-1343.g2965081f.el9 ceph 849 k 2024-05-15T15:42:22.814 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-15T15:42:22.814 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-15T15:42:22.815 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:22.815 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:42:22.815 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:22.815 INFO:teuthology.orchestra.run.smithi144.stdout:Install 2 Packages 2024-05-15T15:42:22.815 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:22.815 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 929 k 2024-05-15T15:42:22.815 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 2.7 M 2024-05-15T15:42:22.815 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:42:22.993 INFO:teuthology.orchestra.run.smithi144.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 722 kB/s | 80 kB 00:00 2024-05-15T15:42:23.128 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:45 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:23.212 INFO:teuthology.orchestra.run.smithi049.stdout:Package libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:23.257 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:23.258 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-15T15:42:23.258 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:23.319 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install libcephfs-devel 2024-05-15T15:42:23.552 INFO:teuthology.orchestra.run.smithi144.stdout:(2/2): ceph-fuse-18.2.2-1343.g2965081f.el9.x86_ 1.2 MB/s | 849 kB 00:00 2024-05-15T15:42:23.552 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:23.552 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.2 MB/s | 929 kB 00:00 2024-05-15T15:42:23.564 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:42:23.586 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:42:23.586 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:42:23.663 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:42:23.663 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:42:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:42:23.923 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:45 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-15T15:42:24.049 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:18.2.2-1343.g2965081f.el9 ceph 31 k 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 31 k 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 139 k 2024-05-15T15:42:24.050 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:42:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 2/2 2024-05-15T15:42:24.372 INFO:teuthology.orchestra.run.smithi049.stdout:libcephfs-devel-18.2.2-1343.g2965081f.el9.x86_6 97 kB/s | 31 kB 00:00 2024-05-15T15:42:24.372 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:24.372 INFO:teuthology.orchestra.run.smithi049.stdout:Total 96 kB/s | 31 kB 00:00 2024-05-15T15:42:24.373 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:42:24.376 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:42:24.376 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:42:24.397 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:42:24.398 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:42:24.483 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:42:24.597 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:24.709 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 2/2 2024-05-15T15:42:24.709 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 1/2 2024-05-15T15:42:24.960 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:24.970 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-15T15:42:24.970 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:24.970 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:24.970 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-15T15:42:24.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:24.971 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:25.186 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:25.187 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:42:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:25.187 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:25.187 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:25.283 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librados-devel 2024-05-15T15:42:25.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install librados2 2024-05-15T15:42:25.873 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:49 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:25.976 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:47 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:26.001 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:26.001 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:26.001 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-05-15T15:42:26.001 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:18.2.2-1343.g2965081f.el9 ceph 127 k 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 127 k 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 456 k 2024-05-15T15:42:26.002 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:42:26.060 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:26.105 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:26.106 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-15T15:42:26.106 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:26.171 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install librbd1 2024-05-15T15:42:26.482 INFO:teuthology.orchestra.run.smithi144.stdout:librados-devel-18.2.2-1343.g2965081f.el9.x86_64 264 kB/s | 127 kB 00:00 2024-05-15T15:42:26.482 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:26.482 INFO:teuthology.orchestra.run.smithi144.stdout:Total 263 kB/s | 127 kB 00:00 2024-05-15T15:42:26.483 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:42:26.488 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:42:26.488 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:42:26.550 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:42:26.550 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:42:26.698 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:42:26.777 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:48 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:26.796 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:26.860 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:26.903 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:26.904 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-15T15:42:26.905 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:26.968 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-rados 2024-05-15T15:42:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:27.462 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:27.463 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:27.463 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:27.463 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:27.463 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:27.463 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:27.571 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:49 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:27.656 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-rados-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:27.699 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:27.700 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-15T15:42:27.700 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:27.735 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install libcephfs2 2024-05-15T15:42:27.759 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-rgw 2024-05-15T15:42:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:52 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:28.362 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:50 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:28.408 INFO:teuthology.orchestra.run.smithi144.stdout:Package libcephfs2-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:28.446 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-rgw-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:28.449 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:28.450 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-05-15T15:42:28.450 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:28.490 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:28.491 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-15T15:42:28.491 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:28.524 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install libcephfs-devel 2024-05-15T15:42:28.562 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-cephfs 2024-05-15T15:42:29.111 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:53 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:29.164 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:51 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:29.234 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:18.2.2-1343.g2965081f.el9 ceph 31 k 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-05-15T15:42:29.235 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:29.236 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 31 k 2024-05-15T15:42:29.236 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 139 k 2024-05-15T15:42:29.236 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:42:29.246 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-cephfs-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:29.289 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:29.290 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-15T15:42:29.290 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:29.353 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-rbd 2024-05-15T15:42:29.652 INFO:teuthology.orchestra.run.smithi144.stdout:libcephfs-devel-18.2.2-1343.g2965081f.el9.x86_6 74 kB/s | 31 kB 00:00 2024-05-15T15:42:29.653 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:29.653 INFO:teuthology.orchestra.run.smithi144.stdout:Total 74 kB/s | 31 kB 00:00 2024-05-15T15:42:29.653 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:42:29.657 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:42:29.657 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:42:29.670 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:42:29.671 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:42:29.747 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:42:29.865 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:29.958 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:51 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:30.040 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-rbd-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:30.083 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:30.084 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-05-15T15:42:30.085 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:30.145 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install rbd-fuse 2024-05-15T15:42:30.240 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:30.468 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:30.469 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:30.469 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:30.469 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:30.469 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:30.469 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:30.668 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librados2 2024-05-15T15:42:30.750 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:52 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:30.878 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:18.2.2-1343.g2965081f.el9 ceph 87 k 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-05-15T15:42:30.879 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:30.880 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 87 k 2024-05-15T15:42:30.880 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 227 k 2024-05-15T15:42:30.880 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:42:31.243 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:55 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:31.324 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:31.341 INFO:teuthology.orchestra.run.smithi049.stdout:rbd-fuse-18.2.2-1343.g2965081f.el9.x86_64.rpm 188 kB/s | 87 kB 00:00 2024-05-15T15:42:31.342 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:31.342 INFO:teuthology.orchestra.run.smithi049.stdout:Total 187 kB/s | 87 kB 00:00 2024-05-15T15:42:31.343 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:42:31.350 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:42:31.350 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:42:31.367 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:31.368 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-05-15T15:42:31.368 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:31.417 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:42:31.417 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:42:31.435 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librbd1 2024-05-15T15:42:31.565 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:42:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:32.026 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:56 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:32.062 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:32.115 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:32.162 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:32.163 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-05-15T15:42:32.163 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:32.240 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rados 2024-05-15T15:42:32.325 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:32.325 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:32.325 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:42:32.326 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:32.326 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:32.326 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:32.518 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install rbd-mirror 2024-05-15T15:42:32.833 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:56 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:32.918 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rados-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:32.960 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:32.961 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-05-15T15:42:32.961 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:33.022 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rgw 2024-05-15T15:42:33.125 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:55 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:33.256 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:33.257 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-05-15T15:42:33.257 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:33.257 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:42:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:18.2.2-1343.g2965081f.el9 ceph 3.0 M 2024-05-15T15:42:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:33.257 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:42:33.257 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:33.258 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-05-15T15:42:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:33.258 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 3.0 M 2024-05-15T15:42:33.258 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 12 M 2024-05-15T15:42:33.258 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:42:33.603 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:57 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:33.683 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rgw-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:33.724 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:33.726 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-05-15T15:42:33.726 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:33.795 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-cephfs 2024-05-15T15:42:34.380 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:58 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:34.460 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-cephfs-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:34.501 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:34.502 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-05-15T15:42:34.502 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:34.571 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rbd 2024-05-15T15:42:34.787 INFO:teuthology.orchestra.run.smithi049.stdout:rbd-mirror-18.2.2-1343.g2965081f.el9.x86_64.rpm 2.0 MB/s | 3.0 MB 00:01 2024-05-15T15:42:34.787 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:34.787 INFO:teuthology.orchestra.run.smithi049.stdout:Total 2.0 MB/s | 3.0 MB 00:01 2024-05-15T15:42:34.788 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:42:34.799 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:42:34.799 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:42:34.886 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:42:34.887 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:42:35.150 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:59 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:35.230 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rbd-2:18.2.2-1343.g2965081f.el9.x86_64 is already installed. 2024-05-15T15:42:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-05-15T15:42:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:35.329 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:42:35.347 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-fuse 2024-05-15T15:42:35.395 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:35.422 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:35.423 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:42:35.423 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-15T15:42:35.423 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T15:42:35.423 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T15:42:35.423 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:35.917 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:59 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:36.039 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:36.040 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-05-15T15:42:36.040 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:36.040 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:42:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:18.2.2-1343.g2965081f.el9 ceph 87 k 2024-05-15T15:42:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:36.041 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:42:36.041 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:36.041 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-05-15T15:42:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:36.041 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 87 k 2024-05-15T15:42:36.041 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 227 k 2024-05-15T15:42:36.041 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:42:36.534 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:36.534 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:36.534 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:42:36.534 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:36.534 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:36.534 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:36.539 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-fuse-18.2.2-1343.g2965081f.el9.x86_64.rpm 174 kB/s | 87 kB 00:00 2024-05-15T15:42:36.539 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:36.540 INFO:teuthology.orchestra.run.smithi144.stdout:Total 174 kB/s | 87 kB 00:00 2024-05-15T15:42:36.540 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:42:36.547 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:42:36.547 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:42:36.610 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:42:36.610 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:42:36.750 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:42:36.756 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install rbd-nbd 2024-05-15T15:42:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:37.269 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:37.362 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:59 ago on Wed 15 May 2024 03:40:38 PM UTC. 2024-05-15T15:42:37.491 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:18.2.2-1343.g2965081f.el9 ceph 172 k 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:37.492 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 172 k 2024-05-15T15:42:37.493 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 494 k 2024-05-15T15:42:37.493 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-15T15:42:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:37.527 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:37.527 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:37.781 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-mirror 2024-05-15T15:42:37.923 INFO:teuthology.orchestra.run.smithi049.stdout:rbd-nbd-18.2.2-1343.g2965081f.el9.x86_64.rpm 400 kB/s | 172 kB 00:00 2024-05-15T15:42:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:Total 399 kB/s | 172 kB 00:00 2024-05-15T15:42:37.924 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-15T15:42:37.933 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-15T15:42:37.933 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-15T15:42:38.001 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-15T15:42:38.002 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-15T15:42:38.156 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-15T15:42:38.248 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:38.370 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:02 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:38.494 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:38.494 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:18.2.2-1343.g2965081f.el9 ceph 3.0 M 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 3.0 M 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 12 M 2024-05-15T15:42:38.495 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:42:38.628 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:38.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:38.891 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:38.891 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-15T15:42:38.891 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:38.891 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:42:38.891 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-15T15:42:39.115 DEBUG:teuthology.parallel:result is None 2024-05-15T15:42:39.701 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-mirror-18.2.2-1343.g2965081f.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-05-15T15:42:39.702 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:39.702 INFO:teuthology.orchestra.run.smithi144.stdout:Total 2.5 MB/s | 3.0 MB 00:01 2024-05-15T15:42:39.702 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:42:39.712 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:42:39.712 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:42:39.793 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:42:39.793 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:42:40.205 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:42:40.268 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:40.293 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T15:42:40.294 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-15T15:42:40.294 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T15:42:40.294 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T15:42:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:41.358 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:41.359 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:41.603 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-nbd 2024-05-15T15:42:42.197 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:06 ago on Wed 15 May 2024 03:40:36 PM UTC. 2024-05-15T15:42:42.321 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-15T15:42:42.322 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:42.322 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-05-15T15:42:42.322 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:42.322 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-15T15:42:42.323 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:18.2.2-1343.g2965081f.el9 ceph 172 k 2024-05-15T15:42:42.323 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:42.323 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-15T15:42:42.323 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-05-15T15:42:42.323 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-05-15T15:42:42.323 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:42.323 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 172 k 2024-05-15T15:42:42.323 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 494 k 2024-05-15T15:42:42.323 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-15T15:42:42.680 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-nbd-18.2.2-1343.g2965081f.el9.x86_64.rpm 483 kB/s | 172 kB 00:00 2024-05-15T15:42:42.680 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:42:42.680 INFO:teuthology.orchestra.run.smithi144.stdout:Total 481 kB/s | 172 kB 00:00 2024-05-15T15:42:42.681 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-15T15:42:42.688 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-15T15:42:42.688 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-15T15:42:42.750 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-15T15:42:42.751 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-15T15:42:42.900 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-15T15:42:43.004 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:43.418 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:43.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:18.2.2-1343.g2965081f.el9.x86_64 1/1 2024-05-15T15:42:43.692 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:43.692 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-15T15:42:43.692 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:18.2.2-1343.g2965081f.el9.x86_64 2024-05-15T15:42:43.692 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:42:43.692 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-15T15:42:43.975 DEBUG:teuthology.parallel:result is None 2024-05-15T15:42:43.975 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-15T15:42:43.975 INFO:teuthology.packaging:ref: None 2024-05-15T15:42:43.975 INFO:teuthology.packaging:tag: None 2024-05-15T15:42:43.975 INFO:teuthology.packaging:branch: reef 2024-05-15T15:42:43.976 INFO:teuthology.packaging:sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:42:43.976 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-05-15T15:42:44.222 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-15T15:42:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:18.2.2-1343.g2965081f.el9 2024-05-15T15:42:44.250 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1343.g2965081f.el9 2024-05-15T15:42:44.250 INFO:teuthology.task.install:The correct ceph version 18.2.2-1343.g2965081f is installed. 2024-05-15T15:42:44.252 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-15T15:42:44.252 INFO:teuthology.packaging:ref: None 2024-05-15T15:42:44.252 INFO:teuthology.packaging:tag: None 2024-05-15T15:42:44.252 INFO:teuthology.packaging:branch: reef 2024-05-15T15:42:44.252 INFO:teuthology.packaging:sha1: de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:42:44.253 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-05-15T15:42:44.500 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-15T15:42:44.521 INFO:teuthology.orchestra.run.smithi144.stdout:18.2.2-1343.g2965081f.el9 2024-05-15T15:42:44.521 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1343.g2965081f.el9 2024-05-15T15:42:44.521 INFO:teuthology.task.install:The correct ceph version 18.2.2-1343.g2965081f is installed. 2024-05-15T15:42:44.523 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-15T15:42:44.523 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:42:44.524 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-15T15:42:44.555 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:42:44.555 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-15T15:42:44.587 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-15T15:42:44.588 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:42:44.588 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-05-15T15:42:44.625 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-15T15:42:44.693 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:42:44.694 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-05-15T15:42:44.718 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-15T15:42:44.782 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-15T15:42:44.782 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:42:44.782 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-15T15:42:44.811 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-15T15:42:44.878 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:42:44.879 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-15T15:42:44.903 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-15T15:42:44.968 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-15T15:42:44.968 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:42:44.968 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-05-15T15:42:44.996 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-15T15:42:45.063 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:42:45.063 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-05-15T15:42:45.088 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-15T15:42:45.153 INFO:teuthology.run_tasks:Running task print... 2024-05-15T15:42:45.172 INFO:teuthology.task.print:**** done install task... 2024-05-15T15:42:45.172 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-15T15:42:45.280 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': 'de5210d58560a4684fdb5b238ef3e6e0285a1de8'} 2024-05-15T15:42:45.280 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-05-15T15:42:45.281 INFO:tasks.cephadm:Cluster fsid is c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:42:45.281 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-15T15:42:45.281 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-05-15T15:42:45.281 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi049': '172.21.15.49', 'mon.smithi144': '172.21.15.144'} 2024-05-15T15:42:45.281 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-15T15:42:45.281 DEBUG:teuthology.orchestra.run.smithi049:> sudo hostname $(hostname -s) 2024-05-15T15:42:45.310 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-05-15T15:42:45.335 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-05-15T15:42:45.335 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:42:45.490 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'de5210d58560a4684fdb5b238ef3e6e0285a1de8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79397/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2576-gde5210d5', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2576.gde5210d5'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-65480-squid/de5210d58560a4684fdb5b238ef3e6e0285a1de8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-15 12:21:39.904261', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-65480-squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-65480-squid/de5210d58560a4684fdb5b238ef3e6e0285a1de8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-15T15:42:45.704 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 2965081fb1a343a09e00ceeede998525f5c6cb5e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-05-15T15:42:45.705 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/x86_64/flavors/default/cephadm 2024-05-15T15:42:45.706 DEBUG:teuthology.orchestra.run.smithi049:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-15T15:42:46.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-r--r--. 1 ubuntu ubuntu 215331 May 15 15:42 /home/ubuntu/cephtest/cephadm 2024-05-15T15:42:46.106 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-15T15:42:46.667 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r--. 1 ubuntu ubuntu 215331 May 15 15:42 /home/ubuntu/cephtest/cephadm 2024-05-15T15:42:46.667 DEBUG:teuthology.orchestra.run.smithi049:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-15T15:42:46.688 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-15T15:42:46.721 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-05-15T15:42:46.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-05-15T15:42:46.730 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-05-15T15:42:47.483 INFO:teuthology.orchestra.run.smithi049.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-05-15T15:42:47.637 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-05-15T15:43:19.611 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-15T15:43:19.611 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)", 2024-05-15T15:43:19.611 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "a04a73ffe07969b07057db75204012857aea1fdc1d3eeb4e5270ae40eb9fbe46", 2024-05-15T15:43:19.612 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-05-15T15:43:19.612 INFO:teuthology.orchestra.run.smithi144.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1" 2024-05-15T15:43:19.612 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-05-15T15:43:19.612 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-15T15:43:26.746 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:43:26.746 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)", 2024-05-15T15:43:26.746 INFO:teuthology.orchestra.run.smithi049.stdout: "image_id": "a04a73ffe07969b07057db75204012857aea1fdc1d3eeb4e5270ae40eb9fbe46", 2024-05-15T15:43:26.746 INFO:teuthology.orchestra.run.smithi049.stdout: "repo_digests": [ 2024-05-15T15:43:26.746 INFO:teuthology.orchestra.run.smithi049.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1" 2024-05-15T15:43:26.746 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-15T15:43:26.746 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:43:26.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph 2024-05-15T15:43:26.816 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-05-15T15:43:26.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /etc/ceph 2024-05-15T15:43:26.875 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-05-15T15:43:26.918 INFO:tasks.cephadm:Writing seed config... 2024-05-15T15:43:26.919 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-05-15T15:43:26.919 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-05-15T15:43:26.919 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-15T15:43:26.919 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-15T15:43:26.919 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-15T15:43:26.919 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-15T15:43:26.919 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-15T15:43:26.920 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-15T15:43:26.921 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-15T15:43:26.922 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-15T15:43:26.923 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:43:26.923 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-15T15:43:26.939 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c619ac44-12d1-11ef-bc98-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-15T15:43:26.940 DEBUG:teuthology.orchestra.run.smithi049:mon.smithi049> sudo journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049.service 2024-05-15T15:43:26.983 INFO:tasks.cephadm:Bootstrapping... 2024-05-15T15:43:26.984 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid c619ac44-12d1-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.49 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-15T15:43:27.144 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-15T15:43:27.144 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', 'c619ac44-12d1-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.49', '--skip-admin-label'] 2024-05-15T15:43:27.170 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 5.0.2 2024-05-15T15:43:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-15T15:43:27.170 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying podman|docker is present... 2024-05-15T15:43:27.194 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 5.0.2 2024-05-15T15:43:27.194 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying lvm2 is present... 2024-05-15T15:43:27.194 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying time synchronization is in place... 2024-05-15T15:43:27.202 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-15T15:43:27.202 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-15T15:43:27.208 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-15T15:43:27.208 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-05-15T15:43:27.215 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-05-15T15:43:27.222 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-05-15T15:43:27.222 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-05-15T15:43:27.222 INFO:teuthology.orchestra.run.smithi049.stdout:Repeating the final host check... 2024-05-15T15:43:27.246 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 5.0.2 2024-05-15T15:43:27.247 INFO:teuthology.orchestra.run.smithi049.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-15T15:43:27.247 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl is present 2024-05-15T15:43:27.247 INFO:teuthology.orchestra.run.smithi049.stdout:lvcreate is present 2024-05-15T15:43:27.253 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-15T15:43:27.254 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-15T15:43:27.260 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-15T15:43:27.260 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-05-15T15:43:27.266 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-05-15T15:43:27.273 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-05-15T15:43:27.273 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-05-15T15:43:27.273 INFO:teuthology.orchestra.run.smithi049.stdout:Host looks OK 2024-05-15T15:43:27.274 INFO:teuthology.orchestra.run.smithi049.stdout:Cluster fsid: c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:43:27.274 INFO:teuthology.orchestra.run.smithi049.stdout:Acquiring lock 139876603687216 on /run/cephadm/c619ac44-12d1-11ef-bc98-c7b262605968.lock 2024-05-15T15:43:27.274 INFO:teuthology.orchestra.run.smithi049.stdout:Lock 139876603687216 acquired on /run/cephadm/c619ac44-12d1-11ef-bc98-c7b262605968.lock 2024-05-15T15:43:27.274 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 3300 ... 2024-05-15T15:43:27.274 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 6789 ... 2024-05-15T15:43:27.274 INFO:teuthology.orchestra.run.smithi049.stdout:Base mon IP(s) is [172.21.15.49:3300, 172.21.15.49:6789], mon addrv is [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-05-15T15:43:27.278 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.49 metric 100 2024-05-15T15:43:27.278 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.49 metric 100 2024-05-15T15:43:27.280 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-15T15:43:27.280 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-15T15:43:27.280 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-15T15:43:27.283 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-15T15:43:27.283 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-15T15:43:27.283 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-15T15:43:27.283 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-15T15:43:27.283 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-05-15T15:43:27.283 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-15T15:43:27.284 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-05-15T15:43:27.284 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-05-15T15:43:27.284 INFO:teuthology.orchestra.run.smithi049.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-15T15:43:27.284 INFO:teuthology.orchestra.run.smithi049.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-15T15:43:27.285 INFO:teuthology.orchestra.run.smithi049.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-05-15T15:43:27.682 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout a04a73ffe07969b07057db75204012857aea1fdc1d3eeb4e5270ae40eb9fbe46 2024-05-15T15:43:27.682 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-05-15T15:43:27.682 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Getting image source signatures 2024-05-15T15:43:27.682 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:4a53206f1a70dca23a56afb8d3479f4bca659b8fe70577a61d0d3e0bd8d72af0 2024-05-15T15:43:27.682 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-15T15:43:27.682 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying config sha256:a04a73ffe07969b07057db75204012857aea1fdc1d3eeb4e5270ae40eb9fbe46 2024-05-15T15:43:27.682 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-15T15:43:28.661 INFO:teuthology.orchestra.run.smithi049.stdout:ceph: stdout ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable) 2024-05-15T15:43:28.662 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph version: ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable) 2024-05-15T15:43:28.662 INFO:teuthology.orchestra.run.smithi049.stdout:Extracting ceph user uid/gid from container image... 2024-05-15T15:43:29.501 INFO:teuthology.orchestra.run.smithi049.stdout:stat: stdout 167 167 2024-05-15T15:43:29.501 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial keys... 2024-05-15T15:43:30.430 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQAh2ERmS1NaMhAAXwEhirY7a2vM8ynVARjISw== 2024-05-15T15:43:31.419 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQAi2ERmrXaiLBAA+VR5mD/8wA2MCBGWUDHL7Q== 2024-05-15T15:43:32.325 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQAj2ERmKLLuLRAA9x67e/WRjZeREkZW7dUw0Q== 2024-05-15T15:43:32.325 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial monmap... 2024-05-15T15:43:33.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-15T15:43:33.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-15T15:43:33.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:43:33.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-15T15:43:33.298 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool for smithi049 [v2:172.21.15.49:3300,v1:172.21.15.49:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-15T15:43:33.298 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = pacific 2024-05-15T15:43:33.298 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: set fsid to c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:43:33.299 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-15T15:43:33.299 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:43:33.299 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mon... 2024-05-15T15:43:34.186 INFO:teuthology.orchestra.run.smithi049.stdout:create mon.smithi049 on 2024-05-15T15:43:34.402 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-15T15:43:34.547 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-15T15:43:34.720 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c619ac44-12d1-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-c619ac44-12d1-11ef-bc98-c7b262605968.target. 2024-05-15T15:43:34.720 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c619ac44-12d1-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-c619ac44-12d1-11ef-bc98-c7b262605968.target. 2024-05-15T15:43:34.943 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049 2024-05-15T15:43:34.944 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049.service: Unit ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049.service not loaded. 2024-05-15T15:43:35.111 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c619ac44-12d1-11ef-bc98-c7b262605968.target.wants/ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049.service → /etc/systemd/system/ceph-c619ac44-12d1-11ef-bc98-c7b262605968@.service. 2024-05-15T15:43:35.686 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-15T15:43:35.686 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-05-15T15:43:35.694 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-15T15:43:35.694 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-05-15T15:43:35.694 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-05-15T15:43:35.694 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-05-15T15:43:35.694 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon to start... 2024-05-15T15:43:35.695 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon... 2024-05-15T15:43:35.820 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:35 smithi049 systemd[1]: Started Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:43:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout cluster: 2024-05-15T15:43:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout id: c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:43:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-15T15:43:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout services: 2024-05-15T15:43:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi049 (age 0.575791s) 2024-05-15T15:43:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout data: 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pgs: 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.141+0000 7f537ad21700 1 Processor -- start 2024-05-15T15:43:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.142+0000 7f537ad21700 1 -- start start 2024-05-15T15:43:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.142+0000 7f537ad21700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374109530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.142+0000 7f537ad21700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5374062280 con 0x7f5374107140 2024-05-15T15:43:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.142+0000 7f5378abd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.142+0000 7f5378abd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374109530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45374/0 (socket says 172.21.15.49:45374) 2024-05-15T15:43:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.142+0000 7f5378abd700 1 -- 172.21.15.49:0/3345952238 learned_addr learned my addr 172.21.15.49:0/3345952238 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.143+0000 7f5378abd700 1 -- 172.21.15.49:0/3345952238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374109a70 con 0x7f5374107140 2024-05-15T15:43:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.143+0000 7f5378abd700 1 --2- 172.21.15.49:0/3345952238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374109530 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f53680098a0 tx=0x7f5368009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9d1cb08b1af84c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.143+0000 7f53737fe700 1 -- 172.21.15.49:0/3345952238 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5368004180 con 0x7f5374107140 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.143+0000 7f53737fe700 1 -- 172.21.15.49:0/3345952238 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53680042e0 con 0x7f5374107140 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.143+0000 7f53737fe700 1 -- 172.21.15.49:0/3345952238 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53680036a0 con 0x7f5374107140 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.144+0000 7f537ad21700 1 -- 172.21.15.49:0/3345952238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 msgr2=0x7f5374109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.144+0000 7f537ad21700 1 --2- 172.21.15.49:0/3345952238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374109530 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f53680098a0 tx=0x7f5368009bb0 comp rx=0 tx=0).stop 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.144+0000 7f537ad21700 1 -- 172.21.15.49:0/3345952238 shutdown_connections 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.144+0000 7f537ad21700 1 --2- 172.21.15.49:0/3345952238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374109530 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.144+0000 7f537ad21700 1 -- 172.21.15.49:0/3345952238 >> 172.21.15.49:0/3345952238 conn(0x7f5374100af0 msgr2=0x7f5374102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.144+0000 7f537ad21700 1 -- 172.21.15.49:0/3345952238 shutdown_connections 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.144+0000 7f537ad21700 1 -- 172.21.15.49:0/3345952238 wait complete. 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.144+0000 7f537ad21700 1 Processor -- start 2024-05-15T15:43:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.144+0000 7f537ad21700 1 -- start start 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f537ad21700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f537ad21700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537419a090 con 0x7f5374107140 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f5378abd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f5378abd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45378/0 (socket says 172.21.15.49:45378) 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f5378abd700 1 -- 172.21.15.49:0/2860387784 learned_addr learned my addr 172.21.15.49:0/2860387784 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f5378abd700 1 -- 172.21.15.49:0/2860387784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368009550 con 0x7f5374107140 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f5378abd700 1 --2- 172.21.15.49:0/2860387784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374199b50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f53740616c0 tx=0x7f5368004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f5371ffb700 1 -- 172.21.15.49:0/2860387784 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5368003ce0 con 0x7f5374107140 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f5371ffb700 1 -- 172.21.15.49:0/2860387784 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5368020070 con 0x7f5374107140 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.145+0000 7f537ad21700 1 -- 172.21.15.49:0/2860387784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537419a290 con 0x7f5374107140 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.146+0000 7f5371ffb700 1 -- 172.21.15.49:0/2860387784 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5368011630 con 0x7f5374107140 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.146+0000 7f537ad21700 1 -- 172.21.15.49:0/2860387784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537419a730 con 0x7f5374107140 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.146+0000 7f5371ffb700 1 -- 172.21.15.49:0/2860387784 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5368022030 con 0x7f5374107140 2024-05-15T15:43:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.146+0000 7f5371ffb700 1 -- 172.21.15.49:0/2860387784 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5368025d40 con 0x7f5374107140 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.147+0000 7f537ad21700 1 -- 172.21.15.49:0/2860387784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5374193990 con 0x7f5374107140 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.149+0000 7f5371ffb700 1 -- 172.21.15.49:0/2860387784 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f5368025480 con 0x7f5374107140 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.222+0000 7f537ad21700 1 -- 172.21.15.49:0/2860387784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5374109f90 con 0x7f5374107140 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.223+0000 7f5371ffb700 1 -- 172.21.15.49:0/2860387784 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f5368011bd0 con 0x7f5374107140 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.223+0000 7f537ad21700 1 -- 172.21.15.49:0/2860387784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 msgr2=0x7f5374199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.223+0000 7f537ad21700 1 --2- 172.21.15.49:0/2860387784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374199b50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f53740616c0 tx=0x7f5368004750 comp rx=0 tx=0).stop 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.223+0000 7f537ad21700 1 -- 172.21.15.49:0/2860387784 shutdown_connections 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.223+0000 7f537ad21700 1 --2- 172.21.15.49:0/2860387784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5374107140 0x7f5374199b50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.224+0000 7f537ad21700 1 -- 172.21.15.49:0/2860387784 >> 172.21.15.49:0/2860387784 conn(0x7f5374100af0 msgr2=0x7f5374100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.224+0000 7f537ad21700 1 -- 172.21.15.49:0/2860387784 shutdown_connections 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:36.224+0000 7f537ad21700 1 -- 172.21.15.49:0/2860387784 wait complete. 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:mon is available 2024-05-15T15:43:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:Assimilating anything we can from ceph.conf... 2024-05-15T15:43:36.953 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:36 smithi049 ceph-mon[28024]: mon.smithi049 is new leader, mons smithi049 in quorum (ranks 0) 2024-05-15T15:43:36.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:36 smithi049 ceph-mon[28024]: monmap e1: 1 mons at {smithi049=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-15T15:43:36.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:36 smithi049 ceph-mon[28024]: fsmap 2024-05-15T15:43:36.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:36 smithi049 ceph-mon[28024]: osdmap e1: 0 total, 0 up, 0 in 2024-05-15T15:43:36.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:36 smithi049 ceph-mon[28024]: mgrmap e1: no daemons active 2024-05-15T15:43:36.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:36 smithi049 ceph-mon[28024]: from='client.? 172.21.15.49:0/2860387784' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-15T15:43:37.809 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:37 smithi049 ceph-mon[28024]: from='client.? 172.21.15.49:0/3403695177' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-15T15:43:37.809 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:37 smithi049 ceph-mon[28024]: from='client.? 172.21.15.49:0/3403695177' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-15T15:43:37.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:37.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.138+0000 7fc0fa106700 1 Processor -- start 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.138+0000 7fc0fa106700 1 -- start start 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.138+0000 7fc0fa106700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4106b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.138+0000 7fc0fa106700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f41070d0 con 0x7fc0f41066e0 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.138+0000 7fc0f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4106b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.139+0000 7fc0f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4106b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45390/0 (socket says 172.21.15.49:45390) 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.139+0000 7fc0f37fe700 1 -- 172.21.15.49:0/3787234201 learned_addr learned my addr 172.21.15.49:0/3787234201 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.139+0000 7fc0f37fe700 1 -- 172.21.15.49:0/3787234201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4107890 con 0x7fc0f41066e0 2024-05-15T15:43:37.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.139+0000 7fc0f37fe700 1 --2- 172.21.15.49:0/3787234201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4106b00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40098a0 tx=0x7fc0e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6307168c4097d64a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.139+0000 7fc0f27fc700 1 -- 172.21.15.49:0/3787234201 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0e4004180 con 0x7fc0f41066e0 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.139+0000 7fc0f27fc700 1 -- 172.21.15.49:0/3787234201 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e40042e0 con 0x7fc0f41066e0 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.139+0000 7fc0f27fc700 1 -- 172.21.15.49:0/3787234201 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0e40036a0 con 0x7fc0f41066e0 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.140+0000 7fc0fa106700 1 -- 172.21.15.49:0/3787234201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 msgr2=0x7fc0f4106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.140+0000 7fc0fa106700 1 --2- 172.21.15.49:0/3787234201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4106b00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40098a0 tx=0x7fc0e4009bb0 comp rx=0 tx=0).stop 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.140+0000 7fc0fa106700 1 -- 172.21.15.49:0/3787234201 shutdown_connections 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.140+0000 7fc0fa106700 1 --2- 172.21.15.49:0/3787234201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4106b00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.140+0000 7fc0fa106700 1 -- 172.21.15.49:0/3787234201 >> 172.21.15.49:0/3787234201 conn(0x7fc0f4101ce0 msgr2=0x7fc0f4104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.140+0000 7fc0fa106700 1 -- 172.21.15.49:0/3787234201 shutdown_connections 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.140+0000 7fc0fa106700 1 -- 172.21.15.49:0/3787234201 wait complete. 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.140+0000 7fc0fa106700 1 Processor -- start 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0fa106700 1 -- start start 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0fa106700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:37.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0fa106700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f4199fb0 con 0x7fc0f41066e0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45404/0 (socket says 172.21.15.49:45404) 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0f37fe700 1 -- 172.21.15.49:0/3403695177 learned_addr learned my addr 172.21.15.49:0/3403695177 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0f37fe700 1 -- 172.21.15.49:0/3403695177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4009550 con 0x7fc0f41066e0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0f37fe700 1 --2- 172.21.15.49:0/3403695177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4199a70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004000 tx=0x7fc0e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0f0ff9700 1 -- 172.21.15.49:0/3403695177 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0e4003ce0 con 0x7fc0f41066e0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.141+0000 7fc0f0ff9700 1 -- 172.21.15.49:0/3403695177 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e4020070 con 0x7fc0f41066e0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.142+0000 7fc0fa106700 1 -- 172.21.15.49:0/3403695177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f419a1b0 con 0x7fc0f41066e0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.142+0000 7fc0f0ff9700 1 -- 172.21.15.49:0/3403695177 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0e4011630 con 0x7fc0f41066e0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.142+0000 7fc0fa106700 1 -- 172.21.15.49:0/3403695177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f419a650 con 0x7fc0f41066e0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.142+0000 7fc0f0ff9700 1 -- 172.21.15.49:0/3403695177 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fc0e4022030 con 0x7fc0f41066e0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.142+0000 7fc0f0ff9700 1 -- 172.21.15.49:0/3403695177 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc0e4025d40 con 0x7fc0f41066e0 2024-05-15T15:43:37.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.143+0000 7fc0fa106700 1 -- 172.21.15.49:0/3403695177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f41937c0 con 0x7fc0f41066e0 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.145+0000 7fc0f0ff9700 1 -- 172.21.15.49:0/3403695177 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fc0e4025480 con 0x7fc0f41066e0 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.216+0000 7fc0fa106700 1 -- 172.21.15.49:0/3403695177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc0f4107e70 con 0x7fc0f41066e0 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.220+0000 7fc0f0ff9700 1 -- 172.21.15.49:0/3403695177 <== mon.0 v2:172.21.15.49:3300/0 7 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc0e4011bd0 con 0x7fc0f41066e0 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.220+0000 7fc0f0ff9700 1 -- 172.21.15.49:0/3403695177 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fc0e4037590 con 0x7fc0f41066e0 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.221+0000 7fc0fa106700 1 -- 172.21.15.49:0/3403695177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 msgr2=0x7fc0f4199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.221+0000 7fc0fa106700 1 --2- 172.21.15.49:0/3403695177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4199a70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004000 tx=0x7fc0e4004750 comp rx=0 tx=0).stop 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.221+0000 7fc0fa106700 1 -- 172.21.15.49:0/3403695177 shutdown_connections 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.221+0000 7fc0fa106700 1 --2- 172.21.15.49:0/3403695177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0f41066e0 0x7fc0f4199a70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.221+0000 7fc0fa106700 1 -- 172.21.15.49:0/3403695177 >> 172.21.15.49:0/3403695177 conn(0x7fc0f4101ce0 msgr2=0x7fc0f4190900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.221+0000 7fc0fa106700 1 -- 172.21.15.49:0/3403695177 shutdown_connections 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:37.221+0000 7fc0fa106700 1 -- 172.21.15.49:0/3403695177 wait complete. 2024-05-15T15:43:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:Generating new minimal ceph.conf... 2024-05-15T15:43:38.788 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.179+0000 7f3fa7a1b700 1 Processor -- start 2024-05-15T15:43:38.788 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.180+0000 7f3fa7a1b700 1 -- start start 2024-05-15T15:43:38.788 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.180+0000 7f3fa7a1b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa0106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.180+0000 7f3fa7a1b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa0107050 con 0x7f3fa0106660 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.180+0000 7f3fa57b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa0106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.180+0000 7f3fa57b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa0106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45408/0 (socket says 172.21.15.49:45408) 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.180+0000 7f3fa57b7700 1 -- 172.21.15.49:0/2672870229 learned_addr learned my addr 172.21.15.49:0/2672870229 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.180+0000 7f3fa57b7700 1 -- 172.21.15.49:0/2672870229 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0107860 con 0x7f3fa0106660 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.180+0000 7f3fa57b7700 1 --2- 172.21.15.49:0/2672870229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa0106a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0098a0 tx=0x7f3f9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcba1da84b360c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3f97fff700 1 -- 172.21.15.49:0/2672870229 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f9c004180 con 0x7f3fa0106660 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3f97fff700 1 -- 172.21.15.49:0/2672870229 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3f9c0042e0 con 0x7f3fa0106660 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3f97fff700 1 -- 172.21.15.49:0/2672870229 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f9c011a70 con 0x7f3fa0106660 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/2672870229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 msgr2=0x7f3fa0106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3fa7a1b700 1 --2- 172.21.15.49:0/2672870229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa0106a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0098a0 tx=0x7f3f9c009bb0 comp rx=0 tx=0).stop 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/2672870229 shutdown_connections 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3fa7a1b700 1 --2- 172.21.15.49:0/2672870229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa0106a80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:38.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/2672870229 >> 172.21.15.49:0/2672870229 conn(0x7f3fa0101c40 msgr2=0x7f3fa0104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/2672870229 shutdown_connections 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.181+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/2672870229 wait complete. 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.182+0000 7f3fa7a1b700 1 Processor -- start 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.182+0000 7f3fa7a1b700 1 -- start start 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.182+0000 7f3fa7a1b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.182+0000 7f3fa7a1b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa019a840 con 0x7f3fa0106660 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.182+0000 7f3fa57b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.182+0000 7f3fa57b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45418/0 (socket says 172.21.15.49:45418) 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.182+0000 7f3fa57b7700 1 -- 172.21.15.49:0/504610158 learned_addr learned my addr 172.21.15.49:0/504610158 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.182+0000 7f3fa57b7700 1 -- 172.21.15.49:0/504610158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c009550 con 0x7f3fa0106660 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.183+0000 7f3fa57b7700 1 --2- 172.21.15.49:0/504610158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa019a300 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c011710 tx=0x7f3f9c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.183+0000 7f3f967fc700 1 -- 172.21.15.49:0/504610158 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f9c003b10 con 0x7f3fa0106660 2024-05-15T15:43:38.790 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.183+0000 7f3f967fc700 1 -- 172.21.15.49:0/504610158 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3f9c01b8c0 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.183+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/504610158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa019aa40 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.183+0000 7f3f967fc700 1 -- 172.21.15.49:0/504610158 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f9c019710 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.183+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/504610158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa019aee0 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.183+0000 7f3f967fc700 1 -- 172.21.15.49:0/504610158 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f3f9c019be0 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.183+0000 7f3f967fc700 1 -- 172.21.15.49:0/504610158 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3f9c01a710 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.184+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/504610158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa00625d0 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.186+0000 7f3f967fc700 1 -- 172.21.15.49:0/504610158 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f3f9c01ba30 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.256+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/504610158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3fa010aa90 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.256+0000 7f3f967fc700 1 -- 172.21.15.49:0/504610158 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f3f9c01bbd0 con 0x7f3fa0106660 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.257+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/504610158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 msgr2=0x7f3fa019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.257+0000 7f3fa7a1b700 1 --2- 172.21.15.49:0/504610158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa019a300 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c011710 tx=0x7f3f9c0047c0 comp rx=0 tx=0).stop 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.257+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/504610158 shutdown_connections 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.257+0000 7f3fa7a1b700 1 --2- 172.21.15.49:0/504610158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fa0106660 0x7f3fa019a300 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.257+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/504610158 >> 172.21.15.49:0/504610158 conn(0x7f3fa0101c40 msgr2=0x7f3fa0102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:38.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.257+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/504610158 shutdown_connections 2024-05-15T15:43:38.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:38.257+0000 7f3fa7a1b700 1 -- 172.21.15.49:0/504610158 wait complete. 2024-05-15T15:43:38.792 INFO:teuthology.orchestra.run.smithi049.stdout:Restarting the monitor... 2024-05-15T15:43:38.816 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:38 smithi049 ceph-mon[28024]: from='client.? 172.21.15.49:0/504610158' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:43:39.077 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:38 smithi049 systemd[1]: Stopping Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T15:43:39.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:39 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[28015]: 2024-05-15T15:43:39.075+0000 7ffbf1467700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi049 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T15:43:39.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:39 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[28015]: 2024-05-15T15:43:39.075+0000 7ffbf1467700 -1 mon.smithi049@0(leader) e1 *** Got Signal Terminated *** 2024-05-15T15:43:39.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:39 smithi049 podman[28273]: 2024-05-15 15:43:39.259792339 +0000 UTC m=+0.294615854 container died 54282dbbcac62490bdb1a3f56e52591ca01f9c44601930c484b2f789d827675a (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-2965081, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-15T15:43:39.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:39 smithi049 podman[28273]: 2024-05-15 15:43:39.724666833 +0000 UTC m=+0.759490347 container cleanup 54282dbbcac62490bdb1a3f56e52591ca01f9c44601930c484b2f789d827675a (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=reef-2965081, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-15T15:43:39.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:39 smithi049 bash[28273]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049 2024-05-15T15:43:40.277 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:39 smithi049 podman[28285]: 2024-05-15 15:43:39.967304078 +0000 UTC m=+0.705860023 container remove 54282dbbcac62490bdb1a3f56e52591ca01f9c44601930c484b2f789d827675a (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-2965081, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-15T15:43:40.277 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049.service: Deactivated successfully. 2024-05-15T15:43:40.277 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 systemd[1]: Stopped Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:43:40.277 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 systemd[1]: Starting Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T15:43:40.538 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 podman[28381]: 2024-05-15 15:43:40.276383477 +0000 UTC m=+0.084004551 container create 73d4d6879c0ceb100ce0513a9453332c17a54be572b3a38785a4cd10c6cbaf49 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=reef-2965081, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 podman[28381]: 2024-05-15 15:43:40.209836707 +0000 UTC m=+0.017457794 image pull a04a73ffe07969b07057db75204012857aea1fdc1d3eeb4e5270ae40eb9fbe46 quay.ceph.io/ceph-ci/ceph:reef 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 podman[28381]: 2024-05-15 15:43:40.453537575 +0000 UTC m=+0.261158644 container init 73d4d6879c0ceb100ce0513a9453332c17a54be572b3a38785a4cd10c6cbaf49 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=reef-2965081, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 podman[28381]: 2024-05-15 15:43:40.456713761 +0000 UTC m=+0.264334841 container start 73d4d6879c0ceb100ce0513a9453332c17a54be572b3a38785a4cd10c6cbaf49 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=reef-2965081, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: set uid:gid to 167:167 (ceph:ceph) 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable), process ceph-mon, pid 2 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: pidfile_write: ignore empty --pid-file 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: load: jerasure load: lrc 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: RocksDB version: 7.9.2 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Git sha 0 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Compile date 2024-05-13 17:32:21 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: DB SUMMARY 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: DB Session ID: 16GGMGHSVQP46KRHHHJD 2024-05-15T15:43:40.539 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: CURRENT file: CURRENT 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: IDENTITY file: IDENTITY 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi049/store.db dir, Total Num: 1, files: 000008.sst 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi049/store.db: 000009.log size: 98483 ; 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.error_if_exists: 0 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.create_if_missing: 0 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.paranoid_checks: 1 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.env: 0x564b9b8b8820 2024-05-15T15:43:40.540 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.fs: PosixFileSystem 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.info_log: 0x564b9d7193e0 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_file_opening_threads: 16 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.statistics: (nil) 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.use_fsync: 0 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_log_file_size: 0 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.keep_log_file_num: 1000 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.recycle_log_file_num: 0 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.allow_fallocate: 1 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.allow_mmap_reads: 0 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.allow_mmap_writes: 0 2024-05-15T15:43:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.use_direct_reads: 0 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.create_missing_column_families: 0 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.db_log_dir: 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.wal_dir: 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.advise_random_on_open: 1 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.db_write_buffer_size: 0 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.write_buffer_manager: 0x564b9c98e140 2024-05-15T15:43:40.542 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-15T15:43:40.543 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-15T15:43:40.543 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-15T15:43:40.543 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.rate_limiter: (nil) 2024-05-15T15:43:40.543 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-15T15:43:40.543 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.wal_recovery_mode: 2 2024-05-15T15:43:40.543 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.enable_thread_tracking: 0 2024-05-15T15:43:40.543 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.enable_pipelined_write: 0 2024-05-15T15:43:40.544 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.unordered_write: 0 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.row_cache: None 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.wal_filter: None 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.allow_ingest_behind: 0 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.two_write_queues: 0 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.manual_wal_flush: 0 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.wal_compression: 0 2024-05-15T15:43:40.545 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.atomic_flush: 0 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.log_readahead_size: 0 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.best_efforts_recovery: 0 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.allow_data_in_errors: 0 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.db_host_id: __hostname__ 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_background_jobs: 2 2024-05-15T15:43:40.546 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_background_compactions: -1 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_subcompactions: 1 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_total_wal_size: 0 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_open_files: -1 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bytes_per_sync: 0 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-15T15:43:40.547 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_readahead_size: 0 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_background_flushes: -1 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Compression algorithms supported: 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: kZSTD supported: 0 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: kXpressCompression supported: 0 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: kLZ4HCCompression supported: 1 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: kZlibCompression supported: 1 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: kSnappyCompression supported: 1 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: kLZ4Compression supported: 1 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: kBZip2Compression supported: 0 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi049/store.db/MANIFEST-000010 2024-05-15T15:43:40.548 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.merge_operator: 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_filter: None 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_filter_factory: None 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.sst_partitioner_factory: None 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564b9d719500) 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: cache_index_and_filter_blocks: 1 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: pin_top_level_index_and_filter: 1 2024-05-15T15:43:40.549 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: index_type: 0 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: data_block_index_type: 0 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: index_shortening: 1 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: checksum: 4 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: no_block_cache: 0 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_cache: 0x564b9ca311f0 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_cache_name: BinnedLRUCache 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_cache_options: 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: capacity : 536870912 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: num_shard_bits : 4 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: strict_capacity_limit : 0 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: high_pri_pool_ratio: 0.000 2024-05-15T15:43:40.550 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_cache_compressed: (nil) 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: persistent_cache: (nil) 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_size: 4096 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_size_deviation: 10 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_restart_interval: 16 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: index_block_restart_interval: 1 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: metadata_block_size: 4096 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: partition_filters: 0 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: use_delta_encoding: 1 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: filter_policy: bloomfilter 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: whole_key_filtering: 1 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: verify_compression: 0 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: read_amp_bytes_per_bit: 0 2024-05-15T15:43:40.551 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: format_version: 5 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: enable_index_compression: 1 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_align: 0 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: max_auto_readahead_size: 262144 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: prepopulate_block_cache: 0 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: initial_auto_readahead_size: 8192 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.write_buffer_size: 33554432 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_write_buffer_number: 2 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression: NoCompression 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression: Disabled 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.prefix_extractor: nullptr 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.num_levels: 7 2024-05-15T15:43:40.552 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression_opts.level: 32767 2024-05-15T15:43:40.553 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression_opts.strategy: 0 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression_opts.enabled: false 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.target_file_size_base: 67108864 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-15T15:43:40.554 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.arena_block_size: 1048576 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-15T15:43:40.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.disable_auto_compactions: 0 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.table_properties_collectors: 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.inplace_update_support: 0 2024-05-15T15:43:40.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.bloom_locality: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.max_successive_merges: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.paranoid_file_checks: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.force_consistency_checks: 1 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.report_bg_io_stats: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.ttl: 2592000 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-15T15:43:40.557 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.enable_blob_files: false 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.min_blob_size: 0 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.blob_file_size: 268435456 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.blob_file_starting_level: 0 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi049/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-15T15:43:40.558 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 454e95b1-b7c0-4b85-a0b8-7f5a235bc408 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715787820485945, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715787820487077, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93760, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91794, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84358, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715787820, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "454e95b1-b7c0-4b85-a0b8-7f5a235bc408", "db_session_id": "16GGMGHSVQP46KRHHHJD", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715787820487162, "job": 1, "event": "recovery_finished"} 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi049/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564b9cace000 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: DB pointer 0x564b9cabc000 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ** DB Stats ** 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-15T15:43:40.559 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ** Compaction Stats [default] ** 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: L0 2/0 93.44 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 124.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Sum 2/0 93.44 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 124.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T15:43:40.560 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 124.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ** Compaction Stats [default] ** 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 124.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-15T15:43:40.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Cumulative compaction: 0.00 GB write, 27.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Interval compaction: 0.00 GB write, 27.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Block cache BinnedLRUCache@0x564b9ca311f0#2 capacity: 512.00 MB usage: 1.33 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: starting mon.smithi049 rank 0 at public addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] at bind addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi049 fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mon.smithi049@-1(???) e1 preinit fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mon.smithi049@-1(???).mds e1 new map 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mon.smithi049@-1(???).mds e1 print_map 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: e1 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:43:40.562 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: legacy client fscid: -1 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: No filesystems configured 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mon.smithi049@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mon.smithi049@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mon.smithi049@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mon.smithi049@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mon.smithi049@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 bash[28381]: 73d4d6879c0ceb100ce0513a9453332c17a54be572b3a38785a4cd10c6cbaf49 2024-05-15T15:43:40.563 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 systemd[1]: Started Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:43:40.563 INFO:teuthology.orchestra.run.smithi049.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-15T15:43:40.875 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mon.smithi049 is new leader, mons smithi049 in quorum (ranks 0) 2024-05-15T15:43:40.875 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: monmap e1: 1 mons at {smithi049=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-15T15:43:40.875 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: fsmap 2024-05-15T15:43:40.875 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: osdmap e1: 0 total, 0 up, 0 in 2024-05-15T15:43:40.875 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:40 smithi049 ceph-mon[28399]: mgrmap e1: no daemons active 2024-05-15T15:43:41.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.980+0000 7f937b474700 1 Processor -- start 2024-05-15T15:43:41.726 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.980+0000 7f937b474700 1 -- start start 2024-05-15T15:43:41.726 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.980+0000 7f937b474700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f9374105990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:41.726 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.980+0000 7f937b474700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9374105f60 con 0x7f9374105570 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.981+0000 7f9379210700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f9374105990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.981+0000 7f9379210700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f9374105990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45426/0 (socket says 172.21.15.49:45426) 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.981+0000 7f9379210700 1 -- 172.21.15.49:0/4173489015 learned_addr learned my addr 172.21.15.49:0/4173489015 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.981+0000 7f9379210700 1 -- 172.21.15.49:0/4173489015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9374106770 con 0x7f9374105570 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.981+0000 7f9379210700 1 --2- 172.21.15.49:0/4173489015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f9374105990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f93700098a0 tx=0x7f9370009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7655c17996ccdb3f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.981+0000 7f936bfff700 1 -- 172.21.15.49:0/4173489015 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9370004180 con 0x7f9374105570 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.981+0000 7f936bfff700 1 -- 172.21.15.49:0/4173489015 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f93700042e0 con 0x7f9374105570 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.981+0000 7f936bfff700 1 -- 172.21.15.49:0/4173489015 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9370011a70 con 0x7f9374105570 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.982+0000 7f937b474700 1 -- 172.21.15.49:0/4173489015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 msgr2=0x7f9374105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.982+0000 7f937b474700 1 --2- 172.21.15.49:0/4173489015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f9374105990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f93700098a0 tx=0x7f9370009bb0 comp rx=0 tx=0).stop 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.982+0000 7f937b474700 1 -- 172.21.15.49:0/4173489015 shutdown_connections 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.982+0000 7f937b474700 1 --2- 172.21.15.49:0/4173489015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f9374105990 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:41.727 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.982+0000 7f937b474700 1 -- 172.21.15.49:0/4173489015 >> 172.21.15.49:0/4173489015 conn(0x7f9374100b10 msgr2=0x7f9374102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.982+0000 7f937b474700 1 -- 172.21.15.49:0/4173489015 shutdown_connections 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.982+0000 7f937b474700 1 -- 172.21.15.49:0/4173489015 wait complete. 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f937b474700 1 Processor -- start 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f937b474700 1 -- start start 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f937b474700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f93740799f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f937b474700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9374079f30 con 0x7f9374105570 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f9379210700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f93740799f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f9379210700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f93740799f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45432/0 (socket says 172.21.15.49:45432) 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f9379210700 1 -- 172.21.15.49:0/3938668785 learned_addr learned my addr 172.21.15.49:0/3938668785 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f9379210700 1 -- 172.21.15.49:0/3938668785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370009550 con 0x7f9374105570 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f9379210700 1 --2- 172.21.15.49:0/3938668785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f93740799f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9370011710 tx=0x7f93700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.983+0000 7f936a7fc700 1 -- 172.21.15.49:0/3938668785 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9370003b10 con 0x7f9374105570 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.984+0000 7f936a7fc700 1 -- 172.21.15.49:0/3938668785 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f937001b8c0 con 0x7f9374105570 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.984+0000 7f937b474700 1 -- 172.21.15.49:0/3938668785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937419d050 con 0x7f9374105570 2024-05-15T15:43:41.728 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.984+0000 7f936a7fc700 1 -- 172.21.15.49:0/3938668785 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9370019710 con 0x7f9374105570 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.984+0000 7f937b474700 1 -- 172.21.15.49:0/3938668785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937419d490 con 0x7f9374105570 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.984+0000 7f936a7fc700 1 -- 172.21.15.49:0/3938668785 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9370019c90 con 0x7f9374105570 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.984+0000 7f936a7fc700 1 -- 172.21.15.49:0/3938668785 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f937001a710 con 0x7f9374105570 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.985+0000 7f937b474700 1 -- 172.21.15.49:0/3938668785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93741099a0 con 0x7f9374105570 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:40.987+0000 7f936a7fc700 1 -- 172.21.15.49:0/3938668785 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f937001ba30 con 0x7f9374105570 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:41.057+0000 7f937b474700 1 -- 172.21.15.49:0/3938668785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f937419d6d0 con 0x7f9374105570 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:41.059+0000 7f936a7fc700 1 -- 172.21.15.49:0/3938668785 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f93700300c0 con 0x7f9374105570 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:41.060+0000 7f937b474700 1 -- 172.21.15.49:0/3938668785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 msgr2=0x7f93740799f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:41.060+0000 7f937b474700 1 --2- 172.21.15.49:0/3938668785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f93740799f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9370011710 tx=0x7f93700047c0 comp rx=0 tx=0).stop 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:41.060+0000 7f937b474700 1 -- 172.21.15.49:0/3938668785 shutdown_connections 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:41.060+0000 7f937b474700 1 --2- 172.21.15.49:0/3938668785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9374105570 0x7f93740799f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:41.060+0000 7f937b474700 1 -- 172.21.15.49:0/3938668785 >> 172.21.15.49:0/3938668785 conn(0x7f9374100b10 msgr2=0x7f93741015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:41.060+0000 7f937b474700 1 -- 172.21.15.49:0/3938668785 shutdown_connections 2024-05-15T15:43:41.729 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:41.060+0000 7f937b474700 1 -- 172.21.15.49:0/3938668785 wait complete. 2024-05-15T15:43:41.730 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-15T15:43:41.744 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-15T15:43:41.745 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mgr... 2024-05-15T15:43:41.745 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-15T15:43:41.745 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-15T15:43:41.745 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:8443 ... 2024-05-15T15:43:41.990 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mgr.smithi049.vuhzgf 2024-05-15T15:43:41.990 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mgr.smithi049.vuhzgf.service: Unit ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mgr.smithi049.vuhzgf.service not loaded. 2024-05-15T15:43:42.146 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c619ac44-12d1-11ef-bc98-c7b262605968.target.wants/ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mgr.smithi049.vuhzgf.service → /etc/systemd/system/ceph-c619ac44-12d1-11ef-bc98-c7b262605968@.service. 2024-05-15T15:43:42.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:42 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3938668785' entity='client.admin' 2024-05-15T15:43:42.671 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-15T15:43:42.671 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-05-15T15:43:42.679 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-15T15:43:42.679 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-05-15T15:43:42.679 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-05-15T15:43:42.680 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-05-15T15:43:42.688 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-15T15:43:42.688 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-05-15T15:43:42.696 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-15T15:43:42.696 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-05-15T15:43:42.696 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-05-15T15:43:42.696 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-05-15T15:43:42.696 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr to start... 2024-05-15T15:43:42.696 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr... 2024-05-15T15:43:43.381 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:43 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/4229881493' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-15T15:43:43.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:43.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-05-15T15:43:43.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "c619ac44-12d1-11ef-bc98-c7b262605968", 2024-05-15T15:43:43.885 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-15T15:43:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "smithi049" 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-15T15:43:43.889 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-15T15:43:43.890 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-15T15:43:43.890 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:43.890 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-15T15:43:43.897 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-15T15:43:43.898 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-15T15:43:43.898 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-15T15:43:43.898 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-15T15:43:43.898 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-15T15:43:43.898 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-15T15:43:43.898 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-15T15:43:43.898 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-15T15:43:43.898 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:43.898 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:43.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-05-15T15:43:35.648676+0000", 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.146+0000 7fdc33707700 1 Processor -- start 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.146+0000 7fdc33707700 1 -- start start 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.146+0000 7fdc33707700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c109550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.146+0000 7fdc33707700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c079e20 con 0x7fdc2c107160 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.146+0000 7fdc314a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:43.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.146+0000 7fdc314a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c109550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45472/0 (socket says 172.21.15.49:45472) 2024-05-15T15:43:43.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.146+0000 7fdc314a3700 1 -- 172.21.15.49:0/2844396572 learned_addr learned my addr 172.21.15.49:0/2844396572 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:43.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.147+0000 7fdc314a3700 1 -- 172.21.15.49:0/2844396572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c109a90 con 0x7fdc2c107160 2024-05-15T15:43:43.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.147+0000 7fdc314a3700 1 --2- 172.21.15.49:0/2844396572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c109550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdc280098a0 tx=0x7fdc28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cc9193e5a17be1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:43.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.147+0000 7fdc23fff700 1 -- 172.21.15.49:0/2844396572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc28004180 con 0x7fdc2c107160 2024-05-15T15:43:43.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.147+0000 7fdc23fff700 1 -- 172.21.15.49:0/2844396572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdc280042e0 con 0x7fdc2c107160 2024-05-15T15:43:43.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.147+0000 7fdc23fff700 1 -- 172.21.15.49:0/2844396572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc28011a70 con 0x7fdc2c107160 2024-05-15T15:43:43.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.148+0000 7fdc33707700 1 -- 172.21.15.49:0/2844396572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 msgr2=0x7fdc2c109550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:43.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.148+0000 7fdc33707700 1 --2- 172.21.15.49:0/2844396572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c109550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdc280098a0 tx=0x7fdc28009bb0 comp rx=0 tx=0).stop 2024-05-15T15:43:43.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.148+0000 7fdc33707700 1 -- 172.21.15.49:0/2844396572 shutdown_connections 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.148+0000 7fdc33707700 1 --2- 172.21.15.49:0/2844396572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c109550 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.148+0000 7fdc33707700 1 -- 172.21.15.49:0/2844396572 >> 172.21.15.49:0/2844396572 conn(0x7fdc2c100b10 msgr2=0x7fdc2c102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.148+0000 7fdc33707700 1 -- 172.21.15.49:0/2844396572 shutdown_connections 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.148+0000 7fdc33707700 1 -- 172.21.15.49:0/2844396572 wait complete. 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.149+0000 7fdc33707700 1 Processor -- start 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.149+0000 7fdc33707700 1 -- start start 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.149+0000 7fdc33707700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.149+0000 7fdc33707700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c19ca40 con 0x7fdc2c107160 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.149+0000 7fdc314a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.149+0000 7fdc314a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45476/0 (socket says 172.21.15.49:45476) 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.149+0000 7fdc314a3700 1 -- 172.21.15.49:0/4229881493 learned_addr learned my addr 172.21.15.49:0/4229881493 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.149+0000 7fdc314a3700 1 -- 172.21.15.49:0/4229881493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc28009550 con 0x7fdc2c107160 2024-05-15T15:43:43.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.149+0000 7fdc314a3700 1 --2- 172.21.15.49:0/4229881493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c19c500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdc28011710 tx=0x7fdc280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.150+0000 7fdc227fc700 1 -- 172.21.15.49:0/4229881493 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc28003b10 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.150+0000 7fdc227fc700 1 -- 172.21.15.49:0/4229881493 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdc2801b8c0 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.150+0000 7fdc33707700 1 -- 172.21.15.49:0/4229881493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c19cc40 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.150+0000 7fdc227fc700 1 -- 172.21.15.49:0/4229881493 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc280197e0 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.150+0000 7fdc33707700 1 -- 172.21.15.49:0/4229881493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c19d0e0 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.150+0000 7fdc227fc700 1 -- 172.21.15.49:0/4229881493 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdc28019cd0 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.150+0000 7fdc227fc700 1 -- 172.21.15.49:0/4229881493 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdc2801a710 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.151+0000 7fdc33707700 1 -- 172.21.15.49:0/4229881493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c10a7b0 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.153+0000 7fdc227fc700 1 -- 172.21.15.49:0/4229881493 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fdc28029060 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.227+0000 7fdc33707700 1 -- 172.21.15.49:0/4229881493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fdc2c10a970 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.228+0000 7fdc227fc700 1 -- 172.21.15.49:0/4229881493 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fdc2801f070 con 0x7fdc2c107160 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.229+0000 7fdc33707700 1 -- 172.21.15.49:0/4229881493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 msgr2=0x7fdc2c19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.229+0000 7fdc33707700 1 --2- 172.21.15.49:0/4229881493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c19c500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdc28011710 tx=0x7fdc280047c0 comp rx=0 tx=0).stop 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.229+0000 7fdc33707700 1 -- 172.21.15.49:0/4229881493 shutdown_connections 2024-05-15T15:43:43.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.229+0000 7fdc33707700 1 --2- 172.21.15.49:0/4229881493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc2c107160 0x7fdc2c19c500 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:43.904 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.229+0000 7fdc33707700 1 -- 172.21.15.49:0/4229881493 >> 172.21.15.49:0/4229881493 conn(0x7fdc2c100b10 msgr2=0x7fdc2c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:43.904 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.229+0000 7fdc33707700 1 -- 172.21.15.49:0/4229881493 shutdown_connections 2024-05-15T15:43:43.904 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:43.229+0000 7fdc33707700 1 -- 172.21.15.49:0/4229881493 wait complete. 2024-05-15T15:43:43.904 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (1/15)... 2024-05-15T15:43:46.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:46 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1294554969' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-15T15:43:50.098 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: Activating manager daemon smithi049.vuhzgf 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: mgrmap e2: smithi049.vuhzgf(active, starting, since 0.00317275s) 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: Manager daemon smithi049.vuhzgf is now available 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/trash_purge_schedule"}]: dispatch 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:43:50.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:49 smithi049 ceph-mon[28399]: from='mgr.14100 172.21.15.49:0/2844097449' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:43:50.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:50.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-05-15T15:43:50.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "c619ac44-12d1-11ef-bc98-c7b262605968", 2024-05-15T15:43:50.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-05-15T15:43:50.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-15T15:43:50.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-15T15:43:50.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-15T15:43:50.103 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "smithi049" 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-15T15:43:50.104 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:50.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-15T15:43:50.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-15T15:43:50.105 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-15T15:43:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-15T15:43:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-15T15:43:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-15T15:43:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-15T15:43:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-15T15:43:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-15T15:43:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-15T15:43:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-15T15:43:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-05-15T15:43:35.648676+0000", 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.332+0000 7fbac0c7a700 1 Processor -- start 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.332+0000 7fbac0c7a700 1 -- start start 2024-05-15T15:43:50.115 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.332+0000 7fbac0c7a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc109500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.332+0000 7fbac0c7a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc079ea0 con 0x7fbabc107110 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.333+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc109500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.333+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc109500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52298/0 (socket says 172.21.15.49:52298) 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.333+0000 7fbaba59c700 1 -- 172.21.15.49:0/2356606400 learned_addr learned my addr 172.21.15.49:0/2356606400 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.333+0000 7fbaba59c700 1 -- 172.21.15.49:0/2356606400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc109a40 con 0x7fbabc107110 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.333+0000 7fbaba59c700 1 --2- 172.21.15.49:0/2356606400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc109500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40098a0 tx=0x7fbaa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437841058a09e7e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.333+0000 7fbab959a700 1 -- 172.21.15.49:0/2356606400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa4004180 con 0x7fbabc107110 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.334+0000 7fbab959a700 1 -- 172.21.15.49:0/2356606400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbaa40042e0 con 0x7fbabc107110 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.334+0000 7fbab959a700 1 -- 172.21.15.49:0/2356606400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa4011a70 con 0x7fbabc107110 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.334+0000 7fbac0c7a700 1 -- 172.21.15.49:0/2356606400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 msgr2=0x7fbabc109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.334+0000 7fbac0c7a700 1 --2- 172.21.15.49:0/2356606400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc109500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40098a0 tx=0x7fbaa4009bb0 comp rx=0 tx=0).stop 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.334+0000 7fbac0c7a700 1 -- 172.21.15.49:0/2356606400 shutdown_connections 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.334+0000 7fbac0c7a700 1 --2- 172.21.15.49:0/2356606400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc109500 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.334+0000 7fbac0c7a700 1 -- 172.21.15.49:0/2356606400 >> 172.21.15.49:0/2356606400 conn(0x7fbabc100b60 msgr2=0x7fbabc102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.334+0000 7fbac0c7a700 1 -- 172.21.15.49:0/2356606400 shutdown_connections 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.334+0000 7fbac0c7a700 1 -- 172.21.15.49:0/2356606400 wait complete. 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.335+0000 7fbac0c7a700 1 Processor -- start 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.335+0000 7fbac0c7a700 1 -- start start 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbac0c7a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbac0c7a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc19a850 con 0x7fbabc107110 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52312/0 (socket says 172.21.15.49:52312) 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbaba59c700 1 -- 172.21.15.49:0/1294554969 learned_addr learned my addr 172.21.15.49:0/1294554969 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbaba59c700 1 -- 172.21.15.49:0/1294554969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa4009550 con 0x7fbabc107110 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbaba59c700 1 --2- 172.21.15.49:0/1294554969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc19a310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4011710 tx=0x7fbaa40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbab37fe700 1 -- 172.21.15.49:0/1294554969 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa4003b10 con 0x7fbabc107110 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbab37fe700 1 -- 172.21.15.49:0/1294554969 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbaa401b8c0 con 0x7fbabc107110 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbac0c7a700 1 -- 172.21.15.49:0/1294554969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc19aa50 con 0x7fbabc107110 2024-05-15T15:43:50.117 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbac0c7a700 1 -- 172.21.15.49:0/1294554969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc19aef0 con 0x7fbabc107110 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.336+0000 7fbab37fe700 1 -- 172.21.15.49:0/1294554969 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa40196a0 con 0x7fbabc107110 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.337+0000 7fbab37fe700 1 -- 172.21.15.49:0/1294554969 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fbaa4019b00 con 0x7fbabc107110 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.337+0000 7fbab37fe700 1 -- 172.21.15.49:0/1294554969 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbaa401a710 con 0x7fbabc107110 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.337+0000 7fbac0c7a700 1 -- 172.21.15.49:0/1294554969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc10a7c0 con 0x7fbabc107110 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.340+0000 7fbab37fe700 1 -- 172.21.15.49:0/1294554969 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fbaa4029060 con 0x7fbabc107110 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.414+0000 7fbac0c7a700 1 -- 172.21.15.49:0/1294554969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbabc10a980 con 0x7fbabc107110 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.414+0000 7fbab37fe700 1 -- 172.21.15.49:0/1294554969 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fbaa401f070 con 0x7fbabc107110 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.416+0000 7fbac0c7a700 1 -- 172.21.15.49:0/1294554969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 msgr2=0x7fbabc19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.416+0000 7fbac0c7a700 1 --2- 172.21.15.49:0/1294554969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc19a310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4011710 tx=0x7fbaa40047c0 comp rx=0 tx=0).stop 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.416+0000 7fbac0c7a700 1 -- 172.21.15.49:0/1294554969 shutdown_connections 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.416+0000 7fbac0c7a700 1 --2- 172.21.15.49:0/1294554969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbabc107110 0x7fbabc19a310 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.416+0000 7fbac0c7a700 1 -- 172.21.15.49:0/1294554969 >> 172.21.15.49:0/1294554969 conn(0x7fbabc100b60 msgr2=0x7fbabc101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.416+0000 7fbac0c7a700 1 -- 172.21.15.49:0/1294554969 shutdown_connections 2024-05-15T15:43:50.118 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:46.416+0000 7fbac0c7a700 1 -- 172.21.15.49:0/1294554969 wait complete. 2024-05-15T15:43:50.119 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (2/15)... 2024-05-15T15:43:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:51 smithi049 ceph-mon[28399]: mgrmap e3: smithi049.vuhzgf(active, since 1.0073s) 2024-05-15T15:43:53.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:52 smithi049 ceph-mon[28399]: mgrmap e4: smithi049.vuhzgf(active, since 2s) 2024-05-15T15:43:56.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:55 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2072618193' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-15T15:43:58.317 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:58.317 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-05-15T15:43:58.317 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "c619ac44-12d1-11ef-bc98-c7b262605968", 2024-05-15T15:43:58.317 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-05-15T15:43:58.317 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-15T15:43:58.317 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "smithi049" 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:58.318 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-15T15:43:58.319 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-15T15:43:58.319 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:58.319 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-15T15:43:58.319 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:58.319 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-15T15:43:58.319 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-15T15:43:58.320 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-15T15:43:58.320 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-15T15:43:58.320 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-15T15:43:58.320 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-15T15:43:58.320 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:58.320 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-15T15:43:58.321 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-15T15:43:58.321 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-15T15:43:58.321 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-15T15:43:58.321 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-15T15:43:58.321 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-15T15:43:58.321 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-15T15:43:58.322 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-15T15:43:58.322 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-15T15:43:58.322 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:58.322 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-15T15:43:58.322 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:58.322 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-15T15:43:58.323 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-15T15:43:58.323 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:58.325 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-15T15:43:58.326 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-15T15:43:58.326 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-15T15:43:58.326 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-15T15:43:58.326 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-15T15:43:58.326 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-15T15:43:58.327 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-05-15T15:43:58.327 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-05-15T15:43:58.327 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T15:43:58.327 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:58.327 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-15T15:43:58.327 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T15:43:58.327 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-05-15T15:43:35.648676+0000", 2024-05-15T15:43:58.327 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T15:43:58.327 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.565+0000 7fe3bb29f700 1 Processor -- start 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.565+0000 7fe3bb29f700 1 -- start start 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.565+0000 7fe3bb29f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b4109550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.565+0000 7fe3bb29f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b4062260 con 0x7fe3b4107160 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.566+0000 7fe3b903b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b4109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.566+0000 7fe3b903b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b4109550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38074/0 (socket says 172.21.15.49:38074) 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.566+0000 7fe3b903b700 1 -- 172.21.15.49:0/1782195421 learned_addr learned my addr 172.21.15.49:0/1782195421 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.566+0000 7fe3b903b700 1 -- 172.21.15.49:0/1782195421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4109a90 con 0x7fe3b4107160 2024-05-15T15:43:58.328 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.566+0000 7fe3b903b700 1 --2- 172.21.15.49:0/1782195421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b4109550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009e80 tx=0x7fe3a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a412f89722ecc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.566+0000 7fe3b3fff700 1 -- 172.21.15.49:0/1782195421 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3a400e070 con 0x7fe3b4107160 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.566+0000 7fe3b3fff700 1 -- 172.21.15.49:0/1782195421 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe3a400b8c0 con 0x7fe3b4107160 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.566+0000 7fe3b3fff700 1 -- 172.21.15.49:0/1782195421 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3a4004440 con 0x7fe3b4107160 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.567+0000 7fe3bb29f700 1 -- 172.21.15.49:0/1782195421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 msgr2=0x7fe3b4109550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.567+0000 7fe3bb29f700 1 --2- 172.21.15.49:0/1782195421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b4109550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009e80 tx=0x7fe3a400b330 comp rx=0 tx=0).stop 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.567+0000 7fe3bb29f700 1 -- 172.21.15.49:0/1782195421 shutdown_connections 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.567+0000 7fe3bb29f700 1 --2- 172.21.15.49:0/1782195421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b4109550 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.567+0000 7fe3bb29f700 1 -- 172.21.15.49:0/1782195421 >> 172.21.15.49:0/1782195421 conn(0x7fe3b4100af0 msgr2=0x7fe3b4102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.567+0000 7fe3bb29f700 1 -- 172.21.15.49:0/1782195421 shutdown_connections 2024-05-15T15:43:58.329 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.567+0000 7fe3bb29f700 1 -- 172.21.15.49:0/1782195421 wait complete. 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.567+0000 7fe3bb29f700 1 Processor -- start 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3bb29f700 1 -- start start 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3bb29f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3bb29f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b419a8a0 con 0x7fe3b4107160 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3b903b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3b903b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38084/0 (socket says 172.21.15.49:38084) 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3b903b700 1 -- 172.21.15.49:0/2072618193 learned_addr learned my addr 172.21.15.49:0/2072618193 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3b903b700 1 -- 172.21.15.49:0/2072618193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a4009b30 con 0x7fe3b4107160 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3b903b700 1 --2- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b419a360 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40099e0 tx=0x7fe3a4003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3b27fc700 1 -- 172.21.15.49:0/2072618193 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3a400e040 con 0x7fe3b4107160 2024-05-15T15:43:58.330 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b419aaa0 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3b27fc700 1 -- 172.21.15.49:0/2072618193 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe3a400bd50 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3b27fc700 1 -- 172.21.15.49:0/2072618193 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3a40125c0 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.568+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b419af20 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.569+0000 7fe3b27fc700 1 -- 172.21.15.49:0/2072618193 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 45545+0+0 (secure 0 0 0) 0x7fe3a4019040 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.569+0000 7fe3b27fc700 1 --2- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fe39c03caa0 0x7fe39c03ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.569+0000 7fe3b27fc700 1 -- 172.21.15.49:0/2072618193 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe3a404be20 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.569+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b410a7b0 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.570+0000 7fe3b883a700 1 --2- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fe39c03caa0 0x7fe39c03ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.570+0000 7fe3b883a700 1 --2- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fe39c03caa0 0x7fe39c03ef60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8006fd0 tx=0x7fe3a8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.575+0000 7fe3b27fc700 1 -- 172.21.15.49:0/2072618193 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3a401f080 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.863+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe3b41054f0 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.863+0000 7fe3b27fc700 1 -- 172.21.15.49:0/2072618193 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fe3a404a090 con 0x7fe3b4107160 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fe39c03caa0 msgr2=0x7fe39c03ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:58.331 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 --2- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fe39c03caa0 0x7fe39c03ef60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8006fd0 tx=0x7fe3a8006e40 comp rx=0 tx=0).stop 2024-05-15T15:43:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 msgr2=0x7fe3b419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 --2- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b419a360 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40099e0 tx=0x7fe3a4003d00 comp rx=0 tx=0).stop 2024-05-15T15:43:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 shutdown_connections 2024-05-15T15:43:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 --2- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fe39c03caa0 0x7fe39c03ef60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 --2- 172.21.15.49:0/2072618193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b4107160 0x7fe3b419a360 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 >> 172.21.15.49:0/2072618193 conn(0x7fe3b4100af0 msgr2=0x7fe3b41016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 shutdown_connections 2024-05-15T15:43:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:55.866+0000 7fe3bb29f700 1 -- 172.21.15.49:0/2072618193 wait complete. 2024-05-15T15:43:58.332 INFO:teuthology.orchestra.run.smithi049.stdout:mgr is available 2024-05-15T15:43:59.282 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:43:59 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1036991698' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-15T15:43:59.536 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.758+0000 7fd6698bc700 1 Processor -- start 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.758+0000 7fd6698bc700 1 -- start start 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.759+0000 7fd6698bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664105ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.759+0000 7fd6698bc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641064b0 con 0x7fd664105ac0 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.759+0000 7fd662ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664105ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.759+0000 7fd662ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664105ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38086/0 (socket says 172.21.15.49:38086) 2024-05-15T15:43:59.537 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.759+0000 7fd662ffd700 1 -- 172.21.15.49:0/2523196628 learned_addr learned my addr 172.21.15.49:0/2523196628 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.759+0000 7fd662ffd700 1 -- 172.21.15.49:0/2523196628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6641067e0 con 0x7fd664105ac0 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.759+0000 7fd662ffd700 1 --2- 172.21.15.49:0/2523196628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664105ee0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6540098a0 tx=0x7fd654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e786fb5b001dae server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd661ffb700 1 -- 172.21.15.49:0/2523196628 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd654004180 con 0x7fd664105ac0 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd661ffb700 1 -- 172.21.15.49:0/2523196628 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd6540042e0 con 0x7fd664105ac0 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd661ffb700 1 -- 172.21.15.49:0/2523196628 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd654011a70 con 0x7fd664105ac0 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd6698bc700 1 -- 172.21.15.49:0/2523196628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 msgr2=0x7fd664105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd6698bc700 1 --2- 172.21.15.49:0/2523196628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664105ee0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6540098a0 tx=0x7fd654009bb0 comp rx=0 tx=0).stop 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd6698bc700 1 -- 172.21.15.49:0/2523196628 shutdown_connections 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd6698bc700 1 --2- 172.21.15.49:0/2523196628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664105ee0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd6698bc700 1 -- 172.21.15.49:0/2523196628 >> 172.21.15.49:0/2523196628 conn(0x7fd6641002a0 msgr2=0x7fd664102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd6698bc700 1 -- 172.21.15.49:0/2523196628 shutdown_connections 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.760+0000 7fd6698bc700 1 -- 172.21.15.49:0/2523196628 wait complete. 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.761+0000 7fd6698bc700 1 Processor -- start 2024-05-15T15:43:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.761+0000 7fd6698bc700 1 -- start start 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.761+0000 7fd6698bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664190630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.761+0000 7fd6698bc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd664190b70 con 0x7fd664105ac0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd662ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664190630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd662ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664190630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38096/0 (socket says 172.21.15.49:38096) 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd662ffd700 1 -- 172.21.15.49:0/1036991698 learned_addr learned my addr 172.21.15.49:0/1036991698 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd662ffd700 1 -- 172.21.15.49:0/1036991698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd654009550 con 0x7fd664105ac0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd662ffd700 1 --2- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664190630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd654011710 tx=0x7fd6540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd6688ba700 1 -- 172.21.15.49:0/1036991698 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd654003b10 con 0x7fd664105ac0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd6688ba700 1 -- 172.21.15.49:0/1036991698 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd65401b8c0 con 0x7fd664105ac0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd6688ba700 1 -- 172.21.15.49:0/1036991698 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6540197e0 con 0x7fd664105ac0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66418edb0 con 0x7fd664105ac0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.762+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66418f250 con 0x7fd664105ac0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.763+0000 7fd6688ba700 1 -- 172.21.15.49:0/1036991698 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 45545+0+0 (secure 0 0 0) 0x7fd65401b440 con 0x7fd664105ac0 2024-05-15T15:43:59.539 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.764+0000 7fd6688ba700 1 --2- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fd6500387e0 0x7fd65003aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.764+0000 7fd6688ba700 1 -- 172.21.15.49:0/1036991698 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd65404c880 con 0x7fd664105ac0 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.764+0000 7fd6627fc700 1 --2- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fd6500387e0 0x7fd65003aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.764+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66418fc60 con 0x7fd664105ac0 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.764+0000 7fd6627fc700 1 --2- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fd6500387e0 0x7fd65003aca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd64c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.769+0000 7fd6688ba700 1 -- 172.21.15.49:0/1036991698 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd65401f080 con 0x7fd664105ac0 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.971+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd664109b60 con 0x7fd664105ac0 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.971+0000 7fd6688ba700 1 -- 172.21.15.49:0/1036991698 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd654047020 con 0x7fd664105ac0 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fd6500387e0 msgr2=0x7fd65003aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 --2- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fd6500387e0 0x7fd65003aca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd64c006e40 comp rx=0 tx=0).stop 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 msgr2=0x7fd664190630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 --2- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664190630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd654011710 tx=0x7fd6540047c0 comp rx=0 tx=0).stop 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 shutdown_connections 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 --2- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fd6500387e0 0x7fd65003aca0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:59.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 --2- 172.21.15.49:0/1036991698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd664105ac0 0x7fd664190630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:43:59.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 >> 172.21.15.49:0/1036991698 conn(0x7fd6641002a0 msgr2=0x7fd664101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:43:59.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 shutdown_connections 2024-05-15T15:43:59.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:58.974+0000 7fd6698bc700 1 -- 172.21.15.49:0/1036991698 wait complete. 2024-05-15T15:43:59.541 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling cephadm module... 2024-05-15T15:44:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:00 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1848257671' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-15T15:44:00.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.902+0000 7f16cbfda700 1 Processor -- start 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.902+0000 7f16cbfda700 1 -- start start 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.902+0000 7f16cbfda700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c4106a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.902+0000 7f16cbfda700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4107060 con 0x7f16c4106670 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.903+0000 7f16c9d76700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c4106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.903+0000 7f16c9d76700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c4106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38102/0 (socket says 172.21.15.49:38102) 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.903+0000 7f16c9d76700 1 -- 172.21.15.49:0/1769963007 learned_addr learned my addr 172.21.15.49:0/1769963007 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.903+0000 7f16c9d76700 1 -- 172.21.15.49:0/1769963007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c4107870 con 0x7f16c4106670 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.903+0000 7f16c9d76700 1 --2- 172.21.15.49:0/1769963007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c4106a90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f16c00098a0 tx=0x7f16c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720e77fe5fd8cc server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.903+0000 7f16c8d74700 1 -- 172.21.15.49:0/1769963007 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16c0004180 con 0x7f16c4106670 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.903+0000 7f16c8d74700 1 -- 172.21.15.49:0/1769963007 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f16c00042e0 con 0x7f16c4106670 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.903+0000 7f16c8d74700 1 -- 172.21.15.49:0/1769963007 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16c0011a70 con 0x7f16c4106670 2024-05-15T15:44:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.904+0000 7f16cbfda700 1 -- 172.21.15.49:0/1769963007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 msgr2=0x7f16c4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.904+0000 7f16cbfda700 1 --2- 172.21.15.49:0/1769963007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c4106a90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f16c00098a0 tx=0x7f16c0009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.904+0000 7f16cbfda700 1 -- 172.21.15.49:0/1769963007 shutdown_connections 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.904+0000 7f16cbfda700 1 --2- 172.21.15.49:0/1769963007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c4106a90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.904+0000 7f16cbfda700 1 -- 172.21.15.49:0/1769963007 >> 172.21.15.49:0/1769963007 conn(0x7f16c4101c90 msgr2=0x7f16c41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.904+0000 7f16cbfda700 1 -- 172.21.15.49:0/1769963007 shutdown_connections 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.904+0000 7f16cbfda700 1 -- 172.21.15.49:0/1769963007 wait complete. 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.905+0000 7f16cbfda700 1 Processor -- start 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.905+0000 7f16cbfda700 1 -- start start 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.905+0000 7f16cbfda700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c407bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.905+0000 7f16cbfda700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c407c050 con 0x7f16c4106670 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.905+0000 7f16c9d76700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c407bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.905+0000 7f16c9d76700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c407bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38118/0 (socket says 172.21.15.49:38118) 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.905+0000 7f16c9d76700 1 -- 172.21.15.49:0/1848257671 learned_addr learned my addr 172.21.15.49:0/1848257671 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.905+0000 7f16c9d76700 1 -- 172.21.15.49:0/1848257671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c0009550 con 0x7f16c4106670 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.905+0000 7f16c9d76700 1 --2- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c407bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f16c0011710 tx=0x7f16c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.906+0000 7f16baffd700 1 -- 172.21.15.49:0/1848257671 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16c0003b10 con 0x7f16c4106670 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.906+0000 7f16baffd700 1 -- 172.21.15.49:0/1848257671 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f16c001b8c0 con 0x7f16c4106670 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.906+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c407a290 con 0x7f16c4106670 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.906+0000 7f16baffd700 1 -- 172.21.15.49:0/1848257671 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16c0019710 con 0x7f16c4106670 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.906+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c407a730 con 0x7f16c4106670 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.907+0000 7f16baffd700 1 -- 172.21.15.49:0/1848257671 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 45545+0+0 (secure 0 0 0) 0x7f16c001b440 con 0x7f16c4106670 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.907+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c40625d0 con 0x7f16c4106670 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.907+0000 7f16baffd700 1 --2- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7f16b00387a0 0x7f16b003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:00.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.907+0000 7f16baffd700 1 -- 172.21.15.49:0/1848257671 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f16c004ca10 con 0x7f16c4106670 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.907+0000 7f16c9575700 1 --2- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7f16b00387a0 0x7f16b003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.907+0000 7f16c9575700 1 --2- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7f16b00387a0 0x7f16b003ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006fd0 tx=0x7f16b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:43:59.912+0000 7f16baffd700 1 -- 172.21.15.49:0/1848257671 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f16c001b6f0 con 0x7f16c4106670 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.173+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f16c410aaa0 con 0x7f16c4106670 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.226+0000 7f16baffd700 1 -- 172.21.15.49:0/1848257671 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 5) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f16c0019870 con 0x7f16c4106670 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.227+0000 7f16baffd700 1 -- 172.21.15.49:0/1848257671 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f16c004c430 con 0x7f16c4106670 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.229+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7f16b00387a0 msgr2=0x7f16b003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.229+0000 7f16cbfda700 1 --2- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7f16b00387a0 0x7f16b003ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006fd0 tx=0x7f16b4006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.229+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 msgr2=0x7f16c407bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.229+0000 7f16cbfda700 1 --2- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c407bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f16c0011710 tx=0x7f16c00047c0 comp rx=0 tx=0).stop 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.230+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 shutdown_connections 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.230+0000 7f16cbfda700 1 --2- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7f16b00387a0 0x7f16b003ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.230+0000 7f16cbfda700 1 --2- 172.21.15.49:0/1848257671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4106670 0x7f16c407bb10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.230+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 >> 172.21.15.49:0/1848257671 conn(0x7f16c4101c90 msgr2=0x7f16c4102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.230+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 shutdown_connections 2024-05-15T15:44:00.812 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:00.230+0000 7f16cbfda700 1 -- 172.21.15.49:0/1848257671 wait complete. 2024-05-15T15:44:01.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:01 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1848257671' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-15T15:44:01.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:01 smithi049 ceph-mon[28399]: mgrmap e5: smithi049.vuhzgf(active, since 10s) 2024-05-15T15:44:02.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-05-15T15:44:02.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-15T15:44:02.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-15T15:44:02.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "smithi049.vuhzgf", 2024-05-15T15:44:02.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-15T15:44:02.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-05-15T15:44:02.033 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.203+0000 7fb033162700 1 Processor -- start 2024-05-15T15:44:02.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.203+0000 7fb033162700 1 -- start start 2024-05-15T15:44:02.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.203+0000 7fb033162700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c1059c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:02.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.204+0000 7fb033162700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c105f90 con 0x7fb02c1055a0 2024-05-15T15:44:02.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.204+0000 7fb030efe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c1059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:02.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.204+0000 7fb030efe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c1059c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38156/0 (socket says 172.21.15.49:38156) 2024-05-15T15:44:02.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.204+0000 7fb030efe700 1 -- 172.21.15.49:0/2716647858 learned_addr learned my addr 172.21.15.49:0/2716647858 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:02.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.204+0000 7fb030efe700 1 -- 172.21.15.49:0/2716647858 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c1067a0 con 0x7fb02c1055a0 2024-05-15T15:44:02.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.204+0000 7fb030efe700 1 --2- 172.21.15.49:0/2716647858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c1059c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0200098a0 tx=0x7fb020009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90279a69efa182e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:02.034 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.204+0000 7fb02b7fe700 1 -- 172.21.15.49:0/2716647858 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb020004180 con 0x7fb02c1055a0 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.204+0000 7fb02b7fe700 1 -- 172.21.15.49:0/2716647858 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb0200042e0 con 0x7fb02c1055a0 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.204+0000 7fb02b7fe700 1 -- 172.21.15.49:0/2716647858 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb020011a70 con 0x7fb02c1055a0 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.205+0000 7fb033162700 1 -- 172.21.15.49:0/2716647858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 msgr2=0x7fb02c1059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.205+0000 7fb033162700 1 --2- 172.21.15.49:0/2716647858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c1059c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0200098a0 tx=0x7fb020009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.205+0000 7fb033162700 1 -- 172.21.15.49:0/2716647858 shutdown_connections 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.205+0000 7fb033162700 1 --2- 172.21.15.49:0/2716647858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c1059c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.205+0000 7fb033162700 1 -- 172.21.15.49:0/2716647858 >> 172.21.15.49:0/2716647858 conn(0x7fb02c100b60 msgr2=0x7fb02c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.205+0000 7fb033162700 1 -- 172.21.15.49:0/2716647858 shutdown_connections 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.205+0000 7fb033162700 1 -- 172.21.15.49:0/2716647858 wait complete. 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.206+0000 7fb033162700 1 Processor -- start 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.206+0000 7fb033162700 1 -- start start 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.206+0000 7fb033162700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.206+0000 7fb033162700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c19a8b0 con 0x7fb02c1055a0 2024-05-15T15:44:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.206+0000 7fb030efe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.206+0000 7fb030efe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38162/0 (socket says 172.21.15.49:38162) 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.206+0000 7fb030efe700 1 -- 172.21.15.49:0/1504220053 learned_addr learned my addr 172.21.15.49:0/1504220053 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.206+0000 7fb030efe700 1 -- 172.21.15.49:0/1504220053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb020009550 con 0x7fb02c1055a0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.207+0000 7fb030efe700 1 --2- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c19a370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb020011710 tx=0x7fb0200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.207+0000 7fb029ffb700 1 -- 172.21.15.49:0/1504220053 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb020003b10 con 0x7fb02c1055a0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.207+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c19aab0 con 0x7fb02c1055a0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.207+0000 7fb029ffb700 1 -- 172.21.15.49:0/1504220053 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb02001b8c0 con 0x7fb02c1055a0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.207+0000 7fb029ffb700 1 -- 172.21.15.49:0/1504220053 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb020019710 con 0x7fb02c1055a0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.207+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c19af50 con 0x7fb02c1055a0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.208+0000 7fb029ffb700 1 -- 172.21.15.49:0/1504220053 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fb020019910 con 0x7fb02c1055a0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.208+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c1945c0 con 0x7fb02c1055a0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.208+0000 7fb029ffb700 1 --2- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fb01c0387a0 0x7fb01c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.208+0000 7fb029ffb700 1 -- 172.21.15.49:0/1504220053 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb02004cad0 con 0x7fb02c1055a0 2024-05-15T15:44:02.036 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.208+0000 7fb02bfff700 1 -- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fb01c0387a0 msgr2=0x7fb01c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2885879516 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.208+0000 7fb02bfff700 1 --2- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fb01c0387a0 0x7fb01c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.214+0000 7fb029ffb700 1 -- 172.21.15.49:0/1504220053 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb020018b80 con 0x7fb02c1055a0 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.409+0000 7fb02bfff700 1 -- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fb01c0387a0 msgr2=0x7fb01c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2885879516 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.409+0000 7fb02bfff700 1 --2- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fb01c0387a0 0x7fb01c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.480+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb02c1099d0 con 0x7fb02c1055a0 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.480+0000 7fb029ffb700 1 -- 172.21.15.49:0/1504220053 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fb020018350 con 0x7fb02c1055a0 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fb01c0387a0 msgr2=0x7fb01c03ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 --2- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fb01c0387a0 0x7fb01c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 msgr2=0x7fb02c19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 --2- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c19a370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb020011710 tx=0x7fb0200047c0 comp rx=0 tx=0).stop 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 shutdown_connections 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 --2- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fb01c0387a0 0x7fb01c03ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 --2- 172.21.15.49:0/1504220053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb02c1055a0 0x7fb02c19a370 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 >> 172.21.15.49:0/1504220053 conn(0x7fb02c100b60 msgr2=0x7fb02c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:02.037 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 shutdown_connections 2024-05-15T15:44:02.038 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:01.483+0000 7fb033162700 1 -- 172.21.15.49:0/1504220053 wait complete. 2024-05-15T15:44:02.038 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-05-15T15:44:02.038 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 5... 2024-05-15T15:44:02.431 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:02 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1504220053' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-15T15:44:07.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: Active manager daemon smithi049.vuhzgf restarted 2024-05-15T15:44:07.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: Activating manager daemon smithi049.vuhzgf 2024-05-15T15:44:07.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: osdmap e2: 0 total, 0 up, 0 in 2024-05-15T15:44:07.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: mgrmap e6: smithi049.vuhzgf(active, starting, since 0.00338922s) 2024-05-15T15:44:07.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:44:07.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:44:07.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:44:07.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:44:07.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:44:07.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: Manager daemon smithi049.vuhzgf is now available 2024-05-15T15:44:07.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:07.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:07.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:44:07.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:44:07.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:07 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:44:08.703 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:08 smithi049 ceph-mon[28399]: Found migration_current of "None". Setting to last migration. 2024-05-15T15:44:08.703 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:08 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/trash_purge_schedule"}]: dispatch 2024-05-15T15:44:08.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:08 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:08.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:08 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:08.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:08 smithi049 ceph-mon[28399]: mgrmap e7: smithi049.vuhzgf(active, since 1.00508s) 2024-05-15T15:44:08.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-05-15T15:44:08.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-15T15:44:08.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-15T15:44:08.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-05-15T15:44:08.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.427+0000 7fca50c5c700 1 Processor -- start 2024-05-15T15:44:08.947 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.427+0000 7fca50c5c700 1 -- start start 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.428+0000 7fca50c5c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c1059c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.428+0000 7fca50c5c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c105f90 con 0x7fca4c1055a0 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.428+0000 7fca4a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c1059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.428+0000 7fca4a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c1059c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38172/0 (socket says 172.21.15.49:38172) 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.428+0000 7fca4a59c700 1 -- 172.21.15.49:0/3588968985 learned_addr learned my addr 172.21.15.49:0/3588968985 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.428+0000 7fca4a59c700 1 -- 172.21.15.49:0/3588968985 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c1067a0 con 0x7fca4c1055a0 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.428+0000 7fca4a59c700 1 --2- 172.21.15.49:0/3588968985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c1059c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fca340098a0 tx=0x7fca34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26751425074f6da server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.429+0000 7fca4959a700 1 -- 172.21.15.49:0/3588968985 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca34004180 con 0x7fca4c1055a0 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.429+0000 7fca4959a700 1 -- 172.21.15.49:0/3588968985 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fca340042e0 con 0x7fca4c1055a0 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.429+0000 7fca4959a700 1 -- 172.21.15.49:0/3588968985 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca34011a70 con 0x7fca4c1055a0 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.429+0000 7fca50c5c700 1 -- 172.21.15.49:0/3588968985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 msgr2=0x7fca4c1059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.429+0000 7fca50c5c700 1 --2- 172.21.15.49:0/3588968985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c1059c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fca340098a0 tx=0x7fca34009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:08.948 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.429+0000 7fca50c5c700 1 -- 172.21.15.49:0/3588968985 shutdown_connections 2024-05-15T15:44:08.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.429+0000 7fca50c5c700 1 --2- 172.21.15.49:0/3588968985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c1059c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:08.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.429+0000 7fca50c5c700 1 -- 172.21.15.49:0/3588968985 >> 172.21.15.49:0/3588968985 conn(0x7fca4c100b60 msgr2=0x7fca4c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:08.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.430+0000 7fca50c5c700 1 -- 172.21.15.49:0/3588968985 shutdown_connections 2024-05-15T15:44:08.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.430+0000 7fca50c5c700 1 -- 172.21.15.49:0/3588968985 wait complete. 2024-05-15T15:44:08.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.430+0000 7fca50c5c700 1 Processor -- start 2024-05-15T15:44:08.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.430+0000 7fca50c5c700 1 -- start start 2024-05-15T15:44:08.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca50c5c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c079a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:08.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca50c5c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c079f90 con 0x7fca4c1055a0 2024-05-15T15:44:08.949 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca4a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c079a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca4a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c079a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38174/0 (socket says 172.21.15.49:38174) 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca4a59c700 1 -- 172.21.15.49:0/1557515002 learned_addr learned my addr 172.21.15.49:0/1557515002 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca4a59c700 1 -- 172.21.15.49:0/1557515002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009550 con 0x7fca4c1055a0 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca4a59c700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c079a50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca34011710 tx=0x7fca340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca34003b10 con 0x7fca4c1055a0 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fca3401b8c0 con 0x7fca4c1055a0 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca50c5c700 1 -- 172.21.15.49:0/1557515002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c19d050 con 0x7fca4c1055a0 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.431+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca34019770 con 0x7fca4c1055a0 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.432+0000 7fca50c5c700 1 -- 172.21.15.49:0/1557515002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c19d4f0 con 0x7fca4c1055a0 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.432+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fca340199a0 con 0x7fca4c1055a0 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.433+0000 7fca437fe700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 0x7fca3803acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.433+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 --> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca3803b400 con 0x7fca38038830 2024-05-15T15:44:08.950 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.433+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fca3404c8f0 con 0x7fca4c1055a0 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.433+0000 7fca49d9b700 1 -- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 msgr2=0x7fca3803acf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2885879516 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.433+0000 7fca49d9b700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 0x7fca3803acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.634+0000 7fca49d9b700 1 -- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 msgr2=0x7fca3803acf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2885879516 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:02.634+0000 7fca49d9b700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 0x7fca3803acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:03.035+0000 7fca49d9b700 1 -- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 msgr2=0x7fca3803acf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2885879516 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:03.035+0000 7fca49d9b700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 0x7fca3803acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:03.836+0000 7fca49d9b700 1 -- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 msgr2=0x7fca3803acf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2885879516 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:03.836+0000 7fca49d9b700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 0x7fca3803acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:05.437+0000 7fca49d9b700 1 -- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 msgr2=0x7fca3803acf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/2885879516 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:05.437+0000 7fca49d9b700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 0x7fca3803acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:07.361+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 6) v1 ==== 45323+0+0 (secure 0 0 0) 0x7fca34019d40 con 0x7fca4c1055a0 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:07.362+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 msgr2=0x7fca3803acf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:07.362+0000 7fca437fe700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 0x7fca3803acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:08.951 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.363+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 7) v1 ==== 45450+0+0 (secure 0 0 0) 0x7fca3402ad20 con 0x7fca4c1055a0 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.363+0000 7fca437fe700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fca3803bfd0 0x7fca3803e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.363+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca3803b400 con 0x7fca3803bfd0 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.363+0000 7fca49d9b700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fca3803bfd0 0x7fca3803e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.364+0000 7fca49d9b700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fca3803bfd0 0x7fca3803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fca3c003a10 tx=0x7fca3c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.364+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fca3803b400 con 0x7fca3803bfd0 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.371+0000 7fca50c5c700 1 -- 172.21.15.49:0/1557515002 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fca4c0625d0 con 0x7fca3803bfd0 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.371+0000 7fca437fe700 1 -- 172.21.15.49:0/1557515002 <== mgr.14118 v2:172.21.15.49:6800/821763420 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fca4c0625d0 con 0x7fca3803bfd0 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.371+0000 7fca50c5c700 1 -- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fca3803bfd0 msgr2=0x7fca3803e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.371+0000 7fca50c5c700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fca3803bfd0 0x7fca3803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fca3c003a10 tx=0x7fca3c0092b0 comp rx=0 tx=0).stop 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.371+0000 7fca50c5c700 1 -- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 msgr2=0x7fca4c079a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.371+0000 7fca50c5c700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c079a50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca34011710 tx=0x7fca340047c0 comp rx=0 tx=0).stop 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.371+0000 7fca50c5c700 1 -- 172.21.15.49:0/1557515002 shutdown_connections 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.371+0000 7fca50c5c700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fca3803bfd0 0x7fca3803e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.372+0000 7fca50c5c700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca4c1055a0 0x7fca4c079a50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:08.952 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.372+0000 7fca50c5c700 1 --2- 172.21.15.49:0/1557515002 >> [v2:172.21.15.49:6800/2885879516,v1:172.21.15.49:6801/2885879516] conn(0x7fca38038830 0x7fca3803acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:08.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.372+0000 7fca50c5c700 1 -- 172.21.15.49:0/1557515002 >> 172.21.15.49:0/1557515002 conn(0x7fca4c100b60 msgr2=0x7fca4c101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:08.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.372+0000 7fca50c5c700 1 -- 172.21.15.49:0/1557515002 shutdown_connections 2024-05-15T15:44:08.953 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:08.372+0000 7fca50c5c700 1 -- 172.21.15.49:0/1557515002 wait complete. 2024-05-15T15:44:08.953 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 5 is available 2024-05-15T15:44:08.953 INFO:teuthology.orchestra.run.smithi049.stdout:Setting orchestrator backend to cephadm... 2024-05-15T15:44:09.598 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:09 smithi049 ceph-mon[28399]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-15T15:44:09.598 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:09 smithi049 ceph-mon[28399]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-15T15:44:09.598 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:09 smithi049 ceph-mon[28399]: [15/May/2024:15:44:09] ENGINE Bus STARTING 2024-05-15T15:44:10.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c92044700 1 Processor -- start 2024-05-15T15:44:10.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c92044700 1 -- start start 2024-05-15T15:44:10.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c92044700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c106a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:10.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c92044700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c107060 con 0x7f6c8c106670 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c8b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c8b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35106/0 (socket says 172.21.15.49:35106) 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c8b7fe700 1 -- 172.21.15.49:0/1564702729 learned_addr learned my addr 172.21.15.49:0/1564702729 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c8b7fe700 1 -- 172.21.15.49:0/1564702729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c107870 con 0x7f6c8c106670 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c8b7fe700 1 --2- 172.21.15.49:0/1564702729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c106a90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098a0 tx=0x7f6c7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9368217ae59e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.358+0000 7f6c8a7fc700 1 -- 172.21.15.49:0/1564702729 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c7c004180 con 0x7f6c8c106670 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.359+0000 7f6c8a7fc700 1 -- 172.21.15.49:0/1564702729 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6c7c0042e0 con 0x7f6c8c106670 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.359+0000 7f6c8a7fc700 1 -- 172.21.15.49:0/1564702729 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c7c011a70 con 0x7f6c8c106670 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.359+0000 7f6c92044700 1 -- 172.21.15.49:0/1564702729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 msgr2=0x7f6c8c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.359+0000 7f6c92044700 1 --2- 172.21.15.49:0/1564702729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c106a90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098a0 tx=0x7f6c7c009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.359+0000 7f6c92044700 1 -- 172.21.15.49:0/1564702729 shutdown_connections 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.359+0000 7f6c92044700 1 --2- 172.21.15.49:0/1564702729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c106a90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.359+0000 7f6c92044700 1 -- 172.21.15.49:0/1564702729 >> 172.21.15.49:0/1564702729 conn(0x7f6c8c101c90 msgr2=0x7f6c8c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.359+0000 7f6c92044700 1 -- 172.21.15.49:0/1564702729 shutdown_connections 2024-05-15T15:44:10.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.359+0000 7f6c92044700 1 -- 172.21.15.49:0/1564702729 wait complete. 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.360+0000 7f6c92044700 1 Processor -- start 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.360+0000 7f6c92044700 1 -- start start 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.360+0000 7f6c92044700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.360+0000 7f6c92044700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c19ca70 con 0x7f6c8c106670 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.360+0000 7f6c8b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.360+0000 7f6c8b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35118/0 (socket says 172.21.15.49:35118) 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.360+0000 7f6c8b7fe700 1 -- 172.21.15.49:0/1948768727 learned_addr learned my addr 172.21.15.49:0/1948768727 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.361+0000 7f6c8b7fe700 1 -- 172.21.15.49:0/1948768727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c009550 con 0x7f6c8c106670 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.361+0000 7f6c8b7fe700 1 --2- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c19c530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c011710 tx=0x7f6c7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.361+0000 7f6c88ff9700 1 -- 172.21.15.49:0/1948768727 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c7c003b10 con 0x7f6c8c106670 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.361+0000 7f6c88ff9700 1 -- 172.21.15.49:0/1948768727 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6c7c01b8c0 con 0x7f6c8c106670 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.361+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c19cc70 con 0x7f6c8c106670 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.361+0000 7f6c88ff9700 1 -- 172.21.15.49:0/1948768727 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c7c019710 con 0x7f6c8c106670 2024-05-15T15:44:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.361+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c19d110 con 0x7f6c8c106670 2024-05-15T15:44:10.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.362+0000 7f6c88ff9700 1 -- 172.21.15.49:0/1948768727 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 45450+0+0 (secure 0 0 0) 0x7f6c7c019930 con 0x7f6c8c106670 2024-05-15T15:44:10.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.362+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c10aaa0 con 0x7f6c8c106670 2024-05-15T15:44:10.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.362+0000 7f6c88ff9700 1 --2- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f6c74038680 0x7f6c7403ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:10.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.362+0000 7f6c88ff9700 1 -- 172.21.15.49:0/1948768727 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6c7c04c930 con 0x7f6c8c106670 2024-05-15T15:44:10.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.362+0000 7f6c8affd700 1 --2- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f6c74038680 0x7f6c7403ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:10.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.363+0000 7f6c8affd700 1 --2- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f6c74038680 0x7f6c7403ab40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6c80006fd0 tx=0x7f6c80006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:10.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.368+0000 7f6c88ff9700 1 -- 172.21.15.49:0/1948768727 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6c7c018b80 con 0x7f6c8c106670 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.415+0000 7f6c88ff9700 1 -- 172.21.15.49:0/1948768727 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f6c7c01a940 con 0x7f6c8c106670 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.588+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c8c19fa00 con 0x7f6c74038680 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.597+0000 7f6c88ff9700 1 -- 172.21.15.49:0/1948768727 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6c8c19fa00 con 0x7f6c74038680 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.599+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f6c74038680 msgr2=0x7f6c7403ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.599+0000 7f6c92044700 1 --2- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f6c74038680 0x7f6c7403ab40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6c80006fd0 tx=0x7f6c80006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.599+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 msgr2=0x7f6c8c19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.599+0000 7f6c92044700 1 --2- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c19c530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c011710 tx=0x7f6c7c0047c0 comp rx=0 tx=0).stop 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.600+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 shutdown_connections 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.600+0000 7f6c92044700 1 --2- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f6c74038680 0x7f6c7403ab40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.600+0000 7f6c92044700 1 --2- 172.21.15.49:0/1948768727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c106670 0x7f6c8c19c530 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.600+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 >> 172.21.15.49:0/1948768727 conn(0x7f6c8c101c90 msgr2=0x7f6c8c1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.600+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 shutdown_connections 2024-05-15T15:44:10.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:09.600+0000 7f6c92044700 1 -- 172.21.15.49:0/1948768727 wait complete. 2024-05-15T15:44:10.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:10 smithi049 ceph-mon[28399]: [15/May/2024:15:44:09] ENGINE Serving on https://172.21.15.49:7150 2024-05-15T15:44:10.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:10 smithi049 ceph-mon[28399]: [15/May/2024:15:44:09] ENGINE Serving on http://172.21.15.49:8765 2024-05-15T15:44:10.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:10 smithi049 ceph-mon[28399]: [15/May/2024:15:44:09] ENGINE Bus STARTED 2024-05-15T15:44:10.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:10 smithi049 ceph-mon[28399]: mgrmap e8: smithi049.vuhzgf(active, since 2s) 2024-05-15T15:44:10.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:10 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:44:10.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:10 smithi049 ceph-mon[28399]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:10.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:10 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:10.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:10 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:44:11.245 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-15T15:44:11.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.502+0000 7f3b4c286700 1 Processor -- start 2024-05-15T15:44:11.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.502+0000 7f3b4c286700 1 -- start start 2024-05-15T15:44:11.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.503+0000 7f3b4c286700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b44106aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:11.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.503+0000 7f3b4c286700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44107070 con 0x7f3b44106680 2024-05-15T15:44:11.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.503+0000 7f3b4a022700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b44106aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:11.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.503+0000 7f3b4a022700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b44106aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35134/0 (socket says 172.21.15.49:35134) 2024-05-15T15:44:11.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.503+0000 7f3b4a022700 1 -- 172.21.15.49:0/1455284681 learned_addr learned my addr 172.21.15.49:0/1455284681 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:11.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.503+0000 7f3b4a022700 1 -- 172.21.15.49:0/1455284681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b44107830 con 0x7f3b44106680 2024-05-15T15:44:11.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.503+0000 7f3b4a022700 1 --2- 172.21.15.49:0/1455284681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b44106aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b400098a0 tx=0x7f3b40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27d201b7f9d14e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:11.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b49020700 1 -- 172.21.15.49:0/1455284681 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b40004180 con 0x7f3b44106680 2024-05-15T15:44:11.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b49020700 1 -- 172.21.15.49:0/1455284681 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3b400042e0 con 0x7f3b44106680 2024-05-15T15:44:11.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b49020700 1 -- 172.21.15.49:0/1455284681 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b40011a70 con 0x7f3b44106680 2024-05-15T15:44:11.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b4c286700 1 -- 172.21.15.49:0/1455284681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 msgr2=0x7f3b44106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:11.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b4c286700 1 --2- 172.21.15.49:0/1455284681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b44106aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b400098a0 tx=0x7f3b40009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:11.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b4c286700 1 -- 172.21.15.49:0/1455284681 shutdown_connections 2024-05-15T15:44:11.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b4c286700 1 --2- 172.21.15.49:0/1455284681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b44106aa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:11.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b4c286700 1 -- 172.21.15.49:0/1455284681 >> 172.21.15.49:0/1455284681 conn(0x7f3b44101c40 msgr2=0x7f3b44104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:11.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b4c286700 1 -- 172.21.15.49:0/1455284681 shutdown_connections 2024-05-15T15:44:11.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.504+0000 7f3b4c286700 1 -- 172.21.15.49:0/1455284681 wait complete. 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.505+0000 7f3b4c286700 1 Processor -- start 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.505+0000 7f3b4c286700 1 -- start start 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.505+0000 7f3b4c286700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b4419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.505+0000 7f3b4c286700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4419cac0 con 0x7f3b44106680 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.505+0000 7f3b4a022700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b4419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.505+0000 7f3b4a022700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b4419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35150/0 (socket says 172.21.15.49:35150) 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.505+0000 7f3b4a022700 1 -- 172.21.15.49:0/3637894521 learned_addr learned my addr 172.21.15.49:0/3637894521 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.506+0000 7f3b4a022700 1 -- 172.21.15.49:0/3637894521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b40009550 con 0x7f3b44106680 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.506+0000 7f3b4a022700 1 --2- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b4419c580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b40003df0 tx=0x7f3b400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:11.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.506+0000 7f3b377fe700 1 -- 172.21.15.49:0/3637894521 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b40003a00 con 0x7f3b44106680 2024-05-15T15:44:11.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.506+0000 7f3b377fe700 1 -- 172.21.15.49:0/3637894521 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3b40003b60 con 0x7f3b44106680 2024-05-15T15:44:11.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.506+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4419ccc0 con 0x7f3b44106680 2024-05-15T15:44:11.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.506+0000 7f3b377fe700 1 -- 172.21.15.49:0/3637894521 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b40023770 con 0x7f3b44106680 2024-05-15T15:44:11.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.506+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4419d160 con 0x7f3b44106680 2024-05-15T15:44:11.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.507+0000 7f3b377fe700 1 -- 172.21.15.49:0/3637894521 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f3b4002c3f0 con 0x7f3b44106680 2024-05-15T15:44:11.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.507+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4410aa60 con 0x7f3b44106680 2024-05-15T15:44:11.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.507+0000 7f3b377fe700 1 --2- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f3b300387f0 0x7f3b3003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:11.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.507+0000 7f3b377fe700 1 -- 172.21.15.49:0/3637894521 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3b4004cb30 con 0x7f3b44106680 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.508+0000 7f3b49821700 1 --2- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f3b300387f0 0x7f3b3003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.508+0000 7f3b49821700 1 --2- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f3b300387f0 0x7f3b3003acb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b38006fd0 tx=0x7f3b38006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.513+0000 7f3b377fe700 1 -- 172.21.15.49:0/3637894521 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3b4000bb90 con 0x7f3b44106680 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.720+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b4419fa50 con 0x7f3b300387f0 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.721+0000 7f3b377fe700 1 -- 172.21.15.49:0/3637894521 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f3b4419fa50 con 0x7f3b300387f0 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f3b300387f0 msgr2=0x7f3b3003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 --2- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f3b300387f0 0x7f3b3003acb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b38006fd0 tx=0x7f3b38006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 msgr2=0x7f3b4419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 --2- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b4419c580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b40003df0 tx=0x7f3b400047c0 comp rx=0 tx=0).stop 2024-05-15T15:44:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 shutdown_connections 2024-05-15T15:44:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 --2- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f3b300387f0 0x7f3b3003acb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 --2- 172.21.15.49:0/3637894521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b44106680 0x7f3b4419c580 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 >> 172.21.15.49:0/3637894521 conn(0x7f3b44101c40 msgr2=0x7f3b44102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 shutdown_connections 2024-05-15T15:44:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:10.723+0000 7f3b4c286700 1 -- 172.21.15.49:0/3637894521 wait complete. 2024-05-15T15:44:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:Generating ssh key... 2024-05-15T15:44:11.627 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:11 smithi049 ceph-mon[28399]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:12.521 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.658+0000 7f6289c84700 1 Processor -- start 2024-05-15T15:44:12.522 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.658+0000 7f6289c84700 1 -- start start 2024-05-15T15:44:12.522 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.658+0000 7f6289c84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f6284106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:12.522 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.658+0000 7f6289c84700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284107050 con 0x7f6284106660 2024-05-15T15:44:12.522 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.658+0000 7f62837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f6284106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:12.522 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.658+0000 7f62837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f6284106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35162/0 (socket says 172.21.15.49:35162) 2024-05-15T15:44:12.522 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.658+0000 7f62837fe700 1 -- 172.21.15.49:0/2619159185 learned_addr learned my addr 172.21.15.49:0/2619159185 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:12.522 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.659+0000 7f62837fe700 1 -- 172.21.15.49:0/2619159185 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284107860 con 0x7f6284106660 2024-05-15T15:44:12.522 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.659+0000 7f62837fe700 1 --2- 172.21.15.49:0/2619159185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f6284106a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f62740098a0 tx=0x7f6274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d4b59a146cafb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:12.522 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.659+0000 7f62827fc700 1 -- 172.21.15.49:0/2619159185 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6274004180 con 0x7f6284106660 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.659+0000 7f62827fc700 1 -- 172.21.15.49:0/2619159185 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f62740042e0 con 0x7f6284106660 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.659+0000 7f62827fc700 1 -- 172.21.15.49:0/2619159185 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6274011a70 con 0x7f6284106660 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.659+0000 7f6289c84700 1 -- 172.21.15.49:0/2619159185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 msgr2=0x7f6284106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.659+0000 7f6289c84700 1 --2- 172.21.15.49:0/2619159185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f6284106a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f62740098a0 tx=0x7f6274009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.660+0000 7f6289c84700 1 -- 172.21.15.49:0/2619159185 shutdown_connections 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.660+0000 7f6289c84700 1 --2- 172.21.15.49:0/2619159185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f6284106a80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.660+0000 7f6289c84700 1 -- 172.21.15.49:0/2619159185 >> 172.21.15.49:0/2619159185 conn(0x7f6284101c40 msgr2=0x7f6284104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.660+0000 7f6289c84700 1 -- 172.21.15.49:0/2619159185 shutdown_connections 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.660+0000 7f6289c84700 1 -- 172.21.15.49:0/2619159185 wait complete. 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.660+0000 7f6289c84700 1 Processor -- start 2024-05-15T15:44:12.523 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.660+0000 7f6289c84700 1 -- start start 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f6289c84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f628419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f6289c84700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628419a840 con 0x7f6284106660 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f62837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f628419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f62837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f628419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35164/0 (socket says 172.21.15.49:35164) 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f62837fe700 1 -- 172.21.15.49:0/2756362114 learned_addr learned my addr 172.21.15.49:0/2756362114 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f62837fe700 1 -- 172.21.15.49:0/2756362114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009550 con 0x7f6284106660 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f62837fe700 1 --2- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f628419a300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6274011710 tx=0x7f62740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f6280ff9700 1 -- 172.21.15.49:0/2756362114 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6274003b10 con 0x7f6284106660 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628419aa40 con 0x7f6284106660 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f6280ff9700 1 -- 172.21.15.49:0/2756362114 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f627401b8c0 con 0x7f6284106660 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f6280ff9700 1 -- 172.21.15.49:0/2756362114 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6274019770 con 0x7f6284106660 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.661+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628419aee0 con 0x7f6284106660 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.662+0000 7f6280ff9700 1 -- 172.21.15.49:0/2756362114 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f62740199f0 con 0x7f6284106660 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.662+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628410aa90 con 0x7f6284106660 2024-05-15T15:44:12.524 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.663+0000 7f6280ff9700 1 --2- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f626c0387e0 0x7f626c03aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.663+0000 7f6280ff9700 1 -- 172.21.15.49:0/2756362114 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f627404c920 con 0x7f6284106660 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.663+0000 7f6282ffd700 1 --2- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f626c0387e0 0x7f626c03aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.663+0000 7f6282ffd700 1 --2- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f626c0387e0 0x7f626c03aca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6278006fd0 tx=0x7f6278006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.668+0000 7f6280ff9700 1 -- 172.21.15.49:0/2756362114 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6274018db0 con 0x7f6284106660 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.872+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6284102e90 con 0x7f626c0387e0 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.936+0000 7f6280ff9700 1 -- 172.21.15.49:0/2756362114 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6284102e90 con 0x7f626c0387e0 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.938+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f626c0387e0 msgr2=0x7f626c03aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.938+0000 7f6289c84700 1 --2- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f626c0387e0 0x7f626c03aca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6278006fd0 tx=0x7f6278006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.938+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 msgr2=0x7f628419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.938+0000 7f6289c84700 1 --2- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f628419a300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6274011710 tx=0x7f62740047c0 comp rx=0 tx=0).stop 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.938+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 shutdown_connections 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.938+0000 7f6289c84700 1 --2- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f626c0387e0 0x7f626c03aca0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.938+0000 7f6289c84700 1 --2- 172.21.15.49:0/2756362114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6284106660 0x7f628419a300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:12.525 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.938+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 >> 172.21.15.49:0/2756362114 conn(0x7f6284101c40 msgr2=0x7f6284102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:12.526 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.938+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 shutdown_connections 2024-05-15T15:44:12.526 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:11.939+0000 7f6289c84700 1 -- 172.21.15.49:0/2756362114 wait complete. 2024-05-15T15:44:13.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:12 smithi049 ceph-mon[28399]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:13.104 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:12 smithi049 ceph-mon[28399]: Generating ssh key... 2024-05-15T15:44:13.104 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:12 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:13.104 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:12 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:13.638 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfEG9qhxHuubmj2d+B5tK5GzpmTBkzdiBJ/g1LvRDWFffflgX2Cc1lr2G0Er/MlP7/RORAl1cVpTCk7K3Q9cmQOnbB0rjSAt8SaDBSnhq6eNUJzYUKkoPxBu2CNOSYH5dihV0CIVYneejxwqTbt243X6Lyc3isA3+K1XyUwTBm5vsK6vcIZ8346B2CpCHTz1yICCKcroLi2EcTTRFBzP7fyskHV68PQ+yNDLfun7ThQN0enHaRyWJuYbFjqQ387GI9e+DLterBT2tkFp/czKOx/3FFLAM5NIpwpUVeP6W1OSWeohOo3Fe+9X4fst3vtvGlYOJ57CnE1wK0iTWFg9TPltwiFDL43bFoII48TOS95AJM+WVADPLak8BmeuCjTD2XL3RnDbuF5VowI+es7iiPH9nac/3Je/UzdZItaG5ieAm7P3rAwERFA9rt6NUalJgay8rI87Tl66E2dGAN93KiV+aVMXrKT+7sY3GMhFybJh1Ph/WrXED42uDflZNe47E= ceph-c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:44:13.638 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.886+0000 7f63ebf4c700 1 Processor -- start 2024-05-15T15:44:13.638 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.886+0000 7f63ebf4c700 1 -- start start 2024-05-15T15:44:13.638 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.886+0000 7f63ebf4c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e41059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:13.638 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.887+0000 7f63ebf4c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4105f80 con 0x7f63e4105590 2024-05-15T15:44:13.638 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.887+0000 7f63e9ce8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e41059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.887+0000 7f63e9ce8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e41059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35174/0 (socket says 172.21.15.49:35174) 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.887+0000 7f63e9ce8700 1 -- 172.21.15.49:0/609454604 learned_addr learned my addr 172.21.15.49:0/609454604 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.887+0000 7f63e9ce8700 1 -- 172.21.15.49:0/609454604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4106790 con 0x7f63e4105590 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.887+0000 7f63e9ce8700 1 --2- 172.21.15.49:0/609454604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e41059b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63e00098a0 tx=0x7f63e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae12d6b5f553f25b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.887+0000 7f63e8ce6700 1 -- 172.21.15.49:0/609454604 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e0004180 con 0x7f63e4105590 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.887+0000 7f63e8ce6700 1 -- 172.21.15.49:0/609454604 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f63e00042e0 con 0x7f63e4105590 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.887+0000 7f63e8ce6700 1 -- 172.21.15.49:0/609454604 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e0011a70 con 0x7f63e4105590 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.888+0000 7f63ebf4c700 1 -- 172.21.15.49:0/609454604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 msgr2=0x7f63e41059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.888+0000 7f63ebf4c700 1 --2- 172.21.15.49:0/609454604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e41059b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63e00098a0 tx=0x7f63e0009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.888+0000 7f63ebf4c700 1 -- 172.21.15.49:0/609454604 shutdown_connections 2024-05-15T15:44:13.639 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.888+0000 7f63ebf4c700 1 --2- 172.21.15.49:0/609454604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e41059b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.888+0000 7f63ebf4c700 1 -- 172.21.15.49:0/609454604 >> 172.21.15.49:0/609454604 conn(0x7f63e4100af0 msgr2=0x7f63e4102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.888+0000 7f63ebf4c700 1 -- 172.21.15.49:0/609454604 shutdown_connections 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.888+0000 7f63ebf4c700 1 -- 172.21.15.49:0/609454604 wait complete. 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.889+0000 7f63ebf4c700 1 Processor -- start 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.889+0000 7f63ebf4c700 1 -- start start 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.889+0000 7f63ebf4c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.889+0000 7f63ebf4c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e419a960 con 0x7f63e4105590 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.889+0000 7f63e9ce8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.889+0000 7f63e9ce8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35190/0 (socket says 172.21.15.49:35190) 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.889+0000 7f63e9ce8700 1 -- 172.21.15.49:0/1365097595 learned_addr learned my addr 172.21.15.49:0/1365097595 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.889+0000 7f63e9ce8700 1 -- 172.21.15.49:0/1365097595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0009550 con 0x7f63e4105590 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.889+0000 7f63e9ce8700 1 --2- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e419a420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f63e0011710 tx=0x7f63e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.890+0000 7f63daffd700 1 -- 172.21.15.49:0/1365097595 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e0003b10 con 0x7f63e4105590 2024-05-15T15:44:13.640 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.890+0000 7f63daffd700 1 -- 172.21.15.49:0/1365097595 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f63e001b8c0 con 0x7f63e4105590 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.890+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e419ab60 con 0x7f63e4105590 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.890+0000 7f63daffd700 1 -- 172.21.15.49:0/1365097595 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e0019710 con 0x7f63e4105590 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.890+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e419b000 con 0x7f63e4105590 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.890+0000 7f63daffd700 1 -- 172.21.15.49:0/1365097595 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f63e0019930 con 0x7f63e4105590 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.890+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e41945b0 con 0x7f63e4105590 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.891+0000 7f63daffd700 1 --2- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f63d00386e0 0x7f63d003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.891+0000 7f63daffd700 1 -- 172.21.15.49:0/1365097595 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f63e004ccb0 con 0x7f63e4105590 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.891+0000 7f63e94e7700 1 --2- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f63d00386e0 0x7f63d003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.891+0000 7f63e94e7700 1 --2- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f63d00386e0 0x7f63d003aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f63d4006fd0 tx=0x7f63d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:12.896+0000 7f63daffd700 1 -- 172.21.15.49:0/1365097595 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f63e001b440 con 0x7f63e4105590 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.102+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f63e402d050 con 0x7f63d00386e0 2024-05-15T15:44:13.641 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.103+0000 7f63daffd700 1 -- 172.21.15.49:0/1365097595 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f63e402d050 con 0x7f63d00386e0 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f63d00386e0 msgr2=0x7f63d003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 --2- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f63d00386e0 0x7f63d003aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f63d4006fd0 tx=0x7f63d4006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 msgr2=0x7f63e419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 --2- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e419a420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f63e0011710 tx=0x7f63e00047c0 comp rx=0 tx=0).stop 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 shutdown_connections 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 --2- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f63d00386e0 0x7f63d003aba0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 --2- 172.21.15.49:0/1365097595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63e4105590 0x7f63e419a420 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 >> 172.21.15.49:0/1365097595 conn(0x7f63e4100af0 msgr2=0x7f63e4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 shutdown_connections 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:13.106+0000 7f63ebf4c700 1 -- 172.21.15.49:0/1365097595 wait complete. 2024-05-15T15:44:13.642 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-15T15:44:13.643 INFO:teuthology.orchestra.run.smithi049.stdout:Adding key to root@localhost authorized_keys... 2024-05-15T15:44:13.643 INFO:teuthology.orchestra.run.smithi049.stdout:Adding host smithi049... 2024-05-15T15:44:14.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:13 smithi049 ceph-mon[28399]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:15.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:15 smithi049 ceph-mon[28399]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:15.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:15 smithi049 ceph-mon[28399]: Deploying cephadm binary to smithi049 2024-05-15T15:44:17.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Added host 'smithi049' with addr '172.21.15.49' 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.087+0000 7f7ab3a63700 1 Processor -- start 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.088+0000 7f7ab3a63700 1 -- start start 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.088+0000 7f7ab3a63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac1094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.088+0000 7f7ab3a63700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac079e60 con 0x7f7aac107100 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.088+0000 7f7ab17ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac1094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.088+0000 7f7ab17ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac1094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40274/0 (socket says 172.21.15.49:40274) 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.088+0000 7f7ab17ff700 1 -- 172.21.15.49:0/2326850883 learned_addr learned my addr 172.21.15.49:0/2326850883 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.089+0000 7f7ab17ff700 1 -- 172.21.15.49:0/2326850883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac109a30 con 0x7f7aac107100 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.089+0000 7f7ab17ff700 1 --2- 172.21.15.49:0/2326850883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac1094f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80098a0 tx=0x7f7aa8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bf35bbf9c51033 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.089+0000 7f7aa3fff700 1 -- 172.21.15.49:0/2326850883 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7aa8004180 con 0x7f7aac107100 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.089+0000 7f7aa3fff700 1 -- 172.21.15.49:0/2326850883 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7aa80042e0 con 0x7f7aac107100 2024-05-15T15:44:17.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.089+0000 7f7aa3fff700 1 -- 172.21.15.49:0/2326850883 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7aa8011a70 con 0x7f7aac107100 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.090+0000 7f7ab3a63700 1 -- 172.21.15.49:0/2326850883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 msgr2=0x7f7aac1094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.090+0000 7f7ab3a63700 1 --2- 172.21.15.49:0/2326850883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac1094f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80098a0 tx=0x7f7aa8009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.090+0000 7f7ab3a63700 1 -- 172.21.15.49:0/2326850883 shutdown_connections 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.090+0000 7f7ab3a63700 1 --2- 172.21.15.49:0/2326850883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac1094f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.090+0000 7f7ab3a63700 1 -- 172.21.15.49:0/2326850883 >> 172.21.15.49:0/2326850883 conn(0x7f7aac100b50 msgr2=0x7f7aac102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.090+0000 7f7ab3a63700 1 -- 172.21.15.49:0/2326850883 shutdown_connections 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.090+0000 7f7ab3a63700 1 -- 172.21.15.49:0/2326850883 wait complete. 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.091+0000 7f7ab3a63700 1 Processor -- start 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.091+0000 7f7ab3a63700 1 -- start start 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.091+0000 7f7ab3a63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac106930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.091+0000 7f7ab3a63700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac106e70 con 0x7f7aac107100 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.091+0000 7f7ab17ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.091+0000 7f7ab17ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40290/0 (socket says 172.21.15.49:40290) 2024-05-15T15:44:17.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.091+0000 7f7ab17ff700 1 -- 172.21.15.49:0/3945687712 learned_addr learned my addr 172.21.15.49:0/3945687712 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.091+0000 7f7ab17ff700 1 -- 172.21.15.49:0/3945687712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa8009550 con 0x7f7aac107100 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.091+0000 7f7ab17ff700 1 --2- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac106930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8011a20 tx=0x7f7aa80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.092+0000 7f7aa27fc700 1 -- 172.21.15.49:0/3945687712 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7aa8003b10 con 0x7f7aac107100 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.092+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac1050b0 con 0x7f7aac107100 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.092+0000 7f7aa27fc700 1 -- 172.21.15.49:0/3945687712 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7aa801b8c0 con 0x7f7aac107100 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.092+0000 7f7aa27fc700 1 -- 172.21.15.49:0/3945687712 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7aa80237b0 con 0x7f7aac107100 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.092+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac105550 con 0x7f7aac107100 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.092+0000 7f7aa27fc700 1 -- 172.21.15.49:0/3945687712 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f7aa8023a30 con 0x7f7aac107100 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.093+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac10a780 con 0x7f7aac107100 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.093+0000 7f7aa27fc700 1 --2- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f7a980387a0 0x7f7a9803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.093+0000 7f7aa27fc700 1 -- 172.21.15.49:0/3945687712 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7aa804ca50 con 0x7f7aac107100 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.093+0000 7f7ab0ffe700 1 --2- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f7a980387a0 0x7f7a9803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:17.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.093+0000 7f7ab0ffe700 1 --2- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f7a980387a0 0x7f7a9803ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c006fd0 tx=0x7f7a9c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.098+0000 7f7aa27fc700 1 -- 172.21.15.49:0/3945687712 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7aa8023ce0 con 0x7f7aac107100 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:14.308+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}) v1 -- 0x7f7aac105d00 con 0x7f7a980387a0 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.814+0000 7f7aa27fc700 1 -- 172.21.15.49:0/3945687712 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f7aac105d00 con 0x7f7a980387a0 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f7a980387a0 msgr2=0x7f7a9803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 --2- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f7a980387a0 0x7f7a9803ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c006fd0 tx=0x7f7a9c006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 msgr2=0x7f7aac106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 --2- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac106930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8011a20 tx=0x7f7aa80047c0 comp rx=0 tx=0).stop 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 shutdown_connections 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 --2- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f7a980387a0 0x7f7a9803ac60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 --2- 172.21.15.49:0/3945687712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aac107100 0x7f7aac106930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 >> 172.21.15.49:0/3945687712 conn(0x7f7aac100b50 msgr2=0x7f7aac101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 shutdown_connections 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:16.817+0000 7f7ab3a63700 1 -- 172.21.15.49:0/3945687712 wait complete. 2024-05-15T15:44:17.381 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying mon service with default placement... 2024-05-15T15:44:17.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:17 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:17.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:17 smithi049 ceph-mon[28399]: Added host smithi049 2024-05-15T15:44:17.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:17 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:44:19.324 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:19 smithi049 ceph-mon[28399]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:19.324 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:19 smithi049 ceph-mon[28399]: Saving service mon spec with placement count:5 2024-05-15T15:44:19.324 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:19 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:19.324 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:19 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:19.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-15T15:44:19.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.800+0000 7efedbe9f700 1 Processor -- start 2024-05-15T15:44:19.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.800+0000 7efedbe9f700 1 -- start start 2024-05-15T15:44:19.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.800+0000 7efedbe9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed4106b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:19.332 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.800+0000 7efedbe9f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed41070d0 con 0x7efed41066e0 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.800+0000 7efed9c3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed4106b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.800+0000 7efed9c3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed4106b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40292/0 (socket says 172.21.15.49:40292) 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.800+0000 7efed9c3b700 1 -- 172.21.15.49:0/634846715 learned_addr learned my addr 172.21.15.49:0/634846715 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.801+0000 7efed9c3b700 1 -- 172.21.15.49:0/634846715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4107890 con 0x7efed41066e0 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.801+0000 7efed9c3b700 1 --2- 172.21.15.49:0/634846715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed4106b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efed00098a0 tx=0x7efed0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745d6afdf0c2a707 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.801+0000 7efed8c39700 1 -- 172.21.15.49:0/634846715 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efed0004180 con 0x7efed41066e0 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.801+0000 7efed8c39700 1 -- 172.21.15.49:0/634846715 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7efed00042e0 con 0x7efed41066e0 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.801+0000 7efed8c39700 1 -- 172.21.15.49:0/634846715 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efed0011a70 con 0x7efed41066e0 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.801+0000 7efedbe9f700 1 -- 172.21.15.49:0/634846715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 msgr2=0x7efed4106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.801+0000 7efedbe9f700 1 --2- 172.21.15.49:0/634846715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed4106b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efed00098a0 tx=0x7efed0009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.802+0000 7efedbe9f700 1 -- 172.21.15.49:0/634846715 shutdown_connections 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.802+0000 7efedbe9f700 1 --2- 172.21.15.49:0/634846715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed4106b00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:19.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.802+0000 7efedbe9f700 1 -- 172.21.15.49:0/634846715 >> 172.21.15.49:0/634846715 conn(0x7efed4101ce0 msgr2=0x7efed4104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.802+0000 7efedbe9f700 1 -- 172.21.15.49:0/634846715 shutdown_connections 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.802+0000 7efedbe9f700 1 -- 172.21.15.49:0/634846715 wait complete. 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.802+0000 7efedbe9f700 1 Processor -- start 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.802+0000 7efedbe9f700 1 -- start start 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efedbe9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efedbe9f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419a860 con 0x7efed41066e0 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efed9c3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efed9c3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed419a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40296/0 (socket says 172.21.15.49:40296) 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efed9c3b700 1 -- 172.21.15.49:0/1720929133 learned_addr learned my addr 172.21.15.49:0/1720929133 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efed9c3b700 1 -- 172.21.15.49:0/1720929133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0009550 con 0x7efed41066e0 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efed9c3b700 1 --2- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed419a320 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efed0011710 tx=0x7efed00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efecaffd700 1 -- 172.21.15.49:0/1720929133 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efed0003b10 con 0x7efed41066e0 2024-05-15T15:44:19.334 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efecaffd700 1 -- 172.21.15.49:0/1720929133 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7efed001b8c0 con 0x7efed41066e0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efecaffd700 1 -- 172.21.15.49:0/1720929133 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efed0019770 con 0x7efed41066e0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed419aa60 con 0x7efed41066e0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.803+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed419af00 con 0x7efed41066e0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.804+0000 7efecaffd700 1 -- 172.21.15.49:0/1720929133 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7efed00199a0 con 0x7efed41066e0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.804+0000 7efecaffd700 1 --2- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7efec0038830 0x7efec003acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.804+0000 7efecaffd700 1 -- 172.21.15.49:0/1720929133 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efed004c8b0 con 0x7efed41066e0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.804+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed410aac0 con 0x7efed41066e0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.804+0000 7efed943a700 1 --2- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7efec0038830 0x7efec003acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.805+0000 7efed943a700 1 --2- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7efec0038830 0x7efec003acf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efec4006fd0 tx=0x7efec4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:17.810+0000 7efecaffd700 1 -- 172.21.15.49:0/1720929133 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efed001f080 con 0x7efed41066e0 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.029+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7efed4102eb0 con 0x7efec0038830 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.035+0000 7efecaffd700 1 -- 172.21.15.49:0/1720929133 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efed4102eb0 con 0x7efec0038830 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.037+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7efec0038830 msgr2=0x7efec003acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:19.335 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.037+0000 7efedbe9f700 1 --2- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7efec0038830 0x7efec003acf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efec4006fd0 tx=0x7efec4006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.037+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 msgr2=0x7efed419a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.037+0000 7efedbe9f700 1 --2- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed419a320 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efed0011710 tx=0x7efed00047c0 comp rx=0 tx=0).stop 2024-05-15T15:44:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.038+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 shutdown_connections 2024-05-15T15:44:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.038+0000 7efedbe9f700 1 --2- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7efec0038830 0x7efec003acf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.038+0000 7efedbe9f700 1 --2- 172.21.15.49:0/1720929133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed41066e0 0x7efed419a320 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.038+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 >> 172.21.15.49:0/1720929133 conn(0x7efed4101ce0 msgr2=0x7efed41027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.038+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 shutdown_connections 2024-05-15T15:44:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:18.038+0000 7efedbe9f700 1 -- 172.21.15.49:0/1720929133 wait complete. 2024-05-15T15:44:19.336 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying mgr service with default placement... 2024-05-15T15:44:20.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:20 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:20.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:20 smithi049 ceph-mon[28399]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:20.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:20 smithi049 ceph-mon[28399]: Saving service mgr spec with placement count:2 2024-05-15T15:44:20.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:20 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:21.000 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-15T15:44:21.000 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.742+0000 7f42421a9700 1 Processor -- start 2024-05-15T15:44:21.000 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.742+0000 7f42421a9700 1 -- start start 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.743+0000 7f42421a9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c106b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.743+0000 7f42421a9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c107140 con 0x7f423c106750 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.743+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.743+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c106b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40306/0 (socket says 172.21.15.49:40306) 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.743+0000 7f423b7fe700 1 -- 172.21.15.49:0/4223847328 learned_addr learned my addr 172.21.15.49:0/4223847328 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.743+0000 7f423b7fe700 1 -- 172.21.15.49:0/4223847328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c107900 con 0x7f423c106750 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.743+0000 7f423b7fe700 1 --2- 172.21.15.49:0/4223847328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c106b70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f422c0098a0 tx=0x7f422c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b807a6c61c905131 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.744+0000 7f423a7fc700 1 -- 172.21.15.49:0/4223847328 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f422c004180 con 0x7f423c106750 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.744+0000 7f423a7fc700 1 -- 172.21.15.49:0/4223847328 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f422c0042e0 con 0x7f423c106750 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.744+0000 7f423a7fc700 1 -- 172.21.15.49:0/4223847328 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f422c011b10 con 0x7f423c106750 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.745+0000 7f42421a9700 1 -- 172.21.15.49:0/4223847328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 msgr2=0x7f423c106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.745+0000 7f42421a9700 1 --2- 172.21.15.49:0/4223847328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c106b70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f422c0098a0 tx=0x7f422c009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.745+0000 7f42421a9700 1 -- 172.21.15.49:0/4223847328 shutdown_connections 2024-05-15T15:44:21.001 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.745+0000 7f42421a9700 1 --2- 172.21.15.49:0/4223847328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c106b70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:21.002 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.745+0000 7f42421a9700 1 -- 172.21.15.49:0/4223847328 >> 172.21.15.49:0/4223847328 conn(0x7f423c101cd0 msgr2=0x7f423c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:21.002 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.745+0000 7f42421a9700 1 -- 172.21.15.49:0/4223847328 shutdown_connections 2024-05-15T15:44:21.002 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.745+0000 7f42421a9700 1 -- 172.21.15.49:0/4223847328 wait complete. 2024-05-15T15:44:21.002 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.746+0000 7f42421a9700 1 Processor -- start 2024-05-15T15:44:21.002 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.746+0000 7f42421a9700 1 -- start start 2024-05-15T15:44:21.002 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.746+0000 7f42421a9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:21.002 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.746+0000 7f42421a9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c19a920 con 0x7f423c106750 2024-05-15T15:44:21.002 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.746+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:21.002 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.746+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40318/0 (socket says 172.21.15.49:40318) 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.747+0000 7f423b7fe700 1 -- 172.21.15.49:0/1856309888 learned_addr learned my addr 172.21.15.49:0/1856309888 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.747+0000 7f423b7fe700 1 -- 172.21.15.49:0/1856309888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c009550 con 0x7f423c106750 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.747+0000 7f423b7fe700 1 --2- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c19a3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f422c009520 tx=0x7f422c006b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.747+0000 7f4238ff9700 1 -- 172.21.15.49:0/1856309888 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f422c011b10 con 0x7f423c106750 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.747+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c19ab20 con 0x7f423c106750 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.747+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c19afc0 con 0x7f423c106750 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.747+0000 7f4238ff9700 1 -- 172.21.15.49:0/1856309888 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f422c003a70 con 0x7f423c106750 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.747+0000 7f4238ff9700 1 -- 172.21.15.49:0/1856309888 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f422c0196e0 con 0x7f423c106750 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.748+0000 7f4238ff9700 1 -- 172.21.15.49:0/1856309888 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f422c019950 con 0x7f423c106750 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.748+0000 7f4238ff9700 1 --2- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f42240387f0 0x7f422403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.748+0000 7f4238ff9700 1 -- 172.21.15.49:0/1856309888 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f422c04d820 con 0x7f423c106750 2024-05-15T15:44:21.003 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.748+0000 7f423affd700 1 --2- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f42240387f0 0x7f422403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.748+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4228005320 con 0x7f423c106750 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.749+0000 7f423affd700 1 --2- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f42240387f0 0x7f422403acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4230006fd0 tx=0x7f4230006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:19.755+0000 7f4238ff9700 1 -- 172.21.15.49:0/1856309888 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f422c018db0 con 0x7f423c106750 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.009+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f4228000bf0 con 0x7f42240387f0 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.014+0000 7f4238ff9700 1 -- 172.21.15.49:0/1856309888 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4228000bf0 con 0x7f42240387f0 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f42240387f0 msgr2=0x7f422403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 --2- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f42240387f0 0x7f422403acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4230006fd0 tx=0x7f4230006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 msgr2=0x7f423c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 --2- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c19a3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f422c009520 tx=0x7f422c006b40 comp rx=0 tx=0).stop 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 shutdown_connections 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 --2- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f42240387f0 0x7f422403acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 --2- 172.21.15.49:0/1856309888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f423c106750 0x7f423c19a3e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 >> 172.21.15.49:0/1856309888 conn(0x7f423c101cd0 msgr2=0x7f423c1027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:21.004 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 shutdown_connections 2024-05-15T15:44:21.005 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:20.016+0000 7f42421a9700 1 -- 172.21.15.49:0/1856309888 wait complete. 2024-05-15T15:44:21.005 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying crash service with default placement... 2024-05-15T15:44:22.145 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:21 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:22.146 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:21 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:22.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-05-15T15:44:22.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.459+0000 7f78b7ec2700 1 Processor -- start 2024-05-15T15:44:22.258 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.460+0000 7f78b7ec2700 1 -- start start 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.460+0000 7f78b7ec2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b0109500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.460+0000 7f78b7ec2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0079ea0 con 0x7f78b0107110 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.460+0000 7f78b5c5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b0109500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.460+0000 7f78b5c5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b0109500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40328/0 (socket says 172.21.15.49:40328) 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.460+0000 7f78b5c5e700 1 -- 172.21.15.49:0/1625481083 learned_addr learned my addr 172.21.15.49:0/1625481083 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.461+0000 7f78b5c5e700 1 -- 172.21.15.49:0/1625481083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b0109a40 con 0x7f78b0107110 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.461+0000 7f78b5c5e700 1 --2- 172.21.15.49:0/1625481083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b0109500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0098a0 tx=0x7f78ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62781cbfed730ef server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.461+0000 7f78b4c5c700 1 -- 172.21.15.49:0/1625481083 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78ac004180 con 0x7f78b0107110 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.461+0000 7f78b4c5c700 1 -- 172.21.15.49:0/1625481083 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78ac0042e0 con 0x7f78b0107110 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.461+0000 7f78b4c5c700 1 -- 172.21.15.49:0/1625481083 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78ac011b10 con 0x7f78b0107110 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.462+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1625481083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 msgr2=0x7f78b0109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:22.259 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.462+0000 7f78b7ec2700 1 --2- 172.21.15.49:0/1625481083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b0109500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0098a0 tx=0x7f78ac009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.462+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1625481083 shutdown_connections 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.462+0000 7f78b7ec2700 1 --2- 172.21.15.49:0/1625481083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b0109500 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.462+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1625481083 >> 172.21.15.49:0/1625481083 conn(0x7f78b0100b60 msgr2=0x7f78b0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.462+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1625481083 shutdown_connections 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.462+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1625481083 wait complete. 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.462+0000 7f78b7ec2700 1 Processor -- start 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.462+0000 7f78b7ec2700 1 -- start start 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78b7ec2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78b7ec2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b019cab0 con 0x7f78b0107110 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78b5c5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78b5c5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b019c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40342/0 (socket says 172.21.15.49:40342) 2024-05-15T15:44:22.260 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78b5c5e700 1 -- 172.21.15.49:0/1329888002 learned_addr learned my addr 172.21.15.49:0/1329888002 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78b5c5e700 1 -- 172.21.15.49:0/1329888002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac009550 con 0x7f78b0107110 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78b5c5e700 1 --2- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b019c570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009fb0 tx=0x7f78ac011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78a6ffd700 1 -- 172.21.15.49:0/1329888002 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78ac0038b0 con 0x7f78b0107110 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b019ccb0 con 0x7f78b0107110 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78a6ffd700 1 -- 172.21.15.49:0/1329888002 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78ac003a10 con 0x7f78b0107110 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78a6ffd700 1 -- 172.21.15.49:0/1329888002 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78ac019750 con 0x7f78b0107110 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.463+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b019d150 con 0x7f78b0107110 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.464+0000 7f78a6ffd700 1 -- 172.21.15.49:0/1329888002 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f78ac0199c0 con 0x7f78b0107110 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.464+0000 7f78a6ffd700 1 --2- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f789c0387a0 0x7f789c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.464+0000 7f78a6ffd700 1 -- 172.21.15.49:0/1329888002 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f78ac04c810 con 0x7f78b0107110 2024-05-15T15:44:22.261 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.465+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b010a830 con 0x7f78b0107110 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.465+0000 7f78b545d700 1 --2- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f789c0387a0 0x7f789c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.465+0000 7f78b545d700 1 --2- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f789c0387a0 0x7f789c03ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006fd0 tx=0x7f78a0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.474+0000 7f78a6ffd700 1 -- 172.21.15.49:0/1329888002 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f78ac019cc0 con 0x7f78b0107110 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.712+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f78b0101d20 con 0x7f789c0387a0 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.718+0000 7f78a6ffd700 1 -- 172.21.15.49:0/1329888002 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f78b0101d20 con 0x7f789c0387a0 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f789c0387a0 msgr2=0x7f789c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 --2- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f789c0387a0 0x7f789c03ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006fd0 tx=0x7f78a0006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 msgr2=0x7f78b019c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 --2- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b019c570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009fb0 tx=0x7f78ac011890 comp rx=0 tx=0).stop 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 shutdown_connections 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 --2- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f789c0387a0 0x7f789c03ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 --2- 172.21.15.49:0/1329888002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0107110 0x7f78b019c570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:22.262 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 >> 172.21.15.49:0/1329888002 conn(0x7f78b0100b60 msgr2=0x7f78b0101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:22.263 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 shutdown_connections 2024-05-15T15:44:22.263 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:21.720+0000 7f78b7ec2700 1 -- 172.21.15.49:0/1329888002 wait complete. 2024-05-15T15:44:22.263 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying ceph-exporter service with default placement... 2024-05-15T15:44:23.173 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:22 smithi049 ceph-mon[28399]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:23.174 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:22 smithi049 ceph-mon[28399]: Saving service crash spec with placement * 2024-05-15T15:44:23.174 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:22 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:23.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-05-15T15:44:23.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.620+0000 7f88a280f700 1 Processor -- start 2024-05-15T15:44:23.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.620+0000 7f88a280f700 1 -- start start 2024-05-15T15:44:23.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.620+0000 7f88a280f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:23.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.620+0000 7f88a280f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c107050 con 0x7f889c106660 2024-05-15T15:44:23.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.621+0000 7f889bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.621+0000 7f889bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40346/0 (socket says 172.21.15.49:40346) 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.621+0000 7f889bfff700 1 -- 172.21.15.49:0/3548973951 learned_addr learned my addr 172.21.15.49:0/3548973951 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.621+0000 7f889bfff700 1 -- 172.21.15.49:0/3548973951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c107860 con 0x7f889c106660 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.621+0000 7f889bfff700 1 --2- 172.21.15.49:0/3548973951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c106a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f888c0098a0 tx=0x7f888c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4eb5e4e7d4ed915 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.621+0000 7f889b7fe700 1 -- 172.21.15.49:0/3548973951 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f888c004180 con 0x7f889c106660 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.621+0000 7f889b7fe700 1 -- 172.21.15.49:0/3548973951 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f888c0042e0 con 0x7f889c106660 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.621+0000 7f889b7fe700 1 -- 172.21.15.49:0/3548973951 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f888c011b10 con 0x7f889c106660 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.622+0000 7f88a280f700 1 -- 172.21.15.49:0/3548973951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 msgr2=0x7f889c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.622+0000 7f88a280f700 1 --2- 172.21.15.49:0/3548973951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c106a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f888c0098a0 tx=0x7f888c009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.622+0000 7f88a280f700 1 -- 172.21.15.49:0/3548973951 shutdown_connections 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.622+0000 7f88a280f700 1 --2- 172.21.15.49:0/3548973951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c106a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.622+0000 7f88a280f700 1 -- 172.21.15.49:0/3548973951 >> 172.21.15.49:0/3548973951 conn(0x7f889c101c40 msgr2=0x7f889c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.622+0000 7f88a280f700 1 -- 172.21.15.49:0/3548973951 shutdown_connections 2024-05-15T15:44:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.622+0000 7f88a280f700 1 -- 172.21.15.49:0/3548973951 wait complete. 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f88a280f700 1 Processor -- start 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f88a280f700 1 -- start start 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f88a280f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c07bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f88a280f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c07c050 con 0x7f889c106660 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f889bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c07bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f889bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c07bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40360/0 (socket says 172.21.15.49:40360) 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f889bfff700 1 -- 172.21.15.49:0/2987047249 learned_addr learned my addr 172.21.15.49:0/2987047249 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f889bfff700 1 -- 172.21.15.49:0/2987047249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c009550 con 0x7f889c106660 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f889bfff700 1 --2- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c07bb10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f888c0117b0 tx=0x7f888c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.623+0000 7f8899ffb700 1 -- 172.21.15.49:0/2987047249 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f888c0038b0 con 0x7f889c106660 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.624+0000 7f8899ffb700 1 -- 172.21.15.49:0/2987047249 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f888c003a10 con 0x7f889c106660 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.624+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c07a290 con 0x7f889c106660 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.624+0000 7f8899ffb700 1 -- 172.21.15.49:0/2987047249 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f888c0246e0 con 0x7f889c106660 2024-05-15T15:44:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.624+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c07a730 con 0x7f889c106660 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.624+0000 7f8899ffb700 1 -- 172.21.15.49:0/2987047249 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f888c02d3f0 con 0x7f889c106660 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.624+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c0625d0 con 0x7f889c106660 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.625+0000 7f8899ffb700 1 --2- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f88880387e0 0x7f888803aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.625+0000 7f8899ffb700 1 -- 172.21.15.49:0/2987047249 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f888c04c9b0 con 0x7f889c106660 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.625+0000 7f8893fff700 1 --2- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f88880387e0 0x7f888803aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.625+0000 7f8893fff700 1 --2- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f88880387e0 0x7f888803aca0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8884006fd0 tx=0x7f8884006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.630+0000 7f8899ffb700 1 -- 172.21.15.49:0/2987047249 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f888c018b00 con 0x7f889c106660 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.857+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f889c07ae90 con 0x7f88880387e0 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.862+0000 7f8899ffb700 1 -- 172.21.15.49:0/2987047249 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f889c07ae90 con 0x7f88880387e0 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f88880387e0 msgr2=0x7f888803aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 --2- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f88880387e0 0x7f888803aca0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8884006fd0 tx=0x7f8884006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 msgr2=0x7f889c07bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 --2- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c07bb10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f888c0117b0 tx=0x7f888c011890 comp rx=0 tx=0).stop 2024-05-15T15:44:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 shutdown_connections 2024-05-15T15:44:23.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 --2- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f88880387e0 0x7f888803aca0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:23.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 --2- 172.21.15.49:0/2987047249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f889c106660 0x7f889c07bb10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:23.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 >> 172.21.15.49:0/2987047249 conn(0x7f889c101c40 msgr2=0x7f889c1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:23.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 shutdown_connections 2024-05-15T15:44:23.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:22.864+0000 7f88a280f700 1 -- 172.21.15.49:0/2987047249 wait complete. 2024-05-15T15:44:23.464 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying prometheus service with default placement... 2024-05-15T15:44:23.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:23 smithi049 ceph-mon[28399]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:23.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:23 smithi049 ceph-mon[28399]: Saving service ceph-exporter spec with placement * 2024-05-15T15:44:23.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:23 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.702+0000 7f662ca32700 1 Processor -- start 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.702+0000 7f662ca32700 1 -- start start 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.703+0000 7f662ca32700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f6628109540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.703+0000 7f662ca32700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628079e60 con 0x7f6628107150 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.703+0000 7f662659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f6628109540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.703+0000 7f662659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f6628109540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43792/0 (socket says 172.21.15.49:43792) 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.703+0000 7f662659c700 1 -- 172.21.15.49:0/2256917079 learned_addr learned my addr 172.21.15.49:0/2256917079 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.703+0000 7f662659c700 1 -- 172.21.15.49:0/2256917079 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628109a80 con 0x7f6628107150 2024-05-15T15:44:25.384 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.703+0000 7f662659c700 1 --2- 172.21.15.49:0/2256917079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f6628109540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6618009e80 tx=0x7f661800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978dbbf8d62b27fa server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.704+0000 7f662559a700 1 -- 172.21.15.49:0/2256917079 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f661800e070 con 0x7f6628107150 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.704+0000 7f662559a700 1 -- 172.21.15.49:0/2256917079 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f661800b8c0 con 0x7f6628107150 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.704+0000 7f662559a700 1 -- 172.21.15.49:0/2256917079 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66180044d0 con 0x7f6628107150 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.704+0000 7f662ca32700 1 -- 172.21.15.49:0/2256917079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 msgr2=0x7f6628109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.704+0000 7f662ca32700 1 --2- 172.21.15.49:0/2256917079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f6628109540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6618009e80 tx=0x7f661800b330 comp rx=0 tx=0).stop 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.705+0000 7f662ca32700 1 -- 172.21.15.49:0/2256917079 shutdown_connections 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.705+0000 7f662ca32700 1 --2- 172.21.15.49:0/2256917079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f6628109540 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.705+0000 7f662ca32700 1 -- 172.21.15.49:0/2256917079 >> 172.21.15.49:0/2256917079 conn(0x7f6628100b60 msgr2=0x7f6628102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.705+0000 7f662ca32700 1 -- 172.21.15.49:0/2256917079 shutdown_connections 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.705+0000 7f662ca32700 1 -- 172.21.15.49:0/2256917079 wait complete. 2024-05-15T15:44:25.385 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.705+0000 7f662ca32700 1 Processor -- start 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.706+0000 7f662ca32700 1 -- start start 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.706+0000 7f662ca32700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f662819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.706+0000 7f662ca32700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819a810 con 0x7f6628107150 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.706+0000 7f662659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f662819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.706+0000 7f662659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f662819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43800/0 (socket says 172.21.15.49:43800) 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.706+0000 7f662659c700 1 -- 172.21.15.49:0/764570765 learned_addr learned my addr 172.21.15.49:0/764570765 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.706+0000 7f662659c700 1 -- 172.21.15.49:0/764570765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618009b30 con 0x7f6628107150 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.706+0000 7f662659c700 1 --2- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f662819a2d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66180072a0 tx=0x7f661800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.706+0000 7f661f7fe700 1 -- 172.21.15.49:0/764570765 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f661800e040 con 0x7f6628107150 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.707+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662819aa10 con 0x7f6628107150 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.707+0000 7f661f7fe700 1 -- 172.21.15.49:0/764570765 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6618003dc0 con 0x7f6628107150 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.707+0000 7f661f7fe700 1 -- 172.21.15.49:0/764570765 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6618012750 con 0x7f6628107150 2024-05-15T15:44:25.386 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.707+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662819af10 con 0x7f6628107150 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.707+0000 7f661f7fe700 1 -- 172.21.15.49:0/764570765 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f6618019040 con 0x7f6628107150 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.708+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662810a810 con 0x7f6628107150 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.708+0000 7f661f7fe700 1 --2- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f66140387e0 0x7f661403aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.708+0000 7f661f7fe700 1 -- 172.21.15.49:0/764570765 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f661804c150 con 0x7f6628107150 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.708+0000 7f6625d9b700 1 --2- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f66140387e0 0x7f661403aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.708+0000 7f6625d9b700 1 --2- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f66140387e0 0x7f661403aca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6610006fd0 tx=0x7f6610006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.713+0000 7f661f7fe700 1 -- 172.21.15.49:0/764570765 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6618011980 con 0x7f6628107150 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.928+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f6628101ce0 con 0x7f66140387e0 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.932+0000 7f661f7fe700 1 -- 172.21.15.49:0/764570765 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f6628101ce0 con 0x7f66140387e0 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f66140387e0 msgr2=0x7f661403aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 --2- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f66140387e0 0x7f661403aca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6610006fd0 tx=0x7f6610006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 msgr2=0x7f662819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 --2- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f662819a2d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66180072a0 tx=0x7f661800be00 comp rx=0 tx=0).stop 2024-05-15T15:44:25.387 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 shutdown_connections 2024-05-15T15:44:25.388 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 --2- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f66140387e0 0x7f661403aca0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:25.388 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 --2- 172.21.15.49:0/764570765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6628107150 0x7f662819a2d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:25.388 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 >> 172.21.15.49:0/764570765 conn(0x7f6628100b60 msgr2=0x7f66281015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:25.388 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 shutdown_connections 2024-05-15T15:44:25.388 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:24.935+0000 7f662ca32700 1 -- 172.21.15.49:0/764570765 wait complete. 2024-05-15T15:44:25.388 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying grafana service with default placement... 2024-05-15T15:44:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:25 smithi049 ceph-mon[28399]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:26.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:25 smithi049 ceph-mon[28399]: Saving service prometheus spec with placement count:1 2024-05-15T15:44:26.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:25 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:26.565 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-05-15T15:44:26.565 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.771+0000 7fdc83c12700 1 Processor -- start 2024-05-15T15:44:26.565 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.771+0000 7fdc83c12700 1 -- start start 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.772+0000 7fdc83c12700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c109570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.772+0000 7fdc83c12700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c062240 con 0x7fdc7c107180 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.772+0000 7fdc819ae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.772+0000 7fdc819ae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43804/0 (socket says 172.21.15.49:43804) 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.772+0000 7fdc819ae700 1 -- 172.21.15.49:0/18063698 learned_addr learned my addr 172.21.15.49:0/18063698 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.772+0000 7fdc819ae700 1 -- 172.21.15.49:0/18063698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c109ab0 con 0x7fdc7c107180 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.772+0000 7fdc819ae700 1 --2- 172.21.15.49:0/18063698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c109570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0098a0 tx=0x7fdc6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bfe23dee3f76b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.773+0000 7fdc809ac700 1 -- 172.21.15.49:0/18063698 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc6c004180 con 0x7fdc7c107180 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.773+0000 7fdc809ac700 1 -- 172.21.15.49:0/18063698 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc6c0042e0 con 0x7fdc7c107180 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.773+0000 7fdc809ac700 1 -- 172.21.15.49:0/18063698 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc6c011b10 con 0x7fdc7c107180 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.773+0000 7fdc83c12700 1 -- 172.21.15.49:0/18063698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 msgr2=0x7fdc7c109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.773+0000 7fdc83c12700 1 --2- 172.21.15.49:0/18063698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c109570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0098a0 tx=0x7fdc6c009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:26.566 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.773+0000 7fdc83c12700 1 -- 172.21.15.49:0/18063698 shutdown_connections 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.773+0000 7fdc83c12700 1 --2- 172.21.15.49:0/18063698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c109570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.773+0000 7fdc83c12700 1 -- 172.21.15.49:0/18063698 >> 172.21.15.49:0/18063698 conn(0x7fdc7c100af0 msgr2=0x7fdc7c102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.774+0000 7fdc83c12700 1 -- 172.21.15.49:0/18063698 shutdown_connections 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.774+0000 7fdc83c12700 1 -- 172.21.15.49:0/18063698 wait complete. 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.774+0000 7fdc83c12700 1 Processor -- start 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.774+0000 7fdc83c12700 1 -- start start 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.774+0000 7fdc83c12700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.774+0000 7fdc83c12700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c19a920 con 0x7fdc7c107180 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.774+0000 7fdc819ae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.774+0000 7fdc819ae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43810/0 (socket says 172.21.15.49:43810) 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.774+0000 7fdc819ae700 1 -- 172.21.15.49:0/3846724228 learned_addr learned my addr 172.21.15.49:0/3846724228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:26.567 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.775+0000 7fdc819ae700 1 -- 172.21.15.49:0/3846724228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c009550 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.775+0000 7fdc819ae700 1 --2- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c19a3e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009fb0 tx=0x7fdc6c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.775+0000 7fdc72ffd700 1 -- 172.21.15.49:0/3846724228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc6c0038b0 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.775+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c19ab20 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.775+0000 7fdc72ffd700 1 -- 172.21.15.49:0/3846724228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc6c003a10 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.775+0000 7fdc72ffd700 1 -- 172.21.15.49:0/3846724228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc6c019750 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.775+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c19afc0 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.776+0000 7fdc72ffd700 1 -- 172.21.15.49:0/3846724228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fdc6c01ac10 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.776+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c10a7f0 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.776+0000 7fdc72ffd700 1 --2- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fdc680386e0 0x7fdc6803aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.776+0000 7fdc72ffd700 1 -- 172.21.15.49:0/3846724228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdc6c04c830 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.776+0000 7fdc811ad700 1 --2- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fdc680386e0 0x7fdc6803aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.777+0000 7fdc811ad700 1 --2- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fdc680386e0 0x7fdc6803aba0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc78006fd0 tx=0x7fdc78006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:25.782+0000 7fdc72ffd700 1 -- 172.21.15.49:0/3846724228 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdc6c0199c0 con 0x7fdc7c107180 2024-05-15T15:44:26.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.002+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc7c101e00 con 0x7fdc680386e0 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.007+0000 7fdc72ffd700 1 -- 172.21.15.49:0/3846724228 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fdc7c101e00 con 0x7fdc680386e0 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fdc680386e0 msgr2=0x7fdc6803aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 --2- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fdc680386e0 0x7fdc6803aba0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc78006fd0 tx=0x7fdc78006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 msgr2=0x7fdc7c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 --2- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c19a3e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009fb0 tx=0x7fdc6c011890 comp rx=0 tx=0).stop 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 shutdown_connections 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 --2- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fdc680386e0 0x7fdc6803aba0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 --2- 172.21.15.49:0/3846724228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc7c107180 0x7fdc7c19a3e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 >> 172.21.15.49:0/3846724228 conn(0x7fdc7c100af0 msgr2=0x7fdc7c1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 shutdown_connections 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.009+0000 7fdc83c12700 1 -- 172.21.15.49:0/3846724228 wait complete. 2024-05-15T15:44:26.569 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying node-exporter service with default placement... 2024-05-15T15:44:27.166 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:27 smithi049 ceph-mon[28399]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:27.166 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:27 smithi049 ceph-mon[28399]: Saving service grafana spec with placement count:1 2024-05-15T15:44:27.166 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:27 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:27.745 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.937+0000 7fd03a614700 1 Processor -- start 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.937+0000 7fd03a614700 1 -- start start 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.937+0000 7fd03a614700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.937+0000 7fd03a614700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03407a810 con 0x7fd03407be20 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.937+0000 7fd033fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.937+0000 7fd033fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43824/0 (socket says 172.21.15.49:43824) 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.937+0000 7fd033fff700 1 -- 172.21.15.49:0/2672662141 learned_addr learned my addr 172.21.15.49:0/2672662141 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.938+0000 7fd033fff700 1 -- 172.21.15.49:0/2672662141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03407a950 con 0x7fd03407be20 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.938+0000 7fd033fff700 1 --2- 172.21.15.49:0/2672662141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03407a2d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0240098a0 tx=0x7fd024009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c5f21c641d4711 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.938+0000 7fd0337fe700 1 -- 172.21.15.49:0/2672662141 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd024004180 con 0x7fd03407be20 2024-05-15T15:44:27.746 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.938+0000 7fd0337fe700 1 -- 172.21.15.49:0/2672662141 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0240042e0 con 0x7fd03407be20 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.938+0000 7fd0337fe700 1 -- 172.21.15.49:0/2672662141 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd024011b10 con 0x7fd03407be20 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.939+0000 7fd03a614700 1 -- 172.21.15.49:0/2672662141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 msgr2=0x7fd03407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.939+0000 7fd03a614700 1 --2- 172.21.15.49:0/2672662141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03407a2d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0240098a0 tx=0x7fd024009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.939+0000 7fd03a614700 1 -- 172.21.15.49:0/2672662141 shutdown_connections 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.939+0000 7fd03a614700 1 --2- 172.21.15.49:0/2672662141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03407a2d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.939+0000 7fd03a614700 1 -- 172.21.15.49:0/2672662141 >> 172.21.15.49:0/2672662141 conn(0x7fd034101ce0 msgr2=0x7fd034104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.939+0000 7fd03a614700 1 -- 172.21.15.49:0/2672662141 shutdown_connections 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.939+0000 7fd03a614700 1 -- 172.21.15.49:0/2672662141 wait complete. 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.939+0000 7fd03a614700 1 Processor -- start 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.939+0000 7fd03a614700 1 -- start start 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd03a614700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd03a614700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03419caa0 con 0x7fd03407be20 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd033fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:27.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd033fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43836/0 (socket says 172.21.15.49:43836) 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd033fff700 1 -- 172.21.15.49:0/3939972134 learned_addr learned my addr 172.21.15.49:0/3939972134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd033fff700 1 -- 172.21.15.49:0/3939972134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd024009550 con 0x7fd03407be20 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd033fff700 1 --2- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03419c560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd0240117b0 tx=0x7fd024011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd031ffb700 1 -- 172.21.15.49:0/3939972134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd0240038b0 con 0x7fd03407be20 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd031ffb700 1 -- 172.21.15.49:0/3939972134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd024003a10 con 0x7fd03407be20 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03419cca0 con 0x7fd03407be20 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd031ffb700 1 -- 172.21.15.49:0/3939972134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd024024780 con 0x7fd03407be20 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.940+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03419d140 con 0x7fd03407be20 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.941+0000 7fd031ffb700 1 -- 172.21.15.49:0/3939972134 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fd02401ac10 con 0x7fd03407be20 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.941+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03410ca00 con 0x7fd03407be20 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.941+0000 7fd031ffb700 1 --2- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fd020038830 0x7fd02003acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.942+0000 7fd031ffb700 1 -- 172.21.15.49:0/3939972134 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd02404c760 con 0x7fd03407be20 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.942+0000 7fd02bfff700 1 --2- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fd020038830 0x7fd02003acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:27.748 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.942+0000 7fd02bfff700 1 --2- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fd020038830 0x7fd02003acf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd01c006fd0 tx=0x7fd01c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:26.947+0000 7fd031ffb700 1 -- 172.21.15.49:0/3939972134 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd0240248e0 con 0x7fd03407be20 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.163+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fd034102f20 con 0x7fd020038830 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.168+0000 7fd031ffb700 1 -- 172.21.15.49:0/3939972134 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fd034102f20 con 0x7fd020038830 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fd020038830 msgr2=0x7fd02003acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 --2- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fd020038830 0x7fd02003acf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd01c006fd0 tx=0x7fd01c006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 msgr2=0x7fd03419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 --2- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03419c560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd0240117b0 tx=0x7fd024011890 comp rx=0 tx=0).stop 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 shutdown_connections 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 --2- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fd020038830 0x7fd02003acf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 --2- 172.21.15.49:0/3939972134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd03407be20 0x7fd03419c560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 >> 172.21.15.49:0/3939972134 conn(0x7fd034101ce0 msgr2=0x7fd034102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 shutdown_connections 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:27.170+0000 7fd03a614700 1 -- 172.21.15.49:0/3939972134 wait complete. 2024-05-15T15:44:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying alertmanager service with default placement... 2024-05-15T15:44:28.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:28 smithi049 ceph-mon[28399]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:28.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:28 smithi049 ceph-mon[28399]: Saving service node-exporter spec with placement * 2024-05-15T15:44:28.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:28 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:28.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-05-15T15:44:28.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.127+0000 7f9201267700 1 Processor -- start 2024-05-15T15:44:28.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.127+0000 7f9201267700 1 -- start start 2024-05-15T15:44:28.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.127+0000 7f9201267700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc109540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:28.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.127+0000 7f9201267700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc079e60 con 0x7f91fc107150 2024-05-15T15:44:28.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.127+0000 7f91faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc109540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:28.936 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.127+0000 7f91faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc109540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43846/0 (socket says 172.21.15.49:43846) 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.127+0000 7f91faffd700 1 -- 172.21.15.49:0/2426496304 learned_addr learned my addr 172.21.15.49:0/2426496304 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.128+0000 7f91faffd700 1 -- 172.21.15.49:0/2426496304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc109a80 con 0x7f91fc107150 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.128+0000 7f91faffd700 1 --2- 172.21.15.49:0/2426496304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc109540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0098a0 tx=0x7f91ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e65ce036201fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.128+0000 7f91f9ffb700 1 -- 172.21.15.49:0/2426496304 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91ec004180 con 0x7f91fc107150 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.128+0000 7f91f9ffb700 1 -- 172.21.15.49:0/2426496304 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91ec0042e0 con 0x7f91fc107150 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.128+0000 7f91f9ffb700 1 -- 172.21.15.49:0/2426496304 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91ec011b10 con 0x7f91fc107150 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.129+0000 7f9201267700 1 -- 172.21.15.49:0/2426496304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 msgr2=0x7f91fc109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.129+0000 7f9201267700 1 --2- 172.21.15.49:0/2426496304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc109540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0098a0 tx=0x7f91ec009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.129+0000 7f9201267700 1 -- 172.21.15.49:0/2426496304 shutdown_connections 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.129+0000 7f9201267700 1 --2- 172.21.15.49:0/2426496304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc109540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.129+0000 7f9201267700 1 -- 172.21.15.49:0/2426496304 >> 172.21.15.49:0/2426496304 conn(0x7f91fc100b60 msgr2=0x7f91fc102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.129+0000 7f9201267700 1 -- 172.21.15.49:0/2426496304 shutdown_connections 2024-05-15T15:44:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.129+0000 7f9201267700 1 -- 172.21.15.49:0/2426496304 wait complete. 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.129+0000 7f9201267700 1 Processor -- start 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.130+0000 7f9201267700 1 -- start start 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.130+0000 7f9201267700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc106980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.130+0000 7f9201267700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc106ec0 con 0x7f91fc107150 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.130+0000 7f91faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.130+0000 7f91faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43852/0 (socket says 172.21.15.49:43852) 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.130+0000 7f91faffd700 1 -- 172.21.15.49:0/70703918 learned_addr learned my addr 172.21.15.49:0/70703918 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.130+0000 7f91faffd700 1 -- 172.21.15.49:0/70703918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec009550 con 0x7f91fc107150 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.130+0000 7f91faffd700 1 --2- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc106980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009fb0 tx=0x7f91ec011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.131+0000 7f91e3fff700 1 -- 172.21.15.49:0/70703918 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91ec0038b0 con 0x7f91fc107150 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.131+0000 7f91e3fff700 1 -- 172.21.15.49:0/70703918 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91ec003a10 con 0x7f91fc107150 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.131+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc105100 con 0x7f91fc107150 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.131+0000 7f91e3fff700 1 -- 172.21.15.49:0/70703918 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91ec019750 con 0x7f91fc107150 2024-05-15T15:44:28.938 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.131+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc1055a0 con 0x7f91fc107150 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.132+0000 7f91e3fff700 1 -- 172.21.15.49:0/70703918 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f91ec0198b0 con 0x7f91fc107150 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.132+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91fc10a810 con 0x7f91fc107150 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.132+0000 7f91e3fff700 1 --2- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f91e4038840 0x7f91e403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.132+0000 7f91e3fff700 1 -- 172.21.15.49:0/70703918 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f91ec04c770 con 0x7f91fc107150 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.132+0000 7f91fa7fc700 1 --2- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f91e4038840 0x7f91e403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.132+0000 7f91fa7fc700 1 --2- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f91e4038840 0x7f91e403ad00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f91f0006fd0 tx=0x7f91f0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.137+0000 7f91e3fff700 1 -- 172.21.15.49:0/70703918 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f91ec022d70 con 0x7f91fc107150 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.352+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f91fc105d90 con 0x7f91e4038840 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.356+0000 7f91e3fff700 1 -- 172.21.15.49:0/70703918 <== mgr.14118 v2:172.21.15.49:6800/821763420 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f91fc105d90 con 0x7f91e4038840 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f91e4038840 msgr2=0x7f91e403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 --2- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f91e4038840 0x7f91e403ad00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f91f0006fd0 tx=0x7f91f0006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 msgr2=0x7f91fc106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 --2- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc106980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009fb0 tx=0x7f91ec011890 comp rx=0 tx=0).stop 2024-05-15T15:44:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 shutdown_connections 2024-05-15T15:44:28.940 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 --2- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f91e4038840 0x7f91e403ad00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:28.940 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 --2- 172.21.15.49:0/70703918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91fc107150 0x7f91fc106980 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:28.940 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 >> 172.21.15.49:0/70703918 conn(0x7f91fc100b60 msgr2=0x7f91fc101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:28.940 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 shutdown_connections 2024-05-15T15:44:28.940 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:28.359+0000 7f9201267700 1 -- 172.21.15.49:0/70703918 wait complete. 2024-05-15T15:44:29.330 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:29 smithi049 ceph-mon[28399]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:44:29.331 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:29 smithi049 ceph-mon[28399]: from='mgr.14118 172.21.15.49:0/4247096588' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:30.184 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:30 smithi049 ceph-mon[28399]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:30.184 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:30 smithi049 ceph-mon[28399]: Saving service alertmanager spec with placement count:1 2024-05-15T15:44:30.184 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:30 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2832885823' entity='client.admin' 2024-05-15T15:44:30.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.365+0000 7f85eebc6700 1 Processor -- start 2024-05-15T15:44:30.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.365+0000 7f85eebc6700 1 -- start start 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.365+0000 7f85eebc6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e8104df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.365+0000 7f85eebc6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e81053c0 con 0x7f85e81049d0 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.365+0000 7f85ec962700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e8104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.365+0000 7f85ec962700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e8104df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43860/0 (socket says 172.21.15.49:43860) 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.365+0000 7f85ec962700 1 -- 172.21.15.49:0/3053024834 learned_addr learned my addr 172.21.15.49:0/3053024834 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.366+0000 7f85ec962700 1 -- 172.21.15.49:0/3053024834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8107960 con 0x7f85e81049d0 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.366+0000 7f85ec962700 1 --2- 172.21.15.49:0/3053024834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e8104df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0098a0 tx=0x7f85dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908de01ee2b505c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.366+0000 7f85e77fe700 1 -- 172.21.15.49:0/3053024834 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc004180 con 0x7f85e81049d0 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.366+0000 7f85e77fe700 1 -- 172.21.15.49:0/3053024834 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85dc0042e0 con 0x7f85e81049d0 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.366+0000 7f85e77fe700 1 -- 172.21.15.49:0/3053024834 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc011b10 con 0x7f85e81049d0 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.367+0000 7f85eebc6700 1 -- 172.21.15.49:0/3053024834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 msgr2=0x7f85e8104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.367+0000 7f85eebc6700 1 --2- 172.21.15.49:0/3053024834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e8104df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0098a0 tx=0x7f85dc009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:30.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.367+0000 7f85eebc6700 1 -- 172.21.15.49:0/3053024834 shutdown_connections 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.367+0000 7f85eebc6700 1 --2- 172.21.15.49:0/3053024834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e8104df0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.367+0000 7f85eebc6700 1 -- 172.21.15.49:0/3053024834 >> 172.21.15.49:0/3053024834 conn(0x7f85e81002e0 msgr2=0x7f85e8102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.367+0000 7f85eebc6700 1 -- 172.21.15.49:0/3053024834 shutdown_connections 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.367+0000 7f85eebc6700 1 -- 172.21.15.49:0/3053024834 wait complete. 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85eebc6700 1 Processor -- start 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85eebc6700 1 -- start start 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85eebc6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85eebc6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e819cb50 con 0x7f85e81049d0 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85ec962700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85ec962700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43864/0 (socket says 172.21.15.49:43864) 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85ec962700 1 -- 172.21.15.49:0/2832885823 learned_addr learned my addr 172.21.15.49:0/2832885823 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85ec962700 1 -- 172.21.15.49:0/2832885823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc009550 con 0x7f85e81049d0 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85ec962700 1 --2- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e819c610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0117b0 tx=0x7f85dc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85e5ffb700 1 -- 172.21.15.49:0/2832885823 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc0038b0 con 0x7f85e81049d0 2024-05-15T15:44:30.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85e5ffb700 1 -- 172.21.15.49:0/2832885823 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85dc003a10 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.368+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e819cd50 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.369+0000 7f85e5ffb700 1 -- 172.21.15.49:0/2832885823 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc0246e0 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.369+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e819d1f0 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.370+0000 7f85e5ffb700 1 -- 172.21.15.49:0/2832885823 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f85dc02d3f0 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.370+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e810ab90 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.370+0000 7f85e5ffb700 1 --2- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f85d8038830 0x7f85d803acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.370+0000 7f85e5ffb700 1 -- 172.21.15.49:0/2832885823 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f85dc04c800 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.370+0000 7f85e7fff700 1 --2- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f85d8038830 0x7f85d803acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.370+0000 7f85e7fff700 1 --2- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f85d8038830 0x7f85d803acf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f85d4006fd0 tx=0x7f85d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.375+0000 7f85e5ffb700 1 -- 172.21.15.49:0/2832885823 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f85dc010970 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.577+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f85e819fae0 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.580+0000 7f85e5ffb700 1 -- 172.21.15.49:0/2832885823 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f85dc010380 con 0x7f85e81049d0 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f85d8038830 msgr2=0x7f85d803acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:30.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 --2- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f85d8038830 0x7f85d803acf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f85d4006fd0 tx=0x7f85d4006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:30.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 msgr2=0x7f85e819c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:30.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 --2- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e819c610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0117b0 tx=0x7f85dc011890 comp rx=0 tx=0).stop 2024-05-15T15:44:30.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 shutdown_connections 2024-05-15T15:44:30.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 --2- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f85d8038830 0x7f85d803acf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:30.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 --2- 172.21.15.49:0/2832885823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85e81049d0 0x7f85e819c610 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:30.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 >> 172.21.15.49:0/2832885823 conn(0x7f85e81002e0 msgr2=0x7f85e8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:30.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 shutdown_connections 2024-05-15T15:44:30.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:29.582+0000 7f85eebc6700 1 -- 172.21.15.49:0/2832885823 wait complete. 2024-05-15T15:44:31.368 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:31 smithi049 ceph-mon[28399]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:44:31.368 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:31 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2926340740' entity='client.admin' 2024-05-15T15:44:31.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.603+0000 7f17842c5700 1 Processor -- start 2024-05-15T15:44:31.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.603+0000 7f17842c5700 1 -- start start 2024-05-15T15:44:31.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.603+0000 7f17842c5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c104db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.603+0000 7f17842c5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177c105380 con 0x7f177c104990 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.603+0000 7f1782061700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c104db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.603+0000 7f1782061700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c104db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43876/0 (socket says 172.21.15.49:43876) 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.603+0000 7f1782061700 1 -- 172.21.15.49:0/3627017537 learned_addr learned my addr 172.21.15.49:0/3627017537 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.604+0000 7f1782061700 1 -- 172.21.15.49:0/3627017537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177c107920 con 0x7f177c104990 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.604+0000 7f1782061700 1 --2- 172.21.15.49:0/3627017537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c104db0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f17780098a0 tx=0x7f1778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d225d9a71d54c752 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.604+0000 7f178105f700 1 -- 172.21.15.49:0/3627017537 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1778004180 con 0x7f177c104990 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.604+0000 7f178105f700 1 -- 172.21.15.49:0/3627017537 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17780042e0 con 0x7f177c104990 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.604+0000 7f178105f700 1 -- 172.21.15.49:0/3627017537 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1778011b10 con 0x7f177c104990 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.605+0000 7f17842c5700 1 -- 172.21.15.49:0/3627017537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 msgr2=0x7f177c104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.605+0000 7f17842c5700 1 --2- 172.21.15.49:0/3627017537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c104db0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f17780098a0 tx=0x7f1778009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.605+0000 7f17842c5700 1 -- 172.21.15.49:0/3627017537 shutdown_connections 2024-05-15T15:44:31.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.605+0000 7f17842c5700 1 --2- 172.21.15.49:0/3627017537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c104db0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.605+0000 7f17842c5700 1 -- 172.21.15.49:0/3627017537 >> 172.21.15.49:0/3627017537 conn(0x7f177c100280 msgr2=0x7f177c102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.605+0000 7f17842c5700 1 -- 172.21.15.49:0/3627017537 shutdown_connections 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.605+0000 7f17842c5700 1 -- 172.21.15.49:0/3627017537 wait complete. 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.605+0000 7f17842c5700 1 Processor -- start 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.605+0000 7f17842c5700 1 -- start start 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f17842c5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f17842c5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177c19a8a0 con 0x7f177c104990 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f1782061700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f1782061700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43888/0 (socket says 172.21.15.49:43888) 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f1782061700 1 -- 172.21.15.49:0/2926340740 learned_addr learned my addr 172.21.15.49:0/2926340740 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f1782061700 1 -- 172.21.15.49:0/2926340740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1778009550 con 0x7f177c104990 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f1782061700 1 --2- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c19a360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1778009fb0 tx=0x7f1778011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f17737fe700 1 -- 172.21.15.49:0/2926340740 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17780038b0 con 0x7f177c104990 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f17737fe700 1 -- 172.21.15.49:0/2926340740 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1778003a10 con 0x7f177c104990 2024-05-15T15:44:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177c19aaa0 con 0x7f177c104990 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f17737fe700 1 -- 172.21.15.49:0/2926340740 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1778019770 con 0x7f177c104990 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.606+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177c19af40 con 0x7f177c104990 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.607+0000 7f17737fe700 1 -- 172.21.15.49:0/2926340740 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f177801a430 con 0x7f177c104990 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.607+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177c10ab50 con 0x7f177c104990 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.607+0000 7f17737fe700 1 --2- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f17680387f0 0x7f176803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.608+0000 7f17737fe700 1 -- 172.21.15.49:0/2926340740 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f177804c780 con 0x7f177c104990 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.608+0000 7f1781860700 1 --2- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f17680387f0 0x7f176803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.608+0000 7f1781860700 1 --2- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f17680387f0 0x7f176803acb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f176c006fd0 tx=0x7f176c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.613+0000 7f17737fe700 1 -- 172.21.15.49:0/2926340740 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1778019df0 con 0x7f177c104990 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.814+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f177c19d830 con 0x7f177c104990 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.818+0000 7f17737fe700 1 -- 172.21.15.49:0/2926340740 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f1778022310 con 0x7f177c104990 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f17680387f0 msgr2=0x7f176803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 --2- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f17680387f0 0x7f176803acb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f176c006fd0 tx=0x7f176c006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:31.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 msgr2=0x7f177c19a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:31.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 --2- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c19a360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1778009fb0 tx=0x7f1778011890 comp rx=0 tx=0).stop 2024-05-15T15:44:31.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 shutdown_connections 2024-05-15T15:44:31.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 --2- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f17680387f0 0x7f176803acb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:31.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 --2- 172.21.15.49:0/2926340740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177c104990 0x7f177c19a360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:31.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 >> 172.21.15.49:0/2926340740 conn(0x7f177c100280 msgr2=0x7f177c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:31.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 shutdown_connections 2024-05-15T15:44:31.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:30.820+0000 7f17842c5700 1 -- 172.21.15.49:0/2926340740 wait complete. 2024-05-15T15:44:31.377 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling the dashboard module... 2024-05-15T15:44:32.357 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:32 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3605166127' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-15T15:44:32.905 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.806+0000 7fbac4aba700 1 Processor -- start 2024-05-15T15:44:32.905 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.806+0000 7fbac4aba700 1 -- start start 2024-05-15T15:44:32.905 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.806+0000 7fbac4aba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac0106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:32.905 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.806+0000 7fbac4aba700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac0107050 con 0x7fbac0106660 2024-05-15T15:44:32.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.806+0000 7fbabe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac0106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:32.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.806+0000 7fbabe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac0106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43896/0 (socket says 172.21.15.49:43896) 2024-05-15T15:44:32.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.806+0000 7fbabe59c700 1 -- 172.21.15.49:0/1112932757 learned_addr learned my addr 172.21.15.49:0/1112932757 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:32.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.807+0000 7fbabe59c700 1 -- 172.21.15.49:0/1112932757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac0107860 con 0x7fbac0106660 2024-05-15T15:44:32.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.807+0000 7fbabe59c700 1 --2- 172.21.15.49:0/1112932757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac0106a80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbab00098a0 tx=0x7fbab0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6dc65d6240f742 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:32.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.807+0000 7fbabd59a700 1 -- 172.21.15.49:0/1112932757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbab0004180 con 0x7fbac0106660 2024-05-15T15:44:32.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.807+0000 7fbabd59a700 1 -- 172.21.15.49:0/1112932757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbab00042e0 con 0x7fbac0106660 2024-05-15T15:44:32.906 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.807+0000 7fbabd59a700 1 -- 172.21.15.49:0/1112932757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbab0011b10 con 0x7fbac0106660 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.808+0000 7fbac4aba700 1 -- 172.21.15.49:0/1112932757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 msgr2=0x7fbac0106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.808+0000 7fbac4aba700 1 --2- 172.21.15.49:0/1112932757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac0106a80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbab00098a0 tx=0x7fbab0009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.808+0000 7fbac4aba700 1 -- 172.21.15.49:0/1112932757 shutdown_connections 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.808+0000 7fbac4aba700 1 --2- 172.21.15.49:0/1112932757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac0106a80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.808+0000 7fbac4aba700 1 -- 172.21.15.49:0/1112932757 >> 172.21.15.49:0/1112932757 conn(0x7fbac0101c40 msgr2=0x7fbac0104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.808+0000 7fbac4aba700 1 -- 172.21.15.49:0/1112932757 shutdown_connections 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.808+0000 7fbac4aba700 1 -- 172.21.15.49:0/1112932757 wait complete. 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.808+0000 7fbac4aba700 1 Processor -- start 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.808+0000 7fbac4aba700 1 -- start start 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbac4aba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbac4aba700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac019a890 con 0x7fbac0106660 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbabe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbabe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43906/0 (socket says 172.21.15.49:43906) 2024-05-15T15:44:32.907 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbabe59c700 1 -- 172.21.15.49:0/3605166127 learned_addr learned my addr 172.21.15.49:0/3605166127 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbabe59c700 1 -- 172.21.15.49:0/3605166127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0009550 con 0x7fbac0106660 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbabe59c700 1 --2- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac019a350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbab00117b0 tx=0x7fbab0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbab77fe700 1 -- 172.21.15.49:0/3605166127 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbab00038b0 con 0x7fbac0106660 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbab77fe700 1 -- 172.21.15.49:0/3605166127 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbab0003a10 con 0x7fbac0106660 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbab77fe700 1 -- 172.21.15.49:0/3605166127 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbab00246e0 con 0x7fbac0106660 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.809+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac019aa90 con 0x7fbac0106660 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.810+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac019af30 con 0x7fbac0106660 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.810+0000 7fbab77fe700 1 -- 172.21.15.49:0/3605166127 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fbab001ab80 con 0x7fbac0106660 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.811+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac010aa90 con 0x7fbac0106660 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.811+0000 7fbab77fe700 1 --2- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fbaac0387e0 0x7fbaac03aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.811+0000 7fbab77fe700 1 -- 172.21.15.49:0/3605166127 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbab004ca00 con 0x7fbac0106660 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.811+0000 7fbabdd9b700 1 --2- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fbaac0387e0 0x7fbaac03aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:32.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.811+0000 7fbabdd9b700 1 --2- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fbaac0387e0 0x7fbaac03aca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006fd0 tx=0x7fbaa8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:31.816+0000 7fbab77fe700 1 -- 172.21.15.49:0/3605166127 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbab0010380 con 0x7fbac0106660 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.082+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fbac019d7c0 con 0x7fbac0106660 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.175+0000 7fbab77fe700 1 -- 172.21.15.49:0/3605166127 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 9) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fbab002d770 con 0x7fbac0106660 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.176+0000 7fbab77fe700 1 -- 172.21.15.49:0/3605166127 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fbab004c350 con 0x7fbac0106660 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.178+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fbaac0387e0 msgr2=0x7fbaac03aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.178+0000 7fbac4aba700 1 --2- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fbaac0387e0 0x7fbaac03aca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006fd0 tx=0x7fbaa8006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.178+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 msgr2=0x7fbac019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.178+0000 7fbac4aba700 1 --2- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac019a350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbab00117b0 tx=0x7fbab0011890 comp rx=0 tx=0).stop 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.178+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 shutdown_connections 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.178+0000 7fbac4aba700 1 --2- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fbaac0387e0 0x7fbaac03aca0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.178+0000 7fbac4aba700 1 --2- 172.21.15.49:0/3605166127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbac0106660 0x7fbac019a350 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.178+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 >> 172.21.15.49:0/3605166127 conn(0x7fbac0101c40 msgr2=0x7fbac01026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:32.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.178+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 shutdown_connections 2024-05-15T15:44:32.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:32.179+0000 7fbac4aba700 1 -- 172.21.15.49:0/3605166127 wait complete. 2024-05-15T15:44:33.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:33 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3605166127' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-15T15:44:33.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:33 smithi049 ceph-mon[28399]: mgrmap e9: smithi049.vuhzgf(active, since 24s) 2024-05-15T15:44:34.212 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-05-15T15:44:34.212 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-15T15:44:34.212 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "smithi049.vuhzgf", 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.327+0000 7f81c128c700 1 Processor -- start 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.327+0000 7f81c128c700 1 -- start start 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.328+0000 7f81c128c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.328+0000 7f81c128c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc106440 con 0x7f81bc105a50 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.328+0000 7f81baffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.328+0000 7f81baffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43930/0 (socket says 172.21.15.49:43930) 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.328+0000 7f81baffd700 1 -- 172.21.15.49:0/3139096750 learned_addr learned my addr 172.21.15.49:0/3139096750 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.328+0000 7f81baffd700 1 -- 172.21.15.49:0/3139096750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc106770 con 0x7f81bc105a50 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.328+0000 7f81baffd700 1 --2- 172.21.15.49:0/3139096750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105e70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f81b00098a0 tx=0x7f81b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebda7bdcfa907b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:34.213 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81b9ffb700 1 -- 172.21.15.49:0/3139096750 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b0004180 con 0x7f81bc105a50 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81b9ffb700 1 -- 172.21.15.49:0/3139096750 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81b00042e0 con 0x7f81bc105a50 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81b9ffb700 1 -- 172.21.15.49:0/3139096750 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b0011b10 con 0x7f81bc105a50 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81c128c700 1 -- 172.21.15.49:0/3139096750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 msgr2=0x7f81bc105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81c128c700 1 --2- 172.21.15.49:0/3139096750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105e70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f81b00098a0 tx=0x7f81b0009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81c128c700 1 -- 172.21.15.49:0/3139096750 shutdown_connections 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81c128c700 1 --2- 172.21.15.49:0/3139096750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105e70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81c128c700 1 -- 172.21.15.49:0/3139096750 >> 172.21.15.49:0/3139096750 conn(0x7f81bc100290 msgr2=0x7f81bc1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81c128c700 1 -- 172.21.15.49:0/3139096750 shutdown_connections 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.329+0000 7f81c128c700 1 -- 172.21.15.49:0/3139096750 wait complete. 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.330+0000 7f81c128c700 1 Processor -- start 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.330+0000 7f81c128c700 1 -- start start 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81c128c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81c128c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc105540 con 0x7f81bc105a50 2024-05-15T15:44:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81baffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81baffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43938/0 (socket says 172.21.15.49:43938) 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81baffd700 1 -- 172.21.15.49:0/1875769254 learned_addr learned my addr 172.21.15.49:0/1875769254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81baffd700 1 -- 172.21.15.49:0/1875769254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0009550 con 0x7f81bc105a50 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81baffd700 1 --2- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009fb0 tx=0x7f81b0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81a3fff700 1 -- 172.21.15.49:0/1875769254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b00038b0 con 0x7f81bc105a50 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc19d070 con 0x7f81bc105a50 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81a3fff700 1 -- 172.21.15.49:0/1875769254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81b0003a10 con 0x7f81bc105a50 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.331+0000 7f81a3fff700 1 -- 172.21.15.49:0/1875769254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b0019750 con 0x7f81bc105a50 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.332+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc19d510 con 0x7f81bc105a50 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.332+0000 7f81a3fff700 1 -- 172.21.15.49:0/1875769254 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f81b00199c0 con 0x7f81bc105a50 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.333+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc0625d0 con 0x7f81bc105a50 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.333+0000 7f81a3fff700 1 --2- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f81a40387f0 0x7f81a403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.333+0000 7f81ba7fc700 1 -- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f81a40387f0 msgr2=0x7f81a403acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/821763420 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.333+0000 7f81a3fff700 1 -- 172.21.15.49:0/1875769254 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f81b004cb80 con 0x7f81bc105a50 2024-05-15T15:44:34.215 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.333+0000 7f81ba7fc700 1 --2- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f81a40387f0 0x7f81a403acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T15:44:34.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.338+0000 7f81a3fff700 1 -- 172.21.15.49:0/1875769254 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f81b0022b00 con 0x7f81bc105a50 2024-05-15T15:44:34.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.534+0000 7f81ba7fc700 1 -- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f81a40387f0 msgr2=0x7f81a403acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/821763420 2024-05-15T15:44:34.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.534+0000 7f81ba7fc700 1 --2- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f81a40387f0 0x7f81a403acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T15:44:34.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.641+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f81bc19db80 con 0x7f81bc105a50 2024-05-15T15:44:34.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.642+0000 7f81a3fff700 1 -- 172.21.15.49:0/1875769254 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f81b0022b00 con 0x7f81bc105a50 2024-05-15T15:44:34.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f81a40387f0 msgr2=0x7f81a403acb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T15:44:34.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 --2- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f81a40387f0 0x7f81a403acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:34.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 msgr2=0x7f81bc105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:34.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 --2- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009fb0 tx=0x7f81b0011890 comp rx=0 tx=0).stop 2024-05-15T15:44:34.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 shutdown_connections 2024-05-15T15:44:34.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 --2- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7f81a40387f0 0x7f81a403acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:34.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 --2- 172.21.15.49:0/1875769254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81bc105a50 0x7f81bc105000 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:34.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 >> 172.21.15.49:0/1875769254 conn(0x7f81bc100290 msgr2=0x7f81bc100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:34.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 shutdown_connections 2024-05-15T15:44:34.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:33.644+0000 7f81c128c700 1 -- 172.21.15.49:0/1875769254 wait complete. 2024-05-15T15:44:34.217 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-05-15T15:44:34.217 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 9... 2024-05-15T15:44:34.330 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:34 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1875769254' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-15T15:44:39.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: Active manager daemon smithi049.vuhzgf restarted 2024-05-15T15:44:39.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: Activating manager daemon smithi049.vuhzgf 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: osdmap e3: 0 total, 0 up, 0 in 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: mgrmap e10: smithi049.vuhzgf(active, starting, since 0.00328779s) 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: Manager daemon smithi049.vuhzgf is now available 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:44:39.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:39 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:44:40.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:40 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/trash_purge_schedule"}]: dispatch 2024-05-15T15:44:40.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:40 smithi049 ceph-mon[28399]: mgrmap e11: smithi049.vuhzgf(active, since 1.006s) 2024-05-15T15:44:40.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-05-15T15:44:40.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-15T15:44:40.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-15T15:44:40.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-05-15T15:44:40.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.676+0000 7fb6451a6700 1 Processor -- start 2024-05-15T15:44:40.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.676+0000 7fb6451a6700 1 -- start start 2024-05-15T15:44:40.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.676+0000 7fb6451a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640106f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:40.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.676+0000 7fb6451a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6401074f0 con 0x7fb640106b00 2024-05-15T15:44:40.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.676+0000 7fb63ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640106f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:40.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.676+0000 7fb63ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640106f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42684/0 (socket says 172.21.15.49:42684) 2024-05-15T15:44:40.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.676+0000 7fb63ed9d700 1 -- 172.21.15.49:0/895839679 learned_addr learned my addr 172.21.15.49:0/895839679 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:40.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.677+0000 7fb63ed9d700 1 -- 172.21.15.49:0/895839679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640105760 con 0x7fb640106b00 2024-05-15T15:44:40.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.677+0000 7fb63ed9d700 1 --2- 172.21.15.49:0/895839679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640106f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb6300098a0 tx=0x7fb630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b5ff88fc60592b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:40.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.677+0000 7fb63dd9b700 1 -- 172.21.15.49:0/895839679 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb630004180 con 0x7fb640106b00 2024-05-15T15:44:40.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.677+0000 7fb63dd9b700 1 -- 172.21.15.49:0/895839679 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6300042e0 con 0x7fb640106b00 2024-05-15T15:44:40.965 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.677+0000 7fb63dd9b700 1 -- 172.21.15.49:0/895839679 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb630011b10 con 0x7fb640106b00 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.677+0000 7fb6451a6700 1 -- 172.21.15.49:0/895839679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 msgr2=0x7fb640106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.678+0000 7fb6451a6700 1 --2- 172.21.15.49:0/895839679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640106f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb6300098a0 tx=0x7fb630009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.678+0000 7fb6451a6700 1 -- 172.21.15.49:0/895839679 shutdown_connections 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.678+0000 7fb6451a6700 1 --2- 172.21.15.49:0/895839679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640106f20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.678+0000 7fb6451a6700 1 -- 172.21.15.49:0/895839679 >> 172.21.15.49:0/895839679 conn(0x7fb640100290 msgr2=0x7fb6401026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.678+0000 7fb6451a6700 1 -- 172.21.15.49:0/895839679 shutdown_connections 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.678+0000 7fb6451a6700 1 -- 172.21.15.49:0/895839679 wait complete. 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.679+0000 7fb6451a6700 1 Processor -- start 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.679+0000 7fb6451a6700 1 -- start start 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.679+0000 7fb6451a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.679+0000 7fb6451a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb640199760 con 0x7fb640106b00 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.679+0000 7fb63ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.679+0000 7fb63ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42690/0 (socket says 172.21.15.49:42690) 2024-05-15T15:44:40.966 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.679+0000 7fb63ed9d700 1 -- 172.21.15.49:0/3666647149 learned_addr learned my addr 172.21.15.49:0/3666647149 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.680+0000 7fb63ed9d700 1 -- 172.21.15.49:0/3666647149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb630009550 con 0x7fb640106b00 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.680+0000 7fb63ed9d700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640199220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb630009fb0 tx=0x7fb630011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.680+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6300038b0 con 0x7fb640106b00 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.680+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb630003a10 con 0x7fb640106b00 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.680+0000 7fb6451a6700 1 -- 172.21.15.49:0/3666647149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb640199960 con 0x7fb640106b00 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.680+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb630019750 con 0x7fb640106b00 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.680+0000 7fb6451a6700 1 -- 172.21.15.49:0/3666647149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb640199e00 con 0x7fb640106b00 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.681+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fb6300198b0 con 0x7fb640106b00 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.681+0000 7fb627fff700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 0x7fb62803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.681+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 --> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb62803b410 con 0x7fb628038840 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.681+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb63004c830 con 0x7fb640106b00 2024-05-15T15:44:40.967 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.681+0000 7fb63e59c700 1 -- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 msgr2=0x7fb62803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/821763420 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.681+0000 7fb63e59c700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 0x7fb62803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.882+0000 7fb63e59c700 1 -- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 msgr2=0x7fb62803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/821763420 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:34.882+0000 7fb63e59c700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 0x7fb62803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:35.283+0000 7fb63e59c700 1 -- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 msgr2=0x7fb62803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/821763420 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:35.283+0000 7fb63e59c700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 0x7fb62803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:36.084+0000 7fb63e59c700 1 -- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 msgr2=0x7fb62803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/821763420 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:36.084+0000 7fb63e59c700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 0x7fb62803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:37.685+0000 7fb63e59c700 1 -- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 msgr2=0x7fb62803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/821763420 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:37.685+0000 7fb63e59c700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 0x7fb62803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:39.329+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 10) v1 ==== 45336+0+0 (secure 0 0 0) 0x7fb63001ac10 con 0x7fb640106b00 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:39.329+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 msgr2=0x7fb62803ad00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T15:44:40.968 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:39.329+0000 7fb627fff700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 0x7fb62803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.330+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 11) v1 ==== 45463+0+0 (secure 0 0 0) 0x7fb63004dab0 con 0x7fb640106b00 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.330+0000 7fb627fff700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb62803bff0 0x7fb62803e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.330+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 --> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb62803b410 con 0x7fb62803bff0 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.331+0000 7fb63e59c700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb62803bff0 0x7fb62803e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.331+0000 7fb63e59c700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb62803bff0 0x7fb62803e3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb634003a10 tx=0x7fb634009270 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.332+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 <== mgr.14162 v2:172.21.15.49:6800/1508146783 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fb62803b410 con 0x7fb62803bff0 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 -- 172.21.15.49:0/3666647149 --> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb6400625d0 con 0x7fb62803bff0 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb627fff700 1 -- 172.21.15.49:0/3666647149 <== mgr.14162 v2:172.21.15.49:6800/1508146783 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fb6400625d0 con 0x7fb62803bff0 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 -- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb62803bff0 msgr2=0x7fb62803e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb62803bff0 0x7fb62803e3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb634003a10 tx=0x7fb634009270 comp rx=0 tx=0).stop 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 -- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 msgr2=0x7fb640199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640199220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb630009fb0 tx=0x7fb630011890 comp rx=0 tx=0).stop 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 -- 172.21.15.49:0/3666647149 shutdown_connections 2024-05-15T15:44:40.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb62803bff0 0x7fb62803e3e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:40.970 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb640106b00 0x7fb640199220 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:40.970 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 --2- 172.21.15.49:0/3666647149 >> [v2:172.21.15.49:6800/821763420,v1:172.21.15.49:6801/821763420] conn(0x7fb628038840 0x7fb62803ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:40.970 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 -- 172.21.15.49:0/3666647149 >> 172.21.15.49:0/3666647149 conn(0x7fb640100290 msgr2=0x7fb640100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:40.970 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 -- 172.21.15.49:0/3666647149 shutdown_connections 2024-05-15T15:44:40.970 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:40.339+0000 7fb6451a6700 1 -- 172.21.15.49:0/3666647149 wait complete. 2024-05-15T15:44:40.970 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 9 is available 2024-05-15T15:44:40.970 INFO:teuthology.orchestra.run.smithi049.stdout:Generating a dashboard self-signed certificate... 2024-05-15T15:44:41.584 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:41 smithi049 ceph-mon[28399]: [15/May/2024:15:44:40] ENGINE Bus STARTING 2024-05-15T15:44:41.584 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:41 smithi049 ceph-mon[28399]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-15T15:44:41.585 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:41 smithi049 ceph-mon[28399]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-15T15:44:41.585 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:41 smithi049 ceph-mon[28399]: [15/May/2024:15:44:40] ENGINE Serving on http://172.21.15.49:8765 2024-05-15T15:44:41.585 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:41 smithi049 ceph-mon[28399]: [15/May/2024:15:44:40] ENGINE Serving on https://172.21.15.49:7150 2024-05-15T15:44:41.585 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:41 smithi049 ceph-mon[28399]: [15/May/2024:15:44:40] ENGINE Bus STARTED 2024-05-15T15:44:42.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-15T15:44:42.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.358+0000 7fada53a0700 1 Processor -- start 2024-05-15T15:44:42.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.358+0000 7fada53a0700 1 -- start start 2024-05-15T15:44:42.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.358+0000 7fada53a0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0109530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:42.203 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.358+0000 7fada53a0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0062280 con 0x7fada0107140 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.359+0000 7fad9effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.359+0000 7fad9effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0109530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42790/0 (socket says 172.21.15.49:42790) 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.359+0000 7fad9effd700 1 -- 172.21.15.49:0/2916226697 learned_addr learned my addr 172.21.15.49:0/2916226697 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.359+0000 7fad9effd700 1 -- 172.21.15.49:0/2916226697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada0109a70 con 0x7fada0107140 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.359+0000 7fad9effd700 1 --2- 172.21.15.49:0/2916226697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0109530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad90009e80 tx=0x7fad9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b57bffbce9cd723 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.359+0000 7fad9dffb700 1 -- 172.21.15.49:0/2916226697 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad9000e070 con 0x7fada0107140 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.359+0000 7fad9dffb700 1 -- 172.21.15.49:0/2916226697 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad9000b8c0 con 0x7fada0107140 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.359+0000 7fad9dffb700 1 -- 172.21.15.49:0/2916226697 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad900044d0 con 0x7fada0107140 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.360+0000 7fada53a0700 1 -- 172.21.15.49:0/2916226697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 msgr2=0x7fada0109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.360+0000 7fada53a0700 1 --2- 172.21.15.49:0/2916226697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0109530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad90009e80 tx=0x7fad9000b330 comp rx=0 tx=0).stop 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.360+0000 7fada53a0700 1 -- 172.21.15.49:0/2916226697 shutdown_connections 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.360+0000 7fada53a0700 1 --2- 172.21.15.49:0/2916226697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0109530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:42.204 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.360+0000 7fada53a0700 1 -- 172.21.15.49:0/2916226697 >> 172.21.15.49:0/2916226697 conn(0x7fada0100af0 msgr2=0x7fada0102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.360+0000 7fada53a0700 1 -- 172.21.15.49:0/2916226697 shutdown_connections 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.360+0000 7fada53a0700 1 -- 172.21.15.49:0/2916226697 wait complete. 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.361+0000 7fada53a0700 1 Processor -- start 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.361+0000 7fada53a0700 1 -- start start 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.361+0000 7fada53a0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0106970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.361+0000 7fada53a0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0106eb0 con 0x7fada0107140 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.361+0000 7fad9effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.361+0000 7fad9effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42806/0 (socket says 172.21.15.49:42806) 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.361+0000 7fad9effd700 1 -- 172.21.15.49:0/1007667824 learned_addr learned my addr 172.21.15.49:0/1007667824 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.361+0000 7fad9effd700 1 -- 172.21.15.49:0/1007667824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad90009b30 con 0x7fada0107140 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.362+0000 7fad9effd700 1 --2- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0106970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad900072a0 tx=0x7fad9000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.362+0000 7fad97fff700 1 -- 172.21.15.49:0/1007667824 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad9000e040 con 0x7fada0107140 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.362+0000 7fad97fff700 1 -- 172.21.15.49:0/1007667824 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad90003dc0 con 0x7fada0107140 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.362+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada01050f0 con 0x7fada0107140 2024-05-15T15:44:42.205 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.362+0000 7fad97fff700 1 -- 172.21.15.49:0/1007667824 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad90012620 con 0x7fada0107140 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.362+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada01055f0 con 0x7fada0107140 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.363+0000 7fad97fff700 1 -- 172.21.15.49:0/1007667824 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 11) v1 ==== 45463+0+0 (secure 0 0 0) 0x7fad90019040 con 0x7fada0107140 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.363+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada010a800 con 0x7fada0107140 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.363+0000 7fad97fff700 1 --2- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fad8c0385f0 0x7fad8c03aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.363+0000 7fad97fff700 1 -- 172.21.15.49:0/1007667824 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fad9004c260 con 0x7fada0107140 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.363+0000 7fad9e7fc700 1 --2- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fad8c0385f0 0x7fad8c03aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.363+0000 7fad9e7fc700 1 --2- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fad8c0385f0 0x7fad8c03aab0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fad88006fd0 tx=0x7fad88006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.368+0000 7fad97fff700 1 -- 172.21.15.49:0/1007667824 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fad900119e0 con 0x7fada0107140 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.383+0000 7fad97fff700 1 -- 172.21.15.49:0/1007667824 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 12) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fad90018d20 con 0x7fada0107140 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.581+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 --> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fada0105d40 con 0x7fad8c0385f0 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.624+0000 7fad97fff700 1 -- 172.21.15.49:0/1007667824 <== mgr.14162 v2:172.21.15.49:6800/1508146783 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fada0105d40 con 0x7fad8c0385f0 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.626+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fad8c0385f0 msgr2=0x7fad8c03aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.626+0000 7fada53a0700 1 --2- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fad8c0385f0 0x7fad8c03aab0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fad88006fd0 tx=0x7fad88006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:42.206 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.626+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 msgr2=0x7fada0106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:42.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.626+0000 7fada53a0700 1 --2- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0106970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad900072a0 tx=0x7fad9000bf70 comp rx=0 tx=0).stop 2024-05-15T15:44:42.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.626+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 shutdown_connections 2024-05-15T15:44:42.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.626+0000 7fada53a0700 1 --2- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fad8c0385f0 0x7fad8c03aab0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:42.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.626+0000 7fada53a0700 1 --2- 172.21.15.49:0/1007667824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada0107140 0x7fada0106970 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:42.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.626+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 >> 172.21.15.49:0/1007667824 conn(0x7fada0100af0 msgr2=0x7fada01016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:42.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.627+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 shutdown_connections 2024-05-15T15:44:42.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:41.627+0000 7fada53a0700 1 -- 172.21.15.49:0/1007667824 wait complete. 2024-05-15T15:44:42.207 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial admin user... 2024-05-15T15:44:42.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:42 smithi049 ceph-mon[28399]: mgrmap e12: smithi049.vuhzgf(active, since 2s) 2024-05-15T15:44:42.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:42 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:42.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:42 smithi049 ceph-mon[28399]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:42.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:42 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:42.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:42 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:43.736 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:43 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:43.736 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:43 smithi049 ceph-mon[28399]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:43.737 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:43 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:44.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$vN5nFbO5oexMIvoKe8gBa.KlKL46l6jryZO6grmiOpSXnkA0jv9ZO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715787883, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.679+0000 7f1153f91700 1 Processor -- start 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.680+0000 7f1153f91700 1 -- start start 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.680+0000 7f1153f91700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c1059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.680+0000 7f1153f91700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c105f80 con 0x7f114c105590 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.680+0000 7f1151d2d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c1059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.680+0000 7f1151d2d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c1059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42818/0 (socket says 172.21.15.49:42818) 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.680+0000 7f1151d2d700 1 -- 172.21.15.49:0/2166258412 learned_addr learned my addr 172.21.15.49:0/2166258412 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.680+0000 7f1151d2d700 1 -- 172.21.15.49:0/2166258412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c106790 con 0x7f114c105590 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.681+0000 7f1151d2d700 1 --2- 172.21.15.49:0/2166258412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c1059b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f11400098a0 tx=0x7f1140009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424e6b60cb9eb601 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.681+0000 7f1150d2b700 1 -- 172.21.15.49:0/2166258412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1140004180 con 0x7f114c105590 2024-05-15T15:44:44.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.681+0000 7f1150d2b700 1 -- 172.21.15.49:0/2166258412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11400042e0 con 0x7f114c105590 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.681+0000 7f1150d2b700 1 -- 172.21.15.49:0/2166258412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1140011b10 con 0x7f114c105590 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.681+0000 7f1153f91700 1 -- 172.21.15.49:0/2166258412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 msgr2=0x7f114c1059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.681+0000 7f1153f91700 1 --2- 172.21.15.49:0/2166258412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c1059b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f11400098a0 tx=0x7f1140009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.681+0000 7f1153f91700 1 -- 172.21.15.49:0/2166258412 shutdown_connections 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.681+0000 7f1153f91700 1 --2- 172.21.15.49:0/2166258412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c1059b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.681+0000 7f1153f91700 1 -- 172.21.15.49:0/2166258412 >> 172.21.15.49:0/2166258412 conn(0x7f114c100af0 msgr2=0x7f114c102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.682+0000 7f1153f91700 1 -- 172.21.15.49:0/2166258412 shutdown_connections 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.682+0000 7f1153f91700 1 -- 172.21.15.49:0/2166258412 wait complete. 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.682+0000 7f1153f91700 1 Processor -- start 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.682+0000 7f1153f91700 1 -- start start 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.682+0000 7f1153f91700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c190650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.682+0000 7f1153f91700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c190b90 con 0x7f114c105590 2024-05-15T15:44:44.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f1151d2d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c190650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f1151d2d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c190650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42820/0 (socket says 172.21.15.49:42820) 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f1151d2d700 1 -- 172.21.15.49:0/193585061 learned_addr learned my addr 172.21.15.49:0/193585061 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f1151d2d700 1 -- 172.21.15.49:0/193585061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140009550 con 0x7f114c105590 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f1151d2d700 1 --2- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c190650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1140009fb0 tx=0x7f1140011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f113effd700 1 -- 172.21.15.49:0/193585061 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11400038b0 con 0x7f114c105590 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f113effd700 1 -- 172.21.15.49:0/193585061 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1140003a10 con 0x7f114c105590 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f113effd700 1 -- 172.21.15.49:0/193585061 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1140019770 con 0x7f114c105590 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c18edd0 con 0x7f114c105590 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.683+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c18f270 con 0x7f114c105590 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.684+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c0625d0 con 0x7f114c105590 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.684+0000 7f113effd700 1 -- 172.21.15.49:0/193585061 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f114001ac10 con 0x7f114c105590 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.685+0000 7f113effd700 1 --2- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f1138038730 0x7f113803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.685+0000 7f113effd700 1 -- 172.21.15.49:0/193585061 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f114004c950 con 0x7f114c105590 2024-05-15T15:44:44.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.685+0000 7f115152c700 1 --2- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f1138038730 0x7f113803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.685+0000 7f115152c700 1 --2- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f1138038730 0x7f113803abf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1148006fd0 tx=0x7f1148006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.694+0000 7f113effd700 1 -- 172.21.15.49:0/193585061 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f11400198d0 con 0x7f114c105590 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:42.932+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 --> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f114c074c20 con 0x7f1138038730 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.178+0000 7f113effd700 1 -- 172.21.15.49:0/193585061 <== mgr.14162 v2:172.21.15.49:6800/1508146783 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f1148009e20 con 0x7f1138038730 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.180+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f1138038730 msgr2=0x7f113803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.180+0000 7f1153f91700 1 --2- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f1138038730 0x7f113803abf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1148006fd0 tx=0x7f1148006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.180+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 msgr2=0x7f114c190650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.180+0000 7f1153f91700 1 --2- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c190650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1140009fb0 tx=0x7f1140011890 comp rx=0 tx=0).stop 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.181+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 shutdown_connections 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.181+0000 7f1153f91700 1 --2- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f1138038730 0x7f113803abf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.181+0000 7f1153f91700 1 --2- 172.21.15.49:0/193585061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f114c105590 0x7f114c190650 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:44.312 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.181+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 >> 172.21.15.49:0/193585061 conn(0x7f114c100af0 msgr2=0x7f114c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:44.313 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.181+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 shutdown_connections 2024-05-15T15:44:44.313 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:43.181+0000 7f1153f91700 1 -- 172.21.15.49:0/193585061 wait complete. 2024-05-15T15:44:44.313 INFO:teuthology.orchestra.run.smithi049.stdout:Fetching dashboard port number... 2024-05-15T15:44:45.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:44 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/633147479' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-15T15:44:45.470 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 8443 2024-05-15T15:44:45.470 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.711+0000 7fcc4ae0a700 1 Processor -- start 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.711+0000 7fcc4ae0a700 1 -- start start 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.711+0000 7fcc4ae0a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc44106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.711+0000 7fcc4ae0a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc44107050 con 0x7fcc44106660 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.712+0000 7fcc48ba6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc44106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.712+0000 7fcc48ba6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc44106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46576/0 (socket says 172.21.15.49:46576) 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.712+0000 7fcc48ba6700 1 -- 172.21.15.49:0/2908728212 learned_addr learned my addr 172.21.15.49:0/2908728212 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.712+0000 7fcc48ba6700 1 -- 172.21.15.49:0/2908728212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44107860 con 0x7fcc44106660 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.712+0000 7fcc48ba6700 1 --2- 172.21.15.49:0/2908728212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc44106a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc340098a0 tx=0x7fcc34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20de0a731f8a908 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.712+0000 7fcc437fe700 1 -- 172.21.15.49:0/2908728212 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc34004180 con 0x7fcc44106660 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.712+0000 7fcc437fe700 1 -- 172.21.15.49:0/2908728212 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc340042e0 con 0x7fcc44106660 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.712+0000 7fcc437fe700 1 -- 172.21.15.49:0/2908728212 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc34011b10 con 0x7fcc44106660 2024-05-15T15:44:45.471 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.713+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/2908728212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 msgr2=0x7fcc44106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.713+0000 7fcc4ae0a700 1 --2- 172.21.15.49:0/2908728212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc44106a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc340098a0 tx=0x7fcc34009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.713+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/2908728212 shutdown_connections 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.713+0000 7fcc4ae0a700 1 --2- 172.21.15.49:0/2908728212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc44106a80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.713+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/2908728212 >> 172.21.15.49:0/2908728212 conn(0x7fcc44101c40 msgr2=0x7fcc44104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.713+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/2908728212 shutdown_connections 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.713+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/2908728212 wait complete. 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.713+0000 7fcc4ae0a700 1 Processor -- start 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc4ae0a700 1 -- start start 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc4ae0a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc4419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc4ae0a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4419a890 con 0x7fcc44106660 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc48ba6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc4419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc48ba6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc4419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46586/0 (socket says 172.21.15.49:46586) 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc48ba6700 1 -- 172.21.15.49:0/633147479 learned_addr learned my addr 172.21.15.49:0/633147479 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:45.472 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc48ba6700 1 -- 172.21.15.49:0/633147479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc34009550 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc48ba6700 1 --2- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc4419a350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc340117b0 tx=0x7fcc34011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc41ffb700 1 -- 172.21.15.49:0/633147479 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc340038b0 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc41ffb700 1 -- 172.21.15.49:0/633147479 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc34003a10 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4419aa90 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.714+0000 7fcc41ffb700 1 -- 172.21.15.49:0/633147479 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc340246e0 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.715+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4419af30 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.715+0000 7fcc41ffb700 1 -- 172.21.15.49:0/633147479 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fcc3401ab80 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.716+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4410aa90 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.716+0000 7fcc41ffb700 1 --2- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fcc30038880 0x7fcc3003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.716+0000 7fcc41ffb700 1 -- 172.21.15.49:0/633147479 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc3404ca40 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.716+0000 7fcc43fff700 1 --2- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fcc30038880 0x7fcc3003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.716+0000 7fcc43fff700 1 --2- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fcc30038880 0x7fcc3003ad40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c006fd0 tx=0x7fcc2c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.721+0000 7fcc41ffb700 1 -- 172.21.15.49:0/633147479 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcc3401a430 con 0x7fcc44106660 2024-05-15T15:44:45.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.924+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fcc4419d7c0 con 0x7fcc44106660 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.924+0000 7fcc41ffb700 1 -- 172.21.15.49:0/633147479 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fcc34018b40 con 0x7fcc44106660 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.927+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fcc30038880 msgr2=0x7fcc3003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.927+0000 7fcc4ae0a700 1 --2- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fcc30038880 0x7fcc3003ad40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c006fd0 tx=0x7fcc2c006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.927+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 msgr2=0x7fcc4419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.927+0000 7fcc4ae0a700 1 --2- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc4419a350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc340117b0 tx=0x7fcc34011890 comp rx=0 tx=0).stop 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.928+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 shutdown_connections 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.928+0000 7fcc4ae0a700 1 --2- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fcc30038880 0x7fcc3003ad40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.928+0000 7fcc4ae0a700 1 --2- 172.21.15.49:0/633147479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc44106660 0x7fcc4419a350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.928+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 >> 172.21.15.49:0/633147479 conn(0x7fcc44101c40 msgr2=0x7fcc441026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.928+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 shutdown_connections 2024-05-15T15:44:45.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:44.928+0000 7fcc4ae0a700 1 -- 172.21.15.49:0/633147479 wait complete. 2024-05-15T15:44:45.478 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-15T15:44:45.478 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-05-15T15:44:45.486 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-15T15:44:45.486 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-05-15T15:44:45.486 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-05-15T15:44:45.486 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-15T15:44:45.488 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph Dashboard is now available at: 2024-05-15T15:44:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: URL: https://smithi049.front.sepia.ceph.com:8443/ 2024-05-15T15:44:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: User: admin 2024-05-15T15:44:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: Password: z3vmjilsl0 2024-05-15T15:44:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:45.489 INFO:teuthology.orchestra.run.smithi049.stdout:Saving cluster configuration to /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config directory 2024-05-15T15:44:45.489 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling autotune for osd_memory_target 2024-05-15T15:44:46.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a8b9e700 1 Processor -- start 2024-05-15T15:44:46.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a8b9e700 1 -- start start 2024-05-15T15:44:46.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a8b9e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a41059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:46.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a8b9e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4105f80 con 0x7f62a4105590 2024-05-15T15:44:46.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a41059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:46.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a41059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46602/0 (socket says 172.21.15.49:46602) 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a259c700 1 -- 172.21.15.49:0/3640086476 learned_addr learned my addr 172.21.15.49:0/3640086476 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a259c700 1 -- 172.21.15.49:0/3640086476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4106790 con 0x7f62a4105590 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a259c700 1 --2- 172.21.15.49:0/3640086476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a41059b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f62940098a0 tx=0x7f6294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e44be808d76728 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a159a700 1 -- 172.21.15.49:0/3640086476 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6294004180 con 0x7f62a4105590 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a159a700 1 -- 172.21.15.49:0/3640086476 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62940042e0 con 0x7f62a4105590 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.878+0000 7f62a159a700 1 -- 172.21.15.49:0/3640086476 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6294011b10 con 0x7f62a4105590 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.879+0000 7f62a8b9e700 1 -- 172.21.15.49:0/3640086476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 msgr2=0x7f62a41059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.879+0000 7f62a8b9e700 1 --2- 172.21.15.49:0/3640086476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a41059b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f62940098a0 tx=0x7f6294009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.879+0000 7f62a8b9e700 1 -- 172.21.15.49:0/3640086476 shutdown_connections 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.879+0000 7f62a8b9e700 1 --2- 172.21.15.49:0/3640086476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a41059b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.879+0000 7f62a8b9e700 1 -- 172.21.15.49:0/3640086476 >> 172.21.15.49:0/3640086476 conn(0x7f62a4100af0 msgr2=0x7f62a4102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.879+0000 7f62a8b9e700 1 -- 172.21.15.49:0/3640086476 shutdown_connections 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.879+0000 7f62a8b9e700 1 -- 172.21.15.49:0/3640086476 wait complete. 2024-05-15T15:44:46.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.880+0000 7f62a8b9e700 1 Processor -- start 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.880+0000 7f62a8b9e700 1 -- start start 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.880+0000 7f62a8b9e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a4190650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.880+0000 7f62a8b9e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4190b90 con 0x7f62a4105590 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.880+0000 7f62a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a4190650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.880+0000 7f62a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a4190650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46610/0 (socket says 172.21.15.49:46610) 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.880+0000 7f62a259c700 1 -- 172.21.15.49:0/963144746 learned_addr learned my addr 172.21.15.49:0/963144746 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.880+0000 7f62a259c700 1 -- 172.21.15.49:0/963144746 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6294009550 con 0x7f62a4105590 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.881+0000 7f62a259c700 1 --2- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a4190650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6294009fb0 tx=0x7f6294011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.881+0000 7f62937fe700 1 -- 172.21.15.49:0/963144746 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62940038b0 con 0x7f62a4105590 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.881+0000 7f62937fe700 1 -- 172.21.15.49:0/963144746 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6294003a10 con 0x7f62a4105590 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.881+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a418edd0 con 0x7f62a4105590 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.881+0000 7f62937fe700 1 -- 172.21.15.49:0/963144746 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6294019770 con 0x7f62a4105590 2024-05-15T15:44:46.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.881+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a418f270 con 0x7f62a4105590 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.882+0000 7f62937fe700 1 -- 172.21.15.49:0/963144746 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f62940198d0 con 0x7f62a4105590 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.882+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a418fc80 con 0x7f62a4105590 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.882+0000 7f62937fe700 1 --2- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f628c038780 0x7f628c03ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.882+0000 7f62937fe700 1 -- 172.21.15.49:0/963144746 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f629404ca20 con 0x7f62a4105590 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.882+0000 7f62a1d9b700 1 --2- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f628c038780 0x7f628c03ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.883+0000 7f62a1d9b700 1 --2- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f628c038780 0x7f628c03ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6298006fd0 tx=0x7f6298006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.889+0000 7f62937fe700 1 -- 172.21.15.49:0/963144746 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f62940224f0 con 0x7f62a4105590 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:45.979+0000 7f62937fe700 1 -- 172.21.15.49:0/963144746 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f629401ac10 con 0x7f62a4105590 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.094+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f62a40625d0 con 0x7f62a4105590 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.095+0000 7f62937fe700 1 -- 172.21.15.49:0/963144746 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f62940228e0 con 0x7f62a4105590 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f628c038780 msgr2=0x7f628c03ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 --2- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f628c038780 0x7f628c03ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6298006fd0 tx=0x7f6298006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 msgr2=0x7f62a4190650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:46.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 --2- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a4190650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6294009fb0 tx=0x7f6294011890 comp rx=0 tx=0).stop 2024-05-15T15:44:46.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 shutdown_connections 2024-05-15T15:44:46.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 --2- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f628c038780 0x7f628c03ac40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:46.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 --2- 172.21.15.49:0/963144746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a4105590 0x7f62a4190650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:46.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 >> 172.21.15.49:0/963144746 conn(0x7f62a4100af0 msgr2=0x7f62a4101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:46.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 shutdown_connections 2024-05-15T15:44:46.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:46.097+0000 7f62a8b9e700 1 -- 172.21.15.49:0/963144746 wait complete. 2024-05-15T15:44:47.447 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:46 smithi049 ceph-mon[28399]: mgrmap e13: smithi049.vuhzgf(active, since 6s) 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.120+0000 7f9daa330700 1 Processor -- start 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.120+0000 7f9daa330700 1 -- start start 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9daa330700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da4106b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9daa330700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da4107140 con 0x7f9da4106750 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da4106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da4106b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46618/0 (socket says 172.21.15.49:46618) 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9da3fff700 1 -- 172.21.15.49:0/4140687186 learned_addr learned my addr 172.21.15.49:0/4140687186 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9da3fff700 1 -- 172.21.15.49:0/4140687186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da4107900 con 0x7f9da4106750 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9da3fff700 1 --2- 172.21.15.49:0/4140687186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da4106b70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9d940098a0 tx=0x7f9d94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29650d8a2d2435a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9da37fe700 1 -- 172.21.15.49:0/4140687186 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d94004180 con 0x7f9da4106750 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9da37fe700 1 -- 172.21.15.49:0/4140687186 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d940042e0 con 0x7f9da4106750 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.121+0000 7f9da37fe700 1 -- 172.21.15.49:0/4140687186 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d94011b10 con 0x7f9da4106750 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.122+0000 7f9daa330700 1 -- 172.21.15.49:0/4140687186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 msgr2=0x7f9da4106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.122+0000 7f9daa330700 1 --2- 172.21.15.49:0/4140687186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da4106b70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9d940098a0 tx=0x7f9d94009bb0 comp rx=0 tx=0).stop 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.122+0000 7f9daa330700 1 -- 172.21.15.49:0/4140687186 shutdown_connections 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.122+0000 7f9daa330700 1 --2- 172.21.15.49:0/4140687186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da4106b70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.122+0000 7f9daa330700 1 -- 172.21.15.49:0/4140687186 >> 172.21.15.49:0/4140687186 conn(0x7f9da4101cd0 msgr2=0x7f9da4104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.122+0000 7f9daa330700 1 -- 172.21.15.49:0/4140687186 shutdown_connections 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.122+0000 7f9daa330700 1 -- 172.21.15.49:0/4140687186 wait complete. 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.123+0000 7f9daa330700 1 Processor -- start 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.123+0000 7f9daa330700 1 -- start start 2024-05-15T15:44:48.019 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.123+0000 7f9daa330700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da407bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:48.020 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.123+0000 7f9daa330700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da407c050 con 0x7f9da4106750 2024-05-15T15:44:48.020 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.123+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da407bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:48.020 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.123+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da407bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46632/0 (socket says 172.21.15.49:46632) 2024-05-15T15:44:48.020 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.123+0000 7f9da3fff700 1 -- 172.21.15.49:0/2093188683 learned_addr learned my addr 172.21.15.49:0/2093188683 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:48.020 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.123+0000 7f9da3fff700 1 -- 172.21.15.49:0/2093188683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94009550 con 0x7f9da4106750 2024-05-15T15:44:48.020 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.124+0000 7f9da3fff700 1 --2- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da407bb10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009fb0 tx=0x7f9d94011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:48.020 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.124+0000 7f9da1ffb700 1 -- 172.21.15.49:0/2093188683 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d940038b0 con 0x7f9da4106750 2024-05-15T15:44:48.020 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.124+0000 7f9da1ffb700 1 -- 172.21.15.49:0/2093188683 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d94003a10 con 0x7f9da4106750 2024-05-15T15:44:48.020 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.124+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da407a290 con 0x7f9da4106750 2024-05-15T15:44:48.021 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.124+0000 7f9da1ffb700 1 -- 172.21.15.49:0/2093188683 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d94019770 con 0x7f9da4106750 2024-05-15T15:44:48.021 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.124+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da407a730 con 0x7f9da4106750 2024-05-15T15:44:48.021 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.125+0000 7f9da1ffb700 1 -- 172.21.15.49:0/2093188683 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f9d9401ac10 con 0x7f9da4106750 2024-05-15T15:44:48.021 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.125+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da40625d0 con 0x7f9da4106750 2024-05-15T15:44:48.021 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.125+0000 7f9da1ffb700 1 --2- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f9d900388e0 0x7f9d9003ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:48.021 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.125+0000 7f9da1ffb700 1 -- 172.21.15.49:0/2093188683 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9d9404c990 con 0x7f9da4106750 2024-05-15T15:44:48.021 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.125+0000 7f9d9bfff700 1 --2- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f9d900388e0 0x7f9d9003ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.126+0000 7f9d9bfff700 1 --2- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f9d900388e0 0x7f9d9003ada0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c006fd0 tx=0x7f9d8c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.131+0000 7f9da1ffb700 1 -- 172.21.15.49:0/2093188683 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d940198d0 con 0x7f9da4106750 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.446+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f9da410ab30 con 0x7f9da4106750 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.448+0000 7f9da1ffb700 1 -- 172.21.15.49:0/2093188683 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f9d94022310 con 0x7f9da4106750 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f9d900388e0 msgr2=0x7f9d9003ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 --2- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f9d900388e0 0x7f9d9003ada0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c006fd0 tx=0x7f9d8c006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 msgr2=0x7f9da407bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 --2- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da407bb10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009fb0 tx=0x7f9d94011890 comp rx=0 tx=0).stop 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 shutdown_connections 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 --2- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f9d900388e0 0x7f9d9003ada0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 --2- 172.21.15.49:0/2093188683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9da4106750 0x7f9da407bb10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:48.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 >> 172.21.15.49:0/2093188683 conn(0x7f9da4101cd0 msgr2=0x7f9da4102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 shutdown_connections 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-05-15T15:44:47.450+0000 7f9daa330700 1 -- 172.21.15.49:0/2093188683 wait complete. 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout:Or, if you are only running a single cluster on this host: 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-15T15:44:48.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:48.024 INFO:teuthology.orchestra.run.smithi049.stdout: ceph telemetry on 2024-05-15T15:44:48.024 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:48.024 INFO:teuthology.orchestra.run.smithi049.stdout:For more information see: 2024-05-15T15:44:48.024 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:48.024 INFO:teuthology.orchestra.run.smithi049.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-15T15:44:48.024 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:44:48.024 INFO:teuthology.orchestra.run.smithi049.stdout:Bootstrap complete. 2024-05-15T15:44:48.048 INFO:tasks.cephadm:Fetching config... 2024-05-15T15:44:48.049 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:44:48.049 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-15T15:44:48.105 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-15T15:44:48.105 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:44:48.105 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-15T15:44:48.162 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-15T15:44:48.162 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:44:48.163 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/keyring of=/dev/stdout 2024-05-15T15:44:48.228 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-15T15:44:48.228 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:44:48.228 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-15T15:44:48.284 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-15T15:44:48.284 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfEG9qhxHuubmj2d+B5tK5GzpmTBkzdiBJ/g1LvRDWFffflgX2Cc1lr2G0Er/MlP7/RORAl1cVpTCk7K3Q9cmQOnbB0rjSAt8SaDBSnhq6eNUJzYUKkoPxBu2CNOSYH5dihV0CIVYneejxwqTbt243X6Lyc3isA3+K1XyUwTBm5vsK6vcIZ8346B2CpCHTz1yICCKcroLi2EcTTRFBzP7fyskHV68PQ+yNDLfun7ThQN0enHaRyWJuYbFjqQ387GI9e+DLterBT2tkFp/czKOx/3FFLAM5NIpwpUVeP6W1OSWeohOo3Fe+9X4fst3vtvGlYOJ57CnE1wK0iTWFg9TPltwiFDL43bFoII48TOS95AJM+WVADPLak8BmeuCjTD2XL3RnDbuF5VowI+es7iiPH9nac/3Je/UzdZItaG5ieAm7P3rAwERFA9rt6NUalJgay8rI87Tl66E2dGAN93KiV+aVMXrKT+7sY3GMhFybJh1Ph/WrXED42uDflZNe47E= ceph-c619ac44-12d1-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-15T15:44:48.361 INFO:teuthology.orchestra.run.smithi049.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfEG9qhxHuubmj2d+B5tK5GzpmTBkzdiBJ/g1LvRDWFffflgX2Cc1lr2G0Er/MlP7/RORAl1cVpTCk7K3Q9cmQOnbB0rjSAt8SaDBSnhq6eNUJzYUKkoPxBu2CNOSYH5dihV0CIVYneejxwqTbt243X6Lyc3isA3+K1XyUwTBm5vsK6vcIZ8346B2CpCHTz1yICCKcroLi2EcTTRFBzP7fyskHV68PQ+yNDLfun7ThQN0enHaRyWJuYbFjqQ387GI9e+DLterBT2tkFp/czKOx/3FFLAM5NIpwpUVeP6W1OSWeohOo3Fe+9X4fst3vtvGlYOJ57CnE1wK0iTWFg9TPltwiFDL43bFoII48TOS95AJM+WVADPLak8BmeuCjTD2XL3RnDbuF5VowI+es7iiPH9nac/3Je/UzdZItaG5ieAm7P3rAwERFA9rt6NUalJgay8rI87Tl66E2dGAN93KiV+aVMXrKT+7sY3GMhFybJh1Ph/WrXED42uDflZNe47E= ceph-c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:44:48.373 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfEG9qhxHuubmj2d+B5tK5GzpmTBkzdiBJ/g1LvRDWFffflgX2Cc1lr2G0Er/MlP7/RORAl1cVpTCk7K3Q9cmQOnbB0rjSAt8SaDBSnhq6eNUJzYUKkoPxBu2CNOSYH5dihV0CIVYneejxwqTbt243X6Lyc3isA3+K1XyUwTBm5vsK6vcIZ8346B2CpCHTz1yICCKcroLi2EcTTRFBzP7fyskHV68PQ+yNDLfun7ThQN0enHaRyWJuYbFjqQ387GI9e+DLterBT2tkFp/czKOx/3FFLAM5NIpwpUVeP6W1OSWeohOo3Fe+9X4fst3vtvGlYOJ57CnE1wK0iTWFg9TPltwiFDL43bFoII48TOS95AJM+WVADPLak8BmeuCjTD2XL3RnDbuF5VowI+es7iiPH9nac/3Je/UzdZItaG5ieAm7P3rAwERFA9rt6NUalJgay8rI87Tl66E2dGAN93KiV+aVMXrKT+7sY3GMhFybJh1Ph/WrXED42uDflZNe47E= ceph-c619ac44-12d1-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-15T15:44:48.415 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfEG9qhxHuubmj2d+B5tK5GzpmTBkzdiBJ/g1LvRDWFffflgX2Cc1lr2G0Er/MlP7/RORAl1cVpTCk7K3Q9cmQOnbB0rjSAt8SaDBSnhq6eNUJzYUKkoPxBu2CNOSYH5dihV0CIVYneejxwqTbt243X6Lyc3isA3+K1XyUwTBm5vsK6vcIZ8346B2CpCHTz1yICCKcroLi2EcTTRFBzP7fyskHV68PQ+yNDLfun7ThQN0enHaRyWJuYbFjqQ387GI9e+DLterBT2tkFp/czKOx/3FFLAM5NIpwpUVeP6W1OSWeohOo3Fe+9X4fst3vtvGlYOJ57CnE1wK0iTWFg9TPltwiFDL43bFoII48TOS95AJM+WVADPLak8BmeuCjTD2XL3RnDbuF5VowI+es7iiPH9nac/3Je/UzdZItaG5ieAm7P3rAwERFA9rt6NUalJgay8rI87Tl66E2dGAN93KiV+aVMXrKT+7sY3GMhFybJh1Ph/WrXED42uDflZNe47E= ceph-c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:44:48.427 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-15T15:44:48.570 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:48 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2093188683' entity='client.admin' 2024-05-15T15:44:48.638 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:44:51.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.236+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/2797383416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 msgr2=0x7fd1dc0ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.236+0000 7fd1e1f4a700 1 --2- 172.21.15.49:0/2797383416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 0x7fd1dc0ffc50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00b0a0 tx=0x7fd1cc00b3b0 comp rx=0 tx=0).stop 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.236+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/2797383416 shutdown_connections 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.236+0000 7fd1e1f4a700 1 --2- 172.21.15.49:0/2797383416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 0x7fd1dc0ffc50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.236+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/2797383416 >> 172.21.15.49:0/2797383416 conn(0x7fd1dc0fadd0 msgr2=0x7fd1dc0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.237+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/2797383416 shutdown_connections 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.237+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/2797383416 wait complete. 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.237+0000 7fd1e1f4a700 1 Processor -- start 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.237+0000 7fd1e1f4a700 1 -- start start 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.237+0000 7fd1e1f4a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 0x7fd1dc195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.237+0000 7fd1e1f4a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc1962f0 con 0x7fd1dc0ff830 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1db7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 0x7fd1dc195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1db7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 0x7fd1dc195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46660/0 (socket says 172.21.15.49:46660) 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1db7fe700 1 -- 172.21.15.49:0/1800690868 learned_addr learned my addr 172.21.15.49:0/1800690868 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1db7fe700 1 -- 172.21.15.49:0/1800690868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc009d00 con 0x7fd1dc0ff830 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1db7fe700 1 --2- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 0x7fd1dc195db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0062a0 tx=0x7fd1cc00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1d8ff9700 1 -- 172.21.15.49:0/1800690868 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1cc00e040 con 0x7fd1dc0ff830 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1d8ff9700 1 -- 172.21.15.49:0/1800690868 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1cc003d70 con 0x7fd1dc0ff830 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc1964f0 con 0x7fd1dc0ff830 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1d8ff9700 1 -- 172.21.15.49:0/1800690868 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1cc01c890 con 0x7fd1dc0ff830 2024-05-15T15:44:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.238+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc196990 con 0x7fd1dc0ff830 2024-05-15T15:44:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.239+0000 7fd1d8ff9700 1 -- 172.21.15.49:0/1800690868 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fd1cc01cb50 con 0x7fd1dc0ff830 2024-05-15T15:44:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.239+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc18f8d0 con 0x7fd1dc0ff830 2024-05-15T15:44:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.240+0000 7fd1d8ff9700 1 --2- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fd1c80387c0 0x7fd1c803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.240+0000 7fd1d8ff9700 1 -- 172.21.15.49:0/1800690868 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd1cc04cbc0 con 0x7fd1dc0ff830 2024-05-15T15:44:51.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.240+0000 7fd1daffd700 1 --2- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fd1c80387c0 0x7fd1c803ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:51.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.240+0000 7fd1daffd700 1 --2- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fd1c80387c0 0x7fd1c803ac80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4006fd0 tx=0x7fd1c4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.245+0000 7fd1d8ff9700 1 -- 172.21.15.49:0/1800690868 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd1cc01b770 con 0x7fd1dc0ff830 2024-05-15T15:44:51.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.455+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fd1dc0625d0 con 0x7fd1dc0ff830 2024-05-15T15:44:51.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.459+0000 7fd1d8ff9700 1 -- 172.21.15.49:0/1800690868 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd1cc011390 con 0x7fd1dc0ff830 2024-05-15T15:44:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.461+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fd1c80387c0 msgr2=0x7fd1c803ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.461+0000 7fd1e1f4a700 1 --2- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fd1c80387c0 0x7fd1c803ac80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4006fd0 tx=0x7fd1c4006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.461+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 msgr2=0x7fd1dc195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.461+0000 7fd1e1f4a700 1 --2- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 0x7fd1dc195db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0062a0 tx=0x7fd1cc00bd40 comp rx=0 tx=0).stop 2024-05-15T15:44:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.462+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 shutdown_connections 2024-05-15T15:44:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.462+0000 7fd1e1f4a700 1 --2- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fd1c80387c0 0x7fd1c803ac80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.462+0000 7fd1e1f4a700 1 --2- 172.21.15.49:0/1800690868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1dc0ff830 0x7fd1dc195db0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.462+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 >> 172.21.15.49:0/1800690868 conn(0x7fd1dc0fadd0 msgr2=0x7fd1dc0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.462+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 shutdown_connections 2024-05-15T15:44:51.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:51.462+0000 7fd1e1f4a700 1 -- 172.21.15.49:0/1800690868 wait complete. 2024-05-15T15:44:52.583 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-15T15:44:52.584 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-15T15:44:52.763 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:52 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1800690868' entity='client.admin' 2024-05-15T15:44:52.763 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:52 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:52.764 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:52 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:52.764 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:52 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:44:52.764 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:52 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:52.764 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:52 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:44:52.764 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:52 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-15T15:44:52.764 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:52 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:44:52.764 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:52 smithi049 ceph-mon[28399]: Deploying daemon ceph-exporter.smithi049 on smithi049 2024-05-15T15:44:53.670 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:44:55.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.353+0000 7f6be6f39700 1 -- 172.21.15.49:0/3402784524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 msgr2=0x7f6be0102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:55.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.353+0000 7f6be6f39700 1 --2- 172.21.15.49:0/3402784524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 0x7f6be0102350 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009910 tx=0x7f6bd4009c20 comp rx=0 tx=0).stop 2024-05-15T15:44:55.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.353+0000 7f6be6f39700 1 -- 172.21.15.49:0/3402784524 shutdown_connections 2024-05-15T15:44:55.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.353+0000 7f6be6f39700 1 --2- 172.21.15.49:0/3402784524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 0x7f6be0102350 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:55.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.353+0000 7f6be6f39700 1 -- 172.21.15.49:0/3402784524 >> 172.21.15.49:0/3402784524 conn(0x7f6be00fa550 msgr2=0x7f6be00fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:55.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.353+0000 7f6be6f39700 1 -- 172.21.15.49:0/3402784524 shutdown_connections 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.353+0000 7f6be6f39700 1 -- 172.21.15.49:0/3402784524 wait complete. 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.353+0000 7f6be6f39700 1 Processor -- start 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.353+0000 7f6be6f39700 1 -- start start 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6be6f39700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 0x7f6be0195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6be6f39700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be0196330 con 0x7f6be0101f30 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6be4cd5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 0x7f6be0195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6be4cd5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 0x7f6be0195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58298/0 (socket says 172.21.15.49:58298) 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6be4cd5700 1 -- 172.21.15.49:0/2755678823 learned_addr learned my addr 172.21.15.49:0/2755678823 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6be4cd5700 1 -- 172.21.15.49:0/2755678823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd40095f0 con 0x7f6be0101f30 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6be4cd5700 1 --2- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 0x7f6be0195df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4006010 tx=0x7f6bd4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6bddffb700 1 -- 172.21.15.49:0/2755678823 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bd401d070 con 0x7f6be0101f30 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6bddffb700 1 -- 172.21.15.49:0/2755678823 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bd4005dc0 con 0x7f6be0101f30 2024-05-15T15:44:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be0196530 con 0x7f6be0101f30 2024-05-15T15:44:55.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.354+0000 7f6bddffb700 1 -- 172.21.15.49:0/2755678823 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bd4022e90 con 0x7f6be0101f30 2024-05-15T15:44:55.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.355+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be01969d0 con 0x7f6be0101f30 2024-05-15T15:44:55.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.356+0000 7f6bddffb700 1 -- 172.21.15.49:0/2755678823 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f6bd400f4b0 con 0x7f6be0101f30 2024-05-15T15:44:55.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.356+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be018f780 con 0x7f6be0101f30 2024-05-15T15:44:55.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.356+0000 7f6bddffb700 1 --2- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f6bd00387c0 0x7f6bd003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:55.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.356+0000 7f6bddffb700 1 -- 172.21.15.49:0/2755678823 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6bd404cb20 con 0x7f6be0101f30 2024-05-15T15:44:55.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.356+0000 7f6bdffff700 1 --2- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f6bd00387c0 0x7f6bd003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:55.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.356+0000 7f6bdffff700 1 --2- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f6bd00387c0 0x7f6bd003ac80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc006fd0 tx=0x7f6bcc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:55.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.361+0000 7f6bddffb700 1 -- 172.21.15.49:0/2755678823 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6bd4017440 con 0x7f6be0101f30 2024-05-15T15:44:55.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.575+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 --> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f6be002ce10 con 0x7f6bd00387c0 2024-05-15T15:44:55.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.578+0000 7f6bddffb700 1 -- 172.21.15.49:0/2755678823 <== mgr.14162 v2:172.21.15.49:6800/1508146783 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6be002ce10 con 0x7f6bd00387c0 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f6bd00387c0 msgr2=0x7f6bd003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 --2- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f6bd00387c0 0x7f6bd003ac80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc006fd0 tx=0x7f6bcc006e40 comp rx=0 tx=0).stop 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 msgr2=0x7f6be0195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 --2- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 0x7f6be0195df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4006010 tx=0x7f6bd4009f80 comp rx=0 tx=0).stop 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 shutdown_connections 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 --2- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f6bd00387c0 0x7f6bd003ac80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 --2- 172.21.15.49:0/2755678823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6be0101f30 0x7f6be0195df0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 >> 172.21.15.49:0/2755678823 conn(0x7f6be00fa550 msgr2=0x7f6be00fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 shutdown_connections 2024-05-15T15:44:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:55.579+0000 7f6be6f39700 1 -- 172.21.15.49:0/2755678823 wait complete. 2024-05-15T15:44:56.264 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-05-15T15:44:56.264 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:44:56.264 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-05-15T15:44:56.283 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:44:56.283 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:44:56.342 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-05-15T15:44:56.343 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch host add smithi144 2024-05-15T15:44:56.688 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:44:56.688 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:56.688 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:56.688 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:56.688 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:56.688 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:44:56.688 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi049", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:44:56.688 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi049", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-15T15:44:56.689 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:44:56.689 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:56 smithi049 ceph-mon[28399]: Deploying daemon crash.smithi049 on smithi049 2024-05-15T15:44:57.394 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:44:58.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.991+0000 7fdfc39fe700 1 -- 172.21.15.49:0/2444024107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 msgr2=0x7fdfbc0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:44:58.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.991+0000 7fdfc39fe700 1 --2- 172.21.15.49:0/2444024107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 0x7fdfbc0ff5f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009910 tx=0x7fdfb8009c20 comp rx=0 tx=0).stop 2024-05-15T15:44:58.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.991+0000 7fdfc39fe700 1 -- 172.21.15.49:0/2444024107 shutdown_connections 2024-05-15T15:44:58.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.991+0000 7fdfc39fe700 1 --2- 172.21.15.49:0/2444024107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 0x7fdfbc0ff5f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:44:58.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.991+0000 7fdfc39fe700 1 -- 172.21.15.49:0/2444024107 >> 172.21.15.49:0/2444024107 conn(0x7fdfbc0fadf0 msgr2=0x7fdfbc0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:44:58.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.991+0000 7fdfc39fe700 1 -- 172.21.15.49:0/2444024107 shutdown_connections 2024-05-15T15:44:58.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.991+0000 7fdfc39fe700 1 -- 172.21.15.49:0/2444024107 wait complete. 2024-05-15T15:44:58.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.992+0000 7fdfc39fe700 1 Processor -- start 2024-05-15T15:44:58.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.992+0000 7fdfc39fe700 1 -- start start 2024-05-15T15:44:58.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.992+0000 7fdfc39fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 0x7fdfbc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.992+0000 7fdfc39fe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc1984c0 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.992+0000 7fdfc179a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 0x7fdfbc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.992+0000 7fdfc179a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 0x7fdfbc197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58314/0 (socket says 172.21.15.49:58314) 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.992+0000 7fdfc179a700 1 -- 172.21.15.49:0/3195635056 learned_addr learned my addr 172.21.15.49:0/3195635056 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.992+0000 7fdfc179a700 1 -- 172.21.15.49:0/3195635056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb80095f0 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.993+0000 7fdfc179a700 1 --2- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 0x7fdfbc197f80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009ed0 tx=0x7fdfb8004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.993+0000 7fdfb27fc700 1 -- 172.21.15.49:0/3195635056 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfb801d070 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.993+0000 7fdfb27fc700 1 -- 172.21.15.49:0/3195635056 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfb800be20 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.993+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc1986c0 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.993+0000 7fdfb27fc700 1 -- 172.21.15.49:0/3195635056 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfb800f460 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.993+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc198b60 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.994+0000 7fdfb27fc700 1 -- 172.21.15.49:0/3195635056 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fdfb8022800 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.994+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc191970 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.994+0000 7fdfb27fc700 1 --2- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdfa8038820 0x7fdfa803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:44:58.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.994+0000 7fdfb27fc700 1 -- 172.21.15.49:0/3195635056 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdfb804cbb0 con 0x7fdfbc0ff1d0 2024-05-15T15:44:58.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.994+0000 7fdfc0f99700 1 --2- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdfa8038820 0x7fdfa803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:44:58.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:58.995+0000 7fdfc0f99700 1 --2- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdfa8038820 0x7fdfa803ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006fd0 tx=0x7fdfac006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:44:59.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:59.000+0000 7fdfb27fc700 1 -- 172.21.15.49:0/3195635056 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdfb802ba00 con 0x7fdfbc0ff1d0 2024-05-15T15:44:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:44:59.211+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 --> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfbc0613e0 con 0x7fdfa8038820 2024-05-15T15:44:59.439 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:44:59 smithi049 ceph-mon[28399]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:45:00.652 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:00 smithi049 ceph-mon[28399]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:00.653 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:00 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:00.653 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:00 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:00.653 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:00 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:00.653 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:00 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:00.653 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:00 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:00.653 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:00 smithi049 ceph-mon[28399]: Deploying daemon node-exporter.smithi049 on smithi049 2024-05-15T15:45:00.653 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:00 smithi049 ceph-mon[28399]: Deploying cephadm binary to smithi144 2024-05-15T15:45:01.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.785+0000 7fdfb27fc700 1 -- 172.21.15.49:0/3195635056 <== mgr.14162 v2:172.21.15.49:6800/1508146783 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fdfbc0613e0 con 0x7fdfa8038820 2024-05-15T15:45:01.789 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdfa8038820 msgr2=0x7fdfa803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 --2- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdfa8038820 0x7fdfa803ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006fd0 tx=0x7fdfac006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 msgr2=0x7fdfbc197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 --2- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 0x7fdfbc197f80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009ed0 tx=0x7fdfb8004690 comp rx=0 tx=0).stop 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 shutdown_connections 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 --2- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdfa8038820 0x7fdfa803ace0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 --2- 172.21.15.49:0/3195635056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0ff1d0 0x7fdfbc197f80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 >> 172.21.15.49:0/3195635056 conn(0x7fdfbc0fadf0 msgr2=0x7fdfbc0fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 shutdown_connections 2024-05-15T15:45:01.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:01.788+0000 7fdfc39fe700 1 -- 172.21.15.49:0/3195635056 wait complete. 2024-05-15T15:45:03.173 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:02 smithi049 ceph-mon[28399]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:03.174 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:02 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:03.174 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:02 smithi049 ceph-mon[28399]: Added host smithi144 2024-05-15T15:45:03.198 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-15T15:45:03.636 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:45:04.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:04 smithi049 ceph-mon[28399]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:04.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:04 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:04.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:04 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:04.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:04 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:04.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:04 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:04.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:04 smithi049 ceph-mon[28399]: Deploying daemon alertmanager.smithi049 on smithi049 2024-05-15T15:45:04.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:04 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:05.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.138+0000 7f434e609700 1 -- 172.21.15.49:0/1630426572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 msgr2=0x7f43480ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:05.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.138+0000 7f434e609700 1 --2- 172.21.15.49:0/1630426572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 0x7f43480ffc50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4338009910 tx=0x7f4338009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.138+0000 7f434e609700 1 -- 172.21.15.49:0/1630426572 shutdown_connections 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.138+0000 7f434e609700 1 --2- 172.21.15.49:0/1630426572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 0x7f43480ffc50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.138+0000 7f434e609700 1 -- 172.21.15.49:0/1630426572 >> 172.21.15.49:0/1630426572 conn(0x7f43480fadd0 msgr2=0x7f43480fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.138+0000 7f434e609700 1 -- 172.21.15.49:0/1630426572 shutdown_connections 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.138+0000 7f434e609700 1 -- 172.21.15.49:0/1630426572 wait complete. 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.139+0000 7f434e609700 1 Processor -- start 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.139+0000 7f434e609700 1 -- start start 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.139+0000 7f434e609700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 0x7f43480693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.139+0000 7f434e609700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480698f0 con 0x7f43480ff830 2024-05-15T15:45:05.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.139+0000 7f4347fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 0x7f43480693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:05.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.139+0000 7f4347fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 0x7f43480693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55676/0 (socket says 172.21.15.49:55676) 2024-05-15T15:45:05.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.139+0000 7f4347fff700 1 -- 172.21.15.49:0/1589552153 learned_addr learned my addr 172.21.15.49:0/1589552153 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:45:05.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.139+0000 7f4347fff700 1 -- 172.21.15.49:0/1589552153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380095f0 con 0x7f43480ff830 2024-05-15T15:45:05.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.140+0000 7f4347fff700 1 --2- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 0x7f43480693b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4338006010 tx=0x7f43380050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:05.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.140+0000 7f4345ffb700 1 -- 172.21.15.49:0/1589552153 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f433801d070 con 0x7f43480ff830 2024-05-15T15:45:05.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.140+0000 7f4345ffb700 1 -- 172.21.15.49:0/1589552153 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f433800be20 con 0x7f43480ff830 2024-05-15T15:45:05.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.140+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4348198b20 con 0x7f43480ff830 2024-05-15T15:45:05.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.140+0000 7f4345ffb700 1 -- 172.21.15.49:0/1589552153 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4338022de0 con 0x7f43480ff830 2024-05-15T15:45:05.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.140+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4348198f60 con 0x7f43480ff830 2024-05-15T15:45:05.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.141+0000 7f4345ffb700 1 -- 172.21.15.49:0/1589552153 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f433800f460 con 0x7f43480ff830 2024-05-15T15:45:05.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.141+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434818e660 con 0x7f43480ff830 2024-05-15T15:45:05.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.141+0000 7f4345ffb700 1 --2- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f4334038820 0x7f433403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:05.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.141+0000 7f4345ffb700 1 -- 172.21.15.49:0/1589552153 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f433804c9a0 con 0x7f43480ff830 2024-05-15T15:45:05.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.141+0000 7f433ffff700 1 --2- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f4334038820 0x7f433403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:05.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.142+0000 7f433ffff700 1 --2- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f4334038820 0x7f433403ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4330006fd0 tx=0x7f4330006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:05.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.146+0000 7f4345ffb700 1 -- 172.21.15.49:0/1589552153 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4338020550 con 0x7f43480ff830 2024-05-15T15:45:05.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.352+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 --> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f434802ce40 con 0x7f4334038820 2024-05-15T15:45:05.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.353+0000 7f4345ffb700 1 -- 172.21.15.49:0/1589552153 <== mgr.14162 v2:172.21.15.49:6800/1508146783 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f434802ce40 con 0x7f4334038820 2024-05-15T15:45:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:45:05.356 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-05-15T15:45:05.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.355+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f4334038820 msgr2=0x7f433403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:05.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.355+0000 7f434e609700 1 --2- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f4334038820 0x7f433403ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4330006fd0 tx=0x7f4330006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:05.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.355+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 msgr2=0x7f43480693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:05.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.355+0000 7f434e609700 1 --2- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 0x7f43480693b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4338006010 tx=0x7f43380050b0 comp rx=0 tx=0).stop 2024-05-15T15:45:05.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.356+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 shutdown_connections 2024-05-15T15:45:05.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.356+0000 7f434e609700 1 --2- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f4334038820 0x7f433403ace0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:05.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.356+0000 7f434e609700 1 --2- 172.21.15.49:0/1589552153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43480ff830 0x7f43480693b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:05.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.356+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 >> 172.21.15.49:0/1589552153 conn(0x7f43480fadd0 msgr2=0x7f43480fb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:05.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.356+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 shutdown_connections 2024-05-15T15:45:05.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:05.356+0000 7f434e609700 1 -- 172.21.15.49:0/1589552153 wait complete. 2024-05-15T15:45:06.541 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-15T15:45:06.542 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-15T15:45:06.749 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:45:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:06 smithi049 ceph-mon[28399]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:06 smithi049 ceph-mon[28399]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T15:45:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:08 smithi049 ceph-mon[28399]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.676+0000 7fdb292ed700 1 -- 172.21.15.49:0/921977829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 msgr2=0x7fdb240ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.676+0000 7fdb292ed700 1 --2- 172.21.15.49:0/921977829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 0x7fdb240ffc40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009910 tx=0x7fdb14009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.676+0000 7fdb292ed700 1 -- 172.21.15.49:0/921977829 shutdown_connections 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.676+0000 7fdb292ed700 1 --2- 172.21.15.49:0/921977829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 0x7fdb240ffc40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.676+0000 7fdb292ed700 1 -- 172.21.15.49:0/921977829 >> 172.21.15.49:0/921977829 conn(0x7fdb240fadc0 msgr2=0x7fdb240fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.676+0000 7fdb292ed700 1 -- 172.21.15.49:0/921977829 shutdown_connections 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.676+0000 7fdb292ed700 1 -- 172.21.15.49:0/921977829 wait complete. 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.677+0000 7fdb292ed700 1 Processor -- start 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.677+0000 7fdb292ed700 1 -- start start 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.677+0000 7fdb292ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 0x7fdb24195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.677+0000 7fdb292ed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb24196370 con 0x7fdb240ff820 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.677+0000 7fdb22ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 0x7fdb24195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.677+0000 7fdb22ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 0x7fdb24195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55690/0 (socket says 172.21.15.49:55690) 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.677+0000 7fdb22ffd700 1 -- 172.21.15.49:0/58006981 learned_addr learned my addr 172.21.15.49:0/58006981 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.677+0000 7fdb22ffd700 1 -- 172.21.15.49:0/58006981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb140095f0 con 0x7fdb240ff820 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.678+0000 7fdb22ffd700 1 --2- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 0x7fdb24195e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006010 tx=0x7fdb14009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.678+0000 7fdb1bfff700 1 -- 172.21.15.49:0/58006981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb1401d070 con 0x7fdb240ff820 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.678+0000 7fdb1bfff700 1 -- 172.21.15.49:0/58006981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb14005dc0 con 0x7fdb240ff820 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.678+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb24196570 con 0x7fdb240ff820 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.678+0000 7fdb1bfff700 1 -- 172.21.15.49:0/58006981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb14022e90 con 0x7fdb240ff820 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.678+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb24196a10 con 0x7fdb240ff820 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.679+0000 7fdb1bfff700 1 -- 172.21.15.49:0/58006981 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fdb1400f4f0 con 0x7fdb240ff820 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.679+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2418f910 con 0x7fdb240ff820 2024-05-15T15:45:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.679+0000 7fdb1bfff700 1 --2- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdb10038760 0x7fdb1003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:09.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.679+0000 7fdb1bfff700 1 -- 172.21.15.49:0/58006981 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdb1404ca30 con 0x7fdb240ff820 2024-05-15T15:45:09.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.679+0000 7fdb227fc700 1 --2- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdb10038760 0x7fdb1003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:09.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.679+0000 7fdb227fc700 1 --2- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdb10038760 0x7fdb1003ac20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c006fd0 tx=0x7fdb0c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:09.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.684+0000 7fdb1bfff700 1 -- 172.21.15.49:0/58006981 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdb14020d40 con 0x7fdb240ff820 2024-05-15T15:45:09.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:09.897+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdb240625d0 con 0x7fdb240ff820 2024-05-15T15:45:10.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.672+0000 7fdb1bfff700 1 -- 172.21.15.49:0/58006981 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdb14027030 con 0x7fdb240ff820 2024-05-15T15:45:10.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdb10038760 msgr2=0x7fdb1003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:10.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 --2- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdb10038760 0x7fdb1003ac20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c006fd0 tx=0x7fdb0c006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:10.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 msgr2=0x7fdb24195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:10.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 --2- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 0x7fdb24195e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006010 tx=0x7fdb14009f80 comp rx=0 tx=0).stop 2024-05-15T15:45:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 shutdown_connections 2024-05-15T15:45:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 --2- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fdb10038760 0x7fdb1003ac20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 --2- 172.21.15.49:0/58006981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb240ff820 0x7fdb24195e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 >> 172.21.15.49:0/58006981 conn(0x7fdb240fadc0 msgr2=0x7fdb24069180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 shutdown_connections 2024-05-15T15:45:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:45:10.675+0000 7fdb292ed700 1 -- 172.21.15.49:0/58006981 wait complete. 2024-05-15T15:45:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-05-15T15:45:10.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:10 smithi049 ceph-mon[28399]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:10.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:10 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/58006981' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-15T15:45:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:11 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/58006981' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-15T15:45:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:11 smithi049 ceph-mon[28399]: osdmap e4: 0 total, 0 up, 0 in 2024-05-15T15:45:12.203 INFO:tasks.cephadm:Adding mon.smithi049 on smithi049 2024-05-15T15:45:12.203 INFO:tasks.cephadm:Adding mon.smithi144 on smithi144 2024-05-15T15:45:12.203 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch apply mon '2;smithi049:172.21.15.49=smithi049;smithi144:172.21.15.144=smithi144' 2024-05-15T15:45:12.406 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:12.452 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:12.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:12 smithi049 ceph-mon[28399]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:13.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.968+0000 7feb78820700 1 -- 172.21.15.144:0/3159073440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 msgr2=0x7feb740ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:13.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.968+0000 7feb78820700 1 --2- 172.21.15.144:0/3159073440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 0x7feb740ffc40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb64009910 tx=0x7feb64009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:13.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.968+0000 7feb78820700 1 -- 172.21.15.144:0/3159073440 shutdown_connections 2024-05-15T15:45:13.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.968+0000 7feb78820700 1 --2- 172.21.15.144:0/3159073440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 0x7feb740ffc40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:13.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.968+0000 7feb78820700 1 -- 172.21.15.144:0/3159073440 >> 172.21.15.144:0/3159073440 conn(0x7feb740fadc0 msgr2=0x7feb740fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:13.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.968+0000 7feb78820700 1 -- 172.21.15.144:0/3159073440 shutdown_connections 2024-05-15T15:45:13.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.968+0000 7feb78820700 1 -- 172.21.15.144:0/3159073440 wait complete. 2024-05-15T15:45:13.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.969+0000 7feb78820700 1 Processor -- start 2024-05-15T15:45:13.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.969+0000 7feb78820700 1 -- start start 2024-05-15T15:45:13.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.969+0000 7feb78820700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 0x7feb74069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:13.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.969+0000 7feb78820700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74069c70 con 0x7feb740ff820 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb7259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 0x7feb74069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb7259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 0x7feb74069730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:52532/0 (socket says 172.21.15.144:52532) 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb7259c700 1 -- 172.21.15.144:0/2581610357 learned_addr learned my addr 172.21.15.144:0/2581610357 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb7259c700 1 -- 172.21.15.144:0/2581610357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb640095f0 con 0x7feb740ff820 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb7259c700 1 --2- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 0x7feb74069730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feb64006010 tx=0x7feb64009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb637fe700 1 -- 172.21.15.144:0/2581610357 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb6401d070 con 0x7feb740ff820 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb637fe700 1 -- 172.21.15.144:0/2581610357 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb64005dc0 con 0x7feb740ff820 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb637fe700 1 -- 172.21.15.144:0/2581610357 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb64022e90 con 0x7feb740ff820 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb74198b10 con 0x7feb740ff820 2024-05-15T15:45:13.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.970+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb74198f50 con 0x7feb740ff820 2024-05-15T15:45:13.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.971+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7418e650 con 0x7feb740ff820 2024-05-15T15:45:13.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.971+0000 7feb637fe700 1 -- 172.21.15.144:0/2581610357 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7feb64022470 con 0x7feb740ff820 2024-05-15T15:45:13.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.972+0000 7feb637fe700 1 --2- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7feb5c0387c0 0x7feb5c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:13.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.972+0000 7feb637fe700 1 -- 172.21.15.144:0/2581610357 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feb6404c890 con 0x7feb740ff820 2024-05-15T15:45:13.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.972+0000 7feb71d9b700 1 --2- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7feb5c0387c0 0x7feb5c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:13.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.972+0000 7feb71d9b700 1 --2- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7feb5c0387c0 0x7feb5c03ac80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feb68006fd0 tx=0x7feb68006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:13.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:13.977+0000 7feb637fe700 1 -- 172.21.15.144:0/2581610357 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feb6400f4b0 con 0x7feb740ff820 2024-05-15T15:45:14.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.179+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 --> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi049:172.21.15.49=smithi049;smithi144:172.21.15.144=smithi144", "target": ["mon-mgr", ""]}) v1 -- 0x7feb7402cec0 con 0x7feb5c0387c0 2024-05-15T15:45:14.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.185+0000 7feb637fe700 1 -- 172.21.15.144:0/2581610357 <== mgr.14162 v2:172.21.15.49:6800/1508146783 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7feb7402cec0 con 0x7feb5c0387c0 2024-05-15T15:45:14.187 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mon update... 2024-05-15T15:45:14.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7feb5c0387c0 msgr2=0x7feb5c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:14.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 --2- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7feb5c0387c0 0x7feb5c03ac80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feb68006fd0 tx=0x7feb68006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:14.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 msgr2=0x7feb74069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:14.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 --2- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 0x7feb74069730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feb64006010 tx=0x7feb64009f80 comp rx=0 tx=0).stop 2024-05-15T15:45:14.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 shutdown_connections 2024-05-15T15:45:14.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 --2- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7feb5c0387c0 0x7feb5c03ac80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:14.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 --2- 172.21.15.144:0/2581610357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb740ff820 0x7feb74069730 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:14.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 >> 172.21.15.144:0/2581610357 conn(0x7feb740fadc0 msgr2=0x7feb740fb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:14.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 shutdown_connections 2024-05-15T15:45:14.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:14.187+0000 7feb78820700 1 -- 172.21.15.144:0/2581610357 wait complete. 2024-05-15T15:45:14.734 DEBUG:teuthology.orchestra.run.smithi144:mon.smithi144> sudo journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi144.service 2024-05-15T15:45:14.737 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:14.737 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:14.965 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:15.012 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:15.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:15.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi049:172.21.15.49=smithi049;smithi144:172.21.15.144=smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:45:15.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: Saving service mon spec with placement smithi049:172.21.15.49=smithi049;smithi144:172.21.15.144=smithi144;count:2 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:15.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:15 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:16.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.321+0000 7fa306cf5700 1 -- 172.21.15.144:0/541101559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 msgr2=0x7fa300100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.321+0000 7fa306cf5700 1 --2- 172.21.15.144:0/541101559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 0x7fa300100d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009910 tx=0x7fa2f4009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.321+0000 7fa306cf5700 1 -- 172.21.15.144:0/541101559 shutdown_connections 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.321+0000 7fa306cf5700 1 --2- 172.21.15.144:0/541101559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 0x7fa300100d50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.321+0000 7fa306cf5700 1 -- 172.21.15.144:0/541101559 >> 172.21.15.144:0/541101559 conn(0x7fa3000fbef0 msgr2=0x7fa3000fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.321+0000 7fa306cf5700 1 -- 172.21.15.144:0/541101559 shutdown_connections 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.322+0000 7fa306cf5700 1 -- 172.21.15.144:0/541101559 wait complete. 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.322+0000 7fa306cf5700 1 Processor -- start 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.322+0000 7fa306cf5700 1 -- start start 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.322+0000 7fa306cf5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 0x7fa300195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.322+0000 7fa306cf5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3001962f0 con 0x7fa300100930 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.322+0000 7fa304a91700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 0x7fa300195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:16.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.322+0000 7fa304a91700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 0x7fa300195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:52538/0 (socket says 172.21.15.144:52538) 2024-05-15T15:45:16.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.322+0000 7fa304a91700 1 -- 172.21.15.144:0/11941287 learned_addr learned my addr 172.21.15.144:0/11941287 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:16.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.323+0000 7fa304a91700 1 -- 172.21.15.144:0/11941287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f40095f0 con 0x7fa300100930 2024-05-15T15:45:16.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.323+0000 7fa304a91700 1 --2- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 0x7fa300195db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006010 tx=0x7fa2f40050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:16.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.323+0000 7fa2fdffb700 1 -- 172.21.15.144:0/11941287 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2f401d070 con 0x7fa300100930 2024-05-15T15:45:16.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.323+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3001964f0 con 0x7fa300100930 2024-05-15T15:45:16.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.323+0000 7fa2fdffb700 1 -- 172.21.15.144:0/11941287 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2f400be20 con 0x7fa300100930 2024-05-15T15:45:16.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.323+0000 7fa2fdffb700 1 -- 172.21.15.144:0/11941287 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2f4022de0 con 0x7fa300100930 2024-05-15T15:45:16.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.323+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa300196990 con 0x7fa300100930 2024-05-15T15:45:16.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.324+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30018f760 con 0x7fa300100930 2024-05-15T15:45:16.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.324+0000 7fa2fdffb700 1 -- 172.21.15.144:0/11941287 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fa2f40226c0 con 0x7fa300100930 2024-05-15T15:45:16.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.324+0000 7fa2fdffb700 1 --2- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fa2f00387b0 0x7fa2f003ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:16.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.325+0000 7fa2fdffb700 1 -- 172.21.15.144:0/11941287 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa2f404cc50 con 0x7fa300100930 2024-05-15T15:45:16.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.325+0000 7fa2fffff700 1 --2- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fa2f00387b0 0x7fa2f003ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:16.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.325+0000 7fa2fffff700 1 --2- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fa2f00387b0 0x7fa2f003ac70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec006fd0 tx=0x7fa2ec006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:16.332 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.330+0000 7fa2fdffb700 1 -- 172.21.15.144:0/11941287 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa2f402ba00 con 0x7fa300100930 2024-05-15T15:45:16.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:16 smithi049 ceph-mon[28399]: Regenerating cephadm self-signed grafana TLS certificates 2024-05-15T15:45:16.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:16 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-15T15:45:16.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:16 smithi049 ceph-mon[28399]: Deploying daemon grafana.smithi049 on smithi049 2024-05-15T15:45:16.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.600+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa30002cda0 con 0x7fa300100930 2024-05-15T15:45:16.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.601+0000 7fa2fdffb700 1 -- 172.21.15.144:0/11941287 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fa2f402b3b0 con 0x7fa300100930 2024-05-15T15:45:16.604 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:16.604 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:16.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.603+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fa2f00387b0 msgr2=0x7fa2f003ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:16.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.603+0000 7fa306cf5700 1 --2- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fa2f00387b0 0x7fa2f003ac70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec006fd0 tx=0x7fa2ec006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:16.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.603+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 msgr2=0x7fa300195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:16.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.603+0000 7fa306cf5700 1 --2- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 0x7fa300195db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006010 tx=0x7fa2f40050b0 comp rx=0 tx=0).stop 2024-05-15T15:45:16.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.604+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 shutdown_connections 2024-05-15T15:45:16.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.604+0000 7fa306cf5700 1 --2- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fa2f00387b0 0x7fa2f003ac70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:16.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.604+0000 7fa306cf5700 1 --2- 172.21.15.144:0/11941287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa300100930 0x7fa300195db0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:16.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.604+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 >> 172.21.15.144:0/11941287 conn(0x7fa3000fbef0 msgr2=0x7fa3000fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:16.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.604+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 shutdown_connections 2024-05-15T15:45:16.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:16.604+0000 7fa306cf5700 1 -- 172.21.15.144:0/11941287 wait complete. 2024-05-15T15:45:16.607 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:17.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:17 smithi049 ceph-mon[28399]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:17.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:17 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/11941287' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:18.146 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:18.146 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:18.350 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:18.398 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:19.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:19 smithi049 ceph-mon[28399]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:19.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.782+0000 7f567c22c700 1 -- 172.21.15.144:0/326373322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 msgr2=0x7f56740ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:19.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.782+0000 7f567c22c700 1 --2- 172.21.15.144:0/326373322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 0x7f56740ffc50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5664009910 tx=0x7f5664009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:19.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.782+0000 7f567c22c700 1 -- 172.21.15.144:0/326373322 shutdown_connections 2024-05-15T15:45:19.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.782+0000 7f567c22c700 1 --2- 172.21.15.144:0/326373322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 0x7f56740ffc50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:19.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.782+0000 7f567c22c700 1 -- 172.21.15.144:0/326373322 >> 172.21.15.144:0/326373322 conn(0x7f56740fadd0 msgr2=0x7f56740fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:19.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.782+0000 7f567c22c700 1 -- 172.21.15.144:0/326373322 shutdown_connections 2024-05-15T15:45:19.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.782+0000 7f567c22c700 1 -- 172.21.15.144:0/326373322 wait complete. 2024-05-15T15:45:19.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.783+0000 7f567c22c700 1 Processor -- start 2024-05-15T15:45:19.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.783+0000 7f567c22c700 1 -- start start 2024-05-15T15:45:19.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.783+0000 7f567c22c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 0x7f5674194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:19.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.783+0000 7f567c22c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5674195210 con 0x7f56740ff830 2024-05-15T15:45:19.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.783+0000 7f5679fc8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 0x7f5674194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:19.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.783+0000 7f5679fc8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 0x7f5674194cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:52554/0 (socket says 172.21.15.144:52554) 2024-05-15T15:45:19.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.783+0000 7f5679fc8700 1 -- 172.21.15.144:0/3418882869 learned_addr learned my addr 172.21.15.144:0/3418882869 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:19.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.783+0000 7f5679fc8700 1 -- 172.21.15.144:0/3418882869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640095f0 con 0x7f56740ff830 2024-05-15T15:45:19.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.784+0000 7f5679fc8700 1 --2- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 0x7f5674194cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:19.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.784+0000 7f566affd700 1 -- 172.21.15.144:0/3418882869 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f566401d070 con 0x7f56740ff830 2024-05-15T15:45:19.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.784+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5674195410 con 0x7f56740ff830 2024-05-15T15:45:19.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.784+0000 7f566affd700 1 -- 172.21.15.144:0/3418882869 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5664005dc0 con 0x7f56740ff830 2024-05-15T15:45:19.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.784+0000 7f566affd700 1 -- 172.21.15.144:0/3418882869 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5664022e90 con 0x7f56740ff830 2024-05-15T15:45:19.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.784+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56741958b0 con 0x7f56740ff830 2024-05-15T15:45:19.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.785+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56740625d0 con 0x7f56740ff830 2024-05-15T15:45:19.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.785+0000 7f566affd700 1 -- 172.21.15.144:0/3418882869 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f566400b960 con 0x7f56740ff830 2024-05-15T15:45:19.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.785+0000 7f566affd700 1 --2- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f566003a970 0x7f566003ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:19.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.785+0000 7f566affd700 1 -- 172.21.15.144:0/3418882869 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f566404c800 con 0x7f56740ff830 2024-05-15T15:45:19.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.785+0000 7f56797c7700 1 --2- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f566003a970 0x7f566003ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:19.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.786+0000 7f56797c7700 1 --2- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f566003a970 0x7f566003ce30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5670006fd0 tx=0x7f5670006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:19.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:19.791+0000 7f566affd700 1 -- 172.21.15.144:0/3418882869 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f566400f4f0 con 0x7f56740ff830 2024-05-15T15:45:20.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.061+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5674105670 con 0x7f56740ff830 2024-05-15T15:45:20.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.062+0000 7f566affd700 1 -- 172.21.15.144:0/3418882869 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f5664027030 con 0x7f56740ff830 2024-05-15T15:45:20.064 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:20.065 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f566003a970 msgr2=0x7f566003ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 --2- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f566003a970 0x7f566003ce30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5670006fd0 tx=0x7f5670006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 msgr2=0x7f5674194cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 --2- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 0x7f5674194cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664009f80 comp rx=0 tx=0).stop 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 shutdown_connections 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 --2- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f566003a970 0x7f566003ce30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 --2- 172.21.15.144:0/3418882869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56740ff830 0x7f5674194cd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 >> 172.21.15.144:0/3418882869 conn(0x7f56740fadd0 msgr2=0x7f56740fb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 shutdown_connections 2024-05-15T15:45:20.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:20.064+0000 7f567c22c700 1 -- 172.21.15.144:0/3418882869 wait complete. 2024-05-15T15:45:20.068 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:20.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:20 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/3418882869' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:21.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:21 smithi049 ceph-mon[28399]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:21.619 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:21.619 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:21.822 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:23.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.285+0000 7f49e1ba1700 1 -- 172.21.15.144:0/3546219697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 msgr2=0x7f49dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:23.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.285+0000 7f49e1ba1700 1 --2- 172.21.15.144:0/3546219697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 0x7f49dc101200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009910 tx=0x7f49d0009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:23.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.285+0000 7f49e1ba1700 1 -- 172.21.15.144:0/3546219697 shutdown_connections 2024-05-15T15:45:23.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.285+0000 7f49e1ba1700 1 --2- 172.21.15.144:0/3546219697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 0x7f49dc101200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:23.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.285+0000 7f49e1ba1700 1 -- 172.21.15.144:0/3546219697 >> 172.21.15.144:0/3546219697 conn(0x7f49dc0fa550 msgr2=0x7f49dc0fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:23.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.286+0000 7f49e1ba1700 1 -- 172.21.15.144:0/3546219697 shutdown_connections 2024-05-15T15:45:23.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.286+0000 7f49e1ba1700 1 -- 172.21.15.144:0/3546219697 wait complete. 2024-05-15T15:45:23.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.286+0000 7f49e1ba1700 1 Processor -- start 2024-05-15T15:45:23.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.286+0000 7f49e1ba1700 1 -- start start 2024-05-15T15:45:23.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.286+0000 7f49e1ba1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 0x7f49dc195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:23.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.286+0000 7f49e1ba1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc196340 con 0x7f49dc100de0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49db7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 0x7f49dc195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49db7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 0x7f49dc195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:50346/0 (socket says 172.21.15.144:50346) 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49db7fe700 1 -- 172.21.15.144:0/2989250585 learned_addr learned my addr 172.21.15.144:0/2989250585 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49db7fe700 1 -- 172.21.15.144:0/2989250585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d00095f0 con 0x7f49dc100de0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49db7fe700 1 --2- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 0x7f49dc195e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49d0006010 tx=0x7f49d00050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49d8ff9700 1 -- 172.21.15.144:0/2989250585 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49d001d070 con 0x7f49dc100de0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49d8ff9700 1 -- 172.21.15.144:0/2989250585 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49d000be20 con 0x7f49dc100de0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc196540 con 0x7f49dc100de0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49d8ff9700 1 -- 172.21.15.144:0/2989250585 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49d0022de0 con 0x7f49dc100de0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.287+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc1969e0 con 0x7f49dc100de0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.288+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc18f920 con 0x7f49dc100de0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.288+0000 7f49d8ff9700 1 -- 172.21.15.144:0/2989250585 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f49d00226c0 con 0x7f49dc100de0 2024-05-15T15:45:23.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.289+0000 7f49d8ff9700 1 --2- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f49c40387b0 0x7f49c403ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:23.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.289+0000 7f49d8ff9700 1 -- 172.21.15.144:0/2989250585 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f49d004cbc0 con 0x7f49dc100de0 2024-05-15T15:45:23.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.289+0000 7f49daffd700 1 --2- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f49c40387b0 0x7f49c403ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:23.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.289+0000 7f49daffd700 1 --2- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f49c40387b0 0x7f49c403ac70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006fd0 tx=0x7f49cc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:23.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.294+0000 7f49d8ff9700 1 -- 172.21.15.144:0/2989250585 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f49d002ba00 con 0x7f49dc100de0 2024-05-15T15:45:23.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:23 smithi049 ceph-mon[28399]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:23.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.563+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f49dc0625d0 con 0x7f49dc100de0 2024-05-15T15:45:23.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.563+0000 7f49d8ff9700 1 -- 172.21.15.144:0/2989250585 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f49d0027030 con 0x7f49dc100de0 2024-05-15T15:45:23.565 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:23.565 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f49c40387b0 msgr2=0x7f49c403ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 --2- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f49c40387b0 0x7f49c403ac70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006fd0 tx=0x7f49cc006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 msgr2=0x7f49dc195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 --2- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 0x7f49dc195e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49d0006010 tx=0x7f49d00050b0 comp rx=0 tx=0).stop 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 shutdown_connections 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 --2- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f49c40387b0 0x7f49c403ac70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 --2- 172.21.15.144:0/2989250585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49dc100de0 0x7f49dc195e00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 >> 172.21.15.144:0/2989250585 conn(0x7f49dc0fa550 msgr2=0x7f49dc0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 shutdown_connections 2024-05-15T15:45:23.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:23.566+0000 7f49e1ba1700 1 -- 172.21.15.144:0/2989250585 wait complete. 2024-05-15T15:45:23.568 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:24.356 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:24 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/2989250585' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:25.209 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:25.209 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:25.412 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:25.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:25 smithi049 ceph-mon[28399]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:25.457 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:26.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.795+0000 7fe6bb945700 1 -- 172.21.15.144:0/300889391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 msgr2=0x7fe6b41015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:26.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.795+0000 7fe6bb945700 1 --2- 172.21.15.144:0/300889391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 0x7fe6b41015e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009910 tx=0x7fe6a4009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:26.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.795+0000 7fe6bb945700 1 -- 172.21.15.144:0/300889391 shutdown_connections 2024-05-15T15:45:26.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.795+0000 7fe6bb945700 1 --2- 172.21.15.144:0/300889391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 0x7fe6b41015e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:26.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.795+0000 7fe6bb945700 1 -- 172.21.15.144:0/300889391 >> 172.21.15.144:0/300889391 conn(0x7fe6b40fadd0 msgr2=0x7fe6b40fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:26.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.795+0000 7fe6bb945700 1 -- 172.21.15.144:0/300889391 shutdown_connections 2024-05-15T15:45:26.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.795+0000 7fe6bb945700 1 -- 172.21.15.144:0/300889391 wait complete. 2024-05-15T15:45:26.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.796+0000 7fe6bb945700 1 Processor -- start 2024-05-15T15:45:26.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.796+0000 7fe6bb945700 1 -- start start 2024-05-15T15:45:26.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.796+0000 7fe6bb945700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 0x7fe6b419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:26.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.796+0000 7fe6bb945700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b419c8e0 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.796+0000 7fe6b96e1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 0x7fe6b419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:26.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.796+0000 7fe6b96e1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 0x7fe6b419c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:50364/0 (socket says 172.21.15.144:50364) 2024-05-15T15:45:26.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.796+0000 7fe6b96e1700 1 -- 172.21.15.144:0/4178710756 learned_addr learned my addr 172.21.15.144:0/4178710756 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:26.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.797+0000 7fe6b96e1700 1 -- 172.21.15.144:0/4178710756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40095f0 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.797+0000 7fe6b96e1700 1 --2- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 0x7fe6b419c3a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009ed0 tx=0x7fe6a4004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.797+0000 7fe6aa7fc700 1 -- 172.21.15.144:0/4178710756 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6a401d070 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.797+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b419cae0 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.797+0000 7fe6aa7fc700 1 -- 172.21.15.144:0/4178710756 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6a400be20 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.797+0000 7fe6aa7fc700 1 -- 172.21.15.144:0/4178710756 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6a400f460 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.797+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b419cf80 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.798+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b41964a0 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.798+0000 7fe6aa7fc700 1 -- 172.21.15.144:0/4178710756 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fe6a4022800 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.798+0000 7fe6aa7fc700 1 --2- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fe6a00387d0 0x7fe6a003ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.798+0000 7fe6aa7fc700 1 -- 172.21.15.144:0/4178710756 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe6a404cc80 con 0x7fe6b40ff1b0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.799+0000 7fe6b8ee0700 1 --2- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fe6a00387d0 0x7fe6a003ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.799+0000 7fe6b8ee0700 1 --2- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fe6a00387d0 0x7fe6a003ac90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006fd0 tx=0x7fe6b0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:26.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:26.809+0000 7fe6aa7fc700 1 -- 172.21.15.144:0/4178710756 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe6a4053080 con 0x7fe6b40ff1b0 2024-05-15T15:45:27.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.080+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe6b40625d0 con 0x7fe6b40ff1b0 2024-05-15T15:45:27.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.080+0000 7fe6aa7fc700 1 -- 172.21.15.144:0/4178710756 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fe6a402bba0 con 0x7fe6b40ff1b0 2024-05-15T15:45:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:27.083 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:27.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.082+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fe6a00387d0 msgr2=0x7fe6a003ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.082+0000 7fe6bb945700 1 --2- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fe6a00387d0 0x7fe6a003ac90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006fd0 tx=0x7fe6b0006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.082+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 msgr2=0x7fe6b419c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.082+0000 7fe6bb945700 1 --2- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 0x7fe6b419c3a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009ed0 tx=0x7fe6a4004690 comp rx=0 tx=0).stop 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.082+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 shutdown_connections 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.082+0000 7fe6bb945700 1 --2- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fe6a00387d0 0x7fe6a003ac90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.082+0000 7fe6bb945700 1 --2- 172.21.15.144:0/4178710756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6b40ff1b0 0x7fe6b419c3a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.082+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 >> 172.21.15.144:0/4178710756 conn(0x7fe6b40fadd0 msgr2=0x7fe6b40fb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.083+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 shutdown_connections 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:27.083+0000 7fe6bb945700 1 -- 172.21.15.144:0/4178710756 wait complete. 2024-05-15T15:45:27.085 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:27.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:27 smithi049 ceph-mon[28399]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:27.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:27 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/4178710756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:28.584 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:28.584 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:28.784 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:28.830 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:29.354 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:29 smithi049 ceph-mon[28399]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:30.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.285+0000 7f90cb8a8700 1 -- 172.21.15.144:0/1223200975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 msgr2=0x7f90c4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:30.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.285+0000 7f90cb8a8700 1 --2- 172.21.15.144:0/1223200975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 0x7f90c4100d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009910 tx=0x7f90b8009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:30.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.285+0000 7f90cb8a8700 1 -- 172.21.15.144:0/1223200975 shutdown_connections 2024-05-15T15:45:30.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.285+0000 7f90cb8a8700 1 --2- 172.21.15.144:0/1223200975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 0x7f90c4100d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:30.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.285+0000 7f90cb8a8700 1 -- 172.21.15.144:0/1223200975 >> 172.21.15.144:0/1223200975 conn(0x7f90c40fbef0 msgr2=0x7f90c40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.285+0000 7f90cb8a8700 1 -- 172.21.15.144:0/1223200975 shutdown_connections 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.285+0000 7f90cb8a8700 1 -- 172.21.15.144:0/1223200975 wait complete. 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.286+0000 7f90cb8a8700 1 Processor -- start 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.286+0000 7f90cb8a8700 1 -- start start 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.286+0000 7f90cb8a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 0x7f90c4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.286+0000 7f90cb8a8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c4198540 con 0x7f90c4100930 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.286+0000 7f90c9644700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 0x7f90c4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.286+0000 7f90c9644700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 0x7f90c4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:50380/0 (socket says 172.21.15.144:50380) 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.286+0000 7f90c9644700 1 -- 172.21.15.144:0/2675075870 learned_addr learned my addr 172.21.15.144:0/2675075870 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.286+0000 7f90c9644700 1 -- 172.21.15.144:0/2675075870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b80095f0 con 0x7f90c4100930 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.287+0000 7f90c9644700 1 --2- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 0x7f90c4198000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f90b8006010 tx=0x7f90b8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.287+0000 7f90b67fc700 1 -- 172.21.15.144:0/2675075870 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90b801d070 con 0x7f90c4100930 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.287+0000 7f90b67fc700 1 -- 172.21.15.144:0/2675075870 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90b8005dc0 con 0x7f90c4100930 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.287+0000 7f90b67fc700 1 -- 172.21.15.144:0/2675075870 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90b8022e90 con 0x7f90c4100930 2024-05-15T15:45:30.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.287+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c4198740 con 0x7f90c4100930 2024-05-15T15:45:30.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.287+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c4198be0 con 0x7f90c4100930 2024-05-15T15:45:30.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.288+0000 7f90b67fc700 1 -- 172.21.15.144:0/2675075870 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f90b8022470 con 0x7f90c4100930 2024-05-15T15:45:30.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.288+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c4191b60 con 0x7f90c4100930 2024-05-15T15:45:30.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.288+0000 7f90b67fc700 1 --2- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f90b0038760 0x7f90b003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:30.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.288+0000 7f90b67fc700 1 -- 172.21.15.144:0/2675075870 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f90b804c7f0 con 0x7f90c4100930 2024-05-15T15:45:30.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.288+0000 7f90c8e43700 1 --2- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f90b0038760 0x7f90b003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:30.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.289+0000 7f90c8e43700 1 --2- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f90b0038760 0x7f90b003ac20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006fd0 tx=0x7f90c0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:30.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.293+0000 7f90b67fc700 1 -- 172.21.15.144:0/2675075870 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f90b800b960 con 0x7f90c4100930 2024-05-15T15:45:30.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.565+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f90c40625d0 con 0x7f90c4100930 2024-05-15T15:45:30.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.565+0000 7f90b67fc700 1 -- 172.21.15.144:0/2675075870 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f90c40625d0 con 0x7f90c4100930 2024-05-15T15:45:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:30.568 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f90b0038760 msgr2=0x7f90b003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 --2- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f90b0038760 0x7f90b003ac20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006fd0 tx=0x7f90c0006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 msgr2=0x7f90c4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 --2- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 0x7f90c4198000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f90b8006010 tx=0x7f90b8009f80 comp rx=0 tx=0).stop 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 shutdown_connections 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 --2- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f90b0038760 0x7f90b003ac20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 --2- 172.21.15.144:0/2675075870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c4100930 0x7f90c4198000 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 >> 172.21.15.144:0/2675075870 conn(0x7f90c40fbef0 msgr2=0x7f90c40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 shutdown_connections 2024-05-15T15:45:30.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:30.568+0000 7f90cb8a8700 1 -- 172.21.15.144:0/2675075870 wait complete. 2024-05-15T15:45:30.571 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:30.677 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:30.677 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:30.677 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:30.677 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:30.678 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:30.678 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:30.678 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:30.678 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:30.678 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:30.678 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:30 smithi049 ceph-mon[28399]: Deploying daemon prometheus.smithi049 on smithi049 2024-05-15T15:45:31.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:31 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/2675075870' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:32.139 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:32.140 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:32.342 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:32.389 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:32.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:32 smithi049 ceph-mon[28399]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.831+0000 7fb695a78700 1 -- 172.21.15.144:0/4104443434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 msgr2=0x7fb6901037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.831+0000 7fb695a78700 1 --2- 172.21.15.144:0/4104443434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 0x7fb6901037b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb68000b0a0 tx=0x7fb68000b3b0 comp rx=0 tx=0).stop 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.831+0000 7fb695a78700 1 -- 172.21.15.144:0/4104443434 shutdown_connections 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.831+0000 7fb695a78700 1 --2- 172.21.15.144:0/4104443434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 0x7fb6901037b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.831+0000 7fb695a78700 1 -- 172.21.15.144:0/4104443434 >> 172.21.15.144:0/4104443434 conn(0x7fb6900fadf0 msgr2=0x7fb6900fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.831+0000 7fb695a78700 1 -- 172.21.15.144:0/4104443434 shutdown_connections 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.831+0000 7fb695a78700 1 -- 172.21.15.144:0/4104443434 wait complete. 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.832+0000 7fb695a78700 1 Processor -- start 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.832+0000 7fb695a78700 1 -- start start 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.832+0000 7fb695a78700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 0x7fb690195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.832+0000 7fb695a78700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb690196370 con 0x7fb6901013c0 2024-05-15T15:45:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb68f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 0x7fb690195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb68f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 0x7fb690195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:45090/0 (socket says 172.21.15.144:45090) 2024-05-15T15:45:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb68f7fe700 1 -- 172.21.15.144:0/3914444749 learned_addr learned my addr 172.21.15.144:0/3914444749 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb68f7fe700 1 -- 172.21.15.144:0/3914444749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb680009d00 con 0x7fb6901013c0 2024-05-15T15:45:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb68f7fe700 1 --2- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 0x7fb690195e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb6800062a0 tx=0x7fb68000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb68cff9700 1 -- 172.21.15.144:0/3914444749 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb68000e040 con 0x7fb6901013c0 2024-05-15T15:45:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb690196570 con 0x7fb6901013c0 2024-05-15T15:45:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb68cff9700 1 -- 172.21.15.144:0/3914444749 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb680003cd0 con 0x7fb6901013c0 2024-05-15T15:45:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb68cff9700 1 -- 172.21.15.144:0/3914444749 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb680012450 con 0x7fb6901013c0 2024-05-15T15:45:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.833+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb690196a10 con 0x7fb6901013c0 2024-05-15T15:45:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.834+0000 7fb68cff9700 1 -- 172.21.15.144:0/3914444749 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fb6800126c0 con 0x7fb6901013c0 2024-05-15T15:45:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.834+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69018f760 con 0x7fb6901013c0 2024-05-15T15:45:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.835+0000 7fb68cff9700 1 --2- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb678038800 0x7fb67803acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:33.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.835+0000 7fb68cff9700 1 -- 172.21.15.144:0/3914444749 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb68004c820 con 0x7fb6901013c0 2024-05-15T15:45:33.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.835+0000 7fb68effd700 1 --2- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb678038800 0x7fb67803acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:33.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.835+0000 7fb68effd700 1 --2- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb678038800 0x7fb67803acc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb684006fd0 tx=0x7fb684006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:33.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:33.841+0000 7fb68cff9700 1 -- 172.21.15.144:0/3914444749 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb68001b360 con 0x7fb6901013c0 2024-05-15T15:45:34.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.111+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb6900625d0 con 0x7fb6901013c0 2024-05-15T15:45:34.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.111+0000 7fb68cff9700 1 -- 172.21.15.144:0/3914444749 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb6900625d0 con 0x7fb6901013c0 2024-05-15T15:45:34.113 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:34.114 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:34.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb678038800 msgr2=0x7fb67803acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:34.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 --2- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb678038800 0x7fb67803acc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb684006fd0 tx=0x7fb684006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:34.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 msgr2=0x7fb690195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:34.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 --2- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 0x7fb690195e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb6800062a0 tx=0x7fb68000bd40 comp rx=0 tx=0).stop 2024-05-15T15:45:34.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 shutdown_connections 2024-05-15T15:45:34.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 --2- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fb678038800 0x7fb67803acc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:34.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 --2- 172.21.15.144:0/3914444749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6901013c0 0x7fb690195e30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:34.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 >> 172.21.15.144:0/3914444749 conn(0x7fb6900fadf0 msgr2=0x7fb6900fb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:34.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 shutdown_connections 2024-05-15T15:45:34.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:34.113+0000 7fb695a78700 1 -- 172.21.15.144:0/3914444749 wait complete. 2024-05-15T15:45:34.116 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:34.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:34 smithi049 ceph-mon[28399]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:34.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:34 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/3914444749' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:34.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:34 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:35.707 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:35.708 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:35.911 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:35.958 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:36.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:36 smithi049 ceph-mon[28399]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:37.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.385+0000 7f6586f20700 1 -- 172.21.15.144:0/3900138857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 msgr2=0x7f65801037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:37.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.385+0000 7f6586f20700 1 --2- 172.21.15.144:0/3900138857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 0x7f65801037b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6578009910 tx=0x7f6578009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:37.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.385+0000 7f6586f20700 1 -- 172.21.15.144:0/3900138857 shutdown_connections 2024-05-15T15:45:37.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.385+0000 7f6586f20700 1 --2- 172.21.15.144:0/3900138857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 0x7f65801037b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.385+0000 7f6586f20700 1 -- 172.21.15.144:0/3900138857 >> 172.21.15.144:0/3900138857 conn(0x7f65800fadd0 msgr2=0x7f65800fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.385+0000 7f6586f20700 1 -- 172.21.15.144:0/3900138857 shutdown_connections 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.385+0000 7f6586f20700 1 -- 172.21.15.144:0/3900138857 wait complete. 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.386+0000 7f6586f20700 1 Processor -- start 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.386+0000 7f6586f20700 1 -- start start 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.386+0000 7f6586f20700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 0x7f6580195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.386+0000 7f6586f20700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65801962b0 con 0x7f65801013c0 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.386+0000 7f6584cbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 0x7f6580195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.386+0000 7f6584cbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 0x7f6580195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:45108/0 (socket says 172.21.15.144:45108) 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.386+0000 7f6584cbc700 1 -- 172.21.15.144:0/3444587491 learned_addr learned my addr 172.21.15.144:0/3444587491 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:37.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.386+0000 7f6584cbc700 1 -- 172.21.15.144:0/3444587491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65780095f0 con 0x7f65801013c0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.387+0000 7f6584cbc700 1 --2- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 0x7f6580195d70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6578009ed0 tx=0x7f6578004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.387+0000 7f657dffb700 1 -- 172.21.15.144:0/3444587491 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f657801d070 con 0x7f65801013c0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.387+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65801964b0 con 0x7f65801013c0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.387+0000 7f657dffb700 1 -- 172.21.15.144:0/3444587491 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f657800be20 con 0x7f65801013c0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.387+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6580196950 con 0x7f65801013c0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.387+0000 7f657dffb700 1 -- 172.21.15.144:0/3444587491 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f657800f460 con 0x7f65801013c0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.388+0000 7f657dffb700 1 -- 172.21.15.144:0/3444587491 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f657800f5c0 con 0x7f65801013c0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.388+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658018f760 con 0x7f65801013c0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.388+0000 7f657dffb700 1 --2- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f656c0387d0 0x7f656c03ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.388+0000 7f657dffb700 1 -- 172.21.15.144:0/3444587491 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f657804c990 con 0x7f65801013c0 2024-05-15T15:45:37.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.389+0000 7f657ffff700 1 --2- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f656c0387d0 0x7f656c03ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:37.392 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.389+0000 7f657ffff700 1 --2- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f656c0387d0 0x7f656c03ac90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f657000ad30 tx=0x7f65700093f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:37.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.395+0000 7f657dffb700 1 -- 172.21.15.144:0/3444587491 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6578022470 con 0x7f65801013c0 2024-05-15T15:45:37.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.665+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f65800625d0 con 0x7f65801013c0 2024-05-15T15:45:37.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.666+0000 7f657dffb700 1 -- 172.21.15.144:0/3444587491 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f65800625d0 con 0x7f65801013c0 2024-05-15T15:45:37.668 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:37.668 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:37.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.667+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f656c0387d0 msgr2=0x7f656c03ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:37.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.667+0000 7f6586f20700 1 --2- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f656c0387d0 0x7f656c03ac90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f657000ad30 tx=0x7f65700093f0 comp rx=0 tx=0).stop 2024-05-15T15:45:37.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.667+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 msgr2=0x7f6580195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:37.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.667+0000 7f6586f20700 1 --2- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 0x7f6580195d70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6578009ed0 tx=0x7f6578004690 comp rx=0 tx=0).stop 2024-05-15T15:45:37.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.668+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 shutdown_connections 2024-05-15T15:45:37.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.668+0000 7f6586f20700 1 --2- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f656c0387d0 0x7f656c03ac90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:37.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.668+0000 7f6586f20700 1 --2- 172.21.15.144:0/3444587491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65801013c0 0x7f6580195d70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:37.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.668+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 >> 172.21.15.144:0/3444587491 conn(0x7f65800fadd0 msgr2=0x7f65800fb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:37.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.668+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 shutdown_connections 2024-05-15T15:45:37.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:37.668+0000 7f6586f20700 1 -- 172.21.15.144:0/3444587491 wait complete. 2024-05-15T15:45:37.671 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:38.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:38 smithi049 ceph-mon[28399]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:38.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:38 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/3444587491' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:39.243 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:39.244 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:39.445 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:39.491 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:40.701 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:40 smithi049 ceph-mon[28399]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:40.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.894+0000 7ffb50e33700 1 -- 172.21.15.144:0/2007645108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 msgr2=0x7ffb4c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:40.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.894+0000 7ffb50e33700 1 --2- 172.21.15.144:0/2007645108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 0x7ffb4c100d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009910 tx=0x7ffb3c009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:40.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.894+0000 7ffb50e33700 1 -- 172.21.15.144:0/2007645108 shutdown_connections 2024-05-15T15:45:40.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.894+0000 7ffb50e33700 1 --2- 172.21.15.144:0/2007645108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 0x7ffb4c100d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:40.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.894+0000 7ffb50e33700 1 -- 172.21.15.144:0/2007645108 >> 172.21.15.144:0/2007645108 conn(0x7ffb4c0fbef0 msgr2=0x7ffb4c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:40.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.894+0000 7ffb50e33700 1 -- 172.21.15.144:0/2007645108 shutdown_connections 2024-05-15T15:45:40.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.894+0000 7ffb50e33700 1 -- 172.21.15.144:0/2007645108 wait complete. 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.895+0000 7ffb50e33700 1 Processor -- start 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.895+0000 7ffb50e33700 1 -- start start 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.895+0000 7ffb50e33700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 0x7ffb4c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.895+0000 7ffb50e33700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c1962f0 con 0x7ffb4c100930 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb4a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 0x7ffb4c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb4a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 0x7ffb4c195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:45126/0 (socket says 172.21.15.144:45126) 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb4a59c700 1 -- 172.21.15.144:0/4170683670 learned_addr learned my addr 172.21.15.144:0/4170683670 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb4a59c700 1 -- 172.21.15.144:0/4170683670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c0095f0 con 0x7ffb4c100930 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb4a59c700 1 --2- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 0x7ffb4c195db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c006010 tx=0x7ffb3c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb437fe700 1 -- 172.21.15.144:0/4170683670 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb3c01d070 con 0x7ffb4c100930 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c1964f0 con 0x7ffb4c100930 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb437fe700 1 -- 172.21.15.144:0/4170683670 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb3c005dc0 con 0x7ffb4c100930 2024-05-15T15:45:40.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb437fe700 1 -- 172.21.15.144:0/4170683670 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb3c022e90 con 0x7ffb4c100930 2024-05-15T15:45:40.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.896+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c196990 con 0x7ffb4c100930 2024-05-15T15:45:40.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.897+0000 7ffb437fe700 1 -- 172.21.15.144:0/4170683670 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7ffb3c00b960 con 0x7ffb4c100930 2024-05-15T15:45:40.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.897+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c18f760 con 0x7ffb4c100930 2024-05-15T15:45:40.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.898+0000 7ffb437fe700 1 --2- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7ffb38038760 0x7ffb3803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:40.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.898+0000 7ffb437fe700 1 -- 172.21.15.144:0/4170683670 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffb3c04c800 con 0x7ffb4c100930 2024-05-15T15:45:40.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.898+0000 7ffb49d9b700 1 --2- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7ffb38038760 0x7ffb3803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:40.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.898+0000 7ffb49d9b700 1 --2- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7ffb38038760 0x7ffb3803ac20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb34006fd0 tx=0x7ffb34006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:40.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:40.903+0000 7ffb437fe700 1 -- 172.21.15.144:0/4170683670 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffb3c022750 con 0x7ffb4c100930 2024-05-15T15:45:41.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.176+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffb4c0625d0 con 0x7ffb4c100930 2024-05-15T15:45:41.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.177+0000 7ffb437fe700 1 -- 172.21.15.144:0/4170683670 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ffb4c0625d0 con 0x7ffb4c100930 2024-05-15T15:45:41.179 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:41.179 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:41.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7ffb38038760 msgr2=0x7ffb3803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:41.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 --2- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7ffb38038760 0x7ffb3803ac20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb34006fd0 tx=0x7ffb34006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:41.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 msgr2=0x7ffb4c195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:41.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 --2- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 0x7ffb4c195db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c006010 tx=0x7ffb3c009f80 comp rx=0 tx=0).stop 2024-05-15T15:45:41.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 shutdown_connections 2024-05-15T15:45:41.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 --2- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7ffb38038760 0x7ffb3803ac20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:41.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 --2- 172.21.15.144:0/4170683670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c100930 0x7ffb4c195db0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:41.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 >> 172.21.15.144:0/4170683670 conn(0x7ffb4c0fbef0 msgr2=0x7ffb4c0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:41.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 shutdown_connections 2024-05-15T15:45:41.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:41.179+0000 7ffb50e33700 1 -- 172.21.15.144:0/4170683670 wait complete. 2024-05-15T15:45:41.182 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:41.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:41 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/4170683670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:42.625 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:42 smithi049 ceph-mon[28399]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:45:42.729 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:42.730 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:42.978 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:44.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.432+0000 7f8924489700 1 -- 172.21.15.144:0/2713673161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 msgr2=0x7f891c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:44.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.432+0000 7f8924489700 1 --2- 172.21.15.144:0/2713673161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 0x7f891c102340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8918009910 tx=0x7f8918009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:44.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.432+0000 7f8924489700 1 -- 172.21.15.144:0/2713673161 shutdown_connections 2024-05-15T15:45:44.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.432+0000 7f8924489700 1 --2- 172.21.15.144:0/2713673161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 0x7f891c102340 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:44.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.432+0000 7f8924489700 1 -- 172.21.15.144:0/2713673161 >> 172.21.15.144:0/2713673161 conn(0x7f891c0fa560 msgr2=0x7f891c0fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:44.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.432+0000 7f8924489700 1 -- 172.21.15.144:0/2713673161 shutdown_connections 2024-05-15T15:45:44.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.432+0000 7f8924489700 1 -- 172.21.15.144:0/2713673161 wait complete. 2024-05-15T15:45:44.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.432+0000 7f8924489700 1 Processor -- start 2024-05-15T15:45:44.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.432+0000 7f8924489700 1 -- start start 2024-05-15T15:45:44.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f8924489700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 0x7f891c1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:44.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f8924489700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c101a30 con 0x7f891c101f20 2024-05-15T15:45:44.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f8922225700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 0x7f891c1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:44.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f8922225700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 0x7f891c1014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:52114/0 (socket says 172.21.15.144:52114) 2024-05-15T15:45:44.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f8922225700 1 -- 172.21.15.144:0/128278054 learned_addr learned my addr 172.21.15.144:0/128278054 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:44.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f8922225700 1 -- 172.21.15.144:0/128278054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89180095f0 con 0x7f891c101f20 2024-05-15T15:45:44.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f8922225700 1 --2- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 0x7f891c1014f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8918006010 tx=0x7f8918009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:44.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f89137fe700 1 -- 172.21.15.144:0/128278054 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f891801d070 con 0x7f891c101f20 2024-05-15T15:45:44.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f89137fe700 1 -- 172.21.15.144:0/128278054 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8918005dc0 con 0x7f891c101f20 2024-05-15T15:45:44.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c0ffc70 con 0x7f891c101f20 2024-05-15T15:45:44.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f89137fe700 1 -- 172.21.15.144:0/128278054 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8918022e90 con 0x7f891c101f20 2024-05-15T15:45:44.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.433+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c100110 con 0x7f891c101f20 2024-05-15T15:45:44.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.434+0000 7f89137fe700 1 -- 172.21.15.144:0/128278054 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 14) v1 ==== 45626+0+0 (secure 0 0 0) 0x7f891800b960 con 0x7f891c101f20 2024-05-15T15:45:44.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.434+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c1070a0 con 0x7f891c101f20 2024-05-15T15:45:44.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.434+0000 7f89137fe700 1 --2- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f89080387b0 0x7f890803ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:44.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.435+0000 7f89137fe700 1 -- 172.21.15.144:0/128278054 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f891804c830 con 0x7f891c101f20 2024-05-15T15:45:44.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.435+0000 7f8921a24700 1 -- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f89080387b0 msgr2=0x7f890803ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1508146783 2024-05-15T15:45:44.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.435+0000 7f8921a24700 1 --2- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f89080387b0 0x7f890803ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T15:45:44.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.440+0000 7f89137fe700 1 -- 172.21.15.144:0/128278054 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f891802a540 con 0x7f891c101f20 2024-05-15T15:45:44.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:44 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:44.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:44 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:44.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:44 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:44.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:44 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-15T15:45:44.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.636+0000 7f8921a24700 1 -- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f89080387b0 msgr2=0x7f890803ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1508146783 2024-05-15T15:45:44.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.636+0000 7f8921a24700 1 --2- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f89080387b0 0x7f890803ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T15:45:44.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.713+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f891c0625d0 con 0x7f891c101f20 2024-05-15T15:45:44.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.714+0000 7f89137fe700 1 -- 172.21.15.144:0/128278054 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8918027020 con 0x7f891c101f20 2024-05-15T15:45:44.716 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:44.717 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f89080387b0 msgr2=0x7f890803ac70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T15:45:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 --2- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f89080387b0 0x7f890803ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 msgr2=0x7f891c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 --2- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 0x7f891c1014f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8918006010 tx=0x7f8918009f80 comp rx=0 tx=0).stop 2024-05-15T15:45:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 shutdown_connections 2024-05-15T15:45:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 --2- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7f89080387b0 0x7f890803ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 --2- 172.21.15.144:0/128278054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f891c101f20 0x7f891c1014f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 >> 172.21.15.144:0/128278054 conn(0x7f891c0fa560 msgr2=0x7f891c0fb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 shutdown_connections 2024-05-15T15:45:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:44.716+0000 7f8924489700 1 -- 172.21.15.144:0/128278054 wait complete. 2024-05-15T15:45:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:45.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:45 smithi049 ceph-mon[28399]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-15T15:45:45.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:45 smithi049 ceph-mon[28399]: mgrmap e14: smithi049.vuhzgf(active, since 64s) 2024-05-15T15:45:45.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:45 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/128278054' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:46.299 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:46.299 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:46.501 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:46.547 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:48.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.013+0000 7fac35e3c700 1 -- 172.21.15.144:0/4007081531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 msgr2=0x7fac30100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:48.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.013+0000 7fac35e3c700 1 --2- 172.21.15.144:0/4007081531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 0x7fac30100d50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fac24009910 tx=0x7fac24009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:48.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.014+0000 7fac35e3c700 1 -- 172.21.15.144:0/4007081531 shutdown_connections 2024-05-15T15:45:48.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.014+0000 7fac35e3c700 1 --2- 172.21.15.144:0/4007081531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 0x7fac30100d50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:48.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.014+0000 7fac35e3c700 1 -- 172.21.15.144:0/4007081531 >> 172.21.15.144:0/4007081531 conn(0x7fac300fbef0 msgr2=0x7fac300fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:48.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.014+0000 7fac35e3c700 1 -- 172.21.15.144:0/4007081531 shutdown_connections 2024-05-15T15:45:48.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.014+0000 7fac35e3c700 1 -- 172.21.15.144:0/4007081531 wait complete. 2024-05-15T15:45:48.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.014+0000 7fac35e3c700 1 Processor -- start 2024-05-15T15:45:48.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.014+0000 7fac35e3c700 1 -- start start 2024-05-15T15:45:48.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac35e3c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 0x7fac30195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:48.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac35e3c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac301962f0 con 0x7fac30100930 2024-05-15T15:45:48.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac2f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 0x7fac30195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:48.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac2f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 0x7fac30195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:52124/0 (socket says 172.21.15.144:52124) 2024-05-15T15:45:48.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac2f7fe700 1 -- 172.21.15.144:0/2203760742 learned_addr learned my addr 172.21.15.144:0/2203760742 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac2f7fe700 1 -- 172.21.15.144:0/2203760742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac240095f0 con 0x7fac30100930 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac2f7fe700 1 --2- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 0x7fac30195db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fac24006010 tx=0x7fac24009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac2cff9700 1 -- 172.21.15.144:0/2203760742 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac2401d070 con 0x7fac30100930 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac2cff9700 1 -- 172.21.15.144:0/2203760742 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac24005dc0 con 0x7fac30100930 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.015+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac301964f0 con 0x7fac30100930 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.016+0000 7fac2cff9700 1 -- 172.21.15.144:0/2203760742 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac24022e90 con 0x7fac30100930 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.016+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac30196990 con 0x7fac30100930 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.016+0000 7fac2cff9700 1 -- 172.21.15.144:0/2203760742 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 14) v1 ==== 45626+0+0 (secure 0 0 0) 0x7fac2400b960 con 0x7fac30100930 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.016+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3018f760 con 0x7fac30100930 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.017+0000 7fac2cff9700 1 --2- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fac180387b0 0x7fac1803ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.017+0000 7fac2cff9700 1 -- 172.21.15.144:0/2203760742 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac2404c8f0 con 0x7fac30100930 2024-05-15T15:45:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.017+0000 7fac2effd700 1 -- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fac180387b0 msgr2=0x7fac1803ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1508146783 2024-05-15T15:45:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.017+0000 7fac2effd700 1 --2- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fac180387b0 0x7fac1803ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T15:45:48.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.022+0000 7fac2cff9700 1 -- 172.21.15.144:0/2203760742 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fac240227a0 con 0x7fac30100930 2024-05-15T15:45:48.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.218+0000 7fac2effd700 1 -- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fac180387b0 msgr2=0x7fac1803ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1508146783 2024-05-15T15:45:48.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.218+0000 7fac2effd700 1 --2- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fac180387b0 0x7fac1803ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T15:45:48.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.298+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fac300625d0 con 0x7fac30100930 2024-05-15T15:45:48.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.298+0000 7fac2cff9700 1 -- 172.21.15.144:0/2203760742 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fac300625d0 con 0x7fac30100930 2024-05-15T15:45:48.301 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:48.301 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fac180387b0 msgr2=0x7fac1803ac70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 --2- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fac180387b0 0x7fac1803ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 msgr2=0x7fac30195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 --2- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 0x7fac30195db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fac24006010 tx=0x7fac24009f80 comp rx=0 tx=0).stop 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 shutdown_connections 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 --2- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:6800/1508146783,v1:172.21.15.49:6801/1508146783] conn(0x7fac180387b0 0x7fac1803ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 --2- 172.21.15.144:0/2203760742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac30100930 0x7fac30195db0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 >> 172.21.15.144:0/2203760742 conn(0x7fac300fbef0 msgr2=0x7fac300fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 shutdown_connections 2024-05-15T15:45:48.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:48.301+0000 7fac35e3c700 1 -- 172.21.15.144:0/2203760742 wait complete. 2024-05-15T15:45:48.304 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:48.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:48 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/2203760742' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:49.845 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:49.845 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:50.041 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:50.088 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:51.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.549+0000 7fa832bd9700 1 -- 172.21.15.144:0/1755345930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 msgr2=0x7fa82c103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.549+0000 7fa832bd9700 1 --2- 172.21.15.144:0/1755345930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 0x7fa82c103900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa820009910 tx=0x7fa820009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.549+0000 7fa832bd9700 1 -- 172.21.15.144:0/1755345930 shutdown_connections 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.549+0000 7fa832bd9700 1 --2- 172.21.15.144:0/1755345930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 0x7fa82c103900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.549+0000 7fa832bd9700 1 -- 172.21.15.144:0/1755345930 >> 172.21.15.144:0/1755345930 conn(0x7fa82c0faee0 msgr2=0x7fa82c0fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.549+0000 7fa832bd9700 1 -- 172.21.15.144:0/1755345930 shutdown_connections 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.549+0000 7fa832bd9700 1 -- 172.21.15.144:0/1755345930 wait complete. 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.550+0000 7fa832bd9700 1 Processor -- start 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.550+0000 7fa832bd9700 1 -- start start 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.550+0000 7fa832bd9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 0x7fa82c100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.550+0000 7fa832bd9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c101280 con 0x7fa82c101510 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.550+0000 7fa830975700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 0x7fa82c100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.550+0000 7fa830975700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 0x7fa82c100d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:48492/0 (socket says 172.21.15.144:48492) 2024-05-15T15:45:51.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.550+0000 7fa830975700 1 -- 172.21.15.144:0/3580521852 learned_addr learned my addr 172.21.15.144:0/3580521852 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:51.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.550+0000 7fa830975700 1 -- 172.21.15.144:0/3580521852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200095f0 con 0x7fa82c101510 2024-05-15T15:45:51.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.551+0000 7fa830975700 1 --2- 172.21.15.144:0/3580521852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 0x7fa82c100d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa820009ed0 tx=0x7fa820004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:51.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.551+0000 7fa829ffb700 1 -- 172.21.15.144:0/3580521852 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa82001d070 con 0x7fa82c101510 2024-05-15T15:45:51.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.551+0000 7fa832bd9700 1 -- 172.21.15.144:0/3580521852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c0ff4c0 con 0x7fa82c101510 2024-05-15T15:45:51.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.551+0000 7fa829ffb700 1 -- 172.21.15.144:0/3580521852 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa82000be20 con 0x7fa82c101510 2024-05-15T15:45:51.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.551+0000 7fa829ffb700 1 -- 172.21.15.144:0/3580521852 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa82000f460 con 0x7fa82c101510 2024-05-15T15:45:51.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.551+0000 7fa832bd9700 1 -- 172.21.15.144:0/3580521852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c0ff960 con 0x7fa82c101510 2024-05-15T15:45:51.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.552+0000 7fa829ffb700 1 -- 172.21.15.144:0/3580521852 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 45350+0+0 (secure 0 0 0) 0x7fa82000f650 con 0x7fa82c101510 2024-05-15T15:45:51.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.552+0000 7fa832bd9700 1 -- 172.21.15.144:0/3580521852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c107160 con 0x7fa82c101510 2024-05-15T15:45:51.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.552+0000 7fa829ffb700 1 -- 172.21.15.144:0/3580521852 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa82004c870 con 0x7fa82c101510 2024-05-15T15:45:51.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.562+0000 7fa829ffb700 1 -- 172.21.15.144:0/3580521852 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa82002a360 con 0x7fa82c101510 2024-05-15T15:45:51.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: Active manager daemon smithi049.vuhzgf restarted 2024-05-15T15:45:51.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: Activating manager daemon smithi049.vuhzgf 2024-05-15T15:45:51.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: osdmap e5: 0 total, 0 up, 0 in 2024-05-15T15:45:51.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: mgrmap e15: smithi049.vuhzgf(active, starting, since 0.00300442s) 2024-05-15T15:45:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:45:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:45:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:45:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:45:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:45:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: Manager daemon smithi049.vuhzgf is now available 2024-05-15T15:45:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:45:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:45:51.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.847+0000 7fa832bd9700 1 -- 172.21.15.144:0/3580521852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa82c18f210 con 0x7fa82c101510 2024-05-15T15:45:51.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.847+0000 7fa829ffb700 1 -- 172.21.15.144:0/3580521852 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fa820047090 con 0x7fa82c101510 2024-05-15T15:45:51.850 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:51.850 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:51.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.850+0000 7fa832bd9700 1 -- 172.21.15.144:0/3580521852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 msgr2=0x7fa82c100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:51.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.850+0000 7fa832bd9700 1 --2- 172.21.15.144:0/3580521852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 0x7fa82c100d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa820009ed0 tx=0x7fa820004690 comp rx=0 tx=0).stop 2024-05-15T15:45:51.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.850+0000 7fa832bd9700 1 -- 172.21.15.144:0/3580521852 shutdown_connections 2024-05-15T15:45:51.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.850+0000 7fa832bd9700 1 --2- 172.21.15.144:0/3580521852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c101510 0x7fa82c100d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:51.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.850+0000 7fa832bd9700 1 -- 172.21.15.144:0/3580521852 >> 172.21.15.144:0/3580521852 conn(0x7fa82c0faee0 msgr2=0x7fa82c0fb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:51.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.850+0000 7fa832bd9700 1 -- 172.21.15.144:0/3580521852 shutdown_connections 2024-05-15T15:45:51.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:51.850+0000 7fa832bd9700 1 -- 172.21.15.144:0/3580521852 wait complete. 2024-05-15T15:45:51.852 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:45:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/trash_purge_schedule"}]: dispatch 2024-05-15T15:45:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:52 smithi049 ceph-mon[28399]: [15/May/2024:15:45:51] ENGINE Bus STARTING 2024-05-15T15:45:52.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:52 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/3580521852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:52.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:52 smithi049 ceph-mon[28399]: [15/May/2024:15:45:51] ENGINE Serving on https://172.21.15.49:7150 2024-05-15T15:45:52.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:52 smithi049 ceph-mon[28399]: [15/May/2024:15:45:52] ENGINE Serving on http://172.21.15.49:8765 2024-05-15T15:45:52.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:52 smithi049 ceph-mon[28399]: [15/May/2024:15:45:52] ENGINE Bus STARTED 2024-05-15T15:45:52.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:52 smithi049 ceph-mon[28399]: mgrmap e16: smithi049.vuhzgf(active, since 1.00609s) 2024-05-15T15:45:53.431 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:53.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:53.629 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:53.680 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:53.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:53 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:53.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:53 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:54.856 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:54 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:54.857 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:54 smithi049 ceph-mon[28399]: mgrmap e17: smithi049.vuhzgf(active, since 2s) 2024-05-15T15:45:55.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.029+0000 7fbd1c613700 1 -- 172.21.15.144:0/4139409759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 msgr2=0x7fbd140ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:55.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.029+0000 7fbd1c613700 1 --2- 172.21.15.144:0/4139409759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 0x7fbd140ffc50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b0a0 tx=0x7fbd0c00b3b0 comp rx=0 tx=0).stop 2024-05-15T15:45:55.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.030+0000 7fbd1c613700 1 -- 172.21.15.144:0/4139409759 shutdown_connections 2024-05-15T15:45:55.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.030+0000 7fbd1c613700 1 --2- 172.21.15.144:0/4139409759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 0x7fbd140ffc50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:55.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.030+0000 7fbd1c613700 1 -- 172.21.15.144:0/4139409759 >> 172.21.15.144:0/4139409759 conn(0x7fbd140fadd0 msgr2=0x7fbd140fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:55.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.030+0000 7fbd1c613700 1 -- 172.21.15.144:0/4139409759 shutdown_connections 2024-05-15T15:45:55.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.030+0000 7fbd1c613700 1 -- 172.21.15.144:0/4139409759 wait complete. 2024-05-15T15:45:55.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.030+0000 7fbd1c613700 1 Processor -- start 2024-05-15T15:45:55.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.030+0000 7fbd1c613700 1 -- start start 2024-05-15T15:45:55.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd1c613700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 0x7fbd140693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:55.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd1c613700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd14069910 con 0x7fbd140ff830 2024-05-15T15:45:55.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd1a3af700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 0x7fbd140693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:55.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd1a3af700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 0x7fbd140693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:48512/0 (socket says 172.21.15.144:48512) 2024-05-15T15:45:55.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd1a3af700 1 -- 172.21.15.144:0/815279017 learned_addr learned my addr 172.21.15.144:0/815279017 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:55.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd1a3af700 1 -- 172.21.15.144:0/815279017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c009d00 con 0x7fbd140ff830 2024-05-15T15:45:55.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd1a3af700 1 --2- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 0x7fbd140693d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0062a0 tx=0x7fbd0c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:55.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd137fe700 1 -- 172.21.15.144:0/815279017 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd0c00e040 con 0x7fbd140ff830 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd137fe700 1 -- 172.21.15.144:0/815279017 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd0c003d70 con 0x7fbd140ff830 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd14198b00 con 0x7fbd140ff830 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.031+0000 7fbd137fe700 1 -- 172.21.15.144:0/815279017 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd0c01c890 con 0x7fbd140ff830 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.032+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd14198fa0 con 0x7fbd140ff830 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.032+0000 7fbd137fe700 1 -- 172.21.15.144:0/815279017 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7fbd0c01cb50 con 0x7fbd140ff830 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.032+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1418e660 con 0x7fbd140ff830 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.033+0000 7fbd137fe700 1 --2- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fbcfc0388b0 0x7fbcfc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.033+0000 7fbd137fe700 1 -- 172.21.15.144:0/815279017 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fbd0c04dc10 con 0x7fbd140ff830 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.033+0000 7fbd19bae700 1 --2- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fbcfc0388b0 0x7fbcfc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:55.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.033+0000 7fbd19bae700 1 --2- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fbcfc0388b0 0x7fbcfc03ad70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd08006fd0 tx=0x7fbd08006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:55.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.038+0000 7fbd137fe700 1 -- 172.21.15.144:0/815279017 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbd0c01b770 con 0x7fbd140ff830 2024-05-15T15:45:55.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.319+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbd140625d0 con 0x7fbd140ff830 2024-05-15T15:45:55.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.320+0000 7fbd137fe700 1 -- 172.21.15.144:0/815279017 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fbd0c011390 con 0x7fbd140ff830 2024-05-15T15:45:55.322 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:45:55.322 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:45:55.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fbcfc0388b0 msgr2=0x7fbcfc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:55.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 --2- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fbcfc0388b0 0x7fbcfc03ad70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd08006fd0 tx=0x7fbd08006e40 comp rx=0 tx=0).stop 2024-05-15T15:45:55.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 msgr2=0x7fbd140693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:55.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 --2- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 0x7fbd140693d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0062a0 tx=0x7fbd0c00bd40 comp rx=0 tx=0).stop 2024-05-15T15:45:55.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 shutdown_connections 2024-05-15T15:45:55.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 --2- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fbcfc0388b0 0x7fbcfc03ad70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:55.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 --2- 172.21.15.144:0/815279017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd140ff830 0x7fbd140693d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:55.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 >> 172.21.15.144:0/815279017 conn(0x7fbd140fadd0 msgr2=0x7fbd140fb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:55.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 shutdown_connections 2024-05-15T15:45:55.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:55.322+0000 7fbd1c613700 1 -- 172.21.15.144:0/815279017 wait complete. 2024-05-15T15:45:55.325 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:45:56.105 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:55 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/815279017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:45:58.019 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:45:58.019 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:45:58.215 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:58.261 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-15T15:45:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:58.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:58.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:45:58.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:58.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:58.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:58.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:45:58.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:45:58.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:45:58.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:45:59.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.847+0000 7fc11f177700 1 -- 172.21.15.144:0/539929697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 msgr2=0x7fc118105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:45:59.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.847+0000 7fc11f177700 1 --2- 172.21.15.144:0/539929697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 0x7fc118105cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009910 tx=0x7fc10c009c20 comp rx=0 tx=0).stop 2024-05-15T15:45:59.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.847+0000 7fc11f177700 1 -- 172.21.15.144:0/539929697 shutdown_connections 2024-05-15T15:45:59.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.847+0000 7fc11f177700 1 --2- 172.21.15.144:0/539929697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 0x7fc118105cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:45:59.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.847+0000 7fc11f177700 1 -- 172.21.15.144:0/539929697 >> 172.21.15.144:0/539929697 conn(0x7fc1180faa00 msgr2=0x7fc1180fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:45:59.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.847+0000 7fc11f177700 1 -- 172.21.15.144:0/539929697 shutdown_connections 2024-05-15T15:45:59.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.847+0000 7fc11f177700 1 -- 172.21.15.144:0/539929697 wait complete. 2024-05-15T15:45:59.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.848+0000 7fc11f177700 1 Processor -- start 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.848+0000 7fc11f177700 1 -- start start 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.848+0000 7fc11f177700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 0x7fc118198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.848+0000 7fc11f177700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1181989b0 con 0x7fc1181058e0 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.848+0000 7fc11cf13700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 0x7fc118198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.848+0000 7fc11cf13700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 0x7fc118198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:48538/0 (socket says 172.21.15.144:48538) 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.848+0000 7fc11cf13700 1 -- 172.21.15.144:0/4099001569 learned_addr learned my addr 172.21.15.144:0/4099001569 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.849+0000 7fc11cf13700 1 -- 172.21.15.144:0/4099001569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0095f0 con 0x7fc1181058e0 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.849+0000 7fc11cf13700 1 --2- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 0x7fc118198470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc10c005490 tx=0x7fc10c005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.849+0000 7fc115ffb700 1 -- 172.21.15.144:0/4099001569 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc10c01d070 con 0x7fc1181058e0 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.849+0000 7fc115ffb700 1 -- 172.21.15.144:0/4099001569 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc10c005700 con 0x7fc1181058e0 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.849+0000 7fc115ffb700 1 -- 172.21.15.144:0/4099001569 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc10c003950 con 0x7fc1181058e0 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.849+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc118198bb0 con 0x7fc1181058e0 2024-05-15T15:45:59.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.849+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc118198fd0 con 0x7fc1181058e0 2024-05-15T15:45:59.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.850+0000 7fc115ffb700 1 -- 172.21.15.144:0/4099001569 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7fc10c004240 con 0x7fc1181058e0 2024-05-15T15:45:59.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.850+0000 7fc115ffb700 1 --2- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc108038910 0x7fc10803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:45:59.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.850+0000 7fc115ffb700 1 -- 172.21.15.144:0/4099001569 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc10c04c640 con 0x7fc1181058e0 2024-05-15T15:45:59.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.850+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc118102020 con 0x7fc1181058e0 2024-05-15T15:45:59.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.851+0000 7fc117fff700 1 --2- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc108038910 0x7fc10803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:45:59.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.851+0000 7fc117fff700 1 --2- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc108038910 0x7fc10803add0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc104006fd0 tx=0x7fc104006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:45:59.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:45:59.856+0000 7fc115ffb700 1 -- 172.21.15.144:0/4099001569 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc10c026020 con 0x7fc1181058e0 2024-05-15T15:45:59.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:59 smithi049 ceph-mon[28399]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:45:59.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:59 smithi049 ceph-mon[28399]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:45:59.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:59 smithi049 ceph-mon[28399]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:45:59.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:59 smithi049 ceph-mon[28399]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:45:59.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:59 smithi049 ceph-mon[28399]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:45:59.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:45:59 smithi049 ceph-mon[28399]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:46:00.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.149+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc11810ade0 con 0x7fc1181058e0 2024-05-15T15:46:00.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.150+0000 7fc115ffb700 1 -- 172.21.15.144:0/4099001569 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc10c00f460 con 0x7fc1181058e0 2024-05-15T15:46:00.153 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:46:00.153 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:46:00.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc108038910 msgr2=0x7fc10803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 --2- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc108038910 0x7fc10803add0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc104006fd0 tx=0x7fc104006e40 comp rx=0 tx=0).stop 2024-05-15T15:46:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 msgr2=0x7fc118198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 --2- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 0x7fc118198470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc10c005490 tx=0x7fc10c005570 comp rx=0 tx=0).stop 2024-05-15T15:46:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 shutdown_connections 2024-05-15T15:46:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 --2- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc108038910 0x7fc10803add0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 --2- 172.21.15.144:0/4099001569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1181058e0 0x7fc118198470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 >> 172.21.15.144:0/4099001569 conn(0x7fc1180faa00 msgr2=0x7fc118108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 shutdown_connections 2024-05-15T15:46:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:00.152+0000 7fc11f177700 1 -- 172.21.15.144:0/4099001569 wait complete. 2024-05-15T15:46:00.156 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:46:00.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: Deploying daemon ceph-exporter.smithi144 on smithi144 2024-05-15T15:46:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:00 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/4099001569' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:01.814 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:46:01.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:46:02.120 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:03.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.557+0000 7f26e7875700 1 -- 172.21.15.144:0/1850787038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 msgr2=0x7f26e0106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:03.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.557+0000 7f26e7875700 1 --2- 172.21.15.144:0/1850787038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 0x7f26e0106ce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009910 tx=0x7f26dc009c20 comp rx=0 tx=0).stop 2024-05-15T15:46:03.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.557+0000 7f26e7875700 1 -- 172.21.15.144:0/1850787038 shutdown_connections 2024-05-15T15:46:03.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.557+0000 7f26e7875700 1 --2- 172.21.15.144:0/1850787038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 0x7f26e0106ce0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:03.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.557+0000 7f26e7875700 1 -- 172.21.15.144:0/1850787038 >> 172.21.15.144:0/1850787038 conn(0x7f26e00fc410 msgr2=0x7f26e00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:03.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.558+0000 7f26e7875700 1 -- 172.21.15.144:0/1850787038 shutdown_connections 2024-05-15T15:46:03.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.558+0000 7f26e7875700 1 -- 172.21.15.144:0/1850787038 wait complete. 2024-05-15T15:46:03.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.558+0000 7f26e7875700 1 Processor -- start 2024-05-15T15:46:03.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.559+0000 7f26e7875700 1 -- start start 2024-05-15T15:46:03.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.559+0000 7f26e7875700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 0x7f26e01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:03.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.559+0000 7f26e7875700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e0198900 con 0x7f26e0106900 2024-05-15T15:46:03.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.559+0000 7f26e5611700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 0x7f26e01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:03.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.559+0000 7f26e5611700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 0x7f26e01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:33876/0 (socket says 172.21.15.144:33876) 2024-05-15T15:46:03.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.559+0000 7f26e5611700 1 -- 172.21.15.144:0/4022546003 learned_addr learned my addr 172.21.15.144:0/4022546003 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:46:03.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.560+0000 7f26e5611700 1 -- 172.21.15.144:0/4022546003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc0095f0 con 0x7f26e0106900 2024-05-15T15:46:03.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.560+0000 7f26e5611700 1 --2- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 0x7f26e01983c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f26dc005490 tx=0x7f26dc005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:03.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.560+0000 7f26d67fc700 1 -- 172.21.15.144:0/4022546003 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26dc01d070 con 0x7f26e0106900 2024-05-15T15:46:03.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.560+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e0198b60 con 0x7f26e0106900 2024-05-15T15:46:03.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.560+0000 7f26d67fc700 1 -- 172.21.15.144:0/4022546003 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26dc005700 con 0x7f26e0106900 2024-05-15T15:46:03.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.560+0000 7f26d67fc700 1 -- 172.21.15.144:0/4022546003 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26dc0038e0 con 0x7f26e0106900 2024-05-15T15:46:03.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.560+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e0198fe0 con 0x7f26e0106900 2024-05-15T15:46:03.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.561+0000 7f26d67fc700 1 -- 172.21.15.144:0/4022546003 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f26dc003b50 con 0x7f26e0106900 2024-05-15T15:46:03.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.561+0000 7f26d67fc700 1 --2- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f26cc038960 0x7f26cc03ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:03.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.561+0000 7f26d67fc700 1 -- 172.21.15.144:0/4022546003 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f26dc04c8f0 con 0x7f26e0106900 2024-05-15T15:46:03.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.561+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e0102040 con 0x7f26e0106900 2024-05-15T15:46:03.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.561+0000 7f26e4e10700 1 --2- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f26cc038960 0x7f26cc03ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:03.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.562+0000 7f26e4e10700 1 --2- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f26cc038960 0x7f26cc03ae20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26d0006fd0 tx=0x7f26d0006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:03.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.567+0000 7f26d67fc700 1 -- 172.21.15.144:0/4022546003 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f26dc026020 con 0x7f26e0106900 2024-05-15T15:46:03.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.841+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f26e00733f0 con 0x7f26e0106900 2024-05-15T15:46:03.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.842+0000 7f26d67fc700 1 -- 172.21.15.144:0/4022546003 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f26dc029720 con 0x7f26e0106900 2024-05-15T15:46:03.845 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:46:03.845 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:46:03.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f26cc038960 msgr2=0x7f26cc03ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:03.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 --2- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f26cc038960 0x7f26cc03ae20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26d0006fd0 tx=0x7f26d0006e40 comp rx=0 tx=0).stop 2024-05-15T15:46:03.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 msgr2=0x7f26e01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:03.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 --2- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 0x7f26e01983c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f26dc005490 tx=0x7f26dc005570 comp rx=0 tx=0).stop 2024-05-15T15:46:03.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 shutdown_connections 2024-05-15T15:46:03.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 --2- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f26cc038960 0x7f26cc03ae20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:03.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 --2- 172.21.15.144:0/4022546003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26e0106900 0x7f26e01983c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:03.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 >> 172.21.15.144:0/4022546003 conn(0x7f26e00fc410 msgr2=0x7f26e0107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:03.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 shutdown_connections 2024-05-15T15:46:03.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:03.845+0000 7f26e7875700 1 -- 172.21.15.144:0/4022546003 wait complete. 2024-05-15T15:46:03.849 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:46:04.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:03 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/4022546003' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:05.688 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:46:05.689 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:46:05.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:05.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:05.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:05.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:05.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:46:05.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-15T15:46:05.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:05.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:05 smithi049 ceph-mon[28399]: Deploying daemon crash.smithi144 on smithi144 2024-05-15T15:46:05.889 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:07.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.600+0000 7fb6eff58700 1 -- 172.21.15.144:0/1714811696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 msgr2=0x7fb6e8106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:07.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.600+0000 7fb6eff58700 1 --2- 172.21.15.144:0/1714811696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 0x7fb6e8106ca0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009910 tx=0x7fb6d8009c20 comp rx=0 tx=0).stop 2024-05-15T15:46:07.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.600+0000 7fb6eff58700 1 -- 172.21.15.144:0/1714811696 shutdown_connections 2024-05-15T15:46:07.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.600+0000 7fb6eff58700 1 --2- 172.21.15.144:0/1714811696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 0x7fb6e8106ca0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:07.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.600+0000 7fb6eff58700 1 -- 172.21.15.144:0/1714811696 >> 172.21.15.144:0/1714811696 conn(0x7fb6e80fc3d0 msgr2=0x7fb6e80fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:07.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.600+0000 7fb6eff58700 1 -- 172.21.15.144:0/1714811696 shutdown_connections 2024-05-15T15:46:07.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.600+0000 7fb6eff58700 1 -- 172.21.15.144:0/1714811696 wait complete. 2024-05-15T15:46:07.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.601+0000 7fb6eff58700 1 Processor -- start 2024-05-15T15:46:07.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.601+0000 7fb6eff58700 1 -- start start 2024-05-15T15:46:07.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.601+0000 7fb6eff58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 0x7fb6e8196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:07.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.601+0000 7fb6eff58700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8196670 con 0x7fb6e81068c0 2024-05-15T15:46:07.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6edcf4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 0x7fb6e8196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:07.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6edcf4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 0x7fb6e8196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:33912/0 (socket says 172.21.15.144:33912) 2024-05-15T15:46:07.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6edcf4700 1 -- 172.21.15.144:0/1033834848 learned_addr learned my addr 172.21.15.144:0/1033834848 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:46:07.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6edcf4700 1 -- 172.21.15.144:0/1033834848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80095f0 con 0x7fb6e81068c0 2024-05-15T15:46:07.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6edcf4700 1 --2- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 0x7fb6e8196130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005490 tx=0x7fb6d8005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:07.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6deffd700 1 -- 172.21.15.144:0/1033834848 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6d801d070 con 0x7fb6e81068c0 2024-05-15T15:46:07.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e81968d0 con 0x7fb6e81068c0 2024-05-15T15:46:07.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6deffd700 1 -- 172.21.15.144:0/1033834848 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6d8005700 con 0x7fb6e81068c0 2024-05-15T15:46:07.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6deffd700 1 -- 172.21.15.144:0/1033834848 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6d80038e0 con 0x7fb6e81068c0 2024-05-15T15:46:07.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.602+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e8196d50 con 0x7fb6e81068c0 2024-05-15T15:46:07.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.603+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e8102000 con 0x7fb6e81068c0 2024-05-15T15:46:07.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.603+0000 7fb6deffd700 1 -- 172.21.15.144:0/1033834848 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7fb6d8003b50 con 0x7fb6e81068c0 2024-05-15T15:46:07.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.603+0000 7fb6deffd700 1 --2- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb6d4038910 0x7fb6d403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:07.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.604+0000 7fb6deffd700 1 -- 172.21.15.144:0/1033834848 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb6d804c650 con 0x7fb6e81068c0 2024-05-15T15:46:07.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.604+0000 7fb6ed4f3700 1 --2- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb6d4038910 0x7fb6d403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:07.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.604+0000 7fb6ed4f3700 1 --2- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb6d4038910 0x7fb6d403add0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006fd0 tx=0x7fb6e4006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:07.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.610+0000 7fb6deffd700 1 -- 172.21.15.144:0/1033834848 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb6d802a590 con 0x7fb6e81068c0 2024-05-15T15:46:07.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:07 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:07.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:07 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:46:07.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.896+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb6e810acb0 con 0x7fb6e81068c0 2024-05-15T15:46:07.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.897+0000 7fb6deffd700 1 -- 172.21.15.144:0/1033834848 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb6d8026030 con 0x7fb6e81068c0 2024-05-15T15:46:07.899 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:46:07.899 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:46:07.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb6d4038910 msgr2=0x7fb6d403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:07.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 --2- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb6d4038910 0x7fb6d403add0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006fd0 tx=0x7fb6e4006e40 comp rx=0 tx=0).stop 2024-05-15T15:46:07.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 msgr2=0x7fb6e8196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:07.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 --2- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 0x7fb6e8196130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005490 tx=0x7fb6d8005570 comp rx=0 tx=0).stop 2024-05-15T15:46:07.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 shutdown_connections 2024-05-15T15:46:07.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 --2- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb6d4038910 0x7fb6d403add0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:07.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 --2- 172.21.15.144:0/1033834848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6e81068c0 0x7fb6e8196130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:07.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 >> 172.21.15.144:0/1033834848 conn(0x7fb6e80fc3d0 msgr2=0x7fb6e8108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:07.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 shutdown_connections 2024-05-15T15:46:07.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:07.899+0000 7fb6eff58700 1 -- 172.21.15.144:0/1033834848 wait complete. 2024-05-15T15:46:07.902 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:46:09.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:09.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:09.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:09.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:09.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:08 smithi049 ceph-mon[28399]: Deploying daemon node-exporter.smithi144 on smithi144 2024-05-15T15:46:09.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:08 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/1033834848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:09.509 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:46:09.509 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:46:09.708 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:12.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.204+0000 7f6854c13700 1 -- 172.21.15.144:0/1313636960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 msgr2=0x7f6850105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:12.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.204+0000 7f6854c13700 1 --2- 172.21.15.144:0/1313636960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 0x7f6850105bb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6840009910 tx=0x7f6840009c20 comp rx=0 tx=0).stop 2024-05-15T15:46:12.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.204+0000 7f6854c13700 1 -- 172.21.15.144:0/1313636960 shutdown_connections 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.204+0000 7f6854c13700 1 --2- 172.21.15.144:0/1313636960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 0x7f6850105bb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.204+0000 7f6854c13700 1 -- 172.21.15.144:0/1313636960 >> 172.21.15.144:0/1313636960 conn(0x7f68500fb300 msgr2=0x7f68500fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.204+0000 7f6854c13700 1 -- 172.21.15.144:0/1313636960 shutdown_connections 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.204+0000 7f6854c13700 1 -- 172.21.15.144:0/1313636960 wait complete. 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.205+0000 7f6854c13700 1 Processor -- start 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.205+0000 7f6854c13700 1 -- start start 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.205+0000 7f6854c13700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 0x7f6850195080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.205+0000 7f6854c13700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501955c0 con 0x7f68501057d0 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f684e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 0x7f6850195080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f684e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 0x7f6850195080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:53334/0 (socket says 172.21.15.144:53334) 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f684e59c700 1 -- 172.21.15.144:0/3977248711 learned_addr learned my addr 172.21.15.144:0/3977248711 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:46:12.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f684e59c700 1 -- 172.21.15.144:0/3977248711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400095f0 con 0x7f68501057d0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f684e59c700 1 --2- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 0x7f6850195080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68400098e0 tx=0x7f6840005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f683f7fe700 1 -- 172.21.15.144:0/3977248711 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f684001d070 con 0x7f68501057d0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6850195820 con 0x7f68501057d0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f683f7fe700 1 -- 172.21.15.144:0/3977248711 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68400056b0 con 0x7f68501057d0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f683f7fe700 1 -- 172.21.15.144:0/3977248711 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6840022d80 con 0x7f68501057d0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.206+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6850195ca0 con 0x7f68501057d0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.207+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6850100f10 con 0x7f68501057d0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.208+0000 7f683f7fe700 1 -- 172.21.15.144:0/3977248711 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f6840022470 con 0x7f68501057d0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.208+0000 7f683f7fe700 1 --2- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f68380388b0 0x7f683803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.208+0000 7f683f7fe700 1 -- 172.21.15.144:0/3977248711 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f684004c950 con 0x7f68501057d0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.208+0000 7f684dd9b700 1 --2- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f68380388b0 0x7f683803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.208+0000 7f684dd9b700 1 --2- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f68380388b0 0x7f683803ad70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6844006fd0 tx=0x7f6844006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:12.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.214+0000 7f683f7fe700 1 -- 172.21.15.144:0/3977248711 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f684002b590 con 0x7f68501057d0 2024-05-15T15:46:12.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.493+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6850109bc0 con 0x7f68501057d0 2024-05-15T15:46:12.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.493+0000 7f683f7fe700 1 -- 172.21.15.144:0/3977248711 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6840027070 con 0x7f68501057d0 2024-05-15T15:46:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:46:12.496 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:46:12.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.495+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f68380388b0 msgr2=0x7f683803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:12.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.495+0000 7f6854c13700 1 --2- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f68380388b0 0x7f683803ad70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6844006fd0 tx=0x7f6844006e40 comp rx=0 tx=0).stop 2024-05-15T15:46:12.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.495+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 msgr2=0x7f6850195080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:12.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.495+0000 7f6854c13700 1 --2- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 0x7f6850195080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68400098e0 tx=0x7f6840005520 comp rx=0 tx=0).stop 2024-05-15T15:46:12.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.495+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 shutdown_connections 2024-05-15T15:46:12.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.495+0000 7f6854c13700 1 --2- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f68380388b0 0x7f683803ad70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:12.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.495+0000 7f6854c13700 1 --2- 172.21.15.144:0/3977248711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68501057d0 0x7f6850195080 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:12.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.495+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 >> 172.21.15.144:0/3977248711 conn(0x7f68500fb300 msgr2=0x7f6850107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:12.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.496+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 shutdown_connections 2024-05-15T15:46:12.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:12.496+0000 7f6854c13700 1 -- 172.21.15.144:0/3977248711 wait complete. 2024-05-15T15:46:12.499 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:46:12.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:12 smithi049 ceph-mon[28399]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:12.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:13.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:13 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/3977248711' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:14.158 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:46:14.159 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:46:14.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:14.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:14.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:14.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:14.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:14.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:46:14.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-15T15:46:14.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:46:14.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:14.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:14 smithi049 ceph-mon[28399]: Deploying daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:46:15.244 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:16.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:16 smithi049 ceph-mon[28399]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:16.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:17.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.052+0000 7fceedab5700 1 -- 172.21.15.144:0/1881905096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 msgr2=0x7fcee8105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:17.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.052+0000 7fceedab5700 1 --2- 172.21.15.144:0/1881905096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 0x7fcee8105c00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fced800b0a0 tx=0x7fced800b3b0 comp rx=0 tx=0).stop 2024-05-15T15:46:17.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.052+0000 7fceedab5700 1 -- 172.21.15.144:0/1881905096 shutdown_connections 2024-05-15T15:46:17.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.052+0000 7fceedab5700 1 --2- 172.21.15.144:0/1881905096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 0x7fcee8105c00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:17.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.052+0000 7fceedab5700 1 -- 172.21.15.144:0/1881905096 >> 172.21.15.144:0/1881905096 conn(0x7fcee8075350 msgr2=0x7fcee8075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.052+0000 7fceedab5700 1 -- 172.21.15.144:0/1881905096 shutdown_connections 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.052+0000 7fceedab5700 1 -- 172.21.15.144:0/1881905096 wait complete. 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.052+0000 7fceedab5700 1 Processor -- start 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.052+0000 7fceedab5700 1 -- start start 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.053+0000 7fceedab5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 0x7fcee81961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.053+0000 7fceedab5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8196700 con 0x7fcee8105820 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.053+0000 7fcee77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 0x7fcee81961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.053+0000 7fcee77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 0x7fcee81961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:53346/0 (socket says 172.21.15.144:53346) 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.053+0000 7fcee77fe700 1 -- 172.21.15.144:0/3455433885 learned_addr learned my addr 172.21.15.144:0/3455433885 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:46:17.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.053+0000 7fcee77fe700 1 -- 172.21.15.144:0/3455433885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009d00 con 0x7fcee8105820 2024-05-15T15:46:17.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.054+0000 7fcee77fe700 1 --2- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 0x7fcee81961c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fced800b830 tx=0x7fced8009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:17.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.054+0000 7fcee4ff9700 1 -- 172.21.15.144:0/3455433885 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fced800e040 con 0x7fcee8105820 2024-05-15T15:46:17.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.054+0000 7fcee4ff9700 1 -- 172.21.15.144:0/3455433885 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fced8004040 con 0x7fcee8105820 2024-05-15T15:46:17.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.054+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee8196900 con 0x7fcee8105820 2024-05-15T15:46:17.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.054+0000 7fcee4ff9700 1 -- 172.21.15.144:0/3455433885 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fced801d440 con 0x7fcee8105820 2024-05-15T15:46:17.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.054+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee8196d80 con 0x7fcee8105820 2024-05-15T15:46:17.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.055+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee8100ef0 con 0x7fcee8105820 2024-05-15T15:46:17.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.055+0000 7fcee4ff9700 1 -- 172.21.15.144:0/3455433885 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7fced80041b0 con 0x7fcee8105820 2024-05-15T15:46:17.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.055+0000 7fcee4ff9700 1 --2- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fced0038910 0x7fced003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:17.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.055+0000 7fcee4ff9700 1 -- 172.21.15.144:0/3455433885 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fced804c4e0 con 0x7fcee8105820 2024-05-15T15:46:17.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.055+0000 7fcee6ffd700 1 --2- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fced0038910 0x7fced003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:17.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.056+0000 7fcee6ffd700 1 --2- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fced0038910 0x7fced003add0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcedc006fd0 tx=0x7fcedc006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:17.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.061+0000 7fcee4ff9700 1 -- 172.21.15.144:0/3455433885 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fced8011570 con 0x7fcee8105820 2024-05-15T15:46:17.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.346+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcee8109c10 con 0x7fcee8105820 2024-05-15T15:46:17.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.346+0000 7fcee4ff9700 1 -- 172.21.15.144:0/3455433885 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fced8011390 con 0x7fcee8105820 2024-05-15T15:46:17.349 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:46:17.349 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:43:32.672725Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-15T15:46:17.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.348+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fced0038910 msgr2=0x7fced003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:17.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.348+0000 7fceedab5700 1 --2- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fced0038910 0x7fced003add0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcedc006fd0 tx=0x7fcedc006e40 comp rx=0 tx=0).stop 2024-05-15T15:46:17.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.348+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 msgr2=0x7fcee81961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:17.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.348+0000 7fceedab5700 1 --2- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 0x7fcee81961c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fced800b830 tx=0x7fced8009760 comp rx=0 tx=0).stop 2024-05-15T15:46:17.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.348+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 shutdown_connections 2024-05-15T15:46:17.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.348+0000 7fceedab5700 1 --2- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fced0038910 0x7fced003add0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:17.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.348+0000 7fceedab5700 1 --2- 172.21.15.144:0/3455433885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee8105820 0x7fcee81961c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:17.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.348+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 >> 172.21.15.144:0/3455433885 conn(0x7fcee8075350 msgr2=0x7fcee81034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:17.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.349+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 shutdown_connections 2024-05-15T15:46:17.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:17.349+0000 7fceedab5700 1 -- 172.21.15.144:0/3455433885 wait complete. 2024-05-15T15:46:17.352 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-05-15T15:46:18.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:18 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:18.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:18 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:18.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:18 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:18.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:18 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:18.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:18 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:46:18.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:18 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:18.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:18 smithi049 ceph-mon[28399]: Deploying daemon mon.smithi144 on smithi144 2024-05-15T15:46:18.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:18 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/3455433885' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:18.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:18 smithi049 ceph-mon[28399]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:18.942 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-15T15:46:18.943 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T15:46:20.041 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T15:46:20.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:20 smithi049 ceph-mon[28399]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:20 smithi144 systemd[1]: Starting Ceph mon.smithi144 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T15:46:21.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:21 smithi144 podman[36335]: 2024-05-15 15:46:21.524118141 +0000 UTC m=+0.202968388 image pull a04a73ffe07969b07057db75204012857aea1fdc1d3eeb4e5270ae40eb9fbe46 quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1 2024-05-15T15:46:21.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:21 smithi144 podman[36335]: 2024-05-15 15:46:21.636878919 +0000 UTC m=+0.315729143 container create bf0fd6a44ac8f2bc27fa7a7960e690c0104915e5b2bd6edf77ed1353a9eb20c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=reef-2965081, org.label-schema.vendor=CentOS) 2024-05-15T15:46:21.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:21 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:46:21.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:21 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:21.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.970+0000 7f457622d700 1 -- 172.21.15.144:0/3339922682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 msgr2=0x7f4570105cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:21.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.970+0000 7f457622d700 1 --2- 172.21.15.144:0/3339922682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 0x7f4570105cb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4564009910 tx=0x7f4564009c20 comp rx=0 tx=0).stop 2024-05-15T15:46:21.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.970+0000 7f457622d700 1 -- 172.21.15.144:0/3339922682 shutdown_connections 2024-05-15T15:46:21.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.970+0000 7f457622d700 1 --2- 172.21.15.144:0/3339922682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 0x7f4570105cb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:21.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.970+0000 7f457622d700 1 -- 172.21.15.144:0/3339922682 >> 172.21.15.144:0/3339922682 conn(0x7f45700fb400 msgr2=0x7f45700fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:21.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.970+0000 7f457622d700 1 -- 172.21.15.144:0/3339922682 shutdown_connections 2024-05-15T15:46:21.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.970+0000 7f457622d700 1 -- 172.21.15.144:0/3339922682 wait complete. 2024-05-15T15:46:21.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.971+0000 7f457622d700 1 Processor -- start 2024-05-15T15:46:21.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.971+0000 7f457622d700 1 -- start start 2024-05-15T15:46:21.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.971+0000 7f457622d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 0x7f4570197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:21.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.971+0000 7f457622d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45701975d0 con 0x7f45701058d0 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.972+0000 7f456f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 0x7f4570197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.972+0000 7f456f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 0x7f4570197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.144:52632/0 (socket says 172.21.15.144:52632) 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.972+0000 7f456f7fe700 1 -- 172.21.15.144:0/3662485208 learned_addr learned my addr 172.21.15.144:0/3662485208 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.972+0000 7f456f7fe700 1 -- 172.21.15.144:0/3662485208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45640095f0 con 0x7f45701058d0 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.972+0000 7f456f7fe700 1 --2- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 0x7f4570197090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4564005490 tx=0x7f4564005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.972+0000 7f456cff9700 1 -- 172.21.15.144:0/3662485208 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f456401d070 con 0x7f45701058d0 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.972+0000 7f456cff9700 1 -- 172.21.15.144:0/3662485208 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4564005700 con 0x7f45701058d0 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.972+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4570197830 con 0x7f45701058d0 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.972+0000 7f456cff9700 1 -- 172.21.15.144:0/3662485208 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45640038e0 con 0x7f45701058d0 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.973+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4570197cb0 con 0x7f45701058d0 2024-05-15T15:46:21.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.973+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4570197ef0 con 0x7f45701058d0 2024-05-15T15:46:21.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.973+0000 7f456cff9700 1 -- 172.21.15.144:0/3662485208 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f4564003ba0 con 0x7f45701058d0 2024-05-15T15:46:21.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.974+0000 7f456cff9700 1 --2- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f45580388c0 0x7f455803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:21.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.974+0000 7f456cff9700 1 -- 172.21.15.144:0/3662485208 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f456404c630 con 0x7f45701058d0 2024-05-15T15:46:21.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.974+0000 7f456effd700 1 --2- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f45580388c0 0x7f455803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:21.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.974+0000 7f456effd700 1 --2- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f45580388c0 0x7f455803ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4560006fd0 tx=0x7f4560006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:21.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:21.979+0000 7f456cff9700 1 -- 172.21.15.144:0/3662485208 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4564029950 con 0x7f45701058d0 2024-05-15T15:46:22.119 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:22.117+0000 7f456cff9700 1 -- 172.21.15.144:0/3662485208 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4564029950 con 0x7f45701058d0 2024-05-15T15:46:22.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:22.269+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4570069480 con 0x7f45701058d0 2024-05-15T15:46:22.386 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 podman[36335]: 2024-05-15 15:46:22.048958704 +0000 UTC m=+0.727808931 container init bf0fd6a44ac8f2bc27fa7a7960e690c0104915e5b2bd6edf77ed1353a9eb20c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-2965081, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T15:46:22.386 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 podman[36335]: 2024-05-15 15:46:22.05304497 +0000 UTC m=+0.731895200 container start bf0fd6a44ac8f2bc27fa7a7960e690c0104915e5b2bd6edf77ed1353a9eb20c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=reef-2965081, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T15:46:22.386 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: set uid:gid to 167:167 (ceph:ceph) 2024-05-15T15:46:22.386 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable), process ceph-mon, pid 2 2024-05-15T15:46:22.386 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pidfile_write: ignore empty --pid-file 2024-05-15T15:46:22.386 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: load: jerasure load: lrc 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: RocksDB version: 7.9.2 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Git sha 0 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Compile date 2024-05-13 17:32:21 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: DB SUMMARY 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: DB Session ID: YVVMJK03HH8TKGN1YRA6 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: CURRENT file: CURRENT 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: IDENTITY file: IDENTITY 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi144/store.db dir, Total Num: 0, files: 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi144/store.db: 000004.log size: 511 ; 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.error_if_exists: 0 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.create_if_missing: 0 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.paranoid_checks: 1 2024-05-15T15:46:22.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.env: 0x55f8bc089820 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.fs: PosixFileSystem 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.info_log: 0x55f8be9713e0 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_file_opening_threads: 16 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.statistics: (nil) 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.use_fsync: 0 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_log_file_size: 0 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.keep_log_file_num: 1000 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.recycle_log_file_num: 0 2024-05-15T15:46:22.388 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.allow_fallocate: 1 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.allow_mmap_reads: 0 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.allow_mmap_writes: 0 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.use_direct_reads: 0 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.create_missing_column_families: 0 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.db_log_dir: 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.wal_dir: 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-15T15:46:22.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.advise_random_on_open: 1 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.db_write_buffer_size: 0 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.write_buffer_manager: 0x55f8bdbe6140 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.rate_limiter: (nil) 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.wal_recovery_mode: 2 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.enable_thread_tracking: 0 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.enable_pipelined_write: 0 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.unordered_write: 0 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-15T15:46:22.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-15T15:46:22.391 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-15T15:46:22.391 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.row_cache: None 2024-05-15T15:46:22.391 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.wal_filter: None 2024-05-15T15:46:22.393 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-15T15:46:22.393 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.allow_ingest_behind: 0 2024-05-15T15:46:22.393 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.two_write_queues: 0 2024-05-15T15:46:22.393 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.manual_wal_flush: 0 2024-05-15T15:46:22.393 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.wal_compression: 0 2024-05-15T15:46:22.393 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.atomic_flush: 0 2024-05-15T15:46:22.393 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.log_readahead_size: 0 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.best_efforts_recovery: 0 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.allow_data_in_errors: 0 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.db_host_id: __hostname__ 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_background_jobs: 2 2024-05-15T15:46:22.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_background_compactions: -1 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_subcompactions: 1 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_total_wal_size: 0 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_open_files: -1 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bytes_per_sync: 0 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-15T15:46:22.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_readahead_size: 0 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_background_flushes: -1 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Compression algorithms supported: 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: kZSTD supported: 0 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: kXpressCompression supported: 0 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: kLZ4HCCompression supported: 1 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: kZlibCompression supported: 1 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: kSnappyCompression supported: 1 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: kLZ4Compression supported: 1 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: kBZip2Compression supported: 0 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi144/store.db/MANIFEST-000005 2024-05-15T15:46:22.396 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.merge_operator: 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_filter: None 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_filter_factory: None 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.sst_partitioner_factory: None 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f8be971500) 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: cache_index_and_filter_blocks: 1 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: pin_top_level_index_and_filter: 1 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_type: 0 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_block_index_type: 0 2024-05-15T15:46:22.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_shortening: 1 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: checksum: 4 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: no_block_cache: 0 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache: 0x55f8bdc891f0 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_name: BinnedLRUCache 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_options: 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: capacity : 536870912 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: num_shard_bits : 4 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: strict_capacity_limit : 0 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: high_pri_pool_ratio: 0.000 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_compressed: (nil) 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: persistent_cache: (nil) 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_size: 4096 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_size_deviation: 10 2024-05-15T15:46:22.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_restart_interval: 16 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_block_restart_interval: 1 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: metadata_block_size: 4096 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: partition_filters: 0 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: use_delta_encoding: 1 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: filter_policy: bloomfilter 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: whole_key_filtering: 1 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: verify_compression: 0 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: read_amp_bytes_per_bit: 0 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: format_version: 5 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: enable_index_compression: 1 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_align: 0 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_auto_readahead_size: 262144 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: prepopulate_block_cache: 0 2024-05-15T15:46:22.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: initial_auto_readahead_size: 8192 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: num_file_reads_for_auto_readahead: 2 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.write_buffer_size: 33554432 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_write_buffer_number: 2 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression: NoCompression 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression: Disabled 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.prefix_extractor: nullptr 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.num_levels: 7 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-15T15:46:22.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression_opts.level: 32767 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression_opts.strategy: 0 2024-05-15T15:46:22.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression_opts.enabled: false 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.target_file_size_base: 67108864 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-15T15:46:22.403 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.arena_block_size: 1048576 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.disable_auto_compactions: 0 2024-05-15T15:46:22.404 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.table_properties_collectors: 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.inplace_update_support: 0 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-15T15:46:22.405 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.bloom_locality: 0 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.max_successive_merges: 0 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.paranoid_file_checks: 0 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.force_consistency_checks: 1 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.report_bg_io_stats: 0 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.ttl: 2592000 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.enable_blob_files: false 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.min_blob_size: 0 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.blob_file_size: 268435456 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-15T15:46:22.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.blob_file_starting_level: 0 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi144/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 90e1ecc4-6a35-411e-8195-7ed58cc7c8dc 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715787982084859, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715787982085454, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715787982, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "90e1ecc4-6a35-411e-8195-7ed58cc7c8dc", "db_session_id": "YVVMJK03HH8TKGN1YRA6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715787982085562, "job": 1, "event": "recovery_finished"} 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi144/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-15T15:46:22.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f8bdd26000 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: DB pointer 0x55f8bdd14000 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** DB Stats ** 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** Compaction Stats [default] ** 2024-05-15T15:46:22.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** Compaction Stats [default] ** 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-15T15:46:22.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative compaction: 0.00 GB write, 0.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval compaction: 0.00 GB write, 0.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Block cache BinnedLRUCache@0x55f8bdc891f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144 does not exist in monmap, will attempt to join an existing cluster 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: using public_addr v2:172.21.15.144:0/0 -> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: starting mon.smithi144 rank -1 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi144 fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(???) e0 preinit fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:46:22.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).mds e1 new map 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).mds e1 print_map 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: e1 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: legacy client fscid: -1 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: No filesystems configured 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-15T15:46:22.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying daemon node-exporter.smithi049 on smithi049 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying cephadm binary to smithi144 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Added host smithi144 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying daemon alertmanager.smithi049 on smithi049 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T15:46:22.413 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/58006981' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/58006981' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: osdmap e4: 0 total, 0 up, 0 in 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi049:172.21.15.49=smithi049;smithi144:172.21.15.144=smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Saving service mon spec with placement smithi049:172.21.15.49=smithi049;smithi144:172.21.15.144=smithi144;count:2 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.414 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Regenerating cephadm self-signed grafana TLS certificates 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying daemon grafana.smithi049 on smithi049 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/11941287' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/3418882869' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/2989250585' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/4178710756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.416 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying daemon prometheus.smithi049 on smithi049 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/2675075870' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/3914444749' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/3444587491' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/4170683670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14162 172.21.15.49:0/9788814' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mgrmap e14: smithi049.vuhzgf(active, since 64s) 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/128278054' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/2203760742' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Active manager daemon smithi049.vuhzgf restarted 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Activating manager daemon smithi049.vuhzgf 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: osdmap e5: 0 total, 0 up, 0 in 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mgrmap e15: smithi049.vuhzgf(active, starting, since 0.00300442s) 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:46:22.418 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Manager daemon smithi049.vuhzgf is now available 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/trash_purge_schedule"}]: dispatch 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: [15/May/2024:15:45:51] ENGINE Bus STARTING 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/3580521852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: [15/May/2024:15:45:51] ENGINE Serving on https://172.21.15.49:7150 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: [15/May/2024:15:45:52] ENGINE Serving on http://172.21.15.49:8765 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: [15/May/2024:15:45:52] ENGINE Bus STARTED 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mgrmap e16: smithi049.vuhzgf(active, since 1.00609s) 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.419 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mgrmap e17: smithi049.vuhzgf(active, since 2s) 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/815279017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:22.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:46:22.421 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying daemon ceph-exporter.smithi144 on smithi144 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/4099001569' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/4022546003' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying daemon crash.smithi144 on smithi144 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying daemon node-exporter.smithi144 on smithi144 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/1033834848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/3977248711' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:22.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: Deploying daemon mon.smithi144 on smithi144 2024-05-15T15:46:22.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/3455433885' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:22.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:22.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:46:22.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:22.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 ceph-mon[36389]: mon.smithi144@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-05-15T15:46:22.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 bash[36335]: bf0fd6a44ac8f2bc27fa7a7960e690c0104915e5b2bd6edf77ed1353a9eb20c2 2024-05-15T15:46:22.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:22 smithi144 systemd[1]: Started Ceph mon.smithi144 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:46:27.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.127+0000 7f456cff9700 1 -- 172.21.15.144:0/3662485208 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1022 (secure 0 0 0) 0x7f4564026030 con 0x7f45701058d0 2024-05-15T15:46:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-15T15:46:27.130 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":2,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","modified":"2024-05-15T15:46:22.117585Z","created":"2024-05-15T15:43:32.672725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi049","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-15T15:46:27.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.130+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f45580388c0 msgr2=0x7f455803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:27.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.130+0000 7f457622d700 1 --2- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f45580388c0 0x7f455803ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4560006fd0 tx=0x7f4560006e40 comp rx=0 tx=0).stop 2024-05-15T15:46:27.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.130+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 msgr2=0x7f4570197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:27.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.130+0000 7f457622d700 1 --2- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 0x7f4570197090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4564005490 tx=0x7f4564005570 comp rx=0 tx=0).stop 2024-05-15T15:46:27.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.130+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 shutdown_connections 2024-05-15T15:46:27.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.130+0000 7f457622d700 1 --2- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f45580388c0 0x7f455803ad80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:27.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.130+0000 7f457622d700 1 --2- 172.21.15.144:0/3662485208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45701058d0 0x7f4570197090 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:27.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.130+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 >> 172.21.15.144:0/3662485208 conn(0x7f45700fb400 msgr2=0x7f45700fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:27.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.131+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 shutdown_connections 2024-05-15T15:46:27.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:46:27.131+0000 7f457622d700 1 -- 172.21.15.144:0/3662485208 wait complete. 2024-05-15T15:46:27.133 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 2 2024-05-15T15:46:27.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:27.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:46:27.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: mon.smithi049 calling monitor election 2024-05-15T15:46:27.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/3662485208' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: mon.smithi144 calling monitor election 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1869513969' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: mon.smithi049 is new leader, mons smithi049,smithi144 in quorum (ranks 0,1) 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: monmap e2: 2 mons at {smithi049=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],smithi144=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: fsmap 2024-05-15T15:46:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: osdmap e5: 0 total, 0 up, 0 in 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: mgrmap e17: smithi049.vuhzgf(active, since 35s) 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: overall HEALTH_OK 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1869513969' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1869513969' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1869513969' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:27.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: mon.smithi049 calling monitor election 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/3662485208' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: mon.smithi144 calling monitor election 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/1869513969' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: mon.smithi049 is new leader, mons smithi049,smithi144 in quorum (ranks 0,1) 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: monmap e2: 2 mons at {smithi049=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],smithi144=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: fsmap 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: osdmap e5: 0 total, 0 up, 0 in 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: mgrmap e17: smithi049.vuhzgf(active, since 35s) 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: overall HEALTH_OK 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/1869513969' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/1869513969' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/1869513969' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:27.770 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-15T15:46:27.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-15T15:46:27.985 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:46:28.358 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:28 smithi049 ceph-mon[28399]: mgrmap e18: smithi049.vuhzgf(active, since 35s), standbys: smithi144.whcjkr 2024-05-15T15:46:28.358 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:28 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi144.whcjkr", "id": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:46:28.358 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:28 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:46:28.358 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:28 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:28 smithi144 ceph-mon[36389]: mgrmap e18: smithi049.vuhzgf(active, since 35s), standbys: smithi144.whcjkr 2024-05-15T15:46:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:28 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi144.whcjkr", "id": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:46:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:28 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:46:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:28 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:46:29.299 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:29 smithi049 ceph-mon[28399]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:29.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:29 smithi144 ceph-mon[36389]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:29.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.484+0000 7f981000a700 1 -- 172.21.15.49:0/53170111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 msgr2=0x7f9808105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.484+0000 7f981000a700 1 --2- 172.21.15.49:0/53170111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 0x7f9808105bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9804009910 tx=0x7f9804009c20 comp rx=0 tx=0).stop 2024-05-15T15:46:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.484+0000 7f981000a700 1 -- 172.21.15.49:0/53170111 shutdown_connections 2024-05-15T15:46:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.484+0000 7f981000a700 1 --2- 172.21.15.49:0/53170111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 0x7f9808105bb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.484+0000 7f981000a700 1 -- 172.21.15.49:0/53170111 >> 172.21.15.49:0/53170111 conn(0x7f98080fb2e0 msgr2=0x7f98080fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.484+0000 7f981000a700 1 -- 172.21.15.49:0/53170111 shutdown_connections 2024-05-15T15:46:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.484+0000 7f981000a700 1 -- 172.21.15.49:0/53170111 wait complete. 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.485+0000 7f981000a700 1 Processor -- start 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.485+0000 7f981000a700 1 -- start start 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.485+0000 7f981000a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 0x7f9808195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.485+0000 7f981000a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98081957a0 0x7f9808199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.485+0000 7f981000a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9808195dc0 con 0x7f98081057d0 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.485+0000 7f981000a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9808195f30 con 0x7f98081957a0 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.485+0000 7f980dda6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 0x7f9808195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.485+0000 7f980dda6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 0x7f9808195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35420/0 (socket says 172.21.15.49:35420) 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.485+0000 7f980dda6700 1 -- 172.21.15.49:0/686483408 learned_addr learned my addr 172.21.15.49:0/686483408 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f980d5a5700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98081957a0 0x7f9808199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f980dda6700 1 -- 172.21.15.49:0/686483408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98081957a0 msgr2=0x7f9808199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:29.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f980dda6700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98081957a0 0x7f9808199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:29.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f980dda6700 1 -- 172.21.15.49:0/686483408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98040095f0 con 0x7f98081057d0 2024-05-15T15:46:29.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f980d5a5700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98081957a0 0x7f9808199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:46:29.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f980dda6700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 0x7f9808195260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9804005b40 tx=0x7f980400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:29.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f97feffd700 1 -- 172.21.15.49:0/686483408 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f980401d070 con 0x7f98081057d0 2024-05-15T15:46:29.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f97feffd700 1 -- 172.21.15.49:0/686483408 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9804003d50 con 0x7f98081057d0 2024-05-15T15:46:29.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980819a150 con 0x7f98081057d0 2024-05-15T15:46:29.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f97feffd700 1 -- 172.21.15.49:0/686483408 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9804017440 con 0x7f98081057d0 2024-05-15T15:46:29.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.486+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980819a6a0 con 0x7f98081057d0 2024-05-15T15:46:29.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.487+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980819a9d0 con 0x7f98081057d0 2024-05-15T15:46:29.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.488+0000 7f97feffd700 1 -- 172.21.15.49:0/686483408 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f9804003890 con 0x7f98081057d0 2024-05-15T15:46:29.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.489+0000 7f97feffd700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f97f406cdb0 0x7f97f406f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:29.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.489+0000 7f97feffd700 1 -- 172.21.15.49:0/686483408 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f980408c2c0 con 0x7f98081057d0 2024-05-15T15:46:29.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.489+0000 7f980d5a5700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f97f406cdb0 0x7f97f406f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:29.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.489+0000 7f980d5a5700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f97f406cdb0 0x7f97f406f270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f97f80079b0 tx=0x7f97f8008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:29.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.494+0000 7f97feffd700 1 -- 172.21.15.49:0/686483408 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f98040577f0 con 0x7f98081057d0 2024-05-15T15:46:29.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.701+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9808196570 con 0x7f98081057d0 2024-05-15T15:46:29.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.701+0000 7f97feffd700 1 -- 172.21.15.49:0/686483408 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f980408ba80 con 0x7f98081057d0 2024-05-15T15:46:29.704 INFO:teuthology.orchestra.run.smithi049.stdout:# minimal ceph.conf for c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:46:29.704 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-05-15T15:46:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:46:29.704 INFO:teuthology.orchestra.run.smithi049.stdout: mon_host = [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-05-15T15:46:29.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f97f406cdb0 msgr2=0x7f97f406f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:29.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f97f406cdb0 0x7f97f406f270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f97f80079b0 tx=0x7f97f8008040 comp rx=0 tx=0).stop 2024-05-15T15:46:29.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 msgr2=0x7f9808195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:29.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 0x7f9808195260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9804005b40 tx=0x7f980400bdc0 comp rx=0 tx=0).stop 2024-05-15T15:46:29.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 shutdown_connections 2024-05-15T15:46:29.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98081057d0 0x7f9808195260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:29.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f97f406cdb0 0x7f97f406f270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:29.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 --2- 172.21.15.49:0/686483408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98081957a0 0x7f9808199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:29.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 >> 172.21.15.49:0/686483408 conn(0x7f98080fb2e0 msgr2=0x7f98081012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:29.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 shutdown_connections 2024-05-15T15:46:29.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:29.705+0000 7f981000a700 1 -- 172.21.15.49:0/686483408 wait complete. 2024-05-15T15:46:30.331 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-15T15:46:30.331 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:46:30.331 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.conf 2024-05-15T15:46:30.361 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:46:30.361 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:46:30.429 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:46:30.430 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-05-15T15:46:30.457 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:46:30.457 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:46:30.523 INFO:tasks.cephadm:Deploying OSDs... 2024-05-15T15:46:30.523 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:46:30.523 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-05-15T15:46:30.540 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-15T15:46:30.540 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-05-15T15:46:30.596 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-15T15:46:30.596 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T15:46:30.597 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-15T15:46:30.597 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:46:30.597 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T15:46:30.597 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-15 15:45:42.336821578 +0000 2024-05-15T15:46:30.597 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-15 15:39:38.679369735 +0000 2024-05-15T15:46:30.597 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-15 15:39:38.679369735 +0000 2024-05-15T15:46:30.597 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-05-15 15:39:38.679369735 +0000 2024-05-15T15:46:30.597 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-15T15:46:30.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:30.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/686483408' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: Reconfiguring mon.smithi049 (unknown last config time)... 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:46:30.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:46:30.658 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:30.658 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:30 smithi144 ceph-mon[36389]: Reconfiguring daemon mon.smithi049 on smithi049 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/686483408' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:30.660 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.661 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.661 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.661 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.661 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:30.661 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: Reconfiguring mon.smithi049 (unknown last config time)... 2024-05-15T15:46:30.661 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:46:30.661 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:46:30.661 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:30.661 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:30 smithi049 ceph-mon[28399]: Reconfiguring daemon mon.smithi049 on smithi049 2024-05-15T15:46:30.662 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-15T15:46:30.662 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-15T15:46:30.662 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.6213e-05 s, 5.3 MB/s 2024-05-15T15:46:30.663 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-15T15:46:30.720 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-05-15T15:46:30.776 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-15T15:46:30.776 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T15:46:30.776 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-15T15:46:30.776 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:46:30.777 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T15:46:30.777 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-15 15:45:42.336821578 +0000 2024-05-15T15:46:30.777 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-15 15:39:39.020360171 +0000 2024-05-15T15:46:30.777 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-15 15:39:39.020360171 +0000 2024-05-15T15:46:30.777 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-05-15 15:39:39.020360171 +0000 2024-05-15T15:46:30.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-15T15:46:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-15T15:46:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-15T15:46:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000108892 s, 4.7 MB/s 2024-05-15T15:46:30.841 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-15T15:46:30.899 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-05-15T15:46:30.956 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-15T15:46:30.956 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T15:46:30.956 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-15T15:46:30.956 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:46:30.956 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T15:46:30.956 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-15 15:45:42.337821551 +0000 2024-05-15T15:46:30.956 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-15 15:39:39.372350299 +0000 2024-05-15T15:46:30.956 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-15 15:39:39.372350299 +0000 2024-05-15T15:46:30.956 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-05-15 15:39:39.372350299 +0000 2024-05-15T15:46:30.957 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-15T15:46:31.021 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-15T15:46:31.021 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-15T15:46:31.022 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.3852e-05 s, 5.5 MB/s 2024-05-15T15:46:31.023 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-15T15:46:31.082 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-05-15T15:46:31.138 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-15T15:46:31.138 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T15:46:31.138 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-15T15:46:31.138 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:46:31.138 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T15:46:31.138 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-15 15:45:42.337821551 +0000 2024-05-15T15:46:31.139 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-15 15:39:39.727340343 +0000 2024-05-15T15:46:31.139 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-15 15:39:39.727340343 +0000 2024-05-15T15:46:31.139 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-05-15 15:39:39.727340343 +0000 2024-05-15T15:46:31.139 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-15T15:46:31.202 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-15T15:46:31.202 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-15T15:46:31.202 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000107437 s, 4.8 MB/s 2024-05-15T15:46:31.204 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-15T15:46:31.261 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:46:31.261 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-05-15T15:46:31.280 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-15T15:46:31.281 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-05-15T15:46:31.338 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-15T15:46:31.338 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T15:46:31.338 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-15T15:46:31.339 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:46:31.339 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T15:46:31.339 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-15 15:46:20.534126048 +0000 2024-05-15T15:46:31.339 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-15 15:39:34.803140794 +0000 2024-05-15T15:46:31.339 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-15 15:39:34.803140794 +0000 2024-05-15T15:46:31.339 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-15 15:39:34.803140794 +0000 2024-05-15T15:46:31.339 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-15T15:46:31.405 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-15T15:46:31.405 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-15T15:46:31.405 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000216866 s, 2.4 MB/s 2024-05-15T15:46:31.406 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-15T15:46:31.464 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-05-15T15:46:31.521 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-15T15:46:31.521 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T15:46:31.521 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-15T15:46:31.521 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:46:31.522 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T15:46:31.522 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-15 15:46:20.534126048 +0000 2024-05-15T15:46:31.522 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-15 15:39:35.154135128 +0000 2024-05-15T15:46:31.522 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-15 15:39:35.154135128 +0000 2024-05-15T15:46:31.522 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-15 15:39:35.154135128 +0000 2024-05-15T15:46:31.522 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-15T15:46:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-15T15:46:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-15T15:46:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000204987 s, 2.5 MB/s 2024-05-15T15:46:31.588 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-15T15:46:31.645 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-05-15T15:46:31.702 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-15T15:46:31.702 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T15:46:31.702 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-15T15:46:31.702 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:46:31.702 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T15:46:31.703 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-15 15:46:20.535126032 +0000 2024-05-15T15:46:31.703 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-15 15:39:35.497129591 +0000 2024-05-15T15:46:31.703 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-15 15:39:35.497129591 +0000 2024-05-15T15:46:31.703 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-15 15:39:35.497129591 +0000 2024-05-15T15:46:31.703 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-15T15:46:31.766 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-15T15:46:31.767 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-15T15:46:31.767 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000200408 s, 2.6 MB/s 2024-05-15T15:46:31.768 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-15T15:46:31.824 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-05-15T15:46:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-15T15:46:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T15:46:31.881 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-15T15:46:31.881 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:46:31.881 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T15:46:31.881 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-15 15:46:20.535126032 +0000 2024-05-15T15:46:31.881 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-15 15:39:35.836124118 +0000 2024-05-15T15:46:31.881 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-15 15:39:35.836124118 +0000 2024-05-15T15:46:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-15 15:39:35.836124118 +0000 2024-05-15T15:46:31.881 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-15T15:46:31.944 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-15T15:46:31.944 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-15T15:46:31.944 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000194809 s, 2.6 MB/s 2024-05-15T15:46:31.945 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-15T15:46:32.002 INFO:tasks.cephadm:Deploying osd.0 on smithi049 with /dev/vg_nvme/lv_4... 2024-05-15T15:46:32.002 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-15T15:46:32.609 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:32 smithi049 ceph-mon[28399]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:32.610 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:32 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:32.610 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:32 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:32.610 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:32 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:32.610 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:32 smithi049 ceph-mon[28399]: Reconfiguring mgr.smithi049.vuhzgf (unknown last config time)... 2024-05-15T15:46:32.610 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:32 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi049.vuhzgf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:46:32.610 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:32 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:46:32.610 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:32 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:32.610 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:32 smithi049 ceph-mon[28399]: Reconfiguring daemon mgr.smithi049.vuhzgf on smithi049 2024-05-15T15:46:32.799 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:46:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:32 smithi144 ceph-mon[36389]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:32 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:32 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:32 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:32 smithi144 ceph-mon[36389]: Reconfiguring mgr.smithi049.vuhzgf (unknown last config time)... 2024-05-15T15:46:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:32 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi049.vuhzgf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:46:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:32 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:46:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:32 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:32.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:32 smithi144 ceph-mon[36389]: Reconfiguring daemon mgr.smithi049.vuhzgf on smithi049 2024-05-15T15:46:35.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:34 smithi049 ceph-mon[28399]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:35.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:34 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:35.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:34 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:35.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:34 smithi049 ceph-mon[28399]: Reconfiguring ceph-exporter.smithi049 (monmap changed)... 2024-05-15T15:46:35.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:34 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:46:35.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:34 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:35.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:34 smithi049 ceph-mon[28399]: Reconfiguring daemon ceph-exporter.smithi049 on smithi049 2024-05-15T15:46:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:34 smithi144 ceph-mon[36389]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:34 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:34 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:35.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:34 smithi144 ceph-mon[36389]: Reconfiguring ceph-exporter.smithi049 (monmap changed)... 2024-05-15T15:46:35.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:34 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:46:35.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:34 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:35.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:34 smithi144 ceph-mon[36389]: Reconfiguring daemon ceph-exporter.smithi049 on smithi049 2024-05-15T15:46:36.176 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-15T15:46:36.176 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-15T15:46:36.176 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-05-15T15:46:36.176 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-05-15T15:46:36.176 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0083688 s, 1.3 GB/s 2024-05-15T15:46:36.176 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-05-15T15:46:36.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:36 smithi049 ceph-mon[28399]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:36.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:36.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:36.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:36 smithi049 ceph-mon[28399]: Reconfiguring crash.smithi049 (monmap changed)... 2024-05-15T15:46:36.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi049", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:46:36.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:36.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:36 smithi049 ceph-mon[28399]: Reconfiguring daemon crash.smithi049 on smithi049 2024-05-15T15:46:36.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:46:36.907 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_4 2024-05-15T15:46:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:36 smithi144 ceph-mon[36389]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:36 smithi144 ceph-mon[36389]: Reconfiguring crash.smithi049 (monmap changed)... 2024-05-15T15:46:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi049", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:46:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:36 smithi144 ceph-mon[36389]: Reconfiguring daemon crash.smithi049 on smithi049 2024-05-15T15:46:36.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:46:37.888 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:46:38.871 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:38 smithi049 ceph-mon[28399]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:38.871 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:38 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:38.871 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:38 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:38.871 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:38 smithi049 ceph-mon[28399]: Reconfiguring alertmanager.smithi049 (dependencies changed)... 2024-05-15T15:46:38.872 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:38 smithi049 ceph-mon[28399]: Reconfiguring daemon alertmanager.smithi049 on smithi049 2024-05-15T15:46:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:38 smithi144 ceph-mon[36389]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:38 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:38 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:38 smithi144 ceph-mon[36389]: Reconfiguring alertmanager.smithi049 (dependencies changed)... 2024-05-15T15:46:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:38 smithi144 ceph-mon[36389]: Reconfiguring daemon alertmanager.smithi049 on smithi049 2024-05-15T15:46:40.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:40 smithi049 ceph-mon[28399]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:40 smithi144 ceph-mon[36389]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:41.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.279+0000 7fceaaf9f700 1 -- 172.21.15.49:0/3329223472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 msgr2=0x7fcea40ff970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:41.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.279+0000 7fceaaf9f700 1 --2- 172.21.15.49:0/3329223472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 0x7fcea40ff970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fce98009910 tx=0x7fce98009c20 comp rx=0 tx=0).stop 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.279+0000 7fceaaf9f700 1 -- 172.21.15.49:0/3329223472 shutdown_connections 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.279+0000 7fceaaf9f700 1 --2- 172.21.15.49:0/3329223472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcea40ffeb0 0x7fcea4100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.279+0000 7fceaaf9f700 1 --2- 172.21.15.49:0/3329223472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 0x7fcea40ff970 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.279+0000 7fceaaf9f700 1 -- 172.21.15.49:0/3329223472 >> 172.21.15.49:0/3329223472 conn(0x7fcea40fb170 msgr2=0x7fcea40fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.279+0000 7fceaaf9f700 1 -- 172.21.15.49:0/3329223472 shutdown_connections 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.279+0000 7fceaaf9f700 1 -- 172.21.15.49:0/3329223472 wait complete. 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fceaaf9f700 1 Processor -- start 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fceaaf9f700 1 -- start start 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fceaaf9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 0x7fcea4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fceaaf9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcea40ffeb0 0x7fcea4198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fceaaf9f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4199470 con 0x7fcea40ff550 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fceaaf9f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41995b0 con 0x7fcea40ffeb0 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fcea8d3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 0x7fcea4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:41.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fcea8d3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 0x7fcea4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39994/0 (socket says 172.21.15.49:39994) 2024-05-15T15:46:41.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fcea8d3b700 1 -- 172.21.15.49:0/1950277305 learned_addr learned my addr 172.21.15.49:0/1950277305 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:46:41.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fcea3fff700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcea40ffeb0 0x7fcea4198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:41.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fcea8d3b700 1 -- 172.21.15.49:0/1950277305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcea40ffeb0 msgr2=0x7fcea4198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:46:41.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fcea8d3b700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcea40ffeb0 0x7fcea4198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:46:41.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fcea8d3b700 1 -- 172.21.15.49:0/1950277305 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce980095f0 con 0x7fcea40ff550 2024-05-15T15:46:41.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.280+0000 7fcea8d3b700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 0x7fcea4198910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fce98006010 tx=0x7fce9800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:41.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.281+0000 7fcea1ffb700 1 -- 172.21.15.49:0/1950277305 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9801d070 con 0x7fcea40ff550 2024-05-15T15:46:41.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.281+0000 7fcea1ffb700 1 -- 172.21.15.49:0/1950277305 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce98005dc0 con 0x7fcea40ff550 2024-05-15T15:46:41.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.281+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea419df60 con 0x7fcea40ff550 2024-05-15T15:46:41.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.281+0000 7fcea1ffb700 1 -- 172.21.15.49:0/1950277305 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce98021410 con 0x7fcea40ff550 2024-05-15T15:46:41.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.281+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea419e450 con 0x7fcea40ff550 2024-05-15T15:46:41.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.282+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea4192a40 con 0x7fcea40ff550 2024-05-15T15:46:41.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.283+0000 7fcea1ffb700 1 -- 172.21.15.49:0/1950277305 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7fce9800f460 con 0x7fcea40ff550 2024-05-15T15:46:41.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.283+0000 7fcea1ffb700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce9406cc50 0x7fce9406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:46:41.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.283+0000 7fcea1ffb700 1 -- 172.21.15.49:0/1950277305 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fce9808c310 con 0x7fcea40ff550 2024-05-15T15:46:41.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.283+0000 7fcea3fff700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce9406cc50 0x7fce9406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:46:41.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.284+0000 7fcea3fff700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce9406cc50 0x7fce9406f110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce90005950 tx=0x7fce9000b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:46:41.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.288+0000 7fcea1ffb700 1 -- 172.21.15.49:0/1950277305 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fce9805b2d0 con 0x7fcea40ff550 2024-05-15T15:46:41.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:46:41.507+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcea402cf00 con 0x7fce9406cc50 2024-05-15T15:46:41.975 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:46:41.976 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:46:41.976 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:42.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:46:42.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:46:42.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:43.036 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:42 smithi049 ceph-mon[28399]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:43.037 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:42 smithi049 ceph-mon[28399]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:46:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:42 smithi144 ceph-mon[36389]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:42 smithi144 ceph-mon[36389]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:46:45.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:44 smithi049 ceph-mon[28399]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:45.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:44 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:45.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:44 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:44 smithi144 ceph-mon[36389]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:44 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:44 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:46.175 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:45 smithi049 ceph-mon[28399]: Reconfiguring grafana.smithi049 (dependencies changed)... 2024-05-15T15:46:46.175 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:45 smithi049 ceph-mon[28399]: Reconfiguring daemon grafana.smithi049 on smithi049 2024-05-15T15:46:46.175 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:45 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/488487658' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25d1eb06-bee2-48a6-8d86-30b5b3b348d8"}]: dispatch 2024-05-15T15:46:46.175 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:45 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/488487658' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25d1eb06-bee2-48a6-8d86-30b5b3b348d8"}]': finished 2024-05-15T15:46:46.175 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:45 smithi049 ceph-mon[28399]: osdmap e6: 1 total, 0 up, 1 in 2024-05-15T15:46:46.175 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:46:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:45 smithi144 ceph-mon[36389]: Reconfiguring grafana.smithi049 (dependencies changed)... 2024-05-15T15:46:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:45 smithi144 ceph-mon[36389]: Reconfiguring daemon grafana.smithi049 on smithi049 2024-05-15T15:46:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:45 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/488487658' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25d1eb06-bee2-48a6-8d86-30b5b3b348d8"}]: dispatch 2024-05-15T15:46:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:45 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/488487658' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25d1eb06-bee2-48a6-8d86-30b5b3b348d8"}]': finished 2024-05-15T15:46:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:45 smithi144 ceph-mon[36389]: osdmap e6: 1 total, 0 up, 1 in 2024-05-15T15:46:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:46:47.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:46 smithi144 ceph-mon[36389]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:47.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:46 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3326473278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:46:47.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:46 smithi049 ceph-mon[28399]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:47.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:46 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3326473278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:46:49.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:48 smithi144 ceph-mon[36389]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:49.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:48 smithi049 ceph-mon[28399]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:51.123 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:50 smithi049 ceph-mon[28399]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:50 smithi144 ceph-mon[36389]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:52.071 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:46:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:46:53.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:52 smithi049 ceph-mon[28399]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:53.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:52 smithi144 ceph-mon[36389]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:54 smithi144 ceph-mon[36389]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:54 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:54 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:54 smithi049 ceph-mon[28399]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:54 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:54 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:46:56.117 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:55 smithi049 ceph-mon[28399]: Reconfiguring prometheus.smithi049 (dependencies changed)... 2024-05-15T15:46:56.117 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:55 smithi049 ceph-mon[28399]: Reconfiguring daemon prometheus.smithi049 on smithi049 2024-05-15T15:46:56.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:55 smithi144 ceph-mon[36389]: Reconfiguring prometheus.smithi049 (dependencies changed)... 2024-05-15T15:46:56.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:55 smithi144 ceph-mon[36389]: Reconfiguring daemon prometheus.smithi049 on smithi049 2024-05-15T15:46:57.173 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:56 smithi049 ceph-mon[28399]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:57.174 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:56 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-15T15:46:57.174 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:56 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:57.174 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:56 smithi049 ceph-mon[28399]: Deploying daemon osd.0 on smithi049 2024-05-15T15:46:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:56 smithi144 ceph-mon[36389]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:56 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-15T15:46:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:56 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:46:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:56 smithi144 ceph-mon[36389]: Deploying daemon osd.0 on smithi049 2024-05-15T15:46:59.194 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:46:58 smithi049 ceph-mon[28399]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:46:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:46:58 smithi144 ceph-mon[36389]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:01.271 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:01 smithi144 ceph-mon[36389]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:01.271 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:01.271 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:01.271 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:01 smithi144 ceph-mon[36389]: Reconfiguring ceph-exporter.smithi144 (monmap changed)... 2024-05-15T15:47:01.271 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:47:01.271 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:01.271 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:01 smithi144 ceph-mon[36389]: Reconfiguring daemon ceph-exporter.smithi144 on smithi144 2024-05-15T15:47:01.297 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:01 smithi049 ceph-mon[28399]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:01.298 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:01.298 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:01.298 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:01 smithi049 ceph-mon[28399]: Reconfiguring ceph-exporter.smithi144 (monmap changed)... 2024-05-15T15:47:01.298 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:47:01.298 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:01.298 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:01 smithi049 ceph-mon[28399]: Reconfiguring daemon ceph-exporter.smithi144 on smithi144 2024-05-15T15:47:02.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:02 smithi144 ceph-mon[36389]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:02.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:02.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:02.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:02 smithi144 ceph-mon[36389]: Reconfiguring crash.smithi144 (monmap changed)... 2024-05-15T15:47:02.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:47:02.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:02.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:02 smithi144 ceph-mon[36389]: Reconfiguring daemon crash.smithi144 on smithi144 2024-05-15T15:47:02.900 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:02 smithi049 ceph-mon[28399]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:02.901 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:02.901 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:02.901 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:02 smithi049 ceph-mon[28399]: Reconfiguring crash.smithi144 (monmap changed)... 2024-05-15T15:47:02.901 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:47:02.901 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:02.901 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:02 smithi049 ceph-mon[28399]: Reconfiguring daemon crash.smithi144 on smithi144 2024-05-15T15:47:04.365 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:04.365 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:04.365 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:04 smithi049 ceph-mon[28399]: Reconfiguring mgr.smithi144.whcjkr (monmap changed)... 2024-05-15T15:47:04.366 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:47:04.366 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:47:04.366 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:04.366 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:04 smithi049 ceph-mon[28399]: Reconfiguring daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:47:04.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:04.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:04.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:04 smithi144 ceph-mon[36389]: Reconfiguring mgr.smithi144.whcjkr (monmap changed)... 2024-05-15T15:47:04.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:47:04.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:47:04.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:04.412 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:04 smithi144 ceph-mon[36389]: Reconfiguring daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:47:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:05 smithi144 ceph-mon[36389]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:05 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:05 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:05 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:47:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:05 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:47:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:05 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:05 smithi049 ceph-mon[28399]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:47:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:47:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: Reconfiguring mon.smithi144 (monmap changed)... 2024-05-15T15:47:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: Reconfiguring daemon mon.smithi144 on smithi144 2024-05-15T15:47:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"}]: dispatch 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi049.front.sepia.ceph.com:3000"}]: dispatch 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi049.front.sepia.ceph.com:9095"}]: dispatch 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:47:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: Reconfiguring mon.smithi144 (monmap changed)... 2024-05-15T15:47:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: Reconfiguring daemon mon.smithi144 on smithi144 2024-05-15T15:47:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"}]: dispatch 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi049.front.sepia.ceph.com:3000"}]: dispatch 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi049.front.sepia.ceph.com:9095"}]: dispatch 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:47:07.405 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:07 smithi049 ceph-mon[28399]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:07.405 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:07 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-15T15:47:07.405 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:07 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"}]: dispatch 2024-05-15T15:47:07.405 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:07 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-15T15:47:07.405 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:07 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi049.front.sepia.ceph.com:3000"}]: dispatch 2024-05-15T15:47:07.405 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:07 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:47:07.405 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:07 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi049.front.sepia.ceph.com:9095"}]: dispatch 2024-05-15T15:47:07.405 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:07 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:47:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:07 smithi144 ceph-mon[36389]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:07 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-15T15:47:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:07 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"}]: dispatch 2024-05-15T15:47:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:07 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-15T15:47:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:07 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi049.front.sepia.ceph.com:3000"}]: dispatch 2024-05-15T15:47:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:07 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:47:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:07 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi049.front.sepia.ceph.com:9095"}]: dispatch 2024-05-15T15:47:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:07 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:47:08.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.733+0000 7fcea1ffb700 1 -- 172.21.15.49:0/1950277305 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcea402cf00 con 0x7fce9406cc50 2024-05-15T15:47:08.736 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 0 on host 'smithi049' 2024-05-15T15:47:08.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce9406cc50 msgr2=0x7fce9406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:47:08.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce9406cc50 0x7fce9406f110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce90005950 tx=0x7fce9000b410 comp rx=0 tx=0).stop 2024-05-15T15:47:08.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 msgr2=0x7fcea4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:47:08.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 0x7fcea4198910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fce98006010 tx=0x7fce9800bd70 comp rx=0 tx=0).stop 2024-05-15T15:47:08.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 shutdown_connections 2024-05-15T15:47:08.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea40ff550 0x7fcea4198910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:08.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce9406cc50 0x7fce9406f110 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:08.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 --2- 172.21.15.49:0/1950277305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcea40ffeb0 0x7fcea4198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:08.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 >> 172.21.15.49:0/1950277305 conn(0x7fcea40fb170 msgr2=0x7fcea4107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:47:08.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 shutdown_connections 2024-05-15T15:47:08.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:08.736+0000 7fceaaf9f700 1 -- 172.21.15.49:0/1950277305 wait complete. 2024-05-15T15:47:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:08 smithi049 ceph-mon[28399]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:08.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:08.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:08.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:08.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:08 smithi144 ceph-mon[36389]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:08 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:08 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:08 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:08 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:09.230 DEBUG:teuthology.orchestra.run.smithi049:osd.0> sudo journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.0.service 2024-05-15T15:47:09.233 INFO:tasks.cephadm:Deploying osd.1 on smithi049 with /dev/vg_nvme/lv_3... 2024-05-15T15:47:09.233 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-15T15:47:09.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:09 smithi049 ceph-mon[28399]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-15T15:47:10.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:09 smithi144 ceph-mon[36389]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-15T15:47:10.629 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:47:11.068 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:10 smithi049 ceph-mon[28399]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:11.069 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:10 smithi049 ceph-mon[28399]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-15T15:47:11.069 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:10 smithi049 ceph-mon[28399]: osdmap e7: 1 total, 0 up, 1 in 2024-05-15T15:47:11.069 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:10 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:11.069 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:10 smithi049 ceph-mon[28399]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T15:47:11.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:10 smithi144 ceph-mon[36389]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:11.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:10 smithi144 ceph-mon[36389]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-15T15:47:11.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:10 smithi144 ceph-mon[36389]: osdmap e7: 1 total, 0 up, 1 in 2024-05-15T15:47:11.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:10 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:11.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:10 smithi144 ceph-mon[36389]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T15:47:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:11 smithi049 ceph-mon[28399]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-05-15T15:47:11.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:11 smithi049 ceph-mon[28399]: osdmap e8: 1 total, 0 up, 1 in 2024-05-15T15:47:11.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:11.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:11.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:11 smithi049 ceph-mon[28399]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' 2024-05-15T15:47:11.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:47:11 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[47019]: 2024-05-15T15:47:11.635+0000 7f6b8288e700 -1 osd.0 0 waiting for initial osdmap 2024-05-15T15:47:11.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:47:11 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[47019]: 2024-05-15T15:47:11.639+0000 7f6b79679700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T15:47:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:11 smithi144 ceph-mon[36389]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-05-15T15:47:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:11 smithi144 ceph-mon[36389]: osdmap e8: 1 total, 0 up, 1 in 2024-05-15T15:47:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:11 smithi144 ceph-mon[36389]: from='osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370]' entity='osd.0' 2024-05-15T15:47:12.904 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:12 smithi049 ceph-mon[28399]: purged_snaps scrub starts 2024-05-15T15:47:12.904 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:12 smithi049 ceph-mon[28399]: purged_snaps scrub ok 2024-05-15T15:47:12.904 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:12 smithi049 ceph-mon[28399]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:12.904 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:12.904 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:12 smithi049 ceph-mon[28399]: osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] boot 2024-05-15T15:47:12.904 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:12 smithi049 ceph-mon[28399]: osdmap e9: 1 total, 1 up, 1 in 2024-05-15T15:47:12.905 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:12.905 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-15T15:47:12.905 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-15T15:47:12.905 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-05-15T15:47:12.905 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-05-15T15:47:12.905 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00922792 s, 1.1 GB/s 2024-05-15T15:47:12.905 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-05-15T15:47:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:12 smithi144 ceph-mon[36389]: purged_snaps scrub starts 2024-05-15T15:47:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:12 smithi144 ceph-mon[36389]: purged_snaps scrub ok 2024-05-15T15:47:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:12 smithi144 ceph-mon[36389]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T15:47:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:12 smithi144 ceph-mon[36389]: osd.0 [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] boot 2024-05-15T15:47:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:12 smithi144 ceph-mon[36389]: osdmap e9: 1 total, 1 up, 1 in 2024-05-15T15:47:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:47:13.531 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_3 2024-05-15T15:47:14.377 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:47:15.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:14 smithi144 ceph-mon[36389]: pgmap v38: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:15.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:14 smithi144 ceph-mon[36389]: osdmap e10: 1 total, 1 up, 1 in 2024-05-15T15:47:15.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:14 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:15.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:14 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:15.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:14 smithi049 ceph-mon[28399]: pgmap v38: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:15.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:14 smithi049 ceph-mon[28399]: osdmap e10: 1 total, 1 up, 1 in 2024-05-15T15:47:15.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:15.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:14 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.876+0000 7ff678caf700 1 -- 172.21.15.49:0/2027711326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 msgr2=0x7ff674101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:47:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.876+0000 7ff678caf700 1 --2- 172.21.15.49:0/2027711326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 0x7ff674101110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff664009910 tx=0x7ff664009c20 comp rx=0 tx=0).stop 2024-05-15T15:47:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.876+0000 7ff678caf700 1 -- 172.21.15.49:0/2027711326 shutdown_connections 2024-05-15T15:47:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.876+0000 7ff678caf700 1 --2- 172.21.15.49:0/2027711326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff674101ef0 0x7ff674102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.876+0000 7ff678caf700 1 --2- 172.21.15.49:0/2027711326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 0x7ff674101110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.876+0000 7ff678caf700 1 -- 172.21.15.49:0/2027711326 >> 172.21.15.49:0/2027711326 conn(0x7ff6740fc290 msgr2=0x7ff6740fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.876+0000 7ff678caf700 1 -- 172.21.15.49:0/2027711326 shutdown_connections 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.876+0000 7ff678caf700 1 -- 172.21.15.49:0/2027711326 wait complete. 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.877+0000 7ff678caf700 1 Processor -- start 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.877+0000 7ff678caf700 1 -- start start 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.877+0000 7ff678caf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 0x7ff674196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.877+0000 7ff678caf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff674101ef0 0x7ff674196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.877+0000 7ff678caf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6741973a0 con 0x7ff674100cf0 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.877+0000 7ff678caf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6741974e0 con 0x7ff674101ef0 2024-05-15T15:47:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff67259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 0x7ff674196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff67259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 0x7ff674196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41288/0 (socket says 172.21.15.49:41288) 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff67259c700 1 -- 172.21.15.49:0/525478164 learned_addr learned my addr 172.21.15.49:0/525478164 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff67259c700 1 -- 172.21.15.49:0/525478164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff674101ef0 msgr2=0x7ff674196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff67259c700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff674101ef0 0x7ff674196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff67259c700 1 -- 172.21.15.49:0/525478164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640095f0 con 0x7ff674100cf0 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff67259c700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 0x7ff674196840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff6640059d0 tx=0x7ff66400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff6637fe700 1 -- 172.21.15.49:0/525478164 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff66401d070 con 0x7ff674100cf0 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff6637fe700 1 -- 172.21.15.49:0/525478164 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff66400f460 con 0x7ff674100cf0 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67419be90 con 0x7ff674100cf0 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff6637fe700 1 -- 172.21.15.49:0/525478164 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff664021620 con 0x7ff674100cf0 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.878+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67419c3b0 con 0x7ff674100cf0 2024-05-15T15:47:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.879+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6741908f0 con 0x7ff674100cf0 2024-05-15T15:47:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.880+0000 7ff6637fe700 1 -- 172.21.15.49:0/525478164 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7ff66400fa70 con 0x7ff674100cf0 2024-05-15T15:47:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.880+0000 7ff6637fe700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff65c06cc60 0x7ff65c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:47:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.880+0000 7ff6637fe700 1 -- 172.21.15.49:0/525478164 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7ff66408c650 con 0x7ff674100cf0 2024-05-15T15:47:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.880+0000 7ff671d9b700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff65c06cc60 0x7ff65c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:47:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.881+0000 7ff671d9b700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff65c06cc60 0x7ff65c06f120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6680097b0 tx=0x7ff668006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:47:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:15.885+0000 7ff6637fe700 1 -- 172.21.15.49:0/525478164 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff66405b3b0 con 0x7ff674100cf0 2024-05-15T15:47:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:16.104+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff67402cf50 con 0x7ff65c06cc60 2024-05-15T15:47:16.918 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:16 smithi049 ceph-mon[28399]: pgmap v40: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:16.918 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:16 smithi049 ceph-mon[28399]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:47:16.918 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:47:16.918 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:47:16.918 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:16 smithi144 ceph-mon[36389]: pgmap v40: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:16 smithi144 ceph-mon[36389]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:47:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:47:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:47:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:19.073 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:18 smithi049 ceph-mon[28399]: pgmap v41: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:18 smithi144 ceph-mon[36389]: pgmap v41: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:20.265 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: Detected new or changed devices on smithi049 2024-05-15T15:47:20.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:20.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:20.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:47:20.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: Adjusting osd_memory_target on smithi049 to 11840M 2024-05-15T15:47:20.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:20.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:20.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:47:20.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:20.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:47:20.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: Detected new or changed devices on smithi049 2024-05-15T15:47:20.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:20.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:20.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:47:20.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: Adjusting osd_memory_target on smithi049 to 11840M 2024-05-15T15:47:20.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:20.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:20.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:47:20.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:20.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:47:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:21 smithi144 ceph-mon[36389]: pgmap v42: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:21 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1606620131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0e717cf-ae5c-4fa4-a30d-d2235ff27315"}]: dispatch 2024-05-15T15:47:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:21 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1606620131' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0e717cf-ae5c-4fa4-a30d-d2235ff27315"}]': finished 2024-05-15T15:47:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:21 smithi144 ceph-mon[36389]: osdmap e11: 2 total, 1 up, 2 in 2024-05-15T15:47:21.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:21 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:21.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:21 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/4135648827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:47:21.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:21 smithi049 ceph-mon[28399]: pgmap v42: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:21.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:21 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1606620131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0e717cf-ae5c-4fa4-a30d-d2235ff27315"}]: dispatch 2024-05-15T15:47:21.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:21 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1606620131' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0e717cf-ae5c-4fa4-a30d-d2235ff27315"}]': finished 2024-05-15T15:47:21.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:21 smithi049 ceph-mon[28399]: osdmap e11: 2 total, 1 up, 2 in 2024-05-15T15:47:21.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:21 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:21.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:21 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/4135648827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:47:22.489 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:22.489 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:22.489 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:47:22.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:22.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:22.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:47:23.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:23 smithi049 ceph-mon[28399]: pgmap v44: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:23.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:23 smithi144 ceph-mon[36389]: pgmap v44: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:24.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:24 smithi144 ceph-mon[36389]: pgmap v45: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:24.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:24.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:24.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:24.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:47:24.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:24.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:24 smithi049 ceph-mon[28399]: pgmap v45: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:24.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:24.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:24.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:24.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:47:24.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:26.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:26 smithi144 ceph-mon[36389]: pgmap v46: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:26.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:26 smithi049 ceph-mon[28399]: pgmap v46: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:27.809 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-15T15:47:27.809 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-15T15:47:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:28.841 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:28 smithi049 ceph-mon[28399]: Deploying daemon osd.1 on smithi049 2024-05-15T15:47:28.842 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:28 smithi049 ceph-mon[28399]: pgmap v47: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:28.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:28 smithi144 ceph-mon[36389]: Deploying daemon osd.1 on smithi049 2024-05-15T15:47:28.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:28 smithi144 ceph-mon[36389]: pgmap v47: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:30.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:30 smithi144 ceph-mon[36389]: pgmap v48: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:30.842 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:30 smithi049 ceph-mon[28399]: pgmap v48: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:32.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:32 smithi049 ceph-mon[28399]: pgmap v49: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:32 smithi144 ceph-mon[36389]: pgmap v49: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:33.846 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:33 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:33.846 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:33 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:33.846 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:33 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:47:33.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:33 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:33.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:33 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:33.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:33 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:47:34.650 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:34 smithi049 ceph-mon[28399]: pgmap v50: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:34.650 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:34 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:34.650 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:34 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:34.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:34 smithi144 ceph-mon[36389]: pgmap v50: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:34.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:34 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:34.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:34 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:36.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.457+0000 7ff6637fe700 1 -- 172.21.15.49:0/525478164 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff67402cf50 con 0x7ff65c06cc60 2024-05-15T15:47:36.460 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 1 on host 'smithi049' 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff65c06cc60 msgr2=0x7ff65c06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff65c06cc60 0x7ff65c06f120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6680097b0 tx=0x7ff668006d20 comp rx=0 tx=0).stop 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 msgr2=0x7ff674196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 0x7ff674196840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff6640059d0 tx=0x7ff66400be20 comp rx=0 tx=0).stop 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 shutdown_connections 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff674100cf0 0x7ff674196840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff65c06cc60 0x7ff65c06f120 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 --2- 172.21.15.49:0/525478164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff674101ef0 0x7ff674196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 >> 172.21.15.49:0/525478164 conn(0x7ff6740fc290 msgr2=0x7ff674105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:47:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 shutdown_connections 2024-05-15T15:47:36.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:36.459+0000 7ff678caf700 1 -- 172.21.15.49:0/525478164 wait complete. 2024-05-15T15:47:36.655 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:36 smithi049 ceph-mon[28399]: pgmap v51: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:36.655 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:36.655 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:36.655 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:47:36.655 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:36 smithi049 ceph-mon[28399]: from='osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T15:47:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:36 smithi144 ceph-mon[36389]: pgmap v51: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:47:36.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:36 smithi144 ceph-mon[36389]: from='osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T15:47:37.186 DEBUG:teuthology.orchestra.run.smithi049:osd.1> sudo journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.1.service 2024-05-15T15:47:37.188 INFO:tasks.cephadm:Deploying osd.2 on smithi049 with /dev/vg_nvme/lv_2... 2024-05-15T15:47:37.188 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-15T15:47:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:47:38.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:38 smithi144 ceph-mon[36389]: pgmap v52: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:38.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:38 smithi144 ceph-mon[36389]: from='osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-15T15:47:38.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:38 smithi144 ceph-mon[36389]: osdmap e12: 2 total, 1 up, 2 in 2024-05-15T15:47:38.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:38 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:38.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:38 smithi144 ceph-mon[36389]: from='osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T15:47:38.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:38 smithi049 ceph-mon[28399]: pgmap v52: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:38.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:38 smithi049 ceph-mon[28399]: from='osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-15T15:47:38.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:38 smithi049 ceph-mon[28399]: osdmap e12: 2 total, 1 up, 2 in 2024-05-15T15:47:38.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:38 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:38.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:38 smithi049 ceph-mon[28399]: from='osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T15:47:38.706 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:47:38 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[54740]: 2024-05-15T15:47:38.419+0000 7fb610a7c700 -1 osd.1 0 waiting for initial osdmap 2024-05-15T15:47:38.706 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:47:38 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[54740]: 2024-05-15T15:47:38.424+0000 7fb60d072700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T15:47:39.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:39 smithi144 ceph-mon[36389]: from='osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-05-15T15:47:39.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:39 smithi144 ceph-mon[36389]: osdmap e13: 2 total, 1 up, 2 in 2024-05-15T15:47:39.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:39 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:39.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:39 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:39.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:39 smithi049 ceph-mon[28399]: from='osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-05-15T15:47:39.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:39 smithi049 ceph-mon[28399]: osdmap e13: 2 total, 1 up, 2 in 2024-05-15T15:47:39.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:39 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:39.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:39 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:40 smithi144 ceph-mon[36389]: purged_snaps scrub starts 2024-05-15T15:47:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:40 smithi144 ceph-mon[36389]: purged_snaps scrub ok 2024-05-15T15:47:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:40 smithi144 ceph-mon[36389]: pgmap v55: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:40 smithi144 ceph-mon[36389]: osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] boot 2024-05-15T15:47:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:40 smithi144 ceph-mon[36389]: osdmap e14: 2 total, 2 up, 2 in 2024-05-15T15:47:40.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:40 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:40.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:40 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:40.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:40 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:40.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:40 smithi049 ceph-mon[28399]: purged_snaps scrub starts 2024-05-15T15:47:40.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:40 smithi049 ceph-mon[28399]: purged_snaps scrub ok 2024-05-15T15:47:40.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:40 smithi049 ceph-mon[28399]: pgmap v55: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-15T15:47:40.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:40 smithi049 ceph-mon[28399]: osd.1 [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] boot 2024-05-15T15:47:40.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:40 smithi049 ceph-mon[28399]: osdmap e14: 2 total, 2 up, 2 in 2024-05-15T15:47:40.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:40 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:47:40.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:40 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:40.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:40 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:41.001 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-15T15:47:41.001 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-15T15:47:41.001 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-05-15T15:47:41.001 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-05-15T15:47:41.002 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0106842 s, 981 MB/s 2024-05-15T15:47:41.002 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-05-15T15:47:41.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:41 smithi144 ceph-mon[36389]: osdmap e15: 2 total, 2 up, 2 in 2024-05-15T15:47:41.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:41 smithi049 ceph-mon[28399]: osdmap e15: 2 total, 2 up, 2 in 2024-05-15T15:47:42.268 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_2 2024-05-15T15:47:42.530 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:42 smithi049 ceph-mon[28399]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:42 smithi144 ceph-mon[36389]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:42.761 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:47:44.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.950+0000 7f4e35496700 1 -- 172.21.15.49:0/3318587095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30101ef0 msgr2=0x7f4e30102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:47:44.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.950+0000 7f4e35496700 1 --2- 172.21.15.49:0/3318587095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30101ef0 0x7f4e30102350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009910 tx=0x7f4e24009c20 comp rx=0 tx=0).stop 2024-05-15T15:47:44.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.950+0000 7f4e35496700 1 -- 172.21.15.49:0/3318587095 shutdown_connections 2024-05-15T15:47:44.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.950+0000 7f4e35496700 1 --2- 172.21.15.49:0/3318587095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30101ef0 0x7f4e30102350 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:44.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.950+0000 7f4e35496700 1 --2- 172.21.15.49:0/3318587095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e30100cf0 0x7f4e30101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:44.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.950+0000 7f4e35496700 1 -- 172.21.15.49:0/3318587095 >> 172.21.15.49:0/3318587095 conn(0x7f4e300fc290 msgr2=0x7f4e300fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:47:44.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.950+0000 7f4e35496700 1 -- 172.21.15.49:0/3318587095 shutdown_connections 2024-05-15T15:47:44.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.950+0000 7f4e35496700 1 -- 172.21.15.49:0/3318587095 wait complete. 2024-05-15T15:47:44.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.951+0000 7f4e35496700 1 Processor -- start 2024-05-15T15:47:44.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.951+0000 7f4e35496700 1 -- start start 2024-05-15T15:47:44.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e35496700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30100cf0 0x7f4e30196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:47:44.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e35496700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e30101ef0 0x7f4e30196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:47:44.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e35496700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e301971e0 con 0x7f4e30100cf0 2024-05-15T15:47:44.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e35496700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e30197320 con 0x7f4e30101ef0 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e2effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30100cf0 0x7f4e30196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e2effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30100cf0 0x7f4e30196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57312/0 (socket says 172.21.15.49:57312) 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e2effd700 1 -- 172.21.15.49:0/2420929305 learned_addr learned my addr 172.21.15.49:0/2420929305 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e2e7fc700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e30101ef0 0x7f4e30196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e2effd700 1 -- 172.21.15.49:0/2420929305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e30101ef0 msgr2=0x7f4e30196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e2effd700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e30101ef0 0x7f4e30196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e2effd700 1 -- 172.21.15.49:0/2420929305 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240095f0 con 0x7f4e30100cf0 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.952+0000 7f4e2e7fc700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e30101ef0 0x7f4e30196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.953+0000 7f4e2effd700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30100cf0 0x7f4e30196710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000c910 tx=0x7f4e2000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.953+0000 7f4e17fff700 1 -- 172.21.15.49:0/2420929305 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e200041f0 con 0x7f4e30100cf0 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.953+0000 7f4e17fff700 1 -- 172.21.15.49:0/2420929305 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e2000ce60 con 0x7f4e30100cf0 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.953+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3019bd40 con 0x7f4e30100cf0 2024-05-15T15:47:44.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.953+0000 7f4e17fff700 1 -- 172.21.15.49:0/2420929305 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e20003d90 con 0x7f4e30100cf0 2024-05-15T15:47:44.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.953+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3019c280 con 0x7f4e30100cf0 2024-05-15T15:47:44.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.954+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e30190870 con 0x7f4e30100cf0 2024-05-15T15:47:44.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.955+0000 7f4e17fff700 1 -- 172.21.15.49:0/2420929305 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f4e20007900 con 0x7f4e30100cf0 2024-05-15T15:47:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.955+0000 7f4e17fff700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4e1806cc60 0x7f4e1806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:47:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.955+0000 7f4e17fff700 1 -- 172.21.15.49:0/2420929305 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f4e2008b220 con 0x7f4e30100cf0 2024-05-15T15:47:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.955+0000 7f4e2e7fc700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4e1806cc60 0x7f4e1806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:47:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.956+0000 7f4e2e7fc700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4e1806cc60 0x7f4e1806f120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005090 tx=0x7f4e24005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:47:44.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:44.960+0000 7f4e17fff700 1 -- 172.21.15.49:0/2420929305 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4e20059d50 con 0x7f4e30100cf0 2024-05-15T15:47:45.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:47:45.187+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e3002cf50 con 0x7f4e1806cc60 2024-05-15T15:47:45.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: Detected new or changed devices on smithi049 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: Adjusting osd_memory_target on smithi049 to 5920M 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:45.203 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:45 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:47:45.396 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: Detected new or changed devices on smithi049 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: Adjusting osd_memory_target on smithi049 to 5920M 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:45.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:45 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:47:46.198 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:46 smithi049 ceph-mon[28399]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:47:46.198 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:47:46.198 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:47:46.198 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:46.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:46 smithi144 ceph-mon[36389]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:47:46.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:47:46.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:47:46.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:47.386 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:47 smithi049 ceph-mon[28399]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:47.386 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:47 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:47.386 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:47 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:47.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:47 smithi144 ceph-mon[36389]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:47.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:47 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:47.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:47 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:49.140 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:49 smithi049 ceph-mon[28399]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:49.140 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:49 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1482359459' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7752daf-dd5b-49cc-abc2-ca953968e542"}]: dispatch 2024-05-15T15:47:49.140 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:49 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1482359459' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7752daf-dd5b-49cc-abc2-ca953968e542"}]': finished 2024-05-15T15:47:49.140 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:49 smithi049 ceph-mon[28399]: osdmap e16: 3 total, 2 up, 3 in 2024-05-15T15:47:49.141 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:49 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:47:49.316 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:49 smithi144 ceph-mon[36389]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:49.316 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:49 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1482359459' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7752daf-dd5b-49cc-abc2-ca953968e542"}]: dispatch 2024-05-15T15:47:49.316 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:49 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1482359459' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7752daf-dd5b-49cc-abc2-ca953968e542"}]': finished 2024-05-15T15:47:49.316 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:49 smithi144 ceph-mon[36389]: osdmap e16: 3 total, 2 up, 3 in 2024-05-15T15:47:49.316 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:49 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:47:50.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:50 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/4231622721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:47:50.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:50 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/4231622721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:47:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:51 smithi049 ceph-mon[28399]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:51.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:51.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:51.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:51.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:47:51.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:51 smithi144 ceph-mon[36389]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:47:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:47:52.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:47:52.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:47:53.517 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:53 smithi049 ceph-mon[28399]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:53.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:53 smithi144 ceph-mon[36389]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:55.561 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:55 smithi049 ceph-mon[28399]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:55.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:55 smithi144 ceph-mon[36389]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:56.405 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:56 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-15T15:47:56.406 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:56 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:56.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:56 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-15T15:47:56.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:56 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:47:57.320 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:57 smithi049 ceph-mon[28399]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:57.320 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:57 smithi049 ceph-mon[28399]: Deploying daemon osd.2 on smithi049 2024-05-15T15:47:57.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:57 smithi144 ceph-mon[36389]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:57.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:57 smithi144 ceph-mon[36389]: Deploying daemon osd.2 on smithi049 2024-05-15T15:47:58.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:47:58 smithi049 ceph-mon[28399]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:47:58.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:47:58 smithi144 ceph-mon[36389]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:00.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:00 smithi049 ceph-mon[28399]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:00.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:00 smithi144 ceph-mon[36389]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:03.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:02 smithi049 ceph-mon[28399]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:02 smithi144 ceph-mon[36389]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:04.895 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 2 on host 'smithi049' 2024-05-15T15:48:04.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.892+0000 7f4e17fff700 1 -- 172.21.15.49:0/2420929305 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4e3002cf50 con 0x7f4e1806cc60 2024-05-15T15:48:04.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4e1806cc60 msgr2=0x7f4e1806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:04.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4e1806cc60 0x7f4e1806f120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005090 tx=0x7f4e24005960 comp rx=0 tx=0).stop 2024-05-15T15:48:04.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30100cf0 msgr2=0x7f4e30196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:04.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30100cf0 0x7f4e30196710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000c910 tx=0x7f4e2000cc20 comp rx=0 tx=0).stop 2024-05-15T15:48:04.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 shutdown_connections 2024-05-15T15:48:04.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e30100cf0 0x7f4e30196710 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:04.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4e1806cc60 0x7f4e1806f120 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:04.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 --2- 172.21.15.49:0/2420929305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e30101ef0 0x7f4e30196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:04.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 >> 172.21.15.49:0/2420929305 conn(0x7f4e300fc290 msgr2=0x7f4e301054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:48:04.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 shutdown_connections 2024-05-15T15:48:04.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:48:04.895+0000 7f4e35496700 1 -- 172.21.15.49:0/2420929305 wait complete. 2024-05-15T15:48:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:04 smithi049 ceph-mon[28399]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:04 smithi049 ceph-mon[28399]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T15:48:05.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:04 smithi144 ceph-mon[36389]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:05.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:05.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:05.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:04 smithi144 ceph-mon[36389]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T15:48:05.480 DEBUG:teuthology.orchestra.run.smithi049:osd.2> sudo journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.2.service 2024-05-15T15:48:05.483 INFO:tasks.cephadm:Deploying osd.3 on smithi144 with /dev/vg_nvme/lv_4... 2024-05-15T15:48:05.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-15T15:48:05.695 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T15:48:06.868 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:06 smithi049 ceph-mon[28399]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:06.869 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:06 smithi049 ceph-mon[28399]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-15T15:48:06.869 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:06 smithi049 ceph-mon[28399]: osdmap e17: 3 total, 2 up, 3 in 2024-05-15T15:48:06.869 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:06.869 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:06 smithi049 ceph-mon[28399]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T15:48:06.869 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:48:06.995 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:06 smithi144 ceph-mon[36389]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:06.995 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:06 smithi144 ceph-mon[36389]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-15T15:48:06.995 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:06 smithi144 ceph-mon[36389]: osdmap e17: 3 total, 2 up, 3 in 2024-05-15T15:48:06.995 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:06.995 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:06 smithi144 ceph-mon[36389]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T15:48:06.995 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:48:07.667 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-15T15:48:07.667 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-15T15:48:07.667 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-15T15:48:07.667 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-15T15:48:07.667 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013808 s, 759 MB/s 2024-05-15T15:48:07.668 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-15T15:48:07.869 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:07 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[62084]: 2024-05-15T15:48:07.612+0000 7f24f497f700 -1 osd.2 0 waiting for initial osdmap 2024-05-15T15:48:07.869 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:07 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[62084]: 2024-05-15T15:48:07.619+0000 7f24ecf6d700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T15:48:07.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:07 smithi144 ceph-mon[36389]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-05-15T15:48:07.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:07 smithi144 ceph-mon[36389]: osdmap e18: 3 total, 2 up, 3 in 2024-05-15T15:48:07.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:07 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:07.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:07 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:07.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:07 smithi144 ceph-mon[36389]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' 2024-05-15T15:48:07.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:07 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:07.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:07 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:08.196 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:07 smithi049 ceph-mon[28399]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-05-15T15:48:08.196 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:07 smithi049 ceph-mon[28399]: osdmap e18: 3 total, 2 up, 3 in 2024-05-15T15:48:08.196 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:07 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:08.197 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:07 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:08.197 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:07 smithi049 ceph-mon[28399]: from='osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409]' entity='osd.2' 2024-05-15T15:48:08.197 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:07 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:08.197 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:07 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:08.321 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-05-15T15:48:08.530 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T15:48:09.125 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:08 smithi049 ceph-mon[28399]: purged_snaps scrub starts 2024-05-15T15:48:09.125 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:08 smithi049 ceph-mon[28399]: purged_snaps scrub ok 2024-05-15T15:48:09.125 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:08 smithi049 ceph-mon[28399]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:09.125 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:09.125 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:08 smithi049 ceph-mon[28399]: osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] boot 2024-05-15T15:48:09.125 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:08 smithi049 ceph-mon[28399]: osdmap e19: 3 total, 3 up, 3 in 2024-05-15T15:48:09.125 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:08 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:08 smithi144 ceph-mon[36389]: purged_snaps scrub starts 2024-05-15T15:48:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:08 smithi144 ceph-mon[36389]: purged_snaps scrub ok 2024-05-15T15:48:09.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:08 smithi144 ceph-mon[36389]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-15T15:48:09.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:08 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:09.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:08 smithi144 ceph-mon[36389]: osd.2 [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] boot 2024-05-15T15:48:09.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:08 smithi144 ceph-mon[36389]: osdmap e19: 3 total, 3 up, 3 in 2024-05-15T15:48:09.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:08 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:48:10.026 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:09 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-15T15:48:10.027 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:09 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-15T15:48:10.027 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:09 smithi144 ceph-mon[36389]: osdmap e20: 3 total, 3 up, 3 in 2024-05-15T15:48:10.027 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:09 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-15T15:48:10.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.028+0000 7fca74dae700 1 -- 172.21.15.144:0/706541441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca700ffc50 msgr2=0x7fca70100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:10.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.028+0000 7fca74dae700 1 --2- 172.21.15.144:0/706541441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca700ffc50 0x7fca70100070 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fca64009910 tx=0x7fca64009c20 comp rx=0 tx=0).stop 2024-05-15T15:48:10.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.028+0000 7fca74dae700 1 -- 172.21.15.144:0/706541441 shutdown_connections 2024-05-15T15:48:10.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.028+0000 7fca74dae700 1 --2- 172.21.15.144:0/706541441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca70100e50 0x7fca701012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:10.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.028+0000 7fca74dae700 1 --2- 172.21.15.144:0/706541441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca700ffc50 0x7fca70100070 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:10.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.028+0000 7fca74dae700 1 -- 172.21.15.144:0/706541441 >> 172.21.15.144:0/706541441 conn(0x7fca700fb1d0 msgr2=0x7fca700fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:48:10.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.028+0000 7fca74dae700 1 -- 172.21.15.144:0/706541441 shutdown_connections 2024-05-15T15:48:10.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.028+0000 7fca74dae700 1 -- 172.21.15.144:0/706541441 wait complete. 2024-05-15T15:48:10.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca74dae700 1 Processor -- start 2024-05-15T15:48:10.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca74dae700 1 -- start start 2024-05-15T15:48:10.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca74dae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca700ffc50 0x7fca70196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:48:10.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca74dae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70100e50 0x7fca70196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:48:10.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca74dae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca701972c0 con 0x7fca700ffc50 2024-05-15T15:48:10.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca74dae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca70197400 con 0x7fca70100e50 2024-05-15T15:48:10.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca6dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70100e50 0x7fca70196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca6dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70100e50 0x7fca70196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53586/0 (socket says 172.21.15.144:53586) 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca6dd9b700 1 -- 172.21.15.144:0/843792391 learned_addr learned my addr 172.21.15.144:0/843792391 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.029+0000 7fca6e59c700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca700ffc50 0x7fca70196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.030+0000 7fca6dd9b700 1 -- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca700ffc50 msgr2=0x7fca70196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.030+0000 7fca6dd9b700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca700ffc50 0x7fca70196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.030+0000 7fca6dd9b700 1 -- 172.21.15.144:0/843792391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca640095f0 con 0x7fca70100e50 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.030+0000 7fca6dd9b700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70100e50 0x7fca70196ca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d8d0 tx=0x7fca6000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.030+0000 7fca5f7fe700 1 -- 172.21.15.144:0/843792391 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca600098a0 con 0x7fca70100e50 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.030+0000 7fca5f7fe700 1 -- 172.21.15.144:0/843792391 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca60010460 con 0x7fca70100e50 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.030+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7019be10 con 0x7fca70100e50 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.030+0000 7fca5f7fe700 1 -- 172.21.15.144:0/843792391 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca6000f5f0 con 0x7fca70100e50 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.030+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7019c330 con 0x7fca70100e50 2024-05-15T15:48:10.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.031+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7004ec60 con 0x7fca70100e50 2024-05-15T15:48:10.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.031+0000 7fca5f7fe700 1 -- 172.21.15.144:0/843792391 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7fca60009a00 con 0x7fca70100e50 2024-05-15T15:48:10.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.032+0000 7fca5f7fe700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fca5806cc60 0x7fca5806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:48:10.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.032+0000 7fca5f7fe700 1 -- 172.21.15.144:0/843792391 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3221+0+0 (secure 0 0 0) 0x7fca6008b730 con 0x7fca70100e50 2024-05-15T15:48:10.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.032+0000 7fca6e59c700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fca5806cc60 0x7fca5806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:48:10.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.032+0000 7fca6e59c700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fca5806cc60 0x7fca5806f120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca64000c00 tx=0x7fca64005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:48:10.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.037+0000 7fca5f7fe700 1 -- 172.21.15.144:0/843792391 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fca60059f30 con 0x7fca70100e50 2024-05-15T15:48:10.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:09 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-15T15:48:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:09 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-15T15:48:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:09 smithi049 ceph-mon[28399]: osdmap e20: 3 total, 3 up, 3 in 2024-05-15T15:48:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:09 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-15T15:48:10.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:10.241+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fca70068d90 con 0x7fca5806cc60 2024-05-15T15:48:10.997 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:10 smithi144 ceph-mon[36389]: pgmap v76: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:10.997 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:10 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:48:10.997 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:10 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:48:10.997 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:10 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:10.997 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:10 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-15T15:48:10.997 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:10 smithi144 ceph-mon[36389]: osdmap e21: 3 total, 3 up, 3 in 2024-05-15T15:48:11.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:10 smithi049 ceph-mon[28399]: pgmap v76: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:11.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:10 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:48:11.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:10 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:48:11.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:10 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:11.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:10 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-15T15:48:11.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:10 smithi049 ceph-mon[28399]: osdmap e21: 3 total, 3 up, 3 in 2024-05-15T15:48:11.910 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:48:11 smithi049 sudo[66486]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T15:48:11.910 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:48:11 smithi049 sudo[66486]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:11.910 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:48:11 smithi049 sudo[66486]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:11.910 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:48:11 smithi049 sudo[66486]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:12.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:12.103+0000 7fca5f7fe700 1 -- 172.21.15.144:0/843792391 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fca60056870 con 0x7fca70100e50 2024-05-15T15:48:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:48:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: Detected new or changed devices on smithi049 2024-05-15T15:48:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: Adjusting osd_memory_target on smithi049 to 3946M 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: osdmap e22: 3 total, 3 up, 3 in 2024-05-15T15:48:12.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.205 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:48:11 smithi049 sudo[66577]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T15:48:12.205 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:48:11 smithi049 sudo[66577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:12.205 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:48:11 smithi049 sudo[66577]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:12.205 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:48:11 smithi049 sudo[66577]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:12.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: Detected new or changed devices on smithi049 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: Adjusting osd_memory_target on smithi049 to 3946M 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:12.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:48:12.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:12.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: osdmap e22: 3 total, 3 up, 3 in 2024-05-15T15:48:12.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:12.722 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:48:12 smithi049 sudo[66646]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T15:48:12.722 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:48:12 smithi049 sudo[66646]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:12.723 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:48:12 smithi049 sudo[66646]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:12.723 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:48:12 smithi049 sudo[66646]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:13.102 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:48:12 smithi049 sudo[66714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T15:48:13.103 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:48:12 smithi049 sudo[66714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:13.103 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:48:12 smithi049 sudo[66714]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:13.103 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 15:48:12 smithi049 sudo[66714]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:13.373 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:13 smithi049 ceph-mon[28399]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:13.373 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:13 smithi049 ceph-mon[28399]: mgrmap e19: smithi049.vuhzgf(active, since 2m), standbys: smithi144.whcjkr 2024-05-15T15:48:13.373 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:13 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/1083441834' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99"}]: dispatch 2024-05-15T15:48:13.374 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:13 smithi049 ceph-mon[28399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99"}]: dispatch 2024-05-15T15:48:13.374 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:13 smithi049 ceph-mon[28399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99"}]': finished 2024-05-15T15:48:13.374 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:13 smithi049 ceph-mon[28399]: osdmap e23: 4 total, 3 up, 4 in 2024-05-15T15:48:13.374 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:13 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:13 smithi144 ceph-mon[36389]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:13 smithi144 ceph-mon[36389]: mgrmap e19: smithi049.vuhzgf(active, since 2m), standbys: smithi144.whcjkr 2024-05-15T15:48:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:13 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/1083441834' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99"}]: dispatch 2024-05-15T15:48:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:13 smithi144 ceph-mon[36389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99"}]: dispatch 2024-05-15T15:48:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:13 smithi144 ceph-mon[36389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99"}]': finished 2024-05-15T15:48:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:13 smithi144 ceph-mon[36389]: osdmap e23: 4 total, 3 up, 4 in 2024-05-15T15:48:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:13 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:13.642 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:13 smithi049 sudo[66762]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T15:48:13.642 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:13 smithi049 sudo[66762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:13.642 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:13 smithi049 sudo[66762]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:13.642 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:13 smithi049 sudo[66762]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:13.955 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:13 smithi049 sudo[66792]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T15:48:13.955 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:13 smithi049 sudo[66792]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:13.955 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:13 smithi049 sudo[66792]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:13.955 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 15:48:13 smithi049 sudo[66792]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:14.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:14 smithi049 sudo[66857]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T15:48:14.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:14 smithi049 sudo[66857]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:14.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:14 smithi049 sudo[66857]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:14.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:14 smithi049 sudo[66857]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:15.024 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:14 smithi049 sudo[66917]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T15:48:15.024 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:14 smithi049 sudo[66917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:15.024 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:14 smithi049 sudo[66917]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:15.024 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:14 smithi049 sudo[66917]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:15.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 sudo[40839]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T15:48:15.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 sudo[40839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:15.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 sudo[40839]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:15.266 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 sudo[40839]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:15.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:15 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/1751622656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:48:15.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:15 smithi049 ceph-mon[28399]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T15:48:15.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 sudo[41026]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T15:48:15.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 sudo[41026]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T15:48:15.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 sudo[41026]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T15:48:15.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 sudo[41026]: pam_unix(sudo:session): session closed for user root 2024-05-15T15:48:15.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/1751622656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:48:15.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:15 smithi144 ceph-mon[36389]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T15:48:16.558 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 101 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:16.558 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T15:48:16.558 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:48:16.558 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:48:16.558 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T15:48:16.558 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:48:16.558 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:48:16.558 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T15:48:16.558 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:16.559 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 101 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:16.559 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:16.559 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:16.559 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:16.559 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:48:16.559 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:16 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:16.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 101 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:16.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T15:48:16.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 101 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:48:16.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:16 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:18.573 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:18 smithi144 ceph-mon[36389]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:18.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:18 smithi049 ceph-mon[28399]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:20.775 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:20 smithi144 ceph-mon[36389]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:20.775 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-15T15:48:20.775 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:20.775 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:20 smithi144 ceph-mon[36389]: Deploying daemon osd.3 on smithi144 2024-05-15T15:48:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:20 smithi049 ceph-mon[28399]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-15T15:48:20.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:20.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:20 smithi049 ceph-mon[28399]: Deploying daemon osd.3 on smithi144 2024-05-15T15:48:21.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:21 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:48:21.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:21 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:48:22.834 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:22 smithi144 ceph-mon[36389]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:22.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:22 smithi049 ceph-mon[28399]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:24.652 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:24 smithi144 ceph-mon[36389]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:24.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:24 smithi049 ceph-mon[28399]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:26.603 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:26 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:26.603 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:26 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:26.603 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:26 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:26.603 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:26 smithi144 ceph-mon[36389]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:26.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:26 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:26.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:26 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:26.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:26 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:26.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:26 smithi049 ceph-mon[28399]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:28.352 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 3 on host 'smithi144' 2024-05-15T15:48:28.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.350+0000 7fca5f7fe700 1 -- 172.21.15.144:0/843792391 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fca70068d90 con 0x7fca5806cc60 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fca5806cc60 msgr2=0x7fca5806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fca5806cc60 0x7fca5806f120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca64000c00 tx=0x7fca64005960 comp rx=0 tx=0).stop 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70100e50 msgr2=0x7fca70196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70100e50 0x7fca70196ca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d8d0 tx=0x7fca6000dbe0 comp rx=0 tx=0).stop 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 shutdown_connections 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca700ffc50 0x7fca70196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fca5806cc60 0x7fca5806f120 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 --2- 172.21.15.144:0/843792391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca70100e50 0x7fca70196ca0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 >> 172.21.15.144:0/843792391 conn(0x7fca700fb1d0 msgr2=0x7fca70068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 shutdown_connections 2024-05-15T15:48:28.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:28.352+0000 7fca74dae700 1 -- 172.21.15.144:0/843792391 wait complete. 2024-05-15T15:48:28.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:28 smithi144 ceph-mon[36389]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:28.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:28 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:28.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:28 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:28.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:28 smithi049 ceph-mon[28399]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:28.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:28 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:28.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:28 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:28.990 DEBUG:teuthology.orchestra.run.smithi144:osd.3> sudo journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.3.service 2024-05-15T15:48:28.992 INFO:tasks.cephadm:Deploying osd.4 on smithi144 with /dev/vg_nvme/lv_3... 2024-05-15T15:48:28.993 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-15T15:48:29.255 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 15:48:29 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[42263]: 2024-05-15T15:48:29.182+0000 7f3f4c49a7c0 -1 osd.3 0 log_to_monitors true 2024-05-15T15:48:29.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:29 smithi144 ceph-mon[36389]: from='osd.3 [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T15:48:29.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:29 smithi144 ceph-mon[36389]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T15:48:29.753 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T15:48:29.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:29 smithi049 ceph-mon[28399]: from='osd.3 [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T15:48:29.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:29 smithi049 ceph-mon[28399]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T15:48:30.819 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:30 smithi144 ceph-mon[36389]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:30.819 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:30 smithi144 ceph-mon[36389]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-15T15:48:30.819 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:30 smithi144 ceph-mon[36389]: osdmap e24: 4 total, 3 up, 4 in 2024-05-15T15:48:30.819 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:30 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:30.819 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:30 smithi144 ceph-mon[36389]: from='osd.3 [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:48:30.820 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:30 smithi144 ceph-mon[36389]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:48:30.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:30 smithi049 ceph-mon[28399]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:30.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:30 smithi049 ceph-mon[28399]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-15T15:48:30.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:30 smithi049 ceph-mon[28399]: osdmap e24: 4 total, 3 up, 4 in 2024-05-15T15:48:30.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:30 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:30.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:30 smithi049 ceph-mon[28399]: from='osd.3 [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:48:30.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:30 smithi049 ceph-mon[28399]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:48:31.375 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 15:48:31 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[42263]: 2024-05-15T15:48:31.359+0000 7f3f412fd700 -1 osd.3 0 waiting for initial osdmap 2024-05-15T15:48:31.375 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 15:48:31 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[42263]: 2024-05-15T15:48:31.373+0000 7f3f398eb700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T15:48:31.655 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:31 smithi144 ceph-mon[36389]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-15T15:48:31.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:31 smithi144 ceph-mon[36389]: osdmap e25: 4 total, 3 up, 4 in 2024-05-15T15:48:31.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:31 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:31.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:31 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:31.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:31 smithi144 ceph-mon[36389]: osdmap e26: 4 total, 3 up, 4 in 2024-05-15T15:48:31.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:31 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:31.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:31 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:31.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:31 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:31.656 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:31 smithi144 ceph-mon[36389]: from='osd.3 ' entity='osd.3' 2024-05-15T15:48:31.714 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-15T15:48:31.714 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-15T15:48:31.714 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-15T15:48:31.714 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-15T15:48:31.714 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128058 s, 819 MB/s 2024-05-15T15:48:31.714 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-15T15:48:31.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:31 smithi049 ceph-mon[28399]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-15T15:48:31.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:31 smithi049 ceph-mon[28399]: osdmap e25: 4 total, 3 up, 4 in 2024-05-15T15:48:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:31 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:31 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:31 smithi049 ceph-mon[28399]: osdmap e26: 4 total, 3 up, 4 in 2024-05-15T15:48:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:31 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:31 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:31 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:31 smithi049 ceph-mon[28399]: from='osd.3 ' entity='osd.3' 2024-05-15T15:48:32.682 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:32 smithi144 ceph-mon[36389]: purged_snaps scrub starts 2024-05-15T15:48:32.682 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:32 smithi144 ceph-mon[36389]: purged_snaps scrub ok 2024-05-15T15:48:32.682 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:32 smithi144 ceph-mon[36389]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:32.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:32 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:32.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:32 smithi144 ceph-mon[36389]: osd.3 [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] boot 2024-05-15T15:48:32.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:32 smithi144 ceph-mon[36389]: osdmap e27: 4 total, 4 up, 4 in 2024-05-15T15:48:32.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:32 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:32.703 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-05-15T15:48:32.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:32 smithi049 ceph-mon[28399]: purged_snaps scrub starts 2024-05-15T15:48:32.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:32 smithi049 ceph-mon[28399]: purged_snaps scrub ok 2024-05-15T15:48:32.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:32 smithi049 ceph-mon[28399]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-05-15T15:48:32.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:32 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:32.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:32 smithi049 ceph-mon[28399]: osd.3 [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] boot 2024-05-15T15:48:32.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:32 smithi049 ceph-mon[28399]: osdmap e27: 4 total, 4 up, 4 in 2024-05-15T15:48:32.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:32 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:48:33.637 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T15:48:33.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:33 smithi144 ceph-mon[36389]: osdmap e28: 4 total, 4 up, 4 in 2024-05-15T15:48:33.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:33 smithi049 ceph-mon[28399]: osdmap e28: 4 total, 4 up, 4 in 2024-05-15T15:48:34.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:34 smithi144 ceph-mon[36389]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:34.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:34 smithi144 ceph-mon[36389]: osdmap e29: 4 total, 4 up, 4 in 2024-05-15T15:48:34.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:34 smithi049 ceph-mon[28399]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:34.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:34 smithi049 ceph-mon[28399]: osdmap e29: 4 total, 4 up, 4 in 2024-05-15T15:48:35.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.419+0000 7facb0154700 1 -- 172.21.15.144:0/1035007410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca8101180 msgr2=0x7faca81015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:35.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.419+0000 7facb0154700 1 --2- 172.21.15.144:0/1035007410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca8101180 0x7faca81015a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faca4009910 tx=0x7faca4009c20 comp rx=0 tx=0).stop 2024-05-15T15:48:35.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.419+0000 7facb0154700 1 -- 172.21.15.144:0/1035007410 shutdown_connections 2024-05-15T15:48:35.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.419+0000 7facb0154700 1 --2- 172.21.15.144:0/1035007410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faca80ffec0 0x7faca8100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:35.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.419+0000 7facb0154700 1 --2- 172.21.15.144:0/1035007410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca8101180 0x7faca81015a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:35.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.419+0000 7facb0154700 1 -- 172.21.15.144:0/1035007410 >> 172.21.15.144:0/1035007410 conn(0x7faca80fa910 msgr2=0x7faca80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:48:35.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.419+0000 7facb0154700 1 -- 172.21.15.144:0/1035007410 shutdown_connections 2024-05-15T15:48:35.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.419+0000 7facb0154700 1 -- 172.21.15.144:0/1035007410 wait complete. 2024-05-15T15:48:35.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facb0154700 1 Processor -- start 2024-05-15T15:48:35.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facb0154700 1 -- start start 2024-05-15T15:48:35.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facb0154700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca80ffec0 0x7faca8196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:48:35.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facb0154700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faca8101180 0x7faca8196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:48:35.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facb0154700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca81973d0 con 0x7faca8101180 2024-05-15T15:48:35.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facb0154700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca8197510 con 0x7faca80ffec0 2024-05-15T15:48:35.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facadef0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca80ffec0 0x7faca8196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:48:35.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facadef0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca80ffec0 0x7faca8196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48188/0 (socket says 172.21.15.144:48188) 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facadef0700 1 -- 172.21.15.144:0/2767054279 learned_addr learned my addr 172.21.15.144:0/2767054279 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facad6ef700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faca8101180 0x7faca8196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facadef0700 1 -- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faca8101180 msgr2=0x7faca8196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facadef0700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faca8101180 0x7faca8196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facadef0700 1 -- 172.21.15.144:0/2767054279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca40095f0 con 0x7faca80ffec0 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.420+0000 7facad6ef700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faca8101180 0x7faca8196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.421+0000 7facadef0700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca80ffec0 0x7faca8196870 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faca400bd70 tx=0x7faca400be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.421+0000 7fac9effd700 1 -- 172.21.15.144:0/2767054279 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faca401d070 con 0x7faca80ffec0 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.421+0000 7fac9effd700 1 -- 172.21.15.144:0/2767054279 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faca400f460 con 0x7faca80ffec0 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.421+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca819bec0 con 0x7faca80ffec0 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.421+0000 7fac9effd700 1 -- 172.21.15.144:0/2767054279 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faca4021600 con 0x7faca80ffec0 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.421+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca819c3b0 con 0x7faca80ffec0 2024-05-15T15:48:35.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.422+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca804ec60 con 0x7faca80ffec0 2024-05-15T15:48:35.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.422+0000 7fac9effd700 1 -- 172.21.15.144:0/2767054279 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7faca400fa50 con 0x7faca80ffec0 2024-05-15T15:48:35.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.423+0000 7fac9effd700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fac9406ccf0 0x7fac9406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:48:35.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.423+0000 7fac9effd700 1 -- 172.21.15.144:0/2767054279 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7faca408ced0 con 0x7faca80ffec0 2024-05-15T15:48:35.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.423+0000 7facad6ef700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fac9406ccf0 0x7fac9406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:48:35.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.424+0000 7facad6ef700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fac9406ccf0 0x7fac9406f1b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac98007900 tx=0x7fac98008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:48:35.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.427+0000 7fac9effd700 1 -- 172.21.15.144:0/2767054279 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faca405b3d0 con 0x7faca80ffec0 2024-05-15T15:48:35.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:35.637+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7faca80ff4f0 con 0x7fac9406ccf0 2024-05-15T15:48:35.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:35 smithi144 ceph-mon[36389]: osdmap e30: 4 total, 4 up, 4 in 2024-05-15T15:48:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:35 smithi049 ceph-mon[28399]: osdmap e30: 4 total, 4 up, 4 in 2024-05-15T15:48:36.875 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:36.875 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: osdmap e31: 4 total, 4 up, 4 in 2024-05-15T15:48:36.875 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:48:36.876 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:48:36.876 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:48:36.876 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:36.876 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: Detected new or changed devices on smithi144 2024-05-15T15:48:36.876 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.876 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:48:36.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: Adjusting osd_memory_target on smithi144 to 14912M 2024-05-15T15:48:36.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:36.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:48:36.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:36.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:36.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:48:36.878 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.878 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:48:36.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: osdmap e31: 4 total, 4 up, 4 in 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: Detected new or changed devices on smithi144 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: Adjusting osd_memory_target on smithi144 to 14912M 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:48:36.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:36.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:36.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:48:36.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:36.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:48:38.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:38 smithi049 ceph-mon[28399]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-15T15:48:38.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:38 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/2132372224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6659631f-da39-4a2c-9f75-e7b4a06b602f"}]: dispatch 2024-05-15T15:48:38.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:38 smithi049 ceph-mon[28399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6659631f-da39-4a2c-9f75-e7b4a06b602f"}]: dispatch 2024-05-15T15:48:38.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:38 smithi049 ceph-mon[28399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6659631f-da39-4a2c-9f75-e7b4a06b602f"}]': finished 2024-05-15T15:48:38.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:38 smithi049 ceph-mon[28399]: osdmap e32: 5 total, 4 up, 5 in 2024-05-15T15:48:38.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:38 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:38.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:38 smithi144 ceph-mon[36389]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-15T15:48:38.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:38 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/2132372224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6659631f-da39-4a2c-9f75-e7b4a06b602f"}]: dispatch 2024-05-15T15:48:38.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:38 smithi144 ceph-mon[36389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6659631f-da39-4a2c-9f75-e7b4a06b602f"}]: dispatch 2024-05-15T15:48:38.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:38 smithi144 ceph-mon[36389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6659631f-da39-4a2c-9f75-e7b4a06b602f"}]': finished 2024-05-15T15:48:38.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:38 smithi144 ceph-mon[36389]: osdmap e32: 5 total, 4 up, 5 in 2024-05-15T15:48:38.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:38 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:39.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:39 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/1751587071' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:48:39.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:39 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/1751587071' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:48:40.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:40 smithi144 ceph-mon[36389]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-15T15:48:40.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:40 smithi049 ceph-mon[28399]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-15T15:48:42.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:42 smithi144 ceph-mon[36389]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-15T15:48:42.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:42 smithi049 ceph-mon[28399]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-15T15:48:44.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:44 smithi144 ceph-mon[36389]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T15:48:44.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:44 smithi049 ceph-mon[28399]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T15:48:46.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:46 smithi144 ceph-mon[36389]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:46.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-15T15:48:46.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:46.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:46 smithi144 ceph-mon[36389]: Deploying daemon osd.4 on smithi144 2024-05-15T15:48:46.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:46 smithi049 ceph-mon[28399]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:46.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-15T15:48:46.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:48:46.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:46 smithi049 ceph-mon[28399]: Deploying daemon osd.4 on smithi144 2024-05-15T15:48:48.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:48 smithi144 ceph-mon[36389]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:48.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:48 smithi049 ceph-mon[28399]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:50.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:50 smithi144 ceph-mon[36389]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:50 smithi049 ceph-mon[28399]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:51.849 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:51.849 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:51.849 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:51.849 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:48:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:48:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:48:52.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:52 smithi144 ceph-mon[36389]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:53.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:52 smithi049 ceph-mon[28399]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:54.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.556+0000 7fac9effd700 1 -- 172.21.15.144:0/2767054279 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faca80ff4f0 con 0x7fac9406ccf0 2024-05-15T15:48:54.559 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 4 on host 'smithi144' 2024-05-15T15:48:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fac9406ccf0 msgr2=0x7fac9406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fac9406ccf0 0x7fac9406f1b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac98007900 tx=0x7fac98008040 comp rx=0 tx=0).stop 2024-05-15T15:48:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca80ffec0 msgr2=0x7faca8196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:48:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca80ffec0 0x7faca8196870 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faca400bd70 tx=0x7faca400be50 comp rx=0 tx=0).stop 2024-05-15T15:48:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 shutdown_connections 2024-05-15T15:48:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fac9406ccf0 0x7fac9406f1b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:54.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca80ffec0 0x7faca8196870 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:54.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 --2- 172.21.15.144:0/2767054279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faca8101180 0x7faca8196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:48:54.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 >> 172.21.15.144:0/2767054279 conn(0x7faca80fa910 msgr2=0x7faca80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:48:54.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 shutdown_connections 2024-05-15T15:48:54.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:48:54.559+0000 7facb0154700 1 -- 172.21.15.144:0/2767054279 wait complete. 2024-05-15T15:48:55.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:54 smithi144 ceph-mon[36389]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:55.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:54 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:55.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:54 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:55.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:54 smithi144 ceph-mon[36389]: from='osd.4 [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T15:48:55.162 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:54 smithi144 ceph-mon[36389]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T15:48:55.182 DEBUG:teuthology.orchestra.run.smithi144:osd.4> sudo journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.4.service 2024-05-15T15:48:55.185 INFO:tasks.cephadm:Deploying osd.5 on smithi144 with /dev/vg_nvme/lv_2... 2024-05-15T15:48:55.185 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-15T15:48:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:54 smithi049 ceph-mon[28399]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:54 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:54 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:54 smithi049 ceph-mon[28399]: from='osd.4 [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T15:48:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:54 smithi049 ceph-mon[28399]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T15:48:55.511 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T15:48:56.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:56 smithi144 ceph-mon[36389]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:56.888 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:56 smithi144 ceph-mon[36389]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-15T15:48:56.888 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:56 smithi144 ceph-mon[36389]: osdmap e33: 5 total, 4 up, 5 in 2024-05-15T15:48:56.888 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:56 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:56.888 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:56 smithi144 ceph-mon[36389]: from='osd.4 [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:48:56.888 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:56 smithi144 ceph-mon[36389]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:48:56.888 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 15:48:56 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[48698]: 2024-05-15T15:48:56.546+0000 7f7259981700 -1 osd.4 0 waiting for initial osdmap 2024-05-15T15:48:56.889 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 15:48:56 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[48698]: 2024-05-15T15:48:56.554+0000 7f7255f77700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T15:48:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:56 smithi049 ceph-mon[28399]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:56 smithi049 ceph-mon[28399]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-15T15:48:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:56 smithi049 ceph-mon[28399]: osdmap e33: 5 total, 4 up, 5 in 2024-05-15T15:48:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:56 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:56 smithi049 ceph-mon[28399]: from='osd.4 [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:48:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:56 smithi049 ceph-mon[28399]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:48:57.483 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-15T15:48:57.483 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-15T15:48:57.483 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-15T15:48:57.483 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-15T15:48:57.483 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0143533 s, 731 MB/s 2024-05-15T15:48:57.483 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-15T15:48:57.593 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:57 smithi144 ceph-mon[36389]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-15T15:48:57.593 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:57 smithi144 ceph-mon[36389]: osdmap e34: 5 total, 4 up, 5 in 2024-05-15T15:48:57.593 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:57 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:57.593 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:57 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:57.593 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:57 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:57.593 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:57 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:57.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:57 smithi049 ceph-mon[28399]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-15T15:48:57.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:57 smithi049 ceph-mon[28399]: osdmap e34: 5 total, 4 up, 5 in 2024-05-15T15:48:57.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:57 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:57.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:57 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:57.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:57 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:57.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:57 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:48:58.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:58 smithi144 ceph-mon[36389]: purged_snaps scrub starts 2024-05-15T15:48:58.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:58 smithi144 ceph-mon[36389]: purged_snaps scrub ok 2024-05-15T15:48:58.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:58 smithi144 ceph-mon[36389]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:58.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:58 smithi144 ceph-mon[36389]: osd.4 [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] boot 2024-05-15T15:48:58.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:58 smithi144 ceph-mon[36389]: osdmap e35: 5 total, 5 up, 5 in 2024-05-15T15:48:58.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:58 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:58 smithi049 ceph-mon[28399]: purged_snaps scrub starts 2024-05-15T15:48:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:58 smithi049 ceph-mon[28399]: purged_snaps scrub ok 2024-05-15T15:48:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:58 smithi049 ceph-mon[28399]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-15T15:48:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:58 smithi049 ceph-mon[28399]: osd.4 [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] boot 2024-05-15T15:48:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:58 smithi049 ceph-mon[28399]: osdmap e35: 5 total, 5 up, 5 in 2024-05-15T15:48:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:48:58.988 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-05-15T15:48:59.527 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T15:48:59.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:48:59 smithi144 ceph-mon[36389]: osdmap e36: 5 total, 5 up, 5 in 2024-05-15T15:48:59.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:48:59 smithi049 ceph-mon[28399]: osdmap e36: 5 total, 5 up, 5 in 2024-05-15T15:49:00.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:00 smithi144 ceph-mon[36389]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:00.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:00 smithi049 ceph-mon[28399]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:01.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 -- 172.21.15.144:0/3230252200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800100cf0 msgr2=0x7fc800101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:01.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 --2- 172.21.15.144:0/3230252200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800100cf0 0x7fc800101110 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009910 tx=0x7fc7f4009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:01.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 -- 172.21.15.144:0/3230252200 shutdown_connections 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 --2- 172.21.15.144:0/3230252200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc800101ef0 0x7fc800102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 --2- 172.21.15.144:0/3230252200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800100cf0 0x7fc800101110 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 -- 172.21.15.144:0/3230252200 >> 172.21.15.144:0/3230252200 conn(0x7fc8000fc290 msgr2=0x7fc8000fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 -- 172.21.15.144:0/3230252200 shutdown_connections 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 -- 172.21.15.144:0/3230252200 wait complete. 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 Processor -- start 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.810+0000 7fc807188700 1 -- start start 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc807188700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc800100cf0 0x7fc800074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc807188700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800101ef0 0x7fc800073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc807188700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000738d0 con 0x7fc800100cf0 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc807188700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc800073a10 con 0x7fc800101ef0 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800101ef0 0x7fc800073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:01.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800101ef0 0x7fc800073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35600/0 (socket says 172.21.15.144:35600) 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fffff700 1 -- 172.21.15.144:0/135243496 learned_addr learned my addr 172.21.15.144:0/135243496 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc804f24700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc800100cf0 0x7fc800074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fffff700 1 -- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc800100cf0 msgr2=0x7fc800074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fffff700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc800100cf0 0x7fc800074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fffff700 1 -- 172.21.15.144:0/135243496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40095f0 con 0x7fc800101ef0 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fffff700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800101ef0 0x7fc800073390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00b4b0 tx=0x7fc7ec00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fdffb700 1 -- 172.21.15.144:0/135243496 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7ec011800 con 0x7fc800101ef0 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fdffb700 1 -- 172.21.15.144:0/135243496 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7ec011e20 con 0x7fc800101ef0 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc800073cf0 con 0x7fc800101ef0 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.811+0000 7fc7fdffb700 1 -- 172.21.15.144:0/135243496 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7ec00f2e0 con 0x7fc800101ef0 2024-05-15T15:49:01.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.812+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80019e8a0 con 0x7fc800101ef0 2024-05-15T15:49:01.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.813+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8001908f0 con 0x7fc800101ef0 2024-05-15T15:49:01.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.813+0000 7fc7fdffb700 1 -- 172.21.15.144:0/135243496 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc7ec010380 con 0x7fc800101ef0 2024-05-15T15:49:01.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.813+0000 7fc7fdffb700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc7f006cce0 0x7fc7f006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:01.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.814+0000 7fc7fdffb700 1 -- 172.21.15.144:0/135243496 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fc7ec08b940 con 0x7fc800101ef0 2024-05-15T15:49:01.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.814+0000 7fc804f24700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc7f006cce0 0x7fc7f006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:01.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.815+0000 7fc804f24700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc7f006cce0 0x7fc7f006f1a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4006010 tx=0x7fc7f400b580 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:01.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:01.818+0000 7fc7fdffb700 1 -- 172.21.15.144:0/135243496 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc7ec059c90 con 0x7fc800101ef0 2024-05-15T15:49:02.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:02.024+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8000613e0 con 0x7fc7f006cce0 2024-05-15T15:49:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: Detected new or changed devices on smithi144 2024-05-15T15:49:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: Adjusting osd_memory_target on smithi144 to 7456M 2024-05-15T15:49:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:02.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:49:02.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:02.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:49:02.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:02.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:49:02.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:01 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: Detected new or changed devices on smithi144 2024-05-15T15:49:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: Adjusting osd_memory_target on smithi144 to 7456M 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:49:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:01 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:03.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:02 smithi144 ceph-mon[36389]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:03.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:02 smithi144 ceph-mon[36389]: from='client.24177 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:49:03.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:49:03.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:49:03.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:03.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:02 smithi049 ceph-mon[28399]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:03.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:02 smithi049 ceph-mon[28399]: from='client.24177 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:49:03.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T15:49:03.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T15:49:03.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:04 smithi144 ceph-mon[36389]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:04 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/755175638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51ea2d85-6cf7-4a96-8c74-612958083a17"}]: dispatch 2024-05-15T15:49:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:04 smithi144 ceph-mon[36389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51ea2d85-6cf7-4a96-8c74-612958083a17"}]: dispatch 2024-05-15T15:49:05.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:04 smithi144 ceph-mon[36389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51ea2d85-6cf7-4a96-8c74-612958083a17"}]': finished 2024-05-15T15:49:05.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:04 smithi144 ceph-mon[36389]: osdmap e37: 6 total, 5 up, 6 in 2024-05-15T15:49:05.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:04 smithi049 ceph-mon[28399]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:04 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/755175638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51ea2d85-6cf7-4a96-8c74-612958083a17"}]: dispatch 2024-05-15T15:49:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:04 smithi049 ceph-mon[28399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51ea2d85-6cf7-4a96-8c74-612958083a17"}]: dispatch 2024-05-15T15:49:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:04 smithi049 ceph-mon[28399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51ea2d85-6cf7-4a96-8c74-612958083a17"}]': finished 2024-05-15T15:49:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:04 smithi049 ceph-mon[28399]: osdmap e37: 6 total, 5 up, 6 in 2024-05-15T15:49:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:05 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/3975705649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:49:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:05 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/3975705649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T15:49:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:06 smithi144 ceph-mon[36389]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:49:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:06 smithi049 ceph-mon[28399]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:49:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:08 smithi144 ceph-mon[36389]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:09.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:08 smithi049 ceph-mon[28399]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:11.088 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:10 smithi144 ceph-mon[36389]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:11.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:10 smithi049 ceph-mon[28399]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:12 smithi144 ceph-mon[36389]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-15T15:49:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:12 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:12 smithi144 ceph-mon[36389]: Deploying daemon osd.5 on smithi144 2024-05-15T15:49:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:12 smithi049 ceph-mon[28399]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-15T15:49:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:12 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:12 smithi049 ceph-mon[28399]: Deploying daemon osd.5 on smithi144 2024-05-15T15:49:15.018 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:14 smithi144 ceph-mon[36389]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:15.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:14 smithi049 ceph-mon[28399]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:17.051 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:16 smithi144 ceph-mon[36389]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:17.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:16 smithi049 ceph-mon[28399]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:18.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:18 smithi144 ceph-mon[36389]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:18.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:18 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:18.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:18 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:49:18.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:18 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:18.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:18 smithi049 ceph-mon[28399]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:18.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:18 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:18.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:18 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:49:18.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:18 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:20 smithi144 ceph-mon[36389]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:20 smithi049 ceph-mon[28399]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:21.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.744+0000 7fc7fdffb700 1 -- 172.21.15.144:0/135243496 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc8000613e0 con 0x7fc7f006cce0 2024-05-15T15:49:21.746 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 5 on host 'smithi144' 2024-05-15T15:49:21.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc7f006cce0 msgr2=0x7fc7f006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:21.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc7f006cce0 0x7fc7f006f1a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4006010 tx=0x7fc7f400b580 comp rx=0 tx=0).stop 2024-05-15T15:49:21.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800101ef0 msgr2=0x7fc800073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:21.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800101ef0 0x7fc800073390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00b4b0 tx=0x7fc7ec00b870 comp rx=0 tx=0).stop 2024-05-15T15:49:21.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 shutdown_connections 2024-05-15T15:49:21.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc800100cf0 0x7fc800074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:21.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc7f006cce0 0x7fc7f006f1a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:21.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 --2- 172.21.15.144:0/135243496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc800101ef0 0x7fc800073390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:21.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 >> 172.21.15.144:0/135243496 conn(0x7fc8000fc290 msgr2=0x7fc800105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:21.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 shutdown_connections 2024-05-15T15:49:21.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:21.746+0000 7fc807188700 1 -- 172.21.15.144:0/135243496 wait complete. 2024-05-15T15:49:21.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:21 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:49:21.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:21 smithi144 ceph-mon[36389]: from='osd.5 [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T15:49:21.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:21 smithi144 ceph-mon[36389]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T15:49:21.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:21 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:49:21.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:21 smithi049 ceph-mon[28399]: from='osd.5 [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T15:49:21.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:21 smithi049 ceph-mon[28399]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T15:49:22.433 DEBUG:teuthology.orchestra.run.smithi144:osd.5> sudo journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.5.service 2024-05-15T15:49:22.436 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-05-15T15:49:22.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-15T15:49:22.670 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:22.679 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:22.680 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-15T15:49:22.680 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: osdmap e38: 6 total, 5 up, 6 in 2024-05-15T15:49:22.680 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:22.680 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: from='osd.5 [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:49:22.680 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:49:22.680 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:22.680 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:22.680 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:22.680 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:22.788 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:22.789 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-15T15:49:22.789 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: osdmap e38: 6 total, 5 up, 6 in 2024-05-15T15:49:22.789 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:22.789 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: from='osd.5 [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:49:22.789 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T15:49:22.789 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:22.789 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:22.789 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:22.789 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:23.790 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:23 smithi144 ceph-mon[36389]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-15T15:49:23.790 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:23 smithi144 ceph-mon[36389]: osdmap e39: 6 total, 5 up, 6 in 2024-05-15T15:49:23.790 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:23 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:23.790 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:23 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:23.790 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:23 smithi144 ceph-mon[36389]: from='osd.5 ' entity='osd.5' 2024-05-15T15:49:23.791 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 15:49:23 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[55371]: 2024-05-15T15:49:23.490+0000 7f31ec62f700 -1 osd.5 0 waiting for initial osdmap 2024-05-15T15:49:23.791 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 15:49:23 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[55371]: 2024-05-15T15:49:23.501+0000 7f31e4c1d700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T15:49:23.852 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:23 smithi049 ceph-mon[28399]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-15T15:49:23.852 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:23 smithi049 ceph-mon[28399]: osdmap e39: 6 total, 5 up, 6 in 2024-05-15T15:49:23.852 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:23 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:23.852 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:23 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:23.852 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:23 smithi049 ceph-mon[28399]: from='osd.5 ' entity='osd.5' 2024-05-15T15:49:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.343+0000 7fe089fd5700 1 -- 172.21.15.49:0/3950236315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 msgr2=0x7fe0840733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.343+0000 7fe089fd5700 1 --2- 172.21.15.49:0/3950236315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 0x7fe0840733b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe07400b0a0 tx=0x7fe07400b3b0 comp rx=0 tx=0).stop 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.343+0000 7fe089fd5700 1 -- 172.21.15.49:0/3950236315 shutdown_connections 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.343+0000 7fe089fd5700 1 --2- 172.21.15.49:0/3950236315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe084073980 0x7fe084073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.343+0000 7fe089fd5700 1 --2- 172.21.15.49:0/3950236315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 0x7fe0840733b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.343+0000 7fe089fd5700 1 -- 172.21.15.49:0/3950236315 >> 172.21.15.49:0/3950236315 conn(0x7fe0840fc2c0 msgr2=0x7fe0840fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.343+0000 7fe089fd5700 1 -- 172.21.15.49:0/3950236315 shutdown_connections 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.343+0000 7fe089fd5700 1 -- 172.21.15.49:0/3950236315 wait complete. 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe089fd5700 1 Processor -- start 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe089fd5700 1 -- start start 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe089fd5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe084073980 0x7fe084198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe089fd5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 0x7fe084198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe089fd5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0841994a0 con 0x7fe084074f50 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe089fd5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0841995e0 con 0x7fe084073980 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe07affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 0x7fe084198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe07affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 0x7fe084198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34872/0 (socket says 172.21.15.49:34872) 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe07affd700 1 -- 172.21.15.49:0/2642343816 learned_addr learned my addr 172.21.15.49:0/2642343816 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe0837fe700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe084073980 0x7fe084198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe07affd700 1 -- 172.21.15.49:0/2642343816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe084073980 msgr2=0x7fe084198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe07affd700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe084073980 0x7fe084198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.344+0000 7fe07affd700 1 -- 172.21.15.49:0/2642343816 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe074009d00 con 0x7fe084074f50 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.345+0000 7fe07affd700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 0x7fe084198e80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00d8d0 tx=0x7fe06c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.345+0000 7fe0817fa700 1 -- 172.21.15.49:0/2642343816 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe06c0098a0 con 0x7fe084074f50 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.345+0000 7fe0817fa700 1 -- 172.21.15.49:0/2642343816 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe06c010460 con 0x7fe084074f50 2024-05-15T15:49:24.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.345+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08419dff0 con 0x7fe084074f50 2024-05-15T15:49:24.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.345+0000 7fe0817fa700 1 -- 172.21.15.49:0/2642343816 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe06c00f5f0 con 0x7fe084074f50 2024-05-15T15:49:24.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.345+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08419e590 con 0x7fe084074f50 2024-05-15T15:49:24.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.346+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe084192a70 con 0x7fe084074f50 2024-05-15T15:49:24.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.346+0000 7fe0817fa700 1 -- 172.21.15.49:0/2642343816 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe06c009a00 con 0x7fe084074f50 2024-05-15T15:49:24.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.347+0000 7fe0817fa700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe07006ccf0 0x7fe07006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:24.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.347+0000 7fe0817fa700 1 -- 172.21.15.49:0/2642343816 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4358+0+0 (secure 0 0 0) 0x7fe06c08bbf0 con 0x7fe084074f50 2024-05-15T15:49:24.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.347+0000 7fe0837fe700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe07006ccf0 0x7fe07006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:24.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.347+0000 7fe0837fe700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe07006ccf0 0x7fe07006f1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe074000f80 tx=0x7fe074007910 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:24.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.352+0000 7fe0817fa700 1 -- 172.21.15.49:0/2642343816 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe06c059f20 con 0x7fe084074f50 2024-05-15T15:49:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.556+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fe08404ec60 con 0x7fe084074f50 2024-05-15T15:49:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.557+0000 7fe0817fa700 1 -- 172.21.15.49:0/2642343816 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7fe06c059ab0 con 0x7fe084074f50 2024-05-15T15:49:24.560 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe07006ccf0 msgr2=0x7fe07006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe07006ccf0 0x7fe07006f1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe074000f80 tx=0x7fe074007910 comp rx=0 tx=0).stop 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 msgr2=0x7fe084198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 0x7fe084198e80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00d8d0 tx=0x7fe06c00dbe0 comp rx=0 tx=0).stop 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 shutdown_connections 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe07006ccf0 0x7fe07006f1b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe084073980 0x7fe084198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 --2- 172.21.15.49:0/2642343816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe084074f50 0x7fe084198e80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 >> 172.21.15.49:0/2642343816 conn(0x7fe0840fc2c0 msgr2=0x7fe084106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 shutdown_connections 2024-05-15T15:49:24.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:24.559+0000 7fe089fd5700 1 -- 172.21.15.49:0/2642343816 wait complete. 2024-05-15T15:49:24.877 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:24 smithi049 ceph-mon[28399]: purged_snaps scrub starts 2024-05-15T15:49:24.877 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:24 smithi049 ceph-mon[28399]: purged_snaps scrub ok 2024-05-15T15:49:24.877 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:24 smithi049 ceph-mon[28399]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:24.877 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:24.877 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:24 smithi049 ceph-mon[28399]: osd.5 [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] boot 2024-05-15T15:49:24.877 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:24 smithi049 ceph-mon[28399]: osdmap e40: 6 total, 6 up, 6 in 2024-05-15T15:49:24.877 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:24.877 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:24 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2642343816' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-15T15:49:24.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:24 smithi144 ceph-mon[36389]: purged_snaps scrub starts 2024-05-15T15:49:24.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:24 smithi144 ceph-mon[36389]: purged_snaps scrub ok 2024-05-15T15:49:24.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:24 smithi144 ceph-mon[36389]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-15T15:49:24.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:24.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:24 smithi144 ceph-mon[36389]: osd.5 [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] boot 2024-05-15T15:49:24.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:24 smithi144 ceph-mon[36389]: osdmap e40: 6 total, 6 up, 6 in 2024-05-15T15:49:24.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:49:24.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:24 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2642343816' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-15T15:49:25.250 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1715788164,"num_in_osds":6,"osd_in_since":1715788144,"num_remapped_pgs":0} 2024-05-15T15:49:25.255 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T15:49:25.481 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:26.870 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:26 smithi049 ceph-mon[28399]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:26.870 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:26 smithi049 ceph-mon[28399]: osdmap e41: 6 total, 6 up, 6 in 2024-05-15T15:49:26.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:26 smithi144 ceph-mon[36389]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:26.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:26 smithi144 ceph-mon[36389]: osdmap e41: 6 total, 6 up, 6 in 2024-05-15T15:49:27.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.173+0000 7ff247e0b700 1 -- 172.21.15.49:0/4242024862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 msgr2=0x7ff2401001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:27.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.173+0000 7ff247e0b700 1 --2- 172.21.15.49:0/4242024862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 0x7ff2401001f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009910 tx=0x7ff23c009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.174+0000 7ff247e0b700 1 -- 172.21.15.49:0/4242024862 shutdown_connections 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.174+0000 7ff247e0b700 1 --2- 172.21.15.49:0/4242024862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2401007c0 0x7ff240104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.174+0000 7ff247e0b700 1 --2- 172.21.15.49:0/4242024862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 0x7ff2401001f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.174+0000 7ff247e0b700 1 -- 172.21.15.49:0/4242024862 >> 172.21.15.49:0/4242024862 conn(0x7ff2400fb700 msgr2=0x7ff2400fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.174+0000 7ff247e0b700 1 -- 172.21.15.49:0/4242024862 shutdown_connections 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.174+0000 7ff247e0b700 1 -- 172.21.15.49:0/4242024862 wait complete. 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.174+0000 7ff247e0b700 1 Processor -- start 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.174+0000 7ff247e0b700 1 -- start start 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff247e0b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 0x7ff2401018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff247e0b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2401007c0 0x7ff240101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff247e0b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240102330 con 0x7ff2400ffe10 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff247e0b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2401024a0 con 0x7ff2401007c0 2024-05-15T15:49:27.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff245ba7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 0x7ff2401018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff245ba7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 0x7ff2401018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34900/0 (socket says 172.21.15.49:34900) 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff245ba7700 1 -- 172.21.15.49:0/3266679561 learned_addr learned my addr 172.21.15.49:0/3266679561 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff2453a6700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2401007c0 0x7ff240101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff245ba7700 1 -- 172.21.15.49:0/3266679561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2401007c0 msgr2=0x7ff240101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff245ba7700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2401007c0 0x7ff240101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff245ba7700 1 -- 172.21.15.49:0/3266679561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0095f0 con 0x7ff2400ffe10 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.175+0000 7ff245ba7700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 0x7ff2401018b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006010 tx=0x7ff23c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.176+0000 7ff236ffd700 1 -- 172.21.15.49:0/3266679561 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff23c01d070 con 0x7ff2400ffe10 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.176+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff240102720 con 0x7ff2400ffe10 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.176+0000 7ff236ffd700 1 -- 172.21.15.49:0/3266679561 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff23c00f460 con 0x7ff2400ffe10 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.176+0000 7ff236ffd700 1 -- 172.21.15.49:0/3266679561 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff23c017630 con 0x7ff2400ffe10 2024-05-15T15:49:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.176+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2401a30b0 con 0x7ff2400ffe10 2024-05-15T15:49:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.177+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2401012f0 con 0x7ff2400ffe10 2024-05-15T15:49:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.177+0000 7ff236ffd700 1 -- 172.21.15.49:0/3266679561 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff23c00f5d0 con 0x7ff2400ffe10 2024-05-15T15:49:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.178+0000 7ff236ffd700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff22c06ccf0 0x7ff22c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.178+0000 7ff2453a6700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff22c06ccf0 0x7ff22c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.178+0000 7ff236ffd700 1 -- 172.21.15.49:0/3266679561 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff23c08d2d0 con 0x7ff2400ffe10 2024-05-15T15:49:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.178+0000 7ff2453a6700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff22c06ccf0 0x7ff22c06f1b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff240102f90 tx=0x7ff230005eb0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.183+0000 7ff236ffd700 1 -- 172.21.15.49:0/3266679561 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff23c057aa0 con 0x7ff2400ffe10 2024-05-15T15:49:27.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.388+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff240102720 con 0x7ff2400ffe10 2024-05-15T15:49:27.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.389+0000 7ff236ffd700 1 -- 172.21.15.49:0/3266679561 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11321 (secure 0 0 0) 0x7ff240102720 con 0x7ff2400ffe10 2024-05-15T15:49:27.392 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:49:27.392 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":41,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","created":"2024-05-15T15:43:35.648208+0000","modified":"2024-05-15T15:49:25.493977+0000","last_up_change":"2024-05-15T15:49:24.492659+0000","last_in_change":"2024-05-15T15:49:04.722203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T15:48:09.899234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25d1eb06-bee2-48a6-8d86-30b5b3b348d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6803","nonce":941734370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6805","nonce":941734370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6809","nonce":941734370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6807","nonce":941734370}]},"public_addr":"172.21.15.49:6803/941734370","cluster_addr":"172.21.15.49:6805/941734370","heartbeat_back_addr":"172.21.15.49:6809/941734370","heartbeat_front_addr":"172.21.15.49:6807/941734370","state":["exists","up"]},{"osd":1,"uuid":"c0e717cf-ae5c-4fa4-a30d-d2235ff27315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6811","nonce":717954912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6813","nonce":717954912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6817","nonce":717954912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6815","nonce":717954912}]},"public_addr":"172.21.15.49:6811/717954912","cluster_addr":"172.21.15.49:6813/717954912","heartbeat_back_addr":"172.21.15.49:6817/717954912","heartbeat_front_addr":"172.21.15.49:6815/717954912","state":["exists","up"]},{"osd":2,"uuid":"d7752daf-dd5b-49cc-abc2-ca953968e542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6819","nonce":2295619409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6821","nonce":2295619409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6825","nonce":2295619409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6823","nonce":2295619409}]},"public_addr":"172.21.15.49:6819/2295619409","cluster_addr":"172.21.15.49:6821/2295619409","heartbeat_back_addr":"172.21.15.49:6825/2295619409","heartbeat_front_addr":"172.21.15.49:6823/2295619409","state":["exists","up"]},{"osd":3,"uuid":"2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6801","nonce":1650511241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6803","nonce":1650511241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6807","nonce":1650511241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6805","nonce":1650511241}]},"public_addr":"172.21.15.144:6801/1650511241","cluster_addr":"172.21.15.144:6803/1650511241","heartbeat_back_addr":"172.21.15.144:6807/1650511241","heartbeat_front_addr":"172.21.15.144:6805/1650511241","state":["exists","up"]},{"osd":4,"uuid":"6659631f-da39-4a2c-9f75-e7b4a06b602f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6809","nonce":4251193981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6811","nonce":4251193981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6815","nonce":4251193981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6813","nonce":4251193981}]},"public_addr":"172.21.15.144:6809/4251193981","cluster_addr":"172.21.15.144:6811/4251193981","heartbeat_back_addr":"172.21.15.144:6815/4251193981","heartbeat_front_addr":"172.21.15.144:6813/4251193981","state":["exists","up"]},{"osd":5,"uuid":"51ea2d85-6cf7-4a96-8c74-612958083a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6817","nonce":1897088829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6819","nonce":1897088829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6823","nonce":1897088829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6821","nonce":1897088829}]},"public_addr":"172.21.15.144:6817/1897088829","cluster_addr":"172.21.15.144:6819/1897088829","heartbeat_back_addr":"172.21.15.144:6823/1897088829","heartbeat_front_addr":"172.21.15.144:6821/1897088829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:10.108689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:37.458152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:05.915664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:30.194669+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:55.748301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:49:22.635948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/644218315":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6800/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/1173274751":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/3014161738":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/75769186":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6801/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/3452717560":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/4010845594":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/2163344179":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/4024404689":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/940476764":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6800/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6801/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6801/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6800/821763420":"2024-05-16T15:44:39.328329+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T15:49:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff22c06ccf0 msgr2=0x7ff22c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff22c06ccf0 0x7ff22c06f1b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff240102f90 tx=0x7ff230005eb0 comp rx=0 tx=0).stop 2024-05-15T15:49:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 msgr2=0x7ff2401018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 0x7ff2401018b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006010 tx=0x7ff23c00be20 comp rx=0 tx=0).stop 2024-05-15T15:49:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 shutdown_connections 2024-05-15T15:49:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2400ffe10 0x7ff2401018b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff22c06ccf0 0x7ff22c06f1b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 --2- 172.21.15.49:0/3266679561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2401007c0 0x7ff240101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 >> 172.21.15.49:0/3266679561 conn(0x7ff2400fb700 msgr2=0x7ff240069670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:27.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 shutdown_connections 2024-05-15T15:49:27.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:27.391+0000 7ff247e0b700 1 -- 172.21.15.49:0/3266679561 wait complete. 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: Detected new or changed devices on smithi144 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: Adjusting osd_memory_target on smithi144 to 4970M 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:49:27.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:27.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:49:27.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:27 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3266679561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T15:49:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: Detected new or changed devices on smithi144 2024-05-15T15:49:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: Adjusting osd_memory_target on smithi144 to 4970M 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:49:27.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:27 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3266679561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T15:49:28.140 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-15T15:48:09.899234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-15T15:49:28.140 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-15T15:49:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:28.745 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:28 smithi049 ceph-mon[28399]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:29.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:28 smithi144 ceph-mon[36389]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:30.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.202+0000 7f51dcfae700 1 -- 172.21.15.49:0/2546713594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 msgr2=0x7f51d8102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:30.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.202+0000 7f51dcfae700 1 --2- 172.21.15.49:0/2546713594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 0x7f51d8102240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009910 tx=0x7f51c8009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:30.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.202+0000 7f51dcfae700 1 -- 172.21.15.49:0/2546713594 shutdown_connections 2024-05-15T15:49:30.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.202+0000 7f51dcfae700 1 --2- 172.21.15.49:0/2546713594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d8102780 0x7f51d810ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:30.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.202+0000 7f51dcfae700 1 --2- 172.21.15.49:0/2546713594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 0x7f51d8102240 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:30.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.202+0000 7f51dcfae700 1 -- 172.21.15.49:0/2546713594 >> 172.21.15.49:0/2546713594 conn(0x7f51d8076800 msgr2=0x7f51d80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:30.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.202+0000 7f51dcfae700 1 -- 172.21.15.49:0/2546713594 shutdown_connections 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.202+0000 7f51dcfae700 1 -- 172.21.15.49:0/2546713594 wait complete. 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.203+0000 7f51dcfae700 1 Processor -- start 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.203+0000 7f51dcfae700 1 -- start start 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.203+0000 7f51dcfae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 0x7f51d819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.203+0000 7f51dcfae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d8102780 0x7f51d819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.203+0000 7f51dcfae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d819d510 con 0x7f51d8101e60 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.203+0000 7f51dcfae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8196af0 con 0x7f51d8102780 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.203+0000 7f51d659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 0x7f51d819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.203+0000 7f51d659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 0x7f51d819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34924/0 (socket says 172.21.15.49:34924) 2024-05-15T15:49:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.203+0000 7f51d659c700 1 -- 172.21.15.49:0/3100115525 learned_addr learned my addr 172.21.15.49:0/3100115525 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:30.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51d5d9b700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d8102780 0x7f51d819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:30.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51d659c700 1 -- 172.21.15.49:0/3100115525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d8102780 msgr2=0x7f51d819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:30.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51d659c700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d8102780 0x7f51d819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:30.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51d659c700 1 -- 172.21.15.49:0/3100115525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c80095f0 con 0x7f51d8101e60 2024-05-15T15:49:30.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51d659c700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 0x7f51d819c9b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f51c8006010 tx=0x7f51c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:30.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51cf7fe700 1 -- 172.21.15.49:0/3100115525 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c801d070 con 0x7f51d8101e60 2024-05-15T15:49:30.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51cf7fe700 1 -- 172.21.15.49:0/3100115525 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51c800f460 con 0x7f51d8101e60 2024-05-15T15:49:30.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d8196d70 con 0x7f51d8101e60 2024-05-15T15:49:30.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51cf7fe700 1 -- 172.21.15.49:0/3100115525 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c8017630 con 0x7f51d8101e60 2024-05-15T15:49:30.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.204+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d8197260 con 0x7f51d8101e60 2024-05-15T15:49:30.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.205+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d81055f0 con 0x7f51d8101e60 2024-05-15T15:49:30.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.206+0000 7f51cf7fe700 1 -- 172.21.15.49:0/3100115525 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f51c800f5d0 con 0x7f51d8101e60 2024-05-15T15:49:30.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.206+0000 7f51cf7fe700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f51c406ccf0 0x7f51c406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:30.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.207+0000 7f51d5d9b700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f51c406ccf0 0x7f51c406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:30.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.207+0000 7f51cf7fe700 1 -- 172.21.15.49:0/3100115525 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f51c808d460 con 0x7f51d8101e60 2024-05-15T15:49:30.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.207+0000 7f51d5d9b700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f51c406ccf0 0x7f51c406f1b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f51c0006fd0 tx=0x7f51c0008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:30.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.212+0000 7f51cf7fe700 1 -- 172.21.15.49:0/3100115525 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f51c805b6c0 con 0x7f51d8101e60 2024-05-15T15:49:30.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.414+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51d810df10 con 0x7f51d8101e60 2024-05-15T15:49:30.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.415+0000 7f51cf7fe700 1 -- 172.21.15.49:0/3100115525 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51c805b250 con 0x7f51d8101e60 2024-05-15T15:49:30.417 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-05-15T15:49:30.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f51c406ccf0 msgr2=0x7f51c406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:30.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f51c406ccf0 0x7f51c406f1b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f51c0006fd0 tx=0x7f51c0008040 comp rx=0 tx=0).stop 2024-05-15T15:49:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 msgr2=0x7f51d819c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 0x7f51d819c9b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f51c8006010 tx=0x7f51c800be20 comp rx=0 tx=0).stop 2024-05-15T15:49:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 shutdown_connections 2024-05-15T15:49:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d8101e60 0x7f51d819c9b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f51c406ccf0 0x7f51c406f1b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 --2- 172.21.15.49:0/3100115525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d8102780 0x7f51d819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 >> 172.21.15.49:0/3100115525 conn(0x7f51d8076800 msgr2=0x7f51d8103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 shutdown_connections 2024-05-15T15:49:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:30.417+0000 7f51dcfae700 1 -- 172.21.15.49:0/3100115525 wait complete. 2024-05-15T15:49:31.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:30 smithi049 ceph-mon[28399]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:31.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:30 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3100115525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T15:49:31.087 INFO:tasks.cephadm:Setting up client nodes... 2024-05-15T15:49:31.088 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-15T15:49:31.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:30 smithi144 ceph-mon[36389]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:31.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:30 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3100115525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T15:49:31.301 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:32.813 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:32 smithi049 ceph-mon[28399]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:33.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.171+0000 7ff458566700 1 -- 172.21.15.49:0/1810467007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff450101210 msgr2=0x7ff450101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:33.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.171+0000 7ff458566700 1 --2- 172.21.15.49:0/1810467007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff450101210 0x7ff450101630 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009910 tx=0x7ff44c009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:33.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.172+0000 7ff458566700 1 -- 172.21.15.49:0/1810467007 shutdown_connections 2024-05-15T15:49:33.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.172+0000 7ff458566700 1 --2- 172.21.15.49:0/1810467007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4500feda0 0x7ff4500ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:33.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.172+0000 7ff458566700 1 --2- 172.21.15.49:0/1810467007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff450101210 0x7ff450101630 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:33.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.172+0000 7ff458566700 1 -- 172.21.15.49:0/1810467007 >> 172.21.15.49:0/1810467007 conn(0x7ff4500fa940 msgr2=0x7ff4500fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.172+0000 7ff458566700 1 -- 172.21.15.49:0/1810467007 shutdown_connections 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.172+0000 7ff458566700 1 -- 172.21.15.49:0/1810467007 wait complete. 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.172+0000 7ff458566700 1 Processor -- start 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.172+0000 7ff458566700 1 -- start start 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.172+0000 7ff458566700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4500feda0 0x7ff450198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff458566700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff450101210 0x7ff450198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff458566700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff450199490 con 0x7ff4500feda0 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff458566700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501995d0 con 0x7ff450101210 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff456302700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4500feda0 0x7ff450198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff456302700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4500feda0 0x7ff450198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34930/0 (socket says 172.21.15.49:34930) 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff456302700 1 -- 172.21.15.49:0/3942895804 learned_addr learned my addr 172.21.15.49:0/3942895804 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff455b01700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff450101210 0x7ff450198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff456302700 1 -- 172.21.15.49:0/3942895804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff450101210 msgr2=0x7ff450198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff456302700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff450101210 0x7ff450198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff456302700 1 -- 172.21.15.49:0/3942895804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c0095f0 con 0x7ff4500feda0 2024-05-15T15:49:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff456302700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4500feda0 0x7ff450198930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0059d0 tx=0x7ff44c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff4477fe700 1 -- 172.21.15.49:0/3942895804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff44c01d070 con 0x7ff4500feda0 2024-05-15T15:49:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff4477fe700 1 -- 172.21.15.49:0/3942895804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff44c00f460 con 0x7ff4500feda0 2024-05-15T15:49:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff4477fe700 1 -- 172.21.15.49:0/3942895804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff44c021640 con 0x7ff4500feda0 2024-05-15T15:49:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45019df80 con 0x7ff4500feda0 2024-05-15T15:49:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.173+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45019e470 con 0x7ff4500feda0 2024-05-15T15:49:33.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.174+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff450192aa0 con 0x7ff4500feda0 2024-05-15T15:49:33.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.175+0000 7ff4477fe700 1 -- 172.21.15.49:0/3942895804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff44c00f5d0 con 0x7ff4500feda0 2024-05-15T15:49:33.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.176+0000 7ff4477fe700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff43c06cce0 0x7ff43c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:33.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.176+0000 7ff4477fe700 1 -- 172.21.15.49:0/3942895804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff44c08d070 con 0x7ff4500feda0 2024-05-15T15:49:33.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.176+0000 7ff455b01700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff43c06cce0 0x7ff43c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:33.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.176+0000 7ff455b01700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff43c06cce0 0x7ff43c06f1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff440004610 tx=0x7ff44000b3c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:33.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.181+0000 7ff4477fe700 1 -- 172.21.15.49:0/3942895804 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff44c05b290 con 0x7ff4500feda0 2024-05-15T15:49:33.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:32 smithi144 ceph-mon[36389]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:33.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.467+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff45004ecb0 con 0x7ff4500feda0 2024-05-15T15:49:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.471+0000 7ff4477fe700 1 -- 172.21.15.49:0/3942895804 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff44c05ae20 con 0x7ff4500feda0 2024-05-15T15:49:33.474 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-05-15T15:49:33.474 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCN2URmAn//GxAAO/jY+yVHPr27IVXxJkDYAw== 2024-05-15T15:49:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.473+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff43c06cce0 msgr2=0x7ff43c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.473+0000 7ff458566700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff43c06cce0 0x7ff43c06f1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff440004610 tx=0x7ff44000b3c0 comp rx=0 tx=0).stop 2024-05-15T15:49:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.473+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4500feda0 msgr2=0x7ff450198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.473+0000 7ff458566700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4500feda0 0x7ff450198930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0059d0 tx=0x7ff44c00bd70 comp rx=0 tx=0).stop 2024-05-15T15:49:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.473+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 shutdown_connections 2024-05-15T15:49:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.473+0000 7ff458566700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4500feda0 0x7ff450198930 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.473+0000 7ff458566700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff43c06cce0 0x7ff43c06f1a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:33.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.473+0000 7ff458566700 1 --2- 172.21.15.49:0/3942895804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff450101210 0x7ff450198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:33.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.473+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 >> 172.21.15.49:0/3942895804 conn(0x7ff4500fa940 msgr2=0x7ff4501088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:33.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.474+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 shutdown_connections 2024-05-15T15:49:33.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:33.474+0000 7ff458566700 1 -- 172.21.15.49:0/3942895804 wait complete. 2024-05-15T15:49:33.761 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:33 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3942895804' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-15T15:49:33.761 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:33 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3942895804' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-15T15:49:34.104 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:49:34.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-15T15:49:34.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-15T15:49:34.146 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-15T15:49:34.175 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:33 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3942895804' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-15T15:49:34.175 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:33 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3942895804' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-15T15:49:34.366 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T15:49:34.799 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:34 smithi144 ceph-mon[36389]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:35.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:34 smithi049 ceph-mon[28399]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:35.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.929+0000 7f08b3d4d700 1 -- 172.21.15.144:0/25688038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac0ffbe0 msgr2=0x7f08ac100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:35.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.929+0000 7f08b3d4d700 1 --2- 172.21.15.144:0/25688038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac0ffbe0 0x7f08ac100000 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009910 tx=0x7f08a8009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:35.932 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.929+0000 7f08b3d4d700 1 -- 172.21.15.144:0/25688038 shutdown_connections 2024-05-15T15:49:35.932 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.929+0000 7f08b3d4d700 1 --2- 172.21.15.144:0/25688038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08ac100de0 0x7f08ac101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:35.932 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.929+0000 7f08b3d4d700 1 --2- 172.21.15.144:0/25688038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac0ffbe0 0x7f08ac100000 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:35.932 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.929+0000 7f08b3d4d700 1 -- 172.21.15.144:0/25688038 >> 172.21.15.144:0/25688038 conn(0x7f08ac0fb1a0 msgr2=0x7f08ac0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:35.932 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.929+0000 7f08b3d4d700 1 -- 172.21.15.144:0/25688038 shutdown_connections 2024-05-15T15:49:35.932 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.929+0000 7f08b3d4d700 1 -- 172.21.15.144:0/25688038 wait complete. 2024-05-15T15:49:35.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b3d4d700 1 Processor -- start 2024-05-15T15:49:35.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b3d4d700 1 -- start start 2024-05-15T15:49:35.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b3d4d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08ac0ffbe0 0x7f08ac1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:35.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b3d4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac100de0 0x7f08ac198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b3d4d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac199550 con 0x7f08ac0ffbe0 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b3d4d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac199690 con 0x7f08ac100de0 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b12e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac100de0 0x7f08ac198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b12e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac100de0 0x7f08ac198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46136/0 (socket says 172.21.15.144:46136) 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b12e8700 1 -- 172.21.15.144:0/1446319589 learned_addr learned my addr 172.21.15.144:0/1446319589 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b1ae9700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08ac0ffbe0 0x7f08ac1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b12e8700 1 -- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08ac0ffbe0 msgr2=0x7f08ac1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b12e8700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08ac0ffbe0 0x7f08ac1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b12e8700 1 -- 172.21.15.144:0/1446319589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a80095f0 con 0x7f08ac100de0 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b1ae9700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08ac0ffbe0 0x7f08ac1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08b12e8700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac100de0 0x7f08ac198f30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f089c00b4b0 tx=0x7f089c00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.930+0000 7f08a2ffd700 1 -- 172.21.15.144:0/1446319589 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f089c011800 con 0x7f08ac100de0 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.931+0000 7f08a2ffd700 1 -- 172.21.15.144:0/1446319589 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f089c011e20 con 0x7f08ac100de0 2024-05-15T15:49:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.931+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac19e0f0 con 0x7f08ac100de0 2024-05-15T15:49:35.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.931+0000 7f08a2ffd700 1 -- 172.21.15.144:0/1446319589 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f089c00f2e0 con 0x7f08ac100de0 2024-05-15T15:49:35.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.931+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac19e6c0 con 0x7f08ac100de0 2024-05-15T15:49:35.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.931+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac04ec60 con 0x7f08ac100de0 2024-05-15T15:49:35.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.932+0000 7f08a2ffd700 1 -- 172.21.15.144:0/1446319589 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f089c010380 con 0x7f08ac100de0 2024-05-15T15:49:35.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.932+0000 7f08a2ffd700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f089806cce0 0x7f089806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:35.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.933+0000 7f08a2ffd700 1 -- 172.21.15.144:0/1446319589 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f089c08baa0 con 0x7f08ac100de0 2024-05-15T15:49:35.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.933+0000 7f08b1ae9700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f089806cce0 0x7f089806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:35.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.933+0000 7f08b1ae9700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f089806cce0 0x7f089806f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08a8006010 tx=0x7f08a8005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:35.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:35.937+0000 7f08a2ffd700 1 -- 172.21.15.144:0/1446319589 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f089c059d70 con 0x7f08ac100de0 2024-05-15T15:49:36.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.210+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f08ac1087c0 con 0x7f08ac100de0 2024-05-15T15:49:36.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.217+0000 7f08a2ffd700 1 -- 172.21.15.144:0/1446319589 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f089c059900 con 0x7f08ac100de0 2024-05-15T15:49:36.220 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2024-05-15T15:49:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCQ2URmU++sDBAAdGgBaacnTJYfRN2o9aOYvw== 2024-05-15T15:49:36.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f089806cce0 msgr2=0x7f089806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:36.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f089806cce0 0x7f089806f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08a8006010 tx=0x7f08a8005920 comp rx=0 tx=0).stop 2024-05-15T15:49:36.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac100de0 msgr2=0x7f08ac198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:36.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac100de0 0x7f08ac198f30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f089c00b4b0 tx=0x7f089c00b870 comp rx=0 tx=0).stop 2024-05-15T15:49:36.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 shutdown_connections 2024-05-15T15:49:36.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08ac0ffbe0 0x7f08ac1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:36.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f089806cce0 0x7f089806f1a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:36.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 --2- 172.21.15.144:0/1446319589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08ac100de0 0x7f08ac198f30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:36.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 >> 172.21.15.144:0/1446319589 conn(0x7f08ac0fb1a0 msgr2=0x7f08ac069180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:36.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 shutdown_connections 2024-05-15T15:49:36.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:49:36.220+0000 7f08b3d4d700 1 -- 172.21.15.144:0/1446319589 wait complete. 2024-05-15T15:49:36.883 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:36 smithi144 ceph-mon[36389]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:36.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:36 smithi144 ceph-mon[36389]: from='client.? 172.21.15.144:0/1446319589' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-15T15:49:36.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:36 smithi144 ceph-mon[36389]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-15T15:49:36.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:36 smithi144 ceph-mon[36389]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-15T15:49:36.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:49:36.908 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:49:36.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-05-15T15:49:36.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-15T15:49:36.988 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-15T15:49:36.989 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-15T15:49:36.989 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-15T15:49:37.016 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:36 smithi049 ceph-mon[28399]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:37.016 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:36 smithi049 ceph-mon[28399]: from='client.? 172.21.15.144:0/1446319589' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-15T15:49:37.016 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:36 smithi049 ceph-mon[28399]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-15T15:49:37.016 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:36 smithi049 ceph-mon[28399]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-15T15:49:37.016 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:49:37.214 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:38.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.841+0000 7f251fb57700 1 -- 172.21.15.49:0/2902669632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 msgr2=0x7f25181058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.841+0000 7f251fb57700 1 --2- 172.21.15.49:0/2902669632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 0x7f25181058e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2508009910 tx=0x7f2508009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.841+0000 7f251fb57700 1 -- 172.21.15.49:0/2902669632 shutdown_connections 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.841+0000 7f251fb57700 1 --2- 172.21.15.49:0/2902669632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 0x7f25181058e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.841+0000 7f251fb57700 1 --2- 172.21.15.49:0/2902669632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2518100ee0 0x7f25181012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.841+0000 7f251fb57700 1 -- 172.21.15.49:0/2902669632 >> 172.21.15.49:0/2902669632 conn(0x7f25180fc790 msgr2=0x7f25180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.841+0000 7f251fb57700 1 -- 172.21.15.49:0/2902669632 shutdown_connections 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.841+0000 7f251fb57700 1 -- 172.21.15.49:0/2902669632 wait complete. 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251fb57700 1 Processor -- start 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251fb57700 1 -- start start 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251fb57700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2518100ee0 0x7f251819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251fb57700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 0x7f251819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251fb57700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251819d440 con 0x7f2518101890 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251fb57700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2518196a20 con 0x7f2518100ee0 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251d0f2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 0x7f251819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251d0f2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 0x7f251819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34334/0 (socket says 172.21.15.49:34334) 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251d0f2700 1 -- 172.21.15.49:0/384439730 learned_addr learned my addr 172.21.15.49:0/384439730 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.842+0000 7f251d8f3700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2518100ee0 0x7f251819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.843+0000 7f251d0f2700 1 -- 172.21.15.49:0/384439730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2518100ee0 msgr2=0x7f251819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.843+0000 7f251d0f2700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2518100ee0 0x7f251819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.843+0000 7f251d0f2700 1 -- 172.21.15.49:0/384439730 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25080095f0 con 0x7f2518101890 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.843+0000 7f251d0f2700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 0x7f251819ce20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f250800bd70 tx=0x7f250800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.843+0000 7f250effd700 1 -- 172.21.15.49:0/384439730 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f250801d070 con 0x7f2518101890 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.843+0000 7f250effd700 1 -- 172.21.15.49:0/384439730 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25080056e0 con 0x7f2518101890 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.843+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2518196ca0 con 0x7f2518101890 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.843+0000 7f250effd700 1 -- 172.21.15.49:0/384439730 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2508017400 con 0x7f2518101890 2024-05-15T15:49:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.843+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2518197190 con 0x7f2518101890 2024-05-15T15:49:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.844+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25181092e0 con 0x7f2518101890 2024-05-15T15:49:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.845+0000 7f250effd700 1 -- 172.21.15.49:0/384439730 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f250800f460 con 0x7f2518101890 2024-05-15T15:49:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.845+0000 7f250effd700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f250406cce0 0x7f250406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:38.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.845+0000 7f250effd700 1 -- 172.21.15.49:0/384439730 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f250808d260 con 0x7f2518101890 2024-05-15T15:49:38.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.845+0000 7f251d8f3700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f250406cce0 0x7f250406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:38.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.846+0000 7f251d8f3700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f250406cce0 0x7f250406f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2514005950 tx=0x7f25140058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:38.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:38.850+0000 7f250effd700 1 -- 172.21.15.49:0/384439730 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f250805b540 con 0x7f2518101890 2024-05-15T15:49:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:38 smithi049 ceph-mon[28399]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:39.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.107+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f25181023c0 con 0x7f2518101890 2024-05-15T15:49:39.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.111+0000 7f250effd700 1 -- 172.21.15.49:0/384439730 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+174013 (secure 0 0 0) 0x7f250805b0d0 con 0x7f2518101890 2024-05-15T15:49:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.113+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f250406cce0 msgr2=0x7f250406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.113+0000 7f251fb57700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f250406cce0 0x7f250406f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2514005950 tx=0x7f25140058e0 comp rx=0 tx=0).stop 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.113+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 msgr2=0x7f251819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.113+0000 7f251fb57700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 0x7f251819ce20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f250800bd70 tx=0x7f250800be50 comp rx=0 tx=0).stop 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.113+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 shutdown_connections 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.113+0000 7f251fb57700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f250406cce0 0x7f250406f1a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.113+0000 7f251fb57700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2518100ee0 0x7f251819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.113+0000 7f251fb57700 1 --2- 172.21.15.49:0/384439730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2518101890 0x7f251819ce20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.113+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 >> 172.21.15.49:0/384439730 conn(0x7f25180fc790 msgr2=0x7f25180fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.114+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 shutdown_connections 2024-05-15T15:49:39.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:39.114+0000 7f251fb57700 1 -- 172.21.15.49:0/384439730 wait complete. 2024-05-15T15:49:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:38 smithi144 ceph-mon[36389]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:39.764 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":19,"flags":0,"active_gid":14213,"active_name":"smithi049.vuhzgf","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1448243308},{"type":"v1","addr":"172.21.15.49:6801","nonce":1448243308}]},"active_addr":"172.21.15.49:6801/1448243308","active_change":"2024-05-15T15:45:51.392805+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14240,"name":"smithi144.whcjkr","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.49:8443/","prometheus":"http://172.21.15.49:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1166053004}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1904525014}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":2543812499}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":2464608729}]}]} 2024-05-15T15:49:39.769 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-15T15:49:39.769 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-15T15:49:39.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T15:49:40.020 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:40.142 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:39 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/384439730' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-15T15:49:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:39 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/384439730' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-15T15:49:41.122 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:40 smithi049 ceph-mon[28399]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:40 smithi144 ceph-mon[36389]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.639+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/526460491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 msgr2=0x7f5bf01047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.639+0000 7f5bf5ec5700 1 --2- 172.21.15.49:0/526460491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 0x7f5bf01047d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009910 tx=0x7f5bd8009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.639+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/526460491 shutdown_connections 2024-05-15T15:49:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.639+0000 7f5bf5ec5700 1 --2- 172.21.15.49:0/526460491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 0x7f5bf01047d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.639+0000 7f5bf5ec5700 1 --2- 172.21.15.49:0/526460491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bf00ffdd0 0x7f5bf01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.639+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/526460491 >> 172.21.15.49:0/526460491 conn(0x7f5bf00fb6a0 msgr2=0x7f5bf00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.639+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/526460491 shutdown_connections 2024-05-15T15:49:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.639+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/526460491 wait complete. 2024-05-15T15:49:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5bf5ec5700 1 Processor -- start 2024-05-15T15:49:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5bf5ec5700 1 -- start start 2024-05-15T15:49:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5bf5ec5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bf00ffdd0 0x7f5bf019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5bf5ec5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 0x7f5bf019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5bf5ec5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf019d470 con 0x7f5bf0100780 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5bf5ec5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf01969e0 con 0x7f5bf00ffdd0 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5beeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 0x7f5bf019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5beeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 0x7f5bf019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34352/0 (socket says 172.21.15.49:34352) 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5beeffd700 1 -- 172.21.15.49:0/744180141 learned_addr learned my addr 172.21.15.49:0/744180141 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5bef7fe700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bf00ffdd0 0x7f5bf019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5beeffd700 1 -- 172.21.15.49:0/744180141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bf00ffdd0 msgr2=0x7f5bf019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5beeffd700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bf00ffdd0 0x7f5bf019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.640+0000 7f5beeffd700 1 -- 172.21.15.49:0/744180141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd80095f0 con 0x7f5bf0100780 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.641+0000 7f5beeffd700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 0x7f5bf019cde0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80092e0 tx=0x7f5bd800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.641+0000 7f5becff9700 1 -- 172.21.15.49:0/744180141 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bd801d070 con 0x7f5bf0100780 2024-05-15T15:49:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.641+0000 7f5becff9700 1 -- 172.21.15.49:0/744180141 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bd8005700 con 0x7f5bf0100780 2024-05-15T15:49:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.641+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf0196c60 con 0x7f5bf0100780 2024-05-15T15:49:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.641+0000 7f5becff9700 1 -- 172.21.15.49:0/744180141 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bd800fdb0 con 0x7f5bf0100780 2024-05-15T15:49:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.641+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf0197150 con 0x7f5bf0100780 2024-05-15T15:49:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.642+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf01012b0 con 0x7f5bf0100780 2024-05-15T15:49:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.642+0000 7f5becff9700 1 -- 172.21.15.49:0/744180141 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5bd800f5f0 con 0x7f5bf0100780 2024-05-15T15:49:41.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.643+0000 7f5becff9700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f5bdc06cce0 0x7f5bdc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:41.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.643+0000 7f5bef7fe700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f5bdc06cce0 0x7f5bdc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:41.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.643+0000 7f5becff9700 1 -- 172.21.15.49:0/744180141 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5bd808d1f0 con 0x7f5bf0100780 2024-05-15T15:49:41.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.643+0000 7f5bef7fe700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f5bdc06cce0 0x7f5bdc06f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009550 tx=0x7f5be00091b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:41.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.648+0000 7f5becff9700 1 -- 172.21.15.49:0/744180141 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5bd805b390 con 0x7f5bf0100780 2024-05-15T15:49:41.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.853+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bf004ec60 con 0x7f5bf0100780 2024-05-15T15:49:41.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.854+0000 7f5becff9700 1 -- 172.21.15.49:0/744180141 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11321 (secure 0 0 0) 0x7f5bd805af20 con 0x7f5bf0100780 2024-05-15T15:49:41.856 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:49:41.856 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":41,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","created":"2024-05-15T15:43:35.648208+0000","modified":"2024-05-15T15:49:25.493977+0000","last_up_change":"2024-05-15T15:49:24.492659+0000","last_in_change":"2024-05-15T15:49:04.722203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T15:48:09.899234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25d1eb06-bee2-48a6-8d86-30b5b3b348d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6803","nonce":941734370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6805","nonce":941734370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6809","nonce":941734370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6807","nonce":941734370}]},"public_addr":"172.21.15.49:6803/941734370","cluster_addr":"172.21.15.49:6805/941734370","heartbeat_back_addr":"172.21.15.49:6809/941734370","heartbeat_front_addr":"172.21.15.49:6807/941734370","state":["exists","up"]},{"osd":1,"uuid":"c0e717cf-ae5c-4fa4-a30d-d2235ff27315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6811","nonce":717954912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6813","nonce":717954912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6817","nonce":717954912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6815","nonce":717954912}]},"public_addr":"172.21.15.49:6811/717954912","cluster_addr":"172.21.15.49:6813/717954912","heartbeat_back_addr":"172.21.15.49:6817/717954912","heartbeat_front_addr":"172.21.15.49:6815/717954912","state":["exists","up"]},{"osd":2,"uuid":"d7752daf-dd5b-49cc-abc2-ca953968e542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6819","nonce":2295619409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6821","nonce":2295619409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6825","nonce":2295619409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6823","nonce":2295619409}]},"public_addr":"172.21.15.49:6819/2295619409","cluster_addr":"172.21.15.49:6821/2295619409","heartbeat_back_addr":"172.21.15.49:6825/2295619409","heartbeat_front_addr":"172.21.15.49:6823/2295619409","state":["exists","up"]},{"osd":3,"uuid":"2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6801","nonce":1650511241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6803","nonce":1650511241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6807","nonce":1650511241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6805","nonce":1650511241}]},"public_addr":"172.21.15.144:6801/1650511241","cluster_addr":"172.21.15.144:6803/1650511241","heartbeat_back_addr":"172.21.15.144:6807/1650511241","heartbeat_front_addr":"172.21.15.144:6805/1650511241","state":["exists","up"]},{"osd":4,"uuid":"6659631f-da39-4a2c-9f75-e7b4a06b602f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6809","nonce":4251193981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6811","nonce":4251193981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6815","nonce":4251193981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6813","nonce":4251193981}]},"public_addr":"172.21.15.144:6809/4251193981","cluster_addr":"172.21.15.144:6811/4251193981","heartbeat_back_addr":"172.21.15.144:6815/4251193981","heartbeat_front_addr":"172.21.15.144:6813/4251193981","state":["exists","up"]},{"osd":5,"uuid":"51ea2d85-6cf7-4a96-8c74-612958083a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6817","nonce":1897088829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6819","nonce":1897088829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6823","nonce":1897088829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6821","nonce":1897088829}]},"public_addr":"172.21.15.144:6817/1897088829","cluster_addr":"172.21.15.144:6819/1897088829","heartbeat_back_addr":"172.21.15.144:6823/1897088829","heartbeat_front_addr":"172.21.15.144:6821/1897088829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:10.108689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:37.458152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:05.915664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:30.194669+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:55.748301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:49:22.635948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/644218315":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6800/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/1173274751":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/3014161738":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/75769186":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6801/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/3452717560":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/4010845594":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/2163344179":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/4024404689":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/940476764":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6800/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6801/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6801/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6800/821763420":"2024-05-16T15:44:39.328329+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T15:49:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f5bdc06cce0 msgr2=0x7f5bdc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f5bdc06cce0 0x7f5bdc06f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009550 tx=0x7f5be00091b0 comp rx=0 tx=0).stop 2024-05-15T15:49:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 msgr2=0x7f5bf019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 0x7f5bf019cde0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80092e0 tx=0x7f5bd800bd70 comp rx=0 tx=0).stop 2024-05-15T15:49:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 shutdown_connections 2024-05-15T15:49:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f5bdc06cce0 0x7f5bdc06f1a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bf00ffdd0 0x7f5bf019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 --2- 172.21.15.49:0/744180141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5bf0100780 0x7f5bf019cde0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 >> 172.21.15.49:0/744180141 conn(0x7f5bf00fb6a0 msgr2=0x7f5bf00fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:41.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.856+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 shutdown_connections 2024-05-15T15:49:41.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:41.857+0000 7f5bf5ec5700 1 -- 172.21.15.49:0/744180141 wait complete. 2024-05-15T15:49:42.591 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-15T15:49:42.591 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T15:49:42.814 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:42.936 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:42 smithi049 ceph-mon[28399]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:42.936 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:42 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/744180141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T15:49:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:42 smithi144 ceph-mon[36389]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:42 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/744180141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T15:49:44.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.452+0000 7fce7356c700 1 -- 172.21.15.49:0/30405424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c0ff110 msgr2=0x7fce6c0ff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:44.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.452+0000 7fce7356c700 1 --2- 172.21.15.49:0/30405424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c0ff110 0x7fce6c0ff590 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fce60009910 tx=0x7fce60009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:44.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.452+0000 7fce7356c700 1 -- 172.21.15.49:0/30405424 shutdown_connections 2024-05-15T15:49:44.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.452+0000 7fce7356c700 1 --2- 172.21.15.49:0/30405424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c0ff110 0x7fce6c0ff590 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:44.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.452+0000 7fce7356c700 1 --2- 172.21.15.49:0/30405424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c105cf0 0x7fce6c1060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:44.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.452+0000 7fce7356c700 1 -- 172.21.15.49:0/30405424 >> 172.21.15.49:0/30405424 conn(0x7fce6c076000 msgr2=0x7fce6c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:44.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.453+0000 7fce7356c700 1 -- 172.21.15.49:0/30405424 shutdown_connections 2024-05-15T15:49:44.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.453+0000 7fce7356c700 1 -- 172.21.15.49:0/30405424 wait complete. 2024-05-15T15:49:44.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.453+0000 7fce7356c700 1 Processor -- start 2024-05-15T15:49:44.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.453+0000 7fce7356c700 1 -- start start 2024-05-15T15:49:44.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce7356c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c0ff110 0x7fce6c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:44.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce7356c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c105cf0 0x7fce6c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:44.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce7356c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c19d540 con 0x7fce6c105cf0 2024-05-15T15:49:44.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce7356c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c196b20 con 0x7fce6c0ff110 2024-05-15T15:49:44.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce70b07700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c105cf0 0x7fce6c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:44.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce70b07700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c105cf0 0x7fce6c19cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60902/0 (socket says 172.21.15.49:60902) 2024-05-15T15:49:44.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce70b07700 1 -- 172.21.15.49:0/1426712259 learned_addr learned my addr 172.21.15.49:0/1426712259 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:44.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce71308700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c0ff110 0x7fce6c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:44.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce70b07700 1 -- 172.21.15.49:0/1426712259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c0ff110 msgr2=0x7fce6c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:44.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce70b07700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c0ff110 0x7fce6c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:44.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce70b07700 1 -- 172.21.15.49:0/1426712259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600095f0 con 0x7fce6c105cf0 2024-05-15T15:49:44.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.454+0000 7fce70b07700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c105cf0 0x7fce6c19cf20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fce60006010 tx=0x7fce6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:44.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.455+0000 7fce5e7fc700 1 -- 172.21.15.49:0/1426712259 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6001d070 con 0x7fce6c105cf0 2024-05-15T15:49:44.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.455+0000 7fce5e7fc700 1 -- 172.21.15.49:0/1426712259 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce6000f460 con 0x7fce6c105cf0 2024-05-15T15:49:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.455+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c196da0 con 0x7fce6c105cf0 2024-05-15T15:49:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.455+0000 7fce5e7fc700 1 -- 172.21.15.49:0/1426712259 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce60017630 con 0x7fce6c105cf0 2024-05-15T15:49:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.455+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c197290 con 0x7fce6c105cf0 2024-05-15T15:49:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.456+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c1092c0 con 0x7fce6c105cf0 2024-05-15T15:49:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.456+0000 7fce5e7fc700 1 -- 172.21.15.49:0/1426712259 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fce6000fab0 con 0x7fce6c105cf0 2024-05-15T15:49:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.457+0000 7fce5e7fc700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce5806cce0 0x7fce5806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.457+0000 7fce5e7fc700 1 -- 172.21.15.49:0/1426712259 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fce6008d3f0 con 0x7fce6c105cf0 2024-05-15T15:49:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.457+0000 7fce71308700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce5806cce0 0x7fce5806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.457+0000 7fce71308700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce5806cce0 0x7fce5806f1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce68006fd0 tx=0x7fce68008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:44.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.462+0000 7fce5e7fc700 1 -- 172.21.15.49:0/1426712259 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fce6005b590 con 0x7fce6c105cf0 2024-05-15T15:49:44.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.665+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce6c102430 con 0x7fce6c105cf0 2024-05-15T15:49:44.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.666+0000 7fce5e7fc700 1 -- 172.21.15.49:0/1426712259 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11321 (secure 0 0 0) 0x7fce6008d030 con 0x7fce6c105cf0 2024-05-15T15:49:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:49:44.668 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":41,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","created":"2024-05-15T15:43:35.648208+0000","modified":"2024-05-15T15:49:25.493977+0000","last_up_change":"2024-05-15T15:49:24.492659+0000","last_in_change":"2024-05-15T15:49:04.722203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T15:48:09.899234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25d1eb06-bee2-48a6-8d86-30b5b3b348d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6803","nonce":941734370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6805","nonce":941734370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6809","nonce":941734370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6807","nonce":941734370}]},"public_addr":"172.21.15.49:6803/941734370","cluster_addr":"172.21.15.49:6805/941734370","heartbeat_back_addr":"172.21.15.49:6809/941734370","heartbeat_front_addr":"172.21.15.49:6807/941734370","state":["exists","up"]},{"osd":1,"uuid":"c0e717cf-ae5c-4fa4-a30d-d2235ff27315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6811","nonce":717954912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6813","nonce":717954912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6817","nonce":717954912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6815","nonce":717954912}]},"public_addr":"172.21.15.49:6811/717954912","cluster_addr":"172.21.15.49:6813/717954912","heartbeat_back_addr":"172.21.15.49:6817/717954912","heartbeat_front_addr":"172.21.15.49:6815/717954912","state":["exists","up"]},{"osd":2,"uuid":"d7752daf-dd5b-49cc-abc2-ca953968e542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6819","nonce":2295619409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6821","nonce":2295619409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6825","nonce":2295619409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6823","nonce":2295619409}]},"public_addr":"172.21.15.49:6819/2295619409","cluster_addr":"172.21.15.49:6821/2295619409","heartbeat_back_addr":"172.21.15.49:6825/2295619409","heartbeat_front_addr":"172.21.15.49:6823/2295619409","state":["exists","up"]},{"osd":3,"uuid":"2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6801","nonce":1650511241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6803","nonce":1650511241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6807","nonce":1650511241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6805","nonce":1650511241}]},"public_addr":"172.21.15.144:6801/1650511241","cluster_addr":"172.21.15.144:6803/1650511241","heartbeat_back_addr":"172.21.15.144:6807/1650511241","heartbeat_front_addr":"172.21.15.144:6805/1650511241","state":["exists","up"]},{"osd":4,"uuid":"6659631f-da39-4a2c-9f75-e7b4a06b602f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6809","nonce":4251193981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6811","nonce":4251193981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6815","nonce":4251193981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6813","nonce":4251193981}]},"public_addr":"172.21.15.144:6809/4251193981","cluster_addr":"172.21.15.144:6811/4251193981","heartbeat_back_addr":"172.21.15.144:6815/4251193981","heartbeat_front_addr":"172.21.15.144:6813/4251193981","state":["exists","up"]},{"osd":5,"uuid":"51ea2d85-6cf7-4a96-8c74-612958083a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6817","nonce":1897088829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6819","nonce":1897088829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6823","nonce":1897088829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6821","nonce":1897088829}]},"public_addr":"172.21.15.144:6817/1897088829","cluster_addr":"172.21.15.144:6819/1897088829","heartbeat_back_addr":"172.21.15.144:6823/1897088829","heartbeat_front_addr":"172.21.15.144:6821/1897088829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:10.108689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:37.458152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:05.915664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:30.194669+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:55.748301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:49:22.635948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/644218315":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6800/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/1173274751":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/3014161738":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/75769186":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6801/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/3452717560":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/4010845594":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/2163344179":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/4024404689":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/940476764":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6800/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6801/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6801/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6800/821763420":"2024-05-16T15:44:39.328329+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T15:49:44.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce5806cce0 msgr2=0x7fce5806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:44.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce5806cce0 0x7fce5806f1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce68006fd0 tx=0x7fce68008040 comp rx=0 tx=0).stop 2024-05-15T15:49:44.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c105cf0 msgr2=0x7fce6c19cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:44.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c105cf0 0x7fce6c19cf20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fce60006010 tx=0x7fce6000be20 comp rx=0 tx=0).stop 2024-05-15T15:49:44.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 shutdown_connections 2024-05-15T15:49:44.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fce5806cce0 0x7fce5806f1a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:44.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce6c0ff110 0x7fce6c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:44.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 --2- 172.21.15.49:0/1426712259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce6c105cf0 0x7fce6c19cf20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:44.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 >> 172.21.15.49:0/1426712259 conn(0x7fce6c076000 msgr2=0x7fce6c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:44.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 shutdown_connections 2024-05-15T15:49:44.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:44.668+0000 7fce7356c700 1 -- 172.21.15.49:0/1426712259 wait complete. 2024-05-15T15:49:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:44 smithi144 ceph-mon[36389]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:44 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1426712259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T15:49:45.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:44 smithi049 ceph-mon[28399]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:45.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:44 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1426712259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T15:49:45.336 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-15T15:49:45.336 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-15T15:49:45.336 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-15T15:49:45.337 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-15T15:49:45.337 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-15T15:49:45.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-15T15:49:45.620 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:45.649 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:46.558 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:47.053 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:46 smithi049 ceph-mon[28399]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:47.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:46 smithi144 ceph-mon[36389]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:47.720 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:48.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:48 smithi049 ceph-mon[28399]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:49.001 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:48 smithi144 ceph-mon[36389]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:50.910 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:50 smithi049 ceph-mon[28399]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:50.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 -- 172.21.15.49:0/2666026809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68101890 msgr2=0x7f3f681058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:50.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 --2- 172.21.15.49:0/2666026809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68101890 0x7f3f681058e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009910 tx=0x7f3f58009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:50.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 -- 172.21.15.49:0/2666026809 shutdown_connections 2024-05-15T15:49:50.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 --2- 172.21.15.49:0/2666026809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68101890 0x7f3f681058e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:50.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 --2- 172.21.15.49:0/2666026809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f68100ee0 0x7f3f681012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:50.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 -- 172.21.15.49:0/2666026809 >> 172.21.15.49:0/2666026809 conn(0x7f3f680fc790 msgr2=0x7f3f680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:50.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 -- 172.21.15.49:0/2666026809 shutdown_connections 2024-05-15T15:49:50.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 -- 172.21.15.49:0/2666026809 wait complete. 2024-05-15T15:49:50.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 Processor -- start 2024-05-15T15:49:50.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.931+0000 7f3f7027f700 1 -- start start 2024-05-15T15:49:50.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f7027f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68100ee0 0x7f3f6819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:50.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f7027f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f68101890 0x7f3f6819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:50.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f7027f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6819d440 con 0x7f3f68100ee0 2024-05-15T15:49:50.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f7027f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68196a20 con 0x7f3f68101890 2024-05-15T15:49:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f6e01b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68100ee0 0x7f3f6819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f6e01b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68100ee0 0x7f3f6819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60926/0 (socket says 172.21.15.49:60926) 2024-05-15T15:49:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f6e01b700 1 -- 172.21.15.49:0/3138352741 learned_addr learned my addr 172.21.15.49:0/3138352741 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f6d81a700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f68101890 0x7f3f6819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f6e01b700 1 -- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f68101890 msgr2=0x7f3f6819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f6e01b700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f68101890 0x7f3f6819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f6e01b700 1 -- 172.21.15.49:0/3138352741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580095f0 con 0x7f3f68100ee0 2024-05-15T15:49:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f6e01b700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68100ee0 0x7f3f6819c8e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400d900 tx=0x7f3f6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3138352741 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f640041f0 con 0x7f3f68100ee0 2024-05-15T15:49:50.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3138352741 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f64004cf0 con 0x7f3f68100ee0 2024-05-15T15:49:50.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.932+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f68196d00 con 0x7f3f68100ee0 2024-05-15T15:49:50.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.933+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3138352741 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f6400b750 con 0x7f3f68100ee0 2024-05-15T15:49:50.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.933+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f68197250 con 0x7f3f68100ee0 2024-05-15T15:49:50.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.933+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f6810e180 con 0x7f3f68100ee0 2024-05-15T15:49:50.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.934+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3138352741 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3f64004750 con 0x7f3f68100ee0 2024-05-15T15:49:50.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.934+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3f5406cce0 0x7f3f5406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.935+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3138352741 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3f6408c3b0 con 0x7f3f68100ee0 2024-05-15T15:49:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.935+0000 7f3f6d81a700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3f5406cce0 0x7f3f5406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.935+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] conn(0x7f3f540729c0 0x7f3f54074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.935+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3138352741 --> [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f54075490 con 0x7f3f540729c0 2024-05-15T15:49:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.935+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3138352741 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f6408c810 con 0x7f3f68100ee0 2024-05-15T15:49:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.935+0000 7f3f6d81a700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3f5406cce0 0x7f3f5406f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f68197be0 tx=0x7f3f5801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.935+0000 7f3f6e81c700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] conn(0x7f3f540729c0 0x7f3f54074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.936+0000 7f3f6e81c700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] conn(0x7f3f540729c0 0x7f3f54074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.936+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3138352741 <== osd.5 v2:172.21.15.144:6816/1897088829 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3f54075490 con 0x7f3f540729c0 2024-05-15T15:49:50.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.952+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 --> [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3f68197700 con 0x7f3f540729c0 2024-05-15T15:49:50.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.952+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3138352741 <== osd.5 v2:172.21.15.144:6816/1897088829 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3f68197700 con 0x7f3f540729c0 2024-05-15T15:49:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] conn(0x7f3f540729c0 msgr2=0x7f3f54074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] conn(0x7f3f540729c0 0x7f3f54074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3f5406cce0 msgr2=0x7f3f5406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3f5406cce0 0x7f3f5406f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f68197be0 tx=0x7f3f5801a040 comp rx=0 tx=0).stop 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68100ee0 msgr2=0x7f3f6819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68100ee0 0x7f3f6819c8e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400d900 tx=0x7f3f6400dc10 comp rx=0 tx=0).stop 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 shutdown_connections 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:6816/1897088829,v1:172.21.15.144:6817/1897088829] conn(0x7f3f540729c0 0x7f3f54074de0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f68100ee0 0x7f3f6819c8e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3f5406cce0 0x7f3f5406f1a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 --2- 172.21.15.49:0/3138352741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f68101890 0x7f3f6819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 >> 172.21.15.49:0/3138352741 conn(0x7f3f680fc790 msgr2=0x7f3f680fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 shutdown_connections 2024-05-15T15:49:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:50.953+0000 7f3f7027f700 1 -- 172.21.15.49:0/3138352741 wait complete. 2024-05-15T15:49:51.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.154+0000 7f997e0c5700 1 -- 172.21.15.49:0/3099185112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99780ffe10 msgr2=0x7f99781001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.154+0000 7f997e0c5700 1 --2- 172.21.15.49:0/3099185112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99780ffe10 0x7f99781001f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9968009910 tx=0x7f9968009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:51.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.154+0000 7f997e0c5700 1 -- 172.21.15.49:0/3099185112 shutdown_connections 2024-05-15T15:49:51.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.154+0000 7f997e0c5700 1 --2- 172.21.15.49:0/3099185112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99781007c0 0x7f9978104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.154+0000 7f997e0c5700 1 --2- 172.21.15.49:0/3099185112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99780ffe10 0x7f99781001f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.154+0000 7f997e0c5700 1 -- 172.21.15.49:0/3099185112 >> 172.21.15.49:0/3099185112 conn(0x7f99780fb700 msgr2=0x7f99780fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.154+0000 7f997e0c5700 1 -- 172.21.15.49:0/3099185112 shutdown_connections 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.154+0000 7f997e0c5700 1 -- 172.21.15.49:0/3099185112 wait complete. 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.155+0000 7f997e0c5700 1 Processor -- start 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.155+0000 7f997e0c5700 1 -- start start 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.155+0000 7f997e0c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99780ffe10 0x7f9978197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.155+0000 7f997e0c5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99781007c0 0x7f99781980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.155+0000 7f997e0c5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9978198610 con 0x7f99781007c0 2024-05-15T15:49:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.155+0000 7f997e0c5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9978198750 con 0x7f99780ffe10 2024-05-15T15:49:51.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99781007c0 0x7f99781980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:51.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99781007c0 0x7f99781980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60940/0 (socket says 172.21.15.49:60940) 2024-05-15T15:49:51.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9976ffd700 1 -- 172.21.15.49:0/1729954603 learned_addr learned my addr 172.21.15.49:0/1729954603 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:51.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f99777fe700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99780ffe10 0x7f9978197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:51.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9976ffd700 1 -- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99780ffe10 msgr2=0x7f9978197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9976ffd700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99780ffe10 0x7f9978197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9976ffd700 1 -- 172.21.15.49:0/1729954603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680095f0 con 0x7f99781007c0 2024-05-15T15:49:51.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9976ffd700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99781007c0 0x7f99781980d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f996c00b480 tx=0x7f996c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:51.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9974ff9700 1 -- 172.21.15.49:0/1729954603 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f996c0118c0 con 0x7f99781007c0 2024-05-15T15:49:51.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99781a0c00 con 0x7f99781007c0 2024-05-15T15:49:51.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9974ff9700 1 -- 172.21.15.49:0/1729954603 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f996c010380 con 0x7f99781007c0 2024-05-15T15:49:51.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f9974ff9700 1 -- 172.21.15.49:0/1729954603 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f996c00f4f0 con 0x7f99781007c0 2024-05-15T15:49:51.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.156+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99781a1010 con 0x7f99781007c0 2024-05-15T15:49:51.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.157+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f997810d110 con 0x7f99781007c0 2024-05-15T15:49:51.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.158+0000 7f9974ff9700 1 -- 172.21.15.49:0/1729954603 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f996c00f7b0 con 0x7f99781007c0 2024-05-15T15:49:51.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.158+0000 7f9974ff9700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f996006cce0 0x7f996006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:51.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.158+0000 7f9974ff9700 1 -- 172.21.15.49:0/1729954603 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f996c08bbc0 con 0x7f99781007c0 2024-05-15T15:49:51.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.158+0000 7f99777fe700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f996006cce0 0x7f996006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:51.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.158+0000 7f9974ff9700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] conn(0x7f99600729c0 0x7f9960074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:51.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.158+0000 7f9974ff9700 1 -- 172.21.15.49:0/1729954603 --> [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9960075490 con 0x7f99600729c0 2024-05-15T15:49:51.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.158+0000 7f9974ff9700 1 -- 172.21.15.49:0/1729954603 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f996c0912a0 con 0x7f99781007c0 2024-05-15T15:49:51.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.159+0000 7f9977fff700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] conn(0x7f99600729c0 0x7f9960074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:51.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.159+0000 7f99777fe700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f996006cce0 0x7f996006f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9968005f50 tx=0x7f996801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:51.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.159+0000 7f9977fff700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] conn(0x7f99600729c0 0x7f9960074de0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:51.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.160+0000 7f9974ff9700 1 -- 172.21.15.49:0/1729954603 <== osd.4 v2:172.21.15.144:6808/4251193981 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9960075490 con 0x7f99600729c0 2024-05-15T15:49:51.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.176+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 --> [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9978061620 con 0x7f99600729c0 2024-05-15T15:49:51.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f9974ff9700 1 -- 172.21.15.49:0/1729954603 <== osd.4 v2:172.21.15.144:6808/4251193981 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9978061620 con 0x7f99600729c0 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] conn(0x7f99600729c0 msgr2=0x7f9960074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] conn(0x7f99600729c0 0x7f9960074de0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f996006cce0 msgr2=0x7f996006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f996006cce0 0x7f996006f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9968005f50 tx=0x7f996801a040 comp rx=0 tx=0).stop 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99781007c0 msgr2=0x7f99781980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99781007c0 0x7f99781980d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f996c00b480 tx=0x7f996c00b840 comp rx=0 tx=0).stop 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 shutdown_connections 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:6808/4251193981,v1:172.21.15.144:6809/4251193981] conn(0x7f99600729c0 0x7f9960074de0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f996006cce0 0x7f996006f1a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99780ffe10 0x7f9978197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 --2- 172.21.15.49:0/1729954603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99781007c0 0x7f99781980d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.177+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 >> 172.21.15.49:0/1729954603 conn(0x7f99780fb700 msgr2=0x7f99780fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:51.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.178+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 shutdown_connections 2024-05-15T15:49:51.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.178+0000 7f997e0c5700 1 -- 172.21.15.49:0/1729954603 wait complete. 2024-05-15T15:49:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:50 smithi144 ceph-mon[36389]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.969+0000 7f75b29d5700 1 -- 172.21.15.49:0/2608437951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac101ef0 msgr2=0x7f75ac1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.969+0000 7f75b29d5700 1 --2- 172.21.15.49:0/2608437951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac101ef0 0x7f75ac1022d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f759c009910 tx=0x7f759c009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.969+0000 7f75b29d5700 1 -- 172.21.15.49:0/2608437951 shutdown_connections 2024-05-15T15:49:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.969+0000 7f75b29d5700 1 --2- 172.21.15.49:0/2608437951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ac102810 0x7f75ac10ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.969+0000 7f75b29d5700 1 --2- 172.21.15.49:0/2608437951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac101ef0 0x7f75ac1022d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.969+0000 7f75b29d5700 1 -- 172.21.15.49:0/2608437951 >> 172.21.15.49:0/2608437951 conn(0x7f75ac0fb6a0 msgr2=0x7f75ac0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.969+0000 7f75b29d5700 1 -- 172.21.15.49:0/2608437951 shutdown_connections 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.969+0000 7f75b29d5700 1 -- 172.21.15.49:0/2608437951 wait complete. 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.970+0000 7f75b29d5700 1 Processor -- start 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.970+0000 7f75b29d5700 1 -- start start 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.970+0000 7f75b29d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ac101ef0 0x7f75ac198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.970+0000 7f75b29d5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac102810 0x7f75ac199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.970+0000 7f75b29d5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac199a00 con 0x7f75ac102810 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.970+0000 7f75b29d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac19d6d0 con 0x7f75ac101ef0 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75ab7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac102810 0x7f75ac199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75ab7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac102810 0x7f75ac199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60956/0 (socket says 172.21.15.49:60956) 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75ab7fe700 1 -- 172.21.15.49:0/1841414315 learned_addr learned my addr 172.21.15.49:0/1841414315 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75abfff700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ac101ef0 0x7f75ac198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75ab7fe700 1 -- 172.21.15.49:0/1841414315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ac101ef0 msgr2=0x7f75ac198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75ab7fe700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ac101ef0 0x7f75ac198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75ab7fe700 1 -- 172.21.15.49:0/1841414315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c0095f0 con 0x7f75ac102810 2024-05-15T15:49:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75abfff700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ac101ef0 0x7f75ac198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T15:49:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75ab7fe700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac102810 0x7f75ac199320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f759400c8f0 tx=0x7f759400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75a97fa700 1 -- 172.21.15.49:0/1841414315 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7594007a50 con 0x7f75ac102810 2024-05-15T15:49:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75a97fa700 1 -- 172.21.15.49:0/1841414315 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7594007bb0 con 0x7f75ac102810 2024-05-15T15:49:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ac19d9b0 con 0x7f75ac102810 2024-05-15T15:49:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75a97fa700 1 -- 172.21.15.49:0/1841414315 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7594018630 con 0x7f75ac102810 2024-05-15T15:49:51.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.971+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ac19df00 con 0x7f75ac102810 2024-05-15T15:49:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.972+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75ac10e1c0 con 0x7f75ac102810 2024-05-15T15:49:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.973+0000 7f75a97fa700 1 -- 172.21.15.49:0/1841414315 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7594007d20 con 0x7f75ac102810 2024-05-15T15:49:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.973+0000 7f75a97fa700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f759806cce0 0x7f759806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.973+0000 7f75abfff700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f759806cce0 0x7f759806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:51.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.973+0000 7f75a97fa700 1 -- 172.21.15.49:0/1841414315 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f759408c450 con 0x7f75ac102810 2024-05-15T15:49:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.974+0000 7f75a97fa700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] conn(0x7f75980729c0 0x7f7598074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.974+0000 7f75a97fa700 1 -- 172.21.15.49:0/1841414315 --> [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7598075490 con 0x7f75980729c0 2024-05-15T15:49:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.974+0000 7f75a97fa700 1 -- 172.21.15.49:0/1841414315 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f759408c810 con 0x7f75ac102810 2024-05-15T15:49:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.974+0000 7f75b0f72700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] conn(0x7f75980729c0 0x7f7598074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.974+0000 7f75abfff700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f759806cce0 0x7f759806f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f759c0059d0 tx=0x7f759c01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.974+0000 7f75b0f72700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] conn(0x7f75980729c0 0x7f7598074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:51.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.975+0000 7f75a97fa700 1 -- 172.21.15.49:0/1841414315 <== osd.2 v2:172.21.15.49:6818/2295619409 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7598075490 con 0x7f75980729c0 2024-05-15T15:49:51.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.991+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 --> [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f75ac19e400 con 0x7f75980729c0 2024-05-15T15:49:51.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.991+0000 7f75a97fa700 1 -- 172.21.15.49:0/1841414315 <== osd.2 v2:172.21.15.49:6818/2295619409 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f75ac19e400 con 0x7f75980729c0 2024-05-15T15:49:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] conn(0x7f75980729c0 msgr2=0x7f7598074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] conn(0x7f75980729c0 0x7f7598074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f759806cce0 msgr2=0x7f759806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f759806cce0 0x7f759806f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f759c0059d0 tx=0x7f759c01a040 comp rx=0 tx=0).stop 2024-05-15T15:49:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac102810 msgr2=0x7f75ac199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac102810 0x7f75ac199320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f759400c8f0 tx=0x7f759400ccb0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 shutdown_connections 2024-05-15T15:49:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6818/2295619409,v1:172.21.15.49:6819/2295619409] conn(0x7f75980729c0 0x7f7598074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f759806cce0 0x7f759806f1a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75ac101ef0 0x7f75ac198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 --2- 172.21.15.49:0/1841414315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75ac102810 0x7f75ac199320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 >> 172.21.15.49:0/1841414315 conn(0x7f75ac0fb6a0 msgr2=0x7f75ac0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 shutdown_connections 2024-05-15T15:49:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:51.992+0000 7f75b29d5700 1 -- 172.21.15.49:0/1841414315 wait complete. 2024-05-15T15:49:52.005 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:49:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:49:52.906 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:52 smithi049 ceph-mon[28399]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:53.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:52 smithi144 ceph-mon[36389]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:53.316 INFO:teuthology.orchestra.run.smithi049.stdout:150323855373 2024-05-15T15:49:53.316 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-15T15:49:53.773 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.376+0000 7f1552cbf700 1 -- 172.21.15.49:0/1682422216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c0ffdc0 msgr2=0x7f154c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.376+0000 7f1552cbf700 1 --2- 172.21.15.49:0/1682422216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c0ffdc0 0x7f154c1001a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1540009910 tx=0x7f1540009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.377+0000 7f1552cbf700 1 -- 172.21.15.49:0/1682422216 shutdown_connections 2024-05-15T15:49:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.377+0000 7f1552cbf700 1 --2- 172.21.15.49:0/1682422216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154c100770 0x7f154c1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.377+0000 7f1552cbf700 1 --2- 172.21.15.49:0/1682422216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c0ffdc0 0x7f154c1001a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.377+0000 7f1552cbf700 1 -- 172.21.15.49:0/1682422216 >> 172.21.15.49:0/1682422216 conn(0x7f154c076800 msgr2=0x7f154c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.377+0000 7f1552cbf700 1 -- 172.21.15.49:0/1682422216 shutdown_connections 2024-05-15T15:49:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.377+0000 7f1552cbf700 1 -- 172.21.15.49:0/1682422216 wait complete. 2024-05-15T15:49:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.377+0000 7f1552cbf700 1 Processor -- start 2024-05-15T15:49:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f1552cbf700 1 -- start start 2024-05-15T15:49:54.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f1552cbf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154c0ffdc0 0x7f154c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f1552cbf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c100770 0x7f154c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f1552cbf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c198510 con 0x7f154c100770 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f1552cbf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c198650 con 0x7f154c0ffdc0 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f154bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c100770 0x7f154c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f154bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c100770 0x7f154c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57022/0 (socket says 172.21.15.49:57022) 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f154bfff700 1 -- 172.21.15.49:0/118390464 learned_addr learned my addr 172.21.15.49:0/118390464 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f1550a5b700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154c0ffdc0 0x7f154c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f154bfff700 1 -- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154c0ffdc0 msgr2=0x7f154c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f154bfff700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154c0ffdc0 0x7f154c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.378+0000 7f154bfff700 1 -- 172.21.15.49:0/118390464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15400095f0 con 0x7f154c100770 2024-05-15T15:49:54.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.379+0000 7f154bfff700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c100770 0x7f154c197f40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f153800d900 tx=0x7f153800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.379+0000 7f1549ffb700 1 -- 172.21.15.49:0/118390464 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1538009900 con 0x7f154c100770 2024-05-15T15:49:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.379+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c1a0a70 con 0x7f154c100770 2024-05-15T15:49:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.379+0000 7f1549ffb700 1 -- 172.21.15.49:0/118390464 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1538010460 con 0x7f154c100770 2024-05-15T15:49:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.379+0000 7f1549ffb700 1 -- 172.21.15.49:0/118390464 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f153800f5f0 con 0x7f154c100770 2024-05-15T15:49:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.379+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c1a0e80 con 0x7f154c100770 2024-05-15T15:49:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.380+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f154c10d080 con 0x7f154c100770 2024-05-15T15:49:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.381+0000 7f1549ffb700 1 -- 172.21.15.49:0/118390464 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f15380105d0 con 0x7f154c100770 2024-05-15T15:49:54.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.381+0000 7f1549ffb700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f153c06cce0 0x7f153c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.381+0000 7f1549ffb700 1 -- 172.21.15.49:0/118390464 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f153808baa0 con 0x7f154c100770 2024-05-15T15:49:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.381+0000 7f1550a5b700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f153c06cce0 0x7f153c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.381+0000 7f1549ffb700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] conn(0x7f153c0729c0 0x7f153c074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.382+0000 7f1549ffb700 1 -- 172.21.15.49:0/118390464 --> [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f153c075490 con 0x7f153c0729c0 2024-05-15T15:49:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.382+0000 7f1549ffb700 1 -- 172.21.15.49:0/118390464 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15380902a0 con 0x7f154c100770 2024-05-15T15:49:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.382+0000 7f155125c700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] conn(0x7f153c0729c0 0x7f153c074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.382+0000 7f1550a5b700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f153c06cce0 0x7f153c06f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1540000c00 tx=0x7f154001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:54.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.383+0000 7f155125c700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] conn(0x7f153c0729c0 0x7f153c074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:54.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.383+0000 7f1549ffb700 1 -- 172.21.15.49:0/118390464 <== osd.3 v2:172.21.15.144:6800/1650511241 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f153c075490 con 0x7f153c0729c0 2024-05-15T15:49:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.399+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 --> [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f154c061620 con 0x7f153c0729c0 2024-05-15T15:49:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.400+0000 7f1549ffb700 1 -- 172.21.15.49:0/118390464 <== osd.3 v2:172.21.15.144:6800/1650511241 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f154c061620 con 0x7f153c0729c0 2024-05-15T15:49:54.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.400+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] conn(0x7f153c0729c0 msgr2=0x7f153c074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:54.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.400+0000 7f1552cbf700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] conn(0x7f153c0729c0 0x7f153c074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.400+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f153c06cce0 msgr2=0x7f153c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.400+0000 7f1552cbf700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f153c06cce0 0x7f153c06f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1540000c00 tx=0x7f154001a040 comp rx=0 tx=0).stop 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.400+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c100770 msgr2=0x7f154c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.400+0000 7f1552cbf700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c100770 0x7f154c197f40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f153800d900 tx=0x7f153800dc10 comp rx=0 tx=0).stop 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.401+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 shutdown_connections 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.401+0000 7f1552cbf700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:6800/1650511241,v1:172.21.15.144:6801/1650511241] conn(0x7f153c0729c0 0x7f153c074de0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.401+0000 7f1552cbf700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f153c06cce0 0x7f153c06f1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.401+0000 7f1552cbf700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154c0ffdc0 0x7f154c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.401+0000 7f1552cbf700 1 --2- 172.21.15.49:0/118390464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154c100770 0x7f154c197f40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.401+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 >> 172.21.15.49:0/118390464 conn(0x7f154c076800 msgr2=0x7f154c103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.401+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 shutdown_connections 2024-05-15T15:49:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:54.401+0000 7f1552cbf700 1 -- 172.21.15.49:0/118390464 wait complete. 2024-05-15T15:49:55.045 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:54 smithi049 ceph-mon[28399]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:54 smithi144 ceph-mon[36389]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.066+0000 7ff20b246700 1 -- 172.21.15.49:0/3025431056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 msgr2=0x7ff2041047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.066+0000 7ff20b246700 1 --2- 172.21.15.49:0/3025431056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 0x7ff2041047d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009910 tx=0x7ff1f0009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.066+0000 7ff20b246700 1 -- 172.21.15.49:0/3025431056 shutdown_connections 2024-05-15T15:49:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.066+0000 7ff20b246700 1 --2- 172.21.15.49:0/3025431056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 0x7ff2041047d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.066+0000 7ff20b246700 1 --2- 172.21.15.49:0/3025431056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040ffdd0 0x7ff2041001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.067+0000 7ff20b246700 1 -- 172.21.15.49:0/3025431056 >> 172.21.15.49:0/3025431056 conn(0x7ff2040fb6a0 msgr2=0x7ff2040fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.067+0000 7ff20b246700 1 -- 172.21.15.49:0/3025431056 shutdown_connections 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.067+0000 7ff20b246700 1 -- 172.21.15.49:0/3025431056 wait complete. 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.067+0000 7ff20b246700 1 Processor -- start 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.067+0000 7ff20b246700 1 -- start start 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff20b246700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040ffdd0 0x7ff204198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff20b246700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 0x7ff204199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff20b246700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff204199a00 con 0x7ff204100780 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff20b246700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20419d6d0 con 0x7ff2040ffdd0 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff203fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 0x7ff204199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff203fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 0x7ff204199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57036/0 (socket says 172.21.15.49:57036) 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff203fff700 1 -- 172.21.15.49:0/1199374693 learned_addr learned my addr 172.21.15.49:0/1199374693 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff208fe2700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040ffdd0 0x7ff204198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff203fff700 1 -- 172.21.15.49:0/1199374693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040ffdd0 msgr2=0x7ff204198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff203fff700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040ffdd0 0x7ff204198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff203fff700 1 -- 172.21.15.49:0/1199374693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f00095f0 con 0x7ff204100780 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff203fff700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 0x7ff204199320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000bd70 tx=0x7ff1f000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff201ffb700 1 -- 172.21.15.49:0/1199374693 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1f001d070 con 0x7ff204100780 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20419d950 con 0x7ff204100780 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff201ffb700 1 -- 172.21.15.49:0/1199374693 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1f00056e0 con 0x7ff204100780 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.068+0000 7ff201ffb700 1 -- 172.21.15.49:0/1199374693 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1f0017400 con 0x7ff204100780 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.069+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20419de40 con 0x7ff204100780 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.069+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff204110370 con 0x7ff204100780 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.070+0000 7ff201ffb700 1 -- 172.21.15.49:0/1199374693 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff1f0017670 con 0x7ff204100780 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.070+0000 7ff201ffb700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff1f406cce0 0x7ff1f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.071+0000 7ff208fe2700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff1f406cce0 0x7ff1f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:55.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.071+0000 7ff201ffb700 1 -- 172.21.15.49:0/1199374693 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff1f008d350 con 0x7ff204100780 2024-05-15T15:49:55.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.071+0000 7ff201ffb700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] conn(0x7ff1f40729c0 0x7ff1f4074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:55.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.071+0000 7ff201ffb700 1 -- 172.21.15.49:0/1199374693 --> [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1f4075490 con 0x7ff1f40729c0 2024-05-15T15:49:55.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.071+0000 7ff201ffb700 1 -- 172.21.15.49:0/1199374693 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f008fa20 con 0x7ff204100780 2024-05-15T15:49:55.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.071+0000 7ff2097e3700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] conn(0x7ff1f40729c0 0x7ff1f4074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:55.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.071+0000 7ff208fe2700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff1f406cce0 0x7ff1f406f1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8005950 tx=0x7ff1f80058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:55.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.071+0000 7ff2097e3700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] conn(0x7ff1f40729c0 0x7ff1f4074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:55.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.072+0000 7ff201ffb700 1 -- 172.21.15.49:0/1199374693 <== osd.0 v2:172.21.15.49:6802/941734370 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff1f4075490 con 0x7ff1f40729c0 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.088+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 --> [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff20419e360 con 0x7ff1f40729c0 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.088+0000 7ff201ffb700 1 -- 172.21.15.49:0/1199374693 <== osd.0 v2:172.21.15.49:6802/941734370 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff20419e360 con 0x7ff1f40729c0 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] conn(0x7ff1f40729c0 msgr2=0x7ff1f4074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] conn(0x7ff1f40729c0 0x7ff1f4074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff1f406cce0 msgr2=0x7ff1f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff1f406cce0 0x7ff1f406f1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8005950 tx=0x7ff1f80058e0 comp rx=0 tx=0).stop 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 msgr2=0x7ff204199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 0x7ff204199320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000bd70 tx=0x7ff1f000be50 comp rx=0 tx=0).stop 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 shutdown_connections 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6802/941734370,v1:172.21.15.49:6803/941734370] conn(0x7ff1f40729c0 0x7ff1f4074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff1f406cce0 0x7ff1f406f1a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:55.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040ffdd0 0x7ff204198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 --2- 172.21.15.49:0/1199374693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff204100780 0x7ff204199320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 >> 172.21.15.49:0/1199374693 conn(0x7ff2040fb6a0 msgr2=0x7ff2040fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 shutdown_connections 2024-05-15T15:49:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:55.089+0000 7ff20b246700 1 -- 172.21.15.49:0/1199374693 wait complete. 2024-05-15T15:49:56.405 INFO:teuthology.orchestra.run.smithi049.stdout:115964117012 2024-05-15T15:49:56.406 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-15T15:49:56.825 INFO:teuthology.orchestra.run.smithi049.stdout:171798691849 2024-05-15T15:49:56.825 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-15T15:49:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:56 smithi144 ceph-mon[36389]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:57.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:56 smithi049 ceph-mon[28399]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:57.851 INFO:teuthology.orchestra.run.smithi049.stdout:81604378648 2024-05-15T15:49:57.851 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-15T15:49:58.283 INFO:teuthology.orchestra.run.smithi049.stdout:38654705699 2024-05-15T15:49:58.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-15T15:49:58.776 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:58.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.812+0000 7fe691953700 1 -- 172.21.15.49:0/4263272253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 msgr2=0x7fe68c10ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:58.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.812+0000 7fe691953700 1 --2- 172.21.15.49:0/4263272253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 0x7fe68c10ace0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe674009910 tx=0x7fe674009c20 comp rx=0 tx=0).stop 2024-05-15T15:49:58.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.812+0000 7fe691953700 1 -- 172.21.15.49:0/4263272253 shutdown_connections 2024-05-15T15:49:58.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.812+0000 7fe691953700 1 --2- 172.21.15.49:0/4263272253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 0x7fe68c10ace0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:58.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.812+0000 7fe691953700 1 --2- 172.21.15.49:0/4263272253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe68c101ed0 0x7fe68c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:58.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.812+0000 7fe691953700 1 -- 172.21.15.49:0/4263272253 >> 172.21.15.49:0/4263272253 conn(0x7fe68c0fb760 msgr2=0x7fe68c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:58.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.812+0000 7fe691953700 1 -- 172.21.15.49:0/4263272253 shutdown_connections 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.812+0000 7fe691953700 1 -- 172.21.15.49:0/4263272253 wait complete. 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.813+0000 7fe691953700 1 Processor -- start 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.813+0000 7fe691953700 1 -- start start 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.813+0000 7fe691953700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe68c101ed0 0x7fe68c19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.813+0000 7fe691953700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 0x7fe68c194890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.813+0000 7fe691953700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c194f70 con 0x7fe68c1027f0 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.813+0000 7fe691953700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c1950e0 con 0x7fe68c101ed0 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.813+0000 7fe6827fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 0x7fe68c194890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.813+0000 7fe6827fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 0x7fe68c194890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57062/0 (socket says 172.21.15.49:57062) 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.813+0000 7fe6827fc700 1 -- 172.21.15.49:0/2241914862 learned_addr learned my addr 172.21.15.49:0/2241914862 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:49:58.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.814+0000 7fe68affd700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe68c101ed0 0x7fe68c19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.814+0000 7fe6827fc700 1 -- 172.21.15.49:0/2241914862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe68c101ed0 msgr2=0x7fe68c19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.814+0000 7fe6827fc700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe68c101ed0 0x7fe68c19b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.814+0000 7fe6827fc700 1 -- 172.21.15.49:0/2241914862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6740095f0 con 0x7fe68c1027f0 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.814+0000 7fe6827fc700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 0x7fe68c194890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe674009fd0 tx=0x7fe674005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.814+0000 7fe690951700 1 -- 172.21.15.49:0/2241914862 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe67401d070 con 0x7fe68c1027f0 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.814+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c195360 con 0x7fe68c1027f0 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.814+0000 7fe690951700 1 -- 172.21.15.49:0/2241914862 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6740052a0 con 0x7fe68c1027f0 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.815+0000 7fe690951700 1 -- 172.21.15.49:0/2241914862 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe67400fe30 con 0x7fe68c1027f0 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.815+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c199350 con 0x7fe68c1027f0 2024-05-15T15:49:58.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.816+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe68c069a50 con 0x7fe68c1027f0 2024-05-15T15:49:58.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.817+0000 7fe690951700 1 -- 172.21.15.49:0/2241914862 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe67400f670 con 0x7fe68c1027f0 2024-05-15T15:49:58.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.817+0000 7fe690951700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe67806ccf0 0x7fe67806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:58.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.817+0000 7fe690951700 1 -- 172.21.15.49:0/2241914862 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe67408d100 con 0x7fe68c1027f0 2024-05-15T15:49:58.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.817+0000 7fe68affd700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe67806ccf0 0x7fe67806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:58.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.817+0000 7fe690951700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] conn(0x7fe6780729d0 0x7fe678074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:49:58.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.818+0000 7fe690951700 1 -- 172.21.15.49:0/2241914862 --> [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6780754a0 con 0x7fe6780729d0 2024-05-15T15:49:58.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.818+0000 7fe690951700 1 -- 172.21.15.49:0/2241914862 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe67408d560 con 0x7fe68c1027f0 2024-05-15T15:49:58.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.818+0000 7fe68b7fe700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] conn(0x7fe6780729d0 0x7fe678074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:49:58.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.818+0000 7fe68affd700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe67806ccf0 0x7fe67806f1b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00f540 tx=0x7fe67c00b040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:58.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.818+0000 7fe68b7fe700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] conn(0x7fe6780729d0 0x7fe678074df0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:49:58.821 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:49:58 smithi049 ceph-mon[28399]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:58.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.819+0000 7fe690951700 1 -- 172.21.15.49:0/2241914862 <== osd.1 v2:172.21.15.49:6810/717954912 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe6780754a0 con 0x7fe6780729d0 2024-05-15T15:49:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:49:58.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.841+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 --> [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe68c10e170 con 0x7fe6780729d0 2024-05-15T15:49:58.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.841+0000 7fe690951700 1 -- 172.21.15.49:0/2241914862 <== osd.1 v2:172.21.15.49:6810/717954912 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe68c10e170 con 0x7fe6780729d0 2024-05-15T15:49:58.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] conn(0x7fe6780729d0 msgr2=0x7fe678074df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:58.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] conn(0x7fe6780729d0 0x7fe678074df0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe67806ccf0 msgr2=0x7fe67806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe67806ccf0 0x7fe67806f1b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00f540 tx=0x7fe67c00b040 comp rx=0 tx=0).stop 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 msgr2=0x7fe68c194890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 0x7fe68c194890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe674009fd0 tx=0x7fe674005ee0 comp rx=0 tx=0).stop 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 shutdown_connections 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6810/717954912,v1:172.21.15.49:6811/717954912] conn(0x7fe6780729d0 0x7fe678074df0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe67806ccf0 0x7fe67806f1b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe68c101ed0 0x7fe68c19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 --2- 172.21.15.49:0/2241914862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe68c1027f0 0x7fe68c194890 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 >> 172.21.15.49:0/2241914862 conn(0x7fe68c0fb760 msgr2=0x7fe68c0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 shutdown_connections 2024-05-15T15:49:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:49:58.842+0000 7fe691953700 1 -- 172.21.15.49:0/2241914862 wait complete. 2024-05-15T15:49:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:49:58 smithi144 ceph-mon[36389]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:49:59.993 INFO:teuthology.orchestra.run.smithi049.stdout:60129542175 2024-05-15T15:49:59.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-15T15:50:00.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:00 smithi049 ceph-mon[28399]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:00.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:00 smithi049 ceph-mon[28399]: overall HEALTH_OK 2024-05-15T15:50:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:00 smithi144 ceph-mon[36389]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:00 smithi144 ceph-mon[36389]: overall HEALTH_OK 2024-05-15T15:50:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:01.931 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:01.935 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:02.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.620+0000 7f11bd720700 1 -- 172.21.15.49:0/2601019390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 msgr2=0x7f11b81001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:02.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.620+0000 7f11bd720700 1 --2- 172.21.15.49:0/2601019390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 0x7f11b81001a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009910 tx=0x7f11a8009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:02.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.620+0000 7f11bd720700 1 -- 172.21.15.49:0/2601019390 shutdown_connections 2024-05-15T15:50:02.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.620+0000 7f11bd720700 1 --2- 172.21.15.49:0/2601019390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b8100770 0x7f11b81047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:02.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.620+0000 7f11bd720700 1 --2- 172.21.15.49:0/2601019390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 0x7f11b81001a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:02.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.620+0000 7f11bd720700 1 -- 172.21.15.49:0/2601019390 >> 172.21.15.49:0/2601019390 conn(0x7f11b8076800 msgr2=0x7f11b80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:02.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.620+0000 7f11bd720700 1 -- 172.21.15.49:0/2601019390 shutdown_connections 2024-05-15T15:50:02.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.620+0000 7f11bd720700 1 -- 172.21.15.49:0/2601019390 wait complete. 2024-05-15T15:50:02.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11bd720700 1 Processor -- start 2024-05-15T15:50:02.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11bd720700 1 -- start start 2024-05-15T15:50:02.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11bd720700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 0x7f11b80fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:02.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11bd720700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b8100770 0x7f11b80fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:02.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11bd720700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80fc760 con 0x7f11b80ffdc0 2024-05-15T15:50:02.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11bd720700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80fc8a0 con 0x7f11b8100770 2024-05-15T15:50:02.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11b6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 0x7f11b80fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:02.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11b6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 0x7f11b80fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57082/0 (socket says 172.21.15.49:57082) 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11b6ffd700 1 -- 172.21.15.49:0/3517587406 learned_addr learned my addr 172.21.15.49:0/3517587406 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.621+0000 7f11b67fc700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b8100770 0x7f11b80fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.622+0000 7f11b6ffd700 1 -- 172.21.15.49:0/3517587406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b8100770 msgr2=0x7f11b80fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.622+0000 7f11b6ffd700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b8100770 0x7f11b80fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.622+0000 7f11b6ffd700 1 -- 172.21.15.49:0/3517587406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80095f0 con 0x7f11b80ffdc0 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.622+0000 7f11b6ffd700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 0x7f11b80fbce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11a8006010 tx=0x7f11a800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.622+0000 7f119ffff700 1 -- 172.21.15.49:0/3517587406 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a801d070 con 0x7f11b80ffdc0 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.622+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b80fcb20 con 0x7f11b80ffdc0 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.622+0000 7f119ffff700 1 -- 172.21.15.49:0/3517587406 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11a80056e0 con 0x7f11b80ffdc0 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.622+0000 7f119ffff700 1 -- 172.21.15.49:0/3517587406 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a800fdf0 con 0x7f11b80ffdc0 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.622+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b81a3000 con 0x7f11b80ffdc0 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.623+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b81012a0 con 0x7f11b80ffdc0 2024-05-15T15:50:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.624+0000 7f119ffff700 1 -- 172.21.15.49:0/3517587406 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f11a800f5f0 con 0x7f11b80ffdc0 2024-05-15T15:50:02.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.624+0000 7f119ffff700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f11a006cce0 0x7f11a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:02.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.624+0000 7f119ffff700 1 -- 172.21.15.49:0/3517587406 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f11a808d1f0 con 0x7f11b80ffdc0 2024-05-15T15:50:02.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.624+0000 7f11b67fc700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f11a006cce0 0x7f11a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:02.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.625+0000 7f11b67fc700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f11a006cce0 0x7f11a006f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f11b80fd410 tx=0x7f11ac00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:02.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.629+0000 7f119ffff700 1 -- 172.21.15.49:0/3517587406 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f11a8057900 con 0x7f11b80ffdc0 2024-05-15T15:50:02.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.840+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f11b810ce40 con 0x7f11b80ffdc0 2024-05-15T15:50:02.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.841+0000 7f119ffff700 1 -- 172.21.15.49:0/3517587406 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11a805af20 con 0x7f11b80ffdc0 2024-05-15T15:50:02.844 INFO:teuthology.orchestra.run.smithi049.stdout:150323855375 2024-05-15T15:50:02.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f11a006cce0 msgr2=0x7f11a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:02.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f11a006cce0 0x7f11a006f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f11b80fd410 tx=0x7f11ac00b410 comp rx=0 tx=0).stop 2024-05-15T15:50:02.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 msgr2=0x7f11b80fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:02.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 0x7f11b80fbce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11a8006010 tx=0x7f11a800bd70 comp rx=0 tx=0).stop 2024-05-15T15:50:02.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 shutdown_connections 2024-05-15T15:50:02.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11b80ffdc0 0x7f11b80fbce0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:02.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f11a006cce0 0x7f11a006f1a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:02.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 --2- 172.21.15.49:0/3517587406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b8100770 0x7f11b80fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:02.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 >> 172.21.15.49:0/3517587406 conn(0x7f11b8076800 msgr2=0x7f11b80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:02.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 shutdown_connections 2024-05-15T15:50:02.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:02.843+0000 7f11bd720700 1 -- 172.21.15.49:0/3517587406 wait complete. 2024-05-15T15:50:02.942 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:02 smithi049 ceph-mon[28399]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:03.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:02 smithi144 ceph-mon[36389]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:03 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3517587406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-15T15:50:04.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:03 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3517587406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-15T15:50:04.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:04 smithi049 ceph-mon[28399]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:04 smithi144 ceph-mon[36389]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.956+0000 7f60c46c0700 1 -- 172.21.15.49:0/2852931609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 msgr2=0x7f60bc1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.956+0000 7f60c46c0700 1 --2- 172.21.15.49:0/2852931609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 0x7f60bc1001d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009910 tx=0x7f60b8009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.957+0000 7f60c46c0700 1 -- 172.21.15.49:0/2852931609 shutdown_connections 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.957+0000 7f60c46c0700 1 --2- 172.21.15.49:0/2852931609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60bc1007a0 0x7f60bc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.957+0000 7f60c46c0700 1 --2- 172.21.15.49:0/2852931609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 0x7f60bc1001d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.957+0000 7f60c46c0700 1 -- 172.21.15.49:0/2852931609 >> 172.21.15.49:0/2852931609 conn(0x7f60bc0fb6a0 msgr2=0x7f60bc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.957+0000 7f60c46c0700 1 -- 172.21.15.49:0/2852931609 shutdown_connections 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.957+0000 7f60c46c0700 1 -- 172.21.15.49:0/2852931609 wait complete. 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.957+0000 7f60c46c0700 1 Processor -- start 2024-05-15T15:50:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.957+0000 7f60c46c0700 1 -- start start 2024-05-15T15:50:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c46c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 0x7f60bc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c46c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60bc1007a0 0x7f60bc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c46c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc198640 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c46c0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc198780 con 0x7f60bc1007a0 2024-05-15T15:50:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c245c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 0x7f60bc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c245c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 0x7f60bc197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46906/0 (socket says 172.21.15.49:46906) 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c245c700 1 -- 172.21.15.49:0/2412513777 learned_addr learned my addr 172.21.15.49:0/2412513777 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c245c700 1 -- 172.21.15.49:0/2412513777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60bc1007a0 msgr2=0x7f60bc198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c1c5b700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60bc1007a0 0x7f60bc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c245c700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60bc1007a0 0x7f60bc198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c245c700 1 -- 172.21.15.49:0/2412513777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b80095f0 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c1c5b700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60bc1007a0 0x7f60bc198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.958+0000 7f60c245c700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 0x7f60bc197b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60b800bd70 tx=0x7f60b800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.959+0000 7f60af7fe700 1 -- 172.21.15.49:0/2412513777 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60b801d070 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.959+0000 7f60af7fe700 1 -- 172.21.15.49:0/2412513777 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60b800f460 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.959+0000 7f60af7fe700 1 -- 172.21.15.49:0/2412513777 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60b8017630 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.959+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc1989a0 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.959+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc1a0f00 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.960+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc108190 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.960+0000 7f60af7fe700 1 -- 172.21.15.49:0/2412513777 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f60b800f5d0 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.961+0000 7f60af7fe700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f60a806cce0 0x7f60a806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.961+0000 7f60af7fe700 1 -- 172.21.15.49:0/2412513777 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f60b808d280 con 0x7f60bc0ffdf0 2024-05-15T15:50:05.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.961+0000 7f60c1c5b700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f60a806cce0 0x7f60a806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.961+0000 7f60c1c5b700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f60a806cce0 0x7f60a806f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0699e0 tx=0x7f60b0008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:05.966+0000 7f60af7fe700 1 -- 172.21.15.49:0/2412513777 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f60b8057990 con 0x7f60bc0ffdf0 2024-05-15T15:50:06.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.170+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f60bc1012d0 con 0x7f60bc0ffdf0 2024-05-15T15:50:06.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.171+0000 7f60af7fe700 1 -- 172.21.15.49:0/2412513777 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f60b805afb0 con 0x7f60bc0ffdf0 2024-05-15T15:50:06.173 INFO:teuthology.orchestra.run.smithi049.stdout:38654705701 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f60a806cce0 msgr2=0x7f60a806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f60a806cce0 0x7f60a806f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0699e0 tx=0x7f60b0008040 comp rx=0 tx=0).stop 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 msgr2=0x7f60bc197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 0x7f60bc197b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60b800bd70 tx=0x7f60b800be50 comp rx=0 tx=0).stop 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 shutdown_connections 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60bc0ffdf0 0x7f60bc197b30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f60a806cce0 0x7f60a806f1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 --2- 172.21.15.49:0/2412513777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60bc1007a0 0x7f60bc198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 >> 172.21.15.49:0/2412513777 conn(0x7f60bc0fb6a0 msgr2=0x7f60bc0fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 shutdown_connections 2024-05-15T15:50:06.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:06.173+0000 7f60c46c0700 1 -- 172.21.15.49:0/2412513777 wait complete. 2024-05-15T15:50:06.582 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855373 got 150323855375 for osd.4 2024-05-15T15:50:06.582 DEBUG:teuthology.parallel:result is None 2024-05-15T15:50:06.875 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:06 smithi049 ceph-mon[28399]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:06.876 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:06 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2412513777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-15T15:50:06.876 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:50:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:06 smithi144 ceph-mon[36389]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:06 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2412513777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-15T15:50:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:50:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.027+0000 7fab35645700 1 -- 172.21.15.49:0/485952387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 msgr2=0x7fab30102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.027+0000 7fab35645700 1 --2- 172.21.15.49:0/485952387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 0x7fab30102250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab20009910 tx=0x7fab20009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.027+0000 7fab35645700 1 -- 172.21.15.49:0/485952387 shutdown_connections 2024-05-15T15:50:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.027+0000 7fab35645700 1 --2- 172.21.15.49:0/485952387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab30102790 0x7fab3010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.027+0000 7fab35645700 1 --2- 172.21.15.49:0/485952387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 0x7fab30102250 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.027+0000 7fab35645700 1 -- 172.21.15.49:0/485952387 >> 172.21.15.49:0/485952387 conn(0x7fab300fb700 msgr2=0x7fab300fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:08.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.027+0000 7fab35645700 1 -- 172.21.15.49:0/485952387 shutdown_connections 2024-05-15T15:50:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.027+0000 7fab35645700 1 -- 172.21.15.49:0/485952387 wait complete. 2024-05-15T15:50:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.027+0000 7fab35645700 1 Processor -- start 2024-05-15T15:50:08.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab35645700 1 -- start start 2024-05-15T15:50:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab35645700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 0x7fab3019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab35645700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab30102790 0x7fab3019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab35645700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3019d4b0 con 0x7fab30101e70 2024-05-15T15:50:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab35645700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab30196a90 con 0x7fab30102790 2024-05-15T15:50:08.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab2effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 0x7fab3019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab2effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 0x7fab3019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46926/0 (socket says 172.21.15.49:46926) 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab2effd700 1 -- 172.21.15.49:0/2469890911 learned_addr learned my addr 172.21.15.49:0/2469890911 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab2e7fc700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab30102790 0x7fab3019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab2effd700 1 -- 172.21.15.49:0/2469890911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab30102790 msgr2=0x7fab3019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab2effd700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab30102790 0x7fab3019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab2effd700 1 -- 172.21.15.49:0/2469890911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab200095f0 con 0x7fab30101e70 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.028+0000 7fab2effd700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 0x7fab3019c950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fab20005f50 tx=0x7fab2000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.029+0000 7fab17fff700 1 -- 172.21.15.49:0/2469890911 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2001d070 con 0x7fab30101e70 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.029+0000 7fab17fff700 1 -- 172.21.15.49:0/2469890911 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab200052a0 con 0x7fab30101e70 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.029+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab30196d10 con 0x7fab30101e70 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.029+0000 7fab17fff700 1 -- 172.21.15.49:0/2469890911 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab20017400 con 0x7fab30101e70 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.029+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab30197200 con 0x7fab30101e70 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.030+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab30105600 con 0x7fab30101e70 2024-05-15T15:50:08.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.030+0000 7fab17fff700 1 -- 172.21.15.49:0/2469890911 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fab2000f7a0 con 0x7fab30101e70 2024-05-15T15:50:08.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.031+0000 7fab17fff700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fab1806ccf0 0x7fab1806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:08.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.031+0000 7fab17fff700 1 -- 172.21.15.49:0/2469890911 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fab2008d3f0 con 0x7fab30101e70 2024-05-15T15:50:08.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.031+0000 7fab2e7fc700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fab1806ccf0 0x7fab1806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:08.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.031+0000 7fab2e7fc700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fab1806ccf0 0x7fab1806f1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab30197c50 tx=0x7fab24009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:08.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.036+0000 7fab17fff700 1 -- 172.21.15.49:0/2469890911 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fab2005b590 con 0x7fab30101e70 2024-05-15T15:50:08.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.241+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fab30069ab0 con 0x7fab30101e70 2024-05-15T15:50:08.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.242+0000 7fab17fff700 1 -- 172.21.15.49:0/2469890911 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab2008d020 con 0x7fab30101e70 2024-05-15T15:50:08.245 INFO:teuthology.orchestra.run.smithi049.stdout:60129542177 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fab1806ccf0 msgr2=0x7fab1806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fab1806ccf0 0x7fab1806f1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab30197c50 tx=0x7fab24009260 comp rx=0 tx=0).stop 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 msgr2=0x7fab3019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 0x7fab3019c950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fab20005f50 tx=0x7fab2000b650 comp rx=0 tx=0).stop 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 shutdown_connections 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab30101e70 0x7fab3019c950 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fab1806ccf0 0x7fab1806f1b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 --2- 172.21.15.49:0/2469890911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab30102790 0x7fab3019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 >> 172.21.15.49:0/2469890911 conn(0x7fab300fb700 msgr2=0x7fab30103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 shutdown_connections 2024-05-15T15:50:08.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:08.244+0000 7fab35645700 1 -- 172.21.15.49:0/2469890911 wait complete. 2024-05-15T15:50:08.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-05-15T15:50:08.598 DEBUG:teuthology.parallel:result is None 2024-05-15T15:50:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:08 smithi049 ceph-mon[28399]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:08 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2469890911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-15T15:50:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:08 smithi144 ceph-mon[36389]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:08 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2469890911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-15T15:50:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.385+0000 7f377f2ac700 1 -- 172.21.15.49:0/1708704776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 msgr2=0x7f37781058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.385+0000 7f377f2ac700 1 --2- 172.21.15.49:0/1708704776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 0x7f37781058f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f376c009910 tx=0x7f376c009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.385+0000 7f377f2ac700 1 -- 172.21.15.49:0/1708704776 shutdown_connections 2024-05-15T15:50:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.385+0000 7f377f2ac700 1 --2- 172.21.15.49:0/1708704776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 0x7f37781058f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.385+0000 7f377f2ac700 1 --2- 172.21.15.49:0/1708704776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3778100ef0 0x7f37781012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.385+0000 7f377f2ac700 1 -- 172.21.15.49:0/1708704776 >> 172.21.15.49:0/1708704776 conn(0x7f37780fc7c0 msgr2=0x7f37780febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.385+0000 7f377f2ac700 1 -- 172.21.15.49:0/1708704776 shutdown_connections 2024-05-15T15:50:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.385+0000 7f377f2ac700 1 -- 172.21.15.49:0/1708704776 wait complete. 2024-05-15T15:50:09.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377f2ac700 1 Processor -- start 2024-05-15T15:50:09.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377f2ac700 1 -- start start 2024-05-15T15:50:09.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377f2ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3778100ef0 0x7f377819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377f2ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 0x7f377819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377f2ac700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377819d4c0 con 0x7f37781018a0 2024-05-15T15:50:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377f2ac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3778196a30 con 0x7f3778100ef0 2024-05-15T15:50:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377c847700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 0x7f377819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377c847700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 0x7f377819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46936/0 (socket says 172.21.15.49:46936) 2024-05-15T15:50:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377c847700 1 -- 172.21.15.49:0/3208369645 learned_addr learned my addr 172.21.15.49:0/3208369645 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377d048700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3778100ef0 0x7f377819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377c847700 1 -- 172.21.15.49:0/3208369645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3778100ef0 msgr2=0x7f377819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.386+0000 7f377c847700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3778100ef0 0x7f377819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.387+0000 7f377c847700 1 -- 172.21.15.49:0/3208369645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c0095f0 con 0x7f37781018a0 2024-05-15T15:50:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.387+0000 7f377c847700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 0x7f377819ce30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f376c00bd70 tx=0x7f376c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.387+0000 7f37767fc700 1 -- 172.21.15.49:0/3208369645 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f376c01d070 con 0x7f37781018a0 2024-05-15T15:50:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.387+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3778196cb0 con 0x7f37781018a0 2024-05-15T15:50:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.387+0000 7f37767fc700 1 -- 172.21.15.49:0/3208369645 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f376c0056e0 con 0x7f37781018a0 2024-05-15T15:50:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.387+0000 7f37767fc700 1 -- 172.21.15.49:0/3208369645 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f376c017400 con 0x7f37781018a0 2024-05-15T15:50:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.387+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37781971a0 con 0x7f37781018a0 2024-05-15T15:50:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.388+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37781092f0 con 0x7f37781018a0 2024-05-15T15:50:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.388+0000 7f37767fc700 1 -- 172.21.15.49:0/3208369645 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f376c00f460 con 0x7f37781018a0 2024-05-15T15:50:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.389+0000 7f37767fc700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f376006cce0 0x7f376006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.389+0000 7f37767fc700 1 -- 172.21.15.49:0/3208369645 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f376c08d260 con 0x7f37781018a0 2024-05-15T15:50:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.389+0000 7f377d048700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f376006cce0 0x7f376006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.389+0000 7f377d048700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f376006cce0 0x7f376006f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3768009d10 tx=0x7f3768009040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.394+0000 7f37767fc700 1 -- 172.21.15.49:0/3208369645 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f376c05b540 con 0x7f37781018a0 2024-05-15T15:50:09.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.601+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f37781023d0 con 0x7f37781018a0 2024-05-15T15:50:09.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.601+0000 7f37767fc700 1 -- 172.21.15.49:0/3208369645 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f376c05b0d0 con 0x7f37781018a0 2024-05-15T15:50:09.604 INFO:teuthology.orchestra.run.smithi049.stdout:171798691852 2024-05-15T15:50:09.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f376006cce0 msgr2=0x7f376006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f376006cce0 0x7f376006f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3768009d10 tx=0x7f3768009040 comp rx=0 tx=0).stop 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 msgr2=0x7f377819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 0x7f377819ce30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f376c00bd70 tx=0x7f376c00be50 comp rx=0 tx=0).stop 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 shutdown_connections 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f376006cce0 0x7f376006f1a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3778100ef0 0x7f377819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 --2- 172.21.15.49:0/3208369645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37781018a0 0x7f377819ce30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 >> 172.21.15.49:0/3208369645 conn(0x7f37780fc7c0 msgr2=0x7f37780fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 shutdown_connections 2024-05-15T15:50:09.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:09.604+0000 7f377f2ac700 1 -- 172.21.15.49:0/3208369645 wait complete. 2024-05-15T15:50:09.944 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:09 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3208369645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-15T15:50:10.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.156+0000 7f391939d700 1 -- 172.21.15.49:0/1056695669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914102740 msgr2=0x7f391410ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:10.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.156+0000 7f391939d700 1 --2- 172.21.15.49:0/1056695669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914102740 0x7f391410ac30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3908009910 tx=0x7f3908009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:10.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.157+0000 7f391939d700 1 -- 172.21.15.49:0/1056695669 shutdown_connections 2024-05-15T15:50:10.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.157+0000 7f391939d700 1 --2- 172.21.15.49:0/1056695669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914102740 0x7f391410ac30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.157+0000 7f391939d700 1 --2- 172.21.15.49:0/1056695669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914101e20 0x7f3914102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.157+0000 7f391939d700 1 -- 172.21.15.49:0/1056695669 >> 172.21.15.49:0/1056695669 conn(0x7f3914076800 msgr2=0x7f39140fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:10.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.157+0000 7f391939d700 1 -- 172.21.15.49:0/1056695669 shutdown_connections 2024-05-15T15:50:10.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.157+0000 7f391939d700 1 -- 172.21.15.49:0/1056695669 wait complete. 2024-05-15T15:50:10.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.157+0000 7f391939d700 1 Processor -- start 2024-05-15T15:50:10.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.157+0000 7f391939d700 1 -- start start 2024-05-15T15:50:10.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f391939d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914101e20 0x7f391419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:10.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f391939d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914102740 0x7f391419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:10.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f391939d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391419d460 con 0x7f3914101e20 2024-05-15T15:50:10.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f391939d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3914196a40 con 0x7f3914102740 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f3912ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914101e20 0x7f391419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f3912ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914101e20 0x7f391419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46978/0 (socket says 172.21.15.49:46978) 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f3912ffd700 1 -- 172.21.15.49:0/2416311330 learned_addr learned my addr 172.21.15.49:0/2416311330 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f39127fc700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914102740 0x7f391419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f3912ffd700 1 -- 172.21.15.49:0/2416311330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914102740 msgr2=0x7f391419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f3912ffd700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914102740 0x7f391419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f3912ffd700 1 -- 172.21.15.49:0/2416311330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39080095f0 con 0x7f3914101e20 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.158+0000 7f3912ffd700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914101e20 0x7f391419c900 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f390400e920 tx=0x7f390400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.159+0000 7f38fbfff700 1 -- 172.21.15.49:0/2416311330 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f390400c9d0 con 0x7f3914101e20 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.159+0000 7f38fbfff700 1 -- 172.21.15.49:0/2416311330 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3904010430 con 0x7f3914101e20 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.159+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3914196d20 con 0x7f3914101e20 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.159+0000 7f38fbfff700 1 -- 172.21.15.49:0/2416311330 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3904018630 con 0x7f3914101e20 2024-05-15T15:50:10.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.159+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3914197240 con 0x7f3914101e20 2024-05-15T15:50:10.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.160+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3914105540 con 0x7f3914101e20 2024-05-15T15:50:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.160+0000 7f38fbfff700 1 -- 172.21.15.49:0/2416311330 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f39040105a0 con 0x7f3914101e20 2024-05-15T15:50:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.161+0000 7f38fbfff700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f38fc0710f0 0x7f38fc0735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.161+0000 7f38fbfff700 1 -- 172.21.15.49:0/2416311330 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3904014070 con 0x7f3914101e20 2024-05-15T15:50:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.161+0000 7f39127fc700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f38fc0710f0 0x7f38fc0735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.161+0000 7f39127fc700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f38fc0710f0 0x7f38fc0735b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39080059d0 tx=0x7f3908005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.166+0000 7f38fbfff700 1 -- 172.21.15.49:0/2416311330 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3904052d60 con 0x7f3914101e20 2024-05-15T15:50:10.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:09 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3208369645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-15T15:50:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.047+0000 7f34d4649700 1 -- 172.21.15.49:0/3475047710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 msgr2=0x7f34cc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.047+0000 7f34d4649700 1 --2- 172.21.15.49:0/3475047710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 0x7f34cc1012c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009910 tx=0x7f34c8009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.047+0000 7f34d4649700 1 -- 172.21.15.49:0/3475047710 shutdown_connections 2024-05-15T15:50:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.047+0000 7f34d4649700 1 --2- 172.21.15.49:0/3475047710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34cc101890 0x7f34cc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.047+0000 7f34d4649700 1 --2- 172.21.15.49:0/3475047710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 0x7f34cc1012c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.047+0000 7f34d4649700 1 -- 172.21.15.49:0/3475047710 >> 172.21.15.49:0/3475047710 conn(0x7f34cc0fc790 msgr2=0x7f34cc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.048+0000 7f34d4649700 1 -- 172.21.15.49:0/3475047710 shutdown_connections 2024-05-15T15:50:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.048+0000 7f34d4649700 1 -- 172.21.15.49:0/3475047710 wait complete. 2024-05-15T15:50:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.048+0000 7f34d4649700 1 Processor -- start 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.048+0000 7f34d4649700 1 -- start start 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.048+0000 7f34d4649700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 0x7f34cc198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.048+0000 7f34d4649700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34cc101890 0x7f34cc199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.048+0000 7f34d4649700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc199a50 con 0x7f34cc100ee0 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.048+0000 7f34d4649700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc19d6d0 con 0x7f34cc101890 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d23e5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 0x7f34cc198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d23e5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 0x7f34cc198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46960/0 (socket says 172.21.15.49:46960) 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d23e5700 1 -- 172.21.15.49:0/2675772889 learned_addr learned my addr 172.21.15.49:0/2675772889 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d1be4700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34cc101890 0x7f34cc199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d23e5700 1 -- 172.21.15.49:0/2675772889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34cc101890 msgr2=0x7f34cc199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d23e5700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34cc101890 0x7f34cc199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d23e5700 1 -- 172.21.15.49:0/2675772889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c80095f0 con 0x7f34cc100ee0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d23e5700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 0x7f34cc198e30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f34c800bd70 tx=0x7f34c800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34c37fe700 1 -- 172.21.15.49:0/2675772889 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c801d070 con 0x7f34cc100ee0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34c37fe700 1 -- 172.21.15.49:0/2675772889 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34c80056e0 con 0x7f34cc100ee0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc19d950 con 0x7f34cc100ee0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34c37fe700 1 -- 172.21.15.49:0/2675772889 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c800fe40 con 0x7f34cc100ee0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.049+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34cc19de40 con 0x7f34cc100ee0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.050+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34cc109280 con 0x7f34cc100ee0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.051+0000 7f34c37fe700 1 -- 172.21.15.49:0/2675772889 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f34c800f680 con 0x7f34cc100ee0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.052+0000 7f34c37fe700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f34b806cce0 0x7f34b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.052+0000 7f34c37fe700 1 -- 172.21.15.49:0/2675772889 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f34c808d230 con 0x7f34cc100ee0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.052+0000 7f34d1be4700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f34b806cce0 0x7f34b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.052+0000 7f34d1be4700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f34b806cce0 0x7f34b806f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34cc19a340 tx=0x7f34bc00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.058+0000 7f34c37fe700 1 -- 172.21.15.49:0/2675772889 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f34c80579c0 con 0x7f34cc100ee0 2024-05-15T15:50:10.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.287+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f34cc19a0f0 con 0x7f34cc100ee0 2024-05-15T15:50:10.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.288+0000 7f34c37fe700 1 -- 172.21.15.49:0/2675772889 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34c805afe0 con 0x7f34cc100ee0 2024-05-15T15:50:10.290 INFO:teuthology.orchestra.run.smithi049.stdout:115964117015 2024-05-15T15:50:10.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f34b806cce0 msgr2=0x7f34b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:10.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f34b806cce0 0x7f34b806f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34cc19a340 tx=0x7f34bc00b410 comp rx=0 tx=0).stop 2024-05-15T15:50:10.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 msgr2=0x7f34cc198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:10.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 0x7f34cc198e30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f34c800bd70 tx=0x7f34c800bda0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 shutdown_connections 2024-05-15T15:50:10.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34cc100ee0 0x7f34cc198e30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f34b806cce0 0x7f34b806f1a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 --2- 172.21.15.49:0/2675772889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34cc101890 0x7f34cc199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 >> 172.21.15.49:0/2675772889 conn(0x7f34cc0fc790 msgr2=0x7f34cc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:10.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 shutdown_connections 2024-05-15T15:50:10.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.291+0000 7f34d4649700 1 -- 172.21.15.49:0/2675772889 wait complete. 2024-05-15T15:50:10.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.400+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3914197f00 con 0x7f3914101e20 2024-05-15T15:50:10.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.400+0000 7f38fbfff700 1 -- 172.21.15.49:0/2416311330 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f390405a360 con 0x7f3914101e20 2024-05-15T15:50:10.403 INFO:teuthology.orchestra.run.smithi049.stdout:81604378652 2024-05-15T15:50:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f38fc0710f0 msgr2=0x7f38fc0735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f38fc0710f0 0x7f38fc0735b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39080059d0 tx=0x7f3908005940 comp rx=0 tx=0).stop 2024-05-15T15:50:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914101e20 msgr2=0x7f391419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914101e20 0x7f391419c900 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f390400e920 tx=0x7f390400ec30 comp rx=0 tx=0).stop 2024-05-15T15:50:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 shutdown_connections 2024-05-15T15:50:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3914101e20 0x7f391419c900 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f38fc0710f0 0x7f38fc0735b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 --2- 172.21.15.49:0/2416311330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914102740 0x7f391419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 >> 172.21.15.49:0/2416311330 conn(0x7f3914076800 msgr2=0x7f3914103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 shutdown_connections 2024-05-15T15:50:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:10.402+0000 7f391939d700 1 -- 172.21.15.49:0/2416311330 wait complete. 2024-05-15T15:50:10.762 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691852 for osd.5 2024-05-15T15:50:10.762 DEBUG:teuthology.parallel:result is None 2024-05-15T15:50:11.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:10 smithi049 ceph-mon[28399]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:11.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:10 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2675772889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-15T15:50:11.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:10 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2416311330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-15T15:50:11.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:10 smithi144 ceph-mon[36389]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:11.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:10 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2675772889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-15T15:50:11.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:10 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2416311330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-15T15:50:11.607 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378652 for osd.2 2024-05-15T15:50:11.607 DEBUG:teuthology.parallel:result is None 2024-05-15T15:50:11.999 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117012 got 115964117015 for osd.3 2024-05-15T15:50:11.999 DEBUG:teuthology.parallel:result is None 2024-05-15T15:50:12.461 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542175 got 60129542177 for osd.1 2024-05-15T15:50:12.461 DEBUG:teuthology.parallel:result is None 2024-05-15T15:50:12.461 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-15T15:50:12.462 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-15T15:50:12.684 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:13.053 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:12 smithi049 ceph-mon[28399]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:12 smithi144 ceph-mon[36389]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:14.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.275+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/3566269522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 msgr2=0x7f4d281047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:14.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.275+0000 7f4d2c8e0700 1 --2- 172.21.15.49:0/3566269522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 0x7f4d281047c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009910 tx=0x7f4d1c009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:14.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.275+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/3566269522 shutdown_connections 2024-05-15T15:50:14.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.275+0000 7f4d2c8e0700 1 --2- 172.21.15.49:0/3566269522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 0x7f4d281047c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:14.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.275+0000 7f4d2c8e0700 1 --2- 172.21.15.49:0/3566269522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d280ffdc0 0x7f4d281001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:14.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.275+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/3566269522 >> 172.21.15.49:0/3566269522 conn(0x7f4d28076800 msgr2=0x7f4d280fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:14.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.275+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/3566269522 shutdown_connections 2024-05-15T15:50:14.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.275+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/3566269522 wait complete. 2024-05-15T15:50:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d2c8e0700 1 Processor -- start 2024-05-15T15:50:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d2c8e0700 1 -- start start 2024-05-15T15:50:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d2c8e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d280ffdc0 0x7f4d280fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d2c8e0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 0x7f4d280fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d2c8e0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d280fc760 con 0x7f4d28100770 2024-05-15T15:50:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d2c8e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d280fc8a0 con 0x7f4d280ffdc0 2024-05-15T15:50:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d25d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 0x7f4d280fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d25d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 0x7f4d280fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33126/0 (socket says 172.21.15.49:33126) 2024-05-15T15:50:14.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d25d9b700 1 -- 172.21.15.49:0/1345747755 learned_addr learned my addr 172.21.15.49:0/1345747755 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.276+0000 7f4d2659c700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d280ffdc0 0x7f4d280fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.277+0000 7f4d25d9b700 1 -- 172.21.15.49:0/1345747755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d280ffdc0 msgr2=0x7f4d280fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.277+0000 7f4d25d9b700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d280ffdc0 0x7f4d280fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.277+0000 7f4d25d9b700 1 -- 172.21.15.49:0/1345747755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0095f0 con 0x7f4d28100770 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.277+0000 7f4d25d9b700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 0x7f4d280fc220 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c00bd70 tx=0x7f4d1c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.277+0000 7f4d177fe700 1 -- 172.21.15.49:0/1345747755 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1c01d070 con 0x7f4d28100770 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.277+0000 7f4d177fe700 1 -- 172.21.15.49:0/1345747755 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d1c0056c0 con 0x7f4d28100770 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.277+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d280fcb20 con 0x7f4d28100770 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.277+0000 7f4d177fe700 1 -- 172.21.15.49:0/1345747755 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1c017400 con 0x7f4d28100770 2024-05-15T15:50:14.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.277+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d281a30a0 con 0x7f4d28100770 2024-05-15T15:50:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.278+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d281012a0 con 0x7f4d28100770 2024-05-15T15:50:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.279+0000 7f4d177fe700 1 -- 172.21.15.49:0/1345747755 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4d1c003a60 con 0x7f4d28100770 2024-05-15T15:50:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.279+0000 7f4d177fe700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d1006cce0 0x7f4d1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.279+0000 7f4d177fe700 1 -- 172.21.15.49:0/1345747755 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4d1c08d2a0 con 0x7f4d28100770 2024-05-15T15:50:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.279+0000 7f4d2659c700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d1006cce0 0x7f4d1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:14.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.280+0000 7f4d2659c700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d1006cce0 0x7f4d1006f1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009a10 tx=0x7f4d18009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:14.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.284+0000 7f4d177fe700 1 -- 172.21.15.49:0/1345747755 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4d1c0579b0 con 0x7f4d28100770 2024-05-15T15:50:14.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.483+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d2810ce50 con 0x7f4d1006cce0 2024-05-15T15:50:14.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.485+0000 7f4d177fe700 1 -- 172.21.15.49:0/1345747755 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28159 (secure 0 0 0) 0x7f4d2810ce50 con 0x7f4d1006cce0 2024-05-15T15:50:14.489 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:50:14.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d1006cce0 msgr2=0x7f4d1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:14.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d1006cce0 0x7f4d1006f1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009a10 tx=0x7f4d18009190 comp rx=0 tx=0).stop 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 msgr2=0x7f4d280fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 0x7f4d280fc220 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c00bd70 tx=0x7f4d1c00bda0 comp rx=0 tx=0).stop 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 shutdown_connections 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d1006cce0 0x7f4d1006f1a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d280ffdc0 0x7f4d280fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 --2- 172.21.15.49:0/1345747755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d28100770 0x7f4d280fc220 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 >> 172.21.15.49:0/1345747755 conn(0x7f4d28076800 msgr2=0x7f4d28068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 shutdown_connections 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:14.487+0000 7f4d2c8e0700 1 -- 172.21.15.49:0/1345747755 wait complete. 2024-05-15T15:50:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-15T15:50:15.045 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:14 smithi144 ceph-mon[36389]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:15.082 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:14 smithi049 ceph-mon[28399]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:15.105 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-05-15T15:50:13.456261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":194720,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":190710,"kb_avail":562276192,"statfs":{"total":575970213888,"available":575770820608,"internally_reserved":0,"allocated":4018176,"data_stored":2631576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":195287718},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001968"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-15T15:49:24.496189+0000","last_change":"2024-05-15T15:48:57.547343+0000","last_active":"2024-05-15T15:49:24.496189+0000","last_peered":"2024-05-15T15:49:24.496189+0000","last_clean":"2024-05-15T15:49:24.496189+0000","last_became_active":"2024-05-15T15:48:57.546898+0000","last_became_peered":"2024-05-15T15:48:57.546898+0000","last_unstale":"2024-05-15T15:49:24.496189+0000","last_undegraded":"2024-05-15T15:49:24.496189+0000","last_fullsized":"2024-05-15T15:49:24.496189+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T15:48:09.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T15:48:09.899611+0000","last_clean_scrub_stamp":"2024-05-15T15:48:09.899611+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:52:49.400653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31480,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713672,"statfs":{"total":95995035648,"available":95962800128,"internally_reserved":0,"allocated":372736,"data_stored":143412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":143412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63300000000000001}]},{"osd":1,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.36599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":966656,"data_stored":733780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed May 15 15:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.375},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35576,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709576,"statfs":{"total":95995035648,"available":95958605824,"internally_reserved":0,"allocated":372736,"data_stored":143412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 15 15:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed May 15 15:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.32300000000000001}]},{"osd":3,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53800000000000003}]},{"osd":4,"last update":"Wed May 15 15:50:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40252,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704900,"statfs":{"total":95995035648,"available":95953817600,"internally_reserved":0,"allocated":966656,"data_stored":733780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 15 15:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.23699999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.32700000000000001},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.28999999999999998}]},{"osd":2,"last update":"Wed May 15 15:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.309},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed May 15 15:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 15 15:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.34200000000000003,"15min":0.33600000000000002},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.26900000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.36799999999999999}]},{"osd":2,"last update":"Wed May 15 15:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.33200000000000002}]},{"osd":3,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.73699999999999999}]},{"osd":4,"last update":"Wed May 15 15:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.0199999999999996,"5min":4.0199999999999996,"15min":4.0199999999999996},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-15T15:50:15.106 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-15T15:50:15.323 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:16.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:15 smithi144 ceph-mon[36389]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T15:50:16.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:15 smithi049 ceph-mon[28399]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.911+0000 7f58cd36d700 1 -- 172.21.15.49:0/2917353116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8101890 msgr2=0x7f58c81058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.911+0000 7f58cd36d700 1 --2- 172.21.15.49:0/2917353116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8101890 0x7f58c81058e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009910 tx=0x7f58b0009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.911+0000 7f58cd36d700 1 -- 172.21.15.49:0/2917353116 shutdown_connections 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.911+0000 7f58cd36d700 1 --2- 172.21.15.49:0/2917353116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8101890 0x7f58c81058e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.911+0000 7f58cd36d700 1 --2- 172.21.15.49:0/2917353116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c8100ee0 0x7f58c81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.911+0000 7f58cd36d700 1 -- 172.21.15.49:0/2917353116 >> 172.21.15.49:0/2917353116 conn(0x7f58c80fc790 msgr2=0x7f58c80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.911+0000 7f58cd36d700 1 -- 172.21.15.49:0/2917353116 shutdown_connections 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.912+0000 7f58cd36d700 1 -- 172.21.15.49:0/2917353116 wait complete. 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.912+0000 7f58cd36d700 1 Processor -- start 2024-05-15T15:50:16.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.912+0000 7f58cd36d700 1 -- start start 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58cd36d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8100ee0 0x7f58c819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58cd36d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c8101890 0x7f58c819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58cd36d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c819d3b0 con 0x7f58c8100ee0 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58cd36d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c8196a20 con 0x7f58c8101890 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58c6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8100ee0 0x7f58c819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58c6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8100ee0 0x7f58c819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33144/0 (socket says 172.21.15.49:33144) 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58c6ffd700 1 -- 172.21.15.49:0/1322443888 learned_addr learned my addr 172.21.15.49:0/1322443888 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58c67fc700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c8101890 0x7f58c819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58c6ffd700 1 -- 172.21.15.49:0/1322443888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c8101890 msgr2=0x7f58c819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:16.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58c6ffd700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c8101890 0x7f58c819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:16.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58c6ffd700 1 -- 172.21.15.49:0/1322443888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b00095f0 con 0x7f58c8100ee0 2024-05-15T15:50:16.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.913+0000 7f58c6ffd700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8100ee0 0x7f58c819c8e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58b800c930 tx=0x7f58b800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:16.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.914+0000 7f58bffff700 1 -- 172.21.15.49:0/1322443888 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b8007ad0 con 0x7f58c8100ee0 2024-05-15T15:50:16.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.914+0000 7f58bffff700 1 -- 172.21.15.49:0/1322443888 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58b8007c30 con 0x7f58c8100ee0 2024-05-15T15:50:16.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.914+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c8196d00 con 0x7f58c8100ee0 2024-05-15T15:50:16.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.914+0000 7f58bffff700 1 -- 172.21.15.49:0/1322443888 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b8018610 con 0x7f58c8100ee0 2024-05-15T15:50:16.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.914+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c8197250 con 0x7f58c8100ee0 2024-05-15T15:50:16.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.915+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c81092e0 con 0x7f58c8100ee0 2024-05-15T15:50:16.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.915+0000 7f58bffff700 1 -- 172.21.15.49:0/1322443888 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f58b800f450 con 0x7f58c8100ee0 2024-05-15T15:50:16.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.916+0000 7f58bffff700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f58b406cce0 0x7f58b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:16.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.916+0000 7f58c67fc700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f58b406cce0 0x7f58b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:16.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.916+0000 7f58bffff700 1 -- 172.21.15.49:0/1322443888 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f58b808c140 con 0x7f58c8100ee0 2024-05-15T15:50:16.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.916+0000 7f58c67fc700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f58b406cce0 0x7f58b406f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f58c8197be0 tx=0x7f58b001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:16.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:16.921+0000 7f58bffff700 1 -- 172.21.15.49:0/1322443888 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f58b805a2e0 con 0x7f58c8100ee0 2024-05-15T15:50:16.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:16 smithi049 ceph-mon[28399]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:17.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.120+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58c8197ee0 con 0x7f58b406cce0 2024-05-15T15:50:17.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.122+0000 7f58bffff700 1 -- 172.21.15.49:0/1322443888 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28126 (secure 0 0 0) 0x7f58c8197ee0 con 0x7f58b406cce0 2024-05-15T15:50:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:50:17.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.124+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f58b406cce0 msgr2=0x7f58b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:17.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.124+0000 7f58cd36d700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f58b406cce0 0x7f58b406f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f58c8197be0 tx=0x7f58b001a040 comp rx=0 tx=0).stop 2024-05-15T15:50:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.124+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8100ee0 msgr2=0x7f58c819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.124+0000 7f58cd36d700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8100ee0 0x7f58c819c8e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58b800c930 tx=0x7f58b800ccf0 comp rx=0 tx=0).stop 2024-05-15T15:50:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.124+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 shutdown_connections 2024-05-15T15:50:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.124+0000 7f58cd36d700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c8100ee0 0x7f58c819c8e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.124+0000 7f58cd36d700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f58b406cce0 0x7f58b406f1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.124+0000 7f58cd36d700 1 --2- 172.21.15.49:0/1322443888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c8101890 0x7f58c819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.124+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 >> 172.21.15.49:0/1322443888 conn(0x7f58c80fc790 msgr2=0x7f58c80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:17.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.125+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 shutdown_connections 2024-05-15T15:50:17.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:17.125+0000 7f58cd36d700 1 -- 172.21.15.49:0/1322443888 wait complete. 2024-05-15T15:50:17.128 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-15T15:50:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:16 smithi144 ceph-mon[36389]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:17.806 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-05-15T15:50:15.456573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":194720,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":190710,"kb_avail":562276192,"statfs":{"total":575970213888,"available":575770820608,"internally_reserved":0,"allocated":4018176,"data_stored":2631576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":195287718},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002008"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-15T15:49:24.496189+0000","last_change":"2024-05-15T15:48:57.547343+0000","last_active":"2024-05-15T15:49:24.496189+0000","last_peered":"2024-05-15T15:49:24.496189+0000","last_clean":"2024-05-15T15:49:24.496189+0000","last_became_active":"2024-05-15T15:48:57.546898+0000","last_became_peered":"2024-05-15T15:48:57.546898+0000","last_unstale":"2024-05-15T15:49:24.496189+0000","last_undegraded":"2024-05-15T15:49:24.496189+0000","last_fullsized":"2024-05-15T15:49:24.496189+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T15:48:09.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T15:48:09.899611+0000","last_clean_scrub_stamp":"2024-05-15T15:48:09.899611+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:52:49.400653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31480,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713672,"statfs":{"total":95995035648,"available":95962800128,"internally_reserved":0,"allocated":372736,"data_stored":143412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":143412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63300000000000001}]},{"osd":1,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.36599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":966656,"data_stored":733780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed May 15 15:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.375},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35576,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709576,"statfs":{"total":95995035648,"available":95958605824,"internally_reserved":0,"allocated":372736,"data_stored":143412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 15 15:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.374,"15min":0.374},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.378,"15min":0.38200000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.50700000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.36199999999999999}]},{"osd":1,"last update":"Wed May 15 15:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.377,"15min":0.38},"min":{"1min":0.27900000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.437}]},{"osd":3,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.68000000000000005}]},{"osd":4,"last update":"Wed May 15 15:50:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40252,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704900,"statfs":{"total":95995035648,"available":95953817600,"internally_reserved":0,"allocated":966656,"data_stored":733780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 15 15:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.32500000000000001,"15min":0.317},"min":{"1min":0.23699999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.32700000000000001},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.28999999999999998}]},{"osd":2,"last update":"Wed May 15 15:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.309},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed May 15 15:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed May 15 15:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 15 15:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.34200000000000003,"15min":0.33600000000000002},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.26900000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.36799999999999999}]},{"osd":2,"last update":"Wed May 15 15:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.33200000000000002}]},{"osd":3,"last update":"Wed May 15 15:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.73699999999999999}]},{"osd":4,"last update":"Wed May 15 15:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.0199999999999996,"5min":4.0199999999999996,"15min":4.0199999999999996},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-15T15:50:17.807 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-15T15:50:17.807 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-15T15:50:17.807 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-15T15:50:17.807 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-15T15:50:18.031 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:18.154 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:17 smithi049 ceph-mon[28399]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T15:50:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:17 smithi144 ceph-mon[36389]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T15:50:19.100 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:18 smithi144 ceph-mon[36389]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:19.119 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:18 smithi049 ceph-mon[28399]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:19.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.618+0000 7f12de0d5700 1 -- 172.21.15.49:0/501230073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d80ffe10 msgr2=0x7f12d81001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:19.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.618+0000 7f12de0d5700 1 --2- 172.21.15.49:0/501230073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d80ffe10 0x7f12d81001f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009910 tx=0x7f12c8009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:19.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.618+0000 7f12de0d5700 1 -- 172.21.15.49:0/501230073 shutdown_connections 2024-05-15T15:50:19.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.618+0000 7f12de0d5700 1 --2- 172.21.15.49:0/501230073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d81007c0 0x7f12d8104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:19.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.618+0000 7f12de0d5700 1 --2- 172.21.15.49:0/501230073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d80ffe10 0x7f12d81001f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:19.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.618+0000 7f12de0d5700 1 -- 172.21.15.49:0/501230073 >> 172.21.15.49:0/501230073 conn(0x7f12d80fb700 msgr2=0x7f12d80fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:19.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.618+0000 7f12de0d5700 1 -- 172.21.15.49:0/501230073 shutdown_connections 2024-05-15T15:50:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.618+0000 7f12de0d5700 1 -- 172.21.15.49:0/501230073 wait complete. 2024-05-15T15:50:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.618+0000 7f12de0d5700 1 Processor -- start 2024-05-15T15:50:19.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12de0d5700 1 -- start start 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12de0d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d80ffe10 0x7f12d819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12de0d5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d81007c0 0x7f12d819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12de0d5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d819c360 con 0x7f12d81007c0 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12de0d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8195940 con 0x7f12d80ffe10 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12d6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d81007c0 0x7f12d819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12d6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d81007c0 0x7f12d819bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33160/0 (socket says 172.21.15.49:33160) 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12d6ffd700 1 -- 172.21.15.49:0/5491390 learned_addr learned my addr 172.21.15.49:0/5491390 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12d77fe700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d80ffe10 0x7f12d819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12d6ffd700 1 -- 172.21.15.49:0/5491390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d80ffe10 msgr2=0x7f12d819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12d6ffd700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d80ffe10 0x7f12d819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12d6ffd700 1 -- 172.21.15.49:0/5491390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c80095f0 con 0x7f12d81007c0 2024-05-15T15:50:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.619+0000 7f12d77fe700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d80ffe10 0x7f12d819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T15:50:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.620+0000 7f12d6ffd700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d81007c0 0x7f12d819bd40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00e8f0 tx=0x7f12cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.620+0000 7f12d4ff9700 1 -- 172.21.15.49:0/5491390 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12cc009790 con 0x7f12d81007c0 2024-05-15T15:50:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.620+0000 7f12d4ff9700 1 -- 172.21.15.49:0/5491390 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12cc004cf0 con 0x7f12d81007c0 2024-05-15T15:50:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.620+0000 7f12d4ff9700 1 -- 172.21.15.49:0/5491390 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12cc005690 con 0x7f12d81007c0 2024-05-15T15:50:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.620+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d8195c20 con 0x7f12d81007c0 2024-05-15T15:50:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.620+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d8196170 con 0x7f12d81007c0 2024-05-15T15:50:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.621+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d81081b0 con 0x7f12d81007c0 2024-05-15T15:50:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.621+0000 7f12d4ff9700 1 -- 172.21.15.49:0/5491390 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f12cc004750 con 0x7f12d81007c0 2024-05-15T15:50:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.622+0000 7f12d4ff9700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f12c006cce0 0x7f12c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.622+0000 7f12d77fe700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f12c006cce0 0x7f12c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.622+0000 7f12d4ff9700 1 -- 172.21.15.49:0/5491390 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f12cc014070 con 0x7f12d81007c0 2024-05-15T15:50:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.622+0000 7f12d77fe700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f12c006cce0 0x7f12c006f1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f12c800b760 tx=0x7f12c8005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:19.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.627+0000 7f12d4ff9700 1 -- 172.21.15.49:0/5491390 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f12cc05a4f0 con 0x7f12d81007c0 2024-05-15T15:50:19.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.886+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f12d8196e00 con 0x7f12d81007c0 2024-05-15T15:50:19.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.887+0000 7f12d4ff9700 1 -- 172.21.15.49:0/5491390 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f12cc05a080 con 0x7f12d81007c0 2024-05-15T15:50:19.888 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:50:19.888 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f12c006cce0 msgr2=0x7f12c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f12c006cce0 0x7f12c006f1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f12c800b760 tx=0x7f12c8005940 comp rx=0 tx=0).stop 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d81007c0 msgr2=0x7f12d819bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d81007c0 0x7f12d819bd40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00e8f0 tx=0x7f12cc00ecb0 comp rx=0 tx=0).stop 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 shutdown_connections 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f12c006cce0 0x7f12c006f1a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d80ffe10 0x7f12d819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 --2- 172.21.15.49:0/5491390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12d81007c0 0x7f12d819bd40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 >> 172.21.15.49:0/5491390 conn(0x7f12d80fb700 msgr2=0x7f12d80fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 shutdown_connections 2024-05-15T15:50:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:19.889+0000 7f12de0d5700 1 -- 172.21.15.49:0/5491390 wait complete. 2024-05-15T15:50:20.491 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-15T15:50:20.492 INFO:tasks.cephadm:Setup complete, yielding 2024-05-15T15:50:20.492 INFO:teuthology.run_tasks:Running task print... 2024-05-15T15:50:20.502 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-05-15T15:50:20.503 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T15:50:20.512 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:50:20.513 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-05-15T15:50:20.728 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:21.062 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:20 smithi049 ceph-mon[28399]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:21.062 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:20 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/5491390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-15T15:50:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:20 smithi144 ceph-mon[36389]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:20 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/5491390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-15T15:50:22.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:21 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:50:22.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:21 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:50:22.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.313+0000 7fe70c832700 1 -- 172.21.15.49:0/380526541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7080ffbe0 msgr2=0x7fe708100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:22.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.313+0000 7fe70c832700 1 --2- 172.21.15.49:0/380526541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7080ffbe0 0x7fe708100000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009910 tx=0x7fe6f8009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.313+0000 7fe70c832700 1 -- 172.21.15.49:0/380526541 shutdown_connections 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.313+0000 7fe70c832700 1 --2- 172.21.15.49:0/380526541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe708100de0 0x7fe708101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.313+0000 7fe70c832700 1 --2- 172.21.15.49:0/380526541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7080ffbe0 0x7fe708100000 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.313+0000 7fe70c832700 1 -- 172.21.15.49:0/380526541 >> 172.21.15.49:0/380526541 conn(0x7fe7080fb180 msgr2=0x7fe7080fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.313+0000 7fe70c832700 1 -- 172.21.15.49:0/380526541 shutdown_connections 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.313+0000 7fe70c832700 1 -- 172.21.15.49:0/380526541 wait complete. 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe70c832700 1 Processor -- start 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe70c832700 1 -- start start 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe70c832700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7080ffbe0 0x7fe708103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe70c832700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe708100de0 0x7fe7081021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe70c832700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe708102700 con 0x7fe708100de0 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe70c832700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe708102840 con 0x7fe7080ffbe0 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe705d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe708100de0 0x7fe7081021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:22.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe705d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe708100de0 0x7fe7081021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33164/0 (socket says 172.21.15.49:33164) 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe705d9b700 1 -- 172.21.15.49:0/1799677017 learned_addr learned my addr 172.21.15.49:0/1799677017 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe70659c700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7080ffbe0 0x7fe708103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe705d9b700 1 -- 172.21.15.49:0/1799677017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7080ffbe0 msgr2=0x7fe708103b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe705d9b700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7080ffbe0 0x7fe708103b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe705d9b700 1 -- 172.21.15.49:0/1799677017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80095f0 con 0x7fe708100de0 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.314+0000 7fe705d9b700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe708100de0 0x7fe7081021c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00d8d0 tx=0x7fe6fc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.315+0000 7fe6f77fe700 1 -- 172.21.15.49:0/1799677017 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6fc0098a0 con 0x7fe708100de0 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.315+0000 7fe6f77fe700 1 -- 172.21.15.49:0/1799677017 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6fc010460 con 0x7fe708100de0 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.315+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe708102b20 con 0x7fe708100de0 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.315+0000 7fe6f77fe700 1 -- 172.21.15.49:0/1799677017 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6fc00f5f0 con 0x7fe708100de0 2024-05-15T15:50:22.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.315+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70819e790 con 0x7fe708100de0 2024-05-15T15:50:22.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.316+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70804ec60 con 0x7fe708100de0 2024-05-15T15:50:22.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.316+0000 7fe6f77fe700 1 -- 172.21.15.49:0/1799677017 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe6fc009a00 con 0x7fe708100de0 2024-05-15T15:50:22.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.317+0000 7fe6f77fe700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe6f006cce0 0x7fe6f006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:22.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.317+0000 7fe70659c700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe6f006cce0 0x7fe6f006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:22.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.317+0000 7fe6f77fe700 1 -- 172.21.15.49:0/1799677017 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe6fc08bd30 con 0x7fe708100de0 2024-05-15T15:50:22.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.317+0000 7fe70659c700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe6f006cce0 0x7fe6f006f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f8005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:22.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.322+0000 7fe6f77fe700 1 -- 172.21.15.49:0/1799677017 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe6fc059f50 con 0x7fe708100de0 2024-05-15T15:50:22.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.523+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fe708108740 con 0x7fe708100de0 2024-05-15T15:50:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.527+0000 7fe6f77fe700 1 -- 172.21.15.49:0/1799677017 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v24)=0 v24) v1 ==== 143+0+0 (secure 0 0 0) 0x7fe6fc059ae0 con 0x7fe708100de0 2024-05-15T15:50:22.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe6f006cce0 msgr2=0x7fe6f006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:22.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe6f006cce0 0x7fe6f006f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f8005960 comp rx=0 tx=0).stop 2024-05-15T15:50:22.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe708100de0 msgr2=0x7fe7081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe708100de0 0x7fe7081021c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00d8d0 tx=0x7fe6fc00dbe0 comp rx=0 tx=0).stop 2024-05-15T15:50:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 shutdown_connections 2024-05-15T15:50:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe6f006cce0 0x7fe6f006f1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7080ffbe0 0x7fe708103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 --2- 172.21.15.49:0/1799677017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe708100de0 0x7fe7081021c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 >> 172.21.15.49:0/1799677017 conn(0x7fe7080fb180 msgr2=0x7fe708069280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 shutdown_connections 2024-05-15T15:50:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:22.530+0000 7fe70c832700 1 -- 172.21.15.49:0/1799677017 wait complete. 2024-05-15T15:50:23.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:22 smithi144 ceph-mon[36389]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:23.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1799677017' entity='client.admin' 2024-05-15T15:50:23.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:50:23.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:22 smithi049 ceph-mon[28399]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:23.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:22 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1799677017' entity='client.admin' 2024-05-15T15:50:23.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:50:23.245 INFO:teuthology.run_tasks:Running task print... 2024-05-15T15:50:23.254 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-05-15T15:50:23.254 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T15:50:23.266 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:50:23.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-15T15:50:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:23.926 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:23 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:23.927 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:23 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:50:23.927 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:23 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:24.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:23 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:24.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:23 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:50:24.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:23 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:25.073 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:24 smithi049 ceph-mon[28399]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:25.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.071+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2464112415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc1017f0 msgr2=0x7f85dc103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:25.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.071+0000 7f85e0eb7700 1 --2- 172.21.15.49:0/2464112415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc1017f0 0x7f85dc103be0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00b0a0 tx=0x7f85cc00b3b0 comp rx=0 tx=0).stop 2024-05-15T15:50:25.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.071+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2464112415 shutdown_connections 2024-05-15T15:50:25.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.071+0000 7f85e0eb7700 1 --2- 172.21.15.49:0/2464112415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85dc104120 0x7f85dc106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:25.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.071+0000 7f85e0eb7700 1 --2- 172.21.15.49:0/2464112415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc1017f0 0x7f85dc103be0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:25.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.071+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2464112415 >> 172.21.15.49:0/2464112415 conn(0x7f85dc0fb180 msgr2=0x7f85dc0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.071+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2464112415 shutdown_connections 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.071+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2464112415 wait complete. 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.072+0000 7f85e0eb7700 1 Processor -- start 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.072+0000 7f85e0eb7700 1 -- start start 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.072+0000 7f85e0eb7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85dc1017f0 0x7f85dc196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.072+0000 7f85e0eb7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc104120 0x7f85dc196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.072+0000 7f85e0eb7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc1972a0 con 0x7f85dc104120 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.072+0000 7f85e0eb7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc1973e0 con 0x7f85dc1017f0 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.072+0000 7f85d9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc104120 0x7f85dc196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.072+0000 7f85d9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc104120 0x7f85dc196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35008/0 (socket says 172.21.15.49:35008) 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.072+0000 7f85d9d9b700 1 -- 172.21.15.49:0/2755668181 learned_addr learned my addr 172.21.15.49:0/2755668181 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:25.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85da59c700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85dc1017f0 0x7f85dc196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:25.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85d9d9b700 1 -- 172.21.15.49:0/2755668181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85dc1017f0 msgr2=0x7f85dc196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:25.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85d9d9b700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85dc1017f0 0x7f85dc196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:25.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85d9d9b700 1 -- 172.21.15.49:0/2755668181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc009d00 con 0x7f85dc104120 2024-05-15T15:50:25.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85d9d9b700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc104120 0x7f85dc196c80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f85d000d8d0 tx=0x7f85d000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:25.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85cb7fe700 1 -- 172.21.15.49:0/2755668181 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85d00098a0 con 0x7f85dc104120 2024-05-15T15:50:25.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85cb7fe700 1 -- 172.21.15.49:0/2755668181 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85d0010460 con 0x7f85dc104120 2024-05-15T15:50:25.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85cb7fe700 1 -- 172.21.15.49:0/2755668181 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85d000f5f0 con 0x7f85dc104120 2024-05-15T15:50:25.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc19be40 con 0x7f85dc104120 2024-05-15T15:50:25.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.073+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc19c3e0 con 0x7f85dc104120 2024-05-15T15:50:25.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.075+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc04ec60 con 0x7f85dc104120 2024-05-15T15:50:25.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.075+0000 7f85cb7fe700 1 -- 172.21.15.49:0/2755668181 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f85d0009a00 con 0x7f85dc104120 2024-05-15T15:50:25.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.076+0000 7f85cb7fe700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f85c406ccf0 0x7f85c406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:25.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.076+0000 7f85cb7fe700 1 -- 172.21.15.49:0/2755668181 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f85d008bd10 con 0x7f85dc104120 2024-05-15T15:50:25.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.076+0000 7f85da59c700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f85c406ccf0 0x7f85c406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:25.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.076+0000 7f85da59c700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f85c406ccf0 0x7f85c406f1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85cc000f80 tx=0x7f85cc007910 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.081+0000 7f85cb7fe700 1 -- 172.21.15.49:0/2755668181 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f85d0059f30 con 0x7f85dc104120 2024-05-15T15:50:25.122 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:24 smithi144 ceph-mon[36389]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:25.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.301+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85dc104b80 con 0x7f85c406ccf0 2024-05-15T15:50:25.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.302+0000 7f85cb7fe700 1 -- 172.21.15.49:0/2755668181 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f85dc104b80 con 0x7f85c406ccf0 2024-05-15T15:50:25.304 INFO:teuthology.orchestra.run.smithi049.stdout:Backend: cephadm 2024-05-15T15:50:25.304 INFO:teuthology.orchestra.run.smithi049.stdout:Available: Yes 2024-05-15T15:50:25.304 INFO:teuthology.orchestra.run.smithi049.stdout:Paused: No 2024-05-15T15:50:25.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.304+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f85c406ccf0 msgr2=0x7f85c406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:25.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.304+0000 7f85e0eb7700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f85c406ccf0 0x7f85c406f1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85cc000f80 tx=0x7f85cc007910 comp rx=0 tx=0).stop 2024-05-15T15:50:25.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.304+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc104120 msgr2=0x7f85dc196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:25.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.304+0000 7f85e0eb7700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc104120 0x7f85dc196c80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f85d000d8d0 tx=0x7f85d000dbe0 comp rx=0 tx=0).stop 2024-05-15T15:50:25.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.305+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 shutdown_connections 2024-05-15T15:50:25.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.305+0000 7f85e0eb7700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f85c406ccf0 0x7f85c406f1b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:25.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.305+0000 7f85e0eb7700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85dc1017f0 0x7f85dc196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:25.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.305+0000 7f85e0eb7700 1 --2- 172.21.15.49:0/2755668181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85dc104120 0x7f85dc196c80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:25.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.305+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 >> 172.21.15.49:0/2755668181 conn(0x7f85dc0fb180 msgr2=0x7f85dc0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:25.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.305+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 shutdown_connections 2024-05-15T15:50:25.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:25.305+0000 7f85e0eb7700 1 -- 172.21.15.49:0/2755668181 wait complete. 2024-05-15T15:50:25.952 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-15T15:50:26.170 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:27.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:26 smithi144 ceph-mon[36389]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:27.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:26 smithi144 ceph-mon[36389]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:27.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:26 smithi049 ceph-mon[28399]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:27.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:26 smithi049 ceph-mon[28399]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.815+0000 7f2568cdf700 1 -- 172.21.15.49:0/1200646767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564100cf0 msgr2=0x7f2564101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.815+0000 7f2568cdf700 1 --2- 172.21.15.49:0/1200646767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564100cf0 0x7f2564101110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2554009910 tx=0x7f2554009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.815+0000 7f2568cdf700 1 -- 172.21.15.49:0/1200646767 shutdown_connections 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.815+0000 7f2568cdf700 1 --2- 172.21.15.49:0/1200646767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2564101ef0 0x7f2564102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.815+0000 7f2568cdf700 1 --2- 172.21.15.49:0/1200646767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564100cf0 0x7f2564101110 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.815+0000 7f2568cdf700 1 -- 172.21.15.49:0/1200646767 >> 172.21.15.49:0/1200646767 conn(0x7f25640fc290 msgr2=0x7f25640fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.815+0000 7f2568cdf700 1 -- 172.21.15.49:0/1200646767 shutdown_connections 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.815+0000 7f2568cdf700 1 -- 172.21.15.49:0/1200646767 wait complete. 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f2568cdf700 1 Processor -- start 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f2568cdf700 1 -- start start 2024-05-15T15:50:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f2568cdf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2564100cf0 0x7f2564074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f2568cdf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564101ef0 0x7f2564073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f2568cdf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25640738d0 con 0x7f2564101ef0 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f2568cdf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2564073a10 con 0x7f2564100cf0 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f2561d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564101ef0 0x7f2564073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f2561d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564101ef0 0x7f2564073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35016/0 (socket says 172.21.15.49:35016) 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f2561d9b700 1 -- 172.21.15.49:0/2825302356 learned_addr learned my addr 172.21.15.49:0/2825302356 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.816+0000 7f256259c700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2564100cf0 0x7f2564074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.817+0000 7f2561d9b700 1 -- 172.21.15.49:0/2825302356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2564100cf0 msgr2=0x7f2564074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.817+0000 7f2561d9b700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2564100cf0 0x7f2564074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.817+0000 7f2561d9b700 1 -- 172.21.15.49:0/2825302356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25540095f0 con 0x7f2564101ef0 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.817+0000 7f2561d9b700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564101ef0 0x7f2564073390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f255800d8d0 tx=0x7f255800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.817+0000 7f25537fe700 1 -- 172.21.15.49:0/2825302356 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25580098a0 con 0x7f2564101ef0 2024-05-15T15:50:27.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.817+0000 7f25537fe700 1 -- 172.21.15.49:0/2825302356 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2558010460 con 0x7f2564101ef0 2024-05-15T15:50:27.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.817+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2564073cf0 con 0x7f2564101ef0 2024-05-15T15:50:27.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.817+0000 7f25537fe700 1 -- 172.21.15.49:0/2825302356 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f255800f5f0 con 0x7f2564101ef0 2024-05-15T15:50:27.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.817+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256419e790 con 0x7f2564101ef0 2024-05-15T15:50:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.819+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2564190870 con 0x7f2564101ef0 2024-05-15T15:50:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.819+0000 7f25537fe700 1 -- 172.21.15.49:0/2825302356 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2558009a00 con 0x7f2564101ef0 2024-05-15T15:50:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.819+0000 7f25537fe700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f254c06cce0 0x7f254c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.819+0000 7f25537fe700 1 -- 172.21.15.49:0/2825302356 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f255808bdd0 con 0x7f2564101ef0 2024-05-15T15:50:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.819+0000 7f256259c700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f254c06cce0 0x7f254c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.820+0000 7f256259c700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f254c06cce0 0x7f254c06f1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f255400b5c0 tx=0x7f2554005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:27.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:27.824+0000 7f25537fe700 1 -- 172.21.15.49:0/2825302356 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2558059f70 con 0x7f2564101ef0 2024-05-15T15:50:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.038+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f25640613e0 con 0x7f254c06cce0 2024-05-15T15:50:28.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.047+0000 7f25537fe700 1 -- 172.21.15.49:0/2825302356 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3100 (secure 0 0 0) 0x7f25640613e0 con 0x7f254c06cce0 2024-05-15T15:50:28.050 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:50:28.050 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (3m) 2m ago 5m 25.6M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:50:28.050 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (5m) 2m ago 5m 7067k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:50:28.050 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (4m) 65s ago 4m 6966k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:50:28.050 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (5m) 2m ago 5m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:50:28.050 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (4m) 65s ago 4m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:50:28.050 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (3m) 2m ago 4m 80.5M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:50:28.050 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:9283,8765,8443 running (6m) 2m ago 6m 491M - 18.2.2-1343-g2965081f a04a73ffe079 ea7d1a4b9c55 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (4m) 65s ago 4m 451M - 18.2.2-1343-g2965081f a04a73ffe079 eece4ed6d244 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (6m) 2m ago 6m 44.0M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (4m) 65s ago 4m 42.9M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (5m) 2m ago 5m 14.4M - 1.5.0 0da6a335fe13 171e0a34f565 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (4m) 65s ago 4m 15.4M - 1.5.0 0da6a335fe13 be3a2837bc85 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (3m) 2m ago 3m 46.5M 3946M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (2m) 2m ago 2m 39.0M 3946M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (2m) 2m ago 2m 45.5M 3946M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (2m) 65s ago 2m 48.3M 4970M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (97s) 65s ago 97s 39.2M 4970M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (70s) 65s ago 70s 13.9M 4970M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:50:28.051 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (3m) 2m ago 4m 40.0M - 2.43.0 a07b618ecd1d 0299663ce1de 2024-05-15T15:50:28.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f254c06cce0 msgr2=0x7f254c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:28.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f254c06cce0 0x7f254c06f1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f255400b5c0 tx=0x7f2554005960 comp rx=0 tx=0).stop 2024-05-15T15:50:28.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564101ef0 msgr2=0x7f2564073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:28.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564101ef0 0x7f2564073390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f255800d8d0 tx=0x7f255800dbe0 comp rx=0 tx=0).stop 2024-05-15T15:50:28.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 shutdown_connections 2024-05-15T15:50:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f254c06cce0 0x7f254c06f1a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2564100cf0 0x7f2564074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 --2- 172.21.15.49:0/2825302356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2564101ef0 0x7f2564073390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 >> 172.21.15.49:0/2825302356 conn(0x7f25640fc290 msgr2=0x7f2564105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.049+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 shutdown_connections 2024-05-15T15:50:28.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:28.050+0000 7f2568cdf700 1 -- 172.21.15.49:0/2825302356 wait complete. 2024-05-15T15:50:28.814 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-15T15:50:29.033 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:29.152 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:28 smithi049 ceph-mon[28399]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:29.152 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:28 smithi049 ceph-mon[28399]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:29.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:28 smithi144 ceph-mon[36389]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:29.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:28 smithi144 ceph-mon[36389]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.572+0000 7ff4f36da700 1 -- 172.21.15.49:0/598024480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 msgr2=0x7ff4ec106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.572+0000 7ff4f36da700 1 --2- 172.21.15.49:0/598024480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 0x7ff4ec106500 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009910 tx=0x7ff4dc009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.572+0000 7ff4f36da700 1 -- 172.21.15.49:0/598024480 shutdown_connections 2024-05-15T15:50:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.572+0000 7ff4f36da700 1 --2- 172.21.15.49:0/598024480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 0x7ff4ec106500 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.572+0000 7ff4f36da700 1 --2- 172.21.15.49:0/598024480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4ec1017e0 0x7ff4ec103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.572+0000 7ff4f36da700 1 -- 172.21.15.49:0/598024480 >> 172.21.15.49:0/598024480 conn(0x7ff4ec0fb1d0 msgr2=0x7ff4ec0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.572+0000 7ff4f36da700 1 -- 172.21.15.49:0/598024480 shutdown_connections 2024-05-15T15:50:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.572+0000 7ff4f36da700 1 -- 172.21.15.49:0/598024480 wait complete. 2024-05-15T15:50:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f36da700 1 Processor -- start 2024-05-15T15:50:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f36da700 1 -- start start 2024-05-15T15:50:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f36da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4ec1017e0 0x7ff4ec1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f36da700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 0x7ff4ec196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f36da700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec197220 con 0x7ff4ec104110 2024-05-15T15:50:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f36da700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec197360 con 0x7ff4ec1017e0 2024-05-15T15:50:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f0c75700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 0x7ff4ec196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f0c75700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 0x7ff4ec196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35034/0 (socket says 172.21.15.49:35034) 2024-05-15T15:50:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f0c75700 1 -- 172.21.15.49:0/2962897278 learned_addr learned my addr 172.21.15.49:0/2962897278 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.573+0000 7ff4f1476700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4ec1017e0 0x7ff4ec1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.574+0000 7ff4f0c75700 1 -- 172.21.15.49:0/2962897278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4ec1017e0 msgr2=0x7ff4ec1966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.574+0000 7ff4f0c75700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4ec1017e0 0x7ff4ec1966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.574+0000 7ff4f0c75700 1 -- 172.21.15.49:0/2962897278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc0095f0 con 0x7ff4ec104110 2024-05-15T15:50:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.574+0000 7ff4f0c75700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 0x7ff4ec196c00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0092e0 tx=0x7ff4dc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.574+0000 7ff4e27fc700 1 -- 172.21.15.49:0/2962897278 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4dc01d070 con 0x7ff4ec104110 2024-05-15T15:50:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.574+0000 7ff4e27fc700 1 -- 172.21.15.49:0/2962897278 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4dc00f460 con 0x7ff4ec104110 2024-05-15T15:50:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.574+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec19bd60 con 0x7ff4ec104110 2024-05-15T15:50:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.574+0000 7ff4e27fc700 1 -- 172.21.15.49:0/2962897278 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4dc021620 con 0x7ff4ec104110 2024-05-15T15:50:30.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.574+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec19c250 con 0x7ff4ec104110 2024-05-15T15:50:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.575+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec04ec60 con 0x7ff4ec104110 2024-05-15T15:50:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.576+0000 7ff4e27fc700 1 -- 172.21.15.49:0/2962897278 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff4dc00f5d0 con 0x7ff4ec104110 2024-05-15T15:50:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.576+0000 7ff4e27fc700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff4d806cce0 0x7ff4d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.576+0000 7ff4e27fc700 1 -- 172.21.15.49:0/2962897278 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff4dc08d060 con 0x7ff4ec104110 2024-05-15T15:50:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.576+0000 7ff4f1476700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff4d806cce0 0x7ff4d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.577+0000 7ff4f1476700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff4d806cce0 0x7ff4d806f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8005950 tx=0x7ff4e800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.581+0000 7ff4e27fc700 1 -- 172.21.15.49:0/2962897278 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff4dc05b280 con 0x7ff4ec104110 2024-05-15T15:50:30.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.793+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4ec105fa0 con 0x7ff4d806cce0 2024-05-15T15:50:30.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.797+0000 7ff4e27fc700 1 -- 172.21.15.49:0/2962897278 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1300 (secure 0 0 0) 0x7ff4ec105fa0 con 0x7ff4d806cce0 2024-05-15T15:50:30.800 INFO:teuthology.orchestra.run.smithi049.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-15T15:50:30.801 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-05-15T15:50:30.801 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter 2/2 2m ago 6m * 2024-05-15T15:50:30.801 INFO:teuthology.orchestra.run.smithi049.stdout:crash 2/2 2m ago 6m * 2024-05-15T15:50:30.801 INFO:teuthology.orchestra.run.smithi049.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-05-15T15:50:30.801 INFO:teuthology.orchestra.run.smithi049.stdout:mgr 2/2 2m ago 6m count:2 2024-05-15T15:50:30.801 INFO:teuthology.orchestra.run.smithi049.stdout:mon 2/2 2m ago 5m smithi049:172.21.15.49=smithi049;smithi144:172.21.15.144=smithi144;count:2 2024-05-15T15:50:30.801 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-05-15T15:50:30.801 INFO:teuthology.orchestra.run.smithi049.stdout:osd 6 2m ago - 2024-05-15T15:50:30.801 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-05-15T15:50:30.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.799+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff4d806cce0 msgr2=0x7ff4d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:30.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.799+0000 7ff4f36da700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff4d806cce0 0x7ff4d806f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8005950 tx=0x7ff4e800b410 comp rx=0 tx=0).stop 2024-05-15T15:50:30.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.799+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 msgr2=0x7ff4ec196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:30.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.799+0000 7ff4f36da700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 0x7ff4ec196c00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0092e0 tx=0x7ff4dc00bd70 comp rx=0 tx=0).stop 2024-05-15T15:50:30.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.800+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 shutdown_connections 2024-05-15T15:50:30.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.800+0000 7ff4f36da700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff4d806cce0 0x7ff4d806f1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:30.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.800+0000 7ff4f36da700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4ec1017e0 0x7ff4ec1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:30.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.800+0000 7ff4f36da700 1 --2- 172.21.15.49:0/2962897278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4ec104110 0x7ff4ec196c00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:30.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.800+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 >> 172.21.15.49:0/2962897278 conn(0x7ff4ec0fb1d0 msgr2=0x7ff4ec0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:30.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.800+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 shutdown_connections 2024-05-15T15:50:30.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:30.800+0000 7ff4f36da700 1 -- 172.21.15.49:0/2962897278 wait complete. 2024-05-15T15:50:31.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:30 smithi049 ceph-mon[28399]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:31.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:30 smithi144 ceph-mon[36389]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:31.499 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-15T15:50:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:32.143 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:31 smithi049 ceph-mon[28399]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:32.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:31 smithi144 ceph-mon[36389]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:33.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:32 smithi049 ceph-mon[28399]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:33.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:32 smithi144 ceph-mon[36389]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:33.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.349+0000 7f9e17a77700 1 -- 172.21.15.49:0/4249137150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10100cf0 msgr2=0x7f9e10101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:33.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.349+0000 7f9e17a77700 1 --2- 172.21.15.49:0/4249137150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10100cf0 0x7f9e10101110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009910 tx=0x7f9e04009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:33.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.349+0000 7f9e17a77700 1 -- 172.21.15.49:0/4249137150 shutdown_connections 2024-05-15T15:50:33.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.349+0000 7f9e17a77700 1 --2- 172.21.15.49:0/4249137150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e10101ef0 0x7f9e10102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:33.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.349+0000 7f9e17a77700 1 --2- 172.21.15.49:0/4249137150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10100cf0 0x7f9e10101110 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:33.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.349+0000 7f9e17a77700 1 -- 172.21.15.49:0/4249137150 >> 172.21.15.49:0/4249137150 conn(0x7f9e100fc290 msgr2=0x7f9e100fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:33.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.349+0000 7f9e17a77700 1 -- 172.21.15.49:0/4249137150 shutdown_connections 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.349+0000 7f9e17a77700 1 -- 172.21.15.49:0/4249137150 wait complete. 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.350+0000 7f9e17a77700 1 Processor -- start 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.350+0000 7f9e17a77700 1 -- start start 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.350+0000 7f9e17a77700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e10100cf0 0x7f9e10074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.350+0000 7f9e17a77700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10101ef0 0x7f9e10073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.350+0000 7f9e17a77700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e100738d0 con 0x7f9e10101ef0 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.350+0000 7f9e17a77700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e10073a10 con 0x7f9e10100cf0 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.350+0000 7f9e15012700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10101ef0 0x7f9e10073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e15012700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10101ef0 0x7f9e10073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35054/0 (socket says 172.21.15.49:35054) 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e15012700 1 -- 172.21.15.49:0/3283030482 learned_addr learned my addr 172.21.15.49:0/3283030482 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e15813700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e10100cf0 0x7f9e10074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e15012700 1 -- 172.21.15.49:0/3283030482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e10100cf0 msgr2=0x7f9e10074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e15012700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e10100cf0 0x7f9e10074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:33.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e15012700 1 -- 172.21.15.49:0/3283030482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040095f0 con 0x7f9e10101ef0 2024-05-15T15:50:33.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e15012700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10101ef0 0x7f9e10073390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00d8d0 tx=0x7f9e0c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:33.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e02ffd700 1 -- 172.21.15.49:0/3283030482 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e0c0098a0 con 0x7f9e10101ef0 2024-05-15T15:50:33.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e10073cf0 con 0x7f9e10101ef0 2024-05-15T15:50:33.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e02ffd700 1 -- 172.21.15.49:0/3283030482 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e0c010460 con 0x7f9e10101ef0 2024-05-15T15:50:33.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e02ffd700 1 -- 172.21.15.49:0/3283030482 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e0c00f5f0 con 0x7f9e10101ef0 2024-05-15T15:50:33.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.351+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1019e790 con 0x7f9e10101ef0 2024-05-15T15:50:33.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.352+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e10190870 con 0x7f9e10101ef0 2024-05-15T15:50:33.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.353+0000 7f9e02ffd700 1 -- 172.21.15.49:0/3283030482 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9e0c009a00 con 0x7f9e10101ef0 2024-05-15T15:50:33.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.353+0000 7f9e02ffd700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9dfc06cce0 0x7f9dfc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:33.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.354+0000 7f9e02ffd700 1 -- 172.21.15.49:0/3283030482 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9e0c08bd30 con 0x7f9e10101ef0 2024-05-15T15:50:33.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.354+0000 7f9e15813700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9dfc06cce0 0x7f9dfc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:33.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.354+0000 7f9e15813700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9dfc06cce0 0x7f9dfc06f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000c00 tx=0x7f9e04005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:33.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.359+0000 7f9e02ffd700 1 -- 172.21.15.49:0/3283030482 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9e0c059f50 con 0x7f9e10101ef0 2024-05-15T15:50:33.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.568+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e1002ce60 con 0x7f9dfc06cce0 2024-05-15T15:50:33.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.569+0000 7f9e02ffd700 1 -- 172.21.15.49:0/3283030482 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f9e1002ce60 con 0x7f9dfc06cce0 2024-05-15T15:50:33.572 INFO:teuthology.orchestra.run.smithi049.stdout:HOST ADDR LABELS STATUS 2024-05-15T15:50:33.572 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 172.21.15.49 2024-05-15T15:50:33.572 INFO:teuthology.orchestra.run.smithi049.stdout:smithi144 172.21.15.144 2024-05-15T15:50:33.572 INFO:teuthology.orchestra.run.smithi049.stdout:2 hosts in cluster 2024-05-15T15:50:33.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.571+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9dfc06cce0 msgr2=0x7f9dfc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:33.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.571+0000 7f9e17a77700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9dfc06cce0 0x7f9dfc06f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000c00 tx=0x7f9e04005960 comp rx=0 tx=0).stop 2024-05-15T15:50:33.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.571+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10101ef0 msgr2=0x7f9e10073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:33.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.571+0000 7f9e17a77700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10101ef0 0x7f9e10073390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00d8d0 tx=0x7f9e0c00dbe0 comp rx=0 tx=0).stop 2024-05-15T15:50:33.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.571+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 shutdown_connections 2024-05-15T15:50:33.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.572+0000 7f9e17a77700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9dfc06cce0 0x7f9dfc06f1a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:33.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.572+0000 7f9e17a77700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e10100cf0 0x7f9e10074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:33.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.572+0000 7f9e17a77700 1 --2- 172.21.15.49:0/3283030482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e10101ef0 0x7f9e10073390 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:33.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.572+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 >> 172.21.15.49:0/3283030482 conn(0x7f9e100fc290 msgr2=0x7f9e10105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:33.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.572+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 shutdown_connections 2024-05-15T15:50:33.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:33.572+0000 7f9e17a77700 1 -- 172.21.15.49:0/3283030482 wait complete. 2024-05-15T15:50:34.220 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-15T15:50:34.432 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:35.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:34 smithi144 ceph-mon[36389]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:35.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:34 smithi144 ceph-mon[36389]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:35.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:34 smithi049 ceph-mon[28399]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:35.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:34 smithi049 ceph-mon[28399]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:36.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.081+0000 7f7f0da7d700 1 -- 172.21.15.49:0/757523280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08101760 msgr2=0x7f7f08103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:36.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.081+0000 7f7f0da7d700 1 --2- 172.21.15.49:0/757523280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08101760 0x7f7f08103b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800b0a0 tx=0x7f7ef800b3b0 comp rx=0 tx=0).stop 2024-05-15T15:50:36.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.081+0000 7f7f0da7d700 1 -- 172.21.15.49:0/757523280 shutdown_connections 2024-05-15T15:50:36.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.081+0000 7f7f0da7d700 1 --2- 172.21.15.49:0/757523280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f08104090 0x7f7f08106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:36.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.081+0000 7f7f0da7d700 1 --2- 172.21.15.49:0/757523280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08101760 0x7f7f08103b50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:36.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.081+0000 7f7f0da7d700 1 -- 172.21.15.49:0/757523280 >> 172.21.15.49:0/757523280 conn(0x7f7f080fb170 msgr2=0x7f7f080fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:36.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.081+0000 7f7f0da7d700 1 -- 172.21.15.49:0/757523280 shutdown_connections 2024-05-15T15:50:36.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.081+0000 7f7f0da7d700 1 -- 172.21.15.49:0/757523280 wait complete. 2024-05-15T15:50:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.082+0000 7f7f0da7d700 1 Processor -- start 2024-05-15T15:50:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.082+0000 7f7f0da7d700 1 -- start start 2024-05-15T15:50:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.082+0000 7f7f0da7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f08101760 0x7f7f08101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.082+0000 7f7f0da7d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08104090 0x7f7f080ff710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.082+0000 7f7f0da7d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819a2e0 con 0x7f7f08104090 2024-05-15T15:50:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.082+0000 7f7f0da7d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819a420 con 0x7f7f08101760 2024-05-15T15:50:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.082+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08104090 0x7f7f080ff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.082+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08104090 0x7f7f080ff710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39996/0 (socket says 172.21.15.49:39996) 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.082+0000 7f7f06ffd700 1 -- 172.21.15.49:0/1070703459 learned_addr learned my addr 172.21.15.49:0/1070703459 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f077fe700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f08101760 0x7f7f08101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f06ffd700 1 -- 172.21.15.49:0/1070703459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f08101760 msgr2=0x7f7f08101070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f06ffd700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f08101760 0x7f7f08101070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f06ffd700 1 -- 172.21.15.49:0/1070703459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef8009d00 con 0x7f7f08104090 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f06ffd700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08104090 0x7f7f080ff710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000c8a0 tx=0x7f7ef000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f04ff9700 1 -- 172.21.15.49:0/1070703459 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef000cea0 con 0x7f7f08104090 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f04ff9700 1 -- 172.21.15.49:0/1070703459 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ef0004830 con 0x7f7f08104090 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f080ffc50 con 0x7f7f08104090 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f04ff9700 1 -- 172.21.15.49:0/1070703459 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef00055b0 con 0x7f7f08104090 2024-05-15T15:50:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.083+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f081001f0 con 0x7f7f08104090 2024-05-15T15:50:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.084+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0804ec60 con 0x7f7f08104090 2024-05-15T15:50:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.085+0000 7f7f04ff9700 1 -- 172.21.15.49:0/1070703459 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7ef00049a0 con 0x7f7f08104090 2024-05-15T15:50:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.085+0000 7f7f04ff9700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7ef406cce0 0x7f7ef406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.085+0000 7f7f077fe700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7ef406cce0 0x7f7ef406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.085+0000 7f7f04ff9700 1 -- 172.21.15.49:0/1070703459 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7ef008bbc0 con 0x7f7f08104090 2024-05-15T15:50:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.086+0000 7f7f077fe700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7ef406cce0 0x7f7ef406f1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009480 tx=0x7f7ef800b7b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.091+0000 7f7f04ff9700 1 -- 172.21.15.49:0/1070703459 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ef0059d60 con 0x7f7f08104090 2024-05-15T15:50:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.300+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f08104a70 con 0x7f7ef406cce0 2024-05-15T15:50:36.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.302+0000 7f7f04ff9700 1 -- 172.21.15.49:0/1070703459 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f7f08104a70 con 0x7f7ef406cce0 2024-05-15T15:50:36.304 INFO:teuthology.orchestra.run.smithi049.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-15T15:50:36.304 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500D2480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-05-15T15:50:36.304 INFO:teuthology.orchestra.run.smithi049.stdout:smithi144 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050085400BGN 372G No 69s ago Has a FileSystem, LVM detected 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7ef406cce0 msgr2=0x7f7ef406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7ef406cce0 0x7f7ef406f1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009480 tx=0x7f7ef800b7b0 comp rx=0 tx=0).stop 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08104090 msgr2=0x7f7f080ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08104090 0x7f7f080ff710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000c8a0 tx=0x7f7ef000cc60 comp rx=0 tx=0).stop 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 shutdown_connections 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7ef406cce0 0x7f7ef406f1a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f08101760 0x7f7f08101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 --2- 172.21.15.49:0/1070703459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f08104090 0x7f7f080ff710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 >> 172.21.15.49:0/1070703459 conn(0x7f7f080fb170 msgr2=0x7f7f080fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 shutdown_connections 2024-05-15T15:50:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:36.304+0000 7f7f0da7d700 1 -- 172.21.15.49:0/1070703459 wait complete. 2024-05-15T15:50:36.901 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:36 smithi049 ceph-mon[28399]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:36.901 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:50:36.925 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T15:50:36.935 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:50:36.935 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-05-15T15:50:37.152 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:37.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:36 smithi144 ceph-mon[36389]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:37.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:50:38.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:37 smithi049 ceph-mon[28399]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:38.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:37 smithi144 ceph-mon[36389]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:38.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.785+0000 7fc8cff47700 1 -- 172.21.15.49:0/360128003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 msgr2=0x7fc8c81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:38.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.785+0000 7fc8cff47700 1 --2- 172.21.15.49:0/360128003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 0x7fc8c81012b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009910 tx=0x7fc8b8009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:38.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.785+0000 7fc8cff47700 1 -- 172.21.15.49:0/360128003 shutdown_connections 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.785+0000 7fc8cff47700 1 --2- 172.21.15.49:0/360128003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 0x7fc8c81012b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.785+0000 7fc8cff47700 1 --2- 172.21.15.49:0/360128003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ffc50 0x7fc8c8100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.785+0000 7fc8cff47700 1 -- 172.21.15.49:0/360128003 >> 172.21.15.49:0/360128003 conn(0x7fc8c80fb1d0 msgr2=0x7fc8c80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.785+0000 7fc8cff47700 1 -- 172.21.15.49:0/360128003 shutdown_connections 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.785+0000 7fc8cff47700 1 -- 172.21.15.49:0/360128003 wait complete. 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cff47700 1 Processor -- start 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cff47700 1 -- start start 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cff47700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ffc50 0x7fc8c8103b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cff47700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 0x7fc8c8102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cff47700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c8102770 con 0x7fc8c8100e50 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cff47700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c81028b0 con 0x7fc8c80ffc50 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cd4e2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 0x7fc8c8102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cd4e2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 0x7fc8c8102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40004/0 (socket says 172.21.15.49:40004) 2024-05-15T15:50:38.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cd4e2700 1 -- 172.21.15.49:0/2490639471 learned_addr learned my addr 172.21.15.49:0/2490639471 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cdce3700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ffc50 0x7fc8c8103b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cd4e2700 1 -- 172.21.15.49:0/2490639471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ffc50 msgr2=0x7fc8c8103b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cd4e2700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ffc50 0x7fc8c8103b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.786+0000 7fc8cd4e2700 1 -- 172.21.15.49:0/2490639471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b80095f0 con 0x7fc8c8100e50 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.787+0000 7fc8cd4e2700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 0x7fc8c8102230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80092e0 tx=0x7fc8b800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.787+0000 7fc8beffd700 1 -- 172.21.15.49:0/2490639471 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8b801d070 con 0x7fc8c8100e50 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.787+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c8102b30 con 0x7fc8c8100e50 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.787+0000 7fc8beffd700 1 -- 172.21.15.49:0/2490639471 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8b800f460 con 0x7fc8c8100e50 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.787+0000 7fc8beffd700 1 -- 172.21.15.49:0/2490639471 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8b8021620 con 0x7fc8c8100e50 2024-05-15T15:50:38.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.787+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c8103020 con 0x7fc8c8100e50 2024-05-15T15:50:38.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.788+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c804ec60 con 0x7fc8c8100e50 2024-05-15T15:50:38.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.788+0000 7fc8beffd700 1 -- 172.21.15.49:0/2490639471 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc8b800f5d0 con 0x7fc8c8100e50 2024-05-15T15:50:38.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.789+0000 7fc8beffd700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc8b406cce0 0x7fc8b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:38.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.789+0000 7fc8beffd700 1 -- 172.21.15.49:0/2490639471 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc8b808d170 con 0x7fc8c8100e50 2024-05-15T15:50:38.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.789+0000 7fc8cdce3700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc8b406cce0 0x7fc8b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:38.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.789+0000 7fc8cdce3700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc8b406cce0 0x7fc8b406f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8100cb0 tx=0x7fc8c400b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:38.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:38.794+0000 7fc8beffd700 1 -- 172.21.15.49:0/2490639471 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc8b805b310 con 0x7fc8c8100e50 2024-05-15T15:50:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:38 smithi049 ceph-mon[28399]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:39.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:39.028+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8c8068e60 con 0x7fc8b406cce0 2024-05-15T15:50:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:38 smithi144 ceph-mon[36389]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:39 smithi144 ceph-mon[36389]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:39 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-15T15:50:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:39 smithi049 ceph-mon[28399]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:50:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:39 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-15T15:50:40.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.873+0000 7fc8beffd700 1 -- 172.21.15.49:0/2490639471 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc8c8068e60 con 0x7fc8b406cce0 2024-05-15T15:50:40.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.876+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc8b406cce0 msgr2=0x7fc8b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:40.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.876+0000 7fc8cff47700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc8b406cce0 0x7fc8b406f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8100cb0 tx=0x7fc8c400b410 comp rx=0 tx=0).stop 2024-05-15T15:50:40.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.876+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 msgr2=0x7fc8c8102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:40.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.876+0000 7fc8cff47700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 0x7fc8c8102230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80092e0 tx=0x7fc8b800bd70 comp rx=0 tx=0).stop 2024-05-15T15:50:40.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.876+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 shutdown_connections 2024-05-15T15:50:40.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.876+0000 7fc8cff47700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc8b406cce0 0x7fc8b406f1a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:40.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.876+0000 7fc8cff47700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ffc50 0x7fc8c8103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:40.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.876+0000 7fc8cff47700 1 --2- 172.21.15.49:0/2490639471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c8100e50 0x7fc8c8102230 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:40.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.876+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 >> 172.21.15.49:0/2490639471 conn(0x7fc8c80fb1d0 msgr2=0x7fc8c8068750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:40.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.877+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 shutdown_connections 2024-05-15T15:50:40.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:40.877+0000 7fc8cff47700 1 -- 172.21.15.49:0/2490639471 wait complete. 2024-05-15T15:50:41.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:40 smithi049 ceph-mon[28399]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:41.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:40 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-15T15:50:41.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:40 smithi049 ceph-mon[28399]: osdmap e42: 6 total, 6 up, 6 in 2024-05-15T15:50:41.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:40 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-15T15:50:41.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:40 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[28395]: 2024-05-15T15:50:40.849+0000 7f08d656b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T15:50:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:40 smithi144 ceph-mon[36389]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:40 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-15T15:50:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:40 smithi144 ceph-mon[36389]: osdmap e42: 6 total, 6 up, 6 in 2024-05-15T15:50:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:40 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-15T15:50:41.596 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: osdmap e43: 6 total, 6 up, 6 in 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: osdmap e44: 6 total, 6 up, 6 in 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: fsmap cephfs:0 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: Saving service mds.cephfs spec with placement count:4 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:50:42.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:42.112 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:50:42.112 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:42.112 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.abbagx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:50:42.112 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.abbagx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T15:50:42.112 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:41 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:42.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-15T15:50:42.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: osdmap e43: 6 total, 6 up, 6 in 2024-05-15T15:50:42.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: osdmap e44: 6 total, 6 up, 6 in 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: fsmap cephfs:0 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: Saving service mds.cephfs spec with placement count:4 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.abbagx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.abbagx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T15:50:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:41 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:42.446 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:43.166 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:42 smithi049 ceph-mon[28399]: Deploying daemon mds.cephfs.smithi049.abbagx on smithi049 2024-05-15T15:50:43.166 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:42 smithi049 ceph-mon[28399]: pgmap v177: 65 pgs: 7 creating+peering, 1 active+clean, 57 unknown; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:43.166 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:42 smithi049 ceph-mon[28399]: osdmap e45: 6 total, 6 up, 6 in 2024-05-15T15:50:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:42 smithi144 ceph-mon[36389]: Deploying daemon mds.cephfs.smithi049.abbagx on smithi049 2024-05-15T15:50:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:42 smithi144 ceph-mon[36389]: pgmap v177: 65 pgs: 7 creating+peering, 1 active+clean, 57 unknown; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:42 smithi144 ceph-mon[36389]: osdmap e45: 6 total, 6 up, 6 in 2024-05-15T15:50:43.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:43 smithi049 ceph-mon[28399]: osdmap e46: 6 total, 6 up, 6 in 2024-05-15T15:50:44.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:43 smithi144 ceph-mon[36389]: osdmap e46: 6 total, 6 up, 6 in 2024-05-15T15:50:44.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.532+0000 7f76d0b3b700 1 -- 172.21.15.49:0/3838850055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc0ffbe0 msgr2=0x7f76cc100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:44.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.532+0000 7f76d0b3b700 1 --2- 172.21.15.49:0/3838850055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc0ffbe0 0x7f76cc100000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009910 tx=0x7f76bc009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.533+0000 7f76d0b3b700 1 -- 172.21.15.49:0/3838850055 shutdown_connections 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.533+0000 7f76d0b3b700 1 --2- 172.21.15.49:0/3838850055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76cc100de0 0x7f76cc101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.533+0000 7f76d0b3b700 1 --2- 172.21.15.49:0/3838850055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc0ffbe0 0x7f76cc100000 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.533+0000 7f76d0b3b700 1 -- 172.21.15.49:0/3838850055 >> 172.21.15.49:0/3838850055 conn(0x7f76cc0fb180 msgr2=0x7f76cc0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.533+0000 7f76d0b3b700 1 -- 172.21.15.49:0/3838850055 shutdown_connections 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.533+0000 7f76d0b3b700 1 -- 172.21.15.49:0/3838850055 wait complete. 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.533+0000 7f76d0b3b700 1 Processor -- start 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76d0b3b700 1 -- start start 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76d0b3b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76cc0ffbe0 0x7f76cc1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76d0b3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc100de0 0x7f76cc196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76d0b3b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc197350 con 0x7f76cc100de0 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76d0b3b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc197490 con 0x7f76cc0ffbe0 2024-05-15T15:50:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76c9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc100de0 0x7f76cc196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:44.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76c9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc100de0 0x7f76cc196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60934/0 (socket says 172.21.15.49:60934) 2024-05-15T15:50:44.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76c9d9b700 1 -- 172.21.15.49:0/1287270773 learned_addr learned my addr 172.21.15.49:0/1287270773 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76ca59c700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76cc0ffbe0 0x7f76cc1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76c9d9b700 1 -- 172.21.15.49:0/1287270773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76cc0ffbe0 msgr2=0x7f76cc1967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76c9d9b700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76cc0ffbe0 0x7f76cc1967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.534+0000 7f76c9d9b700 1 -- 172.21.15.49:0/1287270773 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0095f0 con 0x7f76cc100de0 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.535+0000 7f76c9d9b700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc100de0 0x7f76cc196d30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f76c000b4b0 tx=0x7f76c000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.535+0000 7f76bb7fe700 1 -- 172.21.15.49:0/1287270773 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c0011800 con 0x7f76cc100de0 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.535+0000 7f76bb7fe700 1 -- 172.21.15.49:0/1287270773 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76c0011e20 con 0x7f76cc100de0 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.535+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc19bef0 con 0x7f76cc100de0 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.535+0000 7f76bb7fe700 1 -- 172.21.15.49:0/1287270773 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c000f2e0 con 0x7f76cc100de0 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.535+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc19c490 con 0x7f76cc100de0 2024-05-15T15:50:44.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.536+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76cc04ec60 con 0x7f76cc100de0 2024-05-15T15:50:44.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.537+0000 7f76bb7fe700 1 -- 172.21.15.49:0/1287270773 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f76c0010380 con 0x7f76cc100de0 2024-05-15T15:50:44.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.537+0000 7f76bb7fe700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f76b406cce0 0x7f76b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.537+0000 7f76bb7fe700 1 -- 172.21.15.49:0/1287270773 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f76c008bc60 con 0x7f76cc100de0 2024-05-15T15:50:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.537+0000 7f76ca59c700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f76b406cce0 0x7f76b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.538+0000 7f76ca59c700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f76b406cce0 0x7f76b406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00b5c0 tx=0x7f76bc005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.543+0000 7f76bb7fe700 1 -- 172.21.15.49:0/1287270773 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f76c0059bf0 con 0x7f76cc100de0 2024-05-15T15:50:44.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.794+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f76cc19c760 con 0x7f76cc100de0 2024-05-15T15:50:44.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.794+0000 7f76bb7fe700 1 -- 172.21.15.49:0/1287270773 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1114 (secure 0 0 0) 0x7f76c0015470 con 0x7f76cc100de0 2024-05-15T15:50:44.797 INFO:teuthology.orchestra.run.smithi049.stdout:e2 2024-05-15T15:50:44.797 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:50:44.797 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:50:44.798 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T15:50:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:50:44.799 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T15:50:44.802 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T15:50:44.803 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 2 2024-05-15T15:50:44.803 INFO:teuthology.orchestra.run.smithi049.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-15T15:50:44.803 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T15:50:44.803 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:50:40.850963+0000 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 0 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 1 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:in 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T15:50:44.804 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data disabled 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 0 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:50:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:50:44.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.796+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f76b406cce0 msgr2=0x7f76b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:44.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.796+0000 7f76d0b3b700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f76b406cce0 0x7f76b406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00b5c0 tx=0x7f76bc005960 comp rx=0 tx=0).stop 2024-05-15T15:50:44.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.796+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc100de0 msgr2=0x7f76cc196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:44.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.796+0000 7f76d0b3b700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc100de0 0x7f76cc196d30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f76c000b4b0 tx=0x7f76c000b7c0 comp rx=0 tx=0).stop 2024-05-15T15:50:44.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.796+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 shutdown_connections 2024-05-15T15:50:44.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.796+0000 7f76d0b3b700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f76b406cce0 0x7f76b406f1a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:44.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.796+0000 7f76d0b3b700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76cc0ffbe0 0x7f76cc1967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:44.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.796+0000 7f76d0b3b700 1 --2- 172.21.15.49:0/1287270773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76cc100de0 0x7f76cc196d30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:44.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.796+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 >> 172.21.15.49:0/1287270773 conn(0x7f76cc0fb180 msgr2=0x7f76cc069200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:44.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.797+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 shutdown_connections 2024-05-15T15:50:44.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:44.797+0000 7f76d0b3b700 1 -- 172.21.15.49:0/1287270773 wait complete. 2024-05-15T15:50:44.808 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-05-15T15:50:44.921 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:44 smithi049 ceph-mon[28399]: pgmap v180: 65 pgs: 28 active+clean, 3 creating+activating, 7 creating+peering, 27 unknown; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:44.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:44 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1287270773' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:50:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:44 smithi144 ceph-mon[36389]: pgmap v180: 65 pgs: 28 active+clean, 3 creating+activating, 7 creating+peering, 27 unknown; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:44 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1287270773' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:50:45.622 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T15:50:45.633 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:50:45.633 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-05-15T15:50:45.848 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jgodzs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jgodzs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: Deploying daemon mds.cephfs.smithi144.jgodzs on smithi144 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: daemon mds.cephfs.smithi049.abbagx assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: Cluster is now healthy 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.49:6826/2475999224,v1:172.21.15.49:6827/2475999224] up:boot 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: fsmap cephfs:1 {0=cephfs.smithi049.abbagx=up:creating} 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:50:46.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:46 smithi144 ceph-mon[36389]: daemon mds.cephfs.smithi049.abbagx is now active in filesystem cephfs as rank 0 2024-05-15T15:50:46.312 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:46.312 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:46.312 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:46.312 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jgodzs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:50:46.312 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jgodzs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: Deploying daemon mds.cephfs.smithi144.jgodzs on smithi144 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: daemon mds.cephfs.smithi049.abbagx assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: Cluster is now healthy 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.49:6826/2475999224,v1:172.21.15.49:6827/2475999224] up:boot 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: fsmap cephfs:1 {0=cephfs.smithi049.abbagx=up:creating} 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:50:46.313 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:46 smithi049 ceph-mon[28399]: daemon mds.cephfs.smithi049.abbagx is now active in filesystem cephfs as rank 0 2024-05-15T15:50:47.346 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:47 smithi144 ceph-mon[36389]: pgmap v181: 65 pgs: 62 active+clean, 3 creating+activating; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:47.347 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:47 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.49:6826/2475999224,v1:172.21.15.49:6827/2475999224] up:active 2024-05-15T15:50:47.347 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:47 smithi144 ceph-mon[36389]: fsmap cephfs:1 {0=cephfs.smithi049.abbagx=up:active} 2024-05-15T15:50:47.352 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:47 smithi049 ceph-mon[28399]: pgmap v181: 65 pgs: 62 active+clean, 3 creating+activating; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:47.352 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:47 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.49:6826/2475999224,v1:172.21.15.49:6827/2475999224] up:active 2024-05-15T15:50:47.352 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:47 smithi049 ceph-mon[28399]: fsmap cephfs:1 {0=cephfs.smithi049.abbagx=up:active} 2024-05-15T15:50:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.375+0000 7faf41de3700 1 -- 172.21.15.49:0/1786691074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 msgr2=0x7faf3c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.375+0000 7faf41de3700 1 --2- 172.21.15.49:0/1786691074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 0x7faf3c1012b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf24009910 tx=0x7faf24009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.375+0000 7faf41de3700 1 -- 172.21.15.49:0/1786691074 shutdown_connections 2024-05-15T15:50:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.375+0000 7faf41de3700 1 --2- 172.21.15.49:0/1786691074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 0x7faf3c1012b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.375+0000 7faf41de3700 1 --2- 172.21.15.49:0/1786691074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf3c0ffc50 0x7faf3c100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.375+0000 7faf41de3700 1 -- 172.21.15.49:0/1786691074 >> 172.21.15.49:0/1786691074 conn(0x7faf3c0fb1d0 msgr2=0x7faf3c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.375+0000 7faf41de3700 1 -- 172.21.15.49:0/1786691074 shutdown_connections 2024-05-15T15:50:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.375+0000 7faf41de3700 1 -- 172.21.15.49:0/1786691074 wait complete. 2024-05-15T15:50:47.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.375+0000 7faf41de3700 1 Processor -- start 2024-05-15T15:50:47.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf41de3700 1 -- start start 2024-05-15T15:50:47.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf41de3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf3c0ffc50 0x7faf3c196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:47.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf41de3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 0x7faf3c196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:47.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf41de3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1972c0 con 0x7faf3c100e50 2024-05-15T15:50:47.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf41de3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c197400 con 0x7faf3c0ffc50 2024-05-15T15:50:47.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 0x7faf3c196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:47.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 0x7faf3c196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60958/0 (socket says 172.21.15.49:60958) 2024-05-15T15:50:47.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf32ffd700 1 -- 172.21.15.49:0/785858734 learned_addr learned my addr 172.21.15.49:0/785858734 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:47.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf3b7fe700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf3c0ffc50 0x7faf3c196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:47.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf32ffd700 1 -- 172.21.15.49:0/785858734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf3c0ffc50 msgr2=0x7faf3c196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:47.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf32ffd700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf3c0ffc50 0x7faf3c196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:47.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.376+0000 7faf32ffd700 1 -- 172.21.15.49:0/785858734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240095f0 con 0x7faf3c100e50 2024-05-15T15:50:47.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.377+0000 7faf32ffd700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 0x7faf3c196ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf24006010 tx=0x7faf2400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:47.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.377+0000 7faf397fa700 1 -- 172.21.15.49:0/785858734 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf2401d070 con 0x7faf3c100e50 2024-05-15T15:50:47.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.377+0000 7faf397fa700 1 -- 172.21.15.49:0/785858734 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf2400f460 con 0x7faf3c100e50 2024-05-15T15:50:47.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.377+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c19be00 con 0x7faf3c100e50 2024-05-15T15:50:47.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.377+0000 7faf397fa700 1 -- 172.21.15.49:0/785858734 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf24021640 con 0x7faf3c100e50 2024-05-15T15:50:47.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.377+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c19c2c0 con 0x7faf3c100e50 2024-05-15T15:50:47.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.378+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c04ec60 con 0x7faf3c100e50 2024-05-15T15:50:47.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.378+0000 7faf397fa700 1 -- 172.21.15.49:0/785858734 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7faf24003a60 con 0x7faf3c100e50 2024-05-15T15:50:47.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.379+0000 7faf397fa700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7faf2806cce0 0x7faf2806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:47.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.379+0000 7faf397fa700 1 -- 172.21.15.49:0/785858734 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7faf2408d410 con 0x7faf3c100e50 2024-05-15T15:50:47.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.379+0000 7faf3b7fe700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7faf2806cce0 0x7faf2806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:47.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.379+0000 7faf3b7fe700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7faf2806cce0 0x7faf2806f1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009550 tx=0x7faf2c0091b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:47.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.384+0000 7faf397fa700 1 -- 172.21.15.49:0/785858734 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf2405b270 con 0x7faf3c100e50 2024-05-15T15:50:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:47.634+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7faf3c19c720 con 0x7faf3c100e50 2024-05-15T15:50:48.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.117+0000 7faf397fa700 1 -- 172.21.15.49:0/785858734 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v5) v1 ==== 105+0+0 (secure 0 0 0) 0x7faf2405ae00 con 0x7faf3c100e50 2024-05-15T15:50:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.120+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7faf2806cce0 msgr2=0x7faf2806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.120+0000 7faf41de3700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7faf2806cce0 0x7faf2806f1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009550 tx=0x7faf2c0091b0 comp rx=0 tx=0).stop 2024-05-15T15:50:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.120+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 msgr2=0x7faf3c196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.120+0000 7faf41de3700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 0x7faf3c196ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf24006010 tx=0x7faf2400bd70 comp rx=0 tx=0).stop 2024-05-15T15:50:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.120+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 shutdown_connections 2024-05-15T15:50:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.120+0000 7faf41de3700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7faf2806cce0 0x7faf2806f1a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.120+0000 7faf41de3700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf3c0ffc50 0x7faf3c196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.120+0000 7faf41de3700 1 --2- 172.21.15.49:0/785858734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf3c100e50 0x7faf3c196ca0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.120+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 >> 172.21.15.49:0/785858734 conn(0x7faf3c0fb1d0 msgr2=0x7faf3c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.121+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 shutdown_connections 2024-05-15T15:50:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:48.121+0000 7faf41de3700 1 -- 172.21.15.49:0/785858734 wait complete. 2024-05-15T15:50:48.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:48 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/785858734' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-15T15:50:48.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:48 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:48.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:48 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:48.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:48 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:48.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:48 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.gocadf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:50:48.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:48 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.gocadf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T15:50:48.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:48 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:48.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:48 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/785858734' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-15T15:50:48.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:48 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:48.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:48 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:48.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:48 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:48.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:48 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.gocadf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:50:48.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:48 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.gocadf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T15:50:48.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:48 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:48.803 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T15:50:48.812 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:50:48.813 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-05-15T15:50:49.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: pgmap v182: 65 pgs: 62 active+clean, 3 creating+activating; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:49.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: Deploying daemon mds.cephfs.smithi049.gocadf on smithi049 2024-05-15T15:50:49.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-15T15:50:49.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/785858734' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-15T15:50:49.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] up:boot 2024-05-15T15:50:49.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: daemon mds.cephfs.smithi144.jgodzs assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-15T15:50:49.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-15T15:50:49.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-15T15:50:49.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: fsmap cephfs:1 {0=cephfs.smithi049.abbagx=up:active} 1 up:standby 2024-05-15T15:50:49.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T15:50:49.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:creating} 2024-05-15T15:50:49.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:49 smithi144 ceph-mon[36389]: daemon mds.cephfs.smithi144.jgodzs is now active in filesystem cephfs as rank 1 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: pgmap v182: 65 pgs: 62 active+clean, 3 creating+activating; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: Deploying daemon mds.cephfs.smithi049.gocadf on smithi049 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/785858734' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] up:boot 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: daemon mds.cephfs.smithi144.jgodzs assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: fsmap cephfs:1 {0=cephfs.smithi049.abbagx=up:active} 1 up:standby 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T15:50:49.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:creating} 2024-05-15T15:50:49.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:49 smithi049 ceph-mon[28399]: daemon mds.cephfs.smithi144.jgodzs is now active in filesystem cephfs as rank 1 2024-05-15T15:50:49.978 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:50 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] up:active 2024-05-15T15:50:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:50 smithi049 ceph-mon[28399]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 2024-05-15T15:50:50.318 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:50 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] up:active 2024-05-15T15:50:50.318 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:50 smithi144 ceph-mon[36389]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 2024-05-15T15:50:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:51 smithi144 ceph-mon[36389]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-15T15:50:51.365 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:51 smithi049 ceph-mon[28399]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-15T15:50:51.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.650+0000 7fb45a3da700 1 -- 172.21.15.49:0/110773206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 msgr2=0x7fb4541032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.650+0000 7fb45a3da700 1 --2- 172.21.15.49:0/110773206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 0x7fb4541032c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb444009910 tx=0x7fb444009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.650+0000 7fb45a3da700 1 -- 172.21.15.49:0/110773206 shutdown_connections 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.650+0000 7fb45a3da700 1 --2- 172.21.15.49:0/110773206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb454104090 0x7fb454104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.650+0000 7fb45a3da700 1 --2- 172.21.15.49:0/110773206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 0x7fb4541032c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.650+0000 7fb45a3da700 1 -- 172.21.15.49:0/110773206 >> 172.21.15.49:0/110773206 conn(0x7fb4540fe490 msgr2=0x7fb4541008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.650+0000 7fb45a3da700 1 -- 172.21.15.49:0/110773206 shutdown_connections 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.650+0000 7fb45a3da700 1 -- 172.21.15.49:0/110773206 wait complete. 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb45a3da700 1 Processor -- start 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb45a3da700 1 -- start start 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb45a3da700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 0x7fb4541988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:51.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb45a3da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb454104090 0x7fb454198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb45a3da700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb454199410 con 0x7fb454102ea0 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb45a3da700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb454199550 con 0x7fb454104090 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb453fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 0x7fb4541988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb453fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 0x7fb4541988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60970/0 (socket says 172.21.15.49:60970) 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb453fff700 1 -- 172.21.15.49:0/2549788084 learned_addr learned my addr 172.21.15.49:0/2549788084 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb4537fe700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb454104090 0x7fb454198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb453fff700 1 -- 172.21.15.49:0/2549788084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb454104090 msgr2=0x7fb454198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb453fff700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb454104090 0x7fb454198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.651+0000 7fb453fff700 1 -- 172.21.15.49:0/2549788084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440095f0 con 0x7fb454102ea0 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.652+0000 7fb453fff700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 0x7fb4541988b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb4440059d0 tx=0x7fb44400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.652+0000 7fb4517fa700 1 -- 172.21.15.49:0/2549788084 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb44401d070 con 0x7fb454102ea0 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.652+0000 7fb4517fa700 1 -- 172.21.15.49:0/2549788084 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb444005dc0 con 0x7fb454102ea0 2024-05-15T15:50:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.652+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45419df50 con 0x7fb454102ea0 2024-05-15T15:50:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.652+0000 7fb4517fa700 1 -- 172.21.15.49:0/2549788084 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb444021410 con 0x7fb454102ea0 2024-05-15T15:50:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.652+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45419e440 con 0x7fb454102ea0 2024-05-15T15:50:51.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.653+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4541929f0 con 0x7fb454102ea0 2024-05-15T15:50:51.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.653+0000 7fb4517fa700 1 -- 172.21.15.49:0/2549788084 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb44400f460 con 0x7fb454102ea0 2024-05-15T15:50:51.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.654+0000 7fb4517fa700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb43c06cce0 0x7fb43c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:51.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.654+0000 7fb4517fa700 1 -- 172.21.15.49:0/2549788084 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb44408d460 con 0x7fb454102ea0 2024-05-15T15:50:51.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.654+0000 7fb4537fe700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb43c06cce0 0x7fb43c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:51.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.654+0000 7fb4537fe700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb43c06cce0 0x7fb43c06f1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb454102bd0 tx=0x7fb44800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:51.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.660+0000 7fb4517fa700 1 -- 172.21.15.49:0/2549788084 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb44405b2c0 con 0x7fb454102ea0 2024-05-15T15:50:51.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:51.915+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb45404ec60 con 0x7fb454102ea0 2024-05-15T15:50:52.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.458+0000 7fb4517fa700 1 -- 172.21.15.49:0/2549788084 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v8) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb45404ec60 con 0x7fb454102ea0 2024-05-15T15:50:52.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb43c06cce0 msgr2=0x7fb43c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:52.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb43c06cce0 0x7fb43c06f1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb454102bd0 tx=0x7fb44800b410 comp rx=0 tx=0).stop 2024-05-15T15:50:52.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 msgr2=0x7fb4541988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:52.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 0x7fb4541988b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb4440059d0 tx=0x7fb44400bd70 comp rx=0 tx=0).stop 2024-05-15T15:50:52.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 shutdown_connections 2024-05-15T15:50:52.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb454102ea0 0x7fb4541988b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:52.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb43c06cce0 0x7fb43c06f1a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:52.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 --2- 172.21.15.49:0/2549788084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb454104090 0x7fb454198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:52.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 >> 172.21.15.49:0/2549788084 conn(0x7fb4540fe490 msgr2=0x7fb454107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:52.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 shutdown_connections 2024-05-15T15:50:52.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:52.461+0000 7fb45a3da700 1 -- 172.21.15.49:0/2549788084 wait complete. 2024-05-15T15:50:52.737 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:52.737 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:50:52.737 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 6 op/s 2024-05-15T15:50:52.738 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2549788084' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-15T15:50:52.738 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:52.738 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:52.738 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:52.738 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.kziuke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:50:52.738 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.kziuke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T15:50:52.738 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:52.738 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:52 smithi144 ceph-mon[36389]: Deploying daemon mds.cephfs.smithi144.kziuke on smithi144 2024-05-15T15:50:52.816 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:52.816 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:50:52.816 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 6 op/s 2024-05-15T15:50:52.817 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2549788084' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-15T15:50:52.817 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:52.817 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:52.817 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:52.817 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.kziuke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:50:52.817 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.kziuke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T15:50:52.818 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:50:52.818 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:52 smithi049 ceph-mon[28399]: Deploying daemon mds.cephfs.smithi144.kziuke on smithi144 2024-05-15T15:50:53.071 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T15:50:53.083 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:50:53.083 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-05-15T15:50:53.301 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:53.694 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:53 smithi049 ceph-mon[28399]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-15T15:50:53.694 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:53 smithi049 ceph-mon[28399]: Cluster is now healthy 2024-05-15T15:50:53.694 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:53 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2549788084' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-15T15:50:53.694 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:53 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:boot 2024-05-15T15:50:53.694 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:53 smithi049 ceph-mon[28399]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby 2024-05-15T15:50:53.694 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:53 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T15:50:53.694 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:53 smithi049 ceph-mon[28399]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 2024-05-15T15:50:53.815 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:53 smithi144 ceph-mon[36389]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-15T15:50:53.816 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:53 smithi144 ceph-mon[36389]: Cluster is now healthy 2024-05-15T15:50:53.816 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:53 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2549788084' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-15T15:50:53.816 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:53 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:boot 2024-05-15T15:50:53.816 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:53 smithi144 ceph-mon[36389]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby 2024-05-15T15:50:53.816 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:53 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T15:50:53.817 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:53 smithi144 ceph-mon[36389]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 2024-05-15T15:50:54.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:54 smithi049 ceph-mon[28399]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 771 B/s rd, 2.4 KiB/s wr, 8 op/s 2024-05-15T15:50:54.736 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:54 smithi144 ceph-mon[36389]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 771 B/s rd, 2.4 KiB/s wr, 8 op/s 2024-05-15T15:50:54.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.987+0000 7f9db5d3b700 1 -- 172.21.15.49:0/2494747018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db0101fb0 msgr2=0x7f9db0102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.987+0000 7f9db5d3b700 1 --2- 172.21.15.49:0/2494747018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db0101fb0 0x7f9db0102410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009910 tx=0x7f9da0009c20 comp rx=0 tx=0).stop 2024-05-15T15:50:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.987+0000 7f9db5d3b700 1 -- 172.21.15.49:0/2494747018 shutdown_connections 2024-05-15T15:50:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.987+0000 7f9db5d3b700 1 --2- 172.21.15.49:0/2494747018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db0101fb0 0x7f9db0102410 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.987+0000 7f9db5d3b700 1 --2- 172.21.15.49:0/2494747018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9db00fefd0 0x7f9db00ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.987+0000 7f9db5d3b700 1 -- 172.21.15.49:0/2494747018 >> 172.21.15.49:0/2494747018 conn(0x7f9db00fa950 msgr2=0x7f9db00fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.987+0000 7f9db5d3b700 1 -- 172.21.15.49:0/2494747018 shutdown_connections 2024-05-15T15:50:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.987+0000 7f9db5d3b700 1 -- 172.21.15.49:0/2494747018 wait complete. 2024-05-15T15:50:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.987+0000 7f9db5d3b700 1 Processor -- start 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9db5d3b700 1 -- start start 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9db5d3b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9db00fefd0 0x7f9db0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9db5d3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db0101fb0 0x7f9db0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9db5d3b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0199460 con 0x7f9db0101fb0 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9db5d3b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db01995a0 con 0x7f9db00fefd0 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9daeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db0101fb0 0x7f9db0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9daf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9db00fefd0 0x7f9db0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9daeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db0101fb0 0x7f9db0198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45286/0 (socket says 172.21.15.49:45286) 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9daeffd700 1 -- 172.21.15.49:0/662642737 learned_addr learned my addr 172.21.15.49:0/662642737 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:50:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9daf7fe700 1 -- 172.21.15.49:0/662642737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db0101fb0 msgr2=0x7f9db0198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9daf7fe700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db0101fb0 0x7f9db0198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9daf7fe700 1 -- 172.21.15.49:0/662642737 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da00095f0 con 0x7f9db00fefd0 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.988+0000 7f9daf7fe700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9db00fefd0 0x7f9db0198900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800d6c0 tx=0x7f9d9800d9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.989+0000 7f9dacff9700 1 -- 172.21.15.49:0/662642737 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d980041f0 con 0x7f9db00fefd0 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.989+0000 7f9dacff9700 1 -- 172.21.15.49:0/662642737 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d98004350 con 0x7f9db00fefd0 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.989+0000 7f9dacff9700 1 -- 172.21.15.49:0/662642737 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d98003d20 con 0x7f9db00fefd0 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.989+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db019dfa0 con 0x7f9db00fefd0 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.989+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db019e540 con 0x7f9db00fefd0 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.990+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db004ec60 con 0x7f9db00fefd0 2024-05-15T15:50:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.990+0000 7f9dacff9700 1 -- 172.21.15.49:0/662642737 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9d9800f690 con 0x7f9db00fefd0 2024-05-15T15:50:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.991+0000 7f9dacff9700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9d9c06cce0 0x7f9d9c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:50:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.991+0000 7f9dacff9700 1 -- 172.21.15.49:0/662642737 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f9d9808bc50 con 0x7f9db00fefd0 2024-05-15T15:50:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.991+0000 7f9daeffd700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9d9c06cce0 0x7f9d9c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:50:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.991+0000 7f9daeffd700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9d9c06cce0 0x7f9d9c06f1a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9da00051c0 tx=0x7f9da000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:50:54.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:54.996+0000 7f9dacff9700 1 -- 172.21.15.49:0/662642737 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d98059ad0 con 0x7f9db00fefd0 2024-05-15T15:50:55.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.249+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9db0109870 con 0x7f9db00fefd0 2024-05-15T15:50:55.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.466+0000 7f9dacff9700 1 -- 172.21.15.49:0/662642737 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v10) v1 ==== 168+0+0 (secure 0 0 0) 0x7f9d9801f0d0 con 0x7f9db00fefd0 2024-05-15T15:50:55.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.469+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9d9c06cce0 msgr2=0x7f9d9c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:55.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.469+0000 7f9db5d3b700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9d9c06cce0 0x7f9d9c06f1a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9da00051c0 tx=0x7f9da000b540 comp rx=0 tx=0).stop 2024-05-15T15:50:55.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.469+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9db00fefd0 msgr2=0x7f9db0198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:50:55.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.469+0000 7f9db5d3b700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9db00fefd0 0x7f9db0198900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800d6c0 tx=0x7f9d9800d9d0 comp rx=0 tx=0).stop 2024-05-15T15:50:55.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.470+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 shutdown_connections 2024-05-15T15:50:55.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.470+0000 7f9db5d3b700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9d9c06cce0 0x7f9d9c06f1a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:55.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.470+0000 7f9db5d3b700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9db00fefd0 0x7f9db0198900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:55.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.470+0000 7f9db5d3b700 1 --2- 172.21.15.49:0/662642737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9db0101fb0 0x7f9db0198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:50:55.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.470+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 >> 172.21.15.49:0/662642737 conn(0x7f9db00fa950 msgr2=0x7f9db01000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:50:55.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.470+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 shutdown_connections 2024-05-15T15:50:55.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:50:55.470+0000 7f9db5d3b700 1 -- 172.21.15.49:0/662642737 wait complete. 2024-05-15T15:50:55.472 INFO:teuthology.orchestra.run.smithi049.stderr:inline data enabled 2024-05-15T15:50:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/662642737' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] up:boot 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 1 up:standby 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T15:50:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:55 smithi144 ceph-mon[36389]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:50:56.179 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/662642737' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] up:boot 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 1 up:standby 2024-05-15T15:50:56.180 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T15:50:56.181 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:55 smithi049 ceph-mon[28399]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:50:56.891 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T15:50:56.902 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:50:56.902 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-15T15:50:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:56 smithi144 ceph-mon[36389]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-15T15:50:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:56 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:57.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:56 smithi049 ceph-mon[28399]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-15T15:50:57.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:56 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:57.762 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:50:58.198 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:standby-replay 2024-05-15T15:50:58.198 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] up:active 2024-05-15T15:50:58.198 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: Dropping low affinity active daemon mds.cephfs.smithi049.abbagx in favor of higher affinity standby. 2024-05-15T15:50:58.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: Replacing daemon mds.cephfs.smithi049.abbagx as rank 0 with standby daemon mds.cephfs.smithi049.gocadf 2024-05-15T15:50:58.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-15T15:50:58.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:50:58.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: osdmap e47: 6 total, 6 up, 6 in 2024-05-15T15:50:58.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: fsmap cephfs:2/2 {0=cephfs.smithi049.gocadf=up:replay,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 2024-05-15T15:50:58.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:58.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:57 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:standby-replay 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] up:active 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: Dropping low affinity active daemon mds.cephfs.smithi049.abbagx in favor of higher affinity standby. 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: Replacing daemon mds.cephfs.smithi049.abbagx as rank 0 with standby daemon mds.cephfs.smithi049.gocadf 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: osdmap e47: 6 total, 6 up, 6 in 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: fsmap cephfs:2/2 {0=cephfs.smithi049.gocadf=up:replay,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:57 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:50:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:58 smithi144 ceph-mon[36389]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 921 B/s rd, 2.6 KiB/s wr, 9 op/s 2024-05-15T15:50:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:58 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] up:boot 2024-05-15T15:50:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:58 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:resolve 2024-05-15T15:50:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:58 smithi144 ceph-mon[36389]: fsmap cephfs:2/2 {0=cephfs.smithi049.gocadf=up:resolve,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 1 up:standby 2024-05-15T15:50:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:58 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:50:59.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:58 smithi049 ceph-mon[28399]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 921 B/s rd, 2.6 KiB/s wr, 9 op/s 2024-05-15T15:50:59.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:58 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] up:boot 2024-05-15T15:50:59.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:58 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:resolve 2024-05-15T15:50:59.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:58 smithi049 ceph-mon[28399]: fsmap cephfs:2/2 {0=cephfs.smithi049.gocadf=up:resolve,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 1 up:standby 2024-05-15T15:50:59.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:58 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:51:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.085+0000 7fe1d1d02700 1 -- 172.21.15.49:0/1636385783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc0ffc50 msgr2=0x7fe1cc100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.085+0000 7fe1d1d02700 1 --2- 172.21.15.49:0/1636385783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc0ffc50 0x7fe1cc100070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009910 tx=0x7fe1bc009c20 comp rx=0 tx=0).stop 2024-05-15T15:51:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.085+0000 7fe1d1d02700 1 -- 172.21.15.49:0/1636385783 shutdown_connections 2024-05-15T15:51:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.085+0000 7fe1d1d02700 1 --2- 172.21.15.49:0/1636385783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc100e50 0x7fe1cc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.085+0000 7fe1d1d02700 1 --2- 172.21.15.49:0/1636385783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc0ffc50 0x7fe1cc100070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:00.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.085+0000 7fe1d1d02700 1 -- 172.21.15.49:0/1636385783 >> 172.21.15.49:0/1636385783 conn(0x7fe1cc0fb1d0 msgr2=0x7fe1cc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:00.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.085+0000 7fe1d1d02700 1 -- 172.21.15.49:0/1636385783 shutdown_connections 2024-05-15T15:51:00.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.085+0000 7fe1d1d02700 1 -- 172.21.15.49:0/1636385783 wait complete. 2024-05-15T15:51:00.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1d1d02700 1 Processor -- start 2024-05-15T15:51:00.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1d1d02700 1 -- start start 2024-05-15T15:51:00.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1d1d02700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc0ffc50 0x7fe1cc1967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:00.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1d1d02700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc100e50 0x7fe1cc196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:00.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1d1d02700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc197340 con 0x7fe1cc100e50 2024-05-15T15:51:00.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1d1d02700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc197480 con 0x7fe1cc0ffc50 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1c2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc100e50 0x7fe1cc196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1c2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc100e50 0x7fe1cc196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45336/0 (socket says 172.21.15.49:45336) 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1c2ffd700 1 -- 172.21.15.49:0/473761138 learned_addr learned my addr 172.21.15.49:0/473761138 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.086+0000 7fe1cb7fe700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc0ffc50 0x7fe1cc1967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.087+0000 7fe1c2ffd700 1 -- 172.21.15.49:0/473761138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc0ffc50 msgr2=0x7fe1cc1967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.087+0000 7fe1c2ffd700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc0ffc50 0x7fe1cc1967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.087+0000 7fe1c2ffd700 1 -- 172.21.15.49:0/473761138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0095f0 con 0x7fe1cc100e50 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.087+0000 7fe1c2ffd700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc100e50 0x7fe1cc196d20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400d8d0 tx=0x7fe1b400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.087+0000 7fe1c97fa700 1 -- 172.21.15.49:0/473761138 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b40098a0 con 0x7fe1cc100e50 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.087+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc19be90 con 0x7fe1cc100e50 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.087+0000 7fe1c97fa700 1 -- 172.21.15.49:0/473761138 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1b4010460 con 0x7fe1cc100e50 2024-05-15T15:51:00.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.087+0000 7fe1c97fa700 1 -- 172.21.15.49:0/473761138 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b400f5f0 con 0x7fe1cc100e50 2024-05-15T15:51:00.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.087+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc19c3b0 con 0x7fe1cc100e50 2024-05-15T15:51:00.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.088+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc04ec60 con 0x7fe1cc100e50 2024-05-15T15:51:00.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.089+0000 7fe1c97fa700 1 -- 172.21.15.49:0/473761138 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe1b4009a00 con 0x7fe1cc100e50 2024-05-15T15:51:00.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.089+0000 7fe1c97fa700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe1b806cce0 0x7fe1b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:00.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.089+0000 7fe1c97fa700 1 -- 172.21.15.49:0/473761138 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe1b408c170 con 0x7fe1cc100e50 2024-05-15T15:51:00.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.089+0000 7fe1cb7fe700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe1b806cce0 0x7fe1b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:00.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.090+0000 7fe1cb7fe700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe1b806cce0 0x7fe1b806f1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc000c00 tx=0x7fe1bc005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:00.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.094+0000 7fe1c97fa700 1 -- 172.21.15.49:0/473761138 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe1b4059f70 con 0x7fe1cc100e50 2024-05-15T15:51:00.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:59 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:reconnect 2024-05-15T15:51:00.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:59 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] up:standby-replay 2024-05-15T15:51:00.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:50:59 smithi144 ceph-mon[36389]: fsmap cephfs:2/2 {0=cephfs.smithi049.gocadf=up:reconnect,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 1 up:standby 2024-05-15T15:51:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:59 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:reconnect 2024-05-15T15:51:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:59 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] up:standby-replay 2024-05-15T15:51:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:50:59 smithi049 ceph-mon[28399]: fsmap cephfs:2/2 {0=cephfs.smithi049.gocadf=up:reconnect,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 1 up:standby 2024-05-15T15:51:00.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.360+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe1cc1087b0 con 0x7fe1cc100e50 2024-05-15T15:51:00.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.361+0000 7fe1c97fa700 1 -- 172.21.15.49:0/473761138 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1884 (secure 0 0 0) 0x7fe1b4059b00 con 0x7fe1cc100e50 2024-05-15T15:51:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:e16 2024-05-15T15:51:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:51:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:51:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T15:51:00.363 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:51:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T15:51:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T15:51:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 16 2024-05-15T15:51:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:50:59.902062+0000 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T15:51:00.364 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:rejoin seq 5 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:51:00.365 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:51:00.366 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:51:00.366 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:51:00.366 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:51:00.366 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T15:51:00.366 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:51:00.366 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:14494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:51:00.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.363+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe1b806cce0 msgr2=0x7fe1b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:00.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.363+0000 7fe1d1d02700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe1b806cce0 0x7fe1b806f1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc000c00 tx=0x7fe1bc005960 comp rx=0 tx=0).stop 2024-05-15T15:51:00.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.363+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc100e50 msgr2=0x7fe1cc196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:00.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.363+0000 7fe1d1d02700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc100e50 0x7fe1cc196d20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400d8d0 tx=0x7fe1b400dbe0 comp rx=0 tx=0).stop 2024-05-15T15:51:00.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.364+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 shutdown_connections 2024-05-15T15:51:00.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.364+0000 7fe1d1d02700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe1b806cce0 0x7fe1b806f1a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:00.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.364+0000 7fe1d1d02700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc0ffc50 0x7fe1cc1967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:00.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.364+0000 7fe1d1d02700 1 --2- 172.21.15.49:0/473761138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1cc100e50 0x7fe1cc196d20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:00.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.364+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 >> 172.21.15.49:0/473761138 conn(0x7fe1cc0fb1d0 msgr2=0x7fe1cc068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:00.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.364+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 shutdown_connections 2024-05-15T15:51:00.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:00.364+0000 7fe1d1d02700 1 -- 172.21.15.49:0/473761138 wait complete. 2024-05-15T15:51:00.371 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 16 2024-05-15T15:51:00.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:00 smithi049 ceph-mon[28399]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-15T15:51:00.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:00 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:rejoin 2024-05-15T15:51:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:00 smithi049 ceph-mon[28399]: fsmap cephfs:2/2 {0=cephfs.smithi049.gocadf=up:rejoin,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 1 up:standby 2024-05-15T15:51:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:00 smithi049 ceph-mon[28399]: daemon mds.cephfs.smithi049.gocadf is now active in filesystem cephfs as rank 0 2024-05-15T15:51:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:00 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/473761138' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:51:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:00 smithi049 ceph-mon[28399]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-15T15:51:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:00 smithi049 ceph-mon[28399]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:active 2024-05-15T15:51:00.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:00 smithi049 ceph-mon[28399]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:51:01.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-05-15T15:51:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:00 smithi144 ceph-mon[36389]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-15T15:51:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:00 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:rejoin 2024-05-15T15:51:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:00 smithi144 ceph-mon[36389]: fsmap cephfs:2/2 {0=cephfs.smithi049.gocadf=up:rejoin,1=cephfs.smithi144.jgodzs=up:active} 1 up:standby-replay 1 up:standby 2024-05-15T15:51:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:00 smithi144 ceph-mon[36389]: daemon mds.cephfs.smithi049.gocadf is now active in filesystem cephfs as rank 0 2024-05-15T15:51:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:00 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/473761138' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:51:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:00 smithi144 ceph-mon[36389]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-15T15:51:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:00 smithi144 ceph-mon[36389]: mds.? [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] up:active 2024-05-15T15:51:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:00 smithi144 ceph-mon[36389]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:51:01.427 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:51:03.155 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:02 smithi049 ceph-mon[28399]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.9 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-15T15:51:03.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:02 smithi144 ceph-mon[36389]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.9 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-15T15:51:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.676+0000 7fa0902c9700 1 -- 172.21.15.49:0/3173563646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 msgr2=0x7fa0881010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.676+0000 7fa0902c9700 1 --2- 172.21.15.49:0/3173563646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 0x7fa0881010f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa084009910 tx=0x7fa084009c20 comp rx=0 tx=0).stop 2024-05-15T15:51:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.676+0000 7fa0902c9700 1 -- 172.21.15.49:0/3173563646 shutdown_connections 2024-05-15T15:51:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.676+0000 7fa0902c9700 1 --2- 172.21.15.49:0/3173563646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa088101ed0 0x7fa088102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.676+0000 7fa0902c9700 1 --2- 172.21.15.49:0/3173563646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 0x7fa0881010f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.676+0000 7fa0902c9700 1 -- 172.21.15.49:0/3173563646 >> 172.21.15.49:0/3173563646 conn(0x7fa0880fc2b0 msgr2=0x7fa0880fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.676+0000 7fa0902c9700 1 -- 172.21.15.49:0/3173563646 shutdown_connections 2024-05-15T15:51:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.676+0000 7fa0902c9700 1 -- 172.21.15.49:0/3173563646 wait complete. 2024-05-15T15:51:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.677+0000 7fa0902c9700 1 Processor -- start 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.677+0000 7fa0902c9700 1 -- start start 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.677+0000 7fa0902c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 0x7fa0881966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.677+0000 7fa0902c9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa088101ed0 0x7fa088196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.677+0000 7fa0902c9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa088197250 con 0x7fa088100cd0 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.677+0000 7fa0902c9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa088197390 con 0x7fa088101ed0 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.677+0000 7fa08e065700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 0x7fa0881966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.677+0000 7fa08e065700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 0x7fa0881966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42476/0 (socket says 172.21.15.49:42476) 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.677+0000 7fa08e065700 1 -- 172.21.15.49:0/4242264623 learned_addr learned my addr 172.21.15.49:0/4242264623 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa08d864700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa088101ed0 0x7fa088196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa08d864700 1 -- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 msgr2=0x7fa0881966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa08d864700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 0x7fa0881966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa08d864700 1 -- 172.21.15.49:0/4242264623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0840095f0 con 0x7fa088101ed0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa08e065700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 0x7fa0881966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa08d864700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa088101ed0 0x7fa088196c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa07800e970 tx=0x7fa07800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa07f7fe700 1 -- 172.21.15.49:0/4242264623 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa07800c9d0 con 0x7fa088101ed0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08819bdf0 con 0x7fa088101ed0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa07f7fe700 1 -- 172.21.15.49:0/4242264623 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa078010430 con 0x7fa088101ed0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa07f7fe700 1 -- 172.21.15.49:0/4242264623 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa078018630 con 0x7fa088101ed0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.678+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08819c390 con 0x7fa088101ed0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.679+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa088190850 con 0x7fa088101ed0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.680+0000 7fa07f7fe700 1 -- 172.21.15.49:0/4242264623 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa0780105a0 con 0x7fa088101ed0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.680+0000 7fa07f7fe700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fa07406ccf0 0x7fa07406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.681+0000 7fa07f7fe700 1 -- 172.21.15.49:0/4242264623 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa078014070 con 0x7fa088101ed0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.681+0000 7fa08e065700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fa07406ccf0 0x7fa07406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.681+0000 7fa08e065700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fa07406ccf0 0x7fa07406f1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa08400b800 tx=0x7fa084009f70 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:03.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.685+0000 7fa07f7fe700 1 -- 172.21.15.49:0/4242264623 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa078052d60 con 0x7fa088101ed0 2024-05-15T15:51:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.935+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa08804ecb0 con 0x7fa088101ed0 2024-05-15T15:51:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.936+0000 7fa07f7fe700 1 -- 172.21.15.49:0/4242264623 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4862 (secure 0 0 0) 0x7fa07805a2f0 con 0x7fa088101ed0 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fa07406ccf0 msgr2=0x7fa07406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fa07406ccf0 0x7fa07406f1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa08400b800 tx=0x7fa084009f70 comp rx=0 tx=0).stop 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa088101ed0 msgr2=0x7fa088196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa088101ed0 0x7fa088196c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa07800e970 tx=0x7fa07800ec80 comp rx=0 tx=0).stop 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 shutdown_connections 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa088100cd0 0x7fa0881966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fa07406ccf0 0x7fa07406f1b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 --2- 172.21.15.49:0/4242264623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa088101ed0 0x7fa088196c30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 >> 172.21.15.49:0/4242264623 conn(0x7fa0880fc2b0 msgr2=0x7fa088105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 shutdown_connections 2024-05-15T15:51:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:03.939+0000 7fa0902c9700 1 -- 172.21.15.49:0/4242264623 wait complete. 2024-05-15T15:51:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T15:51:03.954 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:51:04.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:04 smithi049 ceph-mon[28399]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-15T15:51:04.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:04.555 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:04.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:51:04.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:51:04.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:04.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:04 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:51:04.556 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:04 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/4242264623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T15:51:04.577 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-05-15T15:51:04.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:04 smithi144 ceph-mon[36389]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-15T15:51:04.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:04.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:04.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:51:04.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:51:04.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:04.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:04 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:51:04.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:04 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/4242264623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T15:51:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:51:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:06 smithi049 ceph-mon[28399]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-15T15:51:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:06.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:06.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:51:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:06 smithi144 ceph-mon[36389]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-15T15:51:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:51:07.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.665+0000 7fe776046700 1 -- 172.21.15.49:0/2729130695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 msgr2=0x7fe770103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:07.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.665+0000 7fe776046700 1 --2- 172.21.15.49:0/2729130695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 0x7fe770103b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe760009910 tx=0x7fe760009c20 comp rx=0 tx=0).stop 2024-05-15T15:51:07.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.665+0000 7fe776046700 1 -- 172.21.15.49:0/2729130695 shutdown_connections 2024-05-15T15:51:07.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.665+0000 7fe776046700 1 --2- 172.21.15.49:0/2729130695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7701040c0 0x7fe7701064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:07.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.665+0000 7fe776046700 1 --2- 172.21.15.49:0/2729130695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 0x7fe770103b80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:07.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.665+0000 7fe776046700 1 -- 172.21.15.49:0/2729130695 >> 172.21.15.49:0/2729130695 conn(0x7fe7700fb1a0 msgr2=0x7fe7700fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:07.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.665+0000 7fe776046700 1 -- 172.21.15.49:0/2729130695 shutdown_connections 2024-05-15T15:51:07.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.665+0000 7fe776046700 1 -- 172.21.15.49:0/2729130695 wait complete. 2024-05-15T15:51:07.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.665+0000 7fe776046700 1 Processor -- start 2024-05-15T15:51:07.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe776046700 1 -- start start 2024-05-15T15:51:07.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe776046700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 0x7fe7701010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:07.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe776046700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7701040c0 0x7fe7700ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:07.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe776046700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700ffc80 con 0x7fe770101790 2024-05-15T15:51:07.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe776046700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700ffdc0 con 0x7fe7701040c0 2024-05-15T15:51:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe76f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 0x7fe7701010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe76f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 0x7fe7701010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42500/0 (socket says 172.21.15.49:42500) 2024-05-15T15:51:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe76f7fe700 1 -- 172.21.15.49:0/1136782166 learned_addr learned my addr 172.21.15.49:0/1136782166 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe76effd700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7701040c0 0x7fe7700ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe76f7fe700 1 -- 172.21.15.49:0/1136782166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7701040c0 msgr2=0x7fe7700ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe76f7fe700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7701040c0 0x7fe7700ff740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe76f7fe700 1 -- 172.21.15.49:0/1136782166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600095f0 con 0x7fe770101790 2024-05-15T15:51:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe76effd700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7701040c0 0x7fe7700ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:51:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.666+0000 7fe76f7fe700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 0x7fe7701010a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7600092e0 tx=0x7fe76000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.667+0000 7fe76cff9700 1 -- 172.21.15.49:0/1136782166 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76001d070 con 0x7fe770101790 2024-05-15T15:51:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.667+0000 7fe76cff9700 1 -- 172.21.15.49:0/1136782166 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76000f460 con 0x7fe770101790 2024-05-15T15:51:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.667+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe770100040 con 0x7fe770101790 2024-05-15T15:51:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.667+0000 7fe76cff9700 1 -- 172.21.15.49:0/1136782166 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe760021640 con 0x7fe770101790 2024-05-15T15:51:07.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.667+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7701005b0 con 0x7fe770101790 2024-05-15T15:51:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.668+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77004ec60 con 0x7fe770101790 2024-05-15T15:51:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.668+0000 7fe76cff9700 1 -- 172.21.15.49:0/1136782166 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe760003a60 con 0x7fe770101790 2024-05-15T15:51:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.669+0000 7fe76cff9700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe75806cce0 0x7fe75806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.669+0000 7fe76cff9700 1 -- 172.21.15.49:0/1136782166 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe76008d190 con 0x7fe770101790 2024-05-15T15:51:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.669+0000 7fe76effd700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe75806cce0 0x7fe75806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.669+0000 7fe76effd700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe75806cce0 0x7fe75806f1a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe764005fd0 tx=0x7fe764005ee0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.674+0000 7fe76cff9700 1 -- 172.21.15.49:0/1136782166 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe76005b0c0 con 0x7fe770101790 2024-05-15T15:51:07.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.984+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fe7701005b0 con 0x7fe770101790 2024-05-15T15:51:07.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.984+0000 7fe76cff9700 1 -- 172.21.15.49:0/1136782166 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+98 (secure 0 0 0) 0x7fe760017400 con 0x7fe770101790 2024-05-15T15:51:07.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe75806cce0 msgr2=0x7fe75806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:07.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe75806cce0 0x7fe75806f1a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe764005fd0 tx=0x7fe764005ee0 comp rx=0 tx=0).stop 2024-05-15T15:51:07.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 msgr2=0x7fe7701010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:07.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 0x7fe7701010a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7600092e0 tx=0x7fe76000be20 comp rx=0 tx=0).stop 2024-05-15T15:51:07.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 shutdown_connections 2024-05-15T15:51:07.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe770101790 0x7fe7701010a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:07.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe75806cce0 0x7fe75806f1a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:07.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 --2- 172.21.15.49:0/1136782166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7701040c0 0x7fe7700ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:07.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 >> 172.21.15.49:0/1136782166 conn(0x7fe7700fb1a0 msgr2=0x7fe7700fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:07.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 shutdown_connections 2024-05-15T15:51:07.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:07.986+0000 7fe776046700 1 -- 172.21.15.49:0/1136782166 wait complete. 2024-05-15T15:51:07.999 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:51:08.643 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-05-15T15:51:08.658 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-15T15:51:08.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 -- 172.21.15.49:0/4133573549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 msgr2=0x7f9b34101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:08.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 --2- 172.21.15.49:0/4133573549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 0x7f9b34101840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009880 tx=0x7f9b1c02bce0 comp rx=0 tx=0).stop 2024-05-15T15:51:08.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 -- 172.21.15.49:0/4133573549 shutdown_connections 2024-05-15T15:51:08.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 --2- 172.21.15.49:0/4133573549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 0x7f9b34101840 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:08.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 --2- 172.21.15.49:0/4133573549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b341001d0 0x7f9b341005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 -- 172.21.15.49:0/4133573549 >> 172.21.15.49:0/4133573549 conn(0x7f9b340fb740 msgr2=0x7f9b340fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 -- 172.21.15.49:0/4133573549 shutdown_connections 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 -- 172.21.15.49:0/4133573549 wait complete. 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 Processor -- start 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.795+0000 7f9b3b10a640 1 -- start start 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b3b10a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b341001d0 0x7f9b34195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b3b10a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 0x7f9b34196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b3b10a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b34196740 con 0x7f9b341013c0 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b3b10a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b341968b0 con 0x7f9b341001d0 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 0x7f9b34196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 0x7f9b34196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42528/0 (socket says 172.21.15.49:42528) 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b33fff640 1 -- 172.21.15.49:0/2666633750 learned_addr learned my addr 172.21.15.49:0/2666633750 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:08.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b38ea6640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b341001d0 0x7f9b34195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b33fff640 1 -- 172.21.15.49:0/2666633750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b341001d0 msgr2=0x7f9b34195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b33fff640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b341001d0 0x7f9b34195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b33fff640 1 -- 172.21.15.49:0/2666633750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c009560 con 0x7f9b341013c0 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b38ea6640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b341001d0 0x7f9b34195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b33fff640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 0x7f9b34196200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c036040 tx=0x7f9b1c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b31ffb640 1 -- 172.21.15.49:0/2666633750 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b1c0037c0 con 0x7f9b341013c0 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3419b290 con 0x7f9b341013c0 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b31ffb640 1 -- 172.21.15.49:0/2666633750 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b1c003de0 con 0x7f9b341013c0 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b31ffb640 1 -- 172.21.15.49:0/2666633750 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b1c034440 con 0x7f9b341013c0 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.796+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3419b810 con 0x7f9b341013c0 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.797+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3418fd40 con 0x7f9b341013c0 2024-05-15T15:51:08.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.798+0000 7f9b31ffb640 1 -- 172.21.15.49:0/2666633750 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9b1c005060 con 0x7f9b341013c0 2024-05-15T15:51:08.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.798+0000 7f9b31ffb640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9b2006cc50 0x7f9b2006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:08.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.798+0000 7f9b31ffb640 1 -- 172.21.15.49:0/2666633750 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9b1c0ae7c0 con 0x7f9b341013c0 2024-05-15T15:51:08.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.798+0000 7f9b38ea6640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9b2006cc50 0x7f9b2006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:08.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.799+0000 7f9b38ea6640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9b2006cc50 0x7f9b2006f110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9b34197140 tx=0x7f9b24005dd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:08.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.803+0000 7f9b31ffb640 1 -- 172.21.15.49:0/2666633750 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9b1c078b30 con 0x7f9b341013c0 2024-05-15T15:51:08.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:08 smithi049 ceph-mon[28399]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 9.9 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-15T15:51:08.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:08 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1136782166' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-15T15:51:08.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.988+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b34108db0 con 0x7f9b341013c0 2024-05-15T15:51:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.989+0000 7f9b31ffb640 1 -- 172.21.15.49:0/2666633750 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15124 (secure 0 0 0) 0x7f9b1c07c150 con 0x7f9b341013c0 2024-05-15T15:51:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:51:08.992 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":47,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","created":"2024-05-15T15:43:35.648208+0000","modified":"2024-05-15T15:50:56.872370+0000","last_up_change":"2024-05-15T15:49:24.492659+0000","last_in_change":"2024-05-15T15:49:04.722203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T15:48:09.899234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-15T15:50:39.462287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-15T15:50:40.285269+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25d1eb06-bee2-48a6-8d86-30b5b3b348d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6803","nonce":941734370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6805","nonce":941734370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6809","nonce":941734370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":941734370},{"type":"v1","addr":"172.21.15.49:6807","nonce":941734370}]},"public_addr":"172.21.15.49:6803/941734370","cluster_addr":"172.21.15.49:6805/941734370","heartbeat_back_addr":"172.21.15.49:6809/941734370","heartbeat_front_addr":"172.21.15.49:6807/941734370","state":["exists","up"]},{"osd":1,"uuid":"c0e717cf-ae5c-4fa4-a30d-d2235ff27315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6811","nonce":717954912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6813","nonce":717954912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6817","nonce":717954912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":717954912},{"type":"v1","addr":"172.21.15.49:6815","nonce":717954912}]},"public_addr":"172.21.15.49:6811/717954912","cluster_addr":"172.21.15.49:6813/717954912","heartbeat_back_addr":"172.21.15.49:6817/717954912","heartbeat_front_addr":"172.21.15.49:6815/717954912","state":["exists","up"]},{"osd":2,"uuid":"d7752daf-dd5b-49cc-abc2-ca953968e542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6819","nonce":2295619409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6821","nonce":2295619409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6825","nonce":2295619409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2295619409},{"type":"v1","addr":"172.21.15.49:6823","nonce":2295619409}]},"public_addr":"172.21.15.49:6819/2295619409","cluster_addr":"172.21.15.49:6821/2295619409","heartbeat_back_addr":"172.21.15.49:6825/2295619409","heartbeat_front_addr":"172.21.15.49:6823/2295619409","state":["exists","up"]},{"osd":3,"uuid":"2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6801","nonce":1650511241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6803","nonce":1650511241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6807","nonce":1650511241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1650511241},{"type":"v1","addr":"172.21.15.144:6805","nonce":1650511241}]},"public_addr":"172.21.15.144:6801/1650511241","cluster_addr":"172.21.15.144:6803/1650511241","heartbeat_back_addr":"172.21.15.144:6807/1650511241","heartbeat_front_addr":"172.21.15.144:6805/1650511241","state":["exists","up"]},{"osd":4,"uuid":"6659631f-da39-4a2c-9f75-e7b4a06b602f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6809","nonce":4251193981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6811","nonce":4251193981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6815","nonce":4251193981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4251193981},{"type":"v1","addr":"172.21.15.144:6813","nonce":4251193981}]},"public_addr":"172.21.15.144:6809/4251193981","cluster_addr":"172.21.15.144:6811/4251193981","heartbeat_back_addr":"172.21.15.144:6815/4251193981","heartbeat_front_addr":"172.21.15.144:6813/4251193981","state":["exists","up"]},{"osd":5,"uuid":"51ea2d85-6cf7-4a96-8c74-612958083a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6817","nonce":1897088829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6819","nonce":1897088829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6823","nonce":1897088829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1897088829},{"type":"v1","addr":"172.21.15.144:6821","nonce":1897088829}]},"public_addr":"172.21.15.144:6817/1897088829","cluster_addr":"172.21.15.144:6819/1897088829","heartbeat_back_addr":"172.21.15.144:6823/1897088829","heartbeat_front_addr":"172.21.15.144:6821/1897088829","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:10.108689+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:37.458152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:05.915664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:30.194669+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:55.748301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:49:22.635948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6827/2475999224":"2024-05-16T15:50:56.872336+0000","172.21.15.49:6800/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6801/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6801/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/940476764":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/2163344179":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/4010845594":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/3452717560":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/75769186":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/3014161738":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/4024404689":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/1173274751":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6800/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6800/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6801/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/644218315":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6826/2475999224":"2024-05-16T15:50:56.872336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9b2006cc50 msgr2=0x7f9b2006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9b2006cc50 0x7f9b2006f110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9b34197140 tx=0x7f9b24005dd0 comp rx=0 tx=0).stop 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 msgr2=0x7f9b34196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 0x7f9b34196200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c036040 tx=0x7f9b1c0040c0 comp rx=0 tx=0).stop 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 shutdown_connections 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f9b2006cc50 0x7f9b2006f110 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b341013c0 0x7f9b34196200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 --2- 172.21.15.49:0/2666633750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b341001d0 0x7f9b34195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 >> 172.21.15.49:0/2666633750 conn(0x7f9b340fb740 msgr2=0x7f9b34104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 shutdown_connections 2024-05-15T15:51:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:08.991+0000 7f9b3b10a640 1 -- 172.21.15.49:0/2666633750 wait complete. 2024-05-15T15:51:09.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-15T15:48:09.899234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-15T15:50:39.462287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-15T15:50:40.285269+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-15T15:51:09.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-15T15:51:09.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.117+0000 7f7d49c51640 1 -- 172.21.15.49:0/2213937680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d44100ca0 msgr2=0x7f7d44103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.117+0000 7f7d49c51640 1 --2- 172.21.15.49:0/2213937680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d44100ca0 0x7f7d44103090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009880 tx=0x7f7d3402bce0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.117+0000 7f7d49c51640 1 -- 172.21.15.49:0/2213937680 shutdown_connections 2024-05-15T15:51:09.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.117+0000 7f7d49c51640 1 --2- 172.21.15.49:0/2213937680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d441035d0 0x7f7d441059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.117+0000 7f7d49c51640 1 --2- 172.21.15.49:0/2213937680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d44100ca0 0x7f7d44103090 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.117+0000 7f7d49c51640 1 -- 172.21.15.49:0/2213937680 >> 172.21.15.49:0/2213937680 conn(0x7f7d440fa650 msgr2=0x7f7d440fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:09.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.117+0000 7f7d49c51640 1 -- 172.21.15.49:0/2213937680 shutdown_connections 2024-05-15T15:51:09.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.118+0000 7f7d49c51640 1 -- 172.21.15.49:0/2213937680 wait complete. 2024-05-15T15:51:09.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.118+0000 7f7d49c51640 1 Processor -- start 2024-05-15T15:51:09.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.118+0000 7f7d49c51640 1 -- start start 2024-05-15T15:51:09.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d49c51640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d44100ca0 0x7f7d441005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:09.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d49c51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d441035d0 0x7f7d440fec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:09.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d49c51640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440ff190 con 0x7f7d441035d0 2024-05-15T15:51:09.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d49c51640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440ff300 con 0x7f7d44100ca0 2024-05-15T15:51:09.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d441035d0 0x7f7d440fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:09.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d441035d0 0x7f7d440fec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42542/0 (socket says 172.21.15.49:42542) 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d42ffd640 1 -- 172.21.15.49:0/987995270 learned_addr learned my addr 172.21.15.49:0/987995270 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d437fe640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d44100ca0 0x7f7d441005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d42ffd640 1 -- 172.21.15.49:0/987995270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d44100ca0 msgr2=0x7f7d441005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d42ffd640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d44100ca0 0x7f7d441005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d42ffd640 1 -- 172.21.15.49:0/987995270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d34009560 con 0x7f7d441035d0 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d437fe640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d44100ca0 0x7f7d441005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.119+0000 7f7d42ffd640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d441035d0 0x7f7d440fec50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800d6c0 tx=0x7f7d3800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.120+0000 7f7d40ff9640 1 -- 172.21.15.49:0/987995270 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d380051f0 con 0x7f7d441035d0 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.120+0000 7f7d40ff9640 1 -- 172.21.15.49:0/987995270 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d38009ce0 con 0x7f7d441035d0 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.120+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d440ff5e0 con 0x7f7d441035d0 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.120+0000 7f7d40ff9640 1 -- 172.21.15.49:0/987995270 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d38010460 con 0x7f7d441035d0 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.120+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4419dd00 con 0x7f7d441035d0 2024-05-15T15:51:09.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.121+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4404e210 con 0x7f7d441035d0 2024-05-15T15:51:09.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.121+0000 7f7d40ff9640 1 -- 172.21.15.49:0/987995270 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7d38005350 con 0x7f7d441035d0 2024-05-15T15:51:09.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.122+0000 7f7d40ff9640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7d2c06cc70 0x7f7d2c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:09.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.122+0000 7f7d40ff9640 1 -- 172.21.15.49:0/987995270 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7d3808bda0 con 0x7f7d441035d0 2024-05-15T15:51:09.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.122+0000 7f7d437fe640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7d2c06cc70 0x7f7d2c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:09.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.122+0000 7f7d437fe640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7d2c06cc70 0x7f7d2c06f130 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7d34036040 tx=0x7f7d3402cf90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:09.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.127+0000 7f7d40ff9640 1 -- 172.21.15.49:0/987995270 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7d38056110 con 0x7f7d441035d0 2024-05-15T15:51:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:08 smithi144 ceph-mon[36389]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 9.9 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-15T15:51:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:08 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1136782166' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-15T15:51:09.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.308+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d44108f40 con 0x7f7d441035d0 2024-05-15T15:51:09.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.309+0000 7f7d40ff9640 1 -- 172.21.15.49:0/987995270 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d38059730 con 0x7f7d441035d0 2024-05-15T15:51:09.311 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-05-15T15:51:09.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7d2c06cc70 msgr2=0x7f7d2c06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7d2c06cc70 0x7f7d2c06f130 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7d34036040 tx=0x7f7d3402cf90 comp rx=0 tx=0).stop 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d441035d0 msgr2=0x7f7d440fec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d441035d0 0x7f7d440fec50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800d6c0 tx=0x7f7d3800db80 comp rx=0 tx=0).stop 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 shutdown_connections 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7d2c06cc70 0x7f7d2c06f130 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d441035d0 0x7f7d440fec50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 --2- 172.21.15.49:0/987995270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d44100ca0 0x7f7d441005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 >> 172.21.15.49:0/987995270 conn(0x7f7d440fa650 msgr2=0x7f7d440fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 shutdown_connections 2024-05-15T15:51:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.311+0000 7f7d49c51640 1 -- 172.21.15.49:0/987995270 wait complete. 2024-05-15T15:51:09.325 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-15T15:51:09.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.437+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/63473304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc100c30 msgr2=0x7f3fbc103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.437+0000 7f3fc0dd7640 1 --2- 172.21.15.49:0/63473304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc100c30 0x7f3fbc103020 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009c50 tx=0x7f3fac02c190 comp rx=0 tx=0).stop 2024-05-15T15:51:09.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.437+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/63473304 shutdown_connections 2024-05-15T15:51:09.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.437+0000 7f3fc0dd7640 1 --2- 172.21.15.49:0/63473304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fbc103560 0x7f3fbc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.437+0000 7f3fc0dd7640 1 --2- 172.21.15.49:0/63473304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc100c30 0x7f3fbc103020 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.437+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/63473304 >> 172.21.15.49:0/63473304 conn(0x7f3fbc0fa620 msgr2=0x7f3fbc0fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:09.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.437+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/63473304 shutdown_connections 2024-05-15T15:51:09.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.437+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/63473304 wait complete. 2024-05-15T15:51:09.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fc0dd7640 1 Processor -- start 2024-05-15T15:51:09.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fc0dd7640 1 -- start start 2024-05-15T15:51:09.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fc0dd7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fbc100c30 0x7f3fbc100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:09.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fc0dd7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc103560 0x7f3fbc0febe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:09.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fc0dd7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc100ad0 con 0x7f3fbc103560 2024-05-15T15:51:09.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fc0dd7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0ff150 con 0x7f3fbc100c30 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fb9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc103560 0x7f3fbc0febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fb9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc103560 0x7f3fbc0febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42574/0 (socket says 172.21.15.49:42574) 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fb9d9b640 1 -- 172.21.15.49:0/1591776687 learned_addr learned my addr 172.21.15.49:0/1591776687 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fba59c640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fbc100c30 0x7f3fbc100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fb9d9b640 1 -- 172.21.15.49:0/1591776687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fbc100c30 msgr2=0x7f3fbc100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fb9d9b640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fbc100c30 0x7f3fbc100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fb9d9b640 1 -- 172.21.15.49:0/1591776687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac009900 con 0x7f3fbc103560 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fba59c640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fbc100c30 0x7f3fbc100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.438+0000 7f3fb9d9b640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc103560 0x7f3fbc0febe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400d6c0 tx=0x7f3fa400db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.439+0000 7f3fb37fe640 1 -- 172.21.15.49:0/1591776687 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fa40051f0 con 0x7f3fbc103560 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.439+0000 7f3fb37fe640 1 -- 172.21.15.49:0/1591776687 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fa4009ce0 con 0x7f3fbc103560 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.439+0000 7f3fb37fe640 1 -- 172.21.15.49:0/1591776687 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fa4010460 con 0x7f3fbc103560 2024-05-15T15:51:09.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.439+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc0ff430 con 0x7f3fbc103560 2024-05-15T15:51:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.439+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc0ff990 con 0x7f3fbc103560 2024-05-15T15:51:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.440+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc108dd0 con 0x7f3fbc103560 2024-05-15T15:51:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.440+0000 7f3fb37fe640 1 -- 172.21.15.49:0/1591776687 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3fa4005350 con 0x7f3fbc103560 2024-05-15T15:51:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.441+0000 7f3fb37fe640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3fa806cc50 0x7f3fa806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.441+0000 7f3fb37fe640 1 -- 172.21.15.49:0/1591776687 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3fa408bda0 con 0x7f3fbc103560 2024-05-15T15:51:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.441+0000 7f3fba59c640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3fa806cc50 0x7f3fa806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.441+0000 7f3fba59c640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3fa806cc50 0x7f3fa806f110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3fac000f80 tx=0x7f3fac032780 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:09.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.450+0000 7f3fb37fe640 1 -- 172.21.15.49:0/1591776687 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3fa4056110 con 0x7f3fbc103560 2024-05-15T15:51:09.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.636+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f3fbc108f60 con 0x7f3fbc103560 2024-05-15T15:51:09.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.636+0000 7f3fb37fe640 1 -- 172.21.15.49:0/1591776687 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3fa4059730 con 0x7f3fbc103560 2024-05-15T15:51:09.639 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-15T15:51:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3fa806cc50 msgr2=0x7f3fa806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3fa806cc50 0x7f3fa806f110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3fac000f80 tx=0x7f3fac032780 comp rx=0 tx=0).stop 2024-05-15T15:51:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc103560 msgr2=0x7f3fbc0febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc103560 0x7f3fbc0febe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400d6c0 tx=0x7f3fa400db80 comp rx=0 tx=0).stop 2024-05-15T15:51:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 shutdown_connections 2024-05-15T15:51:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f3fa806cc50 0x7f3fa806f110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fbc103560 0x7f3fbc0febe0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 --2- 172.21.15.49:0/1591776687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fbc100c30 0x7f3fbc100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 >> 172.21.15.49:0/1591776687 conn(0x7f3fbc0fa620 msgr2=0x7f3fbc0fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:09.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 shutdown_connections 2024-05-15T15:51:09.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.638+0000 7f3fc0dd7640 1 -- 172.21.15.49:0/1591776687 wait complete. 2024-05-15T15:51:09.652 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-15T15:51:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.756+0000 7ff3f73f2640 1 -- 172.21.15.49:0/3800105964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f00ff0c0 msgr2=0x7ff3f00ff4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.756+0000 7ff3f73f2640 1 --2- 172.21.15.49:0/3800105964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f00ff0c0 0x7ff3f00ff4e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009880 tx=0x7ff3e402bce0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.756+0000 7ff3f73f2640 1 -- 172.21.15.49:0/3800105964 shutdown_connections 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.756+0000 7ff3f73f2640 1 --2- 172.21.15.49:0/3800105964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3f01002b0 0x7ff3f0100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.756+0000 7ff3f73f2640 1 --2- 172.21.15.49:0/3800105964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f00ff0c0 0x7ff3f00ff4e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.756+0000 7ff3f73f2640 1 -- 172.21.15.49:0/3800105964 >> 172.21.15.49:0/3800105964 conn(0x7ff3f00fa630 msgr2=0x7ff3f00fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.756+0000 7ff3f73f2640 1 -- 172.21.15.49:0/3800105964 shutdown_connections 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.756+0000 7ff3f73f2640 1 -- 172.21.15.49:0/3800105964 wait complete. 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f73f2640 1 Processor -- start 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f73f2640 1 -- start start 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f73f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3f00ff0c0 0x7ff3f0069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f73f2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f01002b0 0x7ff3f019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f73f2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f019b030 con 0x7ff3f01002b0 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f73f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f019b1a0 con 0x7ff3f00ff0c0 2024-05-15T15:51:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f498d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f01002b0 0x7ff3f019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f498d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f01002b0 0x7ff3f019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42592/0 (socket says 172.21.15.49:42592) 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f498d640 1 -- 172.21.15.49:0/555435402 learned_addr learned my addr 172.21.15.49:0/555435402 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f518e640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3f00ff0c0 0x7ff3f0069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f498d640 1 -- 172.21.15.49:0/555435402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3f00ff0c0 msgr2=0x7ff3f0069150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f498d640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3f00ff0c0 0x7ff3f0069150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.757+0000 7ff3f498d640 1 -- 172.21.15.49:0/555435402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e4009560 con 0x7ff3f01002b0 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.758+0000 7ff3f518e640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3f00ff0c0 0x7ff3f0069150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.758+0000 7ff3f498d640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f01002b0 0x7ff3f019aaf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00d6f0 tx=0x7ff3ec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.758+0000 7ff3e27fc640 1 -- 172.21.15.49:0/555435402 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3ec009a30 con 0x7ff3f01002b0 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.758+0000 7ff3e27fc640 1 -- 172.21.15.49:0/555435402 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3ec0054e0 con 0x7ff3f01002b0 2024-05-15T15:51:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.758+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f019b480 con 0x7ff3f01002b0 2024-05-15T15:51:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.758+0000 7ff3e27fc640 1 -- 172.21.15.49:0/555435402 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3ec010460 con 0x7ff3f01002b0 2024-05-15T15:51:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.758+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f019ba60 con 0x7ff3f01002b0 2024-05-15T15:51:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.759+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f018ec50 con 0x7ff3f01002b0 2024-05-15T15:51:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.759+0000 7ff3e27fc640 1 -- 172.21.15.49:0/555435402 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff3ec009b90 con 0x7ff3f01002b0 2024-05-15T15:51:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.760+0000 7ff3e27fc640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff3dc06cc50 0x7ff3dc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.760+0000 7ff3e27fc640 1 -- 172.21.15.49:0/555435402 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff3ec08be00 con 0x7ff3f01002b0 2024-05-15T15:51:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.760+0000 7ff3f518e640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff3dc06cc50 0x7ff3dc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.760+0000 7ff3f518e640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff3dc06cc50 0x7ff3dc06f110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4036040 tx=0x7ff3e4000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.765+0000 7ff3e27fc640 1 -- 172.21.15.49:0/555435402 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff3ec056160 con 0x7ff3f01002b0 2024-05-15T15:51:09.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.949+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff3f0107cc0 con 0x7ff3f01002b0 2024-05-15T15:51:09.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.950+0000 7ff3e27fc640 1 -- 172.21.15.49:0/555435402 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff3ec0200b0 con 0x7ff3f01002b0 2024-05-15T15:51:09.952 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff3dc06cc50 msgr2=0x7ff3dc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff3dc06cc50 0x7ff3dc06f110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4036040 tx=0x7ff3e4000bc0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f01002b0 msgr2=0x7ff3f019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f01002b0 0x7ff3f019aaf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00d6f0 tx=0x7ff3ec00dbb0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 shutdown_connections 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff3dc06cc50 0x7ff3dc06f110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3f01002b0 0x7ff3f019aaf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 --2- 172.21.15.49:0/555435402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3f00ff0c0 0x7ff3f0069150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 >> 172.21.15.49:0/555435402 conn(0x7ff3f00fa630 msgr2=0x7ff3f0103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 shutdown_connections 2024-05-15T15:51:09.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:09.952+0000 7ff3f73f2640 1 -- 172.21.15.49:0/555435402 wait complete. 2024-05-15T15:51:09.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:09 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2666633750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T15:51:09.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:09 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/987995270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T15:51:09.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:09 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1591776687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-15T15:51:09.970 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-15T15:51:10.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.074+0000 7f13eca46640 1 -- 172.21.15.49:0/39616767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81001b0 msgr2=0x7f13e81005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:10.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.074+0000 7f13eca46640 1 --2- 172.21.15.49:0/39616767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81001b0 0x7f13e81005d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009880 tx=0x7f13d802bce0 comp rx=0 tx=0).stop 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.074+0000 7f13eca46640 1 -- 172.21.15.49:0/39616767 shutdown_connections 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.074+0000 7f13eca46640 1 --2- 172.21.15.49:0/39616767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13e81013a0 0x7f13e8101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.074+0000 7f13eca46640 1 --2- 172.21.15.49:0/39616767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81001b0 0x7f13e81005d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.074+0000 7f13eca46640 1 -- 172.21.15.49:0/39616767 >> 172.21.15.49:0/39616767 conn(0x7f13e80fb740 msgr2=0x7f13e80fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.074+0000 7f13eca46640 1 -- 172.21.15.49:0/39616767 shutdown_connections 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.074+0000 7f13eca46640 1 -- 172.21.15.49:0/39616767 wait complete. 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13eca46640 1 Processor -- start 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13eca46640 1 -- start start 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13eca46640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13e81001b0 0x7f13e80741c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13eca46640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81013a0 0x7f13e80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:10.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13eca46640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8074790 con 0x7f13e81013a0 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13eca46640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8072e10 con 0x7f13e81001b0 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13e5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81013a0 0x7f13e80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13e5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81013a0 0x7f13e80728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42616/0 (socket says 172.21.15.49:42616) 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13e5d9b640 1 -- 172.21.15.49:0/3134743370 learned_addr learned my addr 172.21.15.49:0/3134743370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13e659c640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13e81001b0 0x7f13e80741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13e5d9b640 1 -- 172.21.15.49:0/3134743370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13e81001b0 msgr2=0x7f13e80741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13e5d9b640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13e81001b0 0x7f13e80741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13e5d9b640 1 -- 172.21.15.49:0/3134743370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8009560 con 0x7f13e81013a0 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13e5d9b640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81013a0 0x7f13e80728a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00d6f0 tx=0x7f13dc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13d77fe640 1 -- 172.21.15.49:0/3134743370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13dc009a30 con 0x7f13e81013a0 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.075+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e80730f0 con 0x7f13e81013a0 2024-05-15T15:51:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.076+0000 7f13d77fe640 1 -- 172.21.15.49:0/3134743370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13dc0054e0 con 0x7f13e81013a0 2024-05-15T15:51:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.076+0000 7f13d77fe640 1 -- 172.21.15.49:0/3134743370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13dc010460 con 0x7f13e81013a0 2024-05-15T15:51:10.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.076+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e80736d0 con 0x7f13e81013a0 2024-05-15T15:51:10.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.076+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e818fd40 con 0x7f13e81013a0 2024-05-15T15:51:10.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.077+0000 7f13d77fe640 1 -- 172.21.15.49:0/3134743370 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f13dc009b90 con 0x7f13e81013a0 2024-05-15T15:51:10.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.078+0000 7f13d77fe640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f13d006cc50 0x7f13d006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:10.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.078+0000 7f13d77fe640 1 -- 172.21.15.49:0/3134743370 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f13dc08bcc0 con 0x7f13e81013a0 2024-05-15T15:51:10.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.078+0000 7f13e659c640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f13d006cc50 0x7f13d006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:10.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.078+0000 7f13e659c640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f13d006cc50 0x7f13d006f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f13d802c960 tx=0x7f13d8005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.083+0000 7f13d77fe640 1 -- 172.21.15.49:0/3134743370 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f13dc056160 con 0x7f13e81013a0 2024-05-15T15:51:10.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:09 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2666633750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T15:51:10.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:09 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/987995270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T15:51:10.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:09 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1591776687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-15T15:51:10.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.311+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13e802c480 con 0x7f13e81013a0 2024-05-15T15:51:10.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.312+0000 7f13d77fe640 1 -- 172.21.15.49:0/3134743370 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4862 (secure 0 0 0) 0x7f13dc017090 con 0x7f13e81013a0 2024-05-15T15:51:10.314 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:51:10.315 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T15:51:00.559034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14484,"mds_1":24303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi049.gocadf","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6829/2512292397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2512292397},{"type":"v1","addr":"172.21.15.49:6829","nonce":2512292397}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14494":{"gid":14494,"name":"cephfs.smithi049.abbagx","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6827/3374515140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":3374515140},{"type":"v1","addr":"172.21.15.49:6827","nonce":3374515140}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24303":{"gid":24303,"name":"cephfs.smithi144.jgodzs","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6825/2552333341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2552333341},{"type":"v1","addr":"172.21.15.144:6825","nonce":2552333341}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24313":{"gid":24313,"name":"cephfs.smithi144.kziuke","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.144:6827/2463281667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2463281667},{"type":"v1","addr":"172.21.15.144:6827","nonce":2463281667}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-15T15:51:10.315 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T15:51:10.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f13d006cc50 msgr2=0x7f13d006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f13d006cc50 0x7f13d006f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f13d802c960 tx=0x7f13d8005fd0 comp rx=0 tx=0).stop 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81013a0 msgr2=0x7f13e80728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81013a0 0x7f13e80728a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00d6f0 tx=0x7f13dc00dbb0 comp rx=0 tx=0).stop 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 shutdown_connections 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f13d006cc50 0x7f13d006f110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13e81013a0 0x7f13e80728a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 --2- 172.21.15.49:0/3134743370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13e81001b0 0x7f13e80741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 >> 172.21.15.49:0/3134743370 conn(0x7f13e80fb740 msgr2=0x7f13e81049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 shutdown_connections 2024-05-15T15:51:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:10.314+0000 7f13eca46640 1 -- 172.21.15.49:0/3134743370 wait complete. 2024-05-15T15:51:10.328 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 50} 2024-05-15T15:51:10.328 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-15T15:51:10.395 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-15T15:51:10.396 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-05-15T15:51:10.396 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-15T15:51:10.396 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-15T15:51:10.396 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:10.396 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-15T15:51:10.396 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-15T15:51:10.396 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:10.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:10.397 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-05-15T15:51:10.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:10.418 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-05-15T15:51:10.493 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2024-05-15T15:51:10.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T15:51:10.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:10.496 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-15T15:51:10.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:10.514 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-05-15T15:51:10.583 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-05-15T15:51:10.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T15:51:10.584 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-15T15:51:10.584 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-15T15:51:10.879 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:51:10.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:10 smithi049 ceph-mon[28399]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 9.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-15T15:51:10.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:10 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/555435402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-15T15:51:10.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:10 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3134743370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T15:51:11.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:10 smithi144 ceph-mon[36389]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 9.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-15T15:51:11.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:10 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/555435402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-15T15:51:11.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:10 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3134743370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T15:51:12.288 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:12.288 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:12.288 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:51:12.288 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:51:12.288 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:12.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:12.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:12.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:51:12.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:51:12.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:51:12.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.511+0000 7ff69bfa9700 1 -- 172.21.15.49:0/3721463898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff694105c90 msgr2=0x7ff694106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:12.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.511+0000 7ff69bfa9700 1 --2- 172.21.15.49:0/3721463898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff694105c90 0x7ff694106070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff684009910 tx=0x7ff684009c20 comp rx=0 tx=0).stop 2024-05-15T15:51:12.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.511+0000 7ff69bfa9700 1 -- 172.21.15.49:0/3721463898 shutdown_connections 2024-05-15T15:51:12.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.511+0000 7ff69bfa9700 1 --2- 172.21.15.49:0/3721463898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6940ff490 0x7ff6940ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:12.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.511+0000 7ff69bfa9700 1 --2- 172.21.15.49:0/3721463898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff694105c90 0x7ff694106070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:12.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.511+0000 7ff69bfa9700 1 -- 172.21.15.49:0/3721463898 >> 172.21.15.49:0/3721463898 conn(0x7ff694074e70 msgr2=0x7ff694075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:12.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.511+0000 7ff69bfa9700 1 -- 172.21.15.49:0/3721463898 shutdown_connections 2024-05-15T15:51:12.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.511+0000 7ff69bfa9700 1 -- 172.21.15.49:0/3721463898 wait complete. 2024-05-15T15:51:12.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff69bfa9700 1 Processor -- start 2024-05-15T15:51:12.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff69bfa9700 1 -- start start 2024-05-15T15:51:12.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff69bfa9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6940ff490 0x7ff69419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:12.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff69bfa9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105c90 0x7ff69419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:12.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff69bfa9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69419d360 con 0x7ff6940ff490 2024-05-15T15:51:12.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff69bfa9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6941969d0 con 0x7ff694105c90 2024-05-15T15:51:12.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff699d45700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6940ff490 0x7ff69419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:12.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff699d45700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6940ff490 0x7ff69419c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42636/0 (socket says 172.21.15.49:42636) 2024-05-15T15:51:12.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff699d45700 1 -- 172.21.15.49:0/2590172573 learned_addr learned my addr 172.21.15.49:0/2590172573 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:12.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.512+0000 7ff699544700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105c90 0x7ff69419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:12.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.513+0000 7ff699d45700 1 -- 172.21.15.49:0/2590172573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105c90 msgr2=0x7ff69419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:12.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.513+0000 7ff699d45700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105c90 0x7ff69419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:12.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.513+0000 7ff699d45700 1 -- 172.21.15.49:0/2590172573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840095f0 con 0x7ff6940ff490 2024-05-15T15:51:12.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.513+0000 7ff699d45700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6940ff490 0x7ff69419c890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff68400bd70 tx=0x7ff68400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:12.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.513+0000 7ff692ffd700 1 -- 172.21.15.49:0/2590172573 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff68401d070 con 0x7ff6940ff490 2024-05-15T15:51:12.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.513+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff694196c50 con 0x7ff6940ff490 2024-05-15T15:51:12.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.513+0000 7ff692ffd700 1 -- 172.21.15.49:0/2590172573 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6840056e0 con 0x7ff6940ff490 2024-05-15T15:51:12.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.513+0000 7ff692ffd700 1 -- 172.21.15.49:0/2590172573 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff68400fe80 con 0x7ff6940ff490 2024-05-15T15:51:12.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.513+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6941971c0 con 0x7ff6940ff490 2024-05-15T15:51:12.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.514+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff694109290 con 0x7ff6940ff490 2024-05-15T15:51:12.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.515+0000 7ff692ffd700 1 -- 172.21.15.49:0/2590172573 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff68400f680 con 0x7ff6940ff490 2024-05-15T15:51:12.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.515+0000 7ff692ffd700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff67c06cce0 0x7ff67c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:12.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.515+0000 7ff699544700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff67c06cce0 0x7ff67c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:12.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.515+0000 7ff692ffd700 1 -- 172.21.15.49:0/2590172573 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff68408d660 con 0x7ff6940ff490 2024-05-15T15:51:12.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.516+0000 7ff699544700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff67c06cce0 0x7ff67c06f1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff694197b90 tx=0x7ff68800a380 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:12.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.521+0000 7ff692ffd700 1 -- 172.21.15.49:0/2590172573 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff68405b460 con 0x7ff6940ff490 2024-05-15T15:51:12.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.770+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff694197e90 con 0x7ff6940ff490 2024-05-15T15:51:12.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.771+0000 7ff692ffd700 1 -- 172.21.15.49:0/2590172573 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7ff68405aff0 con 0x7ff6940ff490 2024-05-15T15:51:12.774 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-15T15:51:12.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff67c06cce0 msgr2=0x7ff67c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:12.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff67c06cce0 0x7ff67c06f1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff694197b90 tx=0x7ff68800a380 comp rx=0 tx=0).stop 2024-05-15T15:51:12.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6940ff490 msgr2=0x7ff69419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:12.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6940ff490 0x7ff69419c890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff68400bd70 tx=0x7ff68400bda0 comp rx=0 tx=0).stop 2024-05-15T15:51:12.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 shutdown_connections 2024-05-15T15:51:12.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6940ff490 0x7ff69419c890 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:12.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7ff67c06cce0 0x7ff67c06f1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:12.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 --2- 172.21.15.49:0/2590172573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105c90 0x7ff69419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:12.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 >> 172.21.15.49:0/2590172573 conn(0x7ff694074e70 msgr2=0x7ff6940fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:12.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 shutdown_connections 2024-05-15T15:51:12.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:12.773+0000 7ff69bfa9700 1 -- 172.21.15.49:0/2590172573 wait complete. 2024-05-15T15:51:13.183 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:12 smithi049 ceph-mon[28399]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-15T15:51:13.183 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:12 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2590172573' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-15T15:51:13.410 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-15T15:51:13.410 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-15T15:51:13.410 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-05-15T15:51:13.410 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-15T15:51:13.410 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:13.411 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-15T15:51:13.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-15T15:51:13.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-15T15:51:13.411 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-15T15:51:13.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:13.411 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-05-15T15:51:13.430 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-15T15:51:13.430 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-15T15:51:13.430 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-05-15T15:51:13.431 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-15T15:51:13.431 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-05-15T15:51:13.431 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-15T15:51:13.431 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-15T15:51:13.431 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-05-15T15:51:13.431 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-05-15T15:51:13.431 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-15T15:51:13.432 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-05-15T15:51:13.432 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-05-15T15:51:13.432 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-15T15:51:13.432 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-05-15T15:51:13.432 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-05-15T15:51:13.432 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-15T15:51:13.432 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42206sec preferred_lft 42206sec 2024-05-15T15:51:13.433 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-05-15T15:51:13.433 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-15T15:51:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-15T15:51:13.433 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-05-15T15:51:13.433 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-05-15T15:51:13.434 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-15T15:51:13.434 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:13.434 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-15T15:51:13.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2024-05-15T15:51:13.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2024-05-15T15:51:13.435 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2024-05-15T15:51:13.435 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-15T15:51:13.435 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-15T15:51:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:12 smithi144 ceph-mon[36389]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-15T15:51:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:12 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2590172573' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-15T15:51:13.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:13.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:13.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:13.635 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-15T15:51:13.661 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-05-15T15:51:13.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:13.663 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-05-15T15:51:13.734 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-05-15T15:51:13.734 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-15T15:51:13.734 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-15T15:51:13.735 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-15T15:51:13.735 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-15T15:51:13.735 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:13.735 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-15T15:51:13.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-15T15:51:13.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-15T15:51:13.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-15T15:51:13.736 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-15T15:51:13.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:14.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:14.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:14.036 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-15T15:51:14.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:14.053 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-05-15T15:51:14.110 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:14.110 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-15T15:51:14.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-15T15:51:14.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-15T15:51:14.110 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-15T15:51:14.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:14.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:14.231 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-15T15:51:14.231 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:14.231 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-15T15:51:14.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-15T15:51:14.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-15T15:51:14.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-15T15:51:14.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-15T15:51:14.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-15T15:51:14.232 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-15T15:51:14.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:14.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:14.359 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:14.359 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-15T15:51:14.359 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-05-15T15:51:14.359 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2024-05-15T15:51:14.359 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-15T15:51:14.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:14.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:14.440 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-15T15:51:14.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-15T15:51:14.441 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:14.457 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-15T15:51:14.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-15T15:51:14.457 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:14.513 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2024-05-15T15:51:14.580 DEBUG:teuthology.orchestra.run.smithi049:> cat /proc/self/mounts | awk '{print $2}' 2024-05-15T15:51:14.638 INFO:teuthology.orchestra.run.smithi049.stdout:/proc 2024-05-15T15:51:14.638 INFO:teuthology.orchestra.run.smithi049.stdout:/sys 2024-05-15T15:51:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:/dev 2024-05-15T15:51:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/security 2024-05-15T15:51:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/shm 2024-05-15T15:51:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/pts 2024-05-15T15:51:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:/run 2024-05-15T15:51:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup 2024-05-15T15:51:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/pstore 2024-05-15T15:51:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/bpf 2024-05-15T15:51:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:/ 2024-05-15T15:51:14.640 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/selinux 2024-05-15T15:51:14.640 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2024-05-15T15:51:14.640 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/hugepages 2024-05-15T15:51:14.640 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/mqueue 2024-05-15T15:51:14.640 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/debug 2024-05-15T15:51:14.640 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/tracing 2024-05-15T15:51:14.640 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/fuse/connections 2024-05-15T15:51:14.640 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/config 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-sysctl.service 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/nfs/rpc_pipefs 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/1000 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/ceph 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/3ac89091c3684e5e8de934704ef1d1dd1e42606a67d654c816ac59fc2d44f430/merged 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/22be059f91f0d12c444a4d22baf48094be3af4e8b39fcdae3b2518290528b37a/merged 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/0 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2024-05-15T15:51:14.641 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/431bd9a771682fe89294b0b110691e183b1b9762417b667d6d1383216ab7b680/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/ae5b239c8ac0ec9f7fcf81ee2f942e40eaf8ddf6461d7b9be91e29de62e6697d/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/a088dca826ced4369fc9974823fa90326b3b4399416cd832c9b7c27f978e748d/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/e051919f8e19d58a77ada84e226c5226245dda4a6d5362982e9d19c2c3af8da7/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/008a3370433459bb01de0c0e170851549ee4a87ea30e3a7c13e46f478613b6ed/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/f0ee989d90fbe2fadb217e83ca2d26ee2d9eed76d07f254a1dc18130363eab6f/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/4556a3005222e77b1a1f443c1101a777a82b5efe364e12e3d64d6f58871f8aa3/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/5d68e229820feb6f8c2b36e1bc31fd25b8d29ec58e10aee16943d514a8b30a25/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/73ad1c8bcade2249221d7f02d4f7d1b746e80318f4aef3c643cec1f6439cc1c8/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/ce17ceccce92b67e0d6c815c87d0e16ae4600f3f291941d3b186e65ab60607b7/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/c14c50671db64fe2182cc7de883b3d1d143df90dfbe55a8a2ab29dbed5e205c3/merged 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-15T15:51:14.642 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-15T15:51:14.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:14.643 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2024-05-15T15:51:14.699 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-15T15:51:14.699 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-15T15:51:14.740 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2024-05-15T15:51:14.766 DEBUG:teuthology.orchestra.run.smithi049:> cat /proc/self/mounts | awk '{print $2}' 2024-05-15T15:51:14.823 INFO:teuthology.orchestra.run.smithi049.stdout:/proc 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/sys 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/dev 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/security 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/shm 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/pts 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/run 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/pstore 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/bpf 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/ 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/selinux 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/hugepages 2024-05-15T15:51:14.824 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/mqueue 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/debug 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/tracing 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/fuse/connections 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/config 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-sysctl.service 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/nfs/rpc_pipefs 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/1000 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/ceph 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/3ac89091c3684e5e8de934704ef1d1dd1e42606a67d654c816ac59fc2d44f430/merged 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/22be059f91f0d12c444a4d22baf48094be3af4e8b39fcdae3b2518290528b37a/merged 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/0 2024-05-15T15:51:14.825 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/431bd9a771682fe89294b0b110691e183b1b9762417b667d6d1383216ab7b680/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/ae5b239c8ac0ec9f7fcf81ee2f942e40eaf8ddf6461d7b9be91e29de62e6697d/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/a088dca826ced4369fc9974823fa90326b3b4399416cd832c9b7c27f978e748d/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/e051919f8e19d58a77ada84e226c5226245dda4a6d5362982e9d19c2c3af8da7/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/008a3370433459bb01de0c0e170851549ee4a87ea30e3a7c13e46f478613b6ed/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/f0ee989d90fbe2fadb217e83ca2d26ee2d9eed76d07f254a1dc18130363eab6f/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/4556a3005222e77b1a1f443c1101a777a82b5efe364e12e3d64d6f58871f8aa3/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/5d68e229820feb6f8c2b36e1bc31fd25b8d29ec58e10aee16943d514a8b30a25/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/73ad1c8bcade2249221d7f02d4f7d1b746e80318f4aef3c643cec1f6439cc1c8/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/ce17ceccce92b67e0d6c815c87d0e16ae4600f3f291941d3b186e65ab60607b7/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/c14c50671db64fe2182cc7de883b3d1d143df90dfbe55a8a2ab29dbed5e205c3/merged 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-15T15:51:14.826 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-15T15:51:14.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:14.827 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2024-05-15T15:51:14.838 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2024-05-15T15:51:14.835+0000 7fa990dff480 -1 init, newargv = 0x559ba9d0d5b0 newargc=15 2024-05-15T15:51:14.838 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[76016]: starting ceph client 2024-05-15T15:51:14.846 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[76016]: starting fuse 2024-05-15T15:51:14.854 INFO:teuthology.orchestra.run.smithi049.stdout:131 2024-05-15T15:51:14.855 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [131] 2024-05-15T15:51:14.855 DEBUG:teuthology.orchestra.run.smithi049:> sudo stdin-killer -- python3 -c ' 2024-05-15T15:51:14.855 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2024-05-15T15:51:14.855 DEBUG:teuthology.orchestra.run.smithi049:> import re 2024-05-15T15:51:14.855 DEBUG:teuthology.orchestra.run.smithi049:> import os 2024-05-15T15:51:14.855 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2024-05-15T15:51:14.855 DEBUG:teuthology.orchestra.run.smithi049:> 2024-05-15T15:51:14.855 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2024-05-15T15:51:14.855 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-15T15:51:14.855 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> return f 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-15T15:51:14.856 DEBUG:teuthology.orchestra.run.smithi049:> 2024-05-15T15:51:14.857 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2024-05-15T15:51:14.857 DEBUG:teuthology.orchestra.run.smithi049:> ' 2024-05-15T15:51:14.976 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.76016.asok 2024-05-15T15:51:14.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:14.987 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76016.asok 2024-05-15T15:51:14.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:14.987 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76016.asok status 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "2965081fb1a343a09e00ceeede998525f5c6cb5e", 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)", 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "76016", 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2024-05-15T15:51:15.127 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 14534, 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 14534 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1252206647 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2024-05-15T15:51:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1252206647 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.14534 192.168.0.1:0/1252206647", 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/1252206647", 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 17, 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 47, 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2024-05-15T15:51:15.129 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:51:15.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-15T15:51:15.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:14 smithi049 ceph-mon[28399]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 8.3 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-15T15:51:15.351 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:51:15.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:14 smithi144 ceph-mon[36389]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 8.3 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-15T15:51:17.008 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:16 smithi049 ceph-mon[28399]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-15T15:51:17.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.021+0000 7f4f8be94700 1 -- 172.21.15.49:0/573130565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 msgr2=0x7f4f8410acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:17.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.021+0000 7f4f8be94700 1 --2- 172.21.15.49:0/573130565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 0x7f4f8410acd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009910 tx=0x7f4f78009c20 comp rx=0 tx=0).stop 2024-05-15T15:51:17.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.021+0000 7f4f8be94700 1 -- 172.21.15.49:0/573130565 shutdown_connections 2024-05-15T15:51:17.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.021+0000 7f4f8be94700 1 --2- 172.21.15.49:0/573130565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 0x7f4f8410acd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:17.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.021+0000 7f4f8be94700 1 --2- 172.21.15.49:0/573130565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f84101ec0 0x7f4f841022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:17.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.021+0000 7f4f8be94700 1 -- 172.21.15.49:0/573130565 >> 172.21.15.49:0/573130565 conn(0x7f4f84076800 msgr2=0x7f4f840fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:17.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.021+0000 7f4f8be94700 1 -- 172.21.15.49:0/573130565 shutdown_connections 2024-05-15T15:51:17.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.022+0000 7f4f8be94700 1 -- 172.21.15.49:0/573130565 wait complete. 2024-05-15T15:51:17.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.022+0000 7f4f8be94700 1 Processor -- start 2024-05-15T15:51:17.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.022+0000 7f4f8be94700 1 -- start start 2024-05-15T15:51:17.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.022+0000 7f4f8be94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f84101ec0 0x7f4f8419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.022+0000 7f4f8be94700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 0x7f4f8419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.022+0000 7f4f8be94700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8419d540 con 0x7f4f841027e0 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.022+0000 7f4f8be94700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84196b20 con 0x7f4f84101ec0 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f8942f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 0x7f4f8419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f8942f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 0x7f4f8419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38294/0 (socket says 172.21.15.49:38294) 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f8942f700 1 -- 172.21.15.49:0/3706537906 learned_addr learned my addr 172.21.15.49:0/3706537906 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f89c30700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f84101ec0 0x7f4f8419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f8942f700 1 -- 172.21.15.49:0/3706537906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f84101ec0 msgr2=0x7f4f8419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f8942f700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f84101ec0 0x7f4f8419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f8942f700 1 -- 172.21.15.49:0/3706537906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f780095f0 con 0x7f4f841027e0 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f89c30700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f84101ec0 0x7f4f8419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f8942f700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 0x7f4f8419cf20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4f780092e0 tx=0x7f4f7800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f82ffd700 1 -- 172.21.15.49:0/3706537906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f7801d070 con 0x7f4f841027e0 2024-05-15T15:51:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f82ffd700 1 -- 172.21.15.49:0/3706537906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f7800f460 con 0x7f4f841027e0 2024-05-15T15:51:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f84196da0 con 0x7f4f841027e0 2024-05-15T15:51:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f82ffd700 1 -- 172.21.15.49:0/3706537906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f78017630 con 0x7f4f841027e0 2024-05-15T15:51:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.023+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f84197290 con 0x7f4f841027e0 2024-05-15T15:51:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.024+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f841056b0 con 0x7f4f841027e0 2024-05-15T15:51:17.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.025+0000 7f4f82ffd700 1 -- 172.21.15.49:0/3706537906 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4f7800f5d0 con 0x7f4f841027e0 2024-05-15T15:51:17.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.025+0000 7f4f82ffd700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4f6c06cce0 0x7f4f6c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:17.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.025+0000 7f4f82ffd700 1 -- 172.21.15.49:0/3706537906 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4f7808d620 con 0x7f4f841027e0 2024-05-15T15:51:17.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.025+0000 7f4f89c30700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4f6c06cce0 0x7f4f6c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:17.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.026+0000 7f4f89c30700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4f6c06cce0 0x7f4f6c06f1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009fa0 tx=0x7f4f74009570 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.031+0000 7f4f82ffd700 1 -- 172.21.15.49:0/3706537906 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4f7805b4a0 con 0x7f4f841027e0 2024-05-15T15:51:17.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.279+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f84068ec0 con 0x7f4f841027e0 2024-05-15T15:51:17.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.279+0000 7f4f82ffd700 1 -- 172.21.15.49:0/3706537906 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4f78005740 con 0x7f4f841027e0 2024-05-15T15:51:17.282 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-15T15:51:17.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.281+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4f6c06cce0 msgr2=0x7f4f6c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:17.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.281+0000 7f4f8be94700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4f6c06cce0 0x7f4f6c06f1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009fa0 tx=0x7f4f74009570 comp rx=0 tx=0).stop 2024-05-15T15:51:17.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.281+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 msgr2=0x7f4f8419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:17.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.281+0000 7f4f8be94700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 0x7f4f8419cf20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4f780092e0 tx=0x7f4f7800be20 comp rx=0 tx=0).stop 2024-05-15T15:51:17.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.281+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 shutdown_connections 2024-05-15T15:51:17.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.281+0000 7f4f8be94700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4f6c06cce0 0x7f4f6c06f1a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:17.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.281+0000 7f4f8be94700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f84101ec0 0x7f4f8419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:17.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.281+0000 7f4f8be94700 1 --2- 172.21.15.49:0/3706537906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f841027e0 0x7f4f8419cf20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:17.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.281+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 >> 172.21.15.49:0/3706537906 conn(0x7f4f84076800 msgr2=0x7f4f841031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:17.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.282+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 shutdown_connections 2024-05-15T15:51:17.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:17.282+0000 7f4f8be94700 1 -- 172.21.15.49:0/3706537906 wait complete. 2024-05-15T15:51:17.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:16 smithi144 ceph-mon[36389]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-15T15:51:17.920 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-15T15:51:17.920 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-15T15:51:17.920 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2024-05-15T15:51:17.920 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-15T15:51:17.920 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:17.920 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-15T15:51:17.920 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-15T15:51:17.920 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-15T15:51:17.920 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-15T15:51:17.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:17.920 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2024-05-15T15:51:17.940 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-15T15:51:17.940 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-15T15:51:17.940 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-05-15T15:51:17.940 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-05-15T15:51:17.940 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-05-15T15:51:17.940 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-05-15T15:51:17.940 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42271sec preferred_lft 42271sec 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2024-05-15T15:51:17.941 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-05-15T15:51:17.942 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-15T15:51:17.942 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:17.942 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-05-15T15:51:17.942 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add name ceph-brx type bridge 2024-05-15T15:51:17.942 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr flush dev ceph-brx 2024-05-15T15:51:17.942 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set ceph-brx up 2024-05-15T15:51:17.942 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-15T15:51:17.942 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-05-15T15:51:18.037 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:18.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:18.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:18.161 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-15T15:51:18.186 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-05-15T15:51:18.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:18.191 DEBUG:teuthology.orchestra.run.smithi144:> route 2024-05-15T15:51:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:17 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3706537906' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-15T15:51:18.263 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2024-05-15T15:51:18.263 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-15T15:51:18.263 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-15T15:51:18.263 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-15T15:51:18.263 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-15T15:51:18.263 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:18.263 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-05-15T15:51:18.263 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-15T15:51:18.264 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-15T15:51:18.264 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-15T15:51:18.264 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-05-15T15:51:18.350 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:17 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3706537906' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-15T15:51:18.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:18.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:18.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:18.584 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-05-15T15:51:18.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:18.600 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2024-05-15T15:51:18.656 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:18.656 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-05-15T15:51:18.656 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-15T15:51:18.656 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-15T15:51:18.656 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-05-15T15:51:18.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:18.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:18.780 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-15T15:51:18.780 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:18.780 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-05-15T15:51:18.780 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-15T15:51:18.780 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-15T15:51:18.780 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-15T15:51:18.781 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-15T15:51:18.781 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-15T15:51:18.781 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-05-15T15:51:18.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:18.908 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T15:51:18.908 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-05-15T15:51:18.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2024-05-15T15:51:18.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.0 master ceph-brx 2024-05-15T15:51:18.908 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-05-15T15:51:18.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T15:51:18.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:18.987 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-15T15:51:18.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-15T15:51:18.987 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:19.002 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-15T15:51:19.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-15T15:51:19.003 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:19.059 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-05-15T15:51:19.124 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-05-15T15:51:19.181 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-05-15T15:51:19.181 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-05-15T15:51:19.181 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-05-15T15:51:19.181 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-05-15T15:51:19.181 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-05-15T15:51:19.181 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-05-15T15:51:19.182 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/0 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/d264a9e994373d7a03b60e8c002476af0c283c6c122e8bf6371f727b5ab0452a/merged 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/fa92f6a63a6fe5ae23b523aad37def5649914dd9af94b18e9b968dd4488fe79c/merged 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/b62f8fcf9c953da72e6ae88c4defcc303a75df70b02aab5ae9876ea109b8fef3/merged 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/3be01d64ea67f40d1fdb49870d8ce41a3c703430a4f811d2e540f334ff1dbbde/merged 2024-05-15T15:51:19.183 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/6563db79bad85ae3a71752e5d3d4a141b381f9958726532d7604b3c655ae25bf/merged 2024-05-15T15:51:19.184 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/05e5d40f029297045f2c10ad4f2c7aa317cad150600c5939ddb43a6fed8133b8/merged 2024-05-15T15:51:19.184 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/282f6d89e41d5e2570d95c1c05d969435f90482d91ba3e10571d9e51c8acff9d/merged 2024-05-15T15:51:19.184 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a3123ac37f8f2ab1b38e8a17a1abfa3ae7fd3173898eb8a7b4bc315b8d4ea34b/merged 2024-05-15T15:51:19.184 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/9ec7e00db964f0dfb8b9923fca8ee014e1e86acbf7e245035f1baa245bef4d8a/merged 2024-05-15T15:51:19.184 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a8da2371b0912f22859539459bce233c9ab86407be340fdc532088da507a2c05/merged 2024-05-15T15:51:19.184 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-05-15T15:51:19.184 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-15T15:51:19.184 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-15T15:51:19.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:19.184 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-05-15T15:51:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:18 smithi144 ceph-mon[36389]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-05-15T15:51:19.214 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-15T15:51:19.214 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-15T15:51:19.256 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-05-15T15:51:19.281 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-05-15T15:51:19.338 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-05-15T15:51:19.339 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/0 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/d264a9e994373d7a03b60e8c002476af0c283c6c122e8bf6371f727b5ab0452a/merged 2024-05-15T15:51:19.340 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/fa92f6a63a6fe5ae23b523aad37def5649914dd9af94b18e9b968dd4488fe79c/merged 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/b62f8fcf9c953da72e6ae88c4defcc303a75df70b02aab5ae9876ea109b8fef3/merged 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/3be01d64ea67f40d1fdb49870d8ce41a3c703430a4f811d2e540f334ff1dbbde/merged 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/6563db79bad85ae3a71752e5d3d4a141b381f9958726532d7604b3c655ae25bf/merged 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/05e5d40f029297045f2c10ad4f2c7aa317cad150600c5939ddb43a6fed8133b8/merged 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/282f6d89e41d5e2570d95c1c05d969435f90482d91ba3e10571d9e51c8acff9d/merged 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a3123ac37f8f2ab1b38e8a17a1abfa3ae7fd3173898eb8a7b4bc315b8d4ea34b/merged 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/9ec7e00db964f0dfb8b9923fca8ee014e1e86acbf7e245035f1baa245bef4d8a/merged 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a8da2371b0912f22859539459bce233c9ab86407be340fdc532088da507a2c05/merged 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-15T15:51:19.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:19.342 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-05-15T15:51:19.361 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:2024-05-15T15:51:19.359+0000 7f7ad823d480 -1 init, newargv = 0x5640095b64c0 newargc=15 2024-05-15T15:51:19.361 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:ceph-fuse[61926]: starting ceph client 2024-05-15T15:51:19.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:ceph-fuse[61926]: starting fuse 2024-05-15T15:51:19.378 INFO:teuthology.orchestra.run.smithi144.stdout:128 2024-05-15T15:51:19.378 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [128] 2024-05-15T15:51:19.378 DEBUG:teuthology.orchestra.run.smithi144:> sudo stdin-killer -- python3 -c ' 2024-05-15T15:51:19.378 DEBUG:teuthology.orchestra.run.smithi144:> import glob 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> import re 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> import subprocess 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> def _find_admin_socket(client_name): 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> files = glob.glob(asok_path) 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> # Given a non-glob path, it better be there 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> if "*" not in asok_path: 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> assert(len(files) == 1) 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> return files[0] 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> for f in files: 2024-05-15T15:51:19.379 DEBUG:teuthology.orchestra.run.smithi144:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-15T15:51:19.380 DEBUG:teuthology.orchestra.run.smithi144:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-15T15:51:19.380 DEBUG:teuthology.orchestra.run.smithi144:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-15T15:51:19.380 DEBUG:teuthology.orchestra.run.smithi144:> contents = proc_f.read() 2024-05-15T15:51:19.380 DEBUG:teuthology.orchestra.run.smithi144:> if mountpoint in contents: 2024-05-15T15:51:19.380 DEBUG:teuthology.orchestra.run.smithi144:> return f 2024-05-15T15:51:19.380 DEBUG:teuthology.orchestra.run.smithi144:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-15T15:51:19.380 DEBUG:teuthology.orchestra.run.smithi144:> 2024-05-15T15:51:19.380 DEBUG:teuthology.orchestra.run.smithi144:> print(_find_admin_socket("client.1")) 2024-05-15T15:51:19.380 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-05-15T15:51:19.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:18 smithi049 ceph-mon[28399]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-05-15T15:51:19.497 INFO:teuthology.orchestra.run.smithi144.stdout:/var/run/ceph/ceph-client.1.61926.asok 2024-05-15T15:51:19.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-15T15:51:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T15:51:19.507 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61926.asok 2024-05-15T15:51:19.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:19.508 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61926.asok status 2024-05-15T15:51:19.645 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-15T15:51:19.645 INFO:teuthology.orchestra.run.smithi144.stdout: "metadata": { 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_sha1": "2965081fb1a343a09e00ceeede998525f5c6cb5e", 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)", 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "entity_id": "1", 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "hostname": "smithi144", 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "pid": "61926", 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "root": "/" 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_count": 0, 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_pinned_count": 0, 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 24353, 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "inst": { 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "name": { 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "client", 2024-05-15T15:51:19.646 INFO:teuthology.orchestra.run.smithi144.stdout: "num": 24353 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 2186632469 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 2186632469 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "inst_str": "client.24353 192.168.0.1:0/2186632469", 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "addr_str": "192.168.0.1:0/2186632469", 2024-05-15T15:51:19.647 INFO:teuthology.orchestra.run.smithi144.stdout: "inode_count": 1, 2024-05-15T15:51:19.648 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_epoch": 17, 2024-05-15T15:51:19.648 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch": 47, 2024-05-15T15:51:19.648 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch_barrier": 0, 2024-05-15T15:51:19.648 INFO:teuthology.orchestra.run.smithi144.stdout: "blocklisted": false, 2024-05-15T15:51:19.648 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs" 2024-05-15T15:51:19.648 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-15T15:51:19.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:19.654 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2024-05-15T15:51:19.654 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:19.676 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2024-05-15T15:51:19.676 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:19.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:19.677 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:19.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:19.752 DEBUG:teuthology.orchestra.run.smithi144:> stat --file-system '--printf=%T 2024-05-15T15:51:19.752 DEBUG:teuthology.orchestra.run.smithi144:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:19.769 INFO:teuthology.orchestra.run.smithi144.stdout:fuseblk 2024-05-15T15:51:19.770 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:19.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T15:51:19.770 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:21 smithi144 ceph-mon[36389]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-05-15T15:51:21.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:21 smithi049 ceph-mon[28399]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-05-15T15:51:22.072 INFO:teuthology.run_tasks:Running task print... 2024-05-15T15:51:22.083 INFO:teuthology.task.print:**** done client 2024-05-15T15:51:22.083 INFO:teuthology.run_tasks:Running task parallel... 2024-05-15T15:51:22.104 INFO:teuthology.task.parallel:starting parallel... 2024-05-15T15:51:22.104 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-15T15:51:22.105 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-15T15:51:22.105 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:51:22.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-15T15:51:22.106 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-15T15:51:22.107 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-15T15:51:22.111 INFO:tasks.workunit:Pulling workunits from ref de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:51:22.112 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-15T15:51:22.112 INFO:tasks.workunit:timeout=3h 2024-05-15T15:51:22.112 INFO:tasks.workunit:cleanup=True 2024-05-15T15:51:22.112 DEBUG:teuthology.orchestra.run.smithi049:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:22.135 INFO:teuthology.orchestra.run.smithi049.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:22.135 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-15T15:51:22.135 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 83h/131d Inode: 1 Links: 2 2024-05-15T15:51:22.135 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:51:22.135 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-15T15:51:22.135 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-15T15:51:22.135 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-15 15:50:45.558463296 +0000 2024-05-15T15:51:22.135 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-15 15:51:19.747625274 +0000 2024-05-15T15:51:22.135 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2024-05-15T15:51:22.136 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-15T15:51:22.136 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-15T15:51:22.210 DEBUG:teuthology.orchestra.run.smithi144:> stat -- /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:22.231 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:22.232 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-15T15:51:22.232 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 80h/128d Inode: 1 Links: 3 2024-05-15T15:51:22.232 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T15:51:22.232 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-15T15:51:22.232 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-15T15:51:22.232 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-15 15:51:22.201701559 +0000 2024-05-15T15:51:22.232 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-15 15:51:22.201701559 +0000 2024-05-15T15:51:22.232 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-05-15T15:51:22.232 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-05-15T15:51:22.232 DEBUG:teuthology.orchestra.run.smithi144:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-05-15T15:51:22.297 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:22 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:51:22.308 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:51:22.308 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:51:22.326 INFO:tasks.workunit.client.0.smithi049.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-15T15:51:22.343 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:51:22.365 INFO:tasks.workunit.client.1.smithi144.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-05-15T15:51:22.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:22 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:51:23.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:23 smithi049 ceph-mon[28399]: pgmap v200: 65 pgs: 65 active+clean; 581 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 426 B/s wr, 4 op/s 2024-05-15T15:51:23.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:23 smithi144 ceph-mon[36389]: pgmap v200: 65 pgs: 65 active+clean; 581 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 426 B/s wr, 4 op/s 2024-05-15T15:51:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.923+0000 7f4d7e69d700 1 -- 172.21.15.49:0/3845282289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d78100de0 msgr2=0x7f4d78101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.923+0000 7f4d7e69d700 1 --2- 172.21.15.49:0/3845282289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d78100de0 0x7f4d78101240 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009910 tx=0x7f4d60009c20 comp rx=0 tx=0).stop 2024-05-15T15:51:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.923+0000 7f4d7e69d700 1 -- 172.21.15.49:0/3845282289 shutdown_connections 2024-05-15T15:51:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.923+0000 7f4d7e69d700 1 --2- 172.21.15.49:0/3845282289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d78100de0 0x7f4d78101240 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.923+0000 7f4d7e69d700 1 --2- 172.21.15.49:0/3845282289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d780ffbe0 0x7f4d78100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.923+0000 7f4d7e69d700 1 -- 172.21.15.49:0/3845282289 >> 172.21.15.49:0/3845282289 conn(0x7f4d780fb180 msgr2=0x7f4d780fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.923+0000 7f4d7e69d700 1 -- 172.21.15.49:0/3845282289 shutdown_connections 2024-05-15T15:51:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.923+0000 7f4d7e69d700 1 -- 172.21.15.49:0/3845282289 wait complete. 2024-05-15T15:51:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.924+0000 7f4d7e69d700 1 Processor -- start 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.924+0000 7f4d7e69d700 1 -- start start 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.924+0000 7f4d7e69d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d780ffbe0 0x7f4d78198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.924+0000 7f4d7e69d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d78100de0 0x7f4d78198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.924+0000 7f4d7e69d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d781993f0 con 0x7f4d780ffbe0 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.924+0000 7f4d7e69d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d78199530 con 0x7f4d78100de0 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d77fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d780ffbe0 0x7f4d78198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d77fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d780ffbe0 0x7f4d78198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44304/0 (socket says 172.21.15.49:44304) 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d77fff700 1 -- 172.21.15.49:0/809747240 learned_addr learned my addr 172.21.15.49:0/809747240 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d6ffff700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d78100de0 0x7f4d78198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d77fff700 1 -- 172.21.15.49:0/809747240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d78100de0 msgr2=0x7f4d78198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d77fff700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d78100de0 0x7f4d78198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d77fff700 1 -- 172.21.15.49:0/809747240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d600095f0 con 0x7f4d780ffbe0 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d6ffff700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d78100de0 0x7f4d78198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T15:51:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d77fff700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d780ffbe0 0x7f4d78198890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800c8f0 tx=0x7f4d6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d75ffb700 1 -- 172.21.15.49:0/809747240 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d68007a70 con 0x7f4d780ffbe0 2024-05-15T15:51:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d75ffb700 1 -- 172.21.15.49:0/809747240 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d68007bd0 con 0x7f4d780ffbe0 2024-05-15T15:51:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.925+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7819df90 con 0x7f4d780ffbe0 2024-05-15T15:51:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.926+0000 7f4d75ffb700 1 -- 172.21.15.49:0/809747240 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d68018630 con 0x7f4d780ffbe0 2024-05-15T15:51:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.926+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7819e4d0 con 0x7f4d780ffbe0 2024-05-15T15:51:23.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.927+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7804ec60 con 0x7f4d780ffbe0 2024-05-15T15:51:23.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.927+0000 7f4d75ffb700 1 -- 172.21.15.49:0/809747240 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4d6800f450 con 0x7f4d780ffbe0 2024-05-15T15:51:23.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.928+0000 7f4d75ffb700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d6406cce0 0x7f4d6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:23.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.928+0000 7f4d75ffb700 1 -- 172.21.15.49:0/809747240 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4d6808c300 con 0x7f4d780ffbe0 2024-05-15T15:51:23.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.928+0000 7f4d6ffff700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d6406cce0 0x7f4d6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:23.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.928+0000 7f4d6ffff700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d6406cce0 0x7f4d6406f1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d600051c0 tx=0x7f4d60005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:23.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:23.933+0000 7f4d75ffb700 1 -- 172.21.15.49:0/809747240 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4d6805a230 con 0x7f4d780ffbe0 2024-05-15T15:51:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.136+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f4d78108740 con 0x7f4d780ffbe0 2024-05-15T15:51:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.137+0000 7f4d75ffb700 1 -- 172.21.15.49:0/809747240 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v25)=0 v25) v1 ==== 155+0+0 (secure 0 0 0) 0x7f4d68059dc0 con 0x7f4d780ffbe0 2024-05-15T15:51:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d6406cce0 msgr2=0x7f4d6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d6406cce0 0x7f4d6406f1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d600051c0 tx=0x7f4d60005920 comp rx=0 tx=0).stop 2024-05-15T15:51:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d780ffbe0 msgr2=0x7f4d78198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d780ffbe0 0x7f4d78198890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800c8f0 tx=0x7f4d6800ccb0 comp rx=0 tx=0).stop 2024-05-15T15:51:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 shutdown_connections 2024-05-15T15:51:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d780ffbe0 0x7f4d78198890 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f4d6406cce0 0x7f4d6406f1a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 --2- 172.21.15.49:0/809747240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d78100de0 0x7f4d78198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 >> 172.21.15.49:0/809747240 conn(0x7f4d780fb180 msgr2=0x7f4d78069170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 shutdown_connections 2024-05-15T15:51:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:24.139+0000 7f4d7e69d700 1 -- 172.21.15.49:0/809747240 wait complete. 2024-05-15T15:51:24.785 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-15T15:51:25.044 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:51:25.158 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:25 smithi049 ceph-mon[28399]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-15T15:51:25.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:25 smithi144 ceph-mon[36389]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-15T15:51:26.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.725+0000 7fb81d150700 1 -- 172.21.15.49:0/845265745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 msgr2=0x7fb8181012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:26.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.725+0000 7fb81d150700 1 --2- 172.21.15.49:0/845265745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 0x7fb8181012b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb800009910 tx=0x7fb800009c20 comp rx=0 tx=0).stop 2024-05-15T15:51:26.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.725+0000 7fb81d150700 1 -- 172.21.15.49:0/845265745 shutdown_connections 2024-05-15T15:51:26.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.725+0000 7fb81d150700 1 --2- 172.21.15.49:0/845265745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 0x7fb8181012b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:26.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.725+0000 7fb81d150700 1 --2- 172.21.15.49:0/845265745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8180ffc50 0x7fb818100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:26.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.725+0000 7fb81d150700 1 -- 172.21.15.49:0/845265745 >> 172.21.15.49:0/845265745 conn(0x7fb8180fb1d0 msgr2=0x7fb8180fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.725+0000 7fb81d150700 1 -- 172.21.15.49:0/845265745 shutdown_connections 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.725+0000 7fb81d150700 1 -- 172.21.15.49:0/845265745 wait complete. 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.726+0000 7fb81d150700 1 Processor -- start 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.726+0000 7fb81d150700 1 -- start start 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.726+0000 7fb81d150700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8180ffc50 0x7fb818069140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.726+0000 7fb81d150700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 0x7fb81819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.726+0000 7fb81d150700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81819ba40 con 0x7fb818100e50 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.726+0000 7fb81d150700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81819bb80 con 0x7fb8180ffc50 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb81659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 0x7fb81819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb81659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 0x7fb81819b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44328/0 (socket says 172.21.15.49:44328) 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb81659c700 1 -- 172.21.15.49:0/2174703621 learned_addr learned my addr 172.21.15.49:0/2174703621 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb816d9d700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8180ffc50 0x7fb818069140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb81659c700 1 -- 172.21.15.49:0/2174703621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8180ffc50 msgr2=0x7fb818069140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb81659c700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8180ffc50 0x7fb818069140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb81659c700 1 -- 172.21.15.49:0/2174703621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8000095f0 con 0x7fb818100e50 2024-05-15T15:51:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb81659c700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 0x7fb81819b500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb8000092e0 tx=0x7fb80000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb80ffff700 1 -- 172.21.15.49:0/2174703621 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb80001d070 con 0x7fb818100e50 2024-05-15T15:51:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81819bdd0 con 0x7fb818100e50 2024-05-15T15:51:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb80ffff700 1 -- 172.21.15.49:0/2174703621 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb80000f460 con 0x7fb818100e50 2024-05-15T15:51:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.727+0000 7fb80ffff700 1 -- 172.21.15.49:0/2174703621 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb800021620 con 0x7fb818100e50 2024-05-15T15:51:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.728+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81819c2c0 con 0x7fb818100e50 2024-05-15T15:51:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.729+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81818f7d0 con 0x7fb818100e50 2024-05-15T15:51:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.729+0000 7fb80ffff700 1 -- 172.21.15.49:0/2174703621 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb80000f5d0 con 0x7fb818100e50 2024-05-15T15:51:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.730+0000 7fb80ffff700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb80406cce0 0x7fb80406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.730+0000 7fb80ffff700 1 -- 172.21.15.49:0/2174703621 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb80008d4c0 con 0x7fb818100e50 2024-05-15T15:51:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.730+0000 7fb816d9d700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb80406cce0 0x7fb80406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.730+0000 7fb816d9d700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb80406cce0 0x7fb80406f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb808009550 tx=0x7fb8080091b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:26.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.735+0000 7fb80ffff700 1 -- 172.21.15.49:0/2174703621 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb80005b2c0 con 0x7fb818100e50 2024-05-15T15:51:26.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.936+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fb81802ce40 con 0x7fb818100e50 2024-05-15T15:51:26.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.937+0000 7fb80ffff700 1 -- 172.21.15.49:0/2174703621 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v25)=0 v25) v1 ==== 163+0+0 (secure 0 0 0) 0x7fb80005ae50 con 0x7fb818100e50 2024-05-15T15:51:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb80406cce0 msgr2=0x7fb80406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb80406cce0 0x7fb80406f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb808009550 tx=0x7fb8080091b0 comp rx=0 tx=0).stop 2024-05-15T15:51:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 msgr2=0x7fb81819b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 0x7fb81819b500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb8000092e0 tx=0x7fb80000bd70 comp rx=0 tx=0).stop 2024-05-15T15:51:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 shutdown_connections 2024-05-15T15:51:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fb80406cce0 0x7fb80406f1a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8180ffc50 0x7fb818069140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 --2- 172.21.15.49:0/2174703621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb818100e50 0x7fb81819b500 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:26.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 >> 172.21.15.49:0/2174703621 conn(0x7fb8180fb1d0 msgr2=0x7fb818104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:26.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 shutdown_connections 2024-05-15T15:51:26.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:26.939+0000 7fb81d150700 1 -- 172.21.15.49:0/2174703621 wait complete. 2024-05-15T15:51:27.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:27 smithi144 ceph-mon[36389]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-15T15:51:27.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:27 smithi049 ceph-mon[28399]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-15T15:51:27.543 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-15T15:51:27.764 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:51:29.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:29 smithi049 ceph-mon[28399]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 938 B/s wr, 4 op/s 2024-05-15T15:51:29.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:29 smithi144 ceph-mon[36389]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 938 B/s wr, 4 op/s 2024-05-15T15:51:29.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.458+0000 7f369328c700 1 -- 172.21.15.49:0/2901112757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c100cf0 msgr2=0x7f368c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:29.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.458+0000 7f369328c700 1 --2- 172.21.15.49:0/2901112757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c100cf0 0x7f368c101110 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3688009910 tx=0x7f3688009c20 comp rx=0 tx=0).stop 2024-05-15T15:51:29.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.458+0000 7f369328c700 1 -- 172.21.15.49:0/2901112757 shutdown_connections 2024-05-15T15:51:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.458+0000 7f369328c700 1 --2- 172.21.15.49:0/2901112757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f368c101ef0 0x7f368c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.458+0000 7f369328c700 1 --2- 172.21.15.49:0/2901112757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c100cf0 0x7f368c101110 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.458+0000 7f369328c700 1 -- 172.21.15.49:0/2901112757 >> 172.21.15.49:0/2901112757 conn(0x7f368c0fc290 msgr2=0x7f368c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.458+0000 7f369328c700 1 -- 172.21.15.49:0/2901112757 shutdown_connections 2024-05-15T15:51:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.458+0000 7f369328c700 1 -- 172.21.15.49:0/2901112757 wait complete. 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.458+0000 7f369328c700 1 Processor -- start 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f369328c700 1 -- start start 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f369328c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f368c100cf0 0x7f368c074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f369328c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c101ef0 0x7f368c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f369328c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0738d0 con 0x7f368c101ef0 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f369328c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c073a10 con 0x7f368c100cf0 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f3690827700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c101ef0 0x7f368c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f3690827700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c101ef0 0x7f368c073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44356/0 (socket says 172.21.15.49:44356) 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f3690827700 1 -- 172.21.15.49:0/2074244778 learned_addr learned my addr 172.21.15.49:0/2074244778 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f3691028700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f368c100cf0 0x7f368c074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f3690827700 1 -- 172.21.15.49:0/2074244778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f368c100cf0 msgr2=0x7f368c074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f3690827700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f368c100cf0 0x7f368c074cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f3690827700 1 -- 172.21.15.49:0/2074244778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36880095f0 con 0x7f368c101ef0 2024-05-15T15:51:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.459+0000 7f3690827700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c101ef0 0x7f368c073390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f367c00b4b0 tx=0x7f367c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.460+0000 7f36827fc700 1 -- 172.21.15.49:0/2074244778 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f367c011800 con 0x7f368c101ef0 2024-05-15T15:51:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.460+0000 7f36827fc700 1 -- 172.21.15.49:0/2074244778 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f367c011e20 con 0x7f368c101ef0 2024-05-15T15:51:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.460+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c073cf0 con 0x7f368c101ef0 2024-05-15T15:51:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.460+0000 7f36827fc700 1 -- 172.21.15.49:0/2074244778 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f367c00f2e0 con 0x7f368c101ef0 2024-05-15T15:51:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.460+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c19e870 con 0x7f368c101ef0 2024-05-15T15:51:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.461+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c1908f0 con 0x7f368c101ef0 2024-05-15T15:51:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.461+0000 7f36827fc700 1 -- 172.21.15.49:0/2074244778 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f367c010380 con 0x7f368c101ef0 2024-05-15T15:51:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.462+0000 7f36827fc700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f367806cce0 0x7f367806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.462+0000 7f36827fc700 1 -- 172.21.15.49:0/2074244778 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f367c08be40 con 0x7f368c101ef0 2024-05-15T15:51:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.462+0000 7f3691028700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f367806cce0 0x7f367806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.462+0000 7f3691028700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f367806cce0 0x7f367806f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f368800b5c0 tx=0x7f3688005160 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:29.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.467+0000 7f36827fc700 1 -- 172.21.15.49:0/2074244778 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f367c059d70 con 0x7f368c101ef0 2024-05-15T15:51:29.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.669+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f368c04ec60 con 0x7f368c101ef0 2024-05-15T15:51:29.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.670+0000 7f36827fc700 1 -- 172.21.15.49:0/2074244778 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v25)=0 v25) v1 ==== 135+0+0 (secure 0 0 0) 0x7f367c059900 con 0x7f368c101ef0 2024-05-15T15:51:29.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f367806cce0 msgr2=0x7f367806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:29.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f367806cce0 0x7f367806f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f368800b5c0 tx=0x7f3688005160 comp rx=0 tx=0).stop 2024-05-15T15:51:29.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c101ef0 msgr2=0x7f368c073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:29.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c101ef0 0x7f368c073390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f367c00b4b0 tx=0x7f367c00b870 comp rx=0 tx=0).stop 2024-05-15T15:51:29.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 shutdown_connections 2024-05-15T15:51:29.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f367806cce0 0x7f367806f1a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:29.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f368c100cf0 0x7f368c074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:29.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 --2- 172.21.15.49:0/2074244778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f368c101ef0 0x7f368c073390 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:29.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 >> 172.21.15.49:0/2074244778 conn(0x7f368c0fc290 msgr2=0x7f368c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:29.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 shutdown_connections 2024-05-15T15:51:29.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:29.672+0000 7f369328c700 1 -- 172.21.15.49:0/2074244778 wait complete. 2024-05-15T15:51:30.387 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-05-15T15:51:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:51:31.384 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:31 smithi049 ceph-mon[28399]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-15T15:51:31.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:31 smithi144 ceph-mon[36389]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-15T15:51:32.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 -- 172.21.15.49:0/3330164712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 msgr2=0x7f7c4c102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:32.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 --2- 172.21.15.49:0/3330164712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 0x7f7c4c102350 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009910 tx=0x7f7c38009c20 comp rx=0 tx=0).stop 2024-05-15T15:51:32.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 -- 172.21.15.49:0/3330164712 shutdown_connections 2024-05-15T15:51:32.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 --2- 172.21.15.49:0/3330164712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 0x7f7c4c102350 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 --2- 172.21.15.49:0/3330164712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c100cf0 0x7f7c4c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 -- 172.21.15.49:0/3330164712 >> 172.21.15.49:0/3330164712 conn(0x7f7c4c0fc290 msgr2=0x7f7c4c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 -- 172.21.15.49:0/3330164712 shutdown_connections 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 -- 172.21.15.49:0/3330164712 wait complete. 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 Processor -- start 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.302+0000 7f7c52c86700 1 -- start start 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c52c86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c100cf0 0x7f7c4c1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c52c86700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 0x7f7c4c196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c52c86700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c197350 con 0x7f7c4c101ef0 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c52c86700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c197490 con 0x7f7c4c100cf0 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c4bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 0x7f7c4c196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c4bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 0x7f7c4c196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44384/0 (socket says 172.21.15.49:44384) 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c4bfff700 1 -- 172.21.15.49:0/1166360036 learned_addr learned my addr 172.21.15.49:0/1166360036 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:51:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c50a22700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c100cf0 0x7f7c4c1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c4bfff700 1 -- 172.21.15.49:0/1166360036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c100cf0 msgr2=0x7f7c4c1967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:51:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c4bfff700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c100cf0 0x7f7c4c1967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:51:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c4bfff700 1 -- 172.21.15.49:0/1166360036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380095f0 con 0x7f7c4c101ef0 2024-05-15T15:51:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c4bfff700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 0x7f7c4c196d30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7c380092e0 tx=0x7f7c3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c49ffb700 1 -- 172.21.15.49:0/1166360036 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3801d070 con 0x7f7c4c101ef0 2024-05-15T15:51:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c49ffb700 1 -- 172.21.15.49:0/1166360036 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c3800f460 con 0x7f7c4c101ef0 2024-05-15T15:51:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4c19be90 con 0x7f7c4c101ef0 2024-05-15T15:51:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.303+0000 7f7c49ffb700 1 -- 172.21.15.49:0/1166360036 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c38021640 con 0x7f7c4c101ef0 2024-05-15T15:51:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.304+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c19c3d0 con 0x7f7c4c101ef0 2024-05-15T15:51:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.305+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4c190910 con 0x7f7c4c101ef0 2024-05-15T15:51:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.305+0000 7f7c49ffb700 1 -- 172.21.15.49:0/1166360036 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7c3800f5d0 con 0x7f7c4c101ef0 2024-05-15T15:51:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.306+0000 7f7c49ffb700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7c3c06cce0 0x7f7c3c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:51:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.306+0000 7f7c50a22700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7c3c06cce0 0x7f7c3c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:51:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.306+0000 7f7c49ffb700 1 -- 172.21.15.49:0/1166360036 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7c3808d630 con 0x7f7c4c101ef0 2024-05-15T15:51:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.306+0000 7f7c50a22700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7c3c06cce0 0x7f7c3c06f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006fd0 tx=0x7f7c40008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:51:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.311+0000 7f7c49ffb700 1 -- 172.21.15.49:0/1166360036 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c3805b430 con 0x7f7c4c101ef0 2024-05-15T15:51:32.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:51:32.535+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c4c02cfb0 con 0x7f7c3c06cce0 2024-05-15T15:51:33.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:33 smithi144 ceph-mon[36389]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-15T15:51:33.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:33 smithi049 ceph-mon[28399]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-15T15:51:34.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:34 smithi144 ceph-mon[36389]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:51:34.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:34 smithi049 ceph-mon[28399]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:51:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:35 smithi144 ceph-mon[36389]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-15T15:51:35.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:35 smithi049 ceph-mon[28399]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-15T15:51:37.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:37 smithi144 ceph-mon[36389]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-05-15T15:51:37.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:37 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:51:37.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:37 smithi049 ceph-mon[28399]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-05-15T15:51:37.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:37 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:51:39.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:39 smithi049 ceph-mon[28399]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 3 op/s 2024-05-15T15:51:39.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:39 smithi144 ceph-mon[36389]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 3 op/s 2024-05-15T15:51:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:41 smithi144 ceph-mon[36389]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-15T15:51:41.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:41 smithi049 ceph-mon[28399]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-15T15:51:43.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:43 smithi144 ceph-mon[36389]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-15T15:51:43.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:43 smithi049 ceph-mon[28399]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-15T15:51:45.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:45 smithi144 ceph-mon[36389]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-15T15:51:45.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:45 smithi049 ceph-mon[28399]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-15T15:51:47.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:47 smithi144 ceph-mon[36389]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-05-15T15:51:47.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:47 smithi049 ceph-mon[28399]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-05-15T15:51:49.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:49 smithi049 ceph-mon[28399]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T15:51:49.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:49 smithi144 ceph-mon[36389]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T15:51:51.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:51 smithi144 ceph-mon[36389]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-15T15:51:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:51 smithi049 ceph-mon[28399]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-15T15:51:52.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:52 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:51:52.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:52 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:51:53.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:53 smithi144 ceph-mon[36389]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:51:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:53 smithi049 ceph-mon[28399]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:51:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:55 smithi144 ceph-mon[36389]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:51:55.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:55 smithi049 ceph-mon[28399]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:51:57.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:57 smithi144 ceph-mon[36389]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:51:57.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:57 smithi049 ceph-mon[28399]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:51:59.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:51:59 smithi144 ceph-mon[36389]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:51:59.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:51:59 smithi049 ceph-mon[28399]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:01.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.347+0000 7f7c49ffb700 1 -- 172.21.15.49:0/1166360036 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f7c4c02cfb0 con 0x7f7c3c06cce0 2024-05-15T15:52:01.350 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:01 smithi049 ceph-mon[28399]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:52:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7c3c06cce0 msgr2=0x7f7c3c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7c3c06cce0 0x7f7c3c06f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006fd0 tx=0x7f7c40008040 comp rx=0 tx=0).stop 2024-05-15T15:52:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 msgr2=0x7f7c4c196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 0x7f7c4c196d30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7c380092e0 tx=0x7f7c3800be20 comp rx=0 tx=0).stop 2024-05-15T15:52:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 shutdown_connections 2024-05-15T15:52:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7c3c06cce0 0x7f7c3c06f1a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c100cf0 0x7f7c4c1967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 --2- 172.21.15.49:0/1166360036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c4c101ef0 0x7f7c4c196d30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 >> 172.21.15.49:0/1166360036 conn(0x7f7c4c0fc290 msgr2=0x7f7c4c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:01.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 shutdown_connections 2024-05-15T15:52:01.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:01.350+0000 7f7c52c86700 1 -- 172.21.15.49:0/1166360036 wait complete. 2024-05-15T15:52:01.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:01 smithi144 ceph-mon[36389]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:01.988 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-05-15T15:52:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:52:02.604 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:02 smithi049 ceph-mon[28399]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:52:02.604 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:52:02.604 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:52:02.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:02 smithi049 ceph-mon[28399]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:02.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:52:02.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:52:02.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:02 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:52:02.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:02 smithi144 ceph-mon[36389]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:52:02.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:52:02.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:52:02.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:02 smithi144 ceph-mon[36389]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:02.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:52:02.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:52:02.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:02 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:52:03.693 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:03 smithi049 ceph-mon[28399]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:52:03.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:03 smithi144 ceph-mon[36389]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:52:04.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:04 smithi144 ceph-mon[36389]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:04.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:04 smithi049 ceph-mon[28399]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:05.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.377+0000 7f7cf99bf700 1 -- 172.21.15.49:0/942636477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 msgr2=0x7f7cf4103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:05.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.377+0000 7f7cf99bf700 1 --2- 172.21.15.49:0/942636477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 0x7f7cf4103bd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009910 tx=0x7f7ce4009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:05.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.377+0000 7f7cf99bf700 1 -- 172.21.15.49:0/942636477 shutdown_connections 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.377+0000 7f7cf99bf700 1 --2- 172.21.15.49:0/942636477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf4104110 0x7f7cf4106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.377+0000 7f7cf99bf700 1 --2- 172.21.15.49:0/942636477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 0x7f7cf4103bd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.377+0000 7f7cf99bf700 1 -- 172.21.15.49:0/942636477 >> 172.21.15.49:0/942636477 conn(0x7f7cf40fb1d0 msgr2=0x7f7cf40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.378+0000 7f7cf99bf700 1 -- 172.21.15.49:0/942636477 shutdown_connections 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.378+0000 7f7cf99bf700 1 -- 172.21.15.49:0/942636477 wait complete. 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.378+0000 7f7cf99bf700 1 Processor -- start 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.378+0000 7f7cf99bf700 1 -- start start 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf99bf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 0x7f7cf41010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf99bf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf4104110 0x7f7cf40ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf99bf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf40ffcd0 con 0x7f7cf41017e0 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf99bf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf40ffe10 con 0x7f7cf4104110 2024-05-15T15:52:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 0x7f7cf41010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 0x7f7cf41010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42366/0 (socket says 172.21.15.49:42366) 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf2ffd700 1 -- 172.21.15.49:0/3017569685 learned_addr learned my addr 172.21.15.49:0/3017569685 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf27fc700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf4104110 0x7f7cf40ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf2ffd700 1 -- 172.21.15.49:0/3017569685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf4104110 msgr2=0x7f7cf40ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf2ffd700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf4104110 0x7f7cf40ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf2ffd700 1 -- 172.21.15.49:0/3017569685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40095f0 con 0x7f7cf41017e0 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf2ffd700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 0x7f7cf41010f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40092e0 tx=0x7f7ce400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.379+0000 7f7cf89bd700 1 -- 172.21.15.49:0/3017569685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ce401d070 con 0x7f7cf41017e0 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.380+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf4100090 con 0x7f7cf41017e0 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.380+0000 7f7cf89bd700 1 -- 172.21.15.49:0/3017569685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ce400f460 con 0x7f7cf41017e0 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.380+0000 7f7cf89bd700 1 -- 172.21.15.49:0/3017569685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ce4021620 con 0x7f7cf41017e0 2024-05-15T15:52:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.380+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf4100580 con 0x7f7cf41017e0 2024-05-15T15:52:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.381+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf404ec60 con 0x7f7cf41017e0 2024-05-15T15:52:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.381+0000 7f7cf89bd700 1 -- 172.21.15.49:0/3017569685 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7ce400f5d0 con 0x7f7cf41017e0 2024-05-15T15:52:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.382+0000 7f7cf89bd700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7cdc06cce0 0x7f7cdc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.382+0000 7f7cf89bd700 1 -- 172.21.15.49:0/3017569685 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7ce408d4c0 con 0x7f7cf41017e0 2024-05-15T15:52:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.382+0000 7f7cf27fc700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7cdc06cce0 0x7f7cdc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:05.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.383+0000 7f7cf27fc700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7cdc06cce0 0x7f7cdc06f1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8005950 tx=0x7f7ce800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:05.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.387+0000 7f7cf89bd700 1 -- 172.21.15.49:0/3017569685 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ce405b2c0 con 0x7f7cf41017e0 2024-05-15T15:52:05.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.610+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cf4105fc0 con 0x7f7cdc06cce0 2024-05-15T15:52:05.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.612+0000 7f7cf89bd700 1 -- 172.21.15.49:0/3017569685 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f7cf4105fc0 con 0x7f7cdc06cce0 2024-05-15T15:52:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.614+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7cdc06cce0 msgr2=0x7f7cdc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.614+0000 7f7cf99bf700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7cdc06cce0 0x7f7cdc06f1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8005950 tx=0x7f7ce800b410 comp rx=0 tx=0).stop 2024-05-15T15:52:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.614+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 msgr2=0x7f7cf41010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.614+0000 7f7cf99bf700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 0x7f7cf41010f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40092e0 tx=0x7f7ce400bd70 comp rx=0 tx=0).stop 2024-05-15T15:52:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.614+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 shutdown_connections 2024-05-15T15:52:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.614+0000 7f7cf99bf700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cf41017e0 0x7f7cf41010f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.614+0000 7f7cf99bf700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f7cdc06cce0 0x7f7cdc06f1a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.614+0000 7f7cf99bf700 1 --2- 172.21.15.49:0/3017569685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf4104110 0x7f7cf40ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.615+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 >> 172.21.15.49:0/3017569685 conn(0x7f7cf40fb1d0 msgr2=0x7f7cf40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.615+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 shutdown_connections 2024-05-15T15:52:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.615+0000 7f7cf99bf700 1 -- 172.21.15.49:0/3017569685 wait complete. 2024-05-15T15:52:05.627 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:52:05.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.727+0000 7f6262ba6700 1 -- 172.21.15.49:0/4292084669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 msgr2=0x7f625c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:05.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.727+0000 7f6262ba6700 1 --2- 172.21.15.49:0/4292084669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 0x7f625c1023d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6244009910 tx=0x7f6244009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:05.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.727+0000 7f6262ba6700 1 -- 172.21.15.49:0/4292084669 shutdown_connections 2024-05-15T15:52:05.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.727+0000 7f6262ba6700 1 --2- 172.21.15.49:0/4292084669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 0x7f625c1023d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.727+0000 7f6262ba6700 1 --2- 172.21.15.49:0/4292084669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f625c0fef90 0x7f625c0ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.727+0000 7f6262ba6700 1 -- 172.21.15.49:0/4292084669 >> 172.21.15.49:0/4292084669 conn(0x7f625c0fa8f0 msgr2=0x7f625c0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:05.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.727+0000 7f6262ba6700 1 -- 172.21.15.49:0/4292084669 shutdown_connections 2024-05-15T15:52:05.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.727+0000 7f6262ba6700 1 -- 172.21.15.49:0/4292084669 wait complete. 2024-05-15T15:52:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.728+0000 7f6262ba6700 1 Processor -- start 2024-05-15T15:52:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.728+0000 7f6262ba6700 1 -- start start 2024-05-15T15:52:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.728+0000 7f6262ba6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f625c0fef90 0x7f625c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.728+0000 7f6262ba6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 0x7f625c198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.728+0000 7f6262ba6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c199480 con 0x7f625c101f70 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.728+0000 7f6262ba6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c1995c0 con 0x7f625c0fef90 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f625bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 0x7f625c198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f625bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 0x7f625c198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42394/0 (socket says 172.21.15.49:42394) 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f625bfff700 1 -- 172.21.15.49:0/1973258922 learned_addr learned my addr 172.21.15.49:0/1973258922 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f6260942700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f625c0fef90 0x7f625c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f625bfff700 1 -- 172.21.15.49:0/1973258922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f625c0fef90 msgr2=0x7f625c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f625bfff700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f625c0fef90 0x7f625c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f625bfff700 1 -- 172.21.15.49:0/1973258922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440095f0 con 0x7f625c101f70 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f625bfff700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 0x7f625c198e60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6244006010 tx=0x7f624400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f6259ffb700 1 -- 172.21.15.49:0/1973258922 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f624401d070 con 0x7f625c101f70 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.729+0000 7f6259ffb700 1 -- 172.21.15.49:0/1973258922 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f624400f460 con 0x7f625c101f70 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.730+0000 7f6259ffb700 1 -- 172.21.15.49:0/1973258922 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6244021620 con 0x7f625c101f70 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.730+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625c19dfc0 con 0x7f625c101f70 2024-05-15T15:52:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.730+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625c19e480 con 0x7f625c101f70 2024-05-15T15:52:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.731+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625c106940 con 0x7f625c101f70 2024-05-15T15:52:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.731+0000 7f6259ffb700 1 -- 172.21.15.49:0/1973258922 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f624400f5d0 con 0x7f625c101f70 2024-05-15T15:52:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.732+0000 7f6259ffb700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f624806cce0 0x7f624806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.732+0000 7f6259ffb700 1 -- 172.21.15.49:0/1973258922 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f624408d420 con 0x7f625c101f70 2024-05-15T15:52:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.732+0000 7f6260942700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f624806cce0 0x7f624806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.733+0000 7f6260942700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f624806cce0 0x7f624806f1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f625c068480 tx=0x7f624c0091c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:05.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.737+0000 7f6259ffb700 1 -- 172.21.15.49:0/1973258922 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f624405b2a0 con 0x7f625c101f70 2024-05-15T15:52:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:52:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:05 smithi049 ceph-mon[28399]: Upgrade: Target is version 19.0.0-2576-gde5210d5 (unknown) 2024-05-15T15:52:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:05 smithi049 ceph-mon[28399]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a'] 2024-05-15T15:52:05.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:52:05.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:05 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:52:05.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:05 smithi049 ceph-mon[28399]: Upgrade: Need to upgrade myself (mgr.smithi049.vuhzgf) 2024-05-15T15:52:05.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:05 smithi049 ceph-mon[28399]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a on smithi144 2024-05-15T15:52:05.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.980+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f625c02cf10 con 0x7f624806cce0 2024-05-15T15:52:05.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.981+0000 7f6259ffb700 1 -- 172.21.15.49:0/1973258922 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f625c02cf10 con 0x7f624806cce0 2024-05-15T15:52:05.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f624806cce0 msgr2=0x7f624806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f624806cce0 0x7f624806f1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f625c068480 tx=0x7f624c0091c0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 msgr2=0x7f625c198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 0x7f625c198e60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6244006010 tx=0x7f624400bd70 comp rx=0 tx=0).stop 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 shutdown_connections 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f624806cce0 0x7f624806f1a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f625c0fef90 0x7f625c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 --2- 172.21.15.49:0/1973258922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625c101f70 0x7f625c198e60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 >> 172.21.15.49:0/1973258922 conn(0x7f625c0fa8f0 msgr2=0x7f625c1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 shutdown_connections 2024-05-15T15:52:05.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:05.983+0000 7f6262ba6700 1 -- 172.21.15.49:0/1973258922 wait complete. 2024-05-15T15:52:06.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.093+0000 7fe2d9950700 1 -- 172.21.15.49:0/637005852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 msgr2=0x7fe2d41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.093+0000 7fe2d9950700 1 --2- 172.21.15.49:0/637005852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 0x7fe2d41012b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009910 tx=0x7fe2c8009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:06.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.093+0000 7fe2d9950700 1 -- 172.21.15.49:0/637005852 shutdown_connections 2024-05-15T15:52:06.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.093+0000 7fe2d9950700 1 --2- 172.21.15.49:0/637005852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 0x7fe2d41012b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.093+0000 7fe2d9950700 1 --2- 172.21.15.49:0/637005852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2d40ffc50 0x7fe2d4100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.093+0000 7fe2d9950700 1 -- 172.21.15.49:0/637005852 >> 172.21.15.49:0/637005852 conn(0x7fe2d40fb1d0 msgr2=0x7fe2d40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:06.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.093+0000 7fe2d9950700 1 -- 172.21.15.49:0/637005852 shutdown_connections 2024-05-15T15:52:06.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.093+0000 7fe2d9950700 1 -- 172.21.15.49:0/637005852 wait complete. 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.094+0000 7fe2d9950700 1 Processor -- start 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.094+0000 7fe2d9950700 1 -- start start 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d9950700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2d40ffc50 0x7fe2d4069140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d9950700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 0x7fe2d419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d9950700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d419ba50 con 0x7fe2d4100e50 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d9950700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d419bb90 con 0x7fe2d40ffc50 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 0x7fe2d419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 0x7fe2d419b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42416/0 (socket says 172.21.15.49:42416) 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d2ffd700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2d40ffc50 0x7fe2d4069140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d27fc700 1 -- 172.21.15.49:0/1285266109 learned_addr learned my addr 172.21.15.49:0/1285266109 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:06.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d27fc700 1 -- 172.21.15.49:0/1285266109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2d40ffc50 msgr2=0x7fe2d4069140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d27fc700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2d40ffc50 0x7fe2d4069140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d27fc700 1 -- 172.21.15.49:0/1285266109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80095f0 con 0x7fe2d4100e50 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d2ffd700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2d40ffc50 0x7fe2d4069140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d27fc700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 0x7fe2d419b510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8005b40 tx=0x7fe2c800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.095+0000 7fe2d894e700 1 -- 172.21.15.49:0/1285266109 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c801d070 con 0x7fe2d4100e50 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.096+0000 7fe2d894e700 1 -- 172.21.15.49:0/1285266109 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2c80050d0 con 0x7fe2d4100e50 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.096+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d419bde0 con 0x7fe2d4100e50 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.096+0000 7fe2d894e700 1 -- 172.21.15.49:0/1285266109 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c8022470 con 0x7fe2d4100e50 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.096+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d419c2d0 con 0x7fe2d4100e50 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.097+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d418f7d0 con 0x7fe2d4100e50 2024-05-15T15:52:06.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.097+0000 7fe2d894e700 1 -- 172.21.15.49:0/1285266109 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe2c80037f0 con 0x7fe2d4100e50 2024-05-15T15:52:06.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.098+0000 7fe2d894e700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe2bc06ccf0 0x7fe2bc06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:06.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.098+0000 7fe2d894e700 1 -- 172.21.15.49:0/1285266109 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe2c808d800 con 0x7fe2d4100e50 2024-05-15T15:52:06.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.098+0000 7fe2d2ffd700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe2bc06ccf0 0x7fe2bc06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:06.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.098+0000 7fe2d2ffd700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe2bc06ccf0 0x7fe2bc06f1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4100cb0 tx=0x7fe2c4009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:06.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.103+0000 7fe2d894e700 1 -- 172.21.15.49:0/1285266109 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe2c805b600 con 0x7fe2d4100e50 2024-05-15T15:52:06.119 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:05 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:52:06.119 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:05 smithi144 ceph-mon[36389]: Upgrade: Target is version 19.0.0-2576-gde5210d5 (unknown) 2024-05-15T15:52:06.119 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:05 smithi144 ceph-mon[36389]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a'] 2024-05-15T15:52:06.120 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:05 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:52:06.120 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:05 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:52:06.120 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:05 smithi144 ceph-mon[36389]: Upgrade: Need to upgrade myself (mgr.smithi049.vuhzgf) 2024-05-15T15:52:06.120 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:05 smithi144 ceph-mon[36389]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a on smithi144 2024-05-15T15:52:06.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.317+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2d402ceb0 con 0x7fe2bc06ccf0 2024-05-15T15:52:06.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.329+0000 7fe2d894e700 1 -- 172.21.15.49:0/1285266109 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe2d402ceb0 con 0x7fe2bc06ccf0 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (5m) 55s ago 6m 24.1M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (7m) 55s ago 7m 7528k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (6m) 60s ago 6m 7424k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (7m) 55s ago 7m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (5m) 60s ago 5m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (5m) 55s ago 6m 85.4M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (81s) 55s ago 81s 13.0M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (74s) 55s ago 74s 16.5M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (79s) 60s ago 78s 16.2M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (71s) 60s ago 71s 17.1M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:9283,8765,8443 running (8m) 55s ago 8m 499M - 18.2.2-1343-g2965081f a04a73ffe079 ea7d1a4b9c55 2024-05-15T15:52:06.331 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (5m) 60s ago 5m 452M - 18.2.2-1343-g2965081f a04a73ffe079 eece4ed6d244 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (8m) 55s ago 8m 45.0M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (5m) 60s ago 5m 38.6M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (7m) 55s ago 7m 14.1M - 1.5.0 0da6a335fe13 171e0a34f565 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (5m) 60s ago 5m 15.4M - 1.5.0 0da6a335fe13 be3a2837bc85 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (5m) 55s ago 5m 53.8M 3946M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (4m) 55s ago 4m 47.9M 3946M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (4m) 55s ago 4m 52.2M 3946M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (3m) 60s ago 3m 52.4M 4970M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (3m) 60s ago 3m 44.5M 4970M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (2m) 60s ago 2m 52.6M 4970M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:52:06.332 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (5m) 55s ago 6m 57.3M - 2.43.0 a07b618ecd1d 0299663ce1de 2024-05-15T15:52:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe2bc06ccf0 msgr2=0x7fe2bc06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe2bc06ccf0 0x7fe2bc06f1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4100cb0 tx=0x7fe2c4009260 comp rx=0 tx=0).stop 2024-05-15T15:52:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 msgr2=0x7fe2d419b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 0x7fe2d419b510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8005b40 tx=0x7fe2c800f690 comp rx=0 tx=0).stop 2024-05-15T15:52:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 shutdown_connections 2024-05-15T15:52:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe2bc06ccf0 0x7fe2bc06f1b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2d40ffc50 0x7fe2d4069140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 --2- 172.21.15.49:0/1285266109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2d4100e50 0x7fe2d419b510 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 >> 172.21.15.49:0/1285266109 conn(0x7fe2d40fb1d0 msgr2=0x7fe2d4104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:06.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 shutdown_connections 2024-05-15T15:52:06.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.331+0000 7fe2d9950700 1 -- 172.21.15.49:0/1285266109 wait complete. 2024-05-15T15:52:06.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.439+0000 7f081b5d8700 1 -- 172.21.15.49:0/844458570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814073a00 msgr2=0x7f0814073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.439+0000 7f081b5d8700 1 --2- 172.21.15.49:0/844458570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814073a00 0x7f0814073e80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0804009910 tx=0x7f0804009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:06.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.439+0000 7f081b5d8700 1 -- 172.21.15.49:0/844458570 shutdown_connections 2024-05-15T15:52:06.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.439+0000 7f081b5d8700 1 --2- 172.21.15.49:0/844458570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814073a00 0x7f0814073e80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.439+0000 7f081b5d8700 1 --2- 172.21.15.49:0/844458570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0814074fd0 0x7f0814073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.439+0000 7f081b5d8700 1 -- 172.21.15.49:0/844458570 >> 172.21.15.49:0/844458570 conn(0x7f08140fc2c0 msgr2=0x7f08140fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:06.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.439+0000 7f081b5d8700 1 -- 172.21.15.49:0/844458570 shutdown_connections 2024-05-15T15:52:06.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.439+0000 7f081b5d8700 1 -- 172.21.15.49:0/844458570 wait complete. 2024-05-15T15:52:06.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.439+0000 7f081b5d8700 1 Processor -- start 2024-05-15T15:52:06.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f081b5d8700 1 -- start start 2024-05-15T15:52:06.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f081b5d8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0814073a00 0x7f0814198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:06.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f081b5d8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814074fd0 0x7f0814198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:06.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f081b5d8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814199470 con 0x7f0814074fd0 2024-05-15T15:52:06.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f081b5d8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141995b0 con 0x7f0814073a00 2024-05-15T15:52:06.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f0818b73700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814074fd0 0x7f0814198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f0818b73700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814074fd0 0x7f0814198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42440/0 (socket says 172.21.15.49:42440) 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f0818b73700 1 -- 172.21.15.49:0/1140586952 learned_addr learned my addr 172.21.15.49:0/1140586952 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f0819374700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0814073a00 0x7f0814198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f0818b73700 1 -- 172.21.15.49:0/1140586952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0814073a00 msgr2=0x7f0814198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f0818b73700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0814073a00 0x7f0814198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f0818b73700 1 -- 172.21.15.49:0/1140586952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08040095f0 con 0x7f0814074fd0 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.440+0000 7f0818b73700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814074fd0 0x7f0814198e50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0804005f00 tx=0x7f080400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.441+0000 7f080a7fc700 1 -- 172.21.15.49:0/1140586952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f080401d070 con 0x7f0814074fd0 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.441+0000 7f080a7fc700 1 -- 172.21.15.49:0/1140586952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08040052a0 con 0x7f0814074fd0 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.441+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081419dfb0 con 0x7f0814074fd0 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.441+0000 7f080a7fc700 1 -- 172.21.15.49:0/1140586952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0804017400 con 0x7f0814074fd0 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.441+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081419e4f0 con 0x7f0814074fd0 2024-05-15T15:52:06.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.442+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0814192ab0 con 0x7f0814074fd0 2024-05-15T15:52:06.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.442+0000 7f080a7fc700 1 -- 172.21.15.49:0/1140586952 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0804003680 con 0x7f0814074fd0 2024-05-15T15:52:06.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.443+0000 7f080a7fc700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f080006cce0 0x7f080006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:06.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.443+0000 7f0819374700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f080006cce0 0x7f080006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:06.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.443+0000 7f080a7fc700 1 -- 172.21.15.49:0/1140586952 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f080408d640 con 0x7f0814074fd0 2024-05-15T15:52:06.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.443+0000 7f0819374700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f080006cce0 0x7f080006f1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f08140753b0 tx=0x7f0810009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:06.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.448+0000 7f080a7fc700 1 -- 172.21.15.49:0/1140586952 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f080405b440 con 0x7f0814074fd0 2024-05-15T15:52:06.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.738+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f081402cda0 con 0x7f0814074fd0 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.739+0000 7f080a7fc700 1 -- 172.21.15.49:0/1140586952 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f080405afd0 con 0x7f0814074fd0 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:52:06.741 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:52:06.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:52:06.742 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:52:06.742 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:52:06.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:52:06.742 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:52:06.742 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 14 2024-05-15T15:52:06.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:52:06.742 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:52:06.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f080006cce0 msgr2=0x7f080006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f080006cce0 0x7f080006f1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f08140753b0 tx=0x7f0810009260 comp rx=0 tx=0).stop 2024-05-15T15:52:06.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814074fd0 msgr2=0x7f0814198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814074fd0 0x7f0814198e50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0804005f00 tx=0x7f080400b5e0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 shutdown_connections 2024-05-15T15:52:06.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f080006cce0 0x7f080006f1a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0814073a00 0x7f0814198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 --2- 172.21.15.49:0/1140586952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0814074fd0 0x7f0814198e50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 >> 172.21.15.49:0/1140586952 conn(0x7f08140fc2c0 msgr2=0x7f0814106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:06.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 shutdown_connections 2024-05-15T15:52:06.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.741+0000 7f081b5d8700 1 -- 172.21.15.49:0/1140586952 wait complete. 2024-05-15T15:52:06.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.850+0000 7f85a4cdb700 1 -- 172.21.15.49:0/2898732135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 msgr2=0x7f85a0100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.850+0000 7f85a4cdb700 1 --2- 172.21.15.49:0/2898732135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 0x7f85a0100070 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8590009910 tx=0x7f8590009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.850+0000 7f85a4cdb700 1 -- 172.21.15.49:0/2898732135 shutdown_connections 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.850+0000 7f85a4cdb700 1 --2- 172.21.15.49:0/2898732135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a0100e50 0x7f85a01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.850+0000 7f85a4cdb700 1 --2- 172.21.15.49:0/2898732135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 0x7f85a0100070 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.850+0000 7f85a4cdb700 1 -- 172.21.15.49:0/2898732135 >> 172.21.15.49:0/2898732135 conn(0x7f85a00fb1d0 msgr2=0x7f85a00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.850+0000 7f85a4cdb700 1 -- 172.21.15.49:0/2898732135 shutdown_connections 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.850+0000 7f85a4cdb700 1 -- 172.21.15.49:0/2898732135 wait complete. 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.851+0000 7f85a4cdb700 1 Processor -- start 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.851+0000 7f85a4cdb700 1 -- start start 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.851+0000 7f85a4cdb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 0x7f85a01967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.851+0000 7f85a4cdb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a0100e50 0x7f85a0196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.851+0000 7f85a4cdb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0197340 con 0x7f85a00ffc50 2024-05-15T15:52:06.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.851+0000 7f85a4cdb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0197480 con 0x7f85a0100e50 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f859e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 0x7f85a01967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f859e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 0x7f85a01967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42462/0 (socket says 172.21.15.49:42462) 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f859e59c700 1 -- 172.21.15.49:0/3199088614 learned_addr learned my addr 172.21.15.49:0/3199088614 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f859dd9b700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a0100e50 0x7f85a0196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f859e59c700 1 -- 172.21.15.49:0/3199088614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a0100e50 msgr2=0x7f85a0196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f859e59c700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a0100e50 0x7f85a0196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f859e59c700 1 -- 172.21.15.49:0/3199088614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85900095f0 con 0x7f85a00ffc50 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f859e59c700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 0x7f85a01967e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f85900059d0 tx=0x7f859000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f858f7fe700 1 -- 172.21.15.49:0/3199088614 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f859001d070 con 0x7f85a00ffc50 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a019be30 con 0x7f85a00ffc50 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f858f7fe700 1 -- 172.21.15.49:0/3199088614 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8590005dc0 con 0x7f85a00ffc50 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f858f7fe700 1 -- 172.21.15.49:0/3199088614 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8590021410 con 0x7f85a00ffc50 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.852+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a019c2a0 con 0x7f85a00ffc50 2024-05-15T15:52:06.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.853+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a004ec60 con 0x7f85a00ffc50 2024-05-15T15:52:06.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.854+0000 7f858f7fe700 1 -- 172.21.15.49:0/3199088614 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f859000f820 con 0x7f85a00ffc50 2024-05-15T15:52:06.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.854+0000 7f858f7fe700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f858806cce0 0x7f858806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:06.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.854+0000 7f858f7fe700 1 -- 172.21.15.49:0/3199088614 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f859008d460 con 0x7f85a00ffc50 2024-05-15T15:52:06.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.854+0000 7f859dd9b700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f858806cce0 0x7f858806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:06.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.855+0000 7f859dd9b700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f858806cce0 0x7f858806f1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8594005950 tx=0x7f859400a300 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:06.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:06.859+0000 7f858f7fe700 1 -- 172.21.15.49:0/3199088614 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f859005b2e0 con 0x7f85a00ffc50 2024-05-15T15:52:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:06 smithi049 ceph-mon[28399]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:06 smithi049 ceph-mon[28399]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:06.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:06 smithi049 ceph-mon[28399]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:06.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:52:06.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:06 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1140586952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:52:07.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.081+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85a0068e40 con 0x7f858806cce0 2024-05-15T15:52:07.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.082+0000 7f858f7fe700 1 -- 172.21.15.49:0/3199088614 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f85a0068e40 con 0x7f858806cce0 2024-05-15T15:52:07.084 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:52:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:52:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:52:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:52:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [], 2024-05-15T15:52:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "0/2 daemons upgraded", 2024-05-15T15:52:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a image on host smithi144", 2024-05-15T15:52:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:52:07.085 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:52:07.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f858806cce0 msgr2=0x7f858806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:07.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f858806cce0 0x7f858806f1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8594005950 tx=0x7f859400a300 comp rx=0 tx=0).stop 2024-05-15T15:52:07.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 msgr2=0x7f85a01967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:07.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 0x7f85a01967e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f85900059d0 tx=0x7f859000bd70 comp rx=0 tx=0).stop 2024-05-15T15:52:07.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 shutdown_connections 2024-05-15T15:52:07.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85a00ffc50 0x7f85a01967e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:07.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f858806cce0 0x7f858806f1a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:07.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 --2- 172.21.15.49:0/3199088614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a0100e50 0x7f85a0196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 >> 172.21.15.49:0/3199088614 conn(0x7f85a00fb1d0 msgr2=0x7f85a0068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 shutdown_connections 2024-05-15T15:52:07.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:07.084+0000 7f85a4cdb700 1 -- 172.21.15.49:0/3199088614 wait complete. 2024-05-15T15:52:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:06 smithi144 ceph-mon[36389]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:06 smithi144 ceph-mon[36389]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:06 smithi144 ceph-mon[36389]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:52:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:06 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1140586952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:52:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:07 smithi144 ceph-mon[36389]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:07 smithi144 ceph-mon[36389]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:08.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:07 smithi049 ceph-mon[28399]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:08.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:07 smithi049 ceph-mon[28399]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:09.160 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:08 smithi049 ceph-mon[28399]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:08 smithi144 ceph-mon[36389]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:11.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:10 smithi049 ceph-mon[28399]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:11.192 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:10 smithi144 ceph-mon[36389]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:12 smithi144 ceph-mon[36389]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:12 smithi049 ceph-mon[28399]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:15.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:14 smithi144 ceph-mon[36389]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:15.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:14 smithi049 ceph-mon[28399]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:16.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:16 smithi049 ceph-mon[28399]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:16 smithi144 ceph-mon[36389]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:18 smithi144 ceph-mon[36389]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:19.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:18 smithi049 ceph-mon[28399]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:21.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:20 smithi049 ceph-mon[28399]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:20 smithi144 ceph-mon[36389]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:22.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:21 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:52:22.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:21 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:52:23.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:22 smithi144 ceph-mon[36389]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:23.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:22 smithi049 ceph-mon[28399]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:25.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:24 smithi144 ceph-mon[36389]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:25.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:24 smithi049 ceph-mon[28399]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:27.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:26 smithi144 ceph-mon[36389]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:27.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:26 smithi049 ceph-mon[28399]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:29.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:28 smithi144 ceph-mon[36389]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:29.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:28 smithi049 ceph-mon[28399]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:31.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:30 smithi049 ceph-mon[28399]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:31.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:30 smithi144 ceph-mon[36389]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:33.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:32 smithi144 ceph-mon[36389]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:33.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:32 smithi049 ceph-mon[28399]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:35.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:34 smithi144 ceph-mon[36389]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:35.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:34 smithi049 ceph-mon[28399]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:37.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:36 smithi144 ceph-mon[36389]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:37.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:36 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:52:37.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:36 smithi049 ceph-mon[28399]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:37.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:36 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:52:37.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.250+0000 7fc745eb2700 1 -- 172.21.15.49:0/921255145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7400ffc50 msgr2=0x7fc740100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:37.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.250+0000 7fc745eb2700 1 --2- 172.21.15.49:0/921255145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7400ffc50 0x7fc740100070 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc730009910 tx=0x7fc730009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:37.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.250+0000 7fc745eb2700 1 -- 172.21.15.49:0/921255145 shutdown_connections 2024-05-15T15:52:37.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.250+0000 7fc745eb2700 1 --2- 172.21.15.49:0/921255145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc740100e50 0x7fc7401012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.250+0000 7fc745eb2700 1 --2- 172.21.15.49:0/921255145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7400ffc50 0x7fc740100070 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.250+0000 7fc745eb2700 1 -- 172.21.15.49:0/921255145 >> 172.21.15.49:0/921255145 conn(0x7fc7400fb1d0 msgr2=0x7fc7400fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:37.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.251+0000 7fc745eb2700 1 -- 172.21.15.49:0/921255145 shutdown_connections 2024-05-15T15:52:37.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.251+0000 7fc745eb2700 1 -- 172.21.15.49:0/921255145 wait complete. 2024-05-15T15:52:37.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.252+0000 7fc745eb2700 1 Processor -- start 2024-05-15T15:52:37.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.252+0000 7fc745eb2700 1 -- start start 2024-05-15T15:52:37.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.252+0000 7fc745eb2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7400ffc50 0x7fc740198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:37.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.252+0000 7fc745eb2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc740100e50 0x7fc740198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:37.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.252+0000 7fc745eb2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7401994a0 con 0x7fc7400ffc50 2024-05-15T15:52:37.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.252+0000 7fc745eb2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7401995e0 con 0x7fc740100e50 2024-05-15T15:52:37.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.253+0000 7fc73f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7400ffc50 0x7fc740198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:37.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.253+0000 7fc73effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc740100e50 0x7fc740198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.253+0000 7fc73effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc740100e50 0x7fc740198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:38032/0 (socket says 172.21.15.49:38032) 2024-05-15T15:52:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.253+0000 7fc73effd700 1 -- 172.21.15.49:0/226567767 learned_addr learned my addr 172.21.15.49:0/226567767 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.253+0000 7fc73effd700 1 -- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7400ffc50 msgr2=0x7fc740198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.253+0000 7fc73effd700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7400ffc50 0x7fc740198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.253+0000 7fc73effd700 1 -- 172.21.15.49:0/226567767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7300095f0 con 0x7fc740100e50 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.253+0000 7fc73f7fe700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7400ffc50 0x7fc740198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.253+0000 7fc73effd700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc740100e50 0x7fc740198e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc73400e920 tx=0x7fc73400ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.254+0000 7fc73cff9700 1 -- 172.21.15.49:0/226567767 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc73400c9d0 con 0x7fc740100e50 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.254+0000 7fc73cff9700 1 -- 172.21.15.49:0/226567767 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc734010430 con 0x7fc740100e50 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.254+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74019e040 con 0x7fc740100e50 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.254+0000 7fc73cff9700 1 -- 172.21.15.49:0/226567767 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc734018650 con 0x7fc740100e50 2024-05-15T15:52:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.254+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74019e510 con 0x7fc740100e50 2024-05-15T15:52:37.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.255+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74018e6c0 con 0x7fc740100e50 2024-05-15T15:52:37.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.256+0000 7fc73cff9700 1 -- 172.21.15.49:0/226567767 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc7340105a0 con 0x7fc740100e50 2024-05-15T15:52:37.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.257+0000 7fc73cff9700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc72806cce0 0x7fc72806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:37.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.257+0000 7fc73cff9700 1 -- 172.21.15.49:0/226567767 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc734014070 con 0x7fc740100e50 2024-05-15T15:52:37.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.257+0000 7fc73f7fe700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc72806cce0 0x7fc72806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:37.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.258+0000 7fc73f7fe700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc72806cce0 0x7fc72806f1a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc730018040 tx=0x7fc730000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:37.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.266+0000 7fc73cff9700 1 -- 172.21.15.49:0/226567767 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc734052d60 con 0x7fc740100e50 2024-05-15T15:52:37.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.569+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc74002ceb0 con 0x7fc72806cce0 2024-05-15T15:52:37.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.571+0000 7fc73cff9700 1 -- 172.21.15.49:0/226567767 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fc74002ceb0 con 0x7fc72806cce0 2024-05-15T15:52:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc72806cce0 msgr2=0x7fc72806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc72806cce0 0x7fc72806f1a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc730018040 tx=0x7fc730000bc0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc740100e50 msgr2=0x7fc740198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc740100e50 0x7fc740198e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc73400e920 tx=0x7fc73400ec30 comp rx=0 tx=0).stop 2024-05-15T15:52:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 shutdown_connections 2024-05-15T15:52:37.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7400ffc50 0x7fc740198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc72806cce0 0x7fc72806f1a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 --2- 172.21.15.49:0/226567767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc740100e50 0x7fc740198e80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 >> 172.21.15.49:0/226567767 conn(0x7fc7400fb1d0 msgr2=0x7fc7401044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 shutdown_connections 2024-05-15T15:52:37.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.574+0000 7fc745eb2700 1 -- 172.21.15.49:0/226567767 wait complete. 2024-05-15T15:52:37.588 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:52:37.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.685+0000 7fc6f41ec700 1 -- 172.21.15.49:0/892661042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 msgr2=0x7fc6ec106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:37.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.685+0000 7fc6f41ec700 1 --2- 172.21.15.49:0/892661042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 0x7fc6ec106560 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009910 tx=0x7fc6dc009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:37.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.686+0000 7fc6f41ec700 1 -- 172.21.15.49:0/892661042 shutdown_connections 2024-05-15T15:52:37.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.686+0000 7fc6f41ec700 1 --2- 172.21.15.49:0/892661042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 0x7fc6ec106560 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.686+0000 7fc6f41ec700 1 --2- 172.21.15.49:0/892661042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ec101840 0x7fc6ec103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.686+0000 7fc6f41ec700 1 -- 172.21.15.49:0/892661042 >> 172.21.15.49:0/892661042 conn(0x7fc6ec0fb1d0 msgr2=0x7fc6ec0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.686+0000 7fc6f41ec700 1 -- 172.21.15.49:0/892661042 shutdown_connections 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.686+0000 7fc6f41ec700 1 -- 172.21.15.49:0/892661042 wait complete. 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f41ec700 1 Processor -- start 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f41ec700 1 -- start start 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f41ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ec101840 0x7fc6ec1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f41ec700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 0x7fc6ec196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f41ec700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec197220 con 0x7fc6ec104170 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f41ec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec197360 con 0x7fc6ec101840 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f1f88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ec101840 0x7fc6ec1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f1787700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 0x7fc6ec196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f1f88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ec101840 0x7fc6ec1966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:38044/0 (socket says 172.21.15.49:38044) 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f1f88700 1 -- 172.21.15.49:0/1589467427 learned_addr learned my addr 172.21.15.49:0/1589467427 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.687+0000 7fc6f1787700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 0x7fc6ec196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51866/0 (socket says 172.21.15.49:51866) 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6f1f88700 1 -- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 msgr2=0x7fc6ec196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6f1f88700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 0x7fc6ec196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6f1f88700 1 -- 172.21.15.49:0/1589467427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc0095f0 con 0x7fc6ec101840 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6f1787700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 0x7fc6ec196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6f1f88700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ec101840 0x7fc6ec1966c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800e920 tx=0x7fc6e800ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6e2ffd700 1 -- 172.21.15.49:0/1589467427 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e800c9d0 con 0x7fc6ec101840 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6e2ffd700 1 -- 172.21.15.49:0/1589467427 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6e8010430 con 0x7fc6ec101840 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec19bdc0 con 0x7fc6ec101840 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6e2ffd700 1 -- 172.21.15.49:0/1589467427 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e8018630 con 0x7fc6ec101840 2024-05-15T15:52:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.688+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec19c240 con 0x7fc6ec101840 2024-05-15T15:52:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.689+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ec04ec60 con 0x7fc6ec101840 2024-05-15T15:52:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.690+0000 7fc6e2ffd700 1 -- 172.21.15.49:0/1589467427 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc6e80105a0 con 0x7fc6ec101840 2024-05-15T15:52:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.690+0000 7fc6e2ffd700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc6d806cce0 0x7fc6d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.691+0000 7fc6e2ffd700 1 -- 172.21.15.49:0/1589467427 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc6e8014070 con 0x7fc6ec101840 2024-05-15T15:52:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.691+0000 7fc6f1787700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc6d806cce0 0x7fc6d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.691+0000 7fc6f1787700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc6d806cce0 0x7fc6d806f1a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00b800 tx=0x7fc6dc009f70 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.699+0000 7fc6e2ffd700 1 -- 172.21.15.49:0/1589467427 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc6e8052d60 con 0x7fc6ec101840 2024-05-15T15:52:37.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.930+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6ec106000 con 0x7fc6d806cce0 2024-05-15T15:52:37.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.932+0000 7fc6e2ffd700 1 -- 172.21.15.49:0/1589467427 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fc6ec106000 con 0x7fc6d806cce0 2024-05-15T15:52:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.935+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc6d806cce0 msgr2=0x7fc6d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.935+0000 7fc6f41ec700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc6d806cce0 0x7fc6d806f1a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00b800 tx=0x7fc6dc009f70 comp rx=0 tx=0).stop 2024-05-15T15:52:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.935+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ec101840 msgr2=0x7fc6ec1966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.935+0000 7fc6f41ec700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ec101840 0x7fc6ec1966c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800e920 tx=0x7fc6e800ec30 comp rx=0 tx=0).stop 2024-05-15T15:52:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.935+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 shutdown_connections 2024-05-15T15:52:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.935+0000 7fc6f41ec700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fc6d806cce0 0x7fc6d806f1a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.936+0000 7fc6f41ec700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ec101840 0x7fc6ec1966c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.936+0000 7fc6f41ec700 1 --2- 172.21.15.49:0/1589467427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6ec104170 0x7fc6ec196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.936+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 >> 172.21.15.49:0/1589467427 conn(0x7fc6ec0fb1d0 msgr2=0x7fc6ec0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:37.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.936+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 shutdown_connections 2024-05-15T15:52:37.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:37.936+0000 7fc6f41ec700 1 -- 172.21.15.49:0/1589467427 wait complete. 2024-05-15T15:52:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.046+0000 7f3466914700 1 -- 172.21.15.49:0/4273849798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3460101760 msgr2=0x7f3460103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.046+0000 7f3466914700 1 --2- 172.21.15.49:0/4273849798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3460101760 0x7f3460103b50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3450009910 tx=0x7f3450009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.046+0000 7f3466914700 1 -- 172.21.15.49:0/4273849798 shutdown_connections 2024-05-15T15:52:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.046+0000 7f3466914700 1 --2- 172.21.15.49:0/4273849798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3460104090 0x7f3460106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.046+0000 7f3466914700 1 --2- 172.21.15.49:0/4273849798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3460101760 0x7f3460103b50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.046+0000 7f3466914700 1 -- 172.21.15.49:0/4273849798 >> 172.21.15.49:0/4273849798 conn(0x7f34600fb170 msgr2=0x7f34600fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.046+0000 7f3466914700 1 -- 172.21.15.49:0/4273849798 shutdown_connections 2024-05-15T15:52:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.046+0000 7f3466914700 1 -- 172.21.15.49:0/4273849798 wait complete. 2024-05-15T15:52:38.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.047+0000 7f3466914700 1 Processor -- start 2024-05-15T15:52:38.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.047+0000 7f3466914700 1 -- start start 2024-05-15T15:52:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f3466914700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3460101760 0x7f3460101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f3466914700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3460104090 0x7f34600ff710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f3466914700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34600ffc50 con 0x7f3460101760 2024-05-15T15:52:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f3466914700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34600ffd90 con 0x7f3460104090 2024-05-15T15:52:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f345f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3460104090 0x7f34600ff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f345f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3460104090 0x7f34600ff710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:38060/0 (socket says 172.21.15.49:38060) 2024-05-15T15:52:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f345f7fe700 1 -- 172.21.15.49:0/570472170 learned_addr learned my addr 172.21.15.49:0/570472170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f345ffff700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3460101760 0x7f3460101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f345f7fe700 1 -- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3460101760 msgr2=0x7f3460101070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f345f7fe700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3460101760 0x7f3460101070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.048+0000 7f345f7fe700 1 -- 172.21.15.49:0/570472170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500095f0 con 0x7f3460104090 2024-05-15T15:52:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.049+0000 7f345f7fe700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3460104090 0x7f34600ff710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f345400d8d0 tx=0x7f345400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.049+0000 7f345d7fa700 1 -- 172.21.15.49:0/570472170 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3454009940 con 0x7f3460104090 2024-05-15T15:52:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.049+0000 7f345d7fa700 1 -- 172.21.15.49:0/570472170 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3454010460 con 0x7f3460104090 2024-05-15T15:52:38.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.049+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3460100070 con 0x7f3460104090 2024-05-15T15:52:38.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.049+0000 7f345d7fa700 1 -- 172.21.15.49:0/570472170 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f345400f5f0 con 0x7f3460104090 2024-05-15T15:52:38.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.049+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3460100580 con 0x7f3460104090 2024-05-15T15:52:38.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.050+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346004ec60 con 0x7f3460104090 2024-05-15T15:52:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.051+0000 7f345d7fa700 1 -- 172.21.15.49:0/570472170 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f34540105d0 con 0x7f3460104090 2024-05-15T15:52:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.052+0000 7f345d7fa700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f344806cce0 0x7f344806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.052+0000 7f345d7fa700 1 -- 172.21.15.49:0/570472170 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f345408c120 con 0x7f3460104090 2024-05-15T15:52:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.052+0000 7f345ffff700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f344806cce0 0x7f344806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.053+0000 7f345ffff700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f344806cce0 0x7f344806f1a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f34500051c0 tx=0x7f3450005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:38.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.061+0000 7f345d7fa700 1 -- 172.21.15.49:0/570472170 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3454059f20 con 0x7f3460104090 2024-05-15T15:52:38.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.275+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3460100580 con 0x7f344806cce0 2024-05-15T15:52:38.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.295+0000 7f345d7fa700 1 -- 172.21.15.49:0/570472170 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3864 (secure 0 0 0) 0x7f3460100580 con 0x7f344806cce0 2024-05-15T15:52:38.297 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (5m) 87s ago 7m 24.1M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (7m) 87s ago 7m 7528k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (6m) 92s ago 6m 7424k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (7m) 87s ago 7m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (6m) 92s ago 6m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (5m) 87s ago 7m 85.4M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (113s) 87s ago 113s 13.0M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (106s) 87s ago 106s 16.5M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (111s) 92s ago 110s 16.2M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (103s) 92s ago 103s 17.1M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:9283,8765,8443 running (8m) 87s ago 8m 499M - 18.2.2-1343-g2965081f a04a73ffe079 ea7d1a4b9c55 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (6m) 92s ago 6m 452M - 18.2.2-1343-g2965081f a04a73ffe079 eece4ed6d244 2024-05-15T15:52:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (8m) 87s ago 9m 45.0M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (6m) 92s ago 6m 38.6M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (7m) 87s ago 7m 14.1M - 1.5.0 0da6a335fe13 171e0a34f565 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (6m) 92s ago 6m 15.4M - 1.5.0 0da6a335fe13 be3a2837bc85 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (5m) 87s ago 5m 53.8M 3946M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (5m) 87s ago 5m 47.9M 3946M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (4m) 87s ago 4m 52.2M 3946M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (4m) 92s ago 4m 52.4M 4970M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (3m) 92s ago 3m 44.5M 4970M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (3m) 92s ago 3m 52.6M 4970M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:52:38.299 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (5m) 87s ago 6m 57.3M - 2.43.0 a07b618ecd1d 0299663ce1de 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.297+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f344806cce0 msgr2=0x7f344806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.297+0000 7f3466914700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f344806cce0 0x7f344806f1a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f34500051c0 tx=0x7f3450005a90 comp rx=0 tx=0).stop 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.297+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3460104090 msgr2=0x7f34600ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.297+0000 7f3466914700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3460104090 0x7f34600ff710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f345400d8d0 tx=0x7f345400dc90 comp rx=0 tx=0).stop 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.298+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 shutdown_connections 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.298+0000 7f3466914700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3460101760 0x7f3460101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.298+0000 7f3466914700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f344806cce0 0x7f344806f1a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.298+0000 7f3466914700 1 --2- 172.21.15.49:0/570472170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3460104090 0x7f34600ff710 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.298+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 >> 172.21.15.49:0/570472170 conn(0x7f34600fb170 msgr2=0x7f34600fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.298+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 shutdown_connections 2024-05-15T15:52:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.298+0000 7f3466914700 1 -- 172.21.15.49:0/570472170 wait complete. 2024-05-15T15:52:38.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.415+0000 7f4264c43700 1 -- 172.21.15.49:0/1677673140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260101250 msgr2=0x7f4260101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.415+0000 7f4264c43700 1 --2- 172.21.15.49:0/1677673140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260101250 0x7f4260101670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4250009910 tx=0x7f4250009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:38.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.415+0000 7f4264c43700 1 -- 172.21.15.49:0/1677673140 shutdown_connections 2024-05-15T15:52:38.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.415+0000 7f4264c43700 1 --2- 172.21.15.49:0/1677673140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42600fed50 0x7f42600ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.415+0000 7f4264c43700 1 --2- 172.21.15.49:0/1677673140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260101250 0x7f4260101670 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.415+0000 7f4264c43700 1 -- 172.21.15.49:0/1677673140 >> 172.21.15.49:0/1677673140 conn(0x7f42600fa8f0 msgr2=0x7f42600fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.415+0000 7f4264c43700 1 -- 172.21.15.49:0/1677673140 shutdown_connections 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.415+0000 7f4264c43700 1 -- 172.21.15.49:0/1677673140 wait complete. 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.416+0000 7f4264c43700 1 Processor -- start 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.416+0000 7f4264c43700 1 -- start start 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.416+0000 7f4264c43700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42600fed50 0x7f42601988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.416+0000 7f4264c43700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4260101250 0x7f4260198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.416+0000 7f4264c43700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260199450 con 0x7f42600fed50 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.416+0000 7f4264c43700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260199590 con 0x7f4260101250 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.417+0000 7f425e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42600fed50 0x7f42601988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.417+0000 7f425e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42600fed50 0x7f42601988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51894/0 (socket says 172.21.15.49:51894) 2024-05-15T15:52:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.417+0000 7f425e59c700 1 -- 172.21.15.49:0/4244377836 learned_addr learned my addr 172.21.15.49:0/4244377836 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:38.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.417+0000 7f425dd9b700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4260101250 0x7f4260198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:38.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.417+0000 7f425e59c700 1 -- 172.21.15.49:0/4244377836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4260101250 msgr2=0x7f4260198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.417+0000 7f425e59c700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4260101250 0x7f4260198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.417+0000 7f425e59c700 1 -- 172.21.15.49:0/4244377836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42500095f0 con 0x7f42600fed50 2024-05-15T15:52:38.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.417+0000 7f425e59c700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42600fed50 0x7f42601988f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f425000b800 tx=0x7f425000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:38.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.418+0000 7f424f7fe700 1 -- 172.21.15.49:0/4244377836 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f425001d070 con 0x7f42600fed50 2024-05-15T15:52:38.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.418+0000 7f424f7fe700 1 -- 172.21.15.49:0/4244377836 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42500052a0 con 0x7f42600fed50 2024-05-15T15:52:38.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.418+0000 7f424f7fe700 1 -- 172.21.15.49:0/4244377836 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4250017400 con 0x7f42600fed50 2024-05-15T15:52:38.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.418+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426019df40 con 0x7f42600fed50 2024-05-15T15:52:38.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.418+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426019e370 con 0x7f42600fed50 2024-05-15T15:52:38.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.419+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4260192b00 con 0x7f42600fed50 2024-05-15T15:52:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.420+0000 7f424f7fe700 1 -- 172.21.15.49:0/4244377836 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f42500037f0 con 0x7f42600fed50 2024-05-15T15:52:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.420+0000 7f424f7fe700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f424806cce0 0x7f424806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.420+0000 7f424f7fe700 1 -- 172.21.15.49:0/4244377836 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f425008d700 con 0x7f42600fed50 2024-05-15T15:52:38.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.423+0000 7f425dd9b700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f424806cce0 0x7f424806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:38.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.430+0000 7f425dd9b700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f424806cce0 0x7f424806f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4254009bf0 tx=0x7f4254009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:38.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.430+0000 7f424f7fe700 1 -- 172.21.15.49:0/4244377836 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f425005b580 con 0x7f42600fed50 2024-05-15T15:52:38.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.730+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f426004ece0 con 0x7f42600fed50 2024-05-15T15:52:38.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.731+0000 7f424f7fe700 1 -- 172.21.15.49:0/4244377836 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f425005b110 con 0x7f42600fed50 2024-05-15T15:52:38.734 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:52:38.734 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:52:38.734 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:52:38.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 14 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:52:38.735 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:52:38.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f424806cce0 msgr2=0x7f424806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f424806cce0 0x7f424806f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4254009bf0 tx=0x7f4254009260 comp rx=0 tx=0).stop 2024-05-15T15:52:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42600fed50 msgr2=0x7f42601988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42600fed50 0x7f42601988f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f425000b800 tx=0x7f425000f510 comp rx=0 tx=0).stop 2024-05-15T15:52:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 shutdown_connections 2024-05-15T15:52:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42600fed50 0x7f42601988f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f424806cce0 0x7f424806f1a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 --2- 172.21.15.49:0/4244377836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4260101250 0x7f4260198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 >> 172.21.15.49:0/4244377836 conn(0x7f42600fa8f0 msgr2=0x7f4260108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.734+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 shutdown_connections 2024-05-15T15:52:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.735+0000 7f4264c43700 1 -- 172.21.15.49:0/4244377836 wait complete. 2024-05-15T15:52:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.847+0000 7f6e79005700 1 -- 172.21.15.49:0/1442723144 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e7000baf0 con 0x7f6e741000f0 2024-05-15T15:52:38.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.848+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1442723144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 msgr2=0x7f6e74100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.848+0000 7f6e7c26b700 1 --2- 172.21.15.49:0/1442723144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 0x7f6e74100510 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009910 tx=0x7f6e70009c20 comp rx=0 tx=0).stop 2024-05-15T15:52:38.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.848+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1442723144 shutdown_connections 2024-05-15T15:52:38.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.848+0000 7f6e7c26b700 1 --2- 172.21.15.49:0/1442723144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e74100e10 0x7f6e74101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.848+0000 7f6e7c26b700 1 --2- 172.21.15.49:0/1442723144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 0x7f6e74100510 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.848+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1442723144 >> 172.21.15.49:0/1442723144 conn(0x7f6e740fa8f0 msgr2=0x7f6e740fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.848+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1442723144 shutdown_connections 2024-05-15T15:52:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.849+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1442723144 wait complete. 2024-05-15T15:52:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.849+0000 7f6e7c26b700 1 Processor -- start 2024-05-15T15:52:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.849+0000 7f6e7c26b700 1 -- start start 2024-05-15T15:52:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.849+0000 7f6e7c26b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 0x7f6e741090f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7c26b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e74100e10 0x7f6e741077e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7c26b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e74107d20 con 0x7f6e741000f0 2024-05-15T15:52:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7c26b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e74107e60 con 0x7f6e74100e10 2024-05-15T15:52:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7a007700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 0x7f6e741090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7a007700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 0x7f6e741090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51900/0 (socket says 172.21.15.49:51900) 2024-05-15T15:52:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7a007700 1 -- 172.21.15.49:0/1082971922 learned_addr learned my addr 172.21.15.49:0/1082971922 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:52:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e79806700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e74100e10 0x7f6e741077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7a007700 1 -- 172.21.15.49:0/1082971922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e74100e10 msgr2=0x7f6e741077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7a007700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e74100e10 0x7f6e741077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7a007700 1 -- 172.21.15.49:0/1082971922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e700095f0 con 0x7f6e741000f0 2024-05-15T15:52:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.850+0000 7f6e7a007700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 0x7f6e741090f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6e70005f00 tx=0x7f6e7000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.851+0000 7f6e6b7fe700 1 -- 172.21.15.49:0/1082971922 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e7001d070 con 0x7f6e741000f0 2024-05-15T15:52:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.851+0000 7f6e6b7fe700 1 -- 172.21.15.49:0/1082971922 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e7000fe30 con 0x7f6e741000f0 2024-05-15T15:52:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.851+0000 7f6e6b7fe700 1 -- 172.21.15.49:0/1082971922 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e70017400 con 0x7f6e741000f0 2024-05-15T15:52:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.851+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e741080e0 con 0x7f6e741000f0 2024-05-15T15:52:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.851+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e74108590 con 0x7f6e741000f0 2024-05-15T15:52:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.852+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e74190880 con 0x7f6e741000f0 2024-05-15T15:52:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.853+0000 7f6e6b7fe700 1 -- 172.21.15.49:0/1082971922 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6e700038a0 con 0x7f6e741000f0 2024-05-15T15:52:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.853+0000 7f6e6b7fe700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f6e6006cce0 0x7f6e6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:52:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.853+0000 7f6e6b7fe700 1 -- 172.21.15.49:0/1082971922 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6e7008d660 con 0x7f6e741000f0 2024-05-15T15:52:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.853+0000 7f6e79806700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f6e6006cce0 0x7f6e6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:52:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.854+0000 7f6e79806700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f6e6006cce0 0x7f6e6006f1a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6e740ffe20 tx=0x7f6e64009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:52:38.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:38.862+0000 7f6e6b7fe700 1 -- 172.21.15.49:0/1082971922 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6e7005b4e0 con 0x7f6e741000f0 2024-05-15T15:52:39.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.087+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e7402cf00 con 0x7f6e6006cce0 2024-05-15T15:52:39.090 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:38 smithi049 ceph-mon[28399]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:39.090 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:38 smithi049 ceph-mon[28399]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:39.091 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:38 smithi049 ceph-mon[28399]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:39.091 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:38 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/4244377836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:52:39.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.091+0000 7f6e6b7fe700 1 -- 172.21.15.49:0/1082971922 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f6e7402cf00 con 0x7f6e6006cce0 2024-05-15T15:52:39.094 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:52:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:52:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:52:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:52:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [], 2024-05-15T15:52:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "0/2 daemons upgraded", 2024-05-15T15:52:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a image on host smithi144", 2024-05-15T15:52:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:52:39.095 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f6e6006cce0 msgr2=0x7f6e6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f6e6006cce0 0x7f6e6006f1a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6e740ffe20 tx=0x7f6e64009260 comp rx=0 tx=0).stop 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 msgr2=0x7f6e741090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 0x7f6e741090f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6e70005f00 tx=0x7f6e7000f510 comp rx=0 tx=0).stop 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 shutdown_connections 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e741000f0 0x7f6e741090f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f6e6006cce0 0x7f6e6006f1a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 --2- 172.21.15.49:0/1082971922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e74100e10 0x7f6e741077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 >> 172.21.15.49:0/1082971922 conn(0x7f6e740fa8f0 msgr2=0x7f6e741043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 shutdown_connections 2024-05-15T15:52:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:52:39.094+0000 7f6e7c26b700 1 -- 172.21.15.49:0/1082971922 wait complete. 2024-05-15T15:52:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:38 smithi144 ceph-mon[36389]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:38 smithi144 ceph-mon[36389]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:38 smithi144 ceph-mon[36389]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:38 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/4244377836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:52:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:39 smithi144 ceph-mon[36389]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:39 smithi144 ceph-mon[36389]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:39 smithi049 ceph-mon[28399]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:39 smithi049 ceph-mon[28399]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:52:41.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:40 smithi049 ceph-mon[28399]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:40 smithi144 ceph-mon[36389]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:42 smithi144 ceph-mon[36389]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:43.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:42 smithi049 ceph-mon[28399]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:44.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:44 smithi049 ceph-mon[28399]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:44 smithi144 ceph-mon[36389]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:47.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:46 smithi144 ceph-mon[36389]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:47.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:46 smithi049 ceph-mon[28399]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:49.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:48 smithi144 ceph-mon[36389]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:49.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:48 smithi049 ceph-mon[28399]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:51.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:50 smithi049 ceph-mon[28399]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:50 smithi144 ceph-mon[36389]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:52.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:51 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:52:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:51 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:52:53.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:52 smithi144 ceph-mon[36389]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:53.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:52 smithi049 ceph-mon[28399]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:54 smithi144 ceph-mon[36389]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:54 smithi049 ceph-mon[28399]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:56.284 INFO:tasks.workunit.client.0.smithi049.stderr:Updating files: 93% (11453/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-15T15:52:56.795 INFO:tasks.workunit.client.0.smithi049.stderr:Note: switching to 'de5210d58560a4684fdb5b238ef3e6e0285a1de8'. 2024-05-15T15:52:56.795 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-15T15:52:56.795 INFO:tasks.workunit.client.0.smithi049.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-15T15:52:56.795 INFO:tasks.workunit.client.0.smithi049.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-15T15:52:56.795 INFO:tasks.workunit.client.0.smithi049.stderr:state without impacting any branches by switching back to a branch. 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr: git switch -c 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr:Or undo this operation with: 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr: git switch - 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-15T15:52:56.796 INFO:tasks.workunit.client.0.smithi049.stderr:HEAD is now at de5210d5856 ceph-volume: drop pytest-xdist usage 2024-05-15T15:52:56.802 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-15T15:52:56.827 INFO:tasks.workunit.client.0.smithi049.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-15T15:52:56.830 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-15T15:52:56.830 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-15T15:52:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:56 smithi144 ceph-mon[36389]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:57.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:56 smithi049 ceph-mon[28399]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:52:57.510 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-15T15:52:57.567 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-15T15:52:57.608 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-15T15:52:57.610 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-15T15:52:57.610 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-15T15:52:57.650 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-15T15:52:57.654 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T15:52:57.654 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-15T15:52:57.711 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-05-15T15:52:57.713 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-15T15:52:57.714 DEBUG:teuthology.orchestra.run.smithi049:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=de5210d58560a4684fdb5b238ef3e6e0285a1de8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-05-15T15:52:57.778 INFO:tasks.workunit.client.0.smithi049.stderr:+ mkdir -p fsstress 2024-05-15T15:52:57.780 INFO:tasks.workunit.client.0.smithi049.stderr:+ pushd fsstress 2024-05-15T15:52:57.781 INFO:tasks.workunit.client.0.smithi049.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-15T15:52:57.781 INFO:tasks.workunit.client.0.smithi049.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-15T15:52:58.298 INFO:tasks.workunit.client.0.smithi049.stderr:+ tar xzf ltp-full.tgz 2024-05-15T15:52:58.975 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:52:58 smithi144 ceph-mon[36389]: pgmap v248: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:59.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:52:58 smithi049 ceph-mon[28399]: pgmap v248: 65 pgs: 65 active+clean; 590 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:52:59.536 INFO:tasks.workunit.client.1.smithi144.stderr:Updating files: 92% (11340/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-15T15:53:00.229 INFO:tasks.workunit.client.1.smithi144.stderr:Note: switching to 'de5210d58560a4684fdb5b238ef3e6e0285a1de8'. 2024-05-15T15:53:00.229 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-05-15T15:53:00.229 INFO:tasks.workunit.client.1.smithi144.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-15T15:53:00.229 INFO:tasks.workunit.client.1.smithi144.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-15T15:53:00.229 INFO:tasks.workunit.client.1.smithi144.stderr:state without impacting any branches by switching back to a branch. 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr: git switch -c 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr:Or undo this operation with: 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr: git switch - 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-15T15:53:00.230 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-05-15T15:53:00.231 INFO:tasks.workunit.client.1.smithi144.stderr:HEAD is now at de5210d5856 ceph-volume: drop pytest-xdist usage 2024-05-15T15:53:00.236 DEBUG:teuthology.orchestra.run.smithi144:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-05-15T15:53:00.348 INFO:tasks.workunit.client.1.smithi144.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-15T15:53:00.351 INFO:tasks.workunit.client.1.smithi144.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-15T15:53:00.351 INFO:tasks.workunit.client.1.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-15T15:53:01.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:00 smithi049 ceph-mon[28399]: pgmap v249: 65 pgs: 65 active+clean; 616 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 2.4 KiB/s wr, 4 op/s 2024-05-15T15:53:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:00 smithi049 ceph-mon[28399]: Upgrade: Updating mgr.smithi144.whcjkr 2024-05-15T15:53:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:53:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:53:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:00 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:53:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:00 smithi049 ceph-mon[28399]: Deploying daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:53:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:00 smithi144 ceph-mon[36389]: pgmap v249: 65 pgs: 65 active+clean; 616 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 2.4 KiB/s wr, 4 op/s 2024-05-15T15:53:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:00 smithi144 ceph-mon[36389]: Upgrade: Updating mgr.smithi144.whcjkr 2024-05-15T15:53:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:00 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:00 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:53:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:00 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:53:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:00 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:53:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:00 smithi144 ceph-mon[36389]: Deploying daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:53:01.998 INFO:tasks.workunit.client.1.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-15T15:53:02.068 INFO:tasks.workunit.client.1.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-15T15:53:02.112 INFO:tasks.workunit.client.1.smithi144.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-15T15:53:02.113 INFO:tasks.workunit.client.1.smithi144.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-15T15:53:02.113 INFO:tasks.workunit.client.1.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-15T15:53:02.152 INFO:tasks.workunit.client.1.smithi144.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-15T15:53:02.156 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-15T15:53:02.156 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-05-15T15:53:02.172 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-05-15T15:53:02.173 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-15T15:53:02.174 DEBUG:teuthology.orchestra.run.smithi144:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=de5210d58560a4684fdb5b238ef3e6e0285a1de8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-05-15T15:53:02.237 INFO:tasks.workunit.client.1.smithi144.stderr:+ mkdir -p fsstress 2024-05-15T15:53:02.240 INFO:tasks.workunit.client.1.smithi144.stderr:+ pushd fsstress 2024-05-15T15:53:02.241 INFO:tasks.workunit.client.1.smithi144.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-15T15:53:02.241 INFO:tasks.workunit.client.1.smithi144.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-15T15:53:02.829 INFO:tasks.workunit.client.1.smithi144.stderr:+ tar xzf ltp-full.tgz 2024-05-15T15:53:03.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:02 smithi144 ceph-mon[36389]: pgmap v250: 65 pgs: 65 active+clean; 5.4 MiB data, 212 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 412 KiB/s wr, 15 op/s 2024-05-15T15:53:03.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:02 smithi049 ceph-mon[28399]: pgmap v250: 65 pgs: 65 active+clean; 5.4 MiB data, 212 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 412 KiB/s wr, 15 op/s 2024-05-15T15:53:04.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:04 smithi144 ceph-mon[36389]: pgmap v251: 65 pgs: 65 active+clean; 16 MiB data, 241 MiB used, 536 GiB / 536 GiB avail; 168 KiB/s rd, 1.3 MiB/s wr, 41 op/s 2024-05-15T15:53:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:04 smithi049 ceph-mon[28399]: pgmap v251: 65 pgs: 65 active+clean; 16 MiB data, 241 MiB used, 536 GiB / 536 GiB avail; 168 KiB/s rd, 1.3 MiB/s wr, 41 op/s 2024-05-15T15:53:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:06 smithi144 ceph-mon[36389]: pgmap v252: 65 pgs: 65 active+clean; 33 MiB data, 286 MiB used, 536 GiB / 536 GiB avail; 168 KiB/s rd, 2.7 MiB/s wr, 80 op/s 2024-05-15T15:53:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:06 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:06 smithi049 ceph-mon[28399]: pgmap v252: 65 pgs: 65 active+clean; 33 MiB data, 286 MiB used, 536 GiB / 536 GiB avail; 168 KiB/s rd, 2.7 MiB/s wr, 80 op/s 2024-05-15T15:53:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:06 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:08 smithi144 ceph-mon[36389]: pgmap v253: 65 pgs: 65 active+clean; 33 MiB data, 286 MiB used, 536 GiB / 536 GiB avail; 168 KiB/s rd, 2.7 MiB/s wr, 79 op/s 2024-05-15T15:53:09.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:08 smithi049 ceph-mon[28399]: pgmap v253: 65 pgs: 65 active+clean; 33 MiB data, 286 MiB used, 536 GiB / 536 GiB avail; 168 KiB/s rd, 2.7 MiB/s wr, 79 op/s 2024-05-15T15:53:09.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.235+0000 7f4099b6f700 1 -- 172.21.15.49:0/1850092663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094101760 msgr2=0x7f4094103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.235+0000 7f4099b6f700 1 --2- 172.21.15.49:0/1850092663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094101760 0x7f4094103b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4084009910 tx=0x7f4084009c20 comp rx=0 tx=0).stop 2024-05-15T15:53:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.235+0000 7f4099b6f700 1 -- 172.21.15.49:0/1850092663 shutdown_connections 2024-05-15T15:53:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.235+0000 7f4099b6f700 1 --2- 172.21.15.49:0/1850092663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4094104090 0x7f4094106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.235+0000 7f4099b6f700 1 --2- 172.21.15.49:0/1850092663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094101760 0x7f4094103b50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.235+0000 7f4099b6f700 1 -- 172.21.15.49:0/1850092663 >> 172.21.15.49:0/1850092663 conn(0x7f40940fb170 msgr2=0x7f40940fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.235+0000 7f4099b6f700 1 -- 172.21.15.49:0/1850092663 shutdown_connections 2024-05-15T15:53:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.236+0000 7f4099b6f700 1 -- 172.21.15.49:0/1850092663 wait complete. 2024-05-15T15:53:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.236+0000 7f4099b6f700 1 Processor -- start 2024-05-15T15:53:09.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.236+0000 7f4099b6f700 1 -- start start 2024-05-15T15:53:09.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4099b6f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4094101760 0x7f4094196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:09.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4099b6f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094104090 0x7f4094196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:09.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094104090 0x7f4094196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:09.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094104090 0x7f4094196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51960/0 (socket says 172.21.15.49:51960) 2024-05-15T15:53:09.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4092ffd700 1 -- 172.21.15.49:0/821846708 learned_addr learned my addr 172.21.15.49:0/821846708 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:09.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f40937fe700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4094101760 0x7f4094196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:09.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941971f0 con 0x7f4094104090 2024-05-15T15:53:09.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4094197330 con 0x7f4094101760 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4092ffd700 1 -- 172.21.15.49:0/821846708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4094101760 msgr2=0x7f4094196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4092ffd700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4094101760 0x7f4094196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f4092ffd700 1 -- 172.21.15.49:0/821846708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840095f0 con 0x7f4094104090 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.237+0000 7f40937fe700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4094101760 0x7f4094196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.238+0000 7f4092ffd700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094104090 0x7f4094196bd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f408800e920 tx=0x7f408800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.238+0000 7f4090ff9700 1 -- 172.21.15.49:0/821846708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4088009790 con 0x7f4094104090 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.238+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409419bd90 con 0x7f4094104090 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.238+0000 7f4090ff9700 1 -- 172.21.15.49:0/821846708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f408800cba0 con 0x7f4094104090 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.238+0000 7f4090ff9700 1 -- 172.21.15.49:0/821846708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4088018630 con 0x7f4094104090 2024-05-15T15:53:09.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.238+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409419c270 con 0x7f4094104090 2024-05-15T15:53:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.239+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409404ec60 con 0x7f4094104090 2024-05-15T15:53:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.240+0000 7f4090ff9700 1 -- 172.21.15.49:0/821846708 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4088018790 con 0x7f4094104090 2024-05-15T15:53:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.240+0000 7f4090ff9700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f407c06cce0 0x7f407c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.241+0000 7f4090ff9700 1 -- 172.21.15.49:0/821846708 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4088014070 con 0x7f4094104090 2024-05-15T15:53:09.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.241+0000 7f40937fe700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f407c06cce0 0x7f407c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:09.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.241+0000 7f40937fe700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f407c06cce0 0x7f407c06f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f40840059d0 tx=0x7f4084005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:09.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.247+0000 7f4090ff9700 1 -- 172.21.15.49:0/821846708 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4088052d60 con 0x7f4094104090 2024-05-15T15:53:09.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.586+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4094104a70 con 0x7f407c06cce0 2024-05-15T15:53:09.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.587+0000 7f4090ff9700 1 -- 172.21.15.49:0/821846708 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f4094104a70 con 0x7f407c06cce0 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f407c06cce0 msgr2=0x7f407c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f407c06cce0 0x7f407c06f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f40840059d0 tx=0x7f4084005940 comp rx=0 tx=0).stop 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094104090 msgr2=0x7f4094196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094104090 0x7f4094196bd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f408800e920 tx=0x7f408800ece0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 shutdown_connections 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f407c06cce0 0x7f407c06f1a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4094101760 0x7f4094196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 --2- 172.21.15.49:0/821846708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4094104090 0x7f4094196bd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 >> 172.21.15.49:0/821846708 conn(0x7f40940fb170 msgr2=0x7f40940fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 shutdown_connections 2024-05-15T15:53:09.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.591+0000 7f4099b6f700 1 -- 172.21.15.49:0/821846708 wait complete. 2024-05-15T15:53:09.610 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:53:09.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.735+0000 7f2985ae7700 1 -- 172.21.15.49:0/1318876619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29800ffbd0 msgr2=0x7f29800ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:09.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.735+0000 7f2985ae7700 1 --2- 172.21.15.49:0/1318876619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29800ffbd0 0x7f29800ffff0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2970009910 tx=0x7f2970009c20 comp rx=0 tx=0).stop 2024-05-15T15:53:09.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.735+0000 7f2985ae7700 1 -- 172.21.15.49:0/1318876619 shutdown_connections 2024-05-15T15:53:09.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.735+0000 7f2985ae7700 1 --2- 172.21.15.49:0/1318876619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2980100dd0 0x7f2980101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.735+0000 7f2985ae7700 1 --2- 172.21.15.49:0/1318876619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29800ffbd0 0x7f29800ffff0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.735+0000 7f2985ae7700 1 -- 172.21.15.49:0/1318876619 >> 172.21.15.49:0/1318876619 conn(0x7f29800fb170 msgr2=0x7f29800fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:09.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.735+0000 7f2985ae7700 1 -- 172.21.15.49:0/1318876619 shutdown_connections 2024-05-15T15:53:09.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.735+0000 7f2985ae7700 1 -- 172.21.15.49:0/1318876619 wait complete. 2024-05-15T15:53:09.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.736+0000 7f2985ae7700 1 Processor -- start 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.736+0000 7f2985ae7700 1 -- start start 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f2985ae7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29800ffbd0 0x7f29801966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f2985ae7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980100dd0 0x7f2980196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f2985ae7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2980197240 con 0x7f2980100dd0 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f2985ae7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2980197380 con 0x7f29800ffbd0 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f297effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980100dd0 0x7f2980196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f297effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980100dd0 0x7f2980196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51968/0 (socket says 172.21.15.49:51968) 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f297effd700 1 -- 172.21.15.49:0/2793685635 learned_addr learned my addr 172.21.15.49:0/2793685635 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f297f7fe700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29800ffbd0 0x7f29801966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f297effd700 1 -- 172.21.15.49:0/2793685635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29800ffbd0 msgr2=0x7f29801966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:09.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f297effd700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29800ffbd0 0x7f29801966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:09.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f297effd700 1 -- 172.21.15.49:0/2793685635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29700095f0 con 0x7f2980100dd0 2024-05-15T15:53:09.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.737+0000 7f297f7fe700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29800ffbd0 0x7f29801966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:53:09.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.738+0000 7f297effd700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980100dd0 0x7f2980196c20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f297400e920 tx=0x7f297400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:09.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.738+0000 7f297cff9700 1 -- 172.21.15.49:0/2793685635 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297400c9d0 con 0x7f2980100dd0 2024-05-15T15:53:09.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.738+0000 7f297cff9700 1 -- 172.21.15.49:0/2793685635 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2974010430 con 0x7f2980100dd0 2024-05-15T15:53:09.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.738+0000 7f297cff9700 1 -- 172.21.15.49:0/2793685635 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2974018630 con 0x7f2980100dd0 2024-05-15T15:53:09.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.738+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298019bde0 con 0x7f2980100dd0 2024-05-15T15:53:09.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.738+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298019c380 con 0x7f2980100dd0 2024-05-15T15:53:09.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.739+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298004ec60 con 0x7f2980100dd0 2024-05-15T15:53:09.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.740+0000 7f297cff9700 1 -- 172.21.15.49:0/2793685635 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f29740105a0 con 0x7f2980100dd0 2024-05-15T15:53:09.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.740+0000 7f297cff9700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f296806ccf0 0x7f296806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:09.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.741+0000 7f297cff9700 1 -- 172.21.15.49:0/2793685635 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2974014070 con 0x7f2980100dd0 2024-05-15T15:53:09.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.741+0000 7f297f7fe700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f296806ccf0 0x7f296806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:09.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.741+0000 7f297f7fe700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f296806ccf0 0x7f296806f1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f297000b800 tx=0x7f2970009f70 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:09.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:09.746+0000 7f297cff9700 1 -- 172.21.15.49:0/2793685635 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2974052d60 con 0x7f2980100dd0 2024-05-15T15:53:10.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.068+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2980069710 con 0x7f296806ccf0 2024-05-15T15:53:10.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.070+0000 7f297cff9700 1 -- 172.21.15.49:0/2793685635 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f2980069710 con 0x7f296806ccf0 2024-05-15T15:53:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.072+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f296806ccf0 msgr2=0x7f296806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.072+0000 7f2985ae7700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f296806ccf0 0x7f296806f1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f297000b800 tx=0x7f2970009f70 comp rx=0 tx=0).stop 2024-05-15T15:53:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.072+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980100dd0 msgr2=0x7f2980196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.072+0000 7f2985ae7700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980100dd0 0x7f2980196c20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f297400e920 tx=0x7f297400ec30 comp rx=0 tx=0).stop 2024-05-15T15:53:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.072+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 shutdown_connections 2024-05-15T15:53:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.072+0000 7f2985ae7700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7f296806ccf0 0x7f296806f1b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.072+0000 7f2985ae7700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29800ffbd0 0x7f29801966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.072+0000 7f2985ae7700 1 --2- 172.21.15.49:0/2793685635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980100dd0 0x7f2980196c20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.072+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 >> 172.21.15.49:0/2793685635 conn(0x7f29800fb170 msgr2=0x7f2980069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:10.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.073+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 shutdown_connections 2024-05-15T15:53:10.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.073+0000 7f2985ae7700 1 -- 172.21.15.49:0/2793685635 wait complete. 2024-05-15T15:53:10.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.195+0000 7fd9278fe700 1 -- 172.21.15.49:0/551753575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 msgr2=0x7fd9201012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:10.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.195+0000 7fd9278fe700 1 --2- 172.21.15.49:0/551753575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 0x7fd9201012b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd910009910 tx=0x7fd910009c20 comp rx=0 tx=0).stop 2024-05-15T15:53:10.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.195+0000 7fd9278fe700 1 -- 172.21.15.49:0/551753575 shutdown_connections 2024-05-15T15:53:10.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.195+0000 7fd9278fe700 1 --2- 172.21.15.49:0/551753575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 0x7fd9201012b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.195+0000 7fd9278fe700 1 --2- 172.21.15.49:0/551753575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9200ffc50 0x7fd920100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.195+0000 7fd9278fe700 1 -- 172.21.15.49:0/551753575 >> 172.21.15.49:0/551753575 conn(0x7fd9200fb1d0 msgr2=0x7fd9200fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:10.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.195+0000 7fd9278fe700 1 -- 172.21.15.49:0/551753575 shutdown_connections 2024-05-15T15:53:10.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.195+0000 7fd9278fe700 1 -- 172.21.15.49:0/551753575 wait complete. 2024-05-15T15:53:10.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd9278fe700 1 Processor -- start 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd9278fe700 1 -- start start 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd9278fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9200ffc50 0x7fd920103b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd9278fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 0x7fd920102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd9278fe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920102770 con 0x7fd920100e50 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd9278fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9201028b0 con 0x7fd9200ffc50 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd924e99700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 0x7fd920102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd924e99700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 0x7fd920102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51982/0 (socket says 172.21.15.49:51982) 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd924e99700 1 -- 172.21.15.49:0/641938247 learned_addr learned my addr 172.21.15.49:0/641938247 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.196+0000 7fd92569a700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9200ffc50 0x7fd920103b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.197+0000 7fd924e99700 1 -- 172.21.15.49:0/641938247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9200ffc50 msgr2=0x7fd920103b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.197+0000 7fd924e99700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9200ffc50 0x7fd920103b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.197+0000 7fd924e99700 1 -- 172.21.15.49:0/641938247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100095f0 con 0x7fd920100e50 2024-05-15T15:53:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.197+0000 7fd924e99700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 0x7fd920102230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b800 tx=0x7fd910005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.197+0000 7fd9167fc700 1 -- 172.21.15.49:0/641938247 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd91001d070 con 0x7fd920100e50 2024-05-15T15:53:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.197+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd920102b30 con 0x7fd920100e50 2024-05-15T15:53:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.197+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd920103020 con 0x7fd920100e50 2024-05-15T15:53:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.197+0000 7fd9167fc700 1 -- 172.21.15.49:0/641938247 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9100052a0 con 0x7fd920100e50 2024-05-15T15:53:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.198+0000 7fd9167fc700 1 -- 172.21.15.49:0/641938247 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd910017400 con 0x7fd920100e50 2024-05-15T15:53:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.198+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9040052f0 con 0x7fd920100e50 2024-05-15T15:53:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.200+0000 7fd9167fc700 1 -- 172.21.15.49:0/641938247 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd910003680 con 0x7fd920100e50 2024-05-15T15:53:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.200+0000 7fd9167fc700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd90c06cce0 0x7fd90c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.200+0000 7fd9167fc700 1 -- 172.21.15.49:0/641938247 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd91008d640 con 0x7fd920100e50 2024-05-15T15:53:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.200+0000 7fd92569a700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd90c06cce0 0x7fd90c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.201+0000 7fd92569a700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd90c06cce0 0x7fd90c06f1a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd920100cb0 tx=0x7fd91c009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.205+0000 7fd9167fc700 1 -- 172.21.15.49:0/641938247 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd91005b420 con 0x7fd920100e50 2024-05-15T15:53:10.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.442+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd904000bc0 con 0x7fd90c06cce0 2024-05-15T15:53:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.453+0000 7fd9167fc700 1 -- 172.21.15.49:0/641938247 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fd904000bc0 con 0x7fd90c06cce0 2024-05-15T15:53:10.456 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:53:10.456 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (6m) 119s ago 7m 24.1M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:53:10.456 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (8m) 119s ago 8m 7528k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:53:10.456 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (7m) 2m ago 7m 7424k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:53:10.456 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (8m) 119s ago 8m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (7m) 2m ago 7m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (6m) 119s ago 7m 85.4M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (2m) 119s ago 2m 13.0M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (2m) 119s ago 2m 16.5M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (2m) 2m ago 2m 16.2M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (2m) 2m ago 2m 17.1M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:9283,8765,8443 running (9m) 119s ago 9m 499M - 18.2.2-1343-g2965081f a04a73ffe079 ea7d1a4b9c55 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (6m) 2m ago 6m 452M - 18.2.2-1343-g2965081f a04a73ffe079 eece4ed6d244 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (9m) 119s ago 9m 45.0M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (6m) 2m ago 6m 38.6M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:53:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (8m) 119s ago 8m 14.1M - 1.5.0 0da6a335fe13 171e0a34f565 2024-05-15T15:53:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (6m) 2m ago 6m 15.4M - 1.5.0 0da6a335fe13 be3a2837bc85 2024-05-15T15:53:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (6m) 119s ago 6m 53.8M 3946M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:53:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (5m) 119s ago 5m 47.9M 3946M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:53:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (5m) 119s ago 5m 52.2M 3946M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:53:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (4m) 2m ago 4m 52.4M 4970M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:53:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (4m) 2m ago 4m 44.5M 4970M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:53:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (3m) 2m ago 3m 52.6M 4970M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:53:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (6m) 119s ago 7m 57.3M - 2.43.0 a07b618ecd1d 0299663ce1de 2024-05-15T15:53:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd90c06cce0 msgr2=0x7fd90c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd90c06cce0 0x7fd90c06f1a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd920100cb0 tx=0x7fd91c009260 comp rx=0 tx=0).stop 2024-05-15T15:53:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 msgr2=0x7fd920102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 0x7fd920102230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b800 tx=0x7fd910005f00 comp rx=0 tx=0).stop 2024-05-15T15:53:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 shutdown_connections 2024-05-15T15:53:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd90c06cce0 0x7fd90c06f1a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9200ffc50 0x7fd920103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 --2- 172.21.15.49:0/641938247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd920100e50 0x7fd920102230 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 >> 172.21.15.49:0/641938247 conn(0x7fd9200fb1d0 msgr2=0x7fd920068750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 shutdown_connections 2024-05-15T15:53:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.456+0000 7fd9278fe700 1 -- 172.21.15.49:0/641938247 wait complete. 2024-05-15T15:53:10.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.617+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2111733844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd554100d20 msgr2=0x7fd554101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:10.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.617+0000 7fd55c6ee700 1 --2- 172.21.15.49:0/2111733844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd554100d20 0x7fd554101140 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd55000b0a0 tx=0x7fd55000b3b0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.617+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2111733844 shutdown_connections 2024-05-15T15:53:10.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.617+0000 7fd55c6ee700 1 --2- 172.21.15.49:0/2111733844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd554101f20 0x7fd554102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.617+0000 7fd55c6ee700 1 --2- 172.21.15.49:0/2111733844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd554100d20 0x7fd554101140 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.617+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2111733844 >> 172.21.15.49:0/2111733844 conn(0x7fd5540fc2c0 msgr2=0x7fd5540fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:10.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.617+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2111733844 shutdown_connections 2024-05-15T15:53:10.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.617+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2111733844 wait complete. 2024-05-15T15:53:10.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.618+0000 7fd55c6ee700 1 Processor -- start 2024-05-15T15:53:10.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.618+0000 7fd55c6ee700 1 -- start start 2024-05-15T15:53:10.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.618+0000 7fd55c6ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd554100d20 0x7fd554074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:10.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.618+0000 7fd55c6ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd554101f20 0x7fd554073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:10.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.618+0000 7fd55c6ee700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55419a300 con 0x7fd554100d20 2024-05-15T15:53:10.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.618+0000 7fd55c6ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55419a440 con 0x7fd554101f20 2024-05-15T15:53:10.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.619+0000 7fd559c89700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd554101f20 0x7fd554073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:10.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.619+0000 7fd55a48a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd554100d20 0x7fd554074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:10.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.619+0000 7fd559c89700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd554101f20 0x7fd554073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:60880/0 (socket says 172.21.15.49:60880) 2024-05-15T15:53:10.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.619+0000 7fd559c89700 1 -- 172.21.15.49:0/2832445193 learned_addr learned my addr 172.21.15.49:0/2832445193 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:10.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.619+0000 7fd559c89700 1 -- 172.21.15.49:0/2832445193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd554100d20 msgr2=0x7fd554074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:10.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.619+0000 7fd559c89700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd554100d20 0x7fd554074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:10.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.619+0000 7fd559c89700 1 -- 172.21.15.49:0/2832445193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd550009d00 con 0x7fd554101f20 2024-05-15T15:53:10.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.619+0000 7fd559c89700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd554101f20 0x7fd554073390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd54400e920 tx=0x7fd54400ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:10.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.620+0000 7fd54b7fe700 1 -- 172.21.15.49:0/2832445193 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54400c9d0 con 0x7fd554101f20 2024-05-15T15:53:10.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.620+0000 7fd54b7fe700 1 -- 172.21.15.49:0/2832445193 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd544010430 con 0x7fd554101f20 2024-05-15T15:53:10.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.620+0000 7fd54b7fe700 1 -- 172.21.15.49:0/2832445193 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd544018630 con 0x7fd554101f20 2024-05-15T15:53:10.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.620+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5540738d0 con 0x7fd554101f20 2024-05-15T15:53:10.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.620+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd554073e20 con 0x7fd554101f20 2024-05-15T15:53:10.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.621+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5541908a0 con 0x7fd554101f20 2024-05-15T15:53:10.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.623+0000 7fd54b7fe700 1 -- 172.21.15.49:0/2832445193 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd54400cb30 con 0x7fd554101f20 2024-05-15T15:53:10.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.624+0000 7fd54b7fe700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd54006cce0 0x7fd54006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:10.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.624+0000 7fd54b7fe700 1 -- 172.21.15.49:0/2832445193 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd544014070 con 0x7fd554101f20 2024-05-15T15:53:10.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.625+0000 7fd55a48a700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd54006cce0 0x7fd54006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:10.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.625+0000 7fd55a48a700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd54006cce0 0x7fd54006f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd550009fd0 tx=0x7fd550006000 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:10.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:10.628+0000 7fd54b7fe700 1 -- 172.21.15.49:0/2832445193 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd544052d60 con 0x7fd554101f20 2024-05-15T15:53:10.833 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:10 smithi144 ceph-mon[36389]: pgmap v254: 65 pgs: 65 active+clean; 43 MiB data, 326 MiB used, 536 GiB / 536 GiB avail; 746 KiB/s rd, 3.5 MiB/s wr, 121 op/s 2024-05-15T15:53:10.833 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:10 smithi144 ceph-mon[36389]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:11.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.037+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd55404ec60 con 0x7fd554101f20 2024-05-15T15:53:11.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.038+0000 7fd54b7fe700 1 -- 172.21.15.49:0/2832445193 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd54405a3b0 con 0x7fd554101f20 2024-05-15T15:53:11.040 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:53:11.040 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:53:11.040 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 14 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:53:11.041 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:53:11.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.041+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd54006cce0 msgr2=0x7fd54006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:11.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.041+0000 7fd55c6ee700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd54006cce0 0x7fd54006f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd550009fd0 tx=0x7fd550006000 comp rx=0 tx=0).stop 2024-05-15T15:53:11.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.041+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd554101f20 msgr2=0x7fd554073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:11.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.041+0000 7fd55c6ee700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd554101f20 0x7fd554073390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd54400e920 tx=0x7fd54400ec30 comp rx=0 tx=0).stop 2024-05-15T15:53:11.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.041+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 shutdown_connections 2024-05-15T15:53:11.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.041+0000 7fd55c6ee700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd554100d20 0x7fd554074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:11.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.041+0000 7fd55c6ee700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fd54006cce0 0x7fd54006f1a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:11.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.041+0000 7fd55c6ee700 1 --2- 172.21.15.49:0/2832445193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd554101f20 0x7fd554073390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:11.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.041+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 >> 172.21.15.49:0/2832445193 conn(0x7fd5540fc2c0 msgr2=0x7fd554105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:11.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.042+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 shutdown_connections 2024-05-15T15:53:11.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.042+0000 7fd55c6ee700 1 -- 172.21.15.49:0/2832445193 wait complete. 2024-05-15T15:53:11.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:10 smithi049 ceph-mon[28399]: pgmap v254: 65 pgs: 65 active+clean; 43 MiB data, 326 MiB used, 536 GiB / 536 GiB avail; 746 KiB/s rd, 3.5 MiB/s wr, 121 op/s 2024-05-15T15:53:11.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:10 smithi049 ceph-mon[28399]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:11.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:10 smithi049 ceph-mon[28399]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:11.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.174+0000 7fe5da238700 1 -- 172.21.15.49:0/3189639245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 msgr2=0x7fe5d40ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.174+0000 7fe5da238700 1 --2- 172.21.15.49:0/3189639245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 0x7fe5d40ff770 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400aa20 tx=0x7fe5c400ad30 comp rx=0 tx=0).stop 2024-05-15T15:53:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.174+0000 7fe5da238700 1 -- 172.21.15.49:0/3189639245 shutdown_connections 2024-05-15T15:53:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.174+0000 7fe5da238700 1 --2- 172.21.15.49:0/3189639245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d4100550 0x7fe5d41009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.174+0000 7fe5da238700 1 --2- 172.21.15.49:0/3189639245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 0x7fe5d40ff770 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.174+0000 7fe5da238700 1 -- 172.21.15.49:0/3189639245 >> 172.21.15.49:0/3189639245 conn(0x7fe5d40fa8f0 msgr2=0x7fe5d40fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.174+0000 7fe5da238700 1 -- 172.21.15.49:0/3189639245 shutdown_connections 2024-05-15T15:53:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.174+0000 7fe5da238700 1 -- 172.21.15.49:0/3189639245 wait complete. 2024-05-15T15:53:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.175+0000 7fe5da238700 1 Processor -- start 2024-05-15T15:53:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.175+0000 7fe5da238700 1 -- start start 2024-05-15T15:53:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.175+0000 7fe5da238700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 0x7fe5d41944e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.175+0000 7fe5da238700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d4100550 0x7fe5d4194a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.175+0000 7fe5da238700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4195040 con 0x7fe5d40ff350 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.175+0000 7fe5da238700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4195180 con 0x7fe5d4100550 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.175+0000 7fe5d37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 0x7fe5d41944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.175+0000 7fe5d37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 0x7fe5d41944e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52000/0 (socket says 172.21.15.49:52000) 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5d37fe700 1 -- 172.21.15.49:0/246767928 learned_addr learned my addr 172.21.15.49:0/246767928 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5d2ffd700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d4100550 0x7fe5d4194a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5d37fe700 1 -- 172.21.15.49:0/246767928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d4100550 msgr2=0x7fe5d4194a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5d37fe700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d4100550 0x7fe5d4194a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5d37fe700 1 -- 172.21.15.49:0/246767928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c400a700 con 0x7fe5d40ff350 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5d37fe700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 0x7fe5d41944e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4006010 tx=0x7fe5c4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5d0ff9700 1 -- 172.21.15.49:0/246767928 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5c40051d0 con 0x7fe5d40ff350 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d4199b30 con 0x7fe5d40ff350 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5d0ff9700 1 -- 172.21.15.49:0/246767928 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5c400e910 con 0x7fe5d40ff350 2024-05-15T15:53:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5d0ff9700 1 -- 172.21.15.49:0/246767928 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5c4016860 con 0x7fe5d40ff350 2024-05-15T15:53:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.176+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d419a020 con 0x7fe5d40ff350 2024-05-15T15:53:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.178+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d418e640 con 0x7fe5d40ff350 2024-05-15T15:53:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.179+0000 7fe5d0ff9700 1 -- 172.21.15.49:0/246767928 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe5c400e450 con 0x7fe5d40ff350 2024-05-15T15:53:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.179+0000 7fe5d0ff9700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe5bc06ef00 0x7fe5bc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.180+0000 7fe5d0ff9700 1 -- 172.21.15.49:0/246767928 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe5c4012070 con 0x7fe5d40ff350 2024-05-15T15:53:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.180+0000 7fe5d2ffd700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe5bc06ef00 0x7fe5bc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.180+0000 7fe5d2ffd700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe5bc06ef00 0x7fe5bc0713c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009c60 tx=0x7fe5c8009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:11.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.185+0000 7fe5d0ff9700 1 -- 172.21.15.49:0/246767928 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe5c405bdb0 con 0x7fe5d40ff350 2024-05-15T15:53:11.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:10 smithi144 ceph-mon[36389]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:11.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.528+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 --> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5d402cf10 con 0x7fe5bc06ef00 2024-05-15T15:53:11.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.530+0000 7fe5d0ff9700 1 -- 172.21.15.49:0/246767928 <== mgr.14213 v2:172.21.15.49:6800/1448243308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe5d402cf10 con 0x7fe5bc06ef00 2024-05-15T15:53:11.532 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:53:11.533 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:53:11.533 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:53:11.533 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:53:11.533 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [], 2024-05-15T15:53:11.533 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "0/2 daemons upgraded", 2024-05-15T15:53:11.533 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mgr daemons", 2024-05-15T15:53:11.533 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:53:11.533 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:53:11.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.533+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe5bc06ef00 msgr2=0x7fe5bc0713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:11.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.533+0000 7fe5da238700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe5bc06ef00 0x7fe5bc0713c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009c60 tx=0x7fe5c8009260 comp rx=0 tx=0).stop 2024-05-15T15:53:11.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.533+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 msgr2=0x7fe5d41944e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:11.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.533+0000 7fe5da238700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 0x7fe5d41944e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4006010 tx=0x7fe5c4005800 comp rx=0 tx=0).stop 2024-05-15T15:53:11.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.533+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 shutdown_connections 2024-05-15T15:53:11.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.533+0000 7fe5da238700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d40ff350 0x7fe5d41944e0 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4006010 tx=0x7fe5c4005800 comp rx=0 tx=0).stop 2024-05-15T15:53:11.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.533+0000 7fe5da238700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.49:6800/1448243308,v1:172.21.15.49:6801/1448243308] conn(0x7fe5bc06ef00 0x7fe5bc0713c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:11.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.533+0000 7fe5da238700 1 --2- 172.21.15.49:0/246767928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d4100550 0x7fe5d4194a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:11.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.533+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 >> 172.21.15.49:0/246767928 conn(0x7fe5d40fa8f0 msgr2=0x7fe5d4103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:11.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.534+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 shutdown_connections 2024-05-15T15:53:11.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:11.534+0000 7fe5da238700 1 -- 172.21.15.49:0/246767928 wait complete. 2024-05-15T15:53:11.823 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:11 smithi049 ceph-mon[28399]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:11.824 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:11 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2832445193' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:53:11.824 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:11.824 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:11 smithi144 ceph-mon[36389]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:11 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2832445193' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:53:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:11 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:53:12.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:11 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:53:13.149 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:12 smithi144 ceph-mon[36389]: pgmap v255: 65 pgs: 65 active+clean; 47 MiB data, 345 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 147 op/s 2024-05-15T15:53:13.150 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:12 smithi144 ceph-mon[36389]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:12 smithi049 ceph-mon[28399]: pgmap v255: 65 pgs: 65 active+clean; 47 MiB data, 345 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 147 op/s 2024-05-15T15:53:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:12 smithi049 ceph-mon[28399]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:15.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:14 smithi144 ceph-mon[36389]: pgmap v256: 65 pgs: 65 active+clean; 60 MiB data, 393 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 180 op/s 2024-05-15T15:53:15.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:14 smithi049 ceph-mon[28399]: pgmap v256: 65 pgs: 65 active+clean; 60 MiB data, 393 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 180 op/s 2024-05-15T15:53:16.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:16 smithi144 ceph-mon[36389]: pgmap v257: 65 pgs: 65 active+clean; 67 MiB data, 422 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-05-15T15:53:17.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:16 smithi049 ceph-mon[28399]: pgmap v257: 65 pgs: 65 active+clean; 67 MiB data, 422 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-05-15T15:53:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:18 smithi144 ceph-mon[36389]: pgmap v258: 65 pgs: 65 active+clean; 67 MiB data, 422 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 147 op/s 2024-05-15T15:53:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:18 smithi144 ceph-mon[36389]: Standby manager daemon smithi144.whcjkr restarted 2024-05-15T15:53:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:18 smithi144 ceph-mon[36389]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:53:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:18 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:53:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:18 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:53:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:18 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:53:19.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:18 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:53:19.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:18 smithi049 ceph-mon[28399]: pgmap v258: 65 pgs: 65 active+clean; 67 MiB data, 422 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 147 op/s 2024-05-15T15:53:19.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:18 smithi049 ceph-mon[28399]: Standby manager daemon smithi144.whcjkr restarted 2024-05-15T15:53:19.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:18 smithi049 ceph-mon[28399]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:53:19.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:18 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:53:19.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:18 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:53:19.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:18 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:53:19.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:18 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:53:20.141 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:19 smithi049 ceph-mon[28399]: mgrmap e20: smithi049.vuhzgf(active, since 7m), standbys: smithi144.whcjkr 2024-05-15T15:53:20.141 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:19 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:20.141 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:19 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:19 smithi144 ceph-mon[36389]: mgrmap e20: smithi049.vuhzgf(active, since 7m), standbys: smithi144.whcjkr 2024-05-15T15:53:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:19 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:19 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:21.028 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:20 smithi144 ceph-mon[36389]: pgmap v259: 65 pgs: 65 active+clean; 87 MiB data, 444 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 196 op/s 2024-05-15T15:53:21.028 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:21.029 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:20 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:20 smithi049 ceph-mon[28399]: pgmap v259: 65 pgs: 65 active+clean; 87 MiB data, 444 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 196 op/s 2024-05-15T15:53:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:20 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:22.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:21 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:22.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:21 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:22.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:22 smithi049 ceph-mon[28399]: pgmap v260: 65 pgs: 65 active+clean; 95 MiB data, 471 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 191 op/s 2024-05-15T15:53:23.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:22 smithi144 ceph-mon[36389]: pgmap v260: 65 pgs: 65 active+clean; 95 MiB data, 471 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 191 op/s 2024-05-15T15:53:24.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: pgmap v261: 65 pgs: 65 active+clean; 106 MiB data, 514 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 5.0 MiB/s wr, 204 op/s 2024-05-15T15:53:24.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:24.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:24.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr fail", "who": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: Activating manager daemon smithi144.whcjkr 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr metadata", "who": "smithi144.whcjkr", "id": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:53:24.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: osdmap e48: 6 total, 6 up, 6 in 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "mgr fail", "who": "smithi049.vuhzgf"}]': finished 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: mgrmap e21: smithi144.whcjkr(active, starting, since 0.0100126s) 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: Manager daemon smithi144.whcjkr is now available 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.whcjkr/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:53:24.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.whcjkr/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:53:24.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.whcjkr/trash_purge_schedule"}]: dispatch 2024-05-15T15:53:24.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:24 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.whcjkr/trash_purge_schedule"}]: dispatch 2024-05-15T15:53:25.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: pgmap v261: 65 pgs: 65 active+clean; 106 MiB data, 514 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 5.0 MiB/s wr, 204 op/s 2024-05-15T15:53:25.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:25.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:25.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:53:25.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:53:25.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:53:25.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:53:25.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr fail", "who": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:53:25.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: Activating manager daemon smithi144.whcjkr 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr metadata", "who": "smithi144.whcjkr", "id": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: osdmap e48: 6 total, 6 up, 6 in 2024-05-15T15:53:25.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.14213 172.21.15.49:0/2959916721' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "mgr fail", "who": "smithi049.vuhzgf"}]': finished 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: mgrmap e21: smithi144.whcjkr(active, starting, since 0.0100126s) 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: Manager daemon smithi144.whcjkr is now available 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.whcjkr/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.whcjkr/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.whcjkr/trash_purge_schedule"}]: dispatch 2024-05-15T15:53:25.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:24 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.whcjkr/trash_purge_schedule"}]: dispatch 2024-05-15T15:53:26.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:25 smithi144 ceph-mon[36389]: Deploying cephadm binary to smithi144 2024-05-15T15:53:26.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:25 smithi144 ceph-mon[36389]: Deploying cephadm binary to smithi049 2024-05-15T15:53:26.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:25 smithi144 ceph-mon[36389]: [15/May/2024:15:53:25] ENGINE Bus STARTING 2024-05-15T15:53:26.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:25 smithi144 ceph-mon[36389]: [15/May/2024:15:53:25] ENGINE Serving on https://172.21.15.144:7150 2024-05-15T15:53:26.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:25 smithi144 ceph-mon[36389]: [15/May/2024:15:53:25] ENGINE Serving on http://172.21.15.144:8765 2024-05-15T15:53:26.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:25 smithi144 ceph-mon[36389]: [15/May/2024:15:53:25] ENGINE Bus STARTED 2024-05-15T15:53:26.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:25 smithi144 ceph-mon[36389]: mgrmap e22: smithi144.whcjkr(active, since 1.01068s) 2024-05-15T15:53:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:25 smithi049 ceph-mon[28399]: Deploying cephadm binary to smithi144 2024-05-15T15:53:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:25 smithi049 ceph-mon[28399]: Deploying cephadm binary to smithi049 2024-05-15T15:53:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:25 smithi049 ceph-mon[28399]: [15/May/2024:15:53:25] ENGINE Bus STARTING 2024-05-15T15:53:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:25 smithi049 ceph-mon[28399]: [15/May/2024:15:53:25] ENGINE Serving on https://172.21.15.144:7150 2024-05-15T15:53:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:25 smithi049 ceph-mon[28399]: [15/May/2024:15:53:25] ENGINE Serving on http://172.21.15.144:8765 2024-05-15T15:53:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:25 smithi049 ceph-mon[28399]: [15/May/2024:15:53:25] ENGINE Bus STARTED 2024-05-15T15:53:26.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:25 smithi049 ceph-mon[28399]: mgrmap e22: smithi144.whcjkr(active, since 1.01068s) 2024-05-15T15:53:27.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:26 smithi144 ceph-mon[36389]: pgmap v3: 65 pgs: 65 active+clean; 114 MiB data, 557 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:53:27.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:26 smithi049 ceph-mon[28399]: pgmap v3: 65 pgs: 65 active+clean; 114 MiB data, 557 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:53:27.903 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:27 smithi049 ceph-mon[28399]: pgmap v4: 65 pgs: 65 active+clean; 114 MiB data, 557 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:53:27.903 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:27 smithi049 ceph-mon[28399]: mgrmap e23: smithi144.whcjkr(active, since 2s) 2024-05-15T15:53:28.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:27 smithi144 ceph-mon[36389]: pgmap v4: 65 pgs: 65 active+clean; 114 MiB data, 557 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:53:28.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:27 smithi144 ceph-mon[36389]: mgrmap e23: smithi144.whcjkr(active, since 2s) 2024-05-15T15:53:30.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:29 smithi144 ceph-mon[36389]: pgmap v5: 65 pgs: 65 active+clean; 114 MiB data, 557 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:53:30.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:29 smithi144 ceph-mon[36389]: mgrmap e24: smithi144.whcjkr(active, since 4s) 2024-05-15T15:53:30.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:29 smithi049 ceph-mon[28399]: pgmap v5: 65 pgs: 65 active+clean; 114 MiB data, 557 MiB used, 536 GiB / 536 GiB avail 2024-05-15T15:53:30.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:29 smithi049 ceph-mon[28399]: mgrmap e24: smithi144.whcjkr(active, since 4s) 2024-05-15T15:53:32.165 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:31 smithi049 ceph-mon[28399]: pgmap v6: 65 pgs: 65 active+clean; 119 MiB data, 570 MiB used, 536 GiB / 536 GiB avail; 822 KiB/s rd, 1.1 MiB/s wr, 38 op/s 2024-05-15T15:53:32.188 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:31 smithi144 ceph-mon[36389]: pgmap v6: 65 pgs: 65 active+clean; 119 MiB data, 570 MiB used, 536 GiB / 536 GiB avail; 822 KiB/s rd, 1.1 MiB/s wr, 38 op/s 2024-05-15T15:53:34.168 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:33 smithi049 ceph-mon[28399]: pgmap v7: 65 pgs: 65 active+clean; 132 MiB data, 608 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 176 op/s 2024-05-15T15:53:34.168 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:33 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:34.168 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:33 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:34.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:33 smithi144 ceph-mon[36389]: pgmap v7: 65 pgs: 65 active+clean; 132 MiB data, 608 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 176 op/s 2024-05-15T15:53:34.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:33 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:34.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:33 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:35.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:34 smithi144 ceph-mon[36389]: Standby manager daemon smithi049.vuhzgf started 2024-05-15T15:53:35.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:34 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.49:0/1371268704' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi049.vuhzgf/crt"}]: dispatch 2024-05-15T15:53:35.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:34 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.49:0/1371268704' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:53:35.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:34 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.49:0/1371268704' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi049.vuhzgf/key"}]: dispatch 2024-05-15T15:53:35.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:34 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.49:0/1371268704' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:53:35.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:34 smithi049 ceph-mon[28399]: Standby manager daemon smithi049.vuhzgf started 2024-05-15T15:53:35.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:34 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.49:0/1371268704' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi049.vuhzgf/crt"}]: dispatch 2024-05-15T15:53:35.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:34 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.49:0/1371268704' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:53:35.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:34 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.49:0/1371268704' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi049.vuhzgf/key"}]: dispatch 2024-05-15T15:53:35.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:34 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.49:0/1371268704' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:53:35.920 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:35 smithi144 ceph-mon[36389]: pgmap v8: 65 pgs: 65 active+clean; 132 MiB data, 608 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 137 op/s 2024-05-15T15:53:35.920 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:35 smithi144 ceph-mon[36389]: mgrmap e25: smithi144.whcjkr(active, since 10s), standbys: smithi049.vuhzgf 2024-05-15T15:53:35.920 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:35 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:53:35.921 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:35 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:35.921 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:35 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:36.196 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:35 smithi049 ceph-mon[28399]: pgmap v8: 65 pgs: 65 active+clean; 132 MiB data, 608 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 137 op/s 2024-05-15T15:53:36.196 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:35 smithi049 ceph-mon[28399]: mgrmap e25: smithi144.whcjkr(active, since 10s), standbys: smithi049.vuhzgf 2024-05-15T15:53:36.196 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:35 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:53:36.196 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:35 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:36.196 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:35 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:38.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:37 smithi144 ceph-mon[36389]: pgmap v9: 65 pgs: 65 active+clean; 145 MiB data, 656 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-05-15T15:53:38.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:37 smithi049 ceph-mon[28399]: pgmap v9: 65 pgs: 65 active+clean; 145 MiB data, 656 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-05-15T15:53:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: pgmap v10: 65 pgs: 65 active+clean; 145 MiB data, 656 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-05-15T15:53:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: Adjusting osd_memory_target on smithi144 to 2240M 2024-05-15T15:53:40.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:39 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: pgmap v10: 65 pgs: 65 active+clean; 145 MiB data, 656 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-05-15T15:53:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: Adjusting osd_memory_target on smithi144 to 2240M 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:40.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:39 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:41.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.690+0000 7fa184958700 1 -- 172.21.15.49:0/3753114791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 msgr2=0x7fa180107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:41.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.690+0000 7fa184958700 1 --2- 172.21.15.49:0/3753114791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 0x7fa180107740 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa170009910 tx=0x7fa170009c20 comp rx=0 tx=0).stop 2024-05-15T15:53:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.690+0000 7fa184958700 1 -- 172.21.15.49:0/3753114791 shutdown_connections 2024-05-15T15:53:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.690+0000 7fa184958700 1 --2- 172.21.15.49:0/3753114791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1801084f0 0x7fa180108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.690+0000 7fa184958700 1 --2- 172.21.15.49:0/3753114791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 0x7fa180107740 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.690+0000 7fa184958700 1 -- 172.21.15.49:0/3753114791 >> 172.21.15.49:0/3753114791 conn(0x7fa1800762e0 msgr2=0x7fa180078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.690+0000 7fa184958700 1 -- 172.21.15.49:0/3753114791 shutdown_connections 2024-05-15T15:53:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.690+0000 7fa184958700 1 -- 172.21.15.49:0/3753114791 wait complete. 2024-05-15T15:53:41.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.691+0000 7fa184958700 1 Processor -- start 2024-05-15T15:53:41.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.691+0000 7fa184958700 1 -- start start 2024-05-15T15:53:41.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa184958700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 0x7fa18019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:41.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa184958700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1801084f0 0x7fa18019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:41.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa184958700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18019d810 con 0x7fa180107320 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa184958700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801a21d0 con 0x7fa1801084f0 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa17e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 0x7fa18019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa177fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1801084f0 0x7fa18019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa17e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 0x7fa18019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44990/0 (socket says 172.21.15.49:44990) 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa17e59c700 1 -- 172.21.15.49:0/4007373076 learned_addr learned my addr 172.21.15.49:0/4007373076 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa17e59c700 1 -- 172.21.15.49:0/4007373076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1801084f0 msgr2=0x7fa18019d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa17e59c700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1801084f0 0x7fa18019d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.692+0000 7fa17e59c700 1 -- 172.21.15.49:0/4007373076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1700095f0 con 0x7fa180107320 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.693+0000 7fa17e59c700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 0x7fa18019ccb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b800 tx=0x7fa170005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.693+0000 7fa1777fe700 1 -- 172.21.15.49:0/4007373076 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa17001d070 con 0x7fa180107320 2024-05-15T15:53:41.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.693+0000 7fa1777fe700 1 -- 172.21.15.49:0/4007373076 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1700052a0 con 0x7fa180107320 2024-05-15T15:53:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.693+0000 7fa1777fe700 1 -- 172.21.15.49:0/4007373076 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa170017400 con 0x7fa180107320 2024-05-15T15:53:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.693+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1801a2370 con 0x7fa180107320 2024-05-15T15:53:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.693+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1801a28b0 con 0x7fa180107320 2024-05-15T15:53:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.695+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa180196e90 con 0x7fa180107320 2024-05-15T15:53:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.695+0000 7fa1777fe700 1 -- 172.21.15.49:0/4007373076 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7fa170003780 con 0x7fa180107320 2024-05-15T15:53:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.696+0000 7fa1777fe700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fa16c06d6f0 0x7fa16c06fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:41.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.696+0000 7fa1777fe700 1 -- 172.21.15.49:0/4007373076 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa17008e4e0 con 0x7fa180107320 2024-05-15T15:53:41.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.696+0000 7fa177fff700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fa16c06d6f0 0x7fa16c06fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:41.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.697+0000 7fa177fff700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fa16c06d6f0 0x7fa16c06fbb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa168009c60 tx=0x7fa168009260 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:41.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:41.704+0000 7fa1777fe700 1 -- 172.21.15.49:0/4007373076 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fa170057ca0 con 0x7fa180107320 2024-05-15T15:53:42.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.058+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 --> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa18002cdf0 con 0x7fa16c06d6f0 2024-05-15T15:53:42.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.060+0000 7fa1777fe700 1 -- 172.21.15.49:0/4007373076 <== mgr.24413 v2:172.21.15.144:6828/177983287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fa18002cdf0 con 0x7fa16c06d6f0 2024-05-15T15:53:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fa16c06d6f0 msgr2=0x7fa16c06fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fa16c06d6f0 0x7fa16c06fbb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa168009c60 tx=0x7fa168009260 comp rx=0 tx=0).stop 2024-05-15T15:53:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 msgr2=0x7fa18019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 0x7fa18019ccb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b800 tx=0x7fa170005f00 comp rx=0 tx=0).stop 2024-05-15T15:53:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 shutdown_connections 2024-05-15T15:53:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fa16c06d6f0 0x7fa16c06fbb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180107320 0x7fa18019ccb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 --2- 172.21.15.49:0/4007373076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1801084f0 0x7fa18019d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 >> 172.21.15.49:0/4007373076 conn(0x7fa1800762e0 msgr2=0x7fa18010bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 shutdown_connections 2024-05-15T15:53:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.063+0000 7fa184958700 1 -- 172.21.15.49:0/4007373076 wait complete. 2024-05-15T15:53:42.080 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:53:42.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.221+0000 7fe46403b700 1 -- 172.21.15.49:0/127301097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45c100e50 msgr2=0x7fe45c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.221+0000 7fe46403b700 1 --2- 172.21.15.49:0/127301097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45c100e50 0x7fe45c1012b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009870 tx=0x7fe44c009b80 comp rx=0 tx=0).stop 2024-05-15T15:53:42.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.221+0000 7fe46403b700 1 -- 172.21.15.49:0/127301097 shutdown_connections 2024-05-15T15:53:42.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.221+0000 7fe46403b700 1 --2- 172.21.15.49:0/127301097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45c100e50 0x7fe45c1012b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.221+0000 7fe46403b700 1 --2- 172.21.15.49:0/127301097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0ffc50 0x7fe45c100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.221+0000 7fe46403b700 1 -- 172.21.15.49:0/127301097 >> 172.21.15.49:0/127301097 conn(0x7fe45c0fb1d0 msgr2=0x7fe45c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:42.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.221+0000 7fe46403b700 1 -- 172.21.15.49:0/127301097 shutdown_connections 2024-05-15T15:53:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.222+0000 7fe46403b700 1 -- 172.21.15.49:0/127301097 wait complete. 2024-05-15T15:53:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.223+0000 7fe46403b700 1 Processor -- start 2024-05-15T15:53:42.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.223+0000 7fe46403b700 1 -- start start 2024-05-15T15:53:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.223+0000 7fe46403b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0ffc50 0x7fe45c103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.223+0000 7fe46403b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45c100e50 0x7fe45c102620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.223+0000 7fe46403b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c102b60 con 0x7fe45c0ffc50 2024-05-15T15:53:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.223+0000 7fe46403b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c102ca0 con 0x7fe45c100e50 2024-05-15T15:53:42.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.224+0000 7fe4615d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45c100e50 0x7fe45c102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.224+0000 7fe4615d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45c100e50 0x7fe45c102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:50792/0 (socket says 172.21.15.49:50792) 2024-05-15T15:53:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.224+0000 7fe4615d6700 1 -- 172.21.15.49:0/2090196195 learned_addr learned my addr 172.21.15.49:0/2090196195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.224+0000 7fe461dd7700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0ffc50 0x7fe45c103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.224+0000 7fe461dd7700 1 -- 172.21.15.49:0/2090196195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45c100e50 msgr2=0x7fe45c102620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.224+0000 7fe461dd7700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45c100e50 0x7fe45c102620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.224+0000 7fe461dd7700 1 -- 172.21.15.49:0/2090196195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c009520 con 0x7fe45c0ffc50 2024-05-15T15:53:42.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.224+0000 7fe461dd7700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0ffc50 0x7fe45c103f80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe458005fd0 tx=0x7fe45800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:42.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.225+0000 7fe452ffd700 1 -- 172.21.15.49:0/2090196195 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4580041f0 con 0x7fe45c0ffc50 2024-05-15T15:53:42.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.225+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c102f80 con 0x7fe45c0ffc50 2024-05-15T15:53:42.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.225+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c19e760 con 0x7fe45c0ffc50 2024-05-15T15:53:42.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.225+0000 7fe452ffd700 1 -- 172.21.15.49:0/2090196195 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe458004350 con 0x7fe45c0ffc50 2024-05-15T15:53:42.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.226+0000 7fe452ffd700 1 -- 172.21.15.49:0/2090196195 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4580041f0 con 0x7fe45c0ffc50 2024-05-15T15:53:42.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.227+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe440005320 con 0x7fe45c0ffc50 2024-05-15T15:53:42.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.228+0000 7fe452ffd700 1 -- 172.21.15.49:0/2090196195 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7fe458003b70 con 0x7fe45c0ffc50 2024-05-15T15:53:42.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.229+0000 7fe452ffd700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fe44806d740 0x7fe44806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:42.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.229+0000 7fe452ffd700 1 -- 172.21.15.49:0/2090196195 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe458014070 con 0x7fe45c0ffc50 2024-05-15T15:53:42.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.229+0000 7fe4615d6700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fe44806d740 0x7fe44806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:42.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.230+0000 7fe4615d6700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fe44806d740 0x7fe44806fc00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009f50 tx=0x7fe44c019040 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:42.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.237+0000 7fe452ffd700 1 -- 172.21.15.49:0/2090196195 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fe45805a4a0 con 0x7fe45c0ffc50 2024-05-15T15:53:42.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: pgmap v11: 65 pgs: 65 active+clean; 148 MiB data, 652 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-05-15T15:53:42.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: Adjusting osd_memory_target on smithi049 to 1216M 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:53:42.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:42 smithi049 ceph-mon[28399]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:53:42.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.558+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 --> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe440000bf0 con 0x7fe44806d740 2024-05-15T15:53:42.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.559+0000 7fe452ffd700 1 -- 172.21.15.49:0/2090196195 <== mgr.24413 v2:172.21.15.144:6828/177983287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fe440000bf0 con 0x7fe44806d740 2024-05-15T15:53:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fe44806d740 msgr2=0x7fe44806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fe44806d740 0x7fe44806fc00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009f50 tx=0x7fe44c019040 comp rx=0 tx=0).stop 2024-05-15T15:53:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0ffc50 msgr2=0x7fe45c103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0ffc50 0x7fe45c103f80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe458005fd0 tx=0x7fe45800ecd0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 shutdown_connections 2024-05-15T15:53:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fe44806d740 0x7fe44806fc00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45c0ffc50 0x7fe45c103f80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 --2- 172.21.15.49:0/2090196195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45c100e50 0x7fe45c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 >> 172.21.15.49:0/2090196195 conn(0x7fe45c0fb1d0 msgr2=0x7fe45c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:42.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 shutdown_connections 2024-05-15T15:53:42.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.562+0000 7fe46403b700 1 -- 172.21.15.49:0/2090196195 wait complete. 2024-05-15T15:53:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: pgmap v11: 65 pgs: 65 active+clean; 148 MiB data, 652 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: Adjusting osd_memory_target on smithi049 to 1216M 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:53:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:53:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:53:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:42 smithi144 ceph-mon[36389]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.698+0000 7f700250b700 1 -- 172.21.15.49:0/4026376980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 msgr2=0x7f6ffc103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.698+0000 7f700250b700 1 --2- 172.21.15.49:0/4026376980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 0x7f6ffc103c30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009910 tx=0x7f6fec009c20 comp rx=0 tx=0).stop 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.698+0000 7f700250b700 1 -- 172.21.15.49:0/4026376980 shutdown_connections 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.698+0000 7f700250b700 1 --2- 172.21.15.49:0/4026376980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ffc104170 0x7f6ffc106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.698+0000 7f700250b700 1 --2- 172.21.15.49:0/4026376980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 0x7f6ffc103c30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.698+0000 7f700250b700 1 -- 172.21.15.49:0/4026376980 >> 172.21.15.49:0/4026376980 conn(0x7f6ffc0fb1d0 msgr2=0x7f6ffc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.698+0000 7f700250b700 1 -- 172.21.15.49:0/4026376980 shutdown_connections 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.698+0000 7f700250b700 1 -- 172.21.15.49:0/4026376980 wait complete. 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.699+0000 7f700250b700 1 Processor -- start 2024-05-15T15:53:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.699+0000 7f700250b700 1 -- start start 2024-05-15T15:53:42.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.699+0000 7f700250b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 0x7f6ffc101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f700250b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ffc104170 0x7f6ffc0ff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f700250b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc19a340 con 0x7f6ffc101840 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f700250b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc19a480 con 0x7f6ffc104170 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f6ffbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 0x7f6ffc101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f6ffbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 0x7f6ffc101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45018/0 (socket says 172.21.15.49:45018) 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f6ffbfff700 1 -- 172.21.15.49:0/3171133245 learned_addr learned my addr 172.21.15.49:0/3171133245 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f6ffb7fe700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ffc104170 0x7f6ffc0ff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f6ffbfff700 1 -- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ffc104170 msgr2=0x7f6ffc0ff7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f6ffbfff700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ffc104170 0x7f6ffc0ff7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f6ffbfff700 1 -- 172.21.15.49:0/3171133245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0095f0 con 0x7f6ffc101840 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.700+0000 7f6ffbfff700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 0x7f6ffc101150 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0092e0 tx=0x7f6fec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.701+0000 7f6ff97fa700 1 -- 172.21.15.49:0/3171133245 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fec01d070 con 0x7f6ffc101840 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.701+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc0ffd30 con 0x7f6ffc101840 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.701+0000 7f6ff97fa700 1 -- 172.21.15.49:0/3171133245 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fec00f460 con 0x7f6ffc101840 2024-05-15T15:53:42.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.701+0000 7f6ff97fa700 1 -- 172.21.15.49:0/3171133245 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fec021620 con 0x7f6ffc101840 2024-05-15T15:53:42.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.701+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc1001f0 con 0x7f6ffc101840 2024-05-15T15:53:42.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.702+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc04ec60 con 0x7f6ffc101840 2024-05-15T15:53:42.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.703+0000 7f6ff97fa700 1 -- 172.21.15.49:0/3171133245 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f6fec00f5d0 con 0x7f6ffc101840 2024-05-15T15:53:42.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.704+0000 7f6ff97fa700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f6fe406d6f0 0x7f6fe406fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:42.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.704+0000 7f6ff97fa700 1 -- 172.21.15.49:0/3171133245 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6fec08e400 con 0x7f6ffc101840 2024-05-15T15:53:42.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.704+0000 7f6ffb7fe700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f6fe406d6f0 0x7f6fe406fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:42.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.705+0000 7f6ffb7fe700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f6fe406d6f0 0x7f6fe406fbb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0005950 tx=0x7f6ff000b410 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:42.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.710+0000 7f6ff97fa700 1 -- 172.21.15.49:0/3171133245 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f6fec05b670 con 0x7f6ffc101840 2024-05-15T15:53:42.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.978+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 --> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ffc0008d0 con 0x7f6fe406d6f0 2024-05-15T15:53:42.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.990+0000 7f6ff97fa700 1 -- 172.21.15.49:0/3171133245 <== mgr.24413 v2:172.21.15.144:6828/177983287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6ffc0008d0 con 0x7f6fe406d6f0 2024-05-15T15:53:42.992 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:53:42.992 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (6m) 3s ago 8m 24.2M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (8m) 3s ago 8m 7864k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (7m) 9s ago 7m 7671k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (8m) 3s ago 8m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (7m) 9s ago 7m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (6m) 3s ago 8m 86.1M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (2m) 3s ago 2m 140M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (2m) 3s ago 2m 153M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (2m) 9s ago 2m 21.4M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (2m) 9s ago 2m 24.4M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:9283,8765,8443 running (10m) 3s ago 10m 314M - 18.2.2-1343-g2965081f a04a73ffe079 ea7d1a4b9c55 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (35s) 9s ago 7m 490M - 19.0.0-2576-gde5210d5 48bc18eb22d8 f3b66ea0d25c 2024-05-15T15:53:42.993 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (10m) 3s ago 10m 48.7M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (7m) 9s ago 7m 46.2M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (8m) 3s ago 8m 15.5M - 1.5.0 0da6a335fe13 171e0a34f565 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (7m) 9s ago 7m 15.5M - 1.5.0 0da6a335fe13 be3a2837bc85 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (6m) 3s ago 6m 86.3M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (6m) 3s ago 6m 84.0M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (5m) 3s ago 5m 76.9M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (5m) 9s ago 5m 88.8M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (4m) 9s ago 4m 79.8M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (4m) 9s ago 4m 87.0M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:53:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (6m) 3s ago 7m 59.5M - 2.43.0 a07b618ecd1d 0299663ce1de 2024-05-15T15:53:42.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f6fe406d6f0 msgr2=0x7f6fe406fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f6fe406d6f0 0x7f6fe406fbb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0005950 tx=0x7f6ff000b410 comp rx=0 tx=0).stop 2024-05-15T15:53:42.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 msgr2=0x7f6ffc101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:42.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 0x7f6ffc101150 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0092e0 tx=0x7f6fec00bd70 comp rx=0 tx=0).stop 2024-05-15T15:53:42.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 shutdown_connections 2024-05-15T15:53:42.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f6fe406d6f0 0x7f6fe406fbb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ffc101840 0x7f6ffc101150 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 --2- 172.21.15.49:0/3171133245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ffc104170 0x7f6ffc0ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:42.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 >> 172.21.15.49:0/3171133245 conn(0x7f6ffc0fb1d0 msgr2=0x7f6ffc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:42.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 shutdown_connections 2024-05-15T15:53:42.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:42.993+0000 7f700250b700 1 -- 172.21.15.49:0/3171133245 wait complete. 2024-05-15T15:53:43.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.116+0000 7efd6fb95700 1 -- 172.21.15.49:0/4269915114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68100e50 msgr2=0x7efd681012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:43.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.116+0000 7efd6fb95700 1 --2- 172.21.15.49:0/4269915114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68100e50 0x7efd681012b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efd58009910 tx=0x7efd58009c20 comp rx=0 tx=0).stop 2024-05-15T15:53:43.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.116+0000 7efd6fb95700 1 -- 172.21.15.49:0/4269915114 shutdown_connections 2024-05-15T15:53:43.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.116+0000 7efd6fb95700 1 --2- 172.21.15.49:0/4269915114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68100e50 0x7efd681012b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:43.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.116+0000 7efd6fb95700 1 --2- 172.21.15.49:0/4269915114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680ffc50 0x7efd68100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:43.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.116+0000 7efd6fb95700 1 -- 172.21.15.49:0/4269915114 >> 172.21.15.49:0/4269915114 conn(0x7efd680fb1d0 msgr2=0x7efd680fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:43.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.116+0000 7efd6fb95700 1 -- 172.21.15.49:0/4269915114 shutdown_connections 2024-05-15T15:53:43.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.116+0000 7efd6fb95700 1 -- 172.21.15.49:0/4269915114 wait complete. 2024-05-15T15:53:43.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.117+0000 7efd6fb95700 1 Processor -- start 2024-05-15T15:53:43.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.117+0000 7efd6fb95700 1 -- start start 2024-05-15T15:53:43.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6fb95700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680ffc50 0x7efd68196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:43.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6fb95700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68100e50 0x7efd68196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:43.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6fb95700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd681973b0 con 0x7efd68100e50 2024-05-15T15:53:43.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6fb95700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd681974f0 con 0x7efd680ffc50 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6d931700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680ffc50 0x7efd68196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6d931700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680ffc50 0x7efd68196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:50826/0 (socket says 172.21.15.49:50826) 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6d130700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68100e50 0x7efd68196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6d931700 1 -- 172.21.15.49:0/3590476465 learned_addr learned my addr 172.21.15.49:0/3590476465 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6d931700 1 -- 172.21.15.49:0/3590476465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68100e50 msgr2=0x7efd68196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6d931700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68100e50 0x7efd68196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6d931700 1 -- 172.21.15.49:0/3590476465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd580095f0 con 0x7efd680ffc50 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.118+0000 7efd6d931700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680ffc50 0x7efd68196850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd6400e8f0 tx=0x7efd6400ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.119+0000 7efd5effd700 1 -- 172.21.15.49:0/3590476465 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd64009790 con 0x7efd680ffc50 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.119+0000 7efd5effd700 1 -- 172.21.15.49:0/3590476465 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd6400c810 con 0x7efd680ffc50 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.119+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6819bf00 con 0x7efd680ffc50 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.119+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6819c3c0 con 0x7efd680ffc50 2024-05-15T15:53:43.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.119+0000 7efd5effd700 1 -- 172.21.15.49:0/3590476465 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd64003d20 con 0x7efd680ffc50 2024-05-15T15:53:43.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.120+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6804ec60 con 0x7efd680ffc50 2024-05-15T15:53:43.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.121+0000 7efd5effd700 1 -- 172.21.15.49:0/3590476465 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7efd640041f0 con 0x7efd680ffc50 2024-05-15T15:53:43.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.121+0000 7efd5effd700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7efd5406d6f0 0x7efd5406fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:43.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.121+0000 7efd5effd700 1 -- 172.21.15.49:0/3590476465 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7efd64014070 con 0x7efd680ffc50 2024-05-15T15:53:43.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.121+0000 7efd6d130700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7efd5406d6f0 0x7efd5406fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:43.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.122+0000 7efd6d130700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7efd5406d6f0 0x7efd5406fbb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd5800b800 tx=0x7efd58009f70 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:43.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.127+0000 7efd5effd700 1 -- 172.21.15.49:0/3590476465 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7efd6405a710 con 0x7efd680ffc50 2024-05-15T15:53:43.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.582+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efd6819c8e0 con 0x7efd680ffc50 2024-05-15T15:53:43.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.583+0000 7efd5effd700 1 -- 172.21.15.49:0/3590476465 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+841 (secure 0 0 0) 0x7efd6405a2a0 con 0x7efd680ffc50 2024-05-15T15:53:43.585 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:53:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 1 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:53:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 13, 2024-05-15T15:53:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 1 2024-05-15T15:53:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:53:43.587 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:53:43.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.585+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7efd5406d6f0 msgr2=0x7efd5406fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.585+0000 7efd6fb95700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7efd5406d6f0 0x7efd5406fbb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd5800b800 tx=0x7efd58009f70 comp rx=0 tx=0).stop 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.585+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680ffc50 msgr2=0x7efd68196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.585+0000 7efd6fb95700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680ffc50 0x7efd68196850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd6400e8f0 tx=0x7efd6400ec00 comp rx=0 tx=0).stop 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.586+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 shutdown_connections 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.586+0000 7efd6fb95700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7efd5406d6f0 0x7efd5406fbb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.586+0000 7efd6fb95700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680ffc50 0x7efd68196850 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.586+0000 7efd6fb95700 1 --2- 172.21.15.49:0/3590476465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68100e50 0x7efd68196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.586+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 >> 172.21.15.49:0/3590476465 conn(0x7efd680fb1d0 msgr2=0x7efd68068710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.586+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 shutdown_connections 2024-05-15T15:53:43.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.586+0000 7efd6fb95700 1 -- 172.21.15.49:0/3590476465 wait complete. 2024-05-15T15:53:43.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:53:43.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:53:43.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:43.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:53:43.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:53:43.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:43.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:43 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:53:43.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:53:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:53:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:53:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:43 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:53:43.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.727+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1350878077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f78104060 msgr2=0x7f7f781044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:43.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.727+0000 7f7f7d17c700 1 --2- 172.21.15.49:0/1350878077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f78104060 0x7f7f781044e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009910 tx=0x7f7f6c009c20 comp rx=0 tx=0).stop 2024-05-15T15:53:43.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.727+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1350878077 shutdown_connections 2024-05-15T15:53:43.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.728+0000 7f7f7d17c700 1 --2- 172.21.15.49:0/1350878077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f78104060 0x7f7f781044e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:43.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.728+0000 7f7f7d17c700 1 --2- 172.21.15.49:0/1350878077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f78102f00 0x7f7f78103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:43.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.728+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1350878077 >> 172.21.15.49:0/1350878077 conn(0x7f7f780fe4a0 msgr2=0x7f7f781008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:43.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.728+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1350878077 shutdown_connections 2024-05-15T15:53:43.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.728+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1350878077 wait complete. 2024-05-15T15:53:43.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.728+0000 7f7f7d17c700 1 Processor -- start 2024-05-15T15:53:43.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.729+0000 7f7f7d17c700 1 -- start start 2024-05-15T15:53:43.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.729+0000 7f7f7d17c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f78102f00 0x7f7f781988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:43.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.729+0000 7f7f7d17c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f78104060 0x7f7f78198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:43.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.729+0000 7f7f7d17c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78199440 con 0x7f7f78102f00 2024-05-15T15:53:43.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.729+0000 7f7f7d17c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78199580 con 0x7f7f78104060 2024-05-15T15:53:43.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.730+0000 7f7f76d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f78102f00 0x7f7f781988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:43.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.730+0000 7f7f76d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f78102f00 0x7f7f781988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53048/0 (socket says 172.21.15.49:53048) 2024-05-15T15:53:43.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.730+0000 7f7f76d9d700 1 -- 172.21.15.49:0/1803382246 learned_addr learned my addr 172.21.15.49:0/1803382246 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:53:43.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.730+0000 7f7f7659c700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f78104060 0x7f7f78198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:43.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.730+0000 7f7f7659c700 1 -- 172.21.15.49:0/1803382246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f78102f00 msgr2=0x7f7f781988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:43.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.730+0000 7f7f7659c700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f78102f00 0x7f7f781988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:43.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.730+0000 7f7f7659c700 1 -- 172.21.15.49:0/1803382246 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f68009520 con 0x7f7f78104060 2024-05-15T15:53:43.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.730+0000 7f7f7659c700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f78104060 0x7f7f78198e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000c00 tx=0x7f7f6c00b600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:43.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.731+0000 7f7f5ffff700 1 -- 172.21.15.49:0/1803382246 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f6c01d070 con 0x7f7f78104060 2024-05-15T15:53:43.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.731+0000 7f7f5ffff700 1 -- 172.21.15.49:0/1803382246 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f6c0052a0 con 0x7f7f78104060 2024-05-15T15:53:43.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.731+0000 7f7f5ffff700 1 -- 172.21.15.49:0/1803382246 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f6c017400 con 0x7f7f78104060 2024-05-15T15:53:43.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.731+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c0095f0 con 0x7f7f78104060 2024-05-15T15:53:43.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.731+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7819e310 con 0x7f7f78104060 2024-05-15T15:53:43.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.732+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f78192a00 con 0x7f7f78104060 2024-05-15T15:53:43.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.733+0000 7f7f5ffff700 1 -- 172.21.15.49:0/1803382246 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f7f6c003710 con 0x7f7f78104060 2024-05-15T15:53:43.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.733+0000 7f7f5ffff700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f7f6006d6f0 0x7f7f6006fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:53:43.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.734+0000 7f7f5ffff700 1 -- 172.21.15.49:0/1803382246 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7f6c08e930 con 0x7f7f78104060 2024-05-15T15:53:43.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.734+0000 7f7f76d9d700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f7f6006d6f0 0x7f7f6006fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:53:43.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.735+0000 7f7f76d9d700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f7f6006d6f0 0x7f7f6006fbb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009ef0 tx=0x7f7f68009260 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:53:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:43.738+0000 7f7f5ffff700 1 -- 172.21.15.49:0/1803382246 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f7f6c05bb80 con 0x7f7f78104060 2024-05-15T15:53:44.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.087+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 --> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f7802ce40 con 0x7f7f6006d6f0 2024-05-15T15:53:44.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.088+0000 7f7f5ffff700 1 -- 172.21.15.49:0/1803382246 <== mgr.24413 v2:172.21.15.144:6828/177983287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f7f7802ce40 con 0x7f7f6006d6f0 2024-05-15T15:53:44.090 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:53:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:53:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:53:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:53:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [], 2024-05-15T15:53:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "1/2 daemons upgraded", 2024-05-15T15:53:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "", 2024-05-15T15:53:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:53:44.091 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:53:44.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.090+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f7f6006d6f0 msgr2=0x7f7f6006fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:44.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.090+0000 7f7f7d17c700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f7f6006d6f0 0x7f7f6006fbb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009ef0 tx=0x7f7f68009260 comp rx=0 tx=0).stop 2024-05-15T15:53:44.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.090+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f78104060 msgr2=0x7f7f78198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:53:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.090+0000 7f7f7d17c700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f78104060 0x7f7f78198e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000c00 tx=0x7f7f6c00b600 comp rx=0 tx=0).stop 2024-05-15T15:53:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.090+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 shutdown_connections 2024-05-15T15:53:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.090+0000 7f7f7d17c700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f7f6006d6f0 0x7f7f6006fbb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.090+0000 7f7f7d17c700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f78102f00 0x7f7f781988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.090+0000 7f7f7d17c700 1 --2- 172.21.15.49:0/1803382246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f78104060 0x7f7f78198e20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:53:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.090+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 >> 172.21.15.49:0/1803382246 conn(0x7f7f780fe4a0 msgr2=0x7f7f78107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:53:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.091+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 shutdown_connections 2024-05-15T15:53:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:53:44.091+0000 7f7f7d17c700 1 -- 172.21.15.49:0/1803382246 wait complete. 2024-05-15T15:53:44.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:44 smithi144 ceph-mon[36389]: pgmap v12: 65 pgs: 65 active+clean; 167 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-05-15T15:53:44.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:44 smithi144 ceph-mon[36389]: Reconfiguring prometheus.smithi049 (dependencies changed)... 2024-05-15T15:53:44.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:44 smithi144 ceph-mon[36389]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:44.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:44 smithi144 ceph-mon[36389]: Reconfiguring daemon prometheus.smithi049 on smithi049 2024-05-15T15:53:44.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:44 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3590476465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:53:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:44 smithi049 ceph-mon[28399]: pgmap v12: 65 pgs: 65 active+clean; 167 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-05-15T15:53:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:44 smithi049 ceph-mon[28399]: Reconfiguring prometheus.smithi049 (dependencies changed)... 2024-05-15T15:53:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:44 smithi049 ceph-mon[28399]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:44 smithi049 ceph-mon[28399]: Reconfiguring daemon prometheus.smithi049 on smithi049 2024-05-15T15:53:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:44 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3590476465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:53:45.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:45 smithi144 ceph-mon[36389]: from='client.24467 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:45.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:45 smithi049 ceph-mon[28399]: from='client.24467 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:53:46.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:46 smithi144 ceph-mon[36389]: pgmap v13: 65 pgs: 65 active+clean; 167 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2024-05-15T15:53:46.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:46 smithi049 ceph-mon[28399]: pgmap v13: 65 pgs: 65 active+clean; 167 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2024-05-15T15:53:48.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:48 smithi144 ceph-mon[36389]: pgmap v14: 65 pgs: 65 active+clean; 186 MiB data, 760 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 377 op/s 2024-05-15T15:53:48.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:48 smithi049 ceph-mon[28399]: pgmap v14: 65 pgs: 65 active+clean; 186 MiB data, 760 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 377 op/s 2024-05-15T15:53:49.939 INFO:tasks.workunit.client.0.smithi049.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-15T15:53:49.945 INFO:tasks.workunit.client.0.smithi049.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-15T15:53:49.945 INFO:tasks.workunit.client.0.smithi049.stderr:+ make 2024-05-15T15:53:50.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:50 smithi049 ceph-mon[28399]: pgmap v15: 65 pgs: 65 active+clean; 186 MiB data, 760 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 292 op/s 2024-05-15T15:53:50.474 INFO:tasks.workunit.client.0.smithi049.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-15T15:53:50.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:50 smithi144 ceph-mon[36389]: pgmap v15: 65 pgs: 65 active+clean; 186 MiB data, 760 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 292 op/s 2024-05-15T15:53:51.493 INFO:tasks.workunit.client.0.smithi049.stderr:++ readlink -f fsstress 2024-05-15T15:53:51.495 INFO:tasks.workunit.client.0.smithi049.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-15T15:53:51.495 INFO:tasks.workunit.client.0.smithi049.stderr:+ popd 2024-05-15T15:53:51.496 INFO:tasks.workunit.client.0.smithi049.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-15T15:53:51.496 INFO:tasks.workunit.client.0.smithi049.stderr:+ popd 2024-05-15T15:53:51.497 INFO:tasks.workunit.client.0.smithi049.stdout:~/cephtest/mnt.0/client.0/tmp 2024-05-15T15:53:51.497 INFO:tasks.workunit.client.0.smithi049.stderr:++ mktemp -d -p . 2024-05-15T15:53:51.500 INFO:tasks.workunit.client.0.smithi049.stderr:+ T=./tmp.HkpAGO5zRK 2024-05-15T15:53:51.500 INFO:tasks.workunit.client.0.smithi049.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.HkpAGO5zRK -l 1 -n 1000 -p 10 -v 2024-05-15T15:53:51.504 INFO:tasks.workunit.client.0.smithi049.stdout:seed = 1715697373 2024-05-15T15:53:51.507 INFO:tasks.workunit.client.0.smithi049.stdout:0/0: stat - no entries 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:0/1: getdents . 0 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:0/2: truncate - no filename 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:3/0: fsync - no filename 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:0/3: read - no filename 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:0/4: read - no filename 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:3/1: write - no filename 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:0/5: rmdir - no directory 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:0/6: dwrite - no filename 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:0/7: dread - no filename 2024-05-15T15:53:51.510 INFO:tasks.workunit.client.0.smithi049.stdout:0/8: unlink - no file 2024-05-15T15:53:51.511 INFO:tasks.workunit.client.0.smithi049.stdout:1/0: mkdir d0 0 2024-05-15T15:53:51.512 INFO:tasks.workunit.client.0.smithi049.stdout:2/0: write - no filename 2024-05-15T15:53:51.513 INFO:tasks.workunit.client.0.smithi049.stdout:3/2: mknod c0 0 2024-05-15T15:53:51.513 INFO:tasks.workunit.client.0.smithi049.stdout:3/3: readlink - no filename 2024-05-15T15:53:51.513 INFO:tasks.workunit.client.0.smithi049.stdout:3/4: truncate - no filename 2024-05-15T15:53:51.513 INFO:tasks.workunit.client.0.smithi049.stdout:3/5: stat c0 0 2024-05-15T15:53:51.513 INFO:tasks.workunit.client.0.smithi049.stdout:3/6: dread - no filename 2024-05-15T15:53:51.513 INFO:tasks.workunit.client.0.smithi049.stdout:3/7: stat c0 0 2024-05-15T15:53:51.514 INFO:tasks.workunit.client.0.smithi049.stdout:1/1: creat d0/f1 x:0 0 0 2024-05-15T15:53:51.514 INFO:tasks.workunit.client.0.smithi049.stdout:3/8: stat c0 0 2024-05-15T15:53:51.514 INFO:tasks.workunit.client.0.smithi049.stdout:3/9: fsync - no filename 2024-05-15T15:53:51.515 INFO:tasks.workunit.client.0.smithi049.stdout:4/0: dread - no filename 2024-05-15T15:53:51.515 INFO:tasks.workunit.client.0.smithi049.stdout:1/2: creat d0/f2 x:0 0 0 2024-05-15T15:53:51.517 INFO:tasks.workunit.client.0.smithi049.stdout:2/1: creat f0 x:0 0 0 2024-05-15T15:53:51.518 INFO:tasks.workunit.client.0.smithi049.stdout:5/0: stat - no entries 2024-05-15T15:53:51.518 INFO:tasks.workunit.client.0.smithi049.stdout:5/1: dwrite - no filename 2024-05-15T15:53:51.519 INFO:tasks.workunit.client.0.smithi049.stdout:1/3: creat d0/f3 x:0 0 0 2024-05-15T15:53:51.520 INFO:tasks.workunit.client.0.smithi049.stdout:4/1: getdents . 0 2024-05-15T15:53:51.521 INFO:tasks.workunit.client.0.smithi049.stdout:6/0: readlink - no filename 2024-05-15T15:53:51.521 INFO:tasks.workunit.client.0.smithi049.stdout:6/1: fsync - no filename 2024-05-15T15:53:51.521 INFO:tasks.workunit.client.0.smithi049.stdout:6/2: dwrite - no filename 2024-05-15T15:53:51.521 INFO:tasks.workunit.client.0.smithi049.stdout:6/3: dwrite - no filename 2024-05-15T15:53:51.521 INFO:tasks.workunit.client.0.smithi049.stdout:6/4: rmdir - no directory 2024-05-15T15:53:51.521 INFO:tasks.workunit.client.0.smithi049.stdout:6/5: truncate - no filename 2024-05-15T15:53:51.521 INFO:tasks.workunit.client.0.smithi049.stdout:6/6: fsync - no filename 2024-05-15T15:53:51.524 INFO:tasks.workunit.client.0.smithi049.stdout:1/4: mkdir d0/d4 0 2024-05-15T15:53:51.524 INFO:tasks.workunit.client.0.smithi049.stdout:5/2: creat f0 x:0 0 0 2024-05-15T15:53:51.527 INFO:tasks.workunit.client.0.smithi049.stdout:7/0: unlink - no file 2024-05-15T15:53:51.527 INFO:tasks.workunit.client.0.smithi049.stdout:7/1: readlink - no filename 2024-05-15T15:53:51.527 INFO:tasks.workunit.client.0.smithi049.stdout:7/2: fdatasync - no filename 2024-05-15T15:53:51.527 INFO:tasks.workunit.client.0.smithi049.stdout:7/3: dwrite - no filename 2024-05-15T15:53:51.528 INFO:tasks.workunit.client.0.smithi049.stdout:4/2: creat f0 x:0 0 0 2024-05-15T15:53:51.531 INFO:tasks.workunit.client.0.smithi049.stdout:1/5: dwrite d0/f3 [0,4194304] 0 2024-05-15T15:53:51.752 INFO:tasks.workunit.client.0.smithi049.stdout:5/3: symlink l1 0 2024-05-15T15:53:51.753 INFO:tasks.workunit.client.0.smithi049.stdout:1/6: unlink d0/f2 0 2024-05-15T15:53:51.754 INFO:tasks.workunit.client.0.smithi049.stdout:8/0: rmdir - no directory 2024-05-15T15:53:51.754 INFO:tasks.workunit.client.0.smithi049.stdout:8/1: chown . 339 1 2024-05-15T15:53:51.754 INFO:tasks.workunit.client.0.smithi049.stdout:8/2: dwrite - no filename 2024-05-15T15:53:51.754 INFO:tasks.workunit.client.0.smithi049.stdout:8/3: stat - no entries 2024-05-15T15:53:51.754 INFO:tasks.workunit.client.0.smithi049.stdout:8/4: chown . 11158664 1 2024-05-15T15:53:51.754 INFO:tasks.workunit.client.0.smithi049.stdout:1/7: chown d0/f1 27720962 1 2024-05-15T15:53:51.868 INFO:tasks.workunit.client.0.smithi049.stdout:3/10: getdents . 0 2024-05-15T15:53:51.869 INFO:tasks.workunit.client.0.smithi049.stdout:3/11: chown c0 90 1 2024-05-15T15:53:51.869 INFO:tasks.workunit.client.0.smithi049.stdout:3/12: write - no filename 2024-05-15T15:53:51.873 INFO:tasks.workunit.client.0.smithi049.stdout:2/2: fsync f0 0 2024-05-15T15:53:51.880 INFO:tasks.workunit.client.0.smithi049.stdout:2/3: dwrite f0 [0,4194304] 0 2024-05-15T15:53:51.917 INFO:tasks.workunit.client.0.smithi049.stdout:7/4: mkdir d0 0 2024-05-15T15:53:51.918 INFO:tasks.workunit.client.0.smithi049.stdout:5/4: creat f2 x:0 0 0 2024-05-15T15:53:51.921 INFO:tasks.workunit.client.0.smithi049.stdout:3/13: creat f1 x:0 0 0 2024-05-15T15:53:51.925 INFO:tasks.workunit.client.0.smithi049.stdout:2/4: rename f0 to f1 0 2024-05-15T15:53:51.929 INFO:tasks.workunit.client.0.smithi049.stdout:5/5: symlink l3 0 2024-05-15T15:53:51.930 INFO:tasks.workunit.client.0.smithi049.stdout:5/6: fsync f0 0 2024-05-15T15:53:51.931 INFO:tasks.workunit.client.0.smithi049.stdout:3/14: dwrite f1 [0,4194304] 0 2024-05-15T15:53:51.931 INFO:tasks.workunit.client.0.smithi049.stdout:3/15: readlink - no filename 2024-05-15T15:53:51.932 INFO:tasks.workunit.client.0.smithi049.stdout:5/7: dread - f2 zero size 2024-05-15T15:53:51.934 INFO:tasks.workunit.client.0.smithi049.stdout:8/5: creat f0 x:0 0 0 2024-05-15T15:53:51.938 INFO:tasks.workunit.client.0.smithi049.stdout:1/8: unlink d0/f1 0 2024-05-15T15:53:51.941 INFO:tasks.workunit.client.0.smithi049.stdout:5/8: dwrite f2 [0,4194304] 0 2024-05-15T15:53:51.941 INFO:tasks.workunit.client.0.smithi049.stdout:5/9: fsync f0 0 2024-05-15T15:53:51.944 INFO:tasks.workunit.client.0.smithi049.stdout:2/5: creat f2 x:0 0 0 2024-05-15T15:53:51.946 INFO:tasks.workunit.client.0.smithi049.stdout:4/3: getdents . 0 2024-05-15T15:53:51.947 INFO:tasks.workunit.client.0.smithi049.stdout:4/4: dread - f0 zero size 2024-05-15T15:53:51.949 INFO:tasks.workunit.client.0.smithi049.stdout:7/5: symlink d0/l1 0 2024-05-15T15:53:51.949 INFO:tasks.workunit.client.0.smithi049.stdout:7/6: dread - no filename 2024-05-15T15:53:51.949 INFO:tasks.workunit.client.0.smithi049.stdout:7/7: truncate - no filename 2024-05-15T15:53:51.950 INFO:tasks.workunit.client.0.smithi049.stdout:2/6: dread f1 [0,4194304] 0 2024-05-15T15:53:51.951 INFO:tasks.workunit.client.0.smithi049.stdout:7/8: chown d0/l1 1395612241 1 2024-05-15T15:53:51.952 INFO:tasks.workunit.client.0.smithi049.stdout:7/9: readlink d0/l1 0 2024-05-15T15:53:51.952 INFO:tasks.workunit.client.0.smithi049.stdout:2/7: write f2 [312168,125531] 0 2024-05-15T15:53:51.953 INFO:tasks.workunit.client.0.smithi049.stdout:9/0: mkdir d0 0 2024-05-15T15:53:51.953 INFO:tasks.workunit.client.0.smithi049.stdout:9/1: dread - no filename 2024-05-15T15:53:51.956 INFO:tasks.workunit.client.0.smithi049.stdout:3/16: creat f2 x:0 0 0 2024-05-15T15:53:51.957 INFO:tasks.workunit.client.0.smithi049.stdout:3/17: fdatasync f1 0 2024-05-15T15:53:51.958 INFO:tasks.workunit.client.0.smithi049.stdout:4/5: dwrite f0 [0,4194304] 0 2024-05-15T15:53:51.958 INFO:tasks.workunit.client.0.smithi049.stdout:8/6: mknod c1 0 2024-05-15T15:53:51.962 INFO:tasks.workunit.client.0.smithi049.stdout:1/9: symlink d0/l5 0 2024-05-15T15:53:51.963 INFO:tasks.workunit.client.0.smithi049.stdout:1/10: chown d0/d4 13902306 1 2024-05-15T15:53:51.963 INFO:tasks.workunit.client.0.smithi049.stdout:1/11: write d0/f3 [372916,75385] 0 2024-05-15T15:53:51.966 INFO:tasks.workunit.client.0.smithi049.stdout:7/10: symlink d0/l2 0 2024-05-15T15:53:51.969 INFO:tasks.workunit.client.0.smithi049.stdout:2/8: mkdir d3 0 2024-05-15T15:53:51.976 INFO:tasks.workunit.client.0.smithi049.stdout:3/18: rename f2 to f3 0 2024-05-15T15:53:51.979 INFO:tasks.workunit.client.0.smithi049.stdout:8/7: mkdir d2 0 2024-05-15T15:53:51.983 INFO:tasks.workunit.client.0.smithi049.stdout:1/12: symlink d0/l6 0 2024-05-15T15:53:51.986 INFO:tasks.workunit.client.0.smithi049.stdout:1/13: write d0/f3 [4042793,6983] 0 2024-05-15T15:53:51.987 INFO:tasks.workunit.client.0.smithi049.stdout:7/11: symlink d0/l3 0 2024-05-15T15:53:51.988 INFO:tasks.workunit.client.0.smithi049.stdout:7/12: dwrite - no filename 2024-05-15T15:53:51.988 INFO:tasks.workunit.client.0.smithi049.stdout:1/14: write d0/f3 [5239221,62300] 0 2024-05-15T15:53:51.989 INFO:tasks.workunit.client.0.smithi049.stdout:8/8: dwrite f0 [0,4194304] 0 2024-05-15T15:53:51.992 INFO:tasks.workunit.client.0.smithi049.stdout:1/15: fsync d0/f3 0 2024-05-15T15:53:51.992 INFO:tasks.workunit.client.0.smithi049.stdout:9/2: creat d0/f1 x:0 0 0 2024-05-15T15:53:51.994 INFO:tasks.workunit.client.0.smithi049.stdout:1/16: write d0/f3 [2111000,70609] 0 2024-05-15T15:53:51.995 INFO:tasks.workunit.client.0.smithi049.stdout:1/17: write d0/f3 [2164897,47338] 0 2024-05-15T15:53:51.997 INFO:tasks.workunit.client.0.smithi049.stdout:3/19: mknod c4 0 2024-05-15T15:53:51.999 INFO:tasks.workunit.client.0.smithi049.stdout:4/6: link f0 f1 0 2024-05-15T15:53:52.005 INFO:tasks.workunit.client.0.smithi049.stdout:5/10: link l3 l4 0 2024-05-15T15:53:52.010 INFO:tasks.workunit.client.0.smithi049.stdout:7/13: creat d0/f4 x:0 0 0 2024-05-15T15:53:52.011 INFO:tasks.workunit.client.0.smithi049.stdout:7/14: truncate d0/f4 293379 0 2024-05-15T15:53:52.013 INFO:tasks.workunit.client.0.smithi049.stdout:7/15: dread d0/f4 [0,4194304] 0 2024-05-15T15:53:52.013 INFO:tasks.workunit.client.0.smithi049.stdout:8/9: unlink c1 0 2024-05-15T15:53:52.013 INFO:tasks.workunit.client.0.smithi049.stdout:8/10: write f0 [2665994,48921] 0 2024-05-15T15:53:52.016 INFO:tasks.workunit.client.0.smithi049.stdout:9/3: creat d0/f2 x:0 0 0 2024-05-15T15:53:52.017 INFO:tasks.workunit.client.0.smithi049.stdout:9/4: dread - d0/f1 zero size 2024-05-15T15:53:52.021 INFO:tasks.workunit.client.0.smithi049.stdout:3/20: fdatasync f3 0 2024-05-15T15:53:52.021 INFO:tasks.workunit.client.0.smithi049.stdout:3/21: write f3 [889408,66303] 0 2024-05-15T15:53:52.025 INFO:tasks.workunit.client.0.smithi049.stdout:5/11: symlink l5 0 2024-05-15T15:53:52.030 INFO:tasks.workunit.client.0.smithi049.stdout:4/7: dwrite f1 [0,4194304] 0 2024-05-15T15:53:52.031 INFO:tasks.workunit.client.0.smithi049.stdout:4/8: chown f1 127 1 2024-05-15T15:53:52.032 INFO:tasks.workunit.client.0.smithi049.stdout:4/9: read f0 [1695350,5207] 0 2024-05-15T15:53:52.032 INFO:tasks.workunit.client.0.smithi049.stdout:5/12: dread f2 [0,4194304] 0 2024-05-15T15:53:52.033 INFO:tasks.workunit.client.0.smithi049.stdout:4/10: write f1 [3390159,93765] 0 2024-05-15T15:53:52.036 INFO:tasks.workunit.client.0.smithi049.stdout:1/18: mknod d0/d4/c7 0 2024-05-15T15:53:52.038 INFO:tasks.workunit.client.0.smithi049.stdout:3/22: creat f5 x:0 0 0 2024-05-15T15:53:52.047 INFO:tasks.workunit.client.0.smithi049.stdout:5/13: rename l1 to l6 0 2024-05-15T15:53:52.052 INFO:tasks.workunit.client.0.smithi049.stdout:4/11: symlink l2 0 2024-05-15T15:53:52.055 INFO:tasks.workunit.client.0.smithi049.stdout:3/23: symlink l6 0 2024-05-15T15:53:52.055 INFO:tasks.workunit.client.0.smithi049.stdout:3/24: fdatasync f3 0 2024-05-15T15:53:52.056 INFO:tasks.workunit.client.0.smithi049.stdout:3/25: write f1 [301689,36481] 0 2024-05-15T15:53:52.062 INFO:tasks.workunit.client.0.smithi049.stdout:5/14: mknod c7 0 2024-05-15T15:53:52.063 INFO:tasks.workunit.client.0.smithi049.stdout:5/15: dread - f0 zero size 2024-05-15T15:53:52.063 INFO:tasks.workunit.client.0.smithi049.stdout:5/16: rmdir - no directory 2024-05-15T15:53:52.065 INFO:tasks.workunit.client.0.smithi049.stdout:3/26: dwrite f5 [0,4194304] 0 2024-05-15T15:53:52.092 INFO:tasks.workunit.client.0.smithi049.stdout:3/27: link f1 f7 0 2024-05-15T15:53:52.093 INFO:tasks.workunit.client.0.smithi049.stdout:3/28: chown f1 14240113 1 2024-05-15T15:53:52.298 INFO:tasks.workunit.client.0.smithi049.stdout:2/9: unlink f1 0 2024-05-15T15:53:52.301 INFO:tasks.workunit.client.0.smithi049.stdout:2/10: creat d3/f4 x:0 0 0 2024-05-15T15:53:52.309 INFO:tasks.workunit.client.0.smithi049.stdout:2/11: dwrite d3/f4 [0,4194304] 0 2024-05-15T15:53:52.313 INFO:tasks.workunit.client.0.smithi049.stdout:2/12: dread f2 [0,4194304] 0 2024-05-15T15:53:52.314 INFO:tasks.workunit.client.0.smithi049.stdout:2/13: chown d3 69113 1 2024-05-15T15:53:52.314 INFO:tasks.workunit.client.0.smithi049.stdout:2/14: readlink - no filename 2024-05-15T15:53:52.318 INFO:tasks.workunit.client.0.smithi049.stdout:2/15: dread d3/f4 [0,4194304] 0 2024-05-15T15:53:52.318 INFO:tasks.workunit.client.0.smithi049.stdout:2/16: read f2 [121075,55591] 0 2024-05-15T15:53:52.320 INFO:tasks.workunit.client.0.smithi049.stdout:2/17: dread f2 [0,4194304] 0 2024-05-15T15:53:52.323 INFO:tasks.workunit.client.0.smithi049.stdout:2/18: unlink d3/f4 0 2024-05-15T15:53:52.325 INFO:tasks.workunit.client.0.smithi049.stdout:2/19: symlink d3/l5 0 2024-05-15T15:53:52.329 INFO:tasks.workunit.client.0.smithi049.stdout:2/20: rename f2 to d3/f6 0 2024-05-15T15:53:52.340 INFO:tasks.workunit.client.0.smithi049.stdout:4/12: fsync f0 0 2024-05-15T15:53:52.340 INFO:tasks.workunit.client.0.smithi049.stdout:4/13: read f0 [3526893,58736] 0 2024-05-15T15:53:52.342 INFO:tasks.workunit.client.0.smithi049.stdout:4/14: mknod c3 0 2024-05-15T15:53:52.346 INFO:tasks.workunit.client.0.smithi049.stdout:7/16: getdents d0 0 2024-05-15T15:53:52.347 INFO:tasks.workunit.client.0.smithi049.stdout:7/17: chown d0/l3 15743 1 2024-05-15T15:53:52.349 INFO:tasks.workunit.client.0.smithi049.stdout:7/18: write d0/f4 [1205725,7360] 0 2024-05-15T15:53:52.352 INFO:tasks.workunit.client.0.smithi049.stdout:7/19: creat d0/f5 x:0 0 0 2024-05-15T15:53:52.355 INFO:tasks.workunit.client.0.smithi049.stdout:7/20: unlink d0/f5 0 2024-05-15T15:53:52.359 INFO:tasks.workunit.client.0.smithi049.stdout:7/21: mkdir d0/d6 0 2024-05-15T15:53:52.359 INFO:tasks.workunit.client.0.smithi049.stdout:4/15: write f1 [2729961,52207] 0 2024-05-15T15:53:52.362 INFO:tasks.workunit.client.0.smithi049.stdout:4/16: creat f4 x:0 0 0 2024-05-15T15:53:52.363 INFO:tasks.workunit.client.0.smithi049.stdout:7/22: mknod d0/d6/c7 0 2024-05-15T15:53:52.365 INFO:tasks.workunit.client.0.smithi049.stdout:4/17: rename f4 to f5 0 2024-05-15T15:53:52.371 INFO:tasks.workunit.client.0.smithi049.stdout:7/23: dwrite d0/f4 [0,4194304] 0 2024-05-15T15:53:52.375 INFO:tasks.workunit.client.0.smithi049.stdout:7/24: dread d0/f4 [0,4194304] 0 2024-05-15T15:53:52.377 INFO:tasks.workunit.client.0.smithi049.stdout:7/25: symlink d0/l8 0 2024-05-15T15:53:52.379 INFO:tasks.workunit.client.0.smithi049.stdout:7/26: creat d0/f9 x:0 0 0 2024-05-15T15:53:52.379 INFO:tasks.workunit.client.0.smithi049.stdout:7/27: chown d0/l3 15332742 1 2024-05-15T15:53:52.381 INFO:tasks.workunit.client.0.smithi049.stdout:8/11: truncate f0 119582 0 2024-05-15T15:53:52.387 INFO:tasks.workunit.client.0.smithi049.stdout:5/17: link l6 l8 0 2024-05-15T15:53:52.389 INFO:tasks.workunit.client.0.smithi049.stdout:5/18: truncate f2 2010723 0 2024-05-15T15:53:52.390 INFO:tasks.workunit.client.0.smithi049.stdout:5/19: readlink l6 0 2024-05-15T15:53:52.391 INFO:tasks.workunit.client.0.smithi049.stdout:5/20: write f0 [645971,116475] 0 2024-05-15T15:53:52.394 INFO:tasks.workunit.client.0.smithi049.stdout:1/19: truncate d0/f3 4517452 0 2024-05-15T15:53:52.394 INFO:tasks.workunit.client.0.smithi049.stdout:1/20: chown d0/l6 328905 1 2024-05-15T15:53:52.398 INFO:tasks.workunit.client.0.smithi049.stdout:1/21: chown d0/f3 11000508 1 2024-05-15T15:53:52.413 INFO:tasks.workunit.client.0.smithi049.stdout:3/29: getdents . 0 2024-05-15T15:53:52.413 INFO:tasks.workunit.client.0.smithi049.stdout:3/30: chown l6 248497122 1 2024-05-15T15:53:52.413 INFO:tasks.workunit.client.0.smithi049.stdout:3/31: rmdir - no directory 2024-05-15T15:53:52.413 INFO:tasks.workunit.client.0.smithi049.stdout:3/32: fsync f3 0 2024-05-15T15:53:52.415 INFO:tasks.workunit.client.0.smithi049.stdout:3/33: rename f3 to f8 0 2024-05-15T15:53:52.451 INFO:tasks.workunit.client.0.smithi049.stdout:2/21: rmdir d3 39 2024-05-15T15:53:52.453 INFO:tasks.workunit.client.0.smithi049.stdout:2/22: mkdir d3/d7 0 2024-05-15T15:53:52.455 INFO:tasks.workunit.client.0.smithi049.stdout:2/23: dread d3/f6 [0,4194304] 0 2024-05-15T15:53:52.455 INFO:tasks.workunit.client.0.smithi049.stdout:2/24: readlink d3/l5 0 2024-05-15T15:53:52.460 INFO:tasks.workunit.client.0.smithi049.stdout:7/28: write d0/f4 [2318654,37870] 0 2024-05-15T15:53:52.464 INFO:tasks.workunit.client.0.smithi049.stdout:2/25: dwrite d3/f6 [0,4194304] 0 2024-05-15T15:53:52.465 INFO:tasks.workunit.client.0.smithi049.stdout:2/26: write d3/f6 [453912,13973] 0 2024-05-15T15:53:52.465 INFO:tasks.workunit.client.0.smithi049.stdout:2/27: chown d3 3 1 2024-05-15T15:53:52.468 INFO:tasks.workunit.client.0.smithi049.stdout:2/28: rename d3/d7 to d3/d8 0 2024-05-15T15:53:52.471 INFO:tasks.workunit.client.0.smithi049.stdout:4/18: truncate f0 1310067 0 2024-05-15T15:53:52.476 INFO:tasks.workunit.client.0.smithi049.stdout:2/29: dwrite d3/f6 [0,4194304] 0 2024-05-15T15:53:52.478 INFO:tasks.workunit.client.0.smithi049.stdout:8/12: dread f0 [0,4194304] 0 2024-05-15T15:53:52.478 INFO:tasks.workunit.client.0.smithi049.stdout:2/30: stat d3 0 2024-05-15T15:53:52.478 INFO:tasks.workunit.client.0.smithi049.stdout:2/31: stat d3/f6 0 2024-05-15T15:53:52.479 INFO:tasks.workunit.client.0.smithi049.stdout:2/32: chown d3 58334699 1 2024-05-15T15:53:52.484 INFO:tasks.workunit.client.0.smithi049.stdout:5/21: dread f2 [0,4194304] 0 2024-05-15T15:53:52.485 INFO:tasks.workunit.client.0.smithi049.stdout:5/22: write f0 [910085,91966] 0 2024-05-15T15:53:52.487 INFO:tasks.workunit.client.0.smithi049.stdout:2/33: mkdir d3/d8/d9 0 2024-05-15T15:53:52.489 INFO:tasks.workunit.client.0.smithi049.stdout:1/22: dread d0/f3 [0,4194304] 0 2024-05-15T15:53:52.498 INFO:tasks.workunit.client.0.smithi049.stdout:5/23: link f0 f9 0 2024-05-15T15:53:52.501 INFO:tasks.workunit.client.0.smithi049.stdout:5/24: fdatasync f9 0 2024-05-15T15:53:52.501 INFO:tasks.workunit.client.0.smithi049.stdout:5/25: chown f2 58206 1 2024-05-15T15:53:52.502 INFO:tasks.workunit.client.0.smithi049.stdout:2/34: rmdir d3/d8/d9 0 2024-05-15T15:53:52.510 INFO:tasks.workunit.client.0.smithi049.stdout:2/35: dwrite d3/f6 [0,4194304] 0 2024-05-15T15:53:52.512 INFO:tasks.workunit.client.0.smithi049.stdout:3/34: dwrite f7 [0,4194304] 0 2024-05-15T15:53:52.513 INFO:tasks.workunit.client.0.smithi049.stdout:2/36: unlink d3/l5 0 2024-05-15T15:53:52.514 INFO:tasks.workunit.client.0.smithi049.stdout:2/37: chown d3 23 1 2024-05-15T15:53:52.516 INFO:tasks.workunit.client.0.smithi049.stdout:3/35: dread f1 [0,4194304] 0 2024-05-15T15:53:52.517 INFO:tasks.workunit.client.0.smithi049.stdout:2/38: mkdir d3/da 0 2024-05-15T15:53:52.523 INFO:tasks.workunit.client.0.smithi049.stdout:2/39: mkdir d3/db 0 2024-05-15T15:53:52.528 INFO:tasks.workunit.client.0.smithi049.stdout:3/36: dwrite f8 [0,4194304] 0 2024-05-15T15:53:52.528 INFO:tasks.workunit.client.0.smithi049.stdout:3/37: rmdir - no directory 2024-05-15T15:53:52.529 INFO:tasks.workunit.client.0.smithi049.stdout:2/40: creat d3/da/fc x:0 0 0 2024-05-15T15:53:52.529 INFO:tasks.workunit.client.0.smithi049.stdout:2/41: readlink - no filename 2024-05-15T15:53:52.530 INFO:tasks.workunit.client.0.smithi049.stdout:2/42: dread - d3/da/fc zero size 2024-05-15T15:53:52.532 INFO:tasks.workunit.client.0.smithi049.stdout:3/38: rename f1 to f9 0 2024-05-15T15:53:52.539 INFO:tasks.workunit.client.0.smithi049.stdout:8/13: write f0 [109980,35037] 0 2024-05-15T15:53:52.540 INFO:tasks.workunit.client.0.smithi049.stdout:5/26: write f2 [1187085,90567] 0 2024-05-15T15:53:52.543 INFO:tasks.workunit.client.0.smithi049.stdout:5/27: mknod ca 0 2024-05-15T15:53:52.544 INFO:tasks.workunit.client.0.smithi049.stdout:8/14: creat d2/f3 x:0 0 0 2024-05-15T15:53:52.549 INFO:tasks.workunit.client.0.smithi049.stdout:5/28: unlink l8 0 2024-05-15T15:53:52.551 INFO:tasks.workunit.client.0.smithi049.stdout:1/23: truncate d0/f3 4835795 0 2024-05-15T15:53:52.552 INFO:tasks.workunit.client.0.smithi049.stdout:8/15: dwrite f0 [0,4194304] 0 2024-05-15T15:53:52.553 INFO:tasks.workunit.client.0.smithi049.stdout:1/24: chown d0/l6 11 1 2024-05-15T15:53:52.555 INFO:tasks.workunit.client.0.smithi049.stdout:5/29: chown l6 139 1 2024-05-15T15:53:52.558 INFO:tasks.workunit.client.0.smithi049.stdout:1/25: creat d0/d4/f8 x:0 0 0 2024-05-15T15:53:52.561 INFO:tasks.workunit.client.0.smithi049.stdout:5/30: creat fb x:0 0 0 2024-05-15T15:53:52.565 INFO:tasks.workunit.client.0.smithi049.stdout:5/31: mkdir dc 0 2024-05-15T15:53:52.566 INFO:tasks.workunit.client.0.smithi049.stdout:1/26: dwrite d0/d4/f8 [0,4194304] 0 2024-05-15T15:53:52.571 INFO:tasks.workunit.client.0.smithi049.stdout:5/32: dread f0 [0,4194304] 0 2024-05-15T15:53:52.577 INFO:tasks.workunit.client.0.smithi049.stdout:5/33: mkdir dc/dd 0 2024-05-15T15:53:52.580 INFO:tasks.workunit.client.0.smithi049.stdout:2/43: fsync d3/da/fc 0 2024-05-15T15:53:52.581 INFO:tasks.workunit.client.0.smithi049.stdout:3/39: fsync f9 0 2024-05-15T15:53:52.588 INFO:tasks.workunit.client.0.smithi049.stdout:5/34: dwrite fb [0,4194304] 0 2024-05-15T15:53:52.588 INFO:tasks.workunit.client.0.smithi049.stdout:8/16: getdents d2 0 2024-05-15T15:53:52.591 INFO:tasks.workunit.client.0.smithi049.stdout:8/17: rename d2 to d2/d4 22 2024-05-15T15:53:52.591 INFO:tasks.workunit.client.0.smithi049.stdout:8/18: readlink - no filename 2024-05-15T15:53:52.591 INFO:tasks.workunit.client.0.smithi049.stdout:8/19: readlink - no filename 2024-05-15T15:53:52.592 INFO:tasks.workunit.client.0.smithi049.stdout:5/35: chown ca 3755 1 2024-05-15T15:53:52.594 INFO:tasks.workunit.client.0.smithi049.stdout:1/27: truncate d0/f3 3839685 0 2024-05-15T15:53:52.595 INFO:tasks.workunit.client.0.smithi049.stdout:5/36: write f0 [652914,50576] 0 2024-05-15T15:53:52.599 INFO:tasks.workunit.client.0.smithi049.stdout:4/19: dwrite f0 [0,4194304] 0 2024-05-15T15:53:52.600 INFO:tasks.workunit.client.0.smithi049.stdout:2/44: dwrite d3/f6 [0,4194304] 0 2024-05-15T15:53:52.601 INFO:tasks.workunit.client.0.smithi049.stdout:1/28: creat d0/d4/f9 x:0 0 0 2024-05-15T15:53:52.603 INFO:tasks.workunit.client.0.smithi049.stdout:3/40: dwrite f5 [0,4194304] 0 2024-05-15T15:53:52.604 INFO:tasks.workunit.client.0.smithi049.stdout:3/41: rmdir - no directory 2024-05-15T15:53:52.604 INFO:tasks.workunit.client.0.smithi049.stdout:5/37: symlink dc/le 0 2024-05-15T15:53:52.605 INFO:tasks.workunit.client.0.smithi049.stdout:3/42: fdatasync f8 0 2024-05-15T15:53:52.607 INFO:tasks.workunit.client.0.smithi049.stdout:2/45: write d3/da/fc [609254,97924] 0 2024-05-15T15:53:52.609 INFO:tasks.workunit.client.0.smithi049.stdout:1/29: write d0/d4/f8 [4803309,78933] 0 2024-05-15T15:53:52.611 INFO:tasks.workunit.client.0.smithi049.stdout:3/43: chown f8 8089 1 2024-05-15T15:53:52.611 INFO:tasks.workunit.client.0.smithi049.stdout:1/30: stat d0/l6 0 2024-05-15T15:53:52.613 INFO:tasks.workunit.client.0.smithi049.stdout:3/44: chown c4 10660617 1 2024-05-15T15:53:52.614 INFO:tasks.workunit.client.0.smithi049.stdout:4/20: dwrite f1 [0,4194304] 0 2024-05-15T15:53:52.615 INFO:tasks.workunit.client.0.smithi049.stdout:3/45: chown l6 15 1 2024-05-15T15:53:52.616 INFO:tasks.workunit.client.0.smithi049.stdout:3/46: fsync f9 0 2024-05-15T15:53:52.617 INFO:tasks.workunit.client.0.smithi049.stdout:2/46: symlink d3/d8/ld 0 2024-05-15T15:53:52.618 INFO:tasks.workunit.client.0.smithi049.stdout:2/47: readlink d3/d8/ld 0 2024-05-15T15:53:52.620 INFO:tasks.workunit.client.0.smithi049.stdout:5/38: dwrite f0 [0,4194304] 0 2024-05-15T15:53:52.623 INFO:tasks.workunit.client.0.smithi049.stdout:2/48: dread d3/f6 [0,4194304] 0 2024-05-15T15:53:52.623 INFO:tasks.workunit.client.0.smithi049.stdout:3/47: symlink la 0 2024-05-15T15:53:52.625 INFO:tasks.workunit.client.0.smithi049.stdout:2/49: write d3/f6 [61621,122639] 0 2024-05-15T15:53:52.630 INFO:tasks.workunit.client.0.smithi049.stdout:5/39: rename l5 to dc/lf 0 2024-05-15T15:53:52.631 INFO:tasks.workunit.client.0.smithi049.stdout:3/48: creat fb x:0 0 0 2024-05-15T15:53:52.642 INFO:tasks.workunit.client.0.smithi049.stdout:5/40: mkdir dc/dd/d10 0 2024-05-15T15:53:52.644 INFO:tasks.workunit.client.0.smithi049.stdout:3/49: truncate f9 2108850 0 2024-05-15T15:53:52.646 INFO:tasks.workunit.client.0.smithi049.stdout:5/41: unlink fb 0 2024-05-15T15:53:52.647 INFO:tasks.workunit.client.0.smithi049.stdout:4/21: fsync f0 0 2024-05-15T15:53:52.651 INFO:tasks.workunit.client.0.smithi049.stdout:4/22: symlink l6 0 2024-05-15T15:53:52.652 INFO:tasks.workunit.client.0.smithi049.stdout:4/23: chown l2 12954078 1 2024-05-15T15:53:52.653 INFO:tasks.workunit.client.0.smithi049.stdout:5/42: dwrite f0 [0,4194304] 0 2024-05-15T15:53:52.656 INFO:tasks.workunit.client.0.smithi049.stdout:4/24: rename c3 to c7 0 2024-05-15T15:53:52.661 INFO:tasks.workunit.client.0.smithi049.stdout:4/25: link f0 f8 0 2024-05-15T15:53:52.662 INFO:tasks.workunit.client.0.smithi049.stdout:4/26: readlink l6 0 2024-05-15T15:53:52.662 INFO:tasks.workunit.client.0.smithi049.stdout:2/50: truncate d3/da/fc 1227 0 2024-05-15T15:53:52.665 INFO:tasks.workunit.client.0.smithi049.stdout:4/27: creat f9 x:0 0 0 2024-05-15T15:53:52.675 INFO:tasks.workunit.client.0.smithi049.stdout:5/43: truncate f0 571893 0 2024-05-15T15:53:52.676 INFO:tasks.workunit.client.0.smithi049.stdout:5/44: rmdir dc 39 2024-05-15T15:53:52.677 INFO:tasks.workunit.client.0.smithi049.stdout:4/28: fsync f0 0 2024-05-15T15:53:52.681 INFO:tasks.workunit.client.0.smithi049.stdout:2/51: dread d3/da/fc [0,4194304] 0 2024-05-15T15:53:52.682 INFO:tasks.workunit.client.0.smithi049.stdout:4/29: creat fa x:0 0 0 2024-05-15T15:53:52.684 INFO:tasks.workunit.client.0.smithi049.stdout:3/50: truncate f9 2097135 0 2024-05-15T15:53:52.687 INFO:tasks.workunit.client.0.smithi049.stdout:4/30: write f8 [3698016,69070] 0 2024-05-15T15:53:52.688 INFO:tasks.workunit.client.0.smithi049.stdout:3/51: dread f8 [0,4194304] 0 2024-05-15T15:53:52.689 INFO:tasks.workunit.client.0.smithi049.stdout:3/52: readlink l6 0 2024-05-15T15:53:52.689 INFO:tasks.workunit.client.0.smithi049.stdout:4/31: write fa [900261,72791] 0 2024-05-15T15:53:52.689 INFO:tasks.workunit.client.0.smithi049.stdout:3/53: readlink l6 0 2024-05-15T15:53:52.690 INFO:tasks.workunit.client.0.smithi049.stdout:4/32: chown f9 535048696 1 2024-05-15T15:53:52.690 INFO:tasks.workunit.client.0.smithi049.stdout:5/45: link f2 dc/f11 0 2024-05-15T15:53:52.690 INFO:tasks.workunit.client.0.smithi049.stdout:3/54: read f5 [3238722,110047] 0 2024-05-15T15:53:52.691 INFO:tasks.workunit.client.0.smithi049.stdout:5/46: rename dc to dc/dd/d10/d12 22 2024-05-15T15:53:52.695 INFO:tasks.workunit.client.0.smithi049.stdout:4/33: rename l6 to lb 0 2024-05-15T15:53:52.695 INFO:tasks.workunit.client.0.smithi049.stdout:4/34: chown f1 1 1 2024-05-15T15:53:52.696 INFO:tasks.workunit.client.0.smithi049.stdout:4/35: fsync f1 0 2024-05-15T15:53:52.697 INFO:tasks.workunit.client.0.smithi049.stdout:5/47: creat dc/dd/f13 x:0 0 0 2024-05-15T15:53:52.700 INFO:tasks.workunit.client.0.smithi049.stdout:2/52: truncate d3/da/fc 507350 0 2024-05-15T15:53:52.703 INFO:tasks.workunit.client.0.smithi049.stdout:4/36: creat fc x:0 0 0 2024-05-15T15:53:52.704 INFO:tasks.workunit.client.0.smithi049.stdout:5/48: creat dc/dd/f14 x:0 0 0 2024-05-15T15:53:52.704 INFO:tasks.workunit.client.0.smithi049.stdout:4/37: write f8 [5211105,42893] 0 2024-05-15T15:53:52.705 INFO:tasks.workunit.client.0.smithi049.stdout:5/49: chown dc/le 24501686 1 2024-05-15T15:53:52.707 INFO:tasks.workunit.client.0.smithi049.stdout:2/53: unlink d3/f6 0 2024-05-15T15:53:52.709 INFO:tasks.workunit.client.0.smithi049.stdout:4/38: mkdir dd 0 2024-05-15T15:53:52.713 INFO:tasks.workunit.client.0.smithi049.stdout:3/55: dwrite f7 [0,4194304] 0 2024-05-15T15:53:52.717 INFO:tasks.workunit.client.0.smithi049.stdout:4/39: stat lb 0 2024-05-15T15:53:52.717 INFO:tasks.workunit.client.0.smithi049.stdout:5/50: mknod dc/dd/d10/c15 0 2024-05-15T15:53:52.724 INFO:tasks.workunit.client.0.smithi049.stdout:5/51: truncate dc/f11 374550 0 2024-05-15T15:53:52.725 INFO:tasks.workunit.client.0.smithi049.stdout:2/54: truncate d3/da/fc 860623 0 2024-05-15T15:53:52.727 INFO:tasks.workunit.client.0.smithi049.stdout:4/40: rmdir dd 0 2024-05-15T15:53:52.729 INFO:tasks.workunit.client.0.smithi049.stdout:3/56: fdatasync f7 0 2024-05-15T15:53:52.732 INFO:tasks.workunit.client.0.smithi049.stdout:5/52: link f9 dc/f16 0 2024-05-15T15:53:52.733 INFO:tasks.workunit.client.0.smithi049.stdout:4/41: dwrite f0 [0,4194304] 0 2024-05-15T15:53:52.733 INFO:tasks.workunit.client.0.smithi049.stdout:3/57: creat fc x:0 0 0 2024-05-15T15:53:52.733 INFO:tasks.workunit.client.0.smithi049.stdout:4/42: chown f9 30072697 1 2024-05-15T15:53:52.735 INFO:tasks.workunit.client.0.smithi049.stdout:3/58: write f9 [4135772,104098] 0 2024-05-15T15:53:52.736 INFO:tasks.workunit.client.0.smithi049.stdout:5/53: creat dc/dd/f17 x:0 0 0 2024-05-15T15:53:52.738 INFO:tasks.workunit.client.0.smithi049.stdout:4/43: creat fe x:0 0 0 2024-05-15T15:53:52.744 INFO:tasks.workunit.client.0.smithi049.stdout:5/54: creat dc/dd/f18 x:0 0 0 2024-05-15T15:53:52.744 INFO:tasks.workunit.client.0.smithi049.stdout:3/59: dwrite f8 [0,4194304] 0 2024-05-15T15:53:52.744 INFO:tasks.workunit.client.0.smithi049.stdout:3/60: rmdir - no directory 2024-05-15T15:53:52.746 INFO:tasks.workunit.client.0.smithi049.stdout:5/55: dread f9 [0,4194304] 0 2024-05-15T15:53:52.746 INFO:tasks.workunit.client.0.smithi049.stdout:4/44: rename fe to ff 0 2024-05-15T15:53:52.752 INFO:tasks.workunit.client.0.smithi049.stdout:3/61: dwrite f8 [0,4194304] 0 2024-05-15T15:53:52.762 INFO:tasks.workunit.client.0.smithi049.stdout:2/55: write d3/da/fc [238436,80168] 0 2024-05-15T15:53:52.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:52 smithi144 ceph-mon[36389]: pgmap v16: 65 pgs: 65 active+clean; 188 MiB data, 775 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 316 op/s 2024-05-15T15:53:52.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:52 smithi049 ceph-mon[28399]: pgmap v16: 65 pgs: 65 active+clean; 188 MiB data, 775 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 316 op/s 2024-05-15T15:53:52.959 INFO:tasks.workunit.client.0.smithi049.stdout:3/62: rename c4 to cd 0 2024-05-15T15:53:52.960 INFO:tasks.workunit.client.0.smithi049.stdout:3/63: write fc [943471,124033] 0 2024-05-15T15:53:52.963 INFO:tasks.workunit.client.0.smithi049.stdout:4/45: link ff f10 0 2024-05-15T15:53:52.963 INFO:tasks.workunit.client.0.smithi049.stdout:4/46: read - fc zero size 2024-05-15T15:53:52.964 INFO:tasks.workunit.client.0.smithi049.stdout:3/64: dread f8 [0,4194304] 0 2024-05-15T15:53:52.964 INFO:tasks.workunit.client.0.smithi049.stdout:4/47: chown ff 48726 1 2024-05-15T15:53:52.964 INFO:tasks.workunit.client.0.smithi049.stdout:2/56: creat d3/d8/fe x:0 0 0 2024-05-15T15:53:52.969 INFO:tasks.workunit.client.0.smithi049.stdout:4/48: dwrite f1 [4194304,4194304] 0 2024-05-15T15:53:52.970 INFO:tasks.workunit.client.0.smithi049.stdout:4/49: rmdir - no directory 2024-05-15T15:53:52.975 INFO:tasks.workunit.client.0.smithi049.stdout:4/50: creat f11 x:0 0 0 2024-05-15T15:53:52.979 INFO:tasks.workunit.client.0.smithi049.stdout:4/51: rename f9 to f12 0 2024-05-15T15:53:53.092 INFO:tasks.workunit.client.0.smithi049.stdout:2/57: getdents d3/d8 0 2024-05-15T15:53:53.093 INFO:tasks.workunit.client.0.smithi049.stdout:2/58: write d3/da/fc [585456,82191] 0 2024-05-15T15:53:53.094 INFO:tasks.workunit.client.0.smithi049.stdout:2/59: dread - d3/d8/fe zero size 2024-05-15T15:53:53.094 INFO:tasks.workunit.client.0.smithi049.stdout:5/56: link f2 dc/f19 0 2024-05-15T15:53:53.101 INFO:tasks.workunit.client.0.smithi049.stdout:2/60: dwrite d3/da/fc [0,4194304] 0 2024-05-15T15:53:53.101 INFO:tasks.workunit.client.0.smithi049.stdout:2/61: chown d3/da 15905970 1 2024-05-15T15:53:53.102 INFO:tasks.workunit.client.0.smithi049.stdout:5/57: getdents dc/dd/d10 0 2024-05-15T15:53:53.102 INFO:tasks.workunit.client.0.smithi049.stdout:2/62: dread - d3/d8/fe zero size 2024-05-15T15:53:53.103 INFO:tasks.workunit.client.0.smithi049.stdout:2/63: dread - d3/d8/fe zero size 2024-05-15T15:53:53.106 INFO:tasks.workunit.client.0.smithi049.stdout:2/64: symlink d3/d8/lf 0 2024-05-15T15:53:53.109 INFO:tasks.workunit.client.0.smithi049.stdout:2/65: truncate d3/d8/fe 879943 0 2024-05-15T15:53:53.110 INFO:tasks.workunit.client.0.smithi049.stdout:2/66: chown d3/d8/ld 15711094 1 2024-05-15T15:53:53.111 INFO:tasks.workunit.client.0.smithi049.stdout:4/52: getdents . 0 2024-05-15T15:53:53.113 INFO:tasks.workunit.client.0.smithi049.stdout:5/58: dwrite f0 [0,4194304] 0 2024-05-15T15:53:53.113 INFO:tasks.workunit.client.0.smithi049.stdout:2/67: truncate d3/d8/fe 1045093 0 2024-05-15T15:53:53.114 INFO:tasks.workunit.client.0.smithi049.stdout:4/53: mknod c13 0 2024-05-15T15:53:53.115 INFO:tasks.workunit.client.0.smithi049.stdout:4/54: chown fc 443 1 2024-05-15T15:53:53.118 INFO:tasks.workunit.client.0.smithi049.stdout:2/68: dread d3/da/fc [0,4194304] 0 2024-05-15T15:53:53.119 INFO:tasks.workunit.client.0.smithi049.stdout:5/59: creat dc/dd/d10/f1a x:0 0 0 2024-05-15T15:53:53.122 INFO:tasks.workunit.client.0.smithi049.stdout:4/55: mknod c14 0 2024-05-15T15:53:53.126 INFO:tasks.workunit.client.0.smithi049.stdout:4/56: unlink c7 0 2024-05-15T15:53:53.126 INFO:tasks.workunit.client.0.smithi049.stdout:4/57: readlink lb 0 2024-05-15T15:53:53.128 INFO:tasks.workunit.client.0.smithi049.stdout:4/58: dread - ff zero size 2024-05-15T15:53:53.128 INFO:tasks.workunit.client.0.smithi049.stdout:5/60: dwrite dc/dd/f13 [0,4194304] 0 2024-05-15T15:53:53.129 INFO:tasks.workunit.client.0.smithi049.stdout:5/61: chown dc/dd/f17 46905200 1 2024-05-15T15:53:53.130 INFO:tasks.workunit.client.0.smithi049.stdout:5/62: write f9 [1603568,95944] 0 2024-05-15T15:53:53.131 INFO:tasks.workunit.client.0.smithi049.stdout:5/63: dread - dc/dd/d10/f1a zero size 2024-05-15T15:53:53.134 INFO:tasks.workunit.client.0.smithi049.stdout:4/59: symlink l15 0 2024-05-15T15:53:53.138 INFO:tasks.workunit.client.0.smithi049.stdout:4/60: creat f16 x:0 0 0 2024-05-15T15:53:53.143 INFO:tasks.workunit.client.0.smithi049.stdout:4/61: dread fa [0,4194304] 0 2024-05-15T15:53:53.145 INFO:tasks.workunit.client.0.smithi049.stdout:4/62: creat f17 x:0 0 0 2024-05-15T15:53:53.145 INFO:tasks.workunit.client.0.smithi049.stdout:4/63: chown ff 0 1 2024-05-15T15:53:53.147 INFO:tasks.workunit.client.0.smithi049.stdout:4/64: fsync f5 0 2024-05-15T15:53:53.147 INFO:tasks.workunit.client.0.smithi049.stdout:4/65: stat f10 0 2024-05-15T15:53:53.147 INFO:tasks.workunit.client.0.smithi049.stdout:4/66: dread - fc zero size 2024-05-15T15:53:53.147 INFO:tasks.workunit.client.0.smithi049.stdout:4/67: truncate f12 433535 0 2024-05-15T15:53:53.148 INFO:tasks.workunit.client.0.smithi049.stdout:4/68: fsync f11 0 2024-05-15T15:53:53.148 INFO:tasks.workunit.client.0.smithi049.stdout:4/69: write f16 [95364,63262] 0 2024-05-15T15:53:53.150 INFO:tasks.workunit.client.0.smithi049.stdout:4/70: mkdir d18 0 2024-05-15T15:53:53.152 INFO:tasks.workunit.client.0.smithi049.stdout:4/71: fdatasync f12 0 2024-05-15T15:53:53.152 INFO:tasks.workunit.client.0.smithi049.stdout:4/72: chown f0 6541 1 2024-05-15T15:53:53.154 INFO:tasks.workunit.client.0.smithi049.stdout:4/73: mknod d18/c19 0 2024-05-15T15:53:53.156 INFO:tasks.workunit.client.0.smithi049.stdout:4/74: creat d18/f1a x:0 0 0 2024-05-15T15:53:53.157 INFO:tasks.workunit.client.0.smithi049.stdout:4/75: read f12 [137267,2121] 0 2024-05-15T15:53:53.158 INFO:tasks.workunit.client.0.smithi049.stdout:4/76: chown f8 6 1 2024-05-15T15:53:53.214 INFO:tasks.workunit.client.0.smithi049.stdout:2/69: truncate d3/da/fc 4015216 0 2024-05-15T15:53:53.215 INFO:tasks.workunit.client.0.smithi049.stdout:2/70: truncate d3/d8/fe 2050125 0 2024-05-15T15:53:53.229 INFO:tasks.workunit.client.0.smithi049.stdout:5/64: dwrite dc/f19 [0,4194304] 0 2024-05-15T15:53:53.229 INFO:tasks.workunit.client.0.smithi049.stdout:4/77: truncate f8 5510922 0 2024-05-15T15:53:53.233 INFO:tasks.workunit.client.0.smithi049.stdout:5/65: creat dc/f1b x:0 0 0 2024-05-15T15:53:53.234 INFO:tasks.workunit.client.0.smithi049.stdout:5/66: stat dc/dd/f14 0 2024-05-15T15:53:53.235 INFO:tasks.workunit.client.0.smithi049.stdout:5/67: write dc/f16 [2540431,45000] 0 2024-05-15T15:53:53.235 INFO:tasks.workunit.client.0.smithi049.stdout:4/78: rename ff to d18/f1b 0 2024-05-15T15:53:53.238 INFO:tasks.workunit.client.0.smithi049.stdout:5/68: fsync f0 0 2024-05-15T15:53:53.246 INFO:tasks.workunit.client.0.smithi049.stdout:5/69: rename l4 to dc/dd/l1c 0 2024-05-15T15:53:53.253 INFO:tasks.workunit.client.0.smithi049.stdout:5/70: getdents dc/dd/d10 0 2024-05-15T15:53:53.257 INFO:tasks.workunit.client.0.smithi049.stdout:5/71: rename f2 to dc/dd/f1d 0 2024-05-15T15:53:53.258 INFO:tasks.workunit.client.0.smithi049.stdout:5/72: read dc/dd/f1d [914660,29847] 0 2024-05-15T15:53:53.259 INFO:tasks.workunit.client.0.smithi049.stdout:5/73: chown dc 758 1 2024-05-15T15:53:53.260 INFO:tasks.workunit.client.0.smithi049.stdout:5/74: chown dc/dd/f13 1740 1 2024-05-15T15:53:53.260 INFO:tasks.workunit.client.0.smithi049.stdout:5/75: dread - dc/dd/f17 zero size 2024-05-15T15:53:53.261 INFO:tasks.workunit.client.0.smithi049.stdout:5/76: chown dc/dd/d10/f1a 1524081 1 2024-05-15T15:53:53.264 INFO:tasks.workunit.client.0.smithi049.stdout:5/77: creat dc/dd/d10/f1e x:0 0 0 2024-05-15T15:53:53.264 INFO:tasks.workunit.client.0.smithi049.stdout:5/78: write dc/f16 [2391630,26457] 0 2024-05-15T15:53:53.266 INFO:tasks.workunit.client.0.smithi049.stdout:5/79: fsync dc/dd/d10/f1a 0 2024-05-15T15:53:53.273 INFO:tasks.workunit.client.0.smithi049.stdout:2/71: link d3/da/fc d3/f10 0 2024-05-15T15:53:53.280 INFO:tasks.workunit.client.0.smithi049.stdout:4/79: write f1 [6030773,29416] 0 2024-05-15T15:53:53.281 INFO:tasks.workunit.client.0.smithi049.stdout:4/80: truncate f17 574991 0 2024-05-15T15:53:53.283 INFO:tasks.workunit.client.0.smithi049.stdout:4/81: creat d18/f1c x:0 0 0 2024-05-15T15:53:53.287 INFO:tasks.workunit.client.0.smithi049.stdout:4/82: dread f0 [0,4194304] 0 2024-05-15T15:53:53.287 INFO:tasks.workunit.client.0.smithi049.stdout:2/72: write d3/da/fc [1185947,126239] 0 2024-05-15T15:53:53.290 INFO:tasks.workunit.client.0.smithi049.stdout:4/83: symlink d18/l1d 0 2024-05-15T15:53:53.291 INFO:tasks.workunit.client.0.smithi049.stdout:2/73: unlink d3/da/fc 0 2024-05-15T15:53:53.294 INFO:tasks.workunit.client.0.smithi049.stdout:2/74: link d3/d8/fe d3/f11 0 2024-05-15T15:53:53.296 INFO:tasks.workunit.client.0.smithi049.stdout:2/75: creat d3/db/f12 x:0 0 0 2024-05-15T15:53:53.304 INFO:tasks.workunit.client.0.smithi049.stdout:2/76: fsync d3/f10 0 2024-05-15T15:53:53.318 INFO:tasks.workunit.client.0.smithi049.stdout:2/77: dwrite d3/f10 [0,4194304] 0 2024-05-15T15:53:54.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:54 smithi144 ceph-mon[36389]: pgmap v17: 65 pgs: 65 active+clean; 205 MiB data, 861 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 432 op/s 2024-05-15T15:53:54.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:54 smithi049 ceph-mon[28399]: pgmap v17: 65 pgs: 65 active+clean; 205 MiB data, 861 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 432 op/s 2024-05-15T15:53:55.192 INFO:tasks.workunit.client.1.smithi144.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-15T15:53:55.196 INFO:tasks.workunit.client.1.smithi144.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-15T15:53:55.196 INFO:tasks.workunit.client.1.smithi144.stderr:+ make 2024-05-15T15:53:55.489 INFO:tasks.workunit.client.1.smithi144.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-15T15:53:55.760 INFO:tasks.workunit.client.0.smithi049.stdout:0/9: sync 2024-05-15T15:53:55.760 INFO:tasks.workunit.client.0.smithi049.stdout:0/10: unlink - no file 2024-05-15T15:53:55.760 INFO:tasks.workunit.client.0.smithi049.stdout:0/11: dwrite - no filename 2024-05-15T15:53:55.760 INFO:tasks.workunit.client.0.smithi049.stdout:0/12: rmdir - no directory 2024-05-15T15:53:55.760 INFO:tasks.workunit.client.0.smithi049.stdout:0/13: dread - no filename 2024-05-15T15:53:55.760 INFO:tasks.workunit.client.0.smithi049.stdout:6/7: sync 2024-05-15T15:53:55.760 INFO:tasks.workunit.client.0.smithi049.stdout:9/5: sync 2024-05-15T15:53:55.760 INFO:tasks.workunit.client.0.smithi049.stdout:6/8: write - no filename 2024-05-15T15:53:55.761 INFO:tasks.workunit.client.0.smithi049.stdout:6/9: write - no filename 2024-05-15T15:53:55.761 INFO:tasks.workunit.client.0.smithi049.stdout:6/10: rename - no filename 2024-05-15T15:53:55.761 INFO:tasks.workunit.client.0.smithi049.stdout:6/11: dwrite - no filename 2024-05-15T15:53:55.761 INFO:tasks.workunit.client.0.smithi049.stdout:6/12: rmdir - no directory 2024-05-15T15:53:55.761 INFO:tasks.workunit.client.0.smithi049.stdout:6/13: fsync - no filename 2024-05-15T15:53:55.761 INFO:tasks.workunit.client.0.smithi049.stdout:6/14: truncate - no filename 2024-05-15T15:53:55.761 INFO:tasks.workunit.client.0.smithi049.stdout:0/14: chown . 143 1 2024-05-15T15:53:55.761 INFO:tasks.workunit.client.0.smithi049.stdout:9/6: stat d0/f1 0 2024-05-15T15:53:55.762 INFO:tasks.workunit.client.0.smithi049.stdout:9/7: write d0/f2 [876943,56562] 0 2024-05-15T15:53:55.762 INFO:tasks.workunit.client.0.smithi049.stdout:9/8: write d0/f2 [811971,103521] 0 2024-05-15T15:53:55.762 INFO:tasks.workunit.client.0.smithi049.stdout:6/15: creat f0 x:0 0 0 2024-05-15T15:53:55.763 INFO:tasks.workunit.client.0.smithi049.stdout:0/15: symlink l0 0 2024-05-15T15:53:55.763 INFO:tasks.workunit.client.0.smithi049.stdout:9/9: chown d0/f2 37 1 2024-05-15T15:53:55.763 INFO:tasks.workunit.client.0.smithi049.stdout:0/16: readlink l0 0 2024-05-15T15:53:55.763 INFO:tasks.workunit.client.0.smithi049.stdout:0/17: dwrite - no filename 2024-05-15T15:53:55.765 INFO:tasks.workunit.client.0.smithi049.stdout:6/16: creat f1 x:0 0 0 2024-05-15T15:53:55.766 INFO:tasks.workunit.client.0.smithi049.stdout:9/10: creat d0/f3 x:0 0 0 2024-05-15T15:53:55.768 INFO:tasks.workunit.client.0.smithi049.stdout:0/18: rename l0 to l1 0 2024-05-15T15:53:55.768 INFO:tasks.workunit.client.0.smithi049.stdout:0/19: dread - no filename 2024-05-15T15:53:55.768 INFO:tasks.workunit.client.0.smithi049.stdout:0/20: write - no filename 2024-05-15T15:53:55.768 INFO:tasks.workunit.client.0.smithi049.stdout:0/21: truncate - no filename 2024-05-15T15:53:55.768 INFO:tasks.workunit.client.0.smithi049.stdout:0/22: dwrite - no filename 2024-05-15T15:53:55.768 INFO:tasks.workunit.client.0.smithi049.stdout:0/23: dwrite - no filename 2024-05-15T15:53:55.768 INFO:tasks.workunit.client.0.smithi049.stdout:0/24: write - no filename 2024-05-15T15:53:55.774 INFO:tasks.workunit.client.0.smithi049.stdout:6/17: link f0 f2 0 2024-05-15T15:53:55.774 INFO:tasks.workunit.client.0.smithi049.stdout:9/11: dwrite d0/f1 [0,4194304] 0 2024-05-15T15:53:55.775 INFO:tasks.workunit.client.0.smithi049.stdout:6/18: truncate f0 784958 0 2024-05-15T15:53:55.777 INFO:tasks.workunit.client.0.smithi049.stdout:6/19: dread f0 [0,4194304] 0 2024-05-15T15:53:55.778 INFO:tasks.workunit.client.0.smithi049.stdout:9/12: rename d0/f3 to d0/f4 0 2024-05-15T15:53:55.784 INFO:tasks.workunit.client.0.smithi049.stdout:9/13: mkdir d0/d5 0 2024-05-15T15:53:55.785 INFO:tasks.workunit.client.0.smithi049.stdout:0/25: chown l1 218989783 1 2024-05-15T15:53:55.785 INFO:tasks.workunit.client.0.smithi049.stdout:0/26: dread - no filename 2024-05-15T15:53:55.785 INFO:tasks.workunit.client.0.smithi049.stdout:0/27: readlink l1 0 2024-05-15T15:53:55.785 INFO:tasks.workunit.client.0.smithi049.stdout:0/28: dwrite - no filename 2024-05-15T15:53:55.792 INFO:tasks.workunit.client.0.smithi049.stdout:0/29: creat f2 x:0 0 0 2024-05-15T15:53:55.793 INFO:tasks.workunit.client.0.smithi049.stdout:0/30: stat l1 0 2024-05-15T15:53:55.795 INFO:tasks.workunit.client.0.smithi049.stdout:9/14: dread d0/f2 [0,4194304] 0 2024-05-15T15:53:55.797 INFO:tasks.workunit.client.0.smithi049.stdout:6/20: dwrite f2 [0,4194304] 0 2024-05-15T15:53:55.800 INFO:tasks.workunit.client.0.smithi049.stdout:6/21: write f1 [784473,94681] 0 2024-05-15T15:53:55.807 INFO:tasks.workunit.client.0.smithi049.stdout:0/31: dwrite f2 [0,4194304] 0 2024-05-15T15:53:55.808 INFO:tasks.workunit.client.0.smithi049.stdout:9/15: dwrite d0/f1 [0,4194304] 0 2024-05-15T15:53:55.835 INFO:tasks.workunit.client.0.smithi049.stdout:2/78: sync 2024-05-15T15:53:55.835 INFO:tasks.workunit.client.0.smithi049.stdout:5/80: sync 2024-05-15T15:53:55.835 INFO:tasks.workunit.client.0.smithi049.stdout:4/84: sync 2024-05-15T15:53:55.835 INFO:tasks.workunit.client.0.smithi049.stdout:1/31: sync 2024-05-15T15:53:55.836 INFO:tasks.workunit.client.0.smithi049.stdout:3/65: sync 2024-05-15T15:53:55.836 INFO:tasks.workunit.client.0.smithi049.stdout:8/20: sync 2024-05-15T15:53:55.836 INFO:tasks.workunit.client.0.smithi049.stdout:7/29: sync 2024-05-15T15:53:55.837 INFO:tasks.workunit.client.0.smithi049.stdout:7/30: fsync d0/f4 0 2024-05-15T15:53:55.837 INFO:tasks.workunit.client.0.smithi049.stdout:4/85: write f11 [316314,40051] 0 2024-05-15T15:53:55.838 INFO:tasks.workunit.client.0.smithi049.stdout:1/32: dread - d0/d4/f9 zero size 2024-05-15T15:53:55.838 INFO:tasks.workunit.client.0.smithi049.stdout:6/22: unlink f1 0 2024-05-15T15:53:55.839 INFO:tasks.workunit.client.0.smithi049.stdout:0/32: creat f3 x:0 0 0 2024-05-15T15:53:55.839 INFO:tasks.workunit.client.0.smithi049.stdout:7/31: chown d0/l1 230 1 2024-05-15T15:53:55.843 INFO:tasks.workunit.client.0.smithi049.stdout:3/66: creat fe x:0 0 0 2024-05-15T15:53:55.846 INFO:tasks.workunit.client.0.smithi049.stdout:3/67: write fe [895341,97100] 0 2024-05-15T15:53:55.846 INFO:tasks.workunit.client.0.smithi049.stdout:0/33: dread f2 [0,4194304] 0 2024-05-15T15:53:55.846 INFO:tasks.workunit.client.0.smithi049.stdout:8/21: symlink d2/l5 0 2024-05-15T15:53:55.849 INFO:tasks.workunit.client.0.smithi049.stdout:0/34: truncate f3 234839 0 2024-05-15T15:53:55.850 INFO:tasks.workunit.client.0.smithi049.stdout:7/32: dwrite d0/f4 [0,4194304] 0 2024-05-15T15:53:55.851 INFO:tasks.workunit.client.0.smithi049.stdout:1/33: creat d0/d4/fa x:0 0 0 2024-05-15T15:53:55.852 INFO:tasks.workunit.client.0.smithi049.stdout:6/23: mknod c3 0 2024-05-15T15:53:55.852 INFO:tasks.workunit.client.0.smithi049.stdout:0/35: dread f3 [0,4194304] 0 2024-05-15T15:53:55.856 INFO:tasks.workunit.client.0.smithi049.stdout:9/16: link d0/f1 d0/d5/f6 0 2024-05-15T15:53:55.856 INFO:tasks.workunit.client.0.smithi049.stdout:2/79: dwrite d3/db/f12 [0,4194304] 0 2024-05-15T15:53:55.858 INFO:tasks.workunit.client.0.smithi049.stdout:8/22: rmdir d2 39 2024-05-15T15:53:55.858 INFO:tasks.workunit.client.0.smithi049.stdout:2/80: chown d3/db/f12 75079288 1 2024-05-15T15:53:55.866 INFO:tasks.workunit.client.0.smithi049.stdout:7/33: mkdir d0/d6/da 0 2024-05-15T15:53:55.869 INFO:tasks.workunit.client.0.smithi049.stdout:1/34: creat d0/d4/fb x:0 0 0 2024-05-15T15:53:55.869 INFO:tasks.workunit.client.0.smithi049.stdout:1/35: chown d0/d4/fa 31 1 2024-05-15T15:53:55.870 INFO:tasks.workunit.client.0.smithi049.stdout:1/36: chown d0/d4/f9 28404463 1 2024-05-15T15:53:55.871 INFO:tasks.workunit.client.0.smithi049.stdout:9/17: unlink d0/f4 0 2024-05-15T15:53:55.873 INFO:tasks.workunit.client.0.smithi049.stdout:8/23: dread - d2/f3 zero size 2024-05-15T15:53:55.874 INFO:tasks.workunit.client.0.smithi049.stdout:8/24: read f0 [2974259,68988] 0 2024-05-15T15:53:55.874 INFO:tasks.workunit.client.0.smithi049.stdout:2/81: mknod d3/db/c13 0 2024-05-15T15:53:55.876 INFO:tasks.workunit.client.0.smithi049.stdout:7/34: unlink d0/l8 0 2024-05-15T15:53:55.879 INFO:tasks.workunit.client.0.smithi049.stdout:8/25: mknod d2/c6 0 2024-05-15T15:53:55.880 INFO:tasks.workunit.client.0.smithi049.stdout:8/26: fdatasync d2/f3 0 2024-05-15T15:53:55.880 INFO:tasks.workunit.client.0.smithi049.stdout:8/27: readlink d2/l5 0 2024-05-15T15:53:55.881 INFO:tasks.workunit.client.0.smithi049.stdout:2/82: chown d3/d8/lf 0 1 2024-05-15T15:53:55.882 INFO:tasks.workunit.client.0.smithi049.stdout:2/83: write d3/f10 [4773135,84362] 0 2024-05-15T15:53:55.883 INFO:tasks.workunit.client.0.smithi049.stdout:1/37: link d0/d4/fa d0/d4/fc 0 2024-05-15T15:53:55.883 INFO:tasks.workunit.client.0.smithi049.stdout:2/84: chown d3/d8/ld 456605420 1 2024-05-15T15:53:55.884 INFO:tasks.workunit.client.0.smithi049.stdout:9/18: mknod d0/d5/c7 0 2024-05-15T15:53:55.885 INFO:tasks.workunit.client.1.smithi144.stderr:++ readlink -f fsstress 2024-05-15T15:53:55.886 INFO:tasks.workunit.client.0.smithi049.stdout:2/85: dread d3/d8/fe [0,4194304] 0 2024-05-15T15:53:55.887 INFO:tasks.workunit.client.1.smithi144.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-15T15:53:55.887 INFO:tasks.workunit.client.1.smithi144.stderr:+ popd 2024-05-15T15:53:55.887 INFO:tasks.workunit.client.0.smithi049.stdout:8/28: creat d2/f7 x:0 0 0 2024-05-15T15:53:55.887 INFO:tasks.workunit.client.0.smithi049.stdout:8/29: dread - d2/f3 zero size 2024-05-15T15:53:55.887 INFO:tasks.workunit.client.0.smithi049.stdout:7/35: getdents d0/d6/da 0 2024-05-15T15:53:55.888 INFO:tasks.workunit.client.1.smithi144.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-15T15:53:55.888 INFO:tasks.workunit.client.1.smithi144.stderr:+ popd 2024-05-15T15:53:55.889 INFO:tasks.workunit.client.1.smithi144.stdout:~/cephtest/mnt.1/client.1/tmp 2024-05-15T15:53:55.889 INFO:tasks.workunit.client.1.smithi144.stderr:++ mktemp -d -p . 2024-05-15T15:53:55.889 INFO:tasks.workunit.client.0.smithi049.stdout:1/38: mkdir d0/d4/dd 0 2024-05-15T15:53:55.891 INFO:tasks.workunit.client.0.smithi049.stdout:8/30: mknod d2/c8 0 2024-05-15T15:53:55.893 INFO:tasks.workunit.client.1.smithi144.stderr:+ T=./tmp.AHccOwf9uB 2024-05-15T15:53:55.893 INFO:tasks.workunit.client.1.smithi144.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.AHccOwf9uB -l 1 -n 1000 -p 10 -v 2024-05-15T15:53:55.893 INFO:tasks.workunit.client.0.smithi049.stdout:7/36: creat d0/fb x:0 0 0 2024-05-15T15:53:55.894 INFO:tasks.workunit.client.0.smithi049.stdout:7/37: chown d0/d6 0 1 2024-05-15T15:53:55.894 INFO:tasks.workunit.client.0.smithi049.stdout:7/38: read - d0/f9 zero size 2024-05-15T15:53:55.895 INFO:tasks.workunit.client.0.smithi049.stdout:2/86: fdatasync d3/f11 0 2024-05-15T15:53:55.896 INFO:tasks.workunit.client.0.smithi049.stdout:8/31: mknod d2/c9 0 2024-05-15T15:53:55.897 INFO:tasks.workunit.client.0.smithi049.stdout:8/32: stat d2 0 2024-05-15T15:53:55.898 INFO:tasks.workunit.client.0.smithi049.stdout:1/39: mkdir d0/d4/dd/de 0 2024-05-15T15:53:55.898 INFO:tasks.workunit.client.0.smithi049.stdout:1/40: read - d0/d4/fa zero size 2024-05-15T15:53:55.899 INFO:tasks.workunit.client.0.smithi049.stdout:7/39: rmdir d0/d6 39 2024-05-15T15:53:55.902 INFO:tasks.workunit.client.1.smithi144.stdout:seed = 1716067294 2024-05-15T15:53:55.904 INFO:tasks.workunit.client.0.smithi049.stdout:8/33: mknod d2/ca 0 2024-05-15T15:53:55.910 INFO:tasks.workunit.client.0.smithi049.stdout:1/41: write d0/d4/fa [368282,105193] 0 2024-05-15T15:53:55.913 INFO:tasks.workunit.client.0.smithi049.stdout:6/24: dwrite f0 [4194304,4194304] 0 2024-05-15T15:53:55.914 INFO:tasks.workunit.client.0.smithi049.stdout:6/25: chown f2 20182 1 2024-05-15T15:53:55.917 INFO:tasks.workunit.client.1.smithi144.stdout:2/0: chown . 3 1 2024-05-15T15:53:55.917 INFO:tasks.workunit.client.1.smithi144.stdout:2/1: dread - no filename 2024-05-15T15:53:55.917 INFO:tasks.workunit.client.1.smithi144.stdout:2/2: truncate - no filename 2024-05-15T15:53:55.917 INFO:tasks.workunit.client.1.smithi144.stdout:2/3: rmdir - no directory 2024-05-15T15:53:55.917 INFO:tasks.workunit.client.1.smithi144.stdout:2/4: write - no filename 2024-05-15T15:53:55.917 INFO:tasks.workunit.client.1.smithi144.stdout:2/5: rename - no filename 2024-05-15T15:53:55.918 INFO:tasks.workunit.client.0.smithi049.stdout:7/40: mknod d0/cc 0 2024-05-15T15:53:55.919 INFO:tasks.workunit.client.1.smithi144.stdout:0/0: creat f0 x:0 0 0 2024-05-15T15:53:55.920 INFO:tasks.workunit.client.0.smithi049.stdout:1/42: mkdir d0/d4/dd/df 0 2024-05-15T15:53:55.920 INFO:tasks.workunit.client.0.smithi049.stdout:7/41: truncate d0/fb 623540 0 2024-05-15T15:53:55.921 INFO:tasks.workunit.client.0.smithi049.stdout:6/26: dread f0 [0,4194304] 0 2024-05-15T15:53:55.922 INFO:tasks.workunit.client.0.smithi049.stdout:8/34: dwrite f0 [0,4194304] 0 2024-05-15T15:53:55.923 INFO:tasks.workunit.client.0.smithi049.stdout:6/27: write f2 [8179127,68191] 0 2024-05-15T15:53:55.924 INFO:tasks.workunit.client.0.smithi049.stdout:8/35: chown d2/f7 57579 1 2024-05-15T15:53:55.924 INFO:tasks.workunit.client.0.smithi049.stdout:3/68: write f5 [1554031,51184] 0 2024-05-15T15:53:55.925 INFO:tasks.workunit.client.0.smithi049.stdout:5/81: truncate f9 1148449 0 2024-05-15T15:53:55.926 INFO:tasks.workunit.client.0.smithi049.stdout:8/36: dread - d2/f7 zero size 2024-05-15T15:53:55.928 INFO:tasks.workunit.client.1.smithi144.stdout:0/1: mknod c1 0 2024-05-15T15:53:55.930 INFO:tasks.workunit.client.1.smithi144.stdout:4/0: dwrite - no filename 2024-05-15T15:53:55.931 INFO:tasks.workunit.client.1.smithi144.stdout:4/1: rename - no filename 2024-05-15T15:53:55.931 INFO:tasks.workunit.client.1.smithi144.stdout:4/2: rmdir - no directory 2024-05-15T15:53:55.932 INFO:tasks.workunit.client.0.smithi049.stdout:1/43: rename d0/d4/fc to d0/d4/dd/de/f10 0 2024-05-15T15:53:55.934 INFO:tasks.workunit.client.0.smithi049.stdout:6/28: creat f4 x:0 0 0 2024-05-15T15:53:55.935 INFO:tasks.workunit.client.1.smithi144.stdout:2/6: symlink l0 0 2024-05-15T15:53:55.935 INFO:tasks.workunit.client.1.smithi144.stdout:2/7: truncate - no filename 2024-05-15T15:53:55.936 INFO:tasks.workunit.client.0.smithi049.stdout:3/69: truncate f8 3649352 0 2024-05-15T15:53:55.938 INFO:tasks.workunit.client.0.smithi049.stdout:6/29: dread f0 [0,4194304] 0 2024-05-15T15:53:55.938 INFO:tasks.workunit.client.1.smithi144.stdout:0/2: rename f0 to f2 0 2024-05-15T15:53:55.938 INFO:tasks.workunit.client.1.smithi144.stdout:0/3: readlink - no filename 2024-05-15T15:53:55.940 INFO:tasks.workunit.client.1.smithi144.stdout:5/0: truncate - no filename 2024-05-15T15:53:55.940 INFO:tasks.workunit.client.1.smithi144.stdout:5/1: read - no filename 2024-05-15T15:53:55.940 INFO:tasks.workunit.client.1.smithi144.stdout:5/2: write - no filename 2024-05-15T15:53:55.940 INFO:tasks.workunit.client.1.smithi144.stdout:5/3: dwrite - no filename 2024-05-15T15:53:55.940 INFO:tasks.workunit.client.1.smithi144.stdout:5/4: dwrite - no filename 2024-05-15T15:53:55.942 INFO:tasks.workunit.client.0.smithi049.stdout:8/37: rename d2/f7 to d2/fb 0 2024-05-15T15:53:55.943 INFO:tasks.workunit.client.0.smithi049.stdout:8/38: write f0 [1630630,100189] 0 2024-05-15T15:53:55.944 INFO:tasks.workunit.client.1.smithi144.stdout:3/0: mkdir d0 0 2024-05-15T15:53:55.944 INFO:tasks.workunit.client.1.smithi144.stdout:3/1: fdatasync - no filename 2024-05-15T15:53:55.944 INFO:tasks.workunit.client.1.smithi144.stdout:3/2: readlink - no filename 2024-05-15T15:53:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:55 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:55.950 INFO:tasks.workunit.client.1.smithi144.stdout:6/0: truncate - no filename 2024-05-15T15:53:55.950 INFO:tasks.workunit.client.1.smithi144.stdout:6/1: write - no filename 2024-05-15T15:53:55.951 INFO:tasks.workunit.client.0.smithi049.stdout:3/70: creat ff x:0 0 0 2024-05-15T15:53:55.953 INFO:tasks.workunit.client.1.smithi144.stdout:4/3: creat f0 x:0 0 0 2024-05-15T15:53:55.953 INFO:tasks.workunit.client.1.smithi144.stdout:4/4: stat f0 0 2024-05-15T15:53:55.955 INFO:tasks.workunit.client.0.smithi049.stdout:2/87: dwrite d3/f11 [0,4194304] 0 2024-05-15T15:53:55.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:55 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:53:55.956 INFO:tasks.workunit.client.0.smithi049.stdout:2/88: stat d3/db/f12 0 2024-05-15T15:53:55.956 INFO:tasks.workunit.client.0.smithi049.stdout:2/89: chown d3 0 1 2024-05-15T15:53:55.959 INFO:tasks.workunit.client.1.smithi144.stdout:7/0: rmdir - no directory 2024-05-15T15:53:55.966 INFO:tasks.workunit.client.0.smithi049.stdout:8/39: dwrite f0 [0,4194304] 0 2024-05-15T15:53:55.969 INFO:tasks.workunit.client.0.smithi049.stdout:3/71: unlink fe 0 2024-05-15T15:53:55.969 INFO:tasks.workunit.client.0.smithi049.stdout:3/72: dread - fb zero size 2024-05-15T15:53:55.973 INFO:tasks.workunit.client.0.smithi049.stdout:6/30: link f0 f5 0 2024-05-15T15:53:55.983 INFO:tasks.workunit.client.1.smithi144.stdout:5/5: mkdir d0 0 2024-05-15T15:53:55.983 INFO:tasks.workunit.client.1.smithi144.stdout:3/3: mkdir d0/d1 0 2024-05-15T15:53:55.985 INFO:tasks.workunit.client.1.smithi144.stdout:6/2: mknod c0 0 2024-05-15T15:53:55.985 INFO:tasks.workunit.client.1.smithi144.stdout:6/3: dwrite - no filename 2024-05-15T15:53:55.985 INFO:tasks.workunit.client.1.smithi144.stdout:8/0: write - no filename 2024-05-15T15:53:55.985 INFO:tasks.workunit.client.1.smithi144.stdout:8/1: dwrite - no filename 2024-05-15T15:53:55.985 INFO:tasks.workunit.client.1.smithi144.stdout:6/4: chown c0 1447764 1 2024-05-15T15:53:55.986 INFO:tasks.workunit.client.1.smithi144.stdout:6/5: chown c0 4134 1 2024-05-15T15:53:55.986 INFO:tasks.workunit.client.0.smithi049.stdout:1/44: link d0/d4/c7 d0/d4/dd/df/c11 0 2024-05-15T15:53:55.986 INFO:tasks.workunit.client.1.smithi144.stdout:4/5: fsync f0 0 2024-05-15T15:53:55.989 INFO:tasks.workunit.client.0.smithi049.stdout:8/40: creat d2/fc x:0 0 0 2024-05-15T15:53:55.991 INFO:tasks.workunit.client.0.smithi049.stdout:3/73: creat f10 x:0 0 0 2024-05-15T15:53:55.991 INFO:tasks.workunit.client.0.smithi049.stdout:3/74: rmdir - no directory 2024-05-15T15:53:55.993 INFO:tasks.workunit.client.0.smithi049.stdout:6/31: symlink l6 0 2024-05-15T15:53:55.994 INFO:tasks.workunit.client.0.smithi049.stdout:3/75: dread f9 [0,4194304] 0 2024-05-15T15:53:55.995 INFO:tasks.workunit.client.0.smithi049.stdout:5/82: getdents dc/dd/d10 0 2024-05-15T15:53:55.997 INFO:tasks.workunit.client.0.smithi049.stdout:7/42: getdents d0/d6 0 2024-05-15T15:53:55.998 INFO:tasks.workunit.client.0.smithi049.stdout:2/90: write d3/d8/fe [4243125,21824] 0 2024-05-15T15:53:55.998 INFO:tasks.workunit.client.0.smithi049.stdout:7/43: stat d0/d6/da 0 2024-05-15T15:53:56.002 INFO:tasks.workunit.client.0.smithi049.stdout:3/76: dread f5 [0,4194304] 0 2024-05-15T15:53:56.002 INFO:tasks.workunit.client.0.smithi049.stdout:7/44: fsync d0/f4 0 2024-05-15T15:53:56.002 INFO:tasks.workunit.client.0.smithi049.stdout:3/77: dread - ff zero size 2024-05-15T15:53:56.002 INFO:tasks.workunit.client.0.smithi049.stdout:1/45: mknod d0/d4/c12 0 2024-05-15T15:53:56.002 INFO:tasks.workunit.client.0.smithi049.stdout:8/41: mknod d2/cd 0 2024-05-15T15:53:56.003 INFO:tasks.workunit.client.0.smithi049.stdout:8/42: write f0 [80431,63145] 0 2024-05-15T15:53:56.007 INFO:tasks.workunit.client.0.smithi049.stdout:6/32: creat f7 x:0 0 0 2024-05-15T15:53:56.008 INFO:tasks.workunit.client.0.smithi049.stdout:6/33: write f7 [484019,25341] 0 2024-05-15T15:53:56.011 INFO:tasks.workunit.client.0.smithi049.stdout:8/43: dwrite d2/f3 [0,4194304] 0 2024-05-15T15:53:56.011 INFO:tasks.workunit.client.0.smithi049.stdout:5/83: fdatasync dc/f19 0 2024-05-15T15:53:56.011 INFO:tasks.workunit.client.0.smithi049.stdout:8/44: stat d2/l5 0 2024-05-15T15:53:56.015 INFO:tasks.workunit.client.0.smithi049.stdout:3/78: mknod c11 0 2024-05-15T15:53:56.018 INFO:tasks.workunit.client.0.smithi049.stdout:5/84: dwrite dc/dd/f1d [0,4194304] 0 2024-05-15T15:53:56.019 INFO:tasks.workunit.client.0.smithi049.stdout:1/46: creat d0/d4/dd/de/f13 x:0 0 0 2024-05-15T15:53:56.020 INFO:tasks.workunit.client.0.smithi049.stdout:5/85: write dc/dd/f1d [4617993,16484] 0 2024-05-15T15:53:56.022 INFO:tasks.workunit.client.0.smithi049.stdout:5/86: write dc/f11 [4054703,109675] 0 2024-05-15T15:53:56.023 INFO:tasks.workunit.client.0.smithi049.stdout:5/87: dread - dc/dd/f18 zero size 2024-05-15T15:53:56.023 INFO:tasks.workunit.client.0.smithi049.stdout:5/88: readlink dc/le 0 2024-05-15T15:53:56.025 INFO:tasks.workunit.client.0.smithi049.stdout:5/89: write dc/dd/f18 [942428,80482] 0 2024-05-15T15:53:56.030 INFO:tasks.workunit.client.0.smithi049.stdout:6/34: write f5 [70904,38500] 0 2024-05-15T15:53:56.032 INFO:tasks.workunit.client.0.smithi049.stdout:8/45: mkdir d2/de 0 2024-05-15T15:53:56.033 INFO:tasks.workunit.client.0.smithi049.stdout:5/90: dread dc/dd/f18 [0,4194304] 0 2024-05-15T15:53:56.034 INFO:tasks.workunit.client.0.smithi049.stdout:8/46: chown d2/cd 177517 1 2024-05-15T15:53:56.034 INFO:tasks.workunit.client.0.smithi049.stdout:6/35: dread f7 [0,4194304] 0 2024-05-15T15:53:56.034 INFO:tasks.workunit.client.0.smithi049.stdout:6/36: rmdir - no directory 2024-05-15T15:53:56.035 INFO:tasks.workunit.client.0.smithi049.stdout:5/91: write dc/dd/d10/f1e [834564,60859] 0 2024-05-15T15:53:56.036 INFO:tasks.workunit.client.0.smithi049.stdout:5/92: dread - dc/dd/f17 zero size 2024-05-15T15:53:56.042 INFO:tasks.workunit.client.0.smithi049.stdout:1/47: symlink d0/d4/l14 0 2024-05-15T15:53:56.048 INFO:tasks.workunit.client.0.smithi049.stdout:8/47: creat d2/ff x:0 0 0 2024-05-15T15:53:56.051 INFO:tasks.workunit.client.0.smithi049.stdout:3/79: unlink cd 0 2024-05-15T15:53:56.054 INFO:tasks.workunit.client.0.smithi049.stdout:7/45: link d0/l1 d0/d6/da/ld 0 2024-05-15T15:53:56.054 INFO:tasks.workunit.client.0.smithi049.stdout:5/93: dread dc/f16 [0,4194304] 0 2024-05-15T15:53:56.061 INFO:tasks.workunit.client.0.smithi049.stdout:8/48: creat d2/f10 x:0 0 0 2024-05-15T15:53:56.068 INFO:tasks.workunit.client.0.smithi049.stdout:5/94: unlink c7 0 2024-05-15T15:53:56.070 INFO:tasks.workunit.client.0.smithi049.stdout:1/48: creat d0/d4/dd/df/f15 x:0 0 0 2024-05-15T15:53:56.071 INFO:tasks.workunit.client.0.smithi049.stdout:7/46: rmdir d0 39 2024-05-15T15:53:56.071 INFO:tasks.workunit.client.0.smithi049.stdout:1/49: write d0/d4/fb [931413,119614] 0 2024-05-15T15:53:56.072 INFO:tasks.workunit.client.0.smithi049.stdout:2/91: getdents d3/da 0 2024-05-15T15:53:56.078 INFO:tasks.workunit.client.0.smithi049.stdout:5/95: creat dc/f1f x:0 0 0 2024-05-15T15:53:56.079 INFO:tasks.workunit.client.0.smithi049.stdout:1/50: dwrite d0/d4/dd/df/f15 [0,4194304] 0 2024-05-15T15:53:56.080 INFO:tasks.workunit.client.0.smithi049.stdout:5/96: write dc/f19 [3887332,8213] 0 2024-05-15T15:53:56.081 INFO:tasks.workunit.client.0.smithi049.stdout:7/47: dread - d0/f9 zero size 2024-05-15T15:53:56.083 INFO:tasks.workunit.client.0.smithi049.stdout:2/92: creat d3/d8/f14 x:0 0 0 2024-05-15T15:53:56.090 INFO:tasks.workunit.client.0.smithi049.stdout:2/93: dwrite d3/d8/fe [0,4194304] 0 2024-05-15T15:53:56.092 INFO:tasks.workunit.client.0.smithi049.stdout:2/94: fdatasync d3/d8/f14 0 2024-05-15T15:53:56.095 INFO:tasks.workunit.client.0.smithi049.stdout:1/51: stat d0/d4/c7 0 2024-05-15T15:53:56.096 INFO:tasks.workunit.client.0.smithi049.stdout:1/52: write d0/d4/f8 [4848814,9630] 0 2024-05-15T15:53:56.097 INFO:tasks.workunit.client.0.smithi049.stdout:5/97: fdatasync f9 0 2024-05-15T15:53:56.098 INFO:tasks.workunit.client.0.smithi049.stdout:5/98: dread - dc/dd/d10/f1a zero size 2024-05-15T15:53:56.101 INFO:tasks.workunit.client.0.smithi049.stdout:7/48: readlink d0/d6/da/ld 0 2024-05-15T15:53:56.103 INFO:tasks.workunit.client.0.smithi049.stdout:6/37: truncate f2 7495338 0 2024-05-15T15:53:56.104 INFO:tasks.workunit.client.0.smithi049.stdout:8/49: link d2/c8 d2/c11 0 2024-05-15T15:53:56.106 INFO:tasks.workunit.client.0.smithi049.stdout:5/99: dwrite dc/f11 [0,4194304] 0 2024-05-15T15:53:56.107 INFO:tasks.workunit.client.0.smithi049.stdout:3/80: getdents . 0 2024-05-15T15:53:56.108 INFO:tasks.workunit.client.0.smithi049.stdout:5/100: dread - dc/dd/f17 zero size 2024-05-15T15:53:56.110 INFO:tasks.workunit.client.0.smithi049.stdout:2/95: mkdir d3/da/d15 0 2024-05-15T15:53:56.115 INFO:tasks.workunit.client.0.smithi049.stdout:1/53: mkdir d0/d4/dd/df/d16 0 2024-05-15T15:53:56.123 INFO:tasks.workunit.client.0.smithi049.stdout:6/38: creat f8 x:0 0 0 2024-05-15T15:53:56.124 INFO:tasks.workunit.client.0.smithi049.stdout:8/50: rmdir d2 39 2024-05-15T15:53:56.125 INFO:tasks.workunit.client.0.smithi049.stdout:4/86: sync 2024-05-15T15:53:56.128 INFO:tasks.workunit.client.0.smithi049.stdout:5/101: mknod dc/c20 0 2024-05-15T15:53:56.129 INFO:tasks.workunit.client.0.smithi049.stdout:2/96: mknod d3/db/c16 0 2024-05-15T15:53:56.132 INFO:tasks.workunit.client.0.smithi049.stdout:1/54: creat d0/d4/dd/de/f17 x:0 0 0 2024-05-15T15:53:56.134 INFO:tasks.workunit.client.0.smithi049.stdout:7/49: symlink d0/d6/da/le 0 2024-05-15T15:53:56.135 INFO:tasks.workunit.client.0.smithi049.stdout:5/102: dwrite dc/f1f [0,4194304] 0 2024-05-15T15:53:56.146 INFO:tasks.workunit.client.0.smithi049.stdout:3/81: getdents . 0 2024-05-15T15:53:56.150 INFO:tasks.workunit.client.0.smithi049.stdout:3/82: write f10 [400058,129090] 0 2024-05-15T15:53:56.153 INFO:tasks.workunit.client.0.smithi049.stdout:7/50: dwrite d0/f9 [0,4194304] 0 2024-05-15T15:53:56.154 INFO:tasks.workunit.client.0.smithi049.stdout:1/55: rename d0/d4/dd/de/f10 to d0/d4/dd/f18 0 2024-05-15T15:53:56.155 INFO:tasks.workunit.client.0.smithi049.stdout:3/83: dread - ff zero size 2024-05-15T15:53:56.155 INFO:tasks.workunit.client.0.smithi049.stdout:6/39: dwrite f7 [0,4194304] 0 2024-05-15T15:53:56.157 INFO:tasks.workunit.client.0.smithi049.stdout:5/103: unlink dc/dd/d10/c15 0 2024-05-15T15:53:56.158 INFO:tasks.workunit.client.0.smithi049.stdout:8/51: dwrite d2/f10 [0,4194304] 0 2024-05-15T15:53:56.161 INFO:tasks.workunit.client.0.smithi049.stdout:5/104: write dc/dd/d10/f1e [608246,107842] 0 2024-05-15T15:53:56.162 INFO:tasks.workunit.client.0.smithi049.stdout:5/105: chown dc 378 1 2024-05-15T15:53:56.165 INFO:tasks.workunit.client.0.smithi049.stdout:2/97: rmdir d3/db 39 2024-05-15T15:53:56.165 INFO:tasks.workunit.client.0.smithi049.stdout:7/51: dread d0/f4 [0,4194304] 0 2024-05-15T15:53:56.166 INFO:tasks.workunit.client.0.smithi049.stdout:5/106: write dc/f1b [815517,23759] 0 2024-05-15T15:53:56.166 INFO:tasks.workunit.client.0.smithi049.stdout:4/87: truncate f10 1004030 0 2024-05-15T15:53:56.168 INFO:tasks.workunit.client.0.smithi049.stdout:5/107: stat dc/dd/d10 0 2024-05-15T15:53:56.170 INFO:tasks.workunit.client.0.smithi049.stdout:8/52: dwrite f0 [0,4194304] 0 2024-05-15T15:53:56.178 INFO:tasks.workunit.client.0.smithi049.stdout:3/84: dread f7 [0,4194304] 0 2024-05-15T15:53:56.186 INFO:tasks.workunit.client.0.smithi049.stdout:4/88: mkdir d18/d1e 0 2024-05-15T15:53:56.187 INFO:tasks.workunit.client.0.smithi049.stdout:4/89: chown f16 17 1 2024-05-15T15:53:56.188 INFO:tasks.workunit.client.0.smithi049.stdout:4/90: write f16 [831660,130906] 0 2024-05-15T15:53:56.188 INFO:tasks.workunit.client.0.smithi049.stdout:4/91: dread - d18/f1c zero size 2024-05-15T15:53:56.192 INFO:tasks.workunit.client.0.smithi049.stdout:1/56: link d0/d4/dd/de/f17 d0/f19 0 2024-05-15T15:53:56.193 INFO:tasks.workunit.client.0.smithi049.stdout:3/85: mkdir d12 0 2024-05-15T15:53:56.195 INFO:tasks.workunit.client.0.smithi049.stdout:7/52: symlink d0/lf 0 2024-05-15T15:53:56.196 INFO:tasks.workunit.client.0.smithi049.stdout:7/53: write d0/f4 [4810489,58988] 0 2024-05-15T15:53:56.197 INFO:tasks.workunit.client.0.smithi049.stdout:3/86: dread f5 [0,4194304] 0 2024-05-15T15:53:56.198 INFO:tasks.workunit.client.0.smithi049.stdout:7/54: truncate d0/f4 5129971 0 2024-05-15T15:53:56.198 INFO:tasks.workunit.client.0.smithi049.stdout:7/55: chown d0/f4 11215 1 2024-05-15T15:53:56.200 INFO:tasks.workunit.client.0.smithi049.stdout:8/53: link d2/ff d2/f12 0 2024-05-15T15:53:56.200 INFO:tasks.workunit.client.0.smithi049.stdout:8/54: chown d2 227 1 2024-05-15T15:53:56.200 INFO:tasks.workunit.client.0.smithi049.stdout:8/55: chown d2/de 104629 1 2024-05-15T15:53:56.203 INFO:tasks.workunit.client.0.smithi049.stdout:2/98: symlink d3/da/l17 0 2024-05-15T15:53:56.212 INFO:tasks.workunit.client.0.smithi049.stdout:7/56: dwrite d0/fb [0,4194304] 0 2024-05-15T15:53:56.217 INFO:tasks.workunit.client.0.smithi049.stdout:1/57: dwrite d0/d4/dd/de/f17 [0,4194304] 0 2024-05-15T15:53:56.219 INFO:tasks.workunit.client.0.smithi049.stdout:8/56: dwrite d2/f3 [4194304,4194304] 0 2024-05-15T15:53:56.219 INFO:tasks.workunit.client.0.smithi049.stdout:1/58: readlink d0/l5 0 2024-05-15T15:53:56.224 INFO:tasks.workunit.client.0.smithi049.stdout:2/99: dread d3/db/f12 [0,4194304] 0 2024-05-15T15:53:56.228 INFO:tasks.workunit.client.0.smithi049.stdout:1/59: symlink d0/d4/dd/df/l1a 0 2024-05-15T15:53:56.231 INFO:tasks.workunit.client.0.smithi049.stdout:2/100: readlink d3/d8/ld 0 2024-05-15T15:53:56.232 INFO:tasks.workunit.client.0.smithi049.stdout:7/57: link d0/d6/da/le d0/d6/l10 0 2024-05-15T15:53:56.234 INFO:tasks.workunit.client.0.smithi049.stdout:1/60: creat d0/d4/f1b x:0 0 0 2024-05-15T15:53:56.235 INFO:tasks.workunit.client.0.smithi049.stdout:2/101: dread d3/d8/fe [0,4194304] 0 2024-05-15T15:53:56.239 INFO:tasks.workunit.client.0.smithi049.stdout:2/102: mknod d3/db/c18 0 2024-05-15T15:53:56.240 INFO:tasks.workunit.client.0.smithi049.stdout:2/103: chown d3/db/c18 4518223 1 2024-05-15T15:53:56.241 INFO:tasks.workunit.client.0.smithi049.stdout:1/61: unlink d0/f3 0 2024-05-15T15:53:56.242 INFO:tasks.workunit.client.0.smithi049.stdout:1/62: read d0/d4/dd/f18 [462817,126758] 0 2024-05-15T15:53:56.242 INFO:tasks.workunit.client.0.smithi049.stdout:1/63: readlink d0/l6 0 2024-05-15T15:53:56.243 INFO:tasks.workunit.client.0.smithi049.stdout:7/58: creat d0/f11 x:0 0 0 2024-05-15T15:53:56.245 INFO:tasks.workunit.client.0.smithi049.stdout:2/104: creat d3/da/f19 x:0 0 0 2024-05-15T15:53:56.246 INFO:tasks.workunit.client.0.smithi049.stdout:1/64: mknod d0/c1c 0 2024-05-15T15:53:56.250 INFO:tasks.workunit.client.0.smithi049.stdout:7/59: dwrite d0/f9 [0,4194304] 0 2024-05-15T15:53:56.252 INFO:tasks.workunit.client.0.smithi049.stdout:7/60: stat d0/f4 0 2024-05-15T15:53:56.260 INFO:tasks.workunit.client.0.smithi049.stdout:1/65: dwrite d0/d4/fb [0,4194304] 0 2024-05-15T15:53:56.262 INFO:tasks.workunit.client.0.smithi049.stdout:2/105: dwrite d3/f10 [0,4194304] 0 2024-05-15T15:53:56.263 INFO:tasks.workunit.client.0.smithi049.stdout:7/61: dread d0/f9 [0,4194304] 0 2024-05-15T15:53:56.264 INFO:tasks.workunit.client.0.smithi049.stdout:7/62: read d0/fb [584153,91330] 0 2024-05-15T15:53:56.268 INFO:tasks.workunit.client.0.smithi049.stdout:2/106: dread d3/db/f12 [0,4194304] 0 2024-05-15T15:53:56.275 INFO:tasks.workunit.client.0.smithi049.stdout:2/107: fsync d3/d8/fe 0 2024-05-15T15:53:56.275 INFO:tasks.workunit.client.0.smithi049.stdout:2/108: chown d3/d8 185040723 1 2024-05-15T15:53:56.278 INFO:tasks.workunit.client.0.smithi049.stdout:2/109: creat d3/db/f1a x:0 0 0 2024-05-15T15:53:56.279 INFO:tasks.workunit.client.0.smithi049.stdout:7/63: link d0/l3 d0/d6/da/l12 0 2024-05-15T15:53:56.281 INFO:tasks.workunit.client.0.smithi049.stdout:2/110: mknod d3/da/c1b 0 2024-05-15T15:53:56.339 INFO:tasks.workunit.client.1.smithi144.stdout:2/8: getdents . 0 2024-05-15T15:53:56.340 INFO:tasks.workunit.client.1.smithi144.stdout:0/4: getdents . 0 2024-05-15T15:53:56.341 INFO:tasks.workunit.client.0.smithi049.stdout:9/19: sync 2024-05-15T15:53:56.341 INFO:tasks.workunit.client.0.smithi049.stdout:0/36: sync 2024-05-15T15:53:56.342 INFO:tasks.workunit.client.1.smithi144.stdout:7/1: mknod c0 0 2024-05-15T15:53:56.344 INFO:tasks.workunit.client.0.smithi049.stdout:0/37: dread f3 [0,4194304] 0 2024-05-15T15:53:56.346 INFO:tasks.workunit.client.1.smithi144.stdout:6/6: creat f1 x:0 0 0 2024-05-15T15:53:56.347 INFO:tasks.workunit.client.1.smithi144.stdout:4/6: symlink l1 0 2024-05-15T15:53:56.350 INFO:tasks.workunit.client.1.smithi144.stdout:2/9: symlink l1 0 2024-05-15T15:53:56.350 INFO:tasks.workunit.client.1.smithi144.stdout:2/10: write - no filename 2024-05-15T15:53:56.351 INFO:tasks.workunit.client.1.smithi144.stdout:0/5: write f2 [17292,32523] 0 2024-05-15T15:53:56.351 INFO:tasks.workunit.client.1.smithi144.stdout:0/6: truncate f2 572482 0 2024-05-15T15:53:56.352 INFO:tasks.workunit.client.1.smithi144.stdout:5/6: symlink d0/l1 0 2024-05-15T15:53:56.352 INFO:tasks.workunit.client.1.smithi144.stdout:5/7: chown d0 18243216 1 2024-05-15T15:53:56.353 INFO:tasks.workunit.client.1.smithi144.stdout:7/2: mknod c1 0 2024-05-15T15:53:56.353 INFO:tasks.workunit.client.1.smithi144.stdout:7/3: chown c0 335589 1 2024-05-15T15:53:56.353 INFO:tasks.workunit.client.0.smithi049.stdout:6/40: truncate f0 242366 0 2024-05-15T15:53:56.354 INFO:tasks.workunit.client.1.smithi144.stdout:3/4: getdents d0/d1 0 2024-05-15T15:53:56.356 INFO:tasks.workunit.client.0.smithi049.stdout:6/41: mknod c9 0 2024-05-15T15:53:56.358 INFO:tasks.workunit.client.1.smithi144.stdout:6/7: unlink c0 0 2024-05-15T15:53:56.361 INFO:tasks.workunit.client.1.smithi144.stdout:2/11: creat f2 x:0 0 0 2024-05-15T15:53:56.362 INFO:tasks.workunit.client.1.smithi144.stdout:2/12: write f2 [775170,126952] 0 2024-05-15T15:53:56.363 INFO:tasks.workunit.client.0.smithi049.stdout:6/42: dwrite f7 [0,4194304] 0 2024-05-15T15:53:56.363 INFO:tasks.workunit.client.1.smithi144.stdout:0/7: mknod c3 0 2024-05-15T15:53:56.363 INFO:tasks.workunit.client.0.smithi049.stdout:6/43: dread - f8 zero size 2024-05-15T15:53:56.365 INFO:tasks.workunit.client.1.smithi144.stdout:7/4: creat f2 x:0 0 0 2024-05-15T15:53:56.366 INFO:tasks.workunit.client.1.smithi144.stdout:3/5: symlink d0/l2 0 2024-05-15T15:53:56.367 INFO:tasks.workunit.client.0.smithi049.stdout:8/57: fsync d2/ff 0 2024-05-15T15:53:56.368 INFO:tasks.workunit.client.0.smithi049.stdout:8/58: stat d2/fc 0 2024-05-15T15:53:56.368 INFO:tasks.workunit.client.0.smithi049.stdout:8/59: fsync d2/fc 0 2024-05-15T15:53:56.371 INFO:tasks.workunit.client.1.smithi144.stdout:9/0: mkdir d0 0 2024-05-15T15:53:56.371 INFO:tasks.workunit.client.1.smithi144.stdout:9/1: fdatasync - no filename 2024-05-15T15:53:56.371 INFO:tasks.workunit.client.1.smithi144.stdout:6/8: creat f2 x:0 0 0 2024-05-15T15:53:56.374 INFO:tasks.workunit.client.0.smithi049.stdout:8/60: mknod d2/c13 0 2024-05-15T15:53:56.375 INFO:tasks.workunit.client.0.smithi049.stdout:8/61: read d2/f10 [712439,32513] 0 2024-05-15T15:53:56.377 INFO:tasks.workunit.client.0.smithi049.stdout:4/92: truncate f11 131043 0 2024-05-15T15:53:56.377 INFO:tasks.workunit.client.0.smithi049.stdout:4/93: readlink l2 0 2024-05-15T15:53:56.378 INFO:tasks.workunit.client.0.smithi049.stdout:4/94: chown f1 17922340 1 2024-05-15T15:53:56.378 INFO:tasks.workunit.client.0.smithi049.stdout:4/95: truncate fc 430511 0 2024-05-15T15:53:56.382 INFO:tasks.workunit.client.1.smithi144.stdout:2/13: mkdir d3 0 2024-05-15T15:53:56.382 INFO:tasks.workunit.client.1.smithi144.stdout:0/8: mkdir d4 0 2024-05-15T15:53:56.383 INFO:tasks.workunit.client.1.smithi144.stdout:7/5: creat f3 x:0 0 0 2024-05-15T15:53:56.384 INFO:tasks.workunit.client.1.smithi144.stdout:7/6: dread - f3 zero size 2024-05-15T15:53:56.386 INFO:tasks.workunit.client.1.smithi144.stdout:0/9: dread f2 [0,4194304] 0 2024-05-15T15:53:56.387 INFO:tasks.workunit.client.0.smithi049.stdout:4/96: dwrite d18/f1a [0,4194304] 0 2024-05-15T15:53:56.388 INFO:tasks.workunit.client.1.smithi144.stdout:0/10: write f2 [815395,92867] 0 2024-05-15T15:53:56.388 INFO:tasks.workunit.client.1.smithi144.stdout:5/8: link d0/l1 d0/l2 0 2024-05-15T15:53:56.389 INFO:tasks.workunit.client.0.smithi049.stdout:3/87: dwrite f5 [0,4194304] 0 2024-05-15T15:53:56.390 INFO:tasks.workunit.client.1.smithi144.stdout:0/11: truncate f2 1624323 0 2024-05-15T15:53:56.390 INFO:tasks.workunit.client.0.smithi049.stdout:3/88: write fb [808806,117671] 0 2024-05-15T15:53:56.392 INFO:tasks.workunit.client.1.smithi144.stdout:2/14: dwrite f2 [0,4194304] 0 2024-05-15T15:53:56.393 INFO:tasks.workunit.client.0.smithi049.stdout:4/97: write f8 [6383650,77405] 0 2024-05-15T15:53:56.396 INFO:tasks.workunit.client.0.smithi049.stdout:3/89: symlink d12/l13 0 2024-05-15T15:53:56.401 INFO:tasks.workunit.client.0.smithi049.stdout:4/98: getdents d18/d1e 0 2024-05-15T15:53:56.401 INFO:tasks.workunit.client.0.smithi049.stdout:4/99: readlink l2 0 2024-05-15T15:53:56.402 INFO:tasks.workunit.client.0.smithi049.stdout:8/62: write d2/ff [422721,4659] 0 2024-05-15T15:53:56.403 INFO:tasks.workunit.client.0.smithi049.stdout:4/100: fsync f0 0 2024-05-15T15:53:56.405 INFO:tasks.workunit.client.1.smithi144.stdout:3/6: mknod d0/d1/c3 0 2024-05-15T15:53:56.405 INFO:tasks.workunit.client.1.smithi144.stdout:3/7: dwrite - no filename 2024-05-15T15:53:56.405 INFO:tasks.workunit.client.0.smithi049.stdout:4/101: chown f0 1342 1 2024-05-15T15:53:56.407 INFO:tasks.workunit.client.0.smithi049.stdout:4/102: read f8 [1825384,109186] 0 2024-05-15T15:53:56.407 INFO:tasks.workunit.client.1.smithi144.stdout:7/7: mkdir d4 0 2024-05-15T15:53:56.409 INFO:tasks.workunit.client.0.smithi049.stdout:3/90: dwrite f10 [0,4194304] 0 2024-05-15T15:53:56.409 INFO:tasks.workunit.client.0.smithi049.stdout:4/103: chown f1 3887500 1 2024-05-15T15:53:56.411 INFO:tasks.workunit.client.1.smithi144.stdout:9/2: symlink d0/l1 0 2024-05-15T15:53:56.414 INFO:tasks.workunit.client.1.smithi144.stdout:5/9: creat d0/f3 x:0 0 0 2024-05-15T15:53:56.414 INFO:tasks.workunit.client.0.smithi049.stdout:8/63: dwrite d2/f3 [0,4194304] 0 2024-05-15T15:53:56.415 INFO:tasks.workunit.client.0.smithi049.stdout:3/91: rmdir d12 39 2024-05-15T15:53:56.420 INFO:tasks.workunit.client.0.smithi049.stdout:8/64: mkdir d2/d14 0 2024-05-15T15:53:56.422 INFO:tasks.workunit.client.0.smithi049.stdout:3/92: truncate f9 1718835 0 2024-05-15T15:53:56.424 INFO:tasks.workunit.client.0.smithi049.stdout:4/104: creat d18/d1e/f1f x:0 0 0 2024-05-15T15:53:56.426 INFO:tasks.workunit.client.1.smithi144.stdout:5/10: mkdir d0/d4 0 2024-05-15T15:53:56.427 INFO:tasks.workunit.client.0.smithi049.stdout:3/93: stat d12 0 2024-05-15T15:53:56.428 INFO:tasks.workunit.client.1.smithi144.stdout:5/11: truncate d0/f3 963785 0 2024-05-15T15:53:56.428 INFO:tasks.workunit.client.1.smithi144.stdout:0/12: creat d4/f5 x:0 0 0 2024-05-15T15:53:56.430 INFO:tasks.workunit.client.1.smithi144.stdout:2/15: mknod d3/c4 0 2024-05-15T15:53:56.431 INFO:tasks.workunit.client.0.smithi049.stdout:8/65: creat d2/d14/f15 x:0 0 0 2024-05-15T15:53:56.432 INFO:tasks.workunit.client.1.smithi144.stdout:7/8: mknod d4/c5 0 2024-05-15T15:53:56.435 INFO:tasks.workunit.client.1.smithi144.stdout:5/12: creat d0/f5 x:0 0 0 2024-05-15T15:53:56.436 INFO:tasks.workunit.client.1.smithi144.stdout:0/13: mkdir d4/d6 0 2024-05-15T15:53:56.437 INFO:tasks.workunit.client.1.smithi144.stdout:2/16: dwrite f2 [0,4194304] 0 2024-05-15T15:53:56.440 INFO:tasks.workunit.client.1.smithi144.stdout:9/3: link d0/l1 d0/l2 0 2024-05-15T15:53:56.440 INFO:tasks.workunit.client.1.smithi144.stdout:9/4: fsync - no filename 2024-05-15T15:53:56.442 INFO:tasks.workunit.client.1.smithi144.stdout:2/17: creat d3/f5 x:0 0 0 2024-05-15T15:53:56.443 INFO:tasks.workunit.client.1.smithi144.stdout:2/18: chown l0 6595108 1 2024-05-15T15:53:56.444 INFO:tasks.workunit.client.0.smithi049.stdout:8/66: dwrite d2/fb [0,4194304] 0 2024-05-15T15:53:56.444 INFO:tasks.workunit.client.1.smithi144.stdout:2/19: chown l1 208427 1 2024-05-15T15:53:56.445 INFO:tasks.workunit.client.1.smithi144.stdout:0/14: dwrite d4/f5 [0,4194304] 0 2024-05-15T15:53:56.445 INFO:tasks.workunit.client.1.smithi144.stdout:9/5: symlink d0/l3 0 2024-05-15T15:53:56.446 INFO:tasks.workunit.client.1.smithi144.stdout:0/15: chown f2 267861888 1 2024-05-15T15:53:56.449 INFO:tasks.workunit.client.0.smithi049.stdout:8/67: unlink d2/ff 0 2024-05-15T15:53:56.453 INFO:tasks.workunit.client.1.smithi144.stdout:5/13: rmdir d0/d4 0 2024-05-15T15:53:56.458 INFO:tasks.workunit.client.1.smithi144.stdout:9/6: creat d0/f4 x:0 0 0 2024-05-15T15:53:56.459 INFO:tasks.workunit.client.1.smithi144.stdout:0/16: symlink d4/l7 0 2024-05-15T15:53:56.459 INFO:tasks.workunit.client.0.smithi049.stdout:8/68: dwrite d2/f3 [0,4194304] 0 2024-05-15T15:53:56.461 INFO:tasks.workunit.client.1.smithi144.stdout:9/7: creat d0/f5 x:0 0 0 2024-05-15T15:53:56.463 INFO:tasks.workunit.client.1.smithi144.stdout:0/17: creat d4/d6/f8 x:0 0 0 2024-05-15T15:53:56.463 INFO:tasks.workunit.client.0.smithi049.stdout:7/64: truncate d0/fb 2194194 0 2024-05-15T15:53:56.464 INFO:tasks.workunit.client.1.smithi144.stdout:0/18: truncate d4/f5 5186993 0 2024-05-15T15:53:56.464 INFO:tasks.workunit.client.1.smithi144.stdout:0/19: chown f2 0 1 2024-05-15T15:53:56.467 INFO:tasks.workunit.client.1.smithi144.stdout:9/8: rename d0/l2 to d0/l6 0 2024-05-15T15:53:56.469 INFO:tasks.workunit.client.1.smithi144.stdout:9/9: creat d0/f7 x:0 0 0 2024-05-15T15:53:56.472 INFO:tasks.workunit.client.0.smithi049.stdout:7/65: dwrite d0/f11 [0,4194304] 0 2024-05-15T15:53:56.473 INFO:tasks.workunit.client.0.smithi049.stdout:2/111: truncate d3/f10 4405240 0 2024-05-15T15:53:56.477 INFO:tasks.workunit.client.1.smithi144.stdout:0/20: dwrite f2 [0,4194304] 0 2024-05-15T15:53:56.479 INFO:tasks.workunit.client.0.smithi049.stdout:2/112: dread d3/d8/fe [0,4194304] 0 2024-05-15T15:53:56.479 INFO:tasks.workunit.client.1.smithi144.stdout:0/21: chown d4/d6/f8 27167934 1 2024-05-15T15:53:56.481 INFO:tasks.workunit.client.1.smithi144.stdout:0/22: readlink d4/l7 0 2024-05-15T15:53:56.481 INFO:tasks.workunit.client.1.smithi144.stdout:9/10: dwrite d0/f5 [0,4194304] 0 2024-05-15T15:53:56.482 INFO:tasks.workunit.client.1.smithi144.stdout:9/11: chown d0/f7 21714745 1 2024-05-15T15:53:56.483 INFO:tasks.workunit.client.1.smithi144.stdout:9/12: dread - d0/f7 zero size 2024-05-15T15:53:56.486 INFO:tasks.workunit.client.1.smithi144.stdout:9/13: mkdir d0/d8 0 2024-05-15T15:53:56.490 INFO:tasks.workunit.client.1.smithi144.stdout:9/14: symlink d0/d8/l9 0 2024-05-15T15:53:56.491 INFO:tasks.workunit.client.1.smithi144.stdout:9/15: write d0/f5 [2911367,60750] 0 2024-05-15T15:53:56.516 INFO:tasks.workunit.client.1.smithi144.stdout:4/7: getdents . 0 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:9/20: dwrite d0/f1 [0,4194304] 0 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:0/38: dwrite f2 [0,4194304] 0 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:9/21: mkdir d0/d8 0 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:9/22: mkdir d0/d8/d9 0 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:4/105: truncate f1 420020 0 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:9/23: dread d0/f1 [0,4194304] 0 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:4/106: read d18/f1a [364990,73055] 0 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:5/108: sync 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:9/24: getdents d0/d8/d9 0 2024-05-15T15:53:56.587 INFO:tasks.workunit.client.0.smithi049.stdout:3/94: dread f9 [0,4194304] 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:5/109: unlink dc/dd/f18 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:3/95: mkdir d12/d14 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:5/110: mknod dc/dd/d10/c21 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:5/111: readlink dc/le 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:9/25: creat d0/d8/d9/fa x:0 0 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:9/26: stat d0/d8/d9 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:9/27: write d0/d5/f6 [2946461,114140] 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:9/28: dwrite d0/d8/d9/fa [0,4194304] 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:9/29: stat d0/d5/f6 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:9/30: dwrite d0/d5/f6 [0,4194304] 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:7/66: dread d0/fb [0,4194304] 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:7/67: dread d0/f4 [0,4194304] 0 2024-05-15T15:53:56.588 INFO:tasks.workunit.client.0.smithi049.stdout:7/68: write d0/f11 [745262,38047] 0 2024-05-15T15:53:56.589 INFO:tasks.workunit.client.1.smithi144.stdout:7/9: fsync f2 0 2024-05-15T15:53:56.589 INFO:tasks.workunit.client.1.smithi144.stdout:7/10: chown f3 217165 1 2024-05-15T15:53:56.589 INFO:tasks.workunit.client.1.smithi144.stdout:4/8: dwrite f0 [0,4194304] 0 2024-05-15T15:53:56.589 INFO:tasks.workunit.client.1.smithi144.stdout:7/11: unlink f3 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:7/12: write f2 [707823,10163] 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:4/9: rename l1 to l2 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:7/13: dread f2 [0,4194304] 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:7/14: dwrite f2 [0,4194304] 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:7/15: chown c0 577 1 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:7/16: creat d4/f6 x:0 0 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:4/10: fdatasync f0 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:4/11: mknod c3 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:5/14: fsync d0/f5 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:4/12: mkdir d4 0 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:1/0: sync 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:1/1: write - no filename 2024-05-15T15:53:56.590 INFO:tasks.workunit.client.1.smithi144.stdout:5/15: mkdir d0/d6 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:4/13: dread f0 [0,4194304] 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/2: symlink l0 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:9/16: fsync d0/f4 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/3: stat l0 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/4: readlink l0 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/5: stat l0 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/6: write - no filename 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:5/16: unlink d0/l2 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:9/17: mkdir d0/da 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/7: creat f1 x:0 0 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/8: dread - f1 zero size 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/9: write f1 [61568,97186] 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/10: stat l0 0 2024-05-15T15:53:56.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/11: rmdir - no directory 2024-05-15T15:53:56.592 INFO:tasks.workunit.client.1.smithi144.stdout:9/18: creat d0/d8/fb x:0 0 0 2024-05-15T15:53:56.592 INFO:tasks.workunit.client.1.smithi144.stdout:9/19: write d0/f5 [2823182,48230] 0 2024-05-15T15:53:56.592 INFO:tasks.workunit.client.1.smithi144.stdout:0/23: truncate d4/f5 242275 0 2024-05-15T15:53:56.593 INFO:tasks.workunit.client.0.smithi049.stdout:9/31: rmdir d0/d8 39 2024-05-15T15:53:56.596 INFO:tasks.workunit.client.1.smithi144.stdout:9/20: dwrite d0/f4 [0,4194304] 0 2024-05-15T15:53:56.604 INFO:tasks.workunit.client.1.smithi144.stdout:5/17: rmdir d0/d6 0 2024-05-15T15:53:56.605 INFO:tasks.workunit.client.1.smithi144.stdout:4/14: dwrite f0 [4194304,4194304] 0 2024-05-15T15:53:56.606 INFO:tasks.workunit.client.0.smithi049.stdout:0/39: truncate f2 3611922 0 2024-05-15T15:53:56.608 INFO:tasks.workunit.client.1.smithi144.stdout:1/12: link l0 l2 0 2024-05-15T15:53:56.609 INFO:tasks.workunit.client.1.smithi144.stdout:1/13: chown f1 5570992 1 2024-05-15T15:53:56.611 INFO:tasks.workunit.client.1.smithi144.stdout:1/14: write f1 [841329,68805] 0 2024-05-15T15:53:56.612 INFO:tasks.workunit.client.1.smithi144.stdout:4/15: dwrite f0 [0,4194304] 0 2024-05-15T15:53:56.613 INFO:tasks.workunit.client.0.smithi049.stdout:9/32: creat d0/d8/d9/fb x:0 0 0 2024-05-15T15:53:56.615 INFO:tasks.workunit.client.1.smithi144.stdout:5/18: creat d0/f7 x:0 0 0 2024-05-15T15:53:56.616 INFO:tasks.workunit.client.0.smithi049.stdout:4/107: fsync f11 0 2024-05-15T15:53:56.617 INFO:tasks.workunit.client.0.smithi049.stdout:5/112: getdents dc/dd 0 2024-05-15T15:53:56.617 INFO:tasks.workunit.client.1.smithi144.stdout:9/21: mkdir d0/da/dc 0 2024-05-15T15:53:56.619 INFO:tasks.workunit.client.1.smithi144.stdout:7/17: rmdir d4 39 2024-05-15T15:53:56.620 INFO:tasks.workunit.client.0.smithi049.stdout:3/96: truncate f7 2611367 0 2024-05-15T15:53:56.621 INFO:tasks.workunit.client.1.smithi144.stdout:5/19: write d0/f7 [379188,76670] 0 2024-05-15T15:53:56.622 INFO:tasks.workunit.client.0.smithi049.stdout:7/69: creat d0/f13 x:0 0 0 2024-05-15T15:53:56.624 INFO:tasks.workunit.client.0.smithi049.stdout:7/70: stat d0/fb 0 2024-05-15T15:53:56.624 INFO:tasks.workunit.client.1.smithi144.stdout:1/15: dwrite f1 [0,4194304] 0 2024-05-15T15:53:56.624 INFO:tasks.workunit.client.1.smithi144.stdout:4/16: dread f0 [0,4194304] 0 2024-05-15T15:53:56.625 INFO:tasks.workunit.client.1.smithi144.stdout:1/16: write f1 [3313264,109322] 0 2024-05-15T15:53:56.626 INFO:tasks.workunit.client.0.smithi049.stdout:5/113: dwrite dc/dd/f13 [0,4194304] 0 2024-05-15T15:53:56.626 INFO:tasks.workunit.client.1.smithi144.stdout:1/17: write f1 [214062,105145] 0 2024-05-15T15:53:56.628 INFO:tasks.workunit.client.0.smithi049.stdout:3/97: dread fc [0,4194304] 0 2024-05-15T15:53:56.630 INFO:tasks.workunit.client.1.smithi144.stdout:7/18: chown d4/f6 0 1 2024-05-15T15:53:56.630 INFO:tasks.workunit.client.1.smithi144.stdout:7/19: chown d4 3471 1 2024-05-15T15:53:56.630 INFO:tasks.workunit.client.0.smithi049.stdout:4/108: mknod d18/d1e/c20 0 2024-05-15T15:53:56.631 INFO:tasks.workunit.client.0.smithi049.stdout:8/69: write d2/f12 [600958,113524] 0 2024-05-15T15:53:56.632 INFO:tasks.workunit.client.0.smithi049.stdout:4/109: write f11 [596704,111388] 0 2024-05-15T15:53:56.634 INFO:tasks.workunit.client.0.smithi049.stdout:4/110: readlink l15 0 2024-05-15T15:53:56.635 INFO:tasks.workunit.client.1.smithi144.stdout:4/17: dwrite f0 [0,4194304] 0 2024-05-15T15:53:56.636 INFO:tasks.workunit.client.1.smithi144.stdout:5/20: creat d0/f8 x:0 0 0 2024-05-15T15:53:56.644 INFO:tasks.workunit.client.0.smithi049.stdout:7/71: creat d0/d6/f14 x:0 0 0 2024-05-15T15:53:56.647 INFO:tasks.workunit.client.0.smithi049.stdout:5/114: chown l3 32463429 1 2024-05-15T15:53:56.650 INFO:tasks.workunit.client.1.smithi144.stdout:7/20: creat d4/f7 x:0 0 0 2024-05-15T15:53:56.650 INFO:tasks.workunit.client.1.smithi144.stdout:7/21: chown d4/f6 5 1 2024-05-15T15:53:56.653 INFO:tasks.workunit.client.1.smithi144.stdout:4/18: readlink l2 0 2024-05-15T15:53:56.654 INFO:tasks.workunit.client.1.smithi144.stdout:4/19: write f0 [1366744,63590] 0 2024-05-15T15:53:56.658 INFO:tasks.workunit.client.1.smithi144.stdout:7/22: symlink d4/l8 0 2024-05-15T15:53:56.659 INFO:tasks.workunit.client.1.smithi144.stdout:7/23: chown c0 2854821 1 2024-05-15T15:53:56.660 INFO:tasks.workunit.client.0.smithi049.stdout:4/111: dread d18/f1a [0,4194304] 0 2024-05-15T15:53:56.660 INFO:tasks.workunit.client.1.smithi144.stdout:4/20: dwrite f0 [0,4194304] 0 2024-05-15T15:53:56.660 INFO:tasks.workunit.client.0.smithi049.stdout:4/112: stat d18/c19 0 2024-05-15T15:53:56.661 INFO:tasks.workunit.client.1.smithi144.stdout:4/21: write f0 [8994147,81747] 0 2024-05-15T15:53:56.666 INFO:tasks.workunit.client.1.smithi144.stdout:5/21: link d0/f7 d0/f9 0 2024-05-15T15:53:56.667 INFO:tasks.workunit.client.0.smithi049.stdout:5/115: mknod dc/dd/d10/c22 0 2024-05-15T15:53:56.669 INFO:tasks.workunit.client.1.smithi144.stdout:7/24: symlink d4/l9 0 2024-05-15T15:53:56.669 INFO:tasks.workunit.client.1.smithi144.stdout:7/25: fdatasync d4/f7 0 2024-05-15T15:53:56.669 INFO:tasks.workunit.client.1.smithi144.stdout:7/26: chown d4/l9 1 1 2024-05-15T15:53:56.670 INFO:tasks.workunit.client.0.smithi049.stdout:9/33: link d0/d5/c7 d0/d5/cc 0 2024-05-15T15:53:56.672 INFO:tasks.workunit.client.0.smithi049.stdout:9/34: write d0/d8/d9/fa [738316,98879] 0 2024-05-15T15:53:56.676 INFO:tasks.workunit.client.0.smithi049.stdout:5/116: dwrite dc/dd/f1d [0,4194304] 0 2024-05-15T15:53:56.678 INFO:tasks.workunit.client.1.smithi144.stdout:5/22: symlink d0/la 0 2024-05-15T15:53:56.683 INFO:tasks.workunit.client.1.smithi144.stdout:7/27: creat d4/fa x:0 0 0 2024-05-15T15:53:56.684 INFO:tasks.workunit.client.1.smithi144.stdout:7/28: readlink d4/l8 0 2024-05-15T15:53:56.685 INFO:tasks.workunit.client.0.smithi049.stdout:8/70: dwrite d2/fb [0,4194304] 0 2024-05-15T15:53:56.685 INFO:tasks.workunit.client.1.smithi144.stdout:5/23: dwrite d0/f3 [0,4194304] 0 2024-05-15T15:53:56.689 INFO:tasks.workunit.client.0.smithi049.stdout:8/71: dread f0 [0,4194304] 0 2024-05-15T15:53:56.692 INFO:tasks.workunit.client.0.smithi049.stdout:9/35: creat d0/d8/fd x:0 0 0 2024-05-15T15:53:56.696 INFO:tasks.workunit.client.1.smithi144.stdout:7/29: symlink d4/lb 0 2024-05-15T15:53:56.697 INFO:tasks.workunit.client.0.smithi049.stdout:9/36: dread d0/d5/f6 [0,4194304] 0 2024-05-15T15:53:56.697 INFO:tasks.workunit.client.1.smithi144.stdout:1/18: dwrite f1 [4194304,4194304] 0 2024-05-15T15:53:56.697 INFO:tasks.workunit.client.0.smithi049.stdout:9/37: fsync d0/d8/d9/fa 0 2024-05-15T15:53:56.700 INFO:tasks.workunit.client.1.smithi144.stdout:5/24: symlink d0/lb 0 2024-05-15T15:53:56.702 INFO:tasks.workunit.client.1.smithi144.stdout:1/19: dread f1 [4194304,4194304] 0 2024-05-15T15:53:56.702 INFO:tasks.workunit.client.1.smithi144.stdout:5/25: write d0/f5 [905021,129291] 0 2024-05-15T15:53:56.702 INFO:tasks.workunit.client.0.smithi049.stdout:5/117: symlink dc/dd/d10/l23 0 2024-05-15T15:53:56.704 INFO:tasks.workunit.client.1.smithi144.stdout:4/22: rmdir d4 0 2024-05-15T15:53:56.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:56 smithi049 ceph-mon[28399]: pgmap v18: 65 pgs: 65 active+clean; 205 MiB data, 861 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 281 op/s 2024-05-15T15:53:56.706 INFO:tasks.workunit.client.0.smithi049.stdout:5/118: dread dc/f1f [0,4194304] 0 2024-05-15T15:53:56.709 INFO:tasks.workunit.client.0.smithi049.stdout:4/113: fdatasync d18/f1b 0 2024-05-15T15:53:56.711 INFO:tasks.workunit.client.0.smithi049.stdout:5/119: dread dc/f1b [0,4194304] 0 2024-05-15T15:53:56.711 INFO:tasks.workunit.client.1.smithi144.stdout:7/30: creat d4/fc x:0 0 0 2024-05-15T15:53:56.711 INFO:tasks.workunit.client.0.smithi049.stdout:5/120: stat dc/f1b 0 2024-05-15T15:53:56.717 INFO:tasks.workunit.client.1.smithi144.stdout:1/20: chown l2 124516306 1 2024-05-15T15:53:56.721 INFO:tasks.workunit.client.1.smithi144.stdout:5/26: mkdir d0/dc 0 2024-05-15T15:53:56.722 INFO:tasks.workunit.client.1.smithi144.stdout:4/23: mknod c5 0 2024-05-15T15:53:56.722 INFO:tasks.workunit.client.0.smithi049.stdout:3/98: link f9 d12/f15 0 2024-05-15T15:53:56.722 INFO:tasks.workunit.client.0.smithi049.stdout:8/72: dwrite d2/fc [0,4194304] 0 2024-05-15T15:53:56.723 INFO:tasks.workunit.client.0.smithi049.stdout:3/99: read f8 [1334916,78583] 0 2024-05-15T15:53:56.723 INFO:tasks.workunit.client.1.smithi144.stdout:4/24: write f0 [6627123,64310] 0 2024-05-15T15:53:56.723 INFO:tasks.workunit.client.1.smithi144.stdout:7/31: symlink d4/ld 0 2024-05-15T15:53:56.724 INFO:tasks.workunit.client.1.smithi144.stdout:7/32: chown d4/fa 667627 1 2024-05-15T15:53:56.726 INFO:tasks.workunit.client.1.smithi144.stdout:1/21: creat f3 x:0 0 0 2024-05-15T15:53:56.727 INFO:tasks.workunit.client.1.smithi144.stdout:5/27: dwrite d0/f3 [0,4194304] 0 2024-05-15T15:53:56.730 INFO:tasks.workunit.client.0.smithi049.stdout:4/114: mkdir d18/d21 0 2024-05-15T15:53:56.731 INFO:tasks.workunit.client.0.smithi049.stdout:3/100: dwrite fb [0,4194304] 0 2024-05-15T15:53:56.731 INFO:tasks.workunit.client.0.smithi049.stdout:3/101: stat d12/l13 0 2024-05-15T15:53:56.732 INFO:tasks.workunit.client.1.smithi144.stdout:4/25: mkdir d6 0 2024-05-15T15:53:56.732 INFO:tasks.workunit.client.0.smithi049.stdout:7/72: getdents d0/d6 0 2024-05-15T15:53:56.735 INFO:tasks.workunit.client.1.smithi144.stdout:1/22: unlink f3 0 2024-05-15T15:53:56.738 INFO:tasks.workunit.client.1.smithi144.stdout:7/33: dwrite d4/fc [0,4194304] 0 2024-05-15T15:53:56.744 INFO:tasks.workunit.client.1.smithi144.stdout:8/2: sync 2024-05-15T15:53:56.745 INFO:tasks.workunit.client.1.smithi144.stdout:4/26: dwrite f0 [4194304,4194304] 0 2024-05-15T15:53:56.745 INFO:tasks.workunit.client.1.smithi144.stdout:1/23: write f1 [3840299,49627] 0 2024-05-15T15:53:56.745 INFO:tasks.workunit.client.0.smithi049.stdout:1/66: sync 2024-05-15T15:53:56.745 INFO:tasks.workunit.client.0.smithi049.stdout:6/44: sync 2024-05-15T15:53:56.746 INFO:tasks.workunit.client.1.smithi144.stdout:7/34: creat d4/fe x:0 0 0 2024-05-15T15:53:56.751 INFO:tasks.workunit.client.0.smithi049.stdout:9/38: mknod d0/ce 0 2024-05-15T15:53:56.753 INFO:tasks.workunit.client.1.smithi144.stdout:7/35: dwrite d4/fa [0,4194304] 0 2024-05-15T15:53:56.754 INFO:tasks.workunit.client.0.smithi049.stdout:1/67: dwrite d0/f19 [4194304,4194304] 0 2024-05-15T15:53:56.759 INFO:tasks.workunit.client.1.smithi144.stdout:1/24: symlink l4 0 2024-05-15T15:53:56.759 INFO:tasks.workunit.client.1.smithi144.stdout:7/36: dwrite f2 [0,4194304] 0 2024-05-15T15:53:56.761 INFO:tasks.workunit.client.1.smithi144.stdout:8/3: mknod c0 0 2024-05-15T15:53:56.763 INFO:tasks.workunit.client.0.smithi049.stdout:4/115: creat d18/f22 x:0 0 0 2024-05-15T15:53:56.766 INFO:tasks.workunit.client.1.smithi144.stdout:4/27: mknod d6/c7 0 2024-05-15T15:53:56.769 INFO:tasks.workunit.client.1.smithi144.stdout:1/25: fdatasync f1 0 2024-05-15T15:53:56.773 INFO:tasks.workunit.client.1.smithi144.stdout:1/26: dread f1 [0,4194304] 0 2024-05-15T15:53:56.773 INFO:tasks.workunit.client.1.smithi144.stdout:1/27: chown l0 73075 1 2024-05-15T15:53:56.775 INFO:tasks.workunit.client.0.smithi049.stdout:8/73: fdatasync d2/f10 0 2024-05-15T15:53:56.777 INFO:tasks.workunit.client.0.smithi049.stdout:7/73: dwrite d0/f9 [4194304,4194304] 0 2024-05-15T15:53:56.778 INFO:tasks.workunit.client.1.smithi144.stdout:7/37: mkdir d4/df 0 2024-05-15T15:53:56.782 INFO:tasks.workunit.client.1.smithi144.stdout:8/4: creat f1 x:0 0 0 2024-05-15T15:53:56.783 INFO:tasks.workunit.client.1.smithi144.stdout:8/5: chown c0 0 1 2024-05-15T15:53:56.785 INFO:tasks.workunit.client.0.smithi049.stdout:6/45: rename l6 to la 0 2024-05-15T15:53:56.787 INFO:tasks.workunit.client.1.smithi144.stdout:4/28: rename l2 to d6/l8 0 2024-05-15T15:53:56.787 INFO:tasks.workunit.client.1.smithi144.stdout:4/29: chown d6 821319 1 2024-05-15T15:53:56.788 INFO:tasks.workunit.client.1.smithi144.stdout:4/30: write f0 [6615110,102924] 0 2024-05-15T15:53:56.788 INFO:tasks.workunit.client.0.smithi049.stdout:9/39: write d0/f1 [5065700,47545] 0 2024-05-15T15:53:56.789 INFO:tasks.workunit.client.0.smithi049.stdout:9/40: fsync d0/d8/fd 0 2024-05-15T15:53:56.790 INFO:tasks.workunit.client.0.smithi049.stdout:9/41: write d0/d5/f6 [5298505,29502] 0 2024-05-15T15:53:56.791 INFO:tasks.workunit.client.1.smithi144.stdout:1/28: rename l4 to l5 0 2024-05-15T15:53:56.791 INFO:tasks.workunit.client.0.smithi049.stdout:1/68: creat d0/d4/dd/df/f1d x:0 0 0 2024-05-15T15:53:56.793 INFO:tasks.workunit.client.1.smithi144.stdout:4/31: dread f0 [0,4194304] 0 2024-05-15T15:53:56.794 INFO:tasks.workunit.client.1.smithi144.stdout:4/32: write f0 [7495682,24637] 0 2024-05-15T15:53:56.794 INFO:tasks.workunit.client.1.smithi144.stdout:7/38: unlink d4/l8 0 2024-05-15T15:53:56.794 INFO:tasks.workunit.client.1.smithi144.stdout:4/33: stat c5 0 2024-05-15T15:53:56.795 INFO:tasks.workunit.client.0.smithi049.stdout:1/69: truncate d0/d4/dd/df/f1d 42078 0 2024-05-15T15:53:56.796 INFO:tasks.workunit.client.0.smithi049.stdout:4/116: creat d18/d1e/f23 x:0 0 0 2024-05-15T15:53:56.797 INFO:tasks.workunit.client.0.smithi049.stdout:4/117: fdatasync f11 0 2024-05-15T15:53:56.797 INFO:tasks.workunit.client.1.smithi144.stdout:8/6: mkdir d2 0 2024-05-15T15:53:56.801 INFO:tasks.workunit.client.0.smithi049.stdout:9/42: dwrite d0/d8/fd [0,4194304] 0 2024-05-15T15:53:56.803 INFO:tasks.workunit.client.1.smithi144.stdout:8/7: dwrite f1 [0,4194304] 0 2024-05-15T15:53:56.806 INFO:tasks.workunit.client.1.smithi144.stdout:1/29: creat f6 x:0 0 0 2024-05-15T15:53:56.806 INFO:tasks.workunit.client.0.smithi049.stdout:5/121: dwrite dc/f16 [0,4194304] 0 2024-05-15T15:53:56.808 INFO:tasks.workunit.client.0.smithi049.stdout:5/122: dread - dc/dd/f14 zero size 2024-05-15T15:53:56.810 INFO:tasks.workunit.client.0.smithi049.stdout:8/74: dread f0 [0,4194304] 0 2024-05-15T15:53:56.811 INFO:tasks.workunit.client.1.smithi144.stdout:4/34: creat d6/f9 x:0 0 0 2024-05-15T15:53:56.812 INFO:tasks.workunit.client.1.smithi144.stdout:8/8: dwrite f1 [0,4194304] 0 2024-05-15T15:53:56.813 INFO:tasks.workunit.client.1.smithi144.stdout:4/35: dread - d6/f9 zero size 2024-05-15T15:53:56.813 INFO:tasks.workunit.client.1.smithi144.stdout:4/36: dread - d6/f9 zero size 2024-05-15T15:53:56.825 INFO:tasks.workunit.client.1.smithi144.stdout:6/9: sync 2024-05-15T15:53:56.830 INFO:tasks.workunit.client.1.smithi144.stdout:4/37: mkdir d6/da 0 2024-05-15T15:53:56.833 INFO:tasks.workunit.client.1.smithi144.stdout:6/10: creat f3 x:0 0 0 2024-05-15T15:53:56.833 INFO:tasks.workunit.client.1.smithi144.stdout:6/11: rmdir - no directory 2024-05-15T15:53:56.833 INFO:tasks.workunit.client.1.smithi144.stdout:6/12: read - f1 zero size 2024-05-15T15:53:56.834 INFO:tasks.workunit.client.0.smithi049.stdout:9/43: mknod d0/d8/cf 0 2024-05-15T15:53:56.835 INFO:tasks.workunit.client.1.smithi144.stdout:1/30: chown l5 0 1 2024-05-15T15:53:56.836 INFO:tasks.workunit.client.1.smithi144.stdout:1/31: fdatasync f1 0 2024-05-15T15:53:56.837 INFO:tasks.workunit.client.0.smithi049.stdout:1/70: rename d0/d4/dd/df to d0/d4/dd/de/d1e 0 2024-05-15T15:53:56.840 INFO:tasks.workunit.client.1.smithi144.stdout:8/9: creat d2/f3 x:0 0 0 2024-05-15T15:53:56.841 INFO:tasks.workunit.client.0.smithi049.stdout:5/123: symlink dc/l24 0 2024-05-15T15:53:56.843 INFO:tasks.workunit.client.1.smithi144.stdout:6/13: symlink l4 0 2024-05-15T15:53:56.845 INFO:tasks.workunit.client.0.smithi049.stdout:0/40: sync 2024-05-15T15:53:56.845 INFO:tasks.workunit.client.0.smithi049.stdout:2/113: sync 2024-05-15T15:53:56.846 INFO:tasks.workunit.client.1.smithi144.stdout:1/32: mkdir d7 0 2024-05-15T15:53:56.846 INFO:tasks.workunit.client.1.smithi144.stdout:1/33: chown f1 24941 1 2024-05-15T15:53:56.848 INFO:tasks.workunit.client.1.smithi144.stdout:7/39: getdents d4 0 2024-05-15T15:53:56.849 INFO:tasks.workunit.client.0.smithi049.stdout:5/124: dwrite dc/dd/f1d [0,4194304] 0 2024-05-15T15:53:56.849 INFO:tasks.workunit.client.0.smithi049.stdout:3/102: getdents d12 0 2024-05-15T15:53:56.850 INFO:tasks.workunit.client.1.smithi144.stdout:8/10: symlink d2/l4 0 2024-05-15T15:53:56.853 INFO:tasks.workunit.client.0.smithi049.stdout:4/118: creat d18/d21/f24 x:0 0 0 2024-05-15T15:53:56.853 INFO:tasks.workunit.client.1.smithi144.stdout:4/38: mkdir d6/da/db 0 2024-05-15T15:53:56.855 INFO:tasks.workunit.client.0.smithi049.stdout:9/44: mkdir d0/d8/d10 0 2024-05-15T15:53:56.857 INFO:tasks.workunit.client.1.smithi144.stdout:6/14: rename f3 to f5 0 2024-05-15T15:53:56.857 INFO:tasks.workunit.client.1.smithi144.stdout:6/15: dread - f5 zero size 2024-05-15T15:53:56.862 INFO:tasks.workunit.client.1.smithi144.stdout:8/11: rename c0 to d2/c5 0 2024-05-15T15:53:56.864 INFO:tasks.workunit.client.0.smithi049.stdout:9/45: dwrite d0/d8/d9/fb [0,4194304] 0 2024-05-15T15:53:56.864 INFO:tasks.workunit.client.0.smithi049.stdout:7/74: creat d0/f15 x:0 0 0 2024-05-15T15:53:56.868 INFO:tasks.workunit.client.0.smithi049.stdout:9/46: write d0/d5/f6 [5117166,116150] 0 2024-05-15T15:53:56.868 INFO:tasks.workunit.client.1.smithi144.stdout:6/16: rename f2 to f6 0 2024-05-15T15:53:56.869 INFO:tasks.workunit.client.1.smithi144.stdout:6/17: rmdir - no directory 2024-05-15T15:53:56.869 INFO:tasks.workunit.client.1.smithi144.stdout:6/18: chown f6 2449 1 2024-05-15T15:53:56.869 INFO:tasks.workunit.client.0.smithi049.stdout:1/71: dwrite d0/d4/fa [0,4194304] 0 2024-05-15T15:53:56.871 INFO:tasks.workunit.client.1.smithi144.stdout:7/40: mkdir d4/df/d10 0 2024-05-15T15:53:56.873 INFO:tasks.workunit.client.0.smithi049.stdout:2/114: unlink d3/db/c16 0 2024-05-15T15:53:56.875 INFO:tasks.workunit.client.1.smithi144.stdout:4/39: rename c3 to d6/cc 0 2024-05-15T15:53:56.878 INFO:tasks.workunit.client.1.smithi144.stdout:6/19: dwrite f1 [0,4194304] 0 2024-05-15T15:53:56.879 INFO:tasks.workunit.client.0.smithi049.stdout:6/46: dwrite f5 [0,4194304] 0 2024-05-15T15:53:56.883 INFO:tasks.workunit.client.0.smithi049.stdout:2/115: dread d3/db/f12 [0,4194304] 0 2024-05-15T15:53:56.884 INFO:tasks.workunit.client.0.smithi049.stdout:3/103: rename la to d12/d14/l16 0 2024-05-15T15:53:56.886 INFO:tasks.workunit.client.0.smithi049.stdout:6/47: dread f2 [0,4194304] 0 2024-05-15T15:53:56.887 INFO:tasks.workunit.client.0.smithi049.stdout:4/119: symlink d18/d21/l25 0 2024-05-15T15:53:56.890 INFO:tasks.workunit.client.0.smithi049.stdout:0/41: write f3 [702549,24822] 0 2024-05-15T15:53:56.892 INFO:tasks.workunit.client.1.smithi144.stdout:4/40: mkdir d6/dd 0 2024-05-15T15:53:56.893 INFO:tasks.workunit.client.0.smithi049.stdout:0/42: write f3 [1454944,123121] 0 2024-05-15T15:53:56.895 INFO:tasks.workunit.client.0.smithi049.stdout:8/75: creat d2/de/f16 x:0 0 0 2024-05-15T15:53:56.896 INFO:tasks.workunit.client.0.smithi049.stdout:7/75: mkdir d0/d6/d16 0 2024-05-15T15:53:56.897 INFO:tasks.workunit.client.0.smithi049.stdout:4/120: dwrite d18/d1e/f1f [0,4194304] 0 2024-05-15T15:53:56.897 INFO:tasks.workunit.client.0.smithi049.stdout:7/76: fsync d0/f11 0 2024-05-15T15:53:56.899 INFO:tasks.workunit.client.1.smithi144.stdout:7/41: link f2 d4/df/f11 0 2024-05-15T15:53:56.901 INFO:tasks.workunit.client.0.smithi049.stdout:1/72: mknod d0/d4/c1f 0 2024-05-15T15:53:56.903 INFO:tasks.workunit.client.0.smithi049.stdout:1/73: chown d0/d4/dd/de/d1e/f1d 159456889 1 2024-05-15T15:53:56.903 INFO:tasks.workunit.client.1.smithi144.stdout:6/20: dwrite f6 [0,4194304] 0 2024-05-15T15:53:56.903 INFO:tasks.workunit.client.1.smithi144.stdout:6/21: dread - f5 zero size 2024-05-15T15:53:56.904 INFO:tasks.workunit.client.0.smithi049.stdout:1/74: write d0/d4/f1b [599648,71084] 0 2024-05-15T15:53:56.905 INFO:tasks.workunit.client.1.smithi144.stdout:7/42: write d4/df/f11 [3647474,40850] 0 2024-05-15T15:53:56.906 INFO:tasks.workunit.client.0.smithi049.stdout:1/75: write d0/d4/dd/f18 [1559494,85987] 0 2024-05-15T15:53:56.908 INFO:tasks.workunit.client.1.smithi144.stdout:6/22: mkdir d7 0 2024-05-15T15:53:56.909 INFO:tasks.workunit.client.1.smithi144.stdout:6/23: truncate f5 1004131 0 2024-05-15T15:53:56.909 INFO:tasks.workunit.client.0.smithi049.stdout:2/116: mknod d3/da/c1c 0 2024-05-15T15:53:56.909 INFO:tasks.workunit.client.0.smithi049.stdout:2/117: read - d3/d8/f14 zero size 2024-05-15T15:53:56.910 INFO:tasks.workunit.client.1.smithi144.stdout:7/43: creat d4/f12 x:0 0 0 2024-05-15T15:53:56.911 INFO:tasks.workunit.client.0.smithi049.stdout:2/118: fsync d3/da/f19 0 2024-05-15T15:53:56.913 INFO:tasks.workunit.client.0.smithi049.stdout:3/104: creat d12/d14/f17 x:0 0 0 2024-05-15T15:53:56.915 INFO:tasks.workunit.client.0.smithi049.stdout:3/105: truncate d12/d14/f17 15911 0 2024-05-15T15:53:56.915 INFO:tasks.workunit.client.0.smithi049.stdout:3/106: chown fb 1 1 2024-05-15T15:53:56.916 INFO:tasks.workunit.client.1.smithi144.stdout:6/24: rename f5 to d7/f8 0 2024-05-15T15:53:56.916 INFO:tasks.workunit.client.1.smithi144.stdout:4/41: chown d6/cc 121157231 1 2024-05-15T15:53:56.918 INFO:tasks.workunit.client.0.smithi049.stdout:6/48: creat fb x:0 0 0 2024-05-15T15:53:56.919 INFO:tasks.workunit.client.1.smithi144.stdout:7/44: creat d4/f13 x:0 0 0 2024-05-15T15:53:56.921 INFO:tasks.workunit.client.0.smithi049.stdout:6/49: write f8 [447102,101117] 0 2024-05-15T15:53:56.921 INFO:tasks.workunit.client.0.smithi049.stdout:6/50: rmdir - no directory 2024-05-15T15:53:56.926 INFO:tasks.workunit.client.0.smithi049.stdout:0/43: rename f2 to f4 0 2024-05-15T15:53:56.926 INFO:tasks.workunit.client.0.smithi049.stdout:3/107: dwrite d12/d14/f17 [0,4194304] 0 2024-05-15T15:53:56.926 INFO:tasks.workunit.client.0.smithi049.stdout:3/108: readlink l6 0 2024-05-15T15:53:56.927 INFO:tasks.workunit.client.1.smithi144.stdout:6/25: symlink d7/l9 0 2024-05-15T15:53:56.927 INFO:tasks.workunit.client.1.smithi144.stdout:6/26: chown d7 808682481 1 2024-05-15T15:53:56.927 INFO:tasks.workunit.client.1.smithi144.stdout:6/27: write f1 [2622697,115466] 0 2024-05-15T15:53:56.927 INFO:tasks.workunit.client.1.smithi144.stdout:8/12: truncate f1 2320005 0 2024-05-15T15:53:56.927 INFO:tasks.workunit.client.1.smithi144.stdout:6/28: dread d7/f8 [0,4194304] 0 2024-05-15T15:53:56.927 INFO:tasks.workunit.client.1.smithi144.stdout:8/13: chown d2/l4 556840223 1 2024-05-15T15:53:56.928 INFO:tasks.workunit.client.1.smithi144.stdout:8/14: dread - d2/f3 zero size 2024-05-15T15:53:56.928 INFO:tasks.workunit.client.1.smithi144.stdout:8/15: dread - d2/f3 zero size 2024-05-15T15:53:56.928 INFO:tasks.workunit.client.1.smithi144.stdout:8/16: chown d2 20348 1 2024-05-15T15:53:56.929 INFO:tasks.workunit.client.1.smithi144.stdout:4/42: rename d6/c7 to d6/da/ce 0 2024-05-15T15:53:56.930 INFO:tasks.workunit.client.0.smithi049.stdout:0/44: dread f4 [0,4194304] 0 2024-05-15T15:53:56.930 INFO:tasks.workunit.client.0.smithi049.stdout:8/76: rmdir d2/de 39 2024-05-15T15:53:56.932 INFO:tasks.workunit.client.0.smithi049.stdout:0/45: write f3 [318765,121434] 0 2024-05-15T15:53:56.932 INFO:tasks.workunit.client.0.smithi049.stdout:3/109: dread f5 [0,4194304] 0 2024-05-15T15:53:56.933 INFO:tasks.workunit.client.0.smithi049.stdout:0/46: write f3 [1889435,82671] 0 2024-05-15T15:53:56.937 INFO:tasks.workunit.client.1.smithi144.stdout:6/29: mkdir d7/da 0 2024-05-15T15:53:56.939 INFO:tasks.workunit.client.1.smithi144.stdout:8/17: rmdir d2 39 2024-05-15T15:53:56.941 INFO:tasks.workunit.client.1.smithi144.stdout:4/43: creat d6/da/ff x:0 0 0 2024-05-15T15:53:56.943 INFO:tasks.workunit.client.1.smithi144.stdout:6/30: rename d7/da to d7/db 0 2024-05-15T15:53:56.947 INFO:tasks.workunit.client.1.smithi144.stdout:4/44: readlink d6/l8 0 2024-05-15T15:53:56.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:56 smithi144 ceph-mon[36389]: pgmap v18: 65 pgs: 65 active+clean; 205 MiB data, 861 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 281 op/s 2024-05-15T15:53:56.953 INFO:tasks.workunit.client.1.smithi144.stdout:4/45: dwrite d6/da/ff [0,4194304] 0 2024-05-15T15:53:56.958 INFO:tasks.workunit.client.1.smithi144.stdout:4/46: dread d6/da/ff [0,4194304] 0 2024-05-15T15:53:56.962 INFO:tasks.workunit.client.0.smithi049.stdout:3/110: creat d12/f18 x:0 0 0 2024-05-15T15:53:56.965 INFO:tasks.workunit.client.0.smithi049.stdout:0/47: symlink l5 0 2024-05-15T15:53:56.966 INFO:tasks.workunit.client.0.smithi049.stdout:4/121: dwrite f10 [0,4194304] 0 2024-05-15T15:53:56.967 INFO:tasks.workunit.client.1.smithi144.stdout:6/31: truncate d7/f8 396906 0 2024-05-15T15:53:56.968 INFO:tasks.workunit.client.1.smithi144.stdout:0/24: sync 2024-05-15T15:53:56.968 INFO:tasks.workunit.client.1.smithi144.stdout:3/8: sync 2024-05-15T15:53:56.968 INFO:tasks.workunit.client.1.smithi144.stdout:5/28: sync 2024-05-15T15:53:56.969 INFO:tasks.workunit.client.1.smithi144.stdout:9/22: sync 2024-05-15T15:53:56.969 INFO:tasks.workunit.client.1.smithi144.stdout:2/20: sync 2024-05-15T15:53:56.969 INFO:tasks.workunit.client.1.smithi144.stdout:3/9: write - no filename 2024-05-15T15:53:56.969 INFO:tasks.workunit.client.1.smithi144.stdout:3/10: write - no filename 2024-05-15T15:53:56.969 INFO:tasks.workunit.client.1.smithi144.stdout:3/11: write - no filename 2024-05-15T15:53:56.969 INFO:tasks.workunit.client.1.smithi144.stdout:3/12: write - no filename 2024-05-15T15:53:56.970 INFO:tasks.workunit.client.0.smithi049.stdout:7/77: creat d0/d6/d16/f17 x:0 0 0 2024-05-15T15:53:56.970 INFO:tasks.workunit.client.1.smithi144.stdout:9/23: write d0/f7 [938628,90784] 0 2024-05-15T15:53:56.971 INFO:tasks.workunit.client.0.smithi049.stdout:4/122: read f11 [533702,92431] 0 2024-05-15T15:53:56.972 INFO:tasks.workunit.client.1.smithi144.stdout:9/24: dread d0/f4 [0,4194304] 0 2024-05-15T15:53:56.974 INFO:tasks.workunit.client.1.smithi144.stdout:8/18: rename f1 to d2/f6 0 2024-05-15T15:53:56.976 INFO:tasks.workunit.client.0.smithi049.stdout:3/111: dwrite ff [0,4194304] 0 2024-05-15T15:53:56.976 INFO:tasks.workunit.client.0.smithi049.stdout:5/125: getdents dc 0 2024-05-15T15:53:56.977 INFO:tasks.workunit.client.0.smithi049.stdout:5/126: write dc/dd/f17 [930879,20987] 0 2024-05-15T15:53:56.978 INFO:tasks.workunit.client.1.smithi144.stdout:6/32: mknod d7/cc 0 2024-05-15T15:53:56.978 INFO:tasks.workunit.client.1.smithi144.stdout:6/33: readlink l4 0 2024-05-15T15:53:56.981 INFO:tasks.workunit.client.1.smithi144.stdout:0/25: chown d4/f5 3 1 2024-05-15T15:53:56.982 INFO:tasks.workunit.client.1.smithi144.stdout:2/21: symlink d3/l6 0 2024-05-15T15:53:56.990 INFO:tasks.workunit.client.0.smithi049.stdout:9/47: getdents d0/d5 0 2024-05-15T15:53:56.992 INFO:tasks.workunit.client.0.smithi049.stdout:7/78: creat d0/d6/f18 x:0 0 0 2024-05-15T15:53:56.996 INFO:tasks.workunit.client.0.smithi049.stdout:4/123: creat d18/d21/f26 x:0 0 0 2024-05-15T15:53:56.998 INFO:tasks.workunit.client.0.smithi049.stdout:7/79: dread d0/f9 [0,4194304] 0 2024-05-15T15:53:56.999 INFO:tasks.workunit.client.1.smithi144.stdout:3/13: creat d0/d1/f4 x:0 0 0 2024-05-15T15:53:57.000 INFO:tasks.workunit.client.1.smithi144.stdout:6/34: creat d7/fd x:0 0 0 2024-05-15T15:53:57.000 INFO:tasks.workunit.client.1.smithi144.stdout:4/47: link d6/da/ce d6/da/c10 0 2024-05-15T15:53:57.000 INFO:tasks.workunit.client.1.smithi144.stdout:0/26: creat d4/f9 x:0 0 0 2024-05-15T15:53:57.000 INFO:tasks.workunit.client.1.smithi144.stdout:4/48: chown d6 4 1 2024-05-15T15:53:57.001 INFO:tasks.workunit.client.0.smithi049.stdout:3/112: dread f9 [0,4194304] 0 2024-05-15T15:53:57.002 INFO:tasks.workunit.client.1.smithi144.stdout:2/22: mkdir d3/d7 0 2024-05-15T15:53:57.002 INFO:tasks.workunit.client.0.smithi049.stdout:3/113: chown d12/d14/f17 1 1 2024-05-15T15:53:57.003 INFO:tasks.workunit.client.1.smithi144.stdout:2/23: truncate d3/f5 978200 0 2024-05-15T15:53:57.003 INFO:tasks.workunit.client.1.smithi144.stdout:2/24: fsync f2 0 2024-05-15T15:53:57.003 INFO:tasks.workunit.client.0.smithi049.stdout:5/127: symlink dc/dd/d10/l25 0 2024-05-15T15:53:57.008 INFO:tasks.workunit.client.0.smithi049.stdout:9/48: mkdir d0/d8/d11 0 2024-05-15T15:53:57.009 INFO:tasks.workunit.client.1.smithi144.stdout:2/25: dwrite d3/f5 [0,4194304] 0 2024-05-15T15:53:57.013 INFO:tasks.workunit.client.0.smithi049.stdout:4/124: symlink d18/l27 0 2024-05-15T15:53:57.013 INFO:tasks.workunit.client.0.smithi049.stdout:9/49: dread d0/d8/d9/fa [0,4194304] 0 2024-05-15T15:53:57.014 INFO:tasks.workunit.client.0.smithi049.stdout:9/50: stat d0/d8/d10 0 2024-05-15T15:53:57.016 INFO:tasks.workunit.client.0.smithi049.stdout:2/119: getdents d3/d8 0 2024-05-15T15:53:57.017 INFO:tasks.workunit.client.0.smithi049.stdout:9/51: dread d0/f2 [0,4194304] 0 2024-05-15T15:53:57.020 INFO:tasks.workunit.client.0.smithi049.stdout:9/52: dread d0/f2 [0,4194304] 0 2024-05-15T15:53:57.020 INFO:tasks.workunit.client.0.smithi049.stdout:6/51: getdents . 0 2024-05-15T15:53:57.020 INFO:tasks.workunit.client.0.smithi049.stdout:0/48: link l1 l6 0 2024-05-15T15:53:57.024 INFO:tasks.workunit.client.0.smithi049.stdout:9/53: dread d0/d8/d9/fa [0,4194304] 0 2024-05-15T15:53:57.028 INFO:tasks.workunit.client.0.smithi049.stdout:2/120: rmdir d3/db 39 2024-05-15T15:53:57.031 INFO:tasks.workunit.client.0.smithi049.stdout:6/52: creat fc x:0 0 0 2024-05-15T15:53:57.033 INFO:tasks.workunit.client.0.smithi049.stdout:0/49: mknod c7 0 2024-05-15T15:53:57.034 INFO:tasks.workunit.client.0.smithi049.stdout:0/50: read f4 [3341263,78134] 0 2024-05-15T15:53:57.035 INFO:tasks.workunit.client.0.smithi049.stdout:6/53: dread f8 [0,4194304] 0 2024-05-15T15:53:57.037 INFO:tasks.workunit.client.0.smithi049.stdout:4/125: unlink f0 0 2024-05-15T15:53:57.037 INFO:tasks.workunit.client.0.smithi049.stdout:0/51: write f3 [2397456,38950] 0 2024-05-15T15:53:57.037 INFO:tasks.workunit.client.0.smithi049.stdout:0/52: rmdir - no directory 2024-05-15T15:53:57.038 INFO:tasks.workunit.client.0.smithi049.stdout:3/114: dwrite f8 [0,4194304] 0 2024-05-15T15:53:57.041 INFO:tasks.workunit.client.0.smithi049.stdout:9/54: rename d0/d5 to d0/d8/d12 0 2024-05-15T15:53:57.046 INFO:tasks.workunit.client.0.smithi049.stdout:6/54: mkdir dd 0 2024-05-15T15:53:57.047 INFO:tasks.workunit.client.0.smithi049.stdout:6/55: chown f0 651364 1 2024-05-15T15:53:57.050 INFO:tasks.workunit.client.1.smithi144.stdout:1/34: sync 2024-05-15T15:53:57.051 INFO:tasks.workunit.client.0.smithi049.stdout:4/126: creat d18/f28 x:0 0 0 2024-05-15T15:53:57.056 INFO:tasks.workunit.client.0.smithi049.stdout:7/80: getdents d0 0 2024-05-15T15:53:57.057 INFO:tasks.workunit.client.1.smithi144.stdout:1/35: dwrite f6 [0,4194304] 0 2024-05-15T15:53:57.059 INFO:tasks.workunit.client.0.smithi049.stdout:7/81: write d0/d6/d16/f17 [94555,68829] 0 2024-05-15T15:53:57.060 INFO:tasks.workunit.client.0.smithi049.stdout:7/82: truncate d0/d6/f18 804555 0 2024-05-15T15:53:57.061 INFO:tasks.workunit.client.0.smithi049.stdout:7/83: readlink d0/d6/da/ld 0 2024-05-15T15:53:57.061 INFO:tasks.workunit.client.0.smithi049.stdout:2/121: mkdir d3/db/d1d 0 2024-05-15T15:53:57.069 INFO:tasks.workunit.client.0.smithi049.stdout:0/53: rename l1 to l8 0 2024-05-15T15:53:57.070 INFO:tasks.workunit.client.0.smithi049.stdout:0/54: write f3 [3312886,43251] 0 2024-05-15T15:53:57.071 INFO:tasks.workunit.client.0.smithi049.stdout:4/127: mkdir d18/d29 0 2024-05-15T15:53:57.115 INFO:tasks.workunit.client.0.smithi049.stdout:7/84: creat d0/f19 x:0 0 0 2024-05-15T15:53:57.116 INFO:tasks.workunit.client.1.smithi144.stdout:7/45: sync 2024-05-15T15:53:57.116 INFO:tasks.workunit.client.1.smithi144.stdout:5/29: sync 2024-05-15T15:53:57.117 INFO:tasks.workunit.client.1.smithi144.stdout:7/46: dread - d4/f6 zero size 2024-05-15T15:53:57.117 INFO:tasks.workunit.client.0.smithi049.stdout:3/115: truncate ff 508757 0 2024-05-15T15:53:57.118 INFO:tasks.workunit.client.0.smithi049.stdout:6/56: mknod dd/ce 0 2024-05-15T15:53:57.125 INFO:tasks.workunit.client.1.smithi144.stdout:7/47: dwrite d4/f13 [0,4194304] 0 2024-05-15T15:53:57.125 INFO:tasks.workunit.client.0.smithi049.stdout:6/57: dwrite f7 [4194304,4194304] 0 2024-05-15T15:53:57.130 INFO:tasks.workunit.client.1.smithi144.stdout:7/48: write d4/fa [4467933,17609] 0 2024-05-15T15:53:57.145 INFO:tasks.workunit.client.0.smithi049.stdout:6/58: chown f2 13842 1 2024-05-15T15:53:57.145 INFO:tasks.workunit.client.0.smithi049.stdout:2/122: mkdir d3/db/d1d/d1e 0 2024-05-15T15:53:57.145 INFO:tasks.workunit.client.0.smithi049.stdout:3/116: creat d12/d14/f19 x:0 0 0 2024-05-15T15:53:57.146 INFO:tasks.workunit.client.0.smithi049.stdout:0/55: rename l8 to l9 0 2024-05-15T15:53:57.147 INFO:tasks.workunit.client.0.smithi049.stdout:4/128: mkdir d18/d1e/d2a 0 2024-05-15T15:53:57.147 INFO:tasks.workunit.client.0.smithi049.stdout:3/117: chown d12/f15 250808 1 2024-05-15T15:53:57.152 INFO:tasks.workunit.client.0.smithi049.stdout:6/59: unlink f7 0 2024-05-15T15:53:57.154 INFO:tasks.workunit.client.0.smithi049.stdout:2/123: rmdir d3/d8 39 2024-05-15T15:53:57.157 INFO:tasks.workunit.client.0.smithi049.stdout:0/56: creat fa x:0 0 0 2024-05-15T15:53:57.157 INFO:tasks.workunit.client.0.smithi049.stdout:0/57: chown c7 1 1 2024-05-15T15:53:57.161 INFO:tasks.workunit.client.0.smithi049.stdout:4/129: fdatasync f16 0 2024-05-15T15:53:57.162 INFO:tasks.workunit.client.0.smithi049.stdout:3/118: creat d12/d14/f1a x:0 0 0 2024-05-15T15:53:57.166 INFO:tasks.workunit.client.0.smithi049.stdout:6/60: creat dd/ff x:0 0 0 2024-05-15T15:53:57.169 INFO:tasks.workunit.client.0.smithi049.stdout:2/124: creat d3/db/f1f x:0 0 0 2024-05-15T15:53:57.171 INFO:tasks.workunit.client.0.smithi049.stdout:0/58: mknod cb 0 2024-05-15T15:53:57.172 INFO:tasks.workunit.client.0.smithi049.stdout:0/59: chown c7 247 1 2024-05-15T15:53:57.175 INFO:tasks.workunit.client.0.smithi049.stdout:4/130: truncate f5 878881 0 2024-05-15T15:53:57.177 INFO:tasks.workunit.client.0.smithi049.stdout:3/119: creat d12/d14/f1b x:0 0 0 2024-05-15T15:53:57.180 INFO:tasks.workunit.client.0.smithi049.stdout:2/125: creat d3/db/d1d/f20 x:0 0 0 2024-05-15T15:53:57.180 INFO:tasks.workunit.client.0.smithi049.stdout:2/126: chown d3/db/d1d 29414 1 2024-05-15T15:53:57.181 INFO:tasks.workunit.client.0.smithi049.stdout:2/127: dread - d3/da/f19 zero size 2024-05-15T15:53:57.182 INFO:tasks.workunit.client.0.smithi049.stdout:0/60: mkdir dc 0 2024-05-15T15:53:57.186 INFO:tasks.workunit.client.0.smithi049.stdout:3/120: mknod d12/d14/c1c 0 2024-05-15T15:53:57.190 INFO:tasks.workunit.client.0.smithi049.stdout:9/55: truncate d0/d8/d9/fb 236212 0 2024-05-15T15:53:57.200 INFO:tasks.workunit.client.0.smithi049.stdout:0/61: creat dc/fd x:0 0 0 2024-05-15T15:53:57.207 INFO:tasks.workunit.client.0.smithi049.stdout:0/62: dwrite dc/fd [0,4194304] 0 2024-05-15T15:53:57.208 INFO:tasks.workunit.client.0.smithi049.stdout:7/85: symlink d0/d6/da/l1a 0 2024-05-15T15:53:57.216 INFO:tasks.workunit.client.0.smithi049.stdout:0/63: rename l5 to dc/le 0 2024-05-15T15:53:57.218 INFO:tasks.workunit.client.0.smithi049.stdout:4/131: getdents d18/d1e 0 2024-05-15T15:53:57.219 INFO:tasks.workunit.client.0.smithi049.stdout:4/132: dread - d18/d21/f24 zero size 2024-05-15T15:53:57.226 INFO:tasks.workunit.client.0.smithi049.stdout:0/64: link c7 dc/cf 0 2024-05-15T15:53:57.230 INFO:tasks.workunit.client.0.smithi049.stdout:0/65: mknod dc/c10 0 2024-05-15T15:53:57.232 INFO:tasks.workunit.client.0.smithi049.stdout:4/133: rmdir d18/d29 0 2024-05-15T15:53:57.234 INFO:tasks.workunit.client.0.smithi049.stdout:4/134: dread f5 [0,4194304] 0 2024-05-15T15:53:57.234 INFO:tasks.workunit.client.0.smithi049.stdout:6/61: getdents dd 0 2024-05-15T15:53:57.234 INFO:tasks.workunit.client.0.smithi049.stdout:4/135: fsync f5 0 2024-05-15T15:53:57.235 INFO:tasks.workunit.client.0.smithi049.stdout:6/62: write dd/ff [414643,30101] 0 2024-05-15T15:53:57.236 INFO:tasks.workunit.client.0.smithi049.stdout:4/136: truncate d18/d21/f26 1038000 0 2024-05-15T15:53:57.242 INFO:tasks.workunit.client.0.smithi049.stdout:4/137: mkdir d18/d21/d2b 0 2024-05-15T15:53:57.244 INFO:tasks.workunit.client.0.smithi049.stdout:4/138: truncate d18/f28 219818 0 2024-05-15T15:53:57.246 INFO:tasks.workunit.client.0.smithi049.stdout:2/128: rmdir d3/db/d1d 39 2024-05-15T15:53:57.247 INFO:tasks.workunit.client.0.smithi049.stdout:6/63: dwrite f2 [0,4194304] 0 2024-05-15T15:53:57.252 INFO:tasks.workunit.client.0.smithi049.stdout:2/129: unlink d3/db/c18 0 2024-05-15T15:53:57.255 INFO:tasks.workunit.client.0.smithi049.stdout:6/64: creat dd/f10 x:0 0 0 2024-05-15T15:53:57.257 INFO:tasks.workunit.client.0.smithi049.stdout:4/139: creat d18/d21/d2b/f2c x:0 0 0 2024-05-15T15:53:57.259 INFO:tasks.workunit.client.0.smithi049.stdout:2/130: chown d3/f10 9057 1 2024-05-15T15:53:57.260 INFO:tasks.workunit.client.0.smithi049.stdout:9/56: read d0/d8/d9/fb [123713,39681] 0 2024-05-15T15:53:57.262 INFO:tasks.workunit.client.0.smithi049.stdout:6/65: symlink dd/l11 0 2024-05-15T15:53:57.262 INFO:tasks.workunit.client.0.smithi049.stdout:6/66: chown f5 57242 1 2024-05-15T15:53:57.263 INFO:tasks.workunit.client.0.smithi049.stdout:6/67: truncate fc 178237 0 2024-05-15T15:53:57.278 INFO:tasks.workunit.client.0.smithi049.stdout:2/131: creat d3/db/d1d/d1e/f21 x:0 0 0 2024-05-15T15:53:57.279 INFO:tasks.workunit.client.0.smithi049.stdout:7/86: write d0/f4 [3818438,119652] 0 2024-05-15T15:53:57.280 INFO:tasks.workunit.client.0.smithi049.stdout:9/57: rmdir d0/d8/d10 0 2024-05-15T15:53:57.283 INFO:tasks.workunit.client.0.smithi049.stdout:2/132: rename d3/da/c1c to d3/db/d1d/c22 0 2024-05-15T15:53:57.286 INFO:tasks.workunit.client.0.smithi049.stdout:7/87: unlink d0/d6/d16/f17 0 2024-05-15T15:53:57.288 INFO:tasks.workunit.client.0.smithi049.stdout:9/58: unlink d0/ce 0 2024-05-15T15:53:57.289 INFO:tasks.workunit.client.0.smithi049.stdout:9/59: rename d0/d8 to d0/d8/d11/d13 22 2024-05-15T15:53:57.290 INFO:tasks.workunit.client.0.smithi049.stdout:0/66: getdents dc 0 2024-05-15T15:53:57.291 INFO:tasks.workunit.client.0.smithi049.stdout:0/67: write f3 [4029116,3848] 0 2024-05-15T15:53:57.291 INFO:tasks.workunit.client.0.smithi049.stdout:0/68: fdatasync dc/fd 0 2024-05-15T15:53:57.304 INFO:tasks.workunit.client.0.smithi049.stdout:7/88: chown d0/fb 6 1 2024-05-15T15:53:57.311 INFO:tasks.workunit.client.0.smithi049.stdout:0/69: mknod dc/c11 0 2024-05-15T15:53:57.312 INFO:tasks.workunit.client.0.smithi049.stdout:0/70: chown f4 0 1 2024-05-15T15:53:57.312 INFO:tasks.workunit.client.0.smithi049.stdout:0/71: read dc/fd [664288,18147] 0 2024-05-15T15:53:57.313 INFO:tasks.workunit.client.0.smithi049.stdout:0/72: fdatasync dc/fd 0 2024-05-15T15:53:57.313 INFO:tasks.workunit.client.0.smithi049.stdout:0/73: dread - fa zero size 2024-05-15T15:53:57.315 INFO:tasks.workunit.client.0.smithi049.stdout:2/133: creat d3/db/f23 x:0 0 0 2024-05-15T15:53:57.315 INFO:tasks.workunit.client.0.smithi049.stdout:2/134: readlink d3/da/l17 0 2024-05-15T15:53:57.316 INFO:tasks.workunit.client.0.smithi049.stdout:2/135: stat d3/db/f23 0 2024-05-15T15:53:57.320 INFO:tasks.workunit.client.0.smithi049.stdout:7/89: link d0/f19 d0/d6/f1b 0 2024-05-15T15:53:57.322 INFO:tasks.workunit.client.0.smithi049.stdout:0/74: symlink dc/l12 0 2024-05-15T15:53:57.327 INFO:tasks.workunit.client.0.smithi049.stdout:0/75: mkdir dc/d13 0 2024-05-15T15:53:57.330 INFO:tasks.workunit.client.0.smithi049.stdout:2/136: mkdir d3/da/d15/d24 0 2024-05-15T15:53:57.334 INFO:tasks.workunit.client.0.smithi049.stdout:7/90: unlink d0/d6/da/le 0 2024-05-15T15:53:57.335 INFO:tasks.workunit.client.0.smithi049.stdout:4/140: dwrite f1 [0,4194304] 0 2024-05-15T15:53:57.337 INFO:tasks.workunit.client.0.smithi049.stdout:2/137: mknod d3/da/c25 0 2024-05-15T15:53:57.344 INFO:tasks.workunit.client.0.smithi049.stdout:0/76: mkdir dc/d13/d14 0 2024-05-15T15:53:57.345 INFO:tasks.workunit.client.0.smithi049.stdout:4/141: dwrite d18/d21/f24 [0,4194304] 0 2024-05-15T15:53:57.345 INFO:tasks.workunit.client.0.smithi049.stdout:0/77: write f3 [3729663,123001] 0 2024-05-15T15:53:57.346 INFO:tasks.workunit.client.0.smithi049.stdout:0/78: dread - fa zero size 2024-05-15T15:53:57.349 INFO:tasks.workunit.client.0.smithi049.stdout:2/138: rename d3/da/f19 to d3/da/d15/f26 0 2024-05-15T15:53:57.350 INFO:tasks.workunit.client.0.smithi049.stdout:2/139: chown d3/da/c25 244 1 2024-05-15T15:53:57.352 INFO:tasks.workunit.client.0.smithi049.stdout:0/79: dread dc/fd [0,4194304] 0 2024-05-15T15:53:57.353 INFO:tasks.workunit.client.0.smithi049.stdout:0/80: chown dc/le 21541 1 2024-05-15T15:53:57.359 INFO:tasks.workunit.client.0.smithi049.stdout:4/142: mknod d18/d21/c2d 0 2024-05-15T15:53:57.362 INFO:tasks.workunit.client.0.smithi049.stdout:0/81: creat dc/f15 x:0 0 0 2024-05-15T15:53:57.365 INFO:tasks.workunit.client.0.smithi049.stdout:7/91: getdents d0 0 2024-05-15T15:53:57.367 INFO:tasks.workunit.client.0.smithi049.stdout:4/143: readlink l2 0 2024-05-15T15:53:57.373 INFO:tasks.workunit.client.0.smithi049.stdout:7/92: creat d0/d6/da/f1c x:0 0 0 2024-05-15T15:53:57.373 INFO:tasks.workunit.client.0.smithi049.stdout:4/144: dread f16 [0,4194304] 0 2024-05-15T15:53:57.373 INFO:tasks.workunit.client.0.smithi049.stdout:9/60: truncate d0/d8/fd 3233073 0 2024-05-15T15:53:57.378 INFO:tasks.workunit.client.0.smithi049.stdout:7/93: mkdir d0/d6/d1d 0 2024-05-15T15:53:57.378 INFO:tasks.workunit.client.0.smithi049.stdout:0/82: dwrite f4 [0,4194304] 0 2024-05-15T15:53:57.380 INFO:tasks.workunit.client.0.smithi049.stdout:0/83: stat dc/fd 0 2024-05-15T15:53:57.385 INFO:tasks.workunit.client.0.smithi049.stdout:4/145: dwrite d18/d21/d2b/f2c [0,4194304] 0 2024-05-15T15:53:57.385 INFO:tasks.workunit.client.0.smithi049.stdout:7/94: mknod d0/d6/da/c1e 0 2024-05-15T15:53:57.387 INFO:tasks.workunit.client.0.smithi049.stdout:0/84: dread dc/fd [0,4194304] 0 2024-05-15T15:53:57.388 INFO:tasks.workunit.client.0.smithi049.stdout:0/85: write dc/f15 [52342,34073] 0 2024-05-15T15:53:57.389 INFO:tasks.workunit.client.0.smithi049.stdout:9/61: mknod d0/d8/d11/c14 0 2024-05-15T15:53:57.400 INFO:tasks.workunit.client.0.smithi049.stdout:4/146: rename f1 to d18/f2e 0 2024-05-15T15:53:57.403 INFO:tasks.workunit.client.0.smithi049.stdout:0/86: chown c7 30310 1 2024-05-15T15:53:57.405 INFO:tasks.workunit.client.0.smithi049.stdout:9/62: creat d0/d8/d9/f15 x:0 0 0 2024-05-15T15:53:57.416 INFO:tasks.workunit.client.0.smithi049.stdout:9/63: creat d0/d8/d11/f16 x:0 0 0 2024-05-15T15:53:57.418 INFO:tasks.workunit.client.0.smithi049.stdout:7/95: rmdir d0/d6/d16 0 2024-05-15T15:53:57.419 INFO:tasks.workunit.client.0.smithi049.stdout:4/147: dwrite f10 [0,4194304] 0 2024-05-15T15:53:57.427 INFO:tasks.workunit.client.0.smithi049.stdout:0/87: truncate dc/fd 2130009 0 2024-05-15T15:53:57.430 INFO:tasks.workunit.client.0.smithi049.stdout:9/64: creat d0/d8/d11/f17 x:0 0 0 2024-05-15T15:53:57.434 INFO:tasks.workunit.client.0.smithi049.stdout:4/148: fdatasync d18/f2e 0 2024-05-15T15:53:57.439 INFO:tasks.workunit.client.0.smithi049.stdout:4/149: creat d18/f2f x:0 0 0 2024-05-15T15:53:57.442 INFO:tasks.workunit.client.0.smithi049.stdout:4/150: creat d18/d21/f30 x:0 0 0 2024-05-15T15:53:57.443 INFO:tasks.workunit.client.0.smithi049.stdout:4/151: read d18/f28 [38220,28279] 0 2024-05-15T15:53:57.443 INFO:tasks.workunit.client.0.smithi049.stdout:4/152: chown d18/d21 102 1 2024-05-15T15:53:57.444 INFO:tasks.workunit.client.0.smithi049.stdout:4/153: write d18/f22 [433298,68365] 0 2024-05-15T15:53:57.538 INFO:tasks.workunit.client.0.smithi049.stdout:1/76: sync 2024-05-15T15:53:57.543 INFO:tasks.workunit.client.0.smithi049.stdout:4/154: chown d18/f1a 3805302 1 2024-05-15T15:53:57.544 INFO:tasks.workunit.client.1.smithi144.stdout:9/25: creat d0/da/dc/fd x:0 0 0 2024-05-15T15:53:57.544 INFO:tasks.workunit.client.0.smithi049.stdout:0/88: rmdir dc 39 2024-05-15T15:53:57.545 INFO:tasks.workunit.client.0.smithi049.stdout:0/89: fdatasync fa 0 2024-05-15T15:53:57.546 INFO:tasks.workunit.client.1.smithi144.stdout:8/19: unlink d2/c5 0 2024-05-15T15:53:57.546 INFO:tasks.workunit.client.1.smithi144.stdout:8/20: dread - d2/f3 zero size 2024-05-15T15:53:57.548 INFO:tasks.workunit.client.1.smithi144.stdout:3/14: creat d0/d1/f5 x:0 0 0 2024-05-15T15:53:57.548 INFO:tasks.workunit.client.1.smithi144.stdout:3/15: fdatasync d0/d1/f5 0 2024-05-15T15:53:57.551 INFO:tasks.workunit.client.1.smithi144.stdout:0/27: mknod d4/ca 0 2024-05-15T15:53:57.553 INFO:tasks.workunit.client.1.smithi144.stdout:4/49: truncate f0 9145920 0 2024-05-15T15:53:57.554 INFO:tasks.workunit.client.0.smithi049.stdout:7/96: write d0/d6/f18 [1802465,122072] 0 2024-05-15T15:53:57.554 INFO:tasks.workunit.client.0.smithi049.stdout:7/97: read - d0/f19 zero size 2024-05-15T15:53:57.558 INFO:tasks.workunit.client.0.smithi049.stdout:1/77: symlink d0/d4/dd/de/d1e/d16/l20 0 2024-05-15T15:53:57.558 INFO:tasks.workunit.client.0.smithi049.stdout:1/78: chown d0 1665238 1 2024-05-15T15:53:57.559 INFO:tasks.workunit.client.1.smithi144.stdout:2/26: unlink l0 0 2024-05-15T15:53:57.562 INFO:tasks.workunit.client.0.smithi049.stdout:4/155: link d18/d21/d2b/f2c d18/f31 0 2024-05-15T15:53:57.564 INFO:tasks.workunit.client.1.smithi144.stdout:5/30: rename d0/lb to d0/ld 0 2024-05-15T15:53:57.564 INFO:tasks.workunit.client.1.smithi144.stdout:5/31: write d0/f8 [457372,62129] 0 2024-05-15T15:53:57.568 INFO:tasks.workunit.client.0.smithi049.stdout:0/90: symlink dc/l16 0 2024-05-15T15:53:57.572 INFO:tasks.workunit.client.1.smithi144.stdout:9/26: symlink d0/le 0 2024-05-15T15:53:57.573 INFO:tasks.workunit.client.0.smithi049.stdout:4/156: creat d18/f32 x:0 0 0 2024-05-15T15:53:57.574 INFO:tasks.workunit.client.1.smithi144.stdout:8/21: symlink d2/l7 0 2024-05-15T15:53:57.574 INFO:tasks.workunit.client.0.smithi049.stdout:4/157: read d18/f22 [285057,107161] 0 2024-05-15T15:53:57.576 INFO:tasks.workunit.client.0.smithi049.stdout:0/91: creat dc/d13/f17 x:0 0 0 2024-05-15T15:53:57.577 INFO:tasks.workunit.client.0.smithi049.stdout:0/92: write fa [486092,116079] 0 2024-05-15T15:53:57.578 INFO:tasks.workunit.client.1.smithi144.stdout:3/16: creat d0/f6 x:0 0 0 2024-05-15T15:53:57.581 INFO:tasks.workunit.client.0.smithi049.stdout:4/158: dwrite d18/f1b [0,4194304] 0 2024-05-15T15:53:57.582 INFO:tasks.workunit.client.0.smithi049.stdout:0/93: dread fa [0,4194304] 0 2024-05-15T15:53:57.582 INFO:tasks.workunit.client.1.smithi144.stdout:0/28: creat d4/d6/fb x:0 0 0 2024-05-15T15:53:57.583 INFO:tasks.workunit.client.0.smithi049.stdout:4/159: write d18/f2f [996678,40175] 0 2024-05-15T15:53:57.584 INFO:tasks.workunit.client.0.smithi049.stdout:4/160: chown d18/d1e/f23 241238 1 2024-05-15T15:53:57.586 INFO:tasks.workunit.client.1.smithi144.stdout:4/50: mkdir d6/d11 0 2024-05-15T15:53:57.589 INFO:tasks.workunit.client.1.smithi144.stdout:4/51: dread d6/da/ff [0,4194304] 0 2024-05-15T15:53:57.591 INFO:tasks.workunit.client.1.smithi144.stdout:1/36: mkdir d7/d8 0 2024-05-15T15:53:57.592 INFO:tasks.workunit.client.0.smithi049.stdout:0/94: rename dc/c11 to dc/d13/c18 0 2024-05-15T15:53:57.598 INFO:tasks.workunit.client.0.smithi049.stdout:4/161: symlink d18/d21/d2b/l33 0 2024-05-15T15:53:57.604 INFO:tasks.workunit.client.1.smithi144.stdout:3/17: symlink d0/l7 0 2024-05-15T15:53:57.605 INFO:tasks.workunit.client.0.smithi049.stdout:0/95: getdents dc/d13/d14 0 2024-05-15T15:53:57.606 INFO:tasks.workunit.client.0.smithi049.stdout:0/96: truncate dc/d13/f17 365875 0 2024-05-15T15:53:57.607 INFO:tasks.workunit.client.0.smithi049.stdout:4/162: symlink d18/d1e/d2a/l34 0 2024-05-15T15:53:57.608 INFO:tasks.workunit.client.1.smithi144.stdout:0/29: mkdir d4/dc 0 2024-05-15T15:53:57.610 INFO:tasks.workunit.client.1.smithi144.stdout:2/27: mknod d3/d7/c8 0 2024-05-15T15:53:57.612 INFO:tasks.workunit.client.1.smithi144.stdout:0/30: dread f2 [0,4194304] 0 2024-05-15T15:53:57.613 INFO:tasks.workunit.client.1.smithi144.stdout:1/37: mkdir d7/d9 0 2024-05-15T15:53:57.613 INFO:tasks.workunit.client.1.smithi144.stdout:1/38: stat d7/d9 0 2024-05-15T15:53:57.614 INFO:tasks.workunit.client.0.smithi049.stdout:0/97: dread f3 [0,4194304] 0 2024-05-15T15:53:57.616 INFO:tasks.workunit.client.1.smithi144.stdout:9/27: unlink d0/l1 0 2024-05-15T15:53:57.617 INFO:tasks.workunit.client.1.smithi144.stdout:4/52: dwrite d6/da/ff [4194304,4194304] 0 2024-05-15T15:53:57.618 INFO:tasks.workunit.client.1.smithi144.stdout:9/28: write d0/da/dc/fd [157103,46093] 0 2024-05-15T15:53:57.620 INFO:tasks.workunit.client.0.smithi049.stdout:4/163: link c13 d18/d21/d2b/c35 0 2024-05-15T15:53:57.622 INFO:tasks.workunit.client.0.smithi049.stdout:0/98: creat dc/d13/d14/f19 x:0 0 0 2024-05-15T15:53:57.623 INFO:tasks.workunit.client.0.smithi049.stdout:0/99: chown dc/cf 4512 1 2024-05-15T15:53:57.624 INFO:tasks.workunit.client.1.smithi144.stdout:2/28: rename f2 to d3/f9 0 2024-05-15T15:53:57.626 INFO:tasks.workunit.client.0.smithi049.stdout:4/164: dread fc [0,4194304] 0 2024-05-15T15:53:57.629 INFO:tasks.workunit.client.1.smithi144.stdout:2/29: dread d3/f9 [0,4194304] 0 2024-05-15T15:53:57.632 INFO:tasks.workunit.client.0.smithi049.stdout:4/165: mkdir d18/d1e/d2a/d36 0 2024-05-15T15:53:57.635 INFO:tasks.workunit.client.1.smithi144.stdout:4/53: symlink d6/da/l12 0 2024-05-15T15:53:57.635 INFO:tasks.workunit.client.1.smithi144.stdout:4/54: stat d6/f9 0 2024-05-15T15:53:57.637 INFO:tasks.workunit.client.1.smithi144.stdout:9/29: mkdir d0/da/dc/df 0 2024-05-15T15:53:57.637 INFO:tasks.workunit.client.0.smithi049.stdout:4/166: creat d18/f37 x:0 0 0 2024-05-15T15:53:57.638 INFO:tasks.workunit.client.0.smithi049.stdout:4/167: readlink d18/d1e/d2a/l34 0 2024-05-15T15:53:57.638 INFO:tasks.workunit.client.1.smithi144.stdout:6/35: dread d7/f8 [0,4194304] 0 2024-05-15T15:53:57.639 INFO:tasks.workunit.client.1.smithi144.stdout:6/36: truncate d7/fd 915331 0 2024-05-15T15:53:57.642 INFO:tasks.workunit.client.0.smithi049.stdout:4/168: write d18/f22 [582695,90424] 0 2024-05-15T15:53:57.643 INFO:tasks.workunit.client.1.smithi144.stdout:1/39: mkdir d7/d8/da 0 2024-05-15T15:53:57.644 INFO:tasks.workunit.client.1.smithi144.stdout:2/30: symlink d3/d7/la 0 2024-05-15T15:53:57.648 INFO:tasks.workunit.client.1.smithi144.stdout:2/31: dread d3/f5 [0,4194304] 0 2024-05-15T15:53:57.653 INFO:tasks.workunit.client.0.smithi049.stdout:7/98: truncate d0/d6/f1b 22430 0 2024-05-15T15:53:57.655 INFO:tasks.workunit.client.0.smithi049.stdout:7/99: read d0/d6/f18 [1077251,33570] 0 2024-05-15T15:53:57.656 INFO:tasks.workunit.client.1.smithi144.stdout:8/22: getdents d2 0 2024-05-15T15:53:57.656 INFO:tasks.workunit.client.1.smithi144.stdout:8/23: dread - d2/f3 zero size 2024-05-15T15:53:57.658 INFO:tasks.workunit.client.0.smithi049.stdout:1/79: truncate d0/d4/dd/f18 1739953 0 2024-05-15T15:53:57.662 INFO:tasks.workunit.client.1.smithi144.stdout:2/32: creat d3/d7/fb x:0 0 0 2024-05-15T15:53:57.662 INFO:tasks.workunit.client.1.smithi144.stdout:2/33: chown d3/l6 189 1 2024-05-15T15:53:57.663 INFO:tasks.workunit.client.0.smithi049.stdout:1/80: dread d0/d4/f1b [0,4194304] 0 2024-05-15T15:53:57.664 INFO:tasks.workunit.client.0.smithi049.stdout:7/100: mkdir d0/d6/d1f 0 2024-05-15T15:53:57.664 INFO:tasks.workunit.client.1.smithi144.stdout:4/55: getdents d6/da/db 0 2024-05-15T15:53:57.666 INFO:tasks.workunit.client.1.smithi144.stdout:3/18: getdents d0/d1 0 2024-05-15T15:53:57.669 INFO:tasks.workunit.client.1.smithi144.stdout:7/49: truncate d4/fa 526456 0 2024-05-15T15:53:57.671 INFO:tasks.workunit.client.0.smithi049.stdout:7/101: dwrite d0/f4 [4194304,4194304] 0 2024-05-15T15:53:57.671 INFO:tasks.workunit.client.0.smithi049.stdout:1/81: rename d0/d4/c12 to d0/d4/dd/de/d1e/c21 0 2024-05-15T15:53:57.672 INFO:tasks.workunit.client.1.smithi144.stdout:5/32: dwrite d0/f9 [0,4194304] 0 2024-05-15T15:53:57.673 INFO:tasks.workunit.client.1.smithi144.stdout:5/33: chown d0/f9 5 1 2024-05-15T15:53:57.674 INFO:tasks.workunit.client.1.smithi144.stdout:8/24: fdatasync d2/f6 0 2024-05-15T15:53:57.674 INFO:tasks.workunit.client.1.smithi144.stdout:1/40: creat d7/d8/da/fb x:0 0 0 2024-05-15T15:53:57.676 INFO:tasks.workunit.client.1.smithi144.stdout:8/25: truncate d2/f3 170129 0 2024-05-15T15:53:57.677 INFO:tasks.workunit.client.1.smithi144.stdout:2/34: symlink d3/d7/lc 0 2024-05-15T15:53:57.678 INFO:tasks.workunit.client.1.smithi144.stdout:8/26: write d2/f3 [56942,105273] 0 2024-05-15T15:53:57.678 INFO:tasks.workunit.client.0.smithi049.stdout:7/102: dwrite d0/f11 [4194304,4194304] 0 2024-05-15T15:53:57.679 INFO:tasks.workunit.client.1.smithi144.stdout:2/35: chown l1 56 1 2024-05-15T15:53:57.680 INFO:tasks.workunit.client.1.smithi144.stdout:2/36: chown d3/f5 114588886 1 2024-05-15T15:53:57.681 INFO:tasks.workunit.client.1.smithi144.stdout:2/37: chown d3/c4 238010376 1 2024-05-15T15:53:57.681 INFO:tasks.workunit.client.1.smithi144.stdout:2/38: stat l1 0 2024-05-15T15:53:57.681 INFO:tasks.workunit.client.1.smithi144.stdout:8/27: chown d2/f6 15329972 1 2024-05-15T15:53:57.682 INFO:tasks.workunit.client.1.smithi144.stdout:3/19: mknod d0/d1/c8 0 2024-05-15T15:53:57.684 INFO:tasks.workunit.client.1.smithi144.stdout:6/37: link d7/cc d7/db/ce 0 2024-05-15T15:53:57.685 INFO:tasks.workunit.client.1.smithi144.stdout:3/20: truncate d0/d1/f4 347856 0 2024-05-15T15:53:57.685 INFO:tasks.workunit.client.1.smithi144.stdout:6/38: chown f1 3 1 2024-05-15T15:53:57.686 INFO:tasks.workunit.client.1.smithi144.stdout:1/41: dread f6 [0,4194304] 0 2024-05-15T15:53:57.689 INFO:tasks.workunit.client.0.smithi049.stdout:7/103: rename d0/l1 to d0/d6/d1f/l20 0 2024-05-15T15:53:57.690 INFO:tasks.workunit.client.1.smithi144.stdout:7/50: unlink d4/lb 0 2024-05-15T15:53:57.691 INFO:tasks.workunit.client.1.smithi144.stdout:7/51: chown d4/df/d10 562 1 2024-05-15T15:53:57.692 INFO:tasks.workunit.client.1.smithi144.stdout:7/52: fdatasync d4/df/f11 0 2024-05-15T15:53:57.693 INFO:tasks.workunit.client.0.smithi049.stdout:0/100: write dc/fd [157648,125661] 0 2024-05-15T15:53:57.694 INFO:tasks.workunit.client.0.smithi049.stdout:0/101: chown dc/le 570682 1 2024-05-15T15:53:57.695 INFO:tasks.workunit.client.1.smithi144.stdout:0/31: dwrite d4/f5 [0,4194304] 0 2024-05-15T15:53:57.696 INFO:tasks.workunit.client.1.smithi144.stdout:9/30: rename d0/da/dc to d0/da/d10 0 2024-05-15T15:53:57.699 INFO:tasks.workunit.client.0.smithi049.stdout:7/104: mknod d0/d6/d1d/c21 0 2024-05-15T15:53:57.701 INFO:tasks.workunit.client.0.smithi049.stdout:7/105: write d0/f11 [3146297,76911] 0 2024-05-15T15:53:57.701 INFO:tasks.workunit.client.1.smithi144.stdout:7/53: dwrite d4/f12 [0,4194304] 0 2024-05-15T15:53:57.702 INFO:tasks.workunit.client.1.smithi144.stdout:4/56: fsync f0 0 2024-05-15T15:53:57.702 INFO:tasks.workunit.client.0.smithi049.stdout:7/106: stat d0/f11 0 2024-05-15T15:53:57.702 INFO:tasks.workunit.client.0.smithi049.stdout:7/107: dread - d0/f15 zero size 2024-05-15T15:53:57.703 INFO:tasks.workunit.client.1.smithi144.stdout:4/57: dread - d6/f9 zero size 2024-05-15T15:53:57.705 INFO:tasks.workunit.client.1.smithi144.stdout:4/58: write d6/f9 [845972,38689] 0 2024-05-15T15:53:57.705 INFO:tasks.workunit.client.0.smithi049.stdout:4/169: write d18/d21/d2b/f2c [2814559,986] 0 2024-05-15T15:53:57.706 INFO:tasks.workunit.client.1.smithi144.stdout:8/28: mknod d2/c8 0 2024-05-15T15:53:57.709 INFO:tasks.workunit.client.1.smithi144.stdout:2/39: mkdir d3/d7/dd 0 2024-05-15T15:53:57.710 INFO:tasks.workunit.client.0.smithi049.stdout:7/108: rmdir d0/d6/d1f 39 2024-05-15T15:53:57.717 INFO:tasks.workunit.client.1.smithi144.stdout:1/42: creat d7/d8/fc x:0 0 0 2024-05-15T15:53:57.718 INFO:tasks.workunit.client.0.smithi049.stdout:4/170: creat d18/d1e/d2a/f38 x:0 0 0 2024-05-15T15:53:57.718 INFO:tasks.workunit.client.0.smithi049.stdout:4/171: fdatasync f16 0 2024-05-15T15:53:57.723 INFO:tasks.workunit.client.0.smithi049.stdout:7/109: dread d0/d6/f1b [0,4194304] 0 2024-05-15T15:53:57.724 INFO:tasks.workunit.client.0.smithi049.stdout:0/102: getdents dc/d13 0 2024-05-15T15:53:57.726 INFO:tasks.workunit.client.0.smithi049.stdout:4/172: symlink d18/d21/l39 0 2024-05-15T15:53:57.727 INFO:tasks.workunit.client.0.smithi049.stdout:4/173: write f16 [762658,51513] 0 2024-05-15T15:53:57.728 INFO:tasks.workunit.client.0.smithi049.stdout:4/174: dread - d18/f1c zero size 2024-05-15T15:53:57.730 INFO:tasks.workunit.client.0.smithi049.stdout:0/103: unlink c7 0 2024-05-15T15:53:57.731 INFO:tasks.workunit.client.0.smithi049.stdout:5/128: sync 2024-05-15T15:53:57.731 INFO:tasks.workunit.client.0.smithi049.stdout:8/77: sync 2024-05-15T15:53:57.739 INFO:tasks.workunit.client.0.smithi049.stdout:4/175: dread f8 [0,4194304] 0 2024-05-15T15:53:57.746 INFO:tasks.workunit.client.0.smithi049.stdout:0/104: mknod dc/c1a 0 2024-05-15T15:53:57.751 INFO:tasks.workunit.client.0.smithi049.stdout:5/129: truncate dc/f1f 3638770 0 2024-05-15T15:53:57.753 INFO:tasks.workunit.client.1.smithi144.stdout:4/59: mknod d6/da/c13 0 2024-05-15T15:53:57.753 INFO:tasks.workunit.client.1.smithi144.stdout:4/60: write d6/f9 [680976,2078] 0 2024-05-15T15:53:57.757 INFO:tasks.workunit.client.0.smithi049.stdout:1/82: dwrite d0/d4/dd/f18 [0,4194304] 0 2024-05-15T15:53:57.762 INFO:tasks.workunit.client.0.smithi049.stdout:0/105: creat dc/f1b x:0 0 0 2024-05-15T15:53:57.763 INFO:tasks.workunit.client.0.smithi049.stdout:0/106: truncate dc/f15 689886 0 2024-05-15T15:53:57.764 INFO:tasks.workunit.client.1.smithi144.stdout:2/40: fsync d3/f5 0 2024-05-15T15:53:57.765 INFO:tasks.workunit.client.1.smithi144.stdout:4/61: dread d6/f9 [0,4194304] 0 2024-05-15T15:53:57.765 INFO:tasks.workunit.client.0.smithi049.stdout:8/78: creat d2/d14/f17 x:0 0 0 2024-05-15T15:53:57.766 INFO:tasks.workunit.client.1.smithi144.stdout:5/34: truncate d0/f8 463473 0 2024-05-15T15:53:57.767 INFO:tasks.workunit.client.1.smithi144.stdout:5/35: write d0/f3 [4292736,94404] 0 2024-05-15T15:53:57.767 INFO:tasks.workunit.client.1.smithi144.stdout:5/36: chown d0/f7 3638 1 2024-05-15T15:53:57.768 INFO:tasks.workunit.client.1.smithi144.stdout:6/39: unlink d7/cc 0 2024-05-15T15:53:57.768 INFO:tasks.workunit.client.1.smithi144.stdout:5/37: write d0/f5 [127216,48873] 0 2024-05-15T15:53:57.770 INFO:tasks.workunit.client.1.smithi144.stdout:7/54: write d4/f12 [4409454,104586] 0 2024-05-15T15:53:57.771 INFO:tasks.workunit.client.0.smithi049.stdout:7/110: getdents d0/d6/d1d 0 2024-05-15T15:53:57.777 INFO:tasks.workunit.client.1.smithi144.stdout:7/55: dwrite d4/f13 [4194304,4194304] 0 2024-05-15T15:53:57.782 INFO:tasks.workunit.client.0.smithi049.stdout:4/176: unlink l2 0 2024-05-15T15:53:57.783 INFO:tasks.workunit.client.1.smithi144.stdout:5/38: dread d0/f5 [0,4194304] 0 2024-05-15T15:53:57.784 INFO:tasks.workunit.client.0.smithi049.stdout:4/177: write d18/d1e/f1f [2140147,120954] 0 2024-05-15T15:53:57.784 INFO:tasks.workunit.client.1.smithi144.stdout:0/32: symlink d4/dc/ld 0 2024-05-15T15:53:57.787 INFO:tasks.workunit.client.0.smithi049.stdout:1/83: creat d0/d4/dd/de/d1e/d16/f22 x:0 0 0 2024-05-15T15:53:57.787 INFO:tasks.workunit.client.1.smithi144.stdout:8/29: dwrite d2/f6 [0,4194304] 0 2024-05-15T15:53:57.788 INFO:tasks.workunit.client.0.smithi049.stdout:1/84: stat d0/d4/c7 0 2024-05-15T15:53:57.788 INFO:tasks.workunit.client.1.smithi144.stdout:8/30: chown d2/l4 112 1 2024-05-15T15:53:57.792 INFO:tasks.workunit.client.0.smithi049.stdout:0/107: creat dc/d13/d14/f1c x:0 0 0 2024-05-15T15:53:57.798 INFO:tasks.workunit.client.1.smithi144.stdout:3/21: truncate d0/d1/f4 342220 0 2024-05-15T15:53:57.799 INFO:tasks.workunit.client.1.smithi144.stdout:9/31: truncate d0/d8/fb 763419 0 2024-05-15T15:53:57.802 INFO:tasks.workunit.client.0.smithi049.stdout:7/111: chown d0/d6/f14 1289 1 2024-05-15T15:53:57.814 INFO:tasks.workunit.client.1.smithi144.stdout:4/62: mknod d6/c14 0 2024-05-15T15:53:57.814 INFO:tasks.workunit.client.1.smithi144.stdout:4/63: stat d6/da/c13 0 2024-05-15T15:53:57.819 INFO:tasks.workunit.client.1.smithi144.stdout:6/40: creat d7/ff x:0 0 0 2024-05-15T15:53:57.825 INFO:tasks.workunit.client.1.smithi144.stdout:1/43: creat d7/d9/fd x:0 0 0 2024-05-15T15:53:57.825 INFO:tasks.workunit.client.1.smithi144.stdout:1/44: write d7/d8/fc [891009,52293] 0 2024-05-15T15:53:57.829 INFO:tasks.workunit.client.0.smithi049.stdout:5/130: dread dc/f1f [0,4194304] 0 2024-05-15T15:53:57.830 INFO:tasks.workunit.client.1.smithi144.stdout:7/56: creat d4/df/f14 x:0 0 0 2024-05-15T15:53:57.832 INFO:tasks.workunit.client.1.smithi144.stdout:0/33: mknod d4/d6/ce 0 2024-05-15T15:53:57.832 INFO:tasks.workunit.client.1.smithi144.stdout:0/34: stat f2 0 2024-05-15T15:53:57.833 INFO:tasks.workunit.client.1.smithi144.stdout:0/35: chown d4/dc/ld 447 1 2024-05-15T15:53:57.834 INFO:tasks.workunit.client.1.smithi144.stdout:5/39: rename d0/f3 to d0/dc/fe 0 2024-05-15T15:53:57.835 INFO:tasks.workunit.client.1.smithi144.stdout:5/40: chown d0/f9 10027 1 2024-05-15T15:53:57.835 INFO:tasks.workunit.client.1.smithi144.stdout:7/57: dwrite d4/fc [0,4194304] 0 2024-05-15T15:53:57.836 INFO:tasks.workunit.client.1.smithi144.stdout:8/31: unlink d2/f6 0 2024-05-15T15:53:57.837 INFO:tasks.workunit.client.1.smithi144.stdout:8/32: chown d2/c8 469559 1 2024-05-15T15:53:57.838 INFO:tasks.workunit.client.1.smithi144.stdout:8/33: write d2/f3 [392670,108091] 0 2024-05-15T15:53:57.840 INFO:tasks.workunit.client.1.smithi144.stdout:7/58: dread d4/f13 [0,4194304] 0 2024-05-15T15:53:57.841 INFO:tasks.workunit.client.1.smithi144.stdout:7/59: dread - d4/f7 zero size 2024-05-15T15:53:57.842 INFO:tasks.workunit.client.1.smithi144.stdout:8/34: dread d2/f3 [0,4194304] 0 2024-05-15T15:53:57.843 INFO:tasks.workunit.client.1.smithi144.stdout:8/35: dread d2/f3 [0,4194304] 0 2024-05-15T15:53:57.844 INFO:tasks.workunit.client.1.smithi144.stdout:8/36: fdatasync d2/f3 0 2024-05-15T15:53:57.844 INFO:tasks.workunit.client.0.smithi049.stdout:5/131: symlink dc/dd/d10/l26 0 2024-05-15T15:53:57.848 INFO:tasks.workunit.client.0.smithi049.stdout:4/178: link d18/d1e/c20 d18/d1e/d2a/d36/c3a 0 2024-05-15T15:53:57.848 INFO:tasks.workunit.client.0.smithi049.stdout:4/179: fsync d18/d1e/f23 0 2024-05-15T15:53:57.849 INFO:tasks.workunit.client.1.smithi144.stdout:9/32: dwrite d0/f7 [0,4194304] 0 2024-05-15T15:53:57.850 INFO:tasks.workunit.client.0.smithi049.stdout:8/79: truncate d2/f3 4803014 0 2024-05-15T15:53:57.854 INFO:tasks.workunit.client.0.smithi049.stdout:8/80: dread d2/fb [0,4194304] 0 2024-05-15T15:53:57.856 INFO:tasks.workunit.client.0.smithi049.stdout:0/108: rmdir dc/d13/d14 39 2024-05-15T15:53:57.857 INFO:tasks.workunit.client.0.smithi049.stdout:7/112: link d0/l2 d0/l22 0 2024-05-15T15:53:57.858 INFO:tasks.workunit.client.0.smithi049.stdout:5/132: symlink dc/dd/d10/l27 0 2024-05-15T15:53:57.861 INFO:tasks.workunit.client.0.smithi049.stdout:4/180: dread fa [0,4194304] 0 2024-05-15T15:53:57.862 INFO:tasks.workunit.client.0.smithi049.stdout:1/85: getdents d0/d4/dd 0 2024-05-15T15:53:57.864 INFO:tasks.workunit.client.1.smithi144.stdout:1/45: creat d7/d9/fe x:0 0 0 2024-05-15T15:53:57.864 INFO:tasks.workunit.client.0.smithi049.stdout:1/86: write d0/d4/dd/de/d1e/f15 [520141,75502] 0 2024-05-15T15:53:57.867 INFO:tasks.workunit.client.0.smithi049.stdout:5/133: symlink dc/dd/l28 0 2024-05-15T15:53:57.868 INFO:tasks.workunit.client.0.smithi049.stdout:4/181: dread f8 [0,4194304] 0 2024-05-15T15:53:57.868 INFO:tasks.workunit.client.0.smithi049.stdout:7/113: read - d0/d6/da/f1c zero size 2024-05-15T15:53:57.869 INFO:tasks.workunit.client.0.smithi049.stdout:5/134: chown dc/f11 4574 1 2024-05-15T15:53:57.872 INFO:tasks.workunit.client.1.smithi144.stdout:0/36: mkdir d4/df 0 2024-05-15T15:53:57.874 INFO:tasks.workunit.client.0.smithi049.stdout:1/87: dwrite d0/d4/fb [0,4194304] 0 2024-05-15T15:53:57.879 INFO:tasks.workunit.client.0.smithi049.stdout:7/114: dwrite d0/f15 [0,4194304] 0 2024-05-15T15:53:57.880 INFO:tasks.workunit.client.1.smithi144.stdout:5/41: write d0/f5 [869575,1831] 0 2024-05-15T15:53:57.880 INFO:tasks.workunit.client.0.smithi049.stdout:5/135: mkdir dc/d29 0 2024-05-15T15:53:57.885 INFO:tasks.workunit.client.0.smithi049.stdout:4/182: mkdir d18/d1e/d2a/d3b 0 2024-05-15T15:53:57.888 INFO:tasks.workunit.client.0.smithi049.stdout:0/109: creat dc/f1d x:0 0 0 2024-05-15T15:53:57.895 INFO:tasks.workunit.client.0.smithi049.stdout:0/110: dwrite f4 [0,4194304] 0 2024-05-15T15:53:57.898 INFO:tasks.workunit.client.0.smithi049.stdout:5/136: creat dc/dd/d10/f2a x:0 0 0 2024-05-15T15:53:57.899 INFO:tasks.workunit.client.1.smithi144.stdout:8/37: mkdir d2/d9 0 2024-05-15T15:53:57.900 INFO:tasks.workunit.client.1.smithi144.stdout:8/38: chown d2/f3 5 1 2024-05-15T15:53:57.900 INFO:tasks.workunit.client.1.smithi144.stdout:8/39: chown d2 497586994 1 2024-05-15T15:53:57.900 INFO:tasks.workunit.client.0.smithi049.stdout:5/137: fdatasync dc/dd/f14 0 2024-05-15T15:53:57.900 INFO:tasks.workunit.client.0.smithi049.stdout:4/183: symlink d18/l3c 0 2024-05-15T15:53:57.901 INFO:tasks.workunit.client.1.smithi144.stdout:9/33: creat d0/d8/f11 x:0 0 0 2024-05-15T15:53:57.904 INFO:tasks.workunit.client.1.smithi144.stdout:2/41: rmdir d3/d7/dd 0 2024-05-15T15:53:57.906 INFO:tasks.workunit.client.0.smithi049.stdout:4/184: dread d18/d21/f24 [0,4194304] 0 2024-05-15T15:53:57.906 INFO:tasks.workunit.client.0.smithi049.stdout:5/138: dread dc/dd/d10/f1e [0,4194304] 0 2024-05-15T15:53:57.907 INFO:tasks.workunit.client.1.smithi144.stdout:4/64: mkdir d6/dd/d15 0 2024-05-15T15:53:57.909 INFO:tasks.workunit.client.0.smithi049.stdout:7/115: mkdir d0/d6/d23 0 2024-05-15T15:53:57.912 INFO:tasks.workunit.client.1.smithi144.stdout:1/46: rename d7/d8/da to d7/df 0 2024-05-15T15:53:57.917 INFO:tasks.workunit.client.1.smithi144.stdout:5/42: symlink d0/dc/lf 0 2024-05-15T15:53:57.921 INFO:tasks.workunit.client.0.smithi049.stdout:4/185: rmdir d18/d1e/d2a 39 2024-05-15T15:53:57.921 INFO:tasks.workunit.client.0.smithi049.stdout:4/186: fsync f10 0 2024-05-15T15:53:57.922 INFO:tasks.workunit.client.0.smithi049.stdout:4/187: dread - d18/f1c zero size 2024-05-15T15:53:57.927 INFO:tasks.workunit.client.0.smithi049.stdout:0/111: creat dc/d13/d14/f1e x:0 0 0 2024-05-15T15:53:57.928 INFO:tasks.workunit.client.0.smithi049.stdout:4/188: dwrite d18/f31 [0,4194304] 0 2024-05-15T15:53:57.929 INFO:tasks.workunit.client.0.smithi049.stdout:4/189: stat d18/d1e/f1f 0 2024-05-15T15:53:57.930 INFO:tasks.workunit.client.0.smithi049.stdout:4/190: truncate d18/f32 391643 0 2024-05-15T15:53:57.931 INFO:tasks.workunit.client.1.smithi144.stdout:7/60: truncate d4/f12 1423595 0 2024-05-15T15:53:57.932 INFO:tasks.workunit.client.0.smithi049.stdout:1/88: getdents d0/d4/dd/de 0 2024-05-15T15:53:57.932 INFO:tasks.workunit.client.0.smithi049.stdout:1/89: stat d0/d4/fa 0 2024-05-15T15:53:57.934 INFO:tasks.workunit.client.1.smithi144.stdout:2/42: creat d3/fe x:0 0 0 2024-05-15T15:53:57.935 INFO:tasks.workunit.client.1.smithi144.stdout:8/40: write d2/f3 [1466704,17827] 0 2024-05-15T15:53:57.935 INFO:tasks.workunit.client.1.smithi144.stdout:4/65: symlink d6/dd/l16 0 2024-05-15T15:53:57.936 INFO:tasks.workunit.client.1.smithi144.stdout:9/34: dread d0/f4 [0,4194304] 0 2024-05-15T15:53:57.939 INFO:tasks.workunit.client.0.smithi049.stdout:7/116: rename d0/d6/da/l12 to d0/d6/d23/l24 0 2024-05-15T15:53:57.940 INFO:tasks.workunit.client.0.smithi049.stdout:1/90: dwrite d0/d4/dd/de/d1e/f15 [0,4194304] 0 2024-05-15T15:53:57.940 INFO:tasks.workunit.client.1.smithi144.stdout:1/47: unlink d7/d8/fc 0 2024-05-15T15:53:57.941 INFO:tasks.workunit.client.1.smithi144.stdout:8/41: dread d2/f3 [0,4194304] 0 2024-05-15T15:53:57.943 INFO:tasks.workunit.client.0.smithi049.stdout:4/191: chown lb 709627 1 2024-05-15T15:53:57.946 INFO:tasks.workunit.client.1.smithi144.stdout:3/22: getdents d0 0 2024-05-15T15:53:57.951 INFO:tasks.workunit.client.1.smithi144.stdout:1/48: dwrite d7/d9/fd [0,4194304] 0 2024-05-15T15:53:57.951 INFO:tasks.workunit.client.1.smithi144.stdout:7/61: rename c1 to d4/df/d10/c15 0 2024-05-15T15:53:57.951 INFO:tasks.workunit.client.1.smithi144.stdout:5/43: dread d0/f8 [0,4194304] 0 2024-05-15T15:53:57.952 INFO:tasks.workunit.client.0.smithi049.stdout:0/112: mkdir dc/d1f 0 2024-05-15T15:53:57.952 INFO:tasks.workunit.client.1.smithi144.stdout:2/43: symlink d3/d7/lf 0 2024-05-15T15:53:57.952 INFO:tasks.workunit.client.1.smithi144.stdout:2/44: dread - d3/d7/fb zero size 2024-05-15T15:53:57.953 INFO:tasks.workunit.client.0.smithi049.stdout:1/91: creat d0/d4/dd/de/d1e/d16/f23 x:0 0 0 2024-05-15T15:53:57.954 INFO:tasks.workunit.client.0.smithi049.stdout:1/92: write d0/d4/dd/f18 [3474144,9406] 0 2024-05-15T15:53:57.956 INFO:tasks.workunit.client.0.smithi049.stdout:7/117: symlink d0/d6/l25 0 2024-05-15T15:53:57.959 INFO:tasks.workunit.client.0.smithi049.stdout:7/118: dread d0/fb [0,4194304] 0 2024-05-15T15:53:57.962 INFO:tasks.workunit.client.1.smithi144.stdout:0/37: link c1 d4/d6/c10 0 2024-05-15T15:53:57.965 INFO:tasks.workunit.client.0.smithi049.stdout:1/93: creat d0/d4/f24 x:0 0 0 2024-05-15T15:53:57.970 INFO:tasks.workunit.client.0.smithi049.stdout:0/113: mknod dc/d1f/c20 0 2024-05-15T15:53:57.970 INFO:tasks.workunit.client.0.smithi049.stdout:0/114: chown fa 516627530 1 2024-05-15T15:53:57.974 INFO:tasks.workunit.client.0.smithi049.stdout:7/119: mknod d0/d6/da/c26 0 2024-05-15T15:53:57.977 INFO:tasks.workunit.client.1.smithi144.stdout:5/44: creat d0/dc/f10 x:0 0 0 2024-05-15T15:53:57.977 INFO:tasks.workunit.client.0.smithi049.stdout:0/115: dwrite dc/d13/f17 [0,4194304] 0 2024-05-15T15:53:57.978 INFO:tasks.workunit.client.1.smithi144.stdout:5/45: chown d0/f7 155030195 1 2024-05-15T15:53:57.978 INFO:tasks.workunit.client.1.smithi144.stdout:4/66: dwrite d6/f9 [0,4194304] 0 2024-05-15T15:53:57.979 INFO:tasks.workunit.client.0.smithi049.stdout:4/192: truncate d18/f31 3419200 0 2024-05-15T15:53:57.979 INFO:tasks.workunit.client.0.smithi049.stdout:0/116: truncate dc/d13/d14/f1e 421746 0 2024-05-15T15:53:57.981 INFO:tasks.workunit.client.1.smithi144.stdout:7/62: rename d4/df/f14 to d4/df/f16 0 2024-05-15T15:53:57.982 INFO:tasks.workunit.client.1.smithi144.stdout:4/67: dread d6/da/ff [4194304,4194304] 0 2024-05-15T15:53:57.982 INFO:tasks.workunit.client.1.smithi144.stdout:7/63: write d4/fc [4421577,60663] 0 2024-05-15T15:53:57.988 INFO:tasks.workunit.client.0.smithi049.stdout:1/94: rename d0/d4/dd/de/d1e/d16/f23 to d0/d4/f25 0 2024-05-15T15:53:57.991 INFO:tasks.workunit.client.0.smithi049.stdout:4/193: rename d18/d21/l39 to d18/d21/l3d 0 2024-05-15T15:53:57.993 INFO:tasks.workunit.client.0.smithi049.stdout:0/117: symlink dc/d1f/l21 0 2024-05-15T15:53:57.995 INFO:tasks.workunit.client.0.smithi049.stdout:1/95: dwrite d0/f19 [0,4194304] 0 2024-05-15T15:53:57.996 INFO:tasks.workunit.client.0.smithi049.stdout:1/96: truncate d0/f19 9428571 0 2024-05-15T15:53:58.005 INFO:tasks.workunit.client.0.smithi049.stdout:1/97: unlink d0/l5 0 2024-05-15T15:53:58.011 INFO:tasks.workunit.client.0.smithi049.stdout:1/98: creat d0/d4/dd/f26 x:0 0 0 2024-05-15T15:53:58.012 INFO:tasks.workunit.client.0.smithi049.stdout:1/99: chown d0/d4/fb 32 1 2024-05-15T15:53:58.013 INFO:tasks.workunit.client.0.smithi049.stdout:4/194: write d18/f28 [938908,102833] 0 2024-05-15T15:53:58.013 INFO:tasks.workunit.client.0.smithi049.stdout:1/100: read d0/d4/dd/de/d1e/f15 [3225168,51525] 0 2024-05-15T15:53:58.015 INFO:tasks.workunit.client.0.smithi049.stdout:0/118: creat dc/d13/f22 x:0 0 0 2024-05-15T15:53:58.020 INFO:tasks.workunit.client.0.smithi049.stdout:4/195: rename d18/f2f to d18/f3e 0 2024-05-15T15:53:58.021 INFO:tasks.workunit.client.0.smithi049.stdout:1/101: rmdir d0/d4/dd 39 2024-05-15T15:53:58.023 INFO:tasks.workunit.client.0.smithi049.stdout:0/119: mkdir dc/d13/d14/d23 0 2024-05-15T15:53:58.024 INFO:tasks.workunit.client.0.smithi049.stdout:0/120: fdatasync dc/d13/d14/f1c 0 2024-05-15T15:53:58.027 INFO:tasks.workunit.client.0.smithi049.stdout:4/196: dwrite f16 [0,4194304] 0 2024-05-15T15:53:58.033 INFO:tasks.workunit.client.0.smithi049.stdout:4/197: dread d18/f2e [0,4194304] 0 2024-05-15T15:53:58.033 INFO:tasks.workunit.client.0.smithi049.stdout:0/121: rmdir dc/d13/d14/d23 0 2024-05-15T15:53:58.035 INFO:tasks.workunit.client.0.smithi049.stdout:1/102: link d0/d4/dd/f26 d0/d4/dd/de/d1e/f27 0 2024-05-15T15:53:58.036 INFO:tasks.workunit.client.0.smithi049.stdout:4/198: creat d18/d21/d2b/f3f x:0 0 0 2024-05-15T15:53:58.038 INFO:tasks.workunit.client.0.smithi049.stdout:0/122: creat dc/d1f/f24 x:0 0 0 2024-05-15T15:53:58.040 INFO:tasks.workunit.client.0.smithi049.stdout:1/103: creat d0/d4/dd/f28 x:0 0 0 2024-05-15T15:53:58.044 INFO:tasks.workunit.client.0.smithi049.stdout:1/104: write d0/d4/dd/de/d1e/f15 [5022459,66335] 0 2024-05-15T15:53:58.047 INFO:tasks.workunit.client.0.smithi049.stdout:4/199: mknod d18/d1e/d2a/d3b/c40 0 2024-05-15T15:53:58.051 INFO:tasks.workunit.client.0.smithi049.stdout:4/200: creat d18/d1e/d2a/f41 x:0 0 0 2024-05-15T15:53:58.053 INFO:tasks.workunit.client.0.smithi049.stdout:1/105: dwrite d0/d4/dd/f28 [0,4194304] 0 2024-05-15T15:53:58.060 INFO:tasks.workunit.client.0.smithi049.stdout:4/201: dread d18/f28 [0,4194304] 0 2024-05-15T15:53:58.069 INFO:tasks.workunit.client.1.smithi144.stdout:6/41: getdents d7/db 0 2024-05-15T15:53:58.069 INFO:tasks.workunit.client.1.smithi144.stdout:1/49: write f6 [2662506,99203] 0 2024-05-15T15:53:58.069 INFO:tasks.workunit.client.1.smithi144.stdout:9/35: mknod d0/c12 0 2024-05-15T15:53:58.070 INFO:tasks.workunit.client.1.smithi144.stdout:0/38: rmdir d4/dc 39 2024-05-15T15:53:58.070 INFO:tasks.workunit.client.1.smithi144.stdout:8/42: mknod d2/d9/ca 0 2024-05-15T15:53:58.071 INFO:tasks.workunit.client.1.smithi144.stdout:5/46: readlink d0/l1 0 2024-05-15T15:53:58.074 INFO:tasks.workunit.client.1.smithi144.stdout:4/68: write f0 [6262299,64017] 0 2024-05-15T15:53:58.075 INFO:tasks.workunit.client.1.smithi144.stdout:5/47: write d0/f9 [2885352,75943] 0 2024-05-15T15:53:58.077 INFO:tasks.workunit.client.1.smithi144.stdout:4/69: chown d6/l8 0 1 2024-05-15T15:53:58.077 INFO:tasks.workunit.client.1.smithi144.stdout:5/48: stat d0/dc 0 2024-05-15T15:53:58.080 INFO:tasks.workunit.client.1.smithi144.stdout:4/70: chown d6/cc 6 1 2024-05-15T15:53:58.081 INFO:tasks.workunit.client.1.smithi144.stdout:7/64: dwrite d4/df/f11 [4194304,4194304] 0 2024-05-15T15:53:58.083 INFO:tasks.workunit.client.1.smithi144.stdout:2/45: dwrite d3/f9 [0,4194304] 0 2024-05-15T15:53:58.083 INFO:tasks.workunit.client.1.smithi144.stdout:4/71: chown d6/cc 1650690 1 2024-05-15T15:53:58.084 INFO:tasks.workunit.client.1.smithi144.stdout:7/65: stat d4/df/d10 0 2024-05-15T15:53:58.085 INFO:tasks.workunit.client.1.smithi144.stdout:0/39: dwrite d4/f9 [0,4194304] 0 2024-05-15T15:53:58.086 INFO:tasks.workunit.client.1.smithi144.stdout:2/46: dread - d3/d7/fb zero size 2024-05-15T15:53:58.086 INFO:tasks.workunit.client.1.smithi144.stdout:2/47: read d3/f9 [647453,48362] 0 2024-05-15T15:53:58.087 INFO:tasks.workunit.client.0.smithi049.stdout:3/121: sync 2024-05-15T15:53:58.088 INFO:tasks.workunit.client.0.smithi049.stdout:3/122: chown fc 519751 1 2024-05-15T15:53:58.093 INFO:tasks.workunit.client.1.smithi144.stdout:4/72: fsync f0 0 2024-05-15T15:53:58.096 INFO:tasks.workunit.client.0.smithi049.stdout:1/106: mkdir d0/d29 0 2024-05-15T15:53:58.100 INFO:tasks.workunit.client.0.smithi049.stdout:4/202: rename d18/f32 to d18/f42 0 2024-05-15T15:53:58.106 INFO:tasks.workunit.client.0.smithi049.stdout:0/123: truncate dc/d13/d14/f1e 349565 0 2024-05-15T15:53:58.108 INFO:tasks.workunit.client.0.smithi049.stdout:4/203: creat d18/d1e/d2a/f43 x:0 0 0 2024-05-15T15:53:58.110 INFO:tasks.workunit.client.0.smithi049.stdout:3/123: link d12/d14/f1a d12/f1d 0 2024-05-15T15:53:58.113 INFO:tasks.workunit.client.0.smithi049.stdout:0/124: dread fa [0,4194304] 0 2024-05-15T15:53:58.119 INFO:tasks.workunit.client.0.smithi049.stdout:0/125: stat dc/c10 0 2024-05-15T15:53:58.119 INFO:tasks.workunit.client.0.smithi049.stdout:3/124: dread f5 [0,4194304] 0 2024-05-15T15:53:58.120 INFO:tasks.workunit.client.0.smithi049.stdout:4/204: fdatasync f12 0 2024-05-15T15:53:58.121 INFO:tasks.workunit.client.0.smithi049.stdout:4/205: write d18/f22 [77098,8036] 0 2024-05-15T15:53:58.122 INFO:tasks.workunit.client.0.smithi049.stdout:4/206: chown d18/f42 1025942333 1 2024-05-15T15:53:58.129 INFO:tasks.workunit.client.0.smithi049.stdout:0/126: rmdir dc 39 2024-05-15T15:53:58.133 INFO:tasks.workunit.client.0.smithi049.stdout:1/107: dwrite d0/d4/f1b [0,4194304] 0 2024-05-15T15:53:58.134 INFO:tasks.workunit.client.0.smithi049.stdout:4/207: rename d18/d21/f24 to d18/d1e/d2a/d3b/f44 0 2024-05-15T15:53:58.141 INFO:tasks.workunit.client.0.smithi049.stdout:1/108: symlink d0/d4/dd/de/d1e/l2a 0 2024-05-15T15:53:58.143 INFO:tasks.workunit.client.0.smithi049.stdout:4/208: creat d18/d21/f45 x:0 0 0 2024-05-15T15:53:58.145 INFO:tasks.workunit.client.0.smithi049.stdout:3/125: link f7 d12/d14/f1e 0 2024-05-15T15:53:58.146 INFO:tasks.workunit.client.0.smithi049.stdout:4/209: dread - d18/d1e/d2a/f41 zero size 2024-05-15T15:53:58.146 INFO:tasks.workunit.client.0.smithi049.stdout:3/126: chown d12/d14/f17 139 1 2024-05-15T15:53:58.148 INFO:tasks.workunit.client.0.smithi049.stdout:0/127: mkdir dc/d13/d14/d25 0 2024-05-15T15:53:58.149 INFO:tasks.workunit.client.0.smithi049.stdout:1/109: dwrite d0/d4/dd/de/f17 [0,4194304] 0 2024-05-15T15:53:58.150 INFO:tasks.workunit.client.0.smithi049.stdout:1/110: readlink d0/d4/l14 0 2024-05-15T15:53:58.152 INFO:tasks.workunit.client.0.smithi049.stdout:1/111: chown d0/d4/dd/de/f17 15364381 1 2024-05-15T15:53:58.156 INFO:tasks.workunit.client.0.smithi049.stdout:3/127: dwrite d12/d14/f19 [0,4194304] 0 2024-05-15T15:53:58.163 INFO:tasks.workunit.client.0.smithi049.stdout:4/210: dwrite f11 [0,4194304] 0 2024-05-15T15:53:58.164 INFO:tasks.workunit.client.0.smithi049.stdout:4/211: readlink d18/d21/d2b/l33 0 2024-05-15T15:53:58.165 INFO:tasks.workunit.client.0.smithi049.stdout:1/112: unlink d0/d4/fb 0 2024-05-15T15:53:58.168 INFO:tasks.workunit.client.0.smithi049.stdout:3/128: symlink d12/d14/l1f 0 2024-05-15T15:53:58.175 INFO:tasks.workunit.client.1.smithi144.stdout:3/23: sync 2024-05-15T15:53:58.176 INFO:tasks.workunit.client.1.smithi144.stdout:3/24: chown d0/d1 0 1 2024-05-15T15:53:58.176 INFO:tasks.workunit.client.1.smithi144.stdout:3/25: truncate d0/f6 265442 0 2024-05-15T15:53:58.177 INFO:tasks.workunit.client.0.smithi049.stdout:1/113: dwrite d0/d4/dd/f28 [4194304,4194304] 0 2024-05-15T15:53:58.182 INFO:tasks.workunit.client.0.smithi049.stdout:3/129: dwrite f8 [0,4194304] 0 2024-05-15T15:53:58.184 INFO:tasks.workunit.client.0.smithi049.stdout:3/130: fdatasync d12/d14/f17 0 2024-05-15T15:53:58.187 INFO:tasks.workunit.client.0.smithi049.stdout:4/212: dwrite f17 [0,4194304] 0 2024-05-15T15:53:58.188 INFO:tasks.workunit.client.0.smithi049.stdout:4/213: read f11 [413042,105300] 0 2024-05-15T15:53:58.190 INFO:tasks.workunit.client.0.smithi049.stdout:1/114: creat d0/d4/dd/de/d1e/d16/f2b x:0 0 0 2024-05-15T15:53:58.198 INFO:tasks.workunit.client.0.smithi049.stdout:3/131: creat d12/d14/f20 x:0 0 0 2024-05-15T15:53:58.203 INFO:tasks.workunit.client.0.smithi049.stdout:4/214: mknod d18/d1e/d2a/c46 0 2024-05-15T15:53:58.204 INFO:tasks.workunit.client.0.smithi049.stdout:4/215: write d18/f22 [53362,63040] 0 2024-05-15T15:53:58.207 INFO:tasks.workunit.client.0.smithi049.stdout:3/132: mknod d12/d14/c21 0 2024-05-15T15:53:58.208 INFO:tasks.workunit.client.0.smithi049.stdout:3/133: truncate d12/d14/f1b 118330 0 2024-05-15T15:53:58.211 INFO:tasks.workunit.client.0.smithi049.stdout:4/216: stat d18/d21/l3d 0 2024-05-15T15:53:58.216 INFO:tasks.workunit.client.0.smithi049.stdout:6/68: sync 2024-05-15T15:53:58.216 INFO:tasks.workunit.client.0.smithi049.stdout:2/140: sync 2024-05-15T15:53:58.216 INFO:tasks.workunit.client.0.smithi049.stdout:6/69: dread - fb zero size 2024-05-15T15:53:58.217 INFO:tasks.workunit.client.0.smithi049.stdout:6/70: dread - dd/f10 zero size 2024-05-15T15:53:58.217 INFO:tasks.workunit.client.0.smithi049.stdout:2/141: truncate d3/db/f1a 141769 0 2024-05-15T15:53:58.218 INFO:tasks.workunit.client.0.smithi049.stdout:1/115: creat d0/d29/f2c x:0 0 0 2024-05-15T15:53:58.220 INFO:tasks.workunit.client.0.smithi049.stdout:3/134: readlink d12/d14/l16 0 2024-05-15T15:53:58.223 INFO:tasks.workunit.client.0.smithi049.stdout:4/217: creat d18/d1e/d2a/d3b/f47 x:0 0 0 2024-05-15T15:53:58.223 INFO:tasks.workunit.client.0.smithi049.stdout:4/218: stat d18/d21/d2b 0 2024-05-15T15:53:58.224 INFO:tasks.workunit.client.0.smithi049.stdout:3/135: dread fb [0,4194304] 0 2024-05-15T15:53:58.225 INFO:tasks.workunit.client.0.smithi049.stdout:3/136: readlink d12/l13 0 2024-05-15T15:53:58.225 INFO:tasks.workunit.client.0.smithi049.stdout:4/219: write d18/d21/d2b/f3f [729506,16295] 0 2024-05-15T15:53:58.226 INFO:tasks.workunit.client.0.smithi049.stdout:6/71: symlink dd/l12 0 2024-05-15T15:53:58.227 INFO:tasks.workunit.client.0.smithi049.stdout:6/72: chown dd/ff 2 1 2024-05-15T15:53:58.228 INFO:tasks.workunit.client.0.smithi049.stdout:6/73: truncate fc 1094934 0 2024-05-15T15:53:58.228 INFO:tasks.workunit.client.0.smithi049.stdout:6/74: fdatasync f2 0 2024-05-15T15:53:58.232 INFO:tasks.workunit.client.0.smithi049.stdout:3/137: dread d12/d14/f1e [0,4194304] 0 2024-05-15T15:53:58.243 INFO:tasks.workunit.client.0.smithi049.stdout:2/142: truncate d3/d8/fe 222159 0 2024-05-15T15:53:58.244 INFO:tasks.workunit.client.0.smithi049.stdout:2/143: dread - d3/db/d1d/f20 zero size 2024-05-15T15:53:58.245 INFO:tasks.workunit.client.0.smithi049.stdout:1/116: creat d0/d4/f2d x:0 0 0 2024-05-15T15:53:58.349 INFO:tasks.workunit.client.0.smithi049.stdout:4/220: creat d18/d1e/f48 x:0 0 0 2024-05-15T15:53:58.350 INFO:tasks.workunit.client.0.smithi049.stdout:4/221: chown f10 460 1 2024-05-15T15:53:58.350 INFO:tasks.workunit.client.0.smithi049.stdout:6/75: truncate f5 3416771 0 2024-05-15T15:53:58.351 INFO:tasks.workunit.client.0.smithi049.stdout:2/144: write d3/da/d15/f26 [636077,102144] 0 2024-05-15T15:53:58.351 INFO:tasks.workunit.client.0.smithi049.stdout:4/222: truncate d18/d1e/d2a/f43 434155 0 2024-05-15T15:53:58.352 INFO:tasks.workunit.client.0.smithi049.stdout:1/117: rmdir d0/d4/dd/de/d1e/d16 39 2024-05-15T15:53:58.354 INFO:tasks.workunit.client.0.smithi049.stdout:1/118: write d0/d4/dd/de/f13 [493435,7659] 0 2024-05-15T15:53:58.360 INFO:tasks.workunit.client.0.smithi049.stdout:1/119: dwrite d0/d4/f24 [0,4194304] 0 2024-05-15T15:53:58.400 INFO:tasks.workunit.client.0.smithi049.stdout:9/65: sync 2024-05-15T15:53:58.405 INFO:tasks.workunit.client.0.smithi049.stdout:4/223: dread f5 [0,4194304] 0 2024-05-15T15:53:58.412 INFO:tasks.workunit.client.0.smithi049.stdout:2/145: dwrite d3/db/f12 [0,4194304] 0 2024-05-15T15:53:58.414 INFO:tasks.workunit.client.0.smithi049.stdout:2/146: write d3/db/f12 [105575,66177] 0 2024-05-15T15:53:58.418 INFO:tasks.workunit.client.0.smithi049.stdout:4/224: rmdir d18/d1e/d2a/d3b 39 2024-05-15T15:53:58.423 INFO:tasks.workunit.client.0.smithi049.stdout:4/225: dread d18/f1a [0,4194304] 0 2024-05-15T15:53:58.460 INFO:tasks.workunit.client.0.smithi049.stdout:6/76: write f5 [1141250,59306] 0 2024-05-15T15:53:58.463 INFO:tasks.workunit.client.0.smithi049.stdout:1/120: rename d0/d29 to d0/d4/dd/de/d1e/d2e 0 2024-05-15T15:53:58.466 INFO:tasks.workunit.client.0.smithi049.stdout:4/226: dread fc [0,4194304] 0 2024-05-15T15:53:58.466 INFO:tasks.workunit.client.0.smithi049.stdout:9/66: write d0/d8/fd [416895,122815] 0 2024-05-15T15:53:58.470 INFO:tasks.workunit.client.0.smithi049.stdout:1/121: dwrite d0/d4/fa [0,4194304] 0 2024-05-15T15:53:58.470 INFO:tasks.workunit.client.0.smithi049.stdout:9/67: symlink d0/d8/d9/l18 0 2024-05-15T15:53:58.472 INFO:tasks.workunit.client.0.smithi049.stdout:9/68: write d0/d8/d11/f16 [296972,72812] 0 2024-05-15T15:53:58.473 INFO:tasks.workunit.client.0.smithi049.stdout:9/69: dread d0/d8/d9/fb [0,4194304] 0 2024-05-15T15:53:58.474 INFO:tasks.workunit.client.0.smithi049.stdout:4/227: rename d18/f42 to d18/f49 0 2024-05-15T15:53:58.475 INFO:tasks.workunit.client.0.smithi049.stdout:4/228: write d18/f22 [1554283,75250] 0 2024-05-15T15:53:58.476 INFO:tasks.workunit.client.0.smithi049.stdout:4/229: chown d18/d21/c2d 0 1 2024-05-15T15:53:58.480 INFO:tasks.workunit.client.0.smithi049.stdout:1/122: creat d0/d4/dd/de/d1e/f2f x:0 0 0 2024-05-15T15:53:58.485 INFO:tasks.workunit.client.0.smithi049.stdout:4/230: dwrite d18/d1e/d2a/f38 [0,4194304] 0 2024-05-15T15:53:58.491 INFO:tasks.workunit.client.0.smithi049.stdout:1/123: dwrite d0/d4/dd/f28 [4194304,4194304] 0 2024-05-15T15:53:58.491 INFO:tasks.workunit.client.0.smithi049.stdout:4/231: write d18/f49 [1121324,11861] 0 2024-05-15T15:53:58.504 INFO:tasks.workunit.client.0.smithi049.stdout:1/124: creat d0/d4/dd/de/d1e/d16/f30 x:0 0 0 2024-05-15T15:53:58.508 INFO:tasks.workunit.client.0.smithi049.stdout:9/70: truncate d0/f2 189849 0 2024-05-15T15:53:58.509 INFO:tasks.workunit.client.0.smithi049.stdout:4/232: getdents d18/d21/d2b 0 2024-05-15T15:53:58.511 INFO:tasks.workunit.client.0.smithi049.stdout:1/125: dwrite d0/d4/dd/de/d1e/d2e/f2c [0,4194304] 0 2024-05-15T15:53:58.512 INFO:tasks.workunit.client.0.smithi049.stdout:1/126: fdatasync d0/d4/dd/de/d1e/f15 0 2024-05-15T15:53:58.513 INFO:tasks.workunit.client.1.smithi144.stdout:9/36: write d0/d8/f11 [401786,111263] 0 2024-05-15T15:53:58.515 INFO:tasks.workunit.client.1.smithi144.stdout:6/42: mknod d7/db/c10 0 2024-05-15T15:53:58.518 INFO:tasks.workunit.client.1.smithi144.stdout:1/50: creat d7/df/f10 x:0 0 0 2024-05-15T15:53:58.518 INFO:tasks.workunit.client.1.smithi144.stdout:1/51: write f1 [1221627,60626] 0 2024-05-15T15:53:58.519 INFO:tasks.workunit.client.0.smithi049.stdout:1/127: dread d0/d4/fa [0,4194304] 0 2024-05-15T15:53:58.522 INFO:tasks.workunit.client.0.smithi049.stdout:9/71: dwrite d0/d8/d9/fb [0,4194304] 0 2024-05-15T15:53:58.526 INFO:tasks.workunit.client.0.smithi049.stdout:8/81: sync 2024-05-15T15:53:58.526 INFO:tasks.workunit.client.0.smithi049.stdout:5/139: sync 2024-05-15T15:53:58.526 INFO:tasks.workunit.client.0.smithi049.stdout:7/120: sync 2024-05-15T15:53:58.528 INFO:tasks.workunit.client.1.smithi144.stdout:5/49: creat d0/f11 x:0 0 0 2024-05-15T15:53:58.532 INFO:tasks.workunit.client.0.smithi049.stdout:7/121: dwrite d0/f15 [0,4194304] 0 2024-05-15T15:53:58.533 INFO:tasks.workunit.client.0.smithi049.stdout:7/122: stat d0/d6/f14 0 2024-05-15T15:53:58.535 INFO:tasks.workunit.client.1.smithi144.stdout:5/50: dwrite d0/f5 [0,4194304] 0 2024-05-15T15:53:58.536 INFO:tasks.workunit.client.1.smithi144.stdout:7/66: mknod d4/c17 0 2024-05-15T15:53:58.536 INFO:tasks.workunit.client.0.smithi049.stdout:7/123: dread d0/fb [0,4194304] 0 2024-05-15T15:53:58.541 INFO:tasks.workunit.client.0.smithi049.stdout:5/140: symlink dc/dd/d10/l2b 0 2024-05-15T15:53:58.542 INFO:tasks.workunit.client.1.smithi144.stdout:0/40: creat d4/d6/f11 x:0 0 0 2024-05-15T15:53:58.542 INFO:tasks.workunit.client.1.smithi144.stdout:2/48: truncate d3/f5 292483 0 2024-05-15T15:53:58.546 INFO:tasks.workunit.client.0.smithi049.stdout:4/233: getdents d18/d1e/d2a/d36 0 2024-05-15T15:53:58.548 INFO:tasks.workunit.client.1.smithi144.stdout:9/37: mknod d0/d8/c13 0 2024-05-15T15:53:58.549 INFO:tasks.workunit.client.1.smithi144.stdout:7/67: dwrite d4/df/f11 [0,4194304] 0 2024-05-15T15:53:58.552 INFO:tasks.workunit.client.0.smithi049.stdout:8/82: mkdir d2/d18 0 2024-05-15T15:53:58.553 INFO:tasks.workunit.client.1.smithi144.stdout:8/43: dwrite d2/f3 [0,4194304] 0 2024-05-15T15:53:58.553 INFO:tasks.workunit.client.0.smithi049.stdout:5/141: rmdir dc/dd 39 2024-05-15T15:53:58.555 INFO:tasks.workunit.client.0.smithi049.stdout:1/128: creat d0/d4/dd/de/d1e/f31 x:0 0 0 2024-05-15T15:53:58.556 INFO:tasks.workunit.client.1.smithi144.stdout:5/51: mkdir d0/dc/d12 0 2024-05-15T15:53:58.557 INFO:tasks.workunit.client.0.smithi049.stdout:9/72: creat d0/f19 x:0 0 0 2024-05-15T15:53:58.557 INFO:tasks.workunit.client.1.smithi144.stdout:5/52: chown d0/dc/f10 58632 1 2024-05-15T15:53:58.561 INFO:tasks.workunit.client.0.smithi049.stdout:4/234: dread d18/f28 [0,4194304] 0 2024-05-15T15:53:58.562 INFO:tasks.workunit.client.0.smithi049.stdout:4/235: fdatasync d18/d1e/f48 0 2024-05-15T15:53:58.562 INFO:tasks.workunit.client.1.smithi144.stdout:0/41: mknod d4/d6/c12 0 2024-05-15T15:53:58.562 INFO:tasks.workunit.client.1.smithi144.stdout:2/49: unlink d3/fe 0 2024-05-15T15:53:58.565 INFO:tasks.workunit.client.0.smithi049.stdout:9/73: creat d0/d8/d12/f1a x:0 0 0 2024-05-15T15:53:58.566 INFO:tasks.workunit.client.0.smithi049.stdout:9/74: dread - d0/d8/d9/f15 zero size 2024-05-15T15:53:58.568 INFO:tasks.workunit.client.0.smithi049.stdout:5/142: mkdir dc/d29/d2c 0 2024-05-15T15:53:58.569 INFO:tasks.workunit.client.1.smithi144.stdout:8/44: mknod d2/d9/cb 0 2024-05-15T15:53:58.571 INFO:tasks.workunit.client.0.smithi049.stdout:5/143: symlink dc/d29/d2c/l2d 0 2024-05-15T15:53:58.572 INFO:tasks.workunit.client.1.smithi144.stdout:2/50: mkdir d3/d10 0 2024-05-15T15:53:58.573 INFO:tasks.workunit.client.0.smithi049.stdout:5/144: stat dc/f1f 0 2024-05-15T15:53:58.573 INFO:tasks.workunit.client.0.smithi049.stdout:9/75: dwrite d0/d8/d9/fa [0,4194304] 0 2024-05-15T15:53:58.573 INFO:tasks.workunit.client.1.smithi144.stdout:9/38: rmdir d0/da/d10 39 2024-05-15T15:53:58.576 INFO:tasks.workunit.client.1.smithi144.stdout:8/45: creat d2/d9/fc x:0 0 0 2024-05-15T15:53:58.578 INFO:tasks.workunit.client.0.smithi049.stdout:9/76: mkdir d0/d8/d1b 0 2024-05-15T15:53:58.579 INFO:tasks.workunit.client.1.smithi144.stdout:5/53: link d0/dc/f10 d0/dc/d12/f13 0 2024-05-15T15:53:58.580 INFO:tasks.workunit.client.1.smithi144.stdout:5/54: read d0/f5 [1936472,97206] 0 2024-05-15T15:53:58.581 INFO:tasks.workunit.client.0.smithi049.stdout:5/145: mknod dc/dd/d10/c2e 0 2024-05-15T15:53:58.581 INFO:tasks.workunit.client.1.smithi144.stdout:0/42: getdents d4/df 0 2024-05-15T15:53:58.582 INFO:tasks.workunit.client.0.smithi049.stdout:9/77: creat d0/d8/d11/f1c x:0 0 0 2024-05-15T15:53:58.583 INFO:tasks.workunit.client.0.smithi049.stdout:9/78: dread - d0/d8/d12/f1a zero size 2024-05-15T15:53:58.583 INFO:tasks.workunit.client.0.smithi049.stdout:9/79: stat d0/f19 0 2024-05-15T15:53:58.584 INFO:tasks.workunit.client.1.smithi144.stdout:5/55: dread d0/f9 [0,4194304] 0 2024-05-15T15:53:58.585 INFO:tasks.workunit.client.1.smithi144.stdout:8/46: creat d2/fd x:0 0 0 2024-05-15T15:53:58.587 INFO:tasks.workunit.client.1.smithi144.stdout:5/56: truncate d0/dc/d12/f13 458457 0 2024-05-15T15:53:58.588 INFO:tasks.workunit.client.0.smithi049.stdout:5/146: link dc/f1b dc/d29/d2c/f2f 0 2024-05-15T15:53:58.588 INFO:tasks.workunit.client.1.smithi144.stdout:9/39: dread d0/f7 [0,4194304] 0 2024-05-15T15:53:58.592 INFO:tasks.workunit.client.1.smithi144.stdout:4/73: write d6/da/ff [3358960,35636] 0 2024-05-15T15:53:58.594 INFO:tasks.workunit.client.1.smithi144.stdout:8/47: dwrite d2/f3 [0,4194304] 0 2024-05-15T15:53:58.595 INFO:tasks.workunit.client.0.smithi049.stdout:7/124: write d0/fb [305402,108662] 0 2024-05-15T15:53:58.596 INFO:tasks.workunit.client.1.smithi144.stdout:6/43: write d7/f8 [1114116,6663] 0 2024-05-15T15:53:58.598 INFO:tasks.workunit.client.1.smithi144.stdout:0/43: symlink d4/df/l13 0 2024-05-15T15:53:58.599 INFO:tasks.workunit.client.0.smithi049.stdout:5/147: creat dc/f30 x:0 0 0 2024-05-15T15:53:58.600 INFO:tasks.workunit.client.1.smithi144.stdout:0/44: write d4/f9 [669925,34571] 0 2024-05-15T15:53:58.600 INFO:tasks.workunit.client.1.smithi144.stdout:8/48: dread d2/f3 [0,4194304] 0 2024-05-15T15:53:58.600 INFO:tasks.workunit.client.0.smithi049.stdout:5/148: write dc/dd/d10/f1a [971117,97074] 0 2024-05-15T15:53:58.601 INFO:tasks.workunit.client.1.smithi144.stdout:5/57: mknod d0/dc/c14 0 2024-05-15T15:53:58.602 INFO:tasks.workunit.client.1.smithi144.stdout:0/45: stat d4/d6/f11 0 2024-05-15T15:53:58.602 INFO:tasks.workunit.client.1.smithi144.stdout:8/49: write d2/f3 [329937,11045] 0 2024-05-15T15:53:58.603 INFO:tasks.workunit.client.1.smithi144.stdout:8/50: read - d2/d9/fc zero size 2024-05-15T15:53:58.607 INFO:tasks.workunit.client.0.smithi049.stdout:1/129: write d0/d4/dd/de/d1e/f27 [314813,14201] 0 2024-05-15T15:53:58.608 INFO:tasks.workunit.client.0.smithi049.stdout:5/149: dwrite dc/dd/f17 [0,4194304] 0 2024-05-15T15:53:58.608 INFO:tasks.workunit.client.1.smithi144.stdout:4/74: rename d6/c14 to d6/d11/c17 0 2024-05-15T15:53:58.609 INFO:tasks.workunit.client.1.smithi144.stdout:4/75: readlink d6/dd/l16 0 2024-05-15T15:53:58.611 INFO:tasks.workunit.client.1.smithi144.stdout:3/26: dwrite d0/d1/f4 [0,4194304] 0 2024-05-15T15:53:58.612 INFO:tasks.workunit.client.0.smithi049.stdout:9/80: getdents d0/d8/d12 0 2024-05-15T15:53:58.613 INFO:tasks.workunit.client.0.smithi049.stdout:8/83: write d2/f3 [5321041,123984] 0 2024-05-15T15:53:58.613 INFO:tasks.workunit.client.1.smithi144.stdout:3/27: write d0/f6 [871318,44243] 0 2024-05-15T15:53:58.615 INFO:tasks.workunit.client.0.smithi049.stdout:5/150: rmdir dc/d29 39 2024-05-15T15:53:58.616 INFO:tasks.workunit.client.1.smithi144.stdout:1/52: truncate d7/d9/fd 451850 0 2024-05-15T15:53:58.619 INFO:tasks.workunit.client.1.smithi144.stdout:1/53: fsync d7/df/f10 0 2024-05-15T15:53:58.621 INFO:tasks.workunit.client.1.smithi144.stdout:4/76: dwrite f0 [8388608,4194304] 0 2024-05-15T15:53:58.622 INFO:tasks.workunit.client.1.smithi144.stdout:5/58: mknod d0/dc/c15 0 2024-05-15T15:53:58.622 INFO:tasks.workunit.client.1.smithi144.stdout:1/54: write f6 [2662949,121097] 0 2024-05-15T15:53:58.623 INFO:tasks.workunit.client.1.smithi144.stdout:7/68: write d4/f12 [1880255,68079] 0 2024-05-15T15:53:58.624 INFO:tasks.workunit.client.1.smithi144.stdout:5/59: stat d0/f5 0 2024-05-15T15:53:58.624 INFO:tasks.workunit.client.1.smithi144.stdout:4/77: write d6/f9 [3506909,1070] 0 2024-05-15T15:53:58.627 INFO:tasks.workunit.client.1.smithi144.stdout:8/51: rename d2/d9 to d2/de 0 2024-05-15T15:53:58.632 INFO:tasks.workunit.client.1.smithi144.stdout:8/52: write d2/de/fc [57442,98494] 0 2024-05-15T15:53:58.635 INFO:tasks.workunit.client.0.smithi049.stdout:5/151: dread dc/f1b [0,4194304] 0 2024-05-15T15:53:58.637 INFO:tasks.workunit.client.1.smithi144.stdout:3/28: dwrite d0/d1/f4 [0,4194304] 0 2024-05-15T15:53:58.637 INFO:tasks.workunit.client.1.smithi144.stdout:8/53: chown d2/de/ca 11829270 1 2024-05-15T15:53:58.638 INFO:tasks.workunit.client.1.smithi144.stdout:9/40: creat d0/f14 x:0 0 0 2024-05-15T15:53:58.639 INFO:tasks.workunit.client.0.smithi049.stdout:5/152: dread dc/dd/f17 [0,4194304] 0 2024-05-15T15:53:58.641 INFO:tasks.workunit.client.1.smithi144.stdout:8/54: rename d2 to d2/de/df 22 2024-05-15T15:53:58.641 INFO:tasks.workunit.client.1.smithi144.stdout:1/55: dwrite f1 [0,4194304] 0 2024-05-15T15:53:58.642 INFO:tasks.workunit.client.1.smithi144.stdout:3/29: stat d0/d1 0 2024-05-15T15:53:58.643 INFO:tasks.workunit.client.0.smithi049.stdout:1/130: getdents d0/d4/dd/de/d1e/d16 0 2024-05-15T15:53:58.645 INFO:tasks.workunit.client.1.smithi144.stdout:4/78: dwrite d6/da/ff [0,4194304] 0 2024-05-15T15:53:58.645 INFO:tasks.workunit.client.0.smithi049.stdout:5/153: dread f0 [0,4194304] 0 2024-05-15T15:53:58.649 INFO:tasks.workunit.client.1.smithi144.stdout:2/51: dwrite d3/f5 [0,4194304] 0 2024-05-15T15:53:58.650 INFO:tasks.workunit.client.0.smithi049.stdout:7/125: dwrite d0/f19 [0,4194304] 0 2024-05-15T15:53:58.655 INFO:tasks.workunit.client.0.smithi049.stdout:9/81: rename d0/d8/d11/f1c to d0/f1d 0 2024-05-15T15:53:58.655 INFO:tasks.workunit.client.0.smithi049.stdout:1/131: dwrite d0/f19 [0,4194304] 0 2024-05-15T15:53:58.656 INFO:tasks.workunit.client.1.smithi144.stdout:5/60: rename d0/f8 to d0/f16 0 2024-05-15T15:53:58.657 INFO:tasks.workunit.client.1.smithi144.stdout:5/61: dread d0/dc/f10 [0,4194304] 0 2024-05-15T15:53:58.662 INFO:tasks.workunit.client.0.smithi049.stdout:1/132: write d0/d4/f1b [4034748,99525] 0 2024-05-15T15:53:58.663 INFO:tasks.workunit.client.1.smithi144.stdout:6/44: mkdir d7/db/d11 0 2024-05-15T15:53:58.664 INFO:tasks.workunit.client.1.smithi144.stdout:6/45: chown d7/f8 30919 1 2024-05-15T15:53:58.671 INFO:tasks.workunit.client.0.smithi049.stdout:7/126: dwrite d0/d6/f14 [0,4194304] 0 2024-05-15T15:53:58.672 INFO:tasks.workunit.client.1.smithi144.stdout:6/46: dwrite d7/ff [0,4194304] 0 2024-05-15T15:53:58.673 INFO:tasks.workunit.client.0.smithi049.stdout:1/133: dread d0/d4/dd/de/d1e/f1d [0,4194304] 0 2024-05-15T15:53:58.676 INFO:tasks.workunit.client.1.smithi144.stdout:1/56: symlink d7/l11 0 2024-05-15T15:53:58.676 INFO:tasks.workunit.client.0.smithi049.stdout:8/84: dwrite d2/fb [0,4194304] 0 2024-05-15T15:53:58.678 INFO:tasks.workunit.client.0.smithi049.stdout:9/82: dwrite d0/d8/fd [0,4194304] 0 2024-05-15T15:53:58.681 INFO:tasks.workunit.client.1.smithi144.stdout:1/57: fsync d7/d9/fe 0 2024-05-15T15:53:58.682 INFO:tasks.workunit.client.1.smithi144.stdout:3/30: mknod d0/d1/c9 0 2024-05-15T15:53:58.684 INFO:tasks.workunit.client.1.smithi144.stdout:7/69: truncate d4/fa 1300023 0 2024-05-15T15:53:58.684 INFO:tasks.workunit.client.1.smithi144.stdout:8/55: creat d2/de/f10 x:0 0 0 2024-05-15T15:53:58.686 INFO:tasks.workunit.client.1.smithi144.stdout:4/79: creat d6/f18 x:0 0 0 2024-05-15T15:53:58.686 INFO:tasks.workunit.client.1.smithi144.stdout:8/56: write d2/fd [401100,35221] 0 2024-05-15T15:53:58.687 INFO:tasks.workunit.client.1.smithi144.stdout:7/70: dread - d4/f6 zero size 2024-05-15T15:53:58.687 INFO:tasks.workunit.client.0.smithi049.stdout:7/127: creat d0/d6/d1d/f27 x:0 0 0 2024-05-15T15:53:58.688 INFO:tasks.workunit.client.1.smithi144.stdout:6/47: dwrite f1 [0,4194304] 0 2024-05-15T15:53:58.690 INFO:tasks.workunit.client.1.smithi144.stdout:7/71: fsync f2 0 2024-05-15T15:53:58.690 INFO:tasks.workunit.client.0.smithi049.stdout:9/83: creat d0/d8/f1e x:0 0 0 2024-05-15T15:53:58.691 INFO:tasks.workunit.client.0.smithi049.stdout:9/84: dread - d0/d8/d12/f1a zero size 2024-05-15T15:53:58.692 INFO:tasks.workunit.client.1.smithi144.stdout:9/41: write d0/f4 [1486021,107456] 0 2024-05-15T15:53:58.696 INFO:tasks.workunit.client.0.smithi049.stdout:8/85: mknod d2/de/c19 0 2024-05-15T15:53:58.696 INFO:tasks.workunit.client.1.smithi144.stdout:0/46: getdents d4 0 2024-05-15T15:53:58.699 INFO:tasks.workunit.client.1.smithi144.stdout:3/31: mkdir d0/da 0 2024-05-15T15:53:58.701 INFO:tasks.workunit.client.0.smithi049.stdout:9/85: creat d0/d8/d11/f1f x:0 0 0 2024-05-15T15:53:58.702 INFO:tasks.workunit.client.0.smithi049.stdout:8/86: dread d2/fb [0,4194304] 0 2024-05-15T15:53:58.702 INFO:tasks.workunit.client.1.smithi144.stdout:8/57: mknod d2/de/c11 0 2024-05-15T15:53:58.702 INFO:tasks.workunit.client.1.smithi144.stdout:6/48: dread d7/f8 [0,4194304] 0 2024-05-15T15:53:58.703 INFO:tasks.workunit.client.1.smithi144.stdout:8/58: write d2/fd [400523,82380] 0 2024-05-15T15:53:58.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:53:58 smithi049 ceph-mon[28399]: pgmap v19: 65 pgs: 65 active+clean; 276 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 363 op/s 2024-05-15T15:53:58.710 INFO:tasks.workunit.client.0.smithi049.stdout:8/87: rename d2/d14/f15 to d2/d18/f1a 0 2024-05-15T15:53:58.710 INFO:tasks.workunit.client.1.smithi144.stdout:8/59: dwrite d2/de/f10 [0,4194304] 0 2024-05-15T15:53:58.710 INFO:tasks.workunit.client.1.smithi144.stdout:9/42: write d0/da/d10/fd [1245072,115137] 0 2024-05-15T15:53:58.710 INFO:tasks.workunit.client.0.smithi049.stdout:7/128: link d0/d6/d1d/c21 d0/d6/d23/c28 0 2024-05-15T15:53:58.711 INFO:tasks.workunit.client.1.smithi144.stdout:0/47: mkdir d4/df/d14 0 2024-05-15T15:53:58.712 INFO:tasks.workunit.client.1.smithi144.stdout:3/32: creat d0/d1/fb x:0 0 0 2024-05-15T15:53:58.714 INFO:tasks.workunit.client.0.smithi049.stdout:9/86: symlink d0/l20 0 2024-05-15T15:53:58.717 INFO:tasks.workunit.client.0.smithi049.stdout:8/88: mkdir d2/d14/d1b 0 2024-05-15T15:53:58.719 INFO:tasks.workunit.client.1.smithi144.stdout:7/72: fdatasync d4/fa 0 2024-05-15T15:53:58.719 INFO:tasks.workunit.client.0.smithi049.stdout:1/134: link d0/d4/dd/de/d1e/c21 d0/d4/c32 0 2024-05-15T15:53:58.719 INFO:tasks.workunit.client.1.smithi144.stdout:7/73: rename d4 to d4/df/d10/d18 22 2024-05-15T15:53:58.720 INFO:tasks.workunit.client.1.smithi144.stdout:7/74: chown d4/df 31 1 2024-05-15T15:53:58.720 INFO:tasks.workunit.client.0.smithi049.stdout:9/87: mkdir d0/d8/d21 0 2024-05-15T15:53:58.723 INFO:tasks.workunit.client.0.smithi049.stdout:7/129: unlink d0/d6/l10 0 2024-05-15T15:53:58.724 INFO:tasks.workunit.client.0.smithi049.stdout:7/130: write d0/f13 [386883,130846] 0 2024-05-15T15:53:58.725 INFO:tasks.workunit.client.1.smithi144.stdout:8/60: mkdir d2/de/d12 0 2024-05-15T15:53:58.727 INFO:tasks.workunit.client.0.smithi049.stdout:9/88: mkdir d0/d8/d9/d22 0 2024-05-15T15:53:58.727 INFO:tasks.workunit.client.0.smithi049.stdout:9/89: stat d0/d8 0 2024-05-15T15:53:58.728 INFO:tasks.workunit.client.1.smithi144.stdout:8/61: dread d2/de/f10 [0,4194304] 0 2024-05-15T15:53:58.729 INFO:tasks.workunit.client.1.smithi144.stdout:8/62: chown d2/de 299 1 2024-05-15T15:53:58.729 INFO:tasks.workunit.client.1.smithi144.stdout:3/33: mknod d0/d1/cc 0 2024-05-15T15:53:58.731 INFO:tasks.workunit.client.1.smithi144.stdout:8/63: truncate d2/f3 4696441 0 2024-05-15T15:53:58.732 INFO:tasks.workunit.client.1.smithi144.stdout:5/62: getdents d0 0 2024-05-15T15:53:58.733 INFO:tasks.workunit.client.1.smithi144.stdout:6/49: chown d7/db/d11 52 1 2024-05-15T15:53:58.733 INFO:tasks.workunit.client.1.smithi144.stdout:7/75: mknod d4/df/c19 0 2024-05-15T15:53:58.733 INFO:tasks.workunit.client.1.smithi144.stdout:9/43: dread d0/da/d10/fd [0,4194304] 0 2024-05-15T15:53:58.735 INFO:tasks.workunit.client.0.smithi049.stdout:1/135: getdents d0/d4/dd/de/d1e/d2e 0 2024-05-15T15:53:58.737 INFO:tasks.workunit.client.1.smithi144.stdout:7/76: dread f2 [4194304,4194304] 0 2024-05-15T15:53:58.737 INFO:tasks.workunit.client.0.smithi049.stdout:1/136: write d0/d4/dd/de/d1e/f15 [145424,79570] 0 2024-05-15T15:53:58.739 INFO:tasks.workunit.client.0.smithi049.stdout:7/131: creat d0/d6/f29 x:0 0 0 2024-05-15T15:53:58.740 INFO:tasks.workunit.client.1.smithi144.stdout:1/58: getdents d7/d8 0 2024-05-15T15:53:58.740 INFO:tasks.workunit.client.0.smithi049.stdout:9/90: link d0/d8/cf d0/d8/d12/c23 0 2024-05-15T15:53:58.741 INFO:tasks.workunit.client.1.smithi144.stdout:1/59: write d7/df/fb [253727,32565] 0 2024-05-15T15:53:58.741 INFO:tasks.workunit.client.1.smithi144.stdout:1/60: dread - d7/df/f10 zero size 2024-05-15T15:53:58.742 INFO:tasks.workunit.client.0.smithi049.stdout:0/128: sync 2024-05-15T15:53:58.742 INFO:tasks.workunit.client.0.smithi049.stdout:3/138: sync 2024-05-15T15:53:58.743 INFO:tasks.workunit.client.1.smithi144.stdout:3/34: rename d0/d1/c9 to d0/d1/cd 0 2024-05-15T15:53:58.743 INFO:tasks.workunit.client.0.smithi049.stdout:0/129: read - dc/d13/d14/f1c zero size 2024-05-15T15:53:58.743 INFO:tasks.workunit.client.0.smithi049.stdout:7/132: dread d0/fb [0,4194304] 0 2024-05-15T15:53:58.744 INFO:tasks.workunit.client.0.smithi049.stdout:3/139: readlink d12/d14/l16 0 2024-05-15T15:53:58.745 INFO:tasks.workunit.client.0.smithi049.stdout:5/154: write dc/d29/d2c/f2f [1760349,6144] 0 2024-05-15T15:53:58.745 INFO:tasks.workunit.client.1.smithi144.stdout:5/63: symlink d0/l17 0 2024-05-15T15:53:58.749 INFO:tasks.workunit.client.0.smithi049.stdout:5/155: dread dc/f1f [0,4194304] 0 2024-05-15T15:53:58.750 INFO:tasks.workunit.client.1.smithi144.stdout:6/50: symlink d7/l12 0 2024-05-15T15:53:58.756 INFO:tasks.workunit.client.0.smithi049.stdout:1/137: mknod d0/d4/dd/de/d1e/c33 0 2024-05-15T15:53:58.757 INFO:tasks.workunit.client.1.smithi144.stdout:7/77: creat d4/df/f1a x:0 0 0 2024-05-15T15:53:58.758 INFO:tasks.workunit.client.1.smithi144.stdout:7/78: read d4/fa [580162,85020] 0 2024-05-15T15:53:58.759 INFO:tasks.workunit.client.1.smithi144.stdout:7/79: read - d4/f6 zero size 2024-05-15T15:53:58.759 INFO:tasks.workunit.client.1.smithi144.stdout:7/80: write d4/fe [909041,49679] 0 2024-05-15T15:53:58.760 INFO:tasks.workunit.client.1.smithi144.stdout:7/81: dread - d4/df/f1a zero size 2024-05-15T15:53:58.760 INFO:tasks.workunit.client.1.smithi144.stdout:1/61: creat d7/d8/f12 x:0 0 0 2024-05-15T15:53:58.762 INFO:tasks.workunit.client.1.smithi144.stdout:1/62: write d7/df/fb [559465,85075] 0 2024-05-15T15:53:58.766 INFO:tasks.workunit.client.1.smithi144.stdout:1/63: dread - d7/d9/fe zero size 2024-05-15T15:53:58.766 INFO:tasks.workunit.client.1.smithi144.stdout:3/35: rename d0/d1/fb to d0/d1/fe 0 2024-05-15T15:53:58.766 INFO:tasks.workunit.client.1.smithi144.stdout:3/36: stat d0/d1/f5 0 2024-05-15T15:53:58.767 INFO:tasks.workunit.client.0.smithi049.stdout:9/91: getdents d0/d8/d9/d22 0 2024-05-15T15:53:58.767 INFO:tasks.workunit.client.1.smithi144.stdout:1/64: dread f1 [0,4194304] 0 2024-05-15T15:53:58.768 INFO:tasks.workunit.client.1.smithi144.stdout:3/37: dread d0/f6 [0,4194304] 0 2024-05-15T15:53:58.769 INFO:tasks.workunit.client.0.smithi049.stdout:7/133: fsync d0/d6/f18 0 2024-05-15T15:53:58.773 INFO:tasks.workunit.client.1.smithi144.stdout:1/65: dread d7/df/fb [0,4194304] 0 2024-05-15T15:53:58.774 INFO:tasks.workunit.client.0.smithi049.stdout:7/134: dread d0/f9 [0,4194304] 0 2024-05-15T15:53:58.775 INFO:tasks.workunit.client.0.smithi049.stdout:7/135: chown d0/d6/d1d 666 1 2024-05-15T15:53:58.775 INFO:tasks.workunit.client.1.smithi144.stdout:3/38: dwrite d0/d1/f4 [0,4194304] 0 2024-05-15T15:53:58.776 INFO:tasks.workunit.client.0.smithi049.stdout:1/138: rename d0/d4 to d0/d4/d34 22 2024-05-15T15:53:58.778 INFO:tasks.workunit.client.1.smithi144.stdout:9/44: symlink d0/l15 0 2024-05-15T15:53:58.779 INFO:tasks.workunit.client.0.smithi049.stdout:3/140: link d12/d14/f17 d12/f22 0 2024-05-15T15:53:58.783 INFO:tasks.workunit.client.1.smithi144.stdout:1/66: dwrite d7/df/f10 [0,4194304] 0 2024-05-15T15:53:58.784 INFO:tasks.workunit.client.0.smithi049.stdout:9/92: unlink d0/d8/d12/f6 0 2024-05-15T15:53:58.785 INFO:tasks.workunit.client.0.smithi049.stdout:9/93: dread - d0/d8/d11/f17 zero size 2024-05-15T15:53:58.786 INFO:tasks.workunit.client.0.smithi049.stdout:9/94: write d0/f19 [536986,5325] 0 2024-05-15T15:53:58.789 INFO:tasks.workunit.client.1.smithi144.stdout:7/82: unlink d4/f7 0 2024-05-15T15:53:58.791 INFO:tasks.workunit.client.0.smithi049.stdout:9/95: dread d0/d8/fd [0,4194304] 0 2024-05-15T15:53:58.795 INFO:tasks.workunit.client.0.smithi049.stdout:1/139: unlink d0/d4/dd/de/d1e/d16/f2b 0 2024-05-15T15:53:58.795 INFO:tasks.workunit.client.0.smithi049.stdout:7/136: fsync d0/d6/f29 0 2024-05-15T15:53:58.797 INFO:tasks.workunit.client.0.smithi049.stdout:1/140: stat d0/d4/l14 0 2024-05-15T15:53:58.797 INFO:tasks.workunit.client.0.smithi049.stdout:3/141: mknod d12/d14/c23 0 2024-05-15T15:53:58.798 INFO:tasks.workunit.client.0.smithi049.stdout:3/142: stat d12 0 2024-05-15T15:53:58.798 INFO:tasks.workunit.client.0.smithi049.stdout:3/143: read f9 [2244611,46651] 0 2024-05-15T15:53:58.803 INFO:tasks.workunit.client.0.smithi049.stdout:7/137: dread d0/d6/f18 [0,4194304] 0 2024-05-15T15:53:58.807 INFO:tasks.workunit.client.0.smithi049.stdout:3/144: truncate d12/f15 690238 0 2024-05-15T15:53:58.809 INFO:tasks.workunit.client.1.smithi144.stdout:0/48: getdents d4 0 2024-05-15T15:53:58.810 INFO:tasks.workunit.client.1.smithi144.stdout:0/49: truncate d4/d6/fb 1019638 0 2024-05-15T15:53:58.810 INFO:tasks.workunit.client.1.smithi144.stdout:0/50: read - d4/d6/f11 zero size 2024-05-15T15:53:58.811 INFO:tasks.workunit.client.0.smithi049.stdout:7/138: rmdir d0/d6 39 2024-05-15T15:53:58.811 INFO:tasks.workunit.client.1.smithi144.stdout:0/51: rename d4 to d4/d15 22 2024-05-15T15:53:58.812 INFO:tasks.workunit.client.1.smithi144.stdout:1/67: mknod d7/d8/c13 0 2024-05-15T15:53:58.812 INFO:tasks.workunit.client.1.smithi144.stdout:1/68: readlink d7/l11 0 2024-05-15T15:53:58.814 INFO:tasks.workunit.client.0.smithi049.stdout:6/77: sync 2024-05-15T15:53:58.814 INFO:tasks.workunit.client.0.smithi049.stdout:2/147: sync 2024-05-15T15:53:58.814 INFO:tasks.workunit.client.0.smithi049.stdout:5/156: dwrite f0 [0,4194304] 0 2024-05-15T15:53:58.815 INFO:tasks.workunit.client.1.smithi144.stdout:5/64: write d0/f16 [101804,125593] 0 2024-05-15T15:53:58.817 INFO:tasks.workunit.client.0.smithi049.stdout:3/145: dread ff [0,4194304] 0 2024-05-15T15:53:58.819 INFO:tasks.workunit.client.0.smithi049.stdout:1/141: dread d0/d4/f8 [0,4194304] 0 2024-05-15T15:53:58.819 INFO:tasks.workunit.client.1.smithi144.stdout:8/64: truncate d2/f3 8932 0 2024-05-15T15:53:58.820 INFO:tasks.workunit.client.0.smithi049.stdout:7/139: write d0/d6/da/f1c [543820,49874] 0 2024-05-15T15:53:58.820 INFO:tasks.workunit.client.0.smithi049.stdout:6/78: symlink dd/l13 0 2024-05-15T15:53:58.824 INFO:tasks.workunit.client.0.smithi049.stdout:2/148: creat d3/da/d15/f27 x:0 0 0 2024-05-15T15:53:58.825 INFO:tasks.workunit.client.0.smithi049.stdout:0/130: dwrite dc/d13/d14/f1e [0,4194304] 0 2024-05-15T15:53:58.827 INFO:tasks.workunit.client.1.smithi144.stdout:3/39: mknod d0/da/cf 0 2024-05-15T15:53:58.834 INFO:tasks.workunit.client.1.smithi144.stdout:1/69: fsync d7/df/fb 0 2024-05-15T15:53:58.834 INFO:tasks.workunit.client.1.smithi144.stdout:1/70: chown f1 591015219 1 2024-05-15T15:53:58.837 INFO:tasks.workunit.client.0.smithi049.stdout:6/79: rename fc to dd/f14 0 2024-05-15T15:53:58.838 INFO:tasks.workunit.client.1.smithi144.stdout:7/83: rename d4/df/d10/c15 to d4/df/d10/c1b 0 2024-05-15T15:53:58.839 INFO:tasks.workunit.client.0.smithi049.stdout:9/96: read d0/f1 [3037002,98087] 0 2024-05-15T15:53:58.839 INFO:tasks.workunit.client.0.smithi049.stdout:2/149: creat d3/db/d1d/f28 x:0 0 0 2024-05-15T15:53:58.840 INFO:tasks.workunit.client.0.smithi049.stdout:9/97: truncate d0/f1d 934643 0 2024-05-15T15:53:58.840 INFO:tasks.workunit.client.1.smithi144.stdout:1/71: dwrite d7/df/f10 [0,4194304] 0 2024-05-15T15:53:58.841 INFO:tasks.workunit.client.1.smithi144.stdout:1/72: chown f6 0 1 2024-05-15T15:53:58.841 INFO:tasks.workunit.client.0.smithi049.stdout:9/98: write d0/d8/d11/f17 [568574,90323] 0 2024-05-15T15:53:58.842 INFO:tasks.workunit.client.0.smithi049.stdout:0/131: write f3 [3369013,51185] 0 2024-05-15T15:53:58.842 INFO:tasks.workunit.client.1.smithi144.stdout:5/65: truncate d0/f9 2694962 0 2024-05-15T15:53:58.843 INFO:tasks.workunit.client.1.smithi144.stdout:6/51: truncate f1 3360819 0 2024-05-15T15:53:58.844 INFO:tasks.workunit.client.1.smithi144.stdout:5/66: write d0/f16 [923260,50972] 0 2024-05-15T15:53:58.845 INFO:tasks.workunit.client.1.smithi144.stdout:8/65: mknod d2/c13 0 2024-05-15T15:53:58.846 INFO:tasks.workunit.client.1.smithi144.stdout:8/66: chown d2 106 1 2024-05-15T15:53:58.846 INFO:tasks.workunit.client.1.smithi144.stdout:7/84: dread d4/df/f11 [0,4194304] 0 2024-05-15T15:53:58.847 INFO:tasks.workunit.client.1.smithi144.stdout:3/40: symlink d0/l10 0 2024-05-15T15:53:58.849 INFO:tasks.workunit.client.0.smithi049.stdout:7/140: unlink d0/d6/d23/l24 0 2024-05-15T15:53:58.850 INFO:tasks.workunit.client.1.smithi144.stdout:2/52: sync 2024-05-15T15:53:58.851 INFO:tasks.workunit.client.0.smithi049.stdout:7/141: truncate d0/d6/da/f1c 1525216 0 2024-05-15T15:53:58.852 INFO:tasks.workunit.client.1.smithi144.stdout:2/53: fsync d3/d7/fb 0 2024-05-15T15:53:58.852 INFO:tasks.workunit.client.0.smithi049.stdout:7/142: fdatasync d0/f4 0 2024-05-15T15:53:58.852 INFO:tasks.workunit.client.0.smithi049.stdout:7/143: stat d0/d6/f14 0 2024-05-15T15:53:58.853 INFO:tasks.workunit.client.0.smithi049.stdout:1/142: mknod d0/d4/dd/de/d1e/c35 0 2024-05-15T15:53:58.853 INFO:tasks.workunit.client.0.smithi049.stdout:7/144: chown d0/f4 2 1 2024-05-15T15:53:58.855 INFO:tasks.workunit.client.1.smithi144.stdout:3/41: dwrite d0/d1/f5 [0,4194304] 0 2024-05-15T15:53:58.855 INFO:tasks.workunit.client.1.smithi144.stdout:3/42: readlink d0/l10 0 2024-05-15T15:53:58.858 INFO:tasks.workunit.client.0.smithi049.stdout:6/80: fsync f8 0 2024-05-15T15:53:58.862 INFO:tasks.workunit.client.0.smithi049.stdout:2/150: rename d3/db/c13 to d3/da/d15/c29 0 2024-05-15T15:53:58.863 INFO:tasks.workunit.client.0.smithi049.stdout:2/151: chown d3/da/d15/f27 3388 1 2024-05-15T15:53:58.863 INFO:tasks.workunit.client.1.smithi144.stdout:1/73: mknod d7/df/c14 0 2024-05-15T15:53:58.864 INFO:tasks.workunit.client.0.smithi049.stdout:9/99: creat d0/d8/f24 x:0 0 0 2024-05-15T15:53:58.866 INFO:tasks.workunit.client.0.smithi049.stdout:9/100: truncate d0/d8/d9/f15 1030317 0 2024-05-15T15:53:58.874 INFO:tasks.workunit.client.1.smithi144.stdout:8/67: creat d2/de/f14 x:0 0 0 2024-05-15T15:53:58.877 INFO:tasks.workunit.client.1.smithi144.stdout:7/85: creat d4/f1c x:0 0 0 2024-05-15T15:53:58.879 INFO:tasks.workunit.client.0.smithi049.stdout:1/143: mkdir d0/d4/dd/de/d36 0 2024-05-15T15:53:58.880 INFO:tasks.workunit.client.1.smithi144.stdout:9/45: getdents d0/d8 0 2024-05-15T15:53:58.882 INFO:tasks.workunit.client.0.smithi049.stdout:1/144: stat d0/d4/dd/de/d1e/d16 0 2024-05-15T15:53:58.883 INFO:tasks.workunit.client.0.smithi049.stdout:1/145: dread - d0/d4/f2d zero size 2024-05-15T15:53:58.884 INFO:tasks.workunit.client.0.smithi049.stdout:3/146: dwrite d12/d14/f1e [0,4194304] 0 2024-05-15T15:53:58.885 INFO:tasks.workunit.client.0.smithi049.stdout:6/81: rmdir dd 39 2024-05-15T15:53:58.887 INFO:tasks.workunit.client.1.smithi144.stdout:0/52: truncate d4/f9 1068416 0 2024-05-15T15:53:58.887 INFO:tasks.workunit.client.0.smithi049.stdout:3/147: write d12/d14/f1b [870704,121403] 0 2024-05-15T15:53:58.889 INFO:tasks.workunit.client.0.smithi049.stdout:2/152: creat d3/db/d1d/d1e/f2a x:0 0 0 2024-05-15T15:53:58.890 INFO:tasks.workunit.client.0.smithi049.stdout:2/153: fdatasync d3/db/d1d/f20 0 2024-05-15T15:53:58.892 INFO:tasks.workunit.client.0.smithi049.stdout:9/101: chown d0/f2 187972 1 2024-05-15T15:53:58.893 INFO:tasks.workunit.client.1.smithi144.stdout:6/52: creat d7/db/d11/f13 x:0 0 0 2024-05-15T15:53:58.895 INFO:tasks.workunit.client.1.smithi144.stdout:3/43: dwrite d0/d1/fe [0,4194304] 0 2024-05-15T15:53:58.895 INFO:tasks.workunit.client.1.smithi144.stdout:5/67: creat d0/dc/d12/f18 x:0 0 0 2024-05-15T15:53:58.897 INFO:tasks.workunit.client.0.smithi049.stdout:7/145: mkdir d0/d6/d1f/d2a 0 2024-05-15T15:53:58.902 INFO:tasks.workunit.client.0.smithi049.stdout:7/146: dread d0/f4 [4194304,4194304] 0 2024-05-15T15:53:58.904 INFO:tasks.workunit.client.0.smithi049.stdout:7/147: write d0/f11 [9421114,88685] 0 2024-05-15T15:53:58.910 INFO:tasks.workunit.client.0.smithi049.stdout:3/148: creat d12/d14/f24 x:0 0 0 2024-05-15T15:53:58.912 INFO:tasks.workunit.client.0.smithi049.stdout:2/154: mkdir d3/db/d1d/d1e/d2b 0 2024-05-15T15:53:58.915 INFO:tasks.workunit.client.1.smithi144.stdout:1/74: write f1 [8682345,109962] 0 2024-05-15T15:53:58.917 INFO:tasks.workunit.client.1.smithi144.stdout:6/53: dread d7/f8 [0,4194304] 0 2024-05-15T15:53:58.917 INFO:tasks.workunit.client.0.smithi049.stdout:0/132: creat dc/d13/f26 x:0 0 0 2024-05-15T15:53:58.918 INFO:tasks.workunit.client.0.smithi049.stdout:0/133: dread - dc/f1d zero size 2024-05-15T15:53:58.919 INFO:tasks.workunit.client.1.smithi144.stdout:3/44: rename d0/da to d0/d11 0 2024-05-15T15:53:58.920 INFO:tasks.workunit.client.0.smithi049.stdout:0/134: dread fa [0,4194304] 0 2024-05-15T15:53:58.921 INFO:tasks.workunit.client.1.smithi144.stdout:5/68: mkdir d0/dc/d19 0 2024-05-15T15:53:58.924 INFO:tasks.workunit.client.1.smithi144.stdout:5/69: dread d0/f5 [0,4194304] 0 2024-05-15T15:53:58.925 INFO:tasks.workunit.client.0.smithi049.stdout:7/148: symlink d0/d6/d1d/l2b 0 2024-05-15T15:53:58.925 INFO:tasks.workunit.client.1.smithi144.stdout:5/70: chown d0/dc/d19 23525 1 2024-05-15T15:53:58.927 INFO:tasks.workunit.client.0.smithi049.stdout:6/82: write dd/f14 [1708680,9125] 0 2024-05-15T15:53:58.927 INFO:tasks.workunit.client.0.smithi049.stdout:0/135: dwrite dc/d13/d14/f19 [0,4194304] 0 2024-05-15T15:53:58.929 INFO:tasks.workunit.client.0.smithi049.stdout:6/83: write f4 [833070,86673] 0 2024-05-15T15:53:58.929 INFO:tasks.workunit.client.1.smithi144.stdout:7/86: truncate f2 4755304 0 2024-05-15T15:53:58.929 INFO:tasks.workunit.client.0.smithi049.stdout:6/84: stat dd/l11 0 2024-05-15T15:53:58.932 INFO:tasks.workunit.client.1.smithi144.stdout:9/46: mkdir d0/da/d16 0 2024-05-15T15:53:58.932 INFO:tasks.workunit.client.1.smithi144.stdout:9/47: chown d0/da 181562 1 2024-05-15T15:53:58.933 INFO:tasks.workunit.client.1.smithi144.stdout:9/48: write d0/f14 [799545,2970] 0 2024-05-15T15:53:58.937 INFO:tasks.workunit.client.0.smithi049.stdout:9/102: mkdir d0/d25 0 2024-05-15T15:53:58.937 INFO:tasks.workunit.client.0.smithi049.stdout:2/155: dread d3/f11 [0,4194304] 0 2024-05-15T15:53:58.938 INFO:tasks.workunit.client.0.smithi049.stdout:6/85: dwrite dd/ff [0,4194304] 0 2024-05-15T15:53:58.941 INFO:tasks.workunit.client.1.smithi144.stdout:1/75: creat d7/df/f15 x:0 0 0 2024-05-15T15:53:58.942 INFO:tasks.workunit.client.0.smithi049.stdout:1/146: creat d0/d4/dd/de/d1e/f37 x:0 0 0 2024-05-15T15:53:58.942 INFO:tasks.workunit.client.1.smithi144.stdout:1/76: dread - d7/df/f15 zero size 2024-05-15T15:53:58.942 INFO:tasks.workunit.client.1.smithi144.stdout:1/77: dread - d7/df/f15 zero size 2024-05-15T15:53:58.943 INFO:tasks.workunit.client.0.smithi049.stdout:3/149: write fc [8390,85051] 0 2024-05-15T15:53:58.945 INFO:tasks.workunit.client.0.smithi049.stdout:7/149: creat d0/d6/d1f/f2c x:0 0 0 2024-05-15T15:53:58.947 INFO:tasks.workunit.client.1.smithi144.stdout:3/45: creat d0/d11/f12 x:0 0 0 2024-05-15T15:53:58.947 INFO:tasks.workunit.client.0.smithi049.stdout:0/136: dread - dc/f1b zero size 2024-05-15T15:53:58.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:53:58 smithi144 ceph-mon[36389]: pgmap v19: 65 pgs: 65 active+clean; 276 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 363 op/s 2024-05-15T15:53:58.949 INFO:tasks.workunit.client.1.smithi144.stdout:8/68: write d2/f3 [374300,112910] 0 2024-05-15T15:53:58.951 INFO:tasks.workunit.client.0.smithi049.stdout:2/156: creat d3/da/f2c x:0 0 0 2024-05-15T15:53:58.951 INFO:tasks.workunit.client.1.smithi144.stdout:5/71: creat d0/f1a x:0 0 0 2024-05-15T15:53:58.952 INFO:tasks.workunit.client.0.smithi049.stdout:2/157: dread d3/f11 [0,4194304] 0 2024-05-15T15:53:58.952 INFO:tasks.workunit.client.1.smithi144.stdout:5/72: chown d0/dc/fe 11542324 1 2024-05-15T15:53:58.953 INFO:tasks.workunit.client.0.smithi049.stdout:2/158: chown d3/db 1588050 1 2024-05-15T15:53:58.953 INFO:tasks.workunit.client.1.smithi144.stdout:7/87: creat d4/df/d10/f1d x:0 0 0 2024-05-15T15:53:58.953 INFO:tasks.workunit.client.0.smithi049.stdout:6/86: rename dd/f10 to dd/f15 0 2024-05-15T15:53:58.955 INFO:tasks.workunit.client.1.smithi144.stdout:9/49: chown d0/l6 235604 1 2024-05-15T15:53:58.956 INFO:tasks.workunit.client.1.smithi144.stdout:0/53: write d4/f9 [951870,37513] 0 2024-05-15T15:53:58.956 INFO:tasks.workunit.client.1.smithi144.stdout:0/54: fsync d4/f5 0 2024-05-15T15:53:58.959 INFO:tasks.workunit.client.0.smithi049.stdout:9/103: mkdir d0/d8/d9/d22/d26 0 2024-05-15T15:53:58.960 INFO:tasks.workunit.client.1.smithi144.stdout:6/54: symlink d7/db/l14 0 2024-05-15T15:53:58.961 INFO:tasks.workunit.client.1.smithi144.stdout:3/46: creat d0/d11/f13 x:0 0 0 2024-05-15T15:53:58.962 INFO:tasks.workunit.client.1.smithi144.stdout:3/47: write d0/d1/f5 [2315458,4409] 0 2024-05-15T15:53:58.962 INFO:tasks.workunit.client.1.smithi144.stdout:3/48: chown d0/d1/c8 12608 1 2024-05-15T15:53:58.963 INFO:tasks.workunit.client.1.smithi144.stdout:3/49: write d0/d1/f5 [1960976,23599] 0 2024-05-15T15:53:58.964 INFO:tasks.workunit.client.1.smithi144.stdout:8/69: write d2/de/f10 [4338275,56712] 0 2024-05-15T15:53:58.965 INFO:tasks.workunit.client.1.smithi144.stdout:5/73: creat d0/dc/f1b x:0 0 0 2024-05-15T15:53:58.965 INFO:tasks.workunit.client.1.smithi144.stdout:7/88: creat d4/df/f1e x:0 0 0 2024-05-15T15:53:58.972 INFO:tasks.workunit.client.1.smithi144.stdout:9/50: creat d0/da/d10/f17 x:0 0 0 2024-05-15T15:53:58.972 INFO:tasks.workunit.client.1.smithi144.stdout:1/78: fsync d7/d9/fd 0 2024-05-15T15:53:58.972 INFO:tasks.workunit.client.1.smithi144.stdout:6/55: creat d7/db/d11/f15 x:0 0 0 2024-05-15T15:53:58.972 INFO:tasks.workunit.client.1.smithi144.stdout:1/79: write d7/df/f15 [385376,12572] 0 2024-05-15T15:53:58.972 INFO:tasks.workunit.client.1.smithi144.stdout:6/56: chown d7/db/c10 18100 1 2024-05-15T15:53:58.973 INFO:tasks.workunit.client.1.smithi144.stdout:8/70: creat d2/f15 x:0 0 0 2024-05-15T15:53:58.973 INFO:tasks.workunit.client.1.smithi144.stdout:3/50: dread d0/f6 [0,4194304] 0 2024-05-15T15:53:58.975 INFO:tasks.workunit.client.1.smithi144.stdout:8/71: write d2/f15 [772151,101029] 0 2024-05-15T15:53:58.977 INFO:tasks.workunit.client.1.smithi144.stdout:1/80: mknod d7/df/c16 0 2024-05-15T15:53:58.979 INFO:tasks.workunit.client.1.smithi144.stdout:5/74: dwrite d0/dc/fe [0,4194304] 0 2024-05-15T15:53:58.982 INFO:tasks.workunit.client.1.smithi144.stdout:3/51: creat d0/d11/f14 x:0 0 0 2024-05-15T15:53:58.984 INFO:tasks.workunit.client.1.smithi144.stdout:8/72: creat d2/f16 x:0 0 0 2024-05-15T15:53:58.985 INFO:tasks.workunit.client.1.smithi144.stdout:8/73: dread - d2/f16 zero size 2024-05-15T15:53:58.985 INFO:tasks.workunit.client.0.smithi049.stdout:4/236: sync 2024-05-15T15:53:58.986 INFO:tasks.workunit.client.0.smithi049.stdout:2/159: mknod d3/db/d1d/d1e/c2d 0 2024-05-15T15:53:58.987 INFO:tasks.workunit.client.1.smithi144.stdout:5/75: chown d0/f9 273 1 2024-05-15T15:53:58.987 INFO:tasks.workunit.client.1.smithi144.stdout:4/80: sync 2024-05-15T15:53:58.987 INFO:tasks.workunit.client.0.smithi049.stdout:3/150: link d12/f15 d12/f25 0 2024-05-15T15:53:58.987 INFO:tasks.workunit.client.0.smithi049.stdout:7/150: rename d0/d6/d1d/c21 to d0/d6/d1f/c2d 0 2024-05-15T15:53:58.988 INFO:tasks.workunit.client.1.smithi144.stdout:9/51: creat d0/f18 x:0 0 0 2024-05-15T15:53:58.988 INFO:tasks.workunit.client.0.smithi049.stdout:0/137: unlink l6 0 2024-05-15T15:53:58.989 INFO:tasks.workunit.client.1.smithi144.stdout:7/89: getdents d4 0 2024-05-15T15:53:58.991 INFO:tasks.workunit.client.1.smithi144.stdout:0/55: dwrite f2 [0,4194304] 0 2024-05-15T15:53:58.992 INFO:tasks.workunit.client.0.smithi049.stdout:3/151: rmdir d12 39 2024-05-15T15:53:58.992 INFO:tasks.workunit.client.0.smithi049.stdout:2/160: dread d3/d8/fe [0,4194304] 0 2024-05-15T15:53:58.992 INFO:tasks.workunit.client.0.smithi049.stdout:9/104: dwrite d0/d8/f1e [0,4194304] 0 2024-05-15T15:53:58.994 INFO:tasks.workunit.client.1.smithi144.stdout:5/76: mkdir d0/d1c 0 2024-05-15T15:53:58.994 INFO:tasks.workunit.client.0.smithi049.stdout:6/87: rename f5 to dd/f16 0 2024-05-15T15:53:58.996 INFO:tasks.workunit.client.0.smithi049.stdout:6/88: chown dd/f16 82649707 1 2024-05-15T15:53:58.996 INFO:tasks.workunit.client.0.smithi049.stdout:0/138: symlink dc/d13/d14/l27 0 2024-05-15T15:53:58.997 INFO:tasks.workunit.client.0.smithi049.stdout:1/147: dwrite d0/d4/dd/de/d1e/f1d [0,4194304] 0 2024-05-15T15:53:58.997 INFO:tasks.workunit.client.0.smithi049.stdout:6/89: stat dd 0 2024-05-15T15:53:58.997 INFO:tasks.workunit.client.0.smithi049.stdout:6/90: stat f2 0 2024-05-15T15:53:58.999 INFO:tasks.workunit.client.1.smithi144.stdout:7/90: dread d4/fe [0,4194304] 0 2024-05-15T15:53:58.999 INFO:tasks.workunit.client.1.smithi144.stdout:4/81: dread f0 [0,4194304] 0 2024-05-15T15:53:58.999 INFO:tasks.workunit.client.1.smithi144.stdout:6/57: dwrite d7/f8 [0,4194304] 0 2024-05-15T15:53:58.999 INFO:tasks.workunit.client.1.smithi144.stdout:9/52: mkdir d0/da/d10/d19 0 2024-05-15T15:53:58.999 INFO:tasks.workunit.client.1.smithi144.stdout:5/77: truncate d0/dc/f1b 474792 0 2024-05-15T15:53:59.000 INFO:tasks.workunit.client.0.smithi049.stdout:7/151: readlink d0/d6/d1f/l20 0 2024-05-15T15:53:59.000 INFO:tasks.workunit.client.1.smithi144.stdout:6/58: chown f6 122499 1 2024-05-15T15:53:59.001 INFO:tasks.workunit.client.1.smithi144.stdout:4/82: truncate d6/f18 453810 0 2024-05-15T15:53:59.001 INFO:tasks.workunit.client.1.smithi144.stdout:5/78: rename d0 to d0/dc/d1d 22 2024-05-15T15:53:59.003 INFO:tasks.workunit.client.1.smithi144.stdout:5/79: write d0/f11 [319891,92472] 0 2024-05-15T15:53:59.004 INFO:tasks.workunit.client.0.smithi049.stdout:6/91: dread f4 [0,4194304] 0 2024-05-15T15:53:59.006 INFO:tasks.workunit.client.1.smithi144.stdout:0/56: creat d4/df/f16 x:0 0 0 2024-05-15T15:53:59.009 INFO:tasks.workunit.client.1.smithi144.stdout:8/74: creat d2/de/d12/f17 x:0 0 0 2024-05-15T15:53:59.010 INFO:tasks.workunit.client.0.smithi049.stdout:9/105: symlink d0/d8/d12/l27 0 2024-05-15T15:53:59.016 INFO:tasks.workunit.client.0.smithi049.stdout:4/237: link d18/d21/f30 d18/d1e/d2a/d36/f4a 0 2024-05-15T15:53:59.017 INFO:tasks.workunit.client.0.smithi049.stdout:0/139: chown dc/f15 1463210579 1 2024-05-15T15:53:59.018 INFO:tasks.workunit.client.1.smithi144.stdout:6/59: symlink d7/db/d11/l16 0 2024-05-15T15:53:59.019 INFO:tasks.workunit.client.1.smithi144.stdout:6/60: write d7/ff [4083544,53641] 0 2024-05-15T15:53:59.019 INFO:tasks.workunit.client.1.smithi144.stdout:5/80: mknod d0/dc/d12/c1e 0 2024-05-15T15:53:59.021 INFO:tasks.workunit.client.0.smithi049.stdout:6/92: mkdir dd/d17 0 2024-05-15T15:53:59.021 INFO:tasks.workunit.client.1.smithi144.stdout:0/57: rmdir d4 39 2024-05-15T15:53:59.023 INFO:tasks.workunit.client.0.smithi049.stdout:3/152: mkdir d12/d26 0 2024-05-15T15:53:59.024 INFO:tasks.workunit.client.1.smithi144.stdout:8/75: rename d2/de/cb to d2/de/d12/c18 0 2024-05-15T15:53:59.025 INFO:tasks.workunit.client.0.smithi049.stdout:9/106: dwrite d0/d8/d11/f17 [0,4194304] 0 2024-05-15T15:53:59.025 INFO:tasks.workunit.client.0.smithi049.stdout:6/93: write dd/f14 [2417635,51400] 0 2024-05-15T15:53:59.025 INFO:tasks.workunit.client.0.smithi049.stdout:2/161: mkdir d3/db/d1d/d1e/d2b/d2e 0 2024-05-15T15:53:59.027 INFO:tasks.workunit.client.1.smithi144.stdout:5/81: dwrite d0/dc/f1b [0,4194304] 0 2024-05-15T15:53:59.027 INFO:tasks.workunit.client.1.smithi144.stdout:8/76: read d2/f15 [686249,78992] 0 2024-05-15T15:53:59.027 INFO:tasks.workunit.client.1.smithi144.stdout:9/53: getdents d0/d8 0 2024-05-15T15:53:59.027 INFO:tasks.workunit.client.1.smithi144.stdout:9/54: stat d0/da/d10 0 2024-05-15T15:53:59.028 INFO:tasks.workunit.client.1.smithi144.stdout:8/77: chown d2/de/c11 92162 1 2024-05-15T15:53:59.028 INFO:tasks.workunit.client.1.smithi144.stdout:9/55: chown d0/d8/l9 11042 1 2024-05-15T15:53:59.028 INFO:tasks.workunit.client.1.smithi144.stdout:4/83: mkdir d6/dd/d15/d19 0 2024-05-15T15:53:59.028 INFO:tasks.workunit.client.1.smithi144.stdout:8/78: chown d2/f15 187042 1 2024-05-15T15:53:59.029 INFO:tasks.workunit.client.0.smithi049.stdout:0/140: dwrite dc/f15 [0,4194304] 0 2024-05-15T15:53:59.029 INFO:tasks.workunit.client.1.smithi144.stdout:4/84: stat d6/da/db 0 2024-05-15T15:53:59.029 INFO:tasks.workunit.client.1.smithi144.stdout:9/56: write d0/f4 [1883512,24913] 0 2024-05-15T15:53:59.031 INFO:tasks.workunit.client.1.smithi144.stdout:4/85: write d6/da/ff [340989,66635] 0 2024-05-15T15:53:59.032 INFO:tasks.workunit.client.1.smithi144.stdout:4/86: chown d6/da/c10 39 1 2024-05-15T15:53:59.033 INFO:tasks.workunit.client.0.smithi049.stdout:4/238: creat d18/d1e/d2a/d36/f4b x:0 0 0 2024-05-15T15:53:59.033 INFO:tasks.workunit.client.1.smithi144.stdout:4/87: chown d6/da/c10 96 1 2024-05-15T15:53:59.034 INFO:tasks.workunit.client.1.smithi144.stdout:6/61: symlink d7/l17 0 2024-05-15T15:53:59.044 INFO:tasks.workunit.client.0.smithi049.stdout:6/94: dwrite dd/f14 [0,4194304] 0 2024-05-15T15:53:59.047 INFO:tasks.workunit.client.0.smithi049.stdout:2/162: dwrite d3/da/d15/f27 [0,4194304] 0 2024-05-15T15:53:59.049 INFO:tasks.workunit.client.1.smithi144.stdout:7/91: getdents d4 0 2024-05-15T15:53:59.049 INFO:tasks.workunit.client.1.smithi144.stdout:7/92: stat d4/fc 0 2024-05-15T15:53:59.050 INFO:tasks.workunit.client.0.smithi049.stdout:0/141: symlink dc/d13/d14/l28 0 2024-05-15T15:53:59.050 INFO:tasks.workunit.client.1.smithi144.stdout:7/93: fsync d4/df/f1e 0 2024-05-15T15:53:59.057 INFO:tasks.workunit.client.0.smithi049.stdout:7/152: creat d0/f2e x:0 0 0 2024-05-15T15:53:59.062 INFO:tasks.workunit.client.1.smithi144.stdout:8/79: rename d2/c13 to d2/de/d12/c19 0 2024-05-15T15:53:59.063 INFO:tasks.workunit.client.0.smithi049.stdout:6/95: write dd/f15 [950883,40087] 0 2024-05-15T15:53:59.064 INFO:tasks.workunit.client.0.smithi049.stdout:6/96: write fb [626961,36078] 0 2024-05-15T15:53:59.066 INFO:tasks.workunit.client.0.smithi049.stdout:2/163: rename d3/db/d1d/d1e/f2a to d3/da/d15/d24/f2f 0 2024-05-15T15:53:59.067 INFO:tasks.workunit.client.0.smithi049.stdout:2/164: chown d3/d8/f14 36 1 2024-05-15T15:53:59.067 INFO:tasks.workunit.client.0.smithi049.stdout:2/165: fdatasync d3/db/f23 0 2024-05-15T15:53:59.067 INFO:tasks.workunit.client.0.smithi049.stdout:9/107: creat d0/d8/d1b/f28 x:0 0 0 2024-05-15T15:53:59.070 INFO:tasks.workunit.client.0.smithi049.stdout:4/239: fdatasync d18/d21/d2b/f2c 0 2024-05-15T15:53:59.072 INFO:tasks.workunit.client.0.smithi049.stdout:9/108: dread d0/d8/d11/f17 [0,4194304] 0 2024-05-15T15:53:59.073 INFO:tasks.workunit.client.0.smithi049.stdout:3/153: write d12/f15 [1797426,1409] 0 2024-05-15T15:53:59.078 INFO:tasks.workunit.client.0.smithi049.stdout:0/142: rename dc/d13/d14/f1e to dc/d1f/f29 0 2024-05-15T15:53:59.084 INFO:tasks.workunit.client.1.smithi144.stdout:0/58: creat d4/dc/f17 x:0 0 0 2024-05-15T15:53:59.084 INFO:tasks.workunit.client.1.smithi144.stdout:5/82: getdents d0/dc/d19 0 2024-05-15T15:53:59.085 INFO:tasks.workunit.client.1.smithi144.stdout:0/59: write f2 [2713499,23181] 0 2024-05-15T15:53:59.086 INFO:tasks.workunit.client.0.smithi049.stdout:4/240: symlink d18/l4c 0 2024-05-15T15:53:59.087 INFO:tasks.workunit.client.0.smithi049.stdout:7/153: write d0/f9 [1182432,6975] 0 2024-05-15T15:53:59.089 INFO:tasks.workunit.client.0.smithi049.stdout:9/109: rmdir d0/d8 39 2024-05-15T15:53:59.092 INFO:tasks.workunit.client.0.smithi049.stdout:3/154: rename d12/d14/f19 to d12/d14/f27 0 2024-05-15T15:53:59.096 INFO:tasks.workunit.client.0.smithi049.stdout:0/143: stat dc/c1a 0 2024-05-15T15:53:59.108 INFO:tasks.workunit.client.0.smithi049.stdout:4/241: write d18/d21/d2b/f2c [3977643,53975] 0 2024-05-15T15:53:59.109 INFO:tasks.workunit.client.0.smithi049.stdout:3/155: mknod d12/c28 0 2024-05-15T15:53:59.111 INFO:tasks.workunit.client.0.smithi049.stdout:0/144: mkdir dc/d13/d14/d2a 0 2024-05-15T15:53:59.112 INFO:tasks.workunit.client.1.smithi144.stdout:2/54: sync 2024-05-15T15:53:59.112 INFO:tasks.workunit.client.1.smithi144.stdout:1/81: sync 2024-05-15T15:53:59.112 INFO:tasks.workunit.client.0.smithi049.stdout:0/145: dread - dc/d13/f26 zero size 2024-05-15T15:53:59.113 INFO:tasks.workunit.client.0.smithi049.stdout:2/166: link d3/db/f1f d3/db/f30 0 2024-05-15T15:53:59.116 INFO:tasks.workunit.client.0.smithi049.stdout:9/110: dwrite d0/f2 [0,4194304] 0 2024-05-15T15:53:59.118 INFO:tasks.workunit.client.0.smithi049.stdout:3/156: rmdir d12 39 2024-05-15T15:53:59.123 INFO:tasks.workunit.client.0.smithi049.stdout:2/167: creat d3/db/d1d/d1e/d2b/f31 x:0 0 0 2024-05-15T15:53:59.126 INFO:tasks.workunit.client.0.smithi049.stdout:9/111: write d0/d8/d11/f16 [99449,129053] 0 2024-05-15T15:53:59.127 INFO:tasks.workunit.client.0.smithi049.stdout:9/112: chown d0/d8/d9/fb 188717 1 2024-05-15T15:53:59.128 INFO:tasks.workunit.client.0.smithi049.stdout:7/154: rmdir d0/d6/d1f/d2a 0 2024-05-15T15:53:59.128 INFO:tasks.workunit.client.0.smithi049.stdout:7/155: read - d0/f2e zero size 2024-05-15T15:53:59.132 INFO:tasks.workunit.client.0.smithi049.stdout:0/146: mkdir dc/d13/d2b 0 2024-05-15T15:53:59.139 INFO:tasks.workunit.client.0.smithi049.stdout:9/113: symlink d0/d8/d9/l29 0 2024-05-15T15:53:59.141 INFO:tasks.workunit.client.0.smithi049.stdout:7/156: fsync d0/d6/f18 0 2024-05-15T15:53:59.143 INFO:tasks.workunit.client.0.smithi049.stdout:2/168: rmdir d3/db/d1d 39 2024-05-15T15:53:59.145 INFO:tasks.workunit.client.0.smithi049.stdout:3/157: creat d12/f29 x:0 0 0 2024-05-15T15:53:59.145 INFO:tasks.workunit.client.0.smithi049.stdout:3/158: chown d12/d14/l16 120061044 1 2024-05-15T15:53:59.145 INFO:tasks.workunit.client.0.smithi049.stdout:3/159: chown d12 4797 1 2024-05-15T15:53:59.152 INFO:tasks.workunit.client.0.smithi049.stdout:3/160: dwrite f7 [0,4194304] 0 2024-05-15T15:53:59.157 INFO:tasks.workunit.client.0.smithi049.stdout:0/147: creat dc/d13/d14/d2a/f2c x:0 0 0 2024-05-15T15:53:59.157 INFO:tasks.workunit.client.0.smithi049.stdout:0/148: stat dc/c10 0 2024-05-15T15:53:59.158 INFO:tasks.workunit.client.1.smithi144.stdout:3/52: sync 2024-05-15T15:53:59.159 INFO:tasks.workunit.client.1.smithi144.stdout:7/94: sync 2024-05-15T15:53:59.160 INFO:tasks.workunit.client.1.smithi144.stdout:7/95: write d4/f6 [87416,13922] 0 2024-05-15T15:53:59.160 INFO:tasks.workunit.client.1.smithi144.stdout:3/53: write d0/d11/f12 [125914,75635] 0 2024-05-15T15:53:59.161 INFO:tasks.workunit.client.0.smithi049.stdout:0/149: dread dc/d1f/f29 [0,4194304] 0 2024-05-15T15:53:59.161 INFO:tasks.workunit.client.1.smithi144.stdout:7/96: dread - d4/f1c zero size 2024-05-15T15:53:59.164 INFO:tasks.workunit.client.1.smithi144.stdout:3/54: dread d0/d1/f4 [0,4194304] 0 2024-05-15T15:53:59.164 INFO:tasks.workunit.client.1.smithi144.stdout:3/55: fdatasync d0/d1/f4 0 2024-05-15T15:53:59.167 INFO:tasks.workunit.client.0.smithi049.stdout:0/150: dwrite dc/d1f/f24 [0,4194304] 0 2024-05-15T15:53:59.172 INFO:tasks.workunit.client.1.smithi144.stdout:3/56: dwrite d0/d11/f12 [0,4194304] 0 2024-05-15T15:53:59.198 INFO:tasks.workunit.client.0.smithi049.stdout:7/157: mkdir d0/d2f 0 2024-05-15T15:53:59.198 INFO:tasks.workunit.client.0.smithi049.stdout:7/158: dread - d0/d6/d1f/f2c zero size 2024-05-15T15:53:59.199 INFO:tasks.workunit.client.0.smithi049.stdout:2/169: mknod d3/db/d1d/c32 0 2024-05-15T15:53:59.202 INFO:tasks.workunit.client.0.smithi049.stdout:0/151: symlink dc/d1f/l2d 0 2024-05-15T15:53:59.205 INFO:tasks.workunit.client.0.smithi049.stdout:2/170: dwrite d3/d8/f14 [0,4194304] 0 2024-05-15T15:53:59.209 INFO:tasks.workunit.client.0.smithi049.stdout:0/152: dread dc/d13/f17 [0,4194304] 0 2024-05-15T15:53:59.212 INFO:tasks.workunit.client.0.smithi049.stdout:2/171: dwrite d3/da/f2c [0,4194304] 0 2024-05-15T15:53:59.214 INFO:tasks.workunit.client.0.smithi049.stdout:2/172: write d3/db/f1a [707597,128097] 0 2024-05-15T15:53:59.221 INFO:tasks.workunit.client.0.smithi049.stdout:2/173: dwrite d3/db/f23 [0,4194304] 0 2024-05-15T15:53:59.222 INFO:tasks.workunit.client.0.smithi049.stdout:2/174: write d3/db/d1d/d1e/d2b/f31 [595942,42582] 0 2024-05-15T15:53:59.287 INFO:tasks.workunit.client.0.smithi049.stdout:9/114: truncate d0/d8/d11/f17 2338058 0 2024-05-15T15:53:59.287 INFO:tasks.workunit.client.0.smithi049.stdout:9/115: rename d0 to d0/d8/d1b/d2a 22 2024-05-15T15:53:59.288 INFO:tasks.workunit.client.0.smithi049.stdout:9/116: write d0/d8/d11/f16 [591687,73173] 0 2024-05-15T15:53:59.293 INFO:tasks.workunit.client.0.smithi049.stdout:3/161: link d12/f22 d12/d14/f2a 0 2024-05-15T15:53:59.293 INFO:tasks.workunit.client.0.smithi049.stdout:3/162: chown ff 18 1 2024-05-15T15:53:59.295 INFO:tasks.workunit.client.0.smithi049.stdout:9/117: dwrite d0/d8/d9/fb [0,4194304] 0 2024-05-15T15:53:59.295 INFO:tasks.workunit.client.0.smithi049.stdout:0/153: creat dc/d13/d14/f2e x:0 0 0 2024-05-15T15:53:59.297 INFO:tasks.workunit.client.0.smithi049.stdout:9/118: rename d0/d8 to d0/d8/d11/d2b 22 2024-05-15T15:53:59.297 INFO:tasks.workunit.client.0.smithi049.stdout:9/119: chown d0/d8/d9 60 1 2024-05-15T15:53:59.298 INFO:tasks.workunit.client.0.smithi049.stdout:9/120: chown d0/l20 20 1 2024-05-15T15:53:59.303 INFO:tasks.workunit.client.0.smithi049.stdout:8/89: sync 2024-05-15T15:53:59.310 INFO:tasks.workunit.client.0.smithi049.stdout:7/159: link d0/d6/f1b d0/d6/d23/f30 0 2024-05-15T15:53:59.313 INFO:tasks.workunit.client.0.smithi049.stdout:0/154: dread f3 [0,4194304] 0 2024-05-15T15:53:59.317 INFO:tasks.workunit.client.0.smithi049.stdout:7/160: dwrite d0/d6/d1d/f27 [0,4194304] 0 2024-05-15T15:53:59.325 INFO:tasks.workunit.client.0.smithi049.stdout:9/121: rmdir d0/d8/d9/d22 39 2024-05-15T15:53:59.329 INFO:tasks.workunit.client.0.smithi049.stdout:2/175: mknod d3/db/c33 0 2024-05-15T15:53:59.330 INFO:tasks.workunit.client.0.smithi049.stdout:2/176: chown d3/d8/lf 16 1 2024-05-15T15:53:59.332 INFO:tasks.workunit.client.0.smithi049.stdout:8/90: dwrite d2/fb [0,4194304] 0 2024-05-15T15:53:59.336 INFO:tasks.workunit.client.0.smithi049.stdout:0/155: rename dc/d1f to dc/d13/d14/d2f 0 2024-05-15T15:53:59.337 INFO:tasks.workunit.client.0.smithi049.stdout:2/177: dread d3/d8/f14 [0,4194304] 0 2024-05-15T15:53:59.337 INFO:tasks.workunit.client.0.smithi049.stdout:0/156: truncate dc/f1b 448800 0 2024-05-15T15:53:59.347 INFO:tasks.workunit.client.0.smithi049.stdout:3/163: dwrite d12/f1d [0,4194304] 0 2024-05-15T15:53:59.350 INFO:tasks.workunit.client.0.smithi049.stdout:2/178: dwrite d3/db/f12 [0,4194304] 0 2024-05-15T15:53:59.350 INFO:tasks.workunit.client.0.smithi049.stdout:8/91: chown d2/f12 42 1 2024-05-15T15:53:59.352 INFO:tasks.workunit.client.0.smithi049.stdout:2/179: write d3/db/d1d/f28 [747763,109823] 0 2024-05-15T15:53:59.353 INFO:tasks.workunit.client.0.smithi049.stdout:2/180: fdatasync d3/da/d15/f27 0 2024-05-15T15:53:59.360 INFO:tasks.workunit.client.0.smithi049.stdout:0/157: creat dc/d13/d14/d2a/f30 x:0 0 0 2024-05-15T15:53:59.365 INFO:tasks.workunit.client.0.smithi049.stdout:9/122: mknod d0/d8/d9/c2c 0 2024-05-15T15:53:59.369 INFO:tasks.workunit.client.0.smithi049.stdout:3/164: symlink d12/l2b 0 2024-05-15T15:53:59.371 INFO:tasks.workunit.client.0.smithi049.stdout:8/92: creat d2/d14/f1c x:0 0 0 2024-05-15T15:53:59.373 INFO:tasks.workunit.client.0.smithi049.stdout:7/161: link d0/l22 d0/d6/d23/l31 0 2024-05-15T15:53:59.377 INFO:tasks.workunit.client.0.smithi049.stdout:2/181: rmdir d3/db/d1d/d1e/d2b 39 2024-05-15T15:53:59.377 INFO:tasks.workunit.client.0.smithi049.stdout:2/182: stat d3/db/d1d/c32 0 2024-05-15T15:53:59.384 INFO:tasks.workunit.client.0.smithi049.stdout:1/148: sync 2024-05-15T15:53:59.384 INFO:tasks.workunit.client.0.smithi049.stdout:5/157: sync 2024-05-15T15:53:59.384 INFO:tasks.workunit.client.0.smithi049.stdout:6/97: sync 2024-05-15T15:53:59.385 INFO:tasks.workunit.client.0.smithi049.stdout:3/165: creat d12/d14/f2c x:0 0 0 2024-05-15T15:53:59.385 INFO:tasks.workunit.client.0.smithi049.stdout:6/98: write fb [1181949,3853] 0 2024-05-15T15:53:59.386 INFO:tasks.workunit.client.0.smithi049.stdout:1/149: fdatasync d0/d4/dd/f28 0 2024-05-15T15:53:59.386 INFO:tasks.workunit.client.0.smithi049.stdout:3/166: chown fc 2127 1 2024-05-15T15:53:59.389 INFO:tasks.workunit.client.0.smithi049.stdout:2/183: truncate d3/f10 2997696 0 2024-05-15T15:53:59.390 INFO:tasks.workunit.client.0.smithi049.stdout:0/158: mkdir dc/d13/d14/d25/d31 0 2024-05-15T15:53:59.391 INFO:tasks.workunit.client.0.smithi049.stdout:9/123: symlink d0/d8/d21/l2d 0 2024-05-15T15:53:59.395 INFO:tasks.workunit.client.0.smithi049.stdout:6/99: write f2 [1672079,91084] 0 2024-05-15T15:53:59.396 INFO:tasks.workunit.client.0.smithi049.stdout:1/150: creat d0/d4/dd/f38 x:0 0 0 2024-05-15T15:53:59.399 INFO:tasks.workunit.client.0.smithi049.stdout:3/167: symlink d12/d14/l2d 0 2024-05-15T15:53:59.403 INFO:tasks.workunit.client.0.smithi049.stdout:2/184: write d3/db/d1d/d1e/d2b/f31 [407271,88782] 0 2024-05-15T15:53:59.405 INFO:tasks.workunit.client.0.smithi049.stdout:0/159: stat dc/l12 0 2024-05-15T15:53:59.406 INFO:tasks.workunit.client.0.smithi049.stdout:0/160: fsync dc/d13/d14/f2e 0 2024-05-15T15:53:59.407 INFO:tasks.workunit.client.0.smithi049.stdout:6/100: dread f2 [0,4194304] 0 2024-05-15T15:53:59.408 INFO:tasks.workunit.client.0.smithi049.stdout:0/161: read dc/d13/d14/f19 [2305548,16139] 0 2024-05-15T15:53:59.411 INFO:tasks.workunit.client.0.smithi049.stdout:6/101: dread f8 [0,4194304] 0 2024-05-15T15:53:59.424 INFO:tasks.workunit.client.0.smithi049.stdout:7/162: truncate d0/f11 4507622 0 2024-05-15T15:53:59.432 INFO:tasks.workunit.client.0.smithi049.stdout:8/93: getdents d2 0 2024-05-15T15:53:59.436 INFO:tasks.workunit.client.0.smithi049.stdout:3/168: symlink d12/d26/l2e 0 2024-05-15T15:53:59.437 INFO:tasks.workunit.client.0.smithi049.stdout:6/102: dwrite f0 [0,4194304] 0 2024-05-15T15:53:59.445 INFO:tasks.workunit.client.0.smithi049.stdout:8/94: dwrite d2/f3 [4194304,4194304] 0 2024-05-15T15:53:59.445 INFO:tasks.workunit.client.0.smithi049.stdout:8/95: stat d2/f12 0 2024-05-15T15:53:59.445 INFO:tasks.workunit.client.0.smithi049.stdout:6/103: dread dd/f15 [0,4194304] 0 2024-05-15T15:53:59.448 INFO:tasks.workunit.client.0.smithi049.stdout:0/162: mkdir dc/d32 0 2024-05-15T15:53:59.450 INFO:tasks.workunit.client.0.smithi049.stdout:5/158: getdents dc/dd 0 2024-05-15T15:53:59.451 INFO:tasks.workunit.client.0.smithi049.stdout:5/159: chown dc/f1b 47591478 1 2024-05-15T15:53:59.451 INFO:tasks.workunit.client.0.smithi049.stdout:0/163: write dc/d13/f26 [922805,88482] 0 2024-05-15T15:53:59.453 INFO:tasks.workunit.client.0.smithi049.stdout:5/160: readlink dc/d29/d2c/l2d 0 2024-05-15T15:53:59.454 INFO:tasks.workunit.client.0.smithi049.stdout:0/164: truncate dc/f1b 1155448 0 2024-05-15T15:53:59.455 INFO:tasks.workunit.client.0.smithi049.stdout:5/161: write f9 [4399988,27029] 0 2024-05-15T15:53:59.457 INFO:tasks.workunit.client.0.smithi049.stdout:6/104: dread dd/ff [0,4194304] 0 2024-05-15T15:53:59.457 INFO:tasks.workunit.client.0.smithi049.stdout:5/162: chown dc/f11 130786 1 2024-05-15T15:53:59.460 INFO:tasks.workunit.client.0.smithi049.stdout:3/169: dread f10 [0,4194304] 0 2024-05-15T15:53:59.460 INFO:tasks.workunit.client.0.smithi049.stdout:8/96: dread d2/f12 [0,4194304] 0 2024-05-15T15:53:59.464 INFO:tasks.workunit.client.0.smithi049.stdout:1/151: rename d0/d4/dd/f26 to d0/d4/f39 0 2024-05-15T15:53:59.465 INFO:tasks.workunit.client.0.smithi049.stdout:1/152: fdatasync d0/d4/dd/de/d1e/f1d 0 2024-05-15T15:53:59.466 INFO:tasks.workunit.client.0.smithi049.stdout:1/153: dread - d0/d4/dd/de/d1e/f2f zero size 2024-05-15T15:53:59.468 INFO:tasks.workunit.client.0.smithi049.stdout:1/154: chown d0/d4/dd/de/d1e/f2f 2986 1 2024-05-15T15:53:59.469 INFO:tasks.workunit.client.0.smithi049.stdout:9/124: getdents d0 0 2024-05-15T15:53:59.474 INFO:tasks.workunit.client.0.smithi049.stdout:1/155: dwrite d0/d4/f2d [0,4194304] 0 2024-05-15T15:53:59.481 INFO:tasks.workunit.client.0.smithi049.stdout:0/165: mkdir dc/d13/d14/d33 0 2024-05-15T15:53:59.483 INFO:tasks.workunit.client.0.smithi049.stdout:7/163: mknod d0/d2f/c32 0 2024-05-15T15:53:59.484 INFO:tasks.workunit.client.0.smithi049.stdout:7/164: read d0/d6/f18 [536562,85515] 0 2024-05-15T15:53:59.485 INFO:tasks.workunit.client.0.smithi049.stdout:5/163: rmdir dc/dd 39 2024-05-15T15:53:59.488 INFO:tasks.workunit.client.0.smithi049.stdout:6/105: mknod dd/c18 0 2024-05-15T15:53:59.488 INFO:tasks.workunit.client.0.smithi049.stdout:6/106: readlink dd/l13 0 2024-05-15T15:53:59.489 INFO:tasks.workunit.client.0.smithi049.stdout:5/164: dread dc/f1b [0,4194304] 0 2024-05-15T15:53:59.491 INFO:tasks.workunit.client.0.smithi049.stdout:3/170: creat d12/d26/f2f x:0 0 0 2024-05-15T15:53:59.493 INFO:tasks.workunit.client.0.smithi049.stdout:8/97: mkdir d2/d14/d1d 0 2024-05-15T15:53:59.502 INFO:tasks.workunit.client.0.smithi049.stdout:3/171: dread f5 [0,4194304] 0 2024-05-15T15:53:59.509 INFO:tasks.workunit.client.0.smithi049.stdout:1/156: symlink d0/d4/dd/de/d1e/d2e/l3a 0 2024-05-15T15:53:59.523 INFO:tasks.workunit.client.0.smithi049.stdout:5/165: dread - dc/dd/f14 zero size 2024-05-15T15:53:59.526 INFO:tasks.workunit.client.0.smithi049.stdout:8/98: unlink d2/d18/f1a 0 2024-05-15T15:53:59.528 INFO:tasks.workunit.client.0.smithi049.stdout:9/125: creat d0/d8/d9/d22/f2e x:0 0 0 2024-05-15T15:53:59.530 INFO:tasks.workunit.client.0.smithi049.stdout:4/242: sync 2024-05-15T15:53:59.530 INFO:tasks.workunit.client.0.smithi049.stdout:3/172: rmdir d12/d14 39 2024-05-15T15:53:59.531 INFO:tasks.workunit.client.0.smithi049.stdout:3/173: dread - d12/f29 zero size 2024-05-15T15:53:59.531 INFO:tasks.workunit.client.0.smithi049.stdout:3/174: stat d12/d26/l2e 0 2024-05-15T15:53:59.533 INFO:tasks.workunit.client.0.smithi049.stdout:1/157: mkdir d0/d4/dd/de/d3b 0 2024-05-15T15:53:59.533 INFO:tasks.workunit.client.0.smithi049.stdout:1/158: dread - d0/d4/f9 zero size 2024-05-15T15:53:59.546 INFO:tasks.workunit.client.0.smithi049.stdout:5/166: mkdir dc/d29/d2c/d31 0 2024-05-15T15:53:59.548 INFO:tasks.workunit.client.0.smithi049.stdout:8/99: creat d2/d14/f1e x:0 0 0 2024-05-15T15:53:59.549 INFO:tasks.workunit.client.0.smithi049.stdout:9/126: rmdir d0 39 2024-05-15T15:53:59.552 INFO:tasks.workunit.client.0.smithi049.stdout:4/243: creat d18/f4d x:0 0 0 2024-05-15T15:53:59.556 INFO:tasks.workunit.client.0.smithi049.stdout:1/159: unlink d0/l6 0 2024-05-15T15:53:59.558 INFO:tasks.workunit.client.0.smithi049.stdout:7/165: truncate d0/d6/da/f1c 313427 0 2024-05-15T15:53:59.562 INFO:tasks.workunit.client.0.smithi049.stdout:5/167: dread dc/d29/d2c/f2f [0,4194304] 0 2024-05-15T15:53:59.562 INFO:tasks.workunit.client.0.smithi049.stdout:2/185: sync 2024-05-15T15:53:59.563 INFO:tasks.workunit.client.0.smithi049.stdout:8/100: mknod d2/d18/c1f 0 2024-05-15T15:53:59.563 INFO:tasks.workunit.client.0.smithi049.stdout:5/168: chown dc/f16 396694 1 2024-05-15T15:53:59.566 INFO:tasks.workunit.client.0.smithi049.stdout:4/244: rmdir d18/d21/d2b 39 2024-05-15T15:53:59.569 INFO:tasks.workunit.client.0.smithi049.stdout:8/101: dwrite d2/de/f16 [0,4194304] 0 2024-05-15T15:53:59.569 INFO:tasks.workunit.client.0.smithi049.stdout:3/175: mkdir d12/d30 0 2024-05-15T15:53:59.570 INFO:tasks.workunit.client.0.smithi049.stdout:8/102: chown f0 15688 1 2024-05-15T15:53:59.574 INFO:tasks.workunit.client.0.smithi049.stdout:0/166: getdents dc/d13 0 2024-05-15T15:53:59.581 INFO:tasks.workunit.client.0.smithi049.stdout:6/107: getdents dd 0 2024-05-15T15:53:59.586 INFO:tasks.workunit.client.0.smithi049.stdout:6/108: chown dd/f16 1786935 1 2024-05-15T15:53:59.587 INFO:tasks.workunit.client.0.smithi049.stdout:0/167: dwrite dc/d13/d14/f1c [0,4194304] 0 2024-05-15T15:53:59.588 INFO:tasks.workunit.client.0.smithi049.stdout:5/169: creat dc/d29/f32 x:0 0 0 2024-05-15T15:53:59.591 INFO:tasks.workunit.client.0.smithi049.stdout:9/127: creat d0/d8/d21/f2f x:0 0 0 2024-05-15T15:53:59.593 INFO:tasks.workunit.client.0.smithi049.stdout:0/168: dwrite dc/f1d [0,4194304] 0 2024-05-15T15:53:59.598 INFO:tasks.workunit.client.0.smithi049.stdout:8/103: dread d2/f3 [0,4194304] 0 2024-05-15T15:53:59.599 INFO:tasks.workunit.client.0.smithi049.stdout:8/104: write d2/f3 [4276001,69472] 0 2024-05-15T15:53:59.601 INFO:tasks.workunit.client.0.smithi049.stdout:7/166: write d0/fb [1642932,96861] 0 2024-05-15T15:53:59.608 INFO:tasks.workunit.client.0.smithi049.stdout:2/186: write d3/f11 [130780,100018] 0 2024-05-15T15:53:59.613 INFO:tasks.workunit.client.0.smithi049.stdout:6/109: creat dd/f19 x:0 0 0 2024-05-15T15:53:59.616 INFO:tasks.workunit.client.0.smithi049.stdout:8/105: dread f0 [0,4194304] 0 2024-05-15T15:53:59.619 INFO:tasks.workunit.client.0.smithi049.stdout:5/170: symlink dc/dd/d10/l33 0 2024-05-15T15:53:59.620 INFO:tasks.workunit.client.0.smithi049.stdout:4/245: dwrite f11 [0,4194304] 0 2024-05-15T15:53:59.625 INFO:tasks.workunit.client.0.smithi049.stdout:3/176: symlink d12/d14/l31 0 2024-05-15T15:53:59.627 INFO:tasks.workunit.client.0.smithi049.stdout:7/167: creat d0/d6/d1f/f33 x:0 0 0 2024-05-15T15:53:59.631 INFO:tasks.workunit.client.0.smithi049.stdout:1/160: link d0/d4/c7 d0/d4/dd/de/d1e/d2e/c3c 0 2024-05-15T15:53:59.633 INFO:tasks.workunit.client.0.smithi049.stdout:1/161: write d0/d4/f1b [2565090,114374] 0 2024-05-15T15:53:59.633 INFO:tasks.workunit.client.0.smithi049.stdout:2/187: write d3/da/d15/d24/f2f [432008,32533] 0 2024-05-15T15:53:59.634 INFO:tasks.workunit.client.0.smithi049.stdout:6/110: fsync dd/ff 0 2024-05-15T15:53:59.635 INFO:tasks.workunit.client.0.smithi049.stdout:2/188: rename d3/db to d3/db/d1d/d34 22 2024-05-15T15:53:59.636 INFO:tasks.workunit.client.0.smithi049.stdout:8/106: write d2/f3 [9039291,32334] 0 2024-05-15T15:53:59.642 INFO:tasks.workunit.client.0.smithi049.stdout:6/111: symlink dd/l1a 0 2024-05-15T15:53:59.644 INFO:tasks.workunit.client.0.smithi049.stdout:1/162: unlink d0/d4/dd/de/d1e/l1a 0 2024-05-15T15:53:59.645 INFO:tasks.workunit.client.0.smithi049.stdout:1/163: write d0/d4/f1b [253302,43741] 0 2024-05-15T15:53:59.646 INFO:tasks.workunit.client.0.smithi049.stdout:5/171: symlink dc/l34 0 2024-05-15T15:53:59.647 INFO:tasks.workunit.client.0.smithi049.stdout:5/172: write dc/dd/d10/f2a [856987,28460] 0 2024-05-15T15:53:59.648 INFO:tasks.workunit.client.0.smithi049.stdout:5/173: chown dc/f19 376 1 2024-05-15T15:53:59.650 INFO:tasks.workunit.client.0.smithi049.stdout:8/107: symlink d2/d18/l20 0 2024-05-15T15:53:59.652 INFO:tasks.workunit.client.0.smithi049.stdout:4/246: mkdir d18/d21/d2b/d4e 0 2024-05-15T15:53:59.653 INFO:tasks.workunit.client.0.smithi049.stdout:4/247: chown d18/d21/d2b/l33 46692278 1 2024-05-15T15:53:59.653 INFO:tasks.workunit.client.0.smithi049.stdout:8/108: dread f0 [0,4194304] 0 2024-05-15T15:53:59.654 INFO:tasks.workunit.client.0.smithi049.stdout:4/248: truncate f10 4709583 0 2024-05-15T15:53:59.655 INFO:tasks.workunit.client.0.smithi049.stdout:0/169: creat dc/f34 x:0 0 0 2024-05-15T15:53:59.655 INFO:tasks.workunit.client.0.smithi049.stdout:4/249: write d18/d21/f26 [886556,83992] 0 2024-05-15T15:53:59.658 INFO:tasks.workunit.client.0.smithi049.stdout:6/112: symlink dd/l1b 0 2024-05-15T15:53:59.664 INFO:tasks.workunit.client.0.smithi049.stdout:5/174: rename dc/d29/d2c/f2f to dc/d29/d2c/d31/f35 0 2024-05-15T15:53:59.668 INFO:tasks.workunit.client.0.smithi049.stdout:8/109: rmdir d2/d14 39 2024-05-15T15:53:59.676 INFO:tasks.workunit.client.0.smithi049.stdout:1/164: creat d0/d4/dd/de/d3b/f3d x:0 0 0 2024-05-15T15:53:59.677 INFO:tasks.workunit.client.0.smithi049.stdout:1/165: write d0/d4/f1b [4678065,88214] 0 2024-05-15T15:53:59.680 INFO:tasks.workunit.client.0.smithi049.stdout:5/175: creat dc/d29/d2c/f36 x:0 0 0 2024-05-15T15:53:59.681 INFO:tasks.workunit.client.0.smithi049.stdout:1/166: dread d0/d4/dd/de/d1e/f27 [0,4194304] 0 2024-05-15T15:53:59.681 INFO:tasks.workunit.client.0.smithi049.stdout:5/176: chown dc/dd/d10/l27 42273 1 2024-05-15T15:53:59.682 INFO:tasks.workunit.client.0.smithi049.stdout:2/189: truncate d3/f11 93767 0 2024-05-15T15:53:59.683 INFO:tasks.workunit.client.0.smithi049.stdout:2/190: read - d3/db/d1d/f20 zero size 2024-05-15T15:53:59.686 INFO:tasks.workunit.client.0.smithi049.stdout:0/170: mknod dc/d13/d2b/c35 0 2024-05-15T15:53:59.689 INFO:tasks.workunit.client.0.smithi049.stdout:3/177: getdents d12/d14 0 2024-05-15T15:53:59.691 INFO:tasks.workunit.client.0.smithi049.stdout:6/113: getdents dd/d17 0 2024-05-15T15:53:59.692 INFO:tasks.workunit.client.0.smithi049.stdout:7/168: dwrite d0/d6/da/f1c [0,4194304] 0 2024-05-15T15:53:59.699 INFO:tasks.workunit.client.0.smithi049.stdout:8/110: write d2/d14/f17 [282799,44542] 0 2024-05-15T15:53:59.703 INFO:tasks.workunit.client.0.smithi049.stdout:5/177: dwrite dc/f1b [0,4194304] 0 2024-05-15T15:53:59.704 INFO:tasks.workunit.client.0.smithi049.stdout:4/250: creat d18/d1e/d2a/f4f x:0 0 0 2024-05-15T15:53:59.710 INFO:tasks.workunit.client.0.smithi049.stdout:1/167: rename d0/d4/dd/de/d3b to d0/d4/dd/de/d1e/d2e/d3e 0 2024-05-15T15:53:59.711 INFO:tasks.workunit.client.0.smithi049.stdout:0/171: dread dc/fd [0,4194304] 0 2024-05-15T15:53:59.714 INFO:tasks.workunit.client.0.smithi049.stdout:7/169: rmdir d0 39 2024-05-15T15:53:59.727 INFO:tasks.workunit.client.0.smithi049.stdout:5/178: rmdir dc/dd/d10 39 2024-05-15T15:53:59.728 INFO:tasks.workunit.client.0.smithi049.stdout:5/179: write dc/f16 [3219102,93605] 0 2024-05-15T15:53:59.728 INFO:tasks.workunit.client.0.smithi049.stdout:5/180: chown dc 771566 1 2024-05-15T15:53:59.737 INFO:tasks.workunit.client.0.smithi049.stdout:0/172: rmdir dc/d13/d14 39 2024-05-15T15:53:59.741 INFO:tasks.workunit.client.0.smithi049.stdout:1/168: rmdir d0 39 2024-05-15T15:53:59.744 INFO:tasks.workunit.client.0.smithi049.stdout:6/114: write f4 [1085369,49608] 0 2024-05-15T15:53:59.753 INFO:tasks.workunit.client.0.smithi049.stdout:4/251: mkdir d18/d1e/d2a/d50 0 2024-05-15T15:53:59.756 INFO:tasks.workunit.client.0.smithi049.stdout:5/181: symlink dc/d29/d2c/d31/l37 0 2024-05-15T15:53:59.759 INFO:tasks.workunit.client.0.smithi049.stdout:4/252: dwrite d18/d21/d2b/f2c [0,4194304] 0 2024-05-15T15:53:59.762 INFO:tasks.workunit.client.0.smithi049.stdout:4/253: write d18/f31 [5074268,75523] 0 2024-05-15T15:53:59.764 INFO:tasks.workunit.client.0.smithi049.stdout:3/178: dwrite d12/d14/f17 [0,4194304] 0 2024-05-15T15:53:59.767 INFO:tasks.workunit.client.0.smithi049.stdout:8/111: write d2/f12 [912035,86768] 0 2024-05-15T15:53:59.772 INFO:tasks.workunit.client.0.smithi049.stdout:2/191: dread d3/f11 [0,4194304] 0 2024-05-15T15:53:59.777 INFO:tasks.workunit.client.0.smithi049.stdout:0/173: write dc/d13/d14/f2e [647328,103889] 0 2024-05-15T15:53:59.779 INFO:tasks.workunit.client.0.smithi049.stdout:0/174: truncate dc/f34 485348 0 2024-05-15T15:53:59.781 INFO:tasks.workunit.client.0.smithi049.stdout:3/179: mkdir d12/d14/d32 0 2024-05-15T15:53:59.784 INFO:tasks.workunit.client.0.smithi049.stdout:7/170: mkdir d0/d34 0 2024-05-15T15:53:59.786 INFO:tasks.workunit.client.0.smithi049.stdout:6/115: mknod dd/d17/c1c 0 2024-05-15T15:53:59.790 INFO:tasks.workunit.client.0.smithi049.stdout:3/180: dwrite d12/f25 [0,4194304] 0 2024-05-15T15:53:59.791 INFO:tasks.workunit.client.0.smithi049.stdout:2/192: mknod d3/da/c35 0 2024-05-15T15:53:59.793 INFO:tasks.workunit.client.0.smithi049.stdout:4/254: creat d18/d1e/d2a/d3b/f51 x:0 0 0 2024-05-15T15:53:59.795 INFO:tasks.workunit.client.0.smithi049.stdout:1/169: creat d0/d4/dd/de/d36/f3f x:0 0 0 2024-05-15T15:53:59.797 INFO:tasks.workunit.client.0.smithi049.stdout:4/255: write d18/d21/f45 [419492,58924] 0 2024-05-15T15:53:59.799 INFO:tasks.workunit.client.0.smithi049.stdout:7/171: dwrite d0/d6/d1f/f33 [0,4194304] 0 2024-05-15T15:53:59.801 INFO:tasks.workunit.client.0.smithi049.stdout:0/175: readlink dc/l16 0 2024-05-15T15:53:59.802 INFO:tasks.workunit.client.0.smithi049.stdout:6/116: dread dd/f14 [0,4194304] 0 2024-05-15T15:53:59.803 INFO:tasks.workunit.client.0.smithi049.stdout:0/176: write dc/f1b [214238,25913] 0 2024-05-15T15:53:59.804 INFO:tasks.workunit.client.0.smithi049.stdout:8/112: mknod d2/d14/d1b/c21 0 2024-05-15T15:53:59.809 INFO:tasks.workunit.client.0.smithi049.stdout:6/117: dread f4 [0,4194304] 0 2024-05-15T15:53:59.810 INFO:tasks.workunit.client.0.smithi049.stdout:3/181: unlink fb 0 2024-05-15T15:53:59.814 INFO:tasks.workunit.client.0.smithi049.stdout:1/170: dwrite d0/d4/dd/de/f13 [0,4194304] 0 2024-05-15T15:53:59.816 INFO:tasks.workunit.client.0.smithi049.stdout:1/171: chown d0/d4/dd/de/d1e 0 1 2024-05-15T15:53:59.822 INFO:tasks.workunit.client.0.smithi049.stdout:3/182: dwrite d12/d14/f1e [0,4194304] 0 2024-05-15T15:53:59.826 INFO:tasks.workunit.client.0.smithi049.stdout:1/172: dwrite d0/d4/f1b [4194304,4194304] 0 2024-05-15T15:53:59.827 INFO:tasks.workunit.client.0.smithi049.stdout:1/173: readlink d0/d4/l14 0 2024-05-15T15:53:59.830 INFO:tasks.workunit.client.0.smithi049.stdout:0/177: fsync fa 0 2024-05-15T15:53:59.836 INFO:tasks.workunit.client.0.smithi049.stdout:6/118: creat dd/d17/f1d x:0 0 0 2024-05-15T15:53:59.846 INFO:tasks.workunit.client.0.smithi049.stdout:3/183: unlink d12/f15 0 2024-05-15T15:53:59.848 INFO:tasks.workunit.client.0.smithi049.stdout:7/172: link d0/d6/d1f/f2c d0/d34/f35 0 2024-05-15T15:53:59.863 INFO:tasks.workunit.client.0.smithi049.stdout:6/119: fdatasync f4 0 2024-05-15T15:53:59.864 INFO:tasks.workunit.client.0.smithi049.stdout:6/120: dread - dd/d17/f1d zero size 2024-05-15T15:53:59.870 INFO:tasks.workunit.client.0.smithi049.stdout:9/128: sync 2024-05-15T15:53:59.870 INFO:tasks.workunit.client.0.smithi049.stdout:7/173: unlink d0/d2f/c32 0 2024-05-15T15:53:59.874 INFO:tasks.workunit.client.0.smithi049.stdout:1/174: link d0/d4/f2d d0/d4/dd/de/d36/f40 0 2024-05-15T15:53:59.875 INFO:tasks.workunit.client.0.smithi049.stdout:1/175: stat d0/d4/dd/f28 0 2024-05-15T15:53:59.876 INFO:tasks.workunit.client.0.smithi049.stdout:1/176: chown d0/d4/dd/de/d1e/f27 2543 1 2024-05-15T15:53:59.876 INFO:tasks.workunit.client.0.smithi049.stdout:0/178: mkdir dc/d13/d2b/d36 0 2024-05-15T15:53:59.877 INFO:tasks.workunit.client.0.smithi049.stdout:0/179: rename dc to dc/d13/d14/d25/d31/d37 22 2024-05-15T15:53:59.878 INFO:tasks.workunit.client.1.smithi144.stdout:4/88: rmdir d6/dd/d15/d19 0 2024-05-15T15:53:59.880 INFO:tasks.workunit.client.0.smithi049.stdout:6/121: write dd/f14 [2850834,108909] 0 2024-05-15T15:53:59.881 INFO:tasks.workunit.client.1.smithi144.stdout:8/80: stat d2/de/d12/c19 0 2024-05-15T15:53:59.884 INFO:tasks.workunit.client.0.smithi049.stdout:6/122: dread dd/ff [0,4194304] 0 2024-05-15T15:53:59.885 INFO:tasks.workunit.client.1.smithi144.stdout:6/62: creat d7/db/f18 x:0 0 0 2024-05-15T15:53:59.885 INFO:tasks.workunit.client.1.smithi144.stdout:6/63: write d7/f8 [4420099,65994] 0 2024-05-15T15:53:59.888 INFO:tasks.workunit.client.1.smithi144.stdout:5/83: mkdir d0/dc/d1f 0 2024-05-15T15:53:59.889 INFO:tasks.workunit.client.1.smithi144.stdout:6/64: dread d7/f8 [0,4194304] 0 2024-05-15T15:53:59.889 INFO:tasks.workunit.client.0.smithi049.stdout:6/123: dread dd/ff [0,4194304] 0 2024-05-15T15:53:59.890 INFO:tasks.workunit.client.0.smithi049.stdout:4/256: truncate d18/d21/d2b/f2c 3145915 0 2024-05-15T15:53:59.891 INFO:tasks.workunit.client.0.smithi049.stdout:1/177: creat d0/d4/dd/de/d1e/d2e/f41 x:0 0 0 2024-05-15T15:53:59.892 INFO:tasks.workunit.client.1.smithi144.stdout:2/55: unlink l1 0 2024-05-15T15:53:59.893 INFO:tasks.workunit.client.1.smithi144.stdout:2/56: fsync d3/f5 0 2024-05-15T15:53:59.893 INFO:tasks.workunit.client.0.smithi049.stdout:0/180: read f3 [544304,92613] 0 2024-05-15T15:53:59.894 INFO:tasks.workunit.client.1.smithi144.stdout:1/82: symlink d7/l17 0 2024-05-15T15:53:59.895 INFO:tasks.workunit.client.1.smithi144.stdout:1/83: truncate d7/df/fb 790279 0 2024-05-15T15:53:59.895 INFO:tasks.workunit.client.0.smithi049.stdout:8/113: rename d2/f10 to d2/f22 0 2024-05-15T15:53:59.896 INFO:tasks.workunit.client.0.smithi049.stdout:3/184: link c11 d12/d14/d32/c33 0 2024-05-15T15:53:59.899 INFO:tasks.workunit.client.1.smithi144.stdout:3/57: creat d0/f15 x:0 0 0 2024-05-15T15:53:59.902 INFO:tasks.workunit.client.1.smithi144.stdout:9/57: getdents d0 0 2024-05-15T15:53:59.905 INFO:tasks.workunit.client.1.smithi144.stdout:4/89: rename d6/da/l12 to d6/dd/d15/l1a 0 2024-05-15T15:53:59.906 INFO:tasks.workunit.client.0.smithi049.stdout:1/178: creat d0/f42 x:0 0 0 2024-05-15T15:53:59.907 INFO:tasks.workunit.client.1.smithi144.stdout:4/90: write d6/da/ff [7155480,28647] 0 2024-05-15T15:53:59.907 INFO:tasks.workunit.client.1.smithi144.stdout:3/58: dwrite d0/d11/f14 [0,4194304] 0 2024-05-15T15:53:59.909 INFO:tasks.workunit.client.0.smithi049.stdout:0/181: fdatasync dc/d13/d14/f19 0 2024-05-15T15:53:59.910 INFO:tasks.workunit.client.0.smithi049.stdout:0/182: fsync dc/d13/d14/d2a/f2c 0 2024-05-15T15:53:59.917 INFO:tasks.workunit.client.1.smithi144.stdout:0/60: getdents d4/df/d14 0 2024-05-15T15:53:59.918 INFO:tasks.workunit.client.1.smithi144.stdout:0/61: truncate d4/d6/f8 631042 0 2024-05-15T15:53:59.918 INFO:tasks.workunit.client.0.smithi049.stdout:8/114: dread d2/de/f16 [0,4194304] 0 2024-05-15T15:53:59.918 INFO:tasks.workunit.client.1.smithi144.stdout:0/62: write d4/f9 [42132,89268] 0 2024-05-15T15:53:59.919 INFO:tasks.workunit.client.0.smithi049.stdout:7/174: dwrite d0/d6/f1b [0,4194304] 0 2024-05-15T15:53:59.926 INFO:tasks.workunit.client.1.smithi144.stdout:2/57: rmdir d3/d7 39 2024-05-15T15:53:59.929 INFO:tasks.workunit.client.0.smithi049.stdout:1/179: creat d0/d4/dd/de/d36/f43 x:0 0 0 2024-05-15T15:53:59.929 INFO:tasks.workunit.client.1.smithi144.stdout:1/84: rmdir d7 39 2024-05-15T15:53:59.929 INFO:tasks.workunit.client.0.smithi049.stdout:7/175: dwrite d0/d6/f1b [4194304,4194304] 0 2024-05-15T15:53:59.930 INFO:tasks.workunit.client.0.smithi049.stdout:6/124: write dd/f15 [1233896,92200] 0 2024-05-15T15:53:59.930 INFO:tasks.workunit.client.0.smithi049.stdout:1/180: stat d0/d4/f1b 0 2024-05-15T15:53:59.931 INFO:tasks.workunit.client.0.smithi049.stdout:4/257: write f5 [941411,121655] 0 2024-05-15T15:53:59.938 INFO:tasks.workunit.client.0.smithi049.stdout:7/176: dread d0/d6/da/f1c [0,4194304] 0 2024-05-15T15:53:59.938 INFO:tasks.workunit.client.0.smithi049.stdout:0/183: chown dc/d13/f26 8921 1 2024-05-15T15:53:59.938 INFO:tasks.workunit.client.0.smithi049.stdout:7/177: chown d0/fb 7851 1 2024-05-15T15:53:59.943 INFO:tasks.workunit.client.1.smithi144.stdout:9/58: mkdir d0/da/d10/d1a 0 2024-05-15T15:53:59.943 INFO:tasks.workunit.client.1.smithi144.stdout:9/59: chown d0/l15 440024 1 2024-05-15T15:53:59.943 INFO:tasks.workunit.client.0.smithi049.stdout:3/185: write f5 [3141580,61134] 0 2024-05-15T15:53:59.945 INFO:tasks.workunit.client.0.smithi049.stdout:9/129: getdents d0/d8/d1b 0 2024-05-15T15:53:59.948 INFO:tasks.workunit.client.1.smithi144.stdout:4/91: creat d6/da/f1b x:0 0 0 2024-05-15T15:53:59.952 INFO:tasks.workunit.client.1.smithi144.stdout:3/59: rmdir d0 39 2024-05-15T15:53:59.954 INFO:tasks.workunit.client.1.smithi144.stdout:7/97: truncate d4/fa 1265579 0 2024-05-15T15:53:59.958 INFO:tasks.workunit.client.0.smithi049.stdout:1/181: creat d0/f44 x:0 0 0 2024-05-15T15:53:59.959 INFO:tasks.workunit.client.1.smithi144.stdout:5/84: symlink d0/d1c/l20 0 2024-05-15T15:53:59.962 INFO:tasks.workunit.client.1.smithi144.stdout:5/85: dread d0/dc/f1b [0,4194304] 0 2024-05-15T15:53:59.962 INFO:tasks.workunit.client.1.smithi144.stdout:0/63: creat d4/dc/f18 x:0 0 0 2024-05-15T15:53:59.962 INFO:tasks.workunit.client.1.smithi144.stdout:0/64: chown d4/ca 314426768 1 2024-05-15T15:53:59.962 INFO:tasks.workunit.client.1.smithi144.stdout:5/86: read d0/dc/f1b [2399809,101297] 0 2024-05-15T15:53:59.965 INFO:tasks.workunit.client.1.smithi144.stdout:6/65: mknod d7/db/c19 0 2024-05-15T15:53:59.966 INFO:tasks.workunit.client.1.smithi144.stdout:5/87: dread d0/f5 [0,4194304] 0 2024-05-15T15:53:59.968 INFO:tasks.workunit.client.0.smithi049.stdout:5/182: sync 2024-05-15T15:53:59.968 INFO:tasks.workunit.client.0.smithi049.stdout:2/193: sync 2024-05-15T15:53:59.970 INFO:tasks.workunit.client.0.smithi049.stdout:2/194: truncate d3/da/f2c 4385065 0 2024-05-15T15:53:59.971 INFO:tasks.workunit.client.1.smithi144.stdout:1/85: readlink d7/l11 0 2024-05-15T15:53:59.971 INFO:tasks.workunit.client.1.smithi144.stdout:1/86: chown d7/d9/fe 0 1 2024-05-15T15:53:59.974 INFO:tasks.workunit.client.1.smithi144.stdout:9/60: rename d0/da/d10/d1a to d0/d8/d1b 0 2024-05-15T15:53:59.977 INFO:tasks.workunit.client.0.smithi049.stdout:9/130: unlink d0/d8/f24 0 2024-05-15T15:53:59.977 INFO:tasks.workunit.client.1.smithi144.stdout:9/61: dread d0/f4 [0,4194304] 0 2024-05-15T15:53:59.978 INFO:tasks.workunit.client.0.smithi049.stdout:8/115: mknod d2/d18/c23 0 2024-05-15T15:53:59.984 INFO:tasks.workunit.client.1.smithi144.stdout:0/65: symlink d4/dc/l19 0 2024-05-15T15:53:59.984 INFO:tasks.workunit.client.0.smithi049.stdout:0/184: getdents dc/d13/d14/d33 0 2024-05-15T15:53:59.984 INFO:tasks.workunit.client.1.smithi144.stdout:0/66: truncate d4/dc/f17 634228 0 2024-05-15T15:53:59.985 INFO:tasks.workunit.client.1.smithi144.stdout:0/67: fdatasync d4/dc/f17 0 2024-05-15T15:53:59.987 INFO:tasks.workunit.client.0.smithi049.stdout:2/195: creat d3/d8/f36 x:0 0 0 2024-05-15T15:53:59.993 INFO:tasks.workunit.client.1.smithi144.stdout:5/88: symlink d0/l21 0 2024-05-15T15:53:59.994 INFO:tasks.workunit.client.1.smithi144.stdout:5/89: dread d0/dc/d12/f13 [0,4194304] 0 2024-05-15T15:53:59.996 INFO:tasks.workunit.client.1.smithi144.stdout:2/58: creat d3/d10/f11 x:0 0 0 2024-05-15T15:53:59.997 INFO:tasks.workunit.client.0.smithi049.stdout:3/186: truncate d12/d14/f17 1993904 0 2024-05-15T15:53:59.997 INFO:tasks.workunit.client.0.smithi049.stdout:3/187: chown d12/d14/l1f 239625442 1 2024-05-15T15:53:59.998 INFO:tasks.workunit.client.0.smithi049.stdout:3/188: chown d12/d26/f2f 1 1 2024-05-15T15:54:00.008 INFO:tasks.workunit.client.0.smithi049.stdout:8/116: dwrite d2/fc [4194304,4194304] 0 2024-05-15T15:54:00.008 INFO:tasks.workunit.client.0.smithi049.stdout:4/258: rename d18/d1e/d2a/d36/c3a to d18/d21/c52 0 2024-05-15T15:54:00.014 INFO:tasks.workunit.client.0.smithi049.stdout:9/131: dwrite d0/d8/fd [0,4194304] 0 2024-05-15T15:54:00.017 INFO:tasks.workunit.client.1.smithi144.stdout:7/98: link d4/fc d4/df/f1f 0 2024-05-15T15:54:00.018 INFO:tasks.workunit.client.0.smithi049.stdout:6/125: getdents dd 0 2024-05-15T15:54:00.019 INFO:tasks.workunit.client.0.smithi049.stdout:8/117: truncate f0 4779483 0 2024-05-15T15:54:00.022 INFO:tasks.workunit.client.0.smithi049.stdout:4/259: creat d18/f53 x:0 0 0 2024-05-15T15:54:00.023 INFO:tasks.workunit.client.1.smithi144.stdout:6/66: symlink d7/db/l1a 0 2024-05-15T15:54:00.023 INFO:tasks.workunit.client.0.smithi049.stdout:4/260: read - d18/f1c zero size 2024-05-15T15:54:00.024 INFO:tasks.workunit.client.0.smithi049.stdout:0/185: mkdir dc/d13/d14/d25/d31/d38 0 2024-05-15T15:54:00.025 INFO:tasks.workunit.client.0.smithi049.stdout:4/261: write d18/f2e [792944,128688] 0 2024-05-15T15:54:00.025 INFO:tasks.workunit.client.1.smithi144.stdout:5/90: creat d0/f22 x:0 0 0 2024-05-15T15:54:00.026 INFO:tasks.workunit.client.1.smithi144.stdout:6/67: dread d7/f8 [0,4194304] 0 2024-05-15T15:54:00.026 INFO:tasks.workunit.client.0.smithi049.stdout:6/126: dwrite dd/f19 [0,4194304] 0 2024-05-15T15:54:00.028 INFO:tasks.workunit.client.0.smithi049.stdout:7/178: rename d0/d6/d1f to d0/d6/d1d/d36 0 2024-05-15T15:54:00.029 INFO:tasks.workunit.client.0.smithi049.stdout:7/179: write d0/fb [1050568,20477] 0 2024-05-15T15:54:00.032 INFO:tasks.workunit.client.0.smithi049.stdout:3/189: creat d12/d14/d32/f34 x:0 0 0 2024-05-15T15:54:00.034 INFO:tasks.workunit.client.0.smithi049.stdout:9/132: creat d0/d8/d21/f30 x:0 0 0 2024-05-15T15:54:00.036 INFO:tasks.workunit.client.0.smithi049.stdout:8/118: mknod d2/d14/d1b/c24 0 2024-05-15T15:54:00.037 INFO:tasks.workunit.client.1.smithi144.stdout:9/62: mknod d0/da/d10/d19/c1c 0 2024-05-15T15:54:00.040 INFO:tasks.workunit.client.0.smithi049.stdout:6/127: symlink dd/d17/l1e 0 2024-05-15T15:54:00.041 INFO:tasks.workunit.client.1.smithi144.stdout:6/68: fsync d7/f8 0 2024-05-15T15:54:00.041 INFO:tasks.workunit.client.0.smithi049.stdout:6/128: truncate dd/f15 1914151 0 2024-05-15T15:54:00.042 INFO:tasks.workunit.client.1.smithi144.stdout:5/91: truncate d0/f7 1994411 0 2024-05-15T15:54:00.043 INFO:tasks.workunit.client.0.smithi049.stdout:6/129: truncate dd/d17/f1d 892713 0 2024-05-15T15:54:00.043 INFO:tasks.workunit.client.0.smithi049.stdout:2/196: rename d3/da/c35 to d3/da/d15/d24/c37 0 2024-05-15T15:54:00.044 INFO:tasks.workunit.client.1.smithi144.stdout:6/69: dread f6 [0,4194304] 0 2024-05-15T15:54:00.045 INFO:tasks.workunit.client.1.smithi144.stdout:6/70: write d7/db/d11/f13 [939689,38870] 0 2024-05-15T15:54:00.045 INFO:tasks.workunit.client.1.smithi144.stdout:2/59: rename d3/c4 to d3/d7/c12 0 2024-05-15T15:54:00.045 INFO:tasks.workunit.client.0.smithi049.stdout:7/180: mknod d0/d6/d23/c37 0 2024-05-15T15:54:00.046 INFO:tasks.workunit.client.1.smithi144.stdout:1/87: link d7/l17 d7/d9/l18 0 2024-05-15T15:54:00.047 INFO:tasks.workunit.client.0.smithi049.stdout:3/190: symlink d12/d14/d32/l35 0 2024-05-15T15:54:00.048 INFO:tasks.workunit.client.1.smithi144.stdout:4/92: getdents d6/dd/d15 0 2024-05-15T15:54:00.048 INFO:tasks.workunit.client.0.smithi049.stdout:2/197: dread d3/da/d15/d24/f2f [0,4194304] 0 2024-05-15T15:54:00.049 INFO:tasks.workunit.client.0.smithi049.stdout:3/191: chown l6 36 1 2024-05-15T15:54:00.051 INFO:tasks.workunit.client.1.smithi144.stdout:3/60: link d0/d1/cc d0/d11/c16 0 2024-05-15T15:54:00.052 INFO:tasks.workunit.client.0.smithi049.stdout:7/181: dread d0/d6/d1d/d36/f33 [0,4194304] 0 2024-05-15T15:54:00.052 INFO:tasks.workunit.client.0.smithi049.stdout:7/182: stat d0/d6/d23/l31 0 2024-05-15T15:54:00.053 INFO:tasks.workunit.client.1.smithi144.stdout:3/61: write d0/d1/f5 [4084871,64981] 0 2024-05-15T15:54:00.054 INFO:tasks.workunit.client.0.smithi049.stdout:6/130: unlink c9 0 2024-05-15T15:54:00.057 INFO:tasks.workunit.client.1.smithi144.stdout:3/62: write d0/d1/f5 [3207257,49129] 0 2024-05-15T15:54:00.058 INFO:tasks.workunit.client.1.smithi144.stdout:9/63: dwrite d0/da/d10/fd [0,4194304] 0 2024-05-15T15:54:00.060 INFO:tasks.workunit.client.1.smithi144.stdout:2/60: symlink d3/d10/l13 0 2024-05-15T15:54:00.061 INFO:tasks.workunit.client.0.smithi049.stdout:6/131: dwrite dd/ff [0,4194304] 0 2024-05-15T15:54:00.061 INFO:tasks.workunit.client.0.smithi049.stdout:9/133: creat d0/d25/f31 x:0 0 0 2024-05-15T15:54:00.063 INFO:tasks.workunit.client.1.smithi144.stdout:6/71: creat d7/db/d11/f1b x:0 0 0 2024-05-15T15:54:00.064 INFO:tasks.workunit.client.0.smithi049.stdout:2/198: write d3/d8/f14 [4416513,19664] 0 2024-05-15T15:54:00.065 INFO:tasks.workunit.client.1.smithi144.stdout:1/88: rmdir d7/df 39 2024-05-15T15:54:00.066 INFO:tasks.workunit.client.1.smithi144.stdout:3/63: dwrite d0/d11/f14 [0,4194304] 0 2024-05-15T15:54:00.066 INFO:tasks.workunit.client.0.smithi049.stdout:2/199: dread - d3/db/d1d/f20 zero size 2024-05-15T15:54:00.067 INFO:tasks.workunit.client.0.smithi049.stdout:3/192: truncate ff 443245 0 2024-05-15T15:54:00.068 INFO:tasks.workunit.client.1.smithi144.stdout:3/64: read d0/f6 [801719,107593] 0 2024-05-15T15:54:00.068 INFO:tasks.workunit.client.0.smithi049.stdout:2/200: write d3/db/f1a [1254583,16648] 0 2024-05-15T15:54:00.068 INFO:tasks.workunit.client.0.smithi049.stdout:3/193: dread - d12/f18 zero size 2024-05-15T15:54:00.069 INFO:tasks.workunit.client.0.smithi049.stdout:6/132: read fb [1041259,92264] 0 2024-05-15T15:54:00.070 INFO:tasks.workunit.client.1.smithi144.stdout:4/93: creat d6/dd/d15/f1c x:0 0 0 2024-05-15T15:54:00.071 INFO:tasks.workunit.client.1.smithi144.stdout:4/94: chown d6/da/ff 1775 1 2024-05-15T15:54:00.074 INFO:tasks.workunit.client.0.smithi049.stdout:8/119: stat f0 0 2024-05-15T15:54:00.076 INFO:tasks.workunit.client.1.smithi144.stdout:6/72: dread d7/db/d11/f13 [0,4194304] 0 2024-05-15T15:54:00.077 INFO:tasks.workunit.client.1.smithi144.stdout:4/95: dread d6/da/ff [0,4194304] 0 2024-05-15T15:54:00.077 INFO:tasks.workunit.client.0.smithi049.stdout:4/262: dwrite d18/f28 [0,4194304] 0 2024-05-15T15:54:00.084 INFO:tasks.workunit.client.1.smithi144.stdout:8/81: sync 2024-05-15T15:54:00.087 INFO:tasks.workunit.client.0.smithi049.stdout:9/134: rename d0/f19 to d0/d8/d1b/f32 0 2024-05-15T15:54:00.093 INFO:tasks.workunit.client.0.smithi049.stdout:8/120: mkdir d2/de/d25 0 2024-05-15T15:54:00.094 INFO:tasks.workunit.client.0.smithi049.stdout:8/121: read d2/de/f16 [4027318,30005] 0 2024-05-15T15:54:00.097 INFO:tasks.workunit.client.1.smithi144.stdout:5/92: mkdir d0/dc/d19/d23 0 2024-05-15T15:54:00.097 INFO:tasks.workunit.client.1.smithi144.stdout:2/61: symlink d3/d10/l14 0 2024-05-15T15:54:00.099 INFO:tasks.workunit.client.1.smithi144.stdout:1/89: rename d7/d9/fd to d7/d9/f19 0 2024-05-15T15:54:00.099 INFO:tasks.workunit.client.1.smithi144.stdout:3/65: mkdir d0/d1/d17 0 2024-05-15T15:54:00.100 INFO:tasks.workunit.client.0.smithi049.stdout:7/183: write d0/d6/f18 [2967452,23330] 0 2024-05-15T15:54:00.101 INFO:tasks.workunit.client.1.smithi144.stdout:7/99: dwrite f2 [0,4194304] 0 2024-05-15T15:54:00.101 INFO:tasks.workunit.client.0.smithi049.stdout:8/122: dwrite d2/d14/f17 [0,4194304] 0 2024-05-15T15:54:00.102 INFO:tasks.workunit.client.1.smithi144.stdout:3/66: truncate d0/d11/f14 4821025 0 2024-05-15T15:54:00.103 INFO:tasks.workunit.client.1.smithi144.stdout:9/64: dread d0/f4 [0,4194304] 0 2024-05-15T15:54:00.103 INFO:tasks.workunit.client.1.smithi144.stdout:3/67: readlink d0/l7 0 2024-05-15T15:54:00.104 INFO:tasks.workunit.client.1.smithi144.stdout:3/68: read - d0/f15 zero size 2024-05-15T15:54:00.110 INFO:tasks.workunit.client.0.smithi049.stdout:3/194: truncate d12/d14/f2a 2867494 0 2024-05-15T15:54:00.112 INFO:tasks.workunit.client.1.smithi144.stdout:7/100: dwrite d4/df/d10/f1d [0,4194304] 0 2024-05-15T15:54:00.114 INFO:tasks.workunit.client.0.smithi049.stdout:4/263: mkdir d18/d1e/d2a/d50/d54 0 2024-05-15T15:54:00.115 INFO:tasks.workunit.client.0.smithi049.stdout:4/264: write d18/d21/f45 [27484,131026] 0 2024-05-15T15:54:00.118 INFO:tasks.workunit.client.0.smithi049.stdout:6/133: getdents dd 0 2024-05-15T15:54:00.120 INFO:tasks.workunit.client.0.smithi049.stdout:9/135: dread - d0/d8/d21/f30 zero size 2024-05-15T15:54:00.121 INFO:tasks.workunit.client.0.smithi049.stdout:9/136: stat d0/d8/d21/f2f 0 2024-05-15T15:54:00.125 INFO:tasks.workunit.client.0.smithi049.stdout:3/195: dread f9 [0,4194304] 0 2024-05-15T15:54:00.126 INFO:tasks.workunit.client.0.smithi049.stdout:4/265: mkdir d18/d1e/d2a/d36/d55 0 2024-05-15T15:54:00.127 INFO:tasks.workunit.client.0.smithi049.stdout:4/266: write d18/d1e/d2a/f38 [2798082,67863] 0 2024-05-15T15:54:00.132 INFO:tasks.workunit.client.0.smithi049.stdout:3/196: dwrite d12/d14/f1b [0,4194304] 0 2024-05-15T15:54:00.141 INFO:tasks.workunit.client.0.smithi049.stdout:3/197: chown d12/d14/d32/c33 0 1 2024-05-15T15:54:00.143 INFO:tasks.workunit.client.0.smithi049.stdout:3/198: write d12/d14/f1a [5187718,49300] 0 2024-05-15T15:54:00.149 INFO:tasks.workunit.client.0.smithi049.stdout:3/199: dwrite f5 [4194304,4194304] 0 2024-05-15T15:54:00.150 INFO:tasks.workunit.client.0.smithi049.stdout:8/123: truncate d2/f3 6284022 0 2024-05-15T15:54:00.153 INFO:tasks.workunit.client.0.smithi049.stdout:4/267: creat d18/d21/f56 x:0 0 0 2024-05-15T15:54:00.155 INFO:tasks.workunit.client.0.smithi049.stdout:6/134: getdents dd/d17 0 2024-05-15T15:54:00.157 INFO:tasks.workunit.client.0.smithi049.stdout:8/124: dwrite d2/fc [4194304,4194304] 0 2024-05-15T15:54:00.157 INFO:tasks.workunit.client.0.smithi049.stdout:3/200: rmdir d12/d14/d32 39 2024-05-15T15:54:00.160 INFO:tasks.workunit.client.0.smithi049.stdout:9/137: truncate d0/d8/d9/fb 4166643 0 2024-05-15T15:54:00.162 INFO:tasks.workunit.client.0.smithi049.stdout:4/268: rmdir d18/d1e/d2a 39 2024-05-15T15:54:00.163 INFO:tasks.workunit.client.0.smithi049.stdout:9/138: dread d0/d8/d1b/f32 [0,4194304] 0 2024-05-15T15:54:00.164 INFO:tasks.workunit.client.0.smithi049.stdout:9/139: dread - d0/d8/d9/d22/f2e zero size 2024-05-15T15:54:00.166 INFO:tasks.workunit.client.0.smithi049.stdout:8/125: fsync d2/fb 0 2024-05-15T15:54:00.171 INFO:tasks.workunit.client.0.smithi049.stdout:6/135: getdents dd/d17 0 2024-05-15T15:54:00.173 INFO:tasks.workunit.client.0.smithi049.stdout:9/140: creat d0/d8/d12/f33 x:0 0 0 2024-05-15T15:54:00.177 INFO:tasks.workunit.client.0.smithi049.stdout:9/141: dread d0/d8/fd [0,4194304] 0 2024-05-15T15:54:00.179 INFO:tasks.workunit.client.0.smithi049.stdout:3/201: rename d12/d26/f2f to d12/d14/d32/f36 0 2024-05-15T15:54:00.181 INFO:tasks.workunit.client.0.smithi049.stdout:3/202: read d12/f1d [4459735,22129] 0 2024-05-15T15:54:00.181 INFO:tasks.workunit.client.0.smithi049.stdout:4/269: creat d18/d1e/d2a/d3b/f57 x:0 0 0 2024-05-15T15:54:00.182 INFO:tasks.workunit.client.0.smithi049.stdout:6/136: symlink dd/l1f 0 2024-05-15T15:54:00.184 INFO:tasks.workunit.client.0.smithi049.stdout:8/126: unlink d2/c6 0 2024-05-15T15:54:00.185 INFO:tasks.workunit.client.0.smithi049.stdout:8/127: readlink d2/d18/l20 0 2024-05-15T15:54:00.191 INFO:tasks.workunit.client.0.smithi049.stdout:8/128: dwrite d2/fc [4194304,4194304] 0 2024-05-15T15:54:00.192 INFO:tasks.workunit.client.0.smithi049.stdout:3/203: fsync f9 0 2024-05-15T15:54:00.194 INFO:tasks.workunit.client.0.smithi049.stdout:4/270: write d18/d1e/d2a/d3b/f44 [3319495,99870] 0 2024-05-15T15:54:00.197 INFO:tasks.workunit.client.0.smithi049.stdout:6/137: creat dd/d17/f20 x:0 0 0 2024-05-15T15:54:00.201 INFO:tasks.workunit.client.1.smithi144.stdout:0/68: sync 2024-05-15T15:54:00.203 INFO:tasks.workunit.client.0.smithi049.stdout:3/204: rmdir d12/d14 39 2024-05-15T15:54:00.203 INFO:tasks.workunit.client.0.smithi049.stdout:3/205: stat fc 0 2024-05-15T15:54:00.206 INFO:tasks.workunit.client.0.smithi049.stdout:4/271: creat d18/d1e/d2a/d36/f58 x:0 0 0 2024-05-15T15:54:00.209 INFO:tasks.workunit.client.0.smithi049.stdout:6/138: creat dd/d17/f21 x:0 0 0 2024-05-15T15:54:00.210 INFO:tasks.workunit.client.0.smithi049.stdout:6/139: chown dd/d17 38293 1 2024-05-15T15:54:00.212 INFO:tasks.workunit.client.0.smithi049.stdout:3/206: mknod d12/d26/c37 0 2024-05-15T15:54:00.214 INFO:tasks.workunit.client.0.smithi049.stdout:8/129: creat d2/de/d25/f26 x:0 0 0 2024-05-15T15:54:00.214 INFO:tasks.workunit.client.0.smithi049.stdout:8/130: write d2/f12 [768554,97174] 0 2024-05-15T15:54:00.222 INFO:tasks.workunit.client.1.smithi144.stdout:4/96: sync 2024-05-15T15:54:00.222 INFO:tasks.workunit.client.0.smithi049.stdout:9/142: write d0/f1d [1377597,130686] 0 2024-05-15T15:54:00.229 INFO:tasks.workunit.client.0.smithi049.stdout:9/143: creat d0/d8/d12/f34 x:0 0 0 2024-05-15T15:54:00.239 INFO:tasks.workunit.client.0.smithi049.stdout:9/144: mknod d0/d8/d12/c35 0 2024-05-15T15:54:00.242 INFO:tasks.workunit.client.0.smithi049.stdout:4/272: getdents d18/d1e 0 2024-05-15T15:54:00.243 INFO:tasks.workunit.client.0.smithi049.stdout:4/273: write f5 [464403,108519] 0 2024-05-15T15:54:00.244 INFO:tasks.workunit.client.0.smithi049.stdout:4/274: readlink d18/d21/d2b/l33 0 2024-05-15T15:54:00.245 INFO:tasks.workunit.client.0.smithi049.stdout:6/140: write f8 [1218527,129767] 0 2024-05-15T15:54:00.249 INFO:tasks.workunit.client.0.smithi049.stdout:9/145: rename d0/f2 to d0/d8/d1b/f36 0 2024-05-15T15:54:00.251 INFO:tasks.workunit.client.0.smithi049.stdout:8/131: write f0 [3823570,94698] 0 2024-05-15T15:54:00.258 INFO:tasks.workunit.client.0.smithi049.stdout:8/132: rmdir d2/de/d25 39 2024-05-15T15:54:00.324 INFO:tasks.workunit.client.0.smithi049.stdout:1/182: sync 2024-05-15T15:54:00.325 INFO:tasks.workunit.client.0.smithi049.stdout:3/207: dwrite d12/f22 [0,4194304] 0 2024-05-15T15:54:00.329 INFO:tasks.workunit.client.0.smithi049.stdout:9/146: write d0/d8/d9/f15 [1783675,12343] 0 2024-05-15T15:54:00.330 INFO:tasks.workunit.client.0.smithi049.stdout:1/183: mkdir d0/d4/dd/de/d1e/d2e/d45 0 2024-05-15T15:54:00.334 INFO:tasks.workunit.client.0.smithi049.stdout:9/147: write d0/d8/d11/f1f [1036521,101616] 0 2024-05-15T15:54:00.334 INFO:tasks.workunit.client.0.smithi049.stdout:6/141: dwrite fb [0,4194304] 0 2024-05-15T15:54:00.335 INFO:tasks.workunit.client.0.smithi049.stdout:6/142: fdatasync dd/ff 0 2024-05-15T15:54:00.335 INFO:tasks.workunit.client.0.smithi049.stdout:8/133: mkdir d2/de/d27 0 2024-05-15T15:54:00.336 INFO:tasks.workunit.client.0.smithi049.stdout:9/148: write d0/d8/d9/f15 [2820146,77839] 0 2024-05-15T15:54:00.336 INFO:tasks.workunit.client.0.smithi049.stdout:6/143: read fb [2843402,123368] 0 2024-05-15T15:54:00.337 INFO:tasks.workunit.client.0.smithi049.stdout:8/134: chown d2/d18/c1f 601815722 1 2024-05-15T15:54:00.338 INFO:tasks.workunit.client.1.smithi144.stdout:0/69: sync 2024-05-15T15:54:00.338 INFO:tasks.workunit.client.0.smithi049.stdout:6/144: write f2 [3138373,46393] 0 2024-05-15T15:54:00.339 INFO:tasks.workunit.client.0.smithi049.stdout:3/208: creat d12/d30/f38 x:0 0 0 2024-05-15T15:54:00.339 INFO:tasks.workunit.client.0.smithi049.stdout:8/135: write d2/d14/f1e [510362,22411] 0 2024-05-15T15:54:00.340 INFO:tasks.workunit.client.0.smithi049.stdout:1/184: rmdir d0/d4/dd/de/d1e 39 2024-05-15T15:54:00.341 INFO:tasks.workunit.client.0.smithi049.stdout:8/136: chown d2/d18/c23 9 1 2024-05-15T15:54:00.344 INFO:tasks.workunit.client.1.smithi144.stdout:8/82: mkdir d2/d1a 0 2024-05-15T15:54:00.347 INFO:tasks.workunit.client.1.smithi144.stdout:8/83: write d2/de/d12/f17 [635453,116670] 0 2024-05-15T15:54:00.348 INFO:tasks.workunit.client.1.smithi144.stdout:5/93: mkdir d0/dc/d24 0 2024-05-15T15:54:00.355 INFO:tasks.workunit.client.0.smithi049.stdout:6/145: symlink dd/d17/l22 0 2024-05-15T15:54:00.355 INFO:tasks.workunit.client.0.smithi049.stdout:9/149: dwrite d0/d8/d1b/f36 [0,4194304] 0 2024-05-15T15:54:00.358 INFO:tasks.workunit.client.1.smithi144.stdout:1/90: dread d7/df/fb [0,4194304] 0 2024-05-15T15:54:00.358 INFO:tasks.workunit.client.1.smithi144.stdout:1/91: readlink l2 0 2024-05-15T15:54:00.362 INFO:tasks.workunit.client.1.smithi144.stdout:9/65: symlink d0/d8/d1b/l1d 0 2024-05-15T15:54:00.364 INFO:tasks.workunit.client.1.smithi144.stdout:3/69: mknod d0/c18 0 2024-05-15T15:54:00.365 INFO:tasks.workunit.client.0.smithi049.stdout:9/150: chown d0/l20 8 1 2024-05-15T15:54:00.365 INFO:tasks.workunit.client.1.smithi144.stdout:3/70: write d0/d1/f5 [657837,112025] 0 2024-05-15T15:54:00.366 INFO:tasks.workunit.client.1.smithi144.stdout:9/66: dread d0/f4 [0,4194304] 0 2024-05-15T15:54:00.367 INFO:tasks.workunit.client.1.smithi144.stdout:3/71: chown d0/d1/f5 173553 1 2024-05-15T15:54:00.367 INFO:tasks.workunit.client.1.smithi144.stdout:9/67: chown d0/d8/c13 234 1 2024-05-15T15:54:00.368 INFO:tasks.workunit.client.0.smithi049.stdout:6/146: rename dd/l11 to dd/l23 0 2024-05-15T15:54:00.368 INFO:tasks.workunit.client.1.smithi144.stdout:3/72: read - d0/f15 zero size 2024-05-15T15:54:00.370 INFO:tasks.workunit.client.0.smithi049.stdout:1/185: rmdir d0/d4/dd/de/d1e 39 2024-05-15T15:54:00.371 INFO:tasks.workunit.client.0.smithi049.stdout:3/209: dread f8 [0,4194304] 0 2024-05-15T15:54:00.372 INFO:tasks.workunit.client.1.smithi144.stdout:0/70: symlink d4/df/l1a 0 2024-05-15T15:54:00.374 INFO:tasks.workunit.client.0.smithi049.stdout:6/147: mkdir dd/d17/d24 0 2024-05-15T15:54:00.375 INFO:tasks.workunit.client.0.smithi049.stdout:6/148: readlink dd/l1a 0 2024-05-15T15:54:00.376 INFO:tasks.workunit.client.1.smithi144.stdout:8/84: creat d2/de/d12/f1b x:0 0 0 2024-05-15T15:54:00.376 INFO:tasks.workunit.client.0.smithi049.stdout:8/137: link d2/d14/d1b/c21 d2/d14/c28 0 2024-05-15T15:54:00.378 INFO:tasks.workunit.client.1.smithi144.stdout:0/71: dwrite d4/d6/fb [0,4194304] 0 2024-05-15T15:54:00.380 INFO:tasks.workunit.client.0.smithi049.stdout:3/210: chown d12/d14/f17 1742 1 2024-05-15T15:54:00.382 INFO:tasks.workunit.client.0.smithi049.stdout:9/151: fdatasync d0/d8/d11/f17 0 2024-05-15T15:54:00.382 INFO:tasks.workunit.client.0.smithi049.stdout:9/152: readlink d0/d8/d21/l2d 0 2024-05-15T15:54:00.384 INFO:tasks.workunit.client.1.smithi144.stdout:6/73: rename d7/db/c10 to d7/db/c1c 0 2024-05-15T15:54:00.386 INFO:tasks.workunit.client.0.smithi049.stdout:8/138: symlink d2/d18/l29 0 2024-05-15T15:54:00.387 INFO:tasks.workunit.client.0.smithi049.stdout:8/139: write d2/fc [1785209,70763] 0 2024-05-15T15:54:00.388 INFO:tasks.workunit.client.1.smithi144.stdout:7/101: link d4/df/f11 d4/df/f20 0 2024-05-15T15:54:00.389 INFO:tasks.workunit.client.0.smithi049.stdout:8/140: write d2/f12 [1598331,24612] 0 2024-05-15T15:54:00.393 INFO:tasks.workunit.client.1.smithi144.stdout:4/97: mkdir d6/da/db/d1d 0 2024-05-15T15:54:00.394 INFO:tasks.workunit.client.1.smithi144.stdout:4/98: readlink d6/dd/l16 0 2024-05-15T15:54:00.398 INFO:tasks.workunit.client.1.smithi144.stdout:8/85: symlink d2/de/d12/l1c 0 2024-05-15T15:54:00.399 INFO:tasks.workunit.client.1.smithi144.stdout:0/72: rmdir d4/d6 39 2024-05-15T15:54:00.406 INFO:tasks.workunit.client.1.smithi144.stdout:7/102: unlink f2 0 2024-05-15T15:54:00.407 INFO:tasks.workunit.client.0.smithi049.stdout:1/186: rmdir d0/d4/dd/de/d1e/d2e/d45 0 2024-05-15T15:54:00.408 INFO:tasks.workunit.client.1.smithi144.stdout:9/68: symlink d0/da/d10/df/l1e 0 2024-05-15T15:54:00.409 INFO:tasks.workunit.client.1.smithi144.stdout:3/73: creat d0/d1/d17/f19 x:0 0 0 2024-05-15T15:54:00.411 INFO:tasks.workunit.client.1.smithi144.stdout:3/74: write d0/d1/f5 [747332,41402] 0 2024-05-15T15:54:00.411 INFO:tasks.workunit.client.1.smithi144.stdout:4/99: mkdir d6/dd/d15/d1e 0 2024-05-15T15:54:00.411 INFO:tasks.workunit.client.0.smithi049.stdout:1/187: symlink d0/d4/dd/de/d1e/d2e/d3e/l46 0 2024-05-15T15:54:00.417 INFO:tasks.workunit.client.0.smithi049.stdout:1/188: symlink d0/d4/dd/de/d1e/l47 0 2024-05-15T15:54:00.421 INFO:tasks.workunit.client.1.smithi144.stdout:7/103: dwrite d4/f13 [4194304,4194304] 0 2024-05-15T15:54:00.422 INFO:tasks.workunit.client.1.smithi144.stdout:3/75: mknod d0/d1/c1a 0 2024-05-15T15:54:00.423 INFO:tasks.workunit.client.1.smithi144.stdout:8/86: creat d2/d1a/f1d x:0 0 0 2024-05-15T15:54:00.424 INFO:tasks.workunit.client.1.smithi144.stdout:8/87: fsync d2/f3 0 2024-05-15T15:54:00.425 INFO:tasks.workunit.client.0.smithi049.stdout:1/189: mkdir d0/d4/dd/de/d48 0 2024-05-15T15:54:00.425 INFO:tasks.workunit.client.1.smithi144.stdout:3/76: write d0/d1/d17/f19 [352338,106790] 0 2024-05-15T15:54:00.426 INFO:tasks.workunit.client.1.smithi144.stdout:8/88: read - d2/de/d12/f1b zero size 2024-05-15T15:54:00.426 INFO:tasks.workunit.client.1.smithi144.stdout:0/73: mknod d4/d6/c1b 0 2024-05-15T15:54:00.428 INFO:tasks.workunit.client.1.smithi144.stdout:6/74: creat d7/db/f1d x:0 0 0 2024-05-15T15:54:00.430 INFO:tasks.workunit.client.1.smithi144.stdout:7/104: fsync d4/fc 0 2024-05-15T15:54:00.432 INFO:tasks.workunit.client.1.smithi144.stdout:7/105: stat d4/df/f1e 0 2024-05-15T15:54:00.432 INFO:tasks.workunit.client.0.smithi049.stdout:1/190: truncate d0/d4/dd/de/d1e/f1d 2551850 0 2024-05-15T15:54:00.433 INFO:tasks.workunit.client.1.smithi144.stdout:7/106: dread - d4/f1c zero size 2024-05-15T15:54:00.434 INFO:tasks.workunit.client.1.smithi144.stdout:0/74: dwrite d4/dc/f17 [0,4194304] 0 2024-05-15T15:54:00.435 INFO:tasks.workunit.client.1.smithi144.stdout:8/89: creat d2/de/d12/f1e x:0 0 0 2024-05-15T15:54:00.435 INFO:tasks.workunit.client.0.smithi049.stdout:1/191: creat d0/f49 x:0 0 0 2024-05-15T15:54:00.436 INFO:tasks.workunit.client.1.smithi144.stdout:7/107: truncate d4/df/f1a 1010945 0 2024-05-15T15:54:00.436 INFO:tasks.workunit.client.0.smithi049.stdout:1/192: chown d0/d4/dd/de/d1e/d2e/c3c 151831526 1 2024-05-15T15:54:00.437 INFO:tasks.workunit.client.1.smithi144.stdout:6/75: mkdir d7/d1e 0 2024-05-15T15:54:00.437 INFO:tasks.workunit.client.1.smithi144.stdout:7/108: truncate d4/f6 560765 0 2024-05-15T15:54:00.440 INFO:tasks.workunit.client.0.smithi049.stdout:1/193: creat d0/d4/dd/de/d48/f4a x:0 0 0 2024-05-15T15:54:00.440 INFO:tasks.workunit.client.1.smithi144.stdout:6/76: write d7/fd [1334019,50314] 0 2024-05-15T15:54:00.441 INFO:tasks.workunit.client.0.smithi049.stdout:1/194: truncate d0/f42 947432 0 2024-05-15T15:54:00.441 INFO:tasks.workunit.client.1.smithi144.stdout:7/109: write d4/f13 [3762667,114694] 0 2024-05-15T15:54:00.441 INFO:tasks.workunit.client.1.smithi144.stdout:8/90: dread d2/de/d12/f17 [0,4194304] 0 2024-05-15T15:54:00.442 INFO:tasks.workunit.client.1.smithi144.stdout:9/69: getdents d0/da/d10/d19 0 2024-05-15T15:54:00.443 INFO:tasks.workunit.client.1.smithi144.stdout:9/70: stat d0/d8/d1b 0 2024-05-15T15:54:00.444 INFO:tasks.workunit.client.0.smithi049.stdout:1/195: symlink d0/l4b 0 2024-05-15T15:54:00.447 INFO:tasks.workunit.client.0.smithi049.stdout:1/196: creat d0/d4/dd/de/d36/f4c x:0 0 0 2024-05-15T15:54:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:00 smithi144 ceph-mon[36389]: pgmap v20: 65 pgs: 65 active+clean; 276 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 15 MiB/s wr, 246 op/s 2024-05-15T15:54:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:00 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:00 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:00 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:54:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:00 smithi144 ceph-mon[36389]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:54:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:00 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:54:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:00 smithi144 ceph-mon[36389]: Upgrade: Need to upgrade myself (mgr.smithi144.whcjkr) 2024-05-15T15:54:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:00 smithi144 ceph-mon[36389]: Upgrade: Need to upgrade myself (mgr.smithi144.whcjkr) 2024-05-15T15:54:00.451 INFO:tasks.workunit.client.1.smithi144.stdout:7/110: write d4/df/f16 [679188,54321] 0 2024-05-15T15:54:00.454 INFO:tasks.workunit.client.0.smithi049.stdout:1/197: dwrite d0/d4/dd/de/d36/f43 [0,4194304] 0 2024-05-15T15:54:00.455 INFO:tasks.workunit.client.0.smithi049.stdout:1/198: chown d0/d4/f39 12278552 1 2024-05-15T15:54:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:00 smithi049 ceph-mon[28399]: pgmap v20: 65 pgs: 65 active+clean; 276 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 15 MiB/s wr, 246 op/s 2024-05-15T15:54:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:00 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:00 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:00 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:54:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:00 smithi049 ceph-mon[28399]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:54:00.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:00 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:54:00.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:00 smithi049 ceph-mon[28399]: Upgrade: Need to upgrade myself (mgr.smithi144.whcjkr) 2024-05-15T15:54:00.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:00 smithi049 ceph-mon[28399]: Upgrade: Need to upgrade myself (mgr.smithi144.whcjkr) 2024-05-15T15:54:00.457 INFO:tasks.workunit.client.1.smithi144.stdout:0/75: rename d4/f9 to d4/f1c 0 2024-05-15T15:54:00.457 INFO:tasks.workunit.client.0.smithi049.stdout:1/199: write d0/d4/dd/de/d36/f43 [790907,51892] 0 2024-05-15T15:54:00.458 INFO:tasks.workunit.client.1.smithi144.stdout:7/111: dwrite d4/f12 [0,4194304] 0 2024-05-15T15:54:00.464 INFO:tasks.workunit.client.1.smithi144.stdout:0/76: mkdir d4/dc/d1d 0 2024-05-15T15:54:00.465 INFO:tasks.workunit.client.0.smithi049.stdout:1/200: mkdir d0/d4/dd/de/d4d 0 2024-05-15T15:54:00.467 INFO:tasks.workunit.client.1.smithi144.stdout:7/112: creat d4/f21 x:0 0 0 2024-05-15T15:54:00.467 INFO:tasks.workunit.client.1.smithi144.stdout:9/71: link d0/l15 d0/d8/l1f 0 2024-05-15T15:54:00.468 INFO:tasks.workunit.client.0.smithi049.stdout:1/201: write d0/d4/f39 [1280315,108122] 0 2024-05-15T15:54:00.470 INFO:tasks.workunit.client.0.smithi049.stdout:1/202: write d0/d4/dd/de/d48/f4a [970819,68940] 0 2024-05-15T15:54:00.471 INFO:tasks.workunit.client.0.smithi049.stdout:1/203: chown d0/d4/f39 1126 1 2024-05-15T15:54:00.472 INFO:tasks.workunit.client.1.smithi144.stdout:0/77: symlink d4/l1e 0 2024-05-15T15:54:00.472 INFO:tasks.workunit.client.0.smithi049.stdout:1/204: readlink d0/d4/dd/de/d1e/d2e/l3a 0 2024-05-15T15:54:00.473 INFO:tasks.workunit.client.0.smithi049.stdout:1/205: truncate d0/d4/dd/de/d36/f3f 248633 0 2024-05-15T15:54:00.474 INFO:tasks.workunit.client.0.smithi049.stdout:1/206: read - d0/d4/dd/de/d1e/d2e/f41 zero size 2024-05-15T15:54:00.475 INFO:tasks.workunit.client.1.smithi144.stdout:7/113: dwrite d4/f13 [8388608,4194304] 0 2024-05-15T15:54:00.475 INFO:tasks.workunit.client.1.smithi144.stdout:0/78: dread d4/d6/f8 [0,4194304] 0 2024-05-15T15:54:00.475 INFO:tasks.workunit.client.1.smithi144.stdout:7/114: stat d4/ld 0 2024-05-15T15:54:00.477 INFO:tasks.workunit.client.1.smithi144.stdout:0/79: mkdir d4/df/d1f 0 2024-05-15T15:54:00.480 INFO:tasks.workunit.client.1.smithi144.stdout:0/80: mknod d4/dc/c20 0 2024-05-15T15:54:00.480 INFO:tasks.workunit.client.1.smithi144.stdout:0/81: dread - d4/dc/f18 zero size 2024-05-15T15:54:00.482 INFO:tasks.workunit.client.0.smithi049.stdout:1/207: dwrite d0/d4/dd/de/d36/f4c [0,4194304] 0 2024-05-15T15:54:00.484 INFO:tasks.workunit.client.1.smithi144.stdout:7/115: fdatasync d4/df/f11 0 2024-05-15T15:54:00.484 INFO:tasks.workunit.client.1.smithi144.stdout:7/116: fdatasync d4/df/f20 0 2024-05-15T15:54:00.486 INFO:tasks.workunit.client.1.smithi144.stdout:0/82: write d4/f1c [563989,76719] 0 2024-05-15T15:54:00.488 INFO:tasks.workunit.client.1.smithi144.stdout:7/117: creat d4/df/f22 x:0 0 0 2024-05-15T15:54:00.489 INFO:tasks.workunit.client.0.smithi049.stdout:1/208: mkdir d0/d4/dd/de/d4e 0 2024-05-15T15:54:00.492 INFO:tasks.workunit.client.1.smithi144.stdout:7/118: rename d4/df/f1e to d4/df/d10/f23 0 2024-05-15T15:54:00.499 INFO:tasks.workunit.client.0.smithi049.stdout:6/149: write fb [5145250,48404] 0 2024-05-15T15:54:00.501 INFO:tasks.workunit.client.0.smithi049.stdout:3/211: getdents d12/d30 0 2024-05-15T15:54:00.503 INFO:tasks.workunit.client.0.smithi049.stdout:3/212: rmdir d12 39 2024-05-15T15:54:00.506 INFO:tasks.workunit.client.0.smithi049.stdout:3/213: write d12/d30/f38 [227429,112047] 0 2024-05-15T15:54:00.506 INFO:tasks.workunit.client.0.smithi049.stdout:6/150: dwrite dd/f15 [0,4194304] 0 2024-05-15T15:54:00.508 INFO:tasks.workunit.client.1.smithi144.stdout:2/62: write d3/f9 [2464356,49097] 0 2024-05-15T15:54:00.508 INFO:tasks.workunit.client.0.smithi049.stdout:6/151: symlink dd/l25 0 2024-05-15T15:54:00.510 INFO:tasks.workunit.client.0.smithi049.stdout:3/214: mknod d12/d14/d32/c39 0 2024-05-15T15:54:00.512 INFO:tasks.workunit.client.1.smithi144.stdout:2/63: mkdir d3/d10/d15 0 2024-05-15T15:54:00.512 INFO:tasks.workunit.client.1.smithi144.stdout:2/64: chown d3/d10 4 1 2024-05-15T15:54:00.515 INFO:tasks.workunit.client.1.smithi144.stdout:4/100: sync 2024-05-15T15:54:00.515 INFO:tasks.workunit.client.0.smithi049.stdout:3/215: link d12/d14/f2c d12/d14/d32/f3a 0 2024-05-15T15:54:00.516 INFO:tasks.workunit.client.0.smithi049.stdout:3/216: write f9 [4842911,45552] 0 2024-05-15T15:54:00.574 INFO:tasks.workunit.client.0.smithi049.stdout:5/183: sync 2024-05-15T15:54:00.575 INFO:tasks.workunit.client.1.smithi144.stdout:5/94: write d0/f5 [1305244,26342] 0 2024-05-15T15:54:00.576 INFO:tasks.workunit.client.0.smithi049.stdout:6/152: getdents dd/d17 0 2024-05-15T15:54:00.577 INFO:tasks.workunit.client.0.smithi049.stdout:3/217: creat d12/d26/f3b x:0 0 0 2024-05-15T15:54:00.579 INFO:tasks.workunit.client.0.smithi049.stdout:5/184: mknod dc/d29/c38 0 2024-05-15T15:54:00.580 INFO:tasks.workunit.client.0.smithi049.stdout:5/185: chown dc/d29/d2c/d31 615438045 1 2024-05-15T15:54:00.580 INFO:tasks.workunit.client.1.smithi144.stdout:5/95: fdatasync d0/dc/f10 0 2024-05-15T15:54:00.581 INFO:tasks.workunit.client.0.smithi049.stdout:3/218: dread f8 [0,4194304] 0 2024-05-15T15:54:00.581 INFO:tasks.workunit.client.1.smithi144.stdout:5/96: chown d0/ld 551252 1 2024-05-15T15:54:00.582 INFO:tasks.workunit.client.0.smithi049.stdout:3/219: fsync d12/f29 0 2024-05-15T15:54:00.582 INFO:tasks.workunit.client.0.smithi049.stdout:6/153: readlink dd/l23 0 2024-05-15T15:54:00.582 INFO:tasks.workunit.client.0.smithi049.stdout:6/154: fsync dd/f15 0 2024-05-15T15:54:00.583 INFO:tasks.workunit.client.0.smithi049.stdout:3/220: dread - d12/d14/f2c zero size 2024-05-15T15:54:00.583 INFO:tasks.workunit.client.1.smithi144.stdout:4/101: creat d6/da/db/d1d/f1f x:0 0 0 2024-05-15T15:54:00.583 INFO:tasks.workunit.client.0.smithi049.stdout:6/155: fsync dd/d17/f20 0 2024-05-15T15:54:00.584 INFO:tasks.workunit.client.0.smithi049.stdout:6/156: read dd/d17/f1d [155079,71615] 0 2024-05-15T15:54:00.585 INFO:tasks.workunit.client.0.smithi049.stdout:6/157: write dd/ff [2623703,44008] 0 2024-05-15T15:54:00.586 INFO:tasks.workunit.client.1.smithi144.stdout:5/97: symlink d0/l25 0 2024-05-15T15:54:00.589 INFO:tasks.workunit.client.1.smithi144.stdout:4/102: unlink d6/f18 0 2024-05-15T15:54:00.591 INFO:tasks.workunit.client.1.smithi144.stdout:5/98: dread d0/dc/f1b [0,4194304] 0 2024-05-15T15:54:00.591 INFO:tasks.workunit.client.0.smithi049.stdout:8/141: write d2/d14/f17 [4338635,95297] 0 2024-05-15T15:54:00.593 INFO:tasks.workunit.client.0.smithi049.stdout:5/186: mkdir dc/dd/d39 0 2024-05-15T15:54:00.596 INFO:tasks.workunit.client.1.smithi144.stdout:1/92: truncate f6 415704 0 2024-05-15T15:54:00.596 INFO:tasks.workunit.client.1.smithi144.stdout:4/103: dwrite d6/f9 [4194304,4194304] 0 2024-05-15T15:54:00.597 INFO:tasks.workunit.client.0.smithi049.stdout:9/153: write d0/f1 [1695759,30812] 0 2024-05-15T15:54:00.597 INFO:tasks.workunit.client.1.smithi144.stdout:4/104: stat d6/cc 0 2024-05-15T15:54:00.598 INFO:tasks.workunit.client.0.smithi049.stdout:3/221: dwrite f8 [4194304,4194304] 0 2024-05-15T15:54:00.599 INFO:tasks.workunit.client.0.smithi049.stdout:9/154: stat d0/d8/d12 0 2024-05-15T15:54:00.599 INFO:tasks.workunit.client.1.smithi144.stdout:5/99: dwrite d0/f11 [0,4194304] 0 2024-05-15T15:54:00.600 INFO:tasks.workunit.client.0.smithi049.stdout:9/155: chown d0/d8/d9/d22 544 1 2024-05-15T15:54:00.603 INFO:tasks.workunit.client.1.smithi144.stdout:8/91: sync 2024-05-15T15:54:00.605 INFO:tasks.workunit.client.0.smithi049.stdout:9/156: truncate d0/d8/d9/d22/f2e 784150 0 2024-05-15T15:54:00.606 INFO:tasks.workunit.client.1.smithi144.stdout:9/72: getdents d0/da/d10/df 0 2024-05-15T15:54:00.606 INFO:tasks.workunit.client.1.smithi144.stdout:5/100: dread d0/f16 [0,4194304] 0 2024-05-15T15:54:00.609 INFO:tasks.workunit.client.1.smithi144.stdout:1/93: rename l0 to d7/d8/l1a 0 2024-05-15T15:54:00.610 INFO:tasks.workunit.client.0.smithi049.stdout:1/209: rmdir d0/d4/dd/de/d1e/d2e 39 2024-05-15T15:54:00.611 INFO:tasks.workunit.client.0.smithi049.stdout:6/158: getdents dd/d17/d24 0 2024-05-15T15:54:00.615 INFO:tasks.workunit.client.1.smithi144.stdout:4/105: mknod d6/dd/c20 0 2024-05-15T15:54:00.616 INFO:tasks.workunit.client.0.smithi049.stdout:6/159: dread dd/f19 [0,4194304] 0 2024-05-15T15:54:00.617 INFO:tasks.workunit.client.1.smithi144.stdout:3/77: getdents d0/d1 0 2024-05-15T15:54:00.618 INFO:tasks.workunit.client.0.smithi049.stdout:8/142: mknod d2/c2a 0 2024-05-15T15:54:00.619 INFO:tasks.workunit.client.1.smithi144.stdout:1/94: dwrite d7/d8/f12 [0,4194304] 0 2024-05-15T15:54:00.622 INFO:tasks.workunit.client.1.smithi144.stdout:6/77: rmdir d7/db 39 2024-05-15T15:54:00.622 INFO:tasks.workunit.client.0.smithi049.stdout:5/187: dread f0 [0,4194304] 0 2024-05-15T15:54:00.623 INFO:tasks.workunit.client.0.smithi049.stdout:5/188: fdatasync dc/dd/f13 0 2024-05-15T15:54:00.624 INFO:tasks.workunit.client.1.smithi144.stdout:9/73: rename d0/c12 to d0/c20 0 2024-05-15T15:54:00.630 INFO:tasks.workunit.client.1.smithi144.stdout:3/78: unlink d0/d11/f12 0 2024-05-15T15:54:00.631 INFO:tasks.workunit.client.1.smithi144.stdout:4/106: dwrite d6/dd/d15/f1c [0,4194304] 0 2024-05-15T15:54:00.631 INFO:tasks.workunit.client.0.smithi049.stdout:9/157: dwrite d0/d25/f31 [0,4194304] 0 2024-05-15T15:54:00.633 INFO:tasks.workunit.client.1.smithi144.stdout:1/95: dwrite d7/d9/fe [0,4194304] 0 2024-05-15T15:54:00.635 INFO:tasks.workunit.client.0.smithi049.stdout:6/160: mkdir dd/d26 0 2024-05-15T15:54:00.636 INFO:tasks.workunit.client.0.smithi049.stdout:9/158: dread d0/d8/d11/f1f [0,4194304] 0 2024-05-15T15:54:00.638 INFO:tasks.workunit.client.0.smithi049.stdout:8/143: stat d2/c13 0 2024-05-15T15:54:00.640 INFO:tasks.workunit.client.1.smithi144.stdout:5/101: mknod d0/dc/d1f/c26 0 2024-05-15T15:54:00.641 INFO:tasks.workunit.client.1.smithi144.stdout:5/102: stat d0/dc/d19/d23 0 2024-05-15T15:54:00.642 INFO:tasks.workunit.client.1.smithi144.stdout:5/103: stat d0/d1c 0 2024-05-15T15:54:00.643 INFO:tasks.workunit.client.1.smithi144.stdout:5/104: fdatasync d0/dc/fe 0 2024-05-15T15:54:00.643 INFO:tasks.workunit.client.1.smithi144.stdout:6/78: write d7/db/f1d [581593,10122] 0 2024-05-15T15:54:00.646 INFO:tasks.workunit.client.0.smithi049.stdout:5/189: creat dc/d29/d2c/f3a x:0 0 0 2024-05-15T15:54:00.648 INFO:tasks.workunit.client.1.smithi144.stdout:9/74: dread d0/f7 [0,4194304] 0 2024-05-15T15:54:00.656 INFO:tasks.workunit.client.1.smithi144.stdout:3/79: symlink d0/l1b 0 2024-05-15T15:54:00.659 INFO:tasks.workunit.client.1.smithi144.stdout:7/119: getdents d4 0 2024-05-15T15:54:00.659 INFO:tasks.workunit.client.1.smithi144.stdout:3/80: dread d0/d1/d17/f19 [0,4194304] 0 2024-05-15T15:54:00.659 INFO:tasks.workunit.client.1.smithi144.stdout:7/120: chown d4/ld 2717475 1 2024-05-15T15:54:00.660 INFO:tasks.workunit.client.1.smithi144.stdout:4/107: unlink d6/da/c13 0 2024-05-15T15:54:00.660 INFO:tasks.workunit.client.1.smithi144.stdout:0/83: sync 2024-05-15T15:54:00.662 INFO:tasks.workunit.client.1.smithi144.stdout:1/96: mknod d7/d8/c1b 0 2024-05-15T15:54:00.663 INFO:tasks.workunit.client.0.smithi049.stdout:5/190: dread dc/f1f [0,4194304] 0 2024-05-15T15:54:00.664 INFO:tasks.workunit.client.1.smithi144.stdout:8/92: link d2/de/ca d2/d1a/c1f 0 2024-05-15T15:54:00.664 INFO:tasks.workunit.client.0.smithi049.stdout:9/159: dread d0/d8/d11/f17 [0,4194304] 0 2024-05-15T15:54:00.665 INFO:tasks.workunit.client.0.smithi049.stdout:8/144: creat d2/d14/d1b/f2b x:0 0 0 2024-05-15T15:54:00.667 INFO:tasks.workunit.client.1.smithi144.stdout:1/97: dread d7/df/f15 [0,4194304] 0 2024-05-15T15:54:00.668 INFO:tasks.workunit.client.1.smithi144.stdout:3/81: dwrite d0/d1/f5 [0,4194304] 0 2024-05-15T15:54:00.668 INFO:tasks.workunit.client.0.smithi049.stdout:3/222: rename d12/d14/d32/f34 to d12/d14/f3c 0 2024-05-15T15:54:00.669 INFO:tasks.workunit.client.1.smithi144.stdout:3/82: readlink d0/l2 0 2024-05-15T15:54:00.669 INFO:tasks.workunit.client.0.smithi049.stdout:3/223: fsync d12/f22 0 2024-05-15T15:54:00.670 INFO:tasks.workunit.client.0.smithi049.stdout:3/224: readlink d12/d26/l2e 0 2024-05-15T15:54:00.671 INFO:tasks.workunit.client.0.smithi049.stdout:9/160: dwrite d0/d8/d21/f2f [0,4194304] 0 2024-05-15T15:54:00.674 INFO:tasks.workunit.client.1.smithi144.stdout:5/105: unlink d0/f5 0 2024-05-15T15:54:00.676 INFO:tasks.workunit.client.0.smithi049.stdout:1/210: creat d0/d4/dd/de/d1e/f4f x:0 0 0 2024-05-15T15:54:00.678 INFO:tasks.workunit.client.1.smithi144.stdout:3/83: dwrite d0/d11/f13 [0,4194304] 0 2024-05-15T15:54:00.679 INFO:tasks.workunit.client.0.smithi049.stdout:6/161: creat dd/d17/d24/f27 x:0 0 0 2024-05-15T15:54:00.679 INFO:tasks.workunit.client.0.smithi049.stdout:6/162: readlink la 0 2024-05-15T15:54:00.681 INFO:tasks.workunit.client.1.smithi144.stdout:9/75: fsync d0/f5 0 2024-05-15T15:54:00.681 INFO:tasks.workunit.client.1.smithi144.stdout:2/65: truncate d3/f9 177274 0 2024-05-15T15:54:00.682 INFO:tasks.workunit.client.0.smithi049.stdout:5/191: truncate dc/dd/f17 4840916 0 2024-05-15T15:54:00.691 INFO:tasks.workunit.client.0.smithi049.stdout:8/145: dwrite d2/de/d25/f26 [0,4194304] 0 2024-05-15T15:54:00.697 INFO:tasks.workunit.client.0.smithi049.stdout:9/161: mknod d0/d8/d1b/c37 0 2024-05-15T15:54:00.697 INFO:tasks.workunit.client.0.smithi049.stdout:9/162: fsync d0/d8/d21/f30 0 2024-05-15T15:54:00.698 INFO:tasks.workunit.client.0.smithi049.stdout:9/163: write d0/d8/f1e [2313310,103785] 0 2024-05-15T15:54:00.700 INFO:tasks.workunit.client.0.smithi049.stdout:1/211: creat d0/d4/dd/de/d1e/d16/f50 x:0 0 0 2024-05-15T15:54:00.707 INFO:tasks.workunit.client.0.smithi049.stdout:5/192: write f0 [2694911,36571] 0 2024-05-15T15:54:00.733 INFO:tasks.workunit.client.0.smithi049.stdout:9/164: unlink d0/d8/d11/c14 0 2024-05-15T15:54:00.735 INFO:tasks.workunit.client.1.smithi144.stdout:3/84: creat d0/d1/f1c x:0 0 0 2024-05-15T15:54:00.736 INFO:tasks.workunit.client.0.smithi049.stdout:1/212: mknod d0/d4/dd/de/d36/c51 0 2024-05-15T15:54:00.742 INFO:tasks.workunit.client.0.smithi049.stdout:9/165: unlink d0/d8/d12/f34 0 2024-05-15T15:54:00.745 INFO:tasks.workunit.client.1.smithi144.stdout:4/108: rename d6/da/db/d1d/f1f to d6/d11/f21 0 2024-05-15T15:54:00.748 INFO:tasks.workunit.client.0.smithi049.stdout:8/146: creat d2/f2c x:0 0 0 2024-05-15T15:54:00.749 INFO:tasks.workunit.client.0.smithi049.stdout:8/147: truncate d2/f2c 211870 0 2024-05-15T15:54:00.754 INFO:tasks.workunit.client.0.smithi049.stdout:1/213: symlink d0/d4/dd/de/d4d/l52 0 2024-05-15T15:54:00.756 INFO:tasks.workunit.client.1.smithi144.stdout:2/66: rmdir d3 39 2024-05-15T15:54:00.761 INFO:tasks.workunit.client.0.smithi049.stdout:6/163: getdents dd 0 2024-05-15T15:54:00.763 INFO:tasks.workunit.client.1.smithi144.stdout:5/106: link d0/dc/f1b d0/dc/d1f/f27 0 2024-05-15T15:54:00.764 INFO:tasks.workunit.client.1.smithi144.stdout:5/107: dread - d0/f1a zero size 2024-05-15T15:54:00.767 INFO:tasks.workunit.client.0.smithi049.stdout:5/193: link dc/d29/d2c/f36 dc/dd/f3b 0 2024-05-15T15:54:00.768 INFO:tasks.workunit.client.1.smithi144.stdout:9/76: mkdir d0/d8/d21 0 2024-05-15T15:54:00.770 INFO:tasks.workunit.client.0.smithi049.stdout:8/148: rename d2/d14/f1c to d2/de/d27/f2d 0 2024-05-15T15:54:00.770 INFO:tasks.workunit.client.0.smithi049.stdout:3/225: write d12/d14/f1a [4050912,80637] 0 2024-05-15T15:54:00.771 INFO:tasks.workunit.client.0.smithi049.stdout:8/149: fdatasync d2/d14/d1b/f2b 0 2024-05-15T15:54:00.772 INFO:tasks.workunit.client.1.smithi144.stdout:4/109: unlink d6/f9 0 2024-05-15T15:54:00.772 INFO:tasks.workunit.client.0.smithi049.stdout:8/150: write d2/f12 [2295800,127145] 0 2024-05-15T15:54:00.774 INFO:tasks.workunit.client.0.smithi049.stdout:8/151: write d2/f12 [1844924,122717] 0 2024-05-15T15:54:00.774 INFO:tasks.workunit.client.0.smithi049.stdout:1/214: creat d0/d4/dd/de/d36/f53 x:0 0 0 2024-05-15T15:54:00.776 INFO:tasks.workunit.client.1.smithi144.stdout:3/85: rename d0/d11/f13 to d0/f1d 0 2024-05-15T15:54:00.779 INFO:tasks.workunit.client.1.smithi144.stdout:2/67: chown d3/f5 6 1 2024-05-15T15:54:00.781 INFO:tasks.workunit.client.1.smithi144.stdout:2/68: fdatasync d3/d7/fb 0 2024-05-15T15:54:00.785 INFO:tasks.workunit.client.1.smithi144.stdout:8/93: dwrite d2/f15 [0,4194304] 0 2024-05-15T15:54:00.785 INFO:tasks.workunit.client.0.smithi049.stdout:5/194: dread dc/dd/d10/f2a [0,4194304] 0 2024-05-15T15:54:00.785 INFO:tasks.workunit.client.1.smithi144.stdout:6/79: dwrite f1 [0,4194304] 0 2024-05-15T15:54:00.786 INFO:tasks.workunit.client.1.smithi144.stdout:4/110: write d6/d11/f21 [938041,24764] 0 2024-05-15T15:54:00.787 INFO:tasks.workunit.client.0.smithi049.stdout:3/226: creat d12/d30/f3d x:0 0 0 2024-05-15T15:54:00.789 INFO:tasks.workunit.client.0.smithi049.stdout:5/195: dread dc/f1b [0,4194304] 0 2024-05-15T15:54:00.790 INFO:tasks.workunit.client.0.smithi049.stdout:5/196: fdatasync dc/dd/f13 0 2024-05-15T15:54:00.791 INFO:tasks.workunit.client.0.smithi049.stdout:5/197: read - dc/dd/f3b zero size 2024-05-15T15:54:00.792 INFO:tasks.workunit.client.0.smithi049.stdout:5/198: chown dc/dd/d10/l2b 28 1 2024-05-15T15:54:00.795 INFO:tasks.workunit.client.1.smithi144.stdout:5/108: mkdir d0/dc/d24/d28 0 2024-05-15T15:54:00.797 INFO:tasks.workunit.client.0.smithi049.stdout:8/152: mknod d2/de/d27/c2e 0 2024-05-15T15:54:00.799 INFO:tasks.workunit.client.1.smithi144.stdout:0/84: truncate f2 2611458 0 2024-05-15T15:54:00.799 INFO:tasks.workunit.client.1.smithi144.stdout:0/85: fsync d4/f1c 0 2024-05-15T15:54:00.799 INFO:tasks.workunit.client.1.smithi144.stdout:0/86: dread - d4/d6/f11 zero size 2024-05-15T15:54:00.800 INFO:tasks.workunit.client.1.smithi144.stdout:0/87: write d4/f5 [2232713,122698] 0 2024-05-15T15:54:00.800 INFO:tasks.workunit.client.1.smithi144.stdout:0/88: read d4/f5 [2997385,61508] 0 2024-05-15T15:54:00.801 INFO:tasks.workunit.client.1.smithi144.stdout:0/89: chown d4/d6/ce 1470563 1 2024-05-15T15:54:00.801 INFO:tasks.workunit.client.1.smithi144.stdout:0/90: chown d4/l7 4229204 1 2024-05-15T15:54:00.805 INFO:tasks.workunit.client.1.smithi144.stdout:9/77: symlink d0/da/d10/df/l22 0 2024-05-15T15:54:00.806 INFO:tasks.workunit.client.1.smithi144.stdout:1/98: write d7/d9/f19 [1330465,81288] 0 2024-05-15T15:54:00.807 INFO:tasks.workunit.client.1.smithi144.stdout:8/94: mknod d2/de/c20 0 2024-05-15T15:54:00.808 INFO:tasks.workunit.client.0.smithi049.stdout:9/166: write d0/d8/d11/f17 [1341550,73774] 0 2024-05-15T15:54:00.809 INFO:tasks.workunit.client.1.smithi144.stdout:6/80: creat d7/db/d11/f1f x:0 0 0 2024-05-15T15:54:00.810 INFO:tasks.workunit.client.1.smithi144.stdout:6/81: write f1 [579582,36610] 0 2024-05-15T15:54:00.811 INFO:tasks.workunit.client.1.smithi144.stdout:1/99: dread d7/d9/f19 [0,4194304] 0 2024-05-15T15:54:00.812 INFO:tasks.workunit.client.1.smithi144.stdout:4/111: creat d6/dd/f22 x:0 0 0 2024-05-15T15:54:00.815 INFO:tasks.workunit.client.1.smithi144.stdout:6/82: dread d7/ff [0,4194304] 0 2024-05-15T15:54:00.815 INFO:tasks.workunit.client.0.smithi049.stdout:8/153: write d2/de/d27/f2d [417365,82049] 0 2024-05-15T15:54:00.815 INFO:tasks.workunit.client.1.smithi144.stdout:4/112: dread f0 [0,4194304] 0 2024-05-15T15:54:00.816 INFO:tasks.workunit.client.1.smithi144.stdout:7/121: mkdir d4/df/d10/d24 0 2024-05-15T15:54:00.816 INFO:tasks.workunit.client.1.smithi144.stdout:7/122: fsync d4/f21 0 2024-05-15T15:54:00.816 INFO:tasks.workunit.client.1.smithi144.stdout:4/113: read - d6/da/f1b zero size 2024-05-15T15:54:00.817 INFO:tasks.workunit.client.1.smithi144.stdout:4/114: dread - d6/da/f1b zero size 2024-05-15T15:54:00.820 INFO:tasks.workunit.client.1.smithi144.stdout:0/91: mknod d4/d6/c21 0 2024-05-15T15:54:00.823 INFO:tasks.workunit.client.0.smithi049.stdout:6/164: link dd/d17/l1e dd/d17/d24/l28 0 2024-05-15T15:54:00.827 INFO:tasks.workunit.client.0.smithi049.stdout:6/165: dread dd/f19 [0,4194304] 0 2024-05-15T15:54:00.828 INFO:tasks.workunit.client.1.smithi144.stdout:8/95: fdatasync d2/de/d12/f17 0 2024-05-15T15:54:00.829 INFO:tasks.workunit.client.0.smithi049.stdout:3/227: mknod d12/c3e 0 2024-05-15T15:54:00.829 INFO:tasks.workunit.client.1.smithi144.stdout:8/96: truncate d2/f3 928194 0 2024-05-15T15:54:00.830 INFO:tasks.workunit.client.1.smithi144.stdout:8/97: chown d2/fd 0 1 2024-05-15T15:54:00.834 INFO:tasks.workunit.client.0.smithi049.stdout:8/154: creat d2/d14/f2f x:0 0 0 2024-05-15T15:54:00.836 INFO:tasks.workunit.client.1.smithi144.stdout:1/100: unlink d7/d8/c13 0 2024-05-15T15:54:00.836 INFO:tasks.workunit.client.1.smithi144.stdout:1/101: chown d7/d9 17082 1 2024-05-15T15:54:00.838 INFO:tasks.workunit.client.1.smithi144.stdout:6/83: mknod d7/db/d11/c20 0 2024-05-15T15:54:00.838 INFO:tasks.workunit.client.1.smithi144.stdout:6/84: fdatasync d7/db/d11/f1f 0 2024-05-15T15:54:00.839 INFO:tasks.workunit.client.0.smithi049.stdout:6/166: symlink dd/l29 0 2024-05-15T15:54:00.840 INFO:tasks.workunit.client.0.smithi049.stdout:3/228: truncate d12/d14/f2c 654659 0 2024-05-15T15:54:00.844 INFO:tasks.workunit.client.0.smithi049.stdout:1/215: getdents d0/d4/dd/de/d48 0 2024-05-15T15:54:00.847 INFO:tasks.workunit.client.1.smithi144.stdout:2/69: creat d3/f16 x:0 0 0 2024-05-15T15:54:00.847 INFO:tasks.workunit.client.1.smithi144.stdout:2/70: stat d3/f9 0 2024-05-15T15:54:00.848 INFO:tasks.workunit.client.1.smithi144.stdout:9/78: symlink d0/da/l23 0 2024-05-15T15:54:00.852 INFO:tasks.workunit.client.0.smithi049.stdout:8/155: mknod d2/d14/d1d/c30 0 2024-05-15T15:54:00.854 INFO:tasks.workunit.client.1.smithi144.stdout:8/98: mkdir d2/d21 0 2024-05-15T15:54:00.855 INFO:tasks.workunit.client.0.smithi049.stdout:6/167: getdents dd/d26 0 2024-05-15T15:54:00.855 INFO:tasks.workunit.client.0.smithi049.stdout:9/167: write d0/d8/fd [3831401,316] 0 2024-05-15T15:54:00.858 INFO:tasks.workunit.client.1.smithi144.stdout:1/102: symlink d7/d9/l1c 0 2024-05-15T15:54:00.859 INFO:tasks.workunit.client.1.smithi144.stdout:1/103: chown d7/d9/fe 73 1 2024-05-15T15:54:00.860 INFO:tasks.workunit.client.0.smithi049.stdout:1/216: mknod d0/d4/dd/de/d1e/d2e/c54 0 2024-05-15T15:54:00.861 INFO:tasks.workunit.client.1.smithi144.stdout:7/123: mkdir d4/df/d25 0 2024-05-15T15:54:00.863 INFO:tasks.workunit.client.0.smithi049.stdout:6/168: symlink dd/l2a 0 2024-05-15T15:54:00.867 INFO:tasks.workunit.client.0.smithi049.stdout:6/169: dread fb [0,4194304] 0 2024-05-15T15:54:00.867 INFO:tasks.workunit.client.0.smithi049.stdout:8/156: symlink d2/de/l31 0 2024-05-15T15:54:00.868 INFO:tasks.workunit.client.1.smithi144.stdout:2/71: creat d3/d7/f17 x:0 0 0 2024-05-15T15:54:00.871 INFO:tasks.workunit.client.0.smithi049.stdout:1/217: rename d0/d4/dd/de/d1e/d2e/d3e/l46 to d0/d4/dd/de/d4d/l55 0 2024-05-15T15:54:00.872 INFO:tasks.workunit.client.1.smithi144.stdout:9/79: creat d0/da/d10/f24 x:0 0 0 2024-05-15T15:54:00.875 INFO:tasks.workunit.client.0.smithi049.stdout:9/168: creat d0/d8/d9/d22/d26/f38 x:0 0 0 2024-05-15T15:54:00.876 INFO:tasks.workunit.client.0.smithi049.stdout:3/229: write d12/d14/d32/f3a [93576,4128] 0 2024-05-15T15:54:00.877 INFO:tasks.workunit.client.1.smithi144.stdout:1/104: fsync d7/df/fb 0 2024-05-15T15:54:00.878 INFO:tasks.workunit.client.0.smithi049.stdout:8/157: write d2/de/f16 [2041954,16567] 0 2024-05-15T15:54:00.878 INFO:tasks.workunit.client.1.smithi144.stdout:6/85: symlink d7/d1e/l21 0 2024-05-15T15:54:00.879 INFO:tasks.workunit.client.0.smithi049.stdout:8/158: fdatasync d2/fc 0 2024-05-15T15:54:00.881 INFO:tasks.workunit.client.1.smithi144.stdout:7/124: rmdir d4/df 39 2024-05-15T15:54:00.883 INFO:tasks.workunit.client.0.smithi049.stdout:8/159: dread d2/de/d25/f26 [0,4194304] 0 2024-05-15T15:54:00.884 INFO:tasks.workunit.client.1.smithi144.stdout:4/115: link d6/cc d6/dd/d15/d1e/c23 0 2024-05-15T15:54:00.885 INFO:tasks.workunit.client.0.smithi049.stdout:8/160: read d2/fb [364707,74122] 0 2024-05-15T15:54:00.885 INFO:tasks.workunit.client.0.smithi049.stdout:8/161: fdatasync d2/f2c 0 2024-05-15T15:54:00.886 INFO:tasks.workunit.client.1.smithi144.stdout:0/92: creat d4/f22 x:0 0 0 2024-05-15T15:54:00.887 INFO:tasks.workunit.client.0.smithi049.stdout:9/169: rename d0/d8/d12/f33 to d0/d8/f39 0 2024-05-15T15:54:00.888 INFO:tasks.workunit.client.1.smithi144.stdout:3/86: sync 2024-05-15T15:54:00.889 INFO:tasks.workunit.client.1.smithi144.stdout:2/72: unlink d3/f16 0 2024-05-15T15:54:00.891 INFO:tasks.workunit.client.1.smithi144.stdout:9/80: creat d0/d8/d1b/f25 x:0 0 0 2024-05-15T15:54:00.893 INFO:tasks.workunit.client.1.smithi144.stdout:2/73: dread d3/f5 [0,4194304] 0 2024-05-15T15:54:00.893 INFO:tasks.workunit.client.1.smithi144.stdout:8/99: mknod d2/d21/c22 0 2024-05-15T15:54:00.893 INFO:tasks.workunit.client.1.smithi144.stdout:2/74: chown d3/f9 54 1 2024-05-15T15:54:00.894 INFO:tasks.workunit.client.1.smithi144.stdout:2/75: chown d3 16662 1 2024-05-15T15:54:00.894 INFO:tasks.workunit.client.1.smithi144.stdout:2/76: chown d3/f9 80045 1 2024-05-15T15:54:00.897 INFO:tasks.workunit.client.0.smithi049.stdout:1/218: dwrite d0/d4/f2d [0,4194304] 0 2024-05-15T15:54:00.898 INFO:tasks.workunit.client.0.smithi049.stdout:1/219: chown d0/d4/dd 385102 1 2024-05-15T15:54:00.899 INFO:tasks.workunit.client.0.smithi049.stdout:1/220: chown d0/d4/f24 9584 1 2024-05-15T15:54:00.900 INFO:tasks.workunit.client.1.smithi144.stdout:4/116: symlink d6/d11/l24 0 2024-05-15T15:54:00.900 INFO:tasks.workunit.client.1.smithi144.stdout:4/117: stat d6/dd 0 2024-05-15T15:54:00.901 INFO:tasks.workunit.client.0.smithi049.stdout:1/221: readlink d0/d4/dd/de/d1e/l47 0 2024-05-15T15:54:00.901 INFO:tasks.workunit.client.0.smithi049.stdout:1/222: dread - d0/f49 zero size 2024-05-15T15:54:00.901 INFO:tasks.workunit.client.0.smithi049.stdout:8/162: dread d2/fb [0,4194304] 0 2024-05-15T15:54:00.902 INFO:tasks.workunit.client.1.smithi144.stdout:0/93: symlink d4/dc/l23 0 2024-05-15T15:54:00.904 INFO:tasks.workunit.client.0.smithi049.stdout:3/230: creat d12/f3f x:0 0 0 2024-05-15T15:54:00.907 INFO:tasks.workunit.client.0.smithi049.stdout:1/223: creat d0/d4/dd/de/d4d/f56 x:0 0 0 2024-05-15T15:54:00.908 INFO:tasks.workunit.client.1.smithi144.stdout:1/105: dwrite d7/d9/f19 [0,4194304] 0 2024-05-15T15:54:00.908 INFO:tasks.workunit.client.0.smithi049.stdout:1/224: read - d0/d4/dd/de/d1e/f37 zero size 2024-05-15T15:54:00.910 INFO:tasks.workunit.client.0.smithi049.stdout:1/225: dread - d0/d4/dd/de/d1e/d2e/d3e/f3d zero size 2024-05-15T15:54:00.911 INFO:tasks.workunit.client.1.smithi144.stdout:8/100: creat d2/de/d12/f23 x:0 0 0 2024-05-15T15:54:00.912 INFO:tasks.workunit.client.0.smithi049.stdout:3/231: rmdir d12/d14 39 2024-05-15T15:54:00.913 INFO:tasks.workunit.client.1.smithi144.stdout:2/77: creat d3/d7/f18 x:0 0 0 2024-05-15T15:54:00.915 INFO:tasks.workunit.client.0.smithi049.stdout:9/170: getdents d0/d8/d9/d22 0 2024-05-15T15:54:00.918 INFO:tasks.workunit.client.0.smithi049.stdout:3/232: creat d12/d26/f40 x:0 0 0 2024-05-15T15:54:00.921 INFO:tasks.workunit.client.0.smithi049.stdout:9/171: unlink d0/d8/d12/f1a 0 2024-05-15T15:54:00.923 INFO:tasks.workunit.client.0.smithi049.stdout:8/163: getdents d2/de/d25 0 2024-05-15T15:54:00.926 INFO:tasks.workunit.client.0.smithi049.stdout:1/226: rename d0/d4/dd/de/d1e/c35 to d0/d4/dd/de/d1e/c57 0 2024-05-15T15:54:00.926 INFO:tasks.workunit.client.0.smithi049.stdout:1/227: dread - d0/d4/dd/de/d1e/d16/f30 zero size 2024-05-15T15:54:00.927 INFO:tasks.workunit.client.0.smithi049.stdout:9/172: creat d0/d8/d1b/f3a x:0 0 0 2024-05-15T15:54:00.928 INFO:tasks.workunit.client.0.smithi049.stdout:9/173: write d0/d8/fd [350356,98417] 0 2024-05-15T15:54:00.929 INFO:tasks.workunit.client.0.smithi049.stdout:1/228: write d0/d4/dd/de/d1e/f31 [416169,98166] 0 2024-05-15T15:54:00.933 INFO:tasks.workunit.client.0.smithi049.stdout:1/229: creat d0/f58 x:0 0 0 2024-05-15T15:54:00.933 INFO:tasks.workunit.client.0.smithi049.stdout:3/233: getdents d12/d30 0 2024-05-15T15:54:00.935 INFO:tasks.workunit.client.0.smithi049.stdout:3/234: readlink d12/d14/d32/l35 0 2024-05-15T15:54:00.936 INFO:tasks.workunit.client.0.smithi049.stdout:1/230: unlink d0/d4/dd/de/d1e/f2f 0 2024-05-15T15:54:00.939 INFO:tasks.workunit.client.0.smithi049.stdout:3/235: write d12/d14/f3c [1030719,129674] 0 2024-05-15T15:54:00.941 INFO:tasks.workunit.client.0.smithi049.stdout:8/164: truncate d2/de/d25/f26 4046871 0 2024-05-15T15:54:00.944 INFO:tasks.workunit.client.1.smithi144.stdout:7/125: symlink d4/l26 0 2024-05-15T15:54:00.944 INFO:tasks.workunit.client.1.smithi144.stdout:4/118: creat d6/d11/f25 x:0 0 0 2024-05-15T15:54:00.944 INFO:tasks.workunit.client.0.smithi049.stdout:1/231: dwrite d0/d4/dd/de/d1e/d2e/f2c [0,4194304] 0 2024-05-15T15:54:00.945 INFO:tasks.workunit.client.1.smithi144.stdout:8/101: rmdir d2/d21 39 2024-05-15T15:54:00.945 INFO:tasks.workunit.client.1.smithi144.stdout:2/78: mknod d3/d10/c19 0 2024-05-15T15:54:00.945 INFO:tasks.workunit.client.1.smithi144.stdout:8/102: stat d2/de 0 2024-05-15T15:54:00.945 INFO:tasks.workunit.client.1.smithi144.stdout:3/87: dwrite d0/d1/f4 [0,4194304] 0 2024-05-15T15:54:00.946 INFO:tasks.workunit.client.0.smithi049.stdout:3/236: rename d12/d14/f2c to d12/f41 0 2024-05-15T15:54:00.946 INFO:tasks.workunit.client.1.smithi144.stdout:1/106: dread d7/df/fb [0,4194304] 0 2024-05-15T15:54:00.947 INFO:tasks.workunit.client.0.smithi049.stdout:9/174: dread d0/d8/d9/fb [0,4194304] 0 2024-05-15T15:54:00.947 INFO:tasks.workunit.client.1.smithi144.stdout:8/103: write d2/f3 [1965620,98385] 0 2024-05-15T15:54:00.948 INFO:tasks.workunit.client.1.smithi144.stdout:8/104: read - d2/f16 zero size 2024-05-15T15:54:00.949 INFO:tasks.workunit.client.1.smithi144.stdout:9/81: creat d0/da/f26 x:0 0 0 2024-05-15T15:54:00.951 INFO:tasks.workunit.client.0.smithi049.stdout:3/237: mkdir d12/d30/d42 0 2024-05-15T15:54:00.952 INFO:tasks.workunit.client.1.smithi144.stdout:4/119: symlink d6/da/db/d1d/l26 0 2024-05-15T15:54:00.954 INFO:tasks.workunit.client.0.smithi049.stdout:8/165: rename d2/d14/f17 to d2/de/d25/f32 0 2024-05-15T15:54:00.954 INFO:tasks.workunit.client.1.smithi144.stdout:4/120: readlink d6/da/db/d1d/l26 0 2024-05-15T15:54:00.956 INFO:tasks.workunit.client.1.smithi144.stdout:9/82: dread d0/f5 [0,4194304] 0 2024-05-15T15:54:00.956 INFO:tasks.workunit.client.1.smithi144.stdout:2/79: unlink d3/d7/fb 0 2024-05-15T15:54:00.957 INFO:tasks.workunit.client.0.smithi049.stdout:9/175: mknod d0/d8/d9/d22/c3b 0 2024-05-15T15:54:00.958 INFO:tasks.workunit.client.0.smithi049.stdout:9/176: chown d0/d8/d9/d22/c3b 8 1 2024-05-15T15:54:00.959 INFO:tasks.workunit.client.0.smithi049.stdout:1/232: mknod d0/d4/c59 0 2024-05-15T15:54:00.960 INFO:tasks.workunit.client.1.smithi144.stdout:3/88: mkdir d0/d11/d1e 0 2024-05-15T15:54:00.961 INFO:tasks.workunit.client.1.smithi144.stdout:3/89: stat d0/f6 0 2024-05-15T15:54:00.962 INFO:tasks.workunit.client.1.smithi144.stdout:4/121: dwrite d6/d11/f21 [0,4194304] 0 2024-05-15T15:54:00.963 INFO:tasks.workunit.client.1.smithi144.stdout:1/107: rmdir d7/d8 39 2024-05-15T15:54:00.967 INFO:tasks.workunit.client.0.smithi049.stdout:9/177: dwrite d0/f1 [4194304,4194304] 0 2024-05-15T15:54:00.967 INFO:tasks.workunit.client.1.smithi144.stdout:6/86: link d7/db/d11/l16 d7/l22 0 2024-05-15T15:54:00.969 INFO:tasks.workunit.client.0.smithi049.stdout:3/238: dwrite d12/d14/d32/f3a [0,4194304] 0 2024-05-15T15:54:00.971 INFO:tasks.workunit.client.0.smithi049.stdout:8/166: rename d2/d14/d1b/c24 to d2/d14/d1b/c33 0 2024-05-15T15:54:00.977 INFO:tasks.workunit.client.0.smithi049.stdout:1/233: creat d0/d4/dd/f5a x:0 0 0 2024-05-15T15:54:00.978 INFO:tasks.workunit.client.1.smithi144.stdout:5/109: sync 2024-05-15T15:54:00.981 INFO:tasks.workunit.client.1.smithi144.stdout:7/126: dwrite d4/fa [0,4194304] 0 2024-05-15T15:54:00.984 INFO:tasks.workunit.client.1.smithi144.stdout:2/80: creat d3/d7/f1a x:0 0 0 2024-05-15T15:54:00.986 INFO:tasks.workunit.client.0.smithi049.stdout:8/167: symlink d2/d14/l34 0 2024-05-15T15:54:00.987 INFO:tasks.workunit.client.1.smithi144.stdout:2/81: write d3/d10/f11 [989783,97316] 0 2024-05-15T15:54:00.987 INFO:tasks.workunit.client.1.smithi144.stdout:5/110: dwrite d0/f22 [0,4194304] 0 2024-05-15T15:54:00.989 INFO:tasks.workunit.client.1.smithi144.stdout:9/83: rename d0/d8/d1b/f25 to d0/da/d10/f27 0 2024-05-15T15:54:00.991 INFO:tasks.workunit.client.1.smithi144.stdout:9/84: truncate d0/d8/fb 1445297 0 2024-05-15T15:54:00.993 INFO:tasks.workunit.client.1.smithi144.stdout:3/90: dread d0/d1/d17/f19 [0,4194304] 0 2024-05-15T15:54:00.999 INFO:tasks.workunit.client.0.smithi049.stdout:3/239: rename f5 to d12/d14/f43 0 2024-05-15T15:54:01.000 INFO:tasks.workunit.client.0.smithi049.stdout:8/168: rmdir d2/d14/d1b 39 2024-05-15T15:54:01.004 INFO:tasks.workunit.client.0.smithi049.stdout:3/240: unlink fc 0 2024-05-15T15:54:01.006 INFO:tasks.workunit.client.1.smithi144.stdout:4/122: rename d6/da/c10 to d6/c27 0 2024-05-15T15:54:01.007 INFO:tasks.workunit.client.0.smithi049.stdout:8/169: creat d2/de/d27/f35 x:0 0 0 2024-05-15T15:54:01.010 INFO:tasks.workunit.client.1.smithi144.stdout:1/108: mkdir d7/d1d 0 2024-05-15T15:54:01.010 INFO:tasks.workunit.client.0.smithi049.stdout:8/170: dread d2/d14/f1e [0,4194304] 0 2024-05-15T15:54:01.011 INFO:tasks.workunit.client.1.smithi144.stdout:0/94: sync 2024-05-15T15:54:01.012 INFO:tasks.workunit.client.0.smithi049.stdout:9/178: truncate d0/d25/f31 3329418 0 2024-05-15T15:54:01.012 INFO:tasks.workunit.client.0.smithi049.stdout:8/171: read d2/fb [387507,115246] 0 2024-05-15T15:54:01.013 INFO:tasks.workunit.client.1.smithi144.stdout:9/85: dwrite d0/f14 [0,4194304] 0 2024-05-15T15:54:01.013 INFO:tasks.workunit.client.1.smithi144.stdout:6/87: mknod d7/c23 0 2024-05-15T15:54:01.014 INFO:tasks.workunit.client.1.smithi144.stdout:6/88: chown d7/db/d11/f15 2322262 1 2024-05-15T15:54:01.015 INFO:tasks.workunit.client.0.smithi049.stdout:3/241: mkdir d12/d30/d42/d44 0 2024-05-15T15:54:01.016 INFO:tasks.workunit.client.0.smithi049.stdout:1/234: dwrite d0/d4/fa [0,4194304] 0 2024-05-15T15:54:01.016 INFO:tasks.workunit.client.1.smithi144.stdout:6/89: write d7/db/d11/f1b [323099,79220] 0 2024-05-15T15:54:01.016 INFO:tasks.workunit.client.1.smithi144.stdout:3/91: symlink d0/d11/l1f 0 2024-05-15T15:54:01.017 INFO:tasks.workunit.client.0.smithi049.stdout:1/235: chown d0/d4/dd/de/d36/f4c 2209 1 2024-05-15T15:54:01.018 INFO:tasks.workunit.client.0.smithi049.stdout:1/236: readlink d0/l4b 0 2024-05-15T15:54:01.019 INFO:tasks.workunit.client.0.smithi049.stdout:1/237: stat d0/d4/dd/de/d36/f4c 0 2024-05-15T15:54:01.020 INFO:tasks.workunit.client.0.smithi049.stdout:9/179: creat d0/d8/d1b/f3c x:0 0 0 2024-05-15T15:54:01.022 INFO:tasks.workunit.client.1.smithi144.stdout:2/82: symlink d3/d10/d15/l1b 0 2024-05-15T15:54:01.023 INFO:tasks.workunit.client.0.smithi049.stdout:3/242: symlink d12/d30/l45 0 2024-05-15T15:54:01.023 INFO:tasks.workunit.client.0.smithi049.stdout:1/238: readlink d0/d4/dd/de/d4d/l55 0 2024-05-15T15:54:01.024 INFO:tasks.workunit.client.0.smithi049.stdout:3/243: write d12/d14/f1e [1958909,17449] 0 2024-05-15T15:54:01.025 INFO:tasks.workunit.client.0.smithi049.stdout:3/244: dread - d12/d26/f3b zero size 2024-05-15T15:54:01.026 INFO:tasks.workunit.client.1.smithi144.stdout:5/111: rename d0/d1c to d0/dc/d1f/d29 0 2024-05-15T15:54:01.027 INFO:tasks.workunit.client.0.smithi049.stdout:8/172: rename d2/f12 to d2/d14/d1b/f36 0 2024-05-15T15:54:01.029 INFO:tasks.workunit.client.1.smithi144.stdout:2/83: dwrite d3/d10/f11 [0,4194304] 0 2024-05-15T15:54:01.031 INFO:tasks.workunit.client.1.smithi144.stdout:2/84: dread d3/f9 [0,4194304] 0 2024-05-15T15:54:01.031 INFO:tasks.workunit.client.1.smithi144.stdout:2/85: stat d3/d10/f11 0 2024-05-15T15:54:01.032 INFO:tasks.workunit.client.1.smithi144.stdout:2/86: write d3/d10/f11 [165611,121917] 0 2024-05-15T15:54:01.075 INFO:tasks.workunit.client.0.smithi049.stdout:0/186: sync 2024-05-15T15:54:01.078 INFO:tasks.workunit.client.0.smithi049.stdout:9/180: truncate d0/d8/d9/fb 3790258 0 2024-05-15T15:54:01.085 INFO:tasks.workunit.client.0.smithi049.stdout:1/239: rename d0/f42 to d0/f5b 0 2024-05-15T15:54:01.087 INFO:tasks.workunit.client.1.smithi144.stdout:9/86: fdatasync d0/f18 0 2024-05-15T15:54:01.087 INFO:tasks.workunit.client.1.smithi144.stdout:9/87: fsync d0/d8/fb 0 2024-05-15T15:54:01.088 INFO:tasks.workunit.client.1.smithi144.stdout:9/88: write d0/f18 [81072,72277] 0 2024-05-15T15:54:01.089 INFO:tasks.workunit.client.1.smithi144.stdout:9/89: read d0/f18 [125846,101536] 0 2024-05-15T15:54:01.090 INFO:tasks.workunit.client.1.smithi144.stdout:3/92: creat d0/d1/d17/f20 x:0 0 0 2024-05-15T15:54:01.092 INFO:tasks.workunit.client.1.smithi144.stdout:9/90: dread d0/f5 [0,4194304] 0 2024-05-15T15:54:01.095 INFO:tasks.workunit.client.1.smithi144.stdout:9/91: dread d0/f14 [0,4194304] 0 2024-05-15T15:54:01.096 INFO:tasks.workunit.client.1.smithi144.stdout:8/105: rename d2/d21 to d2/d24 0 2024-05-15T15:54:01.096 INFO:tasks.workunit.client.1.smithi144.stdout:9/92: rename d0 to d0/d8/d28 22 2024-05-15T15:54:01.097 INFO:tasks.workunit.client.0.smithi049.stdout:9/181: symlink d0/d8/d21/l3d 0 2024-05-15T15:54:01.098 INFO:tasks.workunit.client.0.smithi049.stdout:9/182: stat d0/d8/d1b/f3c 0 2024-05-15T15:54:01.101 INFO:tasks.workunit.client.1.smithi144.stdout:7/127: creat d4/f27 x:0 0 0 2024-05-15T15:54:01.103 INFO:tasks.workunit.client.0.smithi049.stdout:1/240: mkdir d0/d4/dd/de/d48/d5c 0 2024-05-15T15:54:01.107 INFO:tasks.workunit.client.1.smithi144.stdout:1/109: write d7/df/f15 [602854,87534] 0 2024-05-15T15:54:01.108 INFO:tasks.workunit.client.1.smithi144.stdout:1/110: write f1 [3902300,96776] 0 2024-05-15T15:54:01.109 INFO:tasks.workunit.client.0.smithi049.stdout:0/187: creat dc/d32/f39 x:0 0 0 2024-05-15T15:54:01.111 INFO:tasks.workunit.client.1.smithi144.stdout:1/111: fsync f1 0 2024-05-15T15:54:01.112 INFO:tasks.workunit.client.0.smithi049.stdout:9/183: rmdir d0 39 2024-05-15T15:54:01.113 INFO:tasks.workunit.client.1.smithi144.stdout:4/123: chown d6/c27 463 1 2024-05-15T15:54:01.114 INFO:tasks.workunit.client.1.smithi144.stdout:0/95: dwrite d4/d6/f8 [0,4194304] 0 2024-05-15T15:54:01.115 INFO:tasks.workunit.client.0.smithi049.stdout:1/241: mkdir d0/d4/dd/d5d 0 2024-05-15T15:54:01.116 INFO:tasks.workunit.client.0.smithi049.stdout:1/242: chown d0/d4/dd/de/d1e/c11 475533341 1 2024-05-15T15:54:01.119 INFO:tasks.workunit.client.0.smithi049.stdout:0/188: rename dc/fd to dc/d13/d14/d2a/f3a 0 2024-05-15T15:54:01.120 INFO:tasks.workunit.client.1.smithi144.stdout:4/124: dwrite d6/d11/f25 [0,4194304] 0 2024-05-15T15:54:01.121 INFO:tasks.workunit.client.1.smithi144.stdout:6/90: dwrite d7/ff [0,4194304] 0 2024-05-15T15:54:01.122 INFO:tasks.workunit.client.1.smithi144.stdout:4/125: chown d6/dd/l16 2 1 2024-05-15T15:54:01.123 INFO:tasks.workunit.client.1.smithi144.stdout:6/91: truncate d7/db/f18 442975 0 2024-05-15T15:54:01.126 INFO:tasks.workunit.client.1.smithi144.stdout:3/93: creat d0/d1/f21 x:0 0 0 2024-05-15T15:54:01.128 INFO:tasks.workunit.client.0.smithi049.stdout:8/173: getdents d2/de/d27 0 2024-05-15T15:54:01.129 INFO:tasks.workunit.client.1.smithi144.stdout:5/112: write d0/dc/d1f/f27 [3356110,98629] 0 2024-05-15T15:54:01.132 INFO:tasks.workunit.client.0.smithi049.stdout:9/184: creat d0/d8/d9/d22/d26/f3e x:0 0 0 2024-05-15T15:54:01.135 INFO:tasks.workunit.client.1.smithi144.stdout:5/113: dread d0/f11 [0,4194304] 0 2024-05-15T15:54:01.138 INFO:tasks.workunit.client.0.smithi049.stdout:1/243: link d0/d4/f24 d0/f5e 0 2024-05-15T15:54:01.141 INFO:tasks.workunit.client.0.smithi049.stdout:0/189: symlink dc/d13/d14/d25/d31/d38/l3b 0 2024-05-15T15:54:01.146 INFO:tasks.workunit.client.0.smithi049.stdout:1/244: mknod d0/d4/dd/de/d36/c5f 0 2024-05-15T15:54:01.147 INFO:tasks.workunit.client.0.smithi049.stdout:1/245: dread - d0/f58 zero size 2024-05-15T15:54:01.148 INFO:tasks.workunit.client.0.smithi049.stdout:1/246: fdatasync d0/d4/dd/de/d1e/d16/f50 0 2024-05-15T15:54:01.154 INFO:tasks.workunit.client.0.smithi049.stdout:1/247: symlink d0/d4/dd/de/d1e/d16/l60 0 2024-05-15T15:54:01.154 INFO:tasks.workunit.client.0.smithi049.stdout:0/190: dwrite dc/d13/f22 [0,4194304] 0 2024-05-15T15:54:01.155 INFO:tasks.workunit.client.0.smithi049.stdout:1/248: stat d0/d4/dd 0 2024-05-15T15:54:01.156 INFO:tasks.workunit.client.0.smithi049.stdout:1/249: fsync d0/d4/dd/de/d1e/f31 0 2024-05-15T15:54:01.163 INFO:tasks.workunit.client.0.smithi049.stdout:8/174: truncate d2/de/d25/f32 2896306 0 2024-05-15T15:54:01.168 INFO:tasks.workunit.client.0.smithi049.stdout:1/250: dwrite d0/f5b [0,4194304] 0 2024-05-15T15:54:01.168 INFO:tasks.workunit.client.0.smithi049.stdout:0/191: creat dc/d13/f3c x:0 0 0 2024-05-15T15:54:01.170 INFO:tasks.workunit.client.0.smithi049.stdout:8/175: rmdir d2/d14/d1b 39 2024-05-15T15:54:01.177 INFO:tasks.workunit.client.0.smithi049.stdout:1/251: dwrite d0/f58 [0,4194304] 0 2024-05-15T15:54:01.178 INFO:tasks.workunit.client.0.smithi049.stdout:8/176: write d2/d14/d1b/f2b [189784,19899] 0 2024-05-15T15:54:01.179 INFO:tasks.workunit.client.0.smithi049.stdout:1/252: dread - d0/d4/dd/de/d1e/d16/f50 zero size 2024-05-15T15:54:01.179 INFO:tasks.workunit.client.0.smithi049.stdout:1/253: stat d0/f49 0 2024-05-15T15:54:01.185 INFO:tasks.workunit.client.0.smithi049.stdout:0/192: mkdir dc/d13/d3d 0 2024-05-15T15:54:01.186 INFO:tasks.workunit.client.0.smithi049.stdout:8/177: chown d2/d14/d1b/c33 4654 1 2024-05-15T15:54:01.189 INFO:tasks.workunit.client.0.smithi049.stdout:0/193: read dc/d13/d14/f19 [3339440,104105] 0 2024-05-15T15:54:01.190 INFO:tasks.workunit.client.0.smithi049.stdout:0/194: chown dc/d13/d14/d33 54 1 2024-05-15T15:54:01.190 INFO:tasks.workunit.client.0.smithi049.stdout:0/195: chown dc/d13/d14/d2f 5071995 1 2024-05-15T15:54:01.192 INFO:tasks.workunit.client.0.smithi049.stdout:0/196: read dc/d13/f26 [530380,60152] 0 2024-05-15T15:54:01.198 INFO:tasks.workunit.client.0.smithi049.stdout:1/254: rename d0/d4/dd/de/d1e/c33 to d0/d4/c61 0 2024-05-15T15:54:01.200 INFO:tasks.workunit.client.0.smithi049.stdout:1/255: read d0/d4/dd/de/d1e/d2e/f2c [3604821,119308] 0 2024-05-15T15:54:01.206 INFO:tasks.workunit.client.0.smithi049.stdout:0/197: dread dc/d13/d14/d2f/f24 [0,4194304] 0 2024-05-15T15:54:01.210 INFO:tasks.workunit.client.0.smithi049.stdout:8/178: getdents d2/de 0 2024-05-15T15:54:01.213 INFO:tasks.workunit.client.0.smithi049.stdout:7/184: sync 2024-05-15T15:54:01.213 INFO:tasks.workunit.client.0.smithi049.stdout:2/201: sync 2024-05-15T15:54:01.214 INFO:tasks.workunit.client.0.smithi049.stdout:7/185: stat d0/d2f 0 2024-05-15T15:54:01.215 INFO:tasks.workunit.client.0.smithi049.stdout:7/186: fdatasync d0/d6/d23/f30 0 2024-05-15T15:54:01.216 INFO:tasks.workunit.client.0.smithi049.stdout:0/198: mknod dc/c3e 0 2024-05-15T15:54:01.219 INFO:tasks.workunit.client.0.smithi049.stdout:2/202: dread d3/db/f1a [0,4194304] 0 2024-05-15T15:54:01.222 INFO:tasks.workunit.client.0.smithi049.stdout:8/179: symlink d2/l37 0 2024-05-15T15:54:01.232 INFO:tasks.workunit.client.1.smithi144.stdout:8/106: creat d2/d24/f25 x:0 0 0 2024-05-15T15:54:01.232 INFO:tasks.workunit.client.0.smithi049.stdout:1/256: getdents d0/d4/dd/de/d1e/d16 0 2024-05-15T15:54:01.233 INFO:tasks.workunit.client.1.smithi144.stdout:8/107: write d2/d24/f25 [453189,111204] 0 2024-05-15T15:54:01.234 INFO:tasks.workunit.client.1.smithi144.stdout:8/108: truncate d2/de/d12/f23 1022077 0 2024-05-15T15:54:01.235 INFO:tasks.workunit.client.1.smithi144.stdout:1/112: sync 2024-05-15T15:54:01.235 INFO:tasks.workunit.client.0.smithi049.stdout:2/203: symlink d3/db/d1d/l38 0 2024-05-15T15:54:01.236 INFO:tasks.workunit.client.1.smithi144.stdout:1/113: fdatasync d7/d9/fe 0 2024-05-15T15:54:01.237 INFO:tasks.workunit.client.1.smithi144.stdout:9/93: rename d0/da/d10/df to d0/d8/d21/d29 0 2024-05-15T15:54:01.237 INFO:tasks.workunit.client.1.smithi144.stdout:9/94: chown d0/d8/d1b/l1d 220 1 2024-05-15T15:54:01.243 INFO:tasks.workunit.client.0.smithi049.stdout:0/199: creat dc/d13/d3d/f3f x:0 0 0 2024-05-15T15:54:01.245 INFO:tasks.workunit.client.1.smithi144.stdout:2/87: unlink d3/d7/c12 0 2024-05-15T15:54:01.246 INFO:tasks.workunit.client.1.smithi144.stdout:2/88: chown d3/d7/la 91497 1 2024-05-15T15:54:01.247 INFO:tasks.workunit.client.0.smithi049.stdout:1/257: creat d0/d4/dd/de/d1e/d2e/d3e/f62 x:0 0 0 2024-05-15T15:54:01.248 INFO:tasks.workunit.client.0.smithi049.stdout:1/258: dread - d0/d4/dd/de/d1e/d16/f50 zero size 2024-05-15T15:54:01.249 INFO:tasks.workunit.client.0.smithi049.stdout:2/204: creat d3/da/d15/f39 x:0 0 0 2024-05-15T15:54:01.250 INFO:tasks.workunit.client.0.smithi049.stdout:2/205: chown d3/db/f12 581934 1 2024-05-15T15:54:01.253 INFO:tasks.workunit.client.0.smithi049.stdout:8/180: rename d2/d14/c28 to d2/de/d25/c38 0 2024-05-15T15:54:01.257 INFO:tasks.workunit.client.0.smithi049.stdout:2/206: dwrite d3/db/f12 [0,4194304] 0 2024-05-15T15:54:01.258 INFO:tasks.workunit.client.0.smithi049.stdout:2/207: stat d3/d8/ld 0 2024-05-15T15:54:01.259 INFO:tasks.workunit.client.1.smithi144.stdout:4/126: rename d6/da/db/d1d/l26 to d6/da/db/d1d/l28 0 2024-05-15T15:54:01.259 INFO:tasks.workunit.client.0.smithi049.stdout:7/187: dwrite d0/d6/d1d/d36/f2c [0,4194304] 0 2024-05-15T15:54:01.262 INFO:tasks.workunit.client.1.smithi144.stdout:6/92: mknod d7/db/d11/c24 0 2024-05-15T15:54:01.267 INFO:tasks.workunit.client.1.smithi144.stdout:6/93: dwrite f1 [0,4194304] 0 2024-05-15T15:54:01.271 INFO:tasks.workunit.client.0.smithi049.stdout:1/259: mkdir d0/d4/d63 0 2024-05-15T15:54:01.272 INFO:tasks.workunit.client.1.smithi144.stdout:5/114: read d0/f16 [410392,80372] 0 2024-05-15T15:54:01.275 INFO:tasks.workunit.client.0.smithi049.stdout:8/181: truncate d2/f3 6300040 0 2024-05-15T15:54:01.276 INFO:tasks.workunit.client.0.smithi049.stdout:0/200: creat dc/f40 x:0 0 0 2024-05-15T15:54:01.279 INFO:tasks.workunit.client.1.smithi144.stdout:5/115: dwrite d0/dc/f1b [0,4194304] 0 2024-05-15T15:54:01.281 INFO:tasks.workunit.client.1.smithi144.stdout:9/95: unlink d0/d8/d21/d29/l1e 0 2024-05-15T15:54:01.282 INFO:tasks.workunit.client.1.smithi144.stdout:0/96: mkdir d4/df/d1f/d24 0 2024-05-15T15:54:01.284 INFO:tasks.workunit.client.1.smithi144.stdout:3/94: creat d0/d11/d1e/f22 x:0 0 0 2024-05-15T15:54:01.286 INFO:tasks.workunit.client.0.smithi049.stdout:8/182: fdatasync d2/f22 0 2024-05-15T15:54:01.290 INFO:tasks.workunit.client.0.smithi049.stdout:8/183: rename d2/de/d27/f2d to d2/de/d27/f39 0 2024-05-15T15:54:01.291 INFO:tasks.workunit.client.0.smithi049.stdout:8/184: readlink d2/d18/l20 0 2024-05-15T15:54:01.291 INFO:tasks.workunit.client.1.smithi144.stdout:7/128: symlink d4/df/l28 0 2024-05-15T15:54:01.293 INFO:tasks.workunit.client.0.smithi049.stdout:2/208: write d3/db/f30 [453711,54597] 0 2024-05-15T15:54:01.296 INFO:tasks.workunit.client.0.smithi049.stdout:1/260: write d0/f5e [3166694,67918] 0 2024-05-15T15:54:01.297 INFO:tasks.workunit.client.1.smithi144.stdout:1/114: mkdir d7/d8/d1e 0 2024-05-15T15:54:01.297 INFO:tasks.workunit.client.0.smithi049.stdout:1/261: write d0/f19 [5414267,42610] 0 2024-05-15T15:54:01.299 INFO:tasks.workunit.client.0.smithi049.stdout:1/262: write d0/d4/dd/de/d1e/f27 [1377081,106699] 0 2024-05-15T15:54:01.300 INFO:tasks.workunit.client.0.smithi049.stdout:7/188: truncate d0/d6/d1d/f27 1468064 0 2024-05-15T15:54:01.301 INFO:tasks.workunit.client.1.smithi144.stdout:5/116: mknod d0/dc/d19/c2a 0 2024-05-15T15:54:01.303 INFO:tasks.workunit.client.1.smithi144.stdout:1/115: dwrite d7/df/f15 [0,4194304] 0 2024-05-15T15:54:01.304 INFO:tasks.workunit.client.1.smithi144.stdout:9/96: creat d0/d8/d21/f2a x:0 0 0 2024-05-15T15:54:01.307 INFO:tasks.workunit.client.0.smithi049.stdout:7/189: rename d0/d6/da/c1e to d0/d6/da/c38 0 2024-05-15T15:54:01.309 INFO:tasks.workunit.client.1.smithi144.stdout:3/95: creat d0/d1/f23 x:0 0 0 2024-05-15T15:54:01.310 INFO:tasks.workunit.client.1.smithi144.stdout:3/96: write d0/d11/f14 [1394717,87506] 0 2024-05-15T15:54:01.311 INFO:tasks.workunit.client.0.smithi049.stdout:1/263: creat d0/d4/d63/f64 x:0 0 0 2024-05-15T15:54:01.312 INFO:tasks.workunit.client.1.smithi144.stdout:6/94: mknod d7/c25 0 2024-05-15T15:54:01.314 INFO:tasks.workunit.client.1.smithi144.stdout:8/109: truncate d2/f15 2931194 0 2024-05-15T15:54:01.314 INFO:tasks.workunit.client.0.smithi049.stdout:1/264: write d0/d4/dd/f18 [478466,102546] 0 2024-05-15T15:54:01.316 INFO:tasks.workunit.client.0.smithi049.stdout:8/185: dread d2/f3 [0,4194304] 0 2024-05-15T15:54:01.317 INFO:tasks.workunit.client.1.smithi144.stdout:6/95: dread f1 [0,4194304] 0 2024-05-15T15:54:01.318 INFO:tasks.workunit.client.0.smithi049.stdout:7/190: dwrite d0/d6/f14 [4194304,4194304] 0 2024-05-15T15:54:01.321 INFO:tasks.workunit.client.1.smithi144.stdout:2/89: creat d3/f1c x:0 0 0 2024-05-15T15:54:01.321 INFO:tasks.workunit.client.1.smithi144.stdout:0/97: symlink d4/dc/d1d/l25 0 2024-05-15T15:54:01.321 INFO:tasks.workunit.client.0.smithi049.stdout:8/186: mknod d2/de/d25/c3a 0 2024-05-15T15:54:01.322 INFO:tasks.workunit.client.1.smithi144.stdout:9/97: write d0/da/d10/f27 [368353,63542] 0 2024-05-15T15:54:01.324 INFO:tasks.workunit.client.1.smithi144.stdout:3/97: rename d0/d11/d1e to d0/d11/d24 0 2024-05-15T15:54:01.325 INFO:tasks.workunit.client.1.smithi144.stdout:0/98: write d4/d6/f8 [3694670,39249] 0 2024-05-15T15:54:01.326 INFO:tasks.workunit.client.1.smithi144.stdout:2/90: write d3/d7/f17 [262111,56268] 0 2024-05-15T15:54:01.327 INFO:tasks.workunit.client.1.smithi144.stdout:3/98: stat d0/d11 0 2024-05-15T15:54:01.328 INFO:tasks.workunit.client.1.smithi144.stdout:8/110: dwrite d2/de/d12/f1b [0,4194304] 0 2024-05-15T15:54:01.329 INFO:tasks.workunit.client.1.smithi144.stdout:6/96: mknod d7/db/d11/c26 0 2024-05-15T15:54:01.332 INFO:tasks.workunit.client.0.smithi049.stdout:8/187: chown d2/l5 271235 1 2024-05-15T15:54:01.333 INFO:tasks.workunit.client.1.smithi144.stdout:1/116: mkdir d7/d1f 0 2024-05-15T15:54:01.339 INFO:tasks.workunit.client.1.smithi144.stdout:5/117: write d0/f11 [1056504,17412] 0 2024-05-15T15:54:01.341 INFO:tasks.workunit.client.1.smithi144.stdout:9/98: dread d0/f18 [0,4194304] 0 2024-05-15T15:54:01.341 INFO:tasks.workunit.client.1.smithi144.stdout:0/99: mknod d4/d6/c26 0 2024-05-15T15:54:01.344 INFO:tasks.workunit.client.1.smithi144.stdout:9/99: dread d0/d8/f11 [0,4194304] 0 2024-05-15T15:54:01.344 INFO:tasks.workunit.client.1.smithi144.stdout:2/91: creat d3/d10/f1d x:0 0 0 2024-05-15T15:54:01.345 INFO:tasks.workunit.client.1.smithi144.stdout:4/127: sync 2024-05-15T15:54:01.346 INFO:tasks.workunit.client.1.smithi144.stdout:3/99: mknod d0/d11/d24/c25 0 2024-05-15T15:54:01.347 INFO:tasks.workunit.client.0.smithi049.stdout:8/188: getdents d2/de/d27 0 2024-05-15T15:54:01.350 INFO:tasks.workunit.client.1.smithi144.stdout:8/111: rmdir d2/de 39 2024-05-15T15:54:01.351 INFO:tasks.workunit.client.1.smithi144.stdout:2/92: dwrite d3/d10/f1d [0,4194304] 0 2024-05-15T15:54:01.351 INFO:tasks.workunit.client.0.smithi049.stdout:1/265: truncate d0/d4/f2d 4163810 0 2024-05-15T15:54:01.352 INFO:tasks.workunit.client.1.smithi144.stdout:3/100: dread d0/d1/f4 [0,4194304] 0 2024-05-15T15:54:01.352 INFO:tasks.workunit.client.1.smithi144.stdout:2/93: write d3/f1c [738986,15293] 0 2024-05-15T15:54:01.353 INFO:tasks.workunit.client.1.smithi144.stdout:1/117: symlink d7/d8/l20 0 2024-05-15T15:54:01.412 INFO:tasks.workunit.client.1.smithi144.stdout:7/129: sync 2024-05-15T15:54:01.434 INFO:tasks.workunit.client.1.smithi144.stdout:9/100: symlink d0/d8/l2b 0 2024-05-15T15:54:01.436 INFO:tasks.workunit.client.1.smithi144.stdout:4/128: creat d6/f29 x:0 0 0 2024-05-15T15:54:01.436 INFO:tasks.workunit.client.0.smithi049.stdout:8/189: getdents d2/de 0 2024-05-15T15:54:01.438 INFO:tasks.workunit.client.0.smithi049.stdout:7/191: read d0/d6/d1d/f27 [1137984,70940] 0 2024-05-15T15:54:01.438 INFO:tasks.workunit.client.1.smithi144.stdout:3/101: sync 2024-05-15T15:54:01.439 INFO:tasks.workunit.client.0.smithi049.stdout:7/192: truncate d0/fb 2636278 0 2024-05-15T15:54:01.440 INFO:tasks.workunit.client.0.smithi049.stdout:7/193: chown d0/d6/f1b 0 1 2024-05-15T15:54:01.440 INFO:tasks.workunit.client.0.smithi049.stdout:7/194: chown d0/l22 243406737 1 2024-05-15T15:54:01.441 INFO:tasks.workunit.client.1.smithi144.stdout:6/97: mknod d7/db/c27 0 2024-05-15T15:54:01.441 INFO:tasks.workunit.client.0.smithi049.stdout:7/195: read d0/fb [1513400,22387] 0 2024-05-15T15:54:01.442 INFO:tasks.workunit.client.1.smithi144.stdout:5/118: creat d0/dc/d19/d23/f2b x:0 0 0 2024-05-15T15:54:01.445 INFO:tasks.workunit.client.0.smithi049.stdout:8/190: rename d2/de/d27/c2e to d2/d14/c3b 0 2024-05-15T15:54:01.446 INFO:tasks.workunit.client.1.smithi144.stdout:3/102: dwrite d0/d1/f21 [0,4194304] 0 2024-05-15T15:54:01.448 INFO:tasks.workunit.client.1.smithi144.stdout:3/103: chown d0/d11/cf 2042181 1 2024-05-15T15:54:01.449 INFO:tasks.workunit.client.1.smithi144.stdout:0/100: mknod d4/df/d1f/d24/c27 0 2024-05-15T15:54:01.449 INFO:tasks.workunit.client.1.smithi144.stdout:6/98: dwrite d7/db/f1d [0,4194304] 0 2024-05-15T15:54:01.450 INFO:tasks.workunit.client.1.smithi144.stdout:0/101: write d4/d6/f11 [967229,69028] 0 2024-05-15T15:54:01.450 INFO:tasks.workunit.client.1.smithi144.stdout:4/129: mkdir d6/d2a 0 2024-05-15T15:54:01.451 INFO:tasks.workunit.client.1.smithi144.stdout:4/130: dread - d6/dd/f22 zero size 2024-05-15T15:54:01.453 INFO:tasks.workunit.client.0.smithi049.stdout:8/191: dwrite d2/d14/f2f [0,4194304] 0 2024-05-15T15:54:01.453 INFO:tasks.workunit.client.1.smithi144.stdout:4/131: truncate d6/da/f1b 797380 0 2024-05-15T15:54:01.453 INFO:tasks.workunit.client.1.smithi144.stdout:4/132: fdatasync d6/dd/f22 0 2024-05-15T15:54:01.454 INFO:tasks.workunit.client.1.smithi144.stdout:8/112: rename d2/de/d12/f23 to d2/de/f26 0 2024-05-15T15:54:01.455 INFO:tasks.workunit.client.1.smithi144.stdout:8/113: read d2/f3 [1652283,129105] 0 2024-05-15T15:54:01.456 INFO:tasks.workunit.client.0.smithi049.stdout:7/196: dwrite d0/d6/da/f1c [4194304,4194304] 0 2024-05-15T15:54:01.456 INFO:tasks.workunit.client.1.smithi144.stdout:8/114: fdatasync d2/fd 0 2024-05-15T15:54:01.459 INFO:tasks.workunit.client.1.smithi144.stdout:5/119: symlink d0/dc/d1f/l2c 0 2024-05-15T15:54:01.460 INFO:tasks.workunit.client.0.smithi049.stdout:8/192: read d2/de/d25/f26 [3416954,67870] 0 2024-05-15T15:54:01.462 INFO:tasks.workunit.client.1.smithi144.stdout:5/120: write d0/dc/d19/d23/f2b [441206,28586] 0 2024-05-15T15:54:01.465 INFO:tasks.workunit.client.1.smithi144.stdout:8/115: dwrite d2/fd [0,4194304] 0 2024-05-15T15:54:01.465 INFO:tasks.workunit.client.1.smithi144.stdout:1/118: mknod d7/d1f/c21 0 2024-05-15T15:54:01.466 INFO:tasks.workunit.client.0.smithi049.stdout:7/197: mkdir d0/d6/d1d/d36/d39 0 2024-05-15T15:54:01.471 INFO:tasks.workunit.client.1.smithi144.stdout:3/104: unlink d0/f6 0 2024-05-15T15:54:01.473 INFO:tasks.workunit.client.0.smithi049.stdout:7/198: mknod d0/d6/d23/c3a 0 2024-05-15T15:54:01.473 INFO:tasks.workunit.client.0.smithi049.stdout:7/199: chown d0/d6/d23 61852 1 2024-05-15T15:54:01.477 INFO:tasks.workunit.client.1.smithi144.stdout:6/99: readlink d7/db/d11/l16 0 2024-05-15T15:54:01.479 INFO:tasks.workunit.client.0.smithi049.stdout:7/200: dread d0/d6/f1b [4194304,4194304] 0 2024-05-15T15:54:01.481 INFO:tasks.workunit.client.1.smithi144.stdout:0/102: unlink d4/d6/f8 0 2024-05-15T15:54:01.482 INFO:tasks.workunit.client.1.smithi144.stdout:0/103: read d4/d6/f11 [419774,90844] 0 2024-05-15T15:54:01.483 INFO:tasks.workunit.client.0.smithi049.stdout:7/201: symlink d0/d6/d1d/d36/l3b 0 2024-05-15T15:54:01.484 INFO:tasks.workunit.client.1.smithi144.stdout:4/133: mkdir d6/dd/d2b 0 2024-05-15T15:54:01.484 INFO:tasks.workunit.client.0.smithi049.stdout:8/193: getdents d2/d14/d1b 0 2024-05-15T15:54:01.487 INFO:tasks.workunit.client.0.smithi049.stdout:7/202: unlink d0/d6/da/f1c 0 2024-05-15T15:54:01.488 INFO:tasks.workunit.client.1.smithi144.stdout:0/104: dwrite d4/dc/f18 [0,4194304] 0 2024-05-15T15:54:01.488 INFO:tasks.workunit.client.0.smithi049.stdout:8/194: write d2/d14/d1b/f36 [989082,11812] 0 2024-05-15T15:54:01.488 INFO:tasks.workunit.client.0.smithi049.stdout:7/203: write d0/d6/f29 [76246,9933] 0 2024-05-15T15:54:01.491 INFO:tasks.workunit.client.1.smithi144.stdout:1/119: unlink d7/l11 0 2024-05-15T15:54:01.492 INFO:tasks.workunit.client.0.smithi049.stdout:8/195: creat d2/d18/f3c x:0 0 0 2024-05-15T15:54:01.494 INFO:tasks.workunit.client.1.smithi144.stdout:3/105: creat d0/d11/f26 x:0 0 0 2024-05-15T15:54:01.494 INFO:tasks.workunit.client.1.smithi144.stdout:9/101: link d0/f18 d0/da/d10/f2c 0 2024-05-15T15:54:01.495 INFO:tasks.workunit.client.1.smithi144.stdout:6/100: mkdir d7/d1e/d28 0 2024-05-15T15:54:01.495 INFO:tasks.workunit.client.1.smithi144.stdout:9/102: chown d0/d8/d21/d29 11515213 1 2024-05-15T15:54:01.496 INFO:tasks.workunit.client.1.smithi144.stdout:3/106: write d0/d11/d24/f22 [295330,31063] 0 2024-05-15T15:54:01.497 INFO:tasks.workunit.client.1.smithi144.stdout:3/107: stat d0/c18 0 2024-05-15T15:54:01.499 INFO:tasks.workunit.client.1.smithi144.stdout:8/116: dwrite d2/de/f26 [0,4194304] 0 2024-05-15T15:54:01.500 INFO:tasks.workunit.client.1.smithi144.stdout:8/117: write d2/fd [3975050,108514] 0 2024-05-15T15:54:01.538 INFO:tasks.workunit.client.1.smithi144.stdout:4/134: sync 2024-05-15T15:54:01.546 INFO:tasks.workunit.client.0.smithi049.stdout:7/204: creat d0/d6/f3c x:0 0 0 2024-05-15T15:54:01.551 INFO:tasks.workunit.client.0.smithi049.stdout:8/196: creat d2/f3d x:0 0 0 2024-05-15T15:54:01.552 INFO:tasks.workunit.client.1.smithi144.stdout:7/130: dwrite d4/df/f20 [0,4194304] 0 2024-05-15T15:54:01.553 INFO:tasks.workunit.client.1.smithi144.stdout:0/105: symlink d4/df/l28 0 2024-05-15T15:54:01.554 INFO:tasks.workunit.client.1.smithi144.stdout:2/94: getdents d3/d10 0 2024-05-15T15:54:01.554 INFO:tasks.workunit.client.1.smithi144.stdout:2/95: chown d3/d10/d15/l1b 37794 1 2024-05-15T15:54:01.555 INFO:tasks.workunit.client.0.smithi049.stdout:7/205: rename d0/d6/d1d/d36 to d0/d6/da/d3d 0 2024-05-15T15:54:01.555 INFO:tasks.workunit.client.1.smithi144.stdout:2/96: fdatasync d3/f5 0 2024-05-15T15:54:01.555 INFO:tasks.workunit.client.0.smithi049.stdout:8/197: mkdir d2/d14/d1d/d3e 0 2024-05-15T15:54:01.560 INFO:tasks.workunit.client.1.smithi144.stdout:7/131: dwrite d4/df/f20 [0,4194304] 0 2024-05-15T15:54:01.560 INFO:tasks.workunit.client.0.smithi049.stdout:7/206: creat d0/d2f/f3e x:0 0 0 2024-05-15T15:54:01.564 INFO:tasks.workunit.client.1.smithi144.stdout:9/103: mkdir d0/d8/d21/d29/d2d 0 2024-05-15T15:54:01.565 INFO:tasks.workunit.client.0.smithi049.stdout:7/207: rename d0/f19 to d0/d34/f3f 0 2024-05-15T15:54:01.570 INFO:tasks.workunit.client.0.smithi049.stdout:8/198: unlink d2/ca 0 2024-05-15T15:54:01.571 INFO:tasks.workunit.client.0.smithi049.stdout:7/208: dread d0/d6/f1b [4194304,4194304] 0 2024-05-15T15:54:01.571 INFO:tasks.workunit.client.1.smithi144.stdout:3/108: fdatasync d0/d1/d17/f19 0 2024-05-15T15:54:01.572 INFO:tasks.workunit.client.1.smithi144.stdout:3/109: read d0/d1/f5 [2876436,92427] 0 2024-05-15T15:54:01.574 INFO:tasks.workunit.client.1.smithi144.stdout:8/118: creat d2/de/d12/f27 x:0 0 0 2024-05-15T15:54:01.575 INFO:tasks.workunit.client.0.smithi049.stdout:8/199: stat d2/de/d25/f32 0 2024-05-15T15:54:01.576 INFO:tasks.workunit.client.1.smithi144.stdout:4/135: rmdir d6/dd/d15 39 2024-05-15T15:54:01.579 INFO:tasks.workunit.client.0.smithi049.stdout:8/200: mknod d2/d14/d1b/c3f 0 2024-05-15T15:54:01.581 INFO:tasks.workunit.client.0.smithi049.stdout:7/209: link d0/f2e d0/d6/d23/f40 0 2024-05-15T15:54:01.582 INFO:tasks.workunit.client.1.smithi144.stdout:8/119: dwrite d2/de/fc [0,4194304] 0 2024-05-15T15:54:01.584 INFO:tasks.workunit.client.1.smithi144.stdout:1/120: dwrite d7/df/fb [0,4194304] 0 2024-05-15T15:54:01.585 INFO:tasks.workunit.client.1.smithi144.stdout:0/106: rename d4/dc/ld to d4/d6/l29 0 2024-05-15T15:54:01.585 INFO:tasks.workunit.client.0.smithi049.stdout:7/210: dread d0/d6/f1b [4194304,4194304] 0 2024-05-15T15:54:01.587 INFO:tasks.workunit.client.0.smithi049.stdout:8/201: unlink d2/d14/f2f 0 2024-05-15T15:54:01.589 INFO:tasks.workunit.client.0.smithi049.stdout:8/202: dread d2/f2c [0,4194304] 0 2024-05-15T15:54:01.591 INFO:tasks.workunit.client.1.smithi144.stdout:2/97: write d3/f9 [749829,126211] 0 2024-05-15T15:54:01.593 INFO:tasks.workunit.client.1.smithi144.stdout:1/121: dwrite d7/df/f10 [0,4194304] 0 2024-05-15T15:54:01.595 INFO:tasks.workunit.client.0.smithi049.stdout:8/203: creat d2/de/d27/f40 x:0 0 0 2024-05-15T15:54:01.599 INFO:tasks.workunit.client.1.smithi144.stdout:7/132: truncate d4/fc 611969 0 2024-05-15T15:54:01.600 INFO:tasks.workunit.client.1.smithi144.stdout:2/98: dwrite d3/d7/f17 [0,4194304] 0 2024-05-15T15:54:01.600 INFO:tasks.workunit.client.1.smithi144.stdout:2/99: chown d3/d10 226325 1 2024-05-15T15:54:01.601 INFO:tasks.workunit.client.0.smithi049.stdout:8/204: mknod d2/de/c41 0 2024-05-15T15:54:01.604 INFO:tasks.workunit.client.1.smithi144.stdout:9/104: mknod d0/da/d10/d19/c2e 0 2024-05-15T15:54:01.609 INFO:tasks.workunit.client.0.smithi049.stdout:5/199: sync 2024-05-15T15:54:01.609 INFO:tasks.workunit.client.0.smithi049.stdout:4/275: sync 2024-05-15T15:54:01.609 INFO:tasks.workunit.client.1.smithi144.stdout:2/100: dwrite d3/d10/f1d [0,4194304] 0 2024-05-15T15:54:01.610 INFO:tasks.workunit.client.0.smithi049.stdout:8/205: dwrite d2/d18/f3c [0,4194304] 0 2024-05-15T15:54:01.611 INFO:tasks.workunit.client.0.smithi049.stdout:4/276: write d18/f22 [1828521,31223] 0 2024-05-15T15:54:01.612 INFO:tasks.workunit.client.1.smithi144.stdout:3/110: symlink d0/l27 0 2024-05-15T15:54:01.614 INFO:tasks.workunit.client.0.smithi049.stdout:4/277: truncate d18/f2e 4918007 0 2024-05-15T15:54:01.617 INFO:tasks.workunit.client.0.smithi049.stdout:4/278: dread - d18/d1e/d2a/d3b/f51 zero size 2024-05-15T15:54:01.617 INFO:tasks.workunit.client.0.smithi049.stdout:5/200: creat dc/d29/f3c x:0 0 0 2024-05-15T15:54:01.618 INFO:tasks.workunit.client.1.smithi144.stdout:8/120: symlink d2/d1a/l28 0 2024-05-15T15:54:01.619 INFO:tasks.workunit.client.0.smithi049.stdout:8/206: mknod d2/de/d25/c42 0 2024-05-15T15:54:01.620 INFO:tasks.workunit.client.0.smithi049.stdout:7/211: dwrite d0/f11 [4194304,4194304] 0 2024-05-15T15:54:01.621 INFO:tasks.workunit.client.1.smithi144.stdout:0/107: creat d4/df/d1f/d24/f2a x:0 0 0 2024-05-15T15:54:01.622 INFO:tasks.workunit.client.0.smithi049.stdout:7/212: stat d0/d6/l25 0 2024-05-15T15:54:01.622 INFO:tasks.workunit.client.0.smithi049.stdout:8/207: write d2/d14/d1b/f36 [3442397,71882] 0 2024-05-15T15:54:01.624 INFO:tasks.workunit.client.0.smithi049.stdout:4/279: creat d18/d1e/d2a/d36/f59 x:0 0 0 2024-05-15T15:54:01.627 INFO:tasks.workunit.client.1.smithi144.stdout:5/121: getdents d0 0 2024-05-15T15:54:01.628 INFO:tasks.workunit.client.1.smithi144.stdout:1/122: mkdir d7/d9/d22 0 2024-05-15T15:54:01.629 INFO:tasks.workunit.client.1.smithi144.stdout:1/123: write d7/d9/fe [915237,74150] 0 2024-05-15T15:54:01.629 INFO:tasks.workunit.client.1.smithi144.stdout:6/101: creat d7/db/f29 x:0 0 0 2024-05-15T15:54:01.630 INFO:tasks.workunit.client.1.smithi144.stdout:6/102: stat d7/l12 0 2024-05-15T15:54:01.633 INFO:tasks.workunit.client.1.smithi144.stdout:6/103: dread d7/db/d11/f1b [0,4194304] 0 2024-05-15T15:54:01.633 INFO:tasks.workunit.client.0.smithi049.stdout:4/280: rename d18/f2e to d18/d1e/d2a/d50/f5a 0 2024-05-15T15:54:01.634 INFO:tasks.workunit.client.1.smithi144.stdout:9/105: creat d0/d8/d21/f2f x:0 0 0 2024-05-15T15:54:01.636 INFO:tasks.workunit.client.0.smithi049.stdout:5/201: creat dc/dd/f3d x:0 0 0 2024-05-15T15:54:01.636 INFO:tasks.workunit.client.1.smithi144.stdout:9/106: write d0/da/d10/f27 [437141,82419] 0 2024-05-15T15:54:01.636 INFO:tasks.workunit.client.0.smithi049.stdout:5/202: fsync dc/f16 0 2024-05-15T15:54:01.638 INFO:tasks.workunit.client.0.smithi049.stdout:5/203: read dc/f11 [748509,125862] 0 2024-05-15T15:54:01.639 INFO:tasks.workunit.client.0.smithi049.stdout:5/204: readlink dc/dd/d10/l27 0 2024-05-15T15:54:01.639 INFO:tasks.workunit.client.0.smithi049.stdout:5/205: readlink l3 0 2024-05-15T15:54:01.640 INFO:tasks.workunit.client.1.smithi144.stdout:2/101: rmdir d3/d7 39 2024-05-15T15:54:01.641 INFO:tasks.workunit.client.1.smithi144.stdout:2/102: write d3/f5 [4465774,24329] 0 2024-05-15T15:54:01.642 INFO:tasks.workunit.client.1.smithi144.stdout:3/111: mknod d0/d1/c28 0 2024-05-15T15:54:01.643 INFO:tasks.workunit.client.1.smithi144.stdout:3/112: write d0/d11/d24/f22 [320557,111234] 0 2024-05-15T15:54:01.644 INFO:tasks.workunit.client.0.smithi049.stdout:7/213: creat d0/f41 x:0 0 0 2024-05-15T15:54:01.646 INFO:tasks.workunit.client.0.smithi049.stdout:5/206: dwrite f9 [0,4194304] 0 2024-05-15T15:54:01.647 INFO:tasks.workunit.client.0.smithi049.stdout:5/207: dread - dc/d29/d2c/f3a zero size 2024-05-15T15:54:01.652 INFO:tasks.workunit.client.0.smithi049.stdout:7/214: mknod d0/d6/da/c42 0 2024-05-15T15:54:01.652 INFO:tasks.workunit.client.0.smithi049.stdout:7/215: stat d0/f41 0 2024-05-15T15:54:01.655 INFO:tasks.workunit.client.1.smithi144.stdout:7/133: creat d4/df/d25/f29 x:0 0 0 2024-05-15T15:54:01.657 INFO:tasks.workunit.client.0.smithi049.stdout:7/216: rmdir d0/d6/d23 39 2024-05-15T15:54:01.659 INFO:tasks.workunit.client.0.smithi049.stdout:7/217: truncate d0/d34/f35 4456018 0 2024-05-15T15:54:01.660 INFO:tasks.workunit.client.1.smithi144.stdout:6/104: chown d7/db/c1c 1932563275 1 2024-05-15T15:54:01.660 INFO:tasks.workunit.client.1.smithi144.stdout:6/105: dread - d7/db/d11/f1f zero size 2024-05-15T15:54:01.660 INFO:tasks.workunit.client.1.smithi144.stdout:6/106: readlink d7/db/d11/l16 0 2024-05-15T15:54:01.661 INFO:tasks.workunit.client.0.smithi049.stdout:7/218: chown d0/d6/da/d3d/f2c 493924327 1 2024-05-15T15:54:01.662 INFO:tasks.workunit.client.0.smithi049.stdout:8/208: dwrite d2/f3 [0,4194304] 0 2024-05-15T15:54:01.663 INFO:tasks.workunit.client.0.smithi049.stdout:4/281: dread d18/d21/d2b/f2c [0,4194304] 0 2024-05-15T15:54:01.663 INFO:tasks.workunit.client.0.smithi049.stdout:8/209: write f0 [1792092,9504] 0 2024-05-15T15:54:01.664 INFO:tasks.workunit.client.1.smithi144.stdout:9/107: mkdir d0/da/d10/d19/d30 0 2024-05-15T15:54:01.665 INFO:tasks.workunit.client.1.smithi144.stdout:9/108: write d0/da/d10/fd [3512354,21867] 0 2024-05-15T15:54:01.666 INFO:tasks.workunit.client.0.smithi049.stdout:5/208: creat dc/f3e x:0 0 0 2024-05-15T15:54:01.670 INFO:tasks.workunit.client.1.smithi144.stdout:3/113: mknod d0/d11/d24/c29 0 2024-05-15T15:54:01.672 INFO:tasks.workunit.client.1.smithi144.stdout:4/136: rmdir d6/d2a 0 2024-05-15T15:54:01.673 INFO:tasks.workunit.client.0.smithi049.stdout:5/209: mknod dc/d29/d2c/c3f 0 2024-05-15T15:54:01.675 INFO:tasks.workunit.client.1.smithi144.stdout:8/121: symlink d2/l29 0 2024-05-15T15:54:01.676 INFO:tasks.workunit.client.0.smithi049.stdout:4/282: symlink d18/d21/l5b 0 2024-05-15T15:54:01.676 INFO:tasks.workunit.client.1.smithi144.stdout:5/122: unlink d0/f9 0 2024-05-15T15:54:01.678 INFO:tasks.workunit.client.1.smithi144.stdout:1/124: truncate f6 713988 0 2024-05-15T15:54:01.678 INFO:tasks.workunit.client.1.smithi144.stdout:1/125: fdatasync d7/df/f15 0 2024-05-15T15:54:01.679 INFO:tasks.workunit.client.0.smithi049.stdout:5/210: symlink dc/d29/l40 0 2024-05-15T15:54:01.681 INFO:tasks.workunit.client.0.smithi049.stdout:4/283: unlink d18/f37 0 2024-05-15T15:54:01.683 INFO:tasks.workunit.client.0.smithi049.stdout:5/211: mknod dc/dd/d10/c41 0 2024-05-15T15:54:01.689 INFO:tasks.workunit.client.0.smithi049.stdout:8/210: write d2/f22 [2993707,71238] 0 2024-05-15T15:54:01.690 INFO:tasks.workunit.client.0.smithi049.stdout:4/284: dwrite d18/d1e/d2a/f4f [0,4194304] 0 2024-05-15T15:54:01.692 INFO:tasks.workunit.client.1.smithi144.stdout:2/103: mknod d3/c1e 0 2024-05-15T15:54:01.692 INFO:tasks.workunit.client.0.smithi049.stdout:4/285: read - d18/d21/f56 zero size 2024-05-15T15:54:01.693 INFO:tasks.workunit.client.0.smithi049.stdout:5/212: dwrite f9 [0,4194304] 0 2024-05-15T15:54:01.697 INFO:tasks.workunit.client.1.smithi144.stdout:2/104: dwrite d3/f5 [0,4194304] 0 2024-05-15T15:54:01.697 INFO:tasks.workunit.client.1.smithi144.stdout:3/114: dread d0/d1/d17/f19 [0,4194304] 0 2024-05-15T15:54:01.697 INFO:tasks.workunit.client.0.smithi049.stdout:8/211: fsync d2/de/d27/f39 0 2024-05-15T15:54:01.698 INFO:tasks.workunit.client.1.smithi144.stdout:0/108: sync 2024-05-15T15:54:01.699 INFO:tasks.workunit.client.0.smithi049.stdout:8/212: chown d2/d14/d1b 3721 1 2024-05-15T15:54:01.699 INFO:tasks.workunit.client.1.smithi144.stdout:4/137: fsync d6/da/ff 0 2024-05-15T15:54:01.699 INFO:tasks.workunit.client.1.smithi144.stdout:0/109: chown d4/df/l13 62 1 2024-05-15T15:54:01.701 INFO:tasks.workunit.client.1.smithi144.stdout:0/110: write d4/df/d1f/d24/f2a [703134,103196] 0 2024-05-15T15:54:01.703 INFO:tasks.workunit.client.1.smithi144.stdout:8/122: unlink d2/l7 0 2024-05-15T15:54:01.706 INFO:tasks.workunit.client.0.smithi049.stdout:4/286: rename d18/d1e/f48 to d18/d1e/d2a/d50/d54/f5c 0 2024-05-15T15:54:01.710 INFO:tasks.workunit.client.1.smithi144.stdout:5/123: rename d0/dc/d19/d23 to d0/dc/d24/d28/d2d 0 2024-05-15T15:54:01.714 INFO:tasks.workunit.client.1.smithi144.stdout:1/126: readlink d7/d8/l1a 0 2024-05-15T15:54:01.715 INFO:tasks.workunit.client.1.smithi144.stdout:1/127: write d7/df/f10 [3961506,84288] 0 2024-05-15T15:54:01.717 INFO:tasks.workunit.client.1.smithi144.stdout:6/107: mknod d7/c2a 0 2024-05-15T15:54:01.717 INFO:tasks.workunit.client.0.smithi049.stdout:5/213: write dc/dd/d10/f1e [1269050,93602] 0 2024-05-15T15:54:01.719 INFO:tasks.workunit.client.1.smithi144.stdout:9/109: mkdir d0/da/d16/d31 0 2024-05-15T15:54:01.721 INFO:tasks.workunit.client.1.smithi144.stdout:1/128: dwrite d7/df/f10 [0,4194304] 0 2024-05-15T15:54:01.722 INFO:tasks.workunit.client.1.smithi144.stdout:1/129: fsync d7/d9/f19 0 2024-05-15T15:54:01.723 INFO:tasks.workunit.client.1.smithi144.stdout:6/108: dread d7/db/f1d [0,4194304] 0 2024-05-15T15:54:01.723 INFO:tasks.workunit.client.0.smithi049.stdout:5/214: fsync dc/dd/d10/f2a 0 2024-05-15T15:54:01.725 INFO:tasks.workunit.client.0.smithi049.stdout:5/215: write dc/f3e [733576,83211] 0 2024-05-15T15:54:01.726 INFO:tasks.workunit.client.0.smithi049.stdout:5/216: read - dc/dd/f14 zero size 2024-05-15T15:54:01.726 INFO:tasks.workunit.client.0.smithi049.stdout:8/213: creat d2/d14/f43 x:0 0 0 2024-05-15T15:54:01.727 INFO:tasks.workunit.client.1.smithi144.stdout:3/115: truncate d0/d1/f5 3457478 0 2024-05-15T15:54:01.728 INFO:tasks.workunit.client.0.smithi049.stdout:5/217: write dc/dd/f13 [544355,115772] 0 2024-05-15T15:54:01.728 INFO:tasks.workunit.client.1.smithi144.stdout:3/116: chown d0/d1/c3 132 1 2024-05-15T15:54:01.732 INFO:tasks.workunit.client.1.smithi144.stdout:4/138: write d6/dd/d15/f1c [705391,89627] 0 2024-05-15T15:54:01.733 INFO:tasks.workunit.client.0.smithi049.stdout:8/214: creat d2/de/f44 x:0 0 0 2024-05-15T15:54:01.735 INFO:tasks.workunit.client.0.smithi049.stdout:4/287: mkdir d18/d1e/d2a/d36/d5d 0 2024-05-15T15:54:01.736 INFO:tasks.workunit.client.0.smithi049.stdout:5/218: dwrite dc/dd/d10/f1a [0,4194304] 0 2024-05-15T15:54:01.737 INFO:tasks.workunit.client.0.smithi049.stdout:5/219: stat dc/d29/l40 0 2024-05-15T15:54:01.737 INFO:tasks.workunit.client.0.smithi049.stdout:4/288: dread - d18/d1e/d2a/d3b/f51 zero size 2024-05-15T15:54:01.737 INFO:tasks.workunit.client.0.smithi049.stdout:5/220: stat f0 0 2024-05-15T15:54:01.738 INFO:tasks.workunit.client.1.smithi144.stdout:1/130: dread f1 [0,4194304] 0 2024-05-15T15:54:01.738 INFO:tasks.workunit.client.0.smithi049.stdout:5/221: dread - dc/d29/d2c/f36 zero size 2024-05-15T15:54:01.738 INFO:tasks.workunit.client.0.smithi049.stdout:5/222: chown dc/f16 14460777 1 2024-05-15T15:54:01.739 INFO:tasks.workunit.client.0.smithi049.stdout:8/215: rmdir d2/d14/d1d 39 2024-05-15T15:54:01.744 INFO:tasks.workunit.client.0.smithi049.stdout:4/289: dwrite d18/d1e/d2a/d3b/f44 [0,4194304] 0 2024-05-15T15:54:01.746 INFO:tasks.workunit.client.0.smithi049.stdout:4/290: dread - d18/d1e/d2a/d3b/f57 zero size 2024-05-15T15:54:01.748 INFO:tasks.workunit.client.1.smithi144.stdout:7/134: dwrite d4/df/f1f [0,4194304] 0 2024-05-15T15:54:01.749 INFO:tasks.workunit.client.1.smithi144.stdout:7/135: chown d4/fc 496 1 2024-05-15T15:54:01.750 INFO:tasks.workunit.client.0.smithi049.stdout:4/291: truncate d18/d1e/d2a/d36/f4b 548150 0 2024-05-15T15:54:01.755 INFO:tasks.workunit.client.0.smithi049.stdout:4/292: fsync d18/d1e/d2a/d50/d54/f5c 0 2024-05-15T15:54:01.758 INFO:tasks.workunit.client.1.smithi144.stdout:2/105: creat d3/d7/f1f x:0 0 0 2024-05-15T15:54:01.759 INFO:tasks.workunit.client.0.smithi049.stdout:8/216: getdents d2/d14/d1b 0 2024-05-15T15:54:01.759 INFO:tasks.workunit.client.1.smithi144.stdout:2/106: write d3/d7/f17 [2919815,40952] 0 2024-05-15T15:54:01.760 INFO:tasks.workunit.client.0.smithi049.stdout:8/217: read - d2/f3d zero size 2024-05-15T15:54:01.761 INFO:tasks.workunit.client.1.smithi144.stdout:3/117: truncate d0/d1/d17/f19 190107 0 2024-05-15T15:54:01.763 INFO:tasks.workunit.client.0.smithi049.stdout:8/218: read - d2/de/d27/f35 zero size 2024-05-15T15:54:01.763 INFO:tasks.workunit.client.1.smithi144.stdout:2/107: dread d3/d10/f11 [0,4194304] 0 2024-05-15T15:54:01.763 INFO:tasks.workunit.client.1.smithi144.stdout:2/108: chown d3 440 1 2024-05-15T15:54:01.764 INFO:tasks.workunit.client.0.smithi049.stdout:4/293: fsync d18/d1e/d2a/d3b/f44 0 2024-05-15T15:54:01.765 INFO:tasks.workunit.client.1.smithi144.stdout:2/109: fdatasync d3/d7/f17 0 2024-05-15T15:54:01.765 INFO:tasks.workunit.client.0.smithi049.stdout:5/223: dwrite dc/f19 [4194304,4194304] 0 2024-05-15T15:54:01.765 INFO:tasks.workunit.client.1.smithi144.stdout:2/110: write d3/d7/f18 [555494,8893] 0 2024-05-15T15:54:01.765 INFO:tasks.workunit.client.1.smithi144.stdout:9/110: write d0/f7 [2582997,77742] 0 2024-05-15T15:54:01.766 INFO:tasks.workunit.client.1.smithi144.stdout:2/111: fdatasync d3/d10/f11 0 2024-05-15T15:54:01.767 INFO:tasks.workunit.client.1.smithi144.stdout:4/139: symlink d6/d11/l2c 0 2024-05-15T15:54:01.768 INFO:tasks.workunit.client.1.smithi144.stdout:2/112: write d3/f5 [1805727,69827] 0 2024-05-15T15:54:01.768 INFO:tasks.workunit.client.1.smithi144.stdout:4/140: readlink d6/l8 0 2024-05-15T15:54:01.773 INFO:tasks.workunit.client.0.smithi049.stdout:4/294: creat d18/d1e/d2a/d50/f5e x:0 0 0 2024-05-15T15:54:01.774 INFO:tasks.workunit.client.0.smithi049.stdout:5/224: dwrite dc/dd/f3d [0,4194304] 0 2024-05-15T15:54:01.775 INFO:tasks.workunit.client.1.smithi144.stdout:8/123: mknod d2/c2a 0 2024-05-15T15:54:01.776 INFO:tasks.workunit.client.0.smithi049.stdout:5/225: truncate dc/dd/f14 331158 0 2024-05-15T15:54:01.777 INFO:tasks.workunit.client.0.smithi049.stdout:5/226: write dc/f11 [6439381,60185] 0 2024-05-15T15:54:01.782 INFO:tasks.workunit.client.0.smithi049.stdout:5/227: dread f9 [0,4194304] 0 2024-05-15T15:54:01.782 INFO:tasks.workunit.client.1.smithi144.stdout:5/124: fsync d0/f7 0 2024-05-15T15:54:01.783 INFO:tasks.workunit.client.1.smithi144.stdout:5/125: dread d0/dc/d12/f13 [0,4194304] 0 2024-05-15T15:54:01.784 INFO:tasks.workunit.client.0.smithi049.stdout:4/295: symlink d18/d1e/d2a/d36/l5f 0 2024-05-15T15:54:01.784 INFO:tasks.workunit.client.1.smithi144.stdout:5/126: readlink d0/dc/d1f/d29/l20 0 2024-05-15T15:54:01.790 INFO:tasks.workunit.client.0.smithi049.stdout:5/228: mkdir dc/d29/d2c/d42 0 2024-05-15T15:54:01.792 INFO:tasks.workunit.client.0.smithi049.stdout:6/170: sync 2024-05-15T15:54:01.792 INFO:tasks.workunit.client.0.smithi049.stdout:3/245: sync 2024-05-15T15:54:01.792 INFO:tasks.workunit.client.0.smithi049.stdout:9/185: sync 2024-05-15T15:54:01.792 INFO:tasks.workunit.client.0.smithi049.stdout:0/201: sync 2024-05-15T15:54:01.793 INFO:tasks.workunit.client.0.smithi049.stdout:8/219: dwrite d2/d14/f1e [0,4194304] 0 2024-05-15T15:54:01.793 INFO:tasks.workunit.client.1.smithi144.stdout:7/136: mkdir d4/df/d25/d2a 0 2024-05-15T15:54:01.794 INFO:tasks.workunit.client.1.smithi144.stdout:7/137: chown d4/df/d25 42825 1 2024-05-15T15:54:01.795 INFO:tasks.workunit.client.0.smithi049.stdout:5/229: write dc/dd/d10/f1e [2382764,120100] 0 2024-05-15T15:54:01.795 INFO:tasks.workunit.client.0.smithi049.stdout:3/246: chown d12/d14/f20 31950 1 2024-05-15T15:54:01.797 INFO:tasks.workunit.client.0.smithi049.stdout:8/220: chown d2/d14/f1e 26283 1 2024-05-15T15:54:01.797 INFO:tasks.workunit.client.0.smithi049.stdout:4/296: unlink d18/d1e/d2a/d3b/f44 0 2024-05-15T15:54:01.798 INFO:tasks.workunit.client.0.smithi049.stdout:3/247: write f8 [4855232,12492] 0 2024-05-15T15:54:01.806 INFO:tasks.workunit.client.0.smithi049.stdout:5/230: dread dc/f1f [0,4194304] 0 2024-05-15T15:54:01.808 INFO:tasks.workunit.client.0.smithi049.stdout:6/171: fdatasync dd/d17/f1d 0 2024-05-15T15:54:01.810 INFO:tasks.workunit.client.0.smithi049.stdout:8/221: dwrite d2/de/d27/f39 [0,4194304] 0 2024-05-15T15:54:01.817 INFO:tasks.workunit.client.0.smithi049.stdout:0/202: rename dc/d13/f3c to dc/d13/d14/d2a/f41 0 2024-05-15T15:54:01.818 INFO:tasks.workunit.client.0.smithi049.stdout:6/172: dwrite f0 [0,4194304] 0 2024-05-15T15:54:01.819 INFO:tasks.workunit.client.1.smithi144.stdout:2/113: mkdir d3/d7/d20 0 2024-05-15T15:54:01.820 INFO:tasks.workunit.client.1.smithi144.stdout:2/114: chown d3/d7/la 47 1 2024-05-15T15:54:01.824 INFO:tasks.workunit.client.0.smithi049.stdout:3/248: creat d12/d30/d42/f46 x:0 0 0 2024-05-15T15:54:01.825 INFO:tasks.workunit.client.1.smithi144.stdout:2/115: dwrite d3/f9 [0,4194304] 0 2024-05-15T15:54:01.827 INFO:tasks.workunit.client.1.smithi144.stdout:8/124: creat d2/d24/f2b x:0 0 0 2024-05-15T15:54:01.827 INFO:tasks.workunit.client.0.smithi049.stdout:6/173: dwrite dd/d17/f21 [0,4194304] 0 2024-05-15T15:54:01.832 INFO:tasks.workunit.client.1.smithi144.stdout:5/127: mknod d0/dc/d24/d28/c2e 0 2024-05-15T15:54:01.833 INFO:tasks.workunit.client.1.smithi144.stdout:5/128: truncate d0/f1a 356703 0 2024-05-15T15:54:01.840 INFO:tasks.workunit.client.0.smithi049.stdout:2/209: sync 2024-05-15T15:54:01.840 INFO:tasks.workunit.client.0.smithi049.stdout:7/219: sync 2024-05-15T15:54:01.840 INFO:tasks.workunit.client.0.smithi049.stdout:1/266: sync 2024-05-15T15:54:01.841 INFO:tasks.workunit.client.1.smithi144.stdout:9/111: write d0/d8/f11 [82203,59474] 0 2024-05-15T15:54:01.842 INFO:tasks.workunit.client.0.smithi049.stdout:7/220: chown d0/d6/da 4978760 1 2024-05-15T15:54:01.843 INFO:tasks.workunit.client.0.smithi049.stdout:2/210: dread d3/f11 [0,4194304] 0 2024-05-15T15:54:01.844 INFO:tasks.workunit.client.0.smithi049.stdout:4/297: mknod d18/d21/c60 0 2024-05-15T15:54:01.845 INFO:tasks.workunit.client.0.smithi049.stdout:2/211: chown d3/d8/f36 1 1 2024-05-15T15:54:01.845 INFO:tasks.workunit.client.0.smithi049.stdout:8/222: chown d2/d14/d1d/c30 10111 1 2024-05-15T15:54:01.846 INFO:tasks.workunit.client.0.smithi049.stdout:8/223: chown d2/l37 51105 1 2024-05-15T15:54:01.847 INFO:tasks.workunit.client.1.smithi144.stdout:6/109: rename l4 to d7/l2b 0 2024-05-15T15:54:01.847 INFO:tasks.workunit.client.0.smithi049.stdout:2/212: truncate d3/db/d1d/f20 709503 0 2024-05-15T15:54:01.853 INFO:tasks.workunit.client.1.smithi144.stdout:3/118: link d0/f15 d0/d11/f2a 0 2024-05-15T15:54:01.855 INFO:tasks.workunit.client.0.smithi049.stdout:0/203: dwrite dc/d13/d14/d2f/f24 [0,4194304] 0 2024-05-15T15:54:01.856 INFO:tasks.workunit.client.0.smithi049.stdout:6/174: rmdir dd/d17 39 2024-05-15T15:54:01.858 INFO:tasks.workunit.client.1.smithi144.stdout:4/141: mkdir d6/dd/d2d 0 2024-05-15T15:54:01.859 INFO:tasks.workunit.client.1.smithi144.stdout:1/131: dwrite f6 [0,4194304] 0 2024-05-15T15:54:01.861 INFO:tasks.workunit.client.1.smithi144.stdout:0/111: getdents d4/df/d1f/d24 0 2024-05-15T15:54:01.863 INFO:tasks.workunit.client.1.smithi144.stdout:2/116: symlink d3/d7/l21 0 2024-05-15T15:54:01.863 INFO:tasks.workunit.client.1.smithi144.stdout:3/119: dwrite d0/d11/d24/f22 [0,4194304] 0 2024-05-15T15:54:01.863 INFO:tasks.workunit.client.1.smithi144.stdout:0/112: dread - d4/f22 zero size 2024-05-15T15:54:01.864 INFO:tasks.workunit.client.1.smithi144.stdout:8/125: creat d2/de/d12/f2c x:0 0 0 2024-05-15T15:54:01.864 INFO:tasks.workunit.client.0.smithi049.stdout:2/213: symlink d3/da/l3a 0 2024-05-15T15:54:01.865 INFO:tasks.workunit.client.1.smithi144.stdout:2/117: chown d3/d7/f17 22 1 2024-05-15T15:54:01.865 INFO:tasks.workunit.client.0.smithi049.stdout:2/214: write d3/db/f23 [4986038,100548] 0 2024-05-15T15:54:01.867 INFO:tasks.workunit.client.0.smithi049.stdout:0/204: symlink dc/d13/d14/l42 0 2024-05-15T15:54:01.869 INFO:tasks.workunit.client.1.smithi144.stdout:5/129: rmdir d0/dc/d1f 39 2024-05-15T15:54:01.871 INFO:tasks.workunit.client.1.smithi144.stdout:7/138: symlink d4/l2b 0 2024-05-15T15:54:01.872 INFO:tasks.workunit.client.0.smithi049.stdout:9/186: getdents d0 0 2024-05-15T15:54:01.874 INFO:tasks.workunit.client.1.smithi144.stdout:5/130: read d0/dc/f1b [3365232,26921] 0 2024-05-15T15:54:01.875 INFO:tasks.workunit.client.0.smithi049.stdout:1/267: mkdir d0/d4/dd/d5d/d65 0 2024-05-15T15:54:01.875 INFO:tasks.workunit.client.1.smithi144.stdout:7/139: chown d4/df/d25/d2a 73355 1 2024-05-15T15:54:01.877 INFO:tasks.workunit.client.0.smithi049.stdout:1/268: write d0/d4/dd/de/d48/f4a [700365,125200] 0 2024-05-15T15:54:01.877 INFO:tasks.workunit.client.0.smithi049.stdout:4/298: mknod d18/d21/c61 0 2024-05-15T15:54:01.879 INFO:tasks.workunit.client.0.smithi049.stdout:8/224: mkdir d2/d14/d45 0 2024-05-15T15:54:01.879 INFO:tasks.workunit.client.0.smithi049.stdout:8/225: stat d2/de/c41 0 2024-05-15T15:54:01.883 INFO:tasks.workunit.client.0.smithi049.stdout:3/249: creat d12/f47 x:0 0 0 2024-05-15T15:54:01.883 INFO:tasks.workunit.client.1.smithi144.stdout:2/118: dwrite d3/f5 [0,4194304] 0 2024-05-15T15:54:01.884 INFO:tasks.workunit.client.1.smithi144.stdout:0/113: dwrite d4/df/d1f/d24/f2a [0,4194304] 0 2024-05-15T15:54:01.885 INFO:tasks.workunit.client.1.smithi144.stdout:3/120: read d0/d1/f5 [3450830,82541] 0 2024-05-15T15:54:01.887 INFO:tasks.workunit.client.1.smithi144.stdout:8/126: dwrite d2/f16 [0,4194304] 0 2024-05-15T15:54:01.888 INFO:tasks.workunit.client.0.smithi049.stdout:0/205: creat dc/d13/d3d/f43 x:0 0 0 2024-05-15T15:54:01.889 INFO:tasks.workunit.client.1.smithi144.stdout:8/127: stat d2/de/c20 0 2024-05-15T15:54:01.890 INFO:tasks.workunit.client.1.smithi144.stdout:6/110: dwrite d7/db/f1d [0,4194304] 0 2024-05-15T15:54:01.890 INFO:tasks.workunit.client.1.smithi144.stdout:0/114: write d4/df/f16 [1048171,124700] 0 2024-05-15T15:54:01.890 INFO:tasks.workunit.client.0.smithi049.stdout:0/206: write dc/d13/d3d/f43 [371540,52300] 0 2024-05-15T15:54:01.892 INFO:tasks.workunit.client.1.smithi144.stdout:8/128: fdatasync d2/d24/f25 0 2024-05-15T15:54:01.892 INFO:tasks.workunit.client.0.smithi049.stdout:0/207: chown dc/d13/d14/d25/d31 122 1 2024-05-15T15:54:01.893 INFO:tasks.workunit.client.0.smithi049.stdout:0/208: fdatasync dc/d13/d14/f2e 0 2024-05-15T15:54:01.894 INFO:tasks.workunit.client.1.smithi144.stdout:9/112: link d0/da/d10/fd d0/d8/d21/d29/d2d/f32 0 2024-05-15T15:54:01.895 INFO:tasks.workunit.client.0.smithi049.stdout:6/175: mkdir dd/d17/d24/d2b 0 2024-05-15T15:54:01.895 INFO:tasks.workunit.client.1.smithi144.stdout:6/111: write d7/db/f18 [1200538,94089] 0 2024-05-15T15:54:01.895 INFO:tasks.workunit.client.0.smithi049.stdout:6/176: truncate f4 1174765 0 2024-05-15T15:54:01.898 INFO:tasks.workunit.client.1.smithi144.stdout:9/113: chown d0/da/d16 12299210 1 2024-05-15T15:54:01.901 INFO:tasks.workunit.client.1.smithi144.stdout:3/121: dwrite d0/d11/f26 [0,4194304] 0 2024-05-15T15:54:01.903 INFO:tasks.workunit.client.1.smithi144.stdout:0/115: dread d4/d6/fb [0,4194304] 0 2024-05-15T15:54:01.904 INFO:tasks.workunit.client.1.smithi144.stdout:7/140: dread d4/f13 [0,4194304] 0 2024-05-15T15:54:01.907 INFO:tasks.workunit.client.0.smithi049.stdout:1/269: creat d0/d4/dd/de/d36/f66 x:0 0 0 2024-05-15T15:54:01.909 INFO:tasks.workunit.client.1.smithi144.stdout:1/132: mknod d7/d8/d1e/c23 0 2024-05-15T15:54:01.909 INFO:tasks.workunit.client.0.smithi049.stdout:1/270: write d0/d4/dd/de/f17 [4137223,68224] 0 2024-05-15T15:54:01.910 INFO:tasks.workunit.client.1.smithi144.stdout:1/133: write d7/df/fb [1824679,98583] 0 2024-05-15T15:54:01.911 INFO:tasks.workunit.client.0.smithi049.stdout:4/299: chown d18/d1e/f23 162 1 2024-05-15T15:54:01.911 INFO:tasks.workunit.client.0.smithi049.stdout:1/271: stat d0/d4/dd/de/d1e/f15 0 2024-05-15T15:54:01.911 INFO:tasks.workunit.client.1.smithi144.stdout:1/134: read d7/d8/f12 [1926586,28976] 0 2024-05-15T15:54:01.912 INFO:tasks.workunit.client.0.smithi049.stdout:2/215: write d3/d8/fe [992558,130283] 0 2024-05-15T15:54:01.913 INFO:tasks.workunit.client.1.smithi144.stdout:1/135: write d7/df/fb [4807568,46908] 0 2024-05-15T15:54:01.913 INFO:tasks.workunit.client.0.smithi049.stdout:8/226: mknod d2/d14/d1d/c46 0 2024-05-15T15:54:01.915 INFO:tasks.workunit.client.0.smithi049.stdout:1/272: truncate d0/d4/dd/f5a 451093 0 2024-05-15T15:54:01.916 INFO:tasks.workunit.client.0.smithi049.stdout:1/273: fdatasync d0/d4/f39 0 2024-05-15T15:54:01.918 INFO:tasks.workunit.client.0.smithi049.stdout:1/274: chown d0/d4/dd/de/d1e/d16/f22 47593 1 2024-05-15T15:54:01.925 INFO:tasks.workunit.client.1.smithi144.stdout:4/142: truncate f0 8506554 0 2024-05-15T15:54:01.925 INFO:tasks.workunit.client.1.smithi144.stdout:4/143: dread - d6/dd/f22 zero size 2024-05-15T15:54:01.926 INFO:tasks.workunit.client.1.smithi144.stdout:8/129: mkdir d2/d24/d2d 0 2024-05-15T15:54:01.927 INFO:tasks.workunit.client.0.smithi049.stdout:1/275: dwrite d0/d4/dd/de/d1e/d2e/f41 [0,4194304] 0 2024-05-15T15:54:01.929 INFO:tasks.workunit.client.0.smithi049.stdout:4/300: dread d18/f1b [0,4194304] 0 2024-05-15T15:54:01.931 INFO:tasks.workunit.client.0.smithi049.stdout:7/221: getdents d0/d34 0 2024-05-15T15:54:01.931 INFO:tasks.workunit.client.0.smithi049.stdout:7/222: chown d0/d34 27 1 2024-05-15T15:54:01.937 INFO:tasks.workunit.client.1.smithi144.stdout:3/122: symlink d0/l2b 0 2024-05-15T15:54:01.938 INFO:tasks.workunit.client.0.smithi049.stdout:7/223: dwrite d0/d6/f18 [0,4194304] 0 2024-05-15T15:54:01.941 INFO:tasks.workunit.client.0.smithi049.stdout:6/177: write fb [2072895,85545] 0 2024-05-15T15:54:01.942 INFO:tasks.workunit.client.0.smithi049.stdout:6/178: write dd/d17/d24/f27 [531754,28940] 0 2024-05-15T15:54:01.943 INFO:tasks.workunit.client.1.smithi144.stdout:0/116: read d4/f5 [82883,4361] 0 2024-05-15T15:54:01.943 INFO:tasks.workunit.client.1.smithi144.stdout:7/141: mknod d4/df/d25/c2c 0 2024-05-15T15:54:01.944 INFO:tasks.workunit.client.0.smithi049.stdout:2/216: mkdir d3/db/d1d/d1e/d2b/d3b 0 2024-05-15T15:54:01.945 INFO:tasks.workunit.client.0.smithi049.stdout:9/187: truncate d0/d8/d11/f16 193729 0 2024-05-15T15:54:01.946 INFO:tasks.workunit.client.0.smithi049.stdout:8/227: creat d2/d18/f47 x:0 0 0 2024-05-15T15:54:01.950 INFO:tasks.workunit.client.1.smithi144.stdout:2/119: mkdir d3/d7/d20/d22 0 2024-05-15T15:54:01.950 INFO:tasks.workunit.client.1.smithi144.stdout:2/120: fdatasync d3/d10/f1d 0 2024-05-15T15:54:01.951 INFO:tasks.workunit.client.1.smithi144.stdout:2/121: write d3/f5 [1697641,127873] 0 2024-05-15T15:54:01.953 INFO:tasks.workunit.client.1.smithi144.stdout:1/136: rename d7/d1f to d7/df/d24 0 2024-05-15T15:54:01.954 INFO:tasks.workunit.client.1.smithi144.stdout:1/137: write f6 [2957704,61275] 0 2024-05-15T15:54:01.955 INFO:tasks.workunit.client.0.smithi049.stdout:1/276: symlink d0/d4/dd/de/d1e/d2e/d3e/l67 0 2024-05-15T15:54:01.955 INFO:tasks.workunit.client.1.smithi144.stdout:1/138: chown d7/df/d24/c21 339 1 2024-05-15T15:54:01.957 INFO:tasks.workunit.client.0.smithi049.stdout:4/301: mknod d18/d1e/d2a/d36/c62 0 2024-05-15T15:54:01.962 INFO:tasks.workunit.client.1.smithi144.stdout:9/114: link d0/d8/d21/f2f d0/da/d10/d19/f33 0 2024-05-15T15:54:01.966 INFO:tasks.workunit.client.1.smithi144.stdout:3/123: creat d0/f2c x:0 0 0 2024-05-15T15:54:01.970 INFO:tasks.workunit.client.0.smithi049.stdout:1/277: dread d0/d4/f24 [0,4194304] 0 2024-05-15T15:54:01.970 INFO:tasks.workunit.client.0.smithi049.stdout:4/302: dread d18/f28 [0,4194304] 0 2024-05-15T15:54:01.972 INFO:tasks.workunit.client.0.smithi049.stdout:1/278: write d0/d4/dd/de/d1e/d2e/f41 [803788,90260] 0 2024-05-15T15:54:01.973 INFO:tasks.workunit.client.0.smithi049.stdout:9/188: symlink d0/d8/d12/l3f 0 2024-05-15T15:54:01.975 INFO:tasks.workunit.client.0.smithi049.stdout:8/228: mkdir d2/de/d27/d48 0 2024-05-15T15:54:01.975 INFO:tasks.workunit.client.0.smithi049.stdout:8/229: readlink d2/l37 0 2024-05-15T15:54:01.976 INFO:tasks.workunit.client.1.smithi144.stdout:0/117: dread f2 [0,4194304] 0 2024-05-15T15:54:01.983 INFO:tasks.workunit.client.0.smithi049.stdout:8/230: dwrite d2/d14/f1e [0,4194304] 0 2024-05-15T15:54:01.985 INFO:tasks.workunit.client.0.smithi049.stdout:4/303: dread d18/f1a [0,4194304] 0 2024-05-15T15:54:01.985 INFO:tasks.workunit.client.0.smithi049.stdout:8/231: dread - d2/de/d27/f35 zero size 2024-05-15T15:54:01.988 INFO:tasks.workunit.client.0.smithi049.stdout:7/224: dwrite d0/fb [0,4194304] 0 2024-05-15T15:54:01.988 INFO:tasks.workunit.client.0.smithi049.stdout:6/179: mknod dd/c2c 0 2024-05-15T15:54:01.990 INFO:tasks.workunit.client.1.smithi144.stdout:3/124: rename d0/d1/c3 to d0/d1/c2d 0 2024-05-15T15:54:01.992 INFO:tasks.workunit.client.1.smithi144.stdout:7/142: mknod d4/df/c2d 0 2024-05-15T15:54:01.992 INFO:tasks.workunit.client.0.smithi049.stdout:6/180: dread fb [0,4194304] 0 2024-05-15T15:54:01.996 INFO:tasks.workunit.client.0.smithi049.stdout:6/181: dread f0 [0,4194304] 0 2024-05-15T15:54:01.997 INFO:tasks.workunit.client.0.smithi049.stdout:9/189: mkdir d0/d8/d1b/d40 0 2024-05-15T15:54:02.000 INFO:tasks.workunit.client.0.smithi049.stdout:6/182: dread f2 [0,4194304] 0 2024-05-15T15:54:02.001 INFO:tasks.workunit.client.1.smithi144.stdout:1/139: rename d7/df/fb to d7/d8/d1e/f25 0 2024-05-15T15:54:02.002 INFO:tasks.workunit.client.1.smithi144.stdout:1/140: chown d7/df/d24 127 1 2024-05-15T15:54:02.002 INFO:tasks.workunit.client.1.smithi144.stdout:1/141: read d7/df/f10 [684681,96054] 0 2024-05-15T15:54:02.005 INFO:tasks.workunit.client.1.smithi144.stdout:4/144: truncate d6/dd/d15/f1c 2388803 0 2024-05-15T15:54:02.005 INFO:tasks.workunit.client.0.smithi049.stdout:0/209: getdents dc/d13/d14/d25/d31 0 2024-05-15T15:54:02.010 INFO:tasks.workunit.client.1.smithi144.stdout:9/115: rename d0/da to d0/d8/d21/d29/d34 0 2024-05-15T15:54:02.011 INFO:tasks.workunit.client.1.smithi144.stdout:9/116: chown d0/d8/l9 1787303 1 2024-05-15T15:54:02.011 INFO:tasks.workunit.client.1.smithi144.stdout:9/117: rename d0 to d0/d8/d21/d29/d2d/d35 22 2024-05-15T15:54:02.012 INFO:tasks.workunit.client.1.smithi144.stdout:5/131: sync 2024-05-15T15:54:02.013 INFO:tasks.workunit.client.0.smithi049.stdout:2/217: creat d3/db/d1d/d1e/f3c x:0 0 0 2024-05-15T15:54:02.014 INFO:tasks.workunit.client.0.smithi049.stdout:7/225: chown d0/l3 78 1 2024-05-15T15:54:02.019 INFO:tasks.workunit.client.1.smithi144.stdout:2/122: getdents d3/d10/d15 0 2024-05-15T15:54:02.020 INFO:tasks.workunit.client.1.smithi144.stdout:2/123: readlink d3/d10/l14 0 2024-05-15T15:54:02.020 INFO:tasks.workunit.client.1.smithi144.stdout:7/143: dwrite d4/fe [0,4194304] 0 2024-05-15T15:54:02.021 INFO:tasks.workunit.client.0.smithi049.stdout:4/304: truncate d18/d21/d2b/f2c 2131470 0 2024-05-15T15:54:02.024 INFO:tasks.workunit.client.0.smithi049.stdout:1/279: creat d0/d4/dd/de/d48/d5c/f68 x:0 0 0 2024-05-15T15:54:02.024 INFO:tasks.workunit.client.1.smithi144.stdout:9/118: read d0/f4 [1443391,61847] 0 2024-05-15T15:54:02.028 INFO:tasks.workunit.client.0.smithi049.stdout:9/190: creat d0/f41 x:0 0 0 2024-05-15T15:54:02.028 INFO:tasks.workunit.client.1.smithi144.stdout:1/142: mknod d7/c26 0 2024-05-15T15:54:02.030 INFO:tasks.workunit.client.0.smithi049.stdout:6/183: rename dd/l12 to dd/d26/l2d 0 2024-05-15T15:54:02.031 INFO:tasks.workunit.client.0.smithi049.stdout:6/184: write f4 [1559867,42992] 0 2024-05-15T15:54:02.032 INFO:tasks.workunit.client.1.smithi144.stdout:4/145: mkdir d6/dd/d2e 0 2024-05-15T15:54:02.038 INFO:tasks.workunit.client.0.smithi049.stdout:7/226: chown d0/d6/da/c38 841493 1 2024-05-15T15:54:02.038 INFO:tasks.workunit.client.1.smithi144.stdout:5/132: write d0/f7 [969558,105984] 0 2024-05-15T15:54:02.038 INFO:tasks.workunit.client.0.smithi049.stdout:7/227: dread - d0/d2f/f3e zero size 2024-05-15T15:54:02.039 INFO:tasks.workunit.client.1.smithi144.stdout:5/133: fdatasync d0/f7 0 2024-05-15T15:54:02.039 INFO:tasks.workunit.client.1.smithi144.stdout:5/134: fsync d0/f7 0 2024-05-15T15:54:02.041 INFO:tasks.workunit.client.1.smithi144.stdout:9/119: mknod d0/d8/c36 0 2024-05-15T15:54:02.045 INFO:tasks.workunit.client.0.smithi049.stdout:1/280: symlink d0/d4/dd/de/d1e/d2e/d3e/l69 0 2024-05-15T15:54:02.046 INFO:tasks.workunit.client.1.smithi144.stdout:2/124: creat d3/d7/d20/d22/f23 x:0 0 0 2024-05-15T15:54:02.050 INFO:tasks.workunit.client.1.smithi144.stdout:7/144: symlink d4/df/l2e 0 2024-05-15T15:54:02.054 INFO:tasks.workunit.client.0.smithi049.stdout:6/185: creat dd/d26/f2e x:0 0 0 2024-05-15T15:54:02.055 INFO:tasks.workunit.client.1.smithi144.stdout:4/146: dwrite d6/da/ff [4194304,4194304] 0 2024-05-15T15:54:02.055 INFO:tasks.workunit.client.1.smithi144.stdout:4/147: fdatasync d6/dd/f22 0 2024-05-15T15:54:02.056 INFO:tasks.workunit.client.0.smithi049.stdout:2/218: mknod d3/db/d1d/d1e/d2b/d2e/c3d 0 2024-05-15T15:54:02.057 INFO:tasks.workunit.client.0.smithi049.stdout:2/219: fdatasync d3/da/d15/f27 0 2024-05-15T15:54:02.058 INFO:tasks.workunit.client.1.smithi144.stdout:2/125: creat d3/d7/d20/d22/f24 x:0 0 0 2024-05-15T15:54:02.058 INFO:tasks.workunit.client.0.smithi049.stdout:8/232: rmdir d2/de/d27/d48 0 2024-05-15T15:54:02.060 INFO:tasks.workunit.client.0.smithi049.stdout:8/233: read d2/fb [1244172,39186] 0 2024-05-15T15:54:02.061 INFO:tasks.workunit.client.1.smithi144.stdout:5/135: mkdir d0/dc/d2f 0 2024-05-15T15:54:02.061 INFO:tasks.workunit.client.0.smithi049.stdout:8/234: chown d2/de 2086134 1 2024-05-15T15:54:02.062 INFO:tasks.workunit.client.0.smithi049.stdout:0/210: dwrite dc/d13/f17 [0,4194304] 0 2024-05-15T15:54:02.062 INFO:tasks.workunit.client.1.smithi144.stdout:5/136: readlink d0/ld 0 2024-05-15T15:54:02.062 INFO:tasks.workunit.client.1.smithi144.stdout:5/137: fdatasync d0/f1a 0 2024-05-15T15:54:02.065 INFO:tasks.workunit.client.1.smithi144.stdout:4/148: creat d6/da/f2f x:0 0 0 2024-05-15T15:54:02.067 INFO:tasks.workunit.client.0.smithi049.stdout:2/220: dwrite d3/db/f12 [0,4194304] 0 2024-05-15T15:54:02.075 INFO:tasks.workunit.client.1.smithi144.stdout:2/126: rename d3/c1e to d3/d7/d20/d22/c25 0 2024-05-15T15:54:02.081 INFO:tasks.workunit.client.1.smithi144.stdout:2/127: dwrite d3/d7/d20/d22/f23 [0,4194304] 0 2024-05-15T15:54:02.083 INFO:tasks.workunit.client.1.smithi144.stdout:6/112: sync 2024-05-15T15:54:02.098 INFO:tasks.workunit.client.0.smithi049.stdout:0/211: dread f4 [0,4194304] 0 2024-05-15T15:54:02.098 INFO:tasks.workunit.client.0.smithi049.stdout:4/305: chown d18/d21/d2b/f2c 5895955 1 2024-05-15T15:54:02.100 INFO:tasks.workunit.client.0.smithi049.stdout:5/231: sync 2024-05-15T15:54:02.133 INFO:tasks.workunit.client.0.smithi049.stdout:6/186: write dd/f16 [4009777,119187] 0 2024-05-15T15:54:02.138 INFO:tasks.workunit.client.0.smithi049.stdout:8/235: symlink d2/d14/d1b/l49 0 2024-05-15T15:54:02.139 INFO:tasks.workunit.client.0.smithi049.stdout:1/281: truncate d0/d4/dd/de/f13 315022 0 2024-05-15T15:54:02.139 INFO:tasks.workunit.client.1.smithi144.stdout:8/130: sync 2024-05-15T15:54:02.139 INFO:tasks.workunit.client.1.smithi144.stdout:3/125: sync 2024-05-15T15:54:02.139 INFO:tasks.workunit.client.1.smithi144.stdout:0/118: sync 2024-05-15T15:54:02.141 INFO:tasks.workunit.client.0.smithi049.stdout:6/187: dwrite f2 [4194304,4194304] 0 2024-05-15T15:54:02.141 INFO:tasks.workunit.client.1.smithi144.stdout:9/120: rmdir d0/d8/d21/d29/d34/d16/d31 0 2024-05-15T15:54:02.143 INFO:tasks.workunit.client.1.smithi144.stdout:9/121: write d0/d8/f11 [454020,48406] 0 2024-05-15T15:54:02.146 INFO:tasks.workunit.client.1.smithi144.stdout:1/143: getdents d7 0 2024-05-15T15:54:02.147 INFO:tasks.workunit.client.1.smithi144.stdout:8/131: dwrite d2/de/fc [0,4194304] 0 2024-05-15T15:54:02.152 INFO:tasks.workunit.client.1.smithi144.stdout:7/145: rmdir d4/df/d25/d2a 0 2024-05-15T15:54:02.157 INFO:tasks.workunit.client.1.smithi144.stdout:9/122: dwrite d0/d8/d21/d29/d34/d10/f17 [0,4194304] 0 2024-05-15T15:54:02.158 INFO:tasks.workunit.client.0.smithi049.stdout:3/250: sync 2024-05-15T15:54:02.158 INFO:tasks.workunit.client.0.smithi049.stdout:3/251: fdatasync f8 0 2024-05-15T15:54:02.159 INFO:tasks.workunit.client.1.smithi144.stdout:9/123: chown d0/f4 814631205 1 2024-05-15T15:54:02.160 INFO:tasks.workunit.client.1.smithi144.stdout:8/132: dwrite d2/d24/f2b [0,4194304] 0 2024-05-15T15:54:02.162 INFO:tasks.workunit.client.1.smithi144.stdout:8/133: write d2/d1a/f1d [332192,6696] 0 2024-05-15T15:54:02.163 INFO:tasks.workunit.client.0.smithi049.stdout:0/212: symlink dc/d13/d14/d25/d31/d38/l44 0 2024-05-15T15:54:02.163 INFO:tasks.workunit.client.1.smithi144.stdout:9/124: dread d0/d8/fb [0,4194304] 0 2024-05-15T15:54:02.163 INFO:tasks.workunit.client.0.smithi049.stdout:4/306: dwrite d18/d1e/f1f [0,4194304] 0 2024-05-15T15:54:02.164 INFO:tasks.workunit.client.1.smithi144.stdout:8/134: write d2/de/f10 [4919315,72458] 0 2024-05-15T15:54:02.166 INFO:tasks.workunit.client.0.smithi049.stdout:8/236: dread d2/fc [0,4194304] 0 2024-05-15T15:54:02.167 INFO:tasks.workunit.client.1.smithi144.stdout:5/138: symlink d0/dc/d1f/d29/l30 0 2024-05-15T15:54:02.173 INFO:tasks.workunit.client.1.smithi144.stdout:5/139: dwrite d0/f22 [0,4194304] 0 2024-05-15T15:54:02.174 INFO:tasks.workunit.client.1.smithi144.stdout:2/128: sync 2024-05-15T15:54:02.174 INFO:tasks.workunit.client.1.smithi144.stdout:5/140: chown d0/dc/d12/f18 11 1 2024-05-15T15:54:02.175 INFO:tasks.workunit.client.1.smithi144.stdout:2/129: fdatasync d3/d7/d20/d22/f23 0 2024-05-15T15:54:02.176 INFO:tasks.workunit.client.1.smithi144.stdout:2/130: write d3/d7/f18 [1100034,10842] 0 2024-05-15T15:54:02.178 INFO:tasks.workunit.client.1.smithi144.stdout:0/119: mkdir d4/df/d1f/d24/d2b 0 2024-05-15T15:54:02.180 INFO:tasks.workunit.client.1.smithi144.stdout:0/120: write d4/dc/f18 [2526903,17515] 0 2024-05-15T15:54:02.183 INFO:tasks.workunit.client.0.smithi049.stdout:9/191: getdents d0/d8/d1b/d40 0 2024-05-15T15:54:02.185 INFO:tasks.workunit.client.0.smithi049.stdout:9/192: chown d0/d8/d11/f1f 0 1 2024-05-15T15:54:02.193 INFO:tasks.workunit.client.1.smithi144.stdout:1/144: symlink d7/l27 0 2024-05-15T15:54:02.197 INFO:tasks.workunit.client.0.smithi049.stdout:0/213: dread f3 [0,4194304] 0 2024-05-15T15:54:02.198 INFO:tasks.workunit.client.0.smithi049.stdout:0/214: fsync dc/d13/d14/d2a/f2c 0 2024-05-15T15:54:02.199 INFO:tasks.workunit.client.0.smithi049.stdout:0/215: dread - dc/d13/d3d/f3f zero size 2024-05-15T15:54:02.202 INFO:tasks.workunit.client.0.smithi049.stdout:3/252: unlink d12/d14/c21 0 2024-05-15T15:54:02.212 INFO:tasks.workunit.client.0.smithi049.stdout:6/188: mknod dd/d17/d24/d2b/c2f 0 2024-05-15T15:54:02.216 INFO:tasks.workunit.client.0.smithi049.stdout:9/193: unlink d0/d8/d21/f30 0 2024-05-15T15:54:02.217 INFO:tasks.workunit.client.0.smithi049.stdout:9/194: chown d0/d8/fd 350 1 2024-05-15T15:54:02.218 INFO:tasks.workunit.client.0.smithi049.stdout:2/221: link d3/d8/lf d3/da/l3e 0 2024-05-15T15:54:02.222 INFO:tasks.workunit.client.0.smithi049.stdout:0/216: unlink dc/d13/d14/d2f/l21 0 2024-05-15T15:54:02.225 INFO:tasks.workunit.client.0.smithi049.stdout:3/253: creat d12/d30/d42/f48 x:0 0 0 2024-05-15T15:54:02.227 INFO:tasks.workunit.client.1.smithi144.stdout:2/131: creat d3/d10/d15/f26 x:0 0 0 2024-05-15T15:54:02.228 INFO:tasks.workunit.client.0.smithi049.stdout:6/189: dread dd/f15 [0,4194304] 0 2024-05-15T15:54:02.228 INFO:tasks.workunit.client.1.smithi144.stdout:2/132: fdatasync d3/f5 0 2024-05-15T15:54:02.228 INFO:tasks.workunit.client.0.smithi049.stdout:6/190: dread - dd/d17/f20 zero size 2024-05-15T15:54:02.228 INFO:tasks.workunit.client.1.smithi144.stdout:2/133: truncate d3/d7/f1a 392899 0 2024-05-15T15:54:02.229 INFO:tasks.workunit.client.1.smithi144.stdout:2/134: write d3/d7/f17 [1493068,35685] 0 2024-05-15T15:54:02.231 INFO:tasks.workunit.client.0.smithi049.stdout:4/307: mkdir d18/d1e/d63 0 2024-05-15T15:54:02.232 INFO:tasks.workunit.client.1.smithi144.stdout:2/135: dread d3/d10/f1d [0,4194304] 0 2024-05-15T15:54:02.233 INFO:tasks.workunit.client.0.smithi049.stdout:7/228: rename d0/d6/da/d3d/l3b to d0/d6/l43 0 2024-05-15T15:54:02.233 INFO:tasks.workunit.client.0.smithi049.stdout:7/229: dread - d0/f41 zero size 2024-05-15T15:54:02.236 INFO:tasks.workunit.client.1.smithi144.stdout:2/136: dwrite d3/d10/f1d [0,4194304] 0 2024-05-15T15:54:02.238 INFO:tasks.workunit.client.1.smithi144.stdout:0/121: write d4/d6/fb [3083757,26979] 0 2024-05-15T15:54:02.240 INFO:tasks.workunit.client.0.smithi049.stdout:7/230: dwrite d0/f4 [4194304,4194304] 0 2024-05-15T15:54:02.242 INFO:tasks.workunit.client.0.smithi049.stdout:7/231: truncate d0/d2f/f3e 1004991 0 2024-05-15T15:54:02.249 INFO:tasks.workunit.client.0.smithi049.stdout:2/222: creat d3/da/d15/f3f x:0 0 0 2024-05-15T15:54:02.252 INFO:tasks.workunit.client.0.smithi049.stdout:3/254: mknod d12/d30/d42/c49 0 2024-05-15T15:54:02.256 INFO:tasks.workunit.client.0.smithi049.stdout:6/191: symlink dd/d17/d24/l30 0 2024-05-15T15:54:02.258 INFO:tasks.workunit.client.1.smithi144.stdout:6/113: creat d7/db/f2c x:0 0 0 2024-05-15T15:54:02.258 INFO:tasks.workunit.client.0.smithi049.stdout:2/223: dwrite d3/db/d1d/d1e/f21 [0,4194304] 0 2024-05-15T15:54:02.262 INFO:tasks.workunit.client.1.smithi144.stdout:9/125: symlink d0/d8/d21/d29/d34/l37 0 2024-05-15T15:54:02.262 INFO:tasks.workunit.client.0.smithi049.stdout:3/255: dwrite d12/d26/f40 [0,4194304] 0 2024-05-15T15:54:02.264 INFO:tasks.workunit.client.1.smithi144.stdout:4/149: rename d6/da to d6/d11/d30 0 2024-05-15T15:54:02.266 INFO:tasks.workunit.client.0.smithi049.stdout:8/237: link d2/d18/c1f d2/d14/d1b/c4a 0 2024-05-15T15:54:02.270 INFO:tasks.workunit.client.1.smithi144.stdout:9/126: dwrite d0/d8/fb [0,4194304] 0 2024-05-15T15:54:02.395 INFO:tasks.workunit.client.1.smithi144.stdout:2/137: creat d3/d7/f27 x:0 0 0 2024-05-15T15:54:02.396 INFO:tasks.workunit.client.1.smithi144.stdout:2/138: write d3/d7/f1a [606840,48734] 0 2024-05-15T15:54:02.397 INFO:tasks.workunit.client.0.smithi049.stdout:6/192: symlink dd/d26/l31 0 2024-05-15T15:54:02.398 INFO:tasks.workunit.client.0.smithi049.stdout:6/193: fsync dd/f16 0 2024-05-15T15:54:02.399 INFO:tasks.workunit.client.1.smithi144.stdout:0/122: creat d4/df/d1f/f2c x:0 0 0 2024-05-15T15:54:02.399 INFO:tasks.workunit.client.0.smithi049.stdout:6/194: chown dd/d26/f2e 312 1 2024-05-15T15:54:02.399 INFO:tasks.workunit.client.1.smithi144.stdout:0/123: write d4/df/f16 [342354,72896] 0 2024-05-15T15:54:02.400 INFO:tasks.workunit.client.1.smithi144.stdout:0/124: stat d4/df/d1f/d24/d2b 0 2024-05-15T15:54:02.401 INFO:tasks.workunit.client.0.smithi049.stdout:2/224: mknod d3/da/d15/c40 0 2024-05-15T15:54:02.403 INFO:tasks.workunit.client.0.smithi049.stdout:6/195: dread dd/d17/f21 [0,4194304] 0 2024-05-15T15:54:02.404 INFO:tasks.workunit.client.1.smithi144.stdout:6/114: symlink d7/d1e/l2d 0 2024-05-15T15:54:02.405 INFO:tasks.workunit.client.1.smithi144.stdout:0/125: dwrite d4/df/d1f/d24/f2a [0,4194304] 0 2024-05-15T15:54:02.407 INFO:tasks.workunit.client.1.smithi144.stdout:8/135: rename d2/c8 to d2/de/d12/c2e 0 2024-05-15T15:54:02.409 INFO:tasks.workunit.client.1.smithi144.stdout:4/150: creat d6/dd/d15/f31 x:0 0 0 2024-05-15T15:54:02.410 INFO:tasks.workunit.client.1.smithi144.stdout:4/151: chown d6/d11/l2c 2096383 1 2024-05-15T15:54:02.410 INFO:tasks.workunit.client.1.smithi144.stdout:4/152: readlink d6/d11/l2c 0 2024-05-15T15:54:02.411 INFO:tasks.workunit.client.0.smithi049.stdout:1/282: rename d0/d4/f25 to d0/d4/dd/de/d36/f6a 0 2024-05-15T15:54:02.411 INFO:tasks.workunit.client.0.smithi049.stdout:0/217: rename dc/d13 to dc/d13/d14/d2a/d45 22 2024-05-15T15:54:02.411 INFO:tasks.workunit.client.1.smithi144.stdout:4/153: fsync d6/dd/f22 0 2024-05-15T15:54:02.414 INFO:tasks.workunit.client.0.smithi049.stdout:5/232: write dc/dd/f3b [1039922,69533] 0 2024-05-15T15:54:02.417 INFO:tasks.workunit.client.0.smithi049.stdout:2/225: creat d3/db/d1d/d1e/d2b/d2e/f41 x:0 0 0 2024-05-15T15:54:02.418 INFO:tasks.workunit.client.1.smithi144.stdout:9/127: creat d0/d8/d21/f38 x:0 0 0 2024-05-15T15:54:02.419 INFO:tasks.workunit.client.0.smithi049.stdout:2/226: write d3/d8/f14 [1769183,108867] 0 2024-05-15T15:54:02.422 INFO:tasks.workunit.client.1.smithi144.stdout:2/139: mknod d3/d7/d20/d22/c28 0 2024-05-15T15:54:02.426 INFO:tasks.workunit.client.0.smithi049.stdout:8/238: creat d2/d14/d1d/d3e/f4b x:0 0 0 2024-05-15T15:54:02.426 INFO:tasks.workunit.client.0.smithi049.stdout:8/239: stat d2/d18/f3c 0 2024-05-15T15:54:02.427 INFO:tasks.workunit.client.0.smithi049.stdout:8/240: truncate d2/f2c 465084 0 2024-05-15T15:54:02.428 INFO:tasks.workunit.client.1.smithi144.stdout:7/146: rmdir d4/df/d10/d24 0 2024-05-15T15:54:02.428 INFO:tasks.workunit.client.0.smithi049.stdout:8/241: truncate d2/d18/f47 982122 0 2024-05-15T15:54:02.429 INFO:tasks.workunit.client.0.smithi049.stdout:1/283: mknod d0/d4/dd/de/d1e/d16/c6b 0 2024-05-15T15:54:02.430 INFO:tasks.workunit.client.1.smithi144.stdout:7/147: dread d4/f13 [4194304,4194304] 0 2024-05-15T15:54:02.433 INFO:tasks.workunit.client.0.smithi049.stdout:0/218: creat dc/d13/d14/f46 x:0 0 0 2024-05-15T15:54:02.436 INFO:tasks.workunit.client.1.smithi144.stdout:8/136: creat d2/de/d12/f2f x:0 0 0 2024-05-15T15:54:02.436 INFO:tasks.workunit.client.0.smithi049.stdout:8/242: dwrite d2/d14/f1e [4194304,4194304] 0 2024-05-15T15:54:02.437 INFO:tasks.workunit.client.0.smithi049.stdout:8/243: write d2/d18/f3c [4539391,38481] 0 2024-05-15T15:54:02.439 INFO:tasks.workunit.client.1.smithi144.stdout:8/137: dread d2/f16 [0,4194304] 0 2024-05-15T15:54:02.440 INFO:tasks.workunit.client.0.smithi049.stdout:5/233: rename l3 to dc/dd/d10/l43 0 2024-05-15T15:54:02.442 INFO:tasks.workunit.client.1.smithi144.stdout:4/154: creat d6/f32 x:0 0 0 2024-05-15T15:54:02.444 INFO:tasks.workunit.client.1.smithi144.stdout:5/141: getdents d0/dc 0 2024-05-15T15:54:02.445 INFO:tasks.workunit.client.1.smithi144.stdout:9/128: mknod d0/d8/d21/d29/d34/d16/c39 0 2024-05-15T15:54:02.445 INFO:tasks.workunit.client.0.smithi049.stdout:2/227: readlink d3/d8/lf 0 2024-05-15T15:54:02.446 INFO:tasks.workunit.client.0.smithi049.stdout:2/228: write d3/d8/f36 [8563,3212] 0 2024-05-15T15:54:02.447 INFO:tasks.workunit.client.1.smithi144.stdout:8/138: dwrite d2/de/f10 [0,4194304] 0 2024-05-15T15:54:02.449 INFO:tasks.workunit.client.1.smithi144.stdout:8/139: write d2/de/f26 [4924293,93754] 0 2024-05-15T15:54:02.453 INFO:tasks.workunit.client.0.smithi049.stdout:3/256: rmdir d12/d30/d42/d44 0 2024-05-15T15:54:02.453 INFO:tasks.workunit.client.0.smithi049.stdout:3/257: readlink d12/d14/l16 0 2024-05-15T15:54:02.455 INFO:tasks.workunit.client.0.smithi049.stdout:3/258: truncate d12/d14/f24 920913 0 2024-05-15T15:54:02.456 INFO:tasks.workunit.client.0.smithi049.stdout:3/259: stat d12/d30 0 2024-05-15T15:54:02.457 INFO:tasks.workunit.client.0.smithi049.stdout:1/284: symlink d0/d4/dd/d5d/l6c 0 2024-05-15T15:54:02.466 INFO:tasks.workunit.client.0.smithi049.stdout:0/219: dread dc/d13/d14/d2a/f3a [0,4194304] 0 2024-05-15T15:54:02.467 INFO:tasks.workunit.client.0.smithi049.stdout:0/220: write dc/f1d [2604733,104771] 0 2024-05-15T15:54:02.469 INFO:tasks.workunit.client.1.smithi144.stdout:4/155: creat d6/d11/d30/db/f33 x:0 0 0 2024-05-15T15:54:02.470 INFO:tasks.workunit.client.1.smithi144.stdout:4/156: write d6/dd/d15/f31 [569540,93821] 0 2024-05-15T15:54:02.470 INFO:tasks.workunit.client.0.smithi049.stdout:5/234: creat dc/d29/d2c/d31/f44 x:0 0 0 2024-05-15T15:54:02.471 INFO:tasks.workunit.client.1.smithi144.stdout:5/142: rmdir d0/dc/d24 39 2024-05-15T15:54:02.473 INFO:tasks.workunit.client.0.smithi049.stdout:4/308: getdents d18/d1e/d2a 0 2024-05-15T15:54:02.473 INFO:tasks.workunit.client.1.smithi144.stdout:9/129: readlink d0/l3 0 2024-05-15T15:54:02.477 INFO:tasks.workunit.client.1.smithi144.stdout:3/126: truncate d0/d11/f14 1813745 0 2024-05-15T15:54:02.478 INFO:tasks.workunit.client.1.smithi144.stdout:3/127: chown d0/d1/f21 425977059 1 2024-05-15T15:54:02.478 INFO:tasks.workunit.client.1.smithi144.stdout:8/140: rmdir d2/de 39 2024-05-15T15:54:02.479 INFO:tasks.workunit.client.0.smithi049.stdout:4/309: dwrite d18/d1e/d2a/f4f [0,4194304] 0 2024-05-15T15:54:02.479 INFO:tasks.workunit.client.0.smithi049.stdout:3/260: symlink d12/d30/l4a 0 2024-05-15T15:54:02.480 INFO:tasks.workunit.client.1.smithi144.stdout:0/126: mkdir d4/d2d 0 2024-05-15T15:54:02.481 INFO:tasks.workunit.client.0.smithi049.stdout:3/261: truncate d12/d14/f24 1399228 0 2024-05-15T15:54:02.482 INFO:tasks.workunit.client.0.smithi049.stdout:9/195: mkdir d0/d8/d42 0 2024-05-15T15:54:02.494 INFO:tasks.workunit.client.0.smithi049.stdout:2/229: mkdir d3/db/d1d/d1e/d2b/d3b/d42 0 2024-05-15T15:54:02.496 INFO:tasks.workunit.client.1.smithi144.stdout:3/128: dread d0/d1/fe [0,4194304] 0 2024-05-15T15:54:02.497 INFO:tasks.workunit.client.0.smithi049.stdout:1/285: rename d0/d4/dd/de/d1e/c11 to d0/d4/dd/c6d 0 2024-05-15T15:54:02.498 INFO:tasks.workunit.client.0.smithi049.stdout:6/196: getdents dd/d17/d24 0 2024-05-15T15:54:02.500 INFO:tasks.workunit.client.1.smithi144.stdout:8/141: truncate d2/de/f14 405450 0 2024-05-15T15:54:02.501 INFO:tasks.workunit.client.1.smithi144.stdout:8/142: write d2/d24/f25 [279213,81271] 0 2024-05-15T15:54:02.501 INFO:tasks.workunit.client.0.smithi049.stdout:2/230: dwrite d3/da/f2c [0,4194304] 0 2024-05-15T15:54:02.503 INFO:tasks.workunit.client.0.smithi049.stdout:4/310: creat d18/d1e/d2a/f64 x:0 0 0 2024-05-15T15:54:02.506 INFO:tasks.workunit.client.1.smithi144.stdout:8/143: dread d2/de/d12/f17 [0,4194304] 0 2024-05-15T15:54:02.506 INFO:tasks.workunit.client.0.smithi049.stdout:9/196: rename d0/d8/d12/c35 to d0/d8/d12/c43 0 2024-05-15T15:54:02.506 INFO:tasks.workunit.client.1.smithi144.stdout:8/144: chown d2/de/fc 433 1 2024-05-15T15:54:02.507 INFO:tasks.workunit.client.0.smithi049.stdout:6/197: dwrite f0 [4194304,4194304] 0 2024-05-15T15:54:02.507 INFO:tasks.workunit.client.1.smithi144.stdout:1/145: creat d7/d9/d22/f28 x:0 0 0 2024-05-15T15:54:02.508 INFO:tasks.workunit.client.0.smithi049.stdout:6/198: dread - dd/d26/f2e zero size 2024-05-15T15:54:02.509 INFO:tasks.workunit.client.0.smithi049.stdout:8/244: creat d2/d14/f4c x:0 0 0 2024-05-15T15:54:02.512 INFO:tasks.workunit.client.1.smithi144.stdout:7/148: creat d4/f2f x:0 0 0 2024-05-15T15:54:02.512 INFO:tasks.workunit.client.0.smithi049.stdout:2/231: dwrite d3/db/f30 [0,4194304] 0 2024-05-15T15:54:02.513 INFO:tasks.workunit.client.0.smithi049.stdout:0/221: creat dc/d13/d14/d33/f47 x:0 0 0 2024-05-15T15:54:02.521 INFO:tasks.workunit.client.1.smithi144.stdout:9/130: creat d0/d8/d21/d29/d2d/f3a x:0 0 0 2024-05-15T15:54:02.521 INFO:tasks.workunit.client.1.smithi144.stdout:2/140: rmdir d3 39 2024-05-15T15:54:02.522 INFO:tasks.workunit.client.1.smithi144.stdout:9/131: truncate d0/d8/d21/f2f 669208 0 2024-05-15T15:54:02.523 INFO:tasks.workunit.client.0.smithi049.stdout:1/286: mkdir d0/d4/dd/de/d1e/d16/d6e 0 2024-05-15T15:54:02.525 INFO:tasks.workunit.client.1.smithi144.stdout:3/129: creat d0/f2e x:0 0 0 2024-05-15T15:54:02.528 INFO:tasks.workunit.client.1.smithi144.stdout:6/115: write f6 [4803144,43174] 0 2024-05-15T15:54:02.528 INFO:tasks.workunit.client.1.smithi144.stdout:6/116: readlink d7/d1e/l21 0 2024-05-15T15:54:02.529 INFO:tasks.workunit.client.1.smithi144.stdout:6/117: chown d7/db/d11/c26 3587 1 2024-05-15T15:54:02.529 INFO:tasks.workunit.client.0.smithi049.stdout:9/197: creat d0/d8/d1b/f44 x:0 0 0 2024-05-15T15:54:02.530 INFO:tasks.workunit.client.1.smithi144.stdout:6/118: write d7/db/d11/f15 [858679,76042] 0 2024-05-15T15:54:02.530 INFO:tasks.workunit.client.0.smithi049.stdout:1/287: dwrite d0/d4/dd/de/d1e/d2e/f41 [0,4194304] 0 2024-05-15T15:54:02.531 INFO:tasks.workunit.client.1.smithi144.stdout:0/127: creat d4/d2d/f2e x:0 0 0 2024-05-15T15:54:02.537 INFO:tasks.workunit.client.1.smithi144.stdout:7/149: unlink d4/df/d25/c2c 0 2024-05-15T15:54:02.539 INFO:tasks.workunit.client.1.smithi144.stdout:5/143: mkdir d0/dc/d12/d31 0 2024-05-15T15:54:02.542 INFO:tasks.workunit.client.0.smithi049.stdout:8/245: dread d2/de/f16 [0,4194304] 0 2024-05-15T15:54:02.544 INFO:tasks.workunit.client.0.smithi049.stdout:8/246: write d2/de/d27/f39 [1083495,114856] 0 2024-05-15T15:54:02.545 INFO:tasks.workunit.client.1.smithi144.stdout:9/132: symlink d0/d8/d21/l3b 0 2024-05-15T15:54:02.547 INFO:tasks.workunit.client.1.smithi144.stdout:3/130: mknod d0/d1/d17/c2f 0 2024-05-15T15:54:02.547 INFO:tasks.workunit.client.1.smithi144.stdout:2/141: dwrite d3/d7/d20/d22/f23 [0,4194304] 0 2024-05-15T15:54:02.548 INFO:tasks.workunit.client.1.smithi144.stdout:3/131: chown d0/d11/f2a 460 1 2024-05-15T15:54:02.549 INFO:tasks.workunit.client.1.smithi144.stdout:3/132: dread - d0/f2e zero size 2024-05-15T15:54:02.552 INFO:tasks.workunit.client.1.smithi144.stdout:8/145: mkdir d2/d24/d2d/d30 0 2024-05-15T15:54:02.554 INFO:tasks.workunit.client.0.smithi049.stdout:3/262: rename d12/d14/d32/f3a to d12/d14/f4b 0 2024-05-15T15:54:02.556 INFO:tasks.workunit.client.0.smithi049.stdout:9/198: unlink d0/d8/d12/c23 0 2024-05-15T15:54:02.559 INFO:tasks.workunit.client.1.smithi144.stdout:9/133: fdatasync d0/f14 0 2024-05-15T15:54:02.560 INFO:tasks.workunit.client.0.smithi049.stdout:8/247: chown d2/c11 1 1 2024-05-15T15:54:02.563 INFO:tasks.workunit.client.0.smithi049.stdout:1/288: creat d0/d4/dd/d5d/d65/f6f x:0 0 0 2024-05-15T15:54:02.565 INFO:tasks.workunit.client.0.smithi049.stdout:6/199: creat dd/f32 x:0 0 0 2024-05-15T15:54:02.565 INFO:tasks.workunit.client.0.smithi049.stdout:1/289: chown d0/d4/dd/de/d48/f4a 104730155 1 2024-05-15T15:54:02.566 INFO:tasks.workunit.client.1.smithi144.stdout:2/142: creat d3/d10/f29 x:0 0 0 2024-05-15T15:54:02.566 INFO:tasks.workunit.client.1.smithi144.stdout:4/157: truncate d6/d11/f25 2921906 0 2024-05-15T15:54:02.567 INFO:tasks.workunit.client.1.smithi144.stdout:4/158: fsync d6/d11/d30/f1b 0 2024-05-15T15:54:02.567 INFO:tasks.workunit.client.0.smithi049.stdout:2/232: creat d3/db/f43 x:0 0 0 2024-05-15T15:54:02.568 INFO:tasks.workunit.client.1.smithi144.stdout:3/133: mknod d0/d11/c30 0 2024-05-15T15:54:02.571 INFO:tasks.workunit.client.1.smithi144.stdout:0/128: link d4/f1c d4/df/d1f/f2f 0 2024-05-15T15:54:02.571 INFO:tasks.workunit.client.1.smithi144.stdout:2/143: dwrite d3/d7/f1f [0,4194304] 0 2024-05-15T15:54:02.573 INFO:tasks.workunit.client.1.smithi144.stdout:8/146: fdatasync d2/f3 0 2024-05-15T15:54:02.574 INFO:tasks.workunit.client.0.smithi049.stdout:3/263: mknod d12/d30/c4c 0 2024-05-15T15:54:02.577 INFO:tasks.workunit.client.1.smithi144.stdout:7/150: mkdir d4/df/d30 0 2024-05-15T15:54:02.577 INFO:tasks.workunit.client.0.smithi049.stdout:3/264: read d12/d14/f3c [493341,106970] 0 2024-05-15T15:54:02.578 INFO:tasks.workunit.client.0.smithi049.stdout:9/199: write d0/f1d [283546,81123] 0 2024-05-15T15:54:02.578 INFO:tasks.workunit.client.1.smithi144.stdout:8/147: dread d2/fd [0,4194304] 0 2024-05-15T15:54:02.580 INFO:tasks.workunit.client.0.smithi049.stdout:9/200: write d0/f1d [1582971,85353] 0 2024-05-15T15:54:02.583 INFO:tasks.workunit.client.0.smithi049.stdout:9/201: write d0/d8/d9/d22/d26/f3e [1046279,55803] 0 2024-05-15T15:54:02.584 INFO:tasks.workunit.client.0.smithi049.stdout:6/200: rmdir dd/d17 39 2024-05-15T15:54:02.584 INFO:tasks.workunit.client.0.smithi049.stdout:9/202: fdatasync d0/d8/d1b/f36 0 2024-05-15T15:54:02.585 INFO:tasks.workunit.client.0.smithi049.stdout:2/233: dread d3/d8/f14 [0,4194304] 0 2024-05-15T15:54:02.585 INFO:tasks.workunit.client.0.smithi049.stdout:6/201: write f0 [6194030,86924] 0 2024-05-15T15:54:02.587 INFO:tasks.workunit.client.0.smithi049.stdout:1/290: creat d0/d4/d63/f70 x:0 0 0 2024-05-15T15:54:02.587 INFO:tasks.workunit.client.0.smithi049.stdout:9/203: read d0/d8/d9/d22/d26/f3e [739711,41384] 0 2024-05-15T15:54:02.589 INFO:tasks.workunit.client.0.smithi049.stdout:9/204: write d0/d8/d11/f17 [1892419,59423] 0 2024-05-15T15:54:02.592 INFO:tasks.workunit.client.1.smithi144.stdout:6/119: truncate d7/fd 137666 0 2024-05-15T15:54:02.593 INFO:tasks.workunit.client.1.smithi144.stdout:6/120: stat d7/db/d11/c26 0 2024-05-15T15:54:02.593 INFO:tasks.workunit.client.1.smithi144.stdout:6/121: fsync d7/db/f29 0 2024-05-15T15:54:02.600 INFO:tasks.workunit.client.1.smithi144.stdout:8/148: rename d2/d1a/f1d to d2/d1a/f31 0 2024-05-15T15:54:02.602 INFO:tasks.workunit.client.0.smithi049.stdout:4/311: mknod d18/d1e/d2a/d3b/c65 0 2024-05-15T15:54:02.604 INFO:tasks.workunit.client.0.smithi049.stdout:8/248: mknod d2/c4d 0 2024-05-15T15:54:02.606 INFO:tasks.workunit.client.1.smithi144.stdout:8/149: dwrite d2/de/d12/f27 [0,4194304] 0 2024-05-15T15:54:02.611 INFO:tasks.workunit.client.0.smithi049.stdout:2/234: unlink d3/db/d1d/l38 0 2024-05-15T15:54:02.616 INFO:tasks.workunit.client.1.smithi144.stdout:6/122: creat d7/d1e/f2e x:0 0 0 2024-05-15T15:54:02.618 INFO:tasks.workunit.client.1.smithi144.stdout:8/150: dread d2/f3 [0,4194304] 0 2024-05-15T15:54:02.618 INFO:tasks.workunit.client.0.smithi049.stdout:1/291: symlink d0/d4/dd/de/d4d/l71 0 2024-05-15T15:54:02.619 INFO:tasks.workunit.client.0.smithi049.stdout:1/292: read - d0/d4/d63/f70 zero size 2024-05-15T15:54:02.619 INFO:tasks.workunit.client.1.smithi144.stdout:1/146: sync 2024-05-15T15:54:02.622 INFO:tasks.workunit.client.1.smithi144.stdout:9/134: rename d0/d8/d21/d29/d34/d10/d19/d30 to d0/d8/d21/d29/d34/d3c 0 2024-05-15T15:54:02.624 INFO:tasks.workunit.client.1.smithi144.stdout:8/151: dwrite d2/de/d12/f27 [0,4194304] 0 2024-05-15T15:54:02.625 INFO:tasks.workunit.client.0.smithi049.stdout:0/222: rename dc/d13/d2b/d36 to dc/d13/d48 0 2024-05-15T15:54:02.627 INFO:tasks.workunit.client.0.smithi049.stdout:0/223: write dc/d13/d14/d2a/f2c [845777,71874] 0 2024-05-15T15:54:02.628 INFO:tasks.workunit.client.1.smithi144.stdout:4/159: dwrite d6/dd/d15/f1c [0,4194304] 0 2024-05-15T15:54:02.628 INFO:tasks.workunit.client.0.smithi049.stdout:0/224: readlink dc/d13/d14/l27 0 2024-05-15T15:54:02.629 INFO:tasks.workunit.client.0.smithi049.stdout:0/225: truncate dc/f40 833247 0 2024-05-15T15:54:02.632 INFO:tasks.workunit.client.0.smithi049.stdout:4/312: fdatasync f10 0 2024-05-15T15:54:02.635 INFO:tasks.workunit.client.0.smithi049.stdout:8/249: creat d2/d18/f4e x:0 0 0 2024-05-15T15:54:02.636 INFO:tasks.workunit.client.0.smithi049.stdout:0/226: dwrite dc/d13/f22 [0,4194304] 0 2024-05-15T15:54:02.637 INFO:tasks.workunit.client.0.smithi049.stdout:2/235: symlink d3/db/d1d/d1e/d2b/l44 0 2024-05-15T15:54:02.643 INFO:tasks.workunit.client.1.smithi144.stdout:5/144: getdents d0/dc/d1f 0 2024-05-15T15:54:02.645 INFO:tasks.workunit.client.0.smithi049.stdout:0/227: dwrite dc/d13/d14/d33/f47 [0,4194304] 0 2024-05-15T15:54:02.648 INFO:tasks.workunit.client.0.smithi049.stdout:9/205: mkdir d0/d8/d9/d45 0 2024-05-15T15:54:02.650 INFO:tasks.workunit.client.1.smithi144.stdout:1/147: readlink d7/d8/l20 0 2024-05-15T15:54:02.650 INFO:tasks.workunit.client.1.smithi144.stdout:5/145: dwrite d0/dc/d12/f18 [0,4194304] 0 2024-05-15T15:54:02.652 INFO:tasks.workunit.client.0.smithi049.stdout:3/265: creat d12/f4d x:0 0 0 2024-05-15T15:54:02.652 INFO:tasks.workunit.client.0.smithi049.stdout:3/266: chown d12/f18 8101 1 2024-05-15T15:54:02.654 INFO:tasks.workunit.client.1.smithi144.stdout:3/134: rename d0/d11/c30 to d0/d1/d17/c31 0 2024-05-15T15:54:02.654 INFO:tasks.workunit.client.0.smithi049.stdout:1/293: write d0/d4/dd/de/d36/f6a [896308,36238] 0 2024-05-15T15:54:02.656 INFO:tasks.workunit.client.1.smithi144.stdout:8/152: mknod d2/d24/d2d/c32 0 2024-05-15T15:54:02.657 INFO:tasks.workunit.client.1.smithi144.stdout:8/153: fdatasync d2/de/d12/f27 0 2024-05-15T15:54:02.658 INFO:tasks.workunit.client.0.smithi049.stdout:4/313: fdatasync d18/d1e/d2a/f43 0 2024-05-15T15:54:02.662 INFO:tasks.workunit.client.1.smithi144.stdout:0/129: getdents d4/d6 0 2024-05-15T15:54:02.663 INFO:tasks.workunit.client.0.smithi049.stdout:2/236: creat d3/da/f45 x:0 0 0 2024-05-15T15:54:02.664 INFO:tasks.workunit.client.1.smithi144.stdout:2/144: getdents d3 0 2024-05-15T15:54:02.664 INFO:tasks.workunit.client.0.smithi049.stdout:2/237: write d3/da/d15/f27 [4890131,33463] 0 2024-05-15T15:54:02.665 INFO:tasks.workunit.client.0.smithi049.stdout:6/202: creat dd/f33 x:0 0 0 2024-05-15T15:54:02.665 INFO:tasks.workunit.client.0.smithi049.stdout:6/203: chown la 3890 1 2024-05-15T15:54:02.666 INFO:tasks.workunit.client.1.smithi144.stdout:0/130: dread d4/d6/fb [0,4194304] 0 2024-05-15T15:54:02.666 INFO:tasks.workunit.client.1.smithi144.stdout:0/131: chown d4/df/d1f 599 1 2024-05-15T15:54:02.670 INFO:tasks.workunit.client.0.smithi049.stdout:0/228: symlink dc/d13/d14/d25/l49 0 2024-05-15T15:54:02.671 INFO:tasks.workunit.client.0.smithi049.stdout:0/229: stat dc/d13/d14/d2a/f30 0 2024-05-15T15:54:02.671 INFO:tasks.workunit.client.1.smithi144.stdout:1/148: mkdir d7/d8/d29 0 2024-05-15T15:54:02.673 INFO:tasks.workunit.client.1.smithi144.stdout:7/151: rename d4/df/l28 to d4/df/d25/l31 0 2024-05-15T15:54:02.675 INFO:tasks.workunit.client.1.smithi144.stdout:3/135: creat d0/d11/d24/f32 x:0 0 0 2024-05-15T15:54:02.676 INFO:tasks.workunit.client.0.smithi049.stdout:1/294: mkdir d0/d4/dd/d5d/d65/d72 0 2024-05-15T15:54:02.677 INFO:tasks.workunit.client.0.smithi049.stdout:1/295: fdatasync d0/d4/dd/de/d36/f53 0 2024-05-15T15:54:02.677 INFO:tasks.workunit.client.1.smithi144.stdout:4/160: mkdir d6/dd/d34 0 2024-05-15T15:54:02.678 INFO:tasks.workunit.client.1.smithi144.stdout:4/161: write d6/d11/d30/ff [4782910,91197] 0 2024-05-15T15:54:02.679 INFO:tasks.workunit.client.1.smithi144.stdout:8/154: unlink d2/de/c20 0 2024-05-15T15:54:02.681 INFO:tasks.workunit.client.0.smithi049.stdout:8/250: mkdir d2/d4f 0 2024-05-15T15:54:02.682 INFO:tasks.workunit.client.1.smithi144.stdout:4/162: dread d6/dd/d15/f1c [0,4194304] 0 2024-05-15T15:54:02.682 INFO:tasks.workunit.client.1.smithi144.stdout:2/145: symlink d3/l2a 0 2024-05-15T15:54:02.683 INFO:tasks.workunit.client.0.smithi049.stdout:8/251: write d2/d14/d1b/f2b [451796,51187] 0 2024-05-15T15:54:02.683 INFO:tasks.workunit.client.1.smithi144.stdout:4/163: write d6/dd/f22 [916111,48291] 0 2024-05-15T15:54:02.684 INFO:tasks.workunit.client.1.smithi144.stdout:6/123: link d7/d1e/l21 d7/db/d11/l2f 0 2024-05-15T15:54:02.684 INFO:tasks.workunit.client.1.smithi144.stdout:4/164: chown d6/d11/d30/db 11039 1 2024-05-15T15:54:02.684 INFO:tasks.workunit.client.0.smithi049.stdout:0/230: symlink dc/d13/d48/l4a 0 2024-05-15T15:54:02.684 INFO:tasks.workunit.client.1.smithi144.stdout:4/165: chown d6/d11/f21 31551844 1 2024-05-15T15:54:02.685 INFO:tasks.workunit.client.1.smithi144.stdout:0/132: mknod d4/df/c30 0 2024-05-15T15:54:02.685 INFO:tasks.workunit.client.1.smithi144.stdout:4/166: fdatasync d6/d11/d30/f1b 0 2024-05-15T15:54:02.686 INFO:tasks.workunit.client.1.smithi144.stdout:5/146: truncate d0/dc/f1b 4664067 0 2024-05-15T15:54:02.687 INFO:tasks.workunit.client.1.smithi144.stdout:1/149: rename d7/d8/d1e/c23 to d7/d9/c2a 0 2024-05-15T15:54:02.688 INFO:tasks.workunit.client.0.smithi049.stdout:4/314: creat d18/d1e/d2a/d50/f66 x:0 0 0 2024-05-15T15:54:02.689 INFO:tasks.workunit.client.1.smithi144.stdout:3/136: creat d0/d1/d17/f33 x:0 0 0 2024-05-15T15:54:02.689 INFO:tasks.workunit.client.0.smithi049.stdout:6/204: mknod dd/d17/d24/c34 0 2024-05-15T15:54:02.690 INFO:tasks.workunit.client.1.smithi144.stdout:3/137: chown d0/d1/d17 4 1 2024-05-15T15:54:02.690 INFO:tasks.workunit.client.1.smithi144.stdout:3/138: readlink d0/l2b 0 2024-05-15T15:54:02.691 INFO:tasks.workunit.client.1.smithi144.stdout:3/139: chown d0/l27 6941923 1 2024-05-15T15:54:02.691 INFO:tasks.workunit.client.1.smithi144.stdout:5/147: dwrite d0/f22 [0,4194304] 0 2024-05-15T15:54:02.693 INFO:tasks.workunit.client.0.smithi049.stdout:9/206: rename d0/d8/d12/l27 to d0/d8/d9/l46 0 2024-05-15T15:54:02.696 INFO:tasks.workunit.client.0.smithi049.stdout:3/267: rename d12/d14/f4b to d12/d14/f4e 0 2024-05-15T15:54:02.700 INFO:tasks.workunit.client.0.smithi049.stdout:4/315: unlink f8 0 2024-05-15T15:54:02.702 INFO:tasks.workunit.client.0.smithi049.stdout:8/252: symlink d2/d14/l50 0 2024-05-15T15:54:02.704 INFO:tasks.workunit.client.0.smithi049.stdout:8/253: truncate d2/d14/d1b/f2b 727253 0 2024-05-15T15:54:02.704 INFO:tasks.workunit.client.0.smithi049.stdout:8/254: dread - d2/de/f44 zero size 2024-05-15T15:54:02.704 INFO:tasks.workunit.client.0.smithi049.stdout:0/231: mknod dc/c4b 0 2024-05-15T15:54:02.710 INFO:tasks.workunit.client.1.smithi144.stdout:3/140: symlink d0/d1/d17/l34 0 2024-05-15T15:54:02.720 INFO:tasks.workunit.client.1.smithi144.stdout:4/167: fsync d6/d11/f25 0 2024-05-15T15:54:02.723 INFO:tasks.workunit.client.1.smithi144.stdout:1/150: unlink d7/df/d24/c21 0 2024-05-15T15:54:02.723 INFO:tasks.workunit.client.1.smithi144.stdout:1/151: chown d7/df/f15 114909 1 2024-05-15T15:54:02.725 INFO:tasks.workunit.client.1.smithi144.stdout:7/152: creat d4/df/f32 x:0 0 0 2024-05-15T15:54:02.729 INFO:tasks.workunit.client.0.smithi049.stdout:1/296: rename d0/f44 to d0/d4/dd/de/f73 0 2024-05-15T15:54:02.730 INFO:tasks.workunit.client.0.smithi049.stdout:8/255: write d2/de/d25/f26 [3665915,77449] 0 2024-05-15T15:54:02.730 INFO:tasks.workunit.client.0.smithi049.stdout:1/297: write d0/d4/dd/de/d36/f43 [2206685,16659] 0 2024-05-15T15:54:02.731 INFO:tasks.workunit.client.1.smithi144.stdout:2/146: dwrite d3/d10/f1d [0,4194304] 0 2024-05-15T15:54:02.731 INFO:tasks.workunit.client.0.smithi049.stdout:3/268: creat d12/d14/f4f x:0 0 0 2024-05-15T15:54:02.732 INFO:tasks.workunit.client.1.smithi144.stdout:5/148: write d0/dc/d12/f13 [726645,597] 0 2024-05-15T15:54:02.734 INFO:tasks.workunit.client.1.smithi144.stdout:3/141: unlink d0/d1/f1c 0 2024-05-15T15:54:02.734 INFO:tasks.workunit.client.0.smithi049.stdout:0/232: write dc/d13/d14/d2a/f3a [2628666,53228] 0 2024-05-15T15:54:02.736 INFO:tasks.workunit.client.1.smithi144.stdout:8/155: truncate d2/de/fc 293561 0 2024-05-15T15:54:02.741 INFO:tasks.workunit.client.1.smithi144.stdout:4/168: creat d6/d11/d30/db/f35 x:0 0 0 2024-05-15T15:54:02.751 INFO:tasks.workunit.client.0.smithi049.stdout:3/269: mknod d12/d14/d32/c50 0 2024-05-15T15:54:02.752 INFO:tasks.workunit.client.0.smithi049.stdout:0/233: creat dc/d13/d3d/f4c x:0 0 0 2024-05-15T15:54:02.753 INFO:tasks.workunit.client.0.smithi049.stdout:0/234: write dc/f15 [1182899,6409] 0 2024-05-15T15:54:02.753 INFO:tasks.workunit.client.0.smithi049.stdout:4/316: rmdir d18/d1e/d2a/d36/d55 0 2024-05-15T15:54:02.754 INFO:tasks.workunit.client.0.smithi049.stdout:0/235: fsync dc/d13/d14/f1c 0 2024-05-15T15:54:02.759 INFO:tasks.workunit.client.0.smithi049.stdout:0/236: dread dc/d13/f17 [0,4194304] 0 2024-05-15T15:54:02.759 INFO:tasks.workunit.client.0.smithi049.stdout:6/205: rename dd/d26/l2d to dd/d17/d24/l35 0 2024-05-15T15:54:02.761 INFO:tasks.workunit.client.1.smithi144.stdout:8/156: creat d2/de/f33 x:0 0 0 2024-05-15T15:54:02.762 INFO:tasks.workunit.client.0.smithi049.stdout:1/298: symlink d0/d4/dd/de/d1e/l74 0 2024-05-15T15:54:02.763 INFO:tasks.workunit.client.1.smithi144.stdout:1/152: getdents d7/d8/d29 0 2024-05-15T15:54:02.771 INFO:tasks.workunit.client.1.smithi144.stdout:3/142: dwrite d0/d11/f2a [0,4194304] 0 2024-05-15T15:54:02.772 INFO:tasks.workunit.client.0.smithi049.stdout:0/237: creat dc/d13/d2b/f4d x:0 0 0 2024-05-15T15:54:02.772 INFO:tasks.workunit.client.1.smithi144.stdout:5/149: mknod d0/dc/d12/d31/c32 0 2024-05-15T15:54:02.773 INFO:tasks.workunit.client.0.smithi049.stdout:4/317: dwrite d18/f3e [0,4194304] 0 2024-05-15T15:54:02.775 INFO:tasks.workunit.client.0.smithi049.stdout:0/238: write dc/d13/d3d/f3f [491936,63770] 0 2024-05-15T15:54:02.775 INFO:tasks.workunit.client.0.smithi049.stdout:4/318: fdatasync d18/d1e/d2a/f64 0 2024-05-15T15:54:02.776 INFO:tasks.workunit.client.0.smithi049.stdout:6/206: mkdir dd/d17/d24/d2b/d36 0 2024-05-15T15:54:02.778 INFO:tasks.workunit.client.0.smithi049.stdout:1/299: mkdir d0/d4/dd/d75 0 2024-05-15T15:54:02.784 INFO:tasks.workunit.client.1.smithi144.stdout:1/153: dread d7/d9/fe [0,4194304] 0 2024-05-15T15:54:02.784 INFO:tasks.workunit.client.1.smithi144.stdout:7/153: rmdir d4/df/d30 0 2024-05-15T15:54:02.787 INFO:tasks.workunit.client.0.smithi049.stdout:1/300: dwrite d0/d4/dd/de/d1e/f37 [0,4194304] 0 2024-05-15T15:54:02.789 INFO:tasks.workunit.client.0.smithi049.stdout:0/239: creat dc/d13/d14/f4e x:0 0 0 2024-05-15T15:54:02.791 INFO:tasks.workunit.client.1.smithi144.stdout:9/135: sync 2024-05-15T15:54:02.795 INFO:tasks.workunit.client.1.smithi144.stdout:3/143: symlink d0/l35 0 2024-05-15T15:54:02.795 INFO:tasks.workunit.client.0.smithi049.stdout:6/207: rmdir dd/d17/d24/d2b 39 2024-05-15T15:54:02.797 INFO:tasks.workunit.client.1.smithi144.stdout:5/150: mknod d0/dc/d1f/c33 0 2024-05-15T15:54:02.800 INFO:tasks.workunit.client.0.smithi049.stdout:6/208: dread dd/f15 [0,4194304] 0 2024-05-15T15:54:02.802 INFO:tasks.workunit.client.0.smithi049.stdout:6/209: write dd/f16 [809181,93396] 0 2024-05-15T15:54:02.803 INFO:tasks.workunit.client.1.smithi144.stdout:7/154: dread d4/fa [0,4194304] 0 2024-05-15T15:54:02.804 INFO:tasks.workunit.client.0.smithi049.stdout:6/210: fsync dd/ff 0 2024-05-15T15:54:02.804 INFO:tasks.workunit.client.1.smithi144.stdout:7/155: write d4/df/f20 [181885,126466] 0 2024-05-15T15:54:02.808 INFO:tasks.workunit.client.1.smithi144.stdout:2/147: link d3/d10/f11 d3/d7/d20/d22/f2b 0 2024-05-15T15:54:02.809 INFO:tasks.workunit.client.0.smithi049.stdout:3/270: dwrite ff [0,4194304] 0 2024-05-15T15:54:02.809 INFO:tasks.workunit.client.0.smithi049.stdout:4/319: creat d18/d1e/d2a/d36/d5d/f67 x:0 0 0 2024-05-15T15:54:02.810 INFO:tasks.workunit.client.0.smithi049.stdout:0/240: read f4 [521993,65080] 0 2024-05-15T15:54:02.811 INFO:tasks.workunit.client.1.smithi144.stdout:8/157: fdatasync d2/de/fc 0 2024-05-15T15:54:02.811 INFO:tasks.workunit.client.1.smithi144.stdout:8/158: readlink d2/l29 0 2024-05-15T15:54:02.815 INFO:tasks.workunit.client.0.smithi049.stdout:1/301: creat d0/d4/dd/de/d4e/f76 x:0 0 0 2024-05-15T15:54:02.816 INFO:tasks.workunit.client.0.smithi049.stdout:6/211: dwrite dd/d17/d24/f27 [0,4194304] 0 2024-05-15T15:54:02.816 INFO:tasks.workunit.client.1.smithi144.stdout:3/144: symlink d0/d1/l36 0 2024-05-15T15:54:02.817 INFO:tasks.workunit.client.0.smithi049.stdout:6/212: stat dd/f15 0 2024-05-15T15:54:02.867 INFO:tasks.workunit.client.1.smithi144.stdout:6/124: sync 2024-05-15T15:54:02.873 INFO:tasks.workunit.client.0.smithi049.stdout:7/232: sync 2024-05-15T15:54:02.878 INFO:tasks.workunit.client.0.smithi049.stdout:1/302: dwrite d0/d4/dd/de/d1e/f27 [0,4194304] 0 2024-05-15T15:54:02.882 INFO:tasks.workunit.client.1.smithi144.stdout:2/148: fdatasync d3/f5 0 2024-05-15T15:54:02.884 INFO:tasks.workunit.client.0.smithi049.stdout:3/271: symlink d12/d30/d42/l51 0 2024-05-15T15:54:02.884 INFO:tasks.workunit.client.1.smithi144.stdout:2/149: write d3/d10/f1d [4665495,78862] 0 2024-05-15T15:54:02.884 INFO:tasks.workunit.client.0.smithi049.stdout:3/272: fsync d12/f47 0 2024-05-15T15:54:02.885 INFO:tasks.workunit.client.0.smithi049.stdout:3/273: readlink d12/d26/l2e 0 2024-05-15T15:54:02.886 INFO:tasks.workunit.client.1.smithi144.stdout:7/156: dwrite d4/df/d10/f23 [0,4194304] 0 2024-05-15T15:54:02.887 INFO:tasks.workunit.client.1.smithi144.stdout:9/136: fdatasync d0/d8/d21/d29/d34/d10/f2c 0 2024-05-15T15:54:02.887 INFO:tasks.workunit.client.1.smithi144.stdout:7/157: dread - d4/f2f zero size 2024-05-15T15:54:02.888 INFO:tasks.workunit.client.0.smithi049.stdout:7/233: write d0/d34/f3f [3754435,116590] 0 2024-05-15T15:54:02.890 INFO:tasks.workunit.client.0.smithi049.stdout:7/234: chown d0/d6/da/c26 11 1 2024-05-15T15:54:02.890 INFO:tasks.workunit.client.0.smithi049.stdout:0/241: write f3 [2993578,30098] 0 2024-05-15T15:54:02.891 INFO:tasks.workunit.client.1.smithi144.stdout:3/145: write d0/d1/fe [3830824,117794] 0 2024-05-15T15:54:02.892 INFO:tasks.workunit.client.1.smithi144.stdout:6/125: rmdir d7/db 39 2024-05-15T15:54:02.894 INFO:tasks.workunit.client.1.smithi144.stdout:3/146: truncate d0/d1/d17/f33 165943 0 2024-05-15T15:54:02.895 INFO:tasks.workunit.client.1.smithi144.stdout:2/150: dwrite d3/d10/d15/f26 [0,4194304] 0 2024-05-15T15:54:02.896 INFO:tasks.workunit.client.1.smithi144.stdout:1/154: creat d7/d1d/f2b x:0 0 0 2024-05-15T15:54:02.899 INFO:tasks.workunit.client.0.smithi049.stdout:6/213: rename dd/c2c to dd/d17/d24/d2b/c37 0 2024-05-15T15:54:02.900 INFO:tasks.workunit.client.0.smithi049.stdout:6/214: write dd/f14 [1376305,16053] 0 2024-05-15T15:54:02.902 INFO:tasks.workunit.client.0.smithi049.stdout:7/235: mknod d0/d34/c44 0 2024-05-15T15:54:02.903 INFO:tasks.workunit.client.1.smithi144.stdout:7/158: fdatasync d4/df/d10/f23 0 2024-05-15T15:54:02.903 INFO:tasks.workunit.client.0.smithi049.stdout:7/236: write d0/f15 [105934,102339] 0 2024-05-15T15:54:02.904 INFO:tasks.workunit.client.0.smithi049.stdout:1/303: mknod d0/d4/dd/de/d1e/c77 0 2024-05-15T15:54:02.906 INFO:tasks.workunit.client.1.smithi144.stdout:9/137: stat d0/le 0 2024-05-15T15:54:02.907 INFO:tasks.workunit.client.0.smithi049.stdout:3/274: mkdir d12/d14/d52 0 2024-05-15T15:54:02.912 INFO:tasks.workunit.client.0.smithi049.stdout:6/215: creat dd/d17/f38 x:0 0 0 2024-05-15T15:54:02.912 INFO:tasks.workunit.client.1.smithi144.stdout:8/159: truncate d2/de/fc 1311268 0 2024-05-15T15:54:02.913 INFO:tasks.workunit.client.0.smithi049.stdout:6/216: write f8 [769983,19129] 0 2024-05-15T15:54:02.913 INFO:tasks.workunit.client.1.smithi144.stdout:6/126: mknod d7/d1e/c30 0 2024-05-15T15:54:02.914 INFO:tasks.workunit.client.1.smithi144.stdout:5/151: mkdir d0/dc/d2f/d34 0 2024-05-15T15:54:02.922 INFO:tasks.workunit.client.1.smithi144.stdout:5/152: dread d0/f7 [0,4194304] 0 2024-05-15T15:54:02.924 INFO:tasks.workunit.client.0.smithi049.stdout:7/237: dread d0/d6/da/d3d/f33 [0,4194304] 0 2024-05-15T15:54:02.925 INFO:tasks.workunit.client.1.smithi144.stdout:4/169: sync 2024-05-15T15:54:02.925 INFO:tasks.workunit.client.1.smithi144.stdout:0/133: sync 2024-05-15T15:54:02.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:02 smithi144 ceph-mon[36389]: pgmap v21: 65 pgs: 65 active+clean; 355 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 27 MiB/s wr, 256 op/s 2024-05-15T15:54:02.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:02 smithi049 ceph-mon[28399]: pgmap v21: 65 pgs: 65 active+clean; 355 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 27 MiB/s wr, 256 op/s 2024-05-15T15:54:03.043 INFO:tasks.workunit.client.1.smithi144.stdout:3/147: symlink d0/d1/d17/l37 0 2024-05-15T15:54:03.044 INFO:tasks.workunit.client.0.smithi049.stdout:4/320: link d18/d1e/c20 d18/d1e/d2a/d3b/c68 0 2024-05-15T15:54:03.044 INFO:tasks.workunit.client.1.smithi144.stdout:2/151: mknod d3/d10/d15/c2c 0 2024-05-15T15:54:03.045 INFO:tasks.workunit.client.1.smithi144.stdout:2/152: dread - d3/d7/f27 zero size 2024-05-15T15:54:03.046 INFO:tasks.workunit.client.1.smithi144.stdout:7/159: symlink d4/df/d10/l33 0 2024-05-15T15:54:03.046 INFO:tasks.workunit.client.1.smithi144.stdout:2/153: write d3/d7/f27 [235500,27720] 0 2024-05-15T15:54:03.047 INFO:tasks.workunit.client.0.smithi049.stdout:6/217: symlink dd/d26/l39 0 2024-05-15T15:54:03.048 INFO:tasks.workunit.client.0.smithi049.stdout:6/218: write dd/f33 [817806,71642] 0 2024-05-15T15:54:03.049 INFO:tasks.workunit.client.1.smithi144.stdout:9/138: rename d0/d8/fb to d0/d8/d21/d29/d34/d16/f3d 0 2024-05-15T15:54:03.049 INFO:tasks.workunit.client.1.smithi144.stdout:3/148: dwrite d0/d1/f21 [0,4194304] 0 2024-05-15T15:54:03.050 INFO:tasks.workunit.client.0.smithi049.stdout:4/321: mkdir d18/d69 0 2024-05-15T15:54:03.052 INFO:tasks.workunit.client.0.smithi049.stdout:1/304: creat d0/d4/dd/d5d/d65/d72/f78 x:0 0 0 2024-05-15T15:54:03.052 INFO:tasks.workunit.client.1.smithi144.stdout:7/160: dread d4/fc [0,4194304] 0 2024-05-15T15:54:03.053 INFO:tasks.workunit.client.0.smithi049.stdout:3/275: symlink d12/d14/l53 0 2024-05-15T15:54:03.054 INFO:tasks.workunit.client.0.smithi049.stdout:3/276: write d12/f29 [76990,77995] 0 2024-05-15T15:54:03.056 INFO:tasks.workunit.client.1.smithi144.stdout:1/155: creat d7/d8/d29/f2c x:0 0 0 2024-05-15T15:54:03.061 INFO:tasks.workunit.client.1.smithi144.stdout:8/160: symlink d2/l34 0 2024-05-15T15:54:03.063 INFO:tasks.workunit.client.0.smithi049.stdout:6/219: creat dd/f3a x:0 0 0 2024-05-15T15:54:03.066 INFO:tasks.workunit.client.1.smithi144.stdout:0/134: mknod d4/c31 0 2024-05-15T15:54:03.066 INFO:tasks.workunit.client.1.smithi144.stdout:0/135: fdatasync d4/d2d/f2e 0 2024-05-15T15:54:03.068 INFO:tasks.workunit.client.0.smithi049.stdout:7/238: getdents d0 0 2024-05-15T15:54:03.068 INFO:tasks.workunit.client.0.smithi049.stdout:6/220: dread dd/f15 [0,4194304] 0 2024-05-15T15:54:03.069 INFO:tasks.workunit.client.1.smithi144.stdout:7/161: stat d4/df/f11 0 2024-05-15T15:54:03.071 INFO:tasks.workunit.client.0.smithi049.stdout:4/322: rename d18/d1e/d2a/d3b/f51 to d18/d1e/d2a/f6a 0 2024-05-15T15:54:03.075 INFO:tasks.workunit.client.1.smithi144.stdout:1/156: dread d7/d9/f19 [0,4194304] 0 2024-05-15T15:54:03.075 INFO:tasks.workunit.client.1.smithi144.stdout:5/153: rename d0/dc/d19 to d0/dc/d12/d31/d35 0 2024-05-15T15:54:03.075 INFO:tasks.workunit.client.1.smithi144.stdout:4/170: rename d6 to d6/dd/d2d/d36 22 2024-05-15T15:54:03.076 INFO:tasks.workunit.client.0.smithi049.stdout:7/239: dwrite d0/d34/f35 [4194304,4194304] 0 2024-05-15T15:54:03.076 INFO:tasks.workunit.client.1.smithi144.stdout:5/154: chown d0/l1 124 1 2024-05-15T15:54:03.077 INFO:tasks.workunit.client.0.smithi049.stdout:7/240: fdatasync d0/d34/f3f 0 2024-05-15T15:54:03.087 INFO:tasks.workunit.client.0.smithi049.stdout:3/277: getdents d12 0 2024-05-15T15:54:03.089 INFO:tasks.workunit.client.1.smithi144.stdout:0/136: creat d4/d6/f32 x:0 0 0 2024-05-15T15:54:03.095 INFO:tasks.workunit.client.0.smithi049.stdout:4/323: symlink d18/d1e/d2a/d3b/l6b 0 2024-05-15T15:54:03.095 INFO:tasks.workunit.client.0.smithi049.stdout:7/241: dread d0/d6/f14 [0,4194304] 0 2024-05-15T15:54:03.096 INFO:tasks.workunit.client.0.smithi049.stdout:4/324: write d18/f49 [2179236,119882] 0 2024-05-15T15:54:03.096 INFO:tasks.workunit.client.0.smithi049.stdout:7/242: write d0/f4 [3920259,94266] 0 2024-05-15T15:54:03.097 INFO:tasks.workunit.client.1.smithi144.stdout:1/157: symlink d7/d8/d1e/l2d 0 2024-05-15T15:54:03.097 INFO:tasks.workunit.client.1.smithi144.stdout:1/158: fdatasync d7/d9/f19 0 2024-05-15T15:54:03.101 INFO:tasks.workunit.client.1.smithi144.stdout:3/149: rename d0/l35 to d0/d11/l38 0 2024-05-15T15:54:03.101 INFO:tasks.workunit.client.1.smithi144.stdout:3/150: stat d0/d11/d24/f22 0 2024-05-15T15:54:03.102 INFO:tasks.workunit.client.1.smithi144.stdout:2/154: truncate d3/d10/f1d 4173411 0 2024-05-15T15:54:03.102 INFO:tasks.workunit.client.1.smithi144.stdout:3/151: write d0/d11/f26 [2594988,65225] 0 2024-05-15T15:54:03.108 INFO:tasks.workunit.client.1.smithi144.stdout:3/152: dread d0/d11/d24/f22 [0,4194304] 0 2024-05-15T15:54:03.109 INFO:tasks.workunit.client.1.smithi144.stdout:3/153: chown d0/d11/f26 6 1 2024-05-15T15:54:03.110 INFO:tasks.workunit.client.1.smithi144.stdout:3/154: write d0/d11/f26 [4369790,41724] 0 2024-05-15T15:54:03.112 INFO:tasks.workunit.client.0.smithi049.stdout:7/243: chown d0/d6/d23/l31 2 1 2024-05-15T15:54:03.114 INFO:tasks.workunit.client.0.smithi049.stdout:5/235: sync 2024-05-15T15:54:03.114 INFO:tasks.workunit.client.1.smithi144.stdout:0/137: mknod d4/d6/c33 0 2024-05-15T15:54:03.115 INFO:tasks.workunit.client.0.smithi049.stdout:5/236: write dc/f19 [1183238,24249] 0 2024-05-15T15:54:03.115 INFO:tasks.workunit.client.0.smithi049.stdout:6/221: link dd/l29 dd/d17/d24/l3b 0 2024-05-15T15:54:03.116 INFO:tasks.workunit.client.1.smithi144.stdout:0/138: chown d4/d6/c12 499 1 2024-05-15T15:54:03.116 INFO:tasks.workunit.client.0.smithi049.stdout:3/278: dread d12/f1d [0,4194304] 0 2024-05-15T15:54:03.117 INFO:tasks.workunit.client.1.smithi144.stdout:1/159: mknod d7/d1d/c2e 0 2024-05-15T15:54:03.117 INFO:tasks.workunit.client.1.smithi144.stdout:0/139: chown d4/dc/d1d 2 1 2024-05-15T15:54:03.117 INFO:tasks.workunit.client.0.smithi049.stdout:3/279: write d12/f22 [3609779,10762] 0 2024-05-15T15:54:03.118 INFO:tasks.workunit.client.1.smithi144.stdout:3/155: dwrite d0/d1/f21 [0,4194304] 0 2024-05-15T15:54:03.118 INFO:tasks.workunit.client.1.smithi144.stdout:5/155: dwrite d0/f16 [0,4194304] 0 2024-05-15T15:54:03.120 INFO:tasks.workunit.client.0.smithi049.stdout:5/237: mknod dc/d29/c45 0 2024-05-15T15:54:03.122 INFO:tasks.workunit.client.0.smithi049.stdout:5/238: write dc/d29/f32 [1017800,55282] 0 2024-05-15T15:54:03.126 INFO:tasks.workunit.client.0.smithi049.stdout:5/239: write dc/d29/d2c/d31/f44 [719812,19099] 0 2024-05-15T15:54:03.127 INFO:tasks.workunit.client.0.smithi049.stdout:3/280: dwrite f7 [0,4194304] 0 2024-05-15T15:54:03.127 INFO:tasks.workunit.client.0.smithi049.stdout:3/281: dread - d12/f47 zero size 2024-05-15T15:54:03.136 INFO:tasks.workunit.client.0.smithi049.stdout:4/325: rename lb to d18/d1e/d2a/d50/l6c 0 2024-05-15T15:54:03.138 INFO:tasks.workunit.client.1.smithi144.stdout:8/161: creat d2/f35 x:0 0 0 2024-05-15T15:54:03.138 INFO:tasks.workunit.client.0.smithi049.stdout:5/240: dwrite dc/d29/f3c [0,4194304] 0 2024-05-15T15:54:03.139 INFO:tasks.workunit.client.1.smithi144.stdout:8/162: write d2/de/f14 [990754,106924] 0 2024-05-15T15:54:03.140 INFO:tasks.workunit.client.1.smithi144.stdout:9/139: getdents d0/d8 0 2024-05-15T15:54:03.142 INFO:tasks.workunit.client.0.smithi049.stdout:4/326: write d18/d1e/f1f [2189212,94757] 0 2024-05-15T15:54:03.145 INFO:tasks.workunit.client.0.smithi049.stdout:6/222: rename dd/d17/f1d to dd/d26/f3c 0 2024-05-15T15:54:03.146 INFO:tasks.workunit.client.1.smithi144.stdout:6/127: link d7/db/d11/c20 d7/db/c31 0 2024-05-15T15:54:03.150 INFO:tasks.workunit.client.0.smithi049.stdout:6/223: dread dd/f15 [0,4194304] 0 2024-05-15T15:54:03.150 INFO:tasks.workunit.client.1.smithi144.stdout:7/162: creat d4/df/f34 x:0 0 0 2024-05-15T15:54:03.154 INFO:tasks.workunit.client.0.smithi049.stdout:5/241: rename dc/c20 to dc/d29/c46 0 2024-05-15T15:54:03.155 INFO:tasks.workunit.client.0.smithi049.stdout:5/242: stat dc/dd/f14 0 2024-05-15T15:54:03.156 INFO:tasks.workunit.client.1.smithi144.stdout:7/163: dwrite d4/f6 [0,4194304] 0 2024-05-15T15:54:03.157 INFO:tasks.workunit.client.0.smithi049.stdout:3/282: creat d12/d14/f54 x:0 0 0 2024-05-15T15:54:03.157 INFO:tasks.workunit.client.0.smithi049.stdout:3/283: readlink d12/l13 0 2024-05-15T15:54:03.158 INFO:tasks.workunit.client.0.smithi049.stdout:3/284: fsync d12/d30/f3d 0 2024-05-15T15:54:03.158 INFO:tasks.workunit.client.1.smithi144.stdout:3/156: mkdir d0/d1/d17/d39 0 2024-05-15T15:54:03.165 INFO:tasks.workunit.client.1.smithi144.stdout:4/171: link d6/dd/d15/d1e/c23 d6/dd/d2e/c37 0 2024-05-15T15:54:03.165 INFO:tasks.workunit.client.1.smithi144.stdout:4/172: chown d6/dd/d2b 1638332159 1 2024-05-15T15:54:03.167 INFO:tasks.workunit.client.0.smithi049.stdout:4/327: rename f11 to d18/d1e/d2a/d50/d54/f6d 0 2024-05-15T15:54:03.168 INFO:tasks.workunit.client.1.smithi144.stdout:1/160: dwrite d7/d8/d1e/f25 [0,4194304] 0 2024-05-15T15:54:03.169 INFO:tasks.workunit.client.1.smithi144.stdout:8/163: rename d2/de/d12 to d2/de/d36 0 2024-05-15T15:54:03.170 INFO:tasks.workunit.client.0.smithi049.stdout:5/243: mkdir dc/dd/d47 0 2024-05-15T15:54:03.172 INFO:tasks.workunit.client.1.smithi144.stdout:9/140: mkdir d0/d8/d21/d29/d34/d10/d19/d3e 0 2024-05-15T15:54:03.179 INFO:tasks.workunit.client.1.smithi144.stdout:6/128: dread d7/fd [0,4194304] 0 2024-05-15T15:54:03.180 INFO:tasks.workunit.client.1.smithi144.stdout:3/157: write d0/d11/d24/f22 [3680517,10038] 0 2024-05-15T15:54:03.180 INFO:tasks.workunit.client.0.smithi049.stdout:3/285: creat d12/d14/f55 x:0 0 0 2024-05-15T15:54:03.182 INFO:tasks.workunit.client.0.smithi049.stdout:6/224: getdents dd/d17 0 2024-05-15T15:54:03.183 INFO:tasks.workunit.client.0.smithi049.stdout:6/225: write f4 [831623,42842] 0 2024-05-15T15:54:03.184 INFO:tasks.workunit.client.0.smithi049.stdout:4/328: symlink d18/d1e/l6e 0 2024-05-15T15:54:03.185 INFO:tasks.workunit.client.1.smithi144.stdout:2/155: sync 2024-05-15T15:54:03.186 INFO:tasks.workunit.client.1.smithi144.stdout:2/156: chown d3/d10/f29 134 1 2024-05-15T15:54:03.186 INFO:tasks.workunit.client.1.smithi144.stdout:2/157: fdatasync d3/d7/d20/d22/f24 0 2024-05-15T15:54:03.187 INFO:tasks.workunit.client.1.smithi144.stdout:1/161: symlink d7/df/d24/l2f 0 2024-05-15T15:54:03.187 INFO:tasks.workunit.client.1.smithi144.stdout:8/164: symlink d2/d24/d2d/l37 0 2024-05-15T15:54:03.188 INFO:tasks.workunit.client.1.smithi144.stdout:8/165: readlink d2/d24/d2d/l37 0 2024-05-15T15:54:03.189 INFO:tasks.workunit.client.1.smithi144.stdout:8/166: read d2/f3 [1228409,94832] 0 2024-05-15T15:54:03.191 INFO:tasks.workunit.client.1.smithi144.stdout:2/158: dwrite d3/d7/f1f [0,4194304] 0 2024-05-15T15:54:03.192 INFO:tasks.workunit.client.1.smithi144.stdout:2/159: write d3/d7/f18 [1461593,110029] 0 2024-05-15T15:54:03.223 INFO:tasks.workunit.client.0.smithi049.stdout:2/238: sync 2024-05-15T15:54:03.223 INFO:tasks.workunit.client.0.smithi049.stdout:2/239: chown d3/da/d15/d24 61913167 1 2024-05-15T15:54:03.223 INFO:tasks.workunit.client.0.smithi049.stdout:2/240: dread - d3/da/f45 zero size 2024-05-15T15:54:03.266 INFO:tasks.workunit.client.1.smithi144.stdout:7/164: link d4/df/d10/f1d d4/df/d25/f35 0 2024-05-15T15:54:03.266 INFO:tasks.workunit.client.1.smithi144.stdout:7/165: stat d4/df/f20 0 2024-05-15T15:54:03.267 INFO:tasks.workunit.client.0.smithi049.stdout:5/244: truncate dc/dd/f13 3849702 0 2024-05-15T15:54:03.267 INFO:tasks.workunit.client.0.smithi049.stdout:6/226: dread dd/d26/f3c [0,4194304] 0 2024-05-15T15:54:03.267 INFO:tasks.workunit.client.1.smithi144.stdout:3/158: write d0/f1d [708232,75379] 0 2024-05-15T15:54:03.269 INFO:tasks.workunit.client.1.smithi144.stdout:6/129: truncate d7/db/d11/f1b 575728 0 2024-05-15T15:54:03.269 INFO:tasks.workunit.client.1.smithi144.stdout:6/130: dread - d7/db/f2c zero size 2024-05-15T15:54:03.270 INFO:tasks.workunit.client.1.smithi144.stdout:6/131: write d7/db/f2c [897994,4847] 0 2024-05-15T15:54:03.270 INFO:tasks.workunit.client.1.smithi144.stdout:0/140: link d4/d6/c26 d4/df/d1f/d24/d2b/c34 0 2024-05-15T15:54:03.272 INFO:tasks.workunit.client.0.smithi049.stdout:6/227: creat dd/d17/d24/d2b/f3d x:0 0 0 2024-05-15T15:54:03.273 INFO:tasks.workunit.client.1.smithi144.stdout:4/173: creat d6/dd/d2b/f38 x:0 0 0 2024-05-15T15:54:03.279 INFO:tasks.workunit.client.1.smithi144.stdout:4/174: dwrite d6/dd/f22 [0,4194304] 0 2024-05-15T15:54:03.282 INFO:tasks.workunit.client.0.smithi049.stdout:3/286: getdents d12/d14 0 2024-05-15T15:54:03.283 INFO:tasks.workunit.client.1.smithi144.stdout:1/162: dread d7/df/f10 [0,4194304] 0 2024-05-15T15:54:03.283 INFO:tasks.workunit.client.1.smithi144.stdout:8/167: truncate d2/de/d36/f17 228499 0 2024-05-15T15:54:03.284 INFO:tasks.workunit.client.1.smithi144.stdout:8/168: readlink d2/l29 0 2024-05-15T15:54:03.284 INFO:tasks.workunit.client.1.smithi144.stdout:3/159: dread d0/f1d [0,4194304] 0 2024-05-15T15:54:03.284 INFO:tasks.workunit.client.1.smithi144.stdout:1/163: chown d7/d8/l1a 3 1 2024-05-15T15:54:03.284 INFO:tasks.workunit.client.0.smithi049.stdout:5/245: dread dc/dd/f17 [0,4194304] 0 2024-05-15T15:54:03.284 INFO:tasks.workunit.client.0.smithi049.stdout:4/329: dwrite d18/f31 [0,4194304] 0 2024-05-15T15:54:03.285 INFO:tasks.workunit.client.1.smithi144.stdout:2/160: creat d3/d10/d15/f2d x:0 0 0 2024-05-15T15:54:03.287 INFO:tasks.workunit.client.1.smithi144.stdout:9/141: creat d0/d8/d21/d29/d34/d10/d19/d3e/f3f x:0 0 0 2024-05-15T15:54:03.287 INFO:tasks.workunit.client.0.smithi049.stdout:5/246: truncate dc/dd/f3b 1896627 0 2024-05-15T15:54:03.288 INFO:tasks.workunit.client.1.smithi144.stdout:9/142: stat d0/f7 0 2024-05-15T15:54:03.289 INFO:tasks.workunit.client.1.smithi144.stdout:5/156: getdents d0/dc/d12 0 2024-05-15T15:54:03.290 INFO:tasks.workunit.client.1.smithi144.stdout:5/157: stat d0/f11 0 2024-05-15T15:54:03.290 INFO:tasks.workunit.client.0.smithi049.stdout:4/330: dread - d18/d1e/d2a/d50/f66 zero size 2024-05-15T15:54:03.291 INFO:tasks.workunit.client.0.smithi049.stdout:3/287: dwrite d12/f18 [0,4194304] 0 2024-05-15T15:54:03.292 INFO:tasks.workunit.client.0.smithi049.stdout:4/331: truncate d18/d21/f56 466825 0 2024-05-15T15:54:03.295 INFO:tasks.workunit.client.0.smithi049.stdout:5/247: unlink dc/le 0 2024-05-15T15:54:03.296 INFO:tasks.workunit.client.1.smithi144.stdout:9/143: dwrite d0/d8/d21/d29/d2d/f3a [0,4194304] 0 2024-05-15T15:54:03.298 INFO:tasks.workunit.client.1.smithi144.stdout:9/144: dread - d0/d8/d21/d29/d34/d10/d19/d3e/f3f zero size 2024-05-15T15:54:03.300 INFO:tasks.workunit.client.0.smithi049.stdout:3/288: dwrite d12/d14/f1e [0,4194304] 0 2024-05-15T15:54:03.306 INFO:tasks.workunit.client.0.smithi049.stdout:5/248: dwrite dc/f19 [0,4194304] 0 2024-05-15T15:54:03.306 INFO:tasks.workunit.client.1.smithi144.stdout:3/160: symlink d0/d11/l3a 0 2024-05-15T15:54:03.307 INFO:tasks.workunit.client.1.smithi144.stdout:1/164: symlink d7/df/d24/l30 0 2024-05-15T15:54:03.307 INFO:tasks.workunit.client.0.smithi049.stdout:5/249: readlink dc/l34 0 2024-05-15T15:54:03.309 INFO:tasks.workunit.client.1.smithi144.stdout:7/166: dwrite d4/fc [4194304,4194304] 0 2024-05-15T15:54:03.311 INFO:tasks.workunit.client.1.smithi144.stdout:7/167: write d4/f21 [275190,106619] 0 2024-05-15T15:54:03.312 INFO:tasks.workunit.client.1.smithi144.stdout:0/141: write d4/d6/f11 [1848496,8734] 0 2024-05-15T15:54:03.312 INFO:tasks.workunit.client.1.smithi144.stdout:2/161: creat d3/d10/f2e x:0 0 0 2024-05-15T15:54:03.318 INFO:tasks.workunit.client.1.smithi144.stdout:5/158: creat d0/f36 x:0 0 0 2024-05-15T15:54:03.319 INFO:tasks.workunit.client.1.smithi144.stdout:7/168: read d4/df/d10/f1d [2061387,42648] 0 2024-05-15T15:54:03.320 INFO:tasks.workunit.client.0.smithi049.stdout:5/250: creat dc/d29/d2c/d42/f48 x:0 0 0 2024-05-15T15:54:03.320 INFO:tasks.workunit.client.1.smithi144.stdout:5/159: dread d0/f1a [0,4194304] 0 2024-05-15T15:54:03.320 INFO:tasks.workunit.client.1.smithi144.stdout:1/165: dwrite f1 [8388608,4194304] 0 2024-05-15T15:54:03.321 INFO:tasks.workunit.client.1.smithi144.stdout:7/169: stat d4/c5 0 2024-05-15T15:54:03.322 INFO:tasks.workunit.client.1.smithi144.stdout:7/170: chown d4/f6 3834 1 2024-05-15T15:54:03.323 INFO:tasks.workunit.client.1.smithi144.stdout:6/132: mkdir d7/d32 0 2024-05-15T15:54:03.324 INFO:tasks.workunit.client.0.smithi049.stdout:4/332: dwrite d18/d21/f30 [0,4194304] 0 2024-05-15T15:54:03.328 INFO:tasks.workunit.client.1.smithi144.stdout:9/145: symlink d0/d8/d21/d29/d34/d10/d19/l40 0 2024-05-15T15:54:03.329 INFO:tasks.workunit.client.0.smithi049.stdout:3/289: truncate f7 3310891 0 2024-05-15T15:54:03.332 INFO:tasks.workunit.client.1.smithi144.stdout:4/175: link d6/d11/d30/f2f d6/dd/d34/f39 0 2024-05-15T15:54:03.335 INFO:tasks.workunit.client.1.smithi144.stdout:3/161: read d0/f1d [3579778,8953] 0 2024-05-15T15:54:03.336 INFO:tasks.workunit.client.1.smithi144.stdout:3/162: read d0/d1/f5 [87663,93090] 0 2024-05-15T15:54:03.343 INFO:tasks.workunit.client.0.smithi049.stdout:4/333: rename d18/l27 to d18/d1e/d2a/d50/l6f 0 2024-05-15T15:54:03.345 INFO:tasks.workunit.client.0.smithi049.stdout:5/251: truncate dc/dd/f3b 1618562 0 2024-05-15T15:54:03.346 INFO:tasks.workunit.client.1.smithi144.stdout:2/162: creat d3/d10/d15/f2f x:0 0 0 2024-05-15T15:54:03.346 INFO:tasks.workunit.client.1.smithi144.stdout:2/163: read - d3/d10/f29 zero size 2024-05-15T15:54:03.347 INFO:tasks.workunit.client.0.smithi049.stdout:3/290: rename d12/d14/f55 to d12/d14/d52/f56 0 2024-05-15T15:54:03.348 INFO:tasks.workunit.client.1.smithi144.stdout:8/169: truncate d2/de/f26 3997489 0 2024-05-15T15:54:03.348 INFO:tasks.workunit.client.0.smithi049.stdout:3/291: write d12/d30/d42/f46 [603294,63647] 0 2024-05-15T15:54:03.349 INFO:tasks.workunit.client.1.smithi144.stdout:8/170: write d2/f35 [733824,68343] 0 2024-05-15T15:54:03.351 INFO:tasks.workunit.client.0.smithi049.stdout:4/334: write d18/d1e/d2a/d36/f59 [916531,24028] 0 2024-05-15T15:54:03.358 INFO:tasks.workunit.client.0.smithi049.stdout:3/292: write f9 [1731696,110812] 0 2024-05-15T15:54:03.359 INFO:tasks.workunit.client.0.smithi049.stdout:3/293: write d12/d30/f3d [814055,4882] 0 2024-05-15T15:54:03.360 INFO:tasks.workunit.client.1.smithi144.stdout:5/160: dwrite d0/dc/fe [0,4194304] 0 2024-05-15T15:54:03.360 INFO:tasks.workunit.client.1.smithi144.stdout:4/176: mknod d6/d11/c3a 0 2024-05-15T15:54:03.361 INFO:tasks.workunit.client.1.smithi144.stdout:4/177: write d6/dd/f22 [4020900,94649] 0 2024-05-15T15:54:03.363 INFO:tasks.workunit.client.0.smithi049.stdout:4/335: write d18/d1e/d2a/d50/f5a [2735218,123172] 0 2024-05-15T15:54:03.365 INFO:tasks.workunit.client.1.smithi144.stdout:2/164: mknod d3/d7/c30 0 2024-05-15T15:54:03.368 INFO:tasks.workunit.client.0.smithi049.stdout:3/294: symlink d12/d30/d42/l57 0 2024-05-15T15:54:03.368 INFO:tasks.workunit.client.1.smithi144.stdout:4/178: dread d6/dd/f22 [0,4194304] 0 2024-05-15T15:54:03.370 INFO:tasks.workunit.client.0.smithi049.stdout:4/336: creat d18/f70 x:0 0 0 2024-05-15T15:54:03.373 INFO:tasks.workunit.client.1.smithi144.stdout:1/166: mkdir d7/d31 0 2024-05-15T15:54:03.374 INFO:tasks.workunit.client.0.smithi049.stdout:4/337: readlink d18/d1e/d2a/l34 0 2024-05-15T15:54:03.376 INFO:tasks.workunit.client.0.smithi049.stdout:5/252: dread dc/dd/f3d [0,4194304] 0 2024-05-15T15:54:03.376 INFO:tasks.workunit.client.1.smithi144.stdout:6/133: mknod d7/db/c33 0 2024-05-15T15:54:03.378 INFO:tasks.workunit.client.0.smithi049.stdout:4/338: symlink d18/l71 0 2024-05-15T15:54:03.380 INFO:tasks.workunit.client.0.smithi049.stdout:3/295: write d12/d14/f4e [2277724,39132] 0 2024-05-15T15:54:03.382 INFO:tasks.workunit.client.1.smithi144.stdout:6/134: dwrite d7/db/f29 [0,4194304] 0 2024-05-15T15:54:03.382 INFO:tasks.workunit.client.0.smithi049.stdout:5/253: mknod dc/d29/d2c/d31/c49 0 2024-05-15T15:54:03.385 INFO:tasks.workunit.client.0.smithi049.stdout:4/339: rmdir d18/d1e/d2a/d36 39 2024-05-15T15:54:03.390 INFO:tasks.workunit.client.0.smithi049.stdout:4/340: dwrite f17 [0,4194304] 0 2024-05-15T15:54:03.391 INFO:tasks.workunit.client.0.smithi049.stdout:4/341: dread - d18/f4d zero size 2024-05-15T15:54:03.391 INFO:tasks.workunit.client.1.smithi144.stdout:2/165: rename d3/d7/d20/d22/f23 to d3/d10/d15/f31 0 2024-05-15T15:54:03.396 INFO:tasks.workunit.client.0.smithi049.stdout:3/296: creat d12/d14/d32/f58 x:0 0 0 2024-05-15T15:54:03.405 INFO:tasks.workunit.client.0.smithi049.stdout:5/254: rename dc/d29/d2c to dc/d4a 0 2024-05-15T15:54:03.406 INFO:tasks.workunit.client.1.smithi144.stdout:7/171: creat d4/f36 x:0 0 0 2024-05-15T15:54:03.407 INFO:tasks.workunit.client.0.smithi049.stdout:4/342: mknod d18/d21/d2b/d4e/c72 0 2024-05-15T15:54:03.409 INFO:tasks.workunit.client.1.smithi144.stdout:9/146: creat d0/d8/d21/d29/d34/f41 x:0 0 0 2024-05-15T15:54:03.410 INFO:tasks.workunit.client.1.smithi144.stdout:9/147: write d0/d8/d21/f38 [297245,39850] 0 2024-05-15T15:54:03.410 INFO:tasks.workunit.client.0.smithi049.stdout:5/255: fdatasync dc/dd/f3d 0 2024-05-15T15:54:03.411 INFO:tasks.workunit.client.0.smithi049.stdout:5/256: readlink dc/dd/d10/l27 0 2024-05-15T15:54:03.414 INFO:tasks.workunit.client.0.smithi049.stdout:3/297: rename d12/d14/f3c to d12/d14/d32/f59 0 2024-05-15T15:54:03.418 INFO:tasks.workunit.client.1.smithi144.stdout:4/179: dread f0 [0,4194304] 0 2024-05-15T15:54:03.422 INFO:tasks.workunit.client.1.smithi144.stdout:0/142: getdents d4/dc/d1d 0 2024-05-15T15:54:03.422 INFO:tasks.workunit.client.0.smithi049.stdout:3/298: dwrite d12/f22 [4194304,4194304] 0 2024-05-15T15:54:03.422 INFO:tasks.workunit.client.1.smithi144.stdout:0/143: write d4/df/d1f/f2c [875781,33308] 0 2024-05-15T15:54:03.423 INFO:tasks.workunit.client.1.smithi144.stdout:0/144: read - d4/d6/f32 zero size 2024-05-15T15:54:03.426 INFO:tasks.workunit.client.0.smithi049.stdout:4/343: creat d18/d1e/d2a/d50/f73 x:0 0 0 2024-05-15T15:54:03.426 INFO:tasks.workunit.client.0.smithi049.stdout:5/257: getdents dc/dd/d39 0 2024-05-15T15:54:03.427 INFO:tasks.workunit.client.1.smithi144.stdout:1/167: write d7/d9/fe [2755951,98695] 0 2024-05-15T15:54:03.428 INFO:tasks.workunit.client.0.smithi049.stdout:3/299: symlink d12/d14/d32/l5a 0 2024-05-15T15:54:03.430 INFO:tasks.workunit.client.0.smithi049.stdout:4/344: chown d18/d1e/d2a/d36/d5d 0 1 2024-05-15T15:54:03.432 INFO:tasks.workunit.client.0.smithi049.stdout:5/258: mknod dc/d4a/c4b 0 2024-05-15T15:54:03.432 INFO:tasks.workunit.client.1.smithi144.stdout:9/148: write d0/d8/d21/d29/d34/d16/f3d [855059,47026] 0 2024-05-15T15:54:03.435 INFO:tasks.workunit.client.1.smithi144.stdout:3/163: getdents d0 0 2024-05-15T15:54:03.435 INFO:tasks.workunit.client.0.smithi049.stdout:3/300: stat d12/d14/f54 0 2024-05-15T15:54:03.437 INFO:tasks.workunit.client.1.smithi144.stdout:7/172: dwrite d4/df/f16 [0,4194304] 0 2024-05-15T15:54:03.439 INFO:tasks.workunit.client.0.smithi049.stdout:4/345: creat d18/d1e/d2a/d3b/f74 x:0 0 0 2024-05-15T15:54:03.439 INFO:tasks.workunit.client.1.smithi144.stdout:5/161: rename d0/dc/d1f/f27 to d0/dc/d24/f37 0 2024-05-15T15:54:03.440 INFO:tasks.workunit.client.1.smithi144.stdout:4/180: fsync d6/dd/f22 0 2024-05-15T15:54:03.442 INFO:tasks.workunit.client.0.smithi049.stdout:3/301: mkdir d12/d14/d32/d5b 0 2024-05-15T15:54:03.442 INFO:tasks.workunit.client.0.smithi049.stdout:3/302: fdatasync ff 0 2024-05-15T15:54:03.442 INFO:tasks.workunit.client.1.smithi144.stdout:5/162: chown d0/dc/d12/c1e 719061 1 2024-05-15T15:54:03.443 INFO:tasks.workunit.client.1.smithi144.stdout:0/145: symlink d4/df/d1f/l35 0 2024-05-15T15:54:03.443 INFO:tasks.workunit.client.1.smithi144.stdout:4/181: chown d6/d11/d30/f2f 405255 1 2024-05-15T15:54:03.444 INFO:tasks.workunit.client.0.smithi049.stdout:3/303: write d12/f18 [4296997,77970] 0 2024-05-15T15:54:03.445 INFO:tasks.workunit.client.1.smithi144.stdout:3/164: dwrite d0/f2e [0,4194304] 0 2024-05-15T15:54:03.445 INFO:tasks.workunit.client.1.smithi144.stdout:8/171: getdents d2/de 0 2024-05-15T15:54:03.445 INFO:tasks.workunit.client.1.smithi144.stdout:0/146: stat d4/df/d1f/d24/d2b 0 2024-05-15T15:54:03.446 INFO:tasks.workunit.client.0.smithi049.stdout:3/304: stat d12/d30/d42/f48 0 2024-05-15T15:54:03.446 INFO:tasks.workunit.client.1.smithi144.stdout:4/182: write d6/dd/d15/f31 [1593119,120638] 0 2024-05-15T15:54:03.448 INFO:tasks.workunit.client.0.smithi049.stdout:5/259: fdatasync dc/dd/f13 0 2024-05-15T15:54:03.452 INFO:tasks.workunit.client.1.smithi144.stdout:9/149: creat d0/d8/d21/d29/d34/d3c/f42 x:0 0 0 2024-05-15T15:54:03.454 INFO:tasks.workunit.client.0.smithi049.stdout:3/305: dwrite d12/d26/f40 [0,4194304] 0 2024-05-15T15:54:03.461 INFO:tasks.workunit.client.1.smithi144.stdout:6/135: rename d7/l12 to d7/d1e/d28/l34 0 2024-05-15T15:54:03.467 INFO:tasks.workunit.client.1.smithi144.stdout:3/165: mkdir d0/d11/d3b 0 2024-05-15T15:54:03.468 INFO:tasks.workunit.client.1.smithi144.stdout:3/166: write d0/f15 [2447661,114779] 0 2024-05-15T15:54:03.473 INFO:tasks.workunit.client.0.smithi049.stdout:3/306: dread f8 [4194304,4194304] 0 2024-05-15T15:54:03.473 INFO:tasks.workunit.client.1.smithi144.stdout:8/172: dread d2/fd [0,4194304] 0 2024-05-15T15:54:03.474 INFO:tasks.workunit.client.0.smithi049.stdout:3/307: readlink d12/d14/d32/l35 0 2024-05-15T15:54:03.475 INFO:tasks.workunit.client.0.smithi049.stdout:4/346: dwrite d18/f1b [0,4194304] 0 2024-05-15T15:54:03.475 INFO:tasks.workunit.client.1.smithi144.stdout:4/183: unlink d6/d11/f25 0 2024-05-15T15:54:03.479 INFO:tasks.workunit.client.1.smithi144.stdout:1/168: creat d7/d1d/f32 x:0 0 0 2024-05-15T15:54:03.479 INFO:tasks.workunit.client.1.smithi144.stdout:1/169: fdatasync d7/d9/d22/f28 0 2024-05-15T15:54:03.481 INFO:tasks.workunit.client.0.smithi049.stdout:3/308: mknod d12/d30/c5c 0 2024-05-15T15:54:03.482 INFO:tasks.workunit.client.0.smithi049.stdout:3/309: chown d12/d30/d42/c49 9206 1 2024-05-15T15:54:03.483 INFO:tasks.workunit.client.0.smithi049.stdout:8/256: sync 2024-05-15T15:54:03.483 INFO:tasks.workunit.client.0.smithi049.stdout:9/207: sync 2024-05-15T15:54:03.486 INFO:tasks.workunit.client.0.smithi049.stdout:3/310: chown d12/d14/d52 1403863 1 2024-05-15T15:54:03.487 INFO:tasks.workunit.client.1.smithi144.stdout:2/166: getdents d3/d7/d20/d22 0 2024-05-15T15:54:03.489 INFO:tasks.workunit.client.1.smithi144.stdout:6/136: unlink d7/c2a 0 2024-05-15T15:54:03.489 INFO:tasks.workunit.client.0.smithi049.stdout:9/208: dread d0/d8/d1b/f32 [0,4194304] 0 2024-05-15T15:54:03.490 INFO:tasks.workunit.client.0.smithi049.stdout:8/257: creat d2/d4f/f51 x:0 0 0 2024-05-15T15:54:03.491 INFO:tasks.workunit.client.1.smithi144.stdout:3/167: chown d0/d1/cc 114010 1 2024-05-15T15:54:03.492 INFO:tasks.workunit.client.1.smithi144.stdout:3/168: stat d0 0 2024-05-15T15:54:03.492 INFO:tasks.workunit.client.0.smithi049.stdout:8/258: chown d2/d18/f47 840 1 2024-05-15T15:54:03.492 INFO:tasks.workunit.client.0.smithi049.stdout:3/311: rmdir d12/d14/d52 39 2024-05-15T15:54:03.493 INFO:tasks.workunit.client.0.smithi049.stdout:5/260: dwrite dc/d4a/f36 [0,4194304] 0 2024-05-15T15:54:03.494 INFO:tasks.workunit.client.1.smithi144.stdout:0/147: mknod d4/df/d1f/d24/d2b/c36 0 2024-05-15T15:54:03.497 INFO:tasks.workunit.client.0.smithi049.stdout:5/261: dread dc/dd/f14 [0,4194304] 0 2024-05-15T15:54:03.498 INFO:tasks.workunit.client.1.smithi144.stdout:8/173: mkdir d2/d1a/d38 0 2024-05-15T15:54:03.499 INFO:tasks.workunit.client.0.smithi049.stdout:4/347: getdents d18/d1e/d2a/d36/d5d 0 2024-05-15T15:54:03.502 INFO:tasks.workunit.client.0.smithi049.stdout:9/209: chown d0/d8/d9/fb 1792 1 2024-05-15T15:54:03.503 INFO:tasks.workunit.client.1.smithi144.stdout:4/184: truncate d6/dd/d15/f1c 888773 0 2024-05-15T15:54:03.505 INFO:tasks.workunit.client.0.smithi049.stdout:8/259: rmdir d2/d14/d1d/d3e 39 2024-05-15T15:54:03.506 INFO:tasks.workunit.client.0.smithi049.stdout:8/260: read d2/d18/f47 [4002,125452] 0 2024-05-15T15:54:03.512 INFO:tasks.workunit.client.0.smithi049.stdout:3/312: symlink d12/d30/l5d 0 2024-05-15T15:54:03.514 INFO:tasks.workunit.client.1.smithi144.stdout:5/163: truncate d0/dc/d24/d28/d2d/f2b 341207 0 2024-05-15T15:54:03.516 INFO:tasks.workunit.client.1.smithi144.stdout:8/174: dread d2/de/f26 [0,4194304] 0 2024-05-15T15:54:03.517 INFO:tasks.workunit.client.1.smithi144.stdout:7/173: creat d4/f37 x:0 0 0 2024-05-15T15:54:03.517 INFO:tasks.workunit.client.0.smithi049.stdout:5/262: rmdir dc/d4a 39 2024-05-15T15:54:03.519 INFO:tasks.workunit.client.1.smithi144.stdout:2/167: symlink d3/d10/l32 0 2024-05-15T15:54:03.520 INFO:tasks.workunit.client.0.smithi049.stdout:9/210: mknod d0/d8/d21/c47 0 2024-05-15T15:54:03.523 INFO:tasks.workunit.client.1.smithi144.stdout:6/137: truncate d7/fd 982011 0 2024-05-15T15:54:03.527 INFO:tasks.workunit.client.0.smithi049.stdout:3/313: creat d12/d14/d32/f5e x:0 0 0 2024-05-15T15:54:03.527 INFO:tasks.workunit.client.1.smithi144.stdout:0/148: mknod d4/dc/c37 0 2024-05-15T15:54:03.531 INFO:tasks.workunit.client.1.smithi144.stdout:9/150: creat d0/d8/d21/d29/d34/f43 x:0 0 0 2024-05-15T15:54:03.532 INFO:tasks.workunit.client.0.smithi049.stdout:9/211: creat d0/d8/d11/f48 x:0 0 0 2024-05-15T15:54:03.532 INFO:tasks.workunit.client.1.smithi144.stdout:5/164: creat d0/dc/d1f/f38 x:0 0 0 2024-05-15T15:54:03.533 INFO:tasks.workunit.client.1.smithi144.stdout:9/151: truncate d0/d8/d21/d29/d34/d10/f27 1087441 0 2024-05-15T15:54:03.533 INFO:tasks.workunit.client.1.smithi144.stdout:8/175: creat d2/d1a/f39 x:0 0 0 2024-05-15T15:54:03.535 INFO:tasks.workunit.client.1.smithi144.stdout:9/152: chown d0/d8/d1b 129056 1 2024-05-15T15:54:03.535 INFO:tasks.workunit.client.1.smithi144.stdout:2/168: dread d3/f5 [0,4194304] 0 2024-05-15T15:54:03.535 INFO:tasks.workunit.client.1.smithi144.stdout:7/174: dread - d4/df/f22 zero size 2024-05-15T15:54:03.535 INFO:tasks.workunit.client.0.smithi049.stdout:5/263: creat dc/dd/d39/f4c x:0 0 0 2024-05-15T15:54:03.536 INFO:tasks.workunit.client.1.smithi144.stdout:7/175: fdatasync d4/df/f11 0 2024-05-15T15:54:03.537 INFO:tasks.workunit.client.0.smithi049.stdout:5/264: write dc/f19 [3907269,37442] 0 2024-05-15T15:54:03.538 INFO:tasks.workunit.client.0.smithi049.stdout:5/265: chown dc/dd/l28 6 1 2024-05-15T15:54:03.541 INFO:tasks.workunit.client.1.smithi144.stdout:6/138: dread d7/db/d11/f13 [0,4194304] 0 2024-05-15T15:54:03.543 INFO:tasks.workunit.client.1.smithi144.stdout:5/165: dwrite d0/dc/d12/f13 [0,4194304] 0 2024-05-15T15:54:03.544 INFO:tasks.workunit.client.1.smithi144.stdout:2/169: dwrite d3/d7/f27 [0,4194304] 0 2024-05-15T15:54:03.545 INFO:tasks.workunit.client.0.smithi049.stdout:5/266: unlink dc/dd/d10/c2e 0 2024-05-15T15:54:03.546 INFO:tasks.workunit.client.1.smithi144.stdout:2/170: write d3/d7/f1f [673612,111618] 0 2024-05-15T15:54:03.548 INFO:tasks.workunit.client.0.smithi049.stdout:3/314: dread d12/d14/f1b [0,4194304] 0 2024-05-15T15:54:03.554 INFO:tasks.workunit.client.1.smithi144.stdout:5/166: dwrite d0/dc/fe [0,4194304] 0 2024-05-15T15:54:03.554 INFO:tasks.workunit.client.0.smithi049.stdout:5/267: creat dc/dd/d47/f4d x:0 0 0 2024-05-15T15:54:03.557 INFO:tasks.workunit.client.0.smithi049.stdout:3/315: fdatasync d12/d14/f43 0 2024-05-15T15:54:03.558 INFO:tasks.workunit.client.1.smithi144.stdout:7/176: mkdir d4/df/d25/d38 0 2024-05-15T15:54:03.559 INFO:tasks.workunit.client.1.smithi144.stdout:5/167: read d0/f11 [1878232,53574] 0 2024-05-15T15:54:03.560 INFO:tasks.workunit.client.1.smithi144.stdout:7/177: write d4/f2f [978111,55557] 0 2024-05-15T15:54:03.560 INFO:tasks.workunit.client.1.smithi144.stdout:5/168: chown d0/dc/d12/f18 2858 1 2024-05-15T15:54:03.562 INFO:tasks.workunit.client.0.smithi049.stdout:5/268: dwrite dc/dd/f3b [0,4194304] 0 2024-05-15T15:54:03.563 INFO:tasks.workunit.client.1.smithi144.stdout:9/153: rename d0/d8/d21/d29/d34 to d0/d8/d21/d29/d2d/d44 0 2024-05-15T15:54:03.564 INFO:tasks.workunit.client.0.smithi049.stdout:5/269: chown dc/f3e 987219 1 2024-05-15T15:54:03.564 INFO:tasks.workunit.client.1.smithi144.stdout:7/178: dread d4/f21 [0,4194304] 0 2024-05-15T15:54:03.565 INFO:tasks.workunit.client.1.smithi144.stdout:9/154: read - d0/d8/d21/d29/d2d/d44/f26 zero size 2024-05-15T15:54:03.566 INFO:tasks.workunit.client.1.smithi144.stdout:7/179: dread - d4/df/d25/f29 zero size 2024-05-15T15:54:03.566 INFO:tasks.workunit.client.1.smithi144.stdout:9/155: chown d0/d8/l2b 157304179 1 2024-05-15T15:54:03.567 INFO:tasks.workunit.client.1.smithi144.stdout:7/180: readlink d4/l9 0 2024-05-15T15:54:03.569 INFO:tasks.workunit.client.0.smithi049.stdout:3/316: creat d12/d14/d52/f5f x:0 0 0 2024-05-15T15:54:03.569 INFO:tasks.workunit.client.1.smithi144.stdout:4/185: truncate d6/dd/d15/f31 1135026 0 2024-05-15T15:54:03.572 INFO:tasks.workunit.client.0.smithi049.stdout:3/317: mknod d12/d26/c60 0 2024-05-15T15:54:03.572 INFO:tasks.workunit.client.1.smithi144.stdout:9/156: write d0/d8/d21/d29/d2d/d44/d3c/f42 [795567,104436] 0 2024-05-15T15:54:03.573 INFO:tasks.workunit.client.1.smithi144.stdout:4/186: dread - d6/dd/d34/f39 zero size 2024-05-15T15:54:03.574 INFO:tasks.workunit.client.1.smithi144.stdout:3/169: dwrite d0/d11/f14 [0,4194304] 0 2024-05-15T15:54:03.576 INFO:tasks.workunit.client.0.smithi049.stdout:3/318: creat d12/d14/d52/f61 x:0 0 0 2024-05-15T15:54:03.576 INFO:tasks.workunit.client.1.smithi144.stdout:9/157: fsync d0/d8/d21/d29/d2d/d44/d10/f17 0 2024-05-15T15:54:03.577 INFO:tasks.workunit.client.1.smithi144.stdout:2/171: mknod d3/d7/d20/c33 0 2024-05-15T15:54:03.578 INFO:tasks.workunit.client.1.smithi144.stdout:4/187: chown d6/dd/f22 231689 1 2024-05-15T15:54:03.578 INFO:tasks.workunit.client.1.smithi144.stdout:3/170: write d0/d11/f26 [2172026,35832] 0 2024-05-15T15:54:03.578 INFO:tasks.workunit.client.1.smithi144.stdout:0/149: dwrite d4/f5 [0,4194304] 0 2024-05-15T15:54:03.579 INFO:tasks.workunit.client.1.smithi144.stdout:2/172: dread - d3/d10/f29 zero size 2024-05-15T15:54:03.580 INFO:tasks.workunit.client.1.smithi144.stdout:8/176: mkdir d2/d3a 0 2024-05-15T15:54:03.580 INFO:tasks.workunit.client.0.smithi049.stdout:3/319: chown d12/d14/f1a 1 1 2024-05-15T15:54:03.581 INFO:tasks.workunit.client.1.smithi144.stdout:4/188: chown d6/d11/d30/f2f 25221 1 2024-05-15T15:54:03.582 INFO:tasks.workunit.client.1.smithi144.stdout:9/158: chown d0/d8/d21/d29/l22 1348 1 2024-05-15T15:54:03.582 INFO:tasks.workunit.client.0.smithi049.stdout:3/320: write d12/f3f [214983,68902] 0 2024-05-15T15:54:03.582 INFO:tasks.workunit.client.1.smithi144.stdout:8/177: fsync d2/de/d36/f1b 0 2024-05-15T15:54:03.585 INFO:tasks.workunit.client.1.smithi144.stdout:7/181: rmdir d4 39 2024-05-15T15:54:03.586 INFO:tasks.workunit.client.1.smithi144.stdout:5/169: dread d0/dc/f10 [0,4194304] 0 2024-05-15T15:54:03.587 INFO:tasks.workunit.client.1.smithi144.stdout:5/170: write d0/dc/d12/f13 [1204810,111491] 0 2024-05-15T15:54:03.590 INFO:tasks.workunit.client.0.smithi049.stdout:3/321: creat d12/d14/d32/f62 x:0 0 0 2024-05-15T15:54:03.590 INFO:tasks.workunit.client.1.smithi144.stdout:8/178: dread d2/f35 [0,4194304] 0 2024-05-15T15:54:03.592 INFO:tasks.workunit.client.1.smithi144.stdout:6/139: mkdir d7/db/d35 0 2024-05-15T15:54:03.593 INFO:tasks.workunit.client.1.smithi144.stdout:1/170: sync 2024-05-15T15:54:03.595 INFO:tasks.workunit.client.1.smithi144.stdout:5/171: dwrite d0/dc/d12/f13 [0,4194304] 0 2024-05-15T15:54:03.595 INFO:tasks.workunit.client.0.smithi049.stdout:3/322: rmdir d12/d26 39 2024-05-15T15:54:03.595 INFO:tasks.workunit.client.0.smithi049.stdout:3/323: read - d12/d14/f54 zero size 2024-05-15T15:54:03.600 INFO:tasks.workunit.client.0.smithi049.stdout:3/324: mkdir d12/d30/d63 0 2024-05-15T15:54:03.606 INFO:tasks.workunit.client.0.smithi049.stdout:3/325: rename d12/d14/d32/d5b to d12/d14/d32/d64 0 2024-05-15T15:54:03.610 INFO:tasks.workunit.client.0.smithi049.stdout:3/326: read d12/d14/f17 [8081076,41543] 0 2024-05-15T15:54:03.611 INFO:tasks.workunit.client.1.smithi144.stdout:0/150: unlink d4/dc/d1d/l25 0 2024-05-15T15:54:03.611 INFO:tasks.workunit.client.1.smithi144.stdout:0/151: readlink d4/l1e 0 2024-05-15T15:54:03.612 INFO:tasks.workunit.client.1.smithi144.stdout:0/152: chown d4/l7 2806837 1 2024-05-15T15:54:03.612 INFO:tasks.workunit.client.1.smithi144.stdout:0/153: chown d4/df/d1f 1 1 2024-05-15T15:54:03.613 INFO:tasks.workunit.client.1.smithi144.stdout:0/154: truncate d4/df/d1f/f2f 1894021 0 2024-05-15T15:54:03.614 INFO:tasks.workunit.client.1.smithi144.stdout:9/159: creat d0/d8/d21/d29/d2d/d44/d3c/f45 x:0 0 0 2024-05-15T15:54:03.615 INFO:tasks.workunit.client.0.smithi049.stdout:3/327: chown d12/d14/l31 238590 1 2024-05-15T15:54:03.616 INFO:tasks.workunit.client.1.smithi144.stdout:7/182: dread - d4/df/f34 zero size 2024-05-15T15:54:03.618 INFO:tasks.workunit.client.1.smithi144.stdout:8/179: fdatasync d2/de/fc 0 2024-05-15T15:54:03.621 INFO:tasks.workunit.client.1.smithi144.stdout:6/140: dread d7/fd [0,4194304] 0 2024-05-15T15:54:03.621 INFO:tasks.workunit.client.1.smithi144.stdout:1/171: rmdir d7/d9 39 2024-05-15T15:54:03.622 INFO:tasks.workunit.client.1.smithi144.stdout:3/171: creat d0/d1/d17/d39/f3c x:0 0 0 2024-05-15T15:54:03.626 INFO:tasks.workunit.client.1.smithi144.stdout:0/155: mknod d4/d6/c38 0 2024-05-15T15:54:03.629 INFO:tasks.workunit.client.1.smithi144.stdout:3/172: creat d0/f3d x:0 0 0 2024-05-15T15:54:03.631 INFO:tasks.workunit.client.1.smithi144.stdout:2/173: rename d3/d7/f17 to d3/f34 0 2024-05-15T15:54:03.632 INFO:tasks.workunit.client.1.smithi144.stdout:0/156: rmdir d4 39 2024-05-15T15:54:03.634 INFO:tasks.workunit.client.1.smithi144.stdout:6/141: mknod d7/d32/c36 0 2024-05-15T15:54:03.634 INFO:tasks.workunit.client.1.smithi144.stdout:9/160: write d0/f18 [443568,115539] 0 2024-05-15T15:54:03.636 INFO:tasks.workunit.client.1.smithi144.stdout:9/161: read d0/f7 [536900,125820] 0 2024-05-15T15:54:03.637 INFO:tasks.workunit.client.1.smithi144.stdout:3/173: creat d0/d11/d24/f3e x:0 0 0 2024-05-15T15:54:03.639 INFO:tasks.workunit.client.1.smithi144.stdout:3/174: write d0/d1/f21 [158553,40158] 0 2024-05-15T15:54:03.639 INFO:tasks.workunit.client.1.smithi144.stdout:4/189: rename d6/d11 to d6/dd/d34/d3b 0 2024-05-15T15:54:03.640 INFO:tasks.workunit.client.1.smithi144.stdout:8/180: dwrite d2/fd [0,4194304] 0 2024-05-15T15:54:03.645 INFO:tasks.workunit.client.1.smithi144.stdout:2/174: creat d3/d10/f35 x:0 0 0 2024-05-15T15:54:03.645 INFO:tasks.workunit.client.1.smithi144.stdout:2/175: stat d3/d10/l32 0 2024-05-15T15:54:03.645 INFO:tasks.workunit.client.1.smithi144.stdout:9/162: creat d0/d8/d21/d29/d2d/d44/d16/f46 x:0 0 0 2024-05-15T15:54:03.646 INFO:tasks.workunit.client.1.smithi144.stdout:5/172: rename d0/f36 to d0/dc/d1f/f39 0 2024-05-15T15:54:03.649 INFO:tasks.workunit.client.1.smithi144.stdout:0/157: mknod d4/df/d1f/d24/d2b/c39 0 2024-05-15T15:54:03.650 INFO:tasks.workunit.client.1.smithi144.stdout:0/158: dread - d4/f22 zero size 2024-05-15T15:54:03.651 INFO:tasks.workunit.client.1.smithi144.stdout:5/173: dwrite d0/f16 [0,4194304] 0 2024-05-15T15:54:03.652 INFO:tasks.workunit.client.1.smithi144.stdout:9/163: write d0/f7 [4630800,111293] 0 2024-05-15T15:54:03.671 INFO:tasks.workunit.client.0.smithi049.stdout:0/242: sync 2024-05-15T15:54:03.671 INFO:tasks.workunit.client.0.smithi049.stdout:2/241: sync 2024-05-15T15:54:03.671 INFO:tasks.workunit.client.0.smithi049.stdout:7/244: sync 2024-05-15T15:54:03.671 INFO:tasks.workunit.client.0.smithi049.stdout:1/305: sync 2024-05-15T15:54:03.672 INFO:tasks.workunit.client.1.smithi144.stdout:8/181: mknod d2/d24/d2d/d30/c3b 0 2024-05-15T15:54:03.673 INFO:tasks.workunit.client.1.smithi144.stdout:8/182: chown d2/de/f33 161345886 1 2024-05-15T15:54:03.673 INFO:tasks.workunit.client.1.smithi144.stdout:8/183: readlink d2/l4 0 2024-05-15T15:54:03.673 INFO:tasks.workunit.client.0.smithi049.stdout:1/306: chown d0/d4/c59 1 1 2024-05-15T15:54:03.673 INFO:tasks.workunit.client.1.smithi144.stdout:6/142: creat d7/db/f37 x:0 0 0 2024-05-15T15:54:03.674 INFO:tasks.workunit.client.1.smithi144.stdout:0/159: mkdir d4/d6/d3a 0 2024-05-15T15:54:03.675 INFO:tasks.workunit.client.1.smithi144.stdout:2/176: creat d3/d7/f36 x:0 0 0 2024-05-15T15:54:03.677 INFO:tasks.workunit.client.1.smithi144.stdout:2/177: write d3/d10/d15/f26 [214663,68502] 0 2024-05-15T15:54:03.677 INFO:tasks.workunit.client.1.smithi144.stdout:5/174: symlink d0/dc/d1f/l3a 0 2024-05-15T15:54:03.680 INFO:tasks.workunit.client.1.smithi144.stdout:0/160: dwrite d4/f5 [0,4194304] 0 2024-05-15T15:54:03.681 INFO:tasks.workunit.client.1.smithi144.stdout:1/172: rename d7/d8 to d7/d1d/d33 0 2024-05-15T15:54:03.681 INFO:tasks.workunit.client.1.smithi144.stdout:0/161: chown d4/df/l13 4008976 1 2024-05-15T15:54:03.682 INFO:tasks.workunit.client.0.smithi049.stdout:7/245: rmdir d0/d6/da/d3d 39 2024-05-15T15:54:03.683 INFO:tasks.workunit.client.0.smithi049.stdout:7/246: write d0/d6/f18 [4680996,1926] 0 2024-05-15T15:54:03.683 INFO:tasks.workunit.client.1.smithi144.stdout:9/164: symlink d0/d8/d21/d29/d2d/d44/d10/d19/l47 0 2024-05-15T15:54:03.683 INFO:tasks.workunit.client.1.smithi144.stdout:9/165: dread - d0/d8/d21/f2a zero size 2024-05-15T15:54:03.685 INFO:tasks.workunit.client.1.smithi144.stdout:9/166: write d0/d8/d21/d29/d2d/d44/f43 [364275,115054] 0 2024-05-15T15:54:03.686 INFO:tasks.workunit.client.1.smithi144.stdout:9/167: chown d0/d8/c36 1761 1 2024-05-15T15:54:03.687 INFO:tasks.workunit.client.1.smithi144.stdout:6/143: mknod d7/d1e/d28/c38 0 2024-05-15T15:54:03.693 INFO:tasks.workunit.client.1.smithi144.stdout:2/178: mkdir d3/d7/d20/d37 0 2024-05-15T15:54:03.694 INFO:tasks.workunit.client.1.smithi144.stdout:2/179: chown d3/d7/l21 2049881470 1 2024-05-15T15:54:03.695 INFO:tasks.workunit.client.1.smithi144.stdout:4/190: dwrite d6/dd/d15/f1c [0,4194304] 0 2024-05-15T15:54:03.696 INFO:tasks.workunit.client.1.smithi144.stdout:4/191: write d6/f32 [307904,108547] 0 2024-05-15T15:54:03.696 INFO:tasks.workunit.client.0.smithi049.stdout:1/307: creat d0/d4/f79 x:0 0 0 2024-05-15T15:54:03.697 INFO:tasks.workunit.client.0.smithi049.stdout:2/242: write d3/db/f1a [436720,32084] 0 2024-05-15T15:54:03.699 INFO:tasks.workunit.client.0.smithi049.stdout:1/308: write d0/d4/dd/de/d36/f6a [449493,79135] 0 2024-05-15T15:54:03.699 INFO:tasks.workunit.client.1.smithi144.stdout:3/175: rename d0/l2 to d0/d11/d3b/l3f 0 2024-05-15T15:54:03.700 INFO:tasks.workunit.client.1.smithi144.stdout:3/176: chown d0/d1/cd 2222 1 2024-05-15T15:54:03.701 INFO:tasks.workunit.client.1.smithi144.stdout:3/177: dread - d0/d1/f23 zero size 2024-05-15T15:54:03.701 INFO:tasks.workunit.client.1.smithi144.stdout:7/183: sync 2024-05-15T15:54:03.701 INFO:tasks.workunit.client.0.smithi049.stdout:0/243: getdents dc 0 2024-05-15T15:54:03.702 INFO:tasks.workunit.client.1.smithi144.stdout:7/184: chown d4/df/d25 138 1 2024-05-15T15:54:03.702 INFO:tasks.workunit.client.0.smithi049.stdout:0/244: chown dc/d13/d14/d2a/f2c 255868923 1 2024-05-15T15:54:03.703 INFO:tasks.workunit.client.1.smithi144.stdout:1/173: mkdir d7/d1d/d33/d1e/d34 0 2024-05-15T15:54:03.703 INFO:tasks.workunit.client.1.smithi144.stdout:1/174: stat d7/d31 0 2024-05-15T15:54:03.704 INFO:tasks.workunit.client.0.smithi049.stdout:2/243: creat d3/da/f46 x:0 0 0 2024-05-15T15:54:03.705 INFO:tasks.workunit.client.1.smithi144.stdout:0/162: creat d4/df/d1f/d24/d2b/f3b x:0 0 0 2024-05-15T15:54:03.708 INFO:tasks.workunit.client.0.smithi049.stdout:0/245: rmdir dc/d32 39 2024-05-15T15:54:03.710 INFO:tasks.workunit.client.1.smithi144.stdout:6/144: fdatasync d7/fd 0 2024-05-15T15:54:03.712 INFO:tasks.workunit.client.0.smithi049.stdout:2/244: mkdir d3/db/d1d/d1e/d2b/d3b/d47 0 2024-05-15T15:54:03.717 INFO:tasks.workunit.client.1.smithi144.stdout:2/180: rename d3/d7/c8 to d3/d7/c38 0 2024-05-15T15:54:03.717 INFO:tasks.workunit.client.1.smithi144.stdout:1/175: dread f6 [0,4194304] 0 2024-05-15T15:54:03.719 INFO:tasks.workunit.client.0.smithi049.stdout:2/245: rename d3/db/d1d/d1e/d2b/f31 to d3/db/d1d/d1e/d2b/d2e/f48 0 2024-05-15T15:54:03.722 INFO:tasks.workunit.client.1.smithi144.stdout:4/192: readlink d6/dd/d34/d3b/d30/db/d1d/l28 0 2024-05-15T15:54:03.722 INFO:tasks.workunit.client.1.smithi144.stdout:2/181: dwrite d3/d7/f1f [0,4194304] 0 2024-05-15T15:54:03.723 INFO:tasks.workunit.client.1.smithi144.stdout:5/175: truncate d0/dc/d12/f13 3135776 0 2024-05-15T15:54:03.724 INFO:tasks.workunit.client.1.smithi144.stdout:3/178: mkdir d0/d1/d17/d39/d40 0 2024-05-15T15:54:03.725 INFO:tasks.workunit.client.0.smithi049.stdout:0/246: rename dc/d13/d14/f2e to dc/d13/d48/f4f 0 2024-05-15T15:54:03.726 INFO:tasks.workunit.client.1.smithi144.stdout:3/179: write d0/d11/d24/f22 [5134563,64830] 0 2024-05-15T15:54:03.727 INFO:tasks.workunit.client.0.smithi049.stdout:2/246: mknod d3/db/d1d/d1e/c49 0 2024-05-15T15:54:03.727 INFO:tasks.workunit.client.1.smithi144.stdout:3/180: read - d0/d1/f23 zero size 2024-05-15T15:54:03.729 INFO:tasks.workunit.client.1.smithi144.stdout:0/163: symlink d4/df/d1f/d24/l3c 0 2024-05-15T15:54:03.731 INFO:tasks.workunit.client.0.smithi049.stdout:0/247: creat dc/d13/d48/f50 x:0 0 0 2024-05-15T15:54:03.734 INFO:tasks.workunit.client.1.smithi144.stdout:3/181: dwrite d0/d11/d24/f32 [0,4194304] 0 2024-05-15T15:54:03.734 INFO:tasks.workunit.client.1.smithi144.stdout:4/193: mknod d6/dd/d34/c3c 0 2024-05-15T15:54:03.735 INFO:tasks.workunit.client.0.smithi049.stdout:0/248: dread dc/f34 [0,4194304] 0 2024-05-15T15:54:03.735 INFO:tasks.workunit.client.0.smithi049.stdout:2/247: dwrite d3/db/d1d/d1e/d2b/d2e/f41 [0,4194304] 0 2024-05-15T15:54:03.739 INFO:tasks.workunit.client.1.smithi144.stdout:2/182: mknod d3/d10/c39 0 2024-05-15T15:54:03.740 INFO:tasks.workunit.client.1.smithi144.stdout:8/184: rename d2/d1a/c1f to d2/d24/d2d/c3c 0 2024-05-15T15:54:03.742 INFO:tasks.workunit.client.1.smithi144.stdout:8/185: chown d2/de/d36 0 1 2024-05-15T15:54:03.742 INFO:tasks.workunit.client.1.smithi144.stdout:0/164: dread d4/d6/f11 [0,4194304] 0 2024-05-15T15:54:03.746 INFO:tasks.workunit.client.1.smithi144.stdout:1/176: dwrite d7/d9/fe [0,4194304] 0 2024-05-15T15:54:03.747 INFO:tasks.workunit.client.1.smithi144.stdout:7/185: mknod d4/df/d25/d38/c39 0 2024-05-15T15:54:03.747 INFO:tasks.workunit.client.0.smithi049.stdout:0/249: creat dc/d13/d14/d2f/f51 x:0 0 0 2024-05-15T15:54:03.752 INFO:tasks.workunit.client.0.smithi049.stdout:2/248: rmdir d3/db/d1d 39 2024-05-15T15:54:03.753 INFO:tasks.workunit.client.1.smithi144.stdout:7/186: dwrite d4/fe [0,4194304] 0 2024-05-15T15:54:03.756 INFO:tasks.workunit.client.0.smithi049.stdout:0/250: creat dc/d13/d14/d2f/f52 x:0 0 0 2024-05-15T15:54:03.758 INFO:tasks.workunit.client.0.smithi049.stdout:0/251: truncate dc/d13/d14/f4e 140272 0 2024-05-15T15:54:03.760 INFO:tasks.workunit.client.1.smithi144.stdout:5/176: dread d0/dc/d24/f37 [0,4194304] 0 2024-05-15T15:54:03.766 INFO:tasks.workunit.client.0.smithi049.stdout:0/252: creat dc/d13/d14/d33/f53 x:0 0 0 2024-05-15T15:54:03.768 INFO:tasks.workunit.client.0.smithi049.stdout:2/249: creat d3/db/f4a x:0 0 0 2024-05-15T15:54:03.769 INFO:tasks.workunit.client.1.smithi144.stdout:3/182: rmdir d0/d1/d17 39 2024-05-15T15:54:03.771 INFO:tasks.workunit.client.1.smithi144.stdout:3/183: truncate d0/f15 4716614 0 2024-05-15T15:54:03.773 INFO:tasks.workunit.client.0.smithi049.stdout:2/250: unlink d3/da/d15/f3f 0 2024-05-15T15:54:03.773 INFO:tasks.workunit.client.1.smithi144.stdout:4/194: dread d6/dd/d15/f31 [0,4194304] 0 2024-05-15T15:54:03.773 INFO:tasks.workunit.client.1.smithi144.stdout:3/184: read d0/d11/d24/f22 [2383833,61903] 0 2024-05-15T15:54:03.773 INFO:tasks.workunit.client.1.smithi144.stdout:7/187: dread d4/df/d25/f35 [0,4194304] 0 2024-05-15T15:54:03.775 INFO:tasks.workunit.client.1.smithi144.stdout:4/195: chown d6/dd/d34/d3b/d30/f1b 2 1 2024-05-15T15:54:03.776 INFO:tasks.workunit.client.1.smithi144.stdout:2/183: dread d3/f5 [0,4194304] 0 2024-05-15T15:54:03.777 INFO:tasks.workunit.client.0.smithi049.stdout:2/251: mknod d3/da/d15/c4b 0 2024-05-15T15:54:03.778 INFO:tasks.workunit.client.0.smithi049.stdout:2/252: write d3/da/d15/f27 [2627991,57406] 0 2024-05-15T15:54:03.778 INFO:tasks.workunit.client.1.smithi144.stdout:1/177: unlink d7/d1d/f2b 0 2024-05-15T15:54:03.784 INFO:tasks.workunit.client.1.smithi144.stdout:9/168: getdents d0/d8 0 2024-05-15T15:54:03.787 INFO:tasks.workunit.client.1.smithi144.stdout:6/145: creat d7/db/f39 x:0 0 0 2024-05-15T15:54:03.788 INFO:tasks.workunit.client.1.smithi144.stdout:6/146: fsync d7/db/f2c 0 2024-05-15T15:54:03.788 INFO:tasks.workunit.client.1.smithi144.stdout:9/169: dread d0/f18 [0,4194304] 0 2024-05-15T15:54:03.789 INFO:tasks.workunit.client.1.smithi144.stdout:5/177: rmdir d0 39 2024-05-15T15:54:03.791 INFO:tasks.workunit.client.0.smithi049.stdout:2/253: creat d3/f4c x:0 0 0 2024-05-15T15:54:03.793 INFO:tasks.workunit.client.1.smithi144.stdout:8/186: getdents d2/d3a 0 2024-05-15T15:54:03.795 INFO:tasks.workunit.client.1.smithi144.stdout:7/188: creat d4/df/d25/f3a x:0 0 0 2024-05-15T15:54:03.797 INFO:tasks.workunit.client.0.smithi049.stdout:2/254: creat d3/f4d x:0 0 0 2024-05-15T15:54:03.797 INFO:tasks.workunit.client.0.smithi049.stdout:2/255: dread - d3/da/d15/f39 zero size 2024-05-15T15:54:03.798 INFO:tasks.workunit.client.1.smithi144.stdout:0/165: dwrite f2 [0,4194304] 0 2024-05-15T15:54:03.798 INFO:tasks.workunit.client.1.smithi144.stdout:4/196: symlink d6/dd/d2b/l3d 0 2024-05-15T15:54:03.799 INFO:tasks.workunit.client.0.smithi049.stdout:2/256: truncate d3/da/f45 272150 0 2024-05-15T15:54:03.803 INFO:tasks.workunit.client.1.smithi144.stdout:7/189: dwrite d4/df/f34 [0,4194304] 0 2024-05-15T15:54:03.804 INFO:tasks.workunit.client.0.smithi049.stdout:2/257: rmdir d3/db/d1d/d1e/d2b 39 2024-05-15T15:54:03.804 INFO:tasks.workunit.client.1.smithi144.stdout:9/170: symlink d0/d8/d21/d29/d2d/l48 0 2024-05-15T15:54:03.804 INFO:tasks.workunit.client.1.smithi144.stdout:7/190: stat d4/df/f32 0 2024-05-15T15:54:03.804 INFO:tasks.workunit.client.0.smithi049.stdout:2/258: readlink d3/da/l3a 0 2024-05-15T15:54:03.805 INFO:tasks.workunit.client.1.smithi144.stdout:5/178: write d0/dc/fe [3377093,11070] 0 2024-05-15T15:54:03.807 INFO:tasks.workunit.client.1.smithi144.stdout:5/179: chown d0/dc/d12/d31/d35/c2a 90 1 2024-05-15T15:54:03.807 INFO:tasks.workunit.client.1.smithi144.stdout:5/180: fsync d0/f22 0 2024-05-15T15:54:03.810 INFO:tasks.workunit.client.1.smithi144.stdout:3/185: rename d0/d11/d24/f32 to d0/d11/d24/f41 0 2024-05-15T15:54:03.812 INFO:tasks.workunit.client.0.smithi049.stdout:2/259: write d3/db/d1d/d1e/d2b/d2e/f48 [778212,92734] 0 2024-05-15T15:54:03.818 INFO:tasks.workunit.client.1.smithi144.stdout:2/184: mkdir d3/d7/d20/d37/d3a 0 2024-05-15T15:54:03.819 INFO:tasks.workunit.client.0.smithi049.stdout:2/260: dwrite d3/d8/fe [0,4194304] 0 2024-05-15T15:54:03.821 INFO:tasks.workunit.client.1.smithi144.stdout:6/147: mkdir d7/db/d3a 0 2024-05-15T15:54:03.825 INFO:tasks.workunit.client.1.smithi144.stdout:7/191: creat d4/df/d10/f3b x:0 0 0 2024-05-15T15:54:03.828 INFO:tasks.workunit.client.1.smithi144.stdout:7/192: dread - d4/f37 zero size 2024-05-15T15:54:03.828 INFO:tasks.workunit.client.1.smithi144.stdout:8/187: dwrite d2/f3 [0,4194304] 0 2024-05-15T15:54:03.830 INFO:tasks.workunit.client.1.smithi144.stdout:7/193: stat d4/f1c 0 2024-05-15T15:54:03.831 INFO:tasks.workunit.client.1.smithi144.stdout:4/197: mknod d6/dd/c3e 0 2024-05-15T15:54:03.832 INFO:tasks.workunit.client.1.smithi144.stdout:8/188: write d2/de/f14 [342721,71030] 0 2024-05-15T15:54:03.833 INFO:tasks.workunit.client.1.smithi144.stdout:2/185: mkdir d3/d7/d20/d3b 0 2024-05-15T15:54:03.834 INFO:tasks.workunit.client.0.smithi049.stdout:2/261: symlink d3/db/d1d/d1e/d2b/d3b/d47/l4e 0 2024-05-15T15:54:03.835 INFO:tasks.workunit.client.1.smithi144.stdout:6/148: dwrite d7/db/f39 [0,4194304] 0 2024-05-15T15:54:03.836 INFO:tasks.workunit.client.1.smithi144.stdout:4/198: write d6/dd/d34/d3b/d30/ff [8647484,99006] 0 2024-05-15T15:54:03.836 INFO:tasks.workunit.client.1.smithi144.stdout:9/171: truncate d0/d8/d21/d29/d2d/d44/d10/f17 3552470 0 2024-05-15T15:54:03.839 INFO:tasks.workunit.client.1.smithi144.stdout:4/199: write d6/dd/d34/d3b/f21 [3280577,95910] 0 2024-05-15T15:54:03.840 INFO:tasks.workunit.client.0.smithi049.stdout:2/262: unlink d3/da/d15/c40 0 2024-05-15T15:54:03.845 INFO:tasks.workunit.client.1.smithi144.stdout:0/166: rmdir d4/dc/d1d 0 2024-05-15T15:54:03.846 INFO:tasks.workunit.client.1.smithi144.stdout:0/167: write d4/dc/f17 [3531548,36724] 0 2024-05-15T15:54:03.846 INFO:tasks.workunit.client.1.smithi144.stdout:7/194: write d4/df/f1a [1108006,26613] 0 2024-05-15T15:54:03.847 INFO:tasks.workunit.client.1.smithi144.stdout:0/168: read - d4/f22 zero size 2024-05-15T15:54:03.847 INFO:tasks.workunit.client.0.smithi049.stdout:2/263: dwrite d3/d8/f36 [0,4194304] 0 2024-05-15T15:54:03.848 INFO:tasks.workunit.client.1.smithi144.stdout:0/169: truncate f2 4498207 0 2024-05-15T15:54:03.851 INFO:tasks.workunit.client.1.smithi144.stdout:8/189: creat d2/d1a/f3d x:0 0 0 2024-05-15T15:54:03.851 INFO:tasks.workunit.client.1.smithi144.stdout:9/172: rmdir d0/d8 39 2024-05-15T15:54:03.854 INFO:tasks.workunit.client.0.smithi049.stdout:2/264: creat d3/db/d1d/d1e/d2b/d3b/d42/f4f x:0 0 0 2024-05-15T15:54:03.854 INFO:tasks.workunit.client.1.smithi144.stdout:8/190: dread d2/d1a/f31 [0,4194304] 0 2024-05-15T15:54:03.855 INFO:tasks.workunit.client.0.smithi049.stdout:2/265: fdatasync d3/db/f43 0 2024-05-15T15:54:03.859 INFO:tasks.workunit.client.1.smithi144.stdout:3/186: rename d0/d11/l38 to d0/d1/d17/d39/d40/l42 0 2024-05-15T15:54:03.860 INFO:tasks.workunit.client.1.smithi144.stdout:1/178: sync 2024-05-15T15:54:03.861 INFO:tasks.workunit.client.0.smithi049.stdout:2/266: creat d3/d8/f50 x:0 0 0 2024-05-15T15:54:03.861 INFO:tasks.workunit.client.1.smithi144.stdout:1/179: dread - d7/d9/d22/f28 zero size 2024-05-15T15:54:03.863 INFO:tasks.workunit.client.0.smithi049.stdout:2/267: fdatasync d3/d8/f14 0 2024-05-15T15:54:03.865 INFO:tasks.workunit.client.0.smithi049.stdout:6/228: sync 2024-05-15T15:54:03.865 INFO:tasks.workunit.client.1.smithi144.stdout:5/181: write d0/dc/d24/d28/d2d/f2b [926972,57044] 0 2024-05-15T15:54:03.866 INFO:tasks.workunit.client.0.smithi049.stdout:6/229: write f8 [962447,33358] 0 2024-05-15T15:54:03.867 INFO:tasks.workunit.client.1.smithi144.stdout:5/182: write d0/dc/d24/d28/d2d/f2b [995516,22051] 0 2024-05-15T15:54:03.870 INFO:tasks.workunit.client.0.smithi049.stdout:2/268: dwrite d3/db/d1d/f20 [0,4194304] 0 2024-05-15T15:54:03.871 INFO:tasks.workunit.client.0.smithi049.stdout:2/269: fsync d3/da/f2c 0 2024-05-15T15:54:03.872 INFO:tasks.workunit.client.0.smithi049.stdout:2/270: dread d3/da/f45 [0,4194304] 0 2024-05-15T15:54:03.873 INFO:tasks.workunit.client.0.smithi049.stdout:2/271: chown d3/db/f1a 3097 1 2024-05-15T15:54:03.874 INFO:tasks.workunit.client.0.smithi049.stdout:2/272: chown d3/db/d1d/d1e/d2b/d3b/d47/l4e 34912317 1 2024-05-15T15:54:03.876 INFO:tasks.workunit.client.0.smithi049.stdout:2/273: write d3/da/f45 [799074,108578] 0 2024-05-15T15:54:03.962 INFO:tasks.workunit.client.1.smithi144.stdout:0/170: mknod d4/df/d1f/c3d 0 2024-05-15T15:54:03.964 INFO:tasks.workunit.client.1.smithi144.stdout:9/173: chown d0/d8/d21/d29/d2d/f3a 32798171 1 2024-05-15T15:54:04.000 INFO:tasks.workunit.client.0.smithi049.stdout:4/348: sync 2024-05-15T15:54:04.000 INFO:tasks.workunit.client.0.smithi049.stdout:8/261: sync 2024-05-15T15:54:04.001 INFO:tasks.workunit.client.0.smithi049.stdout:8/262: fsync d2/d14/d1b/f2b 0 2024-05-15T15:54:04.006 INFO:tasks.workunit.client.1.smithi144.stdout:4/200: creat d6/dd/d2d/f3f x:0 0 0 2024-05-15T15:54:04.009 INFO:tasks.workunit.client.0.smithi049.stdout:2/274: mknod d3/da/c51 0 2024-05-15T15:54:04.012 INFO:tasks.workunit.client.1.smithi144.stdout:7/195: mkdir d4/d3c 0 2024-05-15T15:54:04.013 INFO:tasks.workunit.client.1.smithi144.stdout:7/196: chown d4/f36 1 1 2024-05-15T15:54:04.013 INFO:tasks.workunit.client.0.smithi049.stdout:2/275: dread d3/db/d1d/d1e/d2b/d2e/f41 [0,4194304] 0 2024-05-15T15:54:04.014 INFO:tasks.workunit.client.0.smithi049.stdout:2/276: dread - d3/da/f46 zero size 2024-05-15T15:54:04.016 INFO:tasks.workunit.client.0.smithi049.stdout:6/230: getdents dd/d17/d24/d2b/d36 0 2024-05-15T15:54:04.016 INFO:tasks.workunit.client.0.smithi049.stdout:8/263: dread d2/d18/f47 [0,4194304] 0 2024-05-15T15:54:04.017 INFO:tasks.workunit.client.0.smithi049.stdout:6/231: chown dd/d17/d24/d2b/f3d 0 1 2024-05-15T15:54:04.017 INFO:tasks.workunit.client.0.smithi049.stdout:8/264: read d2/de/d27/f39 [3169495,117333] 0 2024-05-15T15:54:04.018 INFO:tasks.workunit.client.0.smithi049.stdout:6/232: write f0 [8250866,72394] 0 2024-05-15T15:54:04.018 INFO:tasks.workunit.client.1.smithi144.stdout:5/183: creat d0/dc/d1f/f3b x:0 0 0 2024-05-15T15:54:04.018 INFO:tasks.workunit.client.1.smithi144.stdout:6/149: dwrite d7/db/d11/f1b [0,4194304] 0 2024-05-15T15:54:04.020 INFO:tasks.workunit.client.1.smithi144.stdout:0/171: creat d4/df/d1f/f3e x:0 0 0 2024-05-15T15:54:04.021 INFO:tasks.workunit.client.0.smithi049.stdout:4/349: symlink d18/d1e/d2a/d50/l75 0 2024-05-15T15:54:04.023 INFO:tasks.workunit.client.0.smithi049.stdout:8/265: write d2/d4f/f51 [664747,34558] 0 2024-05-15T15:54:04.027 INFO:tasks.workunit.client.1.smithi144.stdout:8/191: creat d2/d3a/f3e x:0 0 0 2024-05-15T15:54:04.031 INFO:tasks.workunit.client.1.smithi144.stdout:4/201: unlink d6/dd/d2b/f38 0 2024-05-15T15:54:04.032 INFO:tasks.workunit.client.0.smithi049.stdout:8/266: dwrite d2/d14/d1b/f2b [0,4194304] 0 2024-05-15T15:54:04.032 INFO:tasks.workunit.client.1.smithi144.stdout:4/202: write d6/dd/d34/d3b/f21 [1188219,58371] 0 2024-05-15T15:54:04.036 INFO:tasks.workunit.client.0.smithi049.stdout:2/277: mkdir d3/d52 0 2024-05-15T15:54:04.038 INFO:tasks.workunit.client.1.smithi144.stdout:1/180: mknod d7/df/c35 0 2024-05-15T15:54:04.038 INFO:tasks.workunit.client.0.smithi049.stdout:8/267: dwrite f0 [0,4194304] 0 2024-05-15T15:54:04.040 INFO:tasks.workunit.client.0.smithi049.stdout:4/350: mkdir d18/d76 0 2024-05-15T15:54:04.041 INFO:tasks.workunit.client.1.smithi144.stdout:7/197: creat d4/df/d25/d38/f3d x:0 0 0 2024-05-15T15:54:04.043 INFO:tasks.workunit.client.1.smithi144.stdout:2/186: getdents d3 0 2024-05-15T15:54:04.048 INFO:tasks.workunit.client.0.smithi049.stdout:4/351: readlink d18/d1e/d2a/d50/l6f 0 2024-05-15T15:54:04.053 INFO:tasks.workunit.client.1.smithi144.stdout:9/174: link d0/d8/d21/d29/d2d/d44/f26 d0/d8/d21/d29/d2d/d44/d10/d19/f49 0 2024-05-15T15:54:04.055 INFO:tasks.workunit.client.1.smithi144.stdout:8/192: dread d2/de/d36/f17 [0,4194304] 0 2024-05-15T15:54:04.056 INFO:tasks.workunit.client.0.smithi049.stdout:6/233: link dd/l1b dd/d17/d24/d2b/l3e 0 2024-05-15T15:54:04.057 INFO:tasks.workunit.client.0.smithi049.stdout:6/234: chown dd/c18 1245743 1 2024-05-15T15:54:04.058 INFO:tasks.workunit.client.0.smithi049.stdout:6/235: write dd/d17/d24/d2b/f3d [23895,95618] 0 2024-05-15T15:54:04.060 INFO:tasks.workunit.client.0.smithi049.stdout:8/268: dread d2/d18/f3c [0,4194304] 0 2024-05-15T15:54:04.060 INFO:tasks.workunit.client.1.smithi144.stdout:4/203: mkdir d6/dd/d34/d3b/d30/db/d1d/d40 0 2024-05-15T15:54:04.061 INFO:tasks.workunit.client.1.smithi144.stdout:4/204: read d6/dd/d34/d3b/d30/f1b [664287,76032] 0 2024-05-15T15:54:04.062 INFO:tasks.workunit.client.1.smithi144.stdout:3/187: creat d0/d1/f43 x:0 0 0 2024-05-15T15:54:04.066 INFO:tasks.workunit.client.0.smithi049.stdout:4/352: getdents d18/d1e/d2a/d50 0 2024-05-15T15:54:04.066 INFO:tasks.workunit.client.1.smithi144.stdout:5/184: dread d0/f11 [0,4194304] 0 2024-05-15T15:54:04.066 INFO:tasks.workunit.client.1.smithi144.stdout:5/185: truncate d0/dc/d1f/f38 753240 0 2024-05-15T15:54:04.066 INFO:tasks.workunit.client.1.smithi144.stdout:7/198: creat d4/df/d25/d38/f3e x:0 0 0 2024-05-15T15:54:04.067 INFO:tasks.workunit.client.0.smithi049.stdout:4/353: write d18/d21/f30 [5235185,20930] 0 2024-05-15T15:54:04.067 INFO:tasks.workunit.client.1.smithi144.stdout:0/172: write d4/d6/fb [3945141,59762] 0 2024-05-15T15:54:04.067 INFO:tasks.workunit.client.1.smithi144.stdout:2/187: write d3/d10/d15/f31 [2868588,12880] 0 2024-05-15T15:54:04.067 INFO:tasks.workunit.client.0.smithi049.stdout:4/354: write f5 [705434,37678] 0 2024-05-15T15:54:04.073 INFO:tasks.workunit.client.1.smithi144.stdout:9/175: unlink d0/d8/d21/d29/d2d/d44/f43 0 2024-05-15T15:54:04.073 INFO:tasks.workunit.client.0.smithi049.stdout:4/355: rmdir d18/d1e/d2a/d36 39 2024-05-15T15:54:04.074 INFO:tasks.workunit.client.0.smithi049.stdout:4/356: chown d18/f1b 1 1 2024-05-15T15:54:04.075 INFO:tasks.workunit.client.0.smithi049.stdout:4/357: write d18/f1b [5092798,10504] 0 2024-05-15T15:54:04.075 INFO:tasks.workunit.client.1.smithi144.stdout:8/193: mkdir d2/de/d3f 0 2024-05-15T15:54:04.076 INFO:tasks.workunit.client.1.smithi144.stdout:4/205: rmdir d6/dd/d2e 39 2024-05-15T15:54:04.076 INFO:tasks.workunit.client.1.smithi144.stdout:8/194: write d2/d1a/f3d [806031,57714] 0 2024-05-15T15:54:04.077 INFO:tasks.workunit.client.0.smithi049.stdout:8/269: dread d2/fb [0,4194304] 0 2024-05-15T15:54:04.078 INFO:tasks.workunit.client.0.smithi049.stdout:8/270: chown d2/c2a 4621 1 2024-05-15T15:54:04.080 INFO:tasks.workunit.client.0.smithi049.stdout:8/271: mknod d2/de/c52 0 2024-05-15T15:54:04.082 INFO:tasks.workunit.client.1.smithi144.stdout:7/199: creat d4/df/d25/d38/f3f x:0 0 0 2024-05-15T15:54:04.083 INFO:tasks.workunit.client.1.smithi144.stdout:7/200: chown d4/l9 1906530 1 2024-05-15T15:54:04.084 INFO:tasks.workunit.client.1.smithi144.stdout:2/188: creat d3/d7/d20/d37/f3c x:0 0 0 2024-05-15T15:54:04.085 INFO:tasks.workunit.client.1.smithi144.stdout:2/189: fdatasync d3/f9 0 2024-05-15T15:54:04.085 INFO:tasks.workunit.client.0.smithi049.stdout:4/358: dwrite d18/d1e/d2a/d36/f4a [0,4194304] 0 2024-05-15T15:54:04.088 INFO:tasks.workunit.client.1.smithi144.stdout:7/201: dwrite d4/df/d25/f3a [0,4194304] 0 2024-05-15T15:54:04.089 INFO:tasks.workunit.client.0.smithi049.stdout:8/272: dwrite d2/d18/f4e [0,4194304] 0 2024-05-15T15:54:04.093 INFO:tasks.workunit.client.1.smithi144.stdout:2/190: dwrite d3/f5 [0,4194304] 0 2024-05-15T15:54:04.094 INFO:tasks.workunit.client.0.smithi049.stdout:8/273: write d2/fb [2744404,119265] 0 2024-05-15T15:54:04.098 INFO:tasks.workunit.client.0.smithi049.stdout:4/359: dwrite d18/f53 [0,4194304] 0 2024-05-15T15:54:04.099 INFO:tasks.workunit.client.1.smithi144.stdout:1/181: link d7/df/c16 d7/df/c36 0 2024-05-15T15:54:04.100 INFO:tasks.workunit.client.0.smithi049.stdout:8/274: creat d2/de/f53 x:0 0 0 2024-05-15T15:54:04.102 INFO:tasks.workunit.client.1.smithi144.stdout:2/191: dwrite d3/d10/f2e [0,4194304] 0 2024-05-15T15:54:04.103 INFO:tasks.workunit.client.1.smithi144.stdout:5/186: rename d0/dc/f10 to d0/dc/d1f/d29/f3c 0 2024-05-15T15:54:04.104 INFO:tasks.workunit.client.1.smithi144.stdout:2/192: stat d3/d7/l21 0 2024-05-15T15:54:04.105 INFO:tasks.workunit.client.1.smithi144.stdout:6/150: getdents d7/db 0 2024-05-15T15:54:04.109 INFO:tasks.workunit.client.0.smithi049.stdout:4/360: symlink d18/d69/l77 0 2024-05-15T15:54:04.109 INFO:tasks.workunit.client.1.smithi144.stdout:6/151: readlink d7/d1e/l2d 0 2024-05-15T15:54:04.109 INFO:tasks.workunit.client.1.smithi144.stdout:2/193: chown d3/d7/f1a 62 1 2024-05-15T15:54:04.110 INFO:tasks.workunit.client.1.smithi144.stdout:2/194: stat d3/d10/d15/l1b 0 2024-05-15T15:54:04.110 INFO:tasks.workunit.client.1.smithi144.stdout:6/152: chown d7/d1e/l2d 5764 1 2024-05-15T15:54:04.111 INFO:tasks.workunit.client.0.smithi049.stdout:4/361: write d18/d1e/d2a/f64 [70952,116225] 0 2024-05-15T15:54:04.111 INFO:tasks.workunit.client.1.smithi144.stdout:2/195: dread - d3/d10/d15/f2d zero size 2024-05-15T15:54:04.112 INFO:tasks.workunit.client.0.smithi049.stdout:8/275: mkdir d2/d14/d54 0 2024-05-15T15:54:04.113 INFO:tasks.workunit.client.0.smithi049.stdout:8/276: fdatasync d2/d14/f1e 0 2024-05-15T15:54:04.116 INFO:tasks.workunit.client.0.smithi049.stdout:8/277: read - d2/d14/f43 zero size 2024-05-15T15:54:04.117 INFO:tasks.workunit.client.1.smithi144.stdout:7/202: mkdir d4/df/d25/d40 0 2024-05-15T15:54:04.120 INFO:tasks.workunit.client.0.smithi049.stdout:4/362: dwrite f17 [0,4194304] 0 2024-05-15T15:54:04.121 INFO:tasks.workunit.client.1.smithi144.stdout:8/195: creat d2/de/d3f/f40 x:0 0 0 2024-05-15T15:54:04.121 INFO:tasks.workunit.client.0.smithi049.stdout:4/363: readlink d18/l4c 0 2024-05-15T15:54:04.121 INFO:tasks.workunit.client.1.smithi144.stdout:9/176: write d0/f4 [1265546,29713] 0 2024-05-15T15:54:04.123 INFO:tasks.workunit.client.1.smithi144.stdout:9/177: write d0/d8/d21/d29/d2d/d44/d10/d19/d3e/f3f [118903,87401] 0 2024-05-15T15:54:04.124 INFO:tasks.workunit.client.1.smithi144.stdout:3/188: truncate d0/d1/fe 2552848 0 2024-05-15T15:54:04.127 INFO:tasks.workunit.client.0.smithi049.stdout:8/278: dwrite d2/f2c [0,4194304] 0 2024-05-15T15:54:04.128 INFO:tasks.workunit.client.1.smithi144.stdout:4/206: truncate d6/dd/d34/d3b/d30/f1b 118153 0 2024-05-15T15:54:04.130 INFO:tasks.workunit.client.1.smithi144.stdout:1/182: read f6 [750637,56661] 0 2024-05-15T15:54:04.132 INFO:tasks.workunit.client.1.smithi144.stdout:1/183: chown f1 1672 1 2024-05-15T15:54:04.134 INFO:tasks.workunit.client.0.smithi049.stdout:8/279: rename d2/de/f44 to d2/de/f55 0 2024-05-15T15:54:04.135 INFO:tasks.workunit.client.1.smithi144.stdout:3/189: dwrite d0/f2e [4194304,4194304] 0 2024-05-15T15:54:04.135 INFO:tasks.workunit.client.0.smithi049.stdout:4/364: creat d18/d1e/f78 x:0 0 0 2024-05-15T15:54:04.136 INFO:tasks.workunit.client.1.smithi144.stdout:9/178: dwrite d0/d8/d21/d29/d2d/d44/d3c/f42 [0,4194304] 0 2024-05-15T15:54:04.137 INFO:tasks.workunit.client.0.smithi049.stdout:4/365: dread fc [0,4194304] 0 2024-05-15T15:54:04.139 INFO:tasks.workunit.client.0.smithi049.stdout:4/366: write d18/d1e/f1f [3748758,113833] 0 2024-05-15T15:54:04.140 INFO:tasks.workunit.client.0.smithi049.stdout:8/280: dwrite d2/de/d27/f35 [0,4194304] 0 2024-05-15T15:54:04.143 INFO:tasks.workunit.client.1.smithi144.stdout:1/184: dwrite d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:04.145 INFO:tasks.workunit.client.1.smithi144.stdout:3/190: dwrite d0/d11/d24/f3e [0,4194304] 0 2024-05-15T15:54:04.146 INFO:tasks.workunit.client.1.smithi144.stdout:3/191: stat d0/d1/f23 0 2024-05-15T15:54:04.164 INFO:tasks.workunit.client.0.smithi049.stdout:8/281: chown d2/c13 73611 1 2024-05-15T15:54:04.169 INFO:tasks.workunit.client.1.smithi144.stdout:0/173: creat d4/f3f x:0 0 0 2024-05-15T15:54:04.171 INFO:tasks.workunit.client.1.smithi144.stdout:5/187: fdatasync d0/dc/f1b 0 2024-05-15T15:54:04.172 INFO:tasks.workunit.client.0.smithi049.stdout:4/367: dwrite d18/f49 [0,4194304] 0 2024-05-15T15:54:04.177 INFO:tasks.workunit.client.1.smithi144.stdout:6/153: symlink d7/d1e/l3b 0 2024-05-15T15:54:04.177 INFO:tasks.workunit.client.1.smithi144.stdout:6/154: chown d7/db/l1a 2 1 2024-05-15T15:54:04.180 INFO:tasks.workunit.client.0.smithi049.stdout:4/368: dread d18/d1e/d2a/d36/f4a [4194304,4194304] 0 2024-05-15T15:54:04.181 INFO:tasks.workunit.client.0.smithi049.stdout:4/369: write d18/d1e/d2a/f64 [426293,3311] 0 2024-05-15T15:54:04.188 INFO:tasks.workunit.client.1.smithi144.stdout:8/196: mknod d2/d24/d2d/d30/c41 0 2024-05-15T15:54:04.194 INFO:tasks.workunit.client.0.smithi049.stdout:4/370: link d18/d21/f56 d18/d1e/d2a/d50/f79 0 2024-05-15T15:54:04.195 INFO:tasks.workunit.client.0.smithi049.stdout:4/371: fdatasync d18/f49 0 2024-05-15T15:54:04.195 INFO:tasks.workunit.client.0.smithi049.stdout:8/282: dread d2/de/d25/f32 [0,4194304] 0 2024-05-15T15:54:04.196 INFO:tasks.workunit.client.1.smithi144.stdout:4/207: creat d6/dd/d34/d3b/d30/db/d1d/f41 x:0 0 0 2024-05-15T15:54:04.254 INFO:tasks.workunit.client.1.smithi144.stdout:1/185: symlink d7/d1d/d33/d29/l37 0 2024-05-15T15:54:04.255 INFO:tasks.workunit.client.1.smithi144.stdout:3/192: rename d0/d1/d17/c2f to d0/c44 0 2024-05-15T15:54:04.260 INFO:tasks.workunit.client.1.smithi144.stdout:5/188: creat d0/dc/d24/d28/d2d/f3d x:0 0 0 2024-05-15T15:54:04.260 INFO:tasks.workunit.client.1.smithi144.stdout:3/193: dwrite d0/f2c [0,4194304] 0 2024-05-15T15:54:04.261 INFO:tasks.workunit.client.1.smithi144.stdout:6/155: mkdir d7/db/d11/d3c 0 2024-05-15T15:54:04.261 INFO:tasks.workunit.client.1.smithi144.stdout:5/189: stat d0/dc/d24/d28/c2e 0 2024-05-15T15:54:04.262 INFO:tasks.workunit.client.0.smithi049.stdout:8/283: link d2/d14/c3b d2/d18/c56 0 2024-05-15T15:54:04.262 INFO:tasks.workunit.client.0.smithi049.stdout:4/372: rmdir d18/d1e 39 2024-05-15T15:54:04.265 INFO:tasks.workunit.client.0.smithi049.stdout:8/284: rmdir d2/d4f 39 2024-05-15T15:54:04.266 INFO:tasks.workunit.client.1.smithi144.stdout:6/156: dread d7/db/f29 [0,4194304] 0 2024-05-15T15:54:04.267 INFO:tasks.workunit.client.1.smithi144.stdout:6/157: fsync d7/db/d11/f15 0 2024-05-15T15:54:04.267 INFO:tasks.workunit.client.0.smithi049.stdout:4/373: readlink d18/d1e/l6e 0 2024-05-15T15:54:04.269 INFO:tasks.workunit.client.1.smithi144.stdout:7/203: mkdir d4/d41 0 2024-05-15T15:54:04.269 INFO:tasks.workunit.client.1.smithi144.stdout:0/174: write d4/f1c [1373855,87125] 0 2024-05-15T15:54:04.269 INFO:tasks.workunit.client.1.smithi144.stdout:7/204: readlink d4/l26 0 2024-05-15T15:54:04.270 INFO:tasks.workunit.client.0.smithi049.stdout:8/285: mknod d2/d18/c57 0 2024-05-15T15:54:04.270 INFO:tasks.workunit.client.1.smithi144.stdout:7/205: chown d4/df/d25/f35 9618692 1 2024-05-15T15:54:04.271 INFO:tasks.workunit.client.1.smithi144.stdout:8/197: creat d2/d24/d2d/d30/f42 x:0 0 0 2024-05-15T15:54:04.273 INFO:tasks.workunit.client.1.smithi144.stdout:4/208: mkdir d6/d42 0 2024-05-15T15:54:04.274 INFO:tasks.workunit.client.1.smithi144.stdout:2/196: truncate d3/d10/d15/f31 997941 0 2024-05-15T15:54:04.275 INFO:tasks.workunit.client.1.smithi144.stdout:9/179: mkdir d0/d8/d21/d29/d2d/d44/d4a 0 2024-05-15T15:54:04.276 INFO:tasks.workunit.client.1.smithi144.stdout:9/180: read - d0/d8/d21/d29/d2d/d44/d10/f24 zero size 2024-05-15T15:54:04.276 INFO:tasks.workunit.client.0.smithi049.stdout:8/286: mknod d2/d14/c58 0 2024-05-15T15:54:04.277 INFO:tasks.workunit.client.1.smithi144.stdout:1/186: rmdir d7 39 2024-05-15T15:54:04.277 INFO:tasks.workunit.client.0.smithi049.stdout:8/287: dread - d2/f3d zero size 2024-05-15T15:54:04.278 INFO:tasks.workunit.client.0.smithi049.stdout:4/374: rename d18/d1e/d2a/f64 to d18/d21/f7a 0 2024-05-15T15:54:04.282 INFO:tasks.workunit.client.1.smithi144.stdout:9/181: dwrite d0/d8/d21/d29/d2d/d44/d10/f24 [0,4194304] 0 2024-05-15T15:54:04.283 INFO:tasks.workunit.client.1.smithi144.stdout:3/194: creat d0/d11/d3b/f45 x:0 0 0 2024-05-15T15:54:04.283 INFO:tasks.workunit.client.1.smithi144.stdout:3/195: fsync d0/d1/f23 0 2024-05-15T15:54:04.287 INFO:tasks.workunit.client.0.smithi049.stdout:4/375: symlink d18/d1e/l7b 0 2024-05-15T15:54:04.290 INFO:tasks.workunit.client.1.smithi144.stdout:9/182: dread d0/d8/d21/d29/d2d/d44/d10/f27 [0,4194304] 0 2024-05-15T15:54:04.290 INFO:tasks.workunit.client.0.smithi049.stdout:8/288: rmdir d2/d14/d54 0 2024-05-15T15:54:04.293 INFO:tasks.workunit.client.0.smithi049.stdout:4/376: dread d18/d1e/d2a/d36/f59 [0,4194304] 0 2024-05-15T15:54:04.295 INFO:tasks.workunit.client.1.smithi144.stdout:6/158: rmdir d7/d1e/d28 39 2024-05-15T15:54:04.295 INFO:tasks.workunit.client.1.smithi144.stdout:6/159: write d7/db/f18 [832146,25655] 0 2024-05-15T15:54:04.296 INFO:tasks.workunit.client.0.smithi049.stdout:8/289: creat d2/de/d27/f59 x:0 0 0 2024-05-15T15:54:04.300 INFO:tasks.workunit.client.1.smithi144.stdout:6/160: dwrite d7/ff [0,4194304] 0 2024-05-15T15:54:04.301 INFO:tasks.workunit.client.1.smithi144.stdout:6/161: readlink d7/db/d11/l2f 0 2024-05-15T15:54:04.303 INFO:tasks.workunit.client.0.smithi049.stdout:8/290: dwrite d2/f3 [4194304,4194304] 0 2024-05-15T15:54:04.304 INFO:tasks.workunit.client.1.smithi144.stdout:4/209: creat d6/dd/d2b/f43 x:0 0 0 2024-05-15T15:54:04.305 INFO:tasks.workunit.client.0.smithi049.stdout:8/291: write d2/d14/f43 [616352,36573] 0 2024-05-15T15:54:04.306 INFO:tasks.workunit.client.1.smithi144.stdout:2/197: creat d3/d7/f3d x:0 0 0 2024-05-15T15:54:04.308 INFO:tasks.workunit.client.1.smithi144.stdout:6/162: dwrite d7/db/f37 [0,4194304] 0 2024-05-15T15:54:04.308 INFO:tasks.workunit.client.1.smithi144.stdout:4/210: chown d6/dd/d34/d3b/d30/db/d1d/d40 2515788 1 2024-05-15T15:54:04.308 INFO:tasks.workunit.client.0.smithi049.stdout:5/270: sync 2024-05-15T15:54:04.308 INFO:tasks.workunit.client.0.smithi049.stdout:9/212: sync 2024-05-15T15:54:04.309 INFO:tasks.workunit.client.1.smithi144.stdout:2/198: write d3/d10/d15/f26 [1862157,19822] 0 2024-05-15T15:54:04.310 INFO:tasks.workunit.client.1.smithi144.stdout:6/163: read - d7/d1e/f2e zero size 2024-05-15T15:54:04.311 INFO:tasks.workunit.client.1.smithi144.stdout:1/187: dread - d7/d1d/d33/d29/f2c zero size 2024-05-15T15:54:04.312 INFO:tasks.workunit.client.1.smithi144.stdout:6/164: write d7/ff [4673654,54352] 0 2024-05-15T15:54:04.313 INFO:tasks.workunit.client.1.smithi144.stdout:1/188: truncate d7/d1d/f32 952770 0 2024-05-15T15:54:04.313 INFO:tasks.workunit.client.1.smithi144.stdout:6/165: chown d7/db/f37 55053 1 2024-05-15T15:54:04.313 INFO:tasks.workunit.client.0.smithi049.stdout:4/377: write fc [1100640,7229] 0 2024-05-15T15:54:04.315 INFO:tasks.workunit.client.0.smithi049.stdout:4/378: dread d18/d21/f56 [0,4194304] 0 2024-05-15T15:54:04.319 INFO:tasks.workunit.client.0.smithi049.stdout:9/213: unlink d0/d8/d11/f17 0 2024-05-15T15:54:04.319 INFO:tasks.workunit.client.1.smithi144.stdout:6/166: read d7/db/f2c [652289,68132] 0 2024-05-15T15:54:04.319 INFO:tasks.workunit.client.1.smithi144.stdout:3/196: creat d0/f46 x:0 0 0 2024-05-15T15:54:04.322 INFO:tasks.workunit.client.1.smithi144.stdout:9/183: mknod d0/d8/d21/d29/c4b 0 2024-05-15T15:54:04.327 INFO:tasks.workunit.client.0.smithi049.stdout:9/214: write d0/d8/f39 [671823,26761] 0 2024-05-15T15:54:04.327 INFO:tasks.workunit.client.0.smithi049.stdout:5/271: write dc/f16 [2060068,40897] 0 2024-05-15T15:54:04.327 INFO:tasks.workunit.client.1.smithi144.stdout:0/175: mkdir d4/d6/d3a/d40 0 2024-05-15T15:54:04.328 INFO:tasks.workunit.client.1.smithi144.stdout:0/176: stat d4/df/d1f/c3d 0 2024-05-15T15:54:04.331 INFO:tasks.workunit.client.1.smithi144.stdout:7/206: rename d4/df/d25/l31 to d4/df/d25/l42 0 2024-05-15T15:54:04.333 INFO:tasks.workunit.client.1.smithi144.stdout:8/198: mknod d2/d1a/d38/c43 0 2024-05-15T15:54:04.333 INFO:tasks.workunit.client.0.smithi049.stdout:5/272: dwrite dc/dd/f1d [0,4194304] 0 2024-05-15T15:54:04.336 INFO:tasks.workunit.client.1.smithi144.stdout:5/190: dwrite d0/dc/d1f/d29/f3c [0,4194304] 0 2024-05-15T15:54:04.337 INFO:tasks.workunit.client.0.smithi049.stdout:8/292: getdents d2/de 0 2024-05-15T15:54:04.338 INFO:tasks.workunit.client.0.smithi049.stdout:8/293: fsync d2/de/d27/f35 0 2024-05-15T15:54:04.340 INFO:tasks.workunit.client.0.smithi049.stdout:7/247: sync 2024-05-15T15:54:04.341 INFO:tasks.workunit.client.0.smithi049.stdout:3/328: sync 2024-05-15T15:54:04.341 INFO:tasks.workunit.client.0.smithi049.stdout:1/309: sync 2024-05-15T15:54:04.341 INFO:tasks.workunit.client.1.smithi144.stdout:9/184: dread d0/d8/d21/d29/d2d/d44/d10/fd [0,4194304] 0 2024-05-15T15:54:04.342 INFO:tasks.workunit.client.0.smithi049.stdout:4/379: rename d18/d1e/d2a/d36/f59 to d18/f7c 0 2024-05-15T15:54:04.342 INFO:tasks.workunit.client.0.smithi049.stdout:3/329: dread - d12/d14/f54 zero size 2024-05-15T15:54:04.343 INFO:tasks.workunit.client.0.smithi049.stdout:7/248: chown d0/d6/d23/c37 6077018 1 2024-05-15T15:54:04.345 INFO:tasks.workunit.client.0.smithi049.stdout:5/273: read dc/dd/d10/f2a [100501,4894] 0 2024-05-15T15:54:04.346 INFO:tasks.workunit.client.1.smithi144.stdout:2/199: creat d3/d7/d20/f3e x:0 0 0 2024-05-15T15:54:04.346 INFO:tasks.workunit.client.0.smithi049.stdout:5/274: readlink dc/lf 0 2024-05-15T15:54:04.347 INFO:tasks.workunit.client.1.smithi144.stdout:5/191: dread d0/f7 [0,4194304] 0 2024-05-15T15:54:04.347 INFO:tasks.workunit.client.1.smithi144.stdout:2/200: truncate d3/f1c 1595635 0 2024-05-15T15:54:04.349 INFO:tasks.workunit.client.0.smithi049.stdout:5/275: write dc/dd/d47/f4d [413202,123674] 0 2024-05-15T15:54:04.352 INFO:tasks.workunit.client.0.smithi049.stdout:7/249: dwrite d0/f41 [0,4194304] 0 2024-05-15T15:54:04.356 INFO:tasks.workunit.client.1.smithi144.stdout:3/197: symlink d0/d11/l47 0 2024-05-15T15:54:04.362 INFO:tasks.workunit.client.0.smithi049.stdout:9/215: link d0/d8/d9/fb d0/d8/d1b/d40/f49 0 2024-05-15T15:54:04.363 INFO:tasks.workunit.client.0.smithi049.stdout:1/310: dwrite d0/d4/f8 [0,4194304] 0 2024-05-15T15:54:04.364 INFO:tasks.workunit.client.0.smithi049.stdout:3/330: dwrite f10 [0,4194304] 0 2024-05-15T15:54:04.366 INFO:tasks.workunit.client.0.smithi049.stdout:7/250: mkdir d0/d6/d23/d45 0 2024-05-15T15:54:04.367 INFO:tasks.workunit.client.0.smithi049.stdout:7/251: chown d0/d2f 957 1 2024-05-15T15:54:04.367 INFO:tasks.workunit.client.0.smithi049.stdout:8/294: dwrite d2/de/d27/f39 [0,4194304] 0 2024-05-15T15:54:04.368 INFO:tasks.workunit.client.0.smithi049.stdout:8/295: fsync d2/de/d27/f35 0 2024-05-15T15:54:04.369 INFO:tasks.workunit.client.1.smithi144.stdout:8/199: rmdir d2/d24 39 2024-05-15T15:54:04.371 INFO:tasks.workunit.client.0.smithi049.stdout:4/380: symlink d18/d1e/d63/l7d 0 2024-05-15T15:54:04.374 INFO:tasks.workunit.client.1.smithi144.stdout:9/185: mknod d0/d8/d21/d29/d2d/c4c 0 2024-05-15T15:54:04.374 INFO:tasks.workunit.client.0.smithi049.stdout:5/276: symlink dc/d4a/l4e 0 2024-05-15T15:54:04.375 INFO:tasks.workunit.client.1.smithi144.stdout:9/186: stat d0/f4 0 2024-05-15T15:54:04.378 INFO:tasks.workunit.client.0.smithi049.stdout:1/311: chown d0/d4/c7 1198249 1 2024-05-15T15:54:04.379 INFO:tasks.workunit.client.0.smithi049.stdout:4/381: dread f10 [0,4194304] 0 2024-05-15T15:54:04.379 INFO:tasks.workunit.client.1.smithi144.stdout:2/201: mkdir d3/d7/d20/d3f 0 2024-05-15T15:54:04.380 INFO:tasks.workunit.client.1.smithi144.stdout:8/200: dread d2/de/f10 [4194304,4194304] 0 2024-05-15T15:54:04.381 INFO:tasks.workunit.client.0.smithi049.stdout:7/252: creat d0/d34/f46 x:0 0 0 2024-05-15T15:54:04.381 INFO:tasks.workunit.client.1.smithi144.stdout:8/201: write d2/de/f33 [859167,120633] 0 2024-05-15T15:54:04.382 INFO:tasks.workunit.client.1.smithi144.stdout:1/189: creat d7/d1d/d33/d1e/d34/f38 x:0 0 0 2024-05-15T15:54:04.382 INFO:tasks.workunit.client.1.smithi144.stdout:8/202: write d2/fd [3426001,987] 0 2024-05-15T15:54:04.383 INFO:tasks.workunit.client.1.smithi144.stdout:1/190: truncate d7/d1d/d33/d29/f2c 562022 0 2024-05-15T15:54:04.384 INFO:tasks.workunit.client.1.smithi144.stdout:6/167: creat d7/d1e/d28/f3d x:0 0 0 2024-05-15T15:54:04.386 INFO:tasks.workunit.client.1.smithi144.stdout:3/198: creat d0/d1/d17/f48 x:0 0 0 2024-05-15T15:54:04.386 INFO:tasks.workunit.client.1.smithi144.stdout:4/211: write d6/dd/d34/d3b/d30/f1b [702805,82610] 0 2024-05-15T15:54:04.389 INFO:tasks.workunit.client.1.smithi144.stdout:7/207: mknod d4/d41/c43 0 2024-05-15T15:54:04.389 INFO:tasks.workunit.client.0.smithi049.stdout:3/331: symlink d12/d30/d63/l65 0 2024-05-15T15:54:04.392 INFO:tasks.workunit.client.0.smithi049.stdout:1/312: creat d0/d4/dd/d5d/d65/d72/f7a x:0 0 0 2024-05-15T15:54:04.393 INFO:tasks.workunit.client.0.smithi049.stdout:4/382: creat d18/d1e/d2a/d36/f7e x:0 0 0 2024-05-15T15:54:04.394 INFO:tasks.workunit.client.1.smithi144.stdout:7/208: dwrite d4/df/f22 [0,4194304] 0 2024-05-15T15:54:04.395 INFO:tasks.workunit.client.0.smithi049.stdout:7/253: fdatasync d0/d6/da/d3d/f2c 0 2024-05-15T15:54:04.395 INFO:tasks.workunit.client.0.smithi049.stdout:3/332: dwrite f10 [0,4194304] 0 2024-05-15T15:54:04.396 INFO:tasks.workunit.client.1.smithi144.stdout:5/192: mkdir d0/dc/d2f/d34/d3e 0 2024-05-15T15:54:04.398 INFO:tasks.workunit.client.1.smithi144.stdout:2/202: mkdir d3/d7/d20/d40 0 2024-05-15T15:54:04.398 INFO:tasks.workunit.client.1.smithi144.stdout:5/193: dread d0/dc/d1f/f38 [0,4194304] 0 2024-05-15T15:54:04.405 INFO:tasks.workunit.client.1.smithi144.stdout:6/168: creat d7/db/f3e x:0 0 0 2024-05-15T15:54:04.406 INFO:tasks.workunit.client.1.smithi144.stdout:6/169: stat d7/d32 0 2024-05-15T15:54:04.407 INFO:tasks.workunit.client.0.smithi049.stdout:1/313: unlink d0/d4/c1f 0 2024-05-15T15:54:04.408 INFO:tasks.workunit.client.0.smithi049.stdout:1/314: chown d0/d4/d63/f70 15 1 2024-05-15T15:54:04.408 INFO:tasks.workunit.client.0.smithi049.stdout:7/254: dread d0/f9 [4194304,4194304] 0 2024-05-15T15:54:04.408 INFO:tasks.workunit.client.1.smithi144.stdout:3/199: creat d0/d11/d24/f49 x:0 0 0 2024-05-15T15:54:04.409 INFO:tasks.workunit.client.0.smithi049.stdout:7/255: read - d0/d6/d23/f40 zero size 2024-05-15T15:54:04.409 INFO:tasks.workunit.client.0.smithi049.stdout:1/315: chown d0/d4/dd/f28 416 1 2024-05-15T15:54:04.409 INFO:tasks.workunit.client.1.smithi144.stdout:3/200: dread - d0/d1/d17/d39/f3c zero size 2024-05-15T15:54:04.412 INFO:tasks.workunit.client.1.smithi144.stdout:0/177: link d4/d6/ce d4/d6/d3a/c41 0 2024-05-15T15:54:04.413 INFO:tasks.workunit.client.0.smithi049.stdout:6/236: sync 2024-05-15T15:54:04.414 INFO:tasks.workunit.client.0.smithi049.stdout:0/253: sync 2024-05-15T15:54:04.414 INFO:tasks.workunit.client.0.smithi049.stdout:2/278: sync 2024-05-15T15:54:04.415 INFO:tasks.workunit.client.0.smithi049.stdout:6/237: chown dd/d17/c1c 0 1 2024-05-15T15:54:04.415 INFO:tasks.workunit.client.1.smithi144.stdout:0/178: read d4/dc/f18 [1014580,63996] 0 2024-05-15T15:54:04.416 INFO:tasks.workunit.client.0.smithi049.stdout:6/238: write dd/d17/d24/f27 [1391428,10585] 0 2024-05-15T15:54:04.417 INFO:tasks.workunit.client.0.smithi049.stdout:6/239: stat dd/ce 0 2024-05-15T15:54:04.419 INFO:tasks.workunit.client.0.smithi049.stdout:6/240: write dd/f14 [3128490,63187] 0 2024-05-15T15:54:04.422 INFO:tasks.workunit.client.1.smithi144.stdout:7/209: mknod d4/d41/c44 0 2024-05-15T15:54:04.422 INFO:tasks.workunit.client.0.smithi049.stdout:9/216: write d0/d25/f31 [1059113,97722] 0 2024-05-15T15:54:04.423 INFO:tasks.workunit.client.0.smithi049.stdout:5/277: link dc/dd/f13 dc/dd/d39/f4f 0 2024-05-15T15:54:04.424 INFO:tasks.workunit.client.0.smithi049.stdout:5/278: chown dc/l24 1925164 1 2024-05-15T15:54:04.425 INFO:tasks.workunit.client.1.smithi144.stdout:1/191: write d7/df/f10 [4993186,18734] 0 2024-05-15T15:54:04.427 INFO:tasks.workunit.client.0.smithi049.stdout:3/333: dwrite d12/d26/f40 [0,4194304] 0 2024-05-15T15:54:04.428 INFO:tasks.workunit.client.1.smithi144.stdout:7/210: dwrite d4/df/f32 [0,4194304] 0 2024-05-15T15:54:04.429 INFO:tasks.workunit.client.0.smithi049.stdout:2/279: dread d3/db/f12 [0,4194304] 0 2024-05-15T15:54:04.432 INFO:tasks.workunit.client.1.smithi144.stdout:5/194: rename d0/dc/d1f/l3a to d0/dc/d12/l3f 0 2024-05-15T15:54:04.434 INFO:tasks.workunit.client.0.smithi049.stdout:4/383: write d18/d1e/d2a/d50/d54/f5c [552584,79697] 0 2024-05-15T15:54:04.436 INFO:tasks.workunit.client.1.smithi144.stdout:8/203: creat d2/d24/d2d/d30/f44 x:0 0 0 2024-05-15T15:54:04.437 INFO:tasks.workunit.client.0.smithi049.stdout:1/316: rmdir d0/d4/dd/de/d48 39 2024-05-15T15:54:04.438 INFO:tasks.workunit.client.0.smithi049.stdout:1/317: chown d0/d4/dd/de/d36/f53 2 1 2024-05-15T15:54:04.440 INFO:tasks.workunit.client.0.smithi049.stdout:1/318: fdatasync d0/d4/dd/de/d1e/d16/f30 0 2024-05-15T15:54:04.442 INFO:tasks.workunit.client.1.smithi144.stdout:4/212: getdents d6/dd/d34/d3b/d30/db/d1d/d40 0 2024-05-15T15:54:04.444 INFO:tasks.workunit.client.0.smithi049.stdout:0/254: symlink dc/d13/d14/d33/l54 0 2024-05-15T15:54:04.454 INFO:tasks.workunit.client.0.smithi049.stdout:3/334: mkdir d12/d30/d66 0 2024-05-15T15:54:04.455 INFO:tasks.workunit.client.1.smithi144.stdout:1/192: rename d7/d1d/d33/l1a to d7/d31/l39 0 2024-05-15T15:54:04.456 INFO:tasks.workunit.client.0.smithi049.stdout:2/280: symlink d3/db/d1d/l53 0 2024-05-15T15:54:04.457 INFO:tasks.workunit.client.0.smithi049.stdout:2/281: read d3/d8/fe [2379063,70816] 0 2024-05-15T15:54:04.457 INFO:tasks.workunit.client.1.smithi144.stdout:5/195: symlink d0/dc/d2f/d34/l40 0 2024-05-15T15:54:04.458 INFO:tasks.workunit.client.1.smithi144.stdout:2/203: write d3/d10/f11 [293665,101528] 0 2024-05-15T15:54:04.458 INFO:tasks.workunit.client.0.smithi049.stdout:4/384: creat d18/d1e/d63/f7f x:0 0 0 2024-05-15T15:54:04.459 INFO:tasks.workunit.client.1.smithi144.stdout:5/196: read d0/dc/d1f/d29/f3c [2855240,7180] 0 2024-05-15T15:54:04.459 INFO:tasks.workunit.client.1.smithi144.stdout:8/204: creat d2/de/d3f/f45 x:0 0 0 2024-05-15T15:54:04.460 INFO:tasks.workunit.client.0.smithi049.stdout:7/256: link d0/d6/f18 d0/f47 0 2024-05-15T15:54:04.461 INFO:tasks.workunit.client.0.smithi049.stdout:8/296: getdents d2/d4f 0 2024-05-15T15:54:04.462 INFO:tasks.workunit.client.0.smithi049.stdout:1/319: creat d0/d4/dd/d5d/d65/d72/f7b x:0 0 0 2024-05-15T15:54:04.464 INFO:tasks.workunit.client.1.smithi144.stdout:0/179: creat d4/df/d14/f42 x:0 0 0 2024-05-15T15:54:04.464 INFO:tasks.workunit.client.1.smithi144.stdout:9/187: getdents d0/d8/d21/d29/d2d/d44/d10/d19/d3e 0 2024-05-15T15:54:04.464 INFO:tasks.workunit.client.1.smithi144.stdout:2/204: dread d3/f1c [0,4194304] 0 2024-05-15T15:54:04.466 INFO:tasks.workunit.client.1.smithi144.stdout:7/211: mkdir d4/d45 0 2024-05-15T15:54:04.467 INFO:tasks.workunit.client.0.smithi049.stdout:9/217: creat d0/d8/d9/d45/f4a x:0 0 0 2024-05-15T15:54:04.468 INFO:tasks.workunit.client.1.smithi144.stdout:0/180: dread d4/df/f16 [0,4194304] 0 2024-05-15T15:54:04.472 INFO:tasks.workunit.client.1.smithi144.stdout:1/193: creat d7/d9/d22/f3a x:0 0 0 2024-05-15T15:54:04.473 INFO:tasks.workunit.client.0.smithi049.stdout:2/282: creat d3/db/d1d/d1e/d2b/f54 x:0 0 0 2024-05-15T15:54:04.474 INFO:tasks.workunit.client.0.smithi049.stdout:9/218: dwrite d0/d8/d11/f48 [0,4194304] 0 2024-05-15T15:54:04.476 INFO:tasks.workunit.client.1.smithi144.stdout:1/194: dwrite d7/d9/fe [0,4194304] 0 2024-05-15T15:54:04.481 INFO:tasks.workunit.client.1.smithi144.stdout:5/197: mknod d0/dc/d12/d31/c41 0 2024-05-15T15:54:04.483 INFO:tasks.workunit.client.1.smithi144.stdout:8/205: creat d2/de/d36/f46 x:0 0 0 2024-05-15T15:54:04.484 INFO:tasks.workunit.client.0.smithi049.stdout:6/241: truncate dd/f33 472408 0 2024-05-15T15:54:04.492 INFO:tasks.workunit.client.0.smithi049.stdout:3/335: mkdir d12/d67 0 2024-05-15T15:54:04.496 INFO:tasks.workunit.client.1.smithi144.stdout:0/181: rmdir d4/df/d1f 39 2024-05-15T15:54:04.497 INFO:tasks.workunit.client.1.smithi144.stdout:5/198: dread d0/f22 [0,4194304] 0 2024-05-15T15:54:04.498 INFO:tasks.workunit.client.0.smithi049.stdout:9/219: mknod d0/d8/d9/d22/c4b 0 2024-05-15T15:54:04.502 INFO:tasks.workunit.client.0.smithi049.stdout:3/336: dwrite d12/d30/f38 [0,4194304] 0 2024-05-15T15:54:04.502 INFO:tasks.workunit.client.0.smithi049.stdout:3/337: fdatasync d12/d30/d42/f48 0 2024-05-15T15:54:04.505 INFO:tasks.workunit.client.0.smithi049.stdout:0/255: truncate f3 1007255 0 2024-05-15T15:54:04.505 INFO:tasks.workunit.client.1.smithi144.stdout:1/195: symlink d7/df/l3b 0 2024-05-15T15:54:04.506 INFO:tasks.workunit.client.0.smithi049.stdout:0/256: write dc/d13/d14/d2a/f3a [1967391,114797] 0 2024-05-15T15:54:04.507 INFO:tasks.workunit.client.0.smithi049.stdout:8/297: getdents d2/d4f 0 2024-05-15T15:54:04.508 INFO:tasks.workunit.client.0.smithi049.stdout:0/257: chown dc/d13/d2b/c35 3671 1 2024-05-15T15:54:04.509 INFO:tasks.workunit.client.1.smithi144.stdout:3/201: getdents d0/d1/d17 0 2024-05-15T15:54:04.510 INFO:tasks.workunit.client.0.smithi049.stdout:0/258: chown dc/d13/f22 327921 1 2024-05-15T15:54:04.510 INFO:tasks.workunit.client.0.smithi049.stdout:7/257: dread d0/d6/f29 [0,4194304] 0 2024-05-15T15:54:04.511 INFO:tasks.workunit.client.1.smithi144.stdout:3/202: write d0/d1/f23 [114780,28877] 0 2024-05-15T15:54:04.511 INFO:tasks.workunit.client.0.smithi049.stdout:1/320: mknod d0/d4/dd/de/d48/c7c 0 2024-05-15T15:54:04.511 INFO:tasks.workunit.client.1.smithi144.stdout:3/203: chown d0/d11/d24/c29 8195 1 2024-05-15T15:54:04.511 INFO:tasks.workunit.client.0.smithi049.stdout:1/321: readlink d0/d4/l14 0 2024-05-15T15:54:04.514 INFO:tasks.workunit.client.0.smithi049.stdout:6/242: rename dd/d17/f20 to dd/d26/f3f 0 2024-05-15T15:54:04.521 INFO:tasks.workunit.client.1.smithi144.stdout:6/170: rename d7/db/d11/f13 to d7/db/f3f 0 2024-05-15T15:54:04.524 INFO:tasks.workunit.client.1.smithi144.stdout:0/182: fsync d4/df/f16 0 2024-05-15T15:54:04.526 INFO:tasks.workunit.client.0.smithi049.stdout:3/338: mkdir d12/d30/d63/d68 0 2024-05-15T15:54:04.527 INFO:tasks.workunit.client.0.smithi049.stdout:3/339: write d12/d30/d42/f46 [235233,13968] 0 2024-05-15T15:54:04.528 INFO:tasks.workunit.client.1.smithi144.stdout:5/199: mknod d0/dc/d24/d28/d2d/c42 0 2024-05-15T15:54:04.532 INFO:tasks.workunit.client.1.smithi144.stdout:3/204: mkdir d0/d1/d17/d39/d40/d4a 0 2024-05-15T15:54:04.533 INFO:tasks.workunit.client.1.smithi144.stdout:5/200: dwrite d0/dc/d1f/f3b [0,4194304] 0 2024-05-15T15:54:04.533 INFO:tasks.workunit.client.0.smithi049.stdout:2/283: truncate d3/db/f1a 121482 0 2024-05-15T15:54:04.533 INFO:tasks.workunit.client.1.smithi144.stdout:3/205: fdatasync d0/d1/f21 0 2024-05-15T15:54:04.534 INFO:tasks.workunit.client.1.smithi144.stdout:5/201: readlink d0/la 0 2024-05-15T15:54:04.534 INFO:tasks.workunit.client.0.smithi049.stdout:2/284: write d3/db/f1f [3616509,92707] 0 2024-05-15T15:54:04.534 INFO:tasks.workunit.client.0.smithi049.stdout:2/285: readlink d3/d8/lf 0 2024-05-15T15:54:04.536 INFO:tasks.workunit.client.0.smithi049.stdout:2/286: write d3/db/d1d/d1e/f21 [4463337,35631] 0 2024-05-15T15:54:04.537 INFO:tasks.workunit.client.0.smithi049.stdout:2/287: stat d3/d8/ld 0 2024-05-15T15:54:04.543 INFO:tasks.workunit.client.0.smithi049.stdout:4/385: truncate d18/d1e/d2a/f4f 3418602 0 2024-05-15T15:54:04.544 INFO:tasks.workunit.client.0.smithi049.stdout:4/386: chown d18/d21/d2b/d4e 1784472 1 2024-05-15T15:54:04.545 INFO:tasks.workunit.client.0.smithi049.stdout:2/288: dwrite d3/db/f43 [0,4194304] 0 2024-05-15T15:54:04.546 INFO:tasks.workunit.client.0.smithi049.stdout:2/289: chown d3/f4d 949 1 2024-05-15T15:54:04.546 INFO:tasks.workunit.client.0.smithi049.stdout:4/387: write d18/d1e/d2a/d36/f58 [207798,121059] 0 2024-05-15T15:54:04.547 INFO:tasks.workunit.client.0.smithi049.stdout:2/290: readlink d3/da/l3a 0 2024-05-15T15:54:04.550 INFO:tasks.workunit.client.0.smithi049.stdout:1/322: read d0/d4/dd/de/d1e/f1d [967473,69842] 0 2024-05-15T15:54:04.551 INFO:tasks.workunit.client.0.smithi049.stdout:1/323: chown d0/d4/fa 23 1 2024-05-15T15:54:04.567 INFO:tasks.workunit.client.0.smithi049.stdout:0/259: mknod dc/d13/c55 0 2024-05-15T15:54:04.569 INFO:tasks.workunit.client.0.smithi049.stdout:0/260: chown dc/d13/d14/d2f/f52 65304140 1 2024-05-15T15:54:04.570 INFO:tasks.workunit.client.0.smithi049.stdout:0/261: write dc/d13/d14/f46 [48157,109082] 0 2024-05-15T15:54:04.571 INFO:tasks.workunit.client.0.smithi049.stdout:7/258: creat d0/d2f/f48 x:0 0 0 2024-05-15T15:54:04.573 INFO:tasks.workunit.client.0.smithi049.stdout:0/262: write dc/d13/d14/d33/f47 [3221285,103706] 0 2024-05-15T15:54:04.573 INFO:tasks.workunit.client.0.smithi049.stdout:6/243: rename dd/f33 to dd/d26/f40 0 2024-05-15T15:54:04.577 INFO:tasks.workunit.client.0.smithi049.stdout:3/340: creat d12/d67/f69 x:0 0 0 2024-05-15T15:54:04.578 INFO:tasks.workunit.client.0.smithi049.stdout:8/298: creat d2/f5a x:0 0 0 2024-05-15T15:54:04.581 INFO:tasks.workunit.client.0.smithi049.stdout:6/244: dwrite dd/d17/d24/f27 [0,4194304] 0 2024-05-15T15:54:04.583 INFO:tasks.workunit.client.0.smithi049.stdout:7/259: symlink d0/d6/da/d3d/l49 0 2024-05-15T15:54:04.589 INFO:tasks.workunit.client.1.smithi144.stdout:9/188: creat d0/d8/d21/d29/d2d/d44/f4d x:0 0 0 2024-05-15T15:54:04.590 INFO:tasks.workunit.client.1.smithi144.stdout:7/212: creat d4/f46 x:0 0 0 2024-05-15T15:54:04.590 INFO:tasks.workunit.client.0.smithi049.stdout:9/220: truncate d0/f1 3841006 0 2024-05-15T15:54:04.591 INFO:tasks.workunit.client.1.smithi144.stdout:2/205: rename d3/d7/lf to d3/d10/d15/l41 0 2024-05-15T15:54:04.592 INFO:tasks.workunit.client.1.smithi144.stdout:6/171: fdatasync d7/db/f18 0 2024-05-15T15:54:04.592 INFO:tasks.workunit.client.0.smithi049.stdout:1/324: creat d0/d4/dd/d75/f7d x:0 0 0 2024-05-15T15:54:04.592 INFO:tasks.workunit.client.0.smithi049.stdout:9/221: dread d0/d8/d9/d22/f2e [0,4194304] 0 2024-05-15T15:54:04.592 INFO:tasks.workunit.client.1.smithi144.stdout:1/196: symlink d7/l3c 0 2024-05-15T15:54:04.593 INFO:tasks.workunit.client.0.smithi049.stdout:9/222: chown d0/f1d 2852 1 2024-05-15T15:54:04.593 INFO:tasks.workunit.client.1.smithi144.stdout:7/213: write d4/df/f32 [984101,21327] 0 2024-05-15T15:54:04.594 INFO:tasks.workunit.client.0.smithi049.stdout:9/223: readlink d0/d8/d21/l3d 0 2024-05-15T15:54:04.594 INFO:tasks.workunit.client.0.smithi049.stdout:3/341: rmdir d12/d14 39 2024-05-15T15:54:04.596 INFO:tasks.workunit.client.0.smithi049.stdout:9/224: chown d0/d8/d1b/f28 199548900 1 2024-05-15T15:54:04.596 INFO:tasks.workunit.client.1.smithi144.stdout:5/202: dread d0/dc/fe [0,4194304] 0 2024-05-15T15:54:04.597 INFO:tasks.workunit.client.1.smithi144.stdout:3/206: dwrite d0/f3d [0,4194304] 0 2024-05-15T15:54:04.597 INFO:tasks.workunit.client.0.smithi049.stdout:8/299: mkdir d2/de/d27/d5b 0 2024-05-15T15:54:04.598 INFO:tasks.workunit.client.1.smithi144.stdout:3/207: truncate d0/d11/d24/f49 106280 0 2024-05-15T15:54:04.599 INFO:tasks.workunit.client.0.smithi049.stdout:4/388: creat d18/f80 x:0 0 0 2024-05-15T15:54:04.600 INFO:tasks.workunit.client.0.smithi049.stdout:5/279: sync 2024-05-15T15:54:04.601 INFO:tasks.workunit.client.0.smithi049.stdout:2/291: dwrite d3/f10 [0,4194304] 0 2024-05-15T15:54:04.602 INFO:tasks.workunit.client.1.smithi144.stdout:2/206: symlink d3/d7/d20/d22/l42 0 2024-05-15T15:54:04.603 INFO:tasks.workunit.client.0.smithi049.stdout:2/292: readlink d3/da/l3a 0 2024-05-15T15:54:04.604 INFO:tasks.workunit.client.0.smithi049.stdout:6/245: chown dd/d17/d24/l28 8 1 2024-05-15T15:54:04.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:04 smithi049 ceph-mon[28399]: pgmap v22: 65 pgs: 65 active+clean; 847 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.4 MiB/s rd, 95 MiB/s wr, 352 op/s 2024-05-15T15:54:04.606 INFO:tasks.workunit.client.0.smithi049.stdout:2/293: read d3/d8/f36 [3538731,68701] 0 2024-05-15T15:54:04.607 INFO:tasks.workunit.client.0.smithi049.stdout:6/246: chown dd/d17/d24/f27 437143 1 2024-05-15T15:54:04.610 INFO:tasks.workunit.client.0.smithi049.stdout:7/260: mkdir d0/d6/d23/d4a 0 2024-05-15T15:54:04.612 INFO:tasks.workunit.client.0.smithi049.stdout:5/280: dwrite dc/dd/f1d [4194304,4194304] 0 2024-05-15T15:54:04.614 INFO:tasks.workunit.client.0.smithi049.stdout:0/263: mkdir dc/d13/d56 0 2024-05-15T15:54:04.615 INFO:tasks.workunit.client.0.smithi049.stdout:0/264: dread - dc/d13/d48/f50 zero size 2024-05-15T15:54:04.617 INFO:tasks.workunit.client.0.smithi049.stdout:5/281: dread dc/dd/f1d [4194304,4194304] 0 2024-05-15T15:54:04.618 INFO:tasks.workunit.client.0.smithi049.stdout:5/282: write dc/dd/f3b [5157138,46605] 0 2024-05-15T15:54:04.620 INFO:tasks.workunit.client.0.smithi049.stdout:5/283: write dc/d4a/f36 [430505,102600] 0 2024-05-15T15:54:04.620 INFO:tasks.workunit.client.1.smithi144.stdout:5/203: creat d0/dc/d12/f43 x:0 0 0 2024-05-15T15:54:04.621 INFO:tasks.workunit.client.0.smithi049.stdout:5/284: write dc/f3e [712769,47494] 0 2024-05-15T15:54:04.623 INFO:tasks.workunit.client.1.smithi144.stdout:3/208: mknod d0/d11/c4b 0 2024-05-15T15:54:04.627 INFO:tasks.workunit.client.1.smithi144.stdout:5/204: dwrite d0/f7 [0,4194304] 0 2024-05-15T15:54:04.627 INFO:tasks.workunit.client.1.smithi144.stdout:4/213: sync 2024-05-15T15:54:04.627 INFO:tasks.workunit.client.1.smithi144.stdout:8/206: sync 2024-05-15T15:54:04.628 INFO:tasks.workunit.client.1.smithi144.stdout:5/205: dread - d0/dc/d24/d28/d2d/f3d zero size 2024-05-15T15:54:04.628 INFO:tasks.workunit.client.1.smithi144.stdout:4/214: dread - d6/dd/d2d/f3f zero size 2024-05-15T15:54:04.628 INFO:tasks.workunit.client.1.smithi144.stdout:2/207: rmdir d3/d7/d20/d22 39 2024-05-15T15:54:04.628 INFO:tasks.workunit.client.1.smithi144.stdout:8/207: write d2/d1a/f39 [596510,34602] 0 2024-05-15T15:54:04.629 INFO:tasks.workunit.client.1.smithi144.stdout:1/197: rename d7/l17 to d7/df/d24/l3d 0 2024-05-15T15:54:04.630 INFO:tasks.workunit.client.1.smithi144.stdout:5/206: stat d0/dc/d24/d28/d2d 0 2024-05-15T15:54:04.631 INFO:tasks.workunit.client.1.smithi144.stdout:8/208: write d2/de/d36/f27 [2814891,117717] 0 2024-05-15T15:54:04.632 INFO:tasks.workunit.client.1.smithi144.stdout:4/215: read - d6/dd/d34/d3b/d30/db/f33 zero size 2024-05-15T15:54:04.633 INFO:tasks.workunit.client.1.smithi144.stdout:9/189: link d0/d8/d21/f2f d0/d8/d21/d29/d2d/d44/d16/f4e 0 2024-05-15T15:54:04.634 INFO:tasks.workunit.client.1.smithi144.stdout:7/214: mkdir d4/df/d25/d40/d47 0 2024-05-15T15:54:04.636 INFO:tasks.workunit.client.1.smithi144.stdout:7/215: dread - d4/df/d25/d38/f3d zero size 2024-05-15T15:54:04.637 INFO:tasks.workunit.client.1.smithi144.stdout:7/216: chown d4/c5 0 1 2024-05-15T15:54:04.638 INFO:tasks.workunit.client.1.smithi144.stdout:1/198: dread d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:04.641 INFO:tasks.workunit.client.0.smithi049.stdout:9/225: dwrite d0/d8/d1b/d40/f49 [0,4194304] 0 2024-05-15T15:54:04.643 INFO:tasks.workunit.client.1.smithi144.stdout:2/208: dread d3/f1c [0,4194304] 0 2024-05-15T15:54:04.643 INFO:tasks.workunit.client.1.smithi144.stdout:4/216: dwrite d6/dd/d2d/f3f [0,4194304] 0 2024-05-15T15:54:04.644 INFO:tasks.workunit.client.1.smithi144.stdout:7/217: dread d4/fe [0,4194304] 0 2024-05-15T15:54:04.644 INFO:tasks.workunit.client.1.smithi144.stdout:4/217: chown d6/dd/d34/d3b/d30/db/d1d 24 1 2024-05-15T15:54:04.648 INFO:tasks.workunit.client.0.smithi049.stdout:6/247: write dd/d26/f3c [580644,71395] 0 2024-05-15T15:54:04.649 INFO:tasks.workunit.client.1.smithi144.stdout:8/209: creat d2/de/d36/f47 x:0 0 0 2024-05-15T15:54:04.649 INFO:tasks.workunit.client.1.smithi144.stdout:7/218: chown d4/df/d25/f35 2099882 1 2024-05-15T15:54:04.652 INFO:tasks.workunit.client.0.smithi049.stdout:0/265: write dc/d13/d14/f19 [4678336,23603] 0 2024-05-15T15:54:04.652 INFO:tasks.workunit.client.1.smithi144.stdout:5/207: creat d0/dc/d12/d31/d35/f44 x:0 0 0 2024-05-15T15:54:04.653 INFO:tasks.workunit.client.1.smithi144.stdout:4/218: dread d6/dd/d15/f31 [0,4194304] 0 2024-05-15T15:54:04.653 INFO:tasks.workunit.client.1.smithi144.stdout:9/190: symlink d0/d8/d21/d29/d2d/d44/d3c/l4f 0 2024-05-15T15:54:04.655 INFO:tasks.workunit.client.1.smithi144.stdout:2/209: dwrite d3/d10/d15/f2f [0,4194304] 0 2024-05-15T15:54:04.656 INFO:tasks.workunit.client.1.smithi144.stdout:9/191: truncate d0/d8/d21/f38 1024181 0 2024-05-15T15:54:04.657 INFO:tasks.workunit.client.1.smithi144.stdout:2/210: fdatasync d3/d7/f1f 0 2024-05-15T15:54:04.657 INFO:tasks.workunit.client.0.smithi049.stdout:5/285: creat dc/d4a/f50 x:0 0 0 2024-05-15T15:54:04.659 INFO:tasks.workunit.client.1.smithi144.stdout:1/199: symlink d7/d1d/d33/d1e/l3e 0 2024-05-15T15:54:04.660 INFO:tasks.workunit.client.1.smithi144.stdout:1/200: readlink d7/d1d/d33/d1e/l3e 0 2024-05-15T15:54:04.661 INFO:tasks.workunit.client.1.smithi144.stdout:3/209: rename d0/d11/d3b/l3f to d0/d1/d17/d39/l4c 0 2024-05-15T15:54:04.661 INFO:tasks.workunit.client.1.smithi144.stdout:3/210: readlink d0/l27 0 2024-05-15T15:54:04.665 INFO:tasks.workunit.client.0.smithi049.stdout:6/248: dread f0 [0,4194304] 0 2024-05-15T15:54:04.668 INFO:tasks.workunit.client.0.smithi049.stdout:2/294: symlink d3/d52/l55 0 2024-05-15T15:54:04.669 INFO:tasks.workunit.client.1.smithi144.stdout:0/183: sync 2024-05-15T15:54:04.669 INFO:tasks.workunit.client.0.smithi049.stdout:6/249: dread dd/f15 [0,4194304] 0 2024-05-15T15:54:04.669 INFO:tasks.workunit.client.0.smithi049.stdout:2/295: dread - d3/f4d zero size 2024-05-15T15:54:04.670 INFO:tasks.workunit.client.1.smithi144.stdout:6/172: link d7/db/ce d7/db/d3a/c40 0 2024-05-15T15:54:04.670 INFO:tasks.workunit.client.1.smithi144.stdout:7/219: write d4/df/d25/f35 [4052274,6173] 0 2024-05-15T15:54:04.671 INFO:tasks.workunit.client.0.smithi049.stdout:7/261: symlink d0/l4b 0 2024-05-15T15:54:04.671 INFO:tasks.workunit.client.1.smithi144.stdout:6/173: readlink d7/db/d11/l16 0 2024-05-15T15:54:04.675 INFO:tasks.workunit.client.1.smithi144.stdout:5/208: fdatasync d0/dc/fe 0 2024-05-15T15:54:04.677 INFO:tasks.workunit.client.1.smithi144.stdout:5/209: chown d0/l25 50663981 1 2024-05-15T15:54:04.678 INFO:tasks.workunit.client.1.smithi144.stdout:7/220: dwrite d4/df/d10/f23 [0,4194304] 0 2024-05-15T15:54:04.678 INFO:tasks.workunit.client.1.smithi144.stdout:9/192: mkdir d0/d8/d50 0 2024-05-15T15:54:04.680 INFO:tasks.workunit.client.1.smithi144.stdout:2/211: mknod d3/d10/d15/c43 0 2024-05-15T15:54:04.680 INFO:tasks.workunit.client.0.smithi049.stdout:0/266: mkdir dc/d13/d48/d57 0 2024-05-15T15:54:04.681 INFO:tasks.workunit.client.1.smithi144.stdout:9/193: write d0/d8/d21/f2a [919967,112949] 0 2024-05-15T15:54:04.684 INFO:tasks.workunit.client.1.smithi144.stdout:9/194: readlink d0/l3 0 2024-05-15T15:54:04.685 INFO:tasks.workunit.client.1.smithi144.stdout:8/210: dwrite d2/de/f10 [4194304,4194304] 0 2024-05-15T15:54:04.685 INFO:tasks.workunit.client.0.smithi049.stdout:0/267: dread dc/d13/d48/f4f [0,4194304] 0 2024-05-15T15:54:04.689 INFO:tasks.workunit.client.1.smithi144.stdout:4/219: rename d6/dd/d2b to d6/dd/d15/d1e/d44 0 2024-05-15T15:54:04.689 INFO:tasks.workunit.client.1.smithi144.stdout:9/195: dread d0/d8/d21/d29/d2d/f32 [0,4194304] 0 2024-05-15T15:54:04.690 INFO:tasks.workunit.client.1.smithi144.stdout:9/196: stat d0/d8 0 2024-05-15T15:54:04.692 INFO:tasks.workunit.client.0.smithi049.stdout:8/300: creat d2/d14/f5c x:0 0 0 2024-05-15T15:54:04.693 INFO:tasks.workunit.client.0.smithi049.stdout:8/301: dread - d2/de/f53 zero size 2024-05-15T15:54:04.694 INFO:tasks.workunit.client.0.smithi049.stdout:0/268: dread dc/d13/d3d/f3f [0,4194304] 0 2024-05-15T15:54:04.694 INFO:tasks.workunit.client.0.smithi049.stdout:8/302: readlink d2/d14/l34 0 2024-05-15T15:54:04.695 INFO:tasks.workunit.client.0.smithi049.stdout:6/250: fsync dd/d17/f21 0 2024-05-15T15:54:04.696 INFO:tasks.workunit.client.0.smithi049.stdout:6/251: readlink dd/d17/d24/l3b 0 2024-05-15T15:54:04.697 INFO:tasks.workunit.client.0.smithi049.stdout:6/252: truncate dd/d17/f38 499578 0 2024-05-15T15:54:04.698 INFO:tasks.workunit.client.0.smithi049.stdout:6/253: chown la 3 1 2024-05-15T15:54:04.699 INFO:tasks.workunit.client.0.smithi049.stdout:7/262: unlink d0/d6/da/c42 0 2024-05-15T15:54:04.700 INFO:tasks.workunit.client.1.smithi144.stdout:9/197: dwrite d0/d8/d21/d29/d2d/d44/f4d [0,4194304] 0 2024-05-15T15:54:04.701 INFO:tasks.workunit.client.1.smithi144.stdout:1/201: dwrite f6 [4194304,4194304] 0 2024-05-15T15:54:04.702 INFO:tasks.workunit.client.1.smithi144.stdout:7/221: mkdir d4/df/d10/d48 0 2024-05-15T15:54:04.703 INFO:tasks.workunit.client.0.smithi049.stdout:1/325: getdents d0/d4/dd/de/d1e/d2e/d3e 0 2024-05-15T15:54:04.703 INFO:tasks.workunit.client.1.smithi144.stdout:2/212: symlink d3/l44 0 2024-05-15T15:54:04.704 INFO:tasks.workunit.client.0.smithi049.stdout:3/342: link d12/d14/l31 d12/d67/l6a 0 2024-05-15T15:54:04.709 INFO:tasks.workunit.client.0.smithi049.stdout:4/389: getdents d18/d1e 0 2024-05-15T15:54:04.710 INFO:tasks.workunit.client.1.smithi144.stdout:3/211: dwrite d0/d1/f5 [0,4194304] 0 2024-05-15T15:54:04.711 INFO:tasks.workunit.client.0.smithi049.stdout:0/269: dread - dc/d32/f39 zero size 2024-05-15T15:54:04.711 INFO:tasks.workunit.client.0.smithi049.stdout:4/390: chown d18/d1e/d2a/d3b/c68 603517 1 2024-05-15T15:54:04.713 INFO:tasks.workunit.client.1.smithi144.stdout:5/210: mknod d0/dc/c45 0 2024-05-15T15:54:04.714 INFO:tasks.workunit.client.0.smithi049.stdout:6/254: fsync dd/f19 0 2024-05-15T15:54:04.717 INFO:tasks.workunit.client.0.smithi049.stdout:7/263: rename d0/d6/f18 to d0/d6/d23/f4c 0 2024-05-15T15:54:04.717 INFO:tasks.workunit.client.1.smithi144.stdout:5/211: write d0/dc/fe [2589875,118416] 0 2024-05-15T15:54:04.717 INFO:tasks.workunit.client.1.smithi144.stdout:1/202: dwrite d7/d1d/d33/d29/f2c [0,4194304] 0 2024-05-15T15:54:04.719 INFO:tasks.workunit.client.0.smithi049.stdout:1/326: dread d0/d4/dd/de/d36/f4c [0,4194304] 0 2024-05-15T15:54:04.721 INFO:tasks.workunit.client.1.smithi144.stdout:3/212: dread d0/f2c [0,4194304] 0 2024-05-15T15:54:04.721 INFO:tasks.workunit.client.1.smithi144.stdout:8/211: link d2/d3a/f3e d2/de/d36/f48 0 2024-05-15T15:54:04.721 INFO:tasks.workunit.client.0.smithi049.stdout:6/255: dread dd/d26/f3c [0,4194304] 0 2024-05-15T15:54:04.721 INFO:tasks.workunit.client.0.smithi049.stdout:1/327: write d0/d4/dd/f28 [4264365,47202] 0 2024-05-15T15:54:04.722 INFO:tasks.workunit.client.1.smithi144.stdout:8/212: stat d2/d3a 0 2024-05-15T15:54:04.724 INFO:tasks.workunit.client.0.smithi049.stdout:1/328: readlink d0/d4/dd/de/d1e/d2e/d3e/l67 0 2024-05-15T15:54:04.726 INFO:tasks.workunit.client.1.smithi144.stdout:0/184: creat d4/df/f43 x:0 0 0 2024-05-15T15:54:04.727 INFO:tasks.workunit.client.0.smithi049.stdout:5/286: link dc/dd/d10/l26 dc/d4a/d42/l51 0 2024-05-15T15:54:04.728 INFO:tasks.workunit.client.0.smithi049.stdout:3/343: dwrite d12/d14/f1b [0,4194304] 0 2024-05-15T15:54:04.735 INFO:tasks.workunit.client.1.smithi144.stdout:1/203: creat d7/d1d/d33/f3f x:0 0 0 2024-05-15T15:54:04.737 INFO:tasks.workunit.client.0.smithi049.stdout:5/287: dwrite dc/dd/d47/f4d [0,4194304] 0 2024-05-15T15:54:04.740 INFO:tasks.workunit.client.1.smithi144.stdout:8/213: mkdir d2/de/d49 0 2024-05-15T15:54:04.742 INFO:tasks.workunit.client.0.smithi049.stdout:8/303: dread d2/d14/f1e [4194304,4194304] 0 2024-05-15T15:54:04.742 INFO:tasks.workunit.client.0.smithi049.stdout:9/226: getdents d0/d8/d9/d22 0 2024-05-15T15:54:04.748 INFO:tasks.workunit.client.0.smithi049.stdout:0/270: rename dc/d13/d14/d2a/f41 to dc/d13/d14/d25/d31/f58 0 2024-05-15T15:54:04.749 INFO:tasks.workunit.client.0.smithi049.stdout:5/288: dwrite dc/dd/d10/f1a [0,4194304] 0 2024-05-15T15:54:04.751 INFO:tasks.workunit.client.1.smithi144.stdout:6/174: rename d7/d1e/f2e to d7/f41 0 2024-05-15T15:54:04.752 INFO:tasks.workunit.client.0.smithi049.stdout:5/289: chown dc/d4a/d42/f48 0 1 2024-05-15T15:54:04.755 INFO:tasks.workunit.client.1.smithi144.stdout:0/185: creat d4/d2d/f44 x:0 0 0 2024-05-15T15:54:04.755 INFO:tasks.workunit.client.1.smithi144.stdout:0/186: readlink d4/df/l1a 0 2024-05-15T15:54:04.755 INFO:tasks.workunit.client.0.smithi049.stdout:8/304: dwrite d2/f3 [4194304,4194304] 0 2024-05-15T15:54:04.757 INFO:tasks.workunit.client.0.smithi049.stdout:5/290: read dc/f3e [401149,122195] 0 2024-05-15T15:54:04.757 INFO:tasks.workunit.client.1.smithi144.stdout:0/187: dread - d4/f3f zero size 2024-05-15T15:54:04.758 INFO:tasks.workunit.client.1.smithi144.stdout:1/204: dread d7/d1d/d33/f12 [0,4194304] 0 2024-05-15T15:54:04.762 INFO:tasks.workunit.client.0.smithi049.stdout:0/271: dwrite dc/d13/d3d/f43 [0,4194304] 0 2024-05-15T15:54:04.764 INFO:tasks.workunit.client.0.smithi049.stdout:3/344: mkdir d12/d30/d42/d6b 0 2024-05-15T15:54:04.764 INFO:tasks.workunit.client.1.smithi144.stdout:8/214: creat d2/de/d36/f4a x:0 0 0 2024-05-15T15:54:04.765 INFO:tasks.workunit.client.0.smithi049.stdout:3/345: dread - d12/d14/d52/f5f zero size 2024-05-15T15:54:04.765 INFO:tasks.workunit.client.0.smithi049.stdout:4/391: mkdir d18/d76/d81 0 2024-05-15T15:54:04.770 INFO:tasks.workunit.client.0.smithi049.stdout:3/346: write d12/d14/d52/f61 [339984,7594] 0 2024-05-15T15:54:04.770 INFO:tasks.workunit.client.0.smithi049.stdout:7/264: dwrite d0/d6/f14 [4194304,4194304] 0 2024-05-15T15:54:04.771 INFO:tasks.workunit.client.1.smithi144.stdout:9/198: rename d0/d8/d21/f2f to d0/d8/f51 0 2024-05-15T15:54:04.772 INFO:tasks.workunit.client.0.smithi049.stdout:3/347: chown d12/d14/f54 9 1 2024-05-15T15:54:04.773 INFO:tasks.workunit.client.1.smithi144.stdout:9/199: fsync d0/d8/d21/d29/d2d/d44/d16/f46 0 2024-05-15T15:54:04.774 INFO:tasks.workunit.client.1.smithi144.stdout:9/200: fsync d0/d8/f11 0 2024-05-15T15:54:04.775 INFO:tasks.workunit.client.1.smithi144.stdout:6/175: readlink d7/d1e/d28/l34 0 2024-05-15T15:54:04.775 INFO:tasks.workunit.client.1.smithi144.stdout:6/176: stat f6 0 2024-05-15T15:54:04.775 INFO:tasks.workunit.client.0.smithi049.stdout:3/348: dread - d12/d14/d32/f36 zero size 2024-05-15T15:54:04.776 INFO:tasks.workunit.client.1.smithi144.stdout:9/201: read d0/d8/d21/d29/d2d/d44/f4d [702754,82996] 0 2024-05-15T15:54:04.776 INFO:tasks.workunit.client.0.smithi049.stdout:9/227: creat d0/d25/f4c x:0 0 0 2024-05-15T15:54:04.777 INFO:tasks.workunit.client.0.smithi049.stdout:9/228: readlink d0/d8/d12/l3f 0 2024-05-15T15:54:04.786 INFO:tasks.workunit.client.1.smithi144.stdout:0/188: unlink d4/f3f 0 2024-05-15T15:54:04.788 INFO:tasks.workunit.client.1.smithi144.stdout:8/215: mkdir d2/d1a/d38/d4b 0 2024-05-15T15:54:04.789 INFO:tasks.workunit.client.0.smithi049.stdout:6/256: creat dd/d17/d24/d2b/d36/f41 x:0 0 0 2024-05-15T15:54:04.790 INFO:tasks.workunit.client.1.smithi144.stdout:5/212: truncate d0/dc/d1f/f3b 3089826 0 2024-05-15T15:54:04.791 INFO:tasks.workunit.client.1.smithi144.stdout:0/189: dwrite d4/f5 [0,4194304] 0 2024-05-15T15:54:04.792 INFO:tasks.workunit.client.1.smithi144.stdout:5/213: chown d0/dc/d24/d28/c2e 4 1 2024-05-15T15:54:04.792 INFO:tasks.workunit.client.1.smithi144.stdout:7/222: rename d4/f6 to d4/d41/f49 0 2024-05-15T15:54:04.794 INFO:tasks.workunit.client.1.smithi144.stdout:4/220: sync 2024-05-15T15:54:04.795 INFO:tasks.workunit.client.0.smithi049.stdout:1/329: dwrite d0/d4/dd/de/d1e/f1d [0,4194304] 0 2024-05-15T15:54:04.798 INFO:tasks.workunit.client.1.smithi144.stdout:3/213: truncate d0/d11/f26 3902861 0 2024-05-15T15:54:04.798 INFO:tasks.workunit.client.0.smithi049.stdout:1/330: chown d0/d4/f9 366245 1 2024-05-15T15:54:04.799 INFO:tasks.workunit.client.1.smithi144.stdout:7/223: dwrite d4/df/d25/d38/f3d [0,4194304] 0 2024-05-15T15:54:04.800 INFO:tasks.workunit.client.0.smithi049.stdout:5/291: symlink dc/dd/d39/l52 0 2024-05-15T15:54:04.801 INFO:tasks.workunit.client.0.smithi049.stdout:5/292: stat dc/dd/d39 0 2024-05-15T15:54:04.801 INFO:tasks.workunit.client.0.smithi049.stdout:8/305: dwrite d2/de/f55 [0,4194304] 0 2024-05-15T15:54:04.803 INFO:tasks.workunit.client.1.smithi144.stdout:6/177: creat d7/db/d11/f42 x:0 0 0 2024-05-15T15:54:04.806 INFO:tasks.workunit.client.1.smithi144.stdout:2/213: dread d3/d10/f1d [0,4194304] 0 2024-05-15T15:54:04.809 INFO:tasks.workunit.client.1.smithi144.stdout:9/202: mknod d0/d8/d21/d29/d2d/d44/d10/d19/d3e/c52 0 2024-05-15T15:54:04.810 INFO:tasks.workunit.client.0.smithi049.stdout:4/392: rmdir d18/d21/d2b/d4e 39 2024-05-15T15:54:04.816 INFO:tasks.workunit.client.1.smithi144.stdout:9/203: dwrite d0/d8/d21/d29/d2d/d44/d10/d19/d3e/f3f [0,4194304] 0 2024-05-15T15:54:04.817 INFO:tasks.workunit.client.1.smithi144.stdout:8/216: rmdir d2/de/d36 39 2024-05-15T15:54:04.818 INFO:tasks.workunit.client.0.smithi049.stdout:0/272: dwrite dc/f1d [0,4194304] 0 2024-05-15T15:54:04.823 INFO:tasks.workunit.client.0.smithi049.stdout:7/265: write d0/fb [209012,27911] 0 2024-05-15T15:54:04.828 INFO:tasks.workunit.client.1.smithi144.stdout:1/205: rename d7 to d7/d40 22 2024-05-15T15:54:04.828 INFO:tasks.workunit.client.0.smithi049.stdout:6/257: creat dd/d17/d24/d2b/d36/f42 x:0 0 0 2024-05-15T15:54:04.828 INFO:tasks.workunit.client.1.smithi144.stdout:1/206: chown d7/df/d24 461 1 2024-05-15T15:54:04.832 INFO:tasks.workunit.client.0.smithi049.stdout:1/331: creat d0/d4/dd/de/d4e/f7e x:0 0 0 2024-05-15T15:54:04.833 INFO:tasks.workunit.client.1.smithi144.stdout:4/221: mknod d6/dd/d15/d1e/d44/c45 0 2024-05-15T15:54:04.835 INFO:tasks.workunit.client.1.smithi144.stdout:1/207: dread d7/d9/fe [0,4194304] 0 2024-05-15T15:54:04.836 INFO:tasks.workunit.client.0.smithi049.stdout:8/306: unlink d2/d18/f4e 0 2024-05-15T15:54:04.836 INFO:tasks.workunit.client.1.smithi144.stdout:5/214: dwrite d0/dc/d1f/d29/f3c [0,4194304] 0 2024-05-15T15:54:04.837 INFO:tasks.workunit.client.1.smithi144.stdout:1/208: fdatasync d7/d1d/d33/d29/f2c 0 2024-05-15T15:54:04.837 INFO:tasks.workunit.client.1.smithi144.stdout:5/215: readlink d0/ld 0 2024-05-15T15:54:04.839 INFO:tasks.workunit.client.0.smithi049.stdout:6/258: dwrite dd/f3a [0,4194304] 0 2024-05-15T15:54:04.839 INFO:tasks.workunit.client.0.smithi049.stdout:5/293: creat dc/d4a/d42/f53 x:0 0 0 2024-05-15T15:54:04.842 INFO:tasks.workunit.client.1.smithi144.stdout:3/214: mkdir d0/d1/d17/d39/d4d 0 2024-05-15T15:54:04.843 INFO:tasks.workunit.client.1.smithi144.stdout:7/224: rmdir d4/df 39 2024-05-15T15:54:04.852 INFO:tasks.workunit.client.1.smithi144.stdout:6/178: dwrite d7/db/f1d [0,4194304] 0 2024-05-15T15:54:04.855 INFO:tasks.workunit.client.1.smithi144.stdout:4/222: dread d6/dd/d34/d3b/f21 [0,4194304] 0 2024-05-15T15:54:04.857 INFO:tasks.workunit.client.0.smithi049.stdout:5/294: dread dc/dd/f3d [0,4194304] 0 2024-05-15T15:54:04.864 INFO:tasks.workunit.client.1.smithi144.stdout:0/190: creat d4/df/d1f/f45 x:0 0 0 2024-05-15T15:54:04.864 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:04 smithi144 ceph-mon[36389]: pgmap v22: 65 pgs: 65 active+clean; 847 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.4 MiB/s rd, 95 MiB/s wr, 352 op/s 2024-05-15T15:54:04.869 INFO:tasks.workunit.client.0.smithi049.stdout:4/393: mknod d18/d69/c82 0 2024-05-15T15:54:04.873 INFO:tasks.workunit.client.1.smithi144.stdout:5/216: unlink d0/dc/c45 0 2024-05-15T15:54:04.876 INFO:tasks.workunit.client.1.smithi144.stdout:1/209: rename d7/d9 to d7/d1d/d33/d41 0 2024-05-15T15:54:04.878 INFO:tasks.workunit.client.0.smithi049.stdout:0/273: symlink dc/d13/d14/d25/d31/d38/l59 0 2024-05-15T15:54:04.880 INFO:tasks.workunit.client.1.smithi144.stdout:7/225: stat d4/df/f20 0 2024-05-15T15:54:04.883 INFO:tasks.workunit.client.1.smithi144.stdout:6/179: unlink d7/db/f3e 0 2024-05-15T15:54:04.884 INFO:tasks.workunit.client.1.smithi144.stdout:7/226: dread d4/df/f34 [0,4194304] 0 2024-05-15T15:54:04.884 INFO:tasks.workunit.client.1.smithi144.stdout:8/217: write d2/d3a/f3e [621634,38878] 0 2024-05-15T15:54:04.886 INFO:tasks.workunit.client.0.smithi049.stdout:1/332: write d0/d4/dd/de/d36/f4c [2787908,105031] 0 2024-05-15T15:54:04.891 INFO:tasks.workunit.client.0.smithi049.stdout:1/333: dwrite d0/f58 [0,4194304] 0 2024-05-15T15:54:04.899 INFO:tasks.workunit.client.1.smithi144.stdout:9/204: write d0/d8/d21/d29/d2d/d44/d10/d19/f33 [5810,91742] 0 2024-05-15T15:54:04.901 INFO:tasks.workunit.client.0.smithi049.stdout:8/307: mkdir d2/d4f/d5d 0 2024-05-15T15:54:04.903 INFO:tasks.workunit.client.0.smithi049.stdout:8/308: write d2/d14/f43 [1336933,37335] 0 2024-05-15T15:54:04.904 INFO:tasks.workunit.client.1.smithi144.stdout:9/205: dwrite d0/d8/d21/d29/d2d/d44/d16/f4e [0,4194304] 0 2024-05-15T15:54:04.906 INFO:tasks.workunit.client.1.smithi144.stdout:9/206: write d0/d8/f51 [4005251,17488] 0 2024-05-15T15:54:04.907 INFO:tasks.workunit.client.0.smithi049.stdout:7/266: unlink d0/d6/da/d3d/c2d 0 2024-05-15T15:54:04.910 INFO:tasks.workunit.client.1.smithi144.stdout:9/207: dread - d0/d8/d21/d29/d2d/d44/d3c/f45 zero size 2024-05-15T15:54:04.910 INFO:tasks.workunit.client.0.smithi049.stdout:7/267: dread - d0/d6/f3c zero size 2024-05-15T15:54:04.911 INFO:tasks.workunit.client.0.smithi049.stdout:3/349: truncate f10 3600857 0 2024-05-15T15:54:04.915 INFO:tasks.workunit.client.1.smithi144.stdout:5/217: truncate d0/dc/d1f/f38 1303647 0 2024-05-15T15:54:04.916 INFO:tasks.workunit.client.1.smithi144.stdout:5/218: dread - d0/dc/d1f/f39 zero size 2024-05-15T15:54:04.918 INFO:tasks.workunit.client.1.smithi144.stdout:1/210: creat d7/d1d/d33/d29/f42 x:0 0 0 2024-05-15T15:54:04.918 INFO:tasks.workunit.client.1.smithi144.stdout:1/211: write d7/df/f15 [1835573,85492] 0 2024-05-15T15:54:04.919 INFO:tasks.workunit.client.0.smithi049.stdout:1/334: dread d0/d4/dd/de/f13 [0,4194304] 0 2024-05-15T15:54:04.921 INFO:tasks.workunit.client.0.smithi049.stdout:6/259: mknod dd/c43 0 2024-05-15T15:54:04.923 INFO:tasks.workunit.client.0.smithi049.stdout:6/260: truncate dd/d17/d24/d2b/d36/f42 1014625 0 2024-05-15T15:54:04.925 INFO:tasks.workunit.client.0.smithi049.stdout:8/309: rmdir d2 39 2024-05-15T15:54:04.927 INFO:tasks.workunit.client.0.smithi049.stdout:0/274: dread dc/d13/d14/d2f/f29 [0,4194304] 0 2024-05-15T15:54:04.928 INFO:tasks.workunit.client.0.smithi049.stdout:7/268: rename d0/fb to d0/d6/d1d/f4d 0 2024-05-15T15:54:04.931 INFO:tasks.workunit.client.0.smithi049.stdout:9/229: getdents d0/d8/d12 0 2024-05-15T15:54:04.932 INFO:tasks.workunit.client.0.smithi049.stdout:9/230: dread - d0/d8/d1b/f3a zero size 2024-05-15T15:54:04.934 INFO:tasks.workunit.client.1.smithi144.stdout:2/214: creat d3/d7/d20/d37/d3a/f45 x:0 0 0 2024-05-15T15:54:04.934 INFO:tasks.workunit.client.1.smithi144.stdout:2/215: chown d3/d7/d20/f3e 27516 1 2024-05-15T15:54:04.935 INFO:tasks.workunit.client.1.smithi144.stdout:7/227: creat d4/df/d25/f4a x:0 0 0 2024-05-15T15:54:04.936 INFO:tasks.workunit.client.1.smithi144.stdout:8/218: creat d2/de/d3f/f4c x:0 0 0 2024-05-15T15:54:04.938 INFO:tasks.workunit.client.0.smithi049.stdout:1/335: mknod d0/d4/d63/c7f 0 2024-05-15T15:54:04.939 INFO:tasks.workunit.client.1.smithi144.stdout:4/223: mkdir d6/d42/d46 0 2024-05-15T15:54:04.939 INFO:tasks.workunit.client.1.smithi144.stdout:8/219: dread d2/de/f10 [4194304,4194304] 0 2024-05-15T15:54:04.945 INFO:tasks.workunit.client.0.smithi049.stdout:7/269: dread - d0/f2e zero size 2024-05-15T15:54:04.949 INFO:tasks.workunit.client.1.smithi144.stdout:1/212: mknod d7/d1d/d33/d29/c43 0 2024-05-15T15:54:04.951 INFO:tasks.workunit.client.0.smithi049.stdout:1/336: dwrite d0/d4/dd/de/d36/f4c [0,4194304] 0 2024-05-15T15:54:04.951 INFO:tasks.workunit.client.0.smithi049.stdout:6/261: dread f2 [0,4194304] 0 2024-05-15T15:54:04.955 INFO:tasks.workunit.client.0.smithi049.stdout:8/310: dread d2/de/d27/f39 [0,4194304] 0 2024-05-15T15:54:04.956 INFO:tasks.workunit.client.0.smithi049.stdout:0/275: dwrite dc/d13/d14/d25/d31/f58 [0,4194304] 0 2024-05-15T15:54:04.960 INFO:tasks.workunit.client.1.smithi144.stdout:6/180: dwrite d7/db/f29 [0,4194304] 0 2024-05-15T15:54:04.962 INFO:tasks.workunit.client.1.smithi144.stdout:2/216: rename d3/d10/f11 to d3/d7/d20/d37/f46 0 2024-05-15T15:54:04.963 INFO:tasks.workunit.client.1.smithi144.stdout:7/228: unlink d4/f36 0 2024-05-15T15:54:04.965 INFO:tasks.workunit.client.0.smithi049.stdout:6/262: mknod dd/d26/c44 0 2024-05-15T15:54:04.966 INFO:tasks.workunit.client.1.smithi144.stdout:6/181: read d7/db/d11/f15 [419030,89586] 0 2024-05-15T15:54:04.968 INFO:tasks.workunit.client.0.smithi049.stdout:1/337: symlink d0/d4/dd/de/d36/l80 0 2024-05-15T15:54:04.969 INFO:tasks.workunit.client.1.smithi144.stdout:2/217: truncate d3/d10/f29 497922 0 2024-05-15T15:54:04.970 INFO:tasks.workunit.client.1.smithi144.stdout:4/224: unlink d6/dd/c3e 0 2024-05-15T15:54:04.971 INFO:tasks.workunit.client.1.smithi144.stdout:4/225: chown d6/dd/d15/f1c 19 1 2024-05-15T15:54:04.972 INFO:tasks.workunit.client.1.smithi144.stdout:0/191: creat d4/df/f46 x:0 0 0 2024-05-15T15:54:04.972 INFO:tasks.workunit.client.1.smithi144.stdout:8/220: write d2/de/d36/f48 [1141366,111322] 0 2024-05-15T15:54:04.975 INFO:tasks.workunit.client.1.smithi144.stdout:4/226: chown d6/dd/c20 1888 1 2024-05-15T15:54:04.975 INFO:tasks.workunit.client.1.smithi144.stdout:7/229: dwrite d4/df/d10/f1d [0,4194304] 0 2024-05-15T15:54:04.975 INFO:tasks.workunit.client.0.smithi049.stdout:8/311: creat d2/de/d25/f5e x:0 0 0 2024-05-15T15:54:04.976 INFO:tasks.workunit.client.1.smithi144.stdout:7/230: dread - d4/f1c zero size 2024-05-15T15:54:04.976 INFO:tasks.workunit.client.0.smithi049.stdout:0/276: dread f3 [0,4194304] 0 2024-05-15T15:54:04.977 INFO:tasks.workunit.client.0.smithi049.stdout:0/277: chown dc/d13/f26 1824 1 2024-05-15T15:54:04.978 INFO:tasks.workunit.client.0.smithi049.stdout:7/270: rename d0/f41 to d0/d6/d1d/f4e 0 2024-05-15T15:54:04.979 INFO:tasks.workunit.client.1.smithi144.stdout:3/215: link d0/d1/d17/d39/d40/l42 d0/d1/d17/d39/d4d/l4e 0 2024-05-15T15:54:04.980 INFO:tasks.workunit.client.0.smithi049.stdout:3/350: getdents d12/d14 0 2024-05-15T15:54:04.981 INFO:tasks.workunit.client.0.smithi049.stdout:6/263: fsync dd/d17/f21 0 2024-05-15T15:54:04.982 INFO:tasks.workunit.client.0.smithi049.stdout:6/264: chown dd/d17 24 1 2024-05-15T15:54:04.983 INFO:tasks.workunit.client.0.smithi049.stdout:6/265: write dd/d17/f38 [1346145,20342] 0 2024-05-15T15:54:04.984 INFO:tasks.workunit.client.1.smithi144.stdout:1/213: rename d7/df/c35 to d7/d1d/d33/d29/c44 0 2024-05-15T15:54:04.985 INFO:tasks.workunit.client.0.smithi049.stdout:8/312: mknod d2/de/d27/c5f 0 2024-05-15T15:54:04.986 INFO:tasks.workunit.client.0.smithi049.stdout:6/266: dread dd/d17/d24/d2b/d36/f42 [0,4194304] 0 2024-05-15T15:54:04.989 INFO:tasks.workunit.client.0.smithi049.stdout:7/271: chown d0/d34/f46 51761 1 2024-05-15T15:54:04.989 INFO:tasks.workunit.client.1.smithi144.stdout:1/214: dwrite d7/df/f10 [0,4194304] 0 2024-05-15T15:54:04.991 INFO:tasks.workunit.client.1.smithi144.stdout:1/215: read d7/d1d/d33/d41/f19 [1937966,36042] 0 2024-05-15T15:54:04.992 INFO:tasks.workunit.client.1.smithi144.stdout:9/208: creat d0/d8/d21/d29/d2d/d44/f53 x:0 0 0 2024-05-15T15:54:04.993 INFO:tasks.workunit.client.0.smithi049.stdout:1/338: mkdir d0/d4/d81 0 2024-05-15T15:54:04.993 INFO:tasks.workunit.client.1.smithi144.stdout:2/218: read d3/d10/d15/f31 [162090,34027] 0 2024-05-15T15:54:04.994 INFO:tasks.workunit.client.0.smithi049.stdout:1/339: readlink d0/d4/dd/de/d1e/d2e/d3e/l67 0 2024-05-15T15:54:04.994 INFO:tasks.workunit.client.1.smithi144.stdout:0/192: creat d4/df/d1f/d24/f47 x:0 0 0 2024-05-15T15:54:04.996 INFO:tasks.workunit.client.1.smithi144.stdout:8/221: creat d2/de/d36/f4d x:0 0 0 2024-05-15T15:54:04.998 INFO:tasks.workunit.client.1.smithi144.stdout:4/227: creat d6/dd/d15/d1e/d44/f47 x:0 0 0 2024-05-15T15:54:05.001 INFO:tasks.workunit.client.1.smithi144.stdout:0/193: dwrite d4/d6/fb [4194304,4194304] 0 2024-05-15T15:54:05.001 INFO:tasks.workunit.client.0.smithi049.stdout:6/267: mkdir dd/d17/d24/d2b/d36/d45 0 2024-05-15T15:54:05.001 INFO:tasks.workunit.client.1.smithi144.stdout:0/194: fdatasync d4/df/f16 0 2024-05-15T15:54:05.005 INFO:tasks.workunit.client.1.smithi144.stdout:3/216: creat d0/d11/d24/f4f x:0 0 0 2024-05-15T15:54:05.005 INFO:tasks.workunit.client.0.smithi049.stdout:3/351: symlink d12/l6c 0 2024-05-15T15:54:05.006 INFO:tasks.workunit.client.0.smithi049.stdout:3/352: chown d12/d14/c23 227 1 2024-05-15T15:54:05.007 INFO:tasks.workunit.client.0.smithi049.stdout:7/272: fdatasync d0/f2e 0 2024-05-15T15:54:05.008 INFO:tasks.workunit.client.1.smithi144.stdout:4/228: dwrite d6/dd/d34/d3b/d30/db/f35 [0,4194304] 0 2024-05-15T15:54:05.012 INFO:tasks.workunit.client.0.smithi049.stdout:8/313: mkdir d2/de/d27/d5b/d60 0 2024-05-15T15:54:05.013 INFO:tasks.workunit.client.0.smithi049.stdout:8/314: chown d2/de/c19 1 1 2024-05-15T15:54:05.072 INFO:tasks.workunit.client.0.smithi049.stdout:2/296: sync 2024-05-15T15:54:05.074 INFO:tasks.workunit.client.1.smithi144.stdout:1/216: fsync d7/d1d/d33/d41/fe 0 2024-05-15T15:54:05.074 INFO:tasks.workunit.client.1.smithi144.stdout:1/217: fsync d7/d1d/d33/f3f 0 2024-05-15T15:54:05.078 INFO:tasks.workunit.client.1.smithi144.stdout:8/222: rmdir d2/d3a 39 2024-05-15T15:54:05.080 INFO:tasks.workunit.client.1.smithi144.stdout:6/182: truncate d7/db/d11/f1b 3610317 0 2024-05-15T15:54:05.081 INFO:tasks.workunit.client.1.smithi144.stdout:5/219: getdents d0/dc/d24/d28 0 2024-05-15T15:54:05.083 INFO:tasks.workunit.client.0.smithi049.stdout:7/273: write d0/d6/d1d/f4e [2672497,29037] 0 2024-05-15T15:54:05.084 INFO:tasks.workunit.client.0.smithi049.stdout:3/353: dwrite d12/d14/f1a [0,4194304] 0 2024-05-15T15:54:05.087 INFO:tasks.workunit.client.0.smithi049.stdout:3/354: dread d12/d14/f24 [0,4194304] 0 2024-05-15T15:54:05.088 INFO:tasks.workunit.client.0.smithi049.stdout:2/297: dread d3/d8/f14 [0,4194304] 0 2024-05-15T15:54:05.088 INFO:tasks.workunit.client.1.smithi144.stdout:9/209: dwrite d0/f18 [0,4194304] 0 2024-05-15T15:54:05.091 INFO:tasks.workunit.client.1.smithi144.stdout:7/231: truncate d4/df/d10/f1d 3511840 0 2024-05-15T15:54:05.091 INFO:tasks.workunit.client.1.smithi144.stdout:9/210: truncate d0/d8/d21/d29/d2d/d44/d10/d19/f49 788651 0 2024-05-15T15:54:05.092 INFO:tasks.workunit.client.0.smithi049.stdout:0/278: link dc/cf dc/d13/d14/d25/c5a 0 2024-05-15T15:54:05.092 INFO:tasks.workunit.client.1.smithi144.stdout:3/217: truncate d0/d11/d24/f22 1198546 0 2024-05-15T15:54:05.095 INFO:tasks.workunit.client.1.smithi144.stdout:2/219: sync 2024-05-15T15:54:05.097 INFO:tasks.workunit.client.1.smithi144.stdout:2/220: stat d3/d10/f29 0 2024-05-15T15:54:05.097 INFO:tasks.workunit.client.1.smithi144.stdout:1/218: mkdir d7/d1d/d33/d41/d22/d45 0 2024-05-15T15:54:05.098 INFO:tasks.workunit.client.1.smithi144.stdout:2/221: write d3/d7/f1f [2044684,42916] 0 2024-05-15T15:54:05.099 INFO:tasks.workunit.client.0.smithi049.stdout:3/355: creat d12/d14/d32/f6d x:0 0 0 2024-05-15T15:54:05.100 INFO:tasks.workunit.client.1.smithi144.stdout:9/211: dwrite d0/d8/d21/d29/d2d/d44/d10/f2c [0,4194304] 0 2024-05-15T15:54:05.101 INFO:tasks.workunit.client.1.smithi144.stdout:6/183: creat d7/db/d11/f43 x:0 0 0 2024-05-15T15:54:05.106 INFO:tasks.workunit.client.0.smithi049.stdout:6/268: rmdir dd/d17/d24/d2b/d36/d45 0 2024-05-15T15:54:05.106 INFO:tasks.workunit.client.0.smithi049.stdout:6/269: chown dd/d17 8144679 1 2024-05-15T15:54:05.107 INFO:tasks.workunit.client.0.smithi049.stdout:6/270: readlink dd/l23 0 2024-05-15T15:54:05.107 INFO:tasks.workunit.client.1.smithi144.stdout:1/219: dwrite d7/df/f15 [0,4194304] 0 2024-05-15T15:54:05.107 INFO:tasks.workunit.client.1.smithi144.stdout:8/223: dwrite d2/d1a/f31 [0,4194304] 0 2024-05-15T15:54:05.109 INFO:tasks.workunit.client.0.smithi049.stdout:0/279: creat dc/d13/d14/d33/f5b x:0 0 0 2024-05-15T15:54:05.110 INFO:tasks.workunit.client.0.smithi049.stdout:0/280: dread - dc/d13/d2b/f4d zero size 2024-05-15T15:54:05.113 INFO:tasks.workunit.client.0.smithi049.stdout:8/315: rename d2/de/d25/f5e to d2/d14/d1d/f61 0 2024-05-15T15:54:05.114 INFO:tasks.workunit.client.0.smithi049.stdout:8/316: write d2/fc [7410704,43860] 0 2024-05-15T15:54:05.117 INFO:tasks.workunit.client.0.smithi049.stdout:8/317: chown d2/d14 0 1 2024-05-15T15:54:05.118 INFO:tasks.workunit.client.0.smithi049.stdout:8/318: write f0 [1502879,71830] 0 2024-05-15T15:54:05.118 INFO:tasks.workunit.client.0.smithi049.stdout:3/356: creat d12/d30/d42/f6e x:0 0 0 2024-05-15T15:54:05.118 INFO:tasks.workunit.client.1.smithi144.stdout:3/218: readlink d0/d1/d17/d39/l4c 0 2024-05-15T15:54:05.119 INFO:tasks.workunit.client.1.smithi144.stdout:3/219: stat d0/d11/f2a 0 2024-05-15T15:54:05.119 INFO:tasks.workunit.client.1.smithi144.stdout:3/220: chown d0/f46 30 1 2024-05-15T15:54:05.120 INFO:tasks.workunit.client.1.smithi144.stdout:3/221: write d0/d1/d17/f48 [235790,60817] 0 2024-05-15T15:54:05.122 INFO:tasks.workunit.client.0.smithi049.stdout:6/271: creat dd/d17/d24/d2b/f46 x:0 0 0 2024-05-15T15:54:05.125 INFO:tasks.workunit.client.1.smithi144.stdout:3/222: dread d0/f2c [0,4194304] 0 2024-05-15T15:54:05.125 INFO:tasks.workunit.client.1.smithi144.stdout:3/223: fdatasync d0/d1/f5 0 2024-05-15T15:54:05.126 INFO:tasks.workunit.client.1.smithi144.stdout:3/224: write d0/d1/d17/f48 [220985,129529] 0 2024-05-15T15:54:05.128 INFO:tasks.workunit.client.1.smithi144.stdout:2/222: creat d3/d7/f47 x:0 0 0 2024-05-15T15:54:05.130 INFO:tasks.workunit.client.1.smithi144.stdout:2/223: fdatasync d3/d7/f1a 0 2024-05-15T15:54:05.133 INFO:tasks.workunit.client.1.smithi144.stdout:9/212: fdatasync d0/d8/d21/d29/d2d/d44/d10/f2c 0 2024-05-15T15:54:05.134 INFO:tasks.workunit.client.1.smithi144.stdout:3/225: dwrite d0/d1/d17/d39/f3c [0,4194304] 0 2024-05-15T15:54:05.139 INFO:tasks.workunit.client.1.smithi144.stdout:1/220: mknod d7/df/c46 0 2024-05-15T15:54:05.140 INFO:tasks.workunit.client.1.smithi144.stdout:1/221: chown f6 451998 1 2024-05-15T15:54:05.140 INFO:tasks.workunit.client.1.smithi144.stdout:9/213: dread d0/d8/d21/d29/d2d/d44/d10/f2c [0,4194304] 0 2024-05-15T15:54:05.141 INFO:tasks.workunit.client.1.smithi144.stdout:5/220: dread d0/dc/f1b [0,4194304] 0 2024-05-15T15:54:05.144 INFO:tasks.workunit.client.1.smithi144.stdout:1/222: dread d7/d1d/d33/f12 [0,4194304] 0 2024-05-15T15:54:05.146 INFO:tasks.workunit.client.1.smithi144.stdout:5/221: dwrite d0/f16 [0,4194304] 0 2024-05-15T15:54:05.147 INFO:tasks.workunit.client.1.smithi144.stdout:7/232: link d4/f46 d4/d41/f4b 0 2024-05-15T15:54:05.148 INFO:tasks.workunit.client.1.smithi144.stdout:8/224: rename d2/d24/d2d/d30/f42 to d2/de/d3f/f4e 0 2024-05-15T15:54:05.149 INFO:tasks.workunit.client.0.smithi049.stdout:7/274: mknod d0/d6/d23/d4a/c4f 0 2024-05-15T15:54:05.150 INFO:tasks.workunit.client.1.smithi144.stdout:8/225: fdatasync d2/de/d36/f47 0 2024-05-15T15:54:05.150 INFO:tasks.workunit.client.0.smithi049.stdout:7/275: write d0/d34/f35 [6461832,110247] 0 2024-05-15T15:54:05.152 INFO:tasks.workunit.client.1.smithi144.stdout:1/223: dwrite d7/d1d/d33/d29/f2c [0,4194304] 0 2024-05-15T15:54:05.153 INFO:tasks.workunit.client.1.smithi144.stdout:1/224: read d7/d1d/d33/d29/f2c [3818794,57644] 0 2024-05-15T15:54:05.154 INFO:tasks.workunit.client.0.smithi049.stdout:0/281: write fa [806460,49160] 0 2024-05-15T15:54:05.157 INFO:tasks.workunit.client.0.smithi049.stdout:2/298: link d3/da/d15/d24/c37 d3/db/d1d/d1e/d2b/c56 0 2024-05-15T15:54:05.159 INFO:tasks.workunit.client.0.smithi049.stdout:3/357: write d12/d14/f17 [6602668,90068] 0 2024-05-15T15:54:05.160 INFO:tasks.workunit.client.1.smithi144.stdout:0/195: sync 2024-05-15T15:54:05.160 INFO:tasks.workunit.client.0.smithi049.stdout:3/358: chown d12/d30/l5d 20909 1 2024-05-15T15:54:05.163 INFO:tasks.workunit.client.0.smithi049.stdout:0/282: fsync f3 0 2024-05-15T15:54:05.166 INFO:tasks.workunit.client.0.smithi049.stdout:6/272: write dd/d26/f3c [1853702,89546] 0 2024-05-15T15:54:05.167 INFO:tasks.workunit.client.0.smithi049.stdout:2/299: unlink d3/da/c25 0 2024-05-15T15:54:05.169 INFO:tasks.workunit.client.0.smithi049.stdout:7/276: symlink d0/l50 0 2024-05-15T15:54:05.170 INFO:tasks.workunit.client.0.smithi049.stdout:7/277: stat d0/d6/da 0 2024-05-15T15:54:05.170 INFO:tasks.workunit.client.0.smithi049.stdout:6/273: read dd/f14 [1653919,114257] 0 2024-05-15T15:54:05.172 INFO:tasks.workunit.client.0.smithi049.stdout:3/359: creat d12/d30/f6f x:0 0 0 2024-05-15T15:54:05.174 INFO:tasks.workunit.client.1.smithi144.stdout:9/214: creat d0/d8/d21/d29/d2d/d44/d10/d19/d3e/f54 x:0 0 0 2024-05-15T15:54:05.175 INFO:tasks.workunit.client.1.smithi144.stdout:9/215: write d0/d8/d21/d29/d2d/d44/d10/d19/f33 [2447500,7114] 0 2024-05-15T15:54:05.176 INFO:tasks.workunit.client.0.smithi049.stdout:8/319: truncate d2/d18/f3c 2033334 0 2024-05-15T15:54:05.176 INFO:tasks.workunit.client.1.smithi144.stdout:9/216: write d0/f14 [4206974,21798] 0 2024-05-15T15:54:05.177 INFO:tasks.workunit.client.1.smithi144.stdout:9/217: chown d0/d8/c36 161 1 2024-05-15T15:54:05.178 INFO:tasks.workunit.client.0.smithi049.stdout:6/274: dwrite dd/d17/d24/d2b/d36/f41 [0,4194304] 0 2024-05-15T15:54:05.183 INFO:tasks.workunit.client.0.smithi049.stdout:2/300: mknod d3/db/d1d/d1e/d2b/d3b/d47/c57 0 2024-05-15T15:54:05.184 INFO:tasks.workunit.client.0.smithi049.stdout:2/301: chown d3/db/d1d 7323633 1 2024-05-15T15:54:05.185 INFO:tasks.workunit.client.0.smithi049.stdout:8/320: dwrite d2/fc [0,4194304] 0 2024-05-15T15:54:05.185 INFO:tasks.workunit.client.1.smithi144.stdout:7/233: rename d4/fe to d4/df/d25/d40/d47/f4c 0 2024-05-15T15:54:05.186 INFO:tasks.workunit.client.1.smithi144.stdout:7/234: dread - d4/df/d25/f4a zero size 2024-05-15T15:54:05.187 INFO:tasks.workunit.client.1.smithi144.stdout:4/229: getdents d6/dd/d34/d3b/d30/db/d1d 0 2024-05-15T15:54:05.187 INFO:tasks.workunit.client.0.smithi049.stdout:0/283: dread dc/d13/d14/d33/f47 [0,4194304] 0 2024-05-15T15:54:05.190 INFO:tasks.workunit.client.1.smithi144.stdout:8/226: unlink d2/de/f10 0 2024-05-15T15:54:05.193 INFO:tasks.workunit.client.1.smithi144.stdout:1/225: mknod d7/df/d24/c47 0 2024-05-15T15:54:05.194 INFO:tasks.workunit.client.1.smithi144.stdout:1/226: stat d7/d1d/d33/d41/d22/f28 0 2024-05-15T15:54:05.195 INFO:tasks.workunit.client.0.smithi049.stdout:8/321: dwrite d2/fb [0,4194304] 0 2024-05-15T15:54:05.196 INFO:tasks.workunit.client.1.smithi144.stdout:6/184: creat d7/d1e/f44 x:0 0 0 2024-05-15T15:54:05.196 INFO:tasks.workunit.client.0.smithi049.stdout:7/278: dwrite d0/f11 [0,4194304] 0 2024-05-15T15:54:05.198 INFO:tasks.workunit.client.0.smithi049.stdout:8/322: write d2/f3d [417253,289] 0 2024-05-15T15:54:05.198 INFO:tasks.workunit.client.1.smithi144.stdout:0/196: mkdir d4/d2d/d48 0 2024-05-15T15:54:05.202 INFO:tasks.workunit.client.1.smithi144.stdout:2/224: symlink d3/d7/d20/l48 0 2024-05-15T15:54:05.203 INFO:tasks.workunit.client.1.smithi144.stdout:2/225: chown d3/d10/d15/c2c 59402621 1 2024-05-15T15:54:05.204 INFO:tasks.workunit.client.1.smithi144.stdout:2/226: dread - d3/d7/d20/f3e zero size 2024-05-15T15:54:05.208 INFO:tasks.workunit.client.0.smithi049.stdout:3/360: mkdir d12/d30/d63/d68/d70 0 2024-05-15T15:54:05.211 INFO:tasks.workunit.client.1.smithi144.stdout:2/227: dwrite d3/d10/d15/f2d [0,4194304] 0 2024-05-15T15:54:05.212 INFO:tasks.workunit.client.1.smithi144.stdout:9/218: unlink d0/d8/d21/d29/d2d/d44/f4d 0 2024-05-15T15:54:05.214 INFO:tasks.workunit.client.0.smithi049.stdout:6/275: mknod dd/d17/d24/c47 0 2024-05-15T15:54:05.216 INFO:tasks.workunit.client.1.smithi144.stdout:9/219: read d0/d8/d21/f2a [801272,91996] 0 2024-05-15T15:54:05.218 INFO:tasks.workunit.client.0.smithi049.stdout:7/279: creat d0/d2f/f51 x:0 0 0 2024-05-15T15:54:05.219 INFO:tasks.workunit.client.1.smithi144.stdout:5/222: truncate d0/f1a 353155 0 2024-05-15T15:54:05.220 INFO:tasks.workunit.client.1.smithi144.stdout:5/223: truncate d0/dc/d24/d28/d2d/f3d 75912 0 2024-05-15T15:54:05.223 INFO:tasks.workunit.client.0.smithi049.stdout:2/302: rename d3/db/d1d/d1e/d2b/c56 to d3/d8/c58 0 2024-05-15T15:54:05.225 INFO:tasks.workunit.client.1.smithi144.stdout:4/230: unlink d6/l8 0 2024-05-15T15:54:05.228 INFO:tasks.workunit.client.1.smithi144.stdout:7/235: rename d4/df/d25/d38/f3d to d4/df/d25/d40/f4d 0 2024-05-15T15:54:05.229 INFO:tasks.workunit.client.0.smithi049.stdout:8/323: creat d2/d4f/d5d/f62 x:0 0 0 2024-05-15T15:54:05.229 INFO:tasks.workunit.client.1.smithi144.stdout:8/227: mknod d2/d24/d2d/c4f 0 2024-05-15T15:54:05.230 INFO:tasks.workunit.client.1.smithi144.stdout:8/228: dread - d2/de/d36/f47 zero size 2024-05-15T15:54:05.231 INFO:tasks.workunit.client.0.smithi049.stdout:0/284: rename dc/d13/d14/d25/l49 to dc/d13/d14/d33/l5c 0 2024-05-15T15:54:05.232 INFO:tasks.workunit.client.1.smithi144.stdout:3/226: write d0/d11/f26 [4020488,22916] 0 2024-05-15T15:54:05.235 INFO:tasks.workunit.client.1.smithi144.stdout:1/227: chown d7/d1d/d33/d41/l18 20 1 2024-05-15T15:54:05.239 INFO:tasks.workunit.client.0.smithi049.stdout:0/285: readlink dc/le 0 2024-05-15T15:54:05.248 INFO:tasks.workunit.client.1.smithi144.stdout:2/228: dwrite d3/f5 [0,4194304] 0 2024-05-15T15:54:05.249 INFO:tasks.workunit.client.1.smithi144.stdout:1/228: dread f1 [0,4194304] 0 2024-05-15T15:54:05.249 INFO:tasks.workunit.client.0.smithi049.stdout:0/286: rename f4 to dc/d13/d3d/f5d 0 2024-05-15T15:54:05.250 INFO:tasks.workunit.client.1.smithi144.stdout:2/229: write d3/d7/d20/d37/f3c [467778,10030] 0 2024-05-15T15:54:05.251 INFO:tasks.workunit.client.0.smithi049.stdout:0/287: fsync dc/d13/d14/f46 0 2024-05-15T15:54:05.254 INFO:tasks.workunit.client.1.smithi144.stdout:0/197: rename d4/d6/c26 to d4/df/d1f/c49 0 2024-05-15T15:54:05.255 INFO:tasks.workunit.client.1.smithi144.stdout:6/185: write f1 [2547273,46966] 0 2024-05-15T15:54:05.255 INFO:tasks.workunit.client.0.smithi049.stdout:7/280: dread d0/f9 [0,4194304] 0 2024-05-15T15:54:05.256 INFO:tasks.workunit.client.1.smithi144.stdout:7/236: mknod d4/df/d25/d40/d47/c4e 0 2024-05-15T15:54:05.257 INFO:tasks.workunit.client.0.smithi049.stdout:7/281: dread - d0/d2f/f51 zero size 2024-05-15T15:54:05.257 INFO:tasks.workunit.client.0.smithi049.stdout:2/303: dwrite d3/da/d15/d24/f2f [0,4194304] 0 2024-05-15T15:54:05.261 INFO:tasks.workunit.client.0.smithi049.stdout:0/288: dread dc/d13/d14/d33/f47 [0,4194304] 0 2024-05-15T15:54:05.262 INFO:tasks.workunit.client.0.smithi049.stdout:8/324: truncate d2/fc 3339773 0 2024-05-15T15:54:05.262 INFO:tasks.workunit.client.0.smithi049.stdout:8/325: readlink d2/d18/l29 0 2024-05-15T15:54:05.263 INFO:tasks.workunit.client.0.smithi049.stdout:8/326: fdatasync f0 0 2024-05-15T15:54:05.264 INFO:tasks.workunit.client.0.smithi049.stdout:8/327: dread - d2/de/d27/f59 zero size 2024-05-15T15:54:05.264 INFO:tasks.workunit.client.0.smithi049.stdout:7/282: dread d0/d6/d1d/f4e [0,4194304] 0 2024-05-15T15:54:05.267 INFO:tasks.workunit.client.0.smithi049.stdout:2/304: mkdir d3/db/d1d/d1e/d2b/d3b/d47/d59 0 2024-05-15T15:54:05.268 INFO:tasks.workunit.client.0.smithi049.stdout:2/305: chown d3/db/d1d/c32 118 1 2024-05-15T15:54:05.269 INFO:tasks.workunit.client.0.smithi049.stdout:2/306: dread - d3/db/d1d/d1e/d2b/f54 zero size 2024-05-15T15:54:05.269 INFO:tasks.workunit.client.0.smithi049.stdout:2/307: stat d3/d8 0 2024-05-15T15:54:05.271 INFO:tasks.workunit.client.1.smithi144.stdout:2/230: mknod d3/d7/c49 0 2024-05-15T15:54:05.272 INFO:tasks.workunit.client.1.smithi144.stdout:2/231: write d3/f5 [404246,104821] 0 2024-05-15T15:54:05.273 INFO:tasks.workunit.client.1.smithi144.stdout:2/232: stat d3/d7/d20/c33 0 2024-05-15T15:54:05.274 INFO:tasks.workunit.client.1.smithi144.stdout:2/233: chown d3/d10/c19 426236311 1 2024-05-15T15:54:05.275 INFO:tasks.workunit.client.1.smithi144.stdout:1/229: truncate d7/d1d/d33/d1e/f25 5833763 0 2024-05-15T15:54:05.276 INFO:tasks.workunit.client.0.smithi049.stdout:7/283: rmdir d0/d6/da 39 2024-05-15T15:54:05.279 INFO:tasks.workunit.client.0.smithi049.stdout:8/328: dwrite d2/d14/d1d/f61 [0,4194304] 0 2024-05-15T15:54:05.279 INFO:tasks.workunit.client.0.smithi049.stdout:2/308: rename d3 to d3/db/d1d/d1e/d5a 22 2024-05-15T15:54:05.279 INFO:tasks.workunit.client.1.smithi144.stdout:9/220: rename d0/d8/c13 to d0/d8/d50/c55 0 2024-05-15T15:54:05.279 INFO:tasks.workunit.client.1.smithi144.stdout:0/198: dread d4/df/d1f/d24/f2a [0,4194304] 0 2024-05-15T15:54:05.280 INFO:tasks.workunit.client.1.smithi144.stdout:9/221: readlink d0/d8/d1b/l1d 0 2024-05-15T15:54:05.280 INFO:tasks.workunit.client.1.smithi144.stdout:0/199: readlink d4/df/l1a 0 2024-05-15T15:54:05.281 INFO:tasks.workunit.client.1.smithi144.stdout:0/200: fdatasync d4/df/d1f/d24/f47 0 2024-05-15T15:54:05.281 INFO:tasks.workunit.client.1.smithi144.stdout:9/222: read d0/d8/d21/d29/d2d/d44/d10/d19/d3e/f3f [3818213,65362] 0 2024-05-15T15:54:05.318 INFO:tasks.workunit.client.0.smithi049.stdout:5/295: sync 2024-05-15T15:54:05.318 INFO:tasks.workunit.client.0.smithi049.stdout:2/309: dwrite d3/db/d1d/d1e/d2b/f54 [0,4194304] 0 2024-05-15T15:54:05.318 INFO:tasks.workunit.client.1.smithi144.stdout:6/186: mkdir d7/db/d11/d45 0 2024-05-15T15:54:05.318 INFO:tasks.workunit.client.1.smithi144.stdout:6/187: dread - d7/db/d11/f1f zero size 2024-05-15T15:54:05.320 INFO:tasks.workunit.client.1.smithi144.stdout:6/188: read f6 [12914,61825] 0 2024-05-15T15:54:05.320 INFO:tasks.workunit.client.1.smithi144.stdout:8/229: symlink d2/de/l50 0 2024-05-15T15:54:05.321 INFO:tasks.workunit.client.1.smithi144.stdout:8/230: write d2/de/f14 [812025,56687] 0 2024-05-15T15:54:05.323 INFO:tasks.workunit.client.0.smithi049.stdout:7/284: rmdir d0/d6/da 39 2024-05-15T15:54:05.326 INFO:tasks.workunit.client.0.smithi049.stdout:2/310: dwrite d3/db/d1d/d1e/d2b/f54 [0,4194304] 0 2024-05-15T15:54:05.327 INFO:tasks.workunit.client.0.smithi049.stdout:2/311: chown d3/da/d15/d24 25 1 2024-05-15T15:54:05.329 INFO:tasks.workunit.client.0.smithi049.stdout:7/285: mknod d0/d6/d1d/c52 0 2024-05-15T15:54:05.331 INFO:tasks.workunit.client.1.smithi144.stdout:1/230: creat d7/d1d/d33/d41/d22/f48 x:0 0 0 2024-05-15T15:54:05.335 INFO:tasks.workunit.client.1.smithi144.stdout:4/231: rename d6/dd/d34/d3b/d30/ff to d6/dd/d2d/f48 0 2024-05-15T15:54:05.336 INFO:tasks.workunit.client.0.smithi049.stdout:2/312: mkdir d3/db/d5b 0 2024-05-15T15:54:05.337 INFO:tasks.workunit.client.1.smithi144.stdout:4/232: read d6/dd/d34/d3b/d30/db/f35 [1800903,20808] 0 2024-05-15T15:54:05.341 INFO:tasks.workunit.client.1.smithi144.stdout:0/201: mkdir d4/d6/d4a 0 2024-05-15T15:54:05.341 INFO:tasks.workunit.client.1.smithi144.stdout:0/202: chown d4/df/d1f/d24/d2b/f3b 4 1 2024-05-15T15:54:05.345 INFO:tasks.workunit.client.1.smithi144.stdout:0/203: read d4/f1c [423134,76279] 0 2024-05-15T15:54:05.347 INFO:tasks.workunit.client.1.smithi144.stdout:0/204: write d4/d2d/f44 [74203,27379] 0 2024-05-15T15:54:05.347 INFO:tasks.workunit.client.1.smithi144.stdout:2/234: dread d3/f9 [0,4194304] 0 2024-05-15T15:54:05.351 INFO:tasks.workunit.client.1.smithi144.stdout:3/227: rename d0/d1/f21 to d0/d1/d17/d39/d40/f50 0 2024-05-15T15:54:05.351 INFO:tasks.workunit.client.1.smithi144.stdout:8/231: dread d2/f3 [0,4194304] 0 2024-05-15T15:54:05.351 INFO:tasks.workunit.client.1.smithi144.stdout:3/228: chown d0/d1/d17/c31 459842 1 2024-05-15T15:54:05.352 INFO:tasks.workunit.client.1.smithi144.stdout:4/233: mkdir d6/d49 0 2024-05-15T15:54:05.357 INFO:tasks.workunit.client.0.smithi049.stdout:7/286: dread d0/f47 [0,4194304] 0 2024-05-15T15:54:05.358 INFO:tasks.workunit.client.0.smithi049.stdout:7/287: chown d0/f11 467769562 1 2024-05-15T15:54:05.359 INFO:tasks.workunit.client.1.smithi144.stdout:0/205: unlink d4/df/d1f/d24/l3c 0 2024-05-15T15:54:05.360 INFO:tasks.workunit.client.1.smithi144.stdout:2/235: mknod d3/d7/d20/d37/d3a/c4a 0 2024-05-15T15:54:05.361 INFO:tasks.workunit.client.1.smithi144.stdout:8/232: fsync d2/f3 0 2024-05-15T15:54:05.362 INFO:tasks.workunit.client.0.smithi049.stdout:2/313: dread d3/db/f23 [0,4194304] 0 2024-05-15T15:54:05.363 INFO:tasks.workunit.client.1.smithi144.stdout:8/233: truncate d2/de/d3f/f4e 28175 0 2024-05-15T15:54:05.364 INFO:tasks.workunit.client.1.smithi144.stdout:2/236: dread d3/d10/f1d [0,4194304] 0 2024-05-15T15:54:05.365 INFO:tasks.workunit.client.0.smithi049.stdout:7/288: dread d0/d6/d1d/f4e [0,4194304] 0 2024-05-15T15:54:05.365 INFO:tasks.workunit.client.1.smithi144.stdout:8/234: write d2/de/d3f/f4c [768818,113025] 0 2024-05-15T15:54:05.365 INFO:tasks.workunit.client.1.smithi144.stdout:6/189: rmdir d7/db/d11/d45 0 2024-05-15T15:54:05.366 INFO:tasks.workunit.client.1.smithi144.stdout:6/190: readlink d7/d1e/l2d 0 2024-05-15T15:54:05.366 INFO:tasks.workunit.client.1.smithi144.stdout:4/234: dread d6/dd/d34/d3b/f21 [0,4194304] 0 2024-05-15T15:54:05.366 INFO:tasks.workunit.client.0.smithi049.stdout:2/314: creat d3/da/d15/f5c x:0 0 0 2024-05-15T15:54:05.367 INFO:tasks.workunit.client.1.smithi144.stdout:4/235: chown d6/dd/d34/d3b/d30/db/f33 0 1 2024-05-15T15:54:05.367 INFO:tasks.workunit.client.0.smithi049.stdout:2/315: write d3/da/d15/f39 [406476,36032] 0 2024-05-15T15:54:05.368 INFO:tasks.workunit.client.1.smithi144.stdout:4/236: write d6/dd/d34/d3b/d30/db/d1d/f41 [523321,76326] 0 2024-05-15T15:54:05.369 INFO:tasks.workunit.client.1.smithi144.stdout:0/206: creat d4/df/d1f/d24/f4b x:0 0 0 2024-05-15T15:54:05.370 INFO:tasks.workunit.client.1.smithi144.stdout:3/229: mknod d0/d1/c51 0 2024-05-15T15:54:05.371 INFO:tasks.workunit.client.1.smithi144.stdout:9/223: link d0/d8/d50/c55 d0/d8/d1b/c56 0 2024-05-15T15:54:05.374 INFO:tasks.workunit.client.0.smithi049.stdout:2/316: dwrite d3/db/d1d/f20 [0,4194304] 0 2024-05-15T15:54:05.375 INFO:tasks.workunit.client.0.smithi049.stdout:2/317: write d3/db/f43 [4495668,35722] 0 2024-05-15T15:54:05.383 INFO:tasks.workunit.client.0.smithi049.stdout:7/289: write d0/d6/f29 [725803,43154] 0 2024-05-15T15:54:05.384 INFO:tasks.workunit.client.0.smithi049.stdout:7/290: chown d0/d6/f14 80112507 1 2024-05-15T15:54:05.386 INFO:tasks.workunit.client.0.smithi049.stdout:2/318: dread d3/da/f45 [0,4194304] 0 2024-05-15T15:54:05.389 INFO:tasks.workunit.client.0.smithi049.stdout:7/291: dread d0/d6/f14 [4194304,4194304] 0 2024-05-15T15:54:05.392 INFO:tasks.workunit.client.0.smithi049.stdout:2/319: rename d3/f4c to d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d 0 2024-05-15T15:54:05.393 INFO:tasks.workunit.client.1.smithi144.stdout:5/224: sync 2024-05-15T15:54:05.394 INFO:tasks.workunit.client.0.smithi049.stdout:7/292: write d0/f15 [2954839,108092] 0 2024-05-15T15:54:05.399 INFO:tasks.workunit.client.0.smithi049.stdout:7/293: symlink d0/d6/da/d3d/d39/l53 0 2024-05-15T15:54:05.400 INFO:tasks.workunit.client.0.smithi049.stdout:2/320: truncate d3/da/d15/f27 4389213 0 2024-05-15T15:54:05.402 INFO:tasks.workunit.client.0.smithi049.stdout:2/321: unlink d3/da/f46 0 2024-05-15T15:54:05.404 INFO:tasks.workunit.client.1.smithi144.stdout:1/231: getdents d7/d1d/d33/d1e 0 2024-05-15T15:54:05.404 INFO:tasks.workunit.client.0.smithi049.stdout:2/322: creat d3/db/d1d/f5e x:0 0 0 2024-05-15T15:54:05.405 INFO:tasks.workunit.client.1.smithi144.stdout:0/207: mkdir d4/d6/d3a/d4c 0 2024-05-15T15:54:05.406 INFO:tasks.workunit.client.1.smithi144.stdout:8/235: write d2/f16 [1098174,48828] 0 2024-05-15T15:54:05.406 INFO:tasks.workunit.client.1.smithi144.stdout:4/237: write f0 [1709789,63297] 0 2024-05-15T15:54:05.407 INFO:tasks.workunit.client.1.smithi144.stdout:5/225: creat d0/dc/d12/f46 x:0 0 0 2024-05-15T15:54:05.408 INFO:tasks.workunit.client.1.smithi144.stdout:1/232: write d7/d1d/d33/d29/f42 [541116,112385] 0 2024-05-15T15:54:05.410 INFO:tasks.workunit.client.1.smithi144.stdout:3/230: dwrite d0/f46 [0,4194304] 0 2024-05-15T15:54:05.410 INFO:tasks.workunit.client.1.smithi144.stdout:9/224: mknod d0/d8/d21/d29/d2d/d44/d10/d19/c57 0 2024-05-15T15:54:05.411 INFO:tasks.workunit.client.1.smithi144.stdout:5/226: fdatasync d0/dc/d12/d31/d35/f44 0 2024-05-15T15:54:05.417 INFO:tasks.workunit.client.1.smithi144.stdout:8/236: symlink d2/d24/d2d/l51 0 2024-05-15T15:54:05.418 INFO:tasks.workunit.client.1.smithi144.stdout:1/233: dread d7/df/f10 [0,4194304] 0 2024-05-15T15:54:05.421 INFO:tasks.workunit.client.1.smithi144.stdout:9/225: creat d0/d8/d21/d29/d2d/d44/d10/f58 x:0 0 0 2024-05-15T15:54:05.421 INFO:tasks.workunit.client.1.smithi144.stdout:5/227: dwrite d0/dc/d12/f13 [0,4194304] 0 2024-05-15T15:54:05.422 INFO:tasks.workunit.client.1.smithi144.stdout:6/191: getdents d7/db 0 2024-05-15T15:54:05.423 INFO:tasks.workunit.client.1.smithi144.stdout:0/208: mkdir d4/d6/d4a/d4d 0 2024-05-15T15:54:05.425 INFO:tasks.workunit.client.1.smithi144.stdout:1/234: chown l2 203 1 2024-05-15T15:54:05.428 INFO:tasks.workunit.client.1.smithi144.stdout:5/228: symlink d0/dc/d2f/d34/l47 0 2024-05-15T15:54:05.428 INFO:tasks.workunit.client.1.smithi144.stdout:0/209: truncate d4/df/f43 250984 0 2024-05-15T15:54:05.428 INFO:tasks.workunit.client.1.smithi144.stdout:6/192: symlink d7/d1e/l46 0 2024-05-15T15:54:05.430 INFO:tasks.workunit.client.1.smithi144.stdout:9/226: creat d0/f59 x:0 0 0 2024-05-15T15:54:05.431 INFO:tasks.workunit.client.0.smithi049.stdout:2/323: truncate d3/db/d1d/d1e/d2b/d2e/f48 304573 0 2024-05-15T15:54:05.431 INFO:tasks.workunit.client.1.smithi144.stdout:1/235: symlink d7/d1d/d33/l49 0 2024-05-15T15:54:05.431 INFO:tasks.workunit.client.1.smithi144.stdout:9/227: fdatasync d0/f59 0 2024-05-15T15:54:05.432 INFO:tasks.workunit.client.1.smithi144.stdout:5/229: unlink d0/f7 0 2024-05-15T15:54:05.434 INFO:tasks.workunit.client.0.smithi049.stdout:2/324: unlink d3/da/c51 0 2024-05-15T15:54:05.436 INFO:tasks.workunit.client.1.smithi144.stdout:0/210: dread d4/f5 [0,4194304] 0 2024-05-15T15:54:05.436 INFO:tasks.workunit.client.1.smithi144.stdout:1/236: fsync f1 0 2024-05-15T15:54:05.438 INFO:tasks.workunit.client.0.smithi049.stdout:2/325: creat d3/db/d1d/d1e/d2b/d3b/f5f x:0 0 0 2024-05-15T15:54:05.438 INFO:tasks.workunit.client.1.smithi144.stdout:8/237: dread d2/f16 [0,4194304] 0 2024-05-15T15:54:05.438 INFO:tasks.workunit.client.1.smithi144.stdout:9/228: readlink d0/l6 0 2024-05-15T15:54:05.439 INFO:tasks.workunit.client.1.smithi144.stdout:5/230: truncate d0/dc/d24/d28/d2d/f3d 325358 0 2024-05-15T15:54:05.439 INFO:tasks.workunit.client.1.smithi144.stdout:6/193: dwrite d7/db/d11/f1f [0,4194304] 0 2024-05-15T15:54:05.439 INFO:tasks.workunit.client.1.smithi144.stdout:4/238: truncate d6/dd/d2d/f3f 4131455 0 2024-05-15T15:54:05.440 INFO:tasks.workunit.client.1.smithi144.stdout:5/231: chown d0/f16 5 1 2024-05-15T15:54:05.440 INFO:tasks.workunit.client.0.smithi049.stdout:2/326: mknod d3/db/d1d/d1e/d2b/d3b/d47/c60 0 2024-05-15T15:54:05.441 INFO:tasks.workunit.client.0.smithi049.stdout:2/327: stat d3/f11 0 2024-05-15T15:54:05.441 INFO:tasks.workunit.client.0.smithi049.stdout:2/328: chown d3/db/d1d/d1e/d2b/d3b/d47 19 1 2024-05-15T15:54:05.442 INFO:tasks.workunit.client.1.smithi144.stdout:3/231: dread d0/d11/d24/f41 [0,4194304] 0 2024-05-15T15:54:05.442 INFO:tasks.workunit.client.1.smithi144.stdout:5/232: dread - d0/dc/d12/f46 zero size 2024-05-15T15:54:05.443 INFO:tasks.workunit.client.1.smithi144.stdout:0/211: truncate d4/dc/f18 1943025 0 2024-05-15T15:54:05.444 INFO:tasks.workunit.client.1.smithi144.stdout:1/237: creat d7/d1d/d33/d1e/d34/f4a x:0 0 0 2024-05-15T15:54:05.448 INFO:tasks.workunit.client.1.smithi144.stdout:4/239: fsync d6/dd/d15/f31 0 2024-05-15T15:54:05.450 INFO:tasks.workunit.client.1.smithi144.stdout:9/229: mkdir d0/d8/d21/d29/d2d/d44/d16/d5a 0 2024-05-15T15:54:05.450 INFO:tasks.workunit.client.0.smithi049.stdout:2/329: dread d3/da/d15/f26 [0,4194304] 0 2024-05-15T15:54:05.452 INFO:tasks.workunit.client.1.smithi144.stdout:5/233: dread d0/dc/fe [0,4194304] 0 2024-05-15T15:54:05.455 INFO:tasks.workunit.client.1.smithi144.stdout:8/238: dread d2/de/f26 [0,4194304] 0 2024-05-15T15:54:05.457 INFO:tasks.workunit.client.1.smithi144.stdout:0/212: dread d4/df/f16 [0,4194304] 0 2024-05-15T15:54:05.457 INFO:tasks.workunit.client.1.smithi144.stdout:1/238: dwrite d7/d1d/f32 [0,4194304] 0 2024-05-15T15:54:05.457 INFO:tasks.workunit.client.1.smithi144.stdout:9/230: chown d0/d8/d21/d29/d2d/d44/d10/d19/f33 9865 1 2024-05-15T15:54:05.459 INFO:tasks.workunit.client.1.smithi144.stdout:4/240: mknod d6/dd/d15/d1e/c4a 0 2024-05-15T15:54:05.460 INFO:tasks.workunit.client.1.smithi144.stdout:0/213: write d4/d6/f32 [960474,122630] 0 2024-05-15T15:54:05.461 INFO:tasks.workunit.client.1.smithi144.stdout:3/232: mknod d0/d1/d17/d39/d4d/c52 0 2024-05-15T15:54:05.465 INFO:tasks.workunit.client.1.smithi144.stdout:6/194: dread d7/f8 [0,4194304] 0 2024-05-15T15:54:05.466 INFO:tasks.workunit.client.0.smithi049.stdout:2/330: getdents d3/d8 0 2024-05-15T15:54:05.467 INFO:tasks.workunit.client.1.smithi144.stdout:4/241: read d6/dd/d2d/f48 [8252331,128388] 0 2024-05-15T15:54:05.468 INFO:tasks.workunit.client.0.smithi049.stdout:2/331: write d3/db/f4a [183583,6950] 0 2024-05-15T15:54:05.470 INFO:tasks.workunit.client.1.smithi144.stdout:9/231: mkdir d0/d8/d21/d29/d2d/d44/d16/d5b 0 2024-05-15T15:54:05.471 INFO:tasks.workunit.client.0.smithi049.stdout:2/332: write d3/d8/fe [2354519,61945] 0 2024-05-15T15:54:05.472 INFO:tasks.workunit.client.0.smithi049.stdout:2/333: fdatasync d3/db/f43 0 2024-05-15T15:54:05.472 INFO:tasks.workunit.client.1.smithi144.stdout:5/234: creat d0/dc/d2f/d34/d3e/f48 x:0 0 0 2024-05-15T15:54:05.472 INFO:tasks.workunit.client.1.smithi144.stdout:5/235: readlink d0/dc/d1f/d29/l30 0 2024-05-15T15:54:05.474 INFO:tasks.workunit.client.1.smithi144.stdout:6/195: rename d7/db/f37 to d7/db/d35/f47 0 2024-05-15T15:54:05.474 INFO:tasks.workunit.client.1.smithi144.stdout:1/239: mkdir d7/d4b 0 2024-05-15T15:54:05.477 INFO:tasks.workunit.client.0.smithi049.stdout:2/334: rename d3/db/f4a to d3/db/d1d/d1e/d2b/d3b/d42/f61 0 2024-05-15T15:54:05.484 INFO:tasks.workunit.client.1.smithi144.stdout:4/242: rename d6/dd/d15 to d6/d42/d46/d4b 0 2024-05-15T15:54:05.484 INFO:tasks.workunit.client.0.smithi049.stdout:2/335: getdents d3/db/d1d/d1e/d2b/d3b 0 2024-05-15T15:54:05.497 INFO:tasks.workunit.client.0.smithi049.stdout:2/336: dread d3/da/f2c [0,4194304] 0 2024-05-15T15:54:05.498 INFO:tasks.workunit.client.0.smithi049.stdout:2/337: chown d3/db/d1d/d1e/d2b/d3b/d42/f4f 147309 1 2024-05-15T15:54:05.502 INFO:tasks.workunit.client.0.smithi049.stdout:2/338: dwrite d3/f10 [0,4194304] 0 2024-05-15T15:54:05.520 INFO:tasks.workunit.client.1.smithi144.stdout:7/237: sync 2024-05-15T15:54:05.527 INFO:tasks.workunit.client.0.smithi049.stdout:2/339: rename d3/db/d1d/d1e/d2b/d3b/d42/f61 to d3/da/d15/d24/f62 0 2024-05-15T15:54:05.530 INFO:tasks.workunit.client.1.smithi144.stdout:9/232: rename d0/d8/d21/d29/d2d/d44/d3c/l4f to d0/d8/l5c 0 2024-05-15T15:54:05.532 INFO:tasks.workunit.client.0.smithi049.stdout:2/340: mkdir d3/db/d5b/d63 0 2024-05-15T15:54:05.533 INFO:tasks.workunit.client.1.smithi144.stdout:1/240: mknod d7/d4b/c4c 0 2024-05-15T15:54:05.533 INFO:tasks.workunit.client.1.smithi144.stdout:7/238: rmdir d4/df 39 2024-05-15T15:54:05.534 INFO:tasks.workunit.client.1.smithi144.stdout:8/239: link d2/de/d36/c19 d2/d3a/c52 0 2024-05-15T15:54:05.535 INFO:tasks.workunit.client.0.smithi049.stdout:2/341: creat d3/da/d15/d24/f64 x:0 0 0 2024-05-15T15:54:05.535 INFO:tasks.workunit.client.1.smithi144.stdout:3/233: getdents d0/d11/d3b 0 2024-05-15T15:54:05.536 INFO:tasks.workunit.client.1.smithi144.stdout:9/233: dwrite d0/d8/f51 [0,4194304] 0 2024-05-15T15:54:05.537 INFO:tasks.workunit.client.0.smithi049.stdout:2/342: rmdir d3/db/d1d/d1e 39 2024-05-15T15:54:05.538 INFO:tasks.workunit.client.0.smithi049.stdout:2/343: write d3/da/d15/d24/f64 [181751,101191] 0 2024-05-15T15:54:05.544 INFO:tasks.workunit.client.1.smithi144.stdout:5/236: getdents d0/dc 0 2024-05-15T15:54:05.545 INFO:tasks.workunit.client.1.smithi144.stdout:5/237: read - d0/dc/d1f/f39 zero size 2024-05-15T15:54:05.545 INFO:tasks.workunit.client.0.smithi049.stdout:2/344: creat d3/db/d1d/d1e/d2b/d3b/d42/f65 x:0 0 0 2024-05-15T15:54:05.546 INFO:tasks.workunit.client.0.smithi049.stdout:2/345: stat d3/db/d1d/d1e/d2b/d3b/d42 0 2024-05-15T15:54:05.550 INFO:tasks.workunit.client.1.smithi144.stdout:6/196: creat d7/db/f48 x:0 0 0 2024-05-15T15:54:05.550 INFO:tasks.workunit.client.1.smithi144.stdout:9/234: read d0/d8/d21/d29/d2d/d44/d10/f17 [2946987,99666] 0 2024-05-15T15:54:05.551 INFO:tasks.workunit.client.1.smithi144.stdout:8/240: mkdir d2/d53 0 2024-05-15T15:54:05.552 INFO:tasks.workunit.client.1.smithi144.stdout:5/238: rmdir d0/dc/d12/d31 39 2024-05-15T15:54:05.553 INFO:tasks.workunit.client.0.smithi049.stdout:2/346: write d3/db/f12 [1218388,18447] 0 2024-05-15T15:54:05.554 INFO:tasks.workunit.client.1.smithi144.stdout:4/243: getdents d6/d42/d46 0 2024-05-15T15:54:05.554 INFO:tasks.workunit.client.1.smithi144.stdout:7/239: dread d4/df/d25/f35 [0,4194304] 0 2024-05-15T15:54:05.555 INFO:tasks.workunit.client.1.smithi144.stdout:1/241: rename d7/d1d/d33/d41/d22/f28 to d7/f4d 0 2024-05-15T15:54:05.556 INFO:tasks.workunit.client.1.smithi144.stdout:9/235: mknod d0/d8/d21/d29/d2d/d44/d10/d19/c5d 0 2024-05-15T15:54:05.557 INFO:tasks.workunit.client.0.smithi049.stdout:2/347: creat d3/db/d1d/d1e/d2b/d3b/d47/d59/f66 x:0 0 0 2024-05-15T15:54:05.558 INFO:tasks.workunit.client.1.smithi144.stdout:3/234: getdents d0/d1/d17 0 2024-05-15T15:54:05.558 INFO:tasks.workunit.client.1.smithi144.stdout:6/197: symlink d7/db/d11/d3c/l49 0 2024-05-15T15:54:05.559 INFO:tasks.workunit.client.1.smithi144.stdout:8/241: dwrite d2/de/d36/f47 [0,4194304] 0 2024-05-15T15:54:05.559 INFO:tasks.workunit.client.1.smithi144.stdout:4/244: creat d6/dd/d34/d3b/f4c x:0 0 0 2024-05-15T15:54:05.560 INFO:tasks.workunit.client.0.smithi049.stdout:2/348: mkdir d3/db/d1d/d1e/d2b/d2e/d67 0 2024-05-15T15:54:05.560 INFO:tasks.workunit.client.1.smithi144.stdout:6/198: dread - d7/f41 zero size 2024-05-15T15:54:05.561 INFO:tasks.workunit.client.1.smithi144.stdout:4/245: chown d6/dd/d34 41 1 2024-05-15T15:54:05.563 INFO:tasks.workunit.client.1.smithi144.stdout:8/242: write d2/de/f33 [72168,39987] 0 2024-05-15T15:54:05.563 INFO:tasks.workunit.client.1.smithi144.stdout:5/239: link d0/f16 d0/f49 0 2024-05-15T15:54:05.564 INFO:tasks.workunit.client.1.smithi144.stdout:1/242: chown d7/d1d/d33/d41/c2a 16034 1 2024-05-15T15:54:05.568 INFO:tasks.workunit.client.1.smithi144.stdout:9/236: dwrite d0/d8/f11 [0,4194304] 0 2024-05-15T15:54:05.568 INFO:tasks.workunit.client.1.smithi144.stdout:6/199: dread d7/f8 [0,4194304] 0 2024-05-15T15:54:05.569 INFO:tasks.workunit.client.1.smithi144.stdout:4/246: creat d6/d42/d46/d4b/f4d x:0 0 0 2024-05-15T15:54:05.570 INFO:tasks.workunit.client.1.smithi144.stdout:9/237: chown d0/d8/d21/d29/d2d/d44/d10/d19/d3e 267 1 2024-05-15T15:54:05.570 INFO:tasks.workunit.client.1.smithi144.stdout:2/237: sync 2024-05-15T15:54:05.570 INFO:tasks.workunit.client.1.smithi144.stdout:0/214: sync 2024-05-15T15:54:05.571 INFO:tasks.workunit.client.1.smithi144.stdout:7/240: link d4/df/d25/f29 d4/df/d25/d40/d47/f4f 0 2024-05-15T15:54:05.572 INFO:tasks.workunit.client.1.smithi144.stdout:9/238: chown d0/d8/d21/d29/d2d/d44/d10/f2c 8693986 1 2024-05-15T15:54:05.577 INFO:tasks.workunit.client.1.smithi144.stdout:1/243: mkdir d7/d1d/d33/d1e/d4e 0 2024-05-15T15:54:05.577 INFO:tasks.workunit.client.1.smithi144.stdout:1/244: write f1 [9949890,56726] 0 2024-05-15T15:54:05.578 INFO:tasks.workunit.client.1.smithi144.stdout:8/243: rename d2/d1a/f39 to d2/d24/d2d/f54 0 2024-05-15T15:54:05.581 INFO:tasks.workunit.client.1.smithi144.stdout:4/247: mkdir d6/dd/d34/d3b/d30/db/d1d/d4e 0 2024-05-15T15:54:05.583 INFO:tasks.workunit.client.1.smithi144.stdout:5/240: mkdir d0/dc/d12/d4a 0 2024-05-15T15:54:05.585 INFO:tasks.workunit.client.1.smithi144.stdout:3/235: truncate d0/d11/d24/f3e 3967685 0 2024-05-15T15:54:05.587 INFO:tasks.workunit.client.1.smithi144.stdout:7/241: rename d4/df/l2e to d4/df/d25/d40/l50 0 2024-05-15T15:54:05.591 INFO:tasks.workunit.client.1.smithi144.stdout:0/215: mknod d4/d6/d3a/d40/c4e 0 2024-05-15T15:54:05.592 INFO:tasks.workunit.client.1.smithi144.stdout:0/216: dread - d4/df/d1f/d24/f47 zero size 2024-05-15T15:54:05.594 INFO:tasks.workunit.client.1.smithi144.stdout:5/241: mknod d0/dc/d24/d28/d2d/c4b 0 2024-05-15T15:54:05.595 INFO:tasks.workunit.client.1.smithi144.stdout:5/242: chown d0/dc/d24/d28/d2d/f3d 5596 1 2024-05-15T15:54:05.598 INFO:tasks.workunit.client.1.smithi144.stdout:1/245: dwrite d7/d1d/d33/d29/f2c [0,4194304] 0 2024-05-15T15:54:05.598 INFO:tasks.workunit.client.1.smithi144.stdout:9/239: rename d0/d8/d21/d29/d2d to d0/d8/d5e 0 2024-05-15T15:54:05.599 INFO:tasks.workunit.client.1.smithi144.stdout:9/240: chown d0/d8/d1b 34 1 2024-05-15T15:54:05.603 INFO:tasks.workunit.client.1.smithi144.stdout:6/200: creat d7/db/f4a x:0 0 0 2024-05-15T15:54:05.603 INFO:tasks.workunit.client.1.smithi144.stdout:2/238: link d3/d7/l21 d3/d7/l4b 0 2024-05-15T15:54:05.607 INFO:tasks.workunit.client.1.smithi144.stdout:0/217: dread d4/df/d1f/d24/f2a [0,4194304] 0 2024-05-15T15:54:05.611 INFO:tasks.workunit.client.1.smithi144.stdout:4/248: rename d6/f32 to d6/d42/d46/d4b/d1e/f4f 0 2024-05-15T15:54:05.613 INFO:tasks.workunit.client.1.smithi144.stdout:4/249: dread - d6/d42/d46/d4b/f4d zero size 2024-05-15T15:54:05.614 INFO:tasks.workunit.client.1.smithi144.stdout:2/239: chown d3/l6 198715 1 2024-05-15T15:54:05.615 INFO:tasks.workunit.client.1.smithi144.stdout:7/242: creat d4/f51 x:0 0 0 2024-05-15T15:54:05.615 INFO:tasks.workunit.client.1.smithi144.stdout:1/246: write d7/d1d/d33/d41/fe [3655648,61788] 0 2024-05-15T15:54:05.616 INFO:tasks.workunit.client.1.smithi144.stdout:1/247: chown d7/d1d/d33/d1e/f25 5 1 2024-05-15T15:54:05.618 INFO:tasks.workunit.client.1.smithi144.stdout:5/243: creat d0/dc/d12/d31/f4c x:0 0 0 2024-05-15T15:54:05.621 INFO:tasks.workunit.client.1.smithi144.stdout:8/244: getdents d2 0 2024-05-15T15:54:05.621 INFO:tasks.workunit.client.1.smithi144.stdout:8/245: write d2/d24/f25 [174573,29389] 0 2024-05-15T15:54:05.622 INFO:tasks.workunit.client.1.smithi144.stdout:8/246: write d2/de/d36/f46 [849806,79404] 0 2024-05-15T15:54:05.623 INFO:tasks.workunit.client.1.smithi144.stdout:9/241: rename d0/d8/d5e/d44/d10/d19/c2e to d0/d8/d21/d29/c5f 0 2024-05-15T15:54:05.624 INFO:tasks.workunit.client.1.smithi144.stdout:3/236: getdents d0/d1/d17/d39/d4d 0 2024-05-15T15:54:05.631 INFO:tasks.workunit.client.1.smithi144.stdout:2/240: mknod d3/d10/c4c 0 2024-05-15T15:54:05.631 INFO:tasks.workunit.client.1.smithi144.stdout:0/218: symlink d4/d6/d3a/d4c/l4f 0 2024-05-15T15:54:05.631 INFO:tasks.workunit.client.1.smithi144.stdout:7/243: creat d4/d41/f52 x:0 0 0 2024-05-15T15:54:05.632 INFO:tasks.workunit.client.1.smithi144.stdout:5/244: fsync d0/dc/fe 0 2024-05-15T15:54:05.632 INFO:tasks.workunit.client.1.smithi144.stdout:6/201: creat d7/f4b x:0 0 0 2024-05-15T15:54:05.632 INFO:tasks.workunit.client.1.smithi144.stdout:8/247: symlink d2/d24/d2d/d30/l55 0 2024-05-15T15:54:05.633 INFO:tasks.workunit.client.1.smithi144.stdout:4/250: symlink d6/dd/d34/d3b/d30/db/d1d/d4e/l50 0 2024-05-15T15:54:05.634 INFO:tasks.workunit.client.1.smithi144.stdout:3/237: creat d0/d1/d17/f53 x:0 0 0 2024-05-15T15:54:05.635 INFO:tasks.workunit.client.1.smithi144.stdout:7/244: rmdir d4/df 39 2024-05-15T15:54:05.636 INFO:tasks.workunit.client.1.smithi144.stdout:9/242: dread d0/d8/d5e/d44/d10/f27 [0,4194304] 0 2024-05-15T15:54:05.636 INFO:tasks.workunit.client.1.smithi144.stdout:9/243: chown d0/d8/d21 1340919 1 2024-05-15T15:54:05.637 INFO:tasks.workunit.client.1.smithi144.stdout:6/202: truncate d7/db/f1d 4584248 0 2024-05-15T15:54:05.638 INFO:tasks.workunit.client.1.smithi144.stdout:5/245: truncate d0/dc/f1b 4524147 0 2024-05-15T15:54:05.639 INFO:tasks.workunit.client.1.smithi144.stdout:5/246: write d0/dc/d24/d28/d2d/f3d [934618,106985] 0 2024-05-15T15:54:05.640 INFO:tasks.workunit.client.1.smithi144.stdout:4/251: creat d6/d42/d46/d4b/d1e/f51 x:0 0 0 2024-05-15T15:54:05.641 INFO:tasks.workunit.client.1.smithi144.stdout:8/248: truncate d2/de/d36/f17 85212 0 2024-05-15T15:54:05.642 INFO:tasks.workunit.client.1.smithi144.stdout:7/245: write d4/df/d25/d38/f3f [615746,17190] 0 2024-05-15T15:54:05.643 INFO:tasks.workunit.client.1.smithi144.stdout:0/219: mknod d4/d2d/d48/c50 0 2024-05-15T15:54:05.643 INFO:tasks.workunit.client.1.smithi144.stdout:0/220: chown c3 5 1 2024-05-15T15:54:05.646 INFO:tasks.workunit.client.1.smithi144.stdout:2/241: truncate d3/d7/f1a 585145 0 2024-05-15T15:54:05.650 INFO:tasks.workunit.client.1.smithi144.stdout:0/221: dwrite d4/df/d1f/f45 [0,4194304] 0 2024-05-15T15:54:05.650 INFO:tasks.workunit.client.1.smithi144.stdout:5/247: mknod d0/dc/c4d 0 2024-05-15T15:54:05.651 INFO:tasks.workunit.client.1.smithi144.stdout:0/222: dread - d4/df/d1f/d24/d2b/f3b zero size 2024-05-15T15:54:05.651 INFO:tasks.workunit.client.1.smithi144.stdout:4/252: creat d6/dd/d34/d3b/d30/f52 x:0 0 0 2024-05-15T15:54:05.658 INFO:tasks.workunit.client.1.smithi144.stdout:7/246: dread d4/f13 [8388608,4194304] 0 2024-05-15T15:54:05.659 INFO:tasks.workunit.client.1.smithi144.stdout:6/203: mknod d7/db/d35/c4c 0 2024-05-15T15:54:05.659 INFO:tasks.workunit.client.1.smithi144.stdout:7/247: fsync d4/df/f20 0 2024-05-15T15:54:05.660 INFO:tasks.workunit.client.1.smithi144.stdout:4/253: dwrite d6/d42/d46/d4b/f4d [0,4194304] 0 2024-05-15T15:54:05.661 INFO:tasks.workunit.client.1.smithi144.stdout:7/248: chown d4/df/d25/f3a 6683501 1 2024-05-15T15:54:05.662 INFO:tasks.workunit.client.1.smithi144.stdout:7/249: stat d4/df/f20 0 2024-05-15T15:54:05.664 INFO:tasks.workunit.client.1.smithi144.stdout:8/249: creat d2/d53/f56 x:0 0 0 2024-05-15T15:54:05.664 INFO:tasks.workunit.client.1.smithi144.stdout:3/238: link d0/d1/c28 d0/d11/d3b/c54 0 2024-05-15T15:54:05.667 INFO:tasks.workunit.client.1.smithi144.stdout:2/242: mkdir d3/d4d 0 2024-05-15T15:54:05.667 INFO:tasks.workunit.client.1.smithi144.stdout:2/243: readlink d3/l2a 0 2024-05-15T15:54:05.670 INFO:tasks.workunit.client.1.smithi144.stdout:5/248: truncate d0/f16 3090485 0 2024-05-15T15:54:05.670 INFO:tasks.workunit.client.0.smithi049.stdout:4/394: sync 2024-05-15T15:54:05.672 INFO:tasks.workunit.client.1.smithi144.stdout:6/204: creat d7/db/d11/d3c/f4d x:0 0 0 2024-05-15T15:54:05.676 INFO:tasks.workunit.client.0.smithi049.stdout:4/395: creat d18/d1e/d2a/d36/f83 x:0 0 0 2024-05-15T15:54:05.680 INFO:tasks.workunit.client.1.smithi144.stdout:9/244: dwrite d0/d8/d5e/d44/d10/f27 [0,4194304] 0 2024-05-15T15:54:05.680 INFO:tasks.workunit.client.0.smithi049.stdout:4/396: symlink d18/d1e/d2a/d36/l84 0 2024-05-15T15:54:05.681 INFO:tasks.workunit.client.0.smithi049.stdout:4/397: chown d18/f1b 202 1 2024-05-15T15:54:05.685 INFO:tasks.workunit.client.1.smithi144.stdout:3/239: dread d0/f2e [4194304,4194304] 0 2024-05-15T15:54:05.688 INFO:tasks.workunit.client.1.smithi144.stdout:9/245: dwrite d0/f59 [0,4194304] 0 2024-05-15T15:54:05.692 INFO:tasks.workunit.client.1.smithi144.stdout:0/223: dwrite d4/d6/f11 [0,4194304] 0 2024-05-15T15:54:05.695 INFO:tasks.workunit.client.1.smithi144.stdout:9/246: read d0/d8/d5e/f3a [1682453,124329] 0 2024-05-15T15:54:05.696 INFO:tasks.workunit.client.1.smithi144.stdout:2/244: fsync d3/d7/d20/d22/f24 0 2024-05-15T15:54:05.696 INFO:tasks.workunit.client.1.smithi144.stdout:7/250: write d4/f46 [109574,74767] 0 2024-05-15T15:54:05.700 INFO:tasks.workunit.client.1.smithi144.stdout:6/205: symlink d7/db/l4e 0 2024-05-15T15:54:05.700 INFO:tasks.workunit.client.1.smithi144.stdout:8/250: write d2/f15 [81879,14435] 0 2024-05-15T15:54:05.701 INFO:tasks.workunit.client.1.smithi144.stdout:2/245: fdatasync d3/d7/f18 0 2024-05-15T15:54:05.702 INFO:tasks.workunit.client.0.smithi049.stdout:4/398: write d18/d1e/f23 [482461,75559] 0 2024-05-15T15:54:05.703 INFO:tasks.workunit.client.1.smithi144.stdout:2/246: stat d3/d7/d20/l48 0 2024-05-15T15:54:05.704 INFO:tasks.workunit.client.1.smithi144.stdout:4/254: truncate d6/dd/d34/d3b/d30/f1b 521984 0 2024-05-15T15:54:05.705 INFO:tasks.workunit.client.1.smithi144.stdout:6/206: truncate d7/db/f48 1029373 0 2024-05-15T15:54:05.706 INFO:tasks.workunit.client.1.smithi144.stdout:9/247: truncate d0/d8/d5e/d44/f26 1705251 0 2024-05-15T15:54:05.706 INFO:tasks.workunit.client.1.smithi144.stdout:0/224: dwrite f2 [0,4194304] 0 2024-05-15T15:54:05.707 INFO:tasks.workunit.client.1.smithi144.stdout:4/255: chown d6/d42/d46/d4b/l1a 70762 1 2024-05-15T15:54:05.708 INFO:tasks.workunit.client.1.smithi144.stdout:9/248: write d0/d8/f51 [4721831,4700] 0 2024-05-15T15:54:05.709 INFO:tasks.workunit.client.0.smithi049.stdout:4/399: mkdir d18/d85 0 2024-05-15T15:54:05.713 INFO:tasks.workunit.client.1.smithi144.stdout:1/248: sync 2024-05-15T15:54:05.713 INFO:tasks.workunit.client.1.smithi144.stdout:9/249: dread d0/d8/d21/f38 [0,4194304] 0 2024-05-15T15:54:05.715 INFO:tasks.workunit.client.1.smithi144.stdout:1/249: write d7/d1d/d33/d41/f19 [1671259,82510] 0 2024-05-15T15:54:05.718 INFO:tasks.workunit.client.1.smithi144.stdout:0/225: symlink d4/d2d/l51 0 2024-05-15T15:54:05.720 INFO:tasks.workunit.client.1.smithi144.stdout:4/256: creat d6/d42/d46/d4b/d1e/d44/f53 x:0 0 0 2024-05-15T15:54:05.720 INFO:tasks.workunit.client.1.smithi144.stdout:0/226: fdatasync d4/d6/fb 0 2024-05-15T15:54:05.723 INFO:tasks.workunit.client.1.smithi144.stdout:9/250: dwrite d0/f4 [0,4194304] 0 2024-05-15T15:54:05.723 INFO:tasks.workunit.client.1.smithi144.stdout:4/257: chown d6/dd/d2e/c37 304068766 1 2024-05-15T15:54:05.724 INFO:tasks.workunit.client.0.smithi049.stdout:4/400: dread f16 [0,4194304] 0 2024-05-15T15:54:05.725 INFO:tasks.workunit.client.1.smithi144.stdout:6/207: dwrite d7/db/f39 [0,4194304] 0 2024-05-15T15:54:05.730 INFO:tasks.workunit.client.0.smithi049.stdout:4/401: mkdir d18/d1e/d2a/d36/d86 0 2024-05-15T15:54:05.730 INFO:tasks.workunit.client.1.smithi144.stdout:9/251: dread - d0/d8/d5e/d44/d16/f46 zero size 2024-05-15T15:54:05.733 INFO:tasks.workunit.client.1.smithi144.stdout:3/240: truncate d0/f2c 2462372 0 2024-05-15T15:54:05.737 INFO:tasks.workunit.client.0.smithi049.stdout:4/402: creat d18/d21/d2b/d4e/f87 x:0 0 0 2024-05-15T15:54:05.741 INFO:tasks.workunit.client.1.smithi144.stdout:7/251: truncate d4/df/f1f 2674844 0 2024-05-15T15:54:05.742 INFO:tasks.workunit.client.1.smithi144.stdout:5/249: link d0/dc/d1f/c26 d0/dc/d24/d28/c4e 0 2024-05-15T15:54:05.744 INFO:tasks.workunit.client.1.smithi144.stdout:8/251: getdents d2/d24/d2d 0 2024-05-15T15:54:05.744 INFO:tasks.workunit.client.1.smithi144.stdout:8/252: dread - d2/d24/d2d/d30/f44 zero size 2024-05-15T15:54:05.747 INFO:tasks.workunit.client.1.smithi144.stdout:9/252: mknod d0/d8/d5e/d44/d16/d5b/c60 0 2024-05-15T15:54:05.747 INFO:tasks.workunit.client.1.smithi144.stdout:5/250: mknod d0/c4f 0 2024-05-15T15:54:05.748 INFO:tasks.workunit.client.1.smithi144.stdout:3/241: mknod d0/d1/c55 0 2024-05-15T15:54:05.749 INFO:tasks.workunit.client.1.smithi144.stdout:6/208: creat d7/f4f x:0 0 0 2024-05-15T15:54:05.754 INFO:tasks.workunit.client.1.smithi144.stdout:2/247: dread d3/d10/f2e [0,4194304] 0 2024-05-15T15:54:05.755 INFO:tasks.workunit.client.1.smithi144.stdout:3/242: mkdir d0/d1/d56 0 2024-05-15T15:54:05.755 INFO:tasks.workunit.client.1.smithi144.stdout:3/243: stat d0/d11/d3b 0 2024-05-15T15:54:05.757 INFO:tasks.workunit.client.1.smithi144.stdout:9/253: link d0/d8/d5e/f3a d0/d8/d5e/d44/d3c/f61 0 2024-05-15T15:54:05.757 INFO:tasks.workunit.client.1.smithi144.stdout:9/254: chown d0/d8 1 1 2024-05-15T15:54:05.758 INFO:tasks.workunit.client.1.smithi144.stdout:2/248: rename d3/l2a to d3/d7/d20/d22/l4e 0 2024-05-15T15:54:05.759 INFO:tasks.workunit.client.1.smithi144.stdout:1/250: truncate d7/d1d/d33/d29/f2c 88733 0 2024-05-15T15:54:05.760 INFO:tasks.workunit.client.1.smithi144.stdout:3/244: creat d0/d11/d24/f57 x:0 0 0 2024-05-15T15:54:05.762 INFO:tasks.workunit.client.1.smithi144.stdout:0/227: dwrite d4/df/d1f/d24/f2a [0,4194304] 0 2024-05-15T15:54:05.763 INFO:tasks.workunit.client.1.smithi144.stdout:0/228: fdatasync d4/d2d/f44 0 2024-05-15T15:54:05.764 INFO:tasks.workunit.client.1.smithi144.stdout:4/258: truncate d6/d42/d46/d4b/f4d 3673419 0 2024-05-15T15:54:05.766 INFO:tasks.workunit.client.1.smithi144.stdout:1/251: creat d7/d4b/f4f x:0 0 0 2024-05-15T15:54:05.767 INFO:tasks.workunit.client.1.smithi144.stdout:7/252: write d4/f12 [4208516,16277] 0 2024-05-15T15:54:05.769 INFO:tasks.workunit.client.1.smithi144.stdout:2/249: creat d3/d7/d20/d3b/f4f x:0 0 0 2024-05-15T15:54:05.769 INFO:tasks.workunit.client.1.smithi144.stdout:7/253: write d4/df/d10/f3b [298364,14829] 0 2024-05-15T15:54:05.770 INFO:tasks.workunit.client.1.smithi144.stdout:0/229: symlink d4/d6/d4a/l52 0 2024-05-15T15:54:05.772 INFO:tasks.workunit.client.1.smithi144.stdout:4/259: creat d6/dd/d2d/f54 x:0 0 0 2024-05-15T15:54:05.773 INFO:tasks.workunit.client.1.smithi144.stdout:5/251: dread d0/f22 [0,4194304] 0 2024-05-15T15:54:05.774 INFO:tasks.workunit.client.1.smithi144.stdout:8/253: write d2/de/fc [1101972,19511] 0 2024-05-15T15:54:05.776 INFO:tasks.workunit.client.1.smithi144.stdout:6/209: write f6 [3308654,95057] 0 2024-05-15T15:54:05.777 INFO:tasks.workunit.client.1.smithi144.stdout:1/252: truncate d7/d1d/d33/d1e/f25 671811 0 2024-05-15T15:54:05.780 INFO:tasks.workunit.client.1.smithi144.stdout:7/254: creat d4/df/d25/d40/d47/f53 x:0 0 0 2024-05-15T15:54:05.781 INFO:tasks.workunit.client.1.smithi144.stdout:0/230: symlink d4/df/d14/l53 0 2024-05-15T15:54:05.782 INFO:tasks.workunit.client.1.smithi144.stdout:2/250: dwrite d3/d7/f3d [0,4194304] 0 2024-05-15T15:54:05.783 INFO:tasks.workunit.client.1.smithi144.stdout:4/260: write d6/dd/d34/d3b/d30/db/f35 [4716430,10148] 0 2024-05-15T15:54:05.786 INFO:tasks.workunit.client.1.smithi144.stdout:5/252: rename d0/dc/d1f/d29 to d0/dc/d1f/d50 0 2024-05-15T15:54:05.786 INFO:tasks.workunit.client.1.smithi144.stdout:7/255: fsync d4/f13 0 2024-05-15T15:54:05.787 INFO:tasks.workunit.client.1.smithi144.stdout:7/256: stat d4/df/d25/f4a 0 2024-05-15T15:54:05.792 INFO:tasks.workunit.client.1.smithi144.stdout:5/253: dwrite d0/f22 [4194304,4194304] 0 2024-05-15T15:54:05.798 INFO:tasks.workunit.client.1.smithi144.stdout:7/257: creat d4/df/d25/d38/f54 x:0 0 0 2024-05-15T15:54:05.799 INFO:tasks.workunit.client.1.smithi144.stdout:5/254: dread d0/dc/d24/d28/d2d/f3d [0,4194304] 0 2024-05-15T15:54:05.801 INFO:tasks.workunit.client.1.smithi144.stdout:9/255: write d0/d8/d5e/f3a [1366819,73239] 0 2024-05-15T15:54:05.802 INFO:tasks.workunit.client.1.smithi144.stdout:6/210: link d7/f8 d7/d32/f50 0 2024-05-15T15:54:05.805 INFO:tasks.workunit.client.1.smithi144.stdout:3/245: truncate d0/d1/d17/f48 11992 0 2024-05-15T15:54:05.806 INFO:tasks.workunit.client.1.smithi144.stdout:1/253: chown d7/d1d/d33/d1e/f25 12685266 1 2024-05-15T15:54:05.806 INFO:tasks.workunit.client.1.smithi144.stdout:9/256: dread d0/d8/d5e/d44/d10/d19/f33 [4194304,4194304] 0 2024-05-15T15:54:05.809 INFO:tasks.workunit.client.1.smithi144.stdout:0/231: link d4/d6/c1b d4/df/d1f/d24/c54 0 2024-05-15T15:54:05.810 INFO:tasks.workunit.client.1.smithi144.stdout:7/258: readlink d4/df/d25/l42 0 2024-05-15T15:54:05.813 INFO:tasks.workunit.client.1.smithi144.stdout:6/211: dwrite d7/f4b [0,4194304] 0 2024-05-15T15:54:05.813 INFO:tasks.workunit.client.1.smithi144.stdout:8/254: dread d2/de/d36/f1b [0,4194304] 0 2024-05-15T15:54:05.816 INFO:tasks.workunit.client.1.smithi144.stdout:5/255: dread d0/dc/f1b [0,4194304] 0 2024-05-15T15:54:05.818 INFO:tasks.workunit.client.1.smithi144.stdout:3/246: dwrite d0/d11/d24/f49 [0,4194304] 0 2024-05-15T15:54:05.820 INFO:tasks.workunit.client.1.smithi144.stdout:4/261: getdents d6/dd 0 2024-05-15T15:54:05.823 INFO:tasks.workunit.client.1.smithi144.stdout:0/232: mkdir d4/df/d1f/d24/d2b/d55 0 2024-05-15T15:54:05.827 INFO:tasks.workunit.client.1.smithi144.stdout:5/256: dwrite d0/dc/d24/d28/d2d/f2b [0,4194304] 0 2024-05-15T15:54:05.827 INFO:tasks.workunit.client.1.smithi144.stdout:2/251: truncate d3/d10/d15/f2f 2244359 0 2024-05-15T15:54:05.833 INFO:tasks.workunit.client.1.smithi144.stdout:2/252: dwrite d3/d10/d15/f26 [0,4194304] 0 2024-05-15T15:54:05.834 INFO:tasks.workunit.client.1.smithi144.stdout:2/253: chown d3/d10/c19 682821 1 2024-05-15T15:54:05.836 INFO:tasks.workunit.client.1.smithi144.stdout:2/254: dread - d3/d7/d20/d37/d3a/f45 zero size 2024-05-15T15:54:05.838 INFO:tasks.workunit.client.1.smithi144.stdout:1/254: creat d7/d1d/d33/d1e/d4e/f50 x:0 0 0 2024-05-15T15:54:05.842 INFO:tasks.workunit.client.1.smithi144.stdout:7/259: mknod d4/d3c/c55 0 2024-05-15T15:54:05.843 INFO:tasks.workunit.client.1.smithi144.stdout:0/233: mkdir d4/df/d1f/d56 0 2024-05-15T15:54:05.845 INFO:tasks.workunit.client.1.smithi144.stdout:5/257: dread d0/f11 [0,4194304] 0 2024-05-15T15:54:05.846 INFO:tasks.workunit.client.1.smithi144.stdout:6/212: mknod d7/c51 0 2024-05-15T15:54:05.846 INFO:tasks.workunit.client.1.smithi144.stdout:5/258: stat d0/dc/d24/f37 0 2024-05-15T15:54:05.848 INFO:tasks.workunit.client.1.smithi144.stdout:8/255: unlink d2/d24/d2d/c3c 0 2024-05-15T15:54:05.849 INFO:tasks.workunit.client.1.smithi144.stdout:3/247: creat d0/d1/d56/f58 x:0 0 0 2024-05-15T15:54:05.851 INFO:tasks.workunit.client.1.smithi144.stdout:7/260: truncate d4/f21 472736 0 2024-05-15T15:54:05.853 INFO:tasks.workunit.client.1.smithi144.stdout:0/234: rename d4/d6/l29 to d4/d6/d3a/d4c/l57 0 2024-05-15T15:54:05.854 INFO:tasks.workunit.client.1.smithi144.stdout:5/259: stat d0/dc/d12/f13 0 2024-05-15T15:54:05.857 INFO:tasks.workunit.client.1.smithi144.stdout:2/255: symlink d3/l50 0 2024-05-15T15:54:05.860 INFO:tasks.workunit.client.1.smithi144.stdout:5/260: dwrite d0/dc/d12/f43 [0,4194304] 0 2024-05-15T15:54:05.863 INFO:tasks.workunit.client.1.smithi144.stdout:0/235: creat d4/df/d1f/f58 x:0 0 0 2024-05-15T15:54:05.865 INFO:tasks.workunit.client.1.smithi144.stdout:6/213: mknod d7/db/c52 0 2024-05-15T15:54:05.866 INFO:tasks.workunit.client.1.smithi144.stdout:8/256: unlink d2/de/d36/c18 0 2024-05-15T15:54:05.871 INFO:tasks.workunit.client.1.smithi144.stdout:1/255: link d7/df/d24/l2f d7/d1d/d33/l51 0 2024-05-15T15:54:05.874 INFO:tasks.workunit.client.1.smithi144.stdout:7/261: symlink d4/df/d10/d48/l56 0 2024-05-15T15:54:05.876 INFO:tasks.workunit.client.1.smithi144.stdout:0/236: creat d4/df/d1f/d24/f59 x:0 0 0 2024-05-15T15:54:05.876 INFO:tasks.workunit.client.1.smithi144.stdout:0/237: fsync d4/d6/fb 0 2024-05-15T15:54:05.877 INFO:tasks.workunit.client.1.smithi144.stdout:0/238: write d4/df/d1f/d24/f59 [273440,105289] 0 2024-05-15T15:54:05.878 INFO:tasks.workunit.client.1.smithi144.stdout:8/257: mknod d2/d24/d2d/c57 0 2024-05-15T15:54:05.880 INFO:tasks.workunit.client.1.smithi144.stdout:2/256: symlink d3/d7/d20/d3f/l51 0 2024-05-15T15:54:05.881 INFO:tasks.workunit.client.1.smithi144.stdout:3/248: link d0/d1/c1a d0/d1/d56/c59 0 2024-05-15T15:54:05.882 INFO:tasks.workunit.client.1.smithi144.stdout:1/256: mkdir d7/d1d/d52 0 2024-05-15T15:54:05.884 INFO:tasks.workunit.client.1.smithi144.stdout:7/262: creat d4/d3c/f57 x:0 0 0 2024-05-15T15:54:05.888 INFO:tasks.workunit.client.1.smithi144.stdout:0/239: creat d4/d6/d4a/f5a x:0 0 0 2024-05-15T15:54:05.890 INFO:tasks.workunit.client.1.smithi144.stdout:8/258: mkdir d2/de/d36/d58 0 2024-05-15T15:54:05.890 INFO:tasks.workunit.client.1.smithi144.stdout:7/263: dwrite d4/df/d25/d40/d47/f53 [0,4194304] 0 2024-05-15T15:54:05.891 INFO:tasks.workunit.client.1.smithi144.stdout:8/259: fdatasync d2/de/d36/f27 0 2024-05-15T15:54:05.892 INFO:tasks.workunit.client.1.smithi144.stdout:7/264: write d4/f2f [1762425,7107] 0 2024-05-15T15:54:05.894 INFO:tasks.workunit.client.1.smithi144.stdout:3/249: rename d0/d1/cc to d0/d1/d56/c5a 0 2024-05-15T15:54:05.896 INFO:tasks.workunit.client.1.smithi144.stdout:1/257: mkdir d7/d4b/d53 0 2024-05-15T15:54:05.896 INFO:tasks.workunit.client.1.smithi144.stdout:1/258: chown d7/d1d/d33/d1e/l2d 1201707072 1 2024-05-15T15:54:05.897 INFO:tasks.workunit.client.1.smithi144.stdout:1/259: dread - d7/f4d zero size 2024-05-15T15:54:05.898 INFO:tasks.workunit.client.1.smithi144.stdout:1/260: write d7/d1d/d33/d41/d22/f48 [654327,81877] 0 2024-05-15T15:54:05.899 INFO:tasks.workunit.client.1.smithi144.stdout:0/240: mkdir d4/d2d/d48/d5b 0 2024-05-15T15:54:05.902 INFO:tasks.workunit.client.1.smithi144.stdout:7/265: mknod d4/d3c/c58 0 2024-05-15T15:54:05.903 INFO:tasks.workunit.client.1.smithi144.stdout:8/260: rename d2/de/d36/f27 to d2/d3a/f59 0 2024-05-15T15:54:05.904 INFO:tasks.workunit.client.1.smithi144.stdout:3/250: creat d0/d11/d3b/f5b x:0 0 0 2024-05-15T15:54:05.905 INFO:tasks.workunit.client.1.smithi144.stdout:1/261: creat d7/df/d24/f54 x:0 0 0 2024-05-15T15:54:05.907 INFO:tasks.workunit.client.1.smithi144.stdout:5/261: dwrite d0/dc/d1f/f3b [0,4194304] 0 2024-05-15T15:54:05.908 INFO:tasks.workunit.client.1.smithi144.stdout:8/261: fsync d2/de/d36/f1b 0 2024-05-15T15:54:05.909 INFO:tasks.workunit.client.1.smithi144.stdout:6/214: dwrite d7/db/f3f [0,4194304] 0 2024-05-15T15:54:05.912 INFO:tasks.workunit.client.1.smithi144.stdout:2/257: getdents d3/d7/d20/d37 0 2024-05-15T15:54:05.913 INFO:tasks.workunit.client.1.smithi144.stdout:1/262: fsync d7/d1d/d33/f12 0 2024-05-15T15:54:05.914 INFO:tasks.workunit.client.1.smithi144.stdout:5/262: readlink d0/dc/d12/l3f 0 2024-05-15T15:54:05.916 INFO:tasks.workunit.client.1.smithi144.stdout:6/215: mknod d7/d1e/d28/c53 0 2024-05-15T15:54:05.916 INFO:tasks.workunit.client.1.smithi144.stdout:1/263: chown d7/d1d/d33/d1e/d4e 374 1 2024-05-15T15:54:05.917 INFO:tasks.workunit.client.1.smithi144.stdout:2/258: mkdir d3/d7/d20/d22/d52 0 2024-05-15T15:54:05.917 INFO:tasks.workunit.client.1.smithi144.stdout:7/266: dwrite d4/df/f32 [0,4194304] 0 2024-05-15T15:54:05.918 INFO:tasks.workunit.client.1.smithi144.stdout:7/267: readlink d4/ld 0 2024-05-15T15:54:05.919 INFO:tasks.workunit.client.1.smithi144.stdout:8/262: rename d2/d24/d2d/l51 to d2/d3a/l5a 0 2024-05-15T15:54:05.922 INFO:tasks.workunit.client.1.smithi144.stdout:5/263: creat d0/dc/d24/d28/f51 x:0 0 0 2024-05-15T15:54:05.924 INFO:tasks.workunit.client.1.smithi144.stdout:1/264: mkdir d7/d1d/d33/d55 0 2024-05-15T15:54:05.925 INFO:tasks.workunit.client.1.smithi144.stdout:1/265: write d7/df/d24/f54 [630856,45452] 0 2024-05-15T15:54:05.925 INFO:tasks.workunit.client.1.smithi144.stdout:6/216: mknod d7/db/d11/d3c/c54 0 2024-05-15T15:54:05.926 INFO:tasks.workunit.client.1.smithi144.stdout:0/241: truncate d4/dc/f17 1109512 0 2024-05-15T15:54:05.929 INFO:tasks.workunit.client.1.smithi144.stdout:3/251: link d0/d1/d17/d39/d4d/l4e d0/d11/d3b/l5c 0 2024-05-15T15:54:05.930 INFO:tasks.workunit.client.1.smithi144.stdout:5/264: rename d0/dc/d1f/d50/f3c to d0/dc/d12/d31/d35/f52 0 2024-05-15T15:54:05.932 INFO:tasks.workunit.client.1.smithi144.stdout:1/266: dwrite d7/d1d/d33/f3f [0,4194304] 0 2024-05-15T15:54:05.934 INFO:tasks.workunit.client.1.smithi144.stdout:5/265: write d0/dc/d24/d28/d2d/f2b [3546422,35320] 0 2024-05-15T15:54:05.934 INFO:tasks.workunit.client.1.smithi144.stdout:1/267: truncate d7/f4d 472562 0 2024-05-15T15:54:05.934 INFO:tasks.workunit.client.1.smithi144.stdout:8/263: mkdir d2/d1a/d38/d4b/d5b 0 2024-05-15T15:54:05.935 INFO:tasks.workunit.client.1.smithi144.stdout:5/266: readlink d0/l17 0 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:7/294: sync 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:9/231: sync 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:0/289: sync 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:3/361: sync 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:8/329: sync 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:6/276: sync 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:1/340: sync 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:5/296: sync 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:4/403: sync 2024-05-15T15:54:05.936 INFO:tasks.workunit.client.0.smithi049.stdout:2/349: sync 2024-05-15T15:54:05.937 INFO:tasks.workunit.client.1.smithi144.stdout:2/259: rmdir d3/d7/d20 39 2024-05-15T15:54:05.938 INFO:tasks.workunit.client.0.smithi049.stdout:9/232: stat d0/d8/d12 0 2024-05-15T15:54:05.939 INFO:tasks.workunit.client.0.smithi049.stdout:4/404: chown d18/f80 14 1 2024-05-15T15:54:05.940 INFO:tasks.workunit.client.0.smithi049.stdout:2/350: dread - d3/db/d1d/d1e/d2b/d3b/f5f zero size 2024-05-15T15:54:05.942 INFO:tasks.workunit.client.0.smithi049.stdout:5/297: rmdir dc/dd/d10 39 2024-05-15T15:54:05.942 INFO:tasks.workunit.client.0.smithi049.stdout:5/298: chown dc/dd/f3d 0 1 2024-05-15T15:54:05.946 INFO:tasks.workunit.client.1.smithi144.stdout:6/217: dread d7/db/f1d [0,4194304] 0 2024-05-15T15:54:05.947 INFO:tasks.workunit.client.1.smithi144.stdout:1/268: creat d7/df/f56 x:0 0 0 2024-05-15T15:54:05.948 INFO:tasks.workunit.client.1.smithi144.stdout:1/269: fsync d7/d1d/f32 0 2024-05-15T15:54:05.948 INFO:tasks.workunit.client.1.smithi144.stdout:0/242: write d4/df/d1f/f2f [1589852,107117] 0 2024-05-15T15:54:05.949 INFO:tasks.workunit.client.1.smithi144.stdout:1/270: write d7/d1d/f32 [3021099,72124] 0 2024-05-15T15:54:05.951 INFO:tasks.workunit.client.0.smithi049.stdout:3/362: mkdir d12/d30/d42/d71 0 2024-05-15T15:54:05.951 INFO:tasks.workunit.client.0.smithi049.stdout:7/295: dwrite d0/f47 [0,4194304] 0 2024-05-15T15:54:05.953 INFO:tasks.workunit.client.1.smithi144.stdout:5/267: unlink d0/dc/d12/f46 0 2024-05-15T15:54:05.954 INFO:tasks.workunit.client.0.smithi049.stdout:6/277: creat dd/d17/d24/d2b/f48 x:0 0 0 2024-05-15T15:54:05.955 INFO:tasks.workunit.client.0.smithi049.stdout:6/278: fdatasync dd/d17/d24/d2b/f46 0 2024-05-15T15:54:05.956 INFO:tasks.workunit.client.1.smithi144.stdout:5/268: dread d0/dc/d1f/f3b [0,4194304] 0 2024-05-15T15:54:05.958 INFO:tasks.workunit.client.1.smithi144.stdout:2/260: unlink d3/d10/d15/l1b 0 2024-05-15T15:54:05.958 INFO:tasks.workunit.client.0.smithi049.stdout:0/290: mkdir dc/d13/d5e 0 2024-05-15T15:54:05.960 INFO:tasks.workunit.client.1.smithi144.stdout:6/218: mkdir d7/db/d11/d55 0 2024-05-15T15:54:05.962 INFO:tasks.workunit.client.1.smithi144.stdout:5/269: dread d0/dc/d12/d31/d35/f52 [0,4194304] 0 2024-05-15T15:54:05.963 INFO:tasks.workunit.client.1.smithi144.stdout:5/270: write d0/dc/d24/d28/f51 [279817,64640] 0 2024-05-15T15:54:05.965 INFO:tasks.workunit.client.0.smithi049.stdout:4/405: write f10 [1729439,62039] 0 2024-05-15T15:54:05.966 INFO:tasks.workunit.client.0.smithi049.stdout:1/341: creat d0/d4/d81/f82 x:0 0 0 2024-05-15T15:54:05.967 INFO:tasks.workunit.client.1.smithi144.stdout:8/264: creat d2/de/d36/d58/f5c x:0 0 0 2024-05-15T15:54:05.968 INFO:tasks.workunit.client.1.smithi144.stdout:8/265: write d2/d24/f25 [100518,51024] 0 2024-05-15T15:54:05.969 INFO:tasks.workunit.client.0.smithi049.stdout:7/296: mkdir d0/d6/da/d3d/d54 0 2024-05-15T15:54:05.969 INFO:tasks.workunit.client.1.smithi144.stdout:1/271: write d7/f4d [282829,80157] 0 2024-05-15T15:54:05.970 INFO:tasks.workunit.client.1.smithi144.stdout:1/272: readlink d7/l3c 0 2024-05-15T15:54:05.972 INFO:tasks.workunit.client.0.smithi049.stdout:3/363: rename d12/d14/d52/f61 to d12/d30/d63/d68/d70/f72 0 2024-05-15T15:54:05.973 INFO:tasks.workunit.client.1.smithi144.stdout:0/243: mknod d4/d6/d3a/c5c 0 2024-05-15T15:54:05.974 INFO:tasks.workunit.client.0.smithi049.stdout:1/342: dwrite d0/f49 [0,4194304] 0 2024-05-15T15:54:05.974 INFO:tasks.workunit.client.1.smithi144.stdout:8/266: dwrite d2/de/d3f/f4c [0,4194304] 0 2024-05-15T15:54:05.976 INFO:tasks.workunit.client.0.smithi049.stdout:1/343: write d0/d4/dd/de/d36/f6a [15771,90222] 0 2024-05-15T15:54:05.976 INFO:tasks.workunit.client.0.smithi049.stdout:6/279: symlink dd/d17/d24/d2b/d36/l49 0 2024-05-15T15:54:05.976 INFO:tasks.workunit.client.0.smithi049.stdout:2/351: write d3/da/d15/f26 [218877,17328] 0 2024-05-15T15:54:05.978 INFO:tasks.workunit.client.0.smithi049.stdout:1/344: chown d0/d4/d63/c7f 15 1 2024-05-15T15:54:05.979 INFO:tasks.workunit.client.0.smithi049.stdout:6/280: fsync dd/d17/d24/d2b/d36/f41 0 2024-05-15T15:54:05.979 INFO:tasks.workunit.client.0.smithi049.stdout:5/299: fdatasync dc/dd/d39/f4f 0 2024-05-15T15:54:05.980 INFO:tasks.workunit.client.1.smithi144.stdout:9/257: sync 2024-05-15T15:54:05.984 INFO:tasks.workunit.client.1.smithi144.stdout:6/219: write d7/db/f29 [2653587,27447] 0 2024-05-15T15:54:05.985 INFO:tasks.workunit.client.1.smithi144.stdout:6/220: fsync d7/ff 0 2024-05-15T15:54:05.989 INFO:tasks.workunit.client.0.smithi049.stdout:7/297: unlink d0/d6/da/d3d/d39/l53 0 2024-05-15T15:54:05.990 INFO:tasks.workunit.client.0.smithi049.stdout:7/298: stat d0/d6/d1d/c52 0 2024-05-15T15:54:05.993 INFO:tasks.workunit.client.1.smithi144.stdout:1/273: unlink d7/df/f56 0 2024-05-15T15:54:05.993 INFO:tasks.workunit.client.0.smithi049.stdout:3/364: mkdir d12/d26/d73 0 2024-05-15T15:54:05.994 INFO:tasks.workunit.client.0.smithi049.stdout:3/365: chown d12/l2b 398 1 2024-05-15T15:54:05.995 INFO:tasks.workunit.client.1.smithi144.stdout:0/244: mkdir d4/d6/d3a/d5d 0 2024-05-15T15:54:05.997 INFO:tasks.workunit.client.1.smithi144.stdout:3/252: rename d0/d1/d56/c5a to d0/d1/c5d 0 2024-05-15T15:54:05.998 INFO:tasks.workunit.client.1.smithi144.stdout:9/258: readlink d0/d8/l1f 0 2024-05-15T15:54:05.999 INFO:tasks.workunit.client.1.smithi144.stdout:3/253: write d0/d11/f2a [2854060,16362] 0 2024-05-15T15:54:05.999 INFO:tasks.workunit.client.0.smithi049.stdout:1/345: creat d0/d4/d63/f83 x:0 0 0 2024-05-15T15:54:06.000 INFO:tasks.workunit.client.1.smithi144.stdout:3/254: truncate d0/f3d 4247800 0 2024-05-15T15:54:06.000 INFO:tasks.workunit.client.1.smithi144.stdout:2/261: creat d3/d7/d20/d3f/f53 x:0 0 0 2024-05-15T15:54:06.006 INFO:tasks.workunit.client.0.smithi049.stdout:5/300: creat dc/dd/d47/f54 x:0 0 0 2024-05-15T15:54:06.008 INFO:tasks.workunit.client.0.smithi049.stdout:4/406: mkdir d18/d88 0 2024-05-15T15:54:06.011 INFO:tasks.workunit.client.1.smithi144.stdout:6/221: readlink d7/db/l1a 0 2024-05-15T15:54:06.011 INFO:tasks.workunit.client.1.smithi144.stdout:6/222: write f6 [2850415,66923] 0 2024-05-15T15:54:06.012 INFO:tasks.workunit.client.0.smithi049.stdout:2/352: mkdir d3/d68 0 2024-05-15T15:54:06.013 INFO:tasks.workunit.client.0.smithi049.stdout:0/291: link dc/d13/d14/l42 dc/d13/d14/d2a/l5f 0 2024-05-15T15:54:06.016 INFO:tasks.workunit.client.1.smithi144.stdout:5/271: truncate d0/f22 4597624 0 2024-05-15T15:54:06.017 INFO:tasks.workunit.client.1.smithi144.stdout:0/245: creat d4/df/d1f/d24/f5e x:0 0 0 2024-05-15T15:54:06.020 INFO:tasks.workunit.client.0.smithi049.stdout:6/281: dwrite dd/f14 [0,4194304] 0 2024-05-15T15:54:06.021 INFO:tasks.workunit.client.0.smithi049.stdout:0/292: dwrite dc/d13/d14/d2f/f51 [0,4194304] 0 2024-05-15T15:54:06.021 INFO:tasks.workunit.client.0.smithi049.stdout:6/282: write f4 [1125012,39669] 0 2024-05-15T15:54:06.022 INFO:tasks.workunit.client.1.smithi144.stdout:9/259: rename d0/d8/d5e/d44/d16/d5b to d0/d8/d1b/d62 0 2024-05-15T15:54:06.025 INFO:tasks.workunit.client.0.smithi049.stdout:2/353: fdatasync d3/d8/f36 0 2024-05-15T15:54:06.026 INFO:tasks.workunit.client.1.smithi144.stdout:6/223: mkdir d7/db/d11/d3c/d56 0 2024-05-15T15:54:06.027 INFO:tasks.workunit.client.1.smithi144.stdout:6/224: readlink d7/db/d11/d3c/l49 0 2024-05-15T15:54:06.034 INFO:tasks.workunit.client.1.smithi144.stdout:5/272: symlink d0/dc/d24/d28/l53 0 2024-05-15T15:54:06.035 INFO:tasks.workunit.client.1.smithi144.stdout:5/273: stat d0/dc/f1b 0 2024-05-15T15:54:06.036 INFO:tasks.workunit.client.0.smithi049.stdout:6/283: unlink dd/d26/f3c 0 2024-05-15T15:54:06.038 INFO:tasks.workunit.client.1.smithi144.stdout:0/246: creat d4/df/d14/f5f x:0 0 0 2024-05-15T15:54:06.039 INFO:tasks.workunit.client.1.smithi144.stdout:0/247: dread - d4/d6/d4a/f5a zero size 2024-05-15T15:54:06.039 INFO:tasks.workunit.client.0.smithi049.stdout:3/366: truncate d12/d14/f1e 1368526 0 2024-05-15T15:54:06.039 INFO:tasks.workunit.client.0.smithi049.stdout:3/367: readlink d12/d67/l6a 0 2024-05-15T15:54:06.040 INFO:tasks.workunit.client.1.smithi144.stdout:0/248: read d4/d2d/f44 [56523,3597] 0 2024-05-15T15:54:06.040 INFO:tasks.workunit.client.0.smithi049.stdout:0/293: rename dc/d13/d14/d2a/f30 to dc/d32/f60 0 2024-05-15T15:54:06.042 INFO:tasks.workunit.client.1.smithi144.stdout:9/260: creat d0/d8/d5e/d44/d10/d19/f63 x:0 0 0 2024-05-15T15:54:06.045 INFO:tasks.workunit.client.0.smithi049.stdout:2/354: creat d3/db/d1d/d1e/d2b/d3b/d42/f69 x:0 0 0 2024-05-15T15:54:06.045 INFO:tasks.workunit.client.1.smithi144.stdout:2/262: mkdir d3/d7/d54 0 2024-05-15T15:54:06.046 INFO:tasks.workunit.client.0.smithi049.stdout:5/301: link dc/lf dc/dd/d47/l55 0 2024-05-15T15:54:06.047 INFO:tasks.workunit.client.0.smithi049.stdout:2/355: chown d3/db/c33 3811 1 2024-05-15T15:54:06.047 INFO:tasks.workunit.client.1.smithi144.stdout:7/268: sync 2024-05-15T15:54:06.047 INFO:tasks.workunit.client.1.smithi144.stdout:4/262: sync 2024-05-15T15:54:06.047 INFO:tasks.workunit.client.0.smithi049.stdout:0/294: dwrite dc/d13/d14/d2a/f3a [0,4194304] 0 2024-05-15T15:54:06.049 INFO:tasks.workunit.client.1.smithi144.stdout:1/274: link d7/df/c46 d7/d4b/c57 0 2024-05-15T15:54:06.049 INFO:tasks.workunit.client.0.smithi049.stdout:0/295: fdatasync dc/d13/d3d/f43 0 2024-05-15T15:54:06.050 INFO:tasks.workunit.client.0.smithi049.stdout:2/356: dread d3/da/d15/d24/f62 [0,4194304] 0 2024-05-15T15:54:06.051 INFO:tasks.workunit.client.0.smithi049.stdout:6/284: creat dd/d17/f4a x:0 0 0 2024-05-15T15:54:06.058 INFO:tasks.workunit.client.0.smithi049.stdout:1/346: getdents d0/d4/dd/de/d4e 0 2024-05-15T15:54:06.059 INFO:tasks.workunit.client.0.smithi049.stdout:1/347: stat d0/d4/dd/f5a 0 2024-05-15T15:54:06.062 INFO:tasks.workunit.client.1.smithi144.stdout:2/263: fsync d3/d10/f1d 0 2024-05-15T15:54:06.065 INFO:tasks.workunit.client.1.smithi144.stdout:6/225: link d7/f4b d7/d1e/d28/f57 0 2024-05-15T15:54:06.065 INFO:tasks.workunit.client.1.smithi144.stdout:3/255: write d0/d11/d24/f3e [1705,7999] 0 2024-05-15T15:54:06.066 INFO:tasks.workunit.client.0.smithi049.stdout:0/296: write dc/d13/d14/d33/f47 [1500938,6339] 0 2024-05-15T15:54:06.068 INFO:tasks.workunit.client.0.smithi049.stdout:0/297: write dc/d13/d14/f46 [727495,15667] 0 2024-05-15T15:54:06.069 INFO:tasks.workunit.client.0.smithi049.stdout:6/285: creat dd/d26/f4b x:0 0 0 2024-05-15T15:54:06.069 INFO:tasks.workunit.client.0.smithi049.stdout:0/298: write dc/d13/d14/d2f/f51 [916046,28637] 0 2024-05-15T15:54:06.072 INFO:tasks.workunit.client.1.smithi144.stdout:4/263: rename d6/d42/d46/d4b/d1e/d44/f47 to d6/d49/f55 0 2024-05-15T15:54:06.073 INFO:tasks.workunit.client.1.smithi144.stdout:7/269: dread d4/df/f16 [0,4194304] 0 2024-05-15T15:54:06.074 INFO:tasks.workunit.client.1.smithi144.stdout:1/275: mkdir d7/d1d/d58 0 2024-05-15T15:54:06.074 INFO:tasks.workunit.client.0.smithi049.stdout:0/299: dread dc/d13/d14/f46 [0,4194304] 0 2024-05-15T15:54:06.076 INFO:tasks.workunit.client.1.smithi144.stdout:9/261: write d0/d8/d21/f2a [1208567,106278] 0 2024-05-15T15:54:06.076 INFO:tasks.workunit.client.0.smithi049.stdout:5/302: dread dc/dd/d10/f1e [0,4194304] 0 2024-05-15T15:54:06.077 INFO:tasks.workunit.client.0.smithi049.stdout:5/303: stat dc/f1b 0 2024-05-15T15:54:06.078 INFO:tasks.workunit.client.0.smithi049.stdout:1/348: rename d0/d4/dd/d75 to d0/d4/dd/de/d48/d84 0 2024-05-15T15:54:06.082 INFO:tasks.workunit.client.1.smithi144.stdout:9/262: dwrite d0/d8/d5e/d44/d10/d19/d3e/f54 [0,4194304] 0 2024-05-15T15:54:06.082 INFO:tasks.workunit.client.1.smithi144.stdout:0/249: creat d4/df/d1f/d24/d2b/d55/f60 x:0 0 0 2024-05-15T15:54:06.083 INFO:tasks.workunit.client.1.smithi144.stdout:9/263: fdatasync d0/d8/d5e/d44/d3c/f42 0 2024-05-15T15:54:06.087 INFO:tasks.workunit.client.1.smithi144.stdout:9/264: dread d0/d8/d5e/d44/d10/d19/d3e/f54 [0,4194304] 0 2024-05-15T15:54:06.089 INFO:tasks.workunit.client.1.smithi144.stdout:3/256: mkdir d0/d11/d5e 0 2024-05-15T15:54:06.096 INFO:tasks.workunit.client.0.smithi049.stdout:0/300: unlink dc/d13/d14/d2f/f29 0 2024-05-15T15:54:06.097 INFO:tasks.workunit.client.0.smithi049.stdout:3/368: dwrite d12/d14/d32/f59 [0,4194304] 0 2024-05-15T15:54:06.098 INFO:tasks.workunit.client.1.smithi144.stdout:9/265: dread d0/d8/d5e/d44/d10/fd [0,4194304] 0 2024-05-15T15:54:06.099 INFO:tasks.workunit.client.0.smithi049.stdout:5/304: dread dc/f16 [0,4194304] 0 2024-05-15T15:54:06.099 INFO:tasks.workunit.client.1.smithi144.stdout:4/264: mknod d6/d49/c56 0 2024-05-15T15:54:06.099 INFO:tasks.workunit.client.0.smithi049.stdout:5/305: chown dc/dd/d39 7291879 1 2024-05-15T15:54:06.099 INFO:tasks.workunit.client.0.smithi049.stdout:1/349: dread d0/d4/dd/f18 [0,4194304] 0 2024-05-15T15:54:06.100 INFO:tasks.workunit.client.1.smithi144.stdout:4/265: fdatasync d6/dd/d34/d3b/f4c 0 2024-05-15T15:54:06.100 INFO:tasks.workunit.client.0.smithi049.stdout:2/357: truncate d3/d8/fe 578603 0 2024-05-15T15:54:06.100 INFO:tasks.workunit.client.1.smithi144.stdout:4/266: readlink d6/dd/d34/d3b/l2c 0 2024-05-15T15:54:06.102 INFO:tasks.workunit.client.1.smithi144.stdout:2/264: rename d3/d10/d15/f2d to d3/d7/d20/d37/d3a/f55 0 2024-05-15T15:54:06.104 INFO:tasks.workunit.client.1.smithi144.stdout:7/270: symlink d4/d41/l59 0 2024-05-15T15:54:06.105 INFO:tasks.workunit.client.1.smithi144.stdout:7/271: write d4/f51 [26366,34657] 0 2024-05-15T15:54:06.105 INFO:tasks.workunit.client.0.smithi049.stdout:0/301: dwrite dc/d13/d14/d2f/f52 [0,4194304] 0 2024-05-15T15:54:06.106 INFO:tasks.workunit.client.1.smithi144.stdout:7/272: dread - d4/f37 zero size 2024-05-15T15:54:06.106 INFO:tasks.workunit.client.1.smithi144.stdout:1/276: unlink d7/d1d/d33/f12 0 2024-05-15T15:54:06.112 INFO:tasks.workunit.client.0.smithi049.stdout:3/369: symlink d12/d30/d63/l74 0 2024-05-15T15:54:06.113 INFO:tasks.workunit.client.0.smithi049.stdout:5/306: creat dc/dd/d39/f56 x:0 0 0 2024-05-15T15:54:06.114 INFO:tasks.workunit.client.0.smithi049.stdout:5/307: stat dc/dd/d10/l25 0 2024-05-15T15:54:06.115 INFO:tasks.workunit.client.0.smithi049.stdout:1/350: mknod d0/d4/d63/c85 0 2024-05-15T15:54:06.116 INFO:tasks.workunit.client.0.smithi049.stdout:2/358: creat d3/db/d1d/d1e/d2b/d2e/f6a x:0 0 0 2024-05-15T15:54:06.117 INFO:tasks.workunit.client.1.smithi144.stdout:3/257: symlink d0/d1/d17/l5f 0 2024-05-15T15:54:06.120 INFO:tasks.workunit.client.1.smithi144.stdout:5/274: sync 2024-05-15T15:54:06.120 INFO:tasks.workunit.client.1.smithi144.stdout:2/265: creat d3/d7/d20/d3f/f56 x:0 0 0 2024-05-15T15:54:06.122 INFO:tasks.workunit.client.0.smithi049.stdout:6/286: link dd/d17/d24/d2b/f46 dd/f4c 0 2024-05-15T15:54:06.122 INFO:tasks.workunit.client.0.smithi049.stdout:6/287: chown dd/l1f 5756572 1 2024-05-15T15:54:06.123 INFO:tasks.workunit.client.0.smithi049.stdout:3/370: unlink d12/d14/l16 0 2024-05-15T15:54:06.124 INFO:tasks.workunit.client.1.smithi144.stdout:8/267: getdents d2 0 2024-05-15T15:54:06.125 INFO:tasks.workunit.client.0.smithi049.stdout:6/288: write dd/d17/d24/d2b/f3d [841973,27001] 0 2024-05-15T15:54:06.125 INFO:tasks.workunit.client.0.smithi049.stdout:3/371: stat d12/f1d 0 2024-05-15T15:54:06.125 INFO:tasks.workunit.client.1.smithi144.stdout:8/268: rename d2 to d2/d53/d5d 22 2024-05-15T15:54:06.125 INFO:tasks.workunit.client.0.smithi049.stdout:5/308: creat dc/d4a/f57 x:0 0 0 2024-05-15T15:54:06.127 INFO:tasks.workunit.client.0.smithi049.stdout:1/351: symlink d0/d4/dd/d5d/d65/d72/l86 0 2024-05-15T15:54:06.128 INFO:tasks.workunit.client.1.smithi144.stdout:0/250: creat d4/d6/d3a/d5d/f61 x:0 0 0 2024-05-15T15:54:06.128 INFO:tasks.workunit.client.1.smithi144.stdout:0/251: readlink d4/d2d/l51 0 2024-05-15T15:54:06.129 INFO:tasks.workunit.client.1.smithi144.stdout:0/252: write d4/df/d14/f5f [130027,92192] 0 2024-05-15T15:54:06.132 INFO:tasks.workunit.client.0.smithi049.stdout:6/289: dread dd/d26/f40 [0,4194304] 0 2024-05-15T15:54:06.133 INFO:tasks.workunit.client.0.smithi049.stdout:5/309: write f9 [4504106,125339] 0 2024-05-15T15:54:06.133 INFO:tasks.workunit.client.0.smithi049.stdout:6/290: chown dd/l2a 6 1 2024-05-15T15:54:06.133 INFO:tasks.workunit.client.0.smithi049.stdout:6/291: stat dd/d17 0 2024-05-15T15:54:06.134 INFO:tasks.workunit.client.0.smithi049.stdout:5/310: truncate dc/d4a/d42/f48 31972 0 2024-05-15T15:54:06.135 INFO:tasks.workunit.client.0.smithi049.stdout:1/352: creat d0/d4/dd/de/d48/d5c/f87 x:0 0 0 2024-05-15T15:54:06.135 INFO:tasks.workunit.client.0.smithi049.stdout:1/353: stat d0/d4/dd/f38 0 2024-05-15T15:54:06.137 INFO:tasks.workunit.client.0.smithi049.stdout:5/311: dread dc/dd/f17 [4194304,4194304] 0 2024-05-15T15:54:06.138 INFO:tasks.workunit.client.1.smithi144.stdout:6/226: sync 2024-05-15T15:54:06.138 INFO:tasks.workunit.client.1.smithi144.stdout:4/267: mkdir d6/dd/d34/d3b/d30/db/d1d/d40/d57 0 2024-05-15T15:54:06.139 INFO:tasks.workunit.client.1.smithi144.stdout:6/227: read - d7/db/d11/d3c/f4d zero size 2024-05-15T15:54:06.140 INFO:tasks.workunit.client.1.smithi144.stdout:5/275: rmdir d0/dc/d12/d31 39 2024-05-15T15:54:06.141 INFO:tasks.workunit.client.0.smithi049.stdout:6/292: dwrite dd/d17/d24/d2b/d36/f41 [0,4194304] 0 2024-05-15T15:54:06.142 INFO:tasks.workunit.client.0.smithi049.stdout:2/359: rmdir d3/d68 0 2024-05-15T15:54:06.143 INFO:tasks.workunit.client.1.smithi144.stdout:1/277: symlink d7/d1d/d33/d41/d22/d45/l59 0 2024-05-15T15:54:06.144 INFO:tasks.workunit.client.0.smithi049.stdout:1/354: creat d0/d4/dd/de/d1e/d2e/f88 x:0 0 0 2024-05-15T15:54:06.145 INFO:tasks.workunit.client.0.smithi049.stdout:1/355: dread - d0/d4/d63/f64 zero size 2024-05-15T15:54:06.145 INFO:tasks.workunit.client.1.smithi144.stdout:5/276: dwrite d0/dc/d12/f18 [0,4194304] 0 2024-05-15T15:54:06.145 INFO:tasks.workunit.client.1.smithi144.stdout:8/269: creat d2/d24/d2d/f5e x:0 0 0 2024-05-15T15:54:06.146 INFO:tasks.workunit.client.1.smithi144.stdout:1/278: read d7/d1d/d33/d41/d22/f48 [528854,91071] 0 2024-05-15T15:54:06.147 INFO:tasks.workunit.client.1.smithi144.stdout:2/266: dwrite d3/d7/d20/d37/d3a/f55 [0,4194304] 0 2024-05-15T15:54:06.147 INFO:tasks.workunit.client.1.smithi144.stdout:0/253: creat d4/df/d1f/f62 x:0 0 0 2024-05-15T15:54:06.148 INFO:tasks.workunit.client.0.smithi049.stdout:6/293: creat dd/d17/d24/d2b/f4d x:0 0 0 2024-05-15T15:54:06.149 INFO:tasks.workunit.client.0.smithi049.stdout:6/294: write dd/ff [2161726,35988] 0 2024-05-15T15:54:06.149 INFO:tasks.workunit.client.1.smithi144.stdout:9/266: link d0/d8/d5e/d44/l37 d0/d8/d21/l64 0 2024-05-15T15:54:06.149 INFO:tasks.workunit.client.1.smithi144.stdout:1/279: dread - d7/d1d/d33/d1e/d34/f4a zero size 2024-05-15T15:54:06.149 INFO:tasks.workunit.client.1.smithi144.stdout:8/270: truncate d2/de/d36/f48 1892458 0 2024-05-15T15:54:06.150 INFO:tasks.workunit.client.1.smithi144.stdout:9/267: chown d0/d8/f51 6582972 1 2024-05-15T15:54:06.153 INFO:tasks.workunit.client.0.smithi049.stdout:1/356: creat d0/d4/dd/de/d36/f89 x:0 0 0 2024-05-15T15:54:06.154 INFO:tasks.workunit.client.1.smithi144.stdout:1/280: dread d7/d1d/f32 [0,4194304] 0 2024-05-15T15:54:06.154 INFO:tasks.workunit.client.1.smithi144.stdout:6/228: rmdir d7/d1e/d28 39 2024-05-15T15:54:06.158 INFO:tasks.workunit.client.0.smithi049.stdout:6/295: mknod dd/d17/d24/d2b/c4e 0 2024-05-15T15:54:06.161 INFO:tasks.workunit.client.0.smithi049.stdout:1/357: mkdir d0/d4/dd/de/d1e/d16/d8a 0 2024-05-15T15:54:06.163 INFO:tasks.workunit.client.1.smithi144.stdout:2/267: creat d3/d7/d20/d3b/f57 x:0 0 0 2024-05-15T15:54:06.163 INFO:tasks.workunit.client.0.smithi049.stdout:5/312: link dc/dd/d47/l55 dc/d4a/l58 0 2024-05-15T15:54:06.164 INFO:tasks.workunit.client.1.smithi144.stdout:0/254: mknod d4/df/d1f/d24/d2b/c63 0 2024-05-15T15:54:06.165 INFO:tasks.workunit.client.1.smithi144.stdout:0/255: chown d4/d6/d4a/d4d 212 1 2024-05-15T15:54:06.165 INFO:tasks.workunit.client.1.smithi144.stdout:0/256: fsync d4/df/d14/f42 0 2024-05-15T15:54:06.166 INFO:tasks.workunit.client.0.smithi049.stdout:2/360: getdents d3/db/d1d/d1e/d2b/d3b/d47 0 2024-05-15T15:54:06.166 INFO:tasks.workunit.client.1.smithi144.stdout:0/257: chown d4/d6/d3a/d4c/l4f 9830 1 2024-05-15T15:54:06.167 INFO:tasks.workunit.client.0.smithi049.stdout:5/313: creat dc/dd/d10/f59 x:0 0 0 2024-05-15T15:54:06.168 INFO:tasks.workunit.client.1.smithi144.stdout:8/271: fsync d2/d3a/f59 0 2024-05-15T15:54:06.168 INFO:tasks.workunit.client.1.smithi144.stdout:8/272: readlink d2/l34 0 2024-05-15T15:54:06.170 INFO:tasks.workunit.client.1.smithi144.stdout:9/268: creat d0/d8/d5e/d44/d10/d19/d3e/f65 x:0 0 0 2024-05-15T15:54:06.171 INFO:tasks.workunit.client.1.smithi144.stdout:9/269: dread - d0/d8/d5e/d44/d3c/f45 zero size 2024-05-15T15:54:06.171 INFO:tasks.workunit.client.1.smithi144.stdout:7/273: getdents d4/df/d25/d40/d47 0 2024-05-15T15:54:06.172 INFO:tasks.workunit.client.1.smithi144.stdout:5/277: symlink d0/dc/d12/d4a/l54 0 2024-05-15T15:54:06.173 INFO:tasks.workunit.client.1.smithi144.stdout:2/268: mknod d3/d7/d20/d3f/c58 0 2024-05-15T15:54:06.174 INFO:tasks.workunit.client.1.smithi144.stdout:3/258: getdents d0/d1/d17/d39 0 2024-05-15T15:54:06.175 INFO:tasks.workunit.client.1.smithi144.stdout:1/281: mknod d7/c5a 0 2024-05-15T15:54:06.176 INFO:tasks.workunit.client.1.smithi144.stdout:6/229: unlink d7/d1e/d28/c38 0 2024-05-15T15:54:06.177 INFO:tasks.workunit.client.1.smithi144.stdout:5/278: rmdir d0/dc 39 2024-05-15T15:54:06.178 INFO:tasks.workunit.client.1.smithi144.stdout:0/258: mknod d4/d2d/d48/d5b/c64 0 2024-05-15T15:54:06.179 INFO:tasks.workunit.client.1.smithi144.stdout:0/259: stat d4/df/d1f/d24/d2b/d55 0 2024-05-15T15:54:06.180 INFO:tasks.workunit.client.1.smithi144.stdout:9/270: dread d0/d8/d5e/d44/d10/d19/f33 [0,4194304] 0 2024-05-15T15:54:06.180 INFO:tasks.workunit.client.1.smithi144.stdout:8/273: link d2/d1a/f3d d2/de/d3f/f5f 0 2024-05-15T15:54:06.180 INFO:tasks.workunit.client.1.smithi144.stdout:8/274: chown d2/de/d36/f2c 0 1 2024-05-15T15:54:06.181 INFO:tasks.workunit.client.1.smithi144.stdout:3/259: rename d0/d11/l3a to d0/d1/d56/l60 0 2024-05-15T15:54:06.182 INFO:tasks.workunit.client.1.smithi144.stdout:1/282: unlink f6 0 2024-05-15T15:54:06.182 INFO:tasks.workunit.client.1.smithi144.stdout:6/230: read d7/db/f1d [2306768,71520] 0 2024-05-15T15:54:06.183 INFO:tasks.workunit.client.1.smithi144.stdout:2/269: symlink d3/d7/d20/d40/l59 0 2024-05-15T15:54:06.184 INFO:tasks.workunit.client.1.smithi144.stdout:1/283: write d7/d1d/d33/d1e/d34/f38 [612903,43334] 0 2024-05-15T15:54:06.184 INFO:tasks.workunit.client.1.smithi144.stdout:6/231: truncate d7/f4b 4258977 0 2024-05-15T15:54:06.184 INFO:tasks.workunit.client.1.smithi144.stdout:0/260: creat d4/d6/d3a/d5d/f65 x:0 0 0 2024-05-15T15:54:06.184 INFO:tasks.workunit.client.1.smithi144.stdout:6/232: chown d7/ff 126468 1 2024-05-15T15:54:06.185 INFO:tasks.workunit.client.1.smithi144.stdout:3/260: chown d0/f2c 217622 1 2024-05-15T15:54:06.187 INFO:tasks.workunit.client.1.smithi144.stdout:9/271: mknod d0/d8/d5e/d44/d16/d5a/c66 0 2024-05-15T15:54:06.188 INFO:tasks.workunit.client.1.smithi144.stdout:0/261: mknod d4/d6/d3a/d40/c66 0 2024-05-15T15:54:06.190 INFO:tasks.workunit.client.1.smithi144.stdout:8/275: link d2/de/d36/f2c d2/d24/d2d/d30/f60 0 2024-05-15T15:54:06.200 INFO:tasks.workunit.client.1.smithi144.stdout:8/276: creat d2/d1a/d38/d4b/f61 x:0 0 0 2024-05-15T15:54:06.202 INFO:tasks.workunit.client.1.smithi144.stdout:8/277: creat d2/d53/f62 x:0 0 0 2024-05-15T15:54:06.206 INFO:tasks.workunit.client.1.smithi144.stdout:3/261: dread d0/d11/f14 [0,4194304] 0 2024-05-15T15:54:06.208 INFO:tasks.workunit.client.1.smithi144.stdout:8/278: dwrite d2/de/fc [0,4194304] 0 2024-05-15T15:54:06.209 INFO:tasks.workunit.client.1.smithi144.stdout:8/279: chown d2/d1a 61690 1 2024-05-15T15:54:06.209 INFO:tasks.workunit.client.1.smithi144.stdout:0/262: dread d4/d6/fb [0,4194304] 0 2024-05-15T15:54:06.210 INFO:tasks.workunit.client.0.smithi049.stdout:2/361: mkdir d3/db/d1d/d1e/d2b/d6b 0 2024-05-15T15:54:06.210 INFO:tasks.workunit.client.1.smithi144.stdout:3/262: creat d0/d11/f61 x:0 0 0 2024-05-15T15:54:06.211 INFO:tasks.workunit.client.0.smithi049.stdout:1/358: dwrite d0/d4/dd/de/d1e/d16/f30 [0,4194304] 0 2024-05-15T15:54:06.213 INFO:tasks.workunit.client.0.smithi049.stdout:5/314: mknod dc/d4a/d31/c5a 0 2024-05-15T15:54:06.216 INFO:tasks.workunit.client.1.smithi144.stdout:3/263: symlink d0/d11/d24/l62 0 2024-05-15T15:54:06.216 INFO:tasks.workunit.client.0.smithi049.stdout:0/302: read dc/d13/f26 [21809,64935] 0 2024-05-15T15:54:06.217 INFO:tasks.workunit.client.1.smithi144.stdout:3/264: stat d0/d1/d17/f20 0 2024-05-15T15:54:06.219 INFO:tasks.workunit.client.1.smithi144.stdout:6/233: dread f6 [0,4194304] 0 2024-05-15T15:54:06.224 INFO:tasks.workunit.client.1.smithi144.stdout:0/263: dread d4/dc/f17 [0,4194304] 0 2024-05-15T15:54:06.225 INFO:tasks.workunit.client.1.smithi144.stdout:0/264: write d4/df/f46 [688811,35770] 0 2024-05-15T15:54:06.226 INFO:tasks.workunit.client.0.smithi049.stdout:1/359: mkdir d0/d4/dd/de/d4d/d8b 0 2024-05-15T15:54:06.227 INFO:tasks.workunit.client.1.smithi144.stdout:0/265: write d4/df/d1f/f45 [1242331,93609] 0 2024-05-15T15:54:06.227 INFO:tasks.workunit.client.1.smithi144.stdout:0/266: chown d4/f22 152 1 2024-05-15T15:54:06.229 INFO:tasks.workunit.client.0.smithi049.stdout:1/360: write d0/d4/dd/de/d48/d84/f7d [456518,104576] 0 2024-05-15T15:54:06.229 INFO:tasks.workunit.client.0.smithi049.stdout:0/303: dwrite dc/d13/d14/d2f/f51 [4194304,4194304] 0 2024-05-15T15:54:06.231 INFO:tasks.workunit.client.0.smithi049.stdout:2/362: fsync d3/db/d1d/d1e/d2b/d2e/f48 0 2024-05-15T15:54:06.234 INFO:tasks.workunit.client.1.smithi144.stdout:3/265: symlink d0/d11/d24/l63 0 2024-05-15T15:54:06.238 INFO:tasks.workunit.client.1.smithi144.stdout:8/280: link d2/d24/d2d/l37 d2/de/d36/l63 0 2024-05-15T15:54:06.239 INFO:tasks.workunit.client.1.smithi144.stdout:8/281: fdatasync d2/de/d36/f4a 0 2024-05-15T15:54:06.241 INFO:tasks.workunit.client.1.smithi144.stdout:9/272: dread d0/d8/d5e/d44/d10/d19/d3e/f3f [0,4194304] 0 2024-05-15T15:54:06.244 INFO:tasks.workunit.client.0.smithi049.stdout:2/363: creat d3/db/d1d/d1e/d2b/d2e/f6c x:0 0 0 2024-05-15T15:54:06.246 INFO:tasks.workunit.client.0.smithi049.stdout:5/315: link dc/dd/f14 dc/dd/d39/f5b 0 2024-05-15T15:54:06.252 INFO:tasks.workunit.client.1.smithi144.stdout:8/282: symlink d2/d1a/d38/d4b/l64 0 2024-05-15T15:54:06.253 INFO:tasks.workunit.client.1.smithi144.stdout:8/283: write d2/de/d36/f48 [1128367,61550] 0 2024-05-15T15:54:06.253 INFO:tasks.workunit.client.0.smithi049.stdout:1/361: link d0/d4/d63/f83 d0/f8c 0 2024-05-15T15:54:06.255 INFO:tasks.workunit.client.0.smithi049.stdout:1/362: write d0/d4/dd/de/d1e/d2e/f88 [678538,54893] 0 2024-05-15T15:54:06.256 INFO:tasks.workunit.client.0.smithi049.stdout:2/364: creat d3/db/d1d/d1e/d2b/d3b/d42/f6d x:0 0 0 2024-05-15T15:54:06.257 INFO:tasks.workunit.client.1.smithi144.stdout:9/273: creat d0/d8/d5e/d44/d10/d19/f67 x:0 0 0 2024-05-15T15:54:06.258 INFO:tasks.workunit.client.0.smithi049.stdout:2/365: dread - d3/db/d1d/d1e/d2b/d2e/f6c zero size 2024-05-15T15:54:06.259 INFO:tasks.workunit.client.0.smithi049.stdout:5/316: mkdir dc/d4a/d31/d5c 0 2024-05-15T15:54:06.263 INFO:tasks.workunit.client.1.smithi144.stdout:9/274: dwrite d0/d8/d5e/d44/d3c/f61 [4194304,4194304] 0 2024-05-15T15:54:06.265 INFO:tasks.workunit.client.1.smithi144.stdout:1/284: sync 2024-05-15T15:54:06.266 INFO:tasks.workunit.client.0.smithi049.stdout:1/363: rename d0/d4/dd/de/d48/d84/f7d to d0/d4/dd/d5d/d65/d72/f8d 0 2024-05-15T15:54:06.268 INFO:tasks.workunit.client.0.smithi049.stdout:5/317: readlink dc/d4a/l58 0 2024-05-15T15:54:06.277 INFO:tasks.workunit.client.1.smithi144.stdout:6/234: rename d7/db/d3a/c40 to d7/c58 0 2024-05-15T15:54:06.278 INFO:tasks.workunit.client.0.smithi049.stdout:1/364: symlink d0/d4/d63/l8e 0 2024-05-15T15:54:06.280 INFO:tasks.workunit.client.0.smithi049.stdout:1/365: dread - d0/d4/dd/de/d1e/d2e/d3e/f3d zero size 2024-05-15T15:54:06.280 INFO:tasks.workunit.client.0.smithi049.stdout:5/318: creat dc/dd/d47/f5d x:0 0 0 2024-05-15T15:54:06.281 INFO:tasks.workunit.client.1.smithi144.stdout:9/275: mknod d0/d8/d5e/d44/d16/d5a/c68 0 2024-05-15T15:54:06.282 INFO:tasks.workunit.client.0.smithi049.stdout:3/372: stat d12/d14/d32/c39 0 2024-05-15T15:54:06.283 INFO:tasks.workunit.client.0.smithi049.stdout:1/366: truncate d0/d4/dd/de/d1e/d2e/f41 4367635 0 2024-05-15T15:54:06.283 INFO:tasks.workunit.client.0.smithi049.stdout:3/373: chown d12/l13 28 1 2024-05-15T15:54:06.285 INFO:tasks.workunit.client.0.smithi049.stdout:5/319: rename dc/d29/f3c to dc/d4a/d42/f5e 0 2024-05-15T15:54:06.286 INFO:tasks.workunit.client.1.smithi144.stdout:8/284: symlink d2/de/d49/l65 0 2024-05-15T15:54:06.289 INFO:tasks.workunit.client.1.smithi144.stdout:0/267: getdents d4/d6/d3a/d4c 0 2024-05-15T15:54:06.289 INFO:tasks.workunit.client.1.smithi144.stdout:6/235: dread d7/db/d35/f47 [0,4194304] 0 2024-05-15T15:54:06.290 INFO:tasks.workunit.client.1.smithi144.stdout:0/268: write d4/df/d1f/f2c [1292826,25723] 0 2024-05-15T15:54:06.290 INFO:tasks.workunit.client.1.smithi144.stdout:6/236: stat d7/db/d11 0 2024-05-15T15:54:06.291 INFO:tasks.workunit.client.1.smithi144.stdout:0/269: write d4/df/f43 [1070368,10077] 0 2024-05-15T15:54:06.293 INFO:tasks.workunit.client.0.smithi049.stdout:3/374: chown d12/d14/f4e 16747967 1 2024-05-15T15:54:06.296 INFO:tasks.workunit.client.0.smithi049.stdout:5/320: mknod dc/dd/d47/c5f 0 2024-05-15T15:54:06.298 INFO:tasks.workunit.client.1.smithi144.stdout:1/285: link d7/df/f15 d7/d1d/d33/d55/f5b 0 2024-05-15T15:54:06.298 INFO:tasks.workunit.client.1.smithi144.stdout:4/268: write d6/dd/d34/d3b/d30/f2f [66969,88714] 0 2024-05-15T15:54:06.299 INFO:tasks.workunit.client.1.smithi144.stdout:8/285: fdatasync d2/d1a/f3d 0 2024-05-15T15:54:06.300 INFO:tasks.workunit.client.0.smithi049.stdout:3/375: mkdir d12/d30/d63/d68/d70/d75 0 2024-05-15T15:54:06.303 INFO:tasks.workunit.client.0.smithi049.stdout:3/376: symlink d12/d14/d32/l76 0 2024-05-15T15:54:06.305 INFO:tasks.workunit.client.1.smithi144.stdout:8/286: dwrite d2/de/d36/f2f [0,4194304] 0 2024-05-15T15:54:06.311 INFO:tasks.workunit.client.1.smithi144.stdout:6/237: mknod d7/db/d11/d3c/d56/c59 0 2024-05-15T15:54:06.314 INFO:tasks.workunit.client.1.smithi144.stdout:1/286: dwrite d7/d1d/d33/d55/f5b [4194304,4194304] 0 2024-05-15T15:54:06.315 INFO:tasks.workunit.client.1.smithi144.stdout:1/287: truncate d7/d1d/d33/d29/f42 978689 0 2024-05-15T15:54:06.315 INFO:tasks.workunit.client.1.smithi144.stdout:4/269: link d6/d42/d46/d4b/f1c d6/d42/f58 0 2024-05-15T15:54:06.318 INFO:tasks.workunit.client.1.smithi144.stdout:8/287: link d2/d53/f62 d2/d53/f66 0 2024-05-15T15:54:06.319 INFO:tasks.workunit.client.1.smithi144.stdout:8/288: chown d2/d24/d2d/c4f 0 1 2024-05-15T15:54:06.321 INFO:tasks.workunit.client.1.smithi144.stdout:6/238: symlink d7/d1e/l5a 0 2024-05-15T15:54:06.322 INFO:tasks.workunit.client.1.smithi144.stdout:1/288: dread d7/df/f15 [4194304,4194304] 0 2024-05-15T15:54:06.322 INFO:tasks.workunit.client.1.smithi144.stdout:4/270: mkdir d6/dd/d34/d3b/d30/d59 0 2024-05-15T15:54:06.324 INFO:tasks.workunit.client.1.smithi144.stdout:7/274: dwrite d4/df/d25/d40/d47/f4c [0,4194304] 0 2024-05-15T15:54:06.326 INFO:tasks.workunit.client.1.smithi144.stdout:8/289: unlink d2/de/d36/f47 0 2024-05-15T15:54:06.328 INFO:tasks.workunit.client.1.smithi144.stdout:1/289: rmdir d7/df/d24 39 2024-05-15T15:54:06.330 INFO:tasks.workunit.client.1.smithi144.stdout:4/271: chown d6/dd/d2d/f3f 115 1 2024-05-15T15:54:06.330 INFO:tasks.workunit.client.1.smithi144.stdout:7/275: dread d4/df/f32 [0,4194304] 0 2024-05-15T15:54:06.330 INFO:tasks.workunit.client.1.smithi144.stdout:7/276: chown d4/f2f 1 1 2024-05-15T15:54:06.331 INFO:tasks.workunit.client.1.smithi144.stdout:7/277: chown d4/f27 4 1 2024-05-15T15:54:06.332 INFO:tasks.workunit.client.1.smithi144.stdout:5/279: write d0/f22 [3486920,43388] 0 2024-05-15T15:54:06.333 INFO:tasks.workunit.client.1.smithi144.stdout:8/290: stat d2/d3a/l5a 0 2024-05-15T15:54:06.342 INFO:tasks.workunit.client.1.smithi144.stdout:8/291: creat d2/d3a/f67 x:0 0 0 2024-05-15T15:54:06.342 INFO:tasks.workunit.client.1.smithi144.stdout:2/270: write d3/d10/d15/f2f [1920113,76616] 0 2024-05-15T15:54:06.342 INFO:tasks.workunit.client.1.smithi144.stdout:1/290: mkdir d7/d5c 0 2024-05-15T15:54:06.342 INFO:tasks.workunit.client.1.smithi144.stdout:8/292: chown d2/de 126716746 1 2024-05-15T15:54:06.343 INFO:tasks.workunit.client.1.smithi144.stdout:5/280: mknod d0/c55 0 2024-05-15T15:54:06.343 INFO:tasks.workunit.client.1.smithi144.stdout:1/291: truncate d7/d1d/d33/d1e/d4e/f50 410032 0 2024-05-15T15:54:06.344 INFO:tasks.workunit.client.1.smithi144.stdout:1/292: chown d7/d1d/d58 1 1 2024-05-15T15:54:06.344 INFO:tasks.workunit.client.1.smithi144.stdout:6/239: link d7/db/c27 d7/db/d3a/c5b 0 2024-05-15T15:54:06.345 INFO:tasks.workunit.client.1.smithi144.stdout:1/293: write d7/d1d/d33/d41/f19 [3887375,32263] 0 2024-05-15T15:54:06.346 INFO:tasks.workunit.client.1.smithi144.stdout:0/270: sync 2024-05-15T15:54:06.350 INFO:tasks.workunit.client.1.smithi144.stdout:4/272: dread d6/d42/f58 [0,4194304] 0 2024-05-15T15:54:06.350 INFO:tasks.workunit.client.1.smithi144.stdout:2/271: creat d3/d7/d54/f5a x:0 0 0 2024-05-15T15:54:06.353 INFO:tasks.workunit.client.1.smithi144.stdout:0/271: symlink d4/dc/l67 0 2024-05-15T15:54:06.356 INFO:tasks.workunit.client.0.smithi049.stdout:0/304: write dc/d32/f60 [197562,67093] 0 2024-05-15T15:54:06.356 INFO:tasks.workunit.client.1.smithi144.stdout:5/281: dread d0/dc/d1f/f38 [0,4194304] 0 2024-05-15T15:54:06.356 INFO:tasks.workunit.client.1.smithi144.stdout:8/293: dread d2/fd [0,4194304] 0 2024-05-15T15:54:06.357 INFO:tasks.workunit.client.1.smithi144.stdout:8/294: fdatasync d2/f15 0 2024-05-15T15:54:06.358 INFO:tasks.workunit.client.1.smithi144.stdout:6/240: dwrite d7/db/d35/f47 [0,4194304] 0 2024-05-15T15:54:06.359 INFO:tasks.workunit.client.1.smithi144.stdout:6/241: write d7/db/f3f [216234,6233] 0 2024-05-15T15:54:06.360 INFO:tasks.workunit.client.0.smithi049.stdout:0/305: creat dc/d13/d14/d25/d31/f61 x:0 0 0 2024-05-15T15:54:06.362 INFO:tasks.workunit.client.0.smithi049.stdout:0/306: truncate dc/d13/d14/d33/f53 42932 0 2024-05-15T15:54:06.364 INFO:tasks.workunit.client.1.smithi144.stdout:5/282: mknod d0/dc/d12/d4a/c56 0 2024-05-15T15:54:06.370 INFO:tasks.workunit.client.1.smithi144.stdout:8/295: rename d2/d24/f25 to d2/d24/f68 0 2024-05-15T15:54:06.371 INFO:tasks.workunit.client.1.smithi144.stdout:3/266: dwrite d0/d11/d24/f22 [0,4194304] 0 2024-05-15T15:54:06.371 INFO:tasks.workunit.client.0.smithi049.stdout:2/366: dwrite d3/d8/f14 [0,4194304] 0 2024-05-15T15:54:06.372 INFO:tasks.workunit.client.1.smithi144.stdout:3/267: truncate d0/d1/d56/f58 797353 0 2024-05-15T15:54:06.373 INFO:tasks.workunit.client.1.smithi144.stdout:6/242: mknod d7/db/d11/d3c/c5c 0 2024-05-15T15:54:06.377 INFO:tasks.workunit.client.1.smithi144.stdout:5/283: mkdir d0/dc/d24/d57 0 2024-05-15T15:54:06.378 INFO:tasks.workunit.client.0.smithi049.stdout:2/367: mknod d3/da/d15/c6e 0 2024-05-15T15:54:06.382 INFO:tasks.workunit.client.0.smithi049.stdout:0/307: getdents dc/d13/d14/d2f 0 2024-05-15T15:54:06.382 INFO:tasks.workunit.client.1.smithi144.stdout:6/243: dwrite d7/ff [0,4194304] 0 2024-05-15T15:54:06.383 INFO:tasks.workunit.client.1.smithi144.stdout:6/244: stat d7/d1e/d28/f3d 0 2024-05-15T15:54:06.384 INFO:tasks.workunit.client.0.smithi049.stdout:2/368: creat d3/db/d1d/d1e/d2b/d3b/d42/f6f x:0 0 0 2024-05-15T15:54:06.386 INFO:tasks.workunit.client.1.smithi144.stdout:9/276: write d0/d8/d5e/d44/d10/f17 [527093,111192] 0 2024-05-15T15:54:06.387 INFO:tasks.workunit.client.0.smithi049.stdout:0/308: symlink dc/d13/d14/d2a/l62 0 2024-05-15T15:54:06.389 INFO:tasks.workunit.client.0.smithi049.stdout:1/367: dwrite d0/d4/f24 [0,4194304] 0 2024-05-15T15:54:06.391 INFO:tasks.workunit.client.1.smithi144.stdout:8/296: write d2/d1a/f3d [181559,97567] 0 2024-05-15T15:54:06.391 INFO:tasks.workunit.client.0.smithi049.stdout:1/368: write d0/d4/dd/de/d4e/f7e [59874,13186] 0 2024-05-15T15:54:06.393 INFO:tasks.workunit.client.1.smithi144.stdout:2/272: getdents d3/d7/d20/d22 0 2024-05-15T15:54:06.393 INFO:tasks.workunit.client.1.smithi144.stdout:5/284: fsync d0/dc/d1f/f3b 0 2024-05-15T15:54:06.394 INFO:tasks.workunit.client.0.smithi049.stdout:0/309: symlink dc/d13/d2b/l63 0 2024-05-15T15:54:06.394 INFO:tasks.workunit.client.1.smithi144.stdout:2/273: chown d3/d7/d20/c33 331968728 1 2024-05-15T15:54:06.394 INFO:tasks.workunit.client.1.smithi144.stdout:0/272: link d4/df/d1f/d24/c54 d4/d6/d3a/d4c/c68 0 2024-05-15T15:54:06.395 INFO:tasks.workunit.client.1.smithi144.stdout:0/273: dread - d4/df/d1f/d24/d2b/d55/f60 zero size 2024-05-15T15:54:06.396 INFO:tasks.workunit.client.1.smithi144.stdout:6/245: mknod d7/db/d11/d3c/d56/c5d 0 2024-05-15T15:54:06.396 INFO:tasks.workunit.client.0.smithi049.stdout:5/321: truncate f9 821926 0 2024-05-15T15:54:06.399 INFO:tasks.workunit.client.1.smithi144.stdout:9/277: creat d0/d8/d50/f69 x:0 0 0 2024-05-15T15:54:06.400 INFO:tasks.workunit.client.1.smithi144.stdout:9/278: write d0/f18 [2161233,104889] 0 2024-05-15T15:54:06.402 INFO:tasks.workunit.client.0.smithi049.stdout:5/322: creat dc/d4a/d42/f60 x:0 0 0 2024-05-15T15:54:06.403 INFO:tasks.workunit.client.0.smithi049.stdout:5/323: truncate dc/d4a/f57 964340 0 2024-05-15T15:54:06.403 INFO:tasks.workunit.client.0.smithi049.stdout:5/324: readlink dc/d29/l40 0 2024-05-15T15:54:06.408 INFO:tasks.workunit.client.0.smithi049.stdout:5/325: creat dc/d4a/d31/f61 x:0 0 0 2024-05-15T15:54:06.408 INFO:tasks.workunit.client.0.smithi049.stdout:5/326: stat dc/dd/d10/l23 0 2024-05-15T15:54:06.414 INFO:tasks.workunit.client.0.smithi049.stdout:5/327: creat dc/d4a/f62 x:0 0 0 2024-05-15T15:54:06.415 INFO:tasks.workunit.client.0.smithi049.stdout:5/328: readlink dc/dd/d10/l27 0 2024-05-15T15:54:06.422 INFO:tasks.workunit.client.1.smithi144.stdout:0/274: creat d4/df/d1f/d24/f69 x:0 0 0 2024-05-15T15:54:06.423 INFO:tasks.workunit.client.1.smithi144.stdout:7/278: write d4/df/d25/f35 [2249427,58188] 0 2024-05-15T15:54:06.424 INFO:tasks.workunit.client.1.smithi144.stdout:4/273: truncate f0 3391956 0 2024-05-15T15:54:06.424 INFO:tasks.workunit.client.1.smithi144.stdout:5/285: truncate d0/dc/d12/d31/d35/f44 845152 0 2024-05-15T15:54:06.424 INFO:tasks.workunit.client.0.smithi049.stdout:2/369: link d3/f11 d3/db/d5b/f70 0 2024-05-15T15:54:06.425 INFO:tasks.workunit.client.1.smithi144.stdout:9/279: mknod d0/d8/d50/c6a 0 2024-05-15T15:54:06.425 INFO:tasks.workunit.client.0.smithi049.stdout:2/370: chown d3/db/f23 207 1 2024-05-15T15:54:06.426 INFO:tasks.workunit.client.1.smithi144.stdout:0/275: truncate d4/d6/d3a/d5d/f65 579676 0 2024-05-15T15:54:06.427 INFO:tasks.workunit.client.1.smithi144.stdout:9/280: fdatasync d0/d8/d5e/d44/d16/f46 0 2024-05-15T15:54:06.427 INFO:tasks.workunit.client.1.smithi144.stdout:6/246: creat d7/db/d11/d55/f5e x:0 0 0 2024-05-15T15:54:06.429 INFO:tasks.workunit.client.1.smithi144.stdout:0/276: write d4/df/f16 [1496970,17605] 0 2024-05-15T15:54:06.430 INFO:tasks.workunit.client.1.smithi144.stdout:4/274: dread d6/d42/d46/d4b/d1e/f4f [0,4194304] 0 2024-05-15T15:54:06.431 INFO:tasks.workunit.client.1.smithi144.stdout:3/268: getdents d0/d11/d3b 0 2024-05-15T15:54:06.432 INFO:tasks.workunit.client.1.smithi144.stdout:9/281: stat d0/d8/d5e/d44/d3c/f61 0 2024-05-15T15:54:06.433 INFO:tasks.workunit.client.1.smithi144.stdout:5/286: creat d0/dc/d1f/d50/f58 x:0 0 0 2024-05-15T15:54:06.433 INFO:tasks.workunit.client.1.smithi144.stdout:7/279: dwrite d4/df/d25/f3a [0,4194304] 0 2024-05-15T15:54:06.433 INFO:tasks.workunit.client.1.smithi144.stdout:3/269: fsync d0/f3d 0 2024-05-15T15:54:06.434 INFO:tasks.workunit.client.1.smithi144.stdout:2/274: link d3/d10/f2e d3/d7/d20/f5b 0 2024-05-15T15:54:06.434 INFO:tasks.workunit.client.1.smithi144.stdout:7/280: chown d4/d3c/f57 0 1 2024-05-15T15:54:06.438 INFO:tasks.workunit.client.1.smithi144.stdout:0/277: unlink d4/df/d1f/d24/d2b/c63 0 2024-05-15T15:54:06.441 INFO:tasks.workunit.client.0.smithi049.stdout:5/329: rmdir dc/d4a/d42 39 2024-05-15T15:54:06.443 INFO:tasks.workunit.client.0.smithi049.stdout:0/310: write dc/d13/d3d/f5d [4206901,26377] 0 2024-05-15T15:54:06.443 INFO:tasks.workunit.client.1.smithi144.stdout:6/247: mkdir d7/db/d5f 0 2024-05-15T15:54:06.445 INFO:tasks.workunit.client.0.smithi049.stdout:1/369: dwrite d0/f8c [0,4194304] 0 2024-05-15T15:54:06.446 INFO:tasks.workunit.client.0.smithi049.stdout:2/371: dread d3/db/d5b/f70 [0,4194304] 0 2024-05-15T15:54:06.447 INFO:tasks.workunit.client.0.smithi049.stdout:1/370: chown d0/d4/dd/de/d36/l80 175156191 1 2024-05-15T15:54:06.448 INFO:tasks.workunit.client.0.smithi049.stdout:1/371: read d0/d4/dd/de/f17 [9287106,114406] 0 2024-05-15T15:54:06.452 INFO:tasks.workunit.client.0.smithi049.stdout:5/330: rename dc/d4a/d31/f35 to dc/d29/f63 0 2024-05-15T15:54:06.455 INFO:tasks.workunit.client.1.smithi144.stdout:3/270: rename d0/f3d to d0/d11/d3b/f64 0 2024-05-15T15:54:06.456 INFO:tasks.workunit.client.0.smithi049.stdout:0/311: mknod dc/d13/d14/d25/d31/d38/c64 0 2024-05-15T15:54:06.457 INFO:tasks.workunit.client.0.smithi049.stdout:1/372: dwrite d0/d4/dd/de/d4e/f76 [0,4194304] 0 2024-05-15T15:54:06.457 INFO:tasks.workunit.client.1.smithi144.stdout:7/281: creat d4/d41/f5a x:0 0 0 2024-05-15T15:54:06.458 INFO:tasks.workunit.client.1.smithi144.stdout:7/282: write d4/f51 [701822,109971] 0 2024-05-15T15:54:06.460 INFO:tasks.workunit.client.0.smithi049.stdout:0/312: truncate dc/d13/d14/d25/d31/f61 449985 0 2024-05-15T15:54:06.463 INFO:tasks.workunit.client.1.smithi144.stdout:4/275: getdents d6/dd/d34/d3b/d30/db/d1d/d40/d57 0 2024-05-15T15:54:06.465 INFO:tasks.workunit.client.0.smithi049.stdout:1/373: dread d0/d4/dd/de/d48/f4a [0,4194304] 0 2024-05-15T15:54:06.466 INFO:tasks.workunit.client.0.smithi049.stdout:1/374: write d0/d4/dd/de/d1e/f37 [3225162,112864] 0 2024-05-15T15:54:06.467 INFO:tasks.workunit.client.1.smithi144.stdout:6/248: creat d7/db/d11/f60 x:0 0 0 2024-05-15T15:54:06.472 INFO:tasks.workunit.client.1.smithi144.stdout:2/275: symlink d3/d7/d20/l5c 0 2024-05-15T15:54:06.474 INFO:tasks.workunit.client.1.smithi144.stdout:3/271: mknod d0/d11/d24/c65 0 2024-05-15T15:54:06.478 INFO:tasks.workunit.client.1.smithi144.stdout:7/283: chown d4/fc 254 1 2024-05-15T15:54:06.479 INFO:tasks.workunit.client.0.smithi049.stdout:1/375: rename d0/d4/d81/f82 to d0/d4/dd/de/d1e/d16/d8a/f8f 0 2024-05-15T15:54:06.480 INFO:tasks.workunit.client.0.smithi049.stdout:1/376: readlink d0/d4/dd/de/d1e/l74 0 2024-05-15T15:54:06.481 INFO:tasks.workunit.client.0.smithi049.stdout:1/377: read - d0/d4/dd/de/d1e/d2e/d3e/f3d zero size 2024-05-15T15:54:06.482 INFO:tasks.workunit.client.0.smithi049.stdout:5/331: creat dc/f64 x:0 0 0 2024-05-15T15:54:06.483 INFO:tasks.workunit.client.1.smithi144.stdout:6/249: mkdir d7/db/d11/d3c/d61 0 2024-05-15T15:54:06.488 INFO:tasks.workunit.client.1.smithi144.stdout:3/272: rename d0/d11/d24/l62 to d0/d1/d17/l66 0 2024-05-15T15:54:06.490 INFO:tasks.workunit.client.0.smithi049.stdout:5/332: mknod dc/dd/d10/c65 0 2024-05-15T15:54:06.491 INFO:tasks.workunit.client.1.smithi144.stdout:5/287: getdents d0/dc/d24 0 2024-05-15T15:54:06.492 INFO:tasks.workunit.client.0.smithi049.stdout:1/378: dwrite d0/d4/dd/de/d1e/d16/d8a/f8f [0,4194304] 0 2024-05-15T15:54:06.495 INFO:tasks.workunit.client.0.smithi049.stdout:0/313: truncate dc/d13/d48/f4f 207463 0 2024-05-15T15:54:06.496 INFO:tasks.workunit.client.0.smithi049.stdout:0/314: write dc/d32/f39 [319717,22949] 0 2024-05-15T15:54:06.500 INFO:tasks.workunit.client.0.smithi049.stdout:1/379: mkdir d0/d4/dd/de/d1e/d2e/d3e/d90 0 2024-05-15T15:54:06.501 INFO:tasks.workunit.client.1.smithi144.stdout:9/282: getdents d0/d8/d1b/d62 0 2024-05-15T15:54:06.503 INFO:tasks.workunit.client.0.smithi049.stdout:5/333: truncate dc/f16 624495 0 2024-05-15T15:54:06.505 INFO:tasks.workunit.client.1.smithi144.stdout:7/284: rename d4/df/d25/d40/d47 to d4/d41/d5b 0 2024-05-15T15:54:06.506 INFO:tasks.workunit.client.1.smithi144.stdout:3/273: mknod d0/d1/d56/c67 0 2024-05-15T15:54:06.507 INFO:tasks.workunit.client.1.smithi144.stdout:3/274: write d0/d1/f23 [268595,128211] 0 2024-05-15T15:54:06.508 INFO:tasks.workunit.client.0.smithi049.stdout:1/380: rename d0/d4/dd/de/d36/f89 to d0/d4/dd/de/d1e/d2e/d3e/d90/f91 0 2024-05-15T15:54:06.509 INFO:tasks.workunit.client.1.smithi144.stdout:5/288: mkdir d0/dc/d24/d28/d2d/d59 0 2024-05-15T15:54:06.510 INFO:tasks.workunit.client.0.smithi049.stdout:5/334: read dc/dd/f14 [205312,712] 0 2024-05-15T15:54:06.511 INFO:tasks.workunit.client.0.smithi049.stdout:5/335: write dc/d4a/f50 [350169,122836] 0 2024-05-15T15:54:06.511 INFO:tasks.workunit.client.0.smithi049.stdout:0/315: creat dc/d13/d48/d57/f65 x:0 0 0 2024-05-15T15:54:06.512 INFO:tasks.workunit.client.1.smithi144.stdout:6/250: mkdir d7/d62 0 2024-05-15T15:54:06.512 INFO:tasks.workunit.client.0.smithi049.stdout:0/316: chown dc/d13/d48/f50 3028 1 2024-05-15T15:54:06.513 INFO:tasks.workunit.client.1.smithi144.stdout:9/283: mkdir d0/d8/d1b/d62/d6b 0 2024-05-15T15:54:06.514 INFO:tasks.workunit.client.1.smithi144.stdout:7/285: symlink d4/d41/d5b/l5c 0 2024-05-15T15:54:06.515 INFO:tasks.workunit.client.0.smithi049.stdout:1/381: rename d0/d4/d63/c85 to d0/d4/dd/de/d48/c92 0 2024-05-15T15:54:06.516 INFO:tasks.workunit.client.0.smithi049.stdout:1/382: dread - d0/d4/d63/f70 zero size 2024-05-15T15:54:06.518 INFO:tasks.workunit.client.1.smithi144.stdout:6/251: dwrite d7/db/d11/d3c/f4d [0,4194304] 0 2024-05-15T15:54:06.519 INFO:tasks.workunit.client.0.smithi049.stdout:0/317: mknod dc/d13/d14/d25/d31/d38/c66 0 2024-05-15T15:54:06.520 INFO:tasks.workunit.client.1.smithi144.stdout:5/289: fsync d0/dc/d2f/d34/d3e/f48 0 2024-05-15T15:54:06.520 INFO:tasks.workunit.client.0.smithi049.stdout:0/318: read fa [120143,18190] 0 2024-05-15T15:54:06.522 INFO:tasks.workunit.client.0.smithi049.stdout:1/383: fdatasync d0/d4/dd/de/d1e/d2e/d3e/d90/f91 0 2024-05-15T15:54:06.524 INFO:tasks.workunit.client.1.smithi144.stdout:4/276: getdents d6/dd/d2e 0 2024-05-15T15:54:06.524 INFO:tasks.workunit.client.0.smithi049.stdout:1/384: chown d0/d4/d63/f64 492018375 1 2024-05-15T15:54:06.524 INFO:tasks.workunit.client.1.smithi144.stdout:4/277: chown d6/d49/c56 163880390 1 2024-05-15T15:54:06.528 INFO:tasks.workunit.client.1.smithi144.stdout:9/284: stat d0/d8/l5c 0 2024-05-15T15:54:06.529 INFO:tasks.workunit.client.0.smithi049.stdout:0/319: dwrite dc/d32/f39 [0,4194304] 0 2024-05-15T15:54:06.530 INFO:tasks.workunit.client.0.smithi049.stdout:0/320: stat dc/d13/d3d/f5d 0 2024-05-15T15:54:06.531 INFO:tasks.workunit.client.0.smithi049.stdout:0/321: fdatasync dc/d13/d14/d2f/f24 0 2024-05-15T15:54:06.532 INFO:tasks.workunit.client.1.smithi144.stdout:7/286: rmdir d4/d41/d5b 39 2024-05-15T15:54:06.532 INFO:tasks.workunit.client.0.smithi049.stdout:0/322: chown dc/d13/d48 0 1 2024-05-15T15:54:06.533 INFO:tasks.workunit.client.0.smithi049.stdout:0/323: write dc/d13/d14/f1c [2315791,75759] 0 2024-05-15T15:54:06.536 INFO:tasks.workunit.client.0.smithi049.stdout:1/385: unlink d0/d4/f8 0 2024-05-15T15:54:06.540 INFO:tasks.workunit.client.0.smithi049.stdout:0/324: dwrite dc/d13/d14/d2f/f51 [4194304,4194304] 0 2024-05-15T15:54:06.542 INFO:tasks.workunit.client.0.smithi049.stdout:8/330: sync 2024-05-15T15:54:06.546 INFO:tasks.workunit.client.1.smithi144.stdout:4/278: mknod d6/d42/d46/d4b/d1e/c5a 0 2024-05-15T15:54:06.547 INFO:tasks.workunit.client.0.smithi049.stdout:5/336: dwrite dc/dd/d39/f5b [0,4194304] 0 2024-05-15T15:54:06.548 INFO:tasks.workunit.client.1.smithi144.stdout:9/285: symlink d0/d8/l6c 0 2024-05-15T15:54:06.550 INFO:tasks.workunit.client.0.smithi049.stdout:1/386: mknod d0/d4/dd/de/d48/d5c/c93 0 2024-05-15T15:54:06.551 INFO:tasks.workunit.client.1.smithi144.stdout:7/287: rmdir d4/df/d25/d40 39 2024-05-15T15:54:06.553 INFO:tasks.workunit.client.1.smithi144.stdout:3/275: creat d0/d1/f68 x:0 0 0 2024-05-15T15:54:06.553 INFO:tasks.workunit.client.1.smithi144.stdout:3/276: fdatasync d0/d11/d24/f57 0 2024-05-15T15:54:06.554 INFO:tasks.workunit.client.0.smithi049.stdout:8/331: mkdir d2/d4f/d5d/d63 0 2024-05-15T15:54:06.555 INFO:tasks.workunit.client.0.smithi049.stdout:8/332: truncate d2/d14/f4c 536949 0 2024-05-15T15:54:06.556 INFO:tasks.workunit.client.1.smithi144.stdout:6/252: truncate d7/db/d11/f1b 2984736 0 2024-05-15T15:54:06.559 INFO:tasks.workunit.client.1.smithi144.stdout:4/279: mknod d6/d42/c5b 0 2024-05-15T15:54:06.560 INFO:tasks.workunit.client.1.smithi144.stdout:9/286: creat d0/d8/d5e/d44/d16/f6d x:0 0 0 2024-05-15T15:54:06.561 INFO:tasks.workunit.client.1.smithi144.stdout:7/288: mkdir d4/df/d25/d38/d5d 0 2024-05-15T15:54:06.561 INFO:tasks.workunit.client.1.smithi144.stdout:3/277: dwrite d0/d11/d3b/f45 [0,4194304] 0 2024-05-15T15:54:06.562 INFO:tasks.workunit.client.1.smithi144.stdout:6/253: symlink d7/db/d3a/l63 0 2024-05-15T15:54:06.565 INFO:tasks.workunit.client.1.smithi144.stdout:4/280: creat d6/d49/f5c x:0 0 0 2024-05-15T15:54:06.565 INFO:tasks.workunit.client.1.smithi144.stdout:9/287: symlink d0/d8/d50/l6e 0 2024-05-15T15:54:06.566 INFO:tasks.workunit.client.1.smithi144.stdout:3/278: creat d0/f69 x:0 0 0 2024-05-15T15:54:06.566 INFO:tasks.workunit.client.1.smithi144.stdout:6/254: symlink d7/db/d11/d3c/d56/l64 0 2024-05-15T15:54:06.567 INFO:tasks.workunit.client.1.smithi144.stdout:1/294: sync 2024-05-15T15:54:06.568 INFO:tasks.workunit.client.1.smithi144.stdout:4/281: unlink d6/d49/f55 0 2024-05-15T15:54:06.568 INFO:tasks.workunit.client.1.smithi144.stdout:7/289: mknod d4/d41/d5b/c5e 0 2024-05-15T15:54:06.569 INFO:tasks.workunit.client.1.smithi144.stdout:4/282: fdatasync d6/f29 0 2024-05-15T15:54:06.569 INFO:tasks.workunit.client.1.smithi144.stdout:9/288: creat d0/d8/d1b/f6f x:0 0 0 2024-05-15T15:54:06.570 INFO:tasks.workunit.client.1.smithi144.stdout:1/295: fdatasync d7/df/f10 0 2024-05-15T15:54:06.571 INFO:tasks.workunit.client.1.smithi144.stdout:9/289: fsync d0/f5 0 2024-05-15T15:54:06.572 INFO:tasks.workunit.client.1.smithi144.stdout:7/290: write d4/df/f11 [340788,74144] 0 2024-05-15T15:54:06.574 INFO:tasks.workunit.client.1.smithi144.stdout:4/283: creat d6/dd/d34/d3b/f5d x:0 0 0 2024-05-15T15:54:06.574 INFO:tasks.workunit.client.1.smithi144.stdout:9/290: write d0/f18 [1618019,90751] 0 2024-05-15T15:54:06.574 INFO:tasks.workunit.client.1.smithi144.stdout:1/296: write d7/d1d/d33/d41/fe [3224803,130607] 0 2024-05-15T15:54:06.575 INFO:tasks.workunit.client.1.smithi144.stdout:3/279: dwrite d0/f46 [0,4194304] 0 2024-05-15T15:54:06.576 INFO:tasks.workunit.client.1.smithi144.stdout:9/291: chown d0/c20 693 1 2024-05-15T15:54:06.577 INFO:tasks.workunit.client.1.smithi144.stdout:3/280: readlink d0/d11/l47 0 2024-05-15T15:54:06.577 INFO:tasks.workunit.client.1.smithi144.stdout:7/291: creat d4/df/f5f x:0 0 0 2024-05-15T15:54:06.579 INFO:tasks.workunit.client.1.smithi144.stdout:9/292: readlink d0/d8/d5e/d44/d10/d19/l40 0 2024-05-15T15:54:06.580 INFO:tasks.workunit.client.1.smithi144.stdout:9/293: readlink d0/d8/d50/l6e 0 2024-05-15T15:54:06.580 INFO:tasks.workunit.client.1.smithi144.stdout:3/281: write d0/d1/d17/d39/d40/f50 [1574320,99883] 0 2024-05-15T15:54:06.582 INFO:tasks.workunit.client.1.smithi144.stdout:4/284: getdents d6/dd/d34/d3b/d30/db 0 2024-05-15T15:54:06.583 INFO:tasks.workunit.client.1.smithi144.stdout:9/294: mknod d0/d8/d21/c70 0 2024-05-15T15:54:06.583 INFO:tasks.workunit.client.1.smithi144.stdout:3/282: write d0/d1/d17/d39/f3c [1223090,91307] 0 2024-05-15T15:54:06.584 INFO:tasks.workunit.client.1.smithi144.stdout:4/285: dread - d6/d42/d46/d4b/d1e/d44/f43 zero size 2024-05-15T15:54:06.584 INFO:tasks.workunit.client.1.smithi144.stdout:7/292: getdents d4/d3c 0 2024-05-15T15:54:06.586 INFO:tasks.workunit.client.1.smithi144.stdout:7/293: stat d4/df/d25/d40/f4d 0 2024-05-15T15:54:06.587 INFO:tasks.workunit.client.1.smithi144.stdout:3/283: truncate d0/f1d 2696616 0 2024-05-15T15:54:06.587 INFO:tasks.workunit.client.1.smithi144.stdout:7/294: chown d4/df/f20 63 1 2024-05-15T15:54:06.588 INFO:tasks.workunit.client.1.smithi144.stdout:3/284: fsync d0/d1/d56/f58 0 2024-05-15T15:54:06.588 INFO:tasks.workunit.client.1.smithi144.stdout:6/255: dread f1 [0,4194304] 0 2024-05-15T15:54:06.589 INFO:tasks.workunit.client.1.smithi144.stdout:9/295: link d0/d8/d5e/d44/l37 d0/d8/d21/d29/l71 0 2024-05-15T15:54:06.589 INFO:tasks.workunit.client.1.smithi144.stdout:4/286: getdents d6 0 2024-05-15T15:54:06.593 INFO:tasks.workunit.client.1.smithi144.stdout:9/296: dread d0/d8/d5e/f3a [4194304,4194304] 0 2024-05-15T15:54:06.593 INFO:tasks.workunit.client.0.smithi049.stdout:1/387: rename d0/f49 to d0/d4/dd/de/d48/d5c/f94 0 2024-05-15T15:54:06.594 INFO:tasks.workunit.client.1.smithi144.stdout:9/297: dread - d0/d8/d5e/d44/d10/f58 zero size 2024-05-15T15:54:06.595 INFO:tasks.workunit.client.0.smithi049.stdout:1/388: write d0/d4/dd/de/d4d/f56 [189560,12901] 0 2024-05-15T15:54:06.600 INFO:tasks.workunit.client.1.smithi144.stdout:6/256: rmdir d7/d32 39 2024-05-15T15:54:06.604 INFO:tasks.workunit.client.0.smithi049.stdout:8/333: symlink d2/d18/l64 0 2024-05-15T15:54:06.605 INFO:tasks.workunit.client.0.smithi049.stdout:8/334: stat d2/de/d27/c5f 0 2024-05-15T15:54:06.607 INFO:tasks.workunit.client.0.smithi049.stdout:1/389: creat d0/d4/d63/f95 x:0 0 0 2024-05-15T15:54:06.609 INFO:tasks.workunit.client.1.smithi144.stdout:9/298: creat d0/d8/d50/f72 x:0 0 0 2024-05-15T15:54:06.610 INFO:tasks.workunit.client.1.smithi144.stdout:9/299: write d0/d8/d5e/d44/d16/f6d [864801,29501] 0 2024-05-15T15:54:06.610 INFO:tasks.workunit.client.0.smithi049.stdout:0/325: getdents dc/d13/d14/d25/d31 0 2024-05-15T15:54:06.611 INFO:tasks.workunit.client.1.smithi144.stdout:9/300: chown d0/d8/d5e/d44/d10/d19/c57 7 1 2024-05-15T15:54:06.612 INFO:tasks.workunit.client.1.smithi144.stdout:9/301: fdatasync d0/d8/d5e/d44/d10/d19/d3e/f65 0 2024-05-15T15:54:06.615 INFO:tasks.workunit.client.0.smithi049.stdout:5/337: write dc/dd/f14 [4688537,58674] 0 2024-05-15T15:54:06.616 INFO:tasks.workunit.client.0.smithi049.stdout:5/338: dread - dc/dd/d10/f59 zero size 2024-05-15T15:54:06.617 INFO:tasks.workunit.client.0.smithi049.stdout:1/390: rename d0/d4/dd/de/f73 to d0/d4/dd/f96 0 2024-05-15T15:54:06.618 INFO:tasks.workunit.client.1.smithi144.stdout:5/290: truncate d0/dc/d12/f13 3067213 0 2024-05-15T15:54:06.620 INFO:tasks.workunit.client.1.smithi144.stdout:4/287: creat d6/dd/d34/d3b/d30/db/d1d/d40/d57/f5e x:0 0 0 2024-05-15T15:54:06.621 INFO:tasks.workunit.client.0.smithi049.stdout:8/335: fsync d2/d18/f3c 0 2024-05-15T15:54:06.623 INFO:tasks.workunit.client.1.smithi144.stdout:7/295: rmdir d4/df/d25 39 2024-05-15T15:54:06.631 INFO:tasks.workunit.client.1.smithi144.stdout:8/297: sync 2024-05-15T15:54:06.631 INFO:tasks.workunit.client.1.smithi144.stdout:0/278: sync 2024-05-15T15:54:06.635 INFO:tasks.workunit.client.1.smithi144.stdout:7/296: rename d4/d41/d5b/c4e to d4/df/d10/c60 0 2024-05-15T15:54:06.636 INFO:tasks.workunit.client.0.smithi049.stdout:5/339: mknod dc/c66 0 2024-05-15T15:54:06.637 INFO:tasks.workunit.client.1.smithi144.stdout:5/291: dread d0/f16 [0,4194304] 0 2024-05-15T15:54:06.639 INFO:tasks.workunit.client.1.smithi144.stdout:8/298: rmdir d2/d24/d2d/d30 39 2024-05-15T15:54:06.640 INFO:tasks.workunit.client.0.smithi049.stdout:8/336: mknod d2/de/d27/d5b/d60/c65 0 2024-05-15T15:54:06.642 INFO:tasks.workunit.client.1.smithi144.stdout:0/279: mkdir d4/d6/d3a/d6a 0 2024-05-15T15:54:06.644 INFO:tasks.workunit.client.1.smithi144.stdout:8/299: dwrite d2/d1a/f31 [0,4194304] 0 2024-05-15T15:54:06.645 INFO:tasks.workunit.client.1.smithi144.stdout:5/292: symlink d0/dc/d1f/l5a 0 2024-05-15T15:54:06.650 INFO:tasks.workunit.client.0.smithi049.stdout:5/340: rename dc/d4a/l58 to dc/d4a/d31/l67 0 2024-05-15T15:54:06.652 INFO:tasks.workunit.client.0.smithi049.stdout:8/337: creat d2/d14/d1d/d3e/f66 x:0 0 0 2024-05-15T15:54:06.653 INFO:tasks.workunit.client.0.smithi049.stdout:8/338: chown d2/d14/d1b/c33 396240 1 2024-05-15T15:54:06.654 INFO:tasks.workunit.client.1.smithi144.stdout:7/297: creat d4/df/d25/d38/f61 x:0 0 0 2024-05-15T15:54:06.654 INFO:tasks.workunit.client.1.smithi144.stdout:7/298: write d4/df/f11 [3048685,113145] 0 2024-05-15T15:54:06.655 INFO:tasks.workunit.client.0.smithi049.stdout:5/341: dread - dc/d4a/d42/f60 zero size 2024-05-15T15:54:06.657 INFO:tasks.workunit.client.0.smithi049.stdout:5/342: write dc/dd/d47/f54 [803776,53297] 0 2024-05-15T15:54:06.657 INFO:tasks.workunit.client.1.smithi144.stdout:8/300: creat d2/de/d49/f69 x:0 0 0 2024-05-15T15:54:06.658 INFO:tasks.workunit.client.0.smithi049.stdout:5/343: chown dc/d4a/d42/f53 31506783 1 2024-05-15T15:54:06.659 INFO:tasks.workunit.client.1.smithi144.stdout:5/293: mkdir d0/dc/d12/d4a/d5b 0 2024-05-15T15:54:06.659 INFO:tasks.workunit.client.1.smithi144.stdout:0/280: mknod d4/df/d1f/d56/c6b 0 2024-05-15T15:54:06.661 INFO:tasks.workunit.client.1.smithi144.stdout:3/285: dread d0/f1d [0,4194304] 0 2024-05-15T15:54:06.661 INFO:tasks.workunit.client.1.smithi144.stdout:0/281: read d4/df/d1f/f45 [2644920,92754] 0 2024-05-15T15:54:06.663 INFO:tasks.workunit.client.1.smithi144.stdout:7/299: mkdir d4/df/d25/d40/d62 0 2024-05-15T15:54:06.664 INFO:tasks.workunit.client.1.smithi144.stdout:8/301: symlink d2/de/d49/l6a 0 2024-05-15T15:54:06.664 INFO:tasks.workunit.client.1.smithi144.stdout:7/300: write d4/df/d10/f23 [355932,65754] 0 2024-05-15T15:54:06.666 INFO:tasks.workunit.client.1.smithi144.stdout:8/302: write d2/de/d3f/f5f [1657188,59806] 0 2024-05-15T15:54:06.666 INFO:tasks.workunit.client.1.smithi144.stdout:7/301: write d4/df/d25/d38/f54 [320917,98120] 0 2024-05-15T15:54:06.667 INFO:tasks.workunit.client.1.smithi144.stdout:8/303: read - d2/de/d3f/f45 zero size 2024-05-15T15:54:06.668 INFO:tasks.workunit.client.1.smithi144.stdout:6/257: truncate d7/d1e/d28/f57 3037121 0 2024-05-15T15:54:06.668 INFO:tasks.workunit.client.1.smithi144.stdout:8/304: chown d2/d3a 239769 1 2024-05-15T15:54:06.669 INFO:tasks.workunit.client.0.smithi049.stdout:0/326: truncate dc/d13/d3d/f5d 909352 0 2024-05-15T15:54:06.670 INFO:tasks.workunit.client.0.smithi049.stdout:0/327: chown dc/f34 379707117 1 2024-05-15T15:54:06.670 INFO:tasks.workunit.client.1.smithi144.stdout:4/288: write d6/d42/d46/d4b/f1c [1031181,7751] 0 2024-05-15T15:54:06.671 INFO:tasks.workunit.client.1.smithi144.stdout:4/289: stat d6/d49/f5c 0 2024-05-15T15:54:06.672 INFO:tasks.workunit.client.1.smithi144.stdout:9/302: truncate d0/d8/d5e/d44/d10/f17 852647 0 2024-05-15T15:54:06.672 INFO:tasks.workunit.client.1.smithi144.stdout:0/282: mknod d4/d6/c6c 0 2024-05-15T15:54:06.673 INFO:tasks.workunit.client.1.smithi144.stdout:4/290: read d6/dd/d34/f39 [126633,95169] 0 2024-05-15T15:54:06.673 INFO:tasks.workunit.client.1.smithi144.stdout:8/305: dread d2/d1a/f31 [0,4194304] 0 2024-05-15T15:54:06.673 INFO:tasks.workunit.client.0.smithi049.stdout:0/328: creat dc/d13/d14/d2a/f67 x:0 0 0 2024-05-15T15:54:06.674 INFO:tasks.workunit.client.1.smithi144.stdout:2/276: sync 2024-05-15T15:54:06.674 INFO:tasks.workunit.client.1.smithi144.stdout:4/291: fdatasync d6/d42/d46/d4b/f1c 0 2024-05-15T15:54:06.679 INFO:tasks.workunit.client.1.smithi144.stdout:5/294: mkdir d0/dc/d12/d4a/d5b/d5c 0 2024-05-15T15:54:06.681 INFO:tasks.workunit.client.1.smithi144.stdout:7/302: symlink d4/df/d10/l63 0 2024-05-15T15:54:06.681 INFO:tasks.workunit.client.1.smithi144.stdout:6/258: dread d7/db/f39 [0,4194304] 0 2024-05-15T15:54:06.681 INFO:tasks.workunit.client.0.smithi049.stdout:1/391: dwrite d0/d4/dd/de/d36/f40 [0,4194304] 0 2024-05-15T15:54:06.687 INFO:tasks.workunit.client.0.smithi049.stdout:1/392: rmdir d0/d4/dd/de/d1e/d16 39 2024-05-15T15:54:06.694 INFO:tasks.workunit.client.1.smithi144.stdout:8/306: mkdir d2/de/d3f/d6b 0 2024-05-15T15:54:06.694 INFO:tasks.workunit.client.1.smithi144.stdout:8/307: dread - d2/de/d36/f4d zero size 2024-05-15T15:54:06.695 INFO:tasks.workunit.client.0.smithi049.stdout:1/393: dread d0/d4/fa [0,4194304] 0 2024-05-15T15:54:06.696 INFO:tasks.workunit.client.0.smithi049.stdout:5/344: write dc/f19 [4791746,129532] 0 2024-05-15T15:54:06.700 INFO:tasks.workunit.client.0.smithi049.stdout:1/394: creat d0/d4/d81/f97 x:0 0 0 2024-05-15T15:54:06.703 INFO:tasks.workunit.client.0.smithi049.stdout:5/345: dwrite dc/d4a/f50 [0,4194304] 0 2024-05-15T15:54:06.712 INFO:tasks.workunit.client.0.smithi049.stdout:1/395: rename d0/d4/dd/de/d1e/d2e/c54 to d0/d4/dd/de/d48/c98 0 2024-05-15T15:54:06.713 INFO:tasks.workunit.client.0.smithi049.stdout:1/396: write d0/d4/dd/de/d1e/f31 [497239,30457] 0 2024-05-15T15:54:06.715 INFO:tasks.workunit.client.0.smithi049.stdout:1/397: write d0/d4/dd/d5d/d65/d72/f7a [120180,20523] 0 2024-05-15T15:54:06.716 INFO:tasks.workunit.client.0.smithi049.stdout:1/398: chown d0/d4/dd/f28 2 1 2024-05-15T15:54:06.717 INFO:tasks.workunit.client.0.smithi049.stdout:1/399: stat d0/d4/f24 0 2024-05-15T15:54:06.719 INFO:tasks.workunit.client.1.smithi144.stdout:2/277: rmdir d3/d7/d20/d3f 39 2024-05-15T15:54:06.719 INFO:tasks.workunit.client.1.smithi144.stdout:5/295: mkdir d0/dc/d12/d4a/d5d 0 2024-05-15T15:54:06.720 INFO:tasks.workunit.client.1.smithi144.stdout:7/303: mknod d4/df/d10/d48/c64 0 2024-05-15T15:54:06.721 INFO:tasks.workunit.client.1.smithi144.stdout:8/308: truncate d2/f35 1735223 0 2024-05-15T15:54:06.721 INFO:tasks.workunit.client.1.smithi144.stdout:5/296: chown d0/dc/d24/d28/l53 7697187 1 2024-05-15T15:54:06.723 INFO:tasks.workunit.client.1.smithi144.stdout:5/297: dread - d0/dc/d1f/d50/f58 zero size 2024-05-15T15:54:06.724 INFO:tasks.workunit.client.0.smithi049.stdout:0/329: dwrite dc/d13/f17 [4194304,4194304] 0 2024-05-15T15:54:06.724 INFO:tasks.workunit.client.1.smithi144.stdout:9/303: rename d0/d8/d21/l64 to d0/d8/d1b/d62/d6b/l73 0 2024-05-15T15:54:06.727 INFO:tasks.workunit.client.0.smithi049.stdout:5/346: rename dc/d29/l40 to dc/d4a/l68 0 2024-05-15T15:54:06.728 INFO:tasks.workunit.client.0.smithi049.stdout:5/347: write dc/d4a/d42/f48 [209124,102827] 0 2024-05-15T15:54:06.729 INFO:tasks.workunit.client.1.smithi144.stdout:7/304: dwrite d4/df/d25/d38/f3e [0,4194304] 0 2024-05-15T15:54:06.731 INFO:tasks.workunit.client.1.smithi144.stdout:8/309: dwrite d2/de/d36/f4a [0,4194304] 0 2024-05-15T15:54:06.732 INFO:tasks.workunit.client.0.smithi049.stdout:5/348: write dc/dd/f3d [2760144,55961] 0 2024-05-15T15:54:06.733 INFO:tasks.workunit.client.0.smithi049.stdout:5/349: readlink dc/dd/d10/l33 0 2024-05-15T15:54:06.734 INFO:tasks.workunit.client.1.smithi144.stdout:0/283: creat d4/df/f6d x:0 0 0 2024-05-15T15:54:06.737 INFO:tasks.workunit.client.0.smithi049.stdout:1/400: getdents d0/d4/dd/de/d1e/d16/d6e 0 2024-05-15T15:54:06.737 INFO:tasks.workunit.client.1.smithi144.stdout:2/278: rename d3/d10/d15/c2c to d3/d10/d15/c5d 0 2024-05-15T15:54:06.737 INFO:tasks.workunit.client.1.smithi144.stdout:0/284: fdatasync d4/d2d/f44 0 2024-05-15T15:54:06.739 INFO:tasks.workunit.client.0.smithi049.stdout:1/401: write d0/d4/dd/de/d1e/d2e/f88 [598543,25313] 0 2024-05-15T15:54:06.741 INFO:tasks.workunit.client.1.smithi144.stdout:3/286: dwrite d0/d1/fe [0,4194304] 0 2024-05-15T15:54:06.746 INFO:tasks.workunit.client.1.smithi144.stdout:2/279: dread d3/d7/d20/d37/d3a/f55 [0,4194304] 0 2024-05-15T15:54:06.746 INFO:tasks.workunit.client.1.smithi144.stdout:0/285: dwrite d4/df/d14/f5f [0,4194304] 0 2024-05-15T15:54:06.748 INFO:tasks.workunit.client.1.smithi144.stdout:0/286: dread - d4/df/d14/f42 zero size 2024-05-15T15:54:06.752 INFO:tasks.workunit.client.0.smithi049.stdout:1/402: creat d0/d4/dd/de/d1e/d16/d8a/f99 x:0 0 0 2024-05-15T15:54:06.757 INFO:tasks.workunit.client.1.smithi144.stdout:2/280: dwrite d3/d7/d20/d37/d3a/f55 [4194304,4194304] 0 2024-05-15T15:54:06.761 INFO:tasks.workunit.client.0.smithi049.stdout:1/403: symlink d0/d4/d81/l9a 0 2024-05-15T15:54:06.771 INFO:tasks.workunit.client.1.smithi144.stdout:1/297: sync 2024-05-15T15:54:06.772 INFO:tasks.workunit.client.0.smithi049.stdout:5/350: link dc/dd/d10/c41 dc/c69 0 2024-05-15T15:54:06.781 INFO:tasks.workunit.client.1.smithi144.stdout:3/287: symlink d0/d1/d17/d39/d40/l6a 0 2024-05-15T15:54:06.783 INFO:tasks.workunit.client.1.smithi144.stdout:5/298: write d0/dc/d1f/f38 [1914519,109771] 0 2024-05-15T15:54:06.786 INFO:tasks.workunit.client.1.smithi144.stdout:0/287: unlink d4/df/d1f/d24/f59 0 2024-05-15T15:54:06.791 INFO:tasks.workunit.client.1.smithi144.stdout:6/259: write d7/d32/f50 [4522489,67996] 0 2024-05-15T15:54:06.792 INFO:tasks.workunit.client.1.smithi144.stdout:6/260: chown d7/d1e/d28/l34 58107943 1 2024-05-15T15:54:06.792 INFO:tasks.workunit.client.1.smithi144.stdout:4/292: sync 2024-05-15T15:54:06.792 INFO:tasks.workunit.client.0.smithi049.stdout:1/404: getdents d0/d4/dd/de/d1e/d16 0 2024-05-15T15:54:06.793 INFO:tasks.workunit.client.1.smithi144.stdout:4/293: fdatasync d6/dd/d2d/f54 0 2024-05-15T15:54:06.793 INFO:tasks.workunit.client.1.smithi144.stdout:4/294: fsync d6/dd/f22 0 2024-05-15T15:54:06.794 INFO:tasks.workunit.client.1.smithi144.stdout:2/281: mkdir d3/d7/d20/d22/d5e 0 2024-05-15T15:54:06.796 INFO:tasks.workunit.client.0.smithi049.stdout:1/405: creat d0/d4/dd/de/d1e/d16/d8a/f9b x:0 0 0 2024-05-15T15:54:06.797 INFO:tasks.workunit.client.1.smithi144.stdout:8/310: symlink d2/d24/d2d/l6c 0 2024-05-15T15:54:06.798 INFO:tasks.workunit.client.1.smithi144.stdout:1/298: write d7/df/d24/f54 [75111,104662] 0 2024-05-15T15:54:06.799 INFO:tasks.workunit.client.1.smithi144.stdout:1/299: write d7/d1d/d33/d1e/d4e/f50 [798640,30367] 0 2024-05-15T15:54:06.800 INFO:tasks.workunit.client.0.smithi049.stdout:1/406: creat d0/d4/dd/de/d48/d5c/f9c x:0 0 0 2024-05-15T15:54:06.801 INFO:tasks.workunit.client.1.smithi144.stdout:3/288: fsync d0/f1d 0 2024-05-15T15:54:06.803 INFO:tasks.workunit.client.1.smithi144.stdout:5/299: creat d0/dc/d2f/d34/f5e x:0 0 0 2024-05-15T15:54:06.804 INFO:tasks.workunit.client.0.smithi049.stdout:1/407: creat d0/d4/dd/d5d/d65/d72/f9d x:0 0 0 2024-05-15T15:54:06.804 INFO:tasks.workunit.client.0.smithi049.stdout:1/408: fdatasync d0/d4/d63/f95 0 2024-05-15T15:54:06.806 INFO:tasks.workunit.client.1.smithi144.stdout:0/288: rename d4/df/d1f/f62 to d4/d6/d4a/d4d/f6e 0 2024-05-15T15:54:06.806 INFO:tasks.workunit.client.1.smithi144.stdout:0/289: dread - d4/d6/d3a/d5d/f61 zero size 2024-05-15T15:54:06.807 INFO:tasks.workunit.client.0.smithi049.stdout:5/351: truncate dc/dd/f3b 4118630 0 2024-05-15T15:54:06.807 INFO:tasks.workunit.client.1.smithi144.stdout:0/290: chown d4/df/d1f/d24/c27 123 1 2024-05-15T15:54:06.808 INFO:tasks.workunit.client.1.smithi144.stdout:0/291: write d4/d6/f11 [1464148,39661] 0 2024-05-15T15:54:06.808 INFO:tasks.workunit.client.1.smithi144.stdout:7/305: creat d4/df/f65 x:0 0 0 2024-05-15T15:54:06.811 INFO:tasks.workunit.client.1.smithi144.stdout:8/311: rmdir d2 39 2024-05-15T15:54:06.812 INFO:tasks.workunit.client.0.smithi049.stdout:1/409: dwrite d0/d4/dd/de/d1e/d16/d8a/f99 [0,4194304] 0 2024-05-15T15:54:06.813 INFO:tasks.workunit.client.0.smithi049.stdout:1/410: fdatasync d0/d4/dd/de/d36/f53 0 2024-05-15T15:54:06.814 INFO:tasks.workunit.client.1.smithi144.stdout:9/304: truncate d0/f18 1457261 0 2024-05-15T15:54:06.820 INFO:tasks.workunit.client.1.smithi144.stdout:2/282: dwrite d3/d7/d20/d37/d3a/f55 [4194304,4194304] 0 2024-05-15T15:54:06.820 INFO:tasks.workunit.client.0.smithi049.stdout:5/352: creat dc/d4a/f6a x:0 0 0 2024-05-15T15:54:06.821 INFO:tasks.workunit.client.1.smithi144.stdout:0/292: dwrite d4/d2d/f2e [0,4194304] 0 2024-05-15T15:54:06.822 INFO:tasks.workunit.client.0.smithi049.stdout:5/353: stat dc/dd/d39/f56 0 2024-05-15T15:54:06.823 INFO:tasks.workunit.client.1.smithi144.stdout:3/289: creat d0/d11/d3b/f6b x:0 0 0 2024-05-15T15:54:06.824 INFO:tasks.workunit.client.0.smithi049.stdout:1/411: dwrite d0/d4/dd/de/d1e/f37 [4194304,4194304] 0 2024-05-15T15:54:06.826 INFO:tasks.workunit.client.1.smithi144.stdout:3/290: chown d0/d11/d24/c29 539003540 1 2024-05-15T15:54:06.826 INFO:tasks.workunit.client.1.smithi144.stdout:4/295: dread d6/d42/d46/d4b/f4d [0,4194304] 0 2024-05-15T15:54:06.826 INFO:tasks.workunit.client.0.smithi049.stdout:1/412: write d0/d4/f2d [404426,12250] 0 2024-05-15T15:54:06.830 INFO:tasks.workunit.client.1.smithi144.stdout:3/291: write d0/d1/fe [2233302,51587] 0 2024-05-15T15:54:06.830 INFO:tasks.workunit.client.1.smithi144.stdout:0/293: dread d4/df/f46 [0,4194304] 0 2024-05-15T15:54:06.831 INFO:tasks.workunit.client.1.smithi144.stdout:3/292: read d0/d11/d24/f41 [2478917,121337] 0 2024-05-15T15:54:06.832 INFO:tasks.workunit.client.1.smithi144.stdout:6/261: creat d7/db/d5f/f65 x:0 0 0 2024-05-15T15:54:06.835 INFO:tasks.workunit.client.1.smithi144.stdout:4/296: chown d6/dd/d34/d3b/d30/db/d1d/d40/d57 52 1 2024-05-15T15:54:06.836 INFO:tasks.workunit.client.1.smithi144.stdout:4/297: fsync d6/dd/d34/d3b/d30/db/f35 0 2024-05-15T15:54:06.837 INFO:tasks.workunit.client.1.smithi144.stdout:4/298: readlink d6/d42/d46/d4b/l1a 0 2024-05-15T15:54:06.839 INFO:tasks.workunit.client.1.smithi144.stdout:8/312: chown d2/de/d36/f46 0 1 2024-05-15T15:54:06.843 INFO:tasks.workunit.client.1.smithi144.stdout:4/299: read d6/dd/d2d/f48 [550611,45482] 0 2024-05-15T15:54:06.844 INFO:tasks.workunit.client.1.smithi144.stdout:4/300: stat d6/dd/d34/d3b/d30/db/d1d 0 2024-05-15T15:54:06.851 INFO:tasks.workunit.client.1.smithi144.stdout:0/294: symlink d4/df/d1f/d56/l6f 0 2024-05-15T15:54:06.851 INFO:tasks.workunit.client.1.smithi144.stdout:0/295: chown d4/d2d/d48 4 1 2024-05-15T15:54:06.857 INFO:tasks.workunit.client.1.smithi144.stdout:0/296: dwrite d4/df/d14/f42 [0,4194304] 0 2024-05-15T15:54:06.859 INFO:tasks.workunit.client.1.smithi144.stdout:0/297: truncate d4/df/d1f/d24/d2b/d55/f60 927083 0 2024-05-15T15:54:06.859 INFO:tasks.workunit.client.0.smithi049.stdout:5/354: rename dc/d4a/d31/l67 to dc/d29/l6b 0 2024-05-15T15:54:06.862 INFO:tasks.workunit.client.0.smithi049.stdout:1/413: mkdir d0/d4/dd/de/d9e 0 2024-05-15T15:54:06.865 INFO:tasks.workunit.client.1.smithi144.stdout:0/298: dwrite d4/df/d1f/d24/f47 [0,4194304] 0 2024-05-15T15:54:06.866 INFO:tasks.workunit.client.1.smithi144.stdout:2/283: link d3/d7/f27 d3/d10/d15/f5f 0 2024-05-15T15:54:06.868 INFO:tasks.workunit.client.0.smithi049.stdout:1/414: creat d0/d4/d81/f9f x:0 0 0 2024-05-15T15:54:06.869 INFO:tasks.workunit.client.1.smithi144.stdout:6/262: fdatasync d7/d1e/d28/f57 0 2024-05-15T15:54:06.871 INFO:tasks.workunit.client.1.smithi144.stdout:3/293: mkdir d0/d1/d6c 0 2024-05-15T15:54:06.872 INFO:tasks.workunit.client.1.smithi144.stdout:6/263: write d7/db/d5f/f65 [379032,102488] 0 2024-05-15T15:54:06.873 INFO:tasks.workunit.client.1.smithi144.stdout:3/294: stat d0/d11/f26 0 2024-05-15T15:54:06.875 INFO:tasks.workunit.client.0.smithi049.stdout:5/355: rename dc/dd/f3d to dc/f6c 0 2024-05-15T15:54:06.876 INFO:tasks.workunit.client.1.smithi144.stdout:2/284: dwrite d3/d10/d15/f2f [0,4194304] 0 2024-05-15T15:54:06.876 INFO:tasks.workunit.client.1.smithi144.stdout:1/300: truncate d7/d1d/d33/d55/f5b 3038794 0 2024-05-15T15:54:06.876 INFO:tasks.workunit.client.0.smithi049.stdout:5/356: write dc/dd/d47/f54 [901053,27819] 0 2024-05-15T15:54:06.878 INFO:tasks.workunit.client.1.smithi144.stdout:1/301: chown l5 128 1 2024-05-15T15:54:06.879 INFO:tasks.workunit.client.1.smithi144.stdout:7/306: dwrite d4/df/f32 [0,4194304] 0 2024-05-15T15:54:06.879 INFO:tasks.workunit.client.1.smithi144.stdout:3/295: write d0/d1/d17/f33 [856716,37504] 0 2024-05-15T15:54:06.881 INFO:tasks.workunit.client.1.smithi144.stdout:7/307: stat d4/df/f5f 0 2024-05-15T15:54:06.882 INFO:tasks.workunit.client.1.smithi144.stdout:0/299: creat d4/d6/d3a/d4c/f70 x:0 0 0 2024-05-15T15:54:06.884 INFO:tasks.workunit.client.1.smithi144.stdout:3/296: write d0/d1/fe [1035023,84469] 0 2024-05-15T15:54:06.886 INFO:tasks.workunit.client.0.smithi049.stdout:5/357: mknod dc/dd/d39/c6d 0 2024-05-15T15:54:06.888 INFO:tasks.workunit.client.1.smithi144.stdout:7/308: write d4/f51 [1752093,107199] 0 2024-05-15T15:54:06.889 INFO:tasks.workunit.client.0.smithi049.stdout:5/358: creat dc/dd/d39/f6e x:0 0 0 2024-05-15T15:54:06.890 INFO:tasks.workunit.client.1.smithi144.stdout:6/264: creat d7/db/d11/d55/f66 x:0 0 0 2024-05-15T15:54:06.890 INFO:tasks.workunit.client.0.smithi049.stdout:5/359: chown dc/dd/f14 55253288 1 2024-05-15T15:54:06.893 INFO:tasks.workunit.client.0.smithi049.stdout:5/360: creat dc/dd/d10/f6f x:0 0 0 2024-05-15T15:54:06.894 INFO:tasks.workunit.client.1.smithi144.stdout:5/300: dwrite d0/dc/d12/f13 [0,4194304] 0 2024-05-15T15:54:06.896 INFO:tasks.workunit.client.1.smithi144.stdout:2/285: dwrite d3/d7/d20/d37/f3c [0,4194304] 0 2024-05-15T15:54:06.897 INFO:tasks.workunit.client.1.smithi144.stdout:1/302: unlink d7/d1d/d33/d41/c2a 0 2024-05-15T15:54:06.897 INFO:tasks.workunit.client.1.smithi144.stdout:2/286: stat d3 0 2024-05-15T15:54:06.898 INFO:tasks.workunit.client.0.smithi049.stdout:5/361: mknod dc/dd/d47/c70 0 2024-05-15T15:54:06.899 INFO:tasks.workunit.client.0.smithi049.stdout:5/362: stat dc/dd/d39/c6d 0 2024-05-15T15:54:06.905 INFO:tasks.workunit.client.1.smithi144.stdout:4/301: dwrite d6/d42/d46/d4b/f31 [0,4194304] 0 2024-05-15T15:54:06.906 INFO:tasks.workunit.client.1.smithi144.stdout:6/265: write f1 [792328,61498] 0 2024-05-15T15:54:06.907 INFO:tasks.workunit.client.1.smithi144.stdout:5/301: mknod d0/dc/d2f/d34/c5f 0 2024-05-15T15:54:06.907 INFO:tasks.workunit.client.1.smithi144.stdout:9/305: dwrite d0/d8/d5e/d44/d10/f17 [0,4194304] 0 2024-05-15T15:54:06.907 INFO:tasks.workunit.client.0.smithi049.stdout:5/363: fdatasync dc/d4a/f36 0 2024-05-15T15:54:06.909 INFO:tasks.workunit.client.0.smithi049.stdout:5/364: chown dc/d4a/f36 2851944 1 2024-05-15T15:54:06.912 INFO:tasks.workunit.client.0.smithi049.stdout:5/365: creat dc/dd/d10/f71 x:0 0 0 2024-05-15T15:54:06.915 INFO:tasks.workunit.client.0.smithi049.stdout:5/366: stat dc/d29/c46 0 2024-05-15T15:54:06.915 INFO:tasks.workunit.client.1.smithi144.stdout:7/309: symlink d4/df/l66 0 2024-05-15T15:54:06.916 INFO:tasks.workunit.client.1.smithi144.stdout:1/303: dwrite d7/df/d24/f54 [0,4194304] 0 2024-05-15T15:54:06.916 INFO:tasks.workunit.client.1.smithi144.stdout:9/306: fsync d0/d8/d5e/d44/d10/f27 0 2024-05-15T15:54:06.917 INFO:tasks.workunit.client.1.smithi144.stdout:4/302: creat d6/dd/d34/f5f x:0 0 0 2024-05-15T15:54:06.917 INFO:tasks.workunit.client.1.smithi144.stdout:7/310: chown d4/f27 425120 1 2024-05-15T15:54:06.917 INFO:tasks.workunit.client.1.smithi144.stdout:2/287: symlink d3/d7/d20/d22/d52/l60 0 2024-05-15T15:54:06.922 INFO:tasks.workunit.client.1.smithi144.stdout:9/307: dread d0/d8/d5e/d44/d10/fd [0,4194304] 0 2024-05-15T15:54:06.927 INFO:tasks.workunit.client.1.smithi144.stdout:6/266: rename d7/db/f18 to d7/db/d35/f67 0 2024-05-15T15:54:06.931 INFO:tasks.workunit.client.1.smithi144.stdout:5/302: link d0/dc/d12/f43 d0/dc/d12/d4a/f60 0 2024-05-15T15:54:06.932 INFO:tasks.workunit.client.1.smithi144.stdout:4/303: fsync d6/dd/d34/d3b/d30/f2f 0 2024-05-15T15:54:06.935 INFO:tasks.workunit.client.1.smithi144.stdout:5/303: fsync d0/dc/d12/f18 0 2024-05-15T15:54:06.937 INFO:tasks.workunit.client.1.smithi144.stdout:6/267: dwrite d7/d1e/d28/f3d [0,4194304] 0 2024-05-15T15:54:06.938 INFO:tasks.workunit.client.0.smithi049.stdout:1/415: truncate d0/d4/dd/de/d1e/d16/d8a/f99 543273 0 2024-05-15T15:54:06.939 INFO:tasks.workunit.client.1.smithi144.stdout:1/304: getdents d7/d1d 0 2024-05-15T15:54:06.941 INFO:tasks.workunit.client.1.smithi144.stdout:4/304: unlink d6/d42/d46/d4b/d1e/f4f 0 2024-05-15T15:54:06.944 INFO:tasks.workunit.client.0.smithi049.stdout:1/416: creat d0/d4/dd/de/d9e/fa0 x:0 0 0 2024-05-15T15:54:06.945 INFO:tasks.workunit.client.1.smithi144.stdout:7/311: dread d4/df/f20 [0,4194304] 0 2024-05-15T15:54:06.946 INFO:tasks.workunit.client.0.smithi049.stdout:1/417: chown d0/d4/dd/de/d48/d5c/f68 7 1 2024-05-15T15:54:06.946 INFO:tasks.workunit.client.1.smithi144.stdout:9/308: link d0/d8/d5e/d44/d16/f4e d0/d8/d5e/d44/f74 0 2024-05-15T15:54:06.946 INFO:tasks.workunit.client.0.smithi049.stdout:1/418: chown d0/d4/f39 51 1 2024-05-15T15:54:06.947 INFO:tasks.workunit.client.1.smithi144.stdout:8/313: sync 2024-05-15T15:54:06.947 INFO:tasks.workunit.client.1.smithi144.stdout:8/314: dread - d2/de/d49/f69 zero size 2024-05-15T15:54:06.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:06 smithi144 ceph-mon[36389]: pgmap v23: 65 pgs: 65 active+clean; 847 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 7.7 MiB/s rd, 94 MiB/s wr, 210 op/s 2024-05-15T15:54:06.949 INFO:tasks.workunit.client.0.smithi049.stdout:1/419: truncate d0/d4/dd/de/d9e/fa0 734751 0 2024-05-15T15:54:06.951 INFO:tasks.workunit.client.1.smithi144.stdout:9/309: mkdir d0/d8/d5e/d44/d3c/d75 0 2024-05-15T15:54:06.951 INFO:tasks.workunit.client.0.smithi049.stdout:5/367: rmdir dc/dd/d47 39 2024-05-15T15:54:06.952 INFO:tasks.workunit.client.1.smithi144.stdout:3/297: write d0/f1d [909697,89158] 0 2024-05-15T15:54:06.954 INFO:tasks.workunit.client.1.smithi144.stdout:3/298: chown d0/d1/d56/f58 2929 1 2024-05-15T15:54:06.955 INFO:tasks.workunit.client.1.smithi144.stdout:2/288: dread d3/d7/f1f [0,4194304] 0 2024-05-15T15:54:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:06 smithi049 ceph-mon[28399]: pgmap v23: 65 pgs: 65 active+clean; 847 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 7.7 MiB/s rd, 94 MiB/s wr, 210 op/s 2024-05-15T15:54:06.956 INFO:tasks.workunit.client.1.smithi144.stdout:2/289: stat d3/d7/d20/d22/c28 0 2024-05-15T15:54:06.956 INFO:tasks.workunit.client.1.smithi144.stdout:6/268: link d7/f4f d7/f68 0 2024-05-15T15:54:06.959 INFO:tasks.workunit.client.1.smithi144.stdout:9/310: unlink d0/d8/d1b/l1d 0 2024-05-15T15:54:06.960 INFO:tasks.workunit.client.1.smithi144.stdout:8/315: dwrite d2/fd [4194304,4194304] 0 2024-05-15T15:54:06.963 INFO:tasks.workunit.client.1.smithi144.stdout:3/299: symlink d0/d11/l6d 0 2024-05-15T15:54:06.963 INFO:tasks.workunit.client.1.smithi144.stdout:4/305: fsync d6/dd/d34/f5f 0 2024-05-15T15:54:06.964 INFO:tasks.workunit.client.0.smithi049.stdout:5/368: getdents dc/d4a 0 2024-05-15T15:54:06.965 INFO:tasks.workunit.client.0.smithi049.stdout:5/369: chown dc/dd/d10/f2a 251261 1 2024-05-15T15:54:06.967 INFO:tasks.workunit.client.1.smithi144.stdout:7/312: creat d4/f67 x:0 0 0 2024-05-15T15:54:06.969 INFO:tasks.workunit.client.1.smithi144.stdout:3/300: dwrite d0/d11/d24/f22 [0,4194304] 0 2024-05-15T15:54:06.973 INFO:tasks.workunit.client.1.smithi144.stdout:9/311: fdatasync d0/d8/d21/f38 0 2024-05-15T15:54:06.974 INFO:tasks.workunit.client.1.smithi144.stdout:9/312: write d0/d8/d5e/d44/d10/f58 [185753,3498] 0 2024-05-15T15:54:06.979 INFO:tasks.workunit.client.1.smithi144.stdout:7/313: mkdir d4/df/d25/d38/d68 0 2024-05-15T15:54:06.980 INFO:tasks.workunit.client.0.smithi049.stdout:5/370: dread dc/dd/f17 [0,4194304] 0 2024-05-15T15:54:06.983 INFO:tasks.workunit.client.1.smithi144.stdout:5/304: write d0/dc/f1b [776944,36214] 0 2024-05-15T15:54:06.984 INFO:tasks.workunit.client.1.smithi144.stdout:8/316: symlink d2/d24/d2d/l6d 0 2024-05-15T15:54:06.984 INFO:tasks.workunit.client.0.smithi049.stdout:5/371: rmdir dc/d4a 39 2024-05-15T15:54:06.989 INFO:tasks.workunit.client.1.smithi144.stdout:9/313: rename d0/d8/d5e/d44/d10/d19/d3e to d0/d8/d5e/d76 0 2024-05-15T15:54:06.989 INFO:tasks.workunit.client.0.smithi049.stdout:1/420: dwrite d0/d4/f1b [4194304,4194304] 0 2024-05-15T15:54:06.991 INFO:tasks.workunit.client.0.smithi049.stdout:1/421: write d0/d4/dd/de/d36/f53 [894750,131011] 0 2024-05-15T15:54:06.992 INFO:tasks.workunit.client.0.smithi049.stdout:1/422: fdatasync d0/d4/f39 0 2024-05-15T15:54:06.996 INFO:tasks.workunit.client.0.smithi049.stdout:5/372: creat dc/dd/f72 x:0 0 0 2024-05-15T15:54:07.002 INFO:tasks.workunit.client.0.smithi049.stdout:1/423: truncate d0/f5b 2212124 0 2024-05-15T15:54:07.002 INFO:tasks.workunit.client.1.smithi144.stdout:2/290: getdents d3/d7/d20/d3b 0 2024-05-15T15:54:07.006 INFO:tasks.workunit.client.0.smithi049.stdout:5/373: read dc/d4a/f36 [1722753,12374] 0 2024-05-15T15:54:07.007 INFO:tasks.workunit.client.1.smithi144.stdout:8/317: rename d2/d53 to d2/d1a/d38/d4b/d5b/d6e 0 2024-05-15T15:54:07.008 INFO:tasks.workunit.client.0.smithi049.stdout:1/424: creat d0/d4/dd/d5d/fa1 x:0 0 0 2024-05-15T15:54:07.008 INFO:tasks.workunit.client.0.smithi049.stdout:1/425: stat d0/d4/l14 0 2024-05-15T15:54:07.009 INFO:tasks.workunit.client.1.smithi144.stdout:9/314: unlink d0/d8/d21/l3b 0 2024-05-15T15:54:07.011 INFO:tasks.workunit.client.1.smithi144.stdout:7/314: mknod d4/df/d25/d38/d5d/c69 0 2024-05-15T15:54:07.014 INFO:tasks.workunit.client.0.smithi049.stdout:1/426: rmdir d0/d4/dd/de/d4e 39 2024-05-15T15:54:07.015 INFO:tasks.workunit.client.1.smithi144.stdout:1/305: write d7/d1d/d33/d55/f5b [2681227,96358] 0 2024-05-15T15:54:07.017 INFO:tasks.workunit.client.1.smithi144.stdout:3/301: dwrite d0/d1/d17/f48 [0,4194304] 0 2024-05-15T15:54:07.017 INFO:tasks.workunit.client.1.smithi144.stdout:5/305: mknod d0/dc/d24/d57/c61 0 2024-05-15T15:54:07.018 INFO:tasks.workunit.client.0.smithi049.stdout:1/427: chown d0/d4/dd/de/d4e 262151 1 2024-05-15T15:54:07.018 INFO:tasks.workunit.client.1.smithi144.stdout:1/306: write d7/d1d/d33/d1e/d34/f38 [1449864,127331] 0 2024-05-15T15:54:07.020 INFO:tasks.workunit.client.0.smithi049.stdout:1/428: truncate d0/d4/dd/de/d4d/f56 371140 0 2024-05-15T15:54:07.022 INFO:tasks.workunit.client.0.smithi049.stdout:1/429: dread d0/d4/dd/de/d9e/fa0 [0,4194304] 0 2024-05-15T15:54:07.024 INFO:tasks.workunit.client.1.smithi144.stdout:5/306: dread d0/dc/d12/f13 [0,4194304] 0 2024-05-15T15:54:07.025 INFO:tasks.workunit.client.0.smithi049.stdout:1/430: rmdir d0/d4/dd 39 2024-05-15T15:54:07.026 INFO:tasks.workunit.client.0.smithi049.stdout:5/374: write dc/dd/d39/f4f [3253581,6655] 0 2024-05-15T15:54:07.028 INFO:tasks.workunit.client.1.smithi144.stdout:9/315: dwrite d0/d8/d5e/d76/f3f [0,4194304] 0 2024-05-15T15:54:07.028 INFO:tasks.workunit.client.1.smithi144.stdout:6/269: link d7/db/d11/f1b d7/d1e/f69 0 2024-05-15T15:54:07.029 INFO:tasks.workunit.client.1.smithi144.stdout:6/270: readlink d7/d1e/l3b 0 2024-05-15T15:54:07.029 INFO:tasks.workunit.client.1.smithi144.stdout:6/271: chown d7/f41 539 1 2024-05-15T15:54:07.032 INFO:tasks.workunit.client.0.smithi049.stdout:1/431: unlink d0/l4b 0 2024-05-15T15:54:07.032 INFO:tasks.workunit.client.1.smithi144.stdout:0/300: sync 2024-05-15T15:54:07.033 INFO:tasks.workunit.client.1.smithi144.stdout:0/301: chown d4/d2d 7917 1 2024-05-15T15:54:07.033 INFO:tasks.workunit.client.1.smithi144.stdout:2/291: write d3/d10/f1d [4596569,26657] 0 2024-05-15T15:54:07.034 INFO:tasks.workunit.client.1.smithi144.stdout:7/315: dread d4/f12 [0,4194304] 0 2024-05-15T15:54:07.037 INFO:tasks.workunit.client.1.smithi144.stdout:3/302: rmdir d0/d11/d3b 39 2024-05-15T15:54:07.040 INFO:tasks.workunit.client.0.smithi049.stdout:5/375: symlink dc/dd/l73 0 2024-05-15T15:54:07.041 INFO:tasks.workunit.client.1.smithi144.stdout:1/307: mknod d7/d1d/c5d 0 2024-05-15T15:54:07.042 INFO:tasks.workunit.client.1.smithi144.stdout:0/302: dread d4/df/d1f/f2f [0,4194304] 0 2024-05-15T15:54:07.042 INFO:tasks.workunit.client.1.smithi144.stdout:8/318: rmdir d2/d24/d2d/d30 39 2024-05-15T15:54:07.043 INFO:tasks.workunit.client.1.smithi144.stdout:0/303: stat d4/d6/d3a/c5c 0 2024-05-15T15:54:07.043 INFO:tasks.workunit.client.1.smithi144.stdout:8/319: chown d2/de/d36/f2c 58 1 2024-05-15T15:54:07.047 INFO:tasks.workunit.client.1.smithi144.stdout:5/307: creat d0/dc/d12/d31/d35/f62 x:0 0 0 2024-05-15T15:54:07.053 INFO:tasks.workunit.client.1.smithi144.stdout:6/272: dread d7/db/f2c [0,4194304] 0 2024-05-15T15:54:07.054 INFO:tasks.workunit.client.1.smithi144.stdout:8/320: mknod d2/c6f 0 2024-05-15T15:54:07.056 INFO:tasks.workunit.client.1.smithi144.stdout:0/304: rename d4/df/d14/l53 to d4/d6/d4a/l71 0 2024-05-15T15:54:07.057 INFO:tasks.workunit.client.0.smithi049.stdout:1/432: dwrite d0/d4/dd/de/d36/f3f [0,4194304] 0 2024-05-15T15:54:07.057 INFO:tasks.workunit.client.1.smithi144.stdout:0/305: fdatasync d4/f22 0 2024-05-15T15:54:07.058 INFO:tasks.workunit.client.1.smithi144.stdout:4/306: sync 2024-05-15T15:54:07.059 INFO:tasks.workunit.client.1.smithi144.stdout:2/292: dread d3/f1c [0,4194304] 0 2024-05-15T15:54:07.059 INFO:tasks.workunit.client.0.smithi049.stdout:5/376: dread dc/dd/d39/f4f [0,4194304] 0 2024-05-15T15:54:07.060 INFO:tasks.workunit.client.1.smithi144.stdout:4/307: stat d6/dd/d34/d3b/d30/ce 0 2024-05-15T15:54:07.061 INFO:tasks.workunit.client.1.smithi144.stdout:6/273: dwrite d7/d1e/f44 [0,4194304] 0 2024-05-15T15:54:07.061 INFO:tasks.workunit.client.1.smithi144.stdout:2/293: stat d3/d7/d20/d37/d3a/c4a 0 2024-05-15T15:54:07.062 INFO:tasks.workunit.client.1.smithi144.stdout:7/316: symlink d4/d45/l6a 0 2024-05-15T15:54:07.062 INFO:tasks.workunit.client.0.smithi049.stdout:1/433: getdents d0/d4/dd/de/d4d/d8b 0 2024-05-15T15:54:07.064 INFO:tasks.workunit.client.1.smithi144.stdout:1/308: mkdir d7/d1d/d52/d5e 0 2024-05-15T15:54:07.067 INFO:tasks.workunit.client.0.smithi049.stdout:1/434: rmdir d0/d4/dd/de 39 2024-05-15T15:54:07.068 INFO:tasks.workunit.client.0.smithi049.stdout:5/377: dwrite dc/d4a/d42/f48 [0,4194304] 0 2024-05-15T15:54:07.069 INFO:tasks.workunit.client.0.smithi049.stdout:5/378: dread - dc/dd/d10/f59 zero size 2024-05-15T15:54:07.072 INFO:tasks.workunit.client.1.smithi144.stdout:5/308: write d0/dc/d24/d28/d2d/f3d [1409328,69878] 0 2024-05-15T15:54:07.076 INFO:tasks.workunit.client.1.smithi144.stdout:9/316: write d0/d8/d5e/d44/d10/d19/f33 [4144557,84094] 0 2024-05-15T15:54:07.078 INFO:tasks.workunit.client.0.smithi049.stdout:5/379: rename dc/dd/d39/f4f to dc/d4a/d42/f74 0 2024-05-15T15:54:07.137 INFO:tasks.workunit.client.0.smithi049.stdout:9/233: sync 2024-05-15T15:54:07.138 INFO:tasks.workunit.client.0.smithi049.stdout:9/234: readlink d0/d8/d9/l29 0 2024-05-15T15:54:07.138 INFO:tasks.workunit.client.0.smithi049.stdout:1/435: fsync d0/d4/dd/de/d1e/d2e/f2c 0 2024-05-15T15:54:07.139 INFO:tasks.workunit.client.0.smithi049.stdout:9/235: chown d0/d8/d12 460 1 2024-05-15T15:54:07.140 INFO:tasks.workunit.client.1.smithi144.stdout:0/306: creat d4/df/d1f/d24/d2b/d55/f72 x:0 0 0 2024-05-15T15:54:07.141 INFO:tasks.workunit.client.1.smithi144.stdout:3/303: write d0/d11/d3b/f64 [1604787,5194] 0 2024-05-15T15:54:07.142 INFO:tasks.workunit.client.1.smithi144.stdout:3/304: write d0/d1/d17/f48 [911295,58647] 0 2024-05-15T15:54:07.142 INFO:tasks.workunit.client.0.smithi049.stdout:5/380: rmdir dc/dd/d10 39 2024-05-15T15:54:07.144 INFO:tasks.workunit.client.1.smithi144.stdout:3/305: dread d0/d1/d17/f19 [0,4194304] 0 2024-05-15T15:54:07.151 INFO:tasks.workunit.client.0.smithi049.stdout:5/381: dwrite dc/dd/d39/f6e [0,4194304] 0 2024-05-15T15:54:07.151 INFO:tasks.workunit.client.1.smithi144.stdout:6/274: truncate d7/fd 1923102 0 2024-05-15T15:54:07.155 INFO:tasks.workunit.client.0.smithi049.stdout:1/436: dread d0/f58 [0,4194304] 0 2024-05-15T15:54:07.155 INFO:tasks.workunit.client.1.smithi144.stdout:2/294: fsync d3/d7/d20/d22/f2b 0 2024-05-15T15:54:07.159 INFO:tasks.workunit.client.1.smithi144.stdout:6/275: dwrite d7/db/d11/d55/f66 [0,4194304] 0 2024-05-15T15:54:07.160 INFO:tasks.workunit.client.1.smithi144.stdout:7/317: rename d4/d41/f52 to d4/df/d25/d40/d62/f6b 0 2024-05-15T15:54:07.170 INFO:tasks.workunit.client.0.smithi049.stdout:9/236: truncate d0/d8/d11/f16 501479 0 2024-05-15T15:54:07.172 INFO:tasks.workunit.client.1.smithi144.stdout:5/309: mkdir d0/dc/d2f/d34/d63 0 2024-05-15T15:54:07.173 INFO:tasks.workunit.client.1.smithi144.stdout:5/310: write d0/dc/d1f/f3b [2524819,13920] 0 2024-05-15T15:54:07.173 INFO:tasks.workunit.client.0.smithi049.stdout:1/437: mkdir d0/d4/dd/de/d48/d5c/da2 0 2024-05-15T15:54:07.176 INFO:tasks.workunit.client.1.smithi144.stdout:9/317: unlink d0/d8/d5e/f3a 0 2024-05-15T15:54:07.178 INFO:tasks.workunit.client.0.smithi049.stdout:5/382: creat dc/dd/d47/f75 x:0 0 0 2024-05-15T15:54:07.185 INFO:tasks.workunit.client.1.smithi144.stdout:0/307: dread d4/df/f46 [0,4194304] 0 2024-05-15T15:54:07.186 INFO:tasks.workunit.client.0.smithi049.stdout:1/438: creat d0/d4/dd/de/d48/d5c/fa3 x:0 0 0 2024-05-15T15:54:07.187 INFO:tasks.workunit.client.0.smithi049.stdout:9/237: dread d0/d8/d1b/d40/f49 [0,4194304] 0 2024-05-15T15:54:07.190 INFO:tasks.workunit.client.0.smithi049.stdout:5/383: truncate dc/dd/f17 268293 0 2024-05-15T15:54:07.191 INFO:tasks.workunit.client.1.smithi144.stdout:3/306: chown d0/d1/c5d 5 1 2024-05-15T15:54:07.192 INFO:tasks.workunit.client.1.smithi144.stdout:3/307: dread - d0/d11/f61 zero size 2024-05-15T15:54:07.193 INFO:tasks.workunit.client.1.smithi144.stdout:3/308: write d0/d11/f2a [4324637,2902] 0 2024-05-15T15:54:07.195 INFO:tasks.workunit.client.0.smithi049.stdout:7/299: sync 2024-05-15T15:54:07.195 INFO:tasks.workunit.client.0.smithi049.stdout:1/439: dwrite d0/d4/dd/d5d/d65/f6f [0,4194304] 0 2024-05-15T15:54:07.196 INFO:tasks.workunit.client.0.smithi049.stdout:1/440: readlink d0/d4/l14 0 2024-05-15T15:54:07.197 INFO:tasks.workunit.client.0.smithi049.stdout:9/238: mkdir d0/d8/d21/d4d 0 2024-05-15T15:54:07.198 INFO:tasks.workunit.client.1.smithi144.stdout:3/309: dwrite d0/d1/d17/d39/d40/f50 [0,4194304] 0 2024-05-15T15:54:07.200 INFO:tasks.workunit.client.0.smithi049.stdout:5/384: mknod dc/dd/d47/c76 0 2024-05-15T15:54:07.204 INFO:tasks.workunit.client.1.smithi144.stdout:6/276: chown d7/db/c1c 1060066008 1 2024-05-15T15:54:07.206 INFO:tasks.workunit.client.1.smithi144.stdout:1/309: rename d7/d1d/d33/d41/d22/d45/l59 to d7/d1d/d52/l5f 0 2024-05-15T15:54:07.209 INFO:tasks.workunit.client.0.smithi049.stdout:9/239: rmdir d0/d25 39 2024-05-15T15:54:07.212 INFO:tasks.workunit.client.0.smithi049.stdout:5/385: creat dc/dd/d39/f77 x:0 0 0 2024-05-15T15:54:07.217 INFO:tasks.workunit.client.0.smithi049.stdout:1/441: mknod d0/d4/dd/ca4 0 2024-05-15T15:54:07.226 INFO:tasks.workunit.client.1.smithi144.stdout:0/308: dwrite d4/d6/d4a/d4d/f6e [0,4194304] 0 2024-05-15T15:54:07.226 INFO:tasks.workunit.client.0.smithi049.stdout:7/300: creat d0/d6/f55 x:0 0 0 2024-05-15T15:54:07.235 INFO:tasks.workunit.client.0.smithi049.stdout:5/386: symlink dc/l78 0 2024-05-15T15:54:07.244 INFO:tasks.workunit.client.0.smithi049.stdout:7/301: dwrite d0/d6/d23/f30 [0,4194304] 0 2024-05-15T15:54:07.245 INFO:tasks.workunit.client.0.smithi049.stdout:7/302: readlink d0/d6/d1d/l2b 0 2024-05-15T15:54:07.246 INFO:tasks.workunit.client.0.smithi049.stdout:6/296: sync 2024-05-15T15:54:07.247 INFO:tasks.workunit.client.0.smithi049.stdout:8/339: sync 2024-05-15T15:54:07.247 INFO:tasks.workunit.client.0.smithi049.stdout:3/377: sync 2024-05-15T15:54:07.247 INFO:tasks.workunit.client.0.smithi049.stdout:0/330: sync 2024-05-15T15:54:07.247 INFO:tasks.workunit.client.0.smithi049.stdout:4/407: sync 2024-05-15T15:54:07.247 INFO:tasks.workunit.client.0.smithi049.stdout:2/372: sync 2024-05-15T15:54:07.250 INFO:tasks.workunit.client.0.smithi049.stdout:5/387: rename dc/d4a/d31/f61 to dc/dd/d39/f79 0 2024-05-15T15:54:07.250 INFO:tasks.workunit.client.0.smithi049.stdout:3/378: fdatasync d12/d67/f69 0 2024-05-15T15:54:07.252 INFO:tasks.workunit.client.0.smithi049.stdout:2/373: fdatasync d3/db/d1d/d1e/d2b/d3b/f5f 0 2024-05-15T15:54:07.252 INFO:tasks.workunit.client.0.smithi049.stdout:3/379: chown d12/d30/d42/l57 9 1 2024-05-15T15:54:07.253 INFO:tasks.workunit.client.0.smithi049.stdout:9/240: link d0/d8/d1b/d40/f49 d0/d8/d9/d22/d26/f4e 0 2024-05-15T15:54:07.254 INFO:tasks.workunit.client.0.smithi049.stdout:2/374: read d3/f10 [2043660,67065] 0 2024-05-15T15:54:07.254 INFO:tasks.workunit.client.1.smithi144.stdout:0/309: unlink d4/c31 0 2024-05-15T15:54:07.255 INFO:tasks.workunit.client.0.smithi049.stdout:9/241: readlink d0/d8/d12/l3f 0 2024-05-15T15:54:07.255 INFO:tasks.workunit.client.1.smithi144.stdout:0/310: write d4/df/d1f/d24/f4b [916254,120930] 0 2024-05-15T15:54:07.258 INFO:tasks.workunit.client.0.smithi049.stdout:3/380: dread d12/f1d [0,4194304] 0 2024-05-15T15:54:07.261 INFO:tasks.workunit.client.1.smithi144.stdout:0/311: dwrite d4/df/d1f/d24/f2a [0,4194304] 0 2024-05-15T15:54:07.262 INFO:tasks.workunit.client.1.smithi144.stdout:5/311: unlink d0/dc/d12/d31/d35/f52 0 2024-05-15T15:54:07.263 INFO:tasks.workunit.client.1.smithi144.stdout:3/310: mknod d0/d1/d6c/c6e 0 2024-05-15T15:54:07.264 INFO:tasks.workunit.client.0.smithi049.stdout:0/331: symlink dc/d32/l68 0 2024-05-15T15:54:07.265 INFO:tasks.workunit.client.0.smithi049.stdout:0/332: dread - dc/d13/d14/d33/f5b zero size 2024-05-15T15:54:07.267 INFO:tasks.workunit.client.1.smithi144.stdout:5/312: truncate d0/dc/d1f/d50/f58 146503 0 2024-05-15T15:54:07.267 INFO:tasks.workunit.client.1.smithi144.stdout:1/310: symlink d7/d1d/d52/d5e/l60 0 2024-05-15T15:54:07.268 INFO:tasks.workunit.client.1.smithi144.stdout:1/311: chown d7/df/c14 8215 1 2024-05-15T15:54:07.268 INFO:tasks.workunit.client.1.smithi144.stdout:0/312: dwrite d4/d6/d4a/d4d/f6e [0,4194304] 0 2024-05-15T15:54:07.269 INFO:tasks.workunit.client.0.smithi049.stdout:5/388: creat dc/d4a/d31/f7a x:0 0 0 2024-05-15T15:54:07.270 INFO:tasks.workunit.client.1.smithi144.stdout:7/318: link d4/df/d25/l42 d4/d45/l6c 0 2024-05-15T15:54:07.274 INFO:tasks.workunit.client.1.smithi144.stdout:1/312: dwrite d7/d1d/d33/d55/f5b [0,4194304] 0 2024-05-15T15:54:07.278 INFO:tasks.workunit.client.0.smithi049.stdout:4/408: rename d18/d1e/d2a/d36/l84 to d18/d1e/d2a/d3b/l89 0 2024-05-15T15:54:07.279 INFO:tasks.workunit.client.0.smithi049.stdout:4/409: write d18/f49 [3725764,116099] 0 2024-05-15T15:54:07.280 INFO:tasks.workunit.client.0.smithi049.stdout:4/410: readlink d18/d21/d2b/l33 0 2024-05-15T15:54:07.281 INFO:tasks.workunit.client.1.smithi144.stdout:1/313: dwrite d7/d1d/d33/d55/f5b [4194304,4194304] 0 2024-05-15T15:54:07.285 INFO:tasks.workunit.client.0.smithi049.stdout:9/242: symlink d0/d8/d9/d22/d26/l4f 0 2024-05-15T15:54:07.288 INFO:tasks.workunit.client.1.smithi144.stdout:0/313: mknod d4/d6/d3a/d4c/c73 0 2024-05-15T15:54:07.289 INFO:tasks.workunit.client.1.smithi144.stdout:0/314: dread - d4/f22 zero size 2024-05-15T15:54:07.292 INFO:tasks.workunit.client.1.smithi144.stdout:8/321: dwrite d2/f35 [0,4194304] 0 2024-05-15T15:54:07.296 INFO:tasks.workunit.client.0.smithi049.stdout:3/381: dwrite d12/d14/d52/f5f [0,4194304] 0 2024-05-15T15:54:07.308 INFO:tasks.workunit.client.1.smithi144.stdout:3/311: truncate d0/d1/f5 631668 0 2024-05-15T15:54:07.310 INFO:tasks.workunit.client.0.smithi049.stdout:0/333: dwrite dc/f15 [0,4194304] 0 2024-05-15T15:54:07.313 INFO:tasks.workunit.client.0.smithi049.stdout:0/334: truncate dc/d13/d14/d33/f53 411483 0 2024-05-15T15:54:07.314 INFO:tasks.workunit.client.1.smithi144.stdout:3/312: dwrite d0/d11/d3b/f45 [0,4194304] 0 2024-05-15T15:54:07.315 INFO:tasks.workunit.client.0.smithi049.stdout:2/375: mkdir d3/db/d1d/d1e/d2b/d2e/d67/d71 0 2024-05-15T15:54:07.318 INFO:tasks.workunit.client.1.smithi144.stdout:8/322: mknod d2/d1a/d38/d4b/d5b/d6e/c70 0 2024-05-15T15:54:07.331 INFO:tasks.workunit.client.1.smithi144.stdout:0/315: truncate d4/d6/fb 2883488 0 2024-05-15T15:54:07.334 INFO:tasks.workunit.client.1.smithi144.stdout:3/313: mknod d0/c6f 0 2024-05-15T15:54:07.335 INFO:tasks.workunit.client.1.smithi144.stdout:3/314: fsync d0/d11/d3b/f5b 0 2024-05-15T15:54:07.335 INFO:tasks.workunit.client.0.smithi049.stdout:9/243: dread d0/d8/d21/f2f [0,4194304] 0 2024-05-15T15:54:07.336 INFO:tasks.workunit.client.0.smithi049.stdout:5/389: mkdir dc/dd/d7b 0 2024-05-15T15:54:07.337 INFO:tasks.workunit.client.1.smithi144.stdout:8/323: mkdir d2/d1a/d38/d4b/d5b/d6e/d71 0 2024-05-15T15:54:07.339 INFO:tasks.workunit.client.0.smithi049.stdout:0/335: creat dc/d13/d14/d25/f69 x:0 0 0 2024-05-15T15:54:07.341 INFO:tasks.workunit.client.1.smithi144.stdout:0/316: write d4/f5 [3316907,71257] 0 2024-05-15T15:54:07.341 INFO:tasks.workunit.client.0.smithi049.stdout:2/376: dread d3/db/d5b/f70 [0,4194304] 0 2024-05-15T15:54:07.342 INFO:tasks.workunit.client.1.smithi144.stdout:3/315: symlink d0/d1/d56/l70 0 2024-05-15T15:54:07.345 INFO:tasks.workunit.client.1.smithi144.stdout:8/324: chown d2/de/d36/l63 1533220070 1 2024-05-15T15:54:07.345 INFO:tasks.workunit.client.1.smithi144.stdout:8/325: stat d2/l4 0 2024-05-15T15:54:07.346 INFO:tasks.workunit.client.0.smithi049.stdout:3/382: unlink f10 0 2024-05-15T15:54:07.347 INFO:tasks.workunit.client.1.smithi144.stdout:0/317: chown d4/dc/f18 198 1 2024-05-15T15:54:07.348 INFO:tasks.workunit.client.0.smithi049.stdout:9/244: mknod d0/d8/d12/c50 0 2024-05-15T15:54:07.349 INFO:tasks.workunit.client.1.smithi144.stdout:6/277: write d7/d1e/f69 [4020616,44755] 0 2024-05-15T15:54:07.349 INFO:tasks.workunit.client.1.smithi144.stdout:3/316: readlink d0/d1/d17/l66 0 2024-05-15T15:54:07.350 INFO:tasks.workunit.client.0.smithi049.stdout:1/442: dwrite d0/d4/dd/de/f13 [0,4194304] 0 2024-05-15T15:54:07.352 INFO:tasks.workunit.client.0.smithi049.stdout:1/443: fsync d0/d4/dd/de/d36/f66 0 2024-05-15T15:54:07.352 INFO:tasks.workunit.client.1.smithi144.stdout:0/318: mkdir d4/d6/d4a/d74 0 2024-05-15T15:54:07.353 INFO:tasks.workunit.client.1.smithi144.stdout:4/308: symlink d6/dd/d34/d3b/d30/l60 0 2024-05-15T15:54:07.354 INFO:tasks.workunit.client.0.smithi049.stdout:1/444: readlink d0/d4/dd/de/d1e/d2e/l3a 0 2024-05-15T15:54:07.355 INFO:tasks.workunit.client.0.smithi049.stdout:1/445: fdatasync d0/d4/dd/d5d/d65/d72/f7b 0 2024-05-15T15:54:07.355 INFO:tasks.workunit.client.0.smithi049.stdout:0/336: rmdir dc/d13/d14/d2a 39 2024-05-15T15:54:07.356 INFO:tasks.workunit.client.1.smithi144.stdout:0/319: dread d4/df/f16 [0,4194304] 0 2024-05-15T15:54:07.362 INFO:tasks.workunit.client.0.smithi049.stdout:2/377: creat d3/db/d1d/d1e/d2b/d2e/f72 x:0 0 0 2024-05-15T15:54:07.362 INFO:tasks.workunit.client.0.smithi049.stdout:1/446: dwrite d0/d4/d63/f70 [0,4194304] 0 2024-05-15T15:54:07.363 INFO:tasks.workunit.client.0.smithi049.stdout:7/303: link d0/lf d0/d6/l56 0 2024-05-15T15:54:07.365 INFO:tasks.workunit.client.0.smithi049.stdout:2/378: write d3/db/d1d/d1e/d2b/d3b/d42/f6d [1003878,38374] 0 2024-05-15T15:54:07.366 INFO:tasks.workunit.client.1.smithi144.stdout:3/317: dread d0/d11/d3b/f64 [0,4194304] 0 2024-05-15T15:54:07.366 INFO:tasks.workunit.client.1.smithi144.stdout:4/309: unlink c5 0 2024-05-15T15:54:07.369 INFO:tasks.workunit.client.0.smithi049.stdout:3/383: fsync d12/d14/f43 0 2024-05-15T15:54:07.370 INFO:tasks.workunit.client.1.smithi144.stdout:9/318: rename d0/d8/d21/d29 to d0/d8/d5e/d44/d77 0 2024-05-15T15:54:07.371 INFO:tasks.workunit.client.1.smithi144.stdout:6/278: mknod d7/db/c6a 0 2024-05-15T15:54:07.372 INFO:tasks.workunit.client.1.smithi144.stdout:6/279: write d7/db/d11/f43 [59262,91367] 0 2024-05-15T15:54:07.374 INFO:tasks.workunit.client.1.smithi144.stdout:6/280: write d7/db/d11/f42 [218912,34181] 0 2024-05-15T15:54:07.374 INFO:tasks.workunit.client.1.smithi144.stdout:0/320: creat d4/d2d/d48/d5b/f75 x:0 0 0 2024-05-15T15:54:07.377 INFO:tasks.workunit.client.0.smithi049.stdout:6/297: dwrite f2 [4194304,4194304] 0 2024-05-15T15:54:07.377 INFO:tasks.workunit.client.1.smithi144.stdout:0/321: dread d4/df/d1f/d24/d2b/d55/f60 [0,4194304] 0 2024-05-15T15:54:07.378 INFO:tasks.workunit.client.1.smithi144.stdout:2/295: rename d3/d7/f18 to d3/d7/f61 0 2024-05-15T15:54:07.383 INFO:tasks.workunit.client.1.smithi144.stdout:5/313: rename d0/dc/d2f/d34/l40 to d0/dc/d12/d4a/d5b/d5c/l64 0 2024-05-15T15:54:07.383 INFO:tasks.workunit.client.0.smithi049.stdout:4/411: getdents d18/d21/d2b/d4e 0 2024-05-15T15:54:07.385 INFO:tasks.workunit.client.1.smithi144.stdout:6/281: write d7/db/d35/f67 [1254683,112945] 0 2024-05-15T15:54:07.386 INFO:tasks.workunit.client.0.smithi049.stdout:7/304: symlink d0/d6/d23/d4a/l57 0 2024-05-15T15:54:07.388 INFO:tasks.workunit.client.1.smithi144.stdout:3/318: dread d0/d11/d24/f41 [0,4194304] 0 2024-05-15T15:54:07.389 INFO:tasks.workunit.client.0.smithi049.stdout:1/447: dread d0/d4/dd/de/d9e/fa0 [0,4194304] 0 2024-05-15T15:54:07.390 INFO:tasks.workunit.client.1.smithi144.stdout:0/322: symlink d4/d6/d4a/l76 0 2024-05-15T15:54:07.390 INFO:tasks.workunit.client.0.smithi049.stdout:1/448: fdatasync d0/f5e 0 2024-05-15T15:54:07.391 INFO:tasks.workunit.client.1.smithi144.stdout:0/323: write d4/d6/d3a/d5d/f61 [612958,54723] 0 2024-05-15T15:54:07.394 INFO:tasks.workunit.client.1.smithi144.stdout:7/319: write d4/f27 [817442,92857] 0 2024-05-15T15:54:07.395 INFO:tasks.workunit.client.1.smithi144.stdout:7/320: fdatasync d4/d3c/f57 0 2024-05-15T15:54:07.396 INFO:tasks.workunit.client.0.smithi049.stdout:8/340: truncate d2/f3 6671443 0 2024-05-15T15:54:07.396 INFO:tasks.workunit.client.0.smithi049.stdout:3/384: write d12/d14/f4f [850328,106230] 0 2024-05-15T15:54:07.397 INFO:tasks.workunit.client.1.smithi144.stdout:9/319: dread d0/d8/d5e/d44/d10/d19/f33 [0,4194304] 0 2024-05-15T15:54:07.397 INFO:tasks.workunit.client.0.smithi049.stdout:3/385: stat d12/f29 0 2024-05-15T15:54:07.398 INFO:tasks.workunit.client.1.smithi144.stdout:2/296: creat d3/d7/d20/d22/f62 x:0 0 0 2024-05-15T15:54:07.398 INFO:tasks.workunit.client.1.smithi144.stdout:2/297: read d3/d7/f1f [2105223,78264] 0 2024-05-15T15:54:07.400 INFO:tasks.workunit.client.0.smithi049.stdout:6/298: creat dd/d26/f4f x:0 0 0 2024-05-15T15:54:07.402 INFO:tasks.workunit.client.0.smithi049.stdout:0/337: unlink dc/d13/d14/d2a/l62 0 2024-05-15T15:54:07.403 INFO:tasks.workunit.client.0.smithi049.stdout:0/338: write dc/f1d [1406711,35293] 0 2024-05-15T15:54:07.404 INFO:tasks.workunit.client.1.smithi144.stdout:1/314: rename d7/l3c to d7/d31/l61 0 2024-05-15T15:54:07.405 INFO:tasks.workunit.client.0.smithi049.stdout:4/412: rmdir d18/d1e/d2a 39 2024-05-15T15:54:07.406 INFO:tasks.workunit.client.1.smithi144.stdout:5/314: mkdir d0/dc/d1f/d65 0 2024-05-15T15:54:07.412 INFO:tasks.workunit.client.0.smithi049.stdout:1/449: symlink d0/d4/dd/de/d4d/la5 0 2024-05-15T15:54:07.412 INFO:tasks.workunit.client.0.smithi049.stdout:1/450: readlink d0/d4/dd/de/d1e/l2a 0 2024-05-15T15:54:07.413 INFO:tasks.workunit.client.1.smithi144.stdout:3/319: mkdir d0/d1/d56/d71 0 2024-05-15T15:54:07.414 INFO:tasks.workunit.client.1.smithi144.stdout:2/298: dread d3/d10/f2e [0,4194304] 0 2024-05-15T15:54:07.415 INFO:tasks.workunit.client.1.smithi144.stdout:0/324: rmdir d4/df/d1f/d56 39 2024-05-15T15:54:07.419 INFO:tasks.workunit.client.0.smithi049.stdout:1/451: dwrite d0/d4/dd/f28 [0,4194304] 0 2024-05-15T15:54:07.420 INFO:tasks.workunit.client.0.smithi049.stdout:3/386: write d12/d14/f17 [3691152,69730] 0 2024-05-15T15:54:07.426 INFO:tasks.workunit.client.0.smithi049.stdout:6/299: mkdir dd/d17/d24/d2b/d36/d50 0 2024-05-15T15:54:07.428 INFO:tasks.workunit.client.0.smithi049.stdout:6/300: truncate dd/d17/d24/d2b/f46 415334 0 2024-05-15T15:54:07.431 INFO:tasks.workunit.client.1.smithi144.stdout:8/326: rename d2/de/fc to d2/de/d36/d58/f72 0 2024-05-15T15:54:07.432 INFO:tasks.workunit.client.0.smithi049.stdout:0/339: readlink dc/d13/d14/d33/l5c 0 2024-05-15T15:54:07.434 INFO:tasks.workunit.client.0.smithi049.stdout:6/301: dwrite dd/f14 [0,4194304] 0 2024-05-15T15:54:07.437 INFO:tasks.workunit.client.0.smithi049.stdout:4/413: fdatasync d18/f1a 0 2024-05-15T15:54:07.437 INFO:tasks.workunit.client.0.smithi049.stdout:6/302: truncate dd/d17/d24/d2b/f4d 549867 0 2024-05-15T15:54:07.437 INFO:tasks.workunit.client.0.smithi049.stdout:4/414: chown f5 12972 1 2024-05-15T15:54:07.438 INFO:tasks.workunit.client.0.smithi049.stdout:4/415: write f17 [1917235,82053] 0 2024-05-15T15:54:07.441 INFO:tasks.workunit.client.1.smithi144.stdout:3/320: unlink d0/d11/l6d 0 2024-05-15T15:54:07.442 INFO:tasks.workunit.client.1.smithi144.stdout:5/315: dwrite d0/dc/fe [0,4194304] 0 2024-05-15T15:54:07.443 INFO:tasks.workunit.client.0.smithi049.stdout:7/305: symlink d0/l58 0 2024-05-15T15:54:07.444 INFO:tasks.workunit.client.0.smithi049.stdout:4/416: dwrite f17 [0,4194304] 0 2024-05-15T15:54:07.448 INFO:tasks.workunit.client.1.smithi144.stdout:0/325: rmdir d4/d6/d3a/d5d 39 2024-05-15T15:54:07.454 INFO:tasks.workunit.client.0.smithi049.stdout:2/379: dwrite d3/da/f45 [0,4194304] 0 2024-05-15T15:54:07.458 INFO:tasks.workunit.client.1.smithi144.stdout:7/321: symlink d4/df/d25/l6d 0 2024-05-15T15:54:07.460 INFO:tasks.workunit.client.0.smithi049.stdout:9/245: getdents d0/d8/d21 0 2024-05-15T15:54:07.461 INFO:tasks.workunit.client.0.smithi049.stdout:9/246: chown d0/d8/d1b/f3a 1938896 1 2024-05-15T15:54:07.465 INFO:tasks.workunit.client.0.smithi049.stdout:3/387: creat d12/d30/d63/d68/f77 x:0 0 0 2024-05-15T15:54:07.465 INFO:tasks.workunit.client.0.smithi049.stdout:3/388: chown c11 8292175 1 2024-05-15T15:54:07.469 INFO:tasks.workunit.client.1.smithi144.stdout:4/310: sync 2024-05-15T15:54:07.469 INFO:tasks.workunit.client.1.smithi144.stdout:4/311: fsync d6/d42/d46/d4b/f31 0 2024-05-15T15:54:07.470 INFO:tasks.workunit.client.0.smithi049.stdout:0/340: mknod dc/d13/d14/d25/d31/c6a 0 2024-05-15T15:54:07.470 INFO:tasks.workunit.client.1.smithi144.stdout:6/282: write d7/f4f [79995,49654] 0 2024-05-15T15:54:07.470 INFO:tasks.workunit.client.0.smithi049.stdout:0/341: fsync dc/d13/d14/d33/f5b 0 2024-05-15T15:54:07.473 INFO:tasks.workunit.client.1.smithi144.stdout:1/315: truncate d7/d1d/d33/d1e/f25 821857 0 2024-05-15T15:54:07.473 INFO:tasks.workunit.client.1.smithi144.stdout:6/283: dread d7/f8 [4194304,4194304] 0 2024-05-15T15:54:07.477 INFO:tasks.workunit.client.0.smithi049.stdout:8/341: write d2/d18/f3c [888090,54353] 0 2024-05-15T15:54:07.479 INFO:tasks.workunit.client.0.smithi049.stdout:8/342: chown d2/de/d25/f26 7 1 2024-05-15T15:54:07.481 INFO:tasks.workunit.client.1.smithi144.stdout:2/299: symlink d3/d7/d20/d22/d5e/l63 0 2024-05-15T15:54:07.482 INFO:tasks.workunit.client.0.smithi049.stdout:5/390: dwrite dc/dd/f17 [0,4194304] 0 2024-05-15T15:54:07.482 INFO:tasks.workunit.client.1.smithi144.stdout:2/300: write d3/d10/d15/f26 [125026,61258] 0 2024-05-15T15:54:07.483 INFO:tasks.workunit.client.1.smithi144.stdout:2/301: write d3/d7/f47 [76021,33553] 0 2024-05-15T15:54:07.484 INFO:tasks.workunit.client.1.smithi144.stdout:2/302: read - d3/d10/f35 zero size 2024-05-15T15:54:07.484 INFO:tasks.workunit.client.0.smithi049.stdout:7/306: creat d0/d6/d1d/f59 x:0 0 0 2024-05-15T15:54:07.486 INFO:tasks.workunit.client.1.smithi144.stdout:0/326: mknod d4/df/d1f/d24/d2b/c77 0 2024-05-15T15:54:07.487 INFO:tasks.workunit.client.1.smithi144.stdout:0/327: readlink d4/dc/l23 0 2024-05-15T15:54:07.487 INFO:tasks.workunit.client.0.smithi049.stdout:7/307: truncate d0/d2f/f48 941316 0 2024-05-15T15:54:07.488 INFO:tasks.workunit.client.0.smithi049.stdout:1/452: truncate d0/d4/dd/de/d1e/d16/d8a/f99 241315 0 2024-05-15T15:54:07.490 INFO:tasks.workunit.client.0.smithi049.stdout:1/453: chown d0/d4/l14 69846490 1 2024-05-15T15:54:07.492 INFO:tasks.workunit.client.0.smithi049.stdout:5/391: dwrite dc/dd/d47/f5d [0,4194304] 0 2024-05-15T15:54:07.493 INFO:tasks.workunit.client.1.smithi144.stdout:9/320: creat d0/f78 x:0 0 0 2024-05-15T15:54:07.495 INFO:tasks.workunit.client.0.smithi049.stdout:9/247: dread - d0/d25/f4c zero size 2024-05-15T15:54:07.501 INFO:tasks.workunit.client.1.smithi144.stdout:4/312: creat d6/dd/d2e/f61 x:0 0 0 2024-05-15T15:54:07.502 INFO:tasks.workunit.client.0.smithi049.stdout:0/342: symlink dc/d13/d3d/l6b 0 2024-05-15T15:54:07.503 INFO:tasks.workunit.client.1.smithi144.stdout:8/327: mkdir d2/d1a/d38/d4b/d5b/d6e/d71/d73 0 2024-05-15T15:54:07.503 INFO:tasks.workunit.client.0.smithi049.stdout:0/343: write dc/d13/d14/d2a/f3a [4054381,75720] 0 2024-05-15T15:54:07.509 INFO:tasks.workunit.client.1.smithi144.stdout:6/284: rename d7/db/f1d to d7/d62/f6b 0 2024-05-15T15:54:07.510 INFO:tasks.workunit.client.1.smithi144.stdout:6/285: write d7/db/f3f [2091812,61375] 0 2024-05-15T15:54:07.517 INFO:tasks.workunit.client.0.smithi049.stdout:5/392: creat dc/d29/f7c x:0 0 0 2024-05-15T15:54:07.517 INFO:tasks.workunit.client.0.smithi049.stdout:5/393: stat dc/d4a/d42 0 2024-05-15T15:54:07.518 INFO:tasks.workunit.client.0.smithi049.stdout:5/394: dread - dc/dd/d39/f77 zero size 2024-05-15T15:54:07.519 INFO:tasks.workunit.client.0.smithi049.stdout:9/248: symlink d0/d8/d1b/l51 0 2024-05-15T15:54:07.520 INFO:tasks.workunit.client.1.smithi144.stdout:0/328: fdatasync d4/df/f46 0 2024-05-15T15:54:07.521 INFO:tasks.workunit.client.1.smithi144.stdout:7/322: dwrite d4/fc [0,4194304] 0 2024-05-15T15:54:07.522 INFO:tasks.workunit.client.1.smithi144.stdout:5/316: dwrite d0/dc/d12/f18 [0,4194304] 0 2024-05-15T15:54:07.522 INFO:tasks.workunit.client.1.smithi144.stdout:7/323: chown d4/f46 12714 1 2024-05-15T15:54:07.529 INFO:tasks.workunit.client.0.smithi049.stdout:3/389: dwrite d12/f1d [4194304,4194304] 0 2024-05-15T15:54:07.530 INFO:tasks.workunit.client.0.smithi049.stdout:9/249: dwrite d0/d8/d1b/f3a [0,4194304] 0 2024-05-15T15:54:07.535 INFO:tasks.workunit.client.0.smithi049.stdout:9/250: write d0/f41 [454597,116293] 0 2024-05-15T15:54:07.535 INFO:tasks.workunit.client.0.smithi049.stdout:0/344: fdatasync dc/f34 0 2024-05-15T15:54:07.536 INFO:tasks.workunit.client.1.smithi144.stdout:7/324: dread d4/d41/f49 [0,4194304] 0 2024-05-15T15:54:07.536 INFO:tasks.workunit.client.1.smithi144.stdout:8/328: mknod d2/d3a/c74 0 2024-05-15T15:54:07.538 INFO:tasks.workunit.client.0.smithi049.stdout:6/303: link c3 dd/d17/d24/d2b/d36/d50/c51 0 2024-05-15T15:54:07.538 INFO:tasks.workunit.client.1.smithi144.stdout:3/321: truncate d0/d1/f23 227067 0 2024-05-15T15:54:07.539 INFO:tasks.workunit.client.0.smithi049.stdout:3/390: dread d12/d14/f4f [0,4194304] 0 2024-05-15T15:54:07.540 INFO:tasks.workunit.client.1.smithi144.stdout:3/322: dread - d0/d1/f43 zero size 2024-05-15T15:54:07.541 INFO:tasks.workunit.client.1.smithi144.stdout:9/321: dread d0/f18 [0,4194304] 0 2024-05-15T15:54:07.542 INFO:tasks.workunit.client.1.smithi144.stdout:3/323: write d0/d1/f43 [627745,98145] 0 2024-05-15T15:54:07.544 INFO:tasks.workunit.client.0.smithi049.stdout:3/391: write d12/f1d [6394970,3130] 0 2024-05-15T15:54:07.545 INFO:tasks.workunit.client.0.smithi049.stdout:6/304: dread dd/d17/d24/d2b/f4d [0,4194304] 0 2024-05-15T15:54:07.545 INFO:tasks.workunit.client.1.smithi144.stdout:1/316: mknod d7/d1d/d58/c62 0 2024-05-15T15:54:07.548 INFO:tasks.workunit.client.1.smithi144.stdout:4/313: dread d6/dd/d34/d3b/d30/f1b [0,4194304] 0 2024-05-15T15:54:07.548 INFO:tasks.workunit.client.0.smithi049.stdout:0/345: dwrite dc/d13/d48/f50 [0,4194304] 0 2024-05-15T15:54:07.548 INFO:tasks.workunit.client.0.smithi049.stdout:6/305: chown dd/d17/d24/d2b/d36 410 1 2024-05-15T15:54:07.549 INFO:tasks.workunit.client.1.smithi144.stdout:4/314: chown d6/f29 1364704073 1 2024-05-15T15:54:07.550 INFO:tasks.workunit.client.0.smithi049.stdout:1/454: symlink d0/d4/dd/de/d48/d5c/da2/la6 0 2024-05-15T15:54:07.552 INFO:tasks.workunit.client.1.smithi144.stdout:3/324: dwrite d0/d11/d24/f3e [0,4194304] 0 2024-05-15T15:54:07.554 INFO:tasks.workunit.client.1.smithi144.stdout:2/303: rename d3/d10/d15 to d3/d7/d20/d22/d52/d64 0 2024-05-15T15:54:07.558 INFO:tasks.workunit.client.1.smithi144.stdout:4/315: dwrite d6/d49/f5c [0,4194304] 0 2024-05-15T15:54:07.560 INFO:tasks.workunit.client.0.smithi049.stdout:9/251: unlink d0/d8/d9/d22/d26/f4e 0 2024-05-15T15:54:07.565 INFO:tasks.workunit.client.1.smithi144.stdout:7/325: creat d4/df/d10/f6e x:0 0 0 2024-05-15T15:54:07.566 INFO:tasks.workunit.client.0.smithi049.stdout:3/392: dread d12/f41 [0,4194304] 0 2024-05-15T15:54:07.567 INFO:tasks.workunit.client.1.smithi144.stdout:9/322: write d0/d8/d5e/d76/f54 [961005,76104] 0 2024-05-15T15:54:07.568 INFO:tasks.workunit.client.0.smithi049.stdout:3/393: write d12/d30/f3d [999631,117040] 0 2024-05-15T15:54:07.569 INFO:tasks.workunit.client.0.smithi049.stdout:3/394: stat d12/d14/d32/c39 0 2024-05-15T15:54:07.571 INFO:tasks.workunit.client.0.smithi049.stdout:7/308: creat d0/d6/d23/d45/f5a x:0 0 0 2024-05-15T15:54:07.573 INFO:tasks.workunit.client.1.smithi144.stdout:1/317: mknod d7/d1d/d33/c63 0 2024-05-15T15:54:07.576 INFO:tasks.workunit.client.0.smithi049.stdout:6/306: rmdir dd/d17/d24/d2b/d36 39 2024-05-15T15:54:07.577 INFO:tasks.workunit.client.0.smithi049.stdout:6/307: fdatasync dd/d17/d24/f27 0 2024-05-15T15:54:07.578 INFO:tasks.workunit.client.0.smithi049.stdout:6/308: fdatasync dd/f14 0 2024-05-15T15:54:07.578 INFO:tasks.workunit.client.0.smithi049.stdout:8/343: truncate d2/d4f/f51 277349 0 2024-05-15T15:54:07.579 INFO:tasks.workunit.client.0.smithi049.stdout:6/309: chown dd/d17/d24/d2b 1896 1 2024-05-15T15:54:07.579 INFO:tasks.workunit.client.0.smithi049.stdout:1/455: rmdir d0/d4/dd/de 39 2024-05-15T15:54:07.583 INFO:tasks.workunit.client.0.smithi049.stdout:5/395: rename dc/dd/d10/f59 to dc/d4a/d31/d5c/f7d 0 2024-05-15T15:54:07.585 INFO:tasks.workunit.client.1.smithi144.stdout:2/304: mkdir d3/d7/d20/d37/d3a/d65 0 2024-05-15T15:54:07.591 INFO:tasks.workunit.client.0.smithi049.stdout:8/344: dread d2/d14/d1b/f2b [0,4194304] 0 2024-05-15T15:54:07.592 INFO:tasks.workunit.client.1.smithi144.stdout:0/329: symlink d4/d6/d4a/d74/l78 0 2024-05-15T15:54:07.595 INFO:tasks.workunit.client.1.smithi144.stdout:8/329: mkdir d2/de/d3f/d6b/d75 0 2024-05-15T15:54:07.596 INFO:tasks.workunit.client.0.smithi049.stdout:3/395: creat d12/d14/d32/d64/f78 x:0 0 0 2024-05-15T15:54:07.598 INFO:tasks.workunit.client.1.smithi144.stdout:7/326: rename d4/df/d25/f4a to d4/d41/d5b/f6f 0 2024-05-15T15:54:07.601 INFO:tasks.workunit.client.1.smithi144.stdout:9/323: rmdir d0/d8/d5e/d44/d16/d5a 39 2024-05-15T15:54:07.601 INFO:tasks.workunit.client.1.smithi144.stdout:9/324: stat d0/d8/l5c 0 2024-05-15T15:54:07.601 INFO:tasks.workunit.client.1.smithi144.stdout:9/325: fsync d0/d8/d50/f69 0 2024-05-15T15:54:07.602 INFO:tasks.workunit.client.0.smithi049.stdout:0/346: fsync dc/d13/d3d/f5d 0 2024-05-15T15:54:07.604 INFO:tasks.workunit.client.0.smithi049.stdout:0/347: read dc/d13/d14/d25/d31/f61 [161131,37190] 0 2024-05-15T15:54:07.604 INFO:tasks.workunit.client.1.smithi144.stdout:1/318: unlink d7/df/d24/l3d 0 2024-05-15T15:54:07.607 INFO:tasks.workunit.client.1.smithi144.stdout:3/325: mkdir d0/d11/d5e/d72 0 2024-05-15T15:54:07.608 INFO:tasks.workunit.client.1.smithi144.stdout:3/326: chown d0/f2e 959 1 2024-05-15T15:54:07.608 INFO:tasks.workunit.client.0.smithi049.stdout:0/348: dread dc/d13/f17 [4194304,4194304] 0 2024-05-15T15:54:07.609 INFO:tasks.workunit.client.1.smithi144.stdout:3/327: dread d0/d1/d17/f19 [0,4194304] 0 2024-05-15T15:54:07.611 INFO:tasks.workunit.client.1.smithi144.stdout:6/286: link d7/db/c6a d7/db/d11/d3c/d61/c6c 0 2024-05-15T15:54:07.612 INFO:tasks.workunit.client.0.smithi049.stdout:1/456: chown d0/d4/dd/de/d4d/l71 13563 1 2024-05-15T15:54:07.614 INFO:tasks.workunit.client.1.smithi144.stdout:0/330: creat d4/d6/d4a/d74/f79 x:0 0 0 2024-05-15T15:54:07.615 INFO:tasks.workunit.client.1.smithi144.stdout:9/326: dread d0/f59 [0,4194304] 0 2024-05-15T15:54:07.617 INFO:tasks.workunit.client.1.smithi144.stdout:8/330: dread - d2/de/d36/f1e zero size 2024-05-15T15:54:07.618 INFO:tasks.workunit.client.0.smithi049.stdout:9/252: mkdir d0/d8/d9/d52 0 2024-05-15T15:54:07.618 INFO:tasks.workunit.client.1.smithi144.stdout:8/331: truncate d2/de/d49/f69 885076 0 2024-05-15T15:54:07.619 INFO:tasks.workunit.client.1.smithi144.stdout:8/332: fdatasync d2/de/d36/f4d 0 2024-05-15T15:54:07.619 INFO:tasks.workunit.client.1.smithi144.stdout:8/333: write d2/d24/d2d/f5e [467761,121327] 0 2024-05-15T15:54:07.621 INFO:tasks.workunit.client.1.smithi144.stdout:7/327: creat d4/d41/d5b/f70 x:0 0 0 2024-05-15T15:54:07.623 INFO:tasks.workunit.client.0.smithi049.stdout:7/309: link d0/d2f/f51 d0/d6/da/d3d/d39/f5b 0 2024-05-15T15:54:07.625 INFO:tasks.workunit.client.1.smithi144.stdout:1/319: creat d7/d1d/d33/d29/f64 x:0 0 0 2024-05-15T15:54:07.625 INFO:tasks.workunit.client.1.smithi144.stdout:8/334: dwrite d2/de/f14 [0,4194304] 0 2024-05-15T15:54:07.626 INFO:tasks.workunit.client.1.smithi144.stdout:8/335: fdatasync d2/de/d36/f48 0 2024-05-15T15:54:07.627 INFO:tasks.workunit.client.1.smithi144.stdout:3/328: fsync d0/d11/f14 0 2024-05-15T15:54:07.627 INFO:tasks.workunit.client.1.smithi144.stdout:8/336: write d2/f15 [1860998,90214] 0 2024-05-15T15:54:07.630 INFO:tasks.workunit.client.1.smithi144.stdout:2/305: symlink d3/l66 0 2024-05-15T15:54:07.630 INFO:tasks.workunit.client.0.smithi049.stdout:7/310: dwrite d0/d34/f35 [8388608,4194304] 0 2024-05-15T15:54:07.631 INFO:tasks.workunit.client.1.smithi144.stdout:2/306: read d3/d7/f61 [248321,128023] 0 2024-05-15T15:54:07.638 INFO:tasks.workunit.client.1.smithi144.stdout:2/307: dwrite d3/d7/d20/d3b/f4f [0,4194304] 0 2024-05-15T15:54:07.644 INFO:tasks.workunit.client.0.smithi049.stdout:6/310: dread dd/d17/d24/d2b/d36/f42 [0,4194304] 0 2024-05-15T15:54:07.644 INFO:tasks.workunit.client.1.smithi144.stdout:9/327: write d0/d8/d5e/f32 [491126,40122] 0 2024-05-15T15:54:07.645 INFO:tasks.workunit.client.1.smithi144.stdout:9/328: write d0/d8/d5e/d44/f53 [974135,100635] 0 2024-05-15T15:54:07.647 INFO:tasks.workunit.client.1.smithi144.stdout:9/329: truncate d0/d8/d1b/f6f 828337 0 2024-05-15T15:54:07.649 INFO:tasks.workunit.client.0.smithi049.stdout:8/345: truncate f0 2592213 0 2024-05-15T15:54:07.650 INFO:tasks.workunit.client.1.smithi144.stdout:7/328: creat d4/df/d25/d38/f71 x:0 0 0 2024-05-15T15:54:07.650 INFO:tasks.workunit.client.1.smithi144.stdout:6/287: dwrite d7/d1e/d28/f57 [0,4194304] 0 2024-05-15T15:54:07.651 INFO:tasks.workunit.client.0.smithi049.stdout:9/253: mkdir d0/d8/d53 0 2024-05-15T15:54:07.652 INFO:tasks.workunit.client.1.smithi144.stdout:1/320: creat d7/d1d/d52/f65 x:0 0 0 2024-05-15T15:54:07.653 INFO:tasks.workunit.client.1.smithi144.stdout:1/321: fdatasync f1 0 2024-05-15T15:54:07.658 INFO:tasks.workunit.client.0.smithi049.stdout:6/311: dread dd/f16 [0,4194304] 0 2024-05-15T15:54:07.658 INFO:tasks.workunit.client.1.smithi144.stdout:6/288: dwrite d7/d1e/d28/f3d [0,4194304] 0 2024-05-15T15:54:07.659 INFO:tasks.workunit.client.1.smithi144.stdout:3/329: write d0/d11/d3b/f64 [4780263,17702] 0 2024-05-15T15:54:07.660 INFO:tasks.workunit.client.1.smithi144.stdout:6/289: chown d7/db/d11/f43 0 1 2024-05-15T15:54:07.661 INFO:tasks.workunit.client.1.smithi144.stdout:8/337: rename d2/f15 to d2/de/d49/f76 0 2024-05-15T15:54:07.662 INFO:tasks.workunit.client.1.smithi144.stdout:6/290: read f6 [3088455,11555] 0 2024-05-15T15:54:07.663 INFO:tasks.workunit.client.0.smithi049.stdout:1/457: write d0/d4/dd/f38 [171500,10281] 0 2024-05-15T15:54:07.665 INFO:tasks.workunit.client.1.smithi144.stdout:3/330: dread d0/d11/d3b/f45 [0,4194304] 0 2024-05-15T15:54:07.666 INFO:tasks.workunit.client.1.smithi144.stdout:3/331: fsync d0/d11/f26 0 2024-05-15T15:54:07.667 INFO:tasks.workunit.client.0.smithi049.stdout:0/349: link dc/d13/d14/d2a/f2c dc/d13/d48/f6c 0 2024-05-15T15:54:07.673 INFO:tasks.workunit.client.0.smithi049.stdout:0/350: dread dc/d13/d14/d2a/f2c [0,4194304] 0 2024-05-15T15:54:07.677 INFO:tasks.workunit.client.1.smithi144.stdout:4/316: getdents d6/dd/d34/d3b/d30/db/d1d/d40 0 2024-05-15T15:54:07.678 INFO:tasks.workunit.client.1.smithi144.stdout:5/317: getdents d0/dc/d12/d31 0 2024-05-15T15:54:07.679 INFO:tasks.workunit.client.1.smithi144.stdout:9/330: mkdir d0/d8/d1b/d79 0 2024-05-15T15:54:07.680 INFO:tasks.workunit.client.1.smithi144.stdout:7/329: fdatasync d4/d41/d5b/f6f 0 2024-05-15T15:54:07.684 INFO:tasks.workunit.client.0.smithi049.stdout:7/311: unlink d0/f13 0 2024-05-15T15:54:07.684 INFO:tasks.workunit.client.1.smithi144.stdout:9/331: dread d0/d8/d5e/d44/d10/f58 [0,4194304] 0 2024-05-15T15:54:07.685 INFO:tasks.workunit.client.1.smithi144.stdout:4/317: dwrite d6/f29 [0,4194304] 0 2024-05-15T15:54:07.687 INFO:tasks.workunit.client.0.smithi049.stdout:1/458: creat d0/d4/dd/de/d4e/fa7 x:0 0 0 2024-05-15T15:54:07.689 INFO:tasks.workunit.client.0.smithi049.stdout:1/459: write d0/d4/dd/de/d1e/f31 [576944,65496] 0 2024-05-15T15:54:07.690 INFO:tasks.workunit.client.1.smithi144.stdout:7/330: dwrite d4/f27 [0,4194304] 0 2024-05-15T15:54:07.690 INFO:tasks.workunit.client.1.smithi144.stdout:9/332: dread d0/d8/d5e/d44/d10/d19/f49 [0,4194304] 0 2024-05-15T15:54:07.698 INFO:tasks.workunit.client.1.smithi144.stdout:2/308: write d3/d7/d20/d22/d52/d64/f31 [1713124,116032] 0 2024-05-15T15:54:07.702 INFO:tasks.workunit.client.0.smithi049.stdout:3/396: truncate d12/d14/f1b 3254510 0 2024-05-15T15:54:07.703 INFO:tasks.workunit.client.0.smithi049.stdout:8/346: mknod d2/d4f/d5d/d63/c67 0 2024-05-15T15:54:07.705 INFO:tasks.workunit.client.0.smithi049.stdout:6/312: creat dd/d17/d24/d2b/d36/d50/f52 x:0 0 0 2024-05-15T15:54:07.706 INFO:tasks.workunit.client.0.smithi049.stdout:7/312: mkdir d0/d2f/d5c 0 2024-05-15T15:54:07.838 INFO:tasks.workunit.client.0.smithi049.stdout:9/254: creat d0/f54 x:0 0 0 2024-05-15T15:54:07.840 INFO:tasks.workunit.client.1.smithi144.stdout:3/332: write d0/f2c [853730,127373] 0 2024-05-15T15:54:07.841 INFO:tasks.workunit.client.1.smithi144.stdout:3/333: readlink d0/d1/d17/d39/d40/l6a 0 2024-05-15T15:54:07.842 INFO:tasks.workunit.client.1.smithi144.stdout:9/333: rmdir d0/d8/d5e 39 2024-05-15T15:54:07.845 INFO:tasks.workunit.client.0.smithi049.stdout:7/313: dwrite d0/d6/d23/f4c [0,4194304] 0 2024-05-15T15:54:07.856 INFO:tasks.workunit.client.0.smithi049.stdout:1/460: write d0/d4/dd/de/f17 [5906636,117957] 0 2024-05-15T15:54:07.857 INFO:tasks.workunit.client.1.smithi144.stdout:2/309: rmdir d3/d7/d20/d22/d5e 39 2024-05-15T15:54:07.858 INFO:tasks.workunit.client.1.smithi144.stdout:4/318: write d6/d42/d46/d4b/f4d [4112872,80092] 0 2024-05-15T15:54:07.863 INFO:tasks.workunit.client.1.smithi144.stdout:0/331: getdents d4/df/d1f/d24 0 2024-05-15T15:54:07.868 INFO:tasks.workunit.client.1.smithi144.stdout:1/322: mkdir d7/d1d/d33/d41/d22/d66 0 2024-05-15T15:54:07.876 INFO:tasks.workunit.client.1.smithi144.stdout:6/291: link d7/db/d11/c24 d7/db/d35/c6d 0 2024-05-15T15:54:07.877 INFO:tasks.workunit.client.1.smithi144.stdout:6/292: readlink d7/d1e/l46 0 2024-05-15T15:54:07.877 INFO:tasks.workunit.client.1.smithi144.stdout:6/293: read d7/db/d11/f15 [852895,2309] 0 2024-05-15T15:54:07.878 INFO:tasks.workunit.client.1.smithi144.stdout:6/294: fsync d7/db/d11/d55/f5e 0 2024-05-15T15:54:07.881 INFO:tasks.workunit.client.1.smithi144.stdout:4/319: symlink d6/d42/d46/d4b/l62 0 2024-05-15T15:54:07.885 INFO:tasks.workunit.client.1.smithi144.stdout:0/332: rename d4/df/d1f/d24/f2a to d4/d6/d3a/d4c/f7a 0 2024-05-15T15:54:07.887 INFO:tasks.workunit.client.1.smithi144.stdout:3/334: mkdir d0/d1/d56/d71/d73 0 2024-05-15T15:54:07.888 INFO:tasks.workunit.client.1.smithi144.stdout:3/335: write d0/d11/f14 [4008469,127132] 0 2024-05-15T15:54:07.889 INFO:tasks.workunit.client.0.smithi049.stdout:0/351: rename dc/d13/d14/d25/d31/d38 to dc/d6d 0 2024-05-15T15:54:07.891 INFO:tasks.workunit.client.1.smithi144.stdout:9/334: mkdir d0/d8/d7a 0 2024-05-15T15:54:07.892 INFO:tasks.workunit.client.0.smithi049.stdout:3/397: symlink d12/d30/d66/l79 0 2024-05-15T15:54:07.893 INFO:tasks.workunit.client.1.smithi144.stdout:8/338: link d2/d24/d2d/d30/l55 d2/d1a/d38/d4b/d5b/d6e/d71/d73/l77 0 2024-05-15T15:54:07.897 INFO:tasks.workunit.client.1.smithi144.stdout:6/295: rmdir d7/db 39 2024-05-15T15:54:07.897 INFO:tasks.workunit.client.1.smithi144.stdout:6/296: fsync d7/d1e/f69 0 2024-05-15T15:54:07.899 INFO:tasks.workunit.client.1.smithi144.stdout:5/318: getdents d0/dc/d1f 0 2024-05-15T15:54:07.902 INFO:tasks.workunit.client.0.smithi049.stdout:8/347: rename d2/d14/d1d/d3e to d2/d14/d1b/d68 0 2024-05-15T15:54:07.903 INFO:tasks.workunit.client.1.smithi144.stdout:7/331: truncate d4/df/f11 982165 0 2024-05-15T15:54:07.904 INFO:tasks.workunit.client.1.smithi144.stdout:4/320: rename d6/d42/d46/d4b/d1e/c5a to d6/d42/d46/d4b/c63 0 2024-05-15T15:54:07.905 INFO:tasks.workunit.client.0.smithi049.stdout:0/352: fdatasync dc/d13/f22 0 2024-05-15T15:54:07.909 INFO:tasks.workunit.client.1.smithi144.stdout:0/333: unlink d4/d6/c12 0 2024-05-15T15:54:07.912 INFO:tasks.workunit.client.1.smithi144.stdout:3/336: rmdir d0/d1/d17/d39/d40 39 2024-05-15T15:54:07.916 INFO:tasks.workunit.client.0.smithi049.stdout:8/348: mkdir d2/d14/d1b/d69 0 2024-05-15T15:54:07.916 INFO:tasks.workunit.client.0.smithi049.stdout:8/349: chown d2/de/c52 5299104 1 2024-05-15T15:54:07.918 INFO:tasks.workunit.client.1.smithi144.stdout:2/310: link d3/d7/f47 d3/d7/d20/d3f/f67 0 2024-05-15T15:54:07.919 INFO:tasks.workunit.client.1.smithi144.stdout:2/311: chown d3/d7/d20/d22/f62 1 1 2024-05-15T15:54:07.921 INFO:tasks.workunit.client.0.smithi049.stdout:9/255: write d0/d8/d21/f2f [2496689,94441] 0 2024-05-15T15:54:07.925 INFO:tasks.workunit.client.1.smithi144.stdout:5/319: symlink d0/dc/d24/d28/d2d/l66 0 2024-05-15T15:54:07.926 INFO:tasks.workunit.client.1.smithi144.stdout:5/320: dread d0/dc/d12/d31/d35/f44 [0,4194304] 0 2024-05-15T15:54:07.927 INFO:tasks.workunit.client.1.smithi144.stdout:5/321: stat d0/dc/d24/d28/d2d/l66 0 2024-05-15T15:54:07.927 INFO:tasks.workunit.client.1.smithi144.stdout:7/332: readlink d4/d45/l6c 0 2024-05-15T15:54:07.930 INFO:tasks.workunit.client.1.smithi144.stdout:4/321: mknod d6/d42/c64 0 2024-05-15T15:54:07.933 INFO:tasks.workunit.client.1.smithi144.stdout:5/322: dwrite d0/dc/d24/d28/d2d/f2b [0,4194304] 0 2024-05-15T15:54:07.936 INFO:tasks.workunit.client.1.smithi144.stdout:3/337: creat d0/d11/d24/f74 x:0 0 0 2024-05-15T15:54:07.937 INFO:tasks.workunit.client.0.smithi049.stdout:0/353: write dc/d13/f26 [1399517,122912] 0 2024-05-15T15:54:07.938 INFO:tasks.workunit.client.0.smithi049.stdout:0/354: chown dc/d6d 5355 1 2024-05-15T15:54:07.938 INFO:tasks.workunit.client.0.smithi049.stdout:9/256: chown d0/d8/d9/l46 240578270 1 2024-05-15T15:54:07.940 INFO:tasks.workunit.client.1.smithi144.stdout:9/335: mknod d0/d8/d5e/d44/d10/d19/c7b 0 2024-05-15T15:54:07.940 INFO:tasks.workunit.client.0.smithi049.stdout:0/355: write dc/d13/d48/d57/f65 [265368,80097] 0 2024-05-15T15:54:07.941 INFO:tasks.workunit.client.0.smithi049.stdout:0/356: chown dc/d13 103238191 1 2024-05-15T15:54:07.941 INFO:tasks.workunit.client.0.smithi049.stdout:3/398: rmdir d12/d30/d42/d71 0 2024-05-15T15:54:07.942 INFO:tasks.workunit.client.1.smithi144.stdout:9/336: write d0/d8/d5e/d44/d10/fd [2413155,3199] 0 2024-05-15T15:54:07.943 INFO:tasks.workunit.client.0.smithi049.stdout:0/357: fdatasync dc/d13/d14/d25/f69 0 2024-05-15T15:54:07.943 INFO:tasks.workunit.client.0.smithi049.stdout:3/399: chown d12/d30/d42/f48 4072014 1 2024-05-15T15:54:07.944 INFO:tasks.workunit.client.0.smithi049.stdout:1/461: getdents d0/d4/dd/de/d48 0 2024-05-15T15:54:07.944 INFO:tasks.workunit.client.0.smithi049.stdout:0/358: chown dc/d6d/l59 1924 1 2024-05-15T15:54:07.945 INFO:tasks.workunit.client.0.smithi049.stdout:0/359: stat dc/d13/c55 0 2024-05-15T15:54:07.947 INFO:tasks.workunit.client.0.smithi049.stdout:6/313: rename dd/d17/d24/d2b/f48 to dd/f53 0 2024-05-15T15:54:07.950 INFO:tasks.workunit.client.0.smithi049.stdout:8/350: dread d2/de/d27/f39 [0,4194304] 0 2024-05-15T15:54:07.950 INFO:tasks.workunit.client.0.smithi049.stdout:3/400: mkdir d12/d30/d42/d7a 0 2024-05-15T15:54:07.950 INFO:tasks.workunit.client.0.smithi049.stdout:3/401: readlink d12/d26/l2e 0 2024-05-15T15:54:07.951 INFO:tasks.workunit.client.1.smithi144.stdout:7/333: rmdir d4/df/d25/d38 39 2024-05-15T15:54:07.952 INFO:tasks.workunit.client.0.smithi049.stdout:1/462: dread d0/d4/dd/de/d1e/d2e/f88 [0,4194304] 0 2024-05-15T15:54:07.954 INFO:tasks.workunit.client.0.smithi049.stdout:0/360: unlink dc/d6d/l3b 0 2024-05-15T15:54:07.955 INFO:tasks.workunit.client.0.smithi049.stdout:0/361: write dc/d13/d3d/f43 [128067,24352] 0 2024-05-15T15:54:07.956 INFO:tasks.workunit.client.0.smithi049.stdout:9/257: rename d0/d25/f31 to d0/d8/d9/d22/d26/f55 0 2024-05-15T15:54:07.957 INFO:tasks.workunit.client.0.smithi049.stdout:9/258: readlink d0/d8/d9/l18 0 2024-05-15T15:54:07.958 INFO:tasks.workunit.client.0.smithi049.stdout:9/259: dread - d0/d8/d9/d45/f4a zero size 2024-05-15T15:54:07.959 INFO:tasks.workunit.client.1.smithi144.stdout:5/323: rmdir d0/dc/d24 39 2024-05-15T15:54:07.963 INFO:tasks.workunit.client.1.smithi144.stdout:1/323: getdents d7/d1d/d52 0 2024-05-15T15:54:07.964 INFO:tasks.workunit.client.0.smithi049.stdout:3/402: creat d12/d14/d52/f7b x:0 0 0 2024-05-15T15:54:07.967 INFO:tasks.workunit.client.1.smithi144.stdout:9/337: creat d0/d8/d1b/d62/d6b/f7c x:0 0 0 2024-05-15T15:54:07.971 INFO:tasks.workunit.client.1.smithi144.stdout:5/324: dread d0/dc/d1f/f3b [0,4194304] 0 2024-05-15T15:54:07.972 INFO:tasks.workunit.client.0.smithi049.stdout:6/314: dwrite dd/d26/f40 [0,4194304] 0 2024-05-15T15:54:07.972 INFO:tasks.workunit.client.1.smithi144.stdout:5/325: read - d0/dc/d12/d31/d35/f62 zero size 2024-05-15T15:54:08.010 INFO:tasks.workunit.client.0.smithi049.stdout:1/463: rename d0/d4/dd/de/d48/c7c to d0/d4/dd/de/d48/d84/ca8 0 2024-05-15T15:54:08.010 INFO:tasks.workunit.client.0.smithi049.stdout:8/351: write d2/d14/d1b/f36 [2352427,81817] 0 2024-05-15T15:54:08.010 INFO:tasks.workunit.client.0.smithi049.stdout:9/260: rmdir d0/d8/d9 39 2024-05-15T15:54:08.012 INFO:tasks.workunit.client.1.smithi144.stdout:2/312: creat d3/d7/d20/d37/d3a/d65/f68 x:0 0 0 2024-05-15T15:54:08.013 INFO:tasks.workunit.client.1.smithi144.stdout:6/297: link d7/d1e/d28/f57 d7/f6e 0 2024-05-15T15:54:08.014 INFO:tasks.workunit.client.0.smithi049.stdout:3/403: unlink l6 0 2024-05-15T15:54:08.015 INFO:tasks.workunit.client.1.smithi144.stdout:7/334: mkdir d4/df/d10/d48/d72 0 2024-05-15T15:54:08.016 INFO:tasks.workunit.client.0.smithi049.stdout:6/315: mkdir dd/d17/d24/d2b/d36/d50/d54 0 2024-05-15T15:54:08.018 INFO:tasks.workunit.client.0.smithi049.stdout:6/316: truncate dd/d17/d24/d2b/d36/d50/f52 860599 0 2024-05-15T15:54:08.021 INFO:tasks.workunit.client.1.smithi144.stdout:7/335: dwrite d4/d41/d5b/f70 [0,4194304] 0 2024-05-15T15:54:08.022 INFO:tasks.workunit.client.0.smithi049.stdout:1/464: fsync d0/d4/dd/f5a 0 2024-05-15T15:54:08.022 INFO:tasks.workunit.client.1.smithi144.stdout:7/336: dread - d4/df/d25/d40/d62/f6b zero size 2024-05-15T15:54:08.028 INFO:tasks.workunit.client.0.smithi049.stdout:3/404: dread - d12/d14/f54 zero size 2024-05-15T15:54:08.030 INFO:tasks.workunit.client.1.smithi144.stdout:3/338: mknod d0/d1/c75 0 2024-05-15T15:54:08.032 INFO:tasks.workunit.client.0.smithi049.stdout:0/362: rename dc/f1b to dc/d13/d48/d57/f6e 0 2024-05-15T15:54:08.034 INFO:tasks.workunit.client.1.smithi144.stdout:9/338: fsync d0/f18 0 2024-05-15T15:54:08.036 INFO:tasks.workunit.client.0.smithi049.stdout:3/405: dwrite d12/f47 [0,4194304] 0 2024-05-15T15:54:08.037 INFO:tasks.workunit.client.1.smithi144.stdout:5/326: read d0/dc/d24/d28/d2d/f2b [1202340,13297] 0 2024-05-15T15:54:08.038 INFO:tasks.workunit.client.0.smithi049.stdout:8/352: mkdir d2/d6a 0 2024-05-15T15:54:08.039 INFO:tasks.workunit.client.0.smithi049.stdout:8/353: readlink d2/d18/l64 0 2024-05-15T15:54:08.040 INFO:tasks.workunit.client.1.smithi144.stdout:8/339: getdents d2/de/d3f/d6b 0 2024-05-15T15:54:08.040 INFO:tasks.workunit.client.0.smithi049.stdout:6/317: dread fb [4194304,4194304] 0 2024-05-15T15:54:08.040 INFO:tasks.workunit.client.1.smithi144.stdout:8/340: stat d2/de/d36/l63 0 2024-05-15T15:54:08.041 INFO:tasks.workunit.client.0.smithi049.stdout:8/354: write d2/de/d27/f35 [4886405,78610] 0 2024-05-15T15:54:08.041 INFO:tasks.workunit.client.1.smithi144.stdout:8/341: truncate d2/de/d36/f2c 903151 0 2024-05-15T15:54:08.041 INFO:tasks.workunit.client.0.smithi049.stdout:8/355: chown d2/f2c 9089 1 2024-05-15T15:54:08.042 INFO:tasks.workunit.client.1.smithi144.stdout:8/342: chown d2/d24/d2d/d30/f60 7294115 1 2024-05-15T15:54:08.042 INFO:tasks.workunit.client.0.smithi049.stdout:8/356: write d2/de/d27/f35 [293534,2045] 0 2024-05-15T15:54:08.043 INFO:tasks.workunit.client.1.smithi144.stdout:8/343: dread - d2/de/d3f/f40 zero size 2024-05-15T15:54:08.044 INFO:tasks.workunit.client.1.smithi144.stdout:8/344: dread - d2/d3a/f67 zero size 2024-05-15T15:54:08.044 INFO:tasks.workunit.client.1.smithi144.stdout:2/313: unlink d3/d7/d20/l48 0 2024-05-15T15:54:08.045 INFO:tasks.workunit.client.0.smithi049.stdout:1/465: symlink d0/d4/dd/de/d48/d5c/da2/la9 0 2024-05-15T15:54:08.046 INFO:tasks.workunit.client.1.smithi144.stdout:8/345: truncate d2/d1a/d38/d4b/d5b/d6e/f62 927936 0 2024-05-15T15:54:08.047 INFO:tasks.workunit.client.0.smithi049.stdout:9/261: truncate d0/d8/d11/f16 603148 0 2024-05-15T15:54:08.047 INFO:tasks.workunit.client.0.smithi049.stdout:1/466: dread d0/d4/dd/de/d9e/fa0 [0,4194304] 0 2024-05-15T15:54:08.048 INFO:tasks.workunit.client.0.smithi049.stdout:1/467: read - d0/d4/dd/de/d1e/d2e/d3e/d90/f91 zero size 2024-05-15T15:54:08.049 INFO:tasks.workunit.client.0.smithi049.stdout:0/363: fsync dc/d13/d48/f6c 0 2024-05-15T15:54:08.050 INFO:tasks.workunit.client.0.smithi049.stdout:1/468: chown d0/d4/l14 13 1 2024-05-15T15:54:08.050 INFO:tasks.workunit.client.1.smithi144.stdout:0/334: link d4/d6/c10 d4/d6/d3a/d4c/c7b 0 2024-05-15T15:54:08.050 INFO:tasks.workunit.client.1.smithi144.stdout:2/314: dwrite d3/d7/d20/d3b/f4f [0,4194304] 0 2024-05-15T15:54:08.051 INFO:tasks.workunit.client.1.smithi144.stdout:2/315: chown d3/d10/f2e 162874 1 2024-05-15T15:54:08.054 INFO:tasks.workunit.client.0.smithi049.stdout:6/318: truncate dd/d26/f2e 689955 0 2024-05-15T15:54:08.056 INFO:tasks.workunit.client.0.smithi049.stdout:6/319: fsync dd/d17/f38 0 2024-05-15T15:54:08.058 INFO:tasks.workunit.client.1.smithi144.stdout:3/339: read d0/d11/d3b/f45 [727139,112790] 0 2024-05-15T15:54:08.059 INFO:tasks.workunit.client.0.smithi049.stdout:8/357: mknod d2/d4f/d5d/d63/c6b 0 2024-05-15T15:54:08.061 INFO:tasks.workunit.client.0.smithi049.stdout:0/364: dwrite dc/d13/d14/d2a/f3a [0,4194304] 0 2024-05-15T15:54:08.062 INFO:tasks.workunit.client.0.smithi049.stdout:1/469: dwrite d0/d4/dd/de/d4e/f7e [0,4194304] 0 2024-05-15T15:54:08.064 INFO:tasks.workunit.client.1.smithi144.stdout:9/339: unlink d0/d8/f51 0 2024-05-15T15:54:08.073 INFO:tasks.workunit.client.0.smithi049.stdout:0/365: dwrite dc/d13/d14/d2a/f2c [0,4194304] 0 2024-05-15T15:54:08.074 INFO:tasks.workunit.client.1.smithi144.stdout:4/322: getdents d6/d42 0 2024-05-15T15:54:08.078 INFO:tasks.workunit.client.0.smithi049.stdout:9/262: rename d0/d8/d21/d4d to d0/d8/d9/d22/d26/d56 0 2024-05-15T15:54:08.080 INFO:tasks.workunit.client.1.smithi144.stdout:2/316: rename d3/d7/d20/d22/d52/d64/f2f to d3/d7/d20/d22/d52/f69 0 2024-05-15T15:54:08.080 INFO:tasks.workunit.client.0.smithi049.stdout:1/470: mknod d0/d4/dd/de/d36/caa 0 2024-05-15T15:54:08.082 INFO:tasks.workunit.client.1.smithi144.stdout:1/324: link d7/d1d/d33/d1e/l3e d7/d1d/d33/d1e/d4e/l67 0 2024-05-15T15:54:08.085 INFO:tasks.workunit.client.1.smithi144.stdout:3/340: creat d0/d11/d3b/f76 x:0 0 0 2024-05-15T15:54:08.086 INFO:tasks.workunit.client.0.smithi049.stdout:8/358: rmdir d2/d4f/d5d 39 2024-05-15T15:54:08.086 INFO:tasks.workunit.client.0.smithi049.stdout:8/359: readlink d2/d18/l20 0 2024-05-15T15:54:08.086 INFO:tasks.workunit.client.0.smithi049.stdout:8/360: fdatasync d2/f3d 0 2024-05-15T15:54:08.087 INFO:tasks.workunit.client.1.smithi144.stdout:2/317: dwrite d3/d7/d20/f3e [0,4194304] 0 2024-05-15T15:54:08.088 INFO:tasks.workunit.client.1.smithi144.stdout:5/327: dread d0/dc/d12/f43 [0,4194304] 0 2024-05-15T15:54:08.089 INFO:tasks.workunit.client.0.smithi049.stdout:0/366: dread dc/f34 [0,4194304] 0 2024-05-15T15:54:08.092 INFO:tasks.workunit.client.0.smithi049.stdout:6/320: write f0 [9272064,16738] 0 2024-05-15T15:54:08.093 INFO:tasks.workunit.client.0.smithi049.stdout:6/321: readlink dd/d17/d24/l28 0 2024-05-15T15:54:08.098 INFO:tasks.workunit.client.1.smithi144.stdout:8/346: dwrite d2/d24/d2d/d30/f60 [0,4194304] 0 2024-05-15T15:54:08.098 INFO:tasks.workunit.client.0.smithi049.stdout:3/406: rename d12/d14/f24 to d12/d14/f7c 0 2024-05-15T15:54:08.099 INFO:tasks.workunit.client.1.smithi144.stdout:8/347: write d2/de/d3f/f45 [478474,5934] 0 2024-05-15T15:54:08.099 INFO:tasks.workunit.client.1.smithi144.stdout:6/298: mkdir d7/db/d11/d3c/d61/d6f 0 2024-05-15T15:54:08.099 INFO:tasks.workunit.client.1.smithi144.stdout:8/348: chown d2/d3a/f59 201563 1 2024-05-15T15:54:08.100 INFO:tasks.workunit.client.1.smithi144.stdout:8/349: chown d2/d24/d2d/l6c 9234 1 2024-05-15T15:54:08.101 INFO:tasks.workunit.client.1.smithi144.stdout:8/350: stat d2/d1a/d38/d4b/d5b/d6e 0 2024-05-15T15:54:08.101 INFO:tasks.workunit.client.0.smithi049.stdout:1/471: truncate d0/d4/dd/de/d36/f43 5137611 0 2024-05-15T15:54:08.102 INFO:tasks.workunit.client.1.smithi144.stdout:4/323: mknod d6/d42/d46/d4b/d1e/d44/c65 0 2024-05-15T15:54:08.103 INFO:tasks.workunit.client.0.smithi049.stdout:1/472: chown d0/d4/dd/de/d1e/d16/l60 238 1 2024-05-15T15:54:08.105 INFO:tasks.workunit.client.1.smithi144.stdout:2/318: dread d3/d7/d20/d22/d52/d64/f31 [0,4194304] 0 2024-05-15T15:54:08.106 INFO:tasks.workunit.client.1.smithi144.stdout:2/319: chown d3/d7/d20/d40 55473270 1 2024-05-15T15:54:08.109 INFO:tasks.workunit.client.0.smithi049.stdout:1/473: dread d0/d4/d63/f83 [0,4194304] 0 2024-05-15T15:54:08.110 INFO:tasks.workunit.client.0.smithi049.stdout:1/474: chown d0/d4/dd/de/d1e/f31 2204 1 2024-05-15T15:54:08.110 INFO:tasks.workunit.client.0.smithi049.stdout:0/367: unlink dc/d13/d14/d25/d31/f61 0 2024-05-15T15:54:08.112 INFO:tasks.workunit.client.0.smithi049.stdout:1/475: fsync d0/d4/dd/de/d1e/f1d 0 2024-05-15T15:54:08.112 INFO:tasks.workunit.client.0.smithi049.stdout:0/368: write dc/d13/d14/d2f/f51 [6258707,23056] 0 2024-05-15T15:54:08.113 INFO:tasks.workunit.client.1.smithi144.stdout:1/325: mknod d7/d1d/d58/c68 0 2024-05-15T15:54:08.118 INFO:tasks.workunit.client.0.smithi049.stdout:3/407: dread d12/f41 [0,4194304] 0 2024-05-15T15:54:08.118 INFO:tasks.workunit.client.1.smithi144.stdout:3/341: unlink d0/d11/d24/f22 0 2024-05-15T15:54:08.123 INFO:tasks.workunit.client.0.smithi049.stdout:9/263: creat d0/d8/d9/f57 x:0 0 0 2024-05-15T15:54:08.124 INFO:tasks.workunit.client.1.smithi144.stdout:5/328: mknod d0/dc/d12/d31/c67 0 2024-05-15T15:54:08.127 INFO:tasks.workunit.client.0.smithi049.stdout:8/361: dwrite d2/f22 [0,4194304] 0 2024-05-15T15:54:08.127 INFO:tasks.workunit.client.1.smithi144.stdout:9/340: write d0/d8/d5e/d44/d10/d19/f33 [1905157,98961] 0 2024-05-15T15:54:08.128 INFO:tasks.workunit.client.1.smithi144.stdout:9/341: chown d0/d8/d5e/d44/d77/l22 65275216 1 2024-05-15T15:54:08.130 INFO:tasks.workunit.client.1.smithi144.stdout:9/342: write d0/d8/d5e/d44/d16/f3d [2273073,124505] 0 2024-05-15T15:54:08.133 INFO:tasks.workunit.client.0.smithi049.stdout:6/322: truncate dd/f15 2829481 0 2024-05-15T15:54:08.135 INFO:tasks.workunit.client.1.smithi144.stdout:6/299: mkdir d7/db/d11/d55/d70 0 2024-05-15T15:54:08.136 INFO:tasks.workunit.client.0.smithi049.stdout:1/476: mkdir d0/d4/d81/dab 0 2024-05-15T15:54:08.139 INFO:tasks.workunit.client.1.smithi144.stdout:9/343: dwrite d0/d8/d5e/f32 [4194304,4194304] 0 2024-05-15T15:54:08.140 INFO:tasks.workunit.client.1.smithi144.stdout:0/335: link d4/df/d1f/d24/c27 d4/d6/d3a/d40/c7c 0 2024-05-15T15:54:08.141 INFO:tasks.workunit.client.1.smithi144.stdout:0/336: dread - d4/d6/d4a/f5a zero size 2024-05-15T15:54:08.142 INFO:tasks.workunit.client.0.smithi049.stdout:3/408: creat d12/d26/f7d x:0 0 0 2024-05-15T15:54:08.143 INFO:tasks.workunit.client.0.smithi049.stdout:9/264: truncate d0/d8/d1b/f44 665725 0 2024-05-15T15:54:08.146 INFO:tasks.workunit.client.0.smithi049.stdout:8/362: chown d2/d14/d1b/c21 15522922 1 2024-05-15T15:54:08.146 INFO:tasks.workunit.client.1.smithi144.stdout:9/344: dwrite d0/d8/d5e/d76/f65 [0,4194304] 0 2024-05-15T15:54:08.148 INFO:tasks.workunit.client.0.smithi049.stdout:9/265: dread d0/d8/d1b/f32 [0,4194304] 0 2024-05-15T15:54:08.149 INFO:tasks.workunit.client.0.smithi049.stdout:6/323: creat dd/d17/d24/d2b/d36/d50/f55 x:0 0 0 2024-05-15T15:54:08.151 INFO:tasks.workunit.client.0.smithi049.stdout:1/477: rmdir d0/d4/dd/de/d48/d5c/da2 39 2024-05-15T15:54:08.153 INFO:tasks.workunit.client.1.smithi144.stdout:8/351: write d2/d1a/d38/d4b/d5b/d6e/f62 [206394,10908] 0 2024-05-15T15:54:08.153 INFO:tasks.workunit.client.0.smithi049.stdout:1/478: chown d0/d4/dd/de/d4d/l52 9389 1 2024-05-15T15:54:08.154 INFO:tasks.workunit.client.0.smithi049.stdout:1/479: chown d0/d4/dd/de/d1e/d2e/d3e/f3d 62060 1 2024-05-15T15:54:08.156 INFO:tasks.workunit.client.0.smithi049.stdout:0/369: mkdir dc/d13/d56/d6f 0 2024-05-15T15:54:08.157 INFO:tasks.workunit.client.0.smithi049.stdout:3/409: mknod d12/d30/c7e 0 2024-05-15T15:54:08.161 INFO:tasks.workunit.client.0.smithi049.stdout:8/363: read d2/fc [806629,24047] 0 2024-05-15T15:54:08.162 INFO:tasks.workunit.client.0.smithi049.stdout:9/266: stat d0/d8/d12/cc 0 2024-05-15T15:54:08.164 INFO:tasks.workunit.client.0.smithi049.stdout:6/324: creat dd/d17/d24/d2b/d36/f56 x:0 0 0 2024-05-15T15:54:08.165 INFO:tasks.workunit.client.0.smithi049.stdout:1/480: mknod d0/d4/dd/de/d1e/d16/cac 0 2024-05-15T15:54:08.167 INFO:tasks.workunit.client.0.smithi049.stdout:0/370: mknod dc/d13/d14/d2f/c70 0 2024-05-15T15:54:08.168 INFO:tasks.workunit.client.1.smithi144.stdout:4/324: mkdir d6/dd/d34/d3b/d30/d59/d66 0 2024-05-15T15:54:08.173 INFO:tasks.workunit.client.0.smithi049.stdout:8/364: creat d2/de/f6c x:0 0 0 2024-05-15T15:54:08.174 INFO:tasks.workunit.client.0.smithi049.stdout:0/371: dwrite dc/d13/d14/d2a/f67 [0,4194304] 0 2024-05-15T15:54:08.174 INFO:tasks.workunit.client.1.smithi144.stdout:7/337: getdents d4/df/d25/d38/d5d 0 2024-05-15T15:54:08.174 INFO:tasks.workunit.client.0.smithi049.stdout:8/365: fsync d2/fb 0 2024-05-15T15:54:08.175 INFO:tasks.workunit.client.1.smithi144.stdout:0/337: dwrite d4/d6/d3a/d4c/f7a [4194304,4194304] 0 2024-05-15T15:54:08.176 INFO:tasks.workunit.client.1.smithi144.stdout:0/338: write d4/d6/d4a/d74/f79 [947148,36284] 0 2024-05-15T15:54:08.177 INFO:tasks.workunit.client.1.smithi144.stdout:0/339: fsync d4/df/d1f/d24/f5e 0 2024-05-15T15:54:08.186 INFO:tasks.workunit.client.0.smithi049.stdout:1/481: rename d0/d4/dd/d5d to d0/d4/dd/de/d1e/d2e/dad 0 2024-05-15T15:54:08.188 INFO:tasks.workunit.client.1.smithi144.stdout:9/345: mkdir d0/d8/d1b/d62/d7d 0 2024-05-15T15:54:08.189 INFO:tasks.workunit.client.1.smithi144.stdout:9/346: readlink d0/d8/l2b 0 2024-05-15T15:54:08.194 INFO:tasks.workunit.client.1.smithi144.stdout:2/320: truncate d3/d7/f1f 1246754 0 2024-05-15T15:54:08.196 INFO:tasks.workunit.client.1.smithi144.stdout:8/352: unlink d2/de/d36/f4a 0 2024-05-15T15:54:08.199 INFO:tasks.workunit.client.1.smithi144.stdout:5/329: rename d0/dc/d12/d4a/d5b to d0/dc/d2f/d68 0 2024-05-15T15:54:08.201 INFO:tasks.workunit.client.0.smithi049.stdout:0/372: dread - dc/d13/d3d/f4c zero size 2024-05-15T15:54:08.201 INFO:tasks.workunit.client.0.smithi049.stdout:6/325: dwrite fb [0,4194304] 0 2024-05-15T15:54:08.203 INFO:tasks.workunit.client.1.smithi144.stdout:3/342: truncate d0/d11/d24/f3e 369182 0 2024-05-15T15:54:08.203 INFO:tasks.workunit.client.1.smithi144.stdout:3/343: chown d0/d1/d56/d71 1938347 1 2024-05-15T15:54:08.206 INFO:tasks.workunit.client.0.smithi049.stdout:9/267: creat d0/d8/d1b/d40/f58 x:0 0 0 2024-05-15T15:54:08.211 INFO:tasks.workunit.client.0.smithi049.stdout:1/482: mkdir d0/d4/dd/de/d1e/d2e/dad/d65/d72/dae 0 2024-05-15T15:54:08.213 INFO:tasks.workunit.client.1.smithi144.stdout:0/340: creat d4/d6/d3a/d40/f7d x:0 0 0 2024-05-15T15:54:08.213 INFO:tasks.workunit.client.1.smithi144.stdout:0/341: chown d4/df/f6d 395 1 2024-05-15T15:54:08.215 INFO:tasks.workunit.client.0.smithi049.stdout:3/410: unlink d12/d14/d32/f5e 0 2024-05-15T15:54:08.216 INFO:tasks.workunit.client.0.smithi049.stdout:3/411: chown d12/d30/d63/d68/d70/d75 2603344 1 2024-05-15T15:54:08.216 INFO:tasks.workunit.client.1.smithi144.stdout:1/326: link d7/d1d/d33/d29/c43 d7/d1d/d33/d29/c69 0 2024-05-15T15:54:08.217 INFO:tasks.workunit.client.0.smithi049.stdout:3/412: dread - d12/d30/d42/f48 zero size 2024-05-15T15:54:08.220 INFO:tasks.workunit.client.0.smithi049.stdout:8/366: getdents d2/d14/d45 0 2024-05-15T15:54:08.221 INFO:tasks.workunit.client.1.smithi144.stdout:4/325: dwrite d6/dd/d2d/f3f [4194304,4194304] 0 2024-05-15T15:54:08.224 INFO:tasks.workunit.client.1.smithi144.stdout:7/338: write d4/df/d25/f29 [551273,78769] 0 2024-05-15T15:54:08.227 INFO:tasks.workunit.client.1.smithi144.stdout:8/353: truncate d2/de/f26 782073 0 2024-05-15T15:54:08.229 INFO:tasks.workunit.client.0.smithi049.stdout:1/483: symlink d0/d4/dd/de/d48/laf 0 2024-05-15T15:54:08.229 INFO:tasks.workunit.client.1.smithi144.stdout:7/339: dwrite d4/f27 [0,4194304] 0 2024-05-15T15:54:08.232 INFO:tasks.workunit.client.1.smithi144.stdout:5/330: creat d0/dc/d24/d28/d2d/f69 x:0 0 0 2024-05-15T15:54:08.234 INFO:tasks.workunit.client.1.smithi144.stdout:7/340: read d4/df/f16 [1599149,12299] 0 2024-05-15T15:54:08.235 INFO:tasks.workunit.client.0.smithi049.stdout:6/326: getdents dd/d17/d24/d2b/d36/d50/d54 0 2024-05-15T15:54:08.237 INFO:tasks.workunit.client.0.smithi049.stdout:6/327: dread dd/f4c [0,4194304] 0 2024-05-15T15:54:08.238 INFO:tasks.workunit.client.0.smithi049.stdout:9/268: fsync d0/f1 0 2024-05-15T15:54:08.239 INFO:tasks.workunit.client.1.smithi144.stdout:1/327: creat d7/d1d/d33/d41/d22/f6a x:0 0 0 2024-05-15T15:54:08.239 INFO:tasks.workunit.client.1.smithi144.stdout:7/341: dread d4/df/d25/d38/f54 [0,4194304] 0 2024-05-15T15:54:08.240 INFO:tasks.workunit.client.1.smithi144.stdout:1/328: dread - d7/d1d/d33/d41/d22/f6a zero size 2024-05-15T15:54:08.241 INFO:tasks.workunit.client.0.smithi049.stdout:1/484: symlink d0/d4/dd/de/d1e/d2e/lb0 0 2024-05-15T15:54:08.242 INFO:tasks.workunit.client.1.smithi144.stdout:1/329: stat d7/d1d/d33/l20 0 2024-05-15T15:54:08.242 INFO:tasks.workunit.client.1.smithi144.stdout:1/330: write d7/d1d/d33/d41/d22/f3a [80546,61794] 0 2024-05-15T15:54:08.242 INFO:tasks.workunit.client.1.smithi144.stdout:9/347: symlink d0/d8/d7a/l7e 0 2024-05-15T15:54:08.243 INFO:tasks.workunit.client.1.smithi144.stdout:9/348: fdatasync d0/d8/d5e/d76/f54 0 2024-05-15T15:54:08.244 INFO:tasks.workunit.client.1.smithi144.stdout:9/349: read d0/d8/d21/f2a [536651,44176] 0 2024-05-15T15:54:08.244 INFO:tasks.workunit.client.0.smithi049.stdout:3/413: creat d12/d26/d73/f7f x:0 0 0 2024-05-15T15:54:08.247 INFO:tasks.workunit.client.1.smithi144.stdout:2/321: creat d3/d7/d20/d22/d5e/f6a x:0 0 0 2024-05-15T15:54:08.248 INFO:tasks.workunit.client.1.smithi144.stdout:4/326: mknod d6/d42/d46/d4b/d1e/c67 0 2024-05-15T15:54:08.248 INFO:tasks.workunit.client.1.smithi144.stdout:2/322: fsync d3/d7/d20/d37/d3a/f55 0 2024-05-15T15:54:08.248 INFO:tasks.workunit.client.1.smithi144.stdout:4/327: chown d6/d42 1745909 1 2024-05-15T15:54:08.250 INFO:tasks.workunit.client.0.smithi049.stdout:4/417: sync 2024-05-15T15:54:08.250 INFO:tasks.workunit.client.0.smithi049.stdout:3/414: dwrite d12/f47 [0,4194304] 0 2024-05-15T15:54:08.252 INFO:tasks.workunit.client.0.smithi049.stdout:3/415: read d12/f22 [3280517,49129] 0 2024-05-15T15:54:08.254 INFO:tasks.workunit.client.1.smithi144.stdout:4/328: dwrite d6/d42/d46/d4b/f4d [0,4194304] 0 2024-05-15T15:54:08.254 INFO:tasks.workunit.client.1.smithi144.stdout:8/354: rename d2/de/d36/d58/f5c to d2/d1a/d38/d4b/d5b/d6e/f78 0 2024-05-15T15:54:08.255 INFO:tasks.workunit.client.1.smithi144.stdout:8/355: readlink d2/l4 0 2024-05-15T15:54:08.255 INFO:tasks.workunit.client.1.smithi144.stdout:4/329: dread - d6/dd/d2e/f61 zero size 2024-05-15T15:54:08.257 INFO:tasks.workunit.client.0.smithi049.stdout:0/373: dwrite dc/d13/d48/f4f [0,4194304] 0 2024-05-15T15:54:08.261 INFO:tasks.workunit.client.0.smithi049.stdout:1/485: symlink d0/d4/dd/de/d1e/d16/d8a/lb1 0 2024-05-15T15:54:08.261 INFO:tasks.workunit.client.1.smithi144.stdout:6/300: getdents d7/db/d11/d55 0 2024-05-15T15:54:08.262 INFO:tasks.workunit.client.1.smithi144.stdout:6/301: readlink d7/db/l4e 0 2024-05-15T15:54:08.262 INFO:tasks.workunit.client.1.smithi144.stdout:4/330: dread d6/d42/d46/d4b/f31 [0,4194304] 0 2024-05-15T15:54:08.264 INFO:tasks.workunit.client.1.smithi144.stdout:5/331: symlink d0/dc/d24/d28/l6a 0 2024-05-15T15:54:08.268 INFO:tasks.workunit.client.0.smithi049.stdout:8/367: creat d2/d4f/f6d x:0 0 0 2024-05-15T15:54:08.272 INFO:tasks.workunit.client.1.smithi144.stdout:7/342: mkdir d4/df/d25/d40/d73 0 2024-05-15T15:54:08.276 INFO:tasks.workunit.client.0.smithi049.stdout:6/328: fsync dd/d26/f2e 0 2024-05-15T15:54:08.277 INFO:tasks.workunit.client.1.smithi144.stdout:7/343: read d4/df/f16 [1478792,96269] 0 2024-05-15T15:54:08.281 INFO:tasks.workunit.client.0.smithi049.stdout:0/374: rename dc/d13/d14/d2a to dc/d13/d14/d25/d31/d71 0 2024-05-15T15:54:08.282 INFO:tasks.workunit.client.1.smithi144.stdout:6/302: dread f6 [0,4194304] 0 2024-05-15T15:54:08.283 INFO:tasks.workunit.client.1.smithi144.stdout:7/344: dwrite d4/d41/d5b/f6f [0,4194304] 0 2024-05-15T15:54:08.284 INFO:tasks.workunit.client.0.smithi049.stdout:1/486: symlink d0/lb2 0 2024-05-15T15:54:08.289 INFO:tasks.workunit.client.1.smithi144.stdout:1/331: rename d7/d1d/d52 to d7/d1d/d33/d29/d6b 0 2024-05-15T15:54:08.290 INFO:tasks.workunit.client.1.smithi144.stdout:1/332: chown d7/f4d 7615 1 2024-05-15T15:54:08.290 INFO:tasks.workunit.client.1.smithi144.stdout:1/333: chown d7/d31 180 1 2024-05-15T15:54:08.291 INFO:tasks.workunit.client.1.smithi144.stdout:1/334: write d7/d1d/d33/d41/d22/f3a [471905,8254] 0 2024-05-15T15:54:08.291 INFO:tasks.workunit.client.1.smithi144.stdout:8/356: creat d2/d1a/d38/d4b/d5b/d6e/f79 x:0 0 0 2024-05-15T15:54:08.297 INFO:tasks.workunit.client.0.smithi049.stdout:4/418: write d18/f1a [5130995,43836] 0 2024-05-15T15:54:08.297 INFO:tasks.workunit.client.1.smithi144.stdout:8/357: dwrite d2/de/d36/f1b [0,4194304] 0 2024-05-15T15:54:08.298 INFO:tasks.workunit.client.0.smithi049.stdout:3/416: dread d12/d14/f27 [0,4194304] 0 2024-05-15T15:54:08.298 INFO:tasks.workunit.client.1.smithi144.stdout:4/331: symlink d6/d42/l68 0 2024-05-15T15:54:08.299 INFO:tasks.workunit.client.1.smithi144.stdout:8/358: fdatasync d2/d3a/f3e 0 2024-05-15T15:54:08.299 INFO:tasks.workunit.client.0.smithi049.stdout:3/417: write d12/d14/d32/f6d [617343,112172] 0 2024-05-15T15:54:08.299 INFO:tasks.workunit.client.1.smithi144.stdout:8/359: chown d2/de/d3f 334095 1 2024-05-15T15:54:08.300 INFO:tasks.workunit.client.0.smithi049.stdout:3/418: chown d12/l2b 1 1 2024-05-15T15:54:08.301 INFO:tasks.workunit.client.1.smithi144.stdout:8/360: write d2/de/d36/f4d [95329,61655] 0 2024-05-15T15:54:08.301 INFO:tasks.workunit.client.0.smithi049.stdout:3/419: dread - d12/d14/d32/d64/f78 zero size 2024-05-15T15:54:08.302 INFO:tasks.workunit.client.1.smithi144.stdout:4/332: truncate d6/dd/d34/d3b/d30/db/d1d/d40/d57/f5e 238482 0 2024-05-15T15:54:08.302 INFO:tasks.workunit.client.1.smithi144.stdout:4/333: readlink d6/dd/l16 0 2024-05-15T15:54:08.305 INFO:tasks.workunit.client.1.smithi144.stdout:3/344: mkdir d0/d1/d17/d39/d40/d4a/d77 0 2024-05-15T15:54:08.306 INFO:tasks.workunit.client.0.smithi049.stdout:1/487: dread - d0/d4/dd/de/d1e/f4f zero size 2024-05-15T15:54:08.307 INFO:tasks.workunit.client.0.smithi049.stdout:1/488: fsync d0/d4/d63/f64 0 2024-05-15T15:54:08.308 INFO:tasks.workunit.client.0.smithi049.stdout:9/269: truncate d0/d8/d11/f48 3037294 0 2024-05-15T15:54:08.309 INFO:tasks.workunit.client.0.smithi049.stdout:9/270: read - d0/d8/d1b/d40/f58 zero size 2024-05-15T15:54:08.310 INFO:tasks.workunit.client.0.smithi049.stdout:8/368: creat d2/d4f/d5d/d63/f6e x:0 0 0 2024-05-15T15:54:08.310 INFO:tasks.workunit.client.1.smithi144.stdout:3/345: dwrite d0/f2c [0,4194304] 0 2024-05-15T15:54:08.310 INFO:tasks.workunit.client.0.smithi049.stdout:8/369: readlink d2/d18/l64 0 2024-05-15T15:54:08.319 INFO:tasks.workunit.client.0.smithi049.stdout:1/489: mkdir d0/d4/dd/de/d1e/d2e/d3e/d90/db3 0 2024-05-15T15:54:08.320 INFO:tasks.workunit.client.1.smithi144.stdout:6/303: rmdir d7/db/d11/d3c/d56 39 2024-05-15T15:54:08.320 INFO:tasks.workunit.client.0.smithi049.stdout:1/490: dread - d0/d4/dd/de/d1e/d16/d8a/f9b zero size 2024-05-15T15:54:08.321 INFO:tasks.workunit.client.1.smithi144.stdout:6/304: readlink d7/db/d3a/l63 0 2024-05-15T15:54:08.321 INFO:tasks.workunit.client.0.smithi049.stdout:9/271: creat d0/d8/d12/f59 x:0 0 0 2024-05-15T15:54:08.322 INFO:tasks.workunit.client.0.smithi049.stdout:9/272: chown d0/d8/d9/d22/c3b 392665 1 2024-05-15T15:54:08.323 INFO:tasks.workunit.client.0.smithi049.stdout:8/370: unlink d2/de/d25/f26 0 2024-05-15T15:54:08.323 INFO:tasks.workunit.client.1.smithi144.stdout:5/332: dwrite d0/dc/d24/d28/d2d/f2b [0,4194304] 0 2024-05-15T15:54:08.326 INFO:tasks.workunit.client.1.smithi144.stdout:1/335: mknod d7/d1d/d33/d41/d22/c6c 0 2024-05-15T15:54:08.327 INFO:tasks.workunit.client.1.smithi144.stdout:1/336: chown d7/df 930853 1 2024-05-15T15:54:08.327 INFO:tasks.workunit.client.0.smithi049.stdout:0/375: rename dc/d13/d2b/l63 to dc/d13/d14/l72 0 2024-05-15T15:54:08.329 INFO:tasks.workunit.client.1.smithi144.stdout:8/361: symlink d2/d1a/d38/d4b/l7a 0 2024-05-15T15:54:08.329 INFO:tasks.workunit.client.0.smithi049.stdout:0/376: readlink dc/d13/d14/l42 0 2024-05-15T15:54:08.330 INFO:tasks.workunit.client.1.smithi144.stdout:8/362: readlink d2/de/d36/l1c 0 2024-05-15T15:54:08.330 INFO:tasks.workunit.client.1.smithi144.stdout:5/333: dwrite d0/dc/d1f/f38 [0,4194304] 0 2024-05-15T15:54:08.331 INFO:tasks.workunit.client.0.smithi049.stdout:9/273: dwrite d0/d8/f39 [0,4194304] 0 2024-05-15T15:54:08.331 INFO:tasks.workunit.client.1.smithi144.stdout:8/363: fsync d2/de/f14 0 2024-05-15T15:54:08.332 INFO:tasks.workunit.client.0.smithi049.stdout:1/491: truncate d0/d4/dd/de/d1e/f15 4492622 0 2024-05-15T15:54:08.334 INFO:tasks.workunit.client.1.smithi144.stdout:4/334: truncate d6/dd/d34/d3b/d30/f2f 1097001 0 2024-05-15T15:54:08.336 INFO:tasks.workunit.client.0.smithi049.stdout:8/371: truncate d2/d14/d1b/f2b 3041510 0 2024-05-15T15:54:08.337 INFO:tasks.workunit.client.0.smithi049.stdout:8/372: dread - d2/d4f/d5d/f62 zero size 2024-05-15T15:54:08.338 INFO:tasks.workunit.client.0.smithi049.stdout:8/373: chown d2/f2c 2114985 1 2024-05-15T15:54:08.339 INFO:tasks.workunit.client.1.smithi144.stdout:5/334: dwrite d0/dc/f1b [4194304,4194304] 0 2024-05-15T15:54:08.340 INFO:tasks.workunit.client.0.smithi049.stdout:4/419: creat d18/d1e/d2a/d50/d54/f8a x:0 0 0 2024-05-15T15:54:08.340 INFO:tasks.workunit.client.0.smithi049.stdout:6/329: truncate f4 1269493 0 2024-05-15T15:54:08.342 INFO:tasks.workunit.client.1.smithi144.stdout:3/346: mkdir d0/d11/d24/d78 0 2024-05-15T15:54:08.348 INFO:tasks.workunit.client.1.smithi144.stdout:9/350: mknod d0/d8/d5e/d44/d10/d19/c7f 0 2024-05-15T15:54:08.348 INFO:tasks.workunit.client.1.smithi144.stdout:2/323: dwrite d3/d7/f1a [0,4194304] 0 2024-05-15T15:54:08.349 INFO:tasks.workunit.client.0.smithi049.stdout:9/274: truncate d0/d8/d9/d22/f2e 1694985 0 2024-05-15T15:54:08.351 INFO:tasks.workunit.client.0.smithi049.stdout:2/380: sync 2024-05-15T15:54:08.355 INFO:tasks.workunit.client.0.smithi049.stdout:8/374: mknod d2/d4f/d5d/c6f 0 2024-05-15T15:54:08.356 INFO:tasks.workunit.client.0.smithi049.stdout:8/375: dread - d2/f5a zero size 2024-05-15T15:54:08.356 INFO:tasks.workunit.client.1.smithi144.stdout:7/345: creat d4/df/d25/d38/d68/f74 x:0 0 0 2024-05-15T15:54:08.358 INFO:tasks.workunit.client.1.smithi144.stdout:7/346: truncate d4/df/f34 4893499 0 2024-05-15T15:54:08.358 INFO:tasks.workunit.client.1.smithi144.stdout:7/347: stat d4/df/f32 0 2024-05-15T15:54:08.359 INFO:tasks.workunit.client.1.smithi144.stdout:7/348: readlink d4/df/d10/l33 0 2024-05-15T15:54:08.361 INFO:tasks.workunit.client.0.smithi049.stdout:4/420: unlink d18/d21/l3d 0 2024-05-15T15:54:08.363 INFO:tasks.workunit.client.1.smithi144.stdout:1/337: symlink d7/d1d/d58/l6d 0 2024-05-15T15:54:08.364 INFO:tasks.workunit.client.0.smithi049.stdout:3/420: rename d12/d14/f7c to d12/d30/d42/d6b/f80 0 2024-05-15T15:54:08.368 INFO:tasks.workunit.client.1.smithi144.stdout:1/338: dwrite d7/d1d/d33/f3f [0,4194304] 0 2024-05-15T15:54:08.371 INFO:tasks.workunit.client.1.smithi144.stdout:8/364: rename d2/d1a/d38 to d2/d3a/d7b 0 2024-05-15T15:54:08.372 INFO:tasks.workunit.client.0.smithi049.stdout:2/381: mkdir d3/da/d15/d73 0 2024-05-15T15:54:08.378 INFO:tasks.workunit.client.1.smithi144.stdout:5/335: mkdir d0/dc/d12/d31/d35/d6b 0 2024-05-15T15:54:08.379 INFO:tasks.workunit.client.1.smithi144.stdout:5/336: chown d0/dc/d24/d28/d2d/f69 0 1 2024-05-15T15:54:08.381 INFO:tasks.workunit.client.1.smithi144.stdout:3/347: readlink d0/d11/d3b/l5c 0 2024-05-15T15:54:08.382 INFO:tasks.workunit.client.1.smithi144.stdout:0/342: getdents d4/df/d1f/d56 0 2024-05-15T15:54:08.383 INFO:tasks.workunit.client.0.smithi049.stdout:0/377: dread dc/d13/d14/d2f/f51 [4194304,4194304] 0 2024-05-15T15:54:08.384 INFO:tasks.workunit.client.0.smithi049.stdout:7/314: sync 2024-05-15T15:54:08.384 INFO:tasks.workunit.client.0.smithi049.stdout:5/396: sync 2024-05-15T15:54:08.391 INFO:tasks.workunit.client.0.smithi049.stdout:1/492: link d0/d4/dd/de/d36/c51 d0/d4/dd/de/d1e/d2e/d3e/d90/db3/cb4 0 2024-05-15T15:54:08.392 INFO:tasks.workunit.client.0.smithi049.stdout:2/382: creat d3/db/d5b/f74 x:0 0 0 2024-05-15T15:54:08.394 INFO:tasks.workunit.client.0.smithi049.stdout:2/383: chown d3/d8/f14 16035 1 2024-05-15T15:54:08.394 INFO:tasks.workunit.client.0.smithi049.stdout:2/384: fsync d3/db/f43 0 2024-05-15T15:54:08.397 INFO:tasks.workunit.client.0.smithi049.stdout:8/376: creat d2/d14/d45/f70 x:0 0 0 2024-05-15T15:54:08.399 INFO:tasks.workunit.client.1.smithi144.stdout:1/339: symlink d7/d1d/d33/d41/d22/l6e 0 2024-05-15T15:54:08.399 INFO:tasks.workunit.client.0.smithi049.stdout:4/421: dread d18/d21/f30 [0,4194304] 0 2024-05-15T15:54:08.400 INFO:tasks.workunit.client.0.smithi049.stdout:4/422: dread - d18/f4d zero size 2024-05-15T15:54:08.402 INFO:tasks.workunit.client.1.smithi144.stdout:9/351: rename d0/d8/d1b/d62/c60 to d0/d8/d50/c80 0 2024-05-15T15:54:08.403 INFO:tasks.workunit.client.0.smithi049.stdout:8/377: dwrite d2/d14/d1d/f61 [0,4194304] 0 2024-05-15T15:54:08.411 INFO:tasks.workunit.client.0.smithi049.stdout:5/397: creat dc/d4a/d31/d5c/f7e x:0 0 0 2024-05-15T15:54:08.415 INFO:tasks.workunit.client.1.smithi144.stdout:0/343: mknod d4/d6/d3a/d40/c7e 0 2024-05-15T15:54:08.418 INFO:tasks.workunit.client.1.smithi144.stdout:2/324: mkdir d3/d6b 0 2024-05-15T15:54:08.419 INFO:tasks.workunit.client.1.smithi144.stdout:2/325: write d3/d7/d20/d3f/f53 [411311,67637] 0 2024-05-15T15:54:08.419 INFO:tasks.workunit.client.0.smithi049.stdout:2/385: creat d3/db/d1d/d1e/d2b/f75 x:0 0 0 2024-05-15T15:54:08.421 INFO:tasks.workunit.client.1.smithi144.stdout:6/305: creat d7/f71 x:0 0 0 2024-05-15T15:54:08.426 INFO:tasks.workunit.client.1.smithi144.stdout:1/340: creat d7/d1d/d33/d29/d6b/d5e/f6f x:0 0 0 2024-05-15T15:54:08.426 INFO:tasks.workunit.client.1.smithi144.stdout:4/335: dwrite d6/dd/d34/f39 [0,4194304] 0 2024-05-15T15:54:08.427 INFO:tasks.workunit.client.1.smithi144.stdout:1/341: dread - d7/d4b/f4f zero size 2024-05-15T15:54:08.428 INFO:tasks.workunit.client.1.smithi144.stdout:1/342: write f1 [12947619,7316] 0 2024-05-15T15:54:08.432 INFO:tasks.workunit.client.1.smithi144.stdout:4/336: dread d6/dd/d34/f39 [0,4194304] 0 2024-05-15T15:54:08.432 INFO:tasks.workunit.client.1.smithi144.stdout:4/337: fdatasync d6/dd/f22 0 2024-05-15T15:54:08.434 INFO:tasks.workunit.client.0.smithi049.stdout:1/493: dwrite d0/d4/dd/de/d9e/fa0 [0,4194304] 0 2024-05-15T15:54:08.436 INFO:tasks.workunit.client.1.smithi144.stdout:3/348: mknod d0/d11/d5e/d72/c79 0 2024-05-15T15:54:08.437 INFO:tasks.workunit.client.0.smithi049.stdout:3/421: link c11 d12/d30/d63/c81 0 2024-05-15T15:54:08.439 INFO:tasks.workunit.client.1.smithi144.stdout:8/365: truncate d2/de/d49/f76 2759850 0 2024-05-15T15:54:08.441 INFO:tasks.workunit.client.0.smithi049.stdout:5/398: dwrite dc/d4a/d31/d5c/f7d [0,4194304] 0 2024-05-15T15:54:08.441 INFO:tasks.workunit.client.0.smithi049.stdout:2/386: creat d3/db/d1d/d1e/d2b/d3b/f76 x:0 0 0 2024-05-15T15:54:08.442 INFO:tasks.workunit.client.1.smithi144.stdout:3/349: dwrite d0/d11/d3b/f64 [0,4194304] 0 2024-05-15T15:54:08.443 INFO:tasks.workunit.client.0.smithi049.stdout:6/330: link f4 dd/f57 0 2024-05-15T15:54:08.444 INFO:tasks.workunit.client.1.smithi144.stdout:7/349: link d4/df/f32 d4/df/f75 0 2024-05-15T15:54:08.445 INFO:tasks.workunit.client.0.smithi049.stdout:6/331: truncate dd/d26/f4f 108996 0 2024-05-15T15:54:08.445 INFO:tasks.workunit.client.1.smithi144.stdout:7/350: truncate d4/d41/f4b 974965 0 2024-05-15T15:54:08.446 INFO:tasks.workunit.client.0.smithi049.stdout:6/332: stat dd/d26/l39 0 2024-05-15T15:54:08.446 INFO:tasks.workunit.client.0.smithi049.stdout:4/423: creat d18/d85/f8b x:0 0 0 2024-05-15T15:54:08.447 INFO:tasks.workunit.client.0.smithi049.stdout:4/424: truncate d18/d21/f7a 775982 0 2024-05-15T15:54:08.448 INFO:tasks.workunit.client.0.smithi049.stdout:0/378: creat dc/d13/d14/f73 x:0 0 0 2024-05-15T15:54:08.449 INFO:tasks.workunit.client.0.smithi049.stdout:4/425: write d18/d1e/d2a/d3b/f74 [537813,118537] 0 2024-05-15T15:54:08.450 INFO:tasks.workunit.client.1.smithi144.stdout:7/351: dread d4/d41/f4b [0,4194304] 0 2024-05-15T15:54:08.453 INFO:tasks.workunit.client.1.smithi144.stdout:1/343: creat d7/d1d/d33/d41/d22/f70 x:0 0 0 2024-05-15T15:54:08.538 INFO:tasks.workunit.client.1.smithi144.stdout:4/338: creat d6/d49/f69 x:0 0 0 2024-05-15T15:54:08.543 INFO:tasks.workunit.client.0.smithi049.stdout:5/399: chown l6 2 1 2024-05-15T15:54:08.543 INFO:tasks.workunit.client.0.smithi049.stdout:5/400: stat dc/d4a/c4b 0 2024-05-15T15:54:08.544 INFO:tasks.workunit.client.0.smithi049.stdout:5/401: fsync dc/dd/f17 0 2024-05-15T15:54:08.546 INFO:tasks.workunit.client.0.smithi049.stdout:2/387: mknod d3/d52/c77 0 2024-05-15T15:54:08.549 INFO:tasks.workunit.client.0.smithi049.stdout:0/379: mkdir dc/d13/d14/d33/d74 0 2024-05-15T15:54:08.550 INFO:tasks.workunit.client.1.smithi144.stdout:8/366: creat d2/d3a/f7c x:0 0 0 2024-05-15T15:54:08.551 INFO:tasks.workunit.client.1.smithi144.stdout:8/367: write d2/d24/d2d/d30/f44 [527686,117658] 0 2024-05-15T15:54:08.552 INFO:tasks.workunit.client.1.smithi144.stdout:9/352: write d0/d8/d5e/d44/d16/f4e [4301922,5071] 0 2024-05-15T15:54:08.553 INFO:tasks.workunit.client.1.smithi144.stdout:8/368: read d2/d24/d2d/f54 [220293,69631] 0 2024-05-15T15:54:08.554 INFO:tasks.workunit.client.0.smithi049.stdout:9/275: rename d0/d8/d11/f48 to d0/d8/f5a 0 2024-05-15T15:54:08.556 INFO:tasks.workunit.client.1.smithi144.stdout:3/350: rename d0/d11/d5e to d0/d1/d56/d71/d7a 0 2024-05-15T15:54:08.557 INFO:tasks.workunit.client.1.smithi144.stdout:3/351: dread - d0/d11/d3b/f6b zero size 2024-05-15T15:54:08.561 INFO:tasks.workunit.client.1.smithi144.stdout:0/344: truncate d4/d6/d3a/d4c/f7a 5234856 0 2024-05-15T15:54:08.563 INFO:tasks.workunit.client.1.smithi144.stdout:7/352: mkdir d4/d3c/d76 0 2024-05-15T15:54:08.565 INFO:tasks.workunit.client.0.smithi049.stdout:6/333: mkdir dd/d17/d24/d58 0 2024-05-15T15:54:08.566 INFO:tasks.workunit.client.1.smithi144.stdout:8/369: dread d2/f16 [0,4194304] 0 2024-05-15T15:54:08.566 INFO:tasks.workunit.client.0.smithi049.stdout:6/334: stat dd/d17/d24/l28 0 2024-05-15T15:54:08.566 INFO:tasks.workunit.client.1.smithi144.stdout:6/306: truncate f6 2629777 0 2024-05-15T15:54:08.567 INFO:tasks.workunit.client.1.smithi144.stdout:5/337: getdents d0/dc/d2f/d68 0 2024-05-15T15:54:08.568 INFO:tasks.workunit.client.1.smithi144.stdout:5/338: read d0/dc/d12/f43 [3817896,40996] 0 2024-05-15T15:54:08.569 INFO:tasks.workunit.client.0.smithi049.stdout:0/380: read dc/d13/f17 [3605768,67243] 0 2024-05-15T15:54:08.569 INFO:tasks.workunit.client.1.smithi144.stdout:6/307: dread d7/db/d11/f42 [0,4194304] 0 2024-05-15T15:54:08.569 INFO:tasks.workunit.client.1.smithi144.stdout:6/308: stat d7/f68 0 2024-05-15T15:54:08.572 INFO:tasks.workunit.client.1.smithi144.stdout:5/339: dread d0/dc/fe [0,4194304] 0 2024-05-15T15:54:08.572 INFO:tasks.workunit.client.1.smithi144.stdout:2/326: creat d3/f6c x:0 0 0 2024-05-15T15:54:08.573 INFO:tasks.workunit.client.1.smithi144.stdout:2/327: fdatasync d3/d7/d20/d37/d3a/f45 0 2024-05-15T15:54:08.573 INFO:tasks.workunit.client.1.smithi144.stdout:2/328: fsync d3/d7/d20/f3e 0 2024-05-15T15:54:08.575 INFO:tasks.workunit.client.1.smithi144.stdout:0/345: dread d4/f5 [0,4194304] 0 2024-05-15T15:54:08.575 INFO:tasks.workunit.client.0.smithi049.stdout:3/422: dwrite f7 [0,4194304] 0 2024-05-15T15:54:08.576 INFO:tasks.workunit.client.0.smithi049.stdout:2/388: mkdir d3/d78 0 2024-05-15T15:54:08.576 INFO:tasks.workunit.client.0.smithi049.stdout:3/423: read - d12/d30/d63/d68/f77 zero size 2024-05-15T15:54:08.577 INFO:tasks.workunit.client.0.smithi049.stdout:3/424: chown c11 1 1 2024-05-15T15:54:08.579 INFO:tasks.workunit.client.0.smithi049.stdout:6/335: symlink dd/d17/d24/d2b/d36/l59 0 2024-05-15T15:54:08.580 INFO:tasks.workunit.client.0.smithi049.stdout:2/389: read d3/db/d1d/d1e/d2b/d3b/d42/f6d [60413,104567] 0 2024-05-15T15:54:08.581 INFO:tasks.workunit.client.0.smithi049.stdout:2/390: stat d3/d52/l55 0 2024-05-15T15:54:08.582 INFO:tasks.workunit.client.1.smithi144.stdout:7/353: symlink d4/df/d25/d38/d5d/l77 0 2024-05-15T15:54:08.582 INFO:tasks.workunit.client.0.smithi049.stdout:2/391: dread - d3/db/d1d/d1e/d2b/d3b/f5f zero size 2024-05-15T15:54:08.585 INFO:tasks.workunit.client.1.smithi144.stdout:8/370: creat d2/d24/d2d/d30/f7d x:0 0 0 2024-05-15T15:54:08.592 INFO:tasks.workunit.client.1.smithi144.stdout:4/339: symlink d6/dd/d34/d3b/d30/d59/d66/l6a 0 2024-05-15T15:54:08.593 INFO:tasks.workunit.client.1.smithi144.stdout:4/340: dread - d6/d42/d46/d4b/d1e/d44/f53 zero size 2024-05-15T15:54:08.593 INFO:tasks.workunit.client.1.smithi144.stdout:4/341: chown d6/dd/d2d 7853 1 2024-05-15T15:54:08.594 INFO:tasks.workunit.client.0.smithi049.stdout:5/402: creat dc/f7f x:0 0 0 2024-05-15T15:54:08.594 INFO:tasks.workunit.client.1.smithi144.stdout:4/342: fdatasync d6/d49/f69 0 2024-05-15T15:54:08.594 INFO:tasks.workunit.client.1.smithi144.stdout:4/343: readlink d6/d42/l68 0 2024-05-15T15:54:08.598 INFO:tasks.workunit.client.0.smithi049.stdout:4/426: truncate d18/d1e/f1f 917334 0 2024-05-15T15:54:08.598 INFO:tasks.workunit.client.0.smithi049.stdout:4/427: stat f16 0 2024-05-15T15:54:08.599 INFO:tasks.workunit.client.1.smithi144.stdout:4/344: dwrite d6/dd/d34/d3b/d30/db/f35 [0,4194304] 0 2024-05-15T15:54:08.599 INFO:tasks.workunit.client.0.smithi049.stdout:6/336: dread - dd/f32 zero size 2024-05-15T15:54:08.600 INFO:tasks.workunit.client.1.smithi144.stdout:4/345: write d6/d42/d46/d4b/f31 [62085,47005] 0 2024-05-15T15:54:08.601 INFO:tasks.workunit.client.0.smithi049.stdout:4/428: write d18/d1e/d2a/d50/d54/f5c [1156424,125990] 0 2024-05-15T15:54:08.603 INFO:tasks.workunit.client.1.smithi144.stdout:9/353: chown d0/d8/d5e/d44/d10/d19/c1c 0 1 2024-05-15T15:54:08.605 INFO:tasks.workunit.client.0.smithi049.stdout:2/392: mknod d3/d52/c79 0 2024-05-15T15:54:08.607 INFO:tasks.workunit.client.1.smithi144.stdout:3/352: mknod d0/d1/c7b 0 2024-05-15T15:54:08.609 INFO:tasks.workunit.client.1.smithi144.stdout:0/346: symlink d4/df/d1f/d24/d2b/d55/l7f 0 2024-05-15T15:54:08.611 INFO:tasks.workunit.client.0.smithi049.stdout:5/403: unlink dc/dd/d39/f79 0 2024-05-15T15:54:08.618 INFO:tasks.workunit.client.1.smithi144.stdout:4/346: fsync d6/dd/d34/d3b/d30/db/f35 0 2024-05-15T15:54:08.619 INFO:tasks.workunit.client.1.smithi144.stdout:8/371: symlink d2/de/d36/d58/l7e 0 2024-05-15T15:54:08.620 INFO:tasks.workunit.client.0.smithi049.stdout:0/381: write dc/f40 [416798,48118] 0 2024-05-15T15:54:08.621 INFO:tasks.workunit.client.1.smithi144.stdout:5/340: dwrite d0/dc/d1f/f3b [0,4194304] 0 2024-05-15T15:54:08.622 INFO:tasks.workunit.client.0.smithi049.stdout:6/337: symlink dd/d17/d24/d2b/d36/l5a 0 2024-05-15T15:54:08.622 INFO:tasks.workunit.client.1.smithi144.stdout:5/341: dread - d0/dc/d12/d31/d35/f62 zero size 2024-05-15T15:54:08.624 INFO:tasks.workunit.client.1.smithi144.stdout:6/309: mkdir d7/db/d11/d3c/d56/d72 0 2024-05-15T15:54:08.624 INFO:tasks.workunit.client.1.smithi144.stdout:5/342: chown d0/dc/d24/d28/d2d/f69 133 1 2024-05-15T15:54:08.625 INFO:tasks.workunit.client.0.smithi049.stdout:4/429: mknod d18/d85/c8c 0 2024-05-15T15:54:08.625 INFO:tasks.workunit.client.0.smithi049.stdout:4/430: fsync fc 0 2024-05-15T15:54:08.626 INFO:tasks.workunit.client.1.smithi144.stdout:3/353: dread d0/d11/d24/f49 [0,4194304] 0 2024-05-15T15:54:08.628 INFO:tasks.workunit.client.1.smithi144.stdout:3/354: chown d0/d1/d17/d39/d40/l42 32199 1 2024-05-15T15:54:08.628 INFO:tasks.workunit.client.0.smithi049.stdout:3/425: truncate d12/f1d 6871797 0 2024-05-15T15:54:08.629 INFO:tasks.workunit.client.0.smithi049.stdout:3/426: chown d12/d14/d32/f62 1197684 1 2024-05-15T15:54:08.630 INFO:tasks.workunit.client.0.smithi049.stdout:2/393: dread d3/d8/f14 [0,4194304] 0 2024-05-15T15:54:08.631 INFO:tasks.workunit.client.1.smithi144.stdout:6/310: dwrite d7/db/d11/d3c/f4d [0,4194304] 0 2024-05-15T15:54:08.631 INFO:tasks.workunit.client.0.smithi049.stdout:7/315: rename d0/d6/l56 to d0/l5d 0 2024-05-15T15:54:08.633 INFO:tasks.workunit.client.1.smithi144.stdout:9/354: creat d0/d8/d5e/d44/d10/f81 x:0 0 0 2024-05-15T15:54:08.634 INFO:tasks.workunit.client.0.smithi049.stdout:5/404: unlink dc/dd/f72 0 2024-05-15T15:54:08.636 INFO:tasks.workunit.client.0.smithi049.stdout:0/382: truncate dc/d13/d3d/f5d 354936 0 2024-05-15T15:54:08.638 INFO:tasks.workunit.client.0.smithi049.stdout:6/338: mkdir dd/d26/d5b 0 2024-05-15T15:54:08.640 INFO:tasks.workunit.client.1.smithi144.stdout:7/354: mknod d4/df/c78 0 2024-05-15T15:54:08.641 INFO:tasks.workunit.client.1.smithi144.stdout:4/347: fsync d6/dd/d2d/f48 0 2024-05-15T15:54:08.642 INFO:tasks.workunit.client.1.smithi144.stdout:8/372: creat d2/d3a/f7f x:0 0 0 2024-05-15T15:54:08.646 INFO:tasks.workunit.client.0.smithi049.stdout:2/394: fsync d3/f11 0 2024-05-15T15:54:08.648 INFO:tasks.workunit.client.1.smithi144.stdout:5/343: mknod d0/dc/d2f/d68/d5c/c6c 0 2024-05-15T15:54:08.648 INFO:tasks.workunit.client.0.smithi049.stdout:2/395: truncate d3/d8/f50 883297 0 2024-05-15T15:54:08.654 INFO:tasks.workunit.client.0.smithi049.stdout:2/396: read d3/da/d15/f26 [357665,98337] 0 2024-05-15T15:54:08.655 INFO:tasks.workunit.client.1.smithi144.stdout:6/311: unlink d7/d1e/d28/f57 0 2024-05-15T15:54:08.656 INFO:tasks.workunit.client.0.smithi049.stdout:4/431: dread d18/f3e [0,4194304] 0 2024-05-15T15:54:08.658 INFO:tasks.workunit.client.0.smithi049.stdout:3/427: dread d12/d14/f1b [0,4194304] 0 2024-05-15T15:54:08.658 INFO:tasks.workunit.client.1.smithi144.stdout:9/355: creat d0/d8/d21/f82 x:0 0 0 2024-05-15T15:54:08.659 INFO:tasks.workunit.client.1.smithi144.stdout:1/344: sync 2024-05-15T15:54:08.661 INFO:tasks.workunit.client.1.smithi144.stdout:1/345: truncate d7/d1d/d33/d1e/d4e/f50 1002998 0 2024-05-15T15:54:08.664 INFO:tasks.workunit.client.0.smithi049.stdout:0/383: creat dc/d13/d14/d25/d31/f75 x:0 0 0 2024-05-15T15:54:08.667 INFO:tasks.workunit.client.0.smithi049.stdout:6/339: unlink dd/d17/d24/d2b/f3d 0 2024-05-15T15:54:08.674 INFO:tasks.workunit.client.1.smithi144.stdout:3/355: write d0/d1/d17/f19 [128090,30637] 0 2024-05-15T15:54:08.675 INFO:tasks.workunit.client.0.smithi049.stdout:2/397: fdatasync d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d 0 2024-05-15T15:54:08.679 INFO:tasks.workunit.client.1.smithi144.stdout:1/346: unlink d7/d1d/d33/d29/d6b/d5e/f6f 0 2024-05-15T15:54:08.680 INFO:tasks.workunit.client.0.smithi049.stdout:7/316: dwrite d0/d2f/f51 [0,4194304] 0 2024-05-15T15:54:08.682 INFO:tasks.workunit.client.0.smithi049.stdout:6/340: dread dd/d17/d24/f27 [0,4194304] 0 2024-05-15T15:54:08.683 INFO:tasks.workunit.client.1.smithi144.stdout:7/355: mkdir d4/df/d79 0 2024-05-15T15:54:08.684 INFO:tasks.workunit.client.0.smithi049.stdout:6/341: chown dd/d17/d24/d2b/d36/d50/c51 102975612 1 2024-05-15T15:54:08.684 INFO:tasks.workunit.client.1.smithi144.stdout:7/356: truncate d4/df/f1f 4612996 0 2024-05-15T15:54:08.685 INFO:tasks.workunit.client.0.smithi049.stdout:0/384: mkdir dc/d13/d2b/d76 0 2024-05-15T15:54:08.685 INFO:tasks.workunit.client.0.smithi049.stdout:0/385: readlink dc/d13/d14/l27 0 2024-05-15T15:54:08.688 INFO:tasks.workunit.client.1.smithi144.stdout:4/348: write d6/dd/d34/d3b/d30/f1b [1555004,114918] 0 2024-05-15T15:54:08.690 INFO:tasks.workunit.client.0.smithi049.stdout:9/276: rename d0/d8/d9/d22/d26/l4f to d0/l5b 0 2024-05-15T15:54:08.690 INFO:tasks.workunit.client.1.smithi144.stdout:5/344: mknod d0/dc/d1f/d65/c6d 0 2024-05-15T15:54:08.692 INFO:tasks.workunit.client.1.smithi144.stdout:6/312: symlink d7/d1e/l73 0 2024-05-15T15:54:08.693 INFO:tasks.workunit.client.1.smithi144.stdout:6/313: write d7/ff [1141407,5416] 0 2024-05-15T15:54:08.693 INFO:tasks.workunit.client.0.smithi049.stdout:2/398: chown d3/db/f1a 813240767 1 2024-05-15T15:54:08.693 INFO:tasks.workunit.client.1.smithi144.stdout:6/314: fdatasync d7/db/f2c 0 2024-05-15T15:54:08.694 INFO:tasks.workunit.client.1.smithi144.stdout:3/356: mknod d0/d1/d56/d71/d7a/c7c 0 2024-05-15T15:54:08.695 INFO:tasks.workunit.client.0.smithi049.stdout:3/428: symlink d12/l82 0 2024-05-15T15:54:08.695 INFO:tasks.workunit.client.1.smithi144.stdout:3/357: dread - d0/d11/d24/f57 zero size 2024-05-15T15:54:08.695 INFO:tasks.workunit.client.1.smithi144.stdout:9/356: symlink d0/d8/d5e/d44/d16/l83 0 2024-05-15T15:54:08.698 INFO:tasks.workunit.client.1.smithi144.stdout:1/347: creat d7/d1d/d33/f71 x:0 0 0 2024-05-15T15:54:08.698 INFO:tasks.workunit.client.1.smithi144.stdout:9/357: dread d0/f59 [0,4194304] 0 2024-05-15T15:54:08.699 INFO:tasks.workunit.client.0.smithi049.stdout:4/432: mknod d18/d1e/d2a/d36/d86/c8d 0 2024-05-15T15:54:08.700 INFO:tasks.workunit.client.1.smithi144.stdout:0/347: link d4/d6/c1b d4/d2d/d48/c80 0 2024-05-15T15:54:08.700 INFO:tasks.workunit.client.0.smithi049.stdout:4/433: stat d18/d1e/d2a/d36/d5d/f67 0 2024-05-15T15:54:08.701 INFO:tasks.workunit.client.1.smithi144.stdout:0/348: readlink d4/d6/d4a/d74/l78 0 2024-05-15T15:54:08.704 INFO:tasks.workunit.client.0.smithi049.stdout:5/405: link dc/d4a/f57 dc/f80 0 2024-05-15T15:54:08.705 INFO:tasks.workunit.client.1.smithi144.stdout:8/373: link d2/de/d49/l6a d2/d3a/d7b/d4b/d5b/d6e/d71/l80 0 2024-05-15T15:54:08.705 INFO:tasks.workunit.client.1.smithi144.stdout:8/374: chown d2/d24/c22 7940 1 2024-05-15T15:54:08.707 INFO:tasks.workunit.client.1.smithi144.stdout:4/349: mknod d6/c6b 0 2024-05-15T15:54:08.707 INFO:tasks.workunit.client.1.smithi144.stdout:4/350: read - d6/d42/d46/d4b/d1e/f51 zero size 2024-05-15T15:54:08.708 INFO:tasks.workunit.client.1.smithi144.stdout:2/329: sync 2024-05-15T15:54:08.709 INFO:tasks.workunit.client.0.smithi049.stdout:2/399: dread d3/db/f43 [0,4194304] 0 2024-05-15T15:54:08.711 INFO:tasks.workunit.client.1.smithi144.stdout:4/351: dread d6/d42/d46/d4b/f4d [0,4194304] 0 2024-05-15T15:54:08.711 INFO:tasks.workunit.client.0.smithi049.stdout:9/277: mkdir d0/d8/d1b/d40/d5c 0 2024-05-15T15:54:08.714 INFO:tasks.workunit.client.0.smithi049.stdout:7/317: mkdir d0/d6/da/d3d/d54/d5e 0 2024-05-15T15:54:08.715 INFO:tasks.workunit.client.0.smithi049.stdout:7/318: chown d0/d6/d1d/l2b 843 1 2024-05-15T15:54:08.716 INFO:tasks.workunit.client.0.smithi049.stdout:5/406: rename ca to dc/dd/d7b/c81 0 2024-05-15T15:54:08.717 INFO:tasks.workunit.client.1.smithi144.stdout:1/348: mkdir d7/d1d/d33/d1e/d4e/d72 0 2024-05-15T15:54:08.718 INFO:tasks.workunit.client.0.smithi049.stdout:6/342: creat dd/d17/d24/d2b/d36/d50/d54/f5c x:0 0 0 2024-05-15T15:54:08.718 INFO:tasks.workunit.client.1.smithi144.stdout:1/349: write d7/f4d [1361714,14975] 0 2024-05-15T15:54:08.719 INFO:tasks.workunit.client.1.smithi144.stdout:9/358: creat d0/d8/d1b/d62/f84 x:0 0 0 2024-05-15T15:54:08.722 INFO:tasks.workunit.client.1.smithi144.stdout:1/350: dread d7/d1d/d33/d29/f42 [0,4194304] 0 2024-05-15T15:54:08.723 INFO:tasks.workunit.client.1.smithi144.stdout:0/349: fdatasync d4/df/d1f/f2f 0 2024-05-15T15:54:08.726 INFO:tasks.workunit.client.0.smithi049.stdout:8/378: sync 2024-05-15T15:54:08.726 INFO:tasks.workunit.client.0.smithi049.stdout:1/494: sync 2024-05-15T15:54:08.726 INFO:tasks.workunit.client.0.smithi049.stdout:2/400: unlink d3/d52/l55 0 2024-05-15T15:54:08.726 INFO:tasks.workunit.client.0.smithi049.stdout:6/343: read - dd/d17/f4a zero size 2024-05-15T15:54:08.726 INFO:tasks.workunit.client.1.smithi144.stdout:7/357: rename d4/df/d10/c1b to d4/df/d25/d40/c7a 0 2024-05-15T15:54:08.727 INFO:tasks.workunit.client.0.smithi049.stdout:9/278: fsync d0/d8/d9/fa 0 2024-05-15T15:54:08.727 INFO:tasks.workunit.client.1.smithi144.stdout:1/351: dwrite d7/d1d/d33/d41/d22/f70 [0,4194304] 0 2024-05-15T15:54:08.728 INFO:tasks.workunit.client.0.smithi049.stdout:3/429: creat d12/d30/d42/d7a/f83 x:0 0 0 2024-05-15T15:54:08.728 INFO:tasks.workunit.client.1.smithi144.stdout:5/345: write d0/dc/fe [2018649,107934] 0 2024-05-15T15:54:08.730 INFO:tasks.workunit.client.1.smithi144.stdout:8/375: unlink d2/de/d3f/f4c 0 2024-05-15T15:54:08.735 INFO:tasks.workunit.client.0.smithi049.stdout:5/407: creat dc/d4a/d31/f82 x:0 0 0 2024-05-15T15:54:08.737 INFO:tasks.workunit.client.1.smithi144.stdout:2/330: dread d3/d7/d20/d22/d52/d64/f31 [0,4194304] 0 2024-05-15T15:54:08.738 INFO:tasks.workunit.client.1.smithi144.stdout:4/352: mkdir d6/dd/d34/d3b/d30/db/d1d/d40/d6c 0 2024-05-15T15:54:08.738 INFO:tasks.workunit.client.0.smithi049.stdout:1/495: fdatasync d0/d4/d63/f83 0 2024-05-15T15:54:08.739 INFO:tasks.workunit.client.1.smithi144.stdout:4/353: dread d6/dd/d34/d3b/d30/db/d1d/d40/d57/f5e [0,4194304] 0 2024-05-15T15:54:08.740 INFO:tasks.workunit.client.0.smithi049.stdout:8/379: dread - d2/d14/d1b/d68/f4b zero size 2024-05-15T15:54:08.741 INFO:tasks.workunit.client.1.smithi144.stdout:3/358: mknod d0/d1/c7d 0 2024-05-15T15:54:08.741 INFO:tasks.workunit.client.1.smithi144.stdout:3/359: stat d0/l1b 0 2024-05-15T15:54:08.742 INFO:tasks.workunit.client.1.smithi144.stdout:3/360: chown d0/d1/d17/d39/d40/f50 270090 1 2024-05-15T15:54:08.743 INFO:tasks.workunit.client.1.smithi144.stdout:3/361: chown d0/d1/d17/d39/d40/f50 629199 1 2024-05-15T15:54:08.745 INFO:tasks.workunit.client.0.smithi049.stdout:9/279: mkdir d0/d25/d5d 0 2024-05-15T15:54:08.746 INFO:tasks.workunit.client.1.smithi144.stdout:9/359: rmdir d0/d8/d5e/d44/d10/d19 39 2024-05-15T15:54:08.746 INFO:tasks.workunit.client.1.smithi144.stdout:9/360: fdatasync d0/d8/d5e/d76/f65 0 2024-05-15T15:54:08.747 INFO:tasks.workunit.client.1.smithi144.stdout:9/361: write d0/f7 [3759156,10314] 0 2024-05-15T15:54:08.748 INFO:tasks.workunit.client.0.smithi049.stdout:7/319: dwrite d0/d6/d1d/f27 [0,4194304] 0 2024-05-15T15:54:08.755 INFO:tasks.workunit.client.0.smithi049.stdout:8/380: dread - d2/de/d27/f40 zero size 2024-05-15T15:54:08.755 INFO:tasks.workunit.client.1.smithi144.stdout:3/362: dread d0/d1/d17/f48 [0,4194304] 0 2024-05-15T15:54:08.759 INFO:tasks.workunit.client.0.smithi049.stdout:6/344: getdents dd/d26/d5b 0 2024-05-15T15:54:08.759 INFO:tasks.workunit.client.1.smithi144.stdout:8/376: read d2/de/d36/f17 [71366,11385] 0 2024-05-15T15:54:08.760 INFO:tasks.workunit.client.0.smithi049.stdout:6/345: chown dd/d17/f21 8375108 1 2024-05-15T15:54:08.760 INFO:tasks.workunit.client.1.smithi144.stdout:8/377: truncate d2/d3a/f3e 2476240 0 2024-05-15T15:54:08.763 INFO:tasks.workunit.client.1.smithi144.stdout:2/331: unlink d3/d7/d20/d22/f62 0 2024-05-15T15:54:08.763 INFO:tasks.workunit.client.1.smithi144.stdout:1/352: write d7/d1d/d33/d41/d22/f48 [1714243,81501] 0 2024-05-15T15:54:08.764 INFO:tasks.workunit.client.0.smithi049.stdout:9/280: fdatasync d0/d8/d1b/f28 0 2024-05-15T15:54:08.765 INFO:tasks.workunit.client.1.smithi144.stdout:2/332: dread - d3/d7/d20/d37/d3a/d65/f68 zero size 2024-05-15T15:54:08.766 INFO:tasks.workunit.client.1.smithi144.stdout:6/315: creat d7/db/f74 x:0 0 0 2024-05-15T15:54:08.767 INFO:tasks.workunit.client.0.smithi049.stdout:6/346: dwrite dd/ff [4194304,4194304] 0 2024-05-15T15:54:08.768 INFO:tasks.workunit.client.0.smithi049.stdout:6/347: readlink dd/l29 0 2024-05-15T15:54:08.769 INFO:tasks.workunit.client.0.smithi049.stdout:6/348: chown dd/d17/d24/d2b/f46 1426 1 2024-05-15T15:54:08.771 INFO:tasks.workunit.client.0.smithi049.stdout:2/401: mkdir d3/db/d1d/d1e/d2b/d2e/d67/d71/d7a 0 2024-05-15T15:54:08.772 INFO:tasks.workunit.client.1.smithi144.stdout:4/354: mknod d6/d42/c6d 0 2024-05-15T15:54:08.772 INFO:tasks.workunit.client.0.smithi049.stdout:2/402: dread - d3/db/d1d/d1e/d2b/d3b/d42/f4f zero size 2024-05-15T15:54:08.773 INFO:tasks.workunit.client.1.smithi144.stdout:4/355: fdatasync d6/d42/d46/d4b/f31 0 2024-05-15T15:54:08.773 INFO:tasks.workunit.client.0.smithi049.stdout:7/320: creat d0/d6/da/d3d/f5f x:0 0 0 2024-05-15T15:54:08.777 INFO:tasks.workunit.client.0.smithi049.stdout:2/403: dread d3/f11 [0,4194304] 0 2024-05-15T15:54:08.779 INFO:tasks.workunit.client.0.smithi049.stdout:8/381: mkdir d2/de/d71 0 2024-05-15T15:54:08.780 INFO:tasks.workunit.client.0.smithi049.stdout:3/430: dwrite d12/d14/f1a [0,4194304] 0 2024-05-15T15:54:08.789 INFO:tasks.workunit.client.0.smithi049.stdout:7/321: symlink d0/d6/d1d/l60 0 2024-05-15T15:54:08.793 INFO:tasks.workunit.client.1.smithi144.stdout:3/363: mkdir d0/d1/d17/d39/d40/d7e 0 2024-05-15T15:54:08.794 INFO:tasks.workunit.client.0.smithi049.stdout:7/322: chown d0/d6/da/d3d/d54/d5e 21 1 2024-05-15T15:54:08.794 INFO:tasks.workunit.client.0.smithi049.stdout:5/408: dwrite dc/f1b [4194304,4194304] 0 2024-05-15T15:54:08.795 INFO:tasks.workunit.client.1.smithi144.stdout:0/350: truncate d4/df/f43 1007881 0 2024-05-15T15:54:08.795 INFO:tasks.workunit.client.1.smithi144.stdout:0/351: chown d4/dc/f18 115458 1 2024-05-15T15:54:08.796 INFO:tasks.workunit.client.0.smithi049.stdout:2/404: rmdir d3/db/d1d/d1e/d2b/d2e 39 2024-05-15T15:54:08.799 INFO:tasks.workunit.client.0.smithi049.stdout:3/431: mknod d12/d67/c84 0 2024-05-15T15:54:08.801 INFO:tasks.workunit.client.0.smithi049.stdout:7/323: dread d0/d6/f29 [0,4194304] 0 2024-05-15T15:54:08.801 INFO:tasks.workunit.client.1.smithi144.stdout:8/378: creat d2/d24/d2d/d30/f81 x:0 0 0 2024-05-15T15:54:08.803 INFO:tasks.workunit.client.0.smithi049.stdout:9/281: mkdir d0/d8/d9/d52/d5e 0 2024-05-15T15:54:08.803 INFO:tasks.workunit.client.1.smithi144.stdout:8/379: chown d2/de/d36/f2f 5380429 1 2024-05-15T15:54:08.803 INFO:tasks.workunit.client.1.smithi144.stdout:0/352: dwrite d4/df/d1f/f3e [0,4194304] 0 2024-05-15T15:54:08.805 INFO:tasks.workunit.client.0.smithi049.stdout:1/496: getdents d0/d4/dd/de/d1e/d2e/dad/d65 0 2024-05-15T15:54:08.805 INFO:tasks.workunit.client.0.smithi049.stdout:2/405: dwrite d3/d8/f36 [0,4194304] 0 2024-05-15T15:54:08.806 INFO:tasks.workunit.client.0.smithi049.stdout:2/406: chown d3 541 1 2024-05-15T15:54:08.807 INFO:tasks.workunit.client.1.smithi144.stdout:1/353: creat d7/d31/f73 x:0 0 0 2024-05-15T15:54:08.808 INFO:tasks.workunit.client.1.smithi144.stdout:2/333: rename d3/d7/d20 to d3/d7/d20/d22/d6d 22 2024-05-15T15:54:08.809 INFO:tasks.workunit.client.1.smithi144.stdout:2/334: readlink d3/d10/l32 0 2024-05-15T15:54:08.814 INFO:tasks.workunit.client.1.smithi144.stdout:8/380: dread d2/de/d3f/f5f [0,4194304] 0 2024-05-15T15:54:08.817 INFO:tasks.workunit.client.1.smithi144.stdout:4/356: write d6/d42/d46/d4b/f4d [922745,128209] 0 2024-05-15T15:54:08.822 INFO:tasks.workunit.client.1.smithi144.stdout:8/381: dwrite d2/d3a/d7b/d4b/d5b/d6e/f62 [0,4194304] 0 2024-05-15T15:54:08.826 INFO:tasks.workunit.client.1.smithi144.stdout:4/357: dwrite d6/dd/d34/d3b/d30/f1b [0,4194304] 0 2024-05-15T15:54:08.827 INFO:tasks.workunit.client.1.smithi144.stdout:5/346: sync 2024-05-15T15:54:08.827 INFO:tasks.workunit.client.0.smithi049.stdout:3/432: dread d12/d14/f1e [0,4194304] 0 2024-05-15T15:54:08.827 INFO:tasks.workunit.client.0.smithi049.stdout:8/382: truncate d2/f2c 1356893 0 2024-05-15T15:54:08.827 INFO:tasks.workunit.client.0.smithi049.stdout:9/282: chown d0/d8/d11/f16 341506 1 2024-05-15T15:54:08.829 INFO:tasks.workunit.client.1.smithi144.stdout:3/364: creat d0/d1/d56/f7f x:0 0 0 2024-05-15T15:54:08.830 INFO:tasks.workunit.client.0.smithi049.stdout:5/409: dread dc/d29/f63 [0,4194304] 0 2024-05-15T15:54:08.832 INFO:tasks.workunit.client.1.smithi144.stdout:4/358: dread d6/dd/d34/d3b/d30/db/f35 [0,4194304] 0 2024-05-15T15:54:08.833 INFO:tasks.workunit.client.1.smithi144.stdout:3/365: dread d0/d11/d24/f41 [0,4194304] 0 2024-05-15T15:54:08.834 INFO:tasks.workunit.client.1.smithi144.stdout:3/366: write d0/f2c [2574710,102862] 0 2024-05-15T15:54:08.834 INFO:tasks.workunit.client.1.smithi144.stdout:3/367: chown d0/d1/d17/l66 2888 1 2024-05-15T15:54:08.835 INFO:tasks.workunit.client.1.smithi144.stdout:3/368: write d0/d11/d24/f74 [234961,5041] 0 2024-05-15T15:54:08.839 INFO:tasks.workunit.client.0.smithi049.stdout:9/283: dwrite d0/d8/d21/f2f [4194304,4194304] 0 2024-05-15T15:54:08.840 INFO:tasks.workunit.client.1.smithi144.stdout:6/316: truncate f6 2154634 0 2024-05-15T15:54:08.842 INFO:tasks.workunit.client.0.smithi049.stdout:9/284: stat d0/d8/d12/c50 0 2024-05-15T15:54:08.845 INFO:tasks.workunit.client.1.smithi144.stdout:9/362: creat d0/d8/d5e/d44/d16/f85 x:0 0 0 2024-05-15T15:54:08.846 INFO:tasks.workunit.client.1.smithi144.stdout:9/363: chown d0/f4 134240 1 2024-05-15T15:54:08.848 INFO:tasks.workunit.client.1.smithi144.stdout:9/364: read d0/d8/d5e/d44/f74 [3315552,21874] 0 2024-05-15T15:54:08.851 INFO:tasks.workunit.client.1.smithi144.stdout:8/382: unlink d2/d3a/d7b/d4b/d5b/d6e/f79 0 2024-05-15T15:54:08.852 INFO:tasks.workunit.client.1.smithi144.stdout:8/383: write d2/d3a/f7f [905469,10071] 0 2024-05-15T15:54:08.852 INFO:tasks.workunit.client.1.smithi144.stdout:9/365: dread d0/d8/d21/f38 [0,4194304] 0 2024-05-15T15:54:08.853 INFO:tasks.workunit.client.1.smithi144.stdout:7/358: getdents d4/d45 0 2024-05-15T15:54:08.853 INFO:tasks.workunit.client.1.smithi144.stdout:8/384: stat d2/de/d3f 0 2024-05-15T15:54:08.853 INFO:tasks.workunit.client.1.smithi144.stdout:0/353: write d4/dc/f17 [2141913,60389] 0 2024-05-15T15:54:08.855 INFO:tasks.workunit.client.1.smithi144.stdout:7/359: write d4/df/d25/f3a [2648983,45926] 0 2024-05-15T15:54:08.855 INFO:tasks.workunit.client.1.smithi144.stdout:8/385: write d2/de/f14 [3820782,76058] 0 2024-05-15T15:54:08.856 INFO:tasks.workunit.client.1.smithi144.stdout:7/360: truncate d4/df/f75 5056047 0 2024-05-15T15:54:08.856 INFO:tasks.workunit.client.1.smithi144.stdout:7/361: rename d4/d45 to d4/d45/d7b 22 2024-05-15T15:54:08.857 INFO:tasks.workunit.client.1.smithi144.stdout:4/359: symlink d6/d42/d46/d4b/l6e 0 2024-05-15T15:54:08.858 INFO:tasks.workunit.client.1.smithi144.stdout:4/360: write d6/dd/d34/d3b/d30/db/f35 [1490931,65485] 0 2024-05-15T15:54:08.862 INFO:tasks.workunit.client.1.smithi144.stdout:4/361: dread d6/d42/d46/d4b/f31 [0,4194304] 0 2024-05-15T15:54:08.866 INFO:tasks.workunit.client.1.smithi144.stdout:3/369: mknod d0/d1/d17/d39/d4d/c80 0 2024-05-15T15:54:08.869 INFO:tasks.workunit.client.1.smithi144.stdout:2/335: mkdir d3/d6b/d6e 0 2024-05-15T15:54:08.871 INFO:tasks.workunit.client.1.smithi144.stdout:2/336: truncate d3/d7/d54/f5a 603206 0 2024-05-15T15:54:08.879 INFO:tasks.workunit.client.0.smithi049.stdout:1/497: creat d0/d4/d81/dab/fb5 x:0 0 0 2024-05-15T15:54:08.880 INFO:tasks.workunit.client.0.smithi049.stdout:2/407: mknod d3/db/d5b/d63/c7b 0 2024-05-15T15:54:08.882 INFO:tasks.workunit.client.0.smithi049.stdout:2/408: dread d3/db/d5b/f70 [0,4194304] 0 2024-05-15T15:54:08.882 INFO:tasks.workunit.client.1.smithi144.stdout:2/337: dread d3/d7/d20/d37/d3a/f55 [4194304,4194304] 0 2024-05-15T15:54:08.883 INFO:tasks.workunit.client.1.smithi144.stdout:2/338: fdatasync d3/d7/d20/d3f/f53 0 2024-05-15T15:54:08.883 INFO:tasks.workunit.client.1.smithi144.stdout:0/354: creat d4/df/d14/f81 x:0 0 0 2024-05-15T15:54:08.884 INFO:tasks.workunit.client.1.smithi144.stdout:0/355: readlink d4/l1e 0 2024-05-15T15:54:08.885 INFO:tasks.workunit.client.1.smithi144.stdout:7/362: creat d4/df/d25/d38/f7c x:0 0 0 2024-05-15T15:54:08.886 INFO:tasks.workunit.client.1.smithi144.stdout:7/363: dread - d4/df/d25/d40/d62/f6b zero size 2024-05-15T15:54:08.886 INFO:tasks.workunit.client.1.smithi144.stdout:4/362: symlink d6/d42/d46/l6f 0 2024-05-15T15:54:08.889 INFO:tasks.workunit.client.0.smithi049.stdout:7/324: creat d0/d6/da/f61 x:0 0 0 2024-05-15T15:54:08.889 INFO:tasks.workunit.client.1.smithi144.stdout:3/370: chown d0/d1/c1a 86 1 2024-05-15T15:54:08.890 INFO:tasks.workunit.client.0.smithi049.stdout:7/325: read d0/f47 [3991672,38510] 0 2024-05-15T15:54:08.897 INFO:tasks.workunit.client.1.smithi144.stdout:2/339: creat d3/d7/d20/d22/d52/f6f x:0 0 0 2024-05-15T15:54:08.898 INFO:tasks.workunit.client.1.smithi144.stdout:2/340: read d3/d7/d20/d3f/f53 [229456,98197] 0 2024-05-15T15:54:08.899 INFO:tasks.workunit.client.1.smithi144.stdout:2/341: write d3/d7/f3d [162452,30640] 0 2024-05-15T15:54:08.899 INFO:tasks.workunit.client.1.smithi144.stdout:7/364: truncate d4/df/d25/d38/f54 652058 0 2024-05-15T15:54:08.901 INFO:tasks.workunit.client.0.smithi049.stdout:8/383: rename d2/de/d27/f39 to d2/d14/f72 0 2024-05-15T15:54:08.903 INFO:tasks.workunit.client.1.smithi144.stdout:1/354: getdents d7/d1d/d33 0 2024-05-15T15:54:08.907 INFO:tasks.workunit.client.0.smithi049.stdout:9/285: rename d0/d8/d9/d22/d26 to d0/d8/d9/d52/d5e/d5f 0 2024-05-15T15:54:08.908 INFO:tasks.workunit.client.1.smithi144.stdout:1/355: dwrite d7/d1d/d33/d1e/d4e/f50 [0,4194304] 0 2024-05-15T15:54:08.908 INFO:tasks.workunit.client.1.smithi144.stdout:3/371: rmdir d0/d11/d24 39 2024-05-15T15:54:08.909 INFO:tasks.workunit.client.1.smithi144.stdout:3/372: dread - d0/d1/d56/f7f zero size 2024-05-15T15:54:08.910 INFO:tasks.workunit.client.0.smithi049.stdout:8/384: creat d2/de/d25/f73 x:0 0 0 2024-05-15T15:54:08.912 INFO:tasks.workunit.client.0.smithi049.stdout:8/385: chown d2/d14/f43 684177399 1 2024-05-15T15:54:08.912 INFO:tasks.workunit.client.1.smithi144.stdout:6/317: creat d7/d1e/f75 x:0 0 0 2024-05-15T15:54:08.912 INFO:tasks.workunit.client.1.smithi144.stdout:6/318: chown d7/db/d11/d3c/d56 507159905 1 2024-05-15T15:54:08.914 INFO:tasks.workunit.client.1.smithi144.stdout:2/342: dread d3/d7/d20/d22/d52/d64/f5f [0,4194304] 0 2024-05-15T15:54:08.914 INFO:tasks.workunit.client.1.smithi144.stdout:6/319: write d7/db/d11/d55/f5e [400397,10582] 0 2024-05-15T15:54:08.915 INFO:tasks.workunit.client.0.smithi049.stdout:3/433: dwrite d12/d14/f1b [0,4194304] 0 2024-05-15T15:54:08.917 INFO:tasks.workunit.client.1.smithi144.stdout:0/356: creat d4/d6/d3a/d5d/f82 x:0 0 0 2024-05-15T15:54:08.917 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:08 smithi049 ceph-mon[28399]: pgmap v24: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 187 MiB/s wr, 379 op/s 2024-05-15T15:54:08.917 INFO:tasks.workunit.client.1.smithi144.stdout:0/357: chown d4/df/d1f/d56/c6b 67936128 1 2024-05-15T15:54:08.922 INFO:tasks.workunit.client.0.smithi049.stdout:1/498: rename d0/d4/dd/de/d1e/d16/l60 to d0/d4/dd/de/d1e/d2e/dad/d65/lb6 0 2024-05-15T15:54:08.922 INFO:tasks.workunit.client.0.smithi049.stdout:2/409: write d3/f11 [131202,65227] 0 2024-05-15T15:54:08.922 INFO:tasks.workunit.client.0.smithi049.stdout:1/499: readlink d0/d4/dd/de/d1e/d2e/d3e/l69 0 2024-05-15T15:54:08.925 INFO:tasks.workunit.client.0.smithi049.stdout:7/326: link d0/d6/d23/d4a/l57 d0/d6/d23/l62 0 2024-05-15T15:54:08.936 INFO:tasks.workunit.client.1.smithi144.stdout:1/356: mkdir d7/d1d/d33/d29/d6b/d5e/d74 0 2024-05-15T15:54:08.936 INFO:tasks.workunit.client.1.smithi144.stdout:1/357: chown d7/d1d/d33/d29 2059463 1 2024-05-15T15:54:08.940 INFO:tasks.workunit.client.0.smithi049.stdout:7/327: readlink d0/lf 0 2024-05-15T15:54:08.942 INFO:tasks.workunit.client.1.smithi144.stdout:9/366: creat d0/d8/d5e/d44/d3c/d75/f86 x:0 0 0 2024-05-15T15:54:08.945 INFO:tasks.workunit.client.0.smithi049.stdout:1/500: dread d0/d4/dd/de/d1e/f37 [0,4194304] 0 2024-05-15T15:54:08.948 INFO:tasks.workunit.client.1.smithi144.stdout:6/320: fdatasync d7/f8 0 2024-05-15T15:54:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:08 smithi144 ceph-mon[36389]: pgmap v24: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 187 MiB/s wr, 379 op/s 2024-05-15T15:54:08.952 INFO:tasks.workunit.client.1.smithi144.stdout:5/347: sync 2024-05-15T15:54:08.953 INFO:tasks.workunit.client.1.smithi144.stdout:2/343: creat d3/d7/d20/d37/f70 x:0 0 0 2024-05-15T15:54:08.954 INFO:tasks.workunit.client.0.smithi049.stdout:8/386: dwrite d2/de/f16 [0,4194304] 0 2024-05-15T15:54:08.954 INFO:tasks.workunit.client.0.smithi049.stdout:2/410: rename d3/db/d1d/c22 to d3/da/d15/d24/c7c 0 2024-05-15T15:54:08.957 INFO:tasks.workunit.client.0.smithi049.stdout:1/501: dwrite d0/d4/dd/f38 [0,4194304] 0 2024-05-15T15:54:08.957 INFO:tasks.workunit.client.1.smithi144.stdout:7/365: rename d4/df/d25/d40/c7a to d4/df/d10/c7d 0 2024-05-15T15:54:08.958 INFO:tasks.workunit.client.1.smithi144.stdout:7/366: chown d4/df/d25/d40/d62/f6b 35137 1 2024-05-15T15:54:08.961 INFO:tasks.workunit.client.0.smithi049.stdout:7/328: rmdir d0/d6/da 39 2024-05-15T15:54:08.967 INFO:tasks.workunit.client.1.smithi144.stdout:3/373: creat d0/d1/d17/d39/d40/d7e/f81 x:0 0 0 2024-05-15T15:54:08.968 INFO:tasks.workunit.client.0.smithi049.stdout:9/286: getdents d0/d8/d1b 0 2024-05-15T15:54:08.972 INFO:tasks.workunit.client.1.smithi144.stdout:6/321: symlink d7/d62/l76 0 2024-05-15T15:54:08.974 INFO:tasks.workunit.client.1.smithi144.stdout:0/358: symlink d4/d6/d3a/d6a/l83 0 2024-05-15T15:54:08.977 INFO:tasks.workunit.client.0.smithi049.stdout:7/329: symlink d0/d6/d1d/l63 0 2024-05-15T15:54:08.980 INFO:tasks.workunit.client.0.smithi049.stdout:3/434: getdents d12/d26/d73 0 2024-05-15T15:54:08.980 INFO:tasks.workunit.client.0.smithi049.stdout:1/502: dwrite d0/d4/dd/de/d1e/d2e/f88 [0,4194304] 0 2024-05-15T15:54:08.980 INFO:tasks.workunit.client.1.smithi144.stdout:0/359: dwrite d4/df/d1f/f2c [0,4194304] 0 2024-05-15T15:54:08.980 INFO:tasks.workunit.client.1.smithi144.stdout:7/367: creat d4/df/d25/d38/d68/f7e x:0 0 0 2024-05-15T15:54:08.981 INFO:tasks.workunit.client.1.smithi144.stdout:7/368: write d4/d41/d5b/f6f [1143007,90724] 0 2024-05-15T15:54:08.981 INFO:tasks.workunit.client.0.smithi049.stdout:7/330: truncate d0/d6/d23/d45/f5a 283057 0 2024-05-15T15:54:08.982 INFO:tasks.workunit.client.1.smithi144.stdout:4/363: getdents d6/dd/d34 0 2024-05-15T15:54:08.983 INFO:tasks.workunit.client.0.smithi049.stdout:2/411: mknod d3/db/d1d/d1e/d2b/d2e/d67/d71/c7d 0 2024-05-15T15:54:08.983 INFO:tasks.workunit.client.0.smithi049.stdout:1/503: readlink d0/d4/dd/de/d1e/d16/d8a/lb1 0 2024-05-15T15:54:08.983 INFO:tasks.workunit.client.0.smithi049.stdout:2/412: stat d3/da/d15 0 2024-05-15T15:54:08.986 INFO:tasks.workunit.client.0.smithi049.stdout:7/331: write d0/d6/d23/d45/f5a [501326,127801] 0 2024-05-15T15:54:08.987 INFO:tasks.workunit.client.1.smithi144.stdout:3/374: dread - d0/d11/d24/f4f zero size 2024-05-15T15:54:08.989 INFO:tasks.workunit.client.0.smithi049.stdout:1/504: dread - d0/d4/dd/de/d1e/d2e/dad/fa1 zero size 2024-05-15T15:54:08.990 INFO:tasks.workunit.client.0.smithi049.stdout:9/287: creat d0/d8/d1b/d40/d5c/f60 x:0 0 0 2024-05-15T15:54:08.991 INFO:tasks.workunit.client.0.smithi049.stdout:9/288: write d0/f41 [113409,66132] 0 2024-05-15T15:54:08.993 INFO:tasks.workunit.client.0.smithi049.stdout:3/435: chown d12/d14/f4e 1 1 2024-05-15T15:54:08.996 INFO:tasks.workunit.client.0.smithi049.stdout:2/413: creat d3/db/d1d/d1e/d2b/d2e/f7e x:0 0 0 2024-05-15T15:54:08.998 INFO:tasks.workunit.client.1.smithi144.stdout:2/344: symlink d3/d7/d20/l71 0 2024-05-15T15:54:08.998 INFO:tasks.workunit.client.1.smithi144.stdout:3/375: dread d0/d1/d17/f33 [0,4194304] 0 2024-05-15T15:54:08.999 INFO:tasks.workunit.client.1.smithi144.stdout:2/345: dread - d3/d7/d20/d3b/f57 zero size 2024-05-15T15:54:09.001 INFO:tasks.workunit.client.1.smithi144.stdout:5/348: mkdir d0/dc/d2f/d34/d63/d6e 0 2024-05-15T15:54:09.002 INFO:tasks.workunit.client.0.smithi049.stdout:3/436: dwrite d12/d14/f1b [0,4194304] 0 2024-05-15T15:54:09.002 INFO:tasks.workunit.client.0.smithi049.stdout:2/414: write d3/db/d1d/d1e/d2b/d3b/d42/f4f [751540,336] 0 2024-05-15T15:54:09.003 INFO:tasks.workunit.client.1.smithi144.stdout:7/369: creat d4/df/d25/d38/f7f x:0 0 0 2024-05-15T15:54:09.010 INFO:tasks.workunit.client.1.smithi144.stdout:4/364: creat d6/dd/d2e/f70 x:0 0 0 2024-05-15T15:54:09.010 INFO:tasks.workunit.client.0.smithi049.stdout:2/415: dwrite d3/db/f1f [0,4194304] 0 2024-05-15T15:54:09.012 INFO:tasks.workunit.client.1.smithi144.stdout:1/358: write d7/d1d/d33/d1e/f25 [160640,130990] 0 2024-05-15T15:54:09.013 INFO:tasks.workunit.client.1.smithi144.stdout:1/359: readlink l2 0 2024-05-15T15:54:09.013 INFO:tasks.workunit.client.1.smithi144.stdout:1/360: write d7/d31/f73 [312866,59030] 0 2024-05-15T15:54:09.017 INFO:tasks.workunit.client.0.smithi049.stdout:3/437: truncate d12/d14/d52/f7b 321803 0 2024-05-15T15:54:09.019 INFO:tasks.workunit.client.0.smithi049.stdout:1/505: mkdir d0/d4/dd/de/d1e/db7 0 2024-05-15T15:54:09.021 INFO:tasks.workunit.client.0.smithi049.stdout:8/387: dwrite d2/d14/d1b/f2b [0,4194304] 0 2024-05-15T15:54:09.027 INFO:tasks.workunit.client.1.smithi144.stdout:2/346: symlink d3/d7/d20/d3b/l72 0 2024-05-15T15:54:09.029 INFO:tasks.workunit.client.0.smithi049.stdout:1/506: fdatasync d0/d4/dd/de/d48/d5c/f94 0 2024-05-15T15:54:09.031 INFO:tasks.workunit.client.0.smithi049.stdout:8/388: rmdir d2/de/d27/d5b/d60 39 2024-05-15T15:54:09.031 INFO:tasks.workunit.client.0.smithi049.stdout:7/332: dwrite d0/d6/d1d/f4e [0,4194304] 0 2024-05-15T15:54:09.032 INFO:tasks.workunit.client.1.smithi144.stdout:4/365: mkdir d6/dd/d2e/d71 0 2024-05-15T15:54:09.034 INFO:tasks.workunit.client.0.smithi049.stdout:2/416: creat d3/da/d15/d73/f7f x:0 0 0 2024-05-15T15:54:09.034 INFO:tasks.workunit.client.1.smithi144.stdout:8/386: sync 2024-05-15T15:54:09.034 INFO:tasks.workunit.client.1.smithi144.stdout:1/361: mkdir d7/d1d/d33/d1e/d34/d75 0 2024-05-15T15:54:09.034 INFO:tasks.workunit.client.1.smithi144.stdout:9/367: dwrite d0/d8/d5e/d44/d10/d19/f49 [0,4194304] 0 2024-05-15T15:54:09.035 INFO:tasks.workunit.client.1.smithi144.stdout:8/387: chown d2/de/d36/f4d 0 1 2024-05-15T15:54:09.036 INFO:tasks.workunit.client.0.smithi049.stdout:1/507: mkdir d0/d4/dd/de/d4d/db8 0 2024-05-15T15:54:09.038 INFO:tasks.workunit.client.1.smithi144.stdout:6/322: creat d7/db/d11/d3c/f77 x:0 0 0 2024-05-15T15:54:09.038 INFO:tasks.workunit.client.1.smithi144.stdout:8/388: write d2/f35 [524548,3653] 0 2024-05-15T15:54:09.038 INFO:tasks.workunit.client.0.smithi049.stdout:3/438: dwrite d12/d14/f1e [0,4194304] 0 2024-05-15T15:54:09.040 INFO:tasks.workunit.client.1.smithi144.stdout:6/323: chown d7/d1e/l3b 5664777 1 2024-05-15T15:54:09.042 INFO:tasks.workunit.client.1.smithi144.stdout:5/349: mknod d0/dc/d2f/d34/d63/d6e/c6f 0 2024-05-15T15:54:09.044 INFO:tasks.workunit.client.1.smithi144.stdout:1/362: dwrite d7/d1d/d33/f3f [0,4194304] 0 2024-05-15T15:54:09.045 INFO:tasks.workunit.client.1.smithi144.stdout:1/363: write d7/d4b/f4f [219408,52390] 0 2024-05-15T15:54:09.048 INFO:tasks.workunit.client.0.smithi049.stdout:7/333: creat d0/d2f/d5c/f64 x:0 0 0 2024-05-15T15:54:09.049 INFO:tasks.workunit.client.0.smithi049.stdout:2/417: dwrite d3/db/d5b/f70 [0,4194304] 0 2024-05-15T15:54:09.050 INFO:tasks.workunit.client.0.smithi049.stdout:8/389: dread d2/de/f16 [0,4194304] 0 2024-05-15T15:54:09.052 INFO:tasks.workunit.client.0.smithi049.stdout:8/390: dread - d2/f5a zero size 2024-05-15T15:54:09.054 INFO:tasks.workunit.client.0.smithi049.stdout:2/418: readlink d3/db/d1d/d1e/d2b/d3b/d47/l4e 0 2024-05-15T15:54:09.058 INFO:tasks.workunit.client.0.smithi049.stdout:7/334: fdatasync d0/f9 0 2024-05-15T15:54:09.062 INFO:tasks.workunit.client.0.smithi049.stdout:1/508: symlink d0/d4/dd/de/d1e/db7/lb9 0 2024-05-15T15:54:09.065 INFO:tasks.workunit.client.0.smithi049.stdout:3/439: creat d12/d30/d63/d68/d70/d75/f85 x:0 0 0 2024-05-15T15:54:09.067 INFO:tasks.workunit.client.0.smithi049.stdout:8/391: mkdir d2/de/d25/d74 0 2024-05-15T15:54:09.074 INFO:tasks.workunit.client.0.smithi049.stdout:2/419: creat d3/db/d1d/d1e/d2b/d6b/f80 x:0 0 0 2024-05-15T15:54:09.077 INFO:tasks.workunit.client.1.smithi144.stdout:4/366: symlink d6/dd/d2d/l72 0 2024-05-15T15:54:09.078 INFO:tasks.workunit.client.1.smithi144.stdout:8/389: creat d2/d3a/f82 x:0 0 0 2024-05-15T15:54:09.078 INFO:tasks.workunit.client.1.smithi144.stdout:3/376: link d0/d11/c16 d0/d1/d56/d71/c82 0 2024-05-15T15:54:09.080 INFO:tasks.workunit.client.0.smithi049.stdout:2/420: rename d3/db/d1d/d1e/d2b/d3b/d47/c60 to d3/da/d15/d73/c81 0 2024-05-15T15:54:09.080 INFO:tasks.workunit.client.1.smithi144.stdout:5/350: symlink d0/dc/d1f/d50/l70 0 2024-05-15T15:54:09.081 INFO:tasks.workunit.client.1.smithi144.stdout:4/367: chown d6/d49/c56 708062 1 2024-05-15T15:54:09.081 INFO:tasks.workunit.client.1.smithi144.stdout:1/364: unlink d7/df/f15 0 2024-05-15T15:54:09.081 INFO:tasks.workunit.client.1.smithi144.stdout:6/324: dread d7/db/d35/f67 [0,4194304] 0 2024-05-15T15:54:09.083 INFO:tasks.workunit.client.1.smithi144.stdout:3/377: dread d0/f2c [0,4194304] 0 2024-05-15T15:54:09.084 INFO:tasks.workunit.client.1.smithi144.stdout:3/378: chown d0/d11/d24/f74 115327 1 2024-05-15T15:54:09.084 INFO:tasks.workunit.client.1.smithi144.stdout:8/390: mkdir d2/de/d49/d83 0 2024-05-15T15:54:09.086 INFO:tasks.workunit.client.0.smithi049.stdout:2/421: mkdir d3/db/d82 0 2024-05-15T15:54:09.089 INFO:tasks.workunit.client.1.smithi144.stdout:9/368: creat d0/d8/d5e/d44/f87 x:0 0 0 2024-05-15T15:54:09.090 INFO:tasks.workunit.client.0.smithi049.stdout:3/440: truncate f9 2454514 0 2024-05-15T15:54:09.090 INFO:tasks.workunit.client.1.smithi144.stdout:9/369: read - d0/d8/d1b/d62/d6b/f7c zero size 2024-05-15T15:54:09.091 INFO:tasks.workunit.client.1.smithi144.stdout:9/370: read - d0/d8/d50/f69 zero size 2024-05-15T15:54:09.091 INFO:tasks.workunit.client.0.smithi049.stdout:3/441: dread - d12/d30/f6f zero size 2024-05-15T15:54:09.092 INFO:tasks.workunit.client.1.smithi144.stdout:4/368: unlink d6/dd/d34/f39 0 2024-05-15T15:54:09.092 INFO:tasks.workunit.client.0.smithi049.stdout:8/392: dwrite d2/de/d25/f32 [0,4194304] 0 2024-05-15T15:54:09.093 INFO:tasks.workunit.client.1.smithi144.stdout:4/369: chown d6/d42/d46/l6f 1267901 1 2024-05-15T15:54:09.093 INFO:tasks.workunit.client.1.smithi144.stdout:1/365: creat d7/d1d/d33/d1e/d34/f76 x:0 0 0 2024-05-15T15:54:09.094 INFO:tasks.workunit.client.0.smithi049.stdout:3/442: chown d12/d30/d63/d68/f77 24 1 2024-05-15T15:54:09.094 INFO:tasks.workunit.client.1.smithi144.stdout:8/391: rmdir d2 39 2024-05-15T15:54:09.095 INFO:tasks.workunit.client.0.smithi049.stdout:3/443: chown d12/f4d 830508 1 2024-05-15T15:54:09.099 INFO:tasks.workunit.client.0.smithi049.stdout:3/444: dread - d12/d30/d63/d68/d70/d75/f85 zero size 2024-05-15T15:54:09.099 INFO:tasks.workunit.client.1.smithi144.stdout:9/371: truncate d0/d8/d5e/d44/f74 946225 0 2024-05-15T15:54:09.099 INFO:tasks.workunit.client.1.smithi144.stdout:9/372: chown d0/d8/d5e/d76/f3f 54796807 1 2024-05-15T15:54:09.100 INFO:tasks.workunit.client.0.smithi049.stdout:8/393: mknod d2/c75 0 2024-05-15T15:54:09.100 INFO:tasks.workunit.client.1.smithi144.stdout:3/379: dread d0/d11/f26 [0,4194304] 0 2024-05-15T15:54:09.101 INFO:tasks.workunit.client.1.smithi144.stdout:4/370: rename d6/cc to d6/d42/d46/c73 0 2024-05-15T15:54:09.102 INFO:tasks.workunit.client.1.smithi144.stdout:4/371: stat d6/dd/d34/d3b/d30/db/d1d 0 2024-05-15T15:54:09.104 INFO:tasks.workunit.client.1.smithi144.stdout:1/366: unlink d7/d1d/d33/d29/c43 0 2024-05-15T15:54:09.108 INFO:tasks.workunit.client.1.smithi144.stdout:4/372: dwrite d6/dd/d34/d3b/d30/f1b [0,4194304] 0 2024-05-15T15:54:09.109 INFO:tasks.workunit.client.1.smithi144.stdout:5/351: creat d0/dc/f71 x:0 0 0 2024-05-15T15:54:09.109 INFO:tasks.workunit.client.1.smithi144.stdout:4/373: fsync d6/dd/d2d/f3f 0 2024-05-15T15:54:09.113 INFO:tasks.workunit.client.1.smithi144.stdout:6/325: creat d7/f78 x:0 0 0 2024-05-15T15:54:09.123 INFO:tasks.workunit.client.0.smithi049.stdout:3/445: getdents d12/d14 0 2024-05-15T15:54:09.126 INFO:tasks.workunit.client.1.smithi144.stdout:1/367: rename d7/d1d/d33/d41/d22/f48 to d7/d1d/d33/d1e/f77 0 2024-05-15T15:54:09.128 INFO:tasks.workunit.client.0.smithi049.stdout:3/446: fsync d12/d14/f20 0 2024-05-15T15:54:09.130 INFO:tasks.workunit.client.1.smithi144.stdout:8/392: truncate d2/d24/f2b 743706 0 2024-05-15T15:54:09.131 INFO:tasks.workunit.client.0.smithi049.stdout:3/447: write d12/d14/d32/f59 [69258,4813] 0 2024-05-15T15:54:09.132 INFO:tasks.workunit.client.0.smithi049.stdout:8/394: dwrite f0 [0,4194304] 0 2024-05-15T15:54:09.134 INFO:tasks.workunit.client.1.smithi144.stdout:5/352: mknod d0/dc/d2f/d34/d3e/c72 0 2024-05-15T15:54:09.139 INFO:tasks.workunit.client.1.smithi144.stdout:4/374: creat d6/dd/d34/d3b/d30/d59/f74 x:0 0 0 2024-05-15T15:54:09.140 INFO:tasks.workunit.client.1.smithi144.stdout:4/375: chown d6/d49/f5c 15661 1 2024-05-15T15:54:09.140 INFO:tasks.workunit.client.1.smithi144.stdout:4/376: dread - d6/dd/d34/d3b/f4c zero size 2024-05-15T15:54:09.141 INFO:tasks.workunit.client.1.smithi144.stdout:4/377: stat d6/dd/d34/d3b/d30/f1b 0 2024-05-15T15:54:09.142 INFO:tasks.workunit.client.1.smithi144.stdout:4/378: readlink d6/dd/d34/d3b/l2c 0 2024-05-15T15:54:09.144 INFO:tasks.workunit.client.0.smithi049.stdout:3/448: rename d12/d30/d42/c49 to d12/d30/d63/c86 0 2024-05-15T15:54:09.145 INFO:tasks.workunit.client.1.smithi144.stdout:9/373: symlink d0/l88 0 2024-05-15T15:54:09.145 INFO:tasks.workunit.client.1.smithi144.stdout:7/370: sync 2024-05-15T15:54:09.145 INFO:tasks.workunit.client.1.smithi144.stdout:2/347: sync 2024-05-15T15:54:09.145 INFO:tasks.workunit.client.1.smithi144.stdout:0/360: sync 2024-05-15T15:54:09.145 INFO:tasks.workunit.client.0.smithi049.stdout:8/395: creat d2/de/d25/d74/f76 x:0 0 0 2024-05-15T15:54:09.146 INFO:tasks.workunit.client.1.smithi144.stdout:0/361: write d4/df/d1f/f3e [1537338,16338] 0 2024-05-15T15:54:09.146 INFO:tasks.workunit.client.1.smithi144.stdout:3/380: mkdir d0/d1/d17/d39/d83 0 2024-05-15T15:54:09.148 INFO:tasks.workunit.client.1.smithi144.stdout:3/381: truncate d0/d1/f68 450623 0 2024-05-15T15:54:09.149 INFO:tasks.workunit.client.0.smithi049.stdout:3/449: rename c11 to d12/d14/d32/c87 0 2024-05-15T15:54:09.152 INFO:tasks.workunit.client.0.smithi049.stdout:3/450: mkdir d12/d14/d32/d64/d88 0 2024-05-15T15:54:09.153 INFO:tasks.workunit.client.1.smithi144.stdout:3/382: dwrite d0/d11/f14 [0,4194304] 0 2024-05-15T15:54:09.153 INFO:tasks.workunit.client.0.smithi049.stdout:3/451: fsync d12/d14/d52/f5f 0 2024-05-15T15:54:09.162 INFO:tasks.workunit.client.0.smithi049.stdout:3/452: rmdir d12/d30/d42/d7a 39 2024-05-15T15:54:09.166 INFO:tasks.workunit.client.0.smithi049.stdout:3/453: unlink c0 0 2024-05-15T15:54:09.167 INFO:tasks.workunit.client.0.smithi049.stdout:3/454: write d12/d30/d42/f48 [278101,101958] 0 2024-05-15T15:54:09.168 INFO:tasks.workunit.client.1.smithi144.stdout:5/353: read d0/dc/d12/d31/d35/f44 [355480,85788] 0 2024-05-15T15:54:09.169 INFO:tasks.workunit.client.0.smithi049.stdout:8/396: truncate d2/f22 2378274 0 2024-05-15T15:54:09.170 INFO:tasks.workunit.client.0.smithi049.stdout:3/455: truncate d12/d30/d63/d68/f77 127781 0 2024-05-15T15:54:09.171 INFO:tasks.workunit.client.1.smithi144.stdout:4/379: mkdir d6/dd/d2e/d75 0 2024-05-15T15:54:09.171 INFO:tasks.workunit.client.1.smithi144.stdout:4/380: stat d6/d42/l68 0 2024-05-15T15:54:09.171 INFO:tasks.workunit.client.0.smithi049.stdout:3/456: write d12/d14/d52/f7b [339936,106724] 0 2024-05-15T15:54:09.171 INFO:tasks.workunit.client.1.smithi144.stdout:2/348: rmdir d3/d6b 39 2024-05-15T15:54:09.172 INFO:tasks.workunit.client.1.smithi144.stdout:4/381: chown d6/dd/d34/d3b/d30/d59/f74 10 1 2024-05-15T15:54:09.180 INFO:tasks.workunit.client.0.smithi049.stdout:8/397: write d2/de/d27/f40 [615548,98291] 0 2024-05-15T15:54:09.182 INFO:tasks.workunit.client.0.smithi049.stdout:8/398: write d2/d14/f43 [2089986,106336] 0 2024-05-15T15:54:09.186 INFO:tasks.workunit.client.1.smithi144.stdout:3/383: chown d0/d1/d56/d71/c82 5020 1 2024-05-15T15:54:09.188 INFO:tasks.workunit.client.1.smithi144.stdout:7/371: dwrite d4/df/f75 [4194304,4194304] 0 2024-05-15T15:54:09.189 INFO:tasks.workunit.client.0.smithi049.stdout:8/399: symlink d2/de/d27/d5b/d60/l77 0 2024-05-15T15:54:09.191 INFO:tasks.workunit.client.1.smithi144.stdout:5/354: rmdir d0/dc/d12/d31/d35 39 2024-05-15T15:54:09.191 INFO:tasks.workunit.client.1.smithi144.stdout:0/362: dwrite d4/df/d14/f5f [4194304,4194304] 0 2024-05-15T15:54:09.192 INFO:tasks.workunit.client.1.smithi144.stdout:2/349: creat d3/d7/d20/d37/d3a/d65/f73 x:0 0 0 2024-05-15T15:54:09.194 INFO:tasks.workunit.client.1.smithi144.stdout:5/355: write d0/dc/d24/f37 [2245255,93238] 0 2024-05-15T15:54:09.195 INFO:tasks.workunit.client.0.smithi049.stdout:8/400: symlink d2/d14/d1b/d69/l78 0 2024-05-15T15:54:09.199 INFO:tasks.workunit.client.1.smithi144.stdout:5/356: fdatasync d0/dc/d1f/f38 0 2024-05-15T15:54:09.200 INFO:tasks.workunit.client.1.smithi144.stdout:7/372: dwrite d4/f67 [0,4194304] 0 2024-05-15T15:54:09.201 INFO:tasks.workunit.client.0.smithi049.stdout:8/401: dread d2/de/d25/f32 [0,4194304] 0 2024-05-15T15:54:09.202 INFO:tasks.workunit.client.1.smithi144.stdout:1/368: rename d7/d1d/d33/d1e/d34/d75 to d7/d1d/d33/d41/d78 0 2024-05-15T15:54:09.206 INFO:tasks.workunit.client.0.smithi049.stdout:8/402: unlink d2/d18/f3c 0 2024-05-15T15:54:09.206 INFO:tasks.workunit.client.1.smithi144.stdout:6/326: getdents d7/db 0 2024-05-15T15:54:09.206 INFO:tasks.workunit.client.1.smithi144.stdout:1/369: chown d7/c5a 197107646 1 2024-05-15T15:54:09.207 INFO:tasks.workunit.client.1.smithi144.stdout:8/393: dwrite d2/de/f26 [0,4194304] 0 2024-05-15T15:54:09.209 INFO:tasks.workunit.client.1.smithi144.stdout:5/357: truncate d0/dc/d24/d28/f51 576135 0 2024-05-15T15:54:09.209 INFO:tasks.workunit.client.1.smithi144.stdout:6/327: dread - d7/f78 zero size 2024-05-15T15:54:09.212 INFO:tasks.workunit.client.1.smithi144.stdout:6/328: write d7/db/f3f [2784612,124488] 0 2024-05-15T15:54:09.214 INFO:tasks.workunit.client.1.smithi144.stdout:4/382: symlink d6/dd/d2e/d75/l76 0 2024-05-15T15:54:09.216 INFO:tasks.workunit.client.1.smithi144.stdout:6/329: dread - d7/f78 zero size 2024-05-15T15:54:09.218 INFO:tasks.workunit.client.1.smithi144.stdout:2/350: dread d3/d10/f2e [0,4194304] 0 2024-05-15T15:54:09.220 INFO:tasks.workunit.client.1.smithi144.stdout:9/374: dwrite d0/d8/d5e/d44/d3c/f61 [0,4194304] 0 2024-05-15T15:54:09.220 INFO:tasks.workunit.client.1.smithi144.stdout:4/383: write d6/d42/d46/d4b/d1e/f51 [511935,43485] 0 2024-05-15T15:54:09.223 INFO:tasks.workunit.client.1.smithi144.stdout:1/370: dwrite d7/d1d/d33/d1e/d34/f4a [0,4194304] 0 2024-05-15T15:54:09.224 INFO:tasks.workunit.client.1.smithi144.stdout:4/384: write d6/f29 [4529864,19982] 0 2024-05-15T15:54:09.225 INFO:tasks.workunit.client.1.smithi144.stdout:5/358: dread d0/dc/d12/f43 [0,4194304] 0 2024-05-15T15:54:09.226 INFO:tasks.workunit.client.1.smithi144.stdout:3/384: link d0/d1/d17/d39/d40/d7e/f81 d0/d1/d17/f84 0 2024-05-15T15:54:09.229 INFO:tasks.workunit.client.1.smithi144.stdout:7/373: rename d4/df/d25/d38/d68/f7e to d4/d41/d5b/f80 0 2024-05-15T15:54:09.231 INFO:tasks.workunit.client.1.smithi144.stdout:6/330: fdatasync d7/d32/f50 0 2024-05-15T15:54:09.232 INFO:tasks.workunit.client.1.smithi144.stdout:7/374: chown d4/df/d25/d38/d68 2020 1 2024-05-15T15:54:09.233 INFO:tasks.workunit.client.1.smithi144.stdout:6/331: fsync d7/db/f29 0 2024-05-15T15:54:09.233 INFO:tasks.workunit.client.1.smithi144.stdout:0/363: dwrite d4/f1c [0,4194304] 0 2024-05-15T15:54:09.235 INFO:tasks.workunit.client.1.smithi144.stdout:9/375: creat d0/d8/d5e/d44/d77/f89 x:0 0 0 2024-05-15T15:54:09.241 INFO:tasks.workunit.client.1.smithi144.stdout:4/385: rename d6/dd/d2d/f3f to d6/dd/d2d/f77 0 2024-05-15T15:54:09.242 INFO:tasks.workunit.client.1.smithi144.stdout:4/386: write d6/dd/d34/d3b/d30/f1b [2470880,48445] 0 2024-05-15T15:54:09.245 INFO:tasks.workunit.client.1.smithi144.stdout:8/394: mknod d2/de/d49/d83/c84 0 2024-05-15T15:54:09.249 INFO:tasks.workunit.client.1.smithi144.stdout:3/385: write d0/d11/d24/f49 [1765396,26098] 0 2024-05-15T15:54:09.251 INFO:tasks.workunit.client.1.smithi144.stdout:0/364: mknod d4/d6/d4a/d4d/c84 0 2024-05-15T15:54:09.253 INFO:tasks.workunit.client.1.smithi144.stdout:5/359: unlink d0/dc/d12/d31/d35/f44 0 2024-05-15T15:54:09.254 INFO:tasks.workunit.client.1.smithi144.stdout:1/371: dread d7/d1d/d33/d29/f2c [0,4194304] 0 2024-05-15T15:54:09.255 INFO:tasks.workunit.client.1.smithi144.stdout:4/387: mknod d6/dd/d34/d3b/c78 0 2024-05-15T15:54:09.255 INFO:tasks.workunit.client.1.smithi144.stdout:1/372: read - d7/d1d/d33/d41/d22/f6a zero size 2024-05-15T15:54:09.256 INFO:tasks.workunit.client.1.smithi144.stdout:7/375: getdents d4/df/d79 0 2024-05-15T15:54:09.258 INFO:tasks.workunit.client.1.smithi144.stdout:5/360: mkdir d0/dc/d12/d4a/d73 0 2024-05-15T15:54:09.259 INFO:tasks.workunit.client.1.smithi144.stdout:9/376: rmdir d0/d8/d5e/d44/d16/d5a 39 2024-05-15T15:54:09.259 INFO:tasks.workunit.client.1.smithi144.stdout:9/377: fsync d0/d8/d5e/d44/f87 0 2024-05-15T15:54:09.261 INFO:tasks.workunit.client.1.smithi144.stdout:1/373: dread d7/d1d/d33/d29/f42 [0,4194304] 0 2024-05-15T15:54:09.262 INFO:tasks.workunit.client.1.smithi144.stdout:6/332: creat d7/db/d11/d3c/f79 x:0 0 0 2024-05-15T15:54:09.262 INFO:tasks.workunit.client.1.smithi144.stdout:3/386: truncate d0/d1/f5 1594194 0 2024-05-15T15:54:09.263 INFO:tasks.workunit.client.1.smithi144.stdout:6/333: fsync d7/db/f2c 0 2024-05-15T15:54:09.263 INFO:tasks.workunit.client.1.smithi144.stdout:0/365: fdatasync d4/df/f43 0 2024-05-15T15:54:09.263 INFO:tasks.workunit.client.1.smithi144.stdout:3/387: chown d0/d1/d56/f7f 358 1 2024-05-15T15:54:09.264 INFO:tasks.workunit.client.1.smithi144.stdout:6/334: chown d7/db/d35/f67 20574485 1 2024-05-15T15:54:09.264 INFO:tasks.workunit.client.1.smithi144.stdout:5/361: write d0/dc/d1f/f39 [46147,1117] 0 2024-05-15T15:54:09.265 INFO:tasks.workunit.client.1.smithi144.stdout:4/388: truncate f0 1410498 0 2024-05-15T15:54:09.266 INFO:tasks.workunit.client.1.smithi144.stdout:2/351: link d3/l6 d3/l74 0 2024-05-15T15:54:09.269 INFO:tasks.workunit.client.1.smithi144.stdout:0/366: dwrite d4/dc/f17 [0,4194304] 0 2024-05-15T15:54:09.270 INFO:tasks.workunit.client.1.smithi144.stdout:0/367: readlink d4/dc/l19 0 2024-05-15T15:54:09.271 INFO:tasks.workunit.client.1.smithi144.stdout:0/368: chown d4/d6/d3a/c41 1 1 2024-05-15T15:54:09.272 INFO:tasks.workunit.client.1.smithi144.stdout:0/369: stat d4/d6/c10 0 2024-05-15T15:54:09.273 INFO:tasks.workunit.client.1.smithi144.stdout:8/395: getdents d2/d3a/d7b/d4b 0 2024-05-15T15:54:09.275 INFO:tasks.workunit.client.1.smithi144.stdout:0/370: fsync d4/df/d1f/d24/f4b 0 2024-05-15T15:54:09.277 INFO:tasks.workunit.client.1.smithi144.stdout:9/378: mkdir d0/d8/d1b/d79/d8a 0 2024-05-15T15:54:09.279 INFO:tasks.workunit.client.1.smithi144.stdout:1/374: dwrite d7/d1d/d33/d29/d6b/f65 [0,4194304] 0 2024-05-15T15:54:09.280 INFO:tasks.workunit.client.1.smithi144.stdout:9/379: dread - d0/d8/d5e/d44/d77/f89 zero size 2024-05-15T15:54:09.281 INFO:tasks.workunit.client.1.smithi144.stdout:6/335: rename d7/f4f to d7/db/d5f/f7a 0 2024-05-15T15:54:09.281 INFO:tasks.workunit.client.1.smithi144.stdout:3/388: dread d0/d1/d17/f48 [0,4194304] 0 2024-05-15T15:54:09.281 INFO:tasks.workunit.client.1.smithi144.stdout:7/376: truncate d4/df/f22 1182083 0 2024-05-15T15:54:09.283 INFO:tasks.workunit.client.1.smithi144.stdout:5/362: write d0/f16 [108038,60462] 0 2024-05-15T15:54:09.287 INFO:tasks.workunit.client.1.smithi144.stdout:0/371: mkdir d4/df/d1f/d24/d85 0 2024-05-15T15:54:09.288 INFO:tasks.workunit.client.1.smithi144.stdout:0/372: dread - d4/d6/d3a/d5d/f82 zero size 2024-05-15T15:54:09.289 INFO:tasks.workunit.client.1.smithi144.stdout:1/375: symlink d7/d4b/l79 0 2024-05-15T15:54:09.290 INFO:tasks.workunit.client.1.smithi144.stdout:7/377: dread d4/d41/d5b/f4f [0,4194304] 0 2024-05-15T15:54:09.292 INFO:tasks.workunit.client.1.smithi144.stdout:7/378: chown d4/df/d25/d40/d62/f6b 30947993 1 2024-05-15T15:54:09.293 INFO:tasks.workunit.client.1.smithi144.stdout:2/352: rename d3/d7/d20/d37/d3a to d3/d7/d20/d22/d52/d75 0 2024-05-15T15:54:09.295 INFO:tasks.workunit.client.1.smithi144.stdout:6/336: dread d7/db/d11/f42 [0,4194304] 0 2024-05-15T15:54:09.296 INFO:tasks.workunit.client.1.smithi144.stdout:6/337: chown d7/f71 426 1 2024-05-15T15:54:09.298 INFO:tasks.workunit.client.1.smithi144.stdout:0/373: creat d4/df/d1f/d24/d2b/f86 x:0 0 0 2024-05-15T15:54:09.302 INFO:tasks.workunit.client.1.smithi144.stdout:8/396: rename d2/de/c11 to d2/d3a/d7b/d4b/d5b/d6e/d71/d73/c85 0 2024-05-15T15:54:09.302 INFO:tasks.workunit.client.1.smithi144.stdout:8/397: fsync d2/de/f33 0 2024-05-15T15:54:09.303 INFO:tasks.workunit.client.1.smithi144.stdout:2/353: mkdir d3/d7/d20/d37/d76 0 2024-05-15T15:54:09.303 INFO:tasks.workunit.client.1.smithi144.stdout:1/376: fsync d7/d1d/d33/d29/d6b/f65 0 2024-05-15T15:54:09.304 INFO:tasks.workunit.client.1.smithi144.stdout:2/354: truncate d3/d7/d20/d3f/f53 1234851 0 2024-05-15T15:54:09.305 INFO:tasks.workunit.client.1.smithi144.stdout:5/363: mkdir d0/dc/d24/d28/d2d/d59/d74 0 2024-05-15T15:54:09.308 INFO:tasks.workunit.client.1.smithi144.stdout:0/374: symlink d4/d6/d3a/d40/l87 0 2024-05-15T15:54:09.309 INFO:tasks.workunit.client.1.smithi144.stdout:7/379: mkdir d4/df/d81 0 2024-05-15T15:54:09.315 INFO:tasks.workunit.client.1.smithi144.stdout:6/338: write d7/f8 [3414908,9067] 0 2024-05-15T15:54:09.317 INFO:tasks.workunit.client.1.smithi144.stdout:1/377: symlink d7/d1d/d58/l7a 0 2024-05-15T15:54:09.317 INFO:tasks.workunit.client.1.smithi144.stdout:8/398: dread d2/de/f14 [0,4194304] 0 2024-05-15T15:54:09.318 INFO:tasks.workunit.client.1.smithi144.stdout:2/355: creat d3/d7/d20/d3b/f77 x:0 0 0 2024-05-15T15:54:09.318 INFO:tasks.workunit.client.1.smithi144.stdout:5/364: rmdir d0/dc/d2f/d34/d63 39 2024-05-15T15:54:09.319 INFO:tasks.workunit.client.1.smithi144.stdout:6/339: dread d7/db/d5f/f65 [0,4194304] 0 2024-05-15T15:54:09.320 INFO:tasks.workunit.client.1.smithi144.stdout:8/399: truncate d2/d3a/d7b/d4b/d5b/d6e/f78 844795 0 2024-05-15T15:54:09.320 INFO:tasks.workunit.client.1.smithi144.stdout:2/356: dread - d3/d7/d20/d22/d52/d75/f45 zero size 2024-05-15T15:54:09.321 INFO:tasks.workunit.client.1.smithi144.stdout:8/400: write d2/d24/d2d/d30/f81 [195352,49072] 0 2024-05-15T15:54:09.322 INFO:tasks.workunit.client.1.smithi144.stdout:1/378: creat d7/d31/f7b x:0 0 0 2024-05-15T15:54:09.323 INFO:tasks.workunit.client.1.smithi144.stdout:8/401: stat d2/d3a/d7b/d4b/d5b/d6e/f78 0 2024-05-15T15:54:09.324 INFO:tasks.workunit.client.1.smithi144.stdout:5/365: creat d0/dc/d1f/f75 x:0 0 0 2024-05-15T15:54:09.328 INFO:tasks.workunit.client.1.smithi144.stdout:0/375: link d4/df/d1f/d24/d2b/c36 d4/d6/d4a/d4d/c88 0 2024-05-15T15:54:09.328 INFO:tasks.workunit.client.1.smithi144.stdout:7/380: write d4/d41/d5b/f4f [318859,60489] 0 2024-05-15T15:54:09.330 INFO:tasks.workunit.client.1.smithi144.stdout:8/402: symlink d2/d3a/d7b/d4b/d5b/l86 0 2024-05-15T15:54:09.331 INFO:tasks.workunit.client.1.smithi144.stdout:8/403: fsync d2/d3a/f67 0 2024-05-15T15:54:09.331 INFO:tasks.workunit.client.1.smithi144.stdout:9/380: rename d0/d8/d5e/d44/d10/d19/l40 to d0/l8b 0 2024-05-15T15:54:09.332 INFO:tasks.workunit.client.1.smithi144.stdout:2/357: symlink d3/d4d/l78 0 2024-05-15T15:54:09.333 INFO:tasks.workunit.client.1.smithi144.stdout:0/376: symlink d4/d2d/d48/d5b/l89 0 2024-05-15T15:54:09.333 INFO:tasks.workunit.client.1.smithi144.stdout:2/358: chown d3/l50 2152304 1 2024-05-15T15:54:09.334 INFO:tasks.workunit.client.1.smithi144.stdout:5/366: mkdir d0/dc/d12/d31/d76 0 2024-05-15T15:54:09.336 INFO:tasks.workunit.client.1.smithi144.stdout:7/381: mkdir d4/df/d10/d48/d72/d82 0 2024-05-15T15:54:09.337 INFO:tasks.workunit.client.1.smithi144.stdout:2/359: fdatasync d3/d7/f47 0 2024-05-15T15:54:09.339 INFO:tasks.workunit.client.1.smithi144.stdout:7/382: chown d4/c17 62605 1 2024-05-15T15:54:09.340 INFO:tasks.workunit.client.1.smithi144.stdout:5/367: unlink d0/dc/d12/d4a/l54 0 2024-05-15T15:54:09.342 INFO:tasks.workunit.client.1.smithi144.stdout:1/379: getdents d7/d1d/d33/d55 0 2024-05-15T15:54:09.344 INFO:tasks.workunit.client.1.smithi144.stdout:6/340: dread d7/db/f29 [0,4194304] 0 2024-05-15T15:54:09.344 INFO:tasks.workunit.client.1.smithi144.stdout:0/377: dwrite d4/d2d/f44 [0,4194304] 0 2024-05-15T15:54:09.344 INFO:tasks.workunit.client.1.smithi144.stdout:5/368: dread - d0/dc/d2f/d34/f5e zero size 2024-05-15T15:54:09.345 INFO:tasks.workunit.client.1.smithi144.stdout:7/383: truncate d4/d41/d5b/f80 306810 0 2024-05-15T15:54:09.345 INFO:tasks.workunit.client.1.smithi144.stdout:1/380: chown d7/df/l3b 6443 1 2024-05-15T15:54:09.345 INFO:tasks.workunit.client.1.smithi144.stdout:8/404: dwrite d2/de/d3f/f4e [0,4194304] 0 2024-05-15T15:54:09.346 INFO:tasks.workunit.client.1.smithi144.stdout:2/360: creat d3/d7/d20/d37/f79 x:0 0 0 2024-05-15T15:54:09.349 INFO:tasks.workunit.client.1.smithi144.stdout:7/384: stat d4/d41/d5b/f6f 0 2024-05-15T15:54:09.349 INFO:tasks.workunit.client.1.smithi144.stdout:6/341: write d7/f8 [45684,7034] 0 2024-05-15T15:54:09.354 INFO:tasks.workunit.client.1.smithi144.stdout:8/405: dread d2/de/d49/f69 [0,4194304] 0 2024-05-15T15:54:09.354 INFO:tasks.workunit.client.1.smithi144.stdout:5/369: dread d0/dc/d24/d28/f51 [0,4194304] 0 2024-05-15T15:54:09.358 INFO:tasks.workunit.client.0.smithi049.stdout:0/386: sync 2024-05-15T15:54:09.358 INFO:tasks.workunit.client.1.smithi144.stdout:7/385: dread d4/df/d25/f29 [0,4194304] 0 2024-05-15T15:54:09.359 INFO:tasks.workunit.client.1.smithi144.stdout:1/381: mknod d7/d4b/d53/c7c 0 2024-05-15T15:54:09.360 INFO:tasks.workunit.client.1.smithi144.stdout:5/370: rmdir d0/dc/d2f/d34 39 2024-05-15T15:54:09.362 INFO:tasks.workunit.client.1.smithi144.stdout:8/406: mkdir d2/de/d36/d87 0 2024-05-15T15:54:09.364 INFO:tasks.workunit.client.0.smithi049.stdout:0/387: unlink dc/f40 0 2024-05-15T15:54:09.364 INFO:tasks.workunit.client.1.smithi144.stdout:6/342: dwrite d7/db/d11/f1f [4194304,4194304] 0 2024-05-15T15:54:09.364 INFO:tasks.workunit.client.1.smithi144.stdout:9/381: dwrite d0/d8/d5e/d44/f74 [0,4194304] 0 2024-05-15T15:54:09.370 INFO:tasks.workunit.client.1.smithi144.stdout:1/382: dread d7/d1d/d33/d1e/d34/f4a [0,4194304] 0 2024-05-15T15:54:09.372 INFO:tasks.workunit.client.1.smithi144.stdout:7/386: dread d4/f51 [0,4194304] 0 2024-05-15T15:54:09.373 INFO:tasks.workunit.client.1.smithi144.stdout:2/361: creat d3/d7/d20/d22/f7a x:0 0 0 2024-05-15T15:54:09.373 INFO:tasks.workunit.client.1.smithi144.stdout:7/387: dread - d4/d3c/f57 zero size 2024-05-15T15:54:09.375 INFO:tasks.workunit.client.1.smithi144.stdout:0/378: truncate d4/df/d1f/f3e 2647937 0 2024-05-15T15:54:09.376 INFO:tasks.workunit.client.1.smithi144.stdout:7/388: chown d4/d41/d5b/f80 47 1 2024-05-15T15:54:09.376 INFO:tasks.workunit.client.1.smithi144.stdout:2/362: chown d3/d7/d20/d22/c28 18241917 1 2024-05-15T15:54:09.376 INFO:tasks.workunit.client.1.smithi144.stdout:0/379: fsync d4/d6/f11 0 2024-05-15T15:54:09.380 INFO:tasks.workunit.client.1.smithi144.stdout:8/407: symlink d2/d3a/d7b/d4b/d5b/d6e/l88 0 2024-05-15T15:54:09.380 INFO:tasks.workunit.client.1.smithi144.stdout:1/383: dwrite d7/d1d/d33/f71 [0,4194304] 0 2024-05-15T15:54:09.381 INFO:tasks.workunit.client.1.smithi144.stdout:9/382: write d0/d8/d5e/d44/d3c/f42 [688223,629] 0 2024-05-15T15:54:09.381 INFO:tasks.workunit.client.0.smithi049.stdout:0/388: dwrite dc/d13/d3d/f5d [0,4194304] 0 2024-05-15T15:54:09.390 INFO:tasks.workunit.client.1.smithi144.stdout:2/363: mknod d3/d7/d54/c7b 0 2024-05-15T15:54:09.392 INFO:tasks.workunit.client.1.smithi144.stdout:2/364: write d3/d7/d20/d22/d52/d75/f45 [644976,44329] 0 2024-05-15T15:54:09.392 INFO:tasks.workunit.client.1.smithi144.stdout:5/371: rename d0/dc/d2f/d34/f5e to d0/dc/d24/d57/f77 0 2024-05-15T15:54:09.393 INFO:tasks.workunit.client.0.smithi049.stdout:0/389: dwrite dc/d13/d14/d25/f69 [0,4194304] 0 2024-05-15T15:54:09.398 INFO:tasks.workunit.client.1.smithi144.stdout:6/343: mkdir d7/db/d11/d55/d70/d7b 0 2024-05-15T15:54:09.405 INFO:tasks.workunit.client.0.smithi049.stdout:0/390: unlink dc/c3e 0 2024-05-15T15:54:09.406 INFO:tasks.workunit.client.1.smithi144.stdout:7/389: symlink d4/l83 0 2024-05-15T15:54:09.406 INFO:tasks.workunit.client.1.smithi144.stdout:7/390: dread - d4/df/d25/d40/d62/f6b zero size 2024-05-15T15:54:09.406 INFO:tasks.workunit.client.1.smithi144.stdout:4/389: sync 2024-05-15T15:54:09.406 INFO:tasks.workunit.client.1.smithi144.stdout:7/391: chown d4/df/d79 247641514 1 2024-05-15T15:54:09.413 INFO:tasks.workunit.client.1.smithi144.stdout:8/408: dwrite d2/d1a/f31 [0,4194304] 0 2024-05-15T15:54:09.415 INFO:tasks.workunit.client.1.smithi144.stdout:6/344: rename d7/d1e/l21 to d7/db/d3a/l7c 0 2024-05-15T15:54:09.415 INFO:tasks.workunit.client.1.smithi144.stdout:1/384: getdents d7/d1d/d33/d41/d22/d66 0 2024-05-15T15:54:09.417 INFO:tasks.workunit.client.0.smithi049.stdout:0/391: dread dc/d13/d14/f19 [0,4194304] 0 2024-05-15T15:54:09.419 INFO:tasks.workunit.client.1.smithi144.stdout:7/392: unlink d4/df/d25/d38/f3f 0 2024-05-15T15:54:09.420 INFO:tasks.workunit.client.1.smithi144.stdout:7/393: stat d4/df/c2d 0 2024-05-15T15:54:09.422 INFO:tasks.workunit.client.1.smithi144.stdout:8/409: dwrite d2/de/d3f/f40 [0,4194304] 0 2024-05-15T15:54:09.422 INFO:tasks.workunit.client.0.smithi049.stdout:0/392: mknod dc/d13/d48/d57/c77 0 2024-05-15T15:54:09.423 INFO:tasks.workunit.client.0.smithi049.stdout:0/393: chown dc/d13/d14/l28 747 1 2024-05-15T15:54:09.424 INFO:tasks.workunit.client.0.smithi049.stdout:0/394: chown dc/d13/d48/f50 0 1 2024-05-15T15:54:09.425 INFO:tasks.workunit.client.1.smithi144.stdout:7/394: dread d4/f67 [0,4194304] 0 2024-05-15T15:54:09.426 INFO:tasks.workunit.client.1.smithi144.stdout:0/380: link d4/d2d/d48/c50 d4/df/d1f/d24/c8a 0 2024-05-15T15:54:09.428 INFO:tasks.workunit.client.0.smithi049.stdout:0/395: symlink dc/d6d/l78 0 2024-05-15T15:54:09.428 INFO:tasks.workunit.client.1.smithi144.stdout:4/390: write d6/dd/d34/d3b/f21 [4855285,83494] 0 2024-05-15T15:54:09.429 INFO:tasks.workunit.client.1.smithi144.stdout:6/345: chown d7/l2b 2 1 2024-05-15T15:54:09.432 INFO:tasks.workunit.client.1.smithi144.stdout:5/372: truncate d0/dc/d12/f18 2967097 0 2024-05-15T15:54:09.432 INFO:tasks.workunit.client.1.smithi144.stdout:8/410: dread d2/de/d36/f48 [0,4194304] 0 2024-05-15T15:54:09.433 INFO:tasks.workunit.client.1.smithi144.stdout:1/385: fdatasync d7/d1d/d33/d1e/d34/f4a 0 2024-05-15T15:54:09.434 INFO:tasks.workunit.client.1.smithi144.stdout:2/365: creat d3/d7/d20/f7c x:0 0 0 2024-05-15T15:54:09.435 INFO:tasks.workunit.client.1.smithi144.stdout:8/411: dread d2/de/d36/f48 [0,4194304] 0 2024-05-15T15:54:09.436 INFO:tasks.workunit.client.1.smithi144.stdout:8/412: write d2/de/d36/f1b [4416380,50807] 0 2024-05-15T15:54:09.437 INFO:tasks.workunit.client.1.smithi144.stdout:7/395: write d4/d41/d5b/f4f [1479888,83431] 0 2024-05-15T15:54:09.439 INFO:tasks.workunit.client.1.smithi144.stdout:0/381: unlink d4/d6/d4a/f5a 0 2024-05-15T15:54:09.441 INFO:tasks.workunit.client.1.smithi144.stdout:4/391: creat d6/dd/d34/d3b/d30/db/d1d/d40/f79 x:0 0 0 2024-05-15T15:54:09.444 INFO:tasks.workunit.client.1.smithi144.stdout:8/413: dwrite d2/d3a/f82 [0,4194304] 0 2024-05-15T15:54:09.445 INFO:tasks.workunit.client.1.smithi144.stdout:8/414: chown d2/de/d36/f2f 964471848 1 2024-05-15T15:54:09.445 INFO:tasks.workunit.client.1.smithi144.stdout:2/366: dread - d3/d7/f36 zero size 2024-05-15T15:54:09.450 INFO:tasks.workunit.client.1.smithi144.stdout:6/346: symlink d7/d1e/l7d 0 2024-05-15T15:54:09.451 INFO:tasks.workunit.client.1.smithi144.stdout:0/382: mknod d4/df/d1f/d56/c8b 0 2024-05-15T15:54:09.453 INFO:tasks.workunit.client.1.smithi144.stdout:1/386: symlink d7/df/l7d 0 2024-05-15T15:54:09.454 INFO:tasks.workunit.client.1.smithi144.stdout:4/392: symlink d6/dd/d34/d3b/d30/db/d1d/d40/l7a 0 2024-05-15T15:54:09.455 INFO:tasks.workunit.client.1.smithi144.stdout:5/373: mknod d0/dc/d12/d31/d35/d6b/c78 0 2024-05-15T15:54:09.456 INFO:tasks.workunit.client.1.smithi144.stdout:8/415: creat d2/d24/f89 x:0 0 0 2024-05-15T15:54:09.457 INFO:tasks.workunit.client.1.smithi144.stdout:8/416: readlink d2/d24/d2d/l37 0 2024-05-15T15:54:09.458 INFO:tasks.workunit.client.1.smithi144.stdout:7/396: dread d4/d41/d5b/f6f [0,4194304] 0 2024-05-15T15:54:09.458 INFO:tasks.workunit.client.1.smithi144.stdout:8/417: fdatasync d2/d3a/f7f 0 2024-05-15T15:54:09.459 INFO:tasks.workunit.client.1.smithi144.stdout:6/347: mkdir d7/db/d11/d7e 0 2024-05-15T15:54:09.459 INFO:tasks.workunit.client.1.smithi144.stdout:8/418: chown d2/c6f 435384 1 2024-05-15T15:54:09.460 INFO:tasks.workunit.client.1.smithi144.stdout:0/383: symlink d4/df/d1f/d24/d2b/d55/l8c 0 2024-05-15T15:54:09.460 INFO:tasks.workunit.client.1.smithi144.stdout:1/387: symlink d7/d1d/d33/d1e/d4e/l7e 0 2024-05-15T15:54:09.461 INFO:tasks.workunit.client.1.smithi144.stdout:8/419: readlink d2/de/d36/l1c 0 2024-05-15T15:54:09.462 INFO:tasks.workunit.client.1.smithi144.stdout:0/384: readlink d4/df/d1f/d56/l6f 0 2024-05-15T15:54:09.462 INFO:tasks.workunit.client.1.smithi144.stdout:5/374: dread d0/f1a [0,4194304] 0 2024-05-15T15:54:09.463 INFO:tasks.workunit.client.1.smithi144.stdout:1/388: rmdir d7/d1d 39 2024-05-15T15:54:09.464 INFO:tasks.workunit.client.1.smithi144.stdout:6/348: dread d7/db/d11/d55/f5e [0,4194304] 0 2024-05-15T15:54:09.465 INFO:tasks.workunit.client.1.smithi144.stdout:4/393: link d6/d42/d46/d4b/d1e/d44/f53 d6/dd/d34/d3b/d30/db/f7b 0 2024-05-15T15:54:09.465 INFO:tasks.workunit.client.1.smithi144.stdout:8/420: symlink d2/d24/l8a 0 2024-05-15T15:54:09.466 INFO:tasks.workunit.client.1.smithi144.stdout:2/367: link d3/d7/d20/d22/d52/l60 d3/d7/d20/d22/d52/d75/l7d 0 2024-05-15T15:54:09.467 INFO:tasks.workunit.client.1.smithi144.stdout:2/368: dread - d3/d7/d20/d22/f7a zero size 2024-05-15T15:54:09.468 INFO:tasks.workunit.client.1.smithi144.stdout:5/375: dwrite d0/f49 [0,4194304] 0 2024-05-15T15:54:09.468 INFO:tasks.workunit.client.1.smithi144.stdout:1/389: symlink d7/d4b/l7f 0 2024-05-15T15:54:09.470 INFO:tasks.workunit.client.1.smithi144.stdout:4/394: mkdir d6/d7c 0 2024-05-15T15:54:09.471 INFO:tasks.workunit.client.1.smithi144.stdout:5/376: truncate d0/dc/d24/d28/d2d/f69 69774 0 2024-05-15T15:54:09.472 INFO:tasks.workunit.client.1.smithi144.stdout:8/421: mknod d2/d24/c8b 0 2024-05-15T15:54:09.472 INFO:tasks.workunit.client.1.smithi144.stdout:4/395: fsync d6/dd/d2e/f61 0 2024-05-15T15:54:09.475 INFO:tasks.workunit.client.1.smithi144.stdout:2/369: dwrite d3/d7/d54/f5a [0,4194304] 0 2024-05-15T15:54:09.484 INFO:tasks.workunit.client.1.smithi144.stdout:2/370: creat d3/d10/f7e x:0 0 0 2024-05-15T15:54:09.485 INFO:tasks.workunit.client.1.smithi144.stdout:5/377: mkdir d0/dc/d2f/d79 0 2024-05-15T15:54:09.486 INFO:tasks.workunit.client.1.smithi144.stdout:2/371: write d3/d10/f7e [147415,76139] 0 2024-05-15T15:54:09.489 INFO:tasks.workunit.client.1.smithi144.stdout:7/397: truncate d4/df/d10/f23 2738206 0 2024-05-15T15:54:09.489 INFO:tasks.workunit.client.1.smithi144.stdout:7/398: chown d4/d45 222147 1 2024-05-15T15:54:09.489 INFO:tasks.workunit.client.1.smithi144.stdout:6/349: getdents d7/d1e/d28 0 2024-05-15T15:54:09.490 INFO:tasks.workunit.client.1.smithi144.stdout:1/390: mkdir d7/d1d/d33/d29/d6b/d5e/d74/d80 0 2024-05-15T15:54:09.491 INFO:tasks.workunit.client.1.smithi144.stdout:6/350: chown d7/db/d11/d3c/d56/d72 4925739 1 2024-05-15T15:54:09.492 INFO:tasks.workunit.client.1.smithi144.stdout:4/396: dread d6/f29 [0,4194304] 0 2024-05-15T15:54:09.493 INFO:tasks.workunit.client.1.smithi144.stdout:7/399: creat d4/df/d25/d38/d5d/f84 x:0 0 0 2024-05-15T15:54:09.493 INFO:tasks.workunit.client.1.smithi144.stdout:4/397: dread - d6/dd/d34/d3b/d30/db/d1d/d40/f79 zero size 2024-05-15T15:54:09.494 INFO:tasks.workunit.client.1.smithi144.stdout:2/372: mknod d3/d7/d20/c7f 0 2024-05-15T15:54:09.495 INFO:tasks.workunit.client.1.smithi144.stdout:4/398: chown d6/d42/d46/l6f 70 1 2024-05-15T15:54:09.498 INFO:tasks.workunit.client.1.smithi144.stdout:8/422: truncate d2/f35 407636 0 2024-05-15T15:54:09.498 INFO:tasks.workunit.client.1.smithi144.stdout:5/378: dread d0/f22 [4194304,4194304] 0 2024-05-15T15:54:09.498 INFO:tasks.workunit.client.1.smithi144.stdout:4/399: mkdir d6/dd/d34/d3b/d30/d59/d66/d7d 0 2024-05-15T15:54:09.499 INFO:tasks.workunit.client.1.smithi144.stdout:2/373: truncate d3/d7/d20/d22/d52/d75/d65/f68 852539 0 2024-05-15T15:54:09.499 INFO:tasks.workunit.client.1.smithi144.stdout:6/351: rename d7/db/d35/c4c to d7/c7f 0 2024-05-15T15:54:09.501 INFO:tasks.workunit.client.1.smithi144.stdout:7/400: creat d4/df/d25/f85 x:0 0 0 2024-05-15T15:54:09.502 INFO:tasks.workunit.client.1.smithi144.stdout:6/352: symlink d7/d32/l80 0 2024-05-15T15:54:09.502 INFO:tasks.workunit.client.1.smithi144.stdout:8/423: dread d2/d3a/d7b/d4b/d5b/d6e/f78 [0,4194304] 0 2024-05-15T15:54:09.503 INFO:tasks.workunit.client.1.smithi144.stdout:2/374: mkdir d3/d7/d20/d22/d52/d64/d80 0 2024-05-15T15:54:09.506 INFO:tasks.workunit.client.1.smithi144.stdout:7/401: write d4/df/d25/d38/f7f [472039,51566] 0 2024-05-15T15:54:09.512 INFO:tasks.workunit.client.1.smithi144.stdout:5/379: dwrite d0/dc/d1f/f39 [0,4194304] 0 2024-05-15T15:54:09.513 INFO:tasks.workunit.client.1.smithi144.stdout:1/391: dread d7/d1d/d33/d41/f19 [0,4194304] 0 2024-05-15T15:54:09.513 INFO:tasks.workunit.client.1.smithi144.stdout:4/400: dwrite d6/d42/d46/d4b/f1c [0,4194304] 0 2024-05-15T15:54:09.513 INFO:tasks.workunit.client.1.smithi144.stdout:6/353: dread d7/fd [0,4194304] 0 2024-05-15T15:54:09.515 INFO:tasks.workunit.client.1.smithi144.stdout:4/401: fsync d6/dd/d34/d3b/d30/f52 0 2024-05-15T15:54:09.519 INFO:tasks.workunit.client.1.smithi144.stdout:6/354: truncate d7/db/d11/f43 859527 0 2024-05-15T15:54:09.519 INFO:tasks.workunit.client.1.smithi144.stdout:4/402: readlink d6/dd/d2e/d75/l76 0 2024-05-15T15:54:09.523 INFO:tasks.workunit.client.1.smithi144.stdout:1/392: rmdir d7/d5c 0 2024-05-15T15:54:09.525 INFO:tasks.workunit.client.1.smithi144.stdout:5/380: dwrite d0/dc/f1b [0,4194304] 0 2024-05-15T15:54:09.527 INFO:tasks.workunit.client.1.smithi144.stdout:7/402: dread d4/f27 [0,4194304] 0 2024-05-15T15:54:09.527 INFO:tasks.workunit.client.1.smithi144.stdout:4/403: rename d6/dd/d34/d3b/d30/d59/d66/d7d to d6/d7e 0 2024-05-15T15:54:09.529 INFO:tasks.workunit.client.1.smithi144.stdout:4/404: rename d6/dd to d6/dd/d2e/d7f 22 2024-05-15T15:54:09.531 INFO:tasks.workunit.client.1.smithi144.stdout:3/389: sync 2024-05-15T15:54:09.531 INFO:tasks.workunit.client.1.smithi144.stdout:1/393: mkdir d7/d1d/d58/d81 0 2024-05-15T15:54:09.533 INFO:tasks.workunit.client.1.smithi144.stdout:3/390: chown d0/d1/d17/d39/d40/d4a 3451 1 2024-05-15T15:54:09.535 INFO:tasks.workunit.client.1.smithi144.stdout:3/391: write d0/d11/f26 [2694037,95112] 0 2024-05-15T15:54:09.535 INFO:tasks.workunit.client.1.smithi144.stdout:5/381: truncate d0/f11 4616393 0 2024-05-15T15:54:09.537 INFO:tasks.workunit.client.1.smithi144.stdout:4/405: dwrite d6/d42/d46/d4b/f1c [0,4194304] 0 2024-05-15T15:54:09.537 INFO:tasks.workunit.client.1.smithi144.stdout:7/403: mknod d4/df/d25/d38/c86 0 2024-05-15T15:54:09.538 INFO:tasks.workunit.client.1.smithi144.stdout:7/404: read - d4/d3c/f57 zero size 2024-05-15T15:54:09.538 INFO:tasks.workunit.client.1.smithi144.stdout:7/405: dread - d4/f37 zero size 2024-05-15T15:54:09.539 INFO:tasks.workunit.client.1.smithi144.stdout:4/406: dread - d6/dd/d34/d3b/d30/d59/f74 zero size 2024-05-15T15:54:09.542 INFO:tasks.workunit.client.1.smithi144.stdout:8/424: truncate d2/de/d36/d58/f72 113896 0 2024-05-15T15:54:09.544 INFO:tasks.workunit.client.1.smithi144.stdout:2/375: truncate d3/d7/d20/d3f/f53 1052183 0 2024-05-15T15:54:09.546 INFO:tasks.workunit.client.1.smithi144.stdout:6/355: dread d7/db/f3f [0,4194304] 0 2024-05-15T15:54:09.546 INFO:tasks.workunit.client.1.smithi144.stdout:7/406: write d4/f12 [1907918,113007] 0 2024-05-15T15:54:09.548 INFO:tasks.workunit.client.1.smithi144.stdout:2/376: write d3/d7/d20/d22/d52/d75/d65/f68 [782499,23457] 0 2024-05-15T15:54:09.550 INFO:tasks.workunit.client.1.smithi144.stdout:4/407: creat d6/dd/d34/d3b/d30/db/d1d/d40/f80 x:0 0 0 2024-05-15T15:54:09.551 INFO:tasks.workunit.client.1.smithi144.stdout:2/377: dread - d3/d7/f36 zero size 2024-05-15T15:54:09.551 INFO:tasks.workunit.client.1.smithi144.stdout:7/407: truncate d4/d41/d5b/f80 1124448 0 2024-05-15T15:54:09.552 INFO:tasks.workunit.client.1.smithi144.stdout:8/425: dwrite d2/d3a/f59 [0,4194304] 0 2024-05-15T15:54:09.554 INFO:tasks.workunit.client.1.smithi144.stdout:7/408: chown d4/df/d25/f85 23436 1 2024-05-15T15:54:09.557 INFO:tasks.workunit.client.1.smithi144.stdout:1/394: getdents d7/d31 0 2024-05-15T15:54:09.557 INFO:tasks.workunit.client.1.smithi144.stdout:4/408: dread - d6/dd/d34/d3b/d30/d59/f74 zero size 2024-05-15T15:54:09.558 INFO:tasks.workunit.client.1.smithi144.stdout:5/382: fsync d0/f11 0 2024-05-15T15:54:09.558 INFO:tasks.workunit.client.1.smithi144.stdout:8/426: write d2/d24/d2d/f5e [1337484,125082] 0 2024-05-15T15:54:09.560 INFO:tasks.workunit.client.1.smithi144.stdout:6/356: dwrite d7/d1e/f75 [0,4194304] 0 2024-05-15T15:54:09.562 INFO:tasks.workunit.client.1.smithi144.stdout:0/385: sync 2024-05-15T15:54:09.563 INFO:tasks.workunit.client.1.smithi144.stdout:2/378: write d3/f5 [663543,89178] 0 2024-05-15T15:54:09.564 INFO:tasks.workunit.client.1.smithi144.stdout:9/383: sync 2024-05-15T15:54:09.565 INFO:tasks.workunit.client.1.smithi144.stdout:2/379: write d3/d10/f7e [804496,121247] 0 2024-05-15T15:54:09.566 INFO:tasks.workunit.client.1.smithi144.stdout:7/409: symlink d4/df/d25/d38/d5d/l87 0 2024-05-15T15:54:09.571 INFO:tasks.workunit.client.1.smithi144.stdout:4/409: creat d6/d42/d46/d4b/d1e/d44/f81 x:0 0 0 2024-05-15T15:54:09.572 INFO:tasks.workunit.client.1.smithi144.stdout:5/383: symlink d0/dc/d12/d31/d35/l7a 0 2024-05-15T15:54:09.572 INFO:tasks.workunit.client.1.smithi144.stdout:8/427: dwrite d2/d3a/f7c [0,4194304] 0 2024-05-15T15:54:09.576 INFO:tasks.workunit.client.1.smithi144.stdout:0/386: dwrite d4/df/d1f/d24/d2b/d55/f72 [0,4194304] 0 2024-05-15T15:54:09.579 INFO:tasks.workunit.client.1.smithi144.stdout:7/410: creat d4/d41/d5b/f88 x:0 0 0 2024-05-15T15:54:09.580 INFO:tasks.workunit.client.1.smithi144.stdout:3/392: dwrite d0/d11/d24/f3e [0,4194304] 0 2024-05-15T15:54:09.581 INFO:tasks.workunit.client.1.smithi144.stdout:6/357: mkdir d7/db/d11/d55/d70/d7b/d81 0 2024-05-15T15:54:09.581 INFO:tasks.workunit.client.1.smithi144.stdout:3/393: write d0/d11/f26 [3727932,42319] 0 2024-05-15T15:54:09.584 INFO:tasks.workunit.client.1.smithi144.stdout:6/358: chown d7/d1e/c30 125481065 1 2024-05-15T15:54:09.584 INFO:tasks.workunit.client.1.smithi144.stdout:5/384: mknod d0/dc/d1f/c7b 0 2024-05-15T15:54:09.584 INFO:tasks.workunit.client.1.smithi144.stdout:5/385: write d0/dc/d1f/f38 [3942319,10387] 0 2024-05-15T15:54:09.588 INFO:tasks.workunit.client.1.smithi144.stdout:8/428: mkdir d2/d3a/d7b/d4b/d5b/d6e/d71/d8c 0 2024-05-15T15:54:09.589 INFO:tasks.workunit.client.1.smithi144.stdout:8/429: readlink d2/l4 0 2024-05-15T15:54:09.595 INFO:tasks.workunit.client.1.smithi144.stdout:1/395: rename d7/d1d/d33/d29/d6b/d5e/l60 to d7/d1d/d33/d29/l82 0 2024-05-15T15:54:09.596 INFO:tasks.workunit.client.1.smithi144.stdout:2/380: write d3/f1c [1826468,12529] 0 2024-05-15T15:54:09.597 INFO:tasks.workunit.client.1.smithi144.stdout:3/394: symlink d0/d11/d24/l85 0 2024-05-15T15:54:09.600 INFO:tasks.workunit.client.1.smithi144.stdout:4/410: mkdir d6/dd/d34/d3b/d30/db/d1d/d40/d6c/d82 0 2024-05-15T15:54:09.600 INFO:tasks.workunit.client.1.smithi144.stdout:5/386: rmdir d0/dc/d12 39 2024-05-15T15:54:09.601 INFO:tasks.workunit.client.1.smithi144.stdout:7/411: dread d4/df/f22 [0,4194304] 0 2024-05-15T15:54:09.601 INFO:tasks.workunit.client.1.smithi144.stdout:8/430: mkdir d2/d3a/d7b/d4b/d5b/d6e/d8d 0 2024-05-15T15:54:09.604 INFO:tasks.workunit.client.1.smithi144.stdout:8/431: write d2/de/d3f/f4e [5068127,108581] 0 2024-05-15T15:54:09.604 INFO:tasks.workunit.client.1.smithi144.stdout:3/395: unlink d0/d1/d17/d39/d4d/c52 0 2024-05-15T15:54:09.605 INFO:tasks.workunit.client.1.smithi144.stdout:4/411: mknod d6/d42/c83 0 2024-05-15T15:54:09.609 INFO:tasks.workunit.client.1.smithi144.stdout:9/384: rename d0/d8/d5e/d44/d10/d19/c5d to d0/d8/d1b/d62/d7d/c8c 0 2024-05-15T15:54:09.614 INFO:tasks.workunit.client.1.smithi144.stdout:6/359: creat d7/d1e/f82 x:0 0 0 2024-05-15T15:54:09.615 INFO:tasks.workunit.client.1.smithi144.stdout:0/387: getdents d4/d2d 0 2024-05-15T15:54:09.616 INFO:tasks.workunit.client.1.smithi144.stdout:8/432: symlink d2/d3a/d7b/l8e 0 2024-05-15T15:54:09.617 INFO:tasks.workunit.client.1.smithi144.stdout:8/433: chown d2/d3a 0 1 2024-05-15T15:54:09.618 INFO:tasks.workunit.client.1.smithi144.stdout:3/396: creat d0/d1/d56/d71/d7a/f86 x:0 0 0 2024-05-15T15:54:09.620 INFO:tasks.workunit.client.1.smithi144.stdout:4/412: symlink d6/dd/d34/d3b/d30/db/d1d/l84 0 2024-05-15T15:54:09.622 INFO:tasks.workunit.client.1.smithi144.stdout:5/387: creat d0/dc/d2f/d34/d63/d6e/f7c x:0 0 0 2024-05-15T15:54:09.623 INFO:tasks.workunit.client.1.smithi144.stdout:5/388: chown d0/dc/d1f/l5a 1038379553 1 2024-05-15T15:54:09.624 INFO:tasks.workunit.client.1.smithi144.stdout:1/396: creat d7/d1d/d33/d41/f83 x:0 0 0 2024-05-15T15:54:09.624 INFO:tasks.workunit.client.1.smithi144.stdout:1/397: write f1 [6360570,15401] 0 2024-05-15T15:54:09.628 INFO:tasks.workunit.client.1.smithi144.stdout:7/412: rename d4/df/d25/f3a to d4/df/d10/d48/d72/d82/f89 0 2024-05-15T15:54:09.630 INFO:tasks.workunit.client.1.smithi144.stdout:2/381: link d3/d10/l32 d3/d7/d20/d22/d52/d75/d65/l81 0 2024-05-15T15:54:09.630 INFO:tasks.workunit.client.1.smithi144.stdout:2/382: chown d3/d10 0 1 2024-05-15T15:54:09.631 INFO:tasks.workunit.client.1.smithi144.stdout:0/388: creat d4/d2d/f8d x:0 0 0 2024-05-15T15:54:09.632 INFO:tasks.workunit.client.1.smithi144.stdout:0/389: write f2 [268984,41802] 0 2024-05-15T15:54:09.633 INFO:tasks.workunit.client.1.smithi144.stdout:3/397: creat d0/d1/d56/f87 x:0 0 0 2024-05-15T15:54:09.634 INFO:tasks.workunit.client.1.smithi144.stdout:4/413: fsync d6/dd/d2d/f77 0 2024-05-15T15:54:09.635 INFO:tasks.workunit.client.1.smithi144.stdout:4/414: fdatasync d6/dd/d2d/f48 0 2024-05-15T15:54:09.637 INFO:tasks.workunit.client.1.smithi144.stdout:7/413: rmdir d4/d3c 39 2024-05-15T15:54:09.639 INFO:tasks.workunit.client.1.smithi144.stdout:7/414: write d4/df/d10/f6e [524339,41795] 0 2024-05-15T15:54:09.640 INFO:tasks.workunit.client.1.smithi144.stdout:0/390: dwrite d4/d6/d3a/d40/f7d [0,4194304] 0 2024-05-15T15:54:09.642 INFO:tasks.workunit.client.1.smithi144.stdout:0/391: read d4/d6/d3a/d40/f7d [11310,13556] 0 2024-05-15T15:54:09.643 INFO:tasks.workunit.client.1.smithi144.stdout:9/385: dwrite d0/d8/d21/f38 [0,4194304] 0 2024-05-15T15:54:09.645 INFO:tasks.workunit.client.1.smithi144.stdout:9/386: read d0/d8/d5e/d44/d10/f17 [3050379,72780] 0 2024-05-15T15:54:09.646 INFO:tasks.workunit.client.1.smithi144.stdout:9/387: write d0/d8/d5e/d44/d10/f17 [2803565,110848] 0 2024-05-15T15:54:09.649 INFO:tasks.workunit.client.0.smithi049.stdout:4/434: sync 2024-05-15T15:54:09.650 INFO:tasks.workunit.client.1.smithi144.stdout:0/392: dread d4/df/f43 [0,4194304] 0 2024-05-15T15:54:09.651 INFO:tasks.workunit.client.1.smithi144.stdout:0/393: dread d4/d6/d3a/d5d/f65 [0,4194304] 0 2024-05-15T15:54:09.663 INFO:tasks.workunit.client.1.smithi144.stdout:2/383: rmdir d3/d4d 39 2024-05-15T15:54:09.666 INFO:tasks.workunit.client.1.smithi144.stdout:3/398: creat d0/d11/d24/f88 x:0 0 0 2024-05-15T15:54:09.673 INFO:tasks.workunit.client.1.smithi144.stdout:4/415: creat d6/dd/d34/d3b/d30/db/d1d/d40/d57/f85 x:0 0 0 2024-05-15T15:54:09.676 INFO:tasks.workunit.client.0.smithi049.stdout:4/435: write fa [1949154,87239] 0 2024-05-15T15:54:09.678 INFO:tasks.workunit.client.1.smithi144.stdout:9/388: mkdir d0/d8/d1b/d62/d7d/d8d 0 2024-05-15T15:54:09.680 INFO:tasks.workunit.client.1.smithi144.stdout:9/389: readlink d0/d8/l6c 0 2024-05-15T15:54:09.680 INFO:tasks.workunit.client.1.smithi144.stdout:4/416: dwrite d6/dd/d2d/f54 [0,4194304] 0 2024-05-15T15:54:09.682 INFO:tasks.workunit.client.1.smithi144.stdout:8/434: dwrite d2/f35 [0,4194304] 0 2024-05-15T15:54:09.683 INFO:tasks.workunit.client.1.smithi144.stdout:2/384: symlink d3/d10/l82 0 2024-05-15T15:54:09.684 INFO:tasks.workunit.client.0.smithi049.stdout:4/436: link d18/f28 d18/d1e/f8e 0 2024-05-15T15:54:09.684 INFO:tasks.workunit.client.1.smithi144.stdout:3/399: mknod d0/d1/d56/c89 0 2024-05-15T15:54:09.686 INFO:tasks.workunit.client.1.smithi144.stdout:6/360: getdents d7/d32 0 2024-05-15T15:54:09.686 INFO:tasks.workunit.client.0.smithi049.stdout:4/437: fdatasync d18/d21/f56 0 2024-05-15T15:54:09.694 INFO:tasks.workunit.client.1.smithi144.stdout:0/394: getdents d4/df/d1f/d24/d85 0 2024-05-15T15:54:09.695 INFO:tasks.workunit.client.1.smithi144.stdout:9/390: rmdir d0/d8/d5e/d44/d10 39 2024-05-15T15:54:09.695 INFO:tasks.workunit.client.1.smithi144.stdout:0/395: chown d4/df/d1f/d56/c8b 7286 1 2024-05-15T15:54:09.699 INFO:tasks.workunit.client.1.smithi144.stdout:8/435: mknod d2/de/d3f/d6b/c8f 0 2024-05-15T15:54:09.701 INFO:tasks.workunit.client.1.smithi144.stdout:8/436: write d2/de/d36/f2c [1805518,4079] 0 2024-05-15T15:54:09.703 INFO:tasks.workunit.client.1.smithi144.stdout:8/437: truncate d2/d3a/d7b/d4b/d5b/d6e/f56 869532 0 2024-05-15T15:54:09.703 INFO:tasks.workunit.client.1.smithi144.stdout:3/400: symlink d0/d1/d17/d39/d40/l8a 0 2024-05-15T15:54:09.704 INFO:tasks.workunit.client.1.smithi144.stdout:1/398: link l5 d7/d1d/d33/d41/l84 0 2024-05-15T15:54:09.706 INFO:tasks.workunit.client.1.smithi144.stdout:6/361: write d7/db/d35/f67 [1522600,33594] 0 2024-05-15T15:54:09.706 INFO:tasks.workunit.client.1.smithi144.stdout:8/438: dread d2/d3a/f59 [0,4194304] 0 2024-05-15T15:54:09.707 INFO:tasks.workunit.client.1.smithi144.stdout:9/391: creat d0/d8/d1b/d62/f8e x:0 0 0 2024-05-15T15:54:09.709 INFO:tasks.workunit.client.1.smithi144.stdout:0/396: mkdir d4/d2d/d48/d5b/d8e 0 2024-05-15T15:54:09.711 INFO:tasks.workunit.client.1.smithi144.stdout:2/385: rename d3/d7/d20/f5b to d3/d6b/d6e/f83 0 2024-05-15T15:54:09.712 INFO:tasks.workunit.client.1.smithi144.stdout:2/386: chown d3/d10/f1d 32143995 1 2024-05-15T15:54:09.718 INFO:tasks.workunit.client.1.smithi144.stdout:9/392: chown d0/d8/d5e/d44/d10 21486504 1 2024-05-15T15:54:09.718 INFO:tasks.workunit.client.0.smithi049.stdout:4/438: truncate d18/d1e/f23 239194 0 2024-05-15T15:54:09.720 INFO:tasks.workunit.client.1.smithi144.stdout:0/397: creat d4/d6/d3a/d5d/f8f x:0 0 0 2024-05-15T15:54:09.721 INFO:tasks.workunit.client.1.smithi144.stdout:6/362: dread d7/db/d11/d3c/f4d [0,4194304] 0 2024-05-15T15:54:09.723 INFO:tasks.workunit.client.1.smithi144.stdout:3/401: rename d0/d11/d3b/f76 to d0/d1/d17/f8b 0 2024-05-15T15:54:09.725 INFO:tasks.workunit.client.0.smithi049.stdout:4/439: creat d18/d1e/d2a/d36/f8f x:0 0 0 2024-05-15T15:54:09.726 INFO:tasks.workunit.client.1.smithi144.stdout:4/417: link d6/dd/d34/d3b/d30/db/d1d/d40/l7a d6/d49/l86 0 2024-05-15T15:54:09.729 INFO:tasks.workunit.client.1.smithi144.stdout:1/399: unlink d7/df/c46 0 2024-05-15T15:54:09.730 INFO:tasks.workunit.client.1.smithi144.stdout:7/415: sync 2024-05-15T15:54:09.730 INFO:tasks.workunit.client.1.smithi144.stdout:5/389: sync 2024-05-15T15:54:09.732 INFO:tasks.workunit.client.1.smithi144.stdout:7/416: truncate d4/df/d25/d38/f61 366324 0 2024-05-15T15:54:09.733 INFO:tasks.workunit.client.1.smithi144.stdout:5/390: truncate d0/dc/d2f/d34/d63/d6e/f7c 789537 0 2024-05-15T15:54:09.733 INFO:tasks.workunit.client.0.smithi049.stdout:4/440: creat d18/d1e/d2a/d50/f90 x:0 0 0 2024-05-15T15:54:09.734 INFO:tasks.workunit.client.1.smithi144.stdout:5/391: write d0/dc/d2f/d34/d63/d6e/f7c [877959,28043] 0 2024-05-15T15:54:09.735 INFO:tasks.workunit.client.1.smithi144.stdout:0/398: symlink d4/d6/d3a/d6a/l90 0 2024-05-15T15:54:09.735 INFO:tasks.workunit.client.1.smithi144.stdout:0/399: chown d4/l1e 0 1 2024-05-15T15:54:09.736 INFO:tasks.workunit.client.1.smithi144.stdout:0/400: chown d4/df/d1f/d24 134032 1 2024-05-15T15:54:09.736 INFO:tasks.workunit.client.1.smithi144.stdout:7/417: dread d4/df/d25/f29 [0,4194304] 0 2024-05-15T15:54:09.738 INFO:tasks.workunit.client.1.smithi144.stdout:2/387: mknod d3/d7/d20/c84 0 2024-05-15T15:54:09.739 INFO:tasks.workunit.client.1.smithi144.stdout:4/418: symlink d6/dd/d34/d3b/d30/db/d1d/d40/d6c/l87 0 2024-05-15T15:54:09.743 INFO:tasks.workunit.client.1.smithi144.stdout:9/393: rename d0/d8/d5e/d44/d16/d5a/c66 to d0/d8/d1b/d79/c8f 0 2024-05-15T15:54:09.745 INFO:tasks.workunit.client.1.smithi144.stdout:6/363: link d7/db/d11/f60 d7/db/d11/d55/d70/d7b/d81/f83 0 2024-05-15T15:54:09.746 INFO:tasks.workunit.client.0.smithi049.stdout:4/441: dread d18/f1b [0,4194304] 0 2024-05-15T15:54:09.747 INFO:tasks.workunit.client.0.smithi049.stdout:4/442: stat d18/d1e 0 2024-05-15T15:54:09.749 INFO:tasks.workunit.client.0.smithi049.stdout:4/443: chown d18/d1e/d2a/d50 0 1 2024-05-15T15:54:09.749 INFO:tasks.workunit.client.1.smithi144.stdout:9/394: dwrite d0/d8/d50/f69 [0,4194304] 0 2024-05-15T15:54:09.752 INFO:tasks.workunit.client.0.smithi049.stdout:4/444: mknod d18/d85/c91 0 2024-05-15T15:54:09.755 INFO:tasks.workunit.client.1.smithi144.stdout:4/419: dread d6/d49/f5c [0,4194304] 0 2024-05-15T15:54:09.756 INFO:tasks.workunit.client.1.smithi144.stdout:4/420: readlink d6/dd/d34/d3b/d30/l60 0 2024-05-15T15:54:09.757 INFO:tasks.workunit.client.1.smithi144.stdout:4/421: write d6/d42/d46/d4b/f31 [4779227,40021] 0 2024-05-15T15:54:09.758 INFO:tasks.workunit.client.0.smithi049.stdout:4/445: creat d18/d76/f92 x:0 0 0 2024-05-15T15:54:09.759 INFO:tasks.workunit.client.1.smithi144.stdout:8/439: getdents d2/d3a/d7b/d4b/d5b/d6e/d71/d73 0 2024-05-15T15:54:09.760 INFO:tasks.workunit.client.0.smithi049.stdout:4/446: chown d18/f4d 253980 1 2024-05-15T15:54:09.762 INFO:tasks.workunit.client.1.smithi144.stdout:5/392: symlink d0/dc/d2f/d34/l7d 0 2024-05-15T15:54:09.769 INFO:tasks.workunit.client.1.smithi144.stdout:2/388: rmdir d3/d7/d20 39 2024-05-15T15:54:09.770 INFO:tasks.workunit.client.1.smithi144.stdout:1/400: rmdir d7/d1d/d33/d29/d6b/d5e/d74/d80 0 2024-05-15T15:54:09.771 INFO:tasks.workunit.client.1.smithi144.stdout:4/422: symlink d6/dd/d2e/d75/l88 0 2024-05-15T15:54:09.772 INFO:tasks.workunit.client.1.smithi144.stdout:8/440: rmdir d2 39 2024-05-15T15:54:09.776 INFO:tasks.workunit.client.1.smithi144.stdout:2/389: creat d3/d7/d54/f85 x:0 0 0 2024-05-15T15:54:09.778 INFO:tasks.workunit.client.1.smithi144.stdout:6/364: rename d7/db/d11/d55/d70/d7b to d7/db/d35/d84 0 2024-05-15T15:54:09.779 INFO:tasks.workunit.client.1.smithi144.stdout:3/402: truncate d0/d11/f26 934242 0 2024-05-15T15:54:09.779 INFO:tasks.workunit.client.1.smithi144.stdout:3/403: dread - d0/d1/d56/f7f zero size 2024-05-15T15:54:09.780 INFO:tasks.workunit.client.1.smithi144.stdout:4/423: symlink d6/d42/d46/d4b/d1e/d44/l89 0 2024-05-15T15:54:09.781 INFO:tasks.workunit.client.1.smithi144.stdout:4/424: write d6/d42/d46/d4b/d1e/d44/f43 [913787,57905] 0 2024-05-15T15:54:09.783 INFO:tasks.workunit.client.1.smithi144.stdout:2/390: read - d3/d7/d20/d22/f24 zero size 2024-05-15T15:54:09.783 INFO:tasks.workunit.client.1.smithi144.stdout:1/401: mknod d7/d1d/d58/d81/c85 0 2024-05-15T15:54:09.785 INFO:tasks.workunit.client.1.smithi144.stdout:1/402: write d7/d1d/d33/f3f [4375616,20453] 0 2024-05-15T15:54:09.786 INFO:tasks.workunit.client.1.smithi144.stdout:6/365: chown d7/db/c27 57236 1 2024-05-15T15:54:09.786 INFO:tasks.workunit.client.1.smithi144.stdout:1/403: fdatasync d7/d1d/d33/d41/d22/f3a 0 2024-05-15T15:54:09.788 INFO:tasks.workunit.client.1.smithi144.stdout:7/418: truncate d4/df/f1f 151013 0 2024-05-15T15:54:09.789 INFO:tasks.workunit.client.1.smithi144.stdout:3/404: fdatasync d0/d1/d17/d39/d40/d7e/f81 0 2024-05-15T15:54:09.794 INFO:tasks.workunit.client.1.smithi144.stdout:5/393: dread d0/dc/d12/f13 [0,4194304] 0 2024-05-15T15:54:09.794 INFO:tasks.workunit.client.1.smithi144.stdout:6/366: symlink d7/d62/l85 0 2024-05-15T15:54:09.797 INFO:tasks.workunit.client.1.smithi144.stdout:4/425: dread d6/dd/d2d/f77 [4194304,4194304] 0 2024-05-15T15:54:09.798 INFO:tasks.workunit.client.1.smithi144.stdout:1/404: symlink d7/d1d/d33/d41/d78/l86 0 2024-05-15T15:54:09.801 INFO:tasks.workunit.client.1.smithi144.stdout:1/405: dread d7/d1d/d33/d1e/d34/f4a [0,4194304] 0 2024-05-15T15:54:09.806 INFO:tasks.workunit.client.1.smithi144.stdout:7/419: symlink d4/df/d25/d40/d73/l8a 0 2024-05-15T15:54:09.807 INFO:tasks.workunit.client.1.smithi144.stdout:1/406: dwrite d7/df/d24/f54 [0,4194304] 0 2024-05-15T15:54:09.809 INFO:tasks.workunit.client.1.smithi144.stdout:9/395: rename d0/d8/d50/c80 to d0/d8/d5e/d44/d10/d19/c90 0 2024-05-15T15:54:09.812 INFO:tasks.workunit.client.1.smithi144.stdout:5/394: mkdir d0/d7e 0 2024-05-15T15:54:09.814 INFO:tasks.workunit.client.1.smithi144.stdout:6/367: symlink d7/l86 0 2024-05-15T15:54:09.816 INFO:tasks.workunit.client.1.smithi144.stdout:3/405: creat d0/d1/d17/d39/f8c x:0 0 0 2024-05-15T15:54:09.816 INFO:tasks.workunit.client.1.smithi144.stdout:4/426: write d6/d42/d46/d4b/d1e/d44/f53 [288703,49236] 0 2024-05-15T15:54:09.817 INFO:tasks.workunit.client.1.smithi144.stdout:2/391: getdents d3/d7/d20 0 2024-05-15T15:54:09.818 INFO:tasks.workunit.client.1.smithi144.stdout:7/420: creat d4/df/d10/d48/f8b x:0 0 0 2024-05-15T15:54:09.822 INFO:tasks.workunit.client.1.smithi144.stdout:4/427: read - d6/dd/d34/d3b/d30/db/d1d/d40/f79 zero size 2024-05-15T15:54:09.822 INFO:tasks.workunit.client.1.smithi144.stdout:7/421: write d4/df/d25/d38/f7f [390418,39662] 0 2024-05-15T15:54:09.822 INFO:tasks.workunit.client.1.smithi144.stdout:4/428: dread - d6/d49/f69 zero size 2024-05-15T15:54:09.822 INFO:tasks.workunit.client.1.smithi144.stdout:8/441: rename d2/d3a/d7b/d4b/d5b/l86 to d2/d3a/d7b/d4b/d5b/d6e/l90 0 2024-05-15T15:54:09.822 INFO:tasks.workunit.client.1.smithi144.stdout:4/429: dread - d6/dd/d34/d3b/f5d zero size 2024-05-15T15:54:09.822 INFO:tasks.workunit.client.1.smithi144.stdout:5/395: rmdir d0/dc/d2f/d34/d63/d6e 39 2024-05-15T15:54:09.825 INFO:tasks.workunit.client.1.smithi144.stdout:2/392: mkdir d3/d6b/d86 0 2024-05-15T15:54:09.825 INFO:tasks.workunit.client.1.smithi144.stdout:3/406: read d0/d1/f23 [113457,107609] 0 2024-05-15T15:54:09.830 INFO:tasks.workunit.client.1.smithi144.stdout:1/407: rename d7/d1d/d33/d41/d22/d66 to d7/d1d/d33/d1e/d87 0 2024-05-15T15:54:09.832 INFO:tasks.workunit.client.1.smithi144.stdout:8/442: creat d2/d3a/f91 x:0 0 0 2024-05-15T15:54:09.833 INFO:tasks.workunit.client.1.smithi144.stdout:8/443: dread - d2/d3a/f91 zero size 2024-05-15T15:54:09.835 INFO:tasks.workunit.client.1.smithi144.stdout:5/396: creat d0/dc/d24/d57/f7f x:0 0 0 2024-05-15T15:54:09.836 INFO:tasks.workunit.client.1.smithi144.stdout:6/368: dwrite d7/db/d11/f42 [0,4194304] 0 2024-05-15T15:54:09.852 INFO:tasks.workunit.client.0.smithi049.stdout:7/335: sync 2024-05-15T15:54:09.852 INFO:tasks.workunit.client.0.smithi049.stdout:9/289: sync 2024-05-15T15:54:09.852 INFO:tasks.workunit.client.0.smithi049.stdout:1/509: sync 2024-05-15T15:54:09.852 INFO:tasks.workunit.client.0.smithi049.stdout:3/457: sync 2024-05-15T15:54:09.852 INFO:tasks.workunit.client.0.smithi049.stdout:2/422: sync 2024-05-15T15:54:09.852 INFO:tasks.workunit.client.0.smithi049.stdout:6/349: sync 2024-05-15T15:54:09.852 INFO:tasks.workunit.client.0.smithi049.stdout:5/410: sync 2024-05-15T15:54:09.854 INFO:tasks.workunit.client.0.smithi049.stdout:5/411: chown dc/dd/d10 570022573 1 2024-05-15T15:54:09.858 INFO:tasks.workunit.client.0.smithi049.stdout:7/336: dread d0/d2f/f48 [0,4194304] 0 2024-05-15T15:54:09.859 INFO:tasks.workunit.client.0.smithi049.stdout:3/458: dread d12/d14/d32/f6d [0,4194304] 0 2024-05-15T15:54:09.862 INFO:tasks.workunit.client.0.smithi049.stdout:5/412: mkdir dc/dd/d47/d83 0 2024-05-15T15:54:09.864 INFO:tasks.workunit.client.0.smithi049.stdout:6/350: dwrite fb [4194304,4194304] 0 2024-05-15T15:54:09.865 INFO:tasks.workunit.client.0.smithi049.stdout:2/423: rename d3/d52/c79 to d3/db/d1d/d1e/d2b/d3b/c83 0 2024-05-15T15:54:09.867 INFO:tasks.workunit.client.0.smithi049.stdout:1/510: dwrite d0/d4/dd/de/d36/f6a [0,4194304] 0 2024-05-15T15:54:09.868 INFO:tasks.workunit.client.0.smithi049.stdout:6/351: write dd/d17/d24/f27 [4946336,127525] 0 2024-05-15T15:54:09.873 INFO:tasks.workunit.client.0.smithi049.stdout:2/424: dread d3/db/d5b/f70 [0,4194304] 0 2024-05-15T15:54:09.875 INFO:tasks.workunit.client.0.smithi049.stdout:2/425: dread - d3/da/d15/d73/f7f zero size 2024-05-15T15:54:09.876 INFO:tasks.workunit.client.0.smithi049.stdout:3/459: symlink d12/d14/d32/d64/d88/l89 0 2024-05-15T15:54:09.876 INFO:tasks.workunit.client.1.smithi144.stdout:0/401: sync 2024-05-15T15:54:09.877 INFO:tasks.workunit.client.0.smithi049.stdout:7/337: dwrite d0/d6/d1d/f4d [0,4194304] 0 2024-05-15T15:54:09.879 INFO:tasks.workunit.client.1.smithi144.stdout:7/422: write d4/df/d25/f29 [739048,5412] 0 2024-05-15T15:54:09.879 INFO:tasks.workunit.client.0.smithi049.stdout:5/413: rename dc/dd/d47/l55 to dc/d4a/d31/l84 0 2024-05-15T15:54:09.880 INFO:tasks.workunit.client.1.smithi144.stdout:7/423: dread - d4/d41/d5b/f88 zero size 2024-05-15T15:54:09.883 INFO:tasks.workunit.client.1.smithi144.stdout:7/424: chown d4/df/d10/f3b 51061 1 2024-05-15T15:54:09.884 INFO:tasks.workunit.client.0.smithi049.stdout:3/460: dread d12/d14/f4f [0,4194304] 0 2024-05-15T15:54:09.887 INFO:tasks.workunit.client.1.smithi144.stdout:4/430: write f0 [1631623,100691] 0 2024-05-15T15:54:09.888 INFO:tasks.workunit.client.1.smithi144.stdout:1/408: link d7/d1d/d33/d41/f83 d7/df/d24/f88 0 2024-05-15T15:54:09.890 INFO:tasks.workunit.client.1.smithi144.stdout:5/397: dread d0/dc/d24/f37 [0,4194304] 0 2024-05-15T15:54:09.890 INFO:tasks.workunit.client.1.smithi144.stdout:4/431: readlink d6/dd/d34/d3b/d30/db/d1d/l28 0 2024-05-15T15:54:09.891 INFO:tasks.workunit.client.0.smithi049.stdout:5/414: mknod dc/d4a/d31/c85 0 2024-05-15T15:54:09.892 INFO:tasks.workunit.client.1.smithi144.stdout:6/369: mknod d7/db/d11/d55/d70/c87 0 2024-05-15T15:54:09.893 INFO:tasks.workunit.client.1.smithi144.stdout:6/370: dread - d7/db/d11/f60 zero size 2024-05-15T15:54:09.896 INFO:tasks.workunit.client.0.smithi049.stdout:6/352: creat dd/d17/d24/d2b/f5d x:0 0 0 2024-05-15T15:54:09.896 INFO:tasks.workunit.client.0.smithi049.stdout:6/353: stat dd/d26/f4f 0 2024-05-15T15:54:09.897 INFO:tasks.workunit.client.0.smithi049.stdout:6/354: rename dd/d17/d24 to dd/d17/d24/d2b/d36/d5e 22 2024-05-15T15:54:09.898 INFO:tasks.workunit.client.1.smithi144.stdout:8/444: fdatasync d2/de/d49/f76 0 2024-05-15T15:54:09.900 INFO:tasks.workunit.client.0.smithi049.stdout:3/461: creat d12/d30/d63/d68/d70/f8a x:0 0 0 2024-05-15T15:54:09.903 INFO:tasks.workunit.client.0.smithi049.stdout:5/415: symlink dc/dd/d7b/l86 0 2024-05-15T15:54:09.906 INFO:tasks.workunit.client.0.smithi049.stdout:3/462: dread - d12/d14/d32/f36 zero size 2024-05-15T15:54:09.908 INFO:tasks.workunit.client.1.smithi144.stdout:5/398: creat d0/dc/d2f/d68/d5c/f80 x:0 0 0 2024-05-15T15:54:09.909 INFO:tasks.workunit.client.0.smithi049.stdout:5/416: creat dc/d29/f87 x:0 0 0 2024-05-15T15:54:09.910 INFO:tasks.workunit.client.0.smithi049.stdout:6/355: mkdir dd/d17/d24/d58/d5f 0 2024-05-15T15:54:09.914 INFO:tasks.workunit.client.0.smithi049.stdout:3/463: rename d12/d14/f2a to d12/d26/d73/f8b 0 2024-05-15T15:54:09.915 INFO:tasks.workunit.client.0.smithi049.stdout:1/511: write d0/f5b [2163200,94066] 0 2024-05-15T15:54:09.918 INFO:tasks.workunit.client.0.smithi049.stdout:3/464: dread d12/f25 [0,4194304] 0 2024-05-15T15:54:09.924 INFO:tasks.workunit.client.1.smithi144.stdout:9/396: sync 2024-05-15T15:54:09.925 INFO:tasks.workunit.client.1.smithi144.stdout:8/445: symlink d2/l92 0 2024-05-15T15:54:09.925 INFO:tasks.workunit.client.0.smithi049.stdout:6/356: dread dd/f57 [0,4194304] 0 2024-05-15T15:54:09.927 INFO:tasks.workunit.client.0.smithi049.stdout:7/338: dwrite d0/f2e [0,4194304] 0 2024-05-15T15:54:09.928 INFO:tasks.workunit.client.0.smithi049.stdout:1/512: mkdir d0/d4/dd/de/d4d/d8b/dba 0 2024-05-15T15:54:09.930 INFO:tasks.workunit.client.0.smithi049.stdout:3/465: dwrite d12/d30/d63/d68/f77 [0,4194304] 0 2024-05-15T15:54:09.930 INFO:tasks.workunit.client.1.smithi144.stdout:2/393: getdents d3 0 2024-05-15T15:54:09.930 INFO:tasks.workunit.client.1.smithi144.stdout:2/394: chown d3/f1c 84816484 1 2024-05-15T15:54:09.931 INFO:tasks.workunit.client.1.smithi144.stdout:2/395: write d3/d7/d20/d3b/f4f [1285102,25371] 0 2024-05-15T15:54:09.932 INFO:tasks.workunit.client.0.smithi049.stdout:3/466: write d12/d30/d63/d68/d70/d75/f85 [784415,88192] 0 2024-05-15T15:54:09.932 INFO:tasks.workunit.client.1.smithi144.stdout:2/396: write d3/d7/f1a [1359742,80076] 0 2024-05-15T15:54:09.933 INFO:tasks.workunit.client.1.smithi144.stdout:1/409: mkdir d7/df/d89 0 2024-05-15T15:54:09.934 INFO:tasks.workunit.client.1.smithi144.stdout:1/410: read - d7/d1d/d33/d1e/d34/f76 zero size 2024-05-15T15:54:09.941 INFO:tasks.workunit.client.0.smithi049.stdout:7/339: symlink d0/d34/l65 0 2024-05-15T15:54:09.943 INFO:tasks.workunit.client.1.smithi144.stdout:5/399: dwrite d0/dc/d12/f43 [4194304,4194304] 0 2024-05-15T15:54:09.945 INFO:tasks.workunit.client.0.smithi049.stdout:5/417: getdents dc/d4a/d31/d5c 0 2024-05-15T15:54:09.947 INFO:tasks.workunit.client.1.smithi144.stdout:5/400: dread d0/f16 [0,4194304] 0 2024-05-15T15:54:09.950 INFO:tasks.workunit.client.1.smithi144.stdout:0/402: link d4/df/d1f/d24/f69 d4/d6/d3a/f91 0 2024-05-15T15:54:09.951 INFO:tasks.workunit.client.0.smithi049.stdout:7/340: truncate d0/d34/f46 642286 0 2024-05-15T15:54:09.952 INFO:tasks.workunit.client.0.smithi049.stdout:7/341: read d0/d6/d23/f30 [3118738,59838] 0 2024-05-15T15:54:09.953 INFO:tasks.workunit.client.0.smithi049.stdout:3/467: mknod d12/d30/d42/c8c 0 2024-05-15T15:54:09.954 INFO:tasks.workunit.client.0.smithi049.stdout:3/468: chown d12/f18 1859 1 2024-05-15T15:54:09.955 INFO:tasks.workunit.client.0.smithi049.stdout:5/418: truncate dc/d4a/f3a 562700 0 2024-05-15T15:54:09.960 INFO:tasks.workunit.client.1.smithi144.stdout:4/432: link d6/d42/c64 d6/d7e/c8a 0 2024-05-15T15:54:09.961 INFO:tasks.workunit.client.1.smithi144.stdout:6/371: dwrite d7/f4b [4194304,4194304] 0 2024-05-15T15:54:09.962 INFO:tasks.workunit.client.0.smithi049.stdout:7/342: readlink d0/d6/d23/d4a/l57 0 2024-05-15T15:54:09.962 INFO:tasks.workunit.client.1.smithi144.stdout:6/372: chown d7/d1e/f75 1094 1 2024-05-15T15:54:09.963 INFO:tasks.workunit.client.0.smithi049.stdout:3/469: unlink d12/f18 0 2024-05-15T15:54:09.966 INFO:tasks.workunit.client.0.smithi049.stdout:5/419: fdatasync dc/dd/d10/f2a 0 2024-05-15T15:54:09.966 INFO:tasks.workunit.client.1.smithi144.stdout:7/425: link d4/df/f1f d4/df/d25/d38/d5d/f8c 0 2024-05-15T15:54:09.968 INFO:tasks.workunit.client.0.smithi049.stdout:6/357: getdents dd/d17/d24/d2b/d36 0 2024-05-15T15:54:09.969 INFO:tasks.workunit.client.1.smithi144.stdout:9/397: mknod d0/d8/d5e/d44/d3c/c91 0 2024-05-15T15:54:09.973 INFO:tasks.workunit.client.1.smithi144.stdout:8/446: symlink d2/d3a/d7b/d4b/d5b/d6e/d8d/l93 0 2024-05-15T15:54:09.974 INFO:tasks.workunit.client.0.smithi049.stdout:7/343: symlink d0/d2f/d5c/l66 0 2024-05-15T15:54:09.977 INFO:tasks.workunit.client.1.smithi144.stdout:0/403: fdatasync d4/df/f43 0 2024-05-15T15:54:09.978 INFO:tasks.workunit.client.1.smithi144.stdout:0/404: chown d4/d2d/d48/d5b/d8e 391 1 2024-05-15T15:54:09.979 INFO:tasks.workunit.client.1.smithi144.stdout:2/397: mknod d3/d6b/d86/c87 0 2024-05-15T15:54:09.980 INFO:tasks.workunit.client.0.smithi049.stdout:5/420: unlink dc/dd/d47/f75 0 2024-05-15T15:54:09.981 INFO:tasks.workunit.client.0.smithi049.stdout:5/421: chown dc/dd/d10/c65 135 1 2024-05-15T15:54:09.982 INFO:tasks.workunit.client.1.smithi144.stdout:5/401: write d0/f1a [278815,40825] 0 2024-05-15T15:54:09.984 INFO:tasks.workunit.client.0.smithi049.stdout:7/344: symlink d0/d6/d1d/l67 0 2024-05-15T15:54:09.984 INFO:tasks.workunit.client.1.smithi144.stdout:0/405: dwrite d4/df/d1f/d24/f5e [0,4194304] 0 2024-05-15T15:54:09.986 INFO:tasks.workunit.client.0.smithi049.stdout:3/470: dread - d12/d14/d32/f36 zero size 2024-05-15T15:54:09.986 INFO:tasks.workunit.client.1.smithi144.stdout:4/433: creat d6/dd/d34/d3b/d30/db/d1d/d4e/f8b x:0 0 0 2024-05-15T15:54:09.988 INFO:tasks.workunit.client.1.smithi144.stdout:6/373: rmdir d7/db 39 2024-05-15T15:54:09.989 INFO:tasks.workunit.client.1.smithi144.stdout:9/398: mknod d0/d8/d1b/d62/c92 0 2024-05-15T15:54:09.991 INFO:tasks.workunit.client.1.smithi144.stdout:8/447: rmdir d2/d24 39 2024-05-15T15:54:09.993 INFO:tasks.workunit.client.0.smithi049.stdout:3/471: write d12/f25 [1963144,105617] 0 2024-05-15T15:54:09.994 INFO:tasks.workunit.client.1.smithi144.stdout:5/402: dwrite d0/dc/d12/f43 [8388608,4194304] 0 2024-05-15T15:54:09.994 INFO:tasks.workunit.client.1.smithi144.stdout:0/406: dread d4/df/d1f/d24/f4b [0,4194304] 0 2024-05-15T15:54:09.994 INFO:tasks.workunit.client.0.smithi049.stdout:3/472: chown d12/d30/d42/c8c 3 1 2024-05-15T15:54:09.995 INFO:tasks.workunit.client.1.smithi144.stdout:2/398: mknod d3/d7/d20/d40/c88 0 2024-05-15T15:54:09.996 INFO:tasks.workunit.client.1.smithi144.stdout:5/403: dread - d0/dc/d1f/f75 zero size 2024-05-15T15:54:09.997 INFO:tasks.workunit.client.0.smithi049.stdout:6/358: creat dd/f60 x:0 0 0 2024-05-15T15:54:09.998 INFO:tasks.workunit.client.1.smithi144.stdout:2/399: dread - d3/d7/d20/d22/d52/d75/d65/f73 zero size 2024-05-15T15:54:09.999 INFO:tasks.workunit.client.1.smithi144.stdout:1/411: rmdir d7/d1d/d33/d1e/d4e/d72 0 2024-05-15T15:54:09.999 INFO:tasks.workunit.client.1.smithi144.stdout:2/400: read d3/d7/d54/f5a [495570,78492] 0 2024-05-15T15:54:10.002 INFO:tasks.workunit.client.1.smithi144.stdout:2/401: readlink d3/d7/d20/d22/d52/d75/d65/l81 0 2024-05-15T15:54:10.003 INFO:tasks.workunit.client.1.smithi144.stdout:2/402: chown d3/d7/f27 0 1 2024-05-15T15:54:10.003 INFO:tasks.workunit.client.1.smithi144.stdout:5/404: dread d0/dc/f1b [4194304,4194304] 0 2024-05-15T15:54:10.003 INFO:tasks.workunit.client.0.smithi049.stdout:3/473: fsync f8 0 2024-05-15T15:54:10.005 INFO:tasks.workunit.client.1.smithi144.stdout:6/374: unlink d7/f6e 0 2024-05-15T15:54:10.006 INFO:tasks.workunit.client.0.smithi049.stdout:5/422: creat dc/dd/f88 x:0 0 0 2024-05-15T15:54:10.008 INFO:tasks.workunit.client.1.smithi144.stdout:4/434: dread d6/dd/d34/d3b/f21 [0,4194304] 0 2024-05-15T15:54:10.008 INFO:tasks.workunit.client.1.smithi144.stdout:9/399: creat d0/d8/d1b/d79/f93 x:0 0 0 2024-05-15T15:54:10.009 INFO:tasks.workunit.client.1.smithi144.stdout:1/412: dread d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:10.011 INFO:tasks.workunit.client.1.smithi144.stdout:8/448: creat d2/de/d36/d58/f94 x:0 0 0 2024-05-15T15:54:10.014 INFO:tasks.workunit.client.1.smithi144.stdout:4/435: dwrite f0 [0,4194304] 0 2024-05-15T15:54:10.014 INFO:tasks.workunit.client.0.smithi049.stdout:7/345: dwrite d0/d6/da/d3d/f33 [0,4194304] 0 2024-05-15T15:54:10.015 INFO:tasks.workunit.client.0.smithi049.stdout:5/423: symlink dc/d4a/l89 0 2024-05-15T15:54:10.015 INFO:tasks.workunit.client.0.smithi049.stdout:7/346: stat d0/d6/d23/d45/f5a 0 2024-05-15T15:54:10.017 INFO:tasks.workunit.client.0.smithi049.stdout:6/359: mknod dd/d17/d24/d2b/c61 0 2024-05-15T15:54:10.020 INFO:tasks.workunit.client.1.smithi144.stdout:4/436: dread d6/d42/d46/d4b/f31 [4194304,4194304] 0 2024-05-15T15:54:10.026 INFO:tasks.workunit.client.1.smithi144.stdout:9/400: dread d0/f7 [0,4194304] 0 2024-05-15T15:54:10.026 INFO:tasks.workunit.client.1.smithi144.stdout:9/401: stat d0/d8/d5e/d44/d10/f2c 0 2024-05-15T15:54:10.026 INFO:tasks.workunit.client.1.smithi144.stdout:2/403: rename d3/d6b to d3/d7/d20/d89 0 2024-05-15T15:54:10.027 INFO:tasks.workunit.client.0.smithi049.stdout:5/424: dwrite dc/dd/f17 [0,4194304] 0 2024-05-15T15:54:10.027 INFO:tasks.workunit.client.0.smithi049.stdout:5/425: fdatasync dc/dd/f88 0 2024-05-15T15:54:10.030 INFO:tasks.workunit.client.0.smithi049.stdout:7/347: symlink d0/d6/d23/l68 0 2024-05-15T15:54:10.031 INFO:tasks.workunit.client.1.smithi144.stdout:9/402: dwrite d0/d8/d5e/d44/d3c/d75/f86 [0,4194304] 0 2024-05-15T15:54:10.032 INFO:tasks.workunit.client.1.smithi144.stdout:9/403: write d0/d8/d21/f82 [310784,117452] 0 2024-05-15T15:54:10.033 INFO:tasks.workunit.client.0.smithi049.stdout:7/348: fdatasync d0/d6/d23/d45/f5a 0 2024-05-15T15:54:10.033 INFO:tasks.workunit.client.0.smithi049.stdout:3/474: dwrite d12/f3f [0,4194304] 0 2024-05-15T15:54:10.033 INFO:tasks.workunit.client.1.smithi144.stdout:9/404: stat d0/d8/d1b/d62 0 2024-05-15T15:54:10.035 INFO:tasks.workunit.client.0.smithi049.stdout:5/426: mknod dc/dd/d7b/c8a 0 2024-05-15T15:54:10.039 INFO:tasks.workunit.client.1.smithi144.stdout:8/449: creat d2/de/d36/f95 x:0 0 0 2024-05-15T15:54:10.041 INFO:tasks.workunit.client.1.smithi144.stdout:8/450: write d2/de/d36/d58/f94 [537567,73407] 0 2024-05-15T15:54:10.042 INFO:tasks.workunit.client.0.smithi049.stdout:7/349: creat d0/d34/f69 x:0 0 0 2024-05-15T15:54:10.049 INFO:tasks.workunit.client.0.smithi049.stdout:6/360: write dd/d17/d24/d2b/f4d [123205,48679] 0 2024-05-15T15:54:10.050 INFO:tasks.workunit.client.0.smithi049.stdout:6/361: dread - dd/d26/f4b zero size 2024-05-15T15:54:10.051 INFO:tasks.workunit.client.1.smithi144.stdout:6/375: creat d7/db/d11/d3c/d56/f88 x:0 0 0 2024-05-15T15:54:10.052 INFO:tasks.workunit.client.0.smithi049.stdout:6/362: stat dd/d17/d24/d2b/c4e 0 2024-05-15T15:54:10.053 INFO:tasks.workunit.client.0.smithi049.stdout:3/475: dwrite d12/d30/d42/d7a/f83 [0,4194304] 0 2024-05-15T15:54:10.053 INFO:tasks.workunit.client.1.smithi144.stdout:3/407: sync 2024-05-15T15:54:10.054 INFO:tasks.workunit.client.1.smithi144.stdout:3/408: dread - d0/d11/d24/f57 zero size 2024-05-15T15:54:10.054 INFO:tasks.workunit.client.0.smithi049.stdout:6/363: dread f4 [0,4194304] 0 2024-05-15T15:54:10.055 INFO:tasks.workunit.client.1.smithi144.stdout:1/413: symlink d7/d1d/d33/d29/l8a 0 2024-05-15T15:54:10.055 INFO:tasks.workunit.client.1.smithi144.stdout:0/407: link d4/d6/d3a/d40/c4e d4/df/d1f/d56/c92 0 2024-05-15T15:54:10.055 INFO:tasks.workunit.client.1.smithi144.stdout:3/409: read d0/f2e [4344151,105658] 0 2024-05-15T15:54:10.059 INFO:tasks.workunit.client.1.smithi144.stdout:4/437: truncate d6/dd/d2d/f48 2379025 0 2024-05-15T15:54:10.060 INFO:tasks.workunit.client.1.smithi144.stdout:9/405: mkdir d0/d8/d1b/d62/d7d/d8d/d94 0 2024-05-15T15:54:10.061 INFO:tasks.workunit.client.1.smithi144.stdout:1/414: creat d7/d1d/f8b x:0 0 0 2024-05-15T15:54:10.062 INFO:tasks.workunit.client.1.smithi144.stdout:9/406: write d0/d8/d1b/d62/f8e [714258,115251] 0 2024-05-15T15:54:10.063 INFO:tasks.workunit.client.1.smithi144.stdout:6/376: symlink d7/d1e/l89 0 2024-05-15T15:54:10.064 INFO:tasks.workunit.client.1.smithi144.stdout:1/415: dread - d7/d1d/d33/d1e/d34/f76 zero size 2024-05-15T15:54:10.065 INFO:tasks.workunit.client.1.smithi144.stdout:8/451: rename d2/de/f26 to d2/f96 0 2024-05-15T15:54:10.066 INFO:tasks.workunit.client.1.smithi144.stdout:3/410: mknod d0/d1/d17/d39/d40/d4a/d77/c8d 0 2024-05-15T15:54:10.066 INFO:tasks.workunit.client.1.smithi144.stdout:4/438: creat d6/d7c/f8c x:0 0 0 2024-05-15T15:54:10.067 INFO:tasks.workunit.client.1.smithi144.stdout:3/411: stat d0/d1/d56/d71/d7a 0 2024-05-15T15:54:10.069 INFO:tasks.workunit.client.1.smithi144.stdout:8/452: creat d2/d3a/d7b/f97 x:0 0 0 2024-05-15T15:54:10.069 INFO:tasks.workunit.client.1.smithi144.stdout:0/408: getdents d4/d6/d3a 0 2024-05-15T15:54:10.070 INFO:tasks.workunit.client.1.smithi144.stdout:2/404: readlink d3/d7/d20/d22/d52/d75/l7d 0 2024-05-15T15:54:10.072 INFO:tasks.workunit.client.1.smithi144.stdout:4/439: creat d6/dd/d34/d3b/d30/d59/d66/f8d x:0 0 0 2024-05-15T15:54:10.073 INFO:tasks.workunit.client.1.smithi144.stdout:3/412: creat d0/d1/d17/d39/d40/d4a/f8e x:0 0 0 2024-05-15T15:54:10.074 INFO:tasks.workunit.client.1.smithi144.stdout:9/407: rename d0/d8/d21/f38 to d0/d8/d5e/d44/f95 0 2024-05-15T15:54:10.075 INFO:tasks.workunit.client.1.smithi144.stdout:8/453: mkdir d2/d3a/d7b/d4b/d5b/d6e/d71/d73/d98 0 2024-05-15T15:54:10.078 INFO:tasks.workunit.client.1.smithi144.stdout:0/409: dwrite d4/d6/f32 [0,4194304] 0 2024-05-15T15:54:10.079 INFO:tasks.workunit.client.0.smithi049.stdout:6/364: stat dd/d17/d24 0 2024-05-15T15:54:10.080 INFO:tasks.workunit.client.0.smithi049.stdout:6/365: chown dd/c43 712 1 2024-05-15T15:54:10.080 INFO:tasks.workunit.client.1.smithi144.stdout:9/408: truncate d0/d8/d5e/d44/d16/f85 817230 0 2024-05-15T15:54:10.084 INFO:tasks.workunit.client.1.smithi144.stdout:4/440: dwrite d6/dd/d2d/f77 [0,4194304] 0 2024-05-15T15:54:10.086 INFO:tasks.workunit.client.1.smithi144.stdout:1/416: dread d7/d1d/d33/d1e/d4e/f50 [0,4194304] 0 2024-05-15T15:54:10.088 INFO:tasks.workunit.client.1.smithi144.stdout:8/454: write d2/f96 [421136,68376] 0 2024-05-15T15:54:10.088 INFO:tasks.workunit.client.0.smithi049.stdout:3/476: write d12/d14/d52/f56 [368152,63451] 0 2024-05-15T15:54:10.090 INFO:tasks.workunit.client.1.smithi144.stdout:3/413: mkdir d0/d1/d56/d71/d7a/d72/d8f 0 2024-05-15T15:54:10.092 INFO:tasks.workunit.client.0.smithi049.stdout:6/366: dwrite dd/d26/f3f [0,4194304] 0 2024-05-15T15:54:10.093 INFO:tasks.workunit.client.1.smithi144.stdout:0/410: creat d4/df/d1f/d56/f93 x:0 0 0 2024-05-15T15:54:10.099 INFO:tasks.workunit.client.1.smithi144.stdout:7/426: sync 2024-05-15T15:54:10.100 INFO:tasks.workunit.client.0.smithi049.stdout:6/367: symlink dd/d26/l62 0 2024-05-15T15:54:10.101 INFO:tasks.workunit.client.1.smithi144.stdout:7/427: truncate d4/f1c 111480 0 2024-05-15T15:54:10.101 INFO:tasks.workunit.client.1.smithi144.stdout:7/428: chown d4/df/d25/d40 507012 1 2024-05-15T15:54:10.102 INFO:tasks.workunit.client.1.smithi144.stdout:9/409: dread d0/d8/d5e/d44/f26 [0,4194304] 0 2024-05-15T15:54:10.103 INFO:tasks.workunit.client.0.smithi049.stdout:3/477: symlink d12/d30/d42/l8d 0 2024-05-15T15:54:10.104 INFO:tasks.workunit.client.1.smithi144.stdout:9/410: write d0/d8/d1b/d62/d6b/f7c [1043256,22479] 0 2024-05-15T15:54:10.104 INFO:tasks.workunit.client.0.smithi049.stdout:5/427: dwrite dc/d29/f32 [0,4194304] 0 2024-05-15T15:54:10.105 INFO:tasks.workunit.client.0.smithi049.stdout:6/368: dread dd/d17/d24/d2b/d36/d50/f52 [0,4194304] 0 2024-05-15T15:54:10.107 INFO:tasks.workunit.client.1.smithi144.stdout:8/455: creat d2/de/d49/d83/f99 x:0 0 0 2024-05-15T15:54:10.112 INFO:tasks.workunit.client.0.smithi049.stdout:3/478: rmdir d12/d26/d73 39 2024-05-15T15:54:10.116 INFO:tasks.workunit.client.1.smithi144.stdout:3/414: dread d0/d11/f2a [0,4194304] 0 2024-05-15T15:54:10.117 INFO:tasks.workunit.client.1.smithi144.stdout:3/415: write d0/d1/d17/d39/d40/d4a/f8e [342888,115351] 0 2024-05-15T15:54:10.118 INFO:tasks.workunit.client.1.smithi144.stdout:3/416: stat d0/d11/d3b/l5c 0 2024-05-15T15:54:10.119 INFO:tasks.workunit.client.1.smithi144.stdout:4/441: creat d6/dd/d2e/d71/f8e x:0 0 0 2024-05-15T15:54:10.122 INFO:tasks.workunit.client.1.smithi144.stdout:4/442: dread d6/d42/d46/d4b/d1e/d44/f53 [0,4194304] 0 2024-05-15T15:54:10.122 INFO:tasks.workunit.client.0.smithi049.stdout:3/479: truncate d12/d26/d73/f7f 1031560 0 2024-05-15T15:54:10.122 INFO:tasks.workunit.client.1.smithi144.stdout:7/429: creat d4/df/d25/d38/d68/f8d x:0 0 0 2024-05-15T15:54:10.124 INFO:tasks.workunit.client.0.smithi049.stdout:5/428: chown dc/d4a/f57 5794 1 2024-05-15T15:54:10.124 INFO:tasks.workunit.client.1.smithi144.stdout:7/430: chown d4/df/d25/d38/d5d 6 1 2024-05-15T15:54:10.124 INFO:tasks.workunit.client.1.smithi144.stdout:6/377: truncate d7/db/d35/f47 3585145 0 2024-05-15T15:54:10.128 INFO:tasks.workunit.client.1.smithi144.stdout:6/378: fsync d7/d1e/d28/f3d 0 2024-05-15T15:54:10.129 INFO:tasks.workunit.client.0.smithi049.stdout:5/429: dwrite dc/dd/f1d [0,4194304] 0 2024-05-15T15:54:10.129 INFO:tasks.workunit.client.1.smithi144.stdout:3/417: symlink d0/d1/d6c/l90 0 2024-05-15T15:54:10.130 INFO:tasks.workunit.client.1.smithi144.stdout:3/418: write d0/d1/d56/f58 [1491714,127206] 0 2024-05-15T15:54:10.132 INFO:tasks.workunit.client.1.smithi144.stdout:4/443: symlink d6/dd/d34/d3b/d30/db/l8f 0 2024-05-15T15:54:10.136 INFO:tasks.workunit.client.1.smithi144.stdout:2/405: dwrite d3/d7/d20/d3f/f53 [0,4194304] 0 2024-05-15T15:54:10.138 INFO:tasks.workunit.client.1.smithi144.stdout:5/405: sync 2024-05-15T15:54:10.141 INFO:tasks.workunit.client.0.smithi049.stdout:6/369: write dd/d17/d24/d2b/d36/f42 [1571866,65546] 0 2024-05-15T15:54:10.142 INFO:tasks.workunit.client.0.smithi049.stdout:5/430: getdents dc/dd 0 2024-05-15T15:54:10.144 INFO:tasks.workunit.client.0.smithi049.stdout:5/431: write dc/dd/f88 [258414,111000] 0 2024-05-15T15:54:10.144 INFO:tasks.workunit.client.0.smithi049.stdout:6/370: read dd/f16 [2863562,63235] 0 2024-05-15T15:54:10.146 INFO:tasks.workunit.client.0.smithi049.stdout:6/371: read dd/d17/d24/d2b/f4d [468150,363] 0 2024-05-15T15:54:10.147 INFO:tasks.workunit.client.0.smithi049.stdout:6/372: stat dd/f60 0 2024-05-15T15:54:10.147 INFO:tasks.workunit.client.0.smithi049.stdout:6/373: stat fb 0 2024-05-15T15:54:10.150 INFO:tasks.workunit.client.0.smithi049.stdout:5/432: rmdir dc/dd/d10 39 2024-05-15T15:54:10.152 INFO:tasks.workunit.client.1.smithi144.stdout:8/456: mknod d2/de/d36/d87/c9a 0 2024-05-15T15:54:10.155 INFO:tasks.workunit.client.1.smithi144.stdout:0/411: link d4/dc/l23 d4/df/d1f/d24/d2b/d55/l94 0 2024-05-15T15:54:10.156 INFO:tasks.workunit.client.1.smithi144.stdout:0/412: read d4/df/d1f/d24/f5e [423158,78019] 0 2024-05-15T15:54:10.157 INFO:tasks.workunit.client.1.smithi144.stdout:6/379: creat d7/d1e/d28/f8a x:0 0 0 2024-05-15T15:54:10.158 INFO:tasks.workunit.client.1.smithi144.stdout:0/413: read d4/df/d14/f5f [3607417,96504] 0 2024-05-15T15:54:10.158 INFO:tasks.workunit.client.1.smithi144.stdout:6/380: write d7/d1e/d28/f8a [853545,67701] 0 2024-05-15T15:54:10.160 INFO:tasks.workunit.client.1.smithi144.stdout:3/419: creat d0/d1/d17/f91 x:0 0 0 2024-05-15T15:54:10.161 INFO:tasks.workunit.client.1.smithi144.stdout:4/444: creat d6/d42/d46/f90 x:0 0 0 2024-05-15T15:54:10.162 INFO:tasks.workunit.client.1.smithi144.stdout:1/417: getdents d7/d1d/d33/d41/d22/d45 0 2024-05-15T15:54:10.162 INFO:tasks.workunit.client.1.smithi144.stdout:4/445: write d6/dd/d2d/f54 [564353,92049] 0 2024-05-15T15:54:10.163 INFO:tasks.workunit.client.0.smithi049.stdout:6/374: creat dd/d17/d24/d2b/f63 x:0 0 0 2024-05-15T15:54:10.164 INFO:tasks.workunit.client.0.smithi049.stdout:6/375: readlink dd/d17/d24/d2b/d36/l59 0 2024-05-15T15:54:10.165 INFO:tasks.workunit.client.0.smithi049.stdout:5/433: creat dc/d4a/f8b x:0 0 0 2024-05-15T15:54:10.166 INFO:tasks.workunit.client.1.smithi144.stdout:7/431: link d4/f13 d4/df/d25/d38/d68/f8e 0 2024-05-15T15:54:10.167 INFO:tasks.workunit.client.1.smithi144.stdout:1/418: dwrite d7/f4d [0,4194304] 0 2024-05-15T15:54:10.167 INFO:tasks.workunit.client.1.smithi144.stdout:2/406: symlink d3/d7/d54/l8a 0 2024-05-15T15:54:10.168 INFO:tasks.workunit.client.0.smithi049.stdout:6/376: mkdir dd/d17/d64 0 2024-05-15T15:54:10.169 INFO:tasks.workunit.client.1.smithi144.stdout:1/419: write d7/df/d24/f54 [2590118,115927] 0 2024-05-15T15:54:10.169 INFO:tasks.workunit.client.1.smithi144.stdout:2/407: dread - d3/d7/d20/d22/f7a zero size 2024-05-15T15:54:10.171 INFO:tasks.workunit.client.1.smithi144.stdout:5/406: rename d0/dc/d24/d28/d2d/f69 to d0/dc/d2f/d68/d5c/f81 0 2024-05-15T15:54:10.172 INFO:tasks.workunit.client.1.smithi144.stdout:9/411: creat d0/f96 x:0 0 0 2024-05-15T15:54:10.174 INFO:tasks.workunit.client.0.smithi049.stdout:6/377: symlink dd/d17/d24/d58/d5f/l65 0 2024-05-15T15:54:10.179 INFO:tasks.workunit.client.1.smithi144.stdout:6/381: unlink d7/db/d11/d3c/c54 0 2024-05-15T15:54:10.179 INFO:tasks.workunit.client.1.smithi144.stdout:6/382: stat d7/ff 0 2024-05-15T15:54:10.180 INFO:tasks.workunit.client.1.smithi144.stdout:6/383: fsync d7/db/f74 0 2024-05-15T15:54:10.187 INFO:tasks.workunit.client.1.smithi144.stdout:4/446: symlink d6/dd/d34/d3b/d30/db/d1d/d40/d57/l91 0 2024-05-15T15:54:10.188 INFO:tasks.workunit.client.0.smithi049.stdout:6/378: link dd/l1b dd/d17/d24/d2b/d36/d50/d54/l66 0 2024-05-15T15:54:10.192 INFO:tasks.workunit.client.1.smithi144.stdout:3/420: write d0/d1/f4 [4620807,95514] 0 2024-05-15T15:54:10.192 INFO:tasks.workunit.client.0.smithi049.stdout:8/403: sync 2024-05-15T15:54:10.192 INFO:tasks.workunit.client.0.smithi049.stdout:0/396: sync 2024-05-15T15:54:10.193 INFO:tasks.workunit.client.0.smithi049.stdout:6/379: getdents dd/d26/d5b 0 2024-05-15T15:54:10.201 INFO:tasks.workunit.client.0.smithi049.stdout:8/404: dread d2/de/d25/f32 [0,4194304] 0 2024-05-15T15:54:10.201 INFO:tasks.workunit.client.1.smithi144.stdout:5/407: dwrite d0/dc/d2f/d34/d63/d6e/f7c [0,4194304] 0 2024-05-15T15:54:10.203 INFO:tasks.workunit.client.0.smithi049.stdout:0/397: creat dc/d13/d14/f79 x:0 0 0 2024-05-15T15:54:10.204 INFO:tasks.workunit.client.1.smithi144.stdout:6/384: symlink d7/db/d35/l8b 0 2024-05-15T15:54:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:10 smithi049 ceph-mon[28399]: pgmap v25: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 36 MiB/s rd, 174 MiB/s wr, 297 op/s 2024-05-15T15:54:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:10 smithi049 ceph-mon[28399]: Upgrade: Updating mgr.smithi049.vuhzgf 2024-05-15T15:54:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:10 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:10 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi049.vuhzgf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:54:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:10 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi049.vuhzgf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:54:10.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:10 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:54:10.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:10 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:54:10.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:10 smithi049 ceph-mon[28399]: Deploying daemon mgr.smithi049.vuhzgf on smithi049 2024-05-15T15:54:10.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:10 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:54:10.207 INFO:tasks.workunit.client.0.smithi049.stdout:8/405: unlink d2/d14/f1e 0 2024-05-15T15:54:10.207 INFO:tasks.workunit.client.1.smithi144.stdout:5/408: dread d0/dc/d2f/d34/d63/d6e/f7c [0,4194304] 0 2024-05-15T15:54:10.207 INFO:tasks.workunit.client.1.smithi144.stdout:9/412: dread d0/f18 [0,4194304] 0 2024-05-15T15:54:10.207 INFO:tasks.workunit.client.1.smithi144.stdout:5/409: chown d0/f11 7002058 1 2024-05-15T15:54:10.210 INFO:tasks.workunit.client.0.smithi049.stdout:6/380: dwrite dd/f32 [0,4194304] 0 2024-05-15T15:54:10.214 INFO:tasks.workunit.client.0.smithi049.stdout:0/398: dwrite dc/d32/f60 [0,4194304] 0 2024-05-15T15:54:10.214 INFO:tasks.workunit.client.1.smithi144.stdout:3/421: creat d0/d1/d17/d39/d40/f92 x:0 0 0 2024-05-15T15:54:10.217 INFO:tasks.workunit.client.0.smithi049.stdout:8/406: unlink d2/d14/d45/f70 0 2024-05-15T15:54:10.220 INFO:tasks.workunit.client.1.smithi144.stdout:1/420: truncate d7/d1d/d33/d55/f5b 8120932 0 2024-05-15T15:54:10.224 INFO:tasks.workunit.client.1.smithi144.stdout:7/432: write d4/df/f11 [1886200,103345] 0 2024-05-15T15:54:10.229 INFO:tasks.workunit.client.1.smithi144.stdout:5/410: creat d0/dc/d24/d28/f82 x:0 0 0 2024-05-15T15:54:10.229 INFO:tasks.workunit.client.1.smithi144.stdout:5/411: readlink d0/la 0 2024-05-15T15:54:10.230 INFO:tasks.workunit.client.1.smithi144.stdout:5/412: stat d0/dc/d24/d28/d2d/f2b 0 2024-05-15T15:54:10.235 INFO:tasks.workunit.client.1.smithi144.stdout:8/457: write d2/d24/f2b [143576,56064] 0 2024-05-15T15:54:10.235 INFO:tasks.workunit.client.0.smithi049.stdout:0/399: rmdir dc/d13/d14/d33/d74 0 2024-05-15T15:54:10.236 INFO:tasks.workunit.client.1.smithi144.stdout:8/458: write d2/d3a/d7b/d4b/d5b/d6e/f62 [3791809,4921] 0 2024-05-15T15:54:10.238 INFO:tasks.workunit.client.1.smithi144.stdout:8/459: write d2/d3a/d7b/f97 [717029,8769] 0 2024-05-15T15:54:10.239 INFO:tasks.workunit.client.1.smithi144.stdout:8/460: fdatasync d2/de/d36/f1b 0 2024-05-15T15:54:10.240 INFO:tasks.workunit.client.1.smithi144.stdout:2/408: symlink d3/d7/d20/d37/d76/l8b 0 2024-05-15T15:54:10.241 INFO:tasks.workunit.client.0.smithi049.stdout:6/381: getdents dd/d17/d24/d2b 0 2024-05-15T15:54:10.242 INFO:tasks.workunit.client.1.smithi144.stdout:4/447: truncate d6/f29 2229614 0 2024-05-15T15:54:10.243 INFO:tasks.workunit.client.1.smithi144.stdout:4/448: chown d6 64869 1 2024-05-15T15:54:10.243 INFO:tasks.workunit.client.1.smithi144.stdout:4/449: stat d6/d7c/f8c 0 2024-05-15T15:54:10.243 INFO:tasks.workunit.client.0.smithi049.stdout:8/407: getdents d2/de/d27/d5b/d60 0 2024-05-15T15:54:10.245 INFO:tasks.workunit.client.0.smithi049.stdout:8/408: dread d2/d14/f4c [0,4194304] 0 2024-05-15T15:54:10.245 INFO:tasks.workunit.client.1.smithi144.stdout:7/433: rename d4/df/d25/d38/d68/f8e to d4/df/d25/d38/d68/f8f 0 2024-05-15T15:54:10.246 INFO:tasks.workunit.client.1.smithi144.stdout:7/434: truncate d4/f2f 2342026 0 2024-05-15T15:54:10.247 INFO:tasks.workunit.client.1.smithi144.stdout:3/422: dwrite d0/d1/f23 [0,4194304] 0 2024-05-15T15:54:10.247 INFO:tasks.workunit.client.1.smithi144.stdout:6/385: link d7/db/d11/d55/f5e d7/db/d11/d7e/f8c 0 2024-05-15T15:54:10.248 INFO:tasks.workunit.client.1.smithi144.stdout:9/413: creat d0/d8/d1b/d79/d8a/f97 x:0 0 0 2024-05-15T15:54:10.249 INFO:tasks.workunit.client.0.smithi049.stdout:6/382: mkdir dd/d17/d24/d67 0 2024-05-15T15:54:10.250 INFO:tasks.workunit.client.0.smithi049.stdout:6/383: dread - dd/f60 zero size 2024-05-15T15:54:10.251 INFO:tasks.workunit.client.1.smithi144.stdout:5/413: rename d0/l17 to d0/dc/d24/l83 0 2024-05-15T15:54:10.252 INFO:tasks.workunit.client.1.smithi144.stdout:5/414: truncate d0/dc/d12/d31/d35/f62 854759 0 2024-05-15T15:54:10.255 INFO:tasks.workunit.client.0.smithi049.stdout:0/400: rename dc/le to dc/l7a 0 2024-05-15T15:54:10.259 INFO:tasks.workunit.client.0.smithi049.stdout:0/401: read dc/d13/d14/f1c [993292,17969] 0 2024-05-15T15:54:10.260 INFO:tasks.workunit.client.0.smithi049.stdout:6/384: dwrite dd/d17/f38 [0,4194304] 0 2024-05-15T15:54:10.262 INFO:tasks.workunit.client.0.smithi049.stdout:6/385: write dd/d26/f4b [812073,79857] 0 2024-05-15T15:54:10.262 INFO:tasks.workunit.client.1.smithi144.stdout:7/435: fdatasync d4/df/f1f 0 2024-05-15T15:54:10.264 INFO:tasks.workunit.client.1.smithi144.stdout:7/436: chown d4/df/d25/d38/d5d/f8c 1192288 1 2024-05-15T15:54:10.270 INFO:tasks.workunit.client.0.smithi049.stdout:8/409: dread d2/de/f55 [0,4194304] 0 2024-05-15T15:54:10.280 INFO:tasks.workunit.client.0.smithi049.stdout:6/386: mknod dd/d17/c68 0 2024-05-15T15:54:10.281 INFO:tasks.workunit.client.0.smithi049.stdout:0/402: dread dc/d13/d14/f46 [0,4194304] 0 2024-05-15T15:54:10.283 INFO:tasks.workunit.client.0.smithi049.stdout:0/403: write dc/d13/d14/d33/f5b [651814,67822] 0 2024-05-15T15:54:10.290 INFO:tasks.workunit.client.0.smithi049.stdout:0/404: dwrite dc/f1d [0,4194304] 0 2024-05-15T15:54:10.290 INFO:tasks.workunit.client.0.smithi049.stdout:0/405: chown dc/d13/d14 2607 1 2024-05-15T15:54:10.292 INFO:tasks.workunit.client.0.smithi049.stdout:6/387: symlink dd/l69 0 2024-05-15T15:54:10.294 INFO:tasks.workunit.client.0.smithi049.stdout:0/406: rmdir dc/d32 39 2024-05-15T15:54:10.299 INFO:tasks.workunit.client.0.smithi049.stdout:6/388: dread dd/d17/d24/d2b/d36/d50/f52 [0,4194304] 0 2024-05-15T15:54:10.302 INFO:tasks.workunit.client.0.smithi049.stdout:8/410: write d2/d14/f5c [576956,129460] 0 2024-05-15T15:54:10.303 INFO:tasks.workunit.client.1.smithi144.stdout:9/414: creat d0/d8/d5e/d76/f98 x:0 0 0 2024-05-15T15:54:10.303 INFO:tasks.workunit.client.1.smithi144.stdout:4/450: rename d6/d42/c64 to d6/dd/d34/d3b/d30/db/d1d/d40/d6c/c92 0 2024-05-15T15:54:10.303 INFO:tasks.workunit.client.1.smithi144.stdout:5/415: symlink d0/dc/d2f/d34/d63/d6e/l84 0 2024-05-15T15:54:10.304 INFO:tasks.workunit.client.1.smithi144.stdout:1/421: creat d7/d1d/d33/d41/f8c x:0 0 0 2024-05-15T15:54:10.305 INFO:tasks.workunit.client.1.smithi144.stdout:5/416: write d0/dc/d1f/f3b [1618797,78741] 0 2024-05-15T15:54:10.307 INFO:tasks.workunit.client.1.smithi144.stdout:6/386: write d7/d62/f6b [3345643,9893] 0 2024-05-15T15:54:10.308 INFO:tasks.workunit.client.1.smithi144.stdout:5/417: chown d0/dc/d1f/c33 138375 1 2024-05-15T15:54:10.309 INFO:tasks.workunit.client.1.smithi144.stdout:4/451: unlink d6/d42/d46/d4b/f1c 0 2024-05-15T15:54:10.310 INFO:tasks.workunit.client.0.smithi049.stdout:4/447: sync 2024-05-15T15:54:10.310 INFO:tasks.workunit.client.0.smithi049.stdout:9/290: sync 2024-05-15T15:54:10.311 INFO:tasks.workunit.client.1.smithi144.stdout:8/461: dwrite d2/d3a/f59 [0,4194304] 0 2024-05-15T15:54:10.311 INFO:tasks.workunit.client.1.smithi144.stdout:4/452: fsync d6/d42/d46/d4b/d1e/d44/f43 0 2024-05-15T15:54:10.313 INFO:tasks.workunit.client.1.smithi144.stdout:9/415: unlink d0/d8/d5e/d44/f53 0 2024-05-15T15:54:10.314 INFO:tasks.workunit.client.0.smithi049.stdout:8/411: dread d2/d14/d1b/f36 [0,4194304] 0 2024-05-15T15:54:10.315 INFO:tasks.workunit.client.1.smithi144.stdout:1/422: creat d7/d1d/d33/d55/f8d x:0 0 0 2024-05-15T15:54:10.316 INFO:tasks.workunit.client.1.smithi144.stdout:9/416: chown d0/d8/d5e/d44/f74 51849177 1 2024-05-15T15:54:10.317 INFO:tasks.workunit.client.0.smithi049.stdout:6/389: dwrite dd/d26/f2e [0,4194304] 0 2024-05-15T15:54:10.318 INFO:tasks.workunit.client.1.smithi144.stdout:8/462: dwrite d2/f3 [0,4194304] 0 2024-05-15T15:54:10.318 INFO:tasks.workunit.client.0.smithi049.stdout:0/407: dread dc/d13/d14/d2f/f24 [0,4194304] 0 2024-05-15T15:54:10.319 INFO:tasks.workunit.client.0.smithi049.stdout:0/408: chown cb 7748369 1 2024-05-15T15:54:10.319 INFO:tasks.workunit.client.0.smithi049.stdout:0/409: chown dc/d6d 3741999 1 2024-05-15T15:54:10.320 INFO:tasks.workunit.client.1.smithi144.stdout:8/463: write d2/de/d36/f46 [992229,11871] 0 2024-05-15T15:54:10.320 INFO:tasks.workunit.client.0.smithi049.stdout:0/410: dread - dc/d13/d2b/f4d zero size 2024-05-15T15:54:10.322 INFO:tasks.workunit.client.1.smithi144.stdout:7/437: symlink d4/df/d81/l90 0 2024-05-15T15:54:10.323 INFO:tasks.workunit.client.0.smithi049.stdout:8/412: symlink d2/d18/l79 0 2024-05-15T15:54:10.327 INFO:tasks.workunit.client.1.smithi144.stdout:5/418: mknod d0/dc/d24/d57/c85 0 2024-05-15T15:54:10.330 INFO:tasks.workunit.client.0.smithi049.stdout:4/448: rename f12 to d18/d1e/d2a/f93 0 2024-05-15T15:54:10.332 INFO:tasks.workunit.client.0.smithi049.stdout:9/291: creat d0/d8/f61 x:0 0 0 2024-05-15T15:54:10.334 INFO:tasks.workunit.client.1.smithi144.stdout:9/417: fdatasync d0/d8/d21/f2a 0 2024-05-15T15:54:10.334 INFO:tasks.workunit.client.0.smithi049.stdout:0/411: creat dc/d32/f7b x:0 0 0 2024-05-15T15:54:10.335 INFO:tasks.workunit.client.1.smithi144.stdout:8/464: dread d2/d24/f68 [0,4194304] 0 2024-05-15T15:54:10.335 INFO:tasks.workunit.client.1.smithi144.stdout:8/465: chown d2/d3a/f67 117 1 2024-05-15T15:54:10.335 INFO:tasks.workunit.client.1.smithi144.stdout:1/423: mkdir d7/d1d/d33/d29/d6b/d8e 0 2024-05-15T15:54:10.336 INFO:tasks.workunit.client.1.smithi144.stdout:8/466: stat d2/d1a/l28 0 2024-05-15T15:54:10.339 INFO:tasks.workunit.client.1.smithi144.stdout:2/409: rename d3/d10/c4c to d3/d7/c8c 0 2024-05-15T15:54:10.340 INFO:tasks.workunit.client.1.smithi144.stdout:2/410: stat d3/d7/f36 0 2024-05-15T15:54:10.342 INFO:tasks.workunit.client.1.smithi144.stdout:7/438: creat d4/d41/d5b/f91 x:0 0 0 2024-05-15T15:54:10.342 INFO:tasks.workunit.client.1.smithi144.stdout:7/439: chown d4/df/d25/f85 0 1 2024-05-15T15:54:10.344 INFO:tasks.workunit.client.1.smithi144.stdout:0/414: sync 2024-05-15T15:54:10.345 INFO:tasks.workunit.client.1.smithi144.stdout:6/387: symlink d7/db/d35/d84/d81/l8d 0 2024-05-15T15:54:10.346 INFO:tasks.workunit.client.0.smithi049.stdout:6/390: rename dd/f15 to dd/d17/d64/f6a 0 2024-05-15T15:54:10.347 INFO:tasks.workunit.client.1.smithi144.stdout:9/418: dread d0/d8/d5e/d44/d10/f27 [0,4194304] 0 2024-05-15T15:54:10.347 INFO:tasks.workunit.client.1.smithi144.stdout:5/419: rmdir d0/dc/d24/d28 39 2024-05-15T15:54:10.350 INFO:tasks.workunit.client.0.smithi049.stdout:8/413: creat d2/de/d27/d5b/f7a x:0 0 0 2024-05-15T15:54:10.354 INFO:tasks.workunit.client.0.smithi049.stdout:8/414: dread d2/de/d25/f32 [0,4194304] 0 2024-05-15T15:54:10.355 INFO:tasks.workunit.client.0.smithi049.stdout:9/292: write d0/d8/d11/f1f [1413374,68038] 0 2024-05-15T15:54:10.355 INFO:tasks.workunit.client.0.smithi049.stdout:9/293: stat d0/d8/d9/f57 0 2024-05-15T15:54:10.356 INFO:tasks.workunit.client.1.smithi144.stdout:1/424: rmdir d7/df/d24 39 2024-05-15T15:54:10.357 INFO:tasks.workunit.client.1.smithi144.stdout:1/425: truncate d7/d1d/d33/d41/d22/f3a 1259755 0 2024-05-15T15:54:10.357 INFO:tasks.workunit.client.0.smithi049.stdout:9/294: truncate d0/d8/d1b/d40/f58 164028 0 2024-05-15T15:54:10.361 INFO:tasks.workunit.client.0.smithi049.stdout:0/412: rename dc/d32/l68 to dc/d6d/l7c 0 2024-05-15T15:54:10.362 INFO:tasks.workunit.client.1.smithi144.stdout:2/411: fsync d3/d7/d20/f7c 0 2024-05-15T15:54:10.363 INFO:tasks.workunit.client.1.smithi144.stdout:7/440: creat d4/df/d10/d48/d72/d82/f92 x:0 0 0 2024-05-15T15:54:10.363 INFO:tasks.workunit.client.1.smithi144.stdout:2/412: chown d3/d7/d20/d22/d52/d75/d65/l81 15793 1 2024-05-15T15:54:10.366 INFO:tasks.workunit.client.0.smithi049.stdout:4/449: rename d18/d1e/d63/l7d to d18/d1e/d2a/d50/d54/l94 0 2024-05-15T15:54:10.367 INFO:tasks.workunit.client.0.smithi049.stdout:0/413: write dc/d13/d14/f19 [4437124,99558] 0 2024-05-15T15:54:10.367 INFO:tasks.workunit.client.1.smithi144.stdout:5/420: creat d0/dc/d1f/d50/f86 x:0 0 0 2024-05-15T15:54:10.368 INFO:tasks.workunit.client.1.smithi144.stdout:5/421: write d0/dc/f71 [804466,67885] 0 2024-05-15T15:54:10.368 INFO:tasks.workunit.client.0.smithi049.stdout:6/391: creat dd/f6b x:0 0 0 2024-05-15T15:54:10.369 INFO:tasks.workunit.client.0.smithi049.stdout:0/414: write dc/d13/d14/d25/d31/d71/f2c [4354643,113258] 0 2024-05-15T15:54:10.369 INFO:tasks.workunit.client.1.smithi144.stdout:1/426: creat d7/d1d/f8f x:0 0 0 2024-05-15T15:54:10.370 INFO:tasks.workunit.client.0.smithi049.stdout:6/392: fsync dd/d17/d24/d2b/d36/f42 0 2024-05-15T15:54:10.370 INFO:tasks.workunit.client.1.smithi144.stdout:1/427: chown d7/d1d/f8f 3073322 1 2024-05-15T15:54:10.371 INFO:tasks.workunit.client.0.smithi049.stdout:9/295: creat d0/d8/d42/f62 x:0 0 0 2024-05-15T15:54:10.372 INFO:tasks.workunit.client.0.smithi049.stdout:8/415: creat d2/f7b x:0 0 0 2024-05-15T15:54:10.374 INFO:tasks.workunit.client.0.smithi049.stdout:4/450: truncate d18/d1e/d63/f7f 51788 0 2024-05-15T15:54:10.380 INFO:tasks.workunit.client.0.smithi049.stdout:8/416: dwrite d2/de/d27/f40 [0,4194304] 0 2024-05-15T15:54:10.380 INFO:tasks.workunit.client.1.smithi144.stdout:6/388: symlink d7/db/d11/d7e/l8e 0 2024-05-15T15:54:10.382 INFO:tasks.workunit.client.0.smithi049.stdout:0/415: mknod dc/d13/d14/d25/d31/d71/c7d 0 2024-05-15T15:54:10.382 INFO:tasks.workunit.client.0.smithi049.stdout:0/416: chown dc/d13/d5e 255226476 1 2024-05-15T15:54:10.387 INFO:tasks.workunit.client.0.smithi049.stdout:9/296: creat d0/d8/d9/d52/d5e/d5f/f63 x:0 0 0 2024-05-15T15:54:10.392 INFO:tasks.workunit.client.0.smithi049.stdout:4/451: mknod d18/d76/c95 0 2024-05-15T15:54:10.394 INFO:tasks.workunit.client.0.smithi049.stdout:8/417: rename f0 to d2/d14/d45/f7c 0 2024-05-15T15:54:10.395 INFO:tasks.workunit.client.0.smithi049.stdout:6/393: dread f8 [0,4194304] 0 2024-05-15T15:54:10.396 INFO:tasks.workunit.client.0.smithi049.stdout:4/452: rmdir d18/d21 39 2024-05-15T15:54:10.402 INFO:tasks.workunit.client.1.smithi144.stdout:9/419: mkdir d0/d8/d5e/d44/d99 0 2024-05-15T15:54:10.403 INFO:tasks.workunit.client.1.smithi144.stdout:5/422: rename d0/dc/d1f/d50/l20 to d0/dc/d12/d4a/d73/l87 0 2024-05-15T15:54:10.403 INFO:tasks.workunit.client.1.smithi144.stdout:8/467: truncate d2/d3a/d7b/f97 155973 0 2024-05-15T15:54:10.403 INFO:tasks.workunit.client.0.smithi049.stdout:0/417: rename dc/d13/d48/d57/f6e to dc/d13/d14/d25/d31/f7e 0 2024-05-15T15:54:10.403 INFO:tasks.workunit.client.1.smithi144.stdout:1/428: creat d7/d1d/d33/d41/d22/f90 x:0 0 0 2024-05-15T15:54:10.406 INFO:tasks.workunit.client.1.smithi144.stdout:5/423: dread d0/f16 [0,4194304] 0 2024-05-15T15:54:10.410 INFO:tasks.workunit.client.0.smithi049.stdout:0/418: dwrite dc/d32/f60 [0,4194304] 0 2024-05-15T15:54:10.415 INFO:tasks.workunit.client.1.smithi144.stdout:0/415: link d4/d6/d4a/l71 d4/d6/l95 0 2024-05-15T15:54:10.415 INFO:tasks.workunit.client.1.smithi144.stdout:0/416: read d4/d6/f32 [3980005,588] 0 2024-05-15T15:54:10.416 INFO:tasks.workunit.client.1.smithi144.stdout:0/417: read - d4/d6/d3a/d4c/f70 zero size 2024-05-15T15:54:10.417 INFO:tasks.workunit.client.0.smithi049.stdout:8/418: rename d2/d14 to d2/d4f/d5d/d7d 0 2024-05-15T15:54:10.420 INFO:tasks.workunit.client.1.smithi144.stdout:9/420: chown d0/d8/d5e/d44/d3c/f45 176 1 2024-05-15T15:54:10.421 INFO:tasks.workunit.client.0.smithi049.stdout:6/394: creat dd/f6c x:0 0 0 2024-05-15T15:54:10.422 INFO:tasks.workunit.client.1.smithi144.stdout:1/429: dread - d7/d1d/d33/d29/f64 zero size 2024-05-15T15:54:10.424 INFO:tasks.workunit.client.1.smithi144.stdout:5/424: mkdir d0/dc/d12/d31/d35/d6b/d88 0 2024-05-15T15:54:10.427 INFO:tasks.workunit.client.0.smithi049.stdout:9/297: dwrite d0/d8/f5a [0,4194304] 0 2024-05-15T15:54:10.429 INFO:tasks.workunit.client.0.smithi049.stdout:8/419: symlink d2/d4f/d5d/d63/l7e 0 2024-05-15T15:54:10.429 INFO:tasks.workunit.client.1.smithi144.stdout:6/389: dwrite d7/db/f39 [4194304,4194304] 0 2024-05-15T15:54:10.430 INFO:tasks.workunit.client.1.smithi144.stdout:7/441: link d4/d3c/c58 d4/df/d79/c93 0 2024-05-15T15:54:10.433 INFO:tasks.workunit.client.1.smithi144.stdout:5/425: dwrite d0/f1a [0,4194304] 0 2024-05-15T15:54:10.434 INFO:tasks.workunit.client.0.smithi049.stdout:9/298: fdatasync d0/d8/d12/f59 0 2024-05-15T15:54:10.436 INFO:tasks.workunit.client.0.smithi049.stdout:6/395: dwrite dd/d17/d24/d2b/d36/d50/f55 [0,4194304] 0 2024-05-15T15:54:10.439 INFO:tasks.workunit.client.1.smithi144.stdout:1/430: rename d7/f4d to d7/d1d/f91 0 2024-05-15T15:54:10.440 INFO:tasks.workunit.client.0.smithi049.stdout:4/453: dwrite d18/d1e/d2a/d50/f5e [0,4194304] 0 2024-05-15T15:54:10.443 INFO:tasks.workunit.client.0.smithi049.stdout:9/299: mknod d0/d8/d12/c64 0 2024-05-15T15:54:10.447 INFO:tasks.workunit.client.0.smithi049.stdout:8/420: rename d2/d4f/d5d/d7d/d1b/c21 to d2/d4f/d5d/d7d/d1b/c7f 0 2024-05-15T15:54:10.448 INFO:tasks.workunit.client.0.smithi049.stdout:0/419: link dc/d13/d14/l72 dc/d13/d56/d6f/l7f 0 2024-05-15T15:54:10.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:10 smithi144 ceph-mon[36389]: pgmap v25: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 36 MiB/s rd, 174 MiB/s wr, 297 op/s 2024-05-15T15:54:10.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:10 smithi144 ceph-mon[36389]: Upgrade: Updating mgr.smithi049.vuhzgf 2024-05-15T15:54:10.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:10 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:10.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:10 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi049.vuhzgf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:54:10.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:10 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi049.vuhzgf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:54:10.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:10 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:54:10.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:10 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:54:10.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:10 smithi144 ceph-mon[36389]: Deploying daemon mgr.smithi049.vuhzgf on smithi049 2024-05-15T15:54:10.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:10 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:54:10.450 INFO:tasks.workunit.client.0.smithi049.stdout:0/420: read - dc/d13/d14/f79 zero size 2024-05-15T15:54:10.451 INFO:tasks.workunit.client.1.smithi144.stdout:7/442: symlink d4/df/d25/d40/l94 0 2024-05-15T15:54:10.451 INFO:tasks.workunit.client.1.smithi144.stdout:3/423: sync 2024-05-15T15:54:10.451 INFO:tasks.workunit.client.0.smithi049.stdout:9/300: chown d0/l20 21 1 2024-05-15T15:54:10.452 INFO:tasks.workunit.client.1.smithi144.stdout:7/443: write d4/df/d25/d38/d68/f74 [775584,42224] 0 2024-05-15T15:54:10.454 INFO:tasks.workunit.client.0.smithi049.stdout:8/421: creat d2/d4f/d5d/f80 x:0 0 0 2024-05-15T15:54:10.454 INFO:tasks.workunit.client.0.smithi049.stdout:8/422: chown d2/de 13815 1 2024-05-15T15:54:10.458 INFO:tasks.workunit.client.1.smithi144.stdout:3/424: dread d0/d1/d17/d39/d40/d4a/f8e [0,4194304] 0 2024-05-15T15:54:10.460 INFO:tasks.workunit.client.1.smithi144.stdout:3/425: dread - d0/d11/d3b/f6b zero size 2024-05-15T15:54:10.460 INFO:tasks.workunit.client.0.smithi049.stdout:8/423: rmdir d2/d4f/d5d/d7d/d45 39 2024-05-15T15:54:10.461 INFO:tasks.workunit.client.1.smithi144.stdout:0/418: write d4/df/f43 [96156,58462] 0 2024-05-15T15:54:10.464 INFO:tasks.workunit.client.0.smithi049.stdout:0/421: link dc/d13/d14/l28 dc/d13/d14/d33/l80 0 2024-05-15T15:54:10.469 INFO:tasks.workunit.client.1.smithi144.stdout:0/419: dwrite d4/df/f43 [0,4194304] 0 2024-05-15T15:54:10.470 INFO:tasks.workunit.client.1.smithi144.stdout:9/421: dwrite d0/d8/d5e/d44/f41 [0,4194304] 0 2024-05-15T15:54:10.473 INFO:tasks.workunit.client.1.smithi144.stdout:7/444: dread d4/f21 [0,4194304] 0 2024-05-15T15:54:10.474 INFO:tasks.workunit.client.0.smithi049.stdout:0/422: rename dc/d32/f60 to dc/d13/d2b/f81 0 2024-05-15T15:54:10.474 INFO:tasks.workunit.client.0.smithi049.stdout:0/423: stat dc/d13/d14/l42 0 2024-05-15T15:54:10.477 INFO:tasks.workunit.client.0.smithi049.stdout:0/424: truncate dc/d13/d14/d33/f53 1008759 0 2024-05-15T15:54:10.480 INFO:tasks.workunit.client.1.smithi144.stdout:3/426: rename d0/d1/d56/d71/d7a to d0/d11/d3b/d93 0 2024-05-15T15:54:10.482 INFO:tasks.workunit.client.0.smithi049.stdout:6/396: dwrite f8 [0,4194304] 0 2024-05-15T15:54:10.487 INFO:tasks.workunit.client.1.smithi144.stdout:3/427: dwrite d0/d11/d24/f57 [0,4194304] 0 2024-05-15T15:54:10.489 INFO:tasks.workunit.client.0.smithi049.stdout:8/424: truncate d2/d4f/d5d/d7d/f5c 361894 0 2024-05-15T15:54:10.494 INFO:tasks.workunit.client.1.smithi144.stdout:0/420: mkdir d4/df/d1f/d24/d96 0 2024-05-15T15:54:10.496 INFO:tasks.workunit.client.1.smithi144.stdout:0/421: readlink d4/df/l28 0 2024-05-15T15:54:10.496 INFO:tasks.workunit.client.1.smithi144.stdout:6/390: dwrite d7/db/d35/f47 [0,4194304] 0 2024-05-15T15:54:10.497 INFO:tasks.workunit.client.0.smithi049.stdout:8/425: symlink d2/d4f/d5d/l81 0 2024-05-15T15:54:10.499 INFO:tasks.workunit.client.1.smithi144.stdout:6/391: write d7/d1e/d28/f3d [4580290,24659] 0 2024-05-15T15:54:10.500 INFO:tasks.workunit.client.1.smithi144.stdout:5/426: dwrite d0/dc/d12/f18 [0,4194304] 0 2024-05-15T15:54:10.501 INFO:tasks.workunit.client.0.smithi049.stdout:0/425: truncate dc/f15 3820771 0 2024-05-15T15:54:10.502 INFO:tasks.workunit.client.1.smithi144.stdout:5/427: chown d0/dc/f1b 215799915 1 2024-05-15T15:54:10.502 INFO:tasks.workunit.client.1.smithi144.stdout:5/428: stat d0/l1 0 2024-05-15T15:54:10.503 INFO:tasks.workunit.client.0.smithi049.stdout:0/426: write dc/d13/d14/d25/d31/f58 [2850206,99047] 0 2024-05-15T15:54:10.504 INFO:tasks.workunit.client.0.smithi049.stdout:6/397: getdents dd/d17/d24 0 2024-05-15T15:54:10.505 INFO:tasks.workunit.client.0.smithi049.stdout:8/426: creat d2/de/d71/f82 x:0 0 0 2024-05-15T15:54:10.506 INFO:tasks.workunit.client.1.smithi144.stdout:7/445: fdatasync d4/df/d25/d40/d62/f6b 0 2024-05-15T15:54:10.506 INFO:tasks.workunit.client.0.smithi049.stdout:6/398: chown dd/d17/d24/d58/d5f/l65 70240521 1 2024-05-15T15:54:10.507 INFO:tasks.workunit.client.0.smithi049.stdout:8/427: write d2/de/d25/d74/f76 [400977,34971] 0 2024-05-15T15:54:10.508 INFO:tasks.workunit.client.1.smithi144.stdout:9/422: dread d0/f4 [0,4194304] 0 2024-05-15T15:54:10.509 INFO:tasks.workunit.client.0.smithi049.stdout:8/428: dread - d2/d4f/f6d zero size 2024-05-15T15:54:10.509 INFO:tasks.workunit.client.1.smithi144.stdout:3/428: mknod d0/d1/d6c/c94 0 2024-05-15T15:54:10.510 INFO:tasks.workunit.client.1.smithi144.stdout:4/453: sync 2024-05-15T15:54:10.560 INFO:tasks.workunit.client.0.smithi049.stdout:2/426: sync 2024-05-15T15:54:10.561 INFO:tasks.workunit.client.0.smithi049.stdout:6/399: creat dd/d17/f6d x:0 0 0 2024-05-15T15:54:10.564 INFO:tasks.workunit.client.1.smithi144.stdout:3/429: dread d0/d11/d24/f74 [0,4194304] 0 2024-05-15T15:54:10.565 INFO:tasks.workunit.client.1.smithi144.stdout:3/430: chown d0/l2b 1 1 2024-05-15T15:54:10.565 INFO:tasks.workunit.client.0.smithi049.stdout:0/427: mkdir dc/d13/d14/d82 0 2024-05-15T15:54:10.566 INFO:tasks.workunit.client.1.smithi144.stdout:3/431: read d0/d1/f68 [89206,74450] 0 2024-05-15T15:54:10.566 INFO:tasks.workunit.client.1.smithi144.stdout:8/468: sync 2024-05-15T15:54:10.566 INFO:tasks.workunit.client.1.smithi144.stdout:2/413: sync 2024-05-15T15:54:10.566 INFO:tasks.workunit.client.0.smithi049.stdout:6/400: dread dd/d17/f38 [0,4194304] 0 2024-05-15T15:54:10.569 INFO:tasks.workunit.client.1.smithi144.stdout:0/422: unlink d4/d6/d4a/d74/l78 0 2024-05-15T15:54:10.569 INFO:tasks.workunit.client.1.smithi144.stdout:1/431: truncate d7/df/d24/f54 2959979 0 2024-05-15T15:54:10.570 INFO:tasks.workunit.client.0.smithi049.stdout:8/429: mknod d2/d4f/d5d/c83 0 2024-05-15T15:54:10.570 INFO:tasks.workunit.client.1.smithi144.stdout:3/432: dwrite d0/f1d [0,4194304] 0 2024-05-15T15:54:10.570 INFO:tasks.workunit.client.1.smithi144.stdout:0/423: write d4/df/d1f/f58 [659044,36955] 0 2024-05-15T15:54:10.571 INFO:tasks.workunit.client.1.smithi144.stdout:1/432: fdatasync d7/d1d/d33/d29/f64 0 2024-05-15T15:54:10.572 INFO:tasks.workunit.client.0.smithi049.stdout:2/427: mknod d3/db/d1d/d1e/d2b/d6b/c84 0 2024-05-15T15:54:10.573 INFO:tasks.workunit.client.1.smithi144.stdout:3/433: write d0/d11/d3b/f6b [284867,121818] 0 2024-05-15T15:54:10.573 INFO:tasks.workunit.client.1.smithi144.stdout:1/433: stat d7/d1d/d33/d1e/d34/f76 0 2024-05-15T15:54:10.574 INFO:tasks.workunit.client.1.smithi144.stdout:5/429: stat d0/dc/d2f/d68/d5c/l64 0 2024-05-15T15:54:10.574 INFO:tasks.workunit.client.1.smithi144.stdout:1/434: dread - d7/d1d/d33/d41/d22/f6a zero size 2024-05-15T15:54:10.575 INFO:tasks.workunit.client.1.smithi144.stdout:5/430: fsync d0/dc/d1f/f3b 0 2024-05-15T15:54:10.577 INFO:tasks.workunit.client.1.smithi144.stdout:7/446: fsync d4/df/f1a 0 2024-05-15T15:54:10.581 INFO:tasks.workunit.client.1.smithi144.stdout:9/423: creat d0/d8/d5e/f9a x:0 0 0 2024-05-15T15:54:10.582 INFO:tasks.workunit.client.0.smithi049.stdout:0/428: readlink dc/d13/d56/d6f/l7f 0 2024-05-15T15:54:10.588 INFO:tasks.workunit.client.0.smithi049.stdout:8/430: creat d2/de/d71/f84 x:0 0 0 2024-05-15T15:54:10.588 INFO:tasks.workunit.client.0.smithi049.stdout:8/431: fsync d2/de/d27/d5b/f7a 0 2024-05-15T15:54:10.595 INFO:tasks.workunit.client.0.smithi049.stdout:6/401: symlink dd/d26/d5b/l6e 0 2024-05-15T15:54:10.597 INFO:tasks.workunit.client.0.smithi049.stdout:6/402: write dd/d17/d24/d2b/f63 [620307,59774] 0 2024-05-15T15:54:10.597 INFO:tasks.workunit.client.0.smithi049.stdout:2/428: mkdir d3/d85 0 2024-05-15T15:54:10.598 INFO:tasks.workunit.client.1.smithi144.stdout:0/424: rmdir d4/dc 39 2024-05-15T15:54:10.599 INFO:tasks.workunit.client.1.smithi144.stdout:0/425: stat d4/d6/d4a/d4d/c84 0 2024-05-15T15:54:10.599 INFO:tasks.workunit.client.0.smithi049.stdout:6/403: read dd/d17/d24/d2b/f4d [370447,46905] 0 2024-05-15T15:54:10.601 INFO:tasks.workunit.client.0.smithi049.stdout:6/404: dread dd/d17/d24/d2b/f46 [0,4194304] 0 2024-05-15T15:54:10.602 INFO:tasks.workunit.client.1.smithi144.stdout:6/392: truncate d7/ff 7456 0 2024-05-15T15:54:10.603 INFO:tasks.workunit.client.1.smithi144.stdout:2/414: dwrite d3/d7/f47 [0,4194304] 0 2024-05-15T15:54:10.603 INFO:tasks.workunit.client.0.smithi049.stdout:0/429: mknod dc/d13/d14/d82/c83 0 2024-05-15T15:54:10.604 INFO:tasks.workunit.client.1.smithi144.stdout:2/415: truncate d3/d7/d54/f5a 4628073 0 2024-05-15T15:54:10.605 INFO:tasks.workunit.client.0.smithi049.stdout:8/432: write d2/d4f/d5d/d7d/d1b/d68/f4b [198059,99839] 0 2024-05-15T15:54:10.607 INFO:tasks.workunit.client.1.smithi144.stdout:5/431: dread - d0/dc/d24/d28/f82 zero size 2024-05-15T15:54:10.607 INFO:tasks.workunit.client.1.smithi144.stdout:3/434: write d0/d11/d3b/f45 [3284015,117580] 0 2024-05-15T15:54:10.609 INFO:tasks.workunit.client.1.smithi144.stdout:7/447: creat d4/df/d25/d40/d62/f95 x:0 0 0 2024-05-15T15:54:10.610 INFO:tasks.workunit.client.1.smithi144.stdout:7/448: truncate d4/df/d25/d38/f71 783540 0 2024-05-15T15:54:10.610 INFO:tasks.workunit.client.1.smithi144.stdout:9/424: mkdir d0/d8/d5e/d9b 0 2024-05-15T15:54:10.611 INFO:tasks.workunit.client.1.smithi144.stdout:7/449: fsync d4/d41/f5a 0 2024-05-15T15:54:10.612 INFO:tasks.workunit.client.1.smithi144.stdout:9/425: write d0/d8/d5e/d44/d10/f81 [467461,54181] 0 2024-05-15T15:54:10.612 INFO:tasks.workunit.client.0.smithi049.stdout:2/429: link d3/d8/fe d3/db/d1d/d1e/d2b/d3b/d47/f86 0 2024-05-15T15:54:10.613 INFO:tasks.workunit.client.1.smithi144.stdout:9/426: chown d0/d8/d1b 1990153 1 2024-05-15T15:54:10.618 INFO:tasks.workunit.client.0.smithi049.stdout:8/433: rename d2/d4f/d5d/d7d/d1b/d68/f4b to d2/de/d27/f85 0 2024-05-15T15:54:10.619 INFO:tasks.workunit.client.1.smithi144.stdout:4/454: chown d6/d42/d46/d4b 1 1 2024-05-15T15:54:10.619 INFO:tasks.workunit.client.0.smithi049.stdout:0/430: write dc/d13/d3d/f4c [100358,46634] 0 2024-05-15T15:54:10.630 INFO:tasks.workunit.client.0.smithi049.stdout:8/434: rename d2/d4f to d2/de/d71/d86 0 2024-05-15T15:54:10.630 INFO:tasks.workunit.client.0.smithi049.stdout:6/405: dread dd/f16 [4194304,4194304] 0 2024-05-15T15:54:10.631 INFO:tasks.workunit.client.0.smithi049.stdout:0/431: mkdir dc/d13/d3d/d84 0 2024-05-15T15:54:10.632 INFO:tasks.workunit.client.0.smithi049.stdout:6/406: write dd/d26/f2e [4708431,16494] 0 2024-05-15T15:54:10.634 INFO:tasks.workunit.client.0.smithi049.stdout:6/407: dread - dd/d17/f4a zero size 2024-05-15T15:54:10.638 INFO:tasks.workunit.client.0.smithi049.stdout:8/435: unlink d2/de/d71/d86/d5d/c6f 0 2024-05-15T15:54:10.638 INFO:tasks.workunit.client.0.smithi049.stdout:2/430: dwrite d3/db/d1d/d1e/f3c [0,4194304] 0 2024-05-15T15:54:10.640 INFO:tasks.workunit.client.1.smithi144.stdout:6/393: rename d7/db/f39 to d7/db/d35/d84/f8f 0 2024-05-15T15:54:10.640 INFO:tasks.workunit.client.1.smithi144.stdout:6/394: read - d7/f41 zero size 2024-05-15T15:54:10.640 INFO:tasks.workunit.client.0.smithi049.stdout:2/431: truncate d3/da/d15/d24/f62 338131 0 2024-05-15T15:54:10.645 INFO:tasks.workunit.client.0.smithi049.stdout:0/432: symlink dc/d13/d14/d25/d31/d71/l85 0 2024-05-15T15:54:10.645 INFO:tasks.workunit.client.1.smithi144.stdout:6/395: dwrite d7/d32/f50 [4194304,4194304] 0 2024-05-15T15:54:10.646 INFO:tasks.workunit.client.0.smithi049.stdout:0/433: dread - dc/d13/d14/f79 zero size 2024-05-15T15:54:10.647 INFO:tasks.workunit.client.1.smithi144.stdout:6/396: fsync d7/db/d11/d3c/f77 0 2024-05-15T15:54:10.648 INFO:tasks.workunit.client.0.smithi049.stdout:6/408: mknod dd/d17/d64/c6f 0 2024-05-15T15:54:10.649 INFO:tasks.workunit.client.0.smithi049.stdout:6/409: rename dd/d17 to dd/d17/d64/d70 22 2024-05-15T15:54:10.655 INFO:tasks.workunit.client.1.smithi144.stdout:3/435: unlink d0/d1/d17/f53 0 2024-05-15T15:54:10.655 INFO:tasks.workunit.client.1.smithi144.stdout:5/432: dwrite d0/f16 [0,4194304] 0 2024-05-15T15:54:10.655 INFO:tasks.workunit.client.0.smithi049.stdout:2/432: creat d3/db/d1d/d1e/d2b/d2e/f87 x:0 0 0 2024-05-15T15:54:10.656 INFO:tasks.workunit.client.0.smithi049.stdout:6/410: dread dd/d17/d24/d2b/f63 [0,4194304] 0 2024-05-15T15:54:10.656 INFO:tasks.workunit.client.1.smithi144.stdout:7/450: creat d4/d45/f96 x:0 0 0 2024-05-15T15:54:10.657 INFO:tasks.workunit.client.1.smithi144.stdout:7/451: readlink d4/l26 0 2024-05-15T15:54:10.659 INFO:tasks.workunit.client.1.smithi144.stdout:5/433: dread d0/f49 [0,4194304] 0 2024-05-15T15:54:10.663 INFO:tasks.workunit.client.0.smithi049.stdout:8/436: link d2/de/d27/f59 d2/de/d25/d74/f87 0 2024-05-15T15:54:10.664 INFO:tasks.workunit.client.0.smithi049.stdout:8/437: readlink d2/d18/l64 0 2024-05-15T15:54:10.664 INFO:tasks.workunit.client.1.smithi144.stdout:5/434: dwrite d0/dc/d1f/d50/f86 [0,4194304] 0 2024-05-15T15:54:10.665 INFO:tasks.workunit.client.0.smithi049.stdout:2/433: dwrite d3/db/f1f [0,4194304] 0 2024-05-15T15:54:10.665 INFO:tasks.workunit.client.0.smithi049.stdout:0/434: dread dc/d13/d14/d25/d31/d71/f3a [0,4194304] 0 2024-05-15T15:54:10.674 INFO:tasks.workunit.client.0.smithi049.stdout:0/435: rmdir dc/d13/d2b 39 2024-05-15T15:54:10.680 INFO:tasks.workunit.client.1.smithi144.stdout:7/452: mknod d4/df/d10/d48/d72/d82/c97 0 2024-05-15T15:54:10.681 INFO:tasks.workunit.client.0.smithi049.stdout:0/436: symlink dc/d13/d48/d57/l86 0 2024-05-15T15:54:10.681 INFO:tasks.workunit.client.1.smithi144.stdout:8/469: getdents d2/d24 0 2024-05-15T15:54:10.682 INFO:tasks.workunit.client.1.smithi144.stdout:8/470: dread - d2/d24/d2d/d30/f7d zero size 2024-05-15T15:54:10.683 INFO:tasks.workunit.client.1.smithi144.stdout:8/471: readlink d2/de/d36/l63 0 2024-05-15T15:54:10.683 INFO:tasks.workunit.client.0.smithi049.stdout:2/434: rename d3/da/l3e to d3/l88 0 2024-05-15T15:54:10.688 INFO:tasks.workunit.client.1.smithi144.stdout:8/472: dwrite d2/d3a/f7f [0,4194304] 0 2024-05-15T15:54:10.689 INFO:tasks.workunit.client.0.smithi049.stdout:8/438: dread d2/de/d27/f35 [0,4194304] 0 2024-05-15T15:54:10.690 INFO:tasks.workunit.client.0.smithi049.stdout:8/439: stat d2/de/d27/d5b/d60 0 2024-05-15T15:54:10.691 INFO:tasks.workunit.client.1.smithi144.stdout:8/473: dread d2/d24/d2d/d30/f81 [0,4194304] 0 2024-05-15T15:54:10.694 INFO:tasks.workunit.client.1.smithi144.stdout:1/435: sync 2024-05-15T15:54:10.695 INFO:tasks.workunit.client.1.smithi144.stdout:1/436: chown d7/d1d/d33/d29/d6b/l5f 257115772 1 2024-05-15T15:54:10.698 INFO:tasks.workunit.client.1.smithi144.stdout:3/436: mknod d0/d1/d17/d39/d83/c95 0 2024-05-15T15:54:10.700 INFO:tasks.workunit.client.1.smithi144.stdout:2/416: truncate d3/d10/f1d 701600 0 2024-05-15T15:54:10.701 INFO:tasks.workunit.client.1.smithi144.stdout:7/453: creat d4/df/d25/d40/d73/f98 x:0 0 0 2024-05-15T15:54:10.701 INFO:tasks.workunit.client.0.smithi049.stdout:6/411: truncate dd/ff 2691800 0 2024-05-15T15:54:10.707 INFO:tasks.workunit.client.1.smithi144.stdout:7/454: dwrite d4/d41/d5b/f88 [0,4194304] 0 2024-05-15T15:54:10.707 INFO:tasks.workunit.client.1.smithi144.stdout:9/427: rename d0/d8/d5e/d76 to d0/d8/d5e/d44/d10/d9c 0 2024-05-15T15:54:10.708 INFO:tasks.workunit.client.0.smithi049.stdout:8/440: link d2/de/f55 d2/de/d27/d5b/f88 0 2024-05-15T15:54:10.708 INFO:tasks.workunit.client.1.smithi144.stdout:7/455: fsync d4/df/d25/d38/f3e 0 2024-05-15T15:54:10.712 INFO:tasks.workunit.client.1.smithi144.stdout:5/435: unlink d0/dc/d24/d28/c4e 0 2024-05-15T15:54:10.712 INFO:tasks.workunit.client.0.smithi049.stdout:2/435: write d3/db/d1d/d1e/d2b/d2e/f41 [4515325,33832] 0 2024-05-15T15:54:10.713 INFO:tasks.workunit.client.0.smithi049.stdout:2/436: read - d3/db/d1d/d1e/d2b/d2e/f6c zero size 2024-05-15T15:54:10.713 INFO:tasks.workunit.client.0.smithi049.stdout:8/441: creat d2/de/d71/d86/d5d/f89 x:0 0 0 2024-05-15T15:54:10.714 INFO:tasks.workunit.client.0.smithi049.stdout:0/437: write dc/d13/d14/d25/d31/f7e [943461,102868] 0 2024-05-15T15:54:10.715 INFO:tasks.workunit.client.1.smithi144.stdout:8/474: truncate d2/de/d36/f1e 613082 0 2024-05-15T15:54:10.715 INFO:tasks.workunit.client.0.smithi049.stdout:0/438: stat fa 0 2024-05-15T15:54:10.715 INFO:tasks.workunit.client.1.smithi144.stdout:7/456: dwrite d4/df/d10/f1d [0,4194304] 0 2024-05-15T15:54:10.716 INFO:tasks.workunit.client.1.smithi144.stdout:6/397: truncate d7/f8 4214778 0 2024-05-15T15:54:10.716 INFO:tasks.workunit.client.1.smithi144.stdout:8/475: write d2/de/f33 [410934,15407] 0 2024-05-15T15:54:10.717 INFO:tasks.workunit.client.0.smithi049.stdout:0/439: write dc/d13/d14/d25/d31/f58 [226874,77002] 0 2024-05-15T15:54:10.717 INFO:tasks.workunit.client.1.smithi144.stdout:1/437: creat d7/d1d/d33/d1e/d87/f92 x:0 0 0 2024-05-15T15:54:10.720 INFO:tasks.workunit.client.1.smithi144.stdout:1/438: write d7/d1d/d33/d41/d22/f3a [2058298,74618] 0 2024-05-15T15:54:10.721 INFO:tasks.workunit.client.0.smithi049.stdout:2/437: symlink d3/db/d1d/d1e/d2b/d3b/l89 0 2024-05-15T15:54:10.723 INFO:tasks.workunit.client.1.smithi144.stdout:4/455: getdents d6/dd/d34/d3b/d30/db/d1d/d40/d6c 0 2024-05-15T15:54:10.724 INFO:tasks.workunit.client.1.smithi144.stdout:7/457: read d4/df/d10/f6e [504369,41740] 0 2024-05-15T15:54:10.731 INFO:tasks.workunit.client.1.smithi144.stdout:2/417: dwrite d3/d7/d54/f5a [0,4194304] 0 2024-05-15T15:54:10.732 INFO:tasks.workunit.client.0.smithi049.stdout:0/440: mknod dc/d13/d14/d25/d31/c87 0 2024-05-15T15:54:10.733 INFO:tasks.workunit.client.0.smithi049.stdout:6/412: dwrite dd/f19 [0,4194304] 0 2024-05-15T15:54:10.736 INFO:tasks.workunit.client.0.smithi049.stdout:6/413: truncate dd/d17/d24/d2b/f5d 394530 0 2024-05-15T15:54:10.738 INFO:tasks.workunit.client.1.smithi144.stdout:9/428: unlink d0/d8/d5e/d44/d16/l83 0 2024-05-15T15:54:10.740 INFO:tasks.workunit.client.0.smithi049.stdout:6/414: dread dd/d17/d24/d2b/f5d [0,4194304] 0 2024-05-15T15:54:10.741 INFO:tasks.workunit.client.0.smithi049.stdout:6/415: chown dd/d17/d24/d2b/f4d 3 1 2024-05-15T15:54:10.742 INFO:tasks.workunit.client.1.smithi144.stdout:2/418: dwrite d3/d7/d20/d22/d52/d64/f26 [4194304,4194304] 0 2024-05-15T15:54:10.744 INFO:tasks.workunit.client.1.smithi144.stdout:2/419: truncate d3/d10/f7e 1655671 0 2024-05-15T15:54:10.747 INFO:tasks.workunit.client.1.smithi144.stdout:5/436: unlink d0/dc/d12/f43 0 2024-05-15T15:54:10.750 INFO:tasks.workunit.client.1.smithi144.stdout:0/426: sync 2024-05-15T15:54:10.752 INFO:tasks.workunit.client.0.smithi049.stdout:8/442: rename d2/de/d71/d86/d5d/d7d/f5c to d2/de/d71/d86/d5d/d7d/d45/f8a 0 2024-05-15T15:54:10.753 INFO:tasks.workunit.client.1.smithi144.stdout:8/476: mknod d2/de/d36/d87/c9b 0 2024-05-15T15:54:10.754 INFO:tasks.workunit.client.0.smithi049.stdout:8/443: readlink d2/de/d71/d86/d5d/d7d/d1b/d69/l78 0 2024-05-15T15:54:10.754 INFO:tasks.workunit.client.0.smithi049.stdout:0/441: rmdir dc/d13/d2b 39 2024-05-15T15:54:10.759 INFO:tasks.workunit.client.0.smithi049.stdout:2/438: write d3/f10 [4154614,43209] 0 2024-05-15T15:54:10.759 INFO:tasks.workunit.client.1.smithi144.stdout:8/477: dwrite d2/d24/d2d/d30/f7d [0,4194304] 0 2024-05-15T15:54:10.759 INFO:tasks.workunit.client.0.smithi049.stdout:8/444: dread d2/de/d71/d86/d5d/d7d/d1b/f2b [0,4194304] 0 2024-05-15T15:54:10.762 INFO:tasks.workunit.client.0.smithi049.stdout:2/439: truncate d3/db/d1d/d1e/d2b/d6b/f80 520614 0 2024-05-15T15:54:10.762 INFO:tasks.workunit.client.0.smithi049.stdout:8/445: write d2/de/d71/d86/d5d/d63/f6e [721135,37818] 0 2024-05-15T15:54:10.763 INFO:tasks.workunit.client.1.smithi144.stdout:1/439: mknod d7/d1d/c93 0 2024-05-15T15:54:10.765 INFO:tasks.workunit.client.1.smithi144.stdout:4/456: creat d6/d42/d46/d4b/d1e/d44/f93 x:0 0 0 2024-05-15T15:54:10.768 INFO:tasks.workunit.client.1.smithi144.stdout:7/458: creat d4/d41/d5b/f99 x:0 0 0 2024-05-15T15:54:10.772 INFO:tasks.workunit.client.1.smithi144.stdout:3/437: write d0/d11/f26 [1640708,56046] 0 2024-05-15T15:54:10.774 INFO:tasks.workunit.client.1.smithi144.stdout:9/429: chown d0/d8/d5e/d44/d10/d19/c7b 303259571 1 2024-05-15T15:54:10.775 INFO:tasks.workunit.client.0.smithi049.stdout:2/440: creat d3/da/d15/d73/f8a x:0 0 0 2024-05-15T15:54:10.776 INFO:tasks.workunit.client.0.smithi049.stdout:1/513: sync 2024-05-15T15:54:10.777 INFO:tasks.workunit.client.0.smithi049.stdout:0/442: creat dc/d13/d5e/f88 x:0 0 0 2024-05-15T15:54:10.779 INFO:tasks.workunit.client.0.smithi049.stdout:1/514: dread - d0/d4/dd/de/d48/d5c/f87 zero size 2024-05-15T15:54:10.779 INFO:tasks.workunit.client.1.smithi144.stdout:2/420: write d3/f6c [715570,3250] 0 2024-05-15T15:54:10.782 INFO:tasks.workunit.client.0.smithi049.stdout:2/441: dwrite d3/db/f12 [0,4194304] 0 2024-05-15T15:54:10.783 INFO:tasks.workunit.client.0.smithi049.stdout:8/446: mknod d2/de/d71/d86/c8b 0 2024-05-15T15:54:10.784 INFO:tasks.workunit.client.1.smithi144.stdout:5/437: fdatasync d0/f11 0 2024-05-15T15:54:10.787 INFO:tasks.workunit.client.0.smithi049.stdout:0/443: creat dc/d13/d5e/f89 x:0 0 0 2024-05-15T15:54:10.790 INFO:tasks.workunit.client.0.smithi049.stdout:0/444: write dc/d13/d48/d57/f65 [974259,50615] 0 2024-05-15T15:54:10.793 INFO:tasks.workunit.client.0.smithi049.stdout:8/447: dwrite d2/de/d71/d86/d5d/f89 [0,4194304] 0 2024-05-15T15:54:10.794 INFO:tasks.workunit.client.1.smithi144.stdout:0/427: dread d4/dc/f18 [0,4194304] 0 2024-05-15T15:54:10.801 INFO:tasks.workunit.client.0.smithi049.stdout:6/416: link dd/d17/d24/d2b/l3e dd/l71 0 2024-05-15T15:54:10.804 INFO:tasks.workunit.client.1.smithi144.stdout:1/440: rmdir d7/d1d/d33/d1e/d34 39 2024-05-15T15:54:10.808 INFO:tasks.workunit.client.0.smithi049.stdout:2/442: dread d3/da/d15/f27 [0,4194304] 0 2024-05-15T15:54:10.810 INFO:tasks.workunit.client.1.smithi144.stdout:4/457: unlink d6/dd/d34/d3b/d30/db/d1d/d4e/l50 0 2024-05-15T15:54:10.815 INFO:tasks.workunit.client.0.smithi049.stdout:1/515: truncate d0/d4/dd/de/d1e/d2e/f88 1258901 0 2024-05-15T15:54:10.817 INFO:tasks.workunit.client.1.smithi144.stdout:7/459: rename d4/df/d25/d38/f7c to d4/df/d25/d40/d73/f9a 0 2024-05-15T15:54:10.818 INFO:tasks.workunit.client.1.smithi144.stdout:7/460: readlink d4/d41/l59 0 2024-05-15T15:54:10.821 INFO:tasks.workunit.client.1.smithi144.stdout:3/438: creat d0/d11/d3b/f96 x:0 0 0 2024-05-15T15:54:10.822 INFO:tasks.workunit.client.1.smithi144.stdout:3/439: chown d0/d1/d17/c31 0 1 2024-05-15T15:54:10.824 INFO:tasks.workunit.client.0.smithi049.stdout:6/417: creat dd/d17/d24/d2b/f72 x:0 0 0 2024-05-15T15:54:10.825 INFO:tasks.workunit.client.1.smithi144.stdout:9/430: mknod d0/d8/d1b/d79/c9d 0 2024-05-15T15:54:10.827 INFO:tasks.workunit.client.1.smithi144.stdout:2/421: mknod d3/d7/d20/d22/d52/d75/c8d 0 2024-05-15T15:54:10.832 INFO:tasks.workunit.client.1.smithi144.stdout:2/422: dwrite d3/d7/d20/d37/f46 [0,4194304] 0 2024-05-15T15:54:10.833 INFO:tasks.workunit.client.0.smithi049.stdout:6/418: rmdir dd/d26 39 2024-05-15T15:54:10.835 INFO:tasks.workunit.client.0.smithi049.stdout:1/516: dread d0/d4/dd/f5a [0,4194304] 0 2024-05-15T15:54:10.835 INFO:tasks.workunit.client.0.smithi049.stdout:8/448: dwrite d2/d18/f47 [0,4194304] 0 2024-05-15T15:54:10.839 INFO:tasks.workunit.client.1.smithi144.stdout:1/441: unlink d7/df/l3b 0 2024-05-15T15:54:10.841 INFO:tasks.workunit.client.0.smithi049.stdout:0/445: mkdir dc/d13/d2b/d76/d8a 0 2024-05-15T15:54:10.841 INFO:tasks.workunit.client.0.smithi049.stdout:0/446: chown dc/d13/f22 0 1 2024-05-15T15:54:10.845 INFO:tasks.workunit.client.0.smithi049.stdout:0/447: dread - dc/d13/d14/d25/d31/f75 zero size 2024-05-15T15:54:10.846 INFO:tasks.workunit.client.1.smithi144.stdout:7/461: rename d4/df/d25/d38/f61 to d4/df/d79/f9b 0 2024-05-15T15:54:10.847 INFO:tasks.workunit.client.1.smithi144.stdout:3/440: mkdir d0/d1/d17/d39/d40/d97 0 2024-05-15T15:54:10.849 INFO:tasks.workunit.client.1.smithi144.stdout:9/431: fsync d0/d8/d21/f2a 0 2024-05-15T15:54:10.851 INFO:tasks.workunit.client.1.smithi144.stdout:9/432: write d0/d8/d1b/d79/d8a/f97 [631846,105870] 0 2024-05-15T15:54:10.856 INFO:tasks.workunit.client.1.smithi144.stdout:5/438: link d0/f49 d0/dc/d12/d31/d35/f89 0 2024-05-15T15:54:10.860 INFO:tasks.workunit.client.1.smithi144.stdout:0/428: creat d4/d2d/d48/d5b/d8e/f97 x:0 0 0 2024-05-15T15:54:10.864 INFO:tasks.workunit.client.1.smithi144.stdout:6/398: link d7/db/d35/d84/d81/f83 d7/db/d11/d3c/f90 0 2024-05-15T15:54:10.867 INFO:tasks.workunit.client.1.smithi144.stdout:1/442: rmdir d7/d1d/d33/d41/d78 39 2024-05-15T15:54:10.868 INFO:tasks.workunit.client.0.smithi049.stdout:1/517: dread d0/d4/f24 [0,4194304] 0 2024-05-15T15:54:10.873 INFO:tasks.workunit.client.1.smithi144.stdout:7/462: write d4/df/d25/d40/f4d [470524,113213] 0 2024-05-15T15:54:10.874 INFO:tasks.workunit.client.1.smithi144.stdout:3/441: dread - d0/d1/d17/f20 zero size 2024-05-15T15:54:10.877 INFO:tasks.workunit.client.1.smithi144.stdout:9/433: mknod d0/d8/d1b/d62/d7d/d8d/c9e 0 2024-05-15T15:54:10.878 INFO:tasks.workunit.client.1.smithi144.stdout:2/423: dwrite d3/d7/f61 [0,4194304] 0 2024-05-15T15:54:10.880 INFO:tasks.workunit.client.1.smithi144.stdout:0/429: unlink d4/d6/d3a/d5d/f61 0 2024-05-15T15:54:10.882 INFO:tasks.workunit.client.1.smithi144.stdout:6/399: fsync d7/db/d35/d84/f8f 0 2024-05-15T15:54:10.883 INFO:tasks.workunit.client.1.smithi144.stdout:8/478: getdents d2/d3a/d7b/d4b/d5b/d6e/d8d 0 2024-05-15T15:54:10.885 INFO:tasks.workunit.client.1.smithi144.stdout:1/443: write d7/d1d/d33/d41/f19 [2412750,75211] 0 2024-05-15T15:54:10.885 INFO:tasks.workunit.client.0.smithi049.stdout:6/419: write dd/d26/f4f [134781,110569] 0 2024-05-15T15:54:10.885 INFO:tasks.workunit.client.1.smithi144.stdout:1/444: chown d7/d1d/d33/d1e 18024578 1 2024-05-15T15:54:10.888 INFO:tasks.workunit.client.0.smithi049.stdout:6/420: truncate dd/d17/d24/f27 5301089 0 2024-05-15T15:54:10.888 INFO:tasks.workunit.client.0.smithi049.stdout:1/518: creat d0/d4/dd/de/d1e/d2e/dad/fbb x:0 0 0 2024-05-15T15:54:10.888 INFO:tasks.workunit.client.0.smithi049.stdout:2/443: getdents d3/db/d1d/d1e/d2b/d3b/d47 0 2024-05-15T15:54:10.890 INFO:tasks.workunit.client.1.smithi144.stdout:3/442: mkdir d0/d1/d56/d98 0 2024-05-15T15:54:10.890 INFO:tasks.workunit.client.1.smithi144.stdout:3/443: chown d0/d1/d56/d98 487 1 2024-05-15T15:54:10.890 INFO:tasks.workunit.client.0.smithi049.stdout:1/519: write d0/d4/dd/de/d4e/f76 [4442732,31257] 0 2024-05-15T15:54:10.891 INFO:tasks.workunit.client.0.smithi049.stdout:2/444: truncate d3/da/d15/d73/f8a 675475 0 2024-05-15T15:54:10.895 INFO:tasks.workunit.client.1.smithi144.stdout:2/424: write d3/f5 [960238,4842] 0 2024-05-15T15:54:10.899 INFO:tasks.workunit.client.1.smithi144.stdout:5/439: creat d0/dc/d12/d31/d76/f8a x:0 0 0 2024-05-15T15:54:10.901 INFO:tasks.workunit.client.0.smithi049.stdout:2/445: dwrite d3/db/d1d/d1e/d2b/d2e/f87 [0,4194304] 0 2024-05-15T15:54:10.904 INFO:tasks.workunit.client.1.smithi144.stdout:5/440: dwrite d0/dc/d12/f18 [4194304,4194304] 0 2024-05-15T15:54:10.904 INFO:tasks.workunit.client.0.smithi049.stdout:8/449: dread d2/de/d71/d86/d5d/d7d/f72 [0,4194304] 0 2024-05-15T15:54:10.905 INFO:tasks.workunit.client.1.smithi144.stdout:5/441: write d0/dc/d12/f18 [1877936,85819] 0 2024-05-15T15:54:10.909 INFO:tasks.workunit.client.0.smithi049.stdout:1/520: symlink d0/d4/dd/de/d1e/d2e/d3e/d90/db3/lbc 0 2024-05-15T15:54:10.911 INFO:tasks.workunit.client.0.smithi049.stdout:2/446: dwrite d3/db/d1d/d1e/d2b/d2e/f6c [0,4194304] 0 2024-05-15T15:54:10.912 INFO:tasks.workunit.client.1.smithi144.stdout:8/479: mkdir d2/de/d36/d58/d9c 0 2024-05-15T15:54:10.913 INFO:tasks.workunit.client.1.smithi144.stdout:8/480: chown d2/d3a/d7b/d4b/d5b/d6e/d71/d73/d98 99088458 1 2024-05-15T15:54:10.914 INFO:tasks.workunit.client.0.smithi049.stdout:8/450: mkdir d2/de/d25/d8c 0 2024-05-15T15:54:10.916 INFO:tasks.workunit.client.1.smithi144.stdout:4/458: getdents d6/d42/d46/d4b 0 2024-05-15T15:54:10.920 INFO:tasks.workunit.client.0.smithi049.stdout:1/521: symlink d0/d4/dd/de/d4d/lbd 0 2024-05-15T15:54:10.922 INFO:tasks.workunit.client.1.smithi144.stdout:7/463: mkdir d4/d41/d9c 0 2024-05-15T15:54:10.927 INFO:tasks.workunit.client.1.smithi144.stdout:3/444: rename d0/d1/d17/d39/f3c to d0/d1/d6c/f99 0 2024-05-15T15:54:10.927 INFO:tasks.workunit.client.1.smithi144.stdout:7/464: dwrite d4/df/d25/d40/f4d [4194304,4194304] 0 2024-05-15T15:54:10.928 INFO:tasks.workunit.client.1.smithi144.stdout:3/445: chown d0/l1b 106 1 2024-05-15T15:54:10.928 INFO:tasks.workunit.client.1.smithi144.stdout:7/465: chown d4/df/d25/d38/d5d/c69 1307 1 2024-05-15T15:54:10.929 INFO:tasks.workunit.client.0.smithi049.stdout:1/522: dwrite d0/d4/dd/de/d1e/d16/d8a/f9b [0,4194304] 0 2024-05-15T15:54:10.934 INFO:tasks.workunit.client.1.smithi144.stdout:9/434: creat d0/d8/d5e/d9b/f9f x:0 0 0 2024-05-15T15:54:10.935 INFO:tasks.workunit.client.1.smithi144.stdout:3/446: dwrite d0/d11/d24/f88 [0,4194304] 0 2024-05-15T15:54:10.936 INFO:tasks.workunit.client.1.smithi144.stdout:3/447: stat d0/d11/c4b 0 2024-05-15T15:54:10.941 INFO:tasks.workunit.client.0.smithi049.stdout:2/447: write d3/db/d1d/f20 [1566748,108474] 0 2024-05-15T15:54:10.943 INFO:tasks.workunit.client.1.smithi144.stdout:0/430: mknod d4/df/d1f/d24/d96/c98 0 2024-05-15T15:54:10.944 INFO:tasks.workunit.client.1.smithi144.stdout:0/431: chown d4/d6/d3a/d4c/l57 930 1 2024-05-15T15:54:10.944 INFO:tasks.workunit.client.0.smithi049.stdout:8/451: creat d2/de/d71/d86/f8d x:0 0 0 2024-05-15T15:54:10.945 INFO:tasks.workunit.client.1.smithi144.stdout:0/432: dread d4/df/d1f/d24/f4b [0,4194304] 0 2024-05-15T15:54:10.948 INFO:tasks.workunit.client.1.smithi144.stdout:6/400: link d7/d1e/d28/f3d d7/db/d3a/f91 0 2024-05-15T15:54:10.950 INFO:tasks.workunit.client.0.smithi049.stdout:2/448: creat d3/db/d5b/d63/f8b x:0 0 0 2024-05-15T15:54:10.952 INFO:tasks.workunit.client.0.smithi049.stdout:8/452: dwrite d2/de/d71/d86/d5d/d7d/f43 [0,4194304] 0 2024-05-15T15:54:10.953 INFO:tasks.workunit.client.1.smithi144.stdout:4/459: mkdir d6/dd/d2e/d94 0 2024-05-15T15:54:10.954 INFO:tasks.workunit.client.1.smithi144.stdout:4/460: chown d6/d49 2 1 2024-05-15T15:54:10.955 INFO:tasks.workunit.client.0.smithi049.stdout:1/523: link d0/d4/dd/de/d1e/l74 d0/d4/dd/de/d1e/db7/lbe 0 2024-05-15T15:54:10.955 INFO:tasks.workunit.client.1.smithi144.stdout:5/442: dwrite d0/dc/d12/f13 [0,4194304] 0 2024-05-15T15:54:10.957 INFO:tasks.workunit.client.1.smithi144.stdout:1/445: mknod d7/df/d89/c94 0 2024-05-15T15:54:10.957 INFO:tasks.workunit.client.1.smithi144.stdout:1/446: readlink d7/d31/l39 0 2024-05-15T15:54:10.961 INFO:tasks.workunit.client.0.smithi049.stdout:8/453: rename d2/de/f53 to d2/de/d71/d86/d5d/d7d/d1b/d68/f8e 0 2024-05-15T15:54:10.963 INFO:tasks.workunit.client.0.smithi049.stdout:1/524: fsync d0/d4/dd/f96 0 2024-05-15T15:54:10.966 INFO:tasks.workunit.client.0.smithi049.stdout:2/449: mkdir d3/d78/d8c 0 2024-05-15T15:54:10.967 INFO:tasks.workunit.client.0.smithi049.stdout:2/450: chown d3/db/d1d/d1e/d2b 70 1 2024-05-15T15:54:10.971 INFO:tasks.workunit.client.0.smithi049.stdout:8/454: chown d2/de/d71/d86/d5d/d7d/c3b 48 1 2024-05-15T15:54:10.973 INFO:tasks.workunit.client.1.smithi144.stdout:9/435: mknod d0/d8/d5e/ca0 0 2024-05-15T15:54:10.975 INFO:tasks.workunit.client.0.smithi049.stdout:2/451: dwrite d3/db/d1d/f20 [0,4194304] 0 2024-05-15T15:54:10.978 INFO:tasks.workunit.client.0.smithi049.stdout:2/452: read d3/db/f23 [1194697,24466] 0 2024-05-15T15:54:10.981 INFO:tasks.workunit.client.0.smithi049.stdout:8/455: read d2/f3 [4157214,3755] 0 2024-05-15T15:54:10.982 INFO:tasks.workunit.client.0.smithi049.stdout:8/456: fdatasync d2/de/d71/d86/d5d/d63/f6e 0 2024-05-15T15:54:10.983 INFO:tasks.workunit.client.0.smithi049.stdout:1/525: mkdir d0/d4/dd/de/d1e/d16/d6e/dbf 0 2024-05-15T15:54:10.983 INFO:tasks.workunit.client.1.smithi144.stdout:6/401: rmdir d7/db/d11/d3c/d61 39 2024-05-15T15:54:10.985 INFO:tasks.workunit.client.1.smithi144.stdout:4/461: creat d6/f95 x:0 0 0 2024-05-15T15:54:10.987 INFO:tasks.workunit.client.0.smithi049.stdout:8/457: chown d2/f22 6454292 1 2024-05-15T15:54:10.988 INFO:tasks.workunit.client.1.smithi144.stdout:1/447: creat d7/d1d/d33/d29/d6b/d5e/f95 x:0 0 0 2024-05-15T15:54:10.989 INFO:tasks.workunit.client.1.smithi144.stdout:7/466: mkdir d4/d3c/d76/d9d 0 2024-05-15T15:54:10.989 INFO:tasks.workunit.client.0.smithi049.stdout:2/453: write d3/db/d1d/d1e/d2b/d3b/d47/f86 [1289733,27598] 0 2024-05-15T15:54:10.990 INFO:tasks.workunit.client.1.smithi144.stdout:9/436: write d0/d8/d5e/d44/f26 [2160632,67265] 0 2024-05-15T15:54:10.993 INFO:tasks.workunit.client.0.smithi049.stdout:1/526: symlink d0/d4/lc0 0 2024-05-15T15:54:10.995 INFO:tasks.workunit.client.0.smithi049.stdout:2/454: chown d3/d8/lf 425 1 2024-05-15T15:54:10.995 INFO:tasks.workunit.client.0.smithi049.stdout:2/455: fdatasync d3/db/f12 0 2024-05-15T15:54:10.995 INFO:tasks.workunit.client.0.smithi049.stdout:8/458: truncate d2/de/d71/d86/d5d/d7d/d45/f8a 786770 0 2024-05-15T15:54:10.996 INFO:tasks.workunit.client.1.smithi144.stdout:6/402: symlink d7/db/l92 0 2024-05-15T15:54:10.997 INFO:tasks.workunit.client.1.smithi144.stdout:5/443: mknod d0/dc/d24/c8b 0 2024-05-15T15:54:10.998 INFO:tasks.workunit.client.0.smithi049.stdout:2/456: truncate d3/db/d1d/d1e/d2b/d6b/f80 1352265 0 2024-05-15T15:54:11.000 INFO:tasks.workunit.client.0.smithi049.stdout:8/459: creat d2/de/d71/d86/d5d/f8f x:0 0 0 2024-05-15T15:54:11.001 INFO:tasks.workunit.client.1.smithi144.stdout:7/467: creat d4/d3c/d76/f9e x:0 0 0 2024-05-15T15:54:11.002 INFO:tasks.workunit.client.0.smithi049.stdout:2/457: symlink d3/da/l8d 0 2024-05-15T15:54:11.002 INFO:tasks.workunit.client.0.smithi049.stdout:8/460: fdatasync d2/de/d71/d86/d5d/d7d/f43 0 2024-05-15T15:54:11.003 INFO:tasks.workunit.client.0.smithi049.stdout:1/527: dwrite d0/d4/dd/de/d1e/f37 [4194304,4194304] 0 2024-05-15T15:54:11.003 INFO:tasks.workunit.client.1.smithi144.stdout:3/448: creat d0/d11/f9a x:0 0 0 2024-05-15T15:54:11.004 INFO:tasks.workunit.client.1.smithi144.stdout:1/448: dwrite d7/d1d/f91 [0,4194304] 0 2024-05-15T15:54:11.004 INFO:tasks.workunit.client.0.smithi049.stdout:2/458: dread d3/db/f1a [0,4194304] 0 2024-05-15T15:54:11.006 INFO:tasks.workunit.client.1.smithi144.stdout:2/425: link d3/d7/d20/d3b/l72 d3/d7/d20/d22/d52/d75/l8e 0 2024-05-15T15:54:11.006 INFO:tasks.workunit.client.1.smithi144.stdout:2/426: chown d3/d7/d20/d22/d52/d75/f55 468510158 1 2024-05-15T15:54:11.007 INFO:tasks.workunit.client.0.smithi049.stdout:8/461: mknod d2/de/d27/c90 0 2024-05-15T15:54:11.008 INFO:tasks.workunit.client.0.smithi049.stdout:8/462: truncate d2/de/d71/f82 168261 0 2024-05-15T15:54:11.009 INFO:tasks.workunit.client.1.smithi144.stdout:2/427: chown d3/d7/d20/d3b/f57 478 1 2024-05-15T15:54:11.010 INFO:tasks.workunit.client.0.smithi049.stdout:8/463: write d2/de/d71/d86/d5d/f89 [4344957,24305] 0 2024-05-15T15:54:11.015 INFO:tasks.workunit.client.1.smithi144.stdout:6/403: symlink d7/db/d3a/l93 0 2024-05-15T15:54:11.017 INFO:tasks.workunit.client.0.smithi049.stdout:1/528: rmdir d0/d4 39 2024-05-15T15:54:11.020 INFO:tasks.workunit.client.1.smithi144.stdout:4/462: rename d6/dd/d34/d3b/c17 to d6/dd/d34/d3b/c96 0 2024-05-15T15:54:11.022 INFO:tasks.workunit.client.0.smithi049.stdout:8/464: symlink d2/de/d71/d86/d5d/l91 0 2024-05-15T15:54:11.023 INFO:tasks.workunit.client.1.smithi144.stdout:7/468: creat d4/d41/d5b/f9f x:0 0 0 2024-05-15T15:54:11.025 INFO:tasks.workunit.client.1.smithi144.stdout:9/437: creat d0/d8/d5e/d44/d99/fa1 x:0 0 0 2024-05-15T15:54:11.027 INFO:tasks.workunit.client.1.smithi144.stdout:1/449: creat d7/d1d/d33/d29/d6b/d5e/d74/f96 x:0 0 0 2024-05-15T15:54:11.029 INFO:tasks.workunit.client.1.smithi144.stdout:0/433: getdents d4/d6 0 2024-05-15T15:54:11.029 INFO:tasks.workunit.client.0.smithi049.stdout:8/465: symlink d2/de/d25/l92 0 2024-05-15T15:54:11.032 INFO:tasks.workunit.client.1.smithi144.stdout:6/404: rmdir d7/db/d3a 39 2024-05-15T15:54:11.034 INFO:tasks.workunit.client.1.smithi144.stdout:4/463: dread d6/dd/d2d/f54 [0,4194304] 0 2024-05-15T15:54:11.034 INFO:tasks.workunit.client.1.smithi144.stdout:4/464: dread - d6/dd/d34/d3b/d30/db/d1d/d40/f80 zero size 2024-05-15T15:54:11.035 INFO:tasks.workunit.client.0.smithi049.stdout:8/466: unlink d2/de/d27/d5b/f7a 0 2024-05-15T15:54:11.036 INFO:tasks.workunit.client.0.smithi049.stdout:8/467: write d2/de/d25/f73 [887900,104305] 0 2024-05-15T15:54:11.036 INFO:tasks.workunit.client.0.smithi049.stdout:2/459: getdents d3/db/d1d 0 2024-05-15T15:54:11.038 INFO:tasks.workunit.client.0.smithi049.stdout:8/468: write d2/f3d [530501,47353] 0 2024-05-15T15:54:11.038 INFO:tasks.workunit.client.0.smithi049.stdout:1/529: symlink d0/d4/dd/de/d1e/d16/d6e/dbf/lc1 0 2024-05-15T15:54:11.039 INFO:tasks.workunit.client.0.smithi049.stdout:8/469: dread - d2/de/d71/f84 zero size 2024-05-15T15:54:11.040 INFO:tasks.workunit.client.1.smithi144.stdout:1/450: mknod d7/d1d/d33/d29/d6b/c97 0 2024-05-15T15:54:11.041 INFO:tasks.workunit.client.0.smithi049.stdout:2/460: creat d3/db/d1d/d1e/d2b/d3b/d47/d59/f8e x:0 0 0 2024-05-15T15:54:11.042 INFO:tasks.workunit.client.1.smithi144.stdout:1/451: write d7/d1d/d33/d55/f8d [1029771,15380] 0 2024-05-15T15:54:11.042 INFO:tasks.workunit.client.1.smithi144.stdout:7/469: dwrite d4/df/d10/d48/d72/d82/f89 [0,4194304] 0 2024-05-15T15:54:11.043 INFO:tasks.workunit.client.0.smithi049.stdout:2/461: write d3/da/d15/d73/f7f [843619,75512] 0 2024-05-15T15:54:11.044 INFO:tasks.workunit.client.0.smithi049.stdout:2/462: read - d3/db/d1d/d1e/d2b/d3b/d42/f69 zero size 2024-05-15T15:54:11.045 INFO:tasks.workunit.client.0.smithi049.stdout:7/350: sync 2024-05-15T15:54:11.046 INFO:tasks.workunit.client.1.smithi144.stdout:2/428: rename d3/d7/f1a to d3/d7/d20/d89/d6e/f8f 0 2024-05-15T15:54:11.047 INFO:tasks.workunit.client.1.smithi144.stdout:5/444: creat d0/dc/f8c x:0 0 0 2024-05-15T15:54:11.048 INFO:tasks.workunit.client.0.smithi049.stdout:8/470: dread d2/f2c [0,4194304] 0 2024-05-15T15:54:11.048 INFO:tasks.workunit.client.1.smithi144.stdout:4/465: chown d6/d42/c83 506 1 2024-05-15T15:54:11.048 INFO:tasks.workunit.client.1.smithi144.stdout:2/429: write d3/d7/d20/d37/f46 [3309487,122236] 0 2024-05-15T15:54:11.050 INFO:tasks.workunit.client.1.smithi144.stdout:9/438: mkdir d0/d8/d5e/d44/da2 0 2024-05-15T15:54:11.050 INFO:tasks.workunit.client.1.smithi144.stdout:5/445: truncate d0/dc/d12/d31/d76/f8a 474111 0 2024-05-15T15:54:11.053 INFO:tasks.workunit.client.0.smithi049.stdout:8/471: write d2/de/d71/d86/d5d/d63/f6e [767234,67237] 0 2024-05-15T15:54:11.054 INFO:tasks.workunit.client.1.smithi144.stdout:8/481: sync 2024-05-15T15:54:11.054 INFO:tasks.workunit.client.0.smithi049.stdout:8/472: stat d2/de/d27/d5b/d60 0 2024-05-15T15:54:11.054 INFO:tasks.workunit.client.1.smithi144.stdout:5/446: chown d0/dc/d1f/d65/c6d 101 1 2024-05-15T15:54:11.055 INFO:tasks.workunit.client.1.smithi144.stdout:7/470: mkdir d4/d41/d5b/da0 0 2024-05-15T15:54:11.055 INFO:tasks.workunit.client.1.smithi144.stdout:7/471: chown d4/f27 4137 1 2024-05-15T15:54:11.055 INFO:tasks.workunit.client.0.smithi049.stdout:2/463: getdents d3/db/d1d/d1e 0 2024-05-15T15:54:11.056 INFO:tasks.workunit.client.1.smithi144.stdout:7/472: write d4/df/f5f [406804,114978] 0 2024-05-15T15:54:11.057 INFO:tasks.workunit.client.1.smithi144.stdout:7/473: chown d4/d41/f4b 11 1 2024-05-15T15:54:11.057 INFO:tasks.workunit.client.0.smithi049.stdout:8/473: rmdir d2/de/d71/d86/d5d/d7d/d1b/d69 39 2024-05-15T15:54:11.060 INFO:tasks.workunit.client.1.smithi144.stdout:2/430: read d3/f34 [1264300,2582] 0 2024-05-15T15:54:11.060 INFO:tasks.workunit.client.0.smithi049.stdout:2/464: mkdir d3/db/d1d/d1e/d2b/d3b/d42/d8f 0 2024-05-15T15:54:11.061 INFO:tasks.workunit.client.1.smithi144.stdout:2/431: write d3/d7/d20/f7c [804819,91755] 0 2024-05-15T15:54:11.061 INFO:tasks.workunit.client.0.smithi049.stdout:1/530: dread d0/d4/dd/de/d1e/f27 [0,4194304] 0 2024-05-15T15:54:11.065 INFO:tasks.workunit.client.1.smithi144.stdout:9/439: creat d0/d8/d1b/d62/fa3 x:0 0 0 2024-05-15T15:54:11.066 INFO:tasks.workunit.client.1.smithi144.stdout:9/440: fsync d0/d8/d5e/d44/f41 0 2024-05-15T15:54:11.067 INFO:tasks.workunit.client.0.smithi049.stdout:1/531: creat d0/d4/dd/de/d4d/d8b/fc2 x:0 0 0 2024-05-15T15:54:11.067 INFO:tasks.workunit.client.1.smithi144.stdout:9/441: write d0/f96 [526716,52157] 0 2024-05-15T15:54:11.067 INFO:tasks.workunit.client.0.smithi049.stdout:1/532: dread - d0/d4/dd/de/d48/d5c/f87 zero size 2024-05-15T15:54:11.068 INFO:tasks.workunit.client.1.smithi144.stdout:8/482: symlink d2/d3a/d7b/d4b/d5b/l9d 0 2024-05-15T15:54:11.071 INFO:tasks.workunit.client.0.smithi049.stdout:8/474: mkdir d2/de/d27/d5b/d93 0 2024-05-15T15:54:11.072 INFO:tasks.workunit.client.0.smithi049.stdout:8/475: dread - d2/de/d71/f84 zero size 2024-05-15T15:54:11.073 INFO:tasks.workunit.client.0.smithi049.stdout:1/533: fsync d0/d4/f9 0 2024-05-15T15:54:11.075 INFO:tasks.workunit.client.1.smithi144.stdout:2/432: dread d3/d7/d20/d22/d52/d75/f55 [0,4194304] 0 2024-05-15T15:54:11.075 INFO:tasks.workunit.client.0.smithi049.stdout:1/534: chown d0/d4/dd/de/d1e/d2e/f2c 429638 1 2024-05-15T15:54:11.076 INFO:tasks.workunit.client.1.smithi144.stdout:1/452: mknod d7/df/c98 0 2024-05-15T15:54:11.076 INFO:tasks.workunit.client.1.smithi144.stdout:2/433: readlink d3/d7/d54/l8a 0 2024-05-15T15:54:11.079 INFO:tasks.workunit.client.1.smithi144.stdout:7/474: symlink d4/df/d25/d40/d62/la1 0 2024-05-15T15:54:11.080 INFO:tasks.workunit.client.0.smithi049.stdout:8/476: truncate d2/de/d71/d86/d5d/d7d/f4c 758656 0 2024-05-15T15:54:11.080 INFO:tasks.workunit.client.1.smithi144.stdout:0/434: getdents d4/d6/d3a/d5d 0 2024-05-15T15:54:11.082 INFO:tasks.workunit.client.0.smithi049.stdout:8/477: chown d2/de/d71/d86/d5d/d7d/l50 0 1 2024-05-15T15:54:11.084 INFO:tasks.workunit.client.0.smithi049.stdout:2/465: link d3/db/d1d/d1e/d2b/d6b/c84 d3/db/d1d/d1e/d2b/d2e/d67/c90 0 2024-05-15T15:54:11.085 INFO:tasks.workunit.client.0.smithi049.stdout:7/351: dwrite d0/d34/f46 [0,4194304] 0 2024-05-15T15:54:11.086 INFO:tasks.workunit.client.1.smithi144.stdout:9/442: creat d0/d8/d5e/fa4 x:0 0 0 2024-05-15T15:54:11.086 INFO:tasks.workunit.client.0.smithi049.stdout:2/466: read d3/db/d5b/f70 [2568071,75026] 0 2024-05-15T15:54:11.087 INFO:tasks.workunit.client.1.smithi144.stdout:6/405: dwrite d7/d32/f50 [0,4194304] 0 2024-05-15T15:54:11.088 INFO:tasks.workunit.client.1.smithi144.stdout:9/443: write d0/d8/d5e/d44/d10/fd [4785395,109069] 0 2024-05-15T15:54:11.091 INFO:tasks.workunit.client.1.smithi144.stdout:6/406: dread d7/d32/f50 [0,4194304] 0 2024-05-15T15:54:11.093 INFO:tasks.workunit.client.1.smithi144.stdout:4/466: link d6/dd/d34/d3b/d30/db/d1d/f41 d6/dd/d34/d3b/d30/db/d1d/d40/f97 0 2024-05-15T15:54:11.094 INFO:tasks.workunit.client.0.smithi049.stdout:8/478: mkdir d2/de/d25/d74/d94 0 2024-05-15T15:54:11.096 INFO:tasks.workunit.client.1.smithi144.stdout:5/447: creat d0/dc/d12/d31/d35/d6b/d88/f8d x:0 0 0 2024-05-15T15:54:11.100 INFO:tasks.workunit.client.1.smithi144.stdout:4/467: dwrite d6/dd/d34/f5f [0,4194304] 0 2024-05-15T15:54:11.101 INFO:tasks.workunit.client.1.smithi144.stdout:3/449: sync 2024-05-15T15:54:11.102 INFO:tasks.workunit.client.0.smithi049.stdout:7/352: mkdir d0/d6/d23/d45/d6a 0 2024-05-15T15:54:11.103 INFO:tasks.workunit.client.1.smithi144.stdout:4/468: chown d6/d42/d46/d4b/d1e/d44/c65 6115 1 2024-05-15T15:54:11.104 INFO:tasks.workunit.client.0.smithi049.stdout:7/353: truncate d0/d2f/d5c/f64 863357 0 2024-05-15T15:54:11.104 INFO:tasks.workunit.client.1.smithi144.stdout:3/450: truncate d0/d1/d17/f91 692365 0 2024-05-15T15:54:11.105 INFO:tasks.workunit.client.0.smithi049.stdout:1/535: link d0/d4/dd/de/d4d/l52 d0/d4/dd/de/d48/d84/lc3 0 2024-05-15T15:54:11.105 INFO:tasks.workunit.client.0.smithi049.stdout:7/354: readlink d0/d2f/d5c/l66 0 2024-05-15T15:54:11.105 INFO:tasks.workunit.client.1.smithi144.stdout:3/451: truncate d0/d11/d3b/f6b 731393 0 2024-05-15T15:54:11.106 INFO:tasks.workunit.client.1.smithi144.stdout:3/452: dread - d0/d11/d3b/d93/f86 zero size 2024-05-15T15:54:11.107 INFO:tasks.workunit.client.0.smithi049.stdout:1/536: write d0/d4/dd/de/d1e/d2e/dad/d65/d72/f9d [934639,111542] 0 2024-05-15T15:54:11.107 INFO:tasks.workunit.client.1.smithi144.stdout:7/475: mkdir d4/df/d10/d48/d72/da2 0 2024-05-15T15:54:11.108 INFO:tasks.workunit.client.0.smithi049.stdout:1/537: dread - d0/d4/d81/f9f zero size 2024-05-15T15:54:11.109 INFO:tasks.workunit.client.0.smithi049.stdout:2/467: rename d3/db/d1d/d1e/d2b/d2e/d67/d71 to d3/d78/d8c/d91 0 2024-05-15T15:54:11.110 INFO:tasks.workunit.client.0.smithi049.stdout:2/468: readlink d3/da/l8d 0 2024-05-15T15:54:11.111 INFO:tasks.workunit.client.1.smithi144.stdout:8/483: write d2/d24/d2d/d30/f81 [336252,5281] 0 2024-05-15T15:54:11.117 INFO:tasks.workunit.client.1.smithi144.stdout:9/444: readlink d0/l15 0 2024-05-15T15:54:11.118 INFO:tasks.workunit.client.1.smithi144.stdout:9/445: write d0/f14 [2827138,101570] 0 2024-05-15T15:54:11.119 INFO:tasks.workunit.client.1.smithi144.stdout:9/446: fsync d0/d8/d1b/f6f 0 2024-05-15T15:54:11.120 INFO:tasks.workunit.client.0.smithi049.stdout:1/538: fdatasync d0/d4/dd/de/d1e/d2e/d3e/f62 0 2024-05-15T15:54:11.122 INFO:tasks.workunit.client.0.smithi049.stdout:1/539: write d0/d4/dd/de/d4e/f76 [2730497,70685] 0 2024-05-15T15:54:11.123 INFO:tasks.workunit.client.0.smithi049.stdout:8/479: write d2/de/d71/d86/d5d/d7d/f72 [341835,60545] 0 2024-05-15T15:54:11.124 INFO:tasks.workunit.client.0.smithi049.stdout:2/469: creat d3/db/d1d/d1e/d2b/d3b/d42/f92 x:0 0 0 2024-05-15T15:54:11.127 INFO:tasks.workunit.client.1.smithi144.stdout:5/448: dwrite d0/dc/d24/f37 [0,4194304] 0 2024-05-15T15:54:11.132 INFO:tasks.workunit.client.1.smithi144.stdout:2/434: rename d3/d7/d20/d22/d52/d75/l8e to d3/d7/d20/d89/d6e/l90 0 2024-05-15T15:54:11.133 INFO:tasks.workunit.client.1.smithi144.stdout:2/435: write d3/d7/f3d [462336,106433] 0 2024-05-15T15:54:11.134 INFO:tasks.workunit.client.0.smithi049.stdout:2/470: dwrite d3/db/d1d/d1e/d2b/d2e/f6c [0,4194304] 0 2024-05-15T15:54:11.136 INFO:tasks.workunit.client.0.smithi049.stdout:2/471: chown d3/da/c1b 365896481 1 2024-05-15T15:54:11.137 INFO:tasks.workunit.client.1.smithi144.stdout:1/453: write d7/d1d/d33/d55/f5b [4388760,21269] 0 2024-05-15T15:54:11.137 INFO:tasks.workunit.client.0.smithi049.stdout:2/472: fsync d3/f10 0 2024-05-15T15:54:11.139 INFO:tasks.workunit.client.1.smithi144.stdout:7/476: creat d4/d3c/fa3 x:0 0 0 2024-05-15T15:54:11.140 INFO:tasks.workunit.client.1.smithi144.stdout:7/477: chown d4/d41/d5b/f6f 184469870 1 2024-05-15T15:54:11.141 INFO:tasks.workunit.client.1.smithi144.stdout:7/478: chown d4/df/f16 448192 1 2024-05-15T15:54:11.143 INFO:tasks.workunit.client.1.smithi144.stdout:0/435: link d4/df/d1f/f2f d4/d6/d4a/d74/f99 0 2024-05-15T15:54:11.144 INFO:tasks.workunit.client.0.smithi049.stdout:8/480: mkdir d2/de/d95 0 2024-05-15T15:54:11.145 INFO:tasks.workunit.client.0.smithi049.stdout:1/540: dread d0/d4/dd/de/d4e/f7e [0,4194304] 0 2024-05-15T15:54:11.145 INFO:tasks.workunit.client.0.smithi049.stdout:8/481: fsync d2/de/d27/f40 0 2024-05-15T15:54:11.149 INFO:tasks.workunit.client.0.smithi049.stdout:7/355: creat d0/f6b x:0 0 0 2024-05-15T15:54:11.149 INFO:tasks.workunit.client.0.smithi049.stdout:3/480: sync 2024-05-15T15:54:11.149 INFO:tasks.workunit.client.0.smithi049.stdout:5/434: sync 2024-05-15T15:54:11.151 INFO:tasks.workunit.client.0.smithi049.stdout:8/482: dread d2/de/d71/d86/d5d/d7d/d1b/f36 [0,4194304] 0 2024-05-15T15:54:11.154 INFO:tasks.workunit.client.1.smithi144.stdout:9/447: unlink d0/d8/l9 0 2024-05-15T15:54:11.156 INFO:tasks.workunit.client.1.smithi144.stdout:9/448: write d0/d8/d5e/d44/d16/f6d [926401,54671] 0 2024-05-15T15:54:11.163 INFO:tasks.workunit.client.0.smithi049.stdout:7/356: symlink d0/d2f/l6c 0 2024-05-15T15:54:11.166 INFO:tasks.workunit.client.0.smithi049.stdout:3/481: dwrite d12/d14/f43 [4194304,4194304] 0 2024-05-15T15:54:11.167 INFO:tasks.workunit.client.0.smithi049.stdout:2/473: symlink d3/d85/l93 0 2024-05-15T15:54:11.168 INFO:tasks.workunit.client.0.smithi049.stdout:2/474: dread - d3/db/d1d/d1e/d2b/f75 zero size 2024-05-15T15:54:11.170 INFO:tasks.workunit.client.0.smithi049.stdout:2/475: chown d3/db/d1d/d1e 4 1 2024-05-15T15:54:11.172 INFO:tasks.workunit.client.0.smithi049.stdout:3/482: fsync d12/d26/f3b 0 2024-05-15T15:54:11.173 INFO:tasks.workunit.client.0.smithi049.stdout:1/541: dwrite d0/d4/dd/de/d36/f4c [0,4194304] 0 2024-05-15T15:54:11.174 INFO:tasks.workunit.client.0.smithi049.stdout:7/357: mknod d0/d6/d23/c6d 0 2024-05-15T15:54:11.175 INFO:tasks.workunit.client.0.smithi049.stdout:2/476: dread d3/db/d1d/d1e/d2b/d6b/f80 [0,4194304] 0 2024-05-15T15:54:11.176 INFO:tasks.workunit.client.0.smithi049.stdout:8/483: creat d2/de/d71/d86/f96 x:0 0 0 2024-05-15T15:54:11.178 INFO:tasks.workunit.client.0.smithi049.stdout:2/477: read - d3/db/d1d/d1e/d2b/d3b/d42/f69 zero size 2024-05-15T15:54:11.179 INFO:tasks.workunit.client.1.smithi144.stdout:3/453: creat d0/d11/d3b/d93/d72/d8f/f9b x:0 0 0 2024-05-15T15:54:11.187 INFO:tasks.workunit.client.0.smithi049.stdout:2/478: dread d3/db/d1d/d1e/f3c [0,4194304] 0 2024-05-15T15:54:11.189 INFO:tasks.workunit.client.0.smithi049.stdout:1/542: creat d0/d4/dd/de/d1e/d2e/d3e/d90/db3/fc4 x:0 0 0 2024-05-15T15:54:11.190 INFO:tasks.workunit.client.0.smithi049.stdout:3/483: dwrite d12/d14/f1e [0,4194304] 0 2024-05-15T15:54:11.196 INFO:tasks.workunit.client.0.smithi049.stdout:5/435: dread dc/d4a/f36 [0,4194304] 0 2024-05-15T15:54:11.198 INFO:tasks.workunit.client.0.smithi049.stdout:2/479: truncate d3/db/d1d/d1e/f21 2821209 0 2024-05-15T15:54:11.198 INFO:tasks.workunit.client.1.smithi144.stdout:1/454: symlink d7/d31/l99 0 2024-05-15T15:54:11.198 INFO:tasks.workunit.client.0.smithi049.stdout:2/480: fsync d3/db/d1d/d1e/d2b/f75 0 2024-05-15T15:54:11.200 INFO:tasks.workunit.client.1.smithi144.stdout:9/449: unlink d0/d8/d5e/f32 0 2024-05-15T15:54:11.202 INFO:tasks.workunit.client.1.smithi144.stdout:9/450: readlink d0/d8/l2b 0 2024-05-15T15:54:11.203 INFO:tasks.workunit.client.0.smithi049.stdout:1/543: truncate d0/d4/dd/de/d48/f4a 1053775 0 2024-05-15T15:54:11.204 INFO:tasks.workunit.client.1.smithi144.stdout:0/436: dread d4/df/f46 [0,4194304] 0 2024-05-15T15:54:11.204 INFO:tasks.workunit.client.0.smithi049.stdout:7/358: write d0/d2f/f3e [1369062,99492] 0 2024-05-15T15:54:11.206 INFO:tasks.workunit.client.1.smithi144.stdout:7/479: dwrite d4/d41/f49 [4194304,4194304] 0 2024-05-15T15:54:11.208 INFO:tasks.workunit.client.1.smithi144.stdout:6/407: link d7/db/d35/d84/d81/l8d d7/db/d11/d3c/d61/l94 0 2024-05-15T15:54:11.210 INFO:tasks.workunit.client.0.smithi049.stdout:2/481: mknod d3/db/d5b/d63/c94 0 2024-05-15T15:54:11.210 INFO:tasks.workunit.client.0.smithi049.stdout:1/544: dwrite d0/d4/d81/f97 [0,4194304] 0 2024-05-15T15:54:11.212 INFO:tasks.workunit.client.0.smithi049.stdout:2/482: write d3/d8/f36 [1333792,68145] 0 2024-05-15T15:54:11.217 INFO:tasks.workunit.client.1.smithi144.stdout:5/449: rename d0/dc/d12/d31/d76 to d0/dc/d24/d8e 0 2024-05-15T15:54:11.217 INFO:tasks.workunit.client.0.smithi049.stdout:8/484: write d2/de/d71/d86/d5d/d7d/d1b/d68/f8e [713829,53178] 0 2024-05-15T15:54:11.219 INFO:tasks.workunit.client.1.smithi144.stdout:3/454: mkdir d0/d1/d56/d98/d9c 0 2024-05-15T15:54:11.220 INFO:tasks.workunit.client.1.smithi144.stdout:3/455: chown d0/d1/d17/d39/d40/f50 3275 1 2024-05-15T15:54:11.221 INFO:tasks.workunit.client.0.smithi049.stdout:6/421: sync 2024-05-15T15:54:11.221 INFO:tasks.workunit.client.0.smithi049.stdout:0/448: sync 2024-05-15T15:54:11.221 INFO:tasks.workunit.client.0.smithi049.stdout:9/301: sync 2024-05-15T15:54:11.221 INFO:tasks.workunit.client.0.smithi049.stdout:4/454: sync 2024-05-15T15:54:11.223 INFO:tasks.workunit.client.1.smithi144.stdout:1/455: truncate d7/df/d24/f54 2078839 0 2024-05-15T15:54:11.225 INFO:tasks.workunit.client.1.smithi144.stdout:0/437: creat d4/d6/d3a/d4c/f9a x:0 0 0 2024-05-15T15:54:11.225 INFO:tasks.workunit.client.0.smithi049.stdout:5/436: read - dc/dd/d10/f6f zero size 2024-05-15T15:54:11.226 INFO:tasks.workunit.client.1.smithi144.stdout:0/438: chown d4/d2d 84805 1 2024-05-15T15:54:11.226 INFO:tasks.workunit.client.1.smithi144.stdout:3/456: dwrite d0/d11/d3b/f5b [0,4194304] 0 2024-05-15T15:54:11.227 INFO:tasks.workunit.client.0.smithi049.stdout:6/422: write dd/d17/d24/d2b/d36/f56 [624798,50876] 0 2024-05-15T15:54:11.227 INFO:tasks.workunit.client.1.smithi144.stdout:8/484: getdents d2 0 2024-05-15T15:54:11.227 INFO:tasks.workunit.client.0.smithi049.stdout:9/302: read d0/d8/d11/f16 [132612,90647] 0 2024-05-15T15:54:11.227 INFO:tasks.workunit.client.0.smithi049.stdout:8/485: dwrite d2/de/d71/d86/f8d [0,4194304] 0 2024-05-15T15:54:11.228 INFO:tasks.workunit.client.1.smithi144.stdout:0/439: write d4/d2d/f2e [5166735,43171] 0 2024-05-15T15:54:11.229 INFO:tasks.workunit.client.1.smithi144.stdout:0/440: dread - d4/d2d/d48/d5b/f75 zero size 2024-05-15T15:54:11.234 INFO:tasks.workunit.client.1.smithi144.stdout:4/469: sync 2024-05-15T15:54:11.235 INFO:tasks.workunit.client.1.smithi144.stdout:4/470: chown d6/dd/d2d/l72 643832 1 2024-05-15T15:54:11.236 INFO:tasks.workunit.client.1.smithi144.stdout:0/441: dwrite d4/d6/d3a/d4c/f9a [0,4194304] 0 2024-05-15T15:54:11.241 INFO:tasks.workunit.client.1.smithi144.stdout:0/442: dwrite d4/df/d14/f42 [4194304,4194304] 0 2024-05-15T15:54:11.276 INFO:tasks.workunit.client.1.smithi144.stdout:2/436: sync 2024-05-15T15:54:11.279 INFO:tasks.workunit.client.1.smithi144.stdout:2/437: dread d3/d7/d20/d3f/f53 [0,4194304] 0 2024-05-15T15:54:11.280 INFO:tasks.workunit.client.0.smithi049.stdout:6/423: chown dd/f60 11 1 2024-05-15T15:54:11.280 INFO:tasks.workunit.client.0.smithi049.stdout:9/303: fsync d0/d8/d21/f2f 0 2024-05-15T15:54:11.282 INFO:tasks.workunit.client.0.smithi049.stdout:2/483: symlink d3/da/d15/d24/l95 0 2024-05-15T15:54:11.289 INFO:tasks.workunit.client.1.smithi144.stdout:6/408: rename d7/db/d35/d84/f8f to d7/d62/f95 0 2024-05-15T15:54:11.291 INFO:tasks.workunit.client.0.smithi049.stdout:7/359: dread d0/d6/f1b [0,4194304] 0 2024-05-15T15:54:11.292 INFO:tasks.workunit.client.1.smithi144.stdout:5/450: symlink d0/dc/d24/d8e/l8f 0 2024-05-15T15:54:11.294 INFO:tasks.workunit.client.0.smithi049.stdout:4/455: creat d18/d1e/d63/f96 x:0 0 0 2024-05-15T15:54:11.301 INFO:tasks.workunit.client.1.smithi144.stdout:3/457: dread d0/d1/fe [0,4194304] 0 2024-05-15T15:54:11.302 INFO:tasks.workunit.client.0.smithi049.stdout:5/437: write dc/d4a/d31/d5c/f7d [2199685,25441] 0 2024-05-15T15:54:11.305 INFO:tasks.workunit.client.1.smithi144.stdout:8/485: fsync d2/de/d36/f48 0 2024-05-15T15:54:11.306 INFO:tasks.workunit.client.0.smithi049.stdout:0/449: write f3 [1191484,113874] 0 2024-05-15T15:54:11.310 INFO:tasks.workunit.client.0.smithi049.stdout:6/424: symlink dd/d17/d24/d2b/l73 0 2024-05-15T15:54:11.310 INFO:tasks.workunit.client.1.smithi144.stdout:8/486: dwrite d2/fd [4194304,4194304] 0 2024-05-15T15:54:11.311 INFO:tasks.workunit.client.0.smithi049.stdout:6/425: chown dd/d17/d24/d2b/c4e 199162512 1 2024-05-15T15:54:11.311 INFO:tasks.workunit.client.1.smithi144.stdout:8/487: write d2/d3a/d7b/d4b/d5b/d6e/f62 [1361246,63547] 0 2024-05-15T15:54:11.311 INFO:tasks.workunit.client.1.smithi144.stdout:4/471: symlink d6/dd/l98 0 2024-05-15T15:54:11.317 INFO:tasks.workunit.client.0.smithi049.stdout:3/484: rename d12/d26/d73/f8b to d12/d30/d42/f8e 0 2024-05-15T15:54:11.319 INFO:tasks.workunit.client.1.smithi144.stdout:0/443: unlink d4/df/f6d 0 2024-05-15T15:54:11.320 INFO:tasks.workunit.client.0.smithi049.stdout:7/360: creat d0/d2f/d5c/f6e x:0 0 0 2024-05-15T15:54:11.321 INFO:tasks.workunit.client.1.smithi144.stdout:1/456: dwrite d7/df/d24/f88 [0,4194304] 0 2024-05-15T15:54:11.322 INFO:tasks.workunit.client.0.smithi049.stdout:7/361: dread - d0/d6/f55 zero size 2024-05-15T15:54:11.324 INFO:tasks.workunit.client.0.smithi049.stdout:3/485: dwrite d12/f1d [0,4194304] 0 2024-05-15T15:54:11.326 INFO:tasks.workunit.client.1.smithi144.stdout:7/480: creat d4/d3c/d76/d9d/fa4 x:0 0 0 2024-05-15T15:54:11.328 INFO:tasks.workunit.client.1.smithi144.stdout:7/481: write d4/df/d25/f29 [2227829,70867] 0 2024-05-15T15:54:11.328 INFO:tasks.workunit.client.1.smithi144.stdout:7/482: chown d4/d41/d5b/da0 70365 1 2024-05-15T15:54:11.330 INFO:tasks.workunit.client.0.smithi049.stdout:2/484: write d3/db/d1d/d1e/d2b/f54 [1831683,16061] 0 2024-05-15T15:54:11.333 INFO:tasks.workunit.client.1.smithi144.stdout:6/409: creat d7/d1e/d28/f96 x:0 0 0 2024-05-15T15:54:11.334 INFO:tasks.workunit.client.0.smithi049.stdout:3/486: dwrite d12/d14/f1e [0,4194304] 0 2024-05-15T15:54:11.336 INFO:tasks.workunit.client.1.smithi144.stdout:9/451: creat d0/d8/d5e/d44/d3c/fa5 x:0 0 0 2024-05-15T15:54:11.337 INFO:tasks.workunit.client.1.smithi144.stdout:9/452: stat d0/d8/d1b/d62/d6b 0 2024-05-15T15:54:11.343 INFO:tasks.workunit.client.0.smithi049.stdout:8/486: rename d2/de/d71/d86/d5d/f62 to d2/de/d71/d86/d5d/d7d/d1b/d68/f97 0 2024-05-15T15:54:11.345 INFO:tasks.workunit.client.1.smithi144.stdout:3/458: rename d0/d1/f68 to d0/d1/d17/d39/d83/f9d 0 2024-05-15T15:54:11.346 INFO:tasks.workunit.client.0.smithi049.stdout:0/450: dread fa [0,4194304] 0 2024-05-15T15:54:11.347 INFO:tasks.workunit.client.0.smithi049.stdout:0/451: readlink dc/d13/d14/l42 0 2024-05-15T15:54:11.354 INFO:tasks.workunit.client.0.smithi049.stdout:3/487: creat d12/d30/d66/f8f x:0 0 0 2024-05-15T15:54:11.355 INFO:tasks.workunit.client.1.smithi144.stdout:8/488: mkdir d2/d3a/d7b/d4b/d5b/d9e 0 2024-05-15T15:54:11.356 INFO:tasks.workunit.client.1.smithi144.stdout:8/489: write d2/d24/f2b [1044085,37080] 0 2024-05-15T15:54:11.361 INFO:tasks.workunit.client.0.smithi049.stdout:6/426: rename dd/d17/d24/d2b/d36/d50/f55 to dd/d17/d24/f74 0 2024-05-15T15:54:11.363 INFO:tasks.workunit.client.0.smithi049.stdout:4/456: write d18/d1e/d2a/f43 [359062,26122] 0 2024-05-15T15:54:11.366 INFO:tasks.workunit.client.1.smithi144.stdout:0/444: mknod d4/d6/d3a/d6a/c9b 0 2024-05-15T15:54:11.367 INFO:tasks.workunit.client.1.smithi144.stdout:0/445: write d4/d6/d4a/d74/f79 [1286170,128762] 0 2024-05-15T15:54:11.368 INFO:tasks.workunit.client.0.smithi049.stdout:2/485: write d3/db/d1d/d1e/d2b/d3b/d47/d59/f66 [70202,76710] 0 2024-05-15T15:54:11.370 INFO:tasks.workunit.client.0.smithi049.stdout:2/486: stat d3/da/d15/f26 0 2024-05-15T15:54:11.371 INFO:tasks.workunit.client.1.smithi144.stdout:1/457: mknod d7/d1d/d33/d1e/d4e/c9a 0 2024-05-15T15:54:11.372 INFO:tasks.workunit.client.0.smithi049.stdout:5/438: creat dc/d4a/f8c x:0 0 0 2024-05-15T15:54:11.375 INFO:tasks.workunit.client.0.smithi049.stdout:3/488: rmdir d12/d30/d63 39 2024-05-15T15:54:11.376 INFO:tasks.workunit.client.1.smithi144.stdout:7/483: creat d4/d3c/d76/d9d/fa5 x:0 0 0 2024-05-15T15:54:11.377 INFO:tasks.workunit.client.1.smithi144.stdout:7/484: dread d4/df/d79/f9b [0,4194304] 0 2024-05-15T15:54:11.378 INFO:tasks.workunit.client.1.smithi144.stdout:7/485: truncate d4/df/d10/d48/f8b 491985 0 2024-05-15T15:54:11.379 INFO:tasks.workunit.client.1.smithi144.stdout:6/410: mkdir d7/db/d11/d3c/d56/d97 0 2024-05-15T15:54:11.381 INFO:tasks.workunit.client.0.smithi049.stdout:0/452: dwrite dc/f34 [0,4194304] 0 2024-05-15T15:54:11.382 INFO:tasks.workunit.client.1.smithi144.stdout:5/451: link d0/dc/d1f/d50/f58 d0/dc/d12/d31/d35/f90 0 2024-05-15T15:54:11.384 INFO:tasks.workunit.client.0.smithi049.stdout:8/487: unlink d2/de/d71/d86/d5d/d7d/d1b/d69/l78 0 2024-05-15T15:54:11.385 INFO:tasks.workunit.client.1.smithi144.stdout:9/453: mknod d0/d8/d1b/d62/d7d/d8d/ca6 0 2024-05-15T15:54:11.387 INFO:tasks.workunit.client.0.smithi049.stdout:4/457: truncate d18/f70 991150 0 2024-05-15T15:54:11.390 INFO:tasks.workunit.client.1.smithi144.stdout:3/459: mkdir d0/d1/d17/d39/d40/d4a/d9e 0 2024-05-15T15:54:11.393 INFO:tasks.workunit.client.0.smithi049.stdout:7/362: rename d0/d6/f55 to d0/d6/f6f 0 2024-05-15T15:54:11.394 INFO:tasks.workunit.client.0.smithi049.stdout:7/363: read - d0/d6/d1d/f59 zero size 2024-05-15T15:54:11.396 INFO:tasks.workunit.client.0.smithi049.stdout:7/364: write d0/d6/d1d/f59 [207786,55412] 0 2024-05-15T15:54:11.396 INFO:tasks.workunit.client.0.smithi049.stdout:2/487: creat d3/d78/d8c/f96 x:0 0 0 2024-05-15T15:54:11.398 INFO:tasks.workunit.client.0.smithi049.stdout:2/488: readlink d3/db/d1d/d1e/d2b/d3b/l89 0 2024-05-15T15:54:11.400 INFO:tasks.workunit.client.0.smithi049.stdout:2/489: write d3/d8/f36 [2107407,77873] 0 2024-05-15T15:54:11.401 INFO:tasks.workunit.client.1.smithi144.stdout:0/446: fdatasync d4/df/d1f/d24/f5e 0 2024-05-15T15:54:11.406 INFO:tasks.workunit.client.1.smithi144.stdout:7/486: symlink d4/df/d81/la6 0 2024-05-15T15:54:11.407 INFO:tasks.workunit.client.1.smithi144.stdout:7/487: write d4/df/d10/d48/d72/d82/f89 [2486835,89776] 0 2024-05-15T15:54:11.408 INFO:tasks.workunit.client.0.smithi049.stdout:4/458: truncate d18/d1e/d2a/d3b/f57 491435 0 2024-05-15T15:54:11.409 INFO:tasks.workunit.client.0.smithi049.stdout:4/459: chown d18/d1e/d2a/d3b/c40 1667277 1 2024-05-15T15:54:11.410 INFO:tasks.workunit.client.1.smithi144.stdout:6/411: creat d7/db/d35/d84/f98 x:0 0 0 2024-05-15T15:54:11.412 INFO:tasks.workunit.client.1.smithi144.stdout:3/460: mknod d0/c9f 0 2024-05-15T15:54:11.414 INFO:tasks.workunit.client.0.smithi049.stdout:7/365: creat d0/d2f/d5c/f70 x:0 0 0 2024-05-15T15:54:11.416 INFO:tasks.workunit.client.0.smithi049.stdout:5/439: mkdir dc/dd/d7b/d8d 0 2024-05-15T15:54:11.422 INFO:tasks.workunit.client.0.smithi049.stdout:6/427: creat dd/d17/d24/d58/f75 x:0 0 0 2024-05-15T15:54:11.423 INFO:tasks.workunit.client.1.smithi144.stdout:2/438: getdents d3/d7/d20/d22/d52/d75/d65 0 2024-05-15T15:54:11.425 INFO:tasks.workunit.client.0.smithi049.stdout:0/453: mknod dc/d13/d2b/d76/d8a/c8b 0 2024-05-15T15:54:11.425 INFO:tasks.workunit.client.0.smithi049.stdout:0/454: fdatasync dc/f34 0 2024-05-15T15:54:11.427 INFO:tasks.workunit.client.0.smithi049.stdout:4/460: mknod d18/d69/c97 0 2024-05-15T15:54:11.428 INFO:tasks.workunit.client.1.smithi144.stdout:4/472: getdents d6/d7c 0 2024-05-15T15:54:11.431 INFO:tasks.workunit.client.0.smithi049.stdout:3/489: rename d12/d14/d32/d64/f78 to d12/d30/f90 0 2024-05-15T15:54:11.439 INFO:tasks.workunit.client.1.smithi144.stdout:2/439: unlink d3/d7/d20/d3f/l51 0 2024-05-15T15:54:11.440 INFO:tasks.workunit.client.0.smithi049.stdout:8/488: link d2/de/d71/d86/d5d/d7d/l50 d2/de/d25/d8c/l98 0 2024-05-15T15:54:11.441 INFO:tasks.workunit.client.0.smithi049.stdout:8/489: readlink d2/de/l31 0 2024-05-15T15:54:11.441 INFO:tasks.workunit.client.1.smithi144.stdout:0/447: truncate d4/d6/d3a/d4c/f9a 3721804 0 2024-05-15T15:54:11.444 INFO:tasks.workunit.client.0.smithi049.stdout:0/455: creat dc/d13/d14/d25/d31/f8c x:0 0 0 2024-05-15T15:54:11.445 INFO:tasks.workunit.client.1.smithi144.stdout:6/412: symlink d7/d1e/l99 0 2024-05-15T15:54:11.446 INFO:tasks.workunit.client.0.smithi049.stdout:0/456: fsync dc/d13/d3d/f4c 0 2024-05-15T15:54:11.446 INFO:tasks.workunit.client.0.smithi049.stdout:4/461: mkdir d18/d85/d98 0 2024-05-15T15:54:11.449 INFO:tasks.workunit.client.0.smithi049.stdout:8/490: dwrite d2/de/d71/d86/d5d/d7d/d1b/d68/f8e [0,4194304] 0 2024-05-15T15:54:11.450 INFO:tasks.workunit.client.0.smithi049.stdout:0/457: dread dc/d13/d14/f46 [0,4194304] 0 2024-05-15T15:54:11.450 INFO:tasks.workunit.client.1.smithi144.stdout:4/473: mkdir d6/d42/d46/d4b/d1e/d99 0 2024-05-15T15:54:11.451 INFO:tasks.workunit.client.1.smithi144.stdout:7/488: truncate d4/df/d79/f9b 350034 0 2024-05-15T15:54:11.451 INFO:tasks.workunit.client.0.smithi049.stdout:8/491: write d2/de/d71/d86/d5d/d7d/d1b/d68/f66 [323331,25691] 0 2024-05-15T15:54:11.452 INFO:tasks.workunit.client.0.smithi049.stdout:7/366: symlink d0/d6/da/l71 0 2024-05-15T15:54:11.453 INFO:tasks.workunit.client.0.smithi049.stdout:7/367: chown d0/f6b 12503493 1 2024-05-15T15:54:11.454 INFO:tasks.workunit.client.1.smithi144.stdout:8/490: getdents d2/de/d36 0 2024-05-15T15:54:11.458 INFO:tasks.workunit.client.1.smithi144.stdout:1/458: rename d7/c26 to d7/d1d/d33/d41/c9b 0 2024-05-15T15:54:11.462 INFO:tasks.workunit.client.1.smithi144.stdout:1/459: dread d7/d1d/d33/d29/f2c [0,4194304] 0 2024-05-15T15:54:11.463 INFO:tasks.workunit.client.0.smithi049.stdout:3/490: unlink f8 0 2024-05-15T15:54:11.464 INFO:tasks.workunit.client.1.smithi144.stdout:0/448: mknod d4/d6/d3a/d40/c9c 0 2024-05-15T15:54:11.466 INFO:tasks.workunit.client.0.smithi049.stdout:6/428: creat dd/d17/d24/d2b/f76 x:0 0 0 2024-05-15T15:54:11.466 INFO:tasks.workunit.client.1.smithi144.stdout:1/460: dwrite d7/d1d/d33/d29/d6b/d5e/f95 [0,4194304] 0 2024-05-15T15:54:11.468 INFO:tasks.workunit.client.0.smithi049.stdout:6/429: read dd/d17/d24/d2b/d36/f56 [403743,104272] 0 2024-05-15T15:54:11.469 INFO:tasks.workunit.client.1.smithi144.stdout:6/413: dread d7/ff [0,4194304] 0 2024-05-15T15:54:11.470 INFO:tasks.workunit.client.0.smithi049.stdout:6/430: chown dd/d17/d24/d2b/c4e 10 1 2024-05-15T15:54:11.472 INFO:tasks.workunit.client.0.smithi049.stdout:6/431: write dd/d26/f40 [1463035,123867] 0 2024-05-15T15:54:11.473 INFO:tasks.workunit.client.0.smithi049.stdout:5/440: dwrite dc/f16 [0,4194304] 0 2024-05-15T15:54:11.479 INFO:tasks.workunit.client.1.smithi144.stdout:5/452: rename d0/dc/d2f/d34/l7d to d0/d7e/l91 0 2024-05-15T15:54:11.480 INFO:tasks.workunit.client.1.smithi144.stdout:8/491: dread d2/de/d3f/f40 [0,4194304] 0 2024-05-15T15:54:11.481 INFO:tasks.workunit.client.1.smithi144.stdout:2/440: mknod d3/d4d/c91 0 2024-05-15T15:54:11.482 INFO:tasks.workunit.client.0.smithi049.stdout:8/492: stat d2/de/d27/d5b/f88 0 2024-05-15T15:54:11.483 INFO:tasks.workunit.client.1.smithi144.stdout:5/453: write d0/dc/d12/f13 [1035449,67859] 0 2024-05-15T15:54:11.484 INFO:tasks.workunit.client.1.smithi144.stdout:2/441: fsync d3/d7/d20/d37/f3c 0 2024-05-15T15:54:11.484 INFO:tasks.workunit.client.1.smithi144.stdout:5/454: dread - d0/dc/f8c zero size 2024-05-15T15:54:11.485 INFO:tasks.workunit.client.0.smithi049.stdout:7/368: fdatasync d0/d6/d23/f4c 0 2024-05-15T15:54:11.486 INFO:tasks.workunit.client.1.smithi144.stdout:2/442: chown d3/d7/d20/d3f/c58 8926869 1 2024-05-15T15:54:11.488 INFO:tasks.workunit.client.1.smithi144.stdout:2/443: truncate d3/d7/f61 4335753 0 2024-05-15T15:54:11.488 INFO:tasks.workunit.client.1.smithi144.stdout:8/492: dwrite d2/de/d36/d58/f94 [0,4194304] 0 2024-05-15T15:54:11.489 INFO:tasks.workunit.client.0.smithi049.stdout:7/369: truncate d0/d6/da/f61 361133 0 2024-05-15T15:54:11.489 INFO:tasks.workunit.client.0.smithi049.stdout:7/370: stat d0/d6/d1d/f27 0 2024-05-15T15:54:11.489 INFO:tasks.workunit.client.1.smithi144.stdout:2/444: readlink d3/d7/d20/d22/d52/l60 0 2024-05-15T15:54:11.496 INFO:tasks.workunit.client.1.smithi144.stdout:2/445: dwrite d3/d7/f47 [4194304,4194304] 0 2024-05-15T15:54:11.502 INFO:tasks.workunit.client.1.smithi144.stdout:9/454: rename d0/d8/d5e/d44/d3c/d75/f86 to d0/d8/d1b/d62/d6b/fa7 0 2024-05-15T15:54:11.503 INFO:tasks.workunit.client.0.smithi049.stdout:0/458: mknod dc/d13/c8d 0 2024-05-15T15:54:11.504 INFO:tasks.workunit.client.0.smithi049.stdout:6/432: dread dd/d17/d24/f74 [0,4194304] 0 2024-05-15T15:54:11.505 INFO:tasks.workunit.client.1.smithi144.stdout:9/455: chown d0/d8/d5e/f9a 94 1 2024-05-15T15:54:11.505 INFO:tasks.workunit.client.1.smithi144.stdout:2/446: dwrite d3/f6c [0,4194304] 0 2024-05-15T15:54:11.506 INFO:tasks.workunit.client.0.smithi049.stdout:8/493: symlink d2/de/d71/d86/d5d/d7d/d1b/d68/l99 0 2024-05-15T15:54:11.509 INFO:tasks.workunit.client.0.smithi049.stdout:8/494: write d2/de/d71/d86/d5d/f8f [7850,100596] 0 2024-05-15T15:54:11.510 INFO:tasks.workunit.client.0.smithi049.stdout:6/433: dread f8 [0,4194304] 0 2024-05-15T15:54:11.512 INFO:tasks.workunit.client.1.smithi144.stdout:1/461: mkdir d7/d1d/d33/d1e/d34/d9c 0 2024-05-15T15:54:11.514 INFO:tasks.workunit.client.1.smithi144.stdout:8/493: creat d2/de/d3f/f9f x:0 0 0 2024-05-15T15:54:11.514 INFO:tasks.workunit.client.0.smithi049.stdout:5/441: dread dc/d4a/d42/f5e [0,4194304] 0 2024-05-15T15:54:11.516 INFO:tasks.workunit.client.1.smithi144.stdout:8/494: read d2/d24/d2d/d30/f81 [104409,38767] 0 2024-05-15T15:54:11.518 INFO:tasks.workunit.client.1.smithi144.stdout:1/462: dwrite d7/d1d/d33/d29/d6b/f65 [0,4194304] 0 2024-05-15T15:54:11.518 INFO:tasks.workunit.client.1.smithi144.stdout:1/463: chown d7/d1d/f8f 61658854 1 2024-05-15T15:54:11.520 INFO:tasks.workunit.client.1.smithi144.stdout:6/414: creat d7/db/d11/d3c/d56/d97/f9a x:0 0 0 2024-05-15T15:54:11.522 INFO:tasks.workunit.client.0.smithi049.stdout:7/371: mknod d0/d6/d23/d45/d6a/c72 0 2024-05-15T15:54:11.525 INFO:tasks.workunit.client.0.smithi049.stdout:4/462: dwrite d18/d21/f56 [0,4194304] 0 2024-05-15T15:54:11.528 INFO:tasks.workunit.client.0.smithi049.stdout:3/491: link d12/d30/f6f d12/d30/d63/d68/f91 0 2024-05-15T15:54:11.529 INFO:tasks.workunit.client.0.smithi049.stdout:3/492: chown d12/d14/d52/f7b 1 1 2024-05-15T15:54:11.529 INFO:tasks.workunit.client.0.smithi049.stdout:3/493: readlink d12/d30/d63/l74 0 2024-05-15T15:54:11.530 INFO:tasks.workunit.client.0.smithi049.stdout:6/434: symlink dd/d17/d24/d58/l77 0 2024-05-15T15:54:11.533 INFO:tasks.workunit.client.1.smithi144.stdout:9/456: dread d0/d8/d50/f69 [0,4194304] 0 2024-05-15T15:54:11.534 INFO:tasks.workunit.client.1.smithi144.stdout:9/457: dread - d0/d8/d5e/d44/d3c/fa5 zero size 2024-05-15T15:54:11.534 INFO:tasks.workunit.client.0.smithi049.stdout:8/495: rename d2/de/d25/d8c/l98 to d2/de/d71/d86/d5d/l9a 0 2024-05-15T15:54:11.536 INFO:tasks.workunit.client.1.smithi144.stdout:3/461: rename d0/d11/d3b/d93/d72 to d0/d11/d3b/da0 0 2024-05-15T15:54:11.537 INFO:tasks.workunit.client.1.smithi144.stdout:3/462: stat d0/d1/c51 0 2024-05-15T15:54:11.541 INFO:tasks.workunit.client.1.smithi144.stdout:5/455: symlink d0/dc/d2f/d79/l92 0 2024-05-15T15:54:11.542 INFO:tasks.workunit.client.1.smithi144.stdout:0/449: link d4/d6/d3a/d4c/c73 d4/df/d1f/d24/d96/c9d 0 2024-05-15T15:54:11.543 INFO:tasks.workunit.client.1.smithi144.stdout:4/474: getdents d6/d7e 0 2024-05-15T15:54:11.547 INFO:tasks.workunit.client.0.smithi049.stdout:7/372: mknod d0/d6/d1d/c73 0 2024-05-15T15:54:11.548 INFO:tasks.workunit.client.1.smithi144.stdout:1/464: rmdir d7/d1d/d33/d1e/d87 39 2024-05-15T15:54:11.549 INFO:tasks.workunit.client.1.smithi144.stdout:4/475: dwrite d6/dd/d34/d3b/d30/f52 [0,4194304] 0 2024-05-15T15:54:11.552 INFO:tasks.workunit.client.1.smithi144.stdout:6/415: creat d7/db/d11/d3c/d61/f9b x:0 0 0 2024-05-15T15:54:11.553 INFO:tasks.workunit.client.0.smithi049.stdout:3/494: fsync d12/d30/d42/f6e 0 2024-05-15T15:54:11.555 INFO:tasks.workunit.client.1.smithi144.stdout:7/489: getdents d4/df/d25 0 2024-05-15T15:54:11.555 INFO:tasks.workunit.client.0.smithi049.stdout:6/435: mknod dd/d26/c78 0 2024-05-15T15:54:11.556 INFO:tasks.workunit.client.1.smithi144.stdout:7/490: write d4/df/f11 [34164,25376] 0 2024-05-15T15:54:11.556 INFO:tasks.workunit.client.1.smithi144.stdout:7/491: readlink d4/l26 0 2024-05-15T15:54:11.557 INFO:tasks.workunit.client.1.smithi144.stdout:7/492: write d4/f2f [7259,57717] 0 2024-05-15T15:54:11.558 INFO:tasks.workunit.client.1.smithi144.stdout:9/458: symlink d0/d8/d1b/d79/d8a/la8 0 2024-05-15T15:54:11.558 INFO:tasks.workunit.client.1.smithi144.stdout:7/493: fdatasync d4/df/d25/d38/d68/f8d 0 2024-05-15T15:54:11.559 INFO:tasks.workunit.client.1.smithi144.stdout:7/494: write d4/df/d10/d48/d72/d82/f89 [1568261,90105] 0 2024-05-15T15:54:11.560 INFO:tasks.workunit.client.1.smithi144.stdout:2/447: rename d3/d10/l32 to d3/d7/d20/d3b/l92 0 2024-05-15T15:54:11.562 INFO:tasks.workunit.client.0.smithi049.stdout:8/496: creat d2/de/d27/d5b/d60/f9b x:0 0 0 2024-05-15T15:54:11.564 INFO:tasks.workunit.client.1.smithi144.stdout:3/463: mkdir d0/d1/d17/da1 0 2024-05-15T15:54:11.564 INFO:tasks.workunit.client.1.smithi144.stdout:9/459: dwrite d0/d8/d5e/d44/d16/f85 [0,4194304] 0 2024-05-15T15:54:11.567 INFO:tasks.workunit.client.1.smithi144.stdout:0/450: unlink d4/d6/d3a/c41 0 2024-05-15T15:54:11.569 INFO:tasks.workunit.client.0.smithi049.stdout:3/495: fsync d12/d14/f1b 0 2024-05-15T15:54:11.570 INFO:tasks.workunit.client.1.smithi144.stdout:4/476: creat d6/d7e/f9a x:0 0 0 2024-05-15T15:54:11.573 INFO:tasks.workunit.client.1.smithi144.stdout:1/465: creat d7/d1d/d33/d29/d6b/d5e/f9d x:0 0 0 2024-05-15T15:54:11.573 INFO:tasks.workunit.client.1.smithi144.stdout:0/451: dwrite d4/df/d1f/d24/f4b [0,4194304] 0 2024-05-15T15:54:11.574 INFO:tasks.workunit.client.0.smithi049.stdout:8/497: mkdir d2/de/d25/d8c/d9c 0 2024-05-15T15:54:11.574 INFO:tasks.workunit.client.0.smithi049.stdout:0/459: truncate dc/d13/d14/d33/f53 807337 0 2024-05-15T15:54:11.576 INFO:tasks.workunit.client.1.smithi144.stdout:6/416: symlink d7/d1e/d28/l9c 0 2024-05-15T15:54:11.577 INFO:tasks.workunit.client.1.smithi144.stdout:6/417: stat d7/db/d11 0 2024-05-15T15:54:11.577 INFO:tasks.workunit.client.0.smithi049.stdout:0/460: chown dc/d13/c8d 523850 1 2024-05-15T15:54:11.577 INFO:tasks.workunit.client.1.smithi144.stdout:6/418: readlink d7/d1e/l5a 0 2024-05-15T15:54:11.578 INFO:tasks.workunit.client.1.smithi144.stdout:6/419: dread - d7/db/d11/d3c/f79 zero size 2024-05-15T15:54:11.579 INFO:tasks.workunit.client.0.smithi049.stdout:5/442: rename dc/dd/l1c to dc/dd/d39/l8e 0 2024-05-15T15:54:11.579 INFO:tasks.workunit.client.0.smithi049.stdout:6/436: dwrite dd/f53 [0,4194304] 0 2024-05-15T15:54:11.582 INFO:tasks.workunit.client.0.smithi049.stdout:6/437: stat dd/d17/d24/d58/l77 0 2024-05-15T15:54:11.582 INFO:tasks.workunit.client.0.smithi049.stdout:7/373: mknod d0/c74 0 2024-05-15T15:54:11.583 INFO:tasks.workunit.client.0.smithi049.stdout:4/463: write f16 [2848518,83913] 0 2024-05-15T15:54:11.587 INFO:tasks.workunit.client.0.smithi049.stdout:7/374: dread - d0/d34/f69 zero size 2024-05-15T15:54:11.589 INFO:tasks.workunit.client.0.smithi049.stdout:0/461: dwrite dc/d13/d14/d2f/f24 [4194304,4194304] 0 2024-05-15T15:54:11.591 INFO:tasks.workunit.client.0.smithi049.stdout:8/498: creat d2/de/d25/d74/f9d x:0 0 0 2024-05-15T15:54:11.592 INFO:tasks.workunit.client.0.smithi049.stdout:0/462: write dc/d13/d48/d57/f65 [1139249,72220] 0 2024-05-15T15:54:11.592 INFO:tasks.workunit.client.1.smithi144.stdout:7/495: write d4/fa [2299402,3339] 0 2024-05-15T15:54:11.597 INFO:tasks.workunit.client.0.smithi049.stdout:5/443: creat dc/dd/d7b/f8f x:0 0 0 2024-05-15T15:54:11.599 INFO:tasks.workunit.client.0.smithi049.stdout:4/464: rmdir d18/d1e/d2a/d36/d5d 39 2024-05-15T15:54:11.600 INFO:tasks.workunit.client.1.smithi144.stdout:2/448: creat d3/d7/d20/d22/d52/d75/d65/f93 x:0 0 0 2024-05-15T15:54:11.601 INFO:tasks.workunit.client.1.smithi144.stdout:2/449: chown d3/d7/d20/d22/d52/d75/f55 3073 1 2024-05-15T15:54:11.602 INFO:tasks.workunit.client.0.smithi049.stdout:6/438: creat dd/d17/d24/d58/d5f/f79 x:0 0 0 2024-05-15T15:54:11.602 INFO:tasks.workunit.client.1.smithi144.stdout:2/450: write d3/d7/d20/d3f/f67 [3742303,73328] 0 2024-05-15T15:54:11.603 INFO:tasks.workunit.client.0.smithi049.stdout:6/439: stat dd/l23 0 2024-05-15T15:54:11.608 INFO:tasks.workunit.client.0.smithi049.stdout:7/375: creat d0/d6/d23/d4a/f75 x:0 0 0 2024-05-15T15:54:11.608 INFO:tasks.workunit.client.1.smithi144.stdout:2/451: dwrite d3/d7/d20/d22/f2b [0,4194304] 0 2024-05-15T15:54:11.609 INFO:tasks.workunit.client.0.smithi049.stdout:7/376: fsync d0/d2f/d5c/f64 0 2024-05-15T15:54:11.614 INFO:tasks.workunit.client.1.smithi144.stdout:0/452: creat d4/d2d/f9e x:0 0 0 2024-05-15T15:54:11.614 INFO:tasks.workunit.client.1.smithi144.stdout:0/453: stat d4/df/d14/f42 0 2024-05-15T15:54:11.616 INFO:tasks.workunit.client.1.smithi144.stdout:7/496: creat d4/df/d10/d48/fa7 x:0 0 0 2024-05-15T15:54:11.617 INFO:tasks.workunit.client.0.smithi049.stdout:5/444: chown dc/dd/f1d 427 1 2024-05-15T15:54:11.617 INFO:tasks.workunit.client.1.smithi144.stdout:7/497: stat d4/d41/d9c 0 2024-05-15T15:54:11.618 INFO:tasks.workunit.client.0.smithi049.stdout:4/465: mkdir d18/d1e/d63/d99 0 2024-05-15T15:54:11.619 INFO:tasks.workunit.client.1.smithi144.stdout:1/466: dwrite d7/d1d/d33/d1e/f77 [0,4194304] 0 2024-05-15T15:54:11.620 INFO:tasks.workunit.client.0.smithi049.stdout:6/440: readlink dd/d17/d24/d2b/d36/d50/d54/l66 0 2024-05-15T15:54:11.621 INFO:tasks.workunit.client.0.smithi049.stdout:6/441: chown dd/d17/d24/d2b/f4d 201322476 1 2024-05-15T15:54:11.623 INFO:tasks.workunit.client.0.smithi049.stdout:3/496: link d12/d14/f43 d12/f92 0 2024-05-15T15:54:11.625 INFO:tasks.workunit.client.1.smithi144.stdout:4/477: write d6/dd/d2d/f54 [1021218,96271] 0 2024-05-15T15:54:11.626 INFO:tasks.workunit.client.1.smithi144.stdout:9/460: dwrite d0/d8/d5e/d44/d10/f2c [0,4194304] 0 2024-05-15T15:54:11.627 INFO:tasks.workunit.client.0.smithi049.stdout:7/377: fsync d0/d6/d23/f4c 0 2024-05-15T15:54:11.628 INFO:tasks.workunit.client.1.smithi144.stdout:4/478: dread - d6/dd/d34/d3b/d30/db/d1d/d40/f79 zero size 2024-05-15T15:54:11.628 INFO:tasks.workunit.client.1.smithi144.stdout:2/452: mknod d3/d7/d20/d3f/c94 0 2024-05-15T15:54:11.632 INFO:tasks.workunit.client.1.smithi144.stdout:0/454: symlink d4/d6/d4a/d74/l9f 0 2024-05-15T15:54:11.637 INFO:tasks.workunit.client.1.smithi144.stdout:8/495: rename d2/d3a/d7b/d4b/d5b to d2/d24/d2d/da0 0 2024-05-15T15:54:11.640 INFO:tasks.workunit.client.0.smithi049.stdout:6/442: symlink dd/d17/d24/d2b/l7a 0 2024-05-15T15:54:11.642 INFO:tasks.workunit.client.1.smithi144.stdout:3/464: truncate d0/d11/f2a 1208209 0 2024-05-15T15:54:11.645 INFO:tasks.workunit.client.1.smithi144.stdout:8/496: dread d2/d24/d2d/f5e [0,4194304] 0 2024-05-15T15:54:11.646 INFO:tasks.workunit.client.1.smithi144.stdout:9/461: creat d0/d8/d5e/d44/d10/d9c/fa9 x:0 0 0 2024-05-15T15:54:11.647 INFO:tasks.workunit.client.1.smithi144.stdout:8/497: readlink d2/d3a/d7b/d4b/l7a 0 2024-05-15T15:54:11.648 INFO:tasks.workunit.client.1.smithi144.stdout:0/455: dread d4/d6/d4a/d74/f99 [0,4194304] 0 2024-05-15T15:54:11.653 INFO:tasks.workunit.client.0.smithi049.stdout:3/497: creat d12/d14/d32/d64/f93 x:0 0 0 2024-05-15T15:54:11.654 INFO:tasks.workunit.client.0.smithi049.stdout:8/499: rename d2/de/c52 to d2/de/d71/d86/d5d/d7d/d1b/c9e 0 2024-05-15T15:54:11.654 INFO:tasks.workunit.client.1.smithi144.stdout:9/462: dwrite d0/d8/d5e/d44/f87 [0,4194304] 0 2024-05-15T15:54:11.655 INFO:tasks.workunit.client.0.smithi049.stdout:0/463: link dc/d6d/l59 dc/d13/d3d/d84/l8e 0 2024-05-15T15:54:11.656 INFO:tasks.workunit.client.1.smithi144.stdout:3/465: dread d0/d11/d3b/f45 [0,4194304] 0 2024-05-15T15:54:11.657 INFO:tasks.workunit.client.1.smithi144.stdout:5/456: rename d0/dc/d12/d31/d35 to d0/dc/d24/d8e/d93 0 2024-05-15T15:54:11.657 INFO:tasks.workunit.client.0.smithi049.stdout:8/500: dread d2/de/d71/f82 [0,4194304] 0 2024-05-15T15:54:11.658 INFO:tasks.workunit.client.0.smithi049.stdout:8/501: chown d2/de/l31 267350601 1 2024-05-15T15:54:11.659 INFO:tasks.workunit.client.0.smithi049.stdout:6/443: dwrite dd/f53 [0,4194304] 0 2024-05-15T15:54:11.662 INFO:tasks.workunit.client.1.smithi144.stdout:9/463: dwrite d0/d8/d5e/d44/f26 [0,4194304] 0 2024-05-15T15:54:11.680 INFO:tasks.workunit.client.1.smithi144.stdout:1/467: truncate d7/d1d/d33/d1e/d34/f38 923998 0 2024-05-15T15:54:11.683 INFO:tasks.workunit.client.1.smithi144.stdout:4/479: write d6/d42/f58 [3809910,108298] 0 2024-05-15T15:54:11.683 INFO:tasks.workunit.client.1.smithi144.stdout:2/453: write d3/d10/f2e [4133358,11186] 0 2024-05-15T15:54:11.684 INFO:tasks.workunit.client.1.smithi144.stdout:0/456: mkdir d4/df/da0 0 2024-05-15T15:54:11.686 INFO:tasks.workunit.client.1.smithi144.stdout:7/498: getdents d4/d41 0 2024-05-15T15:54:11.689 INFO:tasks.workunit.client.1.smithi144.stdout:6/420: rename d7/db/d11/d3c/d56 to d7/db/d9d 0 2024-05-15T15:54:11.689 INFO:tasks.workunit.client.1.smithi144.stdout:8/498: dwrite d2/de/d49/f69 [0,4194304] 0 2024-05-15T15:54:11.690 INFO:tasks.workunit.client.1.smithi144.stdout:5/457: symlink d0/dc/d12/d4a/l94 0 2024-05-15T15:54:11.692 INFO:tasks.workunit.client.1.smithi144.stdout:4/480: write d6/dd/d34/d3b/f21 [944228,98216] 0 2024-05-15T15:54:11.694 INFO:tasks.workunit.client.1.smithi144.stdout:2/454: rmdir d3/d7/d20/d89/d6e 39 2024-05-15T15:54:11.695 INFO:tasks.workunit.client.1.smithi144.stdout:7/499: dwrite d4/df/d25/f29 [0,4194304] 0 2024-05-15T15:54:11.698 INFO:tasks.workunit.client.1.smithi144.stdout:3/466: truncate d0/d1/d17/f91 36921 0 2024-05-15T15:54:11.699 INFO:tasks.workunit.client.1.smithi144.stdout:7/500: chown d4/d41/d5b/f91 14 1 2024-05-15T15:54:11.702 INFO:tasks.workunit.client.1.smithi144.stdout:6/421: rmdir d7/db/d35/d84 39 2024-05-15T15:54:11.705 INFO:tasks.workunit.client.1.smithi144.stdout:4/481: dwrite d6/dd/d34/d3b/d30/f52 [0,4194304] 0 2024-05-15T15:54:11.706 INFO:tasks.workunit.client.1.smithi144.stdout:5/458: unlink d0/dc/d12/f13 0 2024-05-15T15:54:11.707 INFO:tasks.workunit.client.1.smithi144.stdout:2/455: dwrite d3/d7/d20/d37/f46 [0,4194304] 0 2024-05-15T15:54:11.707 INFO:tasks.workunit.client.1.smithi144.stdout:4/482: dread - d6/d7e/f9a zero size 2024-05-15T15:54:11.708 INFO:tasks.workunit.client.1.smithi144.stdout:2/456: dread - d3/d7/f36 zero size 2024-05-15T15:54:11.711 INFO:tasks.workunit.client.1.smithi144.stdout:9/464: getdents d0/d8/d5e 0 2024-05-15T15:54:11.713 INFO:tasks.workunit.client.1.smithi144.stdout:0/457: mkdir d4/df/d1f/d24/d85/da1 0 2024-05-15T15:54:11.713 INFO:tasks.workunit.client.1.smithi144.stdout:2/457: read d3/d7/d20/d22/d52/d64/f31 [1086369,17157] 0 2024-05-15T15:54:11.715 INFO:tasks.workunit.client.1.smithi144.stdout:0/458: dread - d4/df/d1f/d24/d2b/f86 zero size 2024-05-15T15:54:11.717 INFO:tasks.workunit.client.1.smithi144.stdout:6/422: mknod d7/db/d11/d7e/c9e 0 2024-05-15T15:54:11.718 INFO:tasks.workunit.client.1.smithi144.stdout:5/459: mkdir d0/dc/d24/d8e/d95 0 2024-05-15T15:54:11.718 INFO:tasks.workunit.client.1.smithi144.stdout:0/459: truncate d4/df/d1f/d56/f93 644320 0 2024-05-15T15:54:11.718 INFO:tasks.workunit.client.1.smithi144.stdout:6/423: write d7/db/d11/f43 [628668,66659] 0 2024-05-15T15:54:11.719 INFO:tasks.workunit.client.1.smithi144.stdout:3/467: mknod d0/d1/d56/d71/d73/ca2 0 2024-05-15T15:54:11.720 INFO:tasks.workunit.client.1.smithi144.stdout:0/460: write d4/f22 [884273,127954] 0 2024-05-15T15:54:11.720 INFO:tasks.workunit.client.1.smithi144.stdout:5/460: read d0/dc/d12/f18 [2580429,88094] 0 2024-05-15T15:54:11.720 INFO:tasks.workunit.client.1.smithi144.stdout:9/465: read d0/d8/d5e/d44/d3c/f61 [3838335,21980] 0 2024-05-15T15:54:11.721 INFO:tasks.workunit.client.1.smithi144.stdout:8/499: creat d2/de/fa1 x:0 0 0 2024-05-15T15:54:11.721 INFO:tasks.workunit.client.1.smithi144.stdout:0/461: read d4/dc/f18 [1075051,46977] 0 2024-05-15T15:54:11.722 INFO:tasks.workunit.client.0.smithi049.stdout:1/545: sync 2024-05-15T15:54:11.723 INFO:tasks.workunit.client.1.smithi144.stdout:9/466: dread - d0/d8/d5e/d44/d16/f46 zero size 2024-05-15T15:54:11.723 INFO:tasks.workunit.client.1.smithi144.stdout:0/462: stat d4/d6/d3a/d4c/l4f 0 2024-05-15T15:54:11.723 INFO:tasks.workunit.client.0.smithi049.stdout:7/378: mkdir d0/d6/d23/d76 0 2024-05-15T15:54:11.724 INFO:tasks.workunit.client.0.smithi049.stdout:3/498: mknod d12/d67/c94 0 2024-05-15T15:54:11.725 INFO:tasks.workunit.client.1.smithi144.stdout:9/467: fdatasync d0/d8/d5e/d44/d16/f85 0 2024-05-15T15:54:11.726 INFO:tasks.workunit.client.1.smithi144.stdout:8/500: creat d2/d24/fa2 x:0 0 0 2024-05-15T15:54:11.728 INFO:tasks.workunit.client.0.smithi049.stdout:6/444: rmdir dd/d17/d24/d2b/d36/d50/d54 39 2024-05-15T15:54:11.728 INFO:tasks.workunit.client.0.smithi049.stdout:7/379: truncate d0/d2f/d5c/f70 474456 0 2024-05-15T15:54:11.729 INFO:tasks.workunit.client.1.smithi144.stdout:5/461: mkdir d0/dc/d24/d57/d96 0 2024-05-15T15:54:11.729 INFO:tasks.workunit.client.1.smithi144.stdout:3/468: dwrite d0/d1/d56/f87 [0,4194304] 0 2024-05-15T15:54:11.731 INFO:tasks.workunit.client.0.smithi049.stdout:6/445: dread dd/f4c [0,4194304] 0 2024-05-15T15:54:11.732 INFO:tasks.workunit.client.0.smithi049.stdout:8/502: dwrite d2/de/d25/d74/f9d [0,4194304] 0 2024-05-15T15:54:11.733 INFO:tasks.workunit.client.0.smithi049.stdout:1/546: mknod d0/d4/dd/de/d48/cc5 0 2024-05-15T15:54:11.735 INFO:tasks.workunit.client.0.smithi049.stdout:3/499: dwrite d12/d14/f1e [0,4194304] 0 2024-05-15T15:54:11.736 INFO:tasks.workunit.client.1.smithi144.stdout:3/469: read d0/d11/d3b/f6b [536942,68158] 0 2024-05-15T15:54:11.737 INFO:tasks.workunit.client.1.smithi144.stdout:0/463: dwrite d4/df/d14/f42 [4194304,4194304] 0 2024-05-15T15:54:11.738 INFO:tasks.workunit.client.0.smithi049.stdout:1/547: write d0/d4/dd/de/f13 [246680,92808] 0 2024-05-15T15:54:11.739 INFO:tasks.workunit.client.1.smithi144.stdout:8/501: dwrite d2/d24/f2b [0,4194304] 0 2024-05-15T15:54:11.744 INFO:tasks.workunit.client.1.smithi144.stdout:8/502: read d2/de/f14 [383597,62144] 0 2024-05-15T15:54:11.745 INFO:tasks.workunit.client.1.smithi144.stdout:8/503: readlink d2/d1a/l28 0 2024-05-15T15:54:11.747 INFO:tasks.workunit.client.0.smithi049.stdout:4/466: link d18/d1e/d2a/d50/l6f d18/d21/d2b/l9a 0 2024-05-15T15:54:11.751 INFO:tasks.workunit.client.1.smithi144.stdout:2/458: write d3/f9 [3456837,19095] 0 2024-05-15T15:54:11.753 INFO:tasks.workunit.client.0.smithi049.stdout:3/500: mkdir d12/d26/d73/d95 0 2024-05-15T15:54:11.756 INFO:tasks.workunit.client.0.smithi049.stdout:4/467: dwrite d18/d1e/d63/f96 [0,4194304] 0 2024-05-15T15:54:11.760 INFO:tasks.workunit.client.1.smithi144.stdout:8/504: creat d2/de/d36/d58/fa3 x:0 0 0 2024-05-15T15:54:11.764 INFO:tasks.workunit.client.0.smithi049.stdout:0/464: getdents dc 0 2024-05-15T15:54:11.765 INFO:tasks.workunit.client.0.smithi049.stdout:4/468: symlink d18/d76/l9b 0 2024-05-15T15:54:11.766 INFO:tasks.workunit.client.1.smithi144.stdout:8/505: dread d2/d24/d2d/d30/f44 [0,4194304] 0 2024-05-15T15:54:11.769 INFO:tasks.workunit.client.1.smithi144.stdout:0/464: unlink d4/d6/d3a/d40/c4e 0 2024-05-15T15:54:11.771 INFO:tasks.workunit.client.0.smithi049.stdout:1/548: creat d0/d4/dd/de/d48/d5c/fc6 x:0 0 0 2024-05-15T15:54:11.772 INFO:tasks.workunit.client.0.smithi049.stdout:4/469: creat d18/d1e/d2a/d3b/f9c x:0 0 0 2024-05-15T15:54:11.772 INFO:tasks.workunit.client.0.smithi049.stdout:1/549: chown d0/d4/dd/de/d36/f6a 862 1 2024-05-15T15:54:11.773 INFO:tasks.workunit.client.0.smithi049.stdout:4/470: stat d18/d1e/d2a/d36/f8f 0 2024-05-15T15:54:11.774 INFO:tasks.workunit.client.0.smithi049.stdout:3/501: creat d12/f96 x:0 0 0 2024-05-15T15:54:11.777 INFO:tasks.workunit.client.0.smithi049.stdout:0/465: dread f3 [0,4194304] 0 2024-05-15T15:54:11.778 INFO:tasks.workunit.client.1.smithi144.stdout:9/468: getdents d0/d8/d5e 0 2024-05-15T15:54:11.779 INFO:tasks.workunit.client.1.smithi144.stdout:9/469: write d0/d8/d1b/f6f [1446172,41718] 0 2024-05-15T15:54:11.779 INFO:tasks.workunit.client.0.smithi049.stdout:0/466: write dc/d13/d14/d25/d31/f75 [521368,8151] 0 2024-05-15T15:54:11.779 INFO:tasks.workunit.client.1.smithi144.stdout:9/470: readlink d0/l15 0 2024-05-15T15:54:11.780 INFO:tasks.workunit.client.1.smithi144.stdout:9/471: fdatasync d0/d8/d5e/d44/d10/f81 0 2024-05-15T15:54:11.781 INFO:tasks.workunit.client.1.smithi144.stdout:9/472: fdatasync d0/d8/d21/f82 0 2024-05-15T15:54:11.781 INFO:tasks.workunit.client.1.smithi144.stdout:3/470: getdents d0/d1/d6c 0 2024-05-15T15:54:11.781 INFO:tasks.workunit.client.0.smithi049.stdout:4/471: mknod d18/d85/d98/c9d 0 2024-05-15T15:54:11.782 INFO:tasks.workunit.client.1.smithi144.stdout:3/471: dread - d0/d1/d17/f84 zero size 2024-05-15T15:54:11.782 INFO:tasks.workunit.client.0.smithi049.stdout:3/502: creat d12/d26/d73/d95/f97 x:0 0 0 2024-05-15T15:54:11.783 INFO:tasks.workunit.client.1.smithi144.stdout:0/465: symlink d4/df/d1f/la2 0 2024-05-15T15:54:11.785 INFO:tasks.workunit.client.0.smithi049.stdout:0/467: mkdir dc/d13/d14/d2f/d8f 0 2024-05-15T15:54:11.786 INFO:tasks.workunit.client.0.smithi049.stdout:0/468: chown dc/d13/d14/f4e 1939 1 2024-05-15T15:54:11.787 INFO:tasks.workunit.client.0.smithi049.stdout:7/380: dread d0/d6/d23/f4c [0,4194304] 0 2024-05-15T15:54:11.789 INFO:tasks.workunit.client.0.smithi049.stdout:3/503: rename d12/d30/d63/l74 to d12/d26/d73/d95/l98 0 2024-05-15T15:54:11.790 INFO:tasks.workunit.client.1.smithi144.stdout:2/459: dread d3/d7/d20/d22/d52/d64/f26 [0,4194304] 0 2024-05-15T15:54:11.791 INFO:tasks.workunit.client.1.smithi144.stdout:9/473: rmdir d0/d8/d5e 39 2024-05-15T15:54:11.791 INFO:tasks.workunit.client.1.smithi144.stdout:9/474: chown d0/d8/d1b/d62/d6b/fa7 2241908 1 2024-05-15T15:54:11.792 INFO:tasks.workunit.client.0.smithi049.stdout:0/469: rename dc/d13/d14/f4e to dc/d13/d56/f90 0 2024-05-15T15:54:11.793 INFO:tasks.workunit.client.1.smithi144.stdout:5/462: truncate d0/dc/d24/d28/d2d/f2b 1548463 0 2024-05-15T15:54:11.794 INFO:tasks.workunit.client.1.smithi144.stdout:3/472: mknod d0/d1/d17/d39/d40/ca3 0 2024-05-15T15:54:11.799 INFO:tasks.workunit.client.1.smithi144.stdout:9/475: symlink d0/d8/d1b/d62/d7d/laa 0 2024-05-15T15:54:11.802 INFO:tasks.workunit.client.0.smithi049.stdout:1/550: dread d0/d4/dd/de/d36/f3f [0,4194304] 0 2024-05-15T15:54:11.803 INFO:tasks.workunit.client.0.smithi049.stdout:1/551: read - d0/d4/d81/f9f zero size 2024-05-15T15:54:11.804 INFO:tasks.workunit.client.1.smithi144.stdout:8/506: write d2/d24/d2d/d30/f81 [490343,50179] 0 2024-05-15T15:54:11.805 INFO:tasks.workunit.client.1.smithi144.stdout:3/473: rmdir d0/d1/d56/d71 39 2024-05-15T15:54:11.805 INFO:tasks.workunit.client.0.smithi049.stdout:0/470: dread dc/d13/f22 [0,4194304] 0 2024-05-15T15:54:11.807 INFO:tasks.workunit.client.0.smithi049.stdout:3/504: getdents d12 0 2024-05-15T15:54:11.808 INFO:tasks.workunit.client.1.smithi144.stdout:0/466: mkdir d4/df/d1f/d24/d85/da1/da3 0 2024-05-15T15:54:11.808 INFO:tasks.workunit.client.1.smithi144.stdout:0/467: chown d4/df/d1f/c3d 2 1 2024-05-15T15:54:11.808 INFO:tasks.workunit.client.1.smithi144.stdout:8/507: dread d2/d24/d2d/f54 [0,4194304] 0 2024-05-15T15:54:11.809 INFO:tasks.workunit.client.0.smithi049.stdout:4/472: dread d18/f3e [0,4194304] 0 2024-05-15T15:54:11.809 INFO:tasks.workunit.client.0.smithi049.stdout:3/505: chown d12/d30/l45 2 1 2024-05-15T15:54:11.810 INFO:tasks.workunit.client.0.smithi049.stdout:4/473: stat d18/d1e/d2a/d36/f83 0 2024-05-15T15:54:11.814 INFO:tasks.workunit.client.1.smithi144.stdout:0/468: dwrite d4/d6/d3a/d5d/f82 [0,4194304] 0 2024-05-15T15:54:11.815 INFO:tasks.workunit.client.1.smithi144.stdout:0/469: chown d4/d6/d4a/d4d/c84 0 1 2024-05-15T15:54:11.816 INFO:tasks.workunit.client.0.smithi049.stdout:3/506: write d12/d30/f90 [383682,77667] 0 2024-05-15T15:54:11.818 INFO:tasks.workunit.client.1.smithi144.stdout:0/470: chown d4/d2d/d48/d5b/c64 123823972 1 2024-05-15T15:54:11.820 INFO:tasks.workunit.client.1.smithi144.stdout:9/476: dwrite d0/d8/d5e/d44/f41 [0,4194304] 0 2024-05-15T15:54:11.821 INFO:tasks.workunit.client.1.smithi144.stdout:3/474: mknod d0/d1/d6c/ca4 0 2024-05-15T15:54:11.824 INFO:tasks.workunit.client.1.smithi144.stdout:8/508: creat d2/d24/fa4 x:0 0 0 2024-05-15T15:54:11.824 INFO:tasks.workunit.client.0.smithi049.stdout:4/474: fsync d18/d21/d2b/f3f 0 2024-05-15T15:54:11.828 INFO:tasks.workunit.client.0.smithi049.stdout:0/471: creat dc/d13/f91 x:0 0 0 2024-05-15T15:54:11.829 INFO:tasks.workunit.client.0.smithi049.stdout:3/507: stat d12/d30/d63/c86 0 2024-05-15T15:54:11.830 INFO:tasks.workunit.client.0.smithi049.stdout:3/508: chown d12/f29 4461215 1 2024-05-15T15:54:11.830 INFO:tasks.workunit.client.0.smithi049.stdout:3/509: chown d12/d30/f90 35 1 2024-05-15T15:54:11.831 INFO:tasks.workunit.client.1.smithi144.stdout:0/471: mknod d4/df/d1f/d24/ca4 0 2024-05-15T15:54:11.831 INFO:tasks.workunit.client.0.smithi049.stdout:4/475: truncate d18/d21/d2b/f2c 789427 0 2024-05-15T15:54:11.832 INFO:tasks.workunit.client.1.smithi144.stdout:9/477: mknod d0/d8/d5e/d44/d16/d5a/cab 0 2024-05-15T15:54:11.833 INFO:tasks.workunit.client.1.smithi144.stdout:9/478: chown d0/d8/d5e/d44/d3c/c91 11509474 1 2024-05-15T15:54:11.833 INFO:tasks.workunit.client.0.smithi049.stdout:0/472: symlink dc/d32/l92 0 2024-05-15T15:54:11.835 INFO:tasks.workunit.client.1.smithi144.stdout:8/509: fsync d2/d1a/f3d 0 2024-05-15T15:54:11.836 INFO:tasks.workunit.client.1.smithi144.stdout:1/468: sync 2024-05-15T15:54:11.837 INFO:tasks.workunit.client.1.smithi144.stdout:3/475: dread d0/d11/f2a [0,4194304] 0 2024-05-15T15:54:11.838 INFO:tasks.workunit.client.0.smithi049.stdout:1/552: truncate d0/d4/dd/de/d36/f40 1455725 0 2024-05-15T15:54:11.838 INFO:tasks.workunit.client.1.smithi144.stdout:2/460: rename d3/d7/c38 to d3/d7/d20/d22/c95 0 2024-05-15T15:54:11.839 INFO:tasks.workunit.client.0.smithi049.stdout:0/473: write dc/d13/d14/d25/d31/f8c [458752,108145] 0 2024-05-15T15:54:11.839 INFO:tasks.workunit.client.1.smithi144.stdout:2/461: dread - d3/d7/d20/d22/d52/d75/d65/f93 zero size 2024-05-15T15:54:11.840 INFO:tasks.workunit.client.0.smithi049.stdout:1/553: write d0/d4/dd/de/d48/d5c/f87 [942707,66805] 0 2024-05-15T15:54:11.840 INFO:tasks.workunit.client.0.smithi049.stdout:0/474: stat dc/d13/d14/f73 0 2024-05-15T15:54:11.843 INFO:tasks.workunit.client.0.smithi049.stdout:0/475: write dc/d13/d14/d25/d31/d71/f2c [4713089,79515] 0 2024-05-15T15:54:11.843 INFO:tasks.workunit.client.1.smithi144.stdout:8/510: dread d2/de/d3f/f5f [0,4194304] 0 2024-05-15T15:54:11.845 INFO:tasks.workunit.client.1.smithi144.stdout:5/463: symlink d0/dc/d12/d4a/d5d/l97 0 2024-05-15T15:54:11.847 INFO:tasks.workunit.client.1.smithi144.stdout:0/472: fdatasync d4/d6/d4a/d74/f99 0 2024-05-15T15:54:11.848 INFO:tasks.workunit.client.1.smithi144.stdout:9/479: symlink d0/d8/d50/lac 0 2024-05-15T15:54:11.849 INFO:tasks.workunit.client.0.smithi049.stdout:0/476: dread dc/d13/d14/d25/d31/d71/f2c [4194304,4194304] 0 2024-05-15T15:54:11.851 INFO:tasks.workunit.client.0.smithi049.stdout:0/477: write dc/d13/d14/f73 [392362,105037] 0 2024-05-15T15:54:11.852 INFO:tasks.workunit.client.1.smithi144.stdout:3/476: dread d0/d1/f4 [0,4194304] 0 2024-05-15T15:54:11.854 INFO:tasks.workunit.client.1.smithi144.stdout:3/477: write d0/d1/f43 [1061317,19920] 0 2024-05-15T15:54:11.854 INFO:tasks.workunit.client.1.smithi144.stdout:3/478: dread - d0/d1/d17/f8b zero size 2024-05-15T15:54:11.860 INFO:tasks.workunit.client.1.smithi144.stdout:2/462: write d3/d7/d20/f3e [2486305,62939] 0 2024-05-15T15:54:11.864 INFO:tasks.workunit.client.1.smithi144.stdout:5/464: creat d0/dc/d2f/d68/d5c/f98 x:0 0 0 2024-05-15T15:54:11.865 INFO:tasks.workunit.client.1.smithi144.stdout:0/473: truncate d4/df/f16 1143548 0 2024-05-15T15:54:11.869 INFO:tasks.workunit.client.1.smithi144.stdout:1/469: chown d7/d4b/f4f 22510 1 2024-05-15T15:54:11.870 INFO:tasks.workunit.client.1.smithi144.stdout:3/479: mkdir d0/d11/d24/da5 0 2024-05-15T15:54:11.870 INFO:tasks.workunit.client.1.smithi144.stdout:3/480: chown d0/d11/f61 21543 1 2024-05-15T15:54:11.871 INFO:tasks.workunit.client.1.smithi144.stdout:3/481: chown d0/d11/d3b/f45 99 1 2024-05-15T15:54:11.873 INFO:tasks.workunit.client.1.smithi144.stdout:2/463: creat d3/d7/d20/d22/d5e/f96 x:0 0 0 2024-05-15T15:54:11.873 INFO:tasks.workunit.client.1.smithi144.stdout:2/464: readlink d3/d7/d20/d22/d52/d75/l7d 0 2024-05-15T15:54:11.876 INFO:tasks.workunit.client.1.smithi144.stdout:0/474: mkdir d4/df/d1f/d24/d2b/d55/da5 0 2024-05-15T15:54:11.877 INFO:tasks.workunit.client.1.smithi144.stdout:0/475: read d4/df/d1f/d24/f4b [2645802,76623] 0 2024-05-15T15:54:11.877 INFO:tasks.workunit.client.0.smithi049.stdout:4/476: symlink d18/l9e 0 2024-05-15T15:54:11.878 INFO:tasks.workunit.client.1.smithi144.stdout:9/480: symlink d0/d8/d5e/d44/d10/d19/lad 0 2024-05-15T15:54:11.878 INFO:tasks.workunit.client.1.smithi144.stdout:9/481: write d0/d8/d1b/d62/f8e [1030671,78424] 0 2024-05-15T15:54:11.879 INFO:tasks.workunit.client.0.smithi049.stdout:3/510: rename d12/d14/f1a to d12/d14/d52/f99 0 2024-05-15T15:54:11.881 INFO:tasks.workunit.client.0.smithi049.stdout:1/554: creat d0/d4/dd/de/d1e/d2e/dad/d65/fc7 x:0 0 0 2024-05-15T15:54:11.882 INFO:tasks.workunit.client.1.smithi144.stdout:2/465: mkdir d3/d7/d20/d22/d52/d64/d97 0 2024-05-15T15:54:11.883 INFO:tasks.workunit.client.1.smithi144.stdout:7/501: sync 2024-05-15T15:54:11.883 INFO:tasks.workunit.client.1.smithi144.stdout:3/482: dread d0/d1/d17/f33 [0,4194304] 0 2024-05-15T15:54:11.884 INFO:tasks.workunit.client.1.smithi144.stdout:8/511: rmdir d2/de/d36/d58/d9c 0 2024-05-15T15:54:11.884 INFO:tasks.workunit.client.1.smithi144.stdout:3/483: write d0/f46 [5015448,127268] 0 2024-05-15T15:54:11.885 INFO:tasks.workunit.client.0.smithi049.stdout:0/478: read dc/d13/d56/f90 [55964,62975] 0 2024-05-15T15:54:11.885 INFO:tasks.workunit.client.1.smithi144.stdout:8/512: write d2/de/d36/d58/f94 [1179785,120379] 0 2024-05-15T15:54:11.885 INFO:tasks.workunit.client.1.smithi144.stdout:5/465: symlink d0/dc/d2f/d34/d63/l99 0 2024-05-15T15:54:11.886 INFO:tasks.workunit.client.1.smithi144.stdout:5/466: stat d0/l1 0 2024-05-15T15:54:11.888 INFO:tasks.workunit.client.1.smithi144.stdout:5/467: dread d0/dc/d24/d8e/f8a [0,4194304] 0 2024-05-15T15:54:11.888 INFO:tasks.workunit.client.1.smithi144.stdout:0/476: fdatasync d4/df/f46 0 2024-05-15T15:54:11.889 INFO:tasks.workunit.client.1.smithi144.stdout:0/477: readlink d4/d6/d4a/l76 0 2024-05-15T15:54:11.891 INFO:tasks.workunit.client.1.smithi144.stdout:0/478: write d4/d6/d4a/d4d/f6e [2252374,1574] 0 2024-05-15T15:54:11.891 INFO:tasks.workunit.client.1.smithi144.stdout:8/513: dwrite d2/de/d36/d58/f94 [0,4194304] 0 2024-05-15T15:54:11.892 INFO:tasks.workunit.client.0.smithi049.stdout:1/555: mknod d0/d4/dd/de/d1e/d16/cc8 0 2024-05-15T15:54:11.893 INFO:tasks.workunit.client.1.smithi144.stdout:0/479: truncate d4/d6/d3a/d5d/f8f 497701 0 2024-05-15T15:54:11.895 INFO:tasks.workunit.client.1.smithi144.stdout:2/466: stat d3/l66 0 2024-05-15T15:54:11.895 INFO:tasks.workunit.client.0.smithi049.stdout:4/477: fsync d18/d1e/d2a/f4f 0 2024-05-15T15:54:11.898 INFO:tasks.workunit.client.1.smithi144.stdout:8/514: dread d2/fd [4194304,4194304] 0 2024-05-15T15:54:11.899 INFO:tasks.workunit.client.0.smithi049.stdout:1/556: write d0/d4/dd/de/d4e/f7e [4172498,88481] 0 2024-05-15T15:54:11.901 INFO:tasks.workunit.client.0.smithi049.stdout:1/557: chown d0/d4/dd/de/d1e/d2e/d3e/d90/f91 15 1 2024-05-15T15:54:11.903 INFO:tasks.workunit.client.0.smithi049.stdout:4/478: symlink d18/d85/l9f 0 2024-05-15T15:54:11.903 INFO:tasks.workunit.client.1.smithi144.stdout:2/467: dwrite d3/d7/d20/d22/d5e/f96 [0,4194304] 0 2024-05-15T15:54:11.905 INFO:tasks.workunit.client.1.smithi144.stdout:7/502: dread d4/f51 [0,4194304] 0 2024-05-15T15:54:11.908 INFO:tasks.workunit.client.0.smithi049.stdout:4/479: mknod d18/d85/d98/ca0 0 2024-05-15T15:54:11.912 INFO:tasks.workunit.client.1.smithi144.stdout:2/468: dwrite d3/d7/d20/d37/f46 [0,4194304] 0 2024-05-15T15:54:11.913 INFO:tasks.workunit.client.0.smithi049.stdout:4/480: symlink d18/d1e/d2a/d36/d86/la1 0 2024-05-15T15:54:11.919 INFO:tasks.workunit.client.1.smithi144.stdout:0/480: creat d4/d6/d3a/d4c/fa6 x:0 0 0 2024-05-15T15:54:11.921 INFO:tasks.workunit.client.1.smithi144.stdout:8/515: rename d2/de/d3f/d6b/d75 to d2/d3a/da5 0 2024-05-15T15:54:11.925 INFO:tasks.workunit.client.1.smithi144.stdout:3/484: symlink d0/d1/d56/d71/d73/la6 0 2024-05-15T15:54:11.926 INFO:tasks.workunit.client.1.smithi144.stdout:2/469: mkdir d3/d7/d54/d98 0 2024-05-15T15:54:11.928 INFO:tasks.workunit.client.1.smithi144.stdout:5/468: mkdir d0/dc/d12/d31/d9a 0 2024-05-15T15:54:11.931 INFO:tasks.workunit.client.1.smithi144.stdout:0/481: unlink d4/df/f46 0 2024-05-15T15:54:11.931 INFO:tasks.workunit.client.1.smithi144.stdout:0/482: fdatasync d4/d6/d3a/d4c/f70 0 2024-05-15T15:54:11.932 INFO:tasks.workunit.client.1.smithi144.stdout:8/516: write d2/f16 [2179517,33548] 0 2024-05-15T15:54:11.933 INFO:tasks.workunit.client.1.smithi144.stdout:1/470: truncate d7/d1d/d33/d41/f19 3097923 0 2024-05-15T15:54:11.933 INFO:tasks.workunit.client.1.smithi144.stdout:8/517: write d2/d24/d2d/d30/f7d [1811014,87000] 0 2024-05-15T15:54:11.933 INFO:tasks.workunit.client.1.smithi144.stdout:3/485: mkdir d0/d11/d3b/da7 0 2024-05-15T15:54:11.935 INFO:tasks.workunit.client.1.smithi144.stdout:8/518: fsync d2/d24/d2d/d30/f60 0 2024-05-15T15:54:11.935 INFO:tasks.workunit.client.1.smithi144.stdout:5/469: mkdir d0/d7e/d9b 0 2024-05-15T15:54:11.938 INFO:tasks.workunit.client.1.smithi144.stdout:8/519: dread d2/de/d49/f69 [0,4194304] 0 2024-05-15T15:54:11.945 INFO:tasks.workunit.client.0.smithi049.stdout:9/304: sync 2024-05-15T15:54:11.945 INFO:tasks.workunit.client.0.smithi049.stdout:9/305: dread - d0/d8/d9/f57 zero size 2024-05-15T15:54:11.946 INFO:tasks.workunit.client.1.smithi144.stdout:3/486: rmdir d0/d1/d17 39 2024-05-15T15:54:11.952 INFO:tasks.workunit.client.0.smithi049.stdout:9/306: dwrite d0/d8/d21/f2f [0,4194304] 0 2024-05-15T15:54:11.954 INFO:tasks.workunit.client.1.smithi144.stdout:1/471: symlink d7/d4b/d53/l9e 0 2024-05-15T15:54:11.955 INFO:tasks.workunit.client.1.smithi144.stdout:1/472: fsync d7/d1d/d33/d1e/f77 0 2024-05-15T15:54:11.955 INFO:tasks.workunit.client.0.smithi049.stdout:1/558: dwrite d0/d4/dd/de/d36/f43 [0,4194304] 0 2024-05-15T15:54:11.959 INFO:tasks.workunit.client.0.smithi049.stdout:1/559: write d0/d4/dd/de/d1e/d2e/dad/d65/f6f [3185123,98022] 0 2024-05-15T15:54:11.962 INFO:tasks.workunit.client.1.smithi144.stdout:1/473: dwrite d7/d1d/d33/d1e/d34/f4a [4194304,4194304] 0 2024-05-15T15:54:11.962 INFO:tasks.workunit.client.1.smithi144.stdout:3/487: symlink d0/d11/d3b/da0/d8f/la8 0 2024-05-15T15:54:11.964 INFO:tasks.workunit.client.1.smithi144.stdout:1/474: read d7/d1d/d33/d1e/f77 [984591,13156] 0 2024-05-15T15:54:11.968 INFO:tasks.workunit.client.0.smithi049.stdout:1/560: symlink d0/d4/dd/de/d1e/d16/d8a/lc9 0 2024-05-15T15:54:11.970 INFO:tasks.workunit.client.0.smithi049.stdout:9/307: creat d0/f65 x:0 0 0 2024-05-15T15:54:11.972 INFO:tasks.workunit.client.1.smithi144.stdout:8/520: mknod d2/d24/d2d/da0/d6e/d71/d73/d98/ca6 0 2024-05-15T15:54:11.973 INFO:tasks.workunit.client.0.smithi049.stdout:1/561: fsync d0/d4/f79 0 2024-05-15T15:54:11.973 INFO:tasks.workunit.client.1.smithi144.stdout:8/521: write d2/d3a/d7b/d4b/f61 [407045,65115] 0 2024-05-15T15:54:11.975 INFO:tasks.workunit.client.0.smithi049.stdout:1/562: write d0/d4/dd/de/d1e/d2e/d3e/d90/f91 [548039,28563] 0 2024-05-15T15:54:11.978 INFO:tasks.workunit.client.1.smithi144.stdout:8/522: dwrite d2/d24/f89 [0,4194304] 0 2024-05-15T15:54:11.979 INFO:tasks.workunit.client.1.smithi144.stdout:2/470: getdents d3/d7/d20/d22/d5e 0 2024-05-15T15:54:11.986 INFO:tasks.workunit.client.1.smithi144.stdout:8/523: dread d2/de/f33 [0,4194304] 0 2024-05-15T15:54:11.987 INFO:tasks.workunit.client.1.smithi144.stdout:8/524: write d2/de/d36/f95 [871325,41446] 0 2024-05-15T15:54:11.988 INFO:tasks.workunit.client.1.smithi144.stdout:0/483: creat d4/df/d1f/d56/fa7 x:0 0 0 2024-05-15T15:54:11.989 INFO:tasks.workunit.client.0.smithi049.stdout:1/563: dread d0/d4/dd/de/d1e/d2e/f88 [0,4194304] 0 2024-05-15T15:54:11.992 INFO:tasks.workunit.client.1.smithi144.stdout:8/525: dwrite d2/d24/f2b [0,4194304] 0 2024-05-15T15:54:11.992 INFO:tasks.workunit.client.1.smithi144.stdout:8/526: readlink d2/l29 0 2024-05-15T15:54:11.992 INFO:tasks.workunit.client.1.smithi144.stdout:8/527: readlink d2/l29 0 2024-05-15T15:54:11.993 INFO:tasks.workunit.client.1.smithi144.stdout:8/528: dread - d2/d3a/f91 zero size 2024-05-15T15:54:11.996 INFO:tasks.workunit.client.0.smithi049.stdout:1/564: dwrite d0/d4/dd/de/d4e/f7e [0,4194304] 0 2024-05-15T15:54:11.996 INFO:tasks.workunit.client.0.smithi049.stdout:9/308: link d0/d8/d1b/d40/f49 d0/d25/d5d/f66 0 2024-05-15T15:54:12.002 INFO:tasks.workunit.client.1.smithi144.stdout:2/471: rename d3/d10/l14 to d3/d7/d20/d22/d52/d64/d80/l99 0 2024-05-15T15:54:12.005 INFO:tasks.workunit.client.0.smithi049.stdout:1/565: dwrite d0/d4/dd/de/d36/f43 [0,4194304] 0 2024-05-15T15:54:12.007 INFO:tasks.workunit.client.0.smithi049.stdout:9/309: mknod d0/d8/d1b/d40/d5c/c67 0 2024-05-15T15:54:12.009 INFO:tasks.workunit.client.1.smithi144.stdout:3/488: write d0/d11/f2a [1171961,41892] 0 2024-05-15T15:54:12.012 INFO:tasks.workunit.client.0.smithi049.stdout:1/566: creat d0/d4/dd/de/d48/d84/fca x:0 0 0 2024-05-15T15:54:12.012 INFO:tasks.workunit.client.1.smithi144.stdout:5/470: link d0/dc/d1f/d50/l30 d0/dc/l9c 0 2024-05-15T15:54:12.014 INFO:tasks.workunit.client.1.smithi144.stdout:6/424: sync 2024-05-15T15:54:12.014 INFO:tasks.workunit.client.1.smithi144.stdout:4/483: sync 2024-05-15T15:54:12.018 INFO:tasks.workunit.client.0.smithi049.stdout:9/310: creat d0/d8/d53/f68 x:0 0 0 2024-05-15T15:54:12.020 INFO:tasks.workunit.client.1.smithi144.stdout:2/472: creat d3/d7/d20/d22/d52/d75/d65/f9a x:0 0 0 2024-05-15T15:54:12.021 INFO:tasks.workunit.client.0.smithi049.stdout:9/311: readlink d0/l5b 0 2024-05-15T15:54:12.021 INFO:tasks.workunit.client.1.smithi144.stdout:8/529: dwrite d2/de/d3f/f40 [0,4194304] 0 2024-05-15T15:54:12.022 INFO:tasks.workunit.client.1.smithi144.stdout:8/530: stat d2/de/d36/f17 0 2024-05-15T15:54:12.023 INFO:tasks.workunit.client.1.smithi144.stdout:8/531: chown d2/d24/d2d/da0/d6e/d71 378403694 1 2024-05-15T15:54:12.023 INFO:tasks.workunit.client.1.smithi144.stdout:8/532: fdatasync d2/d24/d2d/d30/f81 0 2024-05-15T15:54:12.024 INFO:tasks.workunit.client.1.smithi144.stdout:1/475: creat d7/df/f9f x:0 0 0 2024-05-15T15:54:12.024 INFO:tasks.workunit.client.0.smithi049.stdout:9/312: truncate d0/d8/d9/fa 5122349 0 2024-05-15T15:54:12.025 INFO:tasks.workunit.client.1.smithi144.stdout:1/476: chown d7/df/f10 0 1 2024-05-15T15:54:12.025 INFO:tasks.workunit.client.0.smithi049.stdout:1/567: link d0/d4/dd/de/d36/f40 d0/d4/dd/de/d1e/d2e/d3e/d90/fcb 0 2024-05-15T15:54:12.026 INFO:tasks.workunit.client.1.smithi144.stdout:0/484: creat d4/df/da0/fa8 x:0 0 0 2024-05-15T15:54:12.027 INFO:tasks.workunit.client.1.smithi144.stdout:1/477: write d7/d1d/d33/d41/d22/f90 [697499,3042] 0 2024-05-15T15:54:12.028 INFO:tasks.workunit.client.0.smithi049.stdout:1/568: write d0/d4/dd/de/d1e/d2e/dad/d65/d72/f9d [1231967,101187] 0 2024-05-15T15:54:12.028 INFO:tasks.workunit.client.0.smithi049.stdout:9/313: mkdir d0/d8/d1b/d40/d5c/d69 0 2024-05-15T15:54:12.031 INFO:tasks.workunit.client.1.smithi144.stdout:2/473: stat d3/d7/d20/l71 0 2024-05-15T15:54:12.033 INFO:tasks.workunit.client.1.smithi144.stdout:5/471: dread d0/dc/d1f/f38 [0,4194304] 0 2024-05-15T15:54:12.033 INFO:tasks.workunit.client.1.smithi144.stdout:1/478: dread d7/d1d/d33/d55/f8d [0,4194304] 0 2024-05-15T15:54:12.034 INFO:tasks.workunit.client.1.smithi144.stdout:1/479: chown d7/d1d/c93 27120 1 2024-05-15T15:54:12.035 INFO:tasks.workunit.client.1.smithi144.stdout:4/484: dread d6/d42/d46/d4b/d1e/d44/f53 [0,4194304] 0 2024-05-15T15:54:12.036 INFO:tasks.workunit.client.1.smithi144.stdout:3/489: creat d0/d11/d3b/da7/fa9 x:0 0 0 2024-05-15T15:54:12.036 INFO:tasks.workunit.client.1.smithi144.stdout:1/480: fsync d7/d1d/d33/d41/f8c 0 2024-05-15T15:54:12.036 INFO:tasks.workunit.client.0.smithi049.stdout:9/314: dread d0/d8/d11/f1f [0,4194304] 0 2024-05-15T15:54:12.037 INFO:tasks.workunit.client.1.smithi144.stdout:4/485: write d6/dd/d2e/f70 [579447,11463] 0 2024-05-15T15:54:12.037 INFO:tasks.workunit.client.1.smithi144.stdout:0/485: rmdir d4/d6 39 2024-05-15T15:54:12.038 INFO:tasks.workunit.client.0.smithi049.stdout:1/569: creat d0/d4/dd/fcc x:0 0 0 2024-05-15T15:54:12.039 INFO:tasks.workunit.client.0.smithi049.stdout:9/315: fdatasync d0/d8/d9/d52/d5e/d5f/f55 0 2024-05-15T15:54:12.040 INFO:tasks.workunit.client.1.smithi144.stdout:6/425: symlink d7/d1e/l9f 0 2024-05-15T15:54:12.043 INFO:tasks.workunit.client.1.smithi144.stdout:8/533: dread d2/f16 [0,4194304] 0 2024-05-15T15:54:12.043 INFO:tasks.workunit.client.1.smithi144.stdout:2/474: read d3/f1c [741788,109427] 0 2024-05-15T15:54:12.043 INFO:tasks.workunit.client.1.smithi144.stdout:3/490: dwrite d0/d11/f2a [0,4194304] 0 2024-05-15T15:54:12.043 INFO:tasks.workunit.client.1.smithi144.stdout:5/472: mknod d0/dc/d1f/d50/c9d 0 2024-05-15T15:54:12.046 INFO:tasks.workunit.client.1.smithi144.stdout:8/534: write d2/de/d3f/f40 [4379859,55818] 0 2024-05-15T15:54:12.046 INFO:tasks.workunit.client.1.smithi144.stdout:1/481: rmdir d7/d1d/d33/d1e 39 2024-05-15T15:54:12.048 INFO:tasks.workunit.client.1.smithi144.stdout:1/482: read - d7/d31/f7b zero size 2024-05-15T15:54:12.049 INFO:tasks.workunit.client.1.smithi144.stdout:0/486: readlink d4/d6/d4a/l52 0 2024-05-15T15:54:12.049 INFO:tasks.workunit.client.1.smithi144.stdout:1/483: stat d7/d1d/d33/d29/l8a 0 2024-05-15T15:54:12.051 INFO:tasks.workunit.client.1.smithi144.stdout:1/484: chown d7/d1d/d58/c68 18535 1 2024-05-15T15:54:12.053 INFO:tasks.workunit.client.1.smithi144.stdout:6/426: unlink d7/d1e/f75 0 2024-05-15T15:54:12.054 INFO:tasks.workunit.client.1.smithi144.stdout:9/482: sync 2024-05-15T15:54:12.054 INFO:tasks.workunit.client.1.smithi144.stdout:7/503: sync 2024-05-15T15:54:12.059 INFO:tasks.workunit.client.1.smithi144.stdout:8/535: creat d2/de/d49/d83/fa7 x:0 0 0 2024-05-15T15:54:12.059 INFO:tasks.workunit.client.1.smithi144.stdout:2/475: dread d3/f5 [0,4194304] 0 2024-05-15T15:54:12.060 INFO:tasks.workunit.client.1.smithi144.stdout:4/486: link d6/d7c/f8c d6/dd/d34/d3b/d30/d59/d66/f9b 0 2024-05-15T15:54:12.061 INFO:tasks.workunit.client.1.smithi144.stdout:0/487: mknod d4/d6/d4a/ca9 0 2024-05-15T15:54:12.062 INFO:tasks.workunit.client.1.smithi144.stdout:4/487: truncate d6/dd/d34/d3b/d30/db/d1d/d40/f79 659510 0 2024-05-15T15:54:12.065 INFO:tasks.workunit.client.1.smithi144.stdout:7/504: symlink d4/df/d25/d38/d5d/la8 0 2024-05-15T15:54:12.068 INFO:tasks.workunit.client.0.smithi049.stdout:9/316: dwrite d0/d8/fd [0,4194304] 0 2024-05-15T15:54:12.069 INFO:tasks.workunit.client.1.smithi144.stdout:2/476: creat d3/d7/d20/d22/d52/d64/d80/f9b x:0 0 0 2024-05-15T15:54:12.070 INFO:tasks.workunit.client.1.smithi144.stdout:0/488: mknod d4/d6/d4a/d74/caa 0 2024-05-15T15:54:12.071 INFO:tasks.workunit.client.0.smithi049.stdout:1/570: dwrite d0/d4/dd/de/d1e/d16/d8a/f99 [0,4194304] 0 2024-05-15T15:54:12.071 INFO:tasks.workunit.client.1.smithi144.stdout:1/485: mknod d7/ca0 0 2024-05-15T15:54:12.072 INFO:tasks.workunit.client.0.smithi049.stdout:1/571: chown d0/d4/dd/de/d48/d5c/f94 5812036 1 2024-05-15T15:54:12.073 INFO:tasks.workunit.client.1.smithi144.stdout:5/473: write d0/dc/d24/d8e/d93/f89 [4017179,1197] 0 2024-05-15T15:54:12.074 INFO:tasks.workunit.client.0.smithi049.stdout:1/572: write d0/d4/dd/de/d1e/f31 [1078296,25686] 0 2024-05-15T15:54:12.075 INFO:tasks.workunit.client.1.smithi144.stdout:4/488: unlink d6/dd/d34/d3b/d30/d59/d66/f8d 0 2024-05-15T15:54:12.077 INFO:tasks.workunit.client.0.smithi049.stdout:9/317: mknod d0/d8/d9/d52/d5e/d5f/d56/c6a 0 2024-05-15T15:54:12.078 INFO:tasks.workunit.client.0.smithi049.stdout:9/318: chown d0/d8/d12/l3f 26136 1 2024-05-15T15:54:12.081 INFO:tasks.workunit.client.1.smithi144.stdout:6/427: creat d7/db/d35/d84/d81/fa0 x:0 0 0 2024-05-15T15:54:12.081 INFO:tasks.workunit.client.1.smithi144.stdout:1/486: dwrite d7/d1d/f91 [4194304,4194304] 0 2024-05-15T15:54:12.084 INFO:tasks.workunit.client.1.smithi144.stdout:1/487: dread - d7/d1d/f8b zero size 2024-05-15T15:54:12.085 INFO:tasks.workunit.client.1.smithi144.stdout:1/488: dread - d7/d1d/d33/d29/d6b/d5e/d74/f96 zero size 2024-05-15T15:54:12.156 INFO:tasks.workunit.client.1.smithi144.stdout:7/505: mknod d4/d41/d5b/ca9 0 2024-05-15T15:54:12.195 INFO:tasks.workunit.client.0.smithi049.stdout:9/319: mkdir d0/d8/d53/d6b 0 2024-05-15T15:54:12.196 INFO:tasks.workunit.client.0.smithi049.stdout:9/320: chown d0/d8/f61 6 1 2024-05-15T15:54:12.199 INFO:tasks.workunit.client.0.smithi049.stdout:1/573: mknod d0/d4/dd/de/d1e/d2e/dad/d65/d72/dae/ccd 0 2024-05-15T15:54:12.205 INFO:tasks.workunit.client.0.smithi049.stdout:1/574: dwrite d0/d4/f1b [0,4194304] 0 2024-05-15T15:54:12.207 INFO:tasks.workunit.client.0.smithi049.stdout:9/321: mknod d0/d25/d5d/c6c 0 2024-05-15T15:54:12.209 INFO:tasks.workunit.client.0.smithi049.stdout:1/575: symlink d0/d4/dd/de/d4d/d8b/lce 0 2024-05-15T15:54:12.209 INFO:tasks.workunit.client.0.smithi049.stdout:1/576: stat d0/d4/dd/de/d36 0 2024-05-15T15:54:12.212 INFO:tasks.workunit.client.1.smithi144.stdout:5/474: dread - d0/dc/d12/d31/f4c zero size 2024-05-15T15:54:12.213 INFO:tasks.workunit.client.0.smithi049.stdout:9/322: mknod d0/d8/d9/d52/d5e/d5f/c6d 0 2024-05-15T15:54:12.218 INFO:tasks.workunit.client.0.smithi049.stdout:9/323: creat d0/d8/d12/f6e x:0 0 0 2024-05-15T15:54:12.220 INFO:tasks.workunit.client.1.smithi144.stdout:1/489: creat d7/d1d/d33/d29/d6b/d5e/fa1 x:0 0 0 2024-05-15T15:54:12.221 INFO:tasks.workunit.client.1.smithi144.stdout:9/483: sync 2024-05-15T15:54:12.223 INFO:tasks.workunit.client.0.smithi049.stdout:9/324: symlink d0/d8/d1b/l6f 0 2024-05-15T15:54:12.228 INFO:tasks.workunit.client.0.smithi049.stdout:9/325: dwrite d0/d8/d1b/f3a [0,4194304] 0 2024-05-15T15:54:12.244 INFO:tasks.workunit.client.1.smithi144.stdout:2/477: creat d3/d7/d54/d98/f9c x:0 0 0 2024-05-15T15:54:12.244 INFO:tasks.workunit.client.1.smithi144.stdout:0/489: readlink d4/df/d1f/d56/l6f 0 2024-05-15T15:54:12.245 INFO:tasks.workunit.client.1.smithi144.stdout:5/475: readlink d0/d7e/l91 0 2024-05-15T15:54:12.246 INFO:tasks.workunit.client.1.smithi144.stdout:1/490: stat d7/d1d/d33/d41/l18 0 2024-05-15T15:54:12.247 INFO:tasks.workunit.client.1.smithi144.stdout:9/484: readlink d0/d8/d5e/d44/l37 0 2024-05-15T15:54:12.250 INFO:tasks.workunit.client.1.smithi144.stdout:1/491: dread d7/d1d/d33/d29/f42 [0,4194304] 0 2024-05-15T15:54:12.254 INFO:tasks.workunit.client.1.smithi144.stdout:7/506: sync 2024-05-15T15:54:12.255 INFO:tasks.workunit.client.1.smithi144.stdout:2/478: truncate d3/d7/d20/d22/d52/d75/f55 5727357 0 2024-05-15T15:54:12.257 INFO:tasks.workunit.client.1.smithi144.stdout:5/476: unlink d0/dc/d12/d4a/d73/l87 0 2024-05-15T15:54:12.260 INFO:tasks.workunit.client.1.smithi144.stdout:7/507: mknod d4/d41/d9c/caa 0 2024-05-15T15:54:12.261 INFO:tasks.workunit.client.1.smithi144.stdout:5/477: mkdir d0/dc/d24/d8e/d93/d9e 0 2024-05-15T15:54:12.262 INFO:tasks.workunit.client.1.smithi144.stdout:5/478: dread - d0/dc/d24/d28/f82 zero size 2024-05-15T15:54:12.266 INFO:tasks.workunit.client.0.smithi049.stdout:2/490: sync 2024-05-15T15:54:12.267 INFO:tasks.workunit.client.1.smithi144.stdout:5/479: creat d0/dc/d12/d31/f9f x:0 0 0 2024-05-15T15:54:12.268 INFO:tasks.workunit.client.1.smithi144.stdout:5/480: write d0/dc/f8c [914912,39232] 0 2024-05-15T15:54:12.269 INFO:tasks.workunit.client.0.smithi049.stdout:2/491: symlink d3/d52/l97 0 2024-05-15T15:54:12.270 INFO:tasks.workunit.client.1.smithi144.stdout:5/481: mknod d0/dc/d24/d28/d2d/d59/ca0 0 2024-05-15T15:54:12.273 INFO:tasks.workunit.client.1.smithi144.stdout:5/482: creat d0/dc/d24/d8e/d93/fa1 x:0 0 0 2024-05-15T15:54:12.275 INFO:tasks.workunit.client.0.smithi049.stdout:2/492: dwrite d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d [0,4194304] 0 2024-05-15T15:54:12.276 INFO:tasks.workunit.client.1.smithi144.stdout:5/483: unlink d0/dc/d24/d28/d2d/c4b 0 2024-05-15T15:54:12.277 INFO:tasks.workunit.client.1.smithi144.stdout:5/484: read - d0/dc/d12/d31/f4c zero size 2024-05-15T15:54:12.279 INFO:tasks.workunit.client.0.smithi049.stdout:2/493: symlink d3/db/d1d/d1e/d2b/d6b/l98 0 2024-05-15T15:54:12.279 INFO:tasks.workunit.client.1.smithi144.stdout:5/485: truncate d0/dc/d24/d8e/d93/d6b/d88/f8d 194718 0 2024-05-15T15:54:12.281 INFO:tasks.workunit.client.0.smithi049.stdout:2/494: stat d3/db/d1d/d1e/d2b/d3b/d42/f6d 0 2024-05-15T15:54:12.282 INFO:tasks.workunit.client.1.smithi144.stdout:5/486: creat d0/d7e/d9b/fa2 x:0 0 0 2024-05-15T15:54:12.283 INFO:tasks.workunit.client.0.smithi049.stdout:2/495: symlink d3/d85/l99 0 2024-05-15T15:54:12.284 INFO:tasks.workunit.client.0.smithi049.stdout:2/496: chown d3/d8/f14 1147 1 2024-05-15T15:54:12.285 INFO:tasks.workunit.client.1.smithi144.stdout:5/487: creat d0/dc/d1f/fa3 x:0 0 0 2024-05-15T15:54:12.285 INFO:tasks.workunit.client.0.smithi049.stdout:2/497: mkdir d3/db/d1d/d1e/d2b/d3b/d47/d9a 0 2024-05-15T15:54:12.287 INFO:tasks.workunit.client.1.smithi144.stdout:5/488: unlink d0/dc/d24/d8e/d93/f90 0 2024-05-15T15:54:12.288 INFO:tasks.workunit.client.1.smithi144.stdout:5/489: chown d0/dc/d2f/d34/d63/d6e 194625173 1 2024-05-15T15:54:12.290 INFO:tasks.workunit.client.1.smithi144.stdout:5/490: unlink d0/dc/fe 0 2024-05-15T15:54:12.291 INFO:tasks.workunit.client.1.smithi144.stdout:5/491: chown d0/dc/d24/d8e/l8f 348 1 2024-05-15T15:54:12.293 INFO:tasks.workunit.client.1.smithi144.stdout:5/492: creat d0/dc/d12/d4a/d5d/fa4 x:0 0 0 2024-05-15T15:54:12.293 INFO:tasks.workunit.client.1.smithi144.stdout:5/493: fsync d0/dc/d1f/d50/f86 0 2024-05-15T15:54:12.296 INFO:tasks.workunit.client.1.smithi144.stdout:5/494: rmdir d0/dc/d12/d31/d9a 0 2024-05-15T15:54:12.298 INFO:tasks.workunit.client.1.smithi144.stdout:5/495: mkdir d0/dc/d2f/d79/da5 0 2024-05-15T15:54:12.299 INFO:tasks.workunit.client.1.smithi144.stdout:5/496: creat d0/dc/d24/d57/fa6 x:0 0 0 2024-05-15T15:54:12.300 INFO:tasks.workunit.client.1.smithi144.stdout:5/497: creat d0/dc/d24/d8e/d93/fa7 x:0 0 0 2024-05-15T15:54:12.301 INFO:tasks.workunit.client.1.smithi144.stdout:5/498: chown d0/dc/d2f/d68/d5c/c6c 29276 1 2024-05-15T15:54:12.302 INFO:tasks.workunit.client.1.smithi144.stdout:5/499: truncate d0/dc/d24/d57/f77 988480 0 2024-05-15T15:54:12.303 INFO:tasks.workunit.client.1.smithi144.stdout:5/500: write d0/dc/f8c [1333710,116405] 0 2024-05-15T15:54:12.304 INFO:tasks.workunit.client.1.smithi144.stdout:5/501: stat d0/dc/d24/d8e/d95 0 2024-05-15T15:54:12.308 INFO:tasks.workunit.client.1.smithi144.stdout:5/502: dwrite d0/f1a [0,4194304] 0 2024-05-15T15:54:12.316 INFO:tasks.workunit.client.1.smithi144.stdout:5/503: link d0/dc/d24/f37 d0/d7e/fa8 0 2024-05-15T15:54:12.322 INFO:tasks.workunit.client.1.smithi144.stdout:5/504: unlink d0/dc/d1f/f3b 0 2024-05-15T15:54:12.325 INFO:tasks.workunit.client.1.smithi144.stdout:5/505: read d0/f11 [3897426,7603] 0 2024-05-15T15:54:12.332 INFO:tasks.workunit.client.1.smithi144.stdout:3/491: write d0/d1/d17/f91 [897849,29544] 0 2024-05-15T15:54:12.333 INFO:tasks.workunit.client.1.smithi144.stdout:5/506: dread d0/dc/f71 [0,4194304] 0 2024-05-15T15:54:12.337 INFO:tasks.workunit.client.1.smithi144.stdout:8/536: truncate d2/de/f14 1846580 0 2024-05-15T15:54:12.338 INFO:tasks.workunit.client.1.smithi144.stdout:8/537: chown d2/de/d36/f2f 3415 1 2024-05-15T15:54:12.339 INFO:tasks.workunit.client.1.smithi144.stdout:3/492: unlink d0/d1/f5 0 2024-05-15T15:54:12.343 INFO:tasks.workunit.client.1.smithi144.stdout:8/538: mkdir d2/da8 0 2024-05-15T15:54:12.344 INFO:tasks.workunit.client.1.smithi144.stdout:7/508: sync 2024-05-15T15:54:12.344 INFO:tasks.workunit.client.1.smithi144.stdout:5/507: link d0/dc/d2f/d34/c5f d0/dc/d24/d57/ca9 0 2024-05-15T15:54:12.346 INFO:tasks.workunit.client.1.smithi144.stdout:5/508: write d0/dc/d12/d4a/d5d/fa4 [79562,29660] 0 2024-05-15T15:54:12.346 INFO:tasks.workunit.client.0.smithi049.stdout:1/577: rename d0/d4/dd/de/d1e/d16/f22 to d0/d4/dd/de/d4d/fcf 0 2024-05-15T15:54:12.347 INFO:tasks.workunit.client.1.smithi144.stdout:5/509: dread - d0/d7e/d9b/fa2 zero size 2024-05-15T15:54:12.349 INFO:tasks.workunit.client.1.smithi144.stdout:3/493: symlink d0/d1/laa 0 2024-05-15T15:54:12.350 INFO:tasks.workunit.client.1.smithi144.stdout:5/510: read d0/dc/d24/d8e/d93/f62 [129132,28158] 0 2024-05-15T15:54:12.351 INFO:tasks.workunit.client.1.smithi144.stdout:7/509: dwrite d4/d41/d5b/f70 [4194304,4194304] 0 2024-05-15T15:54:12.352 INFO:tasks.workunit.client.0.smithi049.stdout:2/498: rename d3/da/d15/d24/l95 to d3/db/d1d/d1e/d2b/d3b/d42/d8f/l9b 0 2024-05-15T15:54:12.355 INFO:tasks.workunit.client.0.smithi049.stdout:1/578: symlink d0/d4/dd/de/d1e/d2e/d3e/ld0 0 2024-05-15T15:54:12.358 INFO:tasks.workunit.client.1.smithi144.stdout:7/510: dwrite d4/df/d25/d38/f71 [0,4194304] 0 2024-05-15T15:54:12.361 INFO:tasks.workunit.client.1.smithi144.stdout:5/511: chown d0/dc/l9c 240455 1 2024-05-15T15:54:12.365 INFO:tasks.workunit.client.0.smithi049.stdout:2/499: unlink d3/db/d1d/d1e/d2b/d2e/f7e 0 2024-05-15T15:54:12.367 INFO:tasks.workunit.client.0.smithi049.stdout:9/326: getdents d0/d8/d9/d52/d5e/d5f 0 2024-05-15T15:54:12.368 INFO:tasks.workunit.client.0.smithi049.stdout:1/579: read - d0/d4/dd/de/d1e/d2e/dad/d65/d72/f78 zero size 2024-05-15T15:54:12.369 INFO:tasks.workunit.client.0.smithi049.stdout:9/327: write d0/d8/d9/d52/d5e/d5f/f55 [3963778,26766] 0 2024-05-15T15:54:12.373 INFO:tasks.workunit.client.1.smithi144.stdout:4/489: dwrite d6/d42/d46/d4b/f31 [0,4194304] 0 2024-05-15T15:54:12.374 INFO:tasks.workunit.client.1.smithi144.stdout:4/490: readlink d6/dd/d34/d3b/l2c 0 2024-05-15T15:54:12.376 INFO:tasks.workunit.client.0.smithi049.stdout:2/500: creat d3/d78/d8c/d91/f9c x:0 0 0 2024-05-15T15:54:12.381 INFO:tasks.workunit.client.0.smithi049.stdout:9/328: creat d0/d8/d1b/d40/d5c/f70 x:0 0 0 2024-05-15T15:54:12.381 INFO:tasks.workunit.client.1.smithi144.stdout:4/491: dwrite d6/dd/d2e/f61 [0,4194304] 0 2024-05-15T15:54:12.382 INFO:tasks.workunit.client.0.smithi049.stdout:9/329: chown d0/d8/d1b/f3c 467857413 1 2024-05-15T15:54:12.383 INFO:tasks.workunit.client.1.smithi144.stdout:5/512: mknod d0/dc/d24/d28/d2d/d59/caa 0 2024-05-15T15:54:12.387 INFO:tasks.workunit.client.1.smithi144.stdout:3/494: symlink d0/d1/d56/d98/d9c/lab 0 2024-05-15T15:54:12.391 INFO:tasks.workunit.client.0.smithi049.stdout:9/330: chown d0/d8/cf 1892 1 2024-05-15T15:54:12.393 INFO:tasks.workunit.client.0.smithi049.stdout:1/580: creat d0/fd1 x:0 0 0 2024-05-15T15:54:12.398 INFO:tasks.workunit.client.1.smithi144.stdout:5/513: stat d0/dc/d24/d57/ca9 0 2024-05-15T15:54:12.406 INFO:tasks.workunit.client.1.smithi144.stdout:2/479: getdents d3/d7/d54/d98 0 2024-05-15T15:54:12.407 INFO:tasks.workunit.client.1.smithi144.stdout:9/485: write d0/d8/d21/f2a [1016702,75337] 0 2024-05-15T15:54:12.410 INFO:tasks.workunit.client.1.smithi144.stdout:9/486: dread - d0/d8/d5e/d9b/f9f zero size 2024-05-15T15:54:12.410 INFO:tasks.workunit.client.1.smithi144.stdout:3/495: dread d0/d11/f14 [0,4194304] 0 2024-05-15T15:54:12.411 INFO:tasks.workunit.client.0.smithi049.stdout:1/581: dread d0/d4/dd/de/d9e/fa0 [0,4194304] 0 2024-05-15T15:54:12.412 INFO:tasks.workunit.client.1.smithi144.stdout:7/511: link d4/f27 d4/d41/fab 0 2024-05-15T15:54:12.412 INFO:tasks.workunit.client.1.smithi144.stdout:2/480: dread d3/d7/d20/d3f/f53 [0,4194304] 0 2024-05-15T15:54:12.412 INFO:tasks.workunit.client.0.smithi049.stdout:1/582: dread - d0/d4/dd/de/d48/d5c/fc6 zero size 2024-05-15T15:54:12.413 INFO:tasks.workunit.client.1.smithi144.stdout:9/487: mkdir d0/d8/d1b/d79/d8a/dae 0 2024-05-15T15:54:12.413 INFO:tasks.workunit.client.1.smithi144.stdout:9/488: readlink d0/d8/d5e/d44/l37 0 2024-05-15T15:54:12.416 INFO:tasks.workunit.client.1.smithi144.stdout:3/496: rename d0/d1/c7d to d0/d1/d17/d39/d40/d4a/d9e/cac 0 2024-05-15T15:54:12.417 INFO:tasks.workunit.client.0.smithi049.stdout:1/583: rmdir d0/d4/dd/de/d1e/d2e/dad/d65 39 2024-05-15T15:54:12.420 INFO:tasks.workunit.client.1.smithi144.stdout:1/492: truncate d7/d1d/d33/d1e/d4e/f50 2564869 0 2024-05-15T15:54:12.423 INFO:tasks.workunit.client.0.smithi049.stdout:1/584: fsync d0/d4/f2d 0 2024-05-15T15:54:12.424 INFO:tasks.workunit.client.1.smithi144.stdout:2/481: mknod d3/d7/d20/d89/d6e/c9d 0 2024-05-15T15:54:12.430 INFO:tasks.workunit.client.0.smithi049.stdout:9/331: unlink d0/d8/f1e 0 2024-05-15T15:54:12.430 INFO:tasks.workunit.client.0.smithi049.stdout:1/585: dwrite d0/d4/dd/de/d1e/f1d [0,4194304] 0 2024-05-15T15:54:12.435 INFO:tasks.workunit.client.0.smithi049.stdout:1/586: mkdir d0/d4/dd/de/d1e/d16/d8a/dd2 0 2024-05-15T15:54:12.436 INFO:tasks.workunit.client.0.smithi049.stdout:1/587: truncate d0/d4/dd/de/d4d/f56 815384 0 2024-05-15T15:54:12.447 INFO:tasks.workunit.client.0.smithi049.stdout:1/588: dwrite d0/d4/dd/de/d1e/d2e/dad/d65/d72/f9d [0,4194304] 0 2024-05-15T15:54:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:12 smithi144 ceph-mon[36389]: pgmap v26: 65 pgs: 65 active+clean; 1.5 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 40 MiB/s rd, 185 MiB/s wr, 357 op/s 2024-05-15T15:54:12.449 INFO:tasks.workunit.client.0.smithi049.stdout:1/589: chown d0/d4/dd/de/d1e/f1d 1776988 1 2024-05-15T15:54:12.452 INFO:tasks.workunit.client.0.smithi049.stdout:1/590: mkdir d0/d4/dd/de/d36/dd3 0 2024-05-15T15:54:12.454 INFO:tasks.workunit.client.1.smithi144.stdout:5/514: creat d0/dc/d12/fab x:0 0 0 2024-05-15T15:54:12.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:12 smithi049 ceph-mon[28399]: pgmap v26: 65 pgs: 65 active+clean; 1.5 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 40 MiB/s rd, 185 MiB/s wr, 357 op/s 2024-05-15T15:54:12.456 INFO:tasks.workunit.client.0.smithi049.stdout:1/591: creat d0/d4/dd/de/d1e/d2e/d3e/d90/db3/fd4 x:0 0 0 2024-05-15T15:54:12.456 INFO:tasks.workunit.client.1.smithi144.stdout:1/493: symlink d7/d1d/la2 0 2024-05-15T15:54:12.457 INFO:tasks.workunit.client.0.smithi049.stdout:8/503: sync 2024-05-15T15:54:12.457 INFO:tasks.workunit.client.0.smithi049.stdout:6/446: sync 2024-05-15T15:54:12.457 INFO:tasks.workunit.client.0.smithi049.stdout:5/445: sync 2024-05-15T15:54:12.458 INFO:tasks.workunit.client.0.smithi049.stdout:5/446: readlink dc/l78 0 2024-05-15T15:54:12.460 INFO:tasks.workunit.client.0.smithi049.stdout:2/501: rename d3/d52/l97 to d3/db/d1d/d1e/d2b/d3b/l9d 0 2024-05-15T15:54:12.461 INFO:tasks.workunit.client.1.smithi144.stdout:5/515: getdents d0/dc/d24/d8e/d93/d9e 0 2024-05-15T15:54:12.462 INFO:tasks.workunit.client.1.smithi144.stdout:5/516: dread - d0/dc/d2f/d68/d5c/f80 zero size 2024-05-15T15:54:12.466 INFO:tasks.workunit.client.1.smithi144.stdout:6/428: creat d7/db/d11/fa1 x:0 0 0 2024-05-15T15:54:12.467 INFO:tasks.workunit.client.0.smithi049.stdout:5/447: fsync dc/dd/d10/f1e 0 2024-05-15T15:54:12.468 INFO:tasks.workunit.client.1.smithi144.stdout:4/492: mkdir d6/d42/d46/d9c 0 2024-05-15T15:54:12.469 INFO:tasks.workunit.client.0.smithi049.stdout:1/592: rename d0/d4/dd/de/d48/d84/fca to d0/d4/dd/de/d4d/fd5 0 2024-05-15T15:54:12.472 INFO:tasks.workunit.client.0.smithi049.stdout:2/502: mkdir d3/da/d15/d9e 0 2024-05-15T15:54:12.473 INFO:tasks.workunit.client.1.smithi144.stdout:0/490: creat d4/d2d/fab x:0 0 0 2024-05-15T15:54:12.474 INFO:tasks.workunit.client.1.smithi144.stdout:6/429: write d7/db/d5f/f7a [21061,35628] 0 2024-05-15T15:54:12.476 INFO:tasks.workunit.client.1.smithi144.stdout:5/517: dread d0/dc/d2f/d34/d63/d6e/f7c [0,4194304] 0 2024-05-15T15:54:12.479 INFO:tasks.workunit.client.0.smithi049.stdout:5/448: creat dc/dd/d7b/f90 x:0 0 0 2024-05-15T15:54:12.479 INFO:tasks.workunit.client.0.smithi049.stdout:1/593: dwrite d0/d4/dd/de/d4e/f76 [0,4194304] 0 2024-05-15T15:54:12.480 INFO:tasks.workunit.client.1.smithi144.stdout:6/430: dwrite d7/db/d11/d3c/f77 [0,4194304] 0 2024-05-15T15:54:12.486 INFO:tasks.workunit.client.0.smithi049.stdout:2/503: mkdir d3/db/d1d/d9f 0 2024-05-15T15:54:12.490 INFO:tasks.workunit.client.0.smithi049.stdout:6/447: link dd/d17/d24/d58/d5f/f79 dd/d17/d24/d2b/d36/d50/d54/f7b 0 2024-05-15T15:54:12.491 INFO:tasks.workunit.client.1.smithi144.stdout:0/491: dread d4/d6/f11 [0,4194304] 0 2024-05-15T15:54:12.491 INFO:tasks.workunit.client.0.smithi049.stdout:6/448: chown dd/d26 1169 1 2024-05-15T15:54:12.500 INFO:tasks.workunit.client.0.smithi049.stdout:2/504: creat d3/da/d15/d73/fa0 x:0 0 0 2024-05-15T15:54:12.504 INFO:tasks.workunit.client.0.smithi049.stdout:5/449: dread dc/f1f [0,4194304] 0 2024-05-15T15:54:12.505 INFO:tasks.workunit.client.1.smithi144.stdout:5/518: link d0/dc/d1f/c33 d0/dc/d24/d8e/d93/cac 0 2024-05-15T15:54:12.508 INFO:tasks.workunit.client.0.smithi049.stdout:2/505: rename d3/db/d5b/d63 to d3/db/d82/da1 0 2024-05-15T15:54:12.508 INFO:tasks.workunit.client.0.smithi049.stdout:0/479: sync 2024-05-15T15:54:12.508 INFO:tasks.workunit.client.0.smithi049.stdout:7/381: sync 2024-05-15T15:54:12.508 INFO:tasks.workunit.client.0.smithi049.stdout:3/511: sync 2024-05-15T15:54:12.508 INFO:tasks.workunit.client.0.smithi049.stdout:4/481: sync 2024-05-15T15:54:12.510 INFO:tasks.workunit.client.1.smithi144.stdout:8/539: creat d2/d3a/d7b/d4b/fa9 x:0 0 0 2024-05-15T15:54:12.511 INFO:tasks.workunit.client.0.smithi049.stdout:3/512: chown d12/d14/d52/f7b 102 1 2024-05-15T15:54:12.512 INFO:tasks.workunit.client.1.smithi144.stdout:0/492: mknod d4/df/cac 0 2024-05-15T15:54:12.517 INFO:tasks.workunit.client.0.smithi049.stdout:7/382: dwrite d0/f2e [0,4194304] 0 2024-05-15T15:54:12.518 INFO:tasks.workunit.client.1.smithi144.stdout:6/431: rename d7/db/d11/d55 to d7/db/da2 0 2024-05-15T15:54:12.518 INFO:tasks.workunit.client.0.smithi049.stdout:6/449: dread dd/f32 [0,4194304] 0 2024-05-15T15:54:12.518 INFO:tasks.workunit.client.1.smithi144.stdout:6/432: write d7/d62/f6b [1757385,71836] 0 2024-05-15T15:54:12.520 INFO:tasks.workunit.client.1.smithi144.stdout:5/519: dread d0/dc/f1b [4194304,4194304] 0 2024-05-15T15:54:12.523 INFO:tasks.workunit.client.1.smithi144.stdout:2/482: sync 2024-05-15T15:54:12.523 INFO:tasks.workunit.client.1.smithi144.stdout:2/483: read - d3/d7/f36 zero size 2024-05-15T15:54:12.526 INFO:tasks.workunit.client.1.smithi144.stdout:0/493: mknod d4/d6/d3a/cad 0 2024-05-15T15:54:12.527 INFO:tasks.workunit.client.0.smithi049.stdout:0/480: dwrite dc/d13/d14/d25/f69 [4194304,4194304] 0 2024-05-15T15:54:12.529 INFO:tasks.workunit.client.0.smithi049.stdout:0/481: readlink dc/d13/d14/d33/l5c 0 2024-05-15T15:54:12.529 INFO:tasks.workunit.client.0.smithi049.stdout:0/482: chown dc/d6d/l44 13071 1 2024-05-15T15:54:12.530 INFO:tasks.workunit.client.0.smithi049.stdout:2/506: symlink d3/db/d82/la2 0 2024-05-15T15:54:12.534 INFO:tasks.workunit.client.0.smithi049.stdout:1/594: getdents d0/d4/dd/de/d4d/d8b 0 2024-05-15T15:54:12.540 INFO:tasks.workunit.client.1.smithi144.stdout:2/484: dread d3/d7/f27 [0,4194304] 0 2024-05-15T15:54:12.542 INFO:tasks.workunit.client.1.smithi144.stdout:9/489: dwrite d0/d8/d5e/d44/d10/f58 [0,4194304] 0 2024-05-15T15:54:12.542 INFO:tasks.workunit.client.0.smithi049.stdout:6/450: fsync dd/f32 0 2024-05-15T15:54:12.543 INFO:tasks.workunit.client.1.smithi144.stdout:9/490: chown d0/d8/d5e/d44/d10/d9c/f54 3233094 1 2024-05-15T15:54:12.544 INFO:tasks.workunit.client.1.smithi144.stdout:0/494: mkdir d4/d2d/dae 0 2024-05-15T15:54:12.544 INFO:tasks.workunit.client.1.smithi144.stdout:3/497: write d0/d1/d17/d39/d83/f9d [1110284,128721] 0 2024-05-15T15:54:12.545 INFO:tasks.workunit.client.0.smithi049.stdout:6/451: write dd/d26/f40 [3795174,73231] 0 2024-05-15T15:54:12.545 INFO:tasks.workunit.client.1.smithi144.stdout:3/498: stat d0/d1/d17/l37 0 2024-05-15T15:54:12.546 INFO:tasks.workunit.client.0.smithi049.stdout:9/332: write d0/d8/d1b/d40/f49 [3905057,63698] 0 2024-05-15T15:54:12.547 INFO:tasks.workunit.client.0.smithi049.stdout:4/482: fsync d18/d1e/d2a/d36/d5d/f67 0 2024-05-15T15:54:12.548 INFO:tasks.workunit.client.0.smithi049.stdout:9/333: chown d0/d8/d9/d52/d5e/d5f/f63 246513 1 2024-05-15T15:54:12.549 INFO:tasks.workunit.client.0.smithi049.stdout:9/334: dread - d0/d25/f4c zero size 2024-05-15T15:54:12.551 INFO:tasks.workunit.client.0.smithi049.stdout:9/335: chown d0/d8/d9/d52 22613 1 2024-05-15T15:54:12.553 INFO:tasks.workunit.client.0.smithi049.stdout:9/336: truncate d0/d25/f4c 20697 0 2024-05-15T15:54:12.553 INFO:tasks.workunit.client.0.smithi049.stdout:0/483: rename dc/d13/d5e to dc/d32/d93 0 2024-05-15T15:54:12.555 INFO:tasks.workunit.client.1.smithi144.stdout:1/494: sync 2024-05-15T15:54:12.560 INFO:tasks.workunit.client.1.smithi144.stdout:1/495: dwrite d7/d1d/d33/d41/f83 [0,4194304] 0 2024-05-15T15:54:12.562 INFO:tasks.workunit.client.0.smithi049.stdout:8/504: dwrite d2/de/f55 [0,4194304] 0 2024-05-15T15:54:12.564 INFO:tasks.workunit.client.1.smithi144.stdout:4/493: write f0 [4152717,111965] 0 2024-05-15T15:54:12.568 INFO:tasks.workunit.client.0.smithi049.stdout:6/452: creat dd/d17/d24/d58/d5f/f7c x:0 0 0 2024-05-15T15:54:12.571 INFO:tasks.workunit.client.0.smithi049.stdout:7/383: dread d0/f4 [0,4194304] 0 2024-05-15T15:54:12.572 INFO:tasks.workunit.client.1.smithi144.stdout:0/495: symlink d4/dc/laf 0 2024-05-15T15:54:12.573 INFO:tasks.workunit.client.0.smithi049.stdout:4/483: creat d18/d1e/d63/fa2 x:0 0 0 2024-05-15T15:54:12.574 INFO:tasks.workunit.client.0.smithi049.stdout:4/484: fsync d18/d1e/d2a/d50/f79 0 2024-05-15T15:54:12.574 INFO:tasks.workunit.client.1.smithi144.stdout:7/512: rmdir d4/d41/d9c 39 2024-05-15T15:54:12.576 INFO:tasks.workunit.client.0.smithi049.stdout:9/337: mkdir d0/d8/d1b/d71 0 2024-05-15T15:54:12.579 INFO:tasks.workunit.client.1.smithi144.stdout:3/499: mknod d0/d11/d3b/cad 0 2024-05-15T15:54:12.580 INFO:tasks.workunit.client.1.smithi144.stdout:3/500: fsync d0/d1/f23 0 2024-05-15T15:54:12.581 INFO:tasks.workunit.client.0.smithi049.stdout:0/484: unlink dc/d13/d14/d82/c83 0 2024-05-15T15:54:12.583 INFO:tasks.workunit.client.1.smithi144.stdout:6/433: creat d7/d1e/fa3 x:0 0 0 2024-05-15T15:54:12.584 INFO:tasks.workunit.client.1.smithi144.stdout:6/434: fdatasync d7/db/d9d/d97/f9a 0 2024-05-15T15:54:12.584 INFO:tasks.workunit.client.0.smithi049.stdout:2/507: mkdir d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3 0 2024-05-15T15:54:12.585 INFO:tasks.workunit.client.1.smithi144.stdout:9/491: dread d0/d8/f11 [0,4194304] 0 2024-05-15T15:54:12.587 INFO:tasks.workunit.client.1.smithi144.stdout:9/492: truncate d0/d8/d5e/d44/d10/d9c/fa9 452695 0 2024-05-15T15:54:12.593 INFO:tasks.workunit.client.0.smithi049.stdout:3/513: rename d12/d30/d42/l8d to d12/d30/d42/l9a 0 2024-05-15T15:54:12.595 INFO:tasks.workunit.client.0.smithi049.stdout:8/505: symlink d2/de/d27/l9f 0 2024-05-15T15:54:12.597 INFO:tasks.workunit.client.0.smithi049.stdout:5/450: getdents dc/d4a/d42 0 2024-05-15T15:54:12.598 INFO:tasks.workunit.client.1.smithi144.stdout:1/496: dread d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:12.600 INFO:tasks.workunit.client.1.smithi144.stdout:4/494: dwrite d6/d49/f5c [4194304,4194304] 0 2024-05-15T15:54:12.601 INFO:tasks.workunit.client.0.smithi049.stdout:7/384: rmdir d0/d6/d23 39 2024-05-15T15:54:12.602 INFO:tasks.workunit.client.1.smithi144.stdout:0/496: symlink d4/df/d1f/d24/d85/lb0 0 2024-05-15T15:54:12.606 INFO:tasks.workunit.client.1.smithi144.stdout:7/513: rename d4/df/d10 to d4/d3c/d76/d9d/dac 0 2024-05-15T15:54:12.615 INFO:tasks.workunit.client.1.smithi144.stdout:8/540: truncate d2/de/d3f/f4e 2314312 0 2024-05-15T15:54:12.616 INFO:tasks.workunit.client.0.smithi049.stdout:6/453: dread dd/d26/f3f [0,4194304] 0 2024-05-15T15:54:12.619 INFO:tasks.workunit.client.1.smithi144.stdout:6/435: symlink d7/db/d9d/la4 0 2024-05-15T15:54:12.620 INFO:tasks.workunit.client.1.smithi144.stdout:5/520: link d0/dc/d12/d31/c41 d0/dc/d12/cad 0 2024-05-15T15:54:12.622 INFO:tasks.workunit.client.1.smithi144.stdout:5/521: write d0/dc/d24/d8e/d93/fa7 [612872,85115] 0 2024-05-15T15:54:12.623 INFO:tasks.workunit.client.1.smithi144.stdout:9/493: creat d0/d8/d1b/d62/d7d/d8d/faf x:0 0 0 2024-05-15T15:54:12.624 INFO:tasks.workunit.client.0.smithi049.stdout:5/451: unlink dc/d4a/d31/c85 0 2024-05-15T15:54:12.626 INFO:tasks.workunit.client.1.smithi144.stdout:6/436: dwrite d7/db/d35/d84/f98 [0,4194304] 0 2024-05-15T15:54:12.628 INFO:tasks.workunit.client.0.smithi049.stdout:7/385: chown d0/d6/d23/l68 3124 1 2024-05-15T15:54:12.628 INFO:tasks.workunit.client.0.smithi049.stdout:7/386: chown d0/d6/d1d/l67 183 1 2024-05-15T15:54:12.629 INFO:tasks.workunit.client.1.smithi144.stdout:5/522: dwrite d0/dc/d1f/fa3 [0,4194304] 0 2024-05-15T15:54:12.631 INFO:tasks.workunit.client.1.smithi144.stdout:4/495: creat d6/dd/d34/f9d x:0 0 0 2024-05-15T15:54:12.638 INFO:tasks.workunit.client.1.smithi144.stdout:0/497: truncate d4/df/d1f/d24/d2b/f3b 559788 0 2024-05-15T15:54:12.638 INFO:tasks.workunit.client.1.smithi144.stdout:0/498: write d4/f22 [286097,3073] 0 2024-05-15T15:54:12.641 INFO:tasks.workunit.client.1.smithi144.stdout:7/514: creat d4/d3c/d76/d9d/dac/d48/d72/d82/fad x:0 0 0 2024-05-15T15:54:12.646 INFO:tasks.workunit.client.1.smithi144.stdout:3/501: write d0/f69 [902640,4191] 0 2024-05-15T15:54:12.647 INFO:tasks.workunit.client.1.smithi144.stdout:3/502: readlink d0/d1/d17/d39/d40/l42 0 2024-05-15T15:54:12.647 INFO:tasks.workunit.client.1.smithi144.stdout:3/503: stat d0/d1/c55 0 2024-05-15T15:54:12.651 INFO:tasks.workunit.client.1.smithi144.stdout:6/437: write d7/d62/f95 [6969558,34407] 0 2024-05-15T15:54:12.652 INFO:tasks.workunit.client.1.smithi144.stdout:1/497: rename d7/df/d24 to d7/d1d/d33/d41/d78/da3 0 2024-05-15T15:54:12.653 INFO:tasks.workunit.client.1.smithi144.stdout:4/496: creat d6/dd/d34/d3b/d30/d59/d66/f9e x:0 0 0 2024-05-15T15:54:12.654 INFO:tasks.workunit.client.1.smithi144.stdout:5/523: read d0/dc/d12/d4a/f60 [12297375,26777] 0 2024-05-15T15:54:12.654 INFO:tasks.workunit.client.1.smithi144.stdout:4/497: chown d6/dd/d2e/d94 675 1 2024-05-15T15:54:12.654 INFO:tasks.workunit.client.1.smithi144.stdout:7/515: dread - d4/df/f65 zero size 2024-05-15T15:54:12.656 INFO:tasks.workunit.client.1.smithi144.stdout:3/504: mkdir d0/d1/d17/dae 0 2024-05-15T15:54:12.656 INFO:tasks.workunit.client.1.smithi144.stdout:2/485: link d3/d7/d20/d22/c95 d3/d7/c9e 0 2024-05-15T15:54:12.657 INFO:tasks.workunit.client.1.smithi144.stdout:6/438: creat d7/db/d11/fa5 x:0 0 0 2024-05-15T15:54:12.657 INFO:tasks.workunit.client.1.smithi144.stdout:1/498: mkdir d7/d1d/d33/d29/d6b/da4 0 2024-05-15T15:54:12.658 INFO:tasks.workunit.client.1.smithi144.stdout:6/439: stat d7/l17 0 2024-05-15T15:54:12.658 INFO:tasks.workunit.client.1.smithi144.stdout:3/505: dread - d0/d11/d3b/da0/d8f/f9b zero size 2024-05-15T15:54:12.660 INFO:tasks.workunit.client.1.smithi144.stdout:5/524: dread d0/dc/d1f/f38 [0,4194304] 0 2024-05-15T15:54:12.660 INFO:tasks.workunit.client.1.smithi144.stdout:4/498: rename d6/dd/d34/d3b/d30/db/f35 to d6/dd/d34/d3b/d30/db/d1d/f9f 0 2024-05-15T15:54:12.661 INFO:tasks.workunit.client.1.smithi144.stdout:8/541: creat d2/d24/d2d/faa x:0 0 0 2024-05-15T15:54:12.662 INFO:tasks.workunit.client.1.smithi144.stdout:1/499: creat d7/d1d/fa5 x:0 0 0 2024-05-15T15:54:12.663 INFO:tasks.workunit.client.1.smithi144.stdout:2/486: creat d3/d7/d20/d22/d52/d75/d65/f9f x:0 0 0 2024-05-15T15:54:12.664 INFO:tasks.workunit.client.1.smithi144.stdout:5/525: readlink d0/dc/d24/l83 0 2024-05-15T15:54:12.665 INFO:tasks.workunit.client.1.smithi144.stdout:6/440: mkdir d7/da6 0 2024-05-15T15:54:12.665 INFO:tasks.workunit.client.1.smithi144.stdout:0/499: dread d4/dc/f17 [0,4194304] 0 2024-05-15T15:54:12.666 INFO:tasks.workunit.client.1.smithi144.stdout:1/500: stat d7/d1d/d33/d29/d6b/d8e 0 2024-05-15T15:54:12.666 INFO:tasks.workunit.client.1.smithi144.stdout:7/516: truncate d4/d41/fab 54943 0 2024-05-15T15:54:12.667 INFO:tasks.workunit.client.1.smithi144.stdout:3/506: dwrite d0/f46 [0,4194304] 0 2024-05-15T15:54:12.668 INFO:tasks.workunit.client.1.smithi144.stdout:5/526: unlink d0/d7e/fa8 0 2024-05-15T15:54:12.669 INFO:tasks.workunit.client.1.smithi144.stdout:5/527: fsync d0/f49 0 2024-05-15T15:54:12.670 INFO:tasks.workunit.client.1.smithi144.stdout:6/441: mknod d7/db/d11/ca7 0 2024-05-15T15:54:12.670 INFO:tasks.workunit.client.1.smithi144.stdout:3/507: stat d0/d11/d3b/d93 0 2024-05-15T15:54:12.672 INFO:tasks.workunit.client.1.smithi144.stdout:1/501: creat d7/d1d/d58/fa6 x:0 0 0 2024-05-15T15:54:12.674 INFO:tasks.workunit.client.1.smithi144.stdout:7/517: mknod d4/d41/cae 0 2024-05-15T15:54:12.674 INFO:tasks.workunit.client.1.smithi144.stdout:6/442: chown d7/f8 8976 1 2024-05-15T15:54:12.675 INFO:tasks.workunit.client.1.smithi144.stdout:5/528: readlink d0/dc/d12/d4a/l94 0 2024-05-15T15:54:12.676 INFO:tasks.workunit.client.1.smithi144.stdout:1/502: stat d7/d4b/c57 0 2024-05-15T15:54:12.679 INFO:tasks.workunit.client.1.smithi144.stdout:6/443: unlink f1 0 2024-05-15T15:54:12.681 INFO:tasks.workunit.client.1.smithi144.stdout:8/542: dwrite d2/d24/d2d/da0/d6e/f78 [0,4194304] 0 2024-05-15T15:54:12.681 INFO:tasks.workunit.client.1.smithi144.stdout:2/487: dwrite d3/d7/f47 [0,4194304] 0 2024-05-15T15:54:12.684 INFO:tasks.workunit.client.1.smithi144.stdout:5/529: write d0/f22 [2286887,98104] 0 2024-05-15T15:54:12.685 INFO:tasks.workunit.client.1.smithi144.stdout:0/500: dwrite d4/df/d14/f81 [0,4194304] 0 2024-05-15T15:54:12.686 INFO:tasks.workunit.client.1.smithi144.stdout:8/543: chown d2/de/l50 101 1 2024-05-15T15:54:12.686 INFO:tasks.workunit.client.1.smithi144.stdout:1/503: creat d7/d1d/d33/d41/d22/fa7 x:0 0 0 2024-05-15T15:54:12.686 INFO:tasks.workunit.client.1.smithi144.stdout:6/444: mkdir d7/d1e/d28/da8 0 2024-05-15T15:54:12.690 INFO:tasks.workunit.client.1.smithi144.stdout:7/518: mkdir d4/d3c/d76/d9d/dac/d48/d72/da2/daf 0 2024-05-15T15:54:12.690 INFO:tasks.workunit.client.1.smithi144.stdout:7/519: fdatasync d4/df/d25/f85 0 2024-05-15T15:54:12.690 INFO:tasks.workunit.client.1.smithi144.stdout:2/488: unlink d3/d7/d20/d22/d52/d75/c8d 0 2024-05-15T15:54:12.691 INFO:tasks.workunit.client.1.smithi144.stdout:6/445: creat d7/d62/fa9 x:0 0 0 2024-05-15T15:54:12.692 INFO:tasks.workunit.client.1.smithi144.stdout:5/530: symlink d0/dc/lae 0 2024-05-15T15:54:12.693 INFO:tasks.workunit.client.1.smithi144.stdout:1/504: creat d7/d1d/d33/d41/d22/fa8 x:0 0 0 2024-05-15T15:54:12.693 INFO:tasks.workunit.client.1.smithi144.stdout:5/531: chown d0/dc/d24/d28/d2d/c42 91203 1 2024-05-15T15:54:12.696 INFO:tasks.workunit.client.1.smithi144.stdout:2/489: rename d3/d7/d20/d89/d6e/f83 to d3/d7/d54/fa0 0 2024-05-15T15:54:12.697 INFO:tasks.workunit.client.1.smithi144.stdout:0/501: getdents d4/df/d1f/d56 0 2024-05-15T15:54:12.698 INFO:tasks.workunit.client.1.smithi144.stdout:0/502: dread - d4/df/d1f/d24/f69 zero size 2024-05-15T15:54:12.699 INFO:tasks.workunit.client.1.smithi144.stdout:6/446: creat d7/da6/faa x:0 0 0 2024-05-15T15:54:12.699 INFO:tasks.workunit.client.1.smithi144.stdout:0/503: write d4/d2d/f44 [1680235,14317] 0 2024-05-15T15:54:12.699 INFO:tasks.workunit.client.1.smithi144.stdout:1/505: mknod d7/df/ca9 0 2024-05-15T15:54:12.701 INFO:tasks.workunit.client.1.smithi144.stdout:2/490: unlink d3/d7/d20/d22/d52/d75/f45 0 2024-05-15T15:54:12.701 INFO:tasks.workunit.client.1.smithi144.stdout:2/491: stat d3/d7/f36 0 2024-05-15T15:54:12.703 INFO:tasks.workunit.client.1.smithi144.stdout:6/447: creat d7/d1e/d28/fab x:0 0 0 2024-05-15T15:54:12.704 INFO:tasks.workunit.client.1.smithi144.stdout:0/504: mkdir d4/d6/d4a/d4d/db1 0 2024-05-15T15:54:12.705 INFO:tasks.workunit.client.1.smithi144.stdout:2/492: symlink d3/d7/d20/d22/d52/d75/d65/la1 0 2024-05-15T15:54:12.705 INFO:tasks.workunit.client.1.smithi144.stdout:0/505: stat d4/d6 0 2024-05-15T15:54:12.707 INFO:tasks.workunit.client.1.smithi144.stdout:1/506: dwrite d7/d1d/d33/d29/d6b/d5e/f9d [0,4194304] 0 2024-05-15T15:54:12.707 INFO:tasks.workunit.client.1.smithi144.stdout:6/448: truncate d7/db/d9d/d97/f9a 401332 0 2024-05-15T15:54:12.707 INFO:tasks.workunit.client.1.smithi144.stdout:0/506: symlink d4/df/d14/lb2 0 2024-05-15T15:54:12.708 INFO:tasks.workunit.client.1.smithi144.stdout:6/449: fdatasync d7/d1e/fa3 0 2024-05-15T15:54:12.709 INFO:tasks.workunit.client.1.smithi144.stdout:6/450: stat d7/db/d9d/c5d 0 2024-05-15T15:54:12.714 INFO:tasks.workunit.client.1.smithi144.stdout:2/493: dwrite d3/d7/d20/d22/d52/d75/d65/f9f [0,4194304] 0 2024-05-15T15:54:12.714 INFO:tasks.workunit.client.1.smithi144.stdout:1/507: rename d7/d1d/d33/d29/d6b/d5e/d74/f96 to d7/d1d/d58/faa 0 2024-05-15T15:54:12.715 INFO:tasks.workunit.client.1.smithi144.stdout:0/507: getdents d4/d2d 0 2024-05-15T15:54:12.717 INFO:tasks.workunit.client.1.smithi144.stdout:2/494: read d3/d7/d20/d89/d6e/f8f [788844,1409] 0 2024-05-15T15:54:12.718 INFO:tasks.workunit.client.1.smithi144.stdout:6/451: rename d7/db/d11/d3c/f90 to d7/db/d11/d3c/fac 0 2024-05-15T15:54:12.719 INFO:tasks.workunit.client.1.smithi144.stdout:0/508: unlink d4/df/d1f/f45 0 2024-05-15T15:54:12.720 INFO:tasks.workunit.client.1.smithi144.stdout:6/452: fdatasync d7/fd 0 2024-05-15T15:54:12.721 INFO:tasks.workunit.client.1.smithi144.stdout:6/453: dread - d7/d1e/d28/f96 zero size 2024-05-15T15:54:12.722 INFO:tasks.workunit.client.1.smithi144.stdout:6/454: write d7/d1e/d28/fab [674603,124690] 0 2024-05-15T15:54:12.725 INFO:tasks.workunit.client.1.smithi144.stdout:0/509: rename d4/df/d1f/f3e to d4/df/d1f/fb3 0 2024-05-15T15:54:12.726 INFO:tasks.workunit.client.1.smithi144.stdout:6/455: mknod d7/db/d11/d3c/cad 0 2024-05-15T15:54:12.726 INFO:tasks.workunit.client.1.smithi144.stdout:0/510: symlink d4/df/d14/lb4 0 2024-05-15T15:54:12.728 INFO:tasks.workunit.client.1.smithi144.stdout:0/511: truncate d4/d6/d4a/d74/f99 5231244 0 2024-05-15T15:54:12.731 INFO:tasks.workunit.client.1.smithi144.stdout:6/456: link d7/db/d3a/l93 d7/d1e/d28/da8/lae 0 2024-05-15T15:54:12.735 INFO:tasks.workunit.client.1.smithi144.stdout:0/512: dwrite d4/f22 [0,4194304] 0 2024-05-15T15:54:12.736 INFO:tasks.workunit.client.1.smithi144.stdout:2/495: dread d3/d7/d20/d3b/f4f [0,4194304] 0 2024-05-15T15:54:12.737 INFO:tasks.workunit.client.1.smithi144.stdout:0/513: chown d4/d6/d3a/d40/c7c 77483 1 2024-05-15T15:54:12.739 INFO:tasks.workunit.client.1.smithi144.stdout:6/457: dwrite d7/db/d5f/f7a [0,4194304] 0 2024-05-15T15:54:12.740 INFO:tasks.workunit.client.0.smithi049.stdout:9/338: fdatasync d0/f1d 0 2024-05-15T15:54:12.741 INFO:tasks.workunit.client.0.smithi049.stdout:2/508: rename d3/da/d15/c29 to d3/da/d15/d73/ca4 0 2024-05-15T15:54:12.743 INFO:tasks.workunit.client.1.smithi144.stdout:2/496: creat d3/d7/d54/d98/fa2 x:0 0 0 2024-05-15T15:54:12.743 INFO:tasks.workunit.client.1.smithi144.stdout:0/514: mkdir d4/df/d14/db5 0 2024-05-15T15:54:12.743 INFO:tasks.workunit.client.0.smithi049.stdout:2/509: chown d3/d52 488739 1 2024-05-15T15:54:12.746 INFO:tasks.workunit.client.0.smithi049.stdout:1/595: dwrite d0/d4/dd/de/d1e/f15 [0,4194304] 0 2024-05-15T15:54:12.749 INFO:tasks.workunit.client.1.smithi144.stdout:2/497: dwrite d3/d7/d20/d22/d52/d75/d65/f9a [0,4194304] 0 2024-05-15T15:54:12.749 INFO:tasks.workunit.client.0.smithi049.stdout:5/452: fsync dc/d4a/d42/f5e 0 2024-05-15T15:54:12.750 INFO:tasks.workunit.client.1.smithi144.stdout:2/498: chown d3/d7/d54/d98/fa2 58 1 2024-05-15T15:54:12.751 INFO:tasks.workunit.client.1.smithi144.stdout:6/458: unlink d7/c25 0 2024-05-15T15:54:12.752 INFO:tasks.workunit.client.0.smithi049.stdout:8/506: write d2/f2c [684414,58210] 0 2024-05-15T15:54:12.752 INFO:tasks.workunit.client.0.smithi049.stdout:5/453: dread - dc/dd/d7b/f90 zero size 2024-05-15T15:54:12.752 INFO:tasks.workunit.client.0.smithi049.stdout:3/514: rename d12/d67/f69 to d12/d30/d63/d68/f9b 0 2024-05-15T15:54:12.754 INFO:tasks.workunit.client.0.smithi049.stdout:3/515: chown d12 16 1 2024-05-15T15:54:12.756 INFO:tasks.workunit.client.0.smithi049.stdout:6/454: mknod dd/d17/d24/d2b/d36/d50/c7d 0 2024-05-15T15:54:12.761 INFO:tasks.workunit.client.0.smithi049.stdout:4/485: dwrite d18/f28 [4194304,4194304] 0 2024-05-15T15:54:12.762 INFO:tasks.workunit.client.1.smithi144.stdout:0/515: symlink d4/df/d1f/d24/d85/da1/da3/lb6 0 2024-05-15T15:54:12.768 INFO:tasks.workunit.client.1.smithi144.stdout:9/494: sync 2024-05-15T15:54:12.770 INFO:tasks.workunit.client.0.smithi049.stdout:3/516: mkdir d12/d14/d32/d64/d88/d9c 0 2024-05-15T15:54:12.771 INFO:tasks.workunit.client.0.smithi049.stdout:6/455: dwrite dd/d26/f40 [0,4194304] 0 2024-05-15T15:54:12.776 INFO:tasks.workunit.client.1.smithi144.stdout:4/499: dread d6/d42/d46/d4b/d1e/f51 [0,4194304] 0 2024-05-15T15:54:12.792 INFO:tasks.workunit.client.0.smithi049.stdout:8/507: mknod d2/de/d71/d86/d5d/d7d/d45/ca0 0 2024-05-15T15:54:12.794 INFO:tasks.workunit.client.0.smithi049.stdout:4/486: dread - d18/d1e/d2a/d3b/f47 zero size 2024-05-15T15:54:12.799 INFO:tasks.workunit.client.0.smithi049.stdout:1/596: dread d0/d4/dd/de/d1e/d2e/f41 [0,4194304] 0 2024-05-15T15:54:12.799 INFO:tasks.workunit.client.0.smithi049.stdout:9/339: link d0/d8/d12/c43 d0/d8/d42/c72 0 2024-05-15T15:54:12.799 INFO:tasks.workunit.client.1.smithi144.stdout:3/508: sync 2024-05-15T15:54:12.802 INFO:tasks.workunit.client.1.smithi144.stdout:0/516: mkdir d4/df/d1f/db7 0 2024-05-15T15:54:12.802 INFO:tasks.workunit.client.0.smithi049.stdout:1/597: write d0/d4/dd/de/d4e/fa7 [189555,39755] 0 2024-05-15T15:54:12.803 INFO:tasks.workunit.client.0.smithi049.stdout:4/487: dwrite fa [0,4194304] 0 2024-05-15T15:54:12.805 INFO:tasks.workunit.client.0.smithi049.stdout:3/517: mkdir d12/d30/d63/d68/d70/d75/d9d 0 2024-05-15T15:54:12.807 INFO:tasks.workunit.client.0.smithi049.stdout:3/518: stat d12/d30/d63/d68/d70/d75/f85 0 2024-05-15T15:54:12.808 INFO:tasks.workunit.client.1.smithi144.stdout:0/517: dwrite d4/df/d1f/f58 [0,4194304] 0 2024-05-15T15:54:12.808 INFO:tasks.workunit.client.1.smithi144.stdout:9/495: mkdir d0/d8/d5e/d44/d16/db0 0 2024-05-15T15:54:12.808 INFO:tasks.workunit.client.1.smithi144.stdout:9/496: readlink d0/d8/l2b 0 2024-05-15T15:54:12.809 INFO:tasks.workunit.client.1.smithi144.stdout:0/518: fsync d4/df/d1f/f2c 0 2024-05-15T15:54:12.810 INFO:tasks.workunit.client.1.smithi144.stdout:9/497: readlink d0/d8/d5e/d44/d77/l71 0 2024-05-15T15:54:12.810 INFO:tasks.workunit.client.0.smithi049.stdout:7/387: getdents d0/d6/d23/d4a 0 2024-05-15T15:54:12.811 INFO:tasks.workunit.client.1.smithi144.stdout:0/519: truncate d4/d6/f11 5150498 0 2024-05-15T15:54:12.813 INFO:tasks.workunit.client.0.smithi049.stdout:8/508: symlink d2/de/la1 0 2024-05-15T15:54:12.813 INFO:tasks.workunit.client.0.smithi049.stdout:4/488: dwrite d18/d1e/d2a/d50/f90 [0,4194304] 0 2024-05-15T15:54:12.818 INFO:tasks.workunit.client.1.smithi144.stdout:3/509: unlink d0/d11/d3b/cad 0 2024-05-15T15:54:12.820 INFO:tasks.workunit.client.0.smithi049.stdout:5/454: creat dc/d4a/f91 x:0 0 0 2024-05-15T15:54:12.822 INFO:tasks.workunit.client.0.smithi049.stdout:6/456: mkdir dd/d17/d24/d2b/d36/d50/d7e 0 2024-05-15T15:54:12.825 INFO:tasks.workunit.client.0.smithi049.stdout:9/340: symlink d0/d8/d1b/d40/d5c/l73 0 2024-05-15T15:54:12.827 INFO:tasks.workunit.client.0.smithi049.stdout:1/598: symlink d0/d4/dd/de/d1e/db7/ld6 0 2024-05-15T15:54:12.831 INFO:tasks.workunit.client.1.smithi144.stdout:9/498: dread d0/f14 [0,4194304] 0 2024-05-15T15:54:12.832 INFO:tasks.workunit.client.1.smithi144.stdout:4/500: rmdir d6/d42/d46/d9c 0 2024-05-15T15:54:12.834 INFO:tasks.workunit.client.1.smithi144.stdout:5/532: write d0/f11 [1145280,7542] 0 2024-05-15T15:54:12.836 INFO:tasks.workunit.client.0.smithi049.stdout:7/388: symlink d0/d2f/l77 0 2024-05-15T15:54:12.836 INFO:tasks.workunit.client.0.smithi049.stdout:7/389: readlink d0/d6/da/l1a 0 2024-05-15T15:54:12.839 INFO:tasks.workunit.client.0.smithi049.stdout:4/489: creat d18/d85/fa3 x:0 0 0 2024-05-15T15:54:12.840 INFO:tasks.workunit.client.1.smithi144.stdout:5/533: dwrite d0/dc/d1f/fa3 [0,4194304] 0 2024-05-15T15:54:12.841 INFO:tasks.workunit.client.0.smithi049.stdout:4/490: write d18/d1e/d2a/d3b/f74 [359354,96762] 0 2024-05-15T15:54:12.842 INFO:tasks.workunit.client.0.smithi049.stdout:4/491: write d18/f1a [2968861,41640] 0 2024-05-15T15:54:12.843 INFO:tasks.workunit.client.1.smithi144.stdout:0/520: symlink d4/d6/d4a/d4d/db1/lb8 0 2024-05-15T15:54:12.845 INFO:tasks.workunit.client.0.smithi049.stdout:5/455: write dc/d29/f63 [4108023,51813] 0 2024-05-15T15:54:12.849 INFO:tasks.workunit.client.1.smithi144.stdout:3/510: dread d0/d11/d3b/f45 [0,4194304] 0 2024-05-15T15:54:12.849 INFO:tasks.workunit.client.1.smithi144.stdout:4/501: mknod d6/dd/d2e/d75/ca0 0 2024-05-15T15:54:12.850 INFO:tasks.workunit.client.1.smithi144.stdout:3/511: readlink d0/d11/l47 0 2024-05-15T15:54:12.857 INFO:tasks.workunit.client.1.smithi144.stdout:1/508: truncate f1 12666699 0 2024-05-15T15:54:12.858 INFO:tasks.workunit.client.1.smithi144.stdout:1/509: truncate d7/d1d/d33/d41/d22/f6a 646148 0 2024-05-15T15:54:12.862 INFO:tasks.workunit.client.1.smithi144.stdout:8/544: sync 2024-05-15T15:54:12.862 INFO:tasks.workunit.client.1.smithi144.stdout:8/545: fdatasync d2/de/fa1 0 2024-05-15T15:54:12.868 INFO:tasks.workunit.client.1.smithi144.stdout:7/520: symlink d4/d41/d5b/da0/lb0 0 2024-05-15T15:54:12.870 INFO:tasks.workunit.client.1.smithi144.stdout:5/534: symlink d0/dc/d2f/laf 0 2024-05-15T15:54:12.873 INFO:tasks.workunit.client.1.smithi144.stdout:0/521: fsync d4/df/d1f/d24/d2b/f3b 0 2024-05-15T15:54:12.874 INFO:tasks.workunit.client.1.smithi144.stdout:0/522: chown d4/d6/d3a/d4c/fa6 3861319 1 2024-05-15T15:54:12.875 INFO:tasks.workunit.client.1.smithi144.stdout:1/510: creat d7/d1d/d33/d29/d6b/d5e/d74/fab x:0 0 0 2024-05-15T15:54:12.877 INFO:tasks.workunit.client.0.smithi049.stdout:2/510: write d3/d8/fe [2714258,112298] 0 2024-05-15T15:54:12.877 INFO:tasks.workunit.client.1.smithi144.stdout:3/512: mknod d0/d11/caf 0 2024-05-15T15:54:12.882 INFO:tasks.workunit.client.0.smithi049.stdout:3/519: rmdir d12/d14/d32 39 2024-05-15T15:54:12.883 INFO:tasks.workunit.client.1.smithi144.stdout:2/499: truncate d3/f5 2944719 0 2024-05-15T15:54:12.883 INFO:tasks.workunit.client.1.smithi144.stdout:8/546: dread d2/de/d36/f48 [0,4194304] 0 2024-05-15T15:54:12.885 INFO:tasks.workunit.client.1.smithi144.stdout:6/459: write f6 [1907943,39473] 0 2024-05-15T15:54:12.890 INFO:tasks.workunit.client.1.smithi144.stdout:1/511: dread - d7/d1d/d33/d1e/d87/f92 zero size 2024-05-15T15:54:12.894 INFO:tasks.workunit.client.1.smithi144.stdout:3/513: creat d0/d1/d17/d39/d40/d4a/d77/fb0 x:0 0 0 2024-05-15T15:54:12.895 INFO:tasks.workunit.client.1.smithi144.stdout:2/500: creat d3/d7/d20/d22/d52/fa3 x:0 0 0 2024-05-15T15:54:12.897 INFO:tasks.workunit.client.1.smithi144.stdout:8/547: mknod d2/de/d36/cab 0 2024-05-15T15:54:12.904 INFO:tasks.workunit.client.1.smithi144.stdout:1/512: mkdir d7/d1d/dac 0 2024-05-15T15:54:12.906 INFO:tasks.workunit.client.1.smithi144.stdout:3/514: mknod d0/d11/d3b/cb1 0 2024-05-15T15:54:12.908 INFO:tasks.workunit.client.1.smithi144.stdout:7/521: link d4/df/d25/d38/d68/f8d d4/df/fb1 0 2024-05-15T15:54:12.910 INFO:tasks.workunit.client.1.smithi144.stdout:7/522: write d4/d41/d5b/f88 [2694627,66352] 0 2024-05-15T15:54:12.911 INFO:tasks.workunit.client.1.smithi144.stdout:8/548: chown d2/l34 5392 1 2024-05-15T15:54:12.911 INFO:tasks.workunit.client.1.smithi144.stdout:9/499: dwrite d0/d8/d5e/d44/d10/f24 [0,4194304] 0 2024-05-15T15:54:12.912 INFO:tasks.workunit.client.0.smithi049.stdout:5/456: creat dc/d29/f92 x:0 0 0 2024-05-15T15:54:12.916 INFO:tasks.workunit.client.0.smithi049.stdout:9/341: dwrite d0/d8/d9/f15 [0,4194304] 0 2024-05-15T15:54:12.917 INFO:tasks.workunit.client.1.smithi144.stdout:1/513: readlink d7/d4b/l79 0 2024-05-15T15:54:12.922 INFO:tasks.workunit.client.1.smithi144.stdout:3/515: dread d0/d11/d24/f74 [0,4194304] 0 2024-05-15T15:54:12.924 INFO:tasks.workunit.client.0.smithi049.stdout:2/511: truncate d3/db/d1d/d1e/d2b/d2e/f48 338637 0 2024-05-15T15:54:12.925 INFO:tasks.workunit.client.1.smithi144.stdout:2/501: mkdir d3/d7/da4 0 2024-05-15T15:54:12.926 INFO:tasks.workunit.client.0.smithi049.stdout:3/520: dread - d12/d14/d32/f36 zero size 2024-05-15T15:54:12.926 INFO:tasks.workunit.client.1.smithi144.stdout:2/502: write d3/d7/d20/d3f/f67 [7537523,114671] 0 2024-05-15T15:54:12.929 INFO:tasks.workunit.client.0.smithi049.stdout:8/509: link d2/de/d71/d86/d5d/d7d/l34 d2/de/d71/d86/d5d/la2 0 2024-05-15T15:54:12.929 INFO:tasks.workunit.client.1.smithi144.stdout:4/502: write d6/dd/d34/d3b/d30/db/d1d/d40/d57/f5e [791979,105583] 0 2024-05-15T15:54:12.930 INFO:tasks.workunit.client.1.smithi144.stdout:8/549: symlink d2/de/d36/lac 0 2024-05-15T15:54:12.933 INFO:tasks.workunit.client.1.smithi144.stdout:9/500: creat d0/d8/d5e/d44/d3c/d75/fb1 x:0 0 0 2024-05-15T15:54:12.934 INFO:tasks.workunit.client.1.smithi144.stdout:9/501: write d0/d8/d5e/d44/d99/fa1 [1029155,129325] 0 2024-05-15T15:54:12.935 INFO:tasks.workunit.client.1.smithi144.stdout:9/502: stat d0/d8/f11 0 2024-05-15T15:54:12.935 INFO:tasks.workunit.client.1.smithi144.stdout:9/503: chown d0/d8/d5e/d44/d10/d19 7 1 2024-05-15T15:54:12.936 INFO:tasks.workunit.client.0.smithi049.stdout:5/457: rename dc/dd/d39/f5b to dc/dd/d7b/f93 0 2024-05-15T15:54:12.937 INFO:tasks.workunit.client.1.smithi144.stdout:0/523: link d4/d6/d3a/d4c/c68 d4/d2d/d48/cb9 0 2024-05-15T15:54:12.937 INFO:tasks.workunit.client.0.smithi049.stdout:5/458: write dc/d4a/f91 [426956,71057] 0 2024-05-15T15:54:12.939 INFO:tasks.workunit.client.0.smithi049.stdout:5/459: write dc/dd/f1d [6111288,84177] 0 2024-05-15T15:54:12.942 INFO:tasks.workunit.client.1.smithi144.stdout:3/516: write d0/d11/d24/f41 [151174,26077] 0 2024-05-15T15:54:12.944 INFO:tasks.workunit.client.0.smithi049.stdout:2/512: creat d3/db/d82/da1/fa5 x:0 0 0 2024-05-15T15:54:12.946 INFO:tasks.workunit.client.1.smithi144.stdout:2/503: chown d3/d7/d20/d22/d52/d64/l41 297 1 2024-05-15T15:54:12.948 INFO:tasks.workunit.client.0.smithi049.stdout:3/521: unlink d12/d14/d32/f36 0 2024-05-15T15:54:12.949 INFO:tasks.workunit.client.0.smithi049.stdout:8/510: creat d2/de/fa3 x:0 0 0 2024-05-15T15:54:12.949 INFO:tasks.workunit.client.0.smithi049.stdout:8/511: stat d2/f2c 0 2024-05-15T15:54:12.950 INFO:tasks.workunit.client.0.smithi049.stdout:8/512: read - d2/de/d71/f84 zero size 2024-05-15T15:54:12.950 INFO:tasks.workunit.client.1.smithi144.stdout:8/550: mknod d2/de/d36/d87/cad 0 2024-05-15T15:54:12.951 INFO:tasks.workunit.client.1.smithi144.stdout:5/535: dwrite d0/dc/d12/f18 [4194304,4194304] 0 2024-05-15T15:54:12.957 INFO:tasks.workunit.client.0.smithi049.stdout:1/599: getdents d0/d4/dd/de/d36 0 2024-05-15T15:54:12.959 INFO:tasks.workunit.client.0.smithi049.stdout:7/390: getdents d0/d2f/d5c 0 2024-05-15T15:54:12.962 INFO:tasks.workunit.client.0.smithi049.stdout:2/513: truncate d3/f4d 455780 0 2024-05-15T15:54:12.964 INFO:tasks.workunit.client.0.smithi049.stdout:2/514: read d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d [3508646,35305] 0 2024-05-15T15:54:12.965 INFO:tasks.workunit.client.0.smithi049.stdout:3/522: mknod d12/d30/d66/c9e 0 2024-05-15T15:54:12.966 INFO:tasks.workunit.client.0.smithi049.stdout:3/523: stat d12/d30/d63/d68 0 2024-05-15T15:54:12.966 INFO:tasks.workunit.client.0.smithi049.stdout:3/524: dread - d12/d26/f7d zero size 2024-05-15T15:54:12.967 INFO:tasks.workunit.client.1.smithi144.stdout:3/517: mknod d0/d1/d56/cb2 0 2024-05-15T15:54:12.968 INFO:tasks.workunit.client.1.smithi144.stdout:3/518: readlink d0/d11/l1f 0 2024-05-15T15:54:12.969 INFO:tasks.workunit.client.0.smithi049.stdout:9/342: creat d0/d8/d9/f74 x:0 0 0 2024-05-15T15:54:12.971 INFO:tasks.workunit.client.1.smithi144.stdout:2/504: rename d3/d7/d20/d22/d5e/f6a to d3/d7/d20/d40/fa5 0 2024-05-15T15:54:12.971 INFO:tasks.workunit.client.0.smithi049.stdout:5/460: getdents dc/dd/d7b/d8d 0 2024-05-15T15:54:12.973 INFO:tasks.workunit.client.1.smithi144.stdout:3/519: dwrite d0/d1/d56/f87 [0,4194304] 0 2024-05-15T15:54:12.974 INFO:tasks.workunit.client.1.smithi144.stdout:5/536: creat d0/dc/d24/d8e/d95/fb0 x:0 0 0 2024-05-15T15:54:12.976 INFO:tasks.workunit.client.0.smithi049.stdout:6/457: write dd/d17/d64/f6a [111781,104733] 0 2024-05-15T15:54:12.981 INFO:tasks.workunit.client.1.smithi144.stdout:6/460: dwrite d7/db/d11/d3c/fac [0,4194304] 0 2024-05-15T15:54:12.981 INFO:tasks.workunit.client.1.smithi144.stdout:2/505: mkdir d3/d7/d20/d22/d52/d75/da6 0 2024-05-15T15:54:12.984 INFO:tasks.workunit.client.1.smithi144.stdout:3/520: write d0/d11/f14 [324111,37866] 0 2024-05-15T15:54:12.986 INFO:tasks.workunit.client.0.smithi049.stdout:3/525: rmdir d12/d30/d63/d68/d70 39 2024-05-15T15:54:12.986 INFO:tasks.workunit.client.1.smithi144.stdout:7/523: write d4/f21 [878086,5902] 0 2024-05-15T15:54:12.987 INFO:tasks.workunit.client.1.smithi144.stdout:7/524: chown d4/f12 102485585 1 2024-05-15T15:54:12.988 INFO:tasks.workunit.client.1.smithi144.stdout:5/537: symlink d0/dc/d1f/lb1 0 2024-05-15T15:54:12.990 INFO:tasks.workunit.client.0.smithi049.stdout:9/343: mkdir d0/d8/d42/d75 0 2024-05-15T15:54:12.991 INFO:tasks.workunit.client.0.smithi049.stdout:6/458: dread dd/d26/f2e [0,4194304] 0 2024-05-15T15:54:12.991 INFO:tasks.workunit.client.0.smithi049.stdout:6/459: dread - dd/f6b zero size 2024-05-15T15:54:12.993 INFO:tasks.workunit.client.0.smithi049.stdout:4/492: truncate d18/d1e/d2a/d50/f90 3758086 0 2024-05-15T15:54:12.994 INFO:tasks.workunit.client.0.smithi049.stdout:4/493: chown d18/d21/c2d 424535362 1 2024-05-15T15:54:12.996 INFO:tasks.workunit.client.0.smithi049.stdout:6/460: dread dd/f53 [0,4194304] 0 2024-05-15T15:54:12.998 INFO:tasks.workunit.client.1.smithi144.stdout:2/506: mknod d3/d7/d20/d22/d52/ca7 0 2024-05-15T15:54:13.001 INFO:tasks.workunit.client.1.smithi144.stdout:3/521: rename d0/d1/cd to d0/d1/d6c/cb3 0 2024-05-15T15:54:13.003 INFO:tasks.workunit.client.1.smithi144.stdout:1/514: dwrite d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:13.003 INFO:tasks.workunit.client.1.smithi144.stdout:5/538: creat d0/dc/d24/d8e/d93/fb2 x:0 0 0 2024-05-15T15:54:13.004 INFO:tasks.workunit.client.1.smithi144.stdout:5/539: chown d0/dc/l9c 452688645 1 2024-05-15T15:54:13.006 INFO:tasks.workunit.client.1.smithi144.stdout:5/540: read d0/dc/d2f/d68/d5c/f81 [52704,17846] 0 2024-05-15T15:54:13.007 INFO:tasks.workunit.client.1.smithi144.stdout:8/551: write d2/d3a/f3e [825628,39313] 0 2024-05-15T15:54:13.011 INFO:tasks.workunit.client.1.smithi144.stdout:7/525: dread d4/f67 [0,4194304] 0 2024-05-15T15:54:13.013 INFO:tasks.workunit.client.1.smithi144.stdout:1/515: dwrite d7/d1d/d33/d41/d22/f90 [0,4194304] 0 2024-05-15T15:54:13.013 INFO:tasks.workunit.client.1.smithi144.stdout:9/504: dwrite d0/f5 [0,4194304] 0 2024-05-15T15:54:13.013 INFO:tasks.workunit.client.1.smithi144.stdout:7/526: chown d4/d3c/d76 1 1 2024-05-15T15:54:13.014 INFO:tasks.workunit.client.1.smithi144.stdout:3/522: symlink d0/d1/d17/d39/d40/d4a/lb4 0 2024-05-15T15:54:13.016 INFO:tasks.workunit.client.1.smithi144.stdout:9/505: truncate d0/d8/d5e/fa4 580746 0 2024-05-15T15:54:13.016 INFO:tasks.workunit.client.1.smithi144.stdout:7/527: truncate d4/df/d25/d38/d5d/f84 1020597 0 2024-05-15T15:54:13.018 INFO:tasks.workunit.client.0.smithi049.stdout:5/461: creat dc/dd/d7b/d8d/f94 x:0 0 0 2024-05-15T15:54:13.024 INFO:tasks.workunit.client.0.smithi049.stdout:4/494: rename d18/d85/c8c to d18/d1e/d2a/d50/d54/ca4 0 2024-05-15T15:54:13.026 INFO:tasks.workunit.client.1.smithi144.stdout:5/541: mkdir d0/dc/d1f/d50/db3 0 2024-05-15T15:54:13.028 INFO:tasks.workunit.client.0.smithi049.stdout:6/461: readlink dd/d17/d24/l35 0 2024-05-15T15:54:13.029 INFO:tasks.workunit.client.0.smithi049.stdout:6/462: read dd/f4c [55910,109339] 0 2024-05-15T15:54:13.029 INFO:tasks.workunit.client.0.smithi049.stdout:1/600: truncate d0/d4/dd/de/d1e/d2e/dad/d65/f6f 87043 0 2024-05-15T15:54:13.030 INFO:tasks.workunit.client.1.smithi144.stdout:3/523: dread d0/d11/d24/f41 [0,4194304] 0 2024-05-15T15:54:13.031 INFO:tasks.workunit.client.0.smithi049.stdout:7/391: link d0/l50 d0/d6/da/d3d/l78 0 2024-05-15T15:54:13.032 INFO:tasks.workunit.client.1.smithi144.stdout:3/524: read d0/d1/d17/f91 [27576,8675] 0 2024-05-15T15:54:13.032 INFO:tasks.workunit.client.1.smithi144.stdout:6/461: write d7/db/f48 [850748,11620] 0 2024-05-15T15:54:13.035 INFO:tasks.workunit.client.0.smithi049.stdout:0/485: sync 2024-05-15T15:54:13.036 INFO:tasks.workunit.client.1.smithi144.stdout:7/528: rmdir d4/df/d25/d40/d62 39 2024-05-15T15:54:13.036 INFO:tasks.workunit.client.0.smithi049.stdout:0/486: write dc/d13/d14/d25/d31/f75 [1412420,104902] 0 2024-05-15T15:54:13.039 INFO:tasks.workunit.client.0.smithi049.stdout:2/515: creat d3/fa6 x:0 0 0 2024-05-15T15:54:13.040 INFO:tasks.workunit.client.1.smithi144.stdout:9/506: symlink d0/d8/d1b/d62/d7d/d8d/lb2 0 2024-05-15T15:54:13.041 INFO:tasks.workunit.client.0.smithi049.stdout:8/513: link d2/d18/l20 d2/la4 0 2024-05-15T15:54:13.042 INFO:tasks.workunit.client.1.smithi144.stdout:7/529: dwrite d4/f1c [0,4194304] 0 2024-05-15T15:54:13.045 INFO:tasks.workunit.client.0.smithi049.stdout:9/344: write d0/f1 [628801,123589] 0 2024-05-15T15:54:13.046 INFO:tasks.workunit.client.1.smithi144.stdout:1/516: mkdir d7/d1d/dac/dad 0 2024-05-15T15:54:13.050 INFO:tasks.workunit.client.0.smithi049.stdout:5/462: rename dc/f19 to dc/d4a/d31/f95 0 2024-05-15T15:54:13.052 INFO:tasks.workunit.client.1.smithi144.stdout:3/525: mkdir d0/d11/d3b/db5 0 2024-05-15T15:54:13.058 INFO:tasks.workunit.client.1.smithi144.stdout:6/462: dread d7/d1e/d28/f3d [0,4194304] 0 2024-05-15T15:54:13.060 INFO:tasks.workunit.client.0.smithi049.stdout:0/487: truncate dc/d13/d56/f90 199998 0 2024-05-15T15:54:13.061 INFO:tasks.workunit.client.1.smithi144.stdout:9/507: chown d0/d8/d5e/d44/d10/d19/f67 74694586 1 2024-05-15T15:54:13.062 INFO:tasks.workunit.client.1.smithi144.stdout:9/508: write d0/d8/d5e/d44/d10/d9c/f54 [491839,64073] 0 2024-05-15T15:54:13.063 INFO:tasks.workunit.client.0.smithi049.stdout:2/516: creat d3/db/d1d/d1e/d2b/d3b/fa7 x:0 0 0 2024-05-15T15:54:13.064 INFO:tasks.workunit.client.1.smithi144.stdout:2/507: getdents d3/d7/d20/d37/d76 0 2024-05-15T15:54:13.065 INFO:tasks.workunit.client.0.smithi049.stdout:8/514: mkdir d2/de/d27/d5b/d60/da5 0 2024-05-15T15:54:13.065 INFO:tasks.workunit.client.0.smithi049.stdout:1/601: dread d0/d4/dd/de/d1e/d16/d8a/f8f [0,4194304] 0 2024-05-15T15:54:13.070 INFO:tasks.workunit.client.0.smithi049.stdout:9/345: creat d0/d25/f76 x:0 0 0 2024-05-15T15:54:13.071 INFO:tasks.workunit.client.1.smithi144.stdout:5/542: symlink d0/dc/d12/d4a/lb4 0 2024-05-15T15:54:13.072 INFO:tasks.workunit.client.1.smithi144.stdout:5/543: dread - d0/dc/d24/d28/f82 zero size 2024-05-15T15:54:13.072 INFO:tasks.workunit.client.1.smithi144.stdout:5/544: readlink d0/dc/d12/d4a/l94 0 2024-05-15T15:54:13.073 INFO:tasks.workunit.client.1.smithi144.stdout:1/517: dread d7/d1d/d33/d55/f8d [0,4194304] 0 2024-05-15T15:54:13.079 INFO:tasks.workunit.client.1.smithi144.stdout:1/518: dwrite d7/d1d/d33/d41/d22/fa7 [0,4194304] 0 2024-05-15T15:54:13.080 INFO:tasks.workunit.client.1.smithi144.stdout:8/552: truncate d2/f3 1286177 0 2024-05-15T15:54:13.081 INFO:tasks.workunit.client.1.smithi144.stdout:9/509: fsync d0/d8/d5e/d44/d3c/f61 0 2024-05-15T15:54:13.082 INFO:tasks.workunit.client.1.smithi144.stdout:9/510: fdatasync d0/d8/d5e/d44/d16/f46 0 2024-05-15T15:54:13.083 INFO:tasks.workunit.client.1.smithi144.stdout:2/508: mknod d3/d7/d20/d22/d52/ca8 0 2024-05-15T15:54:13.083 INFO:tasks.workunit.client.1.smithi144.stdout:2/509: write d3/f9 [2838480,77820] 0 2024-05-15T15:54:13.084 INFO:tasks.workunit.client.1.smithi144.stdout:2/510: readlink d3/d7/d54/l8a 0 2024-05-15T15:54:13.086 INFO:tasks.workunit.client.1.smithi144.stdout:2/511: write d3/d7/d54/f85 [977638,123073] 0 2024-05-15T15:54:13.087 INFO:tasks.workunit.client.1.smithi144.stdout:5/545: rename d0/dc/d24/d8e/d93/l7a to d0/dc/d1f/d65/lb5 0 2024-05-15T15:54:13.089 INFO:tasks.workunit.client.1.smithi144.stdout:3/526: dwrite d0/d1/d17/d39/d40/d7e/f81 [0,4194304] 0 2024-05-15T15:54:13.090 INFO:tasks.workunit.client.1.smithi144.stdout:5/546: dread - d0/dc/d24/d8e/d93/fa1 zero size 2024-05-15T15:54:13.097 INFO:tasks.workunit.client.1.smithi144.stdout:9/511: rename d0/c20 to d0/d8/d1b/cb3 0 2024-05-15T15:54:13.098 INFO:tasks.workunit.client.1.smithi144.stdout:9/512: dread - d0/d8/d5e/f9a zero size 2024-05-15T15:54:13.098 INFO:tasks.workunit.client.1.smithi144.stdout:1/519: mknod d7/d4b/cae 0 2024-05-15T15:54:13.098 INFO:tasks.workunit.client.1.smithi144.stdout:8/553: rmdir d2/d1a 39 2024-05-15T15:54:13.100 INFO:tasks.workunit.client.1.smithi144.stdout:2/512: symlink d3/d7/la9 0 2024-05-15T15:54:13.102 INFO:tasks.workunit.client.1.smithi144.stdout:5/547: mkdir d0/dc/d24/d28/d2d/d59/d74/db6 0 2024-05-15T15:54:13.102 INFO:tasks.workunit.client.1.smithi144.stdout:9/513: fsync d0/d8/d5e/d44/d10/d19/f67 0 2024-05-15T15:54:13.103 INFO:tasks.workunit.client.1.smithi144.stdout:3/527: link d0/d1/d6c/l90 d0/d1/d6c/lb6 0 2024-05-15T15:54:13.103 INFO:tasks.workunit.client.1.smithi144.stdout:5/548: dread d0/dc/f71 [0,4194304] 0 2024-05-15T15:54:13.105 INFO:tasks.workunit.client.1.smithi144.stdout:1/520: read d7/d1d/f32 [3858979,63372] 0 2024-05-15T15:54:13.105 INFO:tasks.workunit.client.1.smithi144.stdout:3/528: creat d0/d1/d6c/fb7 x:0 0 0 2024-05-15T15:54:13.106 INFO:tasks.workunit.client.1.smithi144.stdout:3/529: chown d0/d11/l47 833210202 1 2024-05-15T15:54:13.107 INFO:tasks.workunit.client.1.smithi144.stdout:1/521: write d7/d31/f73 [1385646,64284] 0 2024-05-15T15:54:13.107 INFO:tasks.workunit.client.1.smithi144.stdout:9/514: rename d0/d8/d1b/d62/d7d/d8d/ca6 to d0/d8/d1b/d62/d7d/d8d/cb4 0 2024-05-15T15:54:13.108 INFO:tasks.workunit.client.1.smithi144.stdout:1/522: chown d7/df/ca9 0 1 2024-05-15T15:54:13.111 INFO:tasks.workunit.client.1.smithi144.stdout:9/515: fsync d0/d8/d5e/d44/f95 0 2024-05-15T15:54:13.112 INFO:tasks.workunit.client.1.smithi144.stdout:9/516: write d0/d8/d5e/d44/d99/fa1 [167762,110303] 0 2024-05-15T15:54:13.114 INFO:tasks.workunit.client.1.smithi144.stdout:5/549: link d0/dc/d12/d4a/c56 d0/dc/d24/d28/cb7 0 2024-05-15T15:54:13.116 INFO:tasks.workunit.client.1.smithi144.stdout:9/517: mknod d0/d8/d5e/d44/d99/cb5 0 2024-05-15T15:54:13.117 INFO:tasks.workunit.client.0.smithi049.stdout:5/463: readlink dc/lf 0 2024-05-15T15:54:13.118 INFO:tasks.workunit.client.1.smithi144.stdout:9/518: creat d0/d8/d5e/fb6 x:0 0 0 2024-05-15T15:54:13.119 INFO:tasks.workunit.client.0.smithi049.stdout:4/495: creat d18/d1e/d63/d99/fa5 x:0 0 0 2024-05-15T15:54:13.120 INFO:tasks.workunit.client.0.smithi049.stdout:2/517: mkdir d3/db/d1d/d1e/d2b/d3b/d42/da8 0 2024-05-15T15:54:13.120 INFO:tasks.workunit.client.0.smithi049.stdout:1/602: dread d0/f19 [4194304,4194304] 0 2024-05-15T15:54:13.122 INFO:tasks.workunit.client.0.smithi049.stdout:3/526: getdents d12/d26/d73 0 2024-05-15T15:54:13.122 INFO:tasks.workunit.client.1.smithi144.stdout:8/554: write d2/de/d3f/f4e [3138773,12729] 0 2024-05-15T15:54:13.130 INFO:tasks.workunit.client.1.smithi144.stdout:3/530: dwrite d0/d1/d17/f20 [0,4194304] 0 2024-05-15T15:54:13.130 INFO:tasks.workunit.client.1.smithi144.stdout:3/531: fdatasync d0/d11/f2a 0 2024-05-15T15:54:13.131 INFO:tasks.workunit.client.1.smithi144.stdout:3/532: write d0/f46 [3505937,32982] 0 2024-05-15T15:54:13.133 INFO:tasks.workunit.client.1.smithi144.stdout:4/503: sync 2024-05-15T15:54:13.133 INFO:tasks.workunit.client.1.smithi144.stdout:5/550: truncate d0/dc/d24/d57/f77 953118 0 2024-05-15T15:54:13.139 INFO:tasks.workunit.client.1.smithi144.stdout:8/555: creat d2/d24/d2d/da0/d9e/fae x:0 0 0 2024-05-15T15:54:13.145 INFO:tasks.workunit.client.1.smithi144.stdout:4/504: dwrite d6/dd/d34/d3b/f5d [0,4194304] 0 2024-05-15T15:54:13.147 INFO:tasks.workunit.client.1.smithi144.stdout:8/556: dwrite d2/de/fa1 [0,4194304] 0 2024-05-15T15:54:13.147 INFO:tasks.workunit.client.1.smithi144.stdout:9/519: link d0/d8/d21/c70 d0/d8/d5e/d44/d10/d19/cb7 0 2024-05-15T15:54:13.148 INFO:tasks.workunit.client.1.smithi144.stdout:4/505: dread d6/d42/d46/d4b/d1e/f51 [0,4194304] 0 2024-05-15T15:54:13.150 INFO:tasks.workunit.client.1.smithi144.stdout:5/551: mkdir d0/dc/d2f/d34/d63/d6e/db8 0 2024-05-15T15:54:13.150 INFO:tasks.workunit.client.1.smithi144.stdout:9/520: write d0/d8/d1b/d79/d8a/f97 [1591842,48804] 0 2024-05-15T15:54:13.151 INFO:tasks.workunit.client.1.smithi144.stdout:8/557: chown d2/d24/d2d/da0/d6e/d71/d73/c85 33380728 1 2024-05-15T15:54:13.154 INFO:tasks.workunit.client.1.smithi144.stdout:5/552: truncate d0/dc/d24/d8e/d93/fa1 789312 0 2024-05-15T15:54:13.155 INFO:tasks.workunit.client.1.smithi144.stdout:8/558: creat d2/de/d36/faf x:0 0 0 2024-05-15T15:54:13.156 INFO:tasks.workunit.client.1.smithi144.stdout:3/533: getdents d0/d11/d3b 0 2024-05-15T15:54:13.157 INFO:tasks.workunit.client.1.smithi144.stdout:8/559: chown d2/d24/d2d/l6c 1010253 1 2024-05-15T15:54:13.160 INFO:tasks.workunit.client.1.smithi144.stdout:8/560: mknod d2/d24/d2d/da0/cb0 0 2024-05-15T15:54:13.161 INFO:tasks.workunit.client.1.smithi144.stdout:9/521: creat d0/d8/fb8 x:0 0 0 2024-05-15T15:54:13.162 INFO:tasks.workunit.client.1.smithi144.stdout:4/506: getdents d6/d42 0 2024-05-15T15:54:13.162 INFO:tasks.workunit.client.1.smithi144.stdout:9/522: read - d0/d8/d1b/d62/fa3 zero size 2024-05-15T15:54:13.163 INFO:tasks.workunit.client.1.smithi144.stdout:8/561: mkdir d2/d24/d2d/da0/d6e/d71/db1 0 2024-05-15T15:54:13.164 INFO:tasks.workunit.client.1.smithi144.stdout:8/562: truncate d2/de/d36/f95 1410676 0 2024-05-15T15:54:13.164 INFO:tasks.workunit.client.1.smithi144.stdout:4/507: mknod d6/d42/d46/d4b/d1e/d44/ca1 0 2024-05-15T15:54:13.168 INFO:tasks.workunit.client.1.smithi144.stdout:8/563: creat d2/d3a/d7b/d4b/fb2 x:0 0 0 2024-05-15T15:54:13.168 INFO:tasks.workunit.client.1.smithi144.stdout:9/523: getdents d0/d8/d21 0 2024-05-15T15:54:13.170 INFO:tasks.workunit.client.1.smithi144.stdout:9/524: read d0/d8/d5e/d44/d99/fa1 [498919,51920] 0 2024-05-15T15:54:13.170 INFO:tasks.workunit.client.1.smithi144.stdout:8/564: mknod d2/d24/d2d/da0/d6e/d8d/cb3 0 2024-05-15T15:54:13.172 INFO:tasks.workunit.client.0.smithi049.stdout:9/346: dwrite d0/f65 [0,4194304] 0 2024-05-15T15:54:13.173 INFO:tasks.workunit.client.1.smithi144.stdout:9/525: unlink d0/d8/d1b/d62/d6b/l73 0 2024-05-15T15:54:13.173 INFO:tasks.workunit.client.0.smithi049.stdout:6/463: rename dd/d17/d24/l28 to dd/d17/d24/d58/d5f/l7f 0 2024-05-15T15:54:13.174 INFO:tasks.workunit.client.1.smithi144.stdout:9/526: truncate d0/d8/d5e/fb6 582680 0 2024-05-15T15:54:13.176 INFO:tasks.workunit.client.1.smithi144.stdout:8/565: creat d2/d24/d2d/da0/d6e/d8d/fb4 x:0 0 0 2024-05-15T15:54:13.176 INFO:tasks.workunit.client.0.smithi049.stdout:8/515: dread d2/fb [0,4194304] 0 2024-05-15T15:54:13.177 INFO:tasks.workunit.client.0.smithi049.stdout:3/527: write d12/d14/d52/f5f [3094361,35262] 0 2024-05-15T15:54:13.177 INFO:tasks.workunit.client.0.smithi049.stdout:7/392: link d0/l50 d0/d34/l79 0 2024-05-15T15:54:13.179 INFO:tasks.workunit.client.1.smithi144.stdout:9/527: truncate d0/d8/d5e/d44/d10/d9c/f3f 451305 0 2024-05-15T15:54:13.180 INFO:tasks.workunit.client.1.smithi144.stdout:9/528: readlink d0/l15 0 2024-05-15T15:54:13.180 INFO:tasks.workunit.client.0.smithi049.stdout:1/603: write d0/d4/dd/de/d1e/d2e/d3e/d90/db3/fc4 [126665,13443] 0 2024-05-15T15:54:13.181 INFO:tasks.workunit.client.0.smithi049.stdout:7/393: stat d0/d6/d1d/l67 0 2024-05-15T15:54:13.185 INFO:tasks.workunit.client.0.smithi049.stdout:2/518: dwrite d3/d78/d8c/d91/f9c [0,4194304] 0 2024-05-15T15:54:13.186 INFO:tasks.workunit.client.0.smithi049.stdout:2/519: chown d3/f11 2 1 2024-05-15T15:54:13.186 INFO:tasks.workunit.client.1.smithi144.stdout:5/553: dwrite d0/dc/d12/d4a/f60 [0,4194304] 0 2024-05-15T15:54:13.188 INFO:tasks.workunit.client.1.smithi144.stdout:9/529: fsync d0/d8/d5e/d44/d10/d19/f63 0 2024-05-15T15:54:13.190 INFO:tasks.workunit.client.0.smithi049.stdout:4/496: creat d18/d1e/fa6 x:0 0 0 2024-05-15T15:54:13.191 INFO:tasks.workunit.client.1.smithi144.stdout:3/534: dwrite d0/d1/d17/f91 [0,4194304] 0 2024-05-15T15:54:13.191 INFO:tasks.workunit.client.0.smithi049.stdout:4/497: read d18/d1e/d63/f96 [596077,19817] 0 2024-05-15T15:54:13.192 INFO:tasks.workunit.client.0.smithi049.stdout:4/498: chown d18/d69/c97 6993 1 2024-05-15T15:54:13.195 INFO:tasks.workunit.client.1.smithi144.stdout:9/530: write d0/d8/d21/f2a [181038,99119] 0 2024-05-15T15:54:13.195 INFO:tasks.workunit.client.0.smithi049.stdout:5/464: rename dc/d29/l6b to dc/d4a/l96 0 2024-05-15T15:54:13.197 INFO:tasks.workunit.client.0.smithi049.stdout:2/520: fdatasync d3/da/d15/d24/f64 0 2024-05-15T15:54:13.198 INFO:tasks.workunit.client.0.smithi049.stdout:6/464: creat dd/f80 x:0 0 0 2024-05-15T15:54:13.198 INFO:tasks.workunit.client.1.smithi144.stdout:4/508: dwrite d6/dd/d34/d3b/d30/db/f7b [0,4194304] 0 2024-05-15T15:54:13.200 INFO:tasks.workunit.client.1.smithi144.stdout:8/566: getdents d2/d1a 0 2024-05-15T15:54:13.205 INFO:tasks.workunit.client.1.smithi144.stdout:4/509: dwrite d6/dd/d2e/d71/f8e [0,4194304] 0 2024-05-15T15:54:13.209 INFO:tasks.workunit.client.0.smithi049.stdout:7/394: dread d0/d34/f35 [0,4194304] 0 2024-05-15T15:54:13.214 INFO:tasks.workunit.client.1.smithi144.stdout:4/510: dwrite d6/f95 [0,4194304] 0 2024-05-15T15:54:13.214 INFO:tasks.workunit.client.1.smithi144.stdout:9/531: creat d0/d8/d1b/fb9 x:0 0 0 2024-05-15T15:54:13.215 INFO:tasks.workunit.client.1.smithi144.stdout:9/532: chown d0/f59 2406820 1 2024-05-15T15:54:13.221 INFO:tasks.workunit.client.1.smithi144.stdout:8/567: creat d2/d3a/d7b/d4b/fb5 x:0 0 0 2024-05-15T15:54:13.222 INFO:tasks.workunit.client.1.smithi144.stdout:8/568: chown d2/de/d3f/f45 1327 1 2024-05-15T15:54:13.223 INFO:tasks.workunit.client.0.smithi049.stdout:9/347: rename d0/d8/d9/f15 to d0/d8/d1b/d40/f77 0 2024-05-15T15:54:13.226 INFO:tasks.workunit.client.0.smithi049.stdout:9/348: write d0/d8/d9/d52/d5e/d5f/f55 [3553490,54789] 0 2024-05-15T15:54:13.227 INFO:tasks.workunit.client.0.smithi049.stdout:9/349: stat d0/d8/d21/f2f 0 2024-05-15T15:54:13.228 INFO:tasks.workunit.client.0.smithi049.stdout:9/350: write d0/f41 [1540590,44930] 0 2024-05-15T15:54:13.234 INFO:tasks.workunit.client.1.smithi144.stdout:4/511: unlink d6/dd/d2e/d75/l88 0 2024-05-15T15:54:13.234 INFO:tasks.workunit.client.1.smithi144.stdout:4/512: readlink d6/dd/d2d/l72 0 2024-05-15T15:54:13.236 INFO:tasks.workunit.client.1.smithi144.stdout:9/533: rename d0/l88 to d0/d8/d5e/d44/d4a/lba 0 2024-05-15T15:54:13.237 INFO:tasks.workunit.client.1.smithi144.stdout:9/534: fsync d0/d8/d5e/d44/d10/f81 0 2024-05-15T15:54:13.238 INFO:tasks.workunit.client.1.smithi144.stdout:9/535: dread - d0/d8/d5e/d44/d3c/fa5 zero size 2024-05-15T15:54:13.239 INFO:tasks.workunit.client.0.smithi049.stdout:3/528: link d12/f29 d12/d14/d32/d64/d88/f9f 0 2024-05-15T15:54:13.239 INFO:tasks.workunit.client.1.smithi144.stdout:4/513: dread d6/dd/d2e/f61 [0,4194304] 0 2024-05-15T15:54:13.240 INFO:tasks.workunit.client.1.smithi144.stdout:3/535: link d0/d11/d3b/f45 d0/d11/d24/fb8 0 2024-05-15T15:54:13.240 INFO:tasks.workunit.client.1.smithi144.stdout:9/536: chown d0/d8/l6c 16489754 1 2024-05-15T15:54:13.241 INFO:tasks.workunit.client.0.smithi049.stdout:3/529: dread - d12/d26/f7d zero size 2024-05-15T15:54:13.241 INFO:tasks.workunit.client.0.smithi049.stdout:3/530: fdatasync d12/d26/d73/d95/f97 0 2024-05-15T15:54:13.241 INFO:tasks.workunit.client.0.smithi049.stdout:7/395: creat d0/d6/d1d/f7a x:0 0 0 2024-05-15T15:54:13.244 INFO:tasks.workunit.client.0.smithi049.stdout:4/499: dwrite d18/d1e/d2a/f4f [0,4194304] 0 2024-05-15T15:54:13.245 INFO:tasks.workunit.client.1.smithi144.stdout:9/537: dwrite d0/d8/d1b/f6f [0,4194304] 0 2024-05-15T15:54:13.247 INFO:tasks.workunit.client.0.smithi049.stdout:3/531: dread d12/f25 [0,4194304] 0 2024-05-15T15:54:13.249 INFO:tasks.workunit.client.0.smithi049.stdout:4/500: write d18/d1e/d2a/d3b/f9c [971966,40374] 0 2024-05-15T15:54:13.249 INFO:tasks.workunit.client.1.smithi144.stdout:4/514: creat d6/dd/fa2 x:0 0 0 2024-05-15T15:54:13.253 INFO:tasks.workunit.client.0.smithi049.stdout:4/501: chown d18/d1e/d2a/d50/d54/f6d 11794 1 2024-05-15T15:54:13.253 INFO:tasks.workunit.client.0.smithi049.stdout:5/465: dwrite dc/f3e [0,4194304] 0 2024-05-15T15:54:13.258 INFO:tasks.workunit.client.0.smithi049.stdout:9/351: mknod d0/d25/c78 0 2024-05-15T15:54:13.260 INFO:tasks.workunit.client.1.smithi144.stdout:9/538: mkdir d0/d8/d1b/d62/d7d/d8d/dbb 0 2024-05-15T15:54:13.261 INFO:tasks.workunit.client.0.smithi049.stdout:6/465: creat dd/d17/d24/d2b/d36/d50/d7e/f81 x:0 0 0 2024-05-15T15:54:13.262 INFO:tasks.workunit.client.1.smithi144.stdout:8/569: dwrite d2/de/f33 [0,4194304] 0 2024-05-15T15:54:13.263 INFO:tasks.workunit.client.0.smithi049.stdout:6/466: read dd/d26/f4f [64502,33538] 0 2024-05-15T15:54:13.263 INFO:tasks.workunit.client.1.smithi144.stdout:8/570: chown d2/d24/d2d/c32 19726 1 2024-05-15T15:54:13.264 INFO:tasks.workunit.client.1.smithi144.stdout:4/515: unlink d6/d42/d46/d4b/d1e/d44/f53 0 2024-05-15T15:54:13.265 INFO:tasks.workunit.client.0.smithi049.stdout:6/467: dread - dd/d17/d24/d2b/d36/d50/d54/f5c zero size 2024-05-15T15:54:13.265 INFO:tasks.workunit.client.1.smithi144.stdout:4/516: write d6/dd/fa2 [135197,83780] 0 2024-05-15T15:54:13.265 INFO:tasks.workunit.client.1.smithi144.stdout:3/536: rename d0/d1/d6c/cb3 to d0/d1/d17/d39/d40/d4a/d77/cb9 0 2024-05-15T15:54:13.266 INFO:tasks.workunit.client.0.smithi049.stdout:1/604: getdents d0/d4/dd/de/d1e/db7 0 2024-05-15T15:54:13.270 INFO:tasks.workunit.client.0.smithi049.stdout:7/396: unlink d0/d34/f69 0 2024-05-15T15:54:13.273 INFO:tasks.workunit.client.0.smithi049.stdout:1/605: dread d0/d4/dd/de/d4e/f76 [0,4194304] 0 2024-05-15T15:54:13.275 INFO:tasks.workunit.client.1.smithi144.stdout:9/539: dread d0/d8/d5e/d44/d99/fa1 [0,4194304] 0 2024-05-15T15:54:13.275 INFO:tasks.workunit.client.1.smithi144.stdout:3/537: symlink d0/d1/d17/d39/d40/d7e/lba 0 2024-05-15T15:54:13.277 INFO:tasks.workunit.client.1.smithi144.stdout:4/517: fsync d6/dd/d34/d3b/d30/db/d1d/d40/f97 0 2024-05-15T15:54:13.282 INFO:tasks.workunit.client.1.smithi144.stdout:9/540: chown d0/d8/d1b/d62/d7d/c8c 3 1 2024-05-15T15:54:13.284 INFO:tasks.workunit.client.1.smithi144.stdout:3/538: chown d0/d1/c2d 37175293 1 2024-05-15T15:54:13.284 INFO:tasks.workunit.client.1.smithi144.stdout:0/524: sync 2024-05-15T15:54:13.284 INFO:tasks.workunit.client.1.smithi144.stdout:7/530: sync 2024-05-15T15:54:13.285 INFO:tasks.workunit.client.0.smithi049.stdout:6/468: creat dd/d17/d24/d2b/f82 x:0 0 0 2024-05-15T15:54:13.286 INFO:tasks.workunit.client.0.smithi049.stdout:7/397: dread d0/f9 [4194304,4194304] 0 2024-05-15T15:54:13.288 INFO:tasks.workunit.client.1.smithi144.stdout:9/541: dread d0/d8/d1b/d79/d8a/f97 [0,4194304] 0 2024-05-15T15:54:13.290 INFO:tasks.workunit.client.1.smithi144.stdout:7/531: dwrite d4/df/f11 [0,4194304] 0 2024-05-15T15:54:13.290 INFO:tasks.workunit.client.1.smithi144.stdout:9/542: read d0/d8/d1b/d79/d8a/f97 [538360,110451] 0 2024-05-15T15:54:13.291 INFO:tasks.workunit.client.1.smithi144.stdout:9/543: write d0/d8/d5e/d44/d3c/f42 [4038827,17074] 0 2024-05-15T15:54:13.293 INFO:tasks.workunit.client.0.smithi049.stdout:6/469: dwrite dd/d17/d24/d2b/f72 [0,4194304] 0 2024-05-15T15:54:13.294 INFO:tasks.workunit.client.1.smithi144.stdout:8/571: rename d2/de/d3f/f40 to d2/fb6 0 2024-05-15T15:54:13.299 INFO:tasks.workunit.client.0.smithi049.stdout:3/532: unlink d12/d30/d63/c81 0 2024-05-15T15:54:13.303 INFO:tasks.workunit.client.0.smithi049.stdout:5/466: mknod dc/dd/d47/c97 0 2024-05-15T15:54:13.305 INFO:tasks.workunit.client.0.smithi049.stdout:9/352: creat d0/d8/d1b/d71/f79 x:0 0 0 2024-05-15T15:54:13.306 INFO:tasks.workunit.client.1.smithi144.stdout:4/518: creat d6/dd/d2e/d94/fa3 x:0 0 0 2024-05-15T15:54:13.307 INFO:tasks.workunit.client.0.smithi049.stdout:2/521: getdents d3/db/d1d/d1e/d2b/d2e 0 2024-05-15T15:54:13.307 INFO:tasks.workunit.client.0.smithi049.stdout:9/353: chown d0/d8/d1b/d40/f77 2505579 1 2024-05-15T15:54:13.308 INFO:tasks.workunit.client.0.smithi049.stdout:7/398: rmdir d0 39 2024-05-15T15:54:13.310 INFO:tasks.workunit.client.1.smithi144.stdout:9/544: rmdir d0/d8/d1b/d62/d7d 39 2024-05-15T15:54:13.311 INFO:tasks.workunit.client.1.smithi144.stdout:9/545: fdatasync d0/d8/d5e/d44/d10/f81 0 2024-05-15T15:54:13.312 INFO:tasks.workunit.client.1.smithi144.stdout:8/572: symlink d2/de/d3f/d6b/lb7 0 2024-05-15T15:54:13.313 INFO:tasks.workunit.client.1.smithi144.stdout:8/573: write d2/d3a/f7c [1541445,111379] 0 2024-05-15T15:54:13.317 INFO:tasks.workunit.client.0.smithi049.stdout:1/606: link d0/fd1 d0/d4/dd/de/d1e/db7/fd7 0 2024-05-15T15:54:13.318 INFO:tasks.workunit.client.1.smithi144.stdout:0/525: symlink d4/d2d/dae/lba 0 2024-05-15T15:54:13.319 INFO:tasks.workunit.client.0.smithi049.stdout:4/502: creat d18/d1e/fa7 x:0 0 0 2024-05-15T15:54:13.319 INFO:tasks.workunit.client.0.smithi049.stdout:1/607: readlink d0/d4/dd/de/d1e/d2e/dad/d65/d72/l86 0 2024-05-15T15:54:13.320 INFO:tasks.workunit.client.0.smithi049.stdout:4/503: chown d18/d1e/d2a/d3b/f74 24345825 1 2024-05-15T15:54:13.322 INFO:tasks.workunit.client.0.smithi049.stdout:2/522: mkdir d3/da/da9 0 2024-05-15T15:54:13.323 INFO:tasks.workunit.client.1.smithi144.stdout:7/532: rmdir d4/d41/d9c 39 2024-05-15T15:54:13.324 INFO:tasks.workunit.client.0.smithi049.stdout:7/399: read d0/d2f/d5c/f70 [425458,47170] 0 2024-05-15T15:54:13.325 INFO:tasks.workunit.client.0.smithi049.stdout:9/354: truncate d0/d8/d9/d52/d5e/d5f/f38 295762 0 2024-05-15T15:54:13.328 INFO:tasks.workunit.client.1.smithi144.stdout:4/519: dread f0 [0,4194304] 0 2024-05-15T15:54:13.329 INFO:tasks.workunit.client.1.smithi144.stdout:8/574: mkdir d2/d24/d2d/da0/d9e/db8 0 2024-05-15T15:54:13.331 INFO:tasks.workunit.client.0.smithi049.stdout:1/608: mkdir d0/d4/dd/de/d48/d84/dd8 0 2024-05-15T15:54:13.332 INFO:tasks.workunit.client.1.smithi144.stdout:3/539: link d0/l7 d0/d1/d17/d39/d83/lbb 0 2024-05-15T15:54:13.333 INFO:tasks.workunit.client.1.smithi144.stdout:4/520: dread d6/d42/d46/d4b/d1e/d44/f43 [0,4194304] 0 2024-05-15T15:54:13.333 INFO:tasks.workunit.client.1.smithi144.stdout:4/521: dread - d6/d42/d46/f90 zero size 2024-05-15T15:54:13.334 INFO:tasks.workunit.client.0.smithi049.stdout:5/467: rename dc/c69 to dc/dd/d39/c98 0 2024-05-15T15:54:13.334 INFO:tasks.workunit.client.0.smithi049.stdout:5/468: fsync dc/f3e 0 2024-05-15T15:54:13.335 INFO:tasks.workunit.client.0.smithi049.stdout:4/504: mkdir d18/d1e/d2a/d3b/da8 0 2024-05-15T15:54:13.337 INFO:tasks.workunit.client.1.smithi144.stdout:7/533: unlink d4/df/f1f 0 2024-05-15T15:54:13.337 INFO:tasks.workunit.client.1.smithi144.stdout:3/540: dwrite d0/d11/f26 [0,4194304] 0 2024-05-15T15:54:13.340 INFO:tasks.workunit.client.1.smithi144.stdout:9/546: link d0/d8/fb8 d0/d8/d5e/fbc 0 2024-05-15T15:54:13.340 INFO:tasks.workunit.client.1.smithi144.stdout:6/463: sync 2024-05-15T15:54:13.341 INFO:tasks.workunit.client.1.smithi144.stdout:6/464: write d7/db/d11/f42 [107530,31208] 0 2024-05-15T15:54:13.343 INFO:tasks.workunit.client.1.smithi144.stdout:0/526: mknod d4/cbb 0 2024-05-15T15:54:13.346 INFO:tasks.workunit.client.0.smithi049.stdout:9/355: symlink d0/d8/d1b/d71/l7a 0 2024-05-15T15:54:13.349 INFO:tasks.workunit.client.0.smithi049.stdout:5/469: dread dc/f11 [0,4194304] 0 2024-05-15T15:54:13.350 INFO:tasks.workunit.client.0.smithi049.stdout:3/533: dwrite d12/d30/f38 [0,4194304] 0 2024-05-15T15:54:13.352 INFO:tasks.workunit.client.0.smithi049.stdout:6/470: creat dd/d17/d24/d2b/d36/d50/f83 x:0 0 0 2024-05-15T15:54:13.357 INFO:tasks.workunit.client.1.smithi144.stdout:7/534: symlink d4/d3c/d76/d9d/lb2 0 2024-05-15T15:54:13.359 INFO:tasks.workunit.client.0.smithi049.stdout:4/505: chown d18/d1e/l7b 494373 1 2024-05-15T15:54:13.360 INFO:tasks.workunit.client.1.smithi144.stdout:3/541: creat d0/d11/d3b/da7/fbc x:0 0 0 2024-05-15T15:54:13.361 INFO:tasks.workunit.client.0.smithi049.stdout:6/471: dwrite dd/d26/f4b [0,4194304] 0 2024-05-15T15:54:13.362 INFO:tasks.workunit.client.1.smithi144.stdout:6/465: dread d7/db/d35/f47 [0,4194304] 0 2024-05-15T15:54:13.369 INFO:tasks.workunit.client.0.smithi049.stdout:5/470: symlink dc/d4a/d31/l99 0 2024-05-15T15:54:13.370 INFO:tasks.workunit.client.1.smithi144.stdout:8/575: mkdir d2/d24/d2d/da0/d6e/d71/d8c/db9 0 2024-05-15T15:54:13.372 INFO:tasks.workunit.client.1.smithi144.stdout:4/522: symlink d6/d42/d46/d4b/d1e/d99/la4 0 2024-05-15T15:54:13.374 INFO:tasks.workunit.client.1.smithi144.stdout:3/542: rmdir d0/d1/d56/d71 39 2024-05-15T15:54:13.375 INFO:tasks.workunit.client.0.smithi049.stdout:7/400: dwrite d0/d6/d23/f4c [0,4194304] 0 2024-05-15T15:54:13.382 INFO:tasks.workunit.client.0.smithi049.stdout:4/506: chown d18/d21/d2b/f2c 20572 1 2024-05-15T15:54:13.383 INFO:tasks.workunit.client.1.smithi144.stdout:8/576: rmdir d2/d24/d2d/da0/d6e/d71/d73/d98 39 2024-05-15T15:54:13.383 INFO:tasks.workunit.client.0.smithi049.stdout:6/472: truncate dd/f16 3985623 0 2024-05-15T15:54:13.386 INFO:tasks.workunit.client.0.smithi049.stdout:9/356: mknod d0/c7b 0 2024-05-15T15:54:13.386 INFO:tasks.workunit.client.0.smithi049.stdout:9/357: chown d0/d8/d9/c2c 253717 1 2024-05-15T15:54:13.387 INFO:tasks.workunit.client.1.smithi144.stdout:3/543: symlink d0/d11/d3b/da7/lbd 0 2024-05-15T15:54:13.388 INFO:tasks.workunit.client.0.smithi049.stdout:5/471: fsync dc/d4a/d42/f60 0 2024-05-15T15:54:13.390 INFO:tasks.workunit.client.0.smithi049.stdout:1/609: truncate d0/d4/dd/de/d1e/f37 5037560 0 2024-05-15T15:54:13.390 INFO:tasks.workunit.client.1.smithi144.stdout:6/466: write d7/db/d11/f15 [1454335,51689] 0 2024-05-15T15:54:13.391 INFO:tasks.workunit.client.0.smithi049.stdout:1/610: write d0/d4/d81/f97 [1265659,65339] 0 2024-05-15T15:54:13.392 INFO:tasks.workunit.client.1.smithi144.stdout:9/547: rename d0/d8/d5e/d44/d10/f17 to d0/d8/d5e/d44/d3c/fbd 0 2024-05-15T15:54:13.393 INFO:tasks.workunit.client.1.smithi144.stdout:9/548: readlink d0/l15 0 2024-05-15T15:54:13.394 INFO:tasks.workunit.client.1.smithi144.stdout:9/549: chown d0/d8/d1b/d79/c9d 40 1 2024-05-15T15:54:13.395 INFO:tasks.workunit.client.0.smithi049.stdout:4/507: symlink d18/d85/d98/la9 0 2024-05-15T15:54:13.397 INFO:tasks.workunit.client.0.smithi049.stdout:9/358: unlink d0/d8/d21/l2d 0 2024-05-15T15:54:13.401 INFO:tasks.workunit.client.1.smithi144.stdout:6/467: mknod d7/db/da2/caf 0 2024-05-15T15:54:13.402 INFO:tasks.workunit.client.1.smithi144.stdout:8/577: dread d2/de/d36/d58/f94 [0,4194304] 0 2024-05-15T15:54:13.402 INFO:tasks.workunit.client.0.smithi049.stdout:5/472: read dc/dd/d47/f5d [408440,73027] 0 2024-05-15T15:54:13.402 INFO:tasks.workunit.client.1.smithi144.stdout:6/468: fsync d7/db/d11/d3c/d61/f9b 0 2024-05-15T15:54:13.403 INFO:tasks.workunit.client.0.smithi049.stdout:5/473: dread - dc/d29/f87 zero size 2024-05-15T15:54:13.404 INFO:tasks.workunit.client.1.smithi144.stdout:0/527: rename d4/df/d1f/d56/f93 to d4/df/d1f/d24/fbc 0 2024-05-15T15:54:13.405 INFO:tasks.workunit.client.0.smithi049.stdout:3/534: creat d12/d30/d63/d68/fa0 x:0 0 0 2024-05-15T15:54:13.409 INFO:tasks.workunit.client.0.smithi049.stdout:1/611: read - d0/d4/dd/de/d4d/fcf zero size 2024-05-15T15:54:13.409 INFO:tasks.workunit.client.0.smithi049.stdout:7/401: dwrite d0/d6/f1b [0,4194304] 0 2024-05-15T15:54:13.411 INFO:tasks.workunit.client.0.smithi049.stdout:7/402: dread - d0/d6/d23/d4a/f75 zero size 2024-05-15T15:54:13.412 INFO:tasks.workunit.client.0.smithi049.stdout:9/359: mknod d0/d8/d9/d52/d5e/d5f/c7c 0 2024-05-15T15:54:13.415 INFO:tasks.workunit.client.1.smithi144.stdout:6/469: mkdir d7/db/d3a/db0 0 2024-05-15T15:54:13.416 INFO:tasks.workunit.client.1.smithi144.stdout:6/470: truncate d7/db/d11/d3c/d61/f9b 747202 0 2024-05-15T15:54:13.416 INFO:tasks.workunit.client.1.smithi144.stdout:8/578: dread d2/f3 [0,4194304] 0 2024-05-15T15:54:13.417 INFO:tasks.workunit.client.1.smithi144.stdout:9/550: rename d0/d8/d5e/d44/f95 to d0/d8/d1b/d79/d8a/dae/fbe 0 2024-05-15T15:54:13.419 INFO:tasks.workunit.client.0.smithi049.stdout:9/360: fsync d0/d8/d11/f1f 0 2024-05-15T15:54:13.421 INFO:tasks.workunit.client.0.smithi049.stdout:1/612: mknod d0/d4/dd/de/d48/d5c/da2/cd9 0 2024-05-15T15:54:13.422 INFO:tasks.workunit.client.0.smithi049.stdout:1/613: readlink d0/d4/dd/de/d4d/l55 0 2024-05-15T15:54:13.422 INFO:tasks.workunit.client.1.smithi144.stdout:9/551: mkdir d0/d8/d21/dbf 0 2024-05-15T15:54:13.423 INFO:tasks.workunit.client.0.smithi049.stdout:4/508: rename d18/f53 to d18/faa 0 2024-05-15T15:54:13.425 INFO:tasks.workunit.client.0.smithi049.stdout:9/361: mkdir d0/d8/d1b/d71/d7d 0 2024-05-15T15:54:13.425 INFO:tasks.workunit.client.1.smithi144.stdout:9/552: rmdir d0/d8/d5e/d44/d10 39 2024-05-15T15:54:13.426 INFO:tasks.workunit.client.0.smithi049.stdout:1/614: creat d0/d4/dd/de/d48/fda x:0 0 0 2024-05-15T15:54:13.429 INFO:tasks.workunit.client.1.smithi144.stdout:6/471: creat d7/db/d35/fb1 x:0 0 0 2024-05-15T15:54:13.431 INFO:tasks.workunit.client.0.smithi049.stdout:1/615: unlink d0/d4/d81/dab/fb5 0 2024-05-15T15:54:13.432 INFO:tasks.workunit.client.1.smithi144.stdout:0/528: getdents d4/d6/d3a/d4c 0 2024-05-15T15:54:13.435 INFO:tasks.workunit.client.1.smithi144.stdout:6/472: link d7/db/d11/f1f d7/db/da2/fb2 0 2024-05-15T15:54:13.435 INFO:tasks.workunit.client.0.smithi049.stdout:1/616: symlink d0/d4/dd/de/d1e/d16/d6e/dbf/ldb 0 2024-05-15T15:54:13.439 INFO:tasks.workunit.client.0.smithi049.stdout:1/617: symlink d0/d4/dd/de/d1e/d2e/dad/d65/d72/ldc 0 2024-05-15T15:54:13.439 INFO:tasks.workunit.client.0.smithi049.stdout:9/362: link d0/d8/d9/d22/f2e d0/d8/f7e 0 2024-05-15T15:54:13.441 INFO:tasks.workunit.client.1.smithi144.stdout:9/553: rename d0/d8/d5e/d44/d10/fd to d0/d8/d5e/d44/d10/d19/fc0 0 2024-05-15T15:54:13.442 INFO:tasks.workunit.client.1.smithi144.stdout:6/473: dwrite d7/f8 [0,4194304] 0 2024-05-15T15:54:13.443 INFO:tasks.workunit.client.1.smithi144.stdout:6/474: readlink d7/db/d11/d3c/d61/l94 0 2024-05-15T15:54:13.445 INFO:tasks.workunit.client.0.smithi049.stdout:9/363: read d0/d8/d9/fa [4250759,51532] 0 2024-05-15T15:54:13.449 INFO:tasks.workunit.client.0.smithi049.stdout:1/618: dread d0/d4/dd/de/d36/f53 [0,4194304] 0 2024-05-15T15:54:13.451 INFO:tasks.workunit.client.1.smithi144.stdout:9/554: dwrite d0/d8/d5e/d44/d10/f58 [0,4194304] 0 2024-05-15T15:54:13.451 INFO:tasks.workunit.client.0.smithi049.stdout:9/364: write d0/d8/d1b/f32 [1431703,28618] 0 2024-05-15T15:54:13.452 INFO:tasks.workunit.client.1.smithi144.stdout:0/529: dread d4/df/d1f/d24/f4b [0,4194304] 0 2024-05-15T15:54:13.454 INFO:tasks.workunit.client.1.smithi144.stdout:6/475: dwrite d7/db/d11/fa1 [0,4194304] 0 2024-05-15T15:54:13.459 INFO:tasks.workunit.client.1.smithi144.stdout:9/555: mknod d0/d8/d5e/d44/d77/cc1 0 2024-05-15T15:54:13.462 INFO:tasks.workunit.client.1.smithi144.stdout:0/530: symlink d4/d6/d3a/d4c/lbd 0 2024-05-15T15:54:13.462 INFO:tasks.workunit.client.0.smithi049.stdout:7/403: dread d0/d6/d1d/f27 [0,4194304] 0 2024-05-15T15:54:13.467 INFO:tasks.workunit.client.1.smithi144.stdout:0/531: symlink d4/df/d1f/d24/d2b/d55/lbe 0 2024-05-15T15:54:13.469 INFO:tasks.workunit.client.1.smithi144.stdout:4/523: write d6/dd/d34/d3b/d30/db/d1d/f41 [1575814,80413] 0 2024-05-15T15:54:13.470 INFO:tasks.workunit.client.1.smithi144.stdout:3/544: truncate d0/d1/d17/d39/d83/f9d 1192776 0 2024-05-15T15:54:13.472 INFO:tasks.workunit.client.0.smithi049.stdout:9/365: rename d0/d8/d9/f74 to d0/d8/d9/f7f 0 2024-05-15T15:54:13.473 INFO:tasks.workunit.client.0.smithi049.stdout:9/366: stat d0/d8/d1b/d40/f58 0 2024-05-15T15:54:13.475 INFO:tasks.workunit.client.0.smithi049.stdout:7/404: mkdir d0/d6/d23/d4a/d7b 0 2024-05-15T15:54:13.479 INFO:tasks.workunit.client.0.smithi049.stdout:3/535: write d12/d14/d32/d64/d88/f9f [727255,14289] 0 2024-05-15T15:54:13.481 INFO:tasks.workunit.client.1.smithi144.stdout:5/554: sync 2024-05-15T15:54:13.481 INFO:tasks.workunit.client.1.smithi144.stdout:1/523: sync 2024-05-15T15:54:13.481 INFO:tasks.workunit.client.1.smithi144.stdout:2/513: sync 2024-05-15T15:54:13.481 INFO:tasks.workunit.client.0.smithi049.stdout:5/474: dwrite dc/d4a/f57 [0,4194304] 0 2024-05-15T15:54:13.482 INFO:tasks.workunit.client.0.smithi049.stdout:5/475: chown dc/dd/d10 333900 1 2024-05-15T15:54:13.483 INFO:tasks.workunit.client.0.smithi049.stdout:7/405: rename d0/d6/d23/l68 to d0/d34/l7c 0 2024-05-15T15:54:13.483 INFO:tasks.workunit.client.0.smithi049.stdout:7/406: dread - d0/d6/d1d/f7a zero size 2024-05-15T15:54:13.485 INFO:tasks.workunit.client.0.smithi049.stdout:3/536: mknod d12/d30/d42/d6b/ca1 0 2024-05-15T15:54:13.487 INFO:tasks.workunit.client.0.smithi049.stdout:9/367: link d0/d8/d9/fb d0/d8/d53/f80 0 2024-05-15T15:54:13.489 INFO:tasks.workunit.client.0.smithi049.stdout:9/368: truncate d0/d8/d1b/d40/f49 4331741 0 2024-05-15T15:54:13.489 INFO:tasks.workunit.client.0.smithi049.stdout:7/407: creat d0/d2f/d5c/f7d x:0 0 0 2024-05-15T15:54:13.490 INFO:tasks.workunit.client.1.smithi144.stdout:0/532: symlink d4/d6/d4a/d74/lbf 0 2024-05-15T15:54:13.490 INFO:tasks.workunit.client.1.smithi144.stdout:9/556: rename d0/d8/d5e/d44/d3c to d0/d8/d5e/d44/dc2 0 2024-05-15T15:54:13.491 INFO:tasks.workunit.client.1.smithi144.stdout:8/579: dread d2/de/d49/f69 [0,4194304] 0 2024-05-15T15:54:13.491 INFO:tasks.workunit.client.0.smithi049.stdout:9/369: chown d0/d8/d53/f68 3 1 2024-05-15T15:54:13.491 INFO:tasks.workunit.client.0.smithi049.stdout:3/537: creat d12/d30/d42/d6b/fa2 x:0 0 0 2024-05-15T15:54:13.492 INFO:tasks.workunit.client.1.smithi144.stdout:5/555: dread - d0/dc/d24/d57/fa6 zero size 2024-05-15T15:54:13.493 INFO:tasks.workunit.client.1.smithi144.stdout:0/533: write d4/d6/d3a/d4c/fa6 [644081,65888] 0 2024-05-15T15:54:13.495 INFO:tasks.workunit.client.0.smithi049.stdout:7/408: mknod d0/d2f/d5c/c7e 0 2024-05-15T15:54:13.496 INFO:tasks.workunit.client.1.smithi144.stdout:4/524: dwrite d6/d49/f5c [0,4194304] 0 2024-05-15T15:54:13.496 INFO:tasks.workunit.client.0.smithi049.stdout:3/538: write d12/d30/d42/f48 [836506,8953] 0 2024-05-15T15:54:13.497 INFO:tasks.workunit.client.1.smithi144.stdout:8/580: dread d2/d3a/f3e [0,4194304] 0 2024-05-15T15:54:13.498 INFO:tasks.workunit.client.1.smithi144.stdout:1/524: stat d7/d1d/d33/d41/c9b 0 2024-05-15T15:54:13.498 INFO:tasks.workunit.client.0.smithi049.stdout:4/509: dwrite d18/d1e/d2a/d36/f4b [0,4194304] 0 2024-05-15T15:54:13.498 INFO:tasks.workunit.client.0.smithi049.stdout:5/476: creat dc/f9a x:0 0 0 2024-05-15T15:54:13.500 INFO:tasks.workunit.client.0.smithi049.stdout:3/539: chown d12/d30/d42/f48 7 1 2024-05-15T15:54:13.502 INFO:tasks.workunit.client.0.smithi049.stdout:3/540: write d12/d26/d73/f7f [1155725,15679] 0 2024-05-15T15:54:13.506 INFO:tasks.workunit.client.1.smithi144.stdout:8/581: dwrite d2/d3a/f3e [0,4194304] 0 2024-05-15T15:54:13.506 INFO:tasks.workunit.client.1.smithi144.stdout:1/525: dread d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:13.506 INFO:tasks.workunit.client.1.smithi144.stdout:1/526: write d7/d1d/d33/d1e/f77 [907875,71399] 0 2024-05-15T15:54:13.511 INFO:tasks.workunit.client.0.smithi049.stdout:9/370: write d0/d8/d1b/d40/f49 [2034107,99652] 0 2024-05-15T15:54:13.513 INFO:tasks.workunit.client.1.smithi144.stdout:3/545: dread d0/d1/f23 [0,4194304] 0 2024-05-15T15:54:13.514 INFO:tasks.workunit.client.0.smithi049.stdout:5/477: dwrite dc/d29/f92 [0,4194304] 0 2024-05-15T15:54:13.515 INFO:tasks.workunit.client.0.smithi049.stdout:7/409: mknod d0/d6/d23/d45/c7f 0 2024-05-15T15:54:13.516 INFO:tasks.workunit.client.0.smithi049.stdout:1/619: write d0/d4/dd/de/d36/f66 [331935,116546] 0 2024-05-15T15:54:13.517 INFO:tasks.workunit.client.1.smithi144.stdout:0/534: creat d4/df/d14/fc0 x:0 0 0 2024-05-15T15:54:13.519 INFO:tasks.workunit.client.0.smithi049.stdout:4/510: fsync f10 0 2024-05-15T15:54:13.521 INFO:tasks.workunit.client.1.smithi144.stdout:4/525: mknod d6/dd/d34/d3b/d30/d59/ca5 0 2024-05-15T15:54:13.526 INFO:tasks.workunit.client.0.smithi049.stdout:7/410: dread d0/d6/d1d/f59 [0,4194304] 0 2024-05-15T15:54:13.529 INFO:tasks.workunit.client.0.smithi049.stdout:4/511: dwrite d18/f4d [0,4194304] 0 2024-05-15T15:54:13.529 INFO:tasks.workunit.client.1.smithi144.stdout:9/557: mknod d0/d8/d5e/d44/d16/db0/cc3 0 2024-05-15T15:54:13.533 INFO:tasks.workunit.client.1.smithi144.stdout:8/582: creat d2/d24/d2d/da0/d6e/d71/fba x:0 0 0 2024-05-15T15:54:13.535 INFO:tasks.workunit.client.0.smithi049.stdout:5/478: chown dc/dd/f17 113608743 1 2024-05-15T15:54:13.540 INFO:tasks.workunit.client.1.smithi144.stdout:2/514: link d3/d7/d54/d98/f9c d3/d7/d20/d37/d76/faa 0 2024-05-15T15:54:13.541 INFO:tasks.workunit.client.0.smithi049.stdout:3/541: mkdir d12/d30/da3 0 2024-05-15T15:54:13.544 INFO:tasks.workunit.client.1.smithi144.stdout:0/535: rmdir d4/df/d1f/d24 39 2024-05-15T15:54:13.547 INFO:tasks.workunit.client.0.smithi049.stdout:7/411: rmdir d0/d2f 39 2024-05-15T15:54:13.549 INFO:tasks.workunit.client.0.smithi049.stdout:7/412: chown d0/d6/d23/d45/c7f 253642383 1 2024-05-15T15:54:13.549 INFO:tasks.workunit.client.0.smithi049.stdout:7/413: read d0/f47 [994934,87409] 0 2024-05-15T15:54:13.552 INFO:tasks.workunit.client.0.smithi049.stdout:4/512: creat d18/d85/fab x:0 0 0 2024-05-15T15:54:13.554 INFO:tasks.workunit.client.0.smithi049.stdout:9/371: symlink d0/d8/l81 0 2024-05-15T15:54:13.555 INFO:tasks.workunit.client.1.smithi144.stdout:9/558: symlink d0/d8/d1b/d62/d6b/lc4 0 2024-05-15T15:54:13.555 INFO:tasks.workunit.client.0.smithi049.stdout:3/542: dread d12/d30/d42/d7a/f83 [0,4194304] 0 2024-05-15T15:54:13.556 INFO:tasks.workunit.client.0.smithi049.stdout:3/543: stat d12/d26/d73/f7f 0 2024-05-15T15:54:13.557 INFO:tasks.workunit.client.0.smithi049.stdout:5/479: write dc/dd/f1d [3188933,27657] 0 2024-05-15T15:54:13.559 INFO:tasks.workunit.client.1.smithi144.stdout:0/536: dread d4/d6/d3a/d40/f7d [0,4194304] 0 2024-05-15T15:54:13.560 INFO:tasks.workunit.client.1.smithi144.stdout:8/583: rename d2/d3a/d7b/d4b/fb2 to d2/d24/d2d/da0/d6e/d8d/fbb 0 2024-05-15T15:54:13.561 INFO:tasks.workunit.client.1.smithi144.stdout:8/584: chown d2/de/d49 1734940 1 2024-05-15T15:54:13.563 INFO:tasks.workunit.client.0.smithi049.stdout:3/544: dwrite d12/d14/d52/f56 [0,4194304] 0 2024-05-15T15:54:13.567 INFO:tasks.workunit.client.1.smithi144.stdout:2/515: truncate d3/d7/d20/d3f/f56 705350 0 2024-05-15T15:54:13.570 INFO:tasks.workunit.client.1.smithi144.stdout:3/546: creat d0/d11/d24/da5/fbe x:0 0 0 2024-05-15T15:54:13.571 INFO:tasks.workunit.client.1.smithi144.stdout:3/547: chown d0/d1/d6c 0 1 2024-05-15T15:54:13.571 INFO:tasks.workunit.client.1.smithi144.stdout:3/548: chown d0 43 1 2024-05-15T15:54:13.573 INFO:tasks.workunit.client.0.smithi049.stdout:7/414: rmdir d0/d6 39 2024-05-15T15:54:13.575 INFO:tasks.workunit.client.1.smithi144.stdout:9/559: fdatasync d0/d8/d1b/d62/d7d/d8d/faf 0 2024-05-15T15:54:13.577 INFO:tasks.workunit.client.0.smithi049.stdout:9/372: creat d0/d8/d21/f82 x:0 0 0 2024-05-15T15:54:13.577 INFO:tasks.workunit.client.0.smithi049.stdout:9/373: fdatasync d0/d8/f61 0 2024-05-15T15:54:13.580 INFO:tasks.workunit.client.1.smithi144.stdout:0/537: fsync d4/d6/fb 0 2024-05-15T15:54:13.585 INFO:tasks.workunit.client.1.smithi144.stdout:7/535: sync 2024-05-15T15:54:13.585 INFO:tasks.workunit.client.1.smithi144.stdout:8/585: unlink d2/d24/d2d/f5e 0 2024-05-15T15:54:13.587 INFO:tasks.workunit.client.0.smithi049.stdout:3/545: unlink d12/c3e 0 2024-05-15T15:54:13.589 INFO:tasks.workunit.client.1.smithi144.stdout:1/527: link d7/d4b/cae d7/d1d/d33/d1e/d87/caf 0 2024-05-15T15:54:13.589 INFO:tasks.workunit.client.1.smithi144.stdout:2/516: dread d3/d7/d20/d22/d5e/f96 [0,4194304] 0 2024-05-15T15:54:13.591 INFO:tasks.workunit.client.0.smithi049.stdout:1/620: rmdir d0/d4/dd/de/d48/d84/dd8 0 2024-05-15T15:54:13.594 INFO:tasks.workunit.client.1.smithi144.stdout:5/556: getdents d0/dc/d24 0 2024-05-15T15:54:13.598 INFO:tasks.workunit.client.1.smithi144.stdout:3/549: stat d0/l7 0 2024-05-15T15:54:13.599 INFO:tasks.workunit.client.0.smithi049.stdout:7/415: write d0/d6/da/d3d/f33 [74559,113556] 0 2024-05-15T15:54:13.603 INFO:tasks.workunit.client.0.smithi049.stdout:5/480: mkdir dc/d4a/d9b 0 2024-05-15T15:54:13.605 INFO:tasks.workunit.client.1.smithi144.stdout:9/560: rmdir d0/d8/d5e/d44/d99 39 2024-05-15T15:54:13.606 INFO:tasks.workunit.client.1.smithi144.stdout:9/561: chown d0/d8/d5e/d44/dc2/f45 46 1 2024-05-15T15:54:13.608 INFO:tasks.workunit.client.1.smithi144.stdout:4/526: truncate d6/dd/d34/d3b/d30/f2f 2634079 0 2024-05-15T15:54:13.609 INFO:tasks.workunit.client.1.smithi144.stdout:0/538: mknod d4/d6/d3a/cc1 0 2024-05-15T15:54:13.610 INFO:tasks.workunit.client.1.smithi144.stdout:0/539: write d4/df/d1f/f2f [1521597,110508] 0 2024-05-15T15:54:13.610 INFO:tasks.workunit.client.1.smithi144.stdout:4/527: dread d6/d42/d46/d4b/d1e/d44/f43 [0,4194304] 0 2024-05-15T15:54:13.611 INFO:tasks.workunit.client.1.smithi144.stdout:7/536: unlink d4/df/d25/d38/d68/f8f 0 2024-05-15T15:54:13.612 INFO:tasks.workunit.client.0.smithi049.stdout:9/374: creat d0/d8/d1b/d71/d7d/f83 x:0 0 0 2024-05-15T15:54:13.612 INFO:tasks.workunit.client.1.smithi144.stdout:8/586: creat d2/d24/d2d/da0/d9e/fbc x:0 0 0 2024-05-15T15:54:13.614 INFO:tasks.workunit.client.0.smithi049.stdout:5/481: creat dc/dd/d47/f9c x:0 0 0 2024-05-15T15:54:13.615 INFO:tasks.workunit.client.1.smithi144.stdout:1/528: symlink d7/d1d/d58/d81/lb0 0 2024-05-15T15:54:13.615 INFO:tasks.workunit.client.1.smithi144.stdout:1/529: readlink l2 0 2024-05-15T15:54:13.615 INFO:tasks.workunit.client.0.smithi049.stdout:5/482: fsync dc/d4a/d31/f82 0 2024-05-15T15:54:13.616 INFO:tasks.workunit.client.1.smithi144.stdout:2/517: rmdir d3/d7/d20/d37/d76 39 2024-05-15T15:54:13.618 INFO:tasks.workunit.client.0.smithi049.stdout:1/621: symlink d0/d4/dd/de/d48/d5c/ldd 0 2024-05-15T15:54:13.626 INFO:tasks.workunit.client.0.smithi049.stdout:3/546: readlink d12/d30/d42/l9a 0 2024-05-15T15:54:13.626 INFO:tasks.workunit.client.0.smithi049.stdout:5/483: rename dc/dd/d47/c70 to dc/d4a/d42/c9d 0 2024-05-15T15:54:13.628 INFO:tasks.workunit.client.1.smithi144.stdout:0/540: unlink d4/d6/d3a/d6a/l83 0 2024-05-15T15:54:13.630 INFO:tasks.workunit.client.0.smithi049.stdout:1/622: dwrite d0/d4/dd/de/d1e/d2e/d3e/d90/db3/fc4 [0,4194304] 0 2024-05-15T15:54:13.631 INFO:tasks.workunit.client.0.smithi049.stdout:9/375: dread d0/d8/d21/f2f [0,4194304] 0 2024-05-15T15:54:13.677 INFO:tasks.workunit.client.0.smithi049.stdout:5/484: stat dc/dd/d39/l8e 0 2024-05-15T15:54:13.678 INFO:tasks.workunit.client.1.smithi144.stdout:8/587: symlink d2/d3a/lbd 0 2024-05-15T15:54:13.679 INFO:tasks.workunit.client.1.smithi144.stdout:8/588: write d2/de/d36/f48 [643222,128488] 0 2024-05-15T15:54:13.680 INFO:tasks.workunit.client.0.smithi049.stdout:3/547: symlink d12/d30/d42/d7a/la4 0 2024-05-15T15:54:13.680 INFO:tasks.workunit.client.1.smithi144.stdout:8/589: fsync d2/de/fa1 0 2024-05-15T15:54:13.681 INFO:tasks.workunit.client.1.smithi144.stdout:1/530: creat d7/d1d/d33/d41/d78/fb1 x:0 0 0 2024-05-15T15:54:13.681 INFO:tasks.workunit.client.1.smithi144.stdout:8/590: dread - d2/d24/d2d/da0/d6e/d8d/fbb zero size 2024-05-15T15:54:13.682 INFO:tasks.workunit.client.1.smithi144.stdout:1/531: fdatasync d7/d1d/fa5 0 2024-05-15T15:54:13.683 INFO:tasks.workunit.client.0.smithi049.stdout:7/416: write d0/d2f/d5c/f70 [1222349,25231] 0 2024-05-15T15:54:13.685 INFO:tasks.workunit.client.0.smithi049.stdout:7/417: read d0/d34/f3f [5316163,102885] 0 2024-05-15T15:54:13.686 INFO:tasks.workunit.client.1.smithi144.stdout:6/476: sync 2024-05-15T15:54:13.686 INFO:tasks.workunit.client.0.smithi049.stdout:1/623: rename d0/d4/dd/de/d1e/d16/f30 to d0/d4/d81/dab/fde 0 2024-05-15T15:54:13.686 INFO:tasks.workunit.client.1.smithi144.stdout:8/591: dwrite d2/d24/f89 [0,4194304] 0 2024-05-15T15:54:13.687 INFO:tasks.workunit.client.0.smithi049.stdout:1/624: dread - d0/d4/dd/de/d48/d5c/fc6 zero size 2024-05-15T15:54:13.687 INFO:tasks.workunit.client.0.smithi049.stdout:7/418: truncate d0/d6/d23/d45/f5a 1203996 0 2024-05-15T15:54:13.688 INFO:tasks.workunit.client.0.smithi049.stdout:7/419: read d0/d34/f35 [2878241,43268] 0 2024-05-15T15:54:13.692 INFO:tasks.workunit.client.1.smithi144.stdout:4/528: rename d6/d7e/c8a to d6/d42/d46/ca6 0 2024-05-15T15:54:13.693 INFO:tasks.workunit.client.1.smithi144.stdout:9/562: dwrite d0/d8/d5e/d44/dc2/f45 [0,4194304] 0 2024-05-15T15:54:13.694 INFO:tasks.workunit.client.0.smithi049.stdout:9/376: unlink d0/d8/d1b/d71/d7d/f83 0 2024-05-15T15:54:13.695 INFO:tasks.workunit.client.0.smithi049.stdout:9/377: stat d0/d8/d1b/d40/d5c/f70 0 2024-05-15T15:54:13.701 INFO:tasks.workunit.client.1.smithi144.stdout:7/537: mknod d4/df/d25/d40/cb3 0 2024-05-15T15:54:13.701 INFO:tasks.workunit.client.1.smithi144.stdout:4/529: write d6/dd/d34/d3b/d30/db/d1d/d4e/f8b [733064,48960] 0 2024-05-15T15:54:13.701 INFO:tasks.workunit.client.0.smithi049.stdout:5/485: mkdir dc/d4a/d9e 0 2024-05-15T15:54:13.702 INFO:tasks.workunit.client.1.smithi144.stdout:9/563: dread d0/f14 [0,4194304] 0 2024-05-15T15:54:13.704 INFO:tasks.workunit.client.1.smithi144.stdout:7/538: truncate d4/d3c/d76/d9d/dac/d48/d72/d82/f92 439985 0 2024-05-15T15:54:13.704 INFO:tasks.workunit.client.0.smithi049.stdout:3/548: dread d12/d30/f3d [0,4194304] 0 2024-05-15T15:54:13.705 INFO:tasks.workunit.client.0.smithi049.stdout:7/420: mkdir d0/d6/da/d3d/d80 0 2024-05-15T15:54:13.705 INFO:tasks.workunit.client.0.smithi049.stdout:7/421: dread - d0/f6b zero size 2024-05-15T15:54:13.710 INFO:tasks.workunit.client.1.smithi144.stdout:7/539: dwrite d4/f2f [0,4194304] 0 2024-05-15T15:54:13.714 INFO:tasks.workunit.client.1.smithi144.stdout:5/557: dwrite d0/dc/f1b [0,4194304] 0 2024-05-15T15:54:13.716 INFO:tasks.workunit.client.1.smithi144.stdout:6/477: mknod d7/d1e/d28/cb3 0 2024-05-15T15:54:13.717 INFO:tasks.workunit.client.1.smithi144.stdout:6/478: chown d7/db/d9d/c5d 0 1 2024-05-15T15:54:13.717 INFO:tasks.workunit.client.1.smithi144.stdout:7/540: dwrite d4/df/d25/d38/d68/f74 [0,4194304] 0 2024-05-15T15:54:13.719 INFO:tasks.workunit.client.1.smithi144.stdout:7/541: write d4/f2f [746255,44090] 0 2024-05-15T15:54:13.719 INFO:tasks.workunit.client.1.smithi144.stdout:7/542: readlink d4/d45/l6c 0 2024-05-15T15:54:13.724 INFO:tasks.workunit.client.0.smithi049.stdout:9/378: dread d0/d8/d53/f80 [0,4194304] 0 2024-05-15T15:54:13.732 INFO:tasks.workunit.client.0.smithi049.stdout:1/625: rename d0/d4/dd/de/d1e/d2e/dad/d65/f6f to d0/d4/dd/de/d1e/d2e/dad/d65/fdf 0 2024-05-15T15:54:13.733 INFO:tasks.workunit.client.1.smithi144.stdout:2/518: truncate d3/d7/d20/d22/d52/f69 67441 0 2024-05-15T15:54:13.734 INFO:tasks.workunit.client.1.smithi144.stdout:4/530: chown d6/f29 18 1 2024-05-15T15:54:13.734 INFO:tasks.workunit.client.1.smithi144.stdout:2/519: write d3/d7/f47 [8183807,47656] 0 2024-05-15T15:54:13.735 INFO:tasks.workunit.client.0.smithi049.stdout:5/486: write dc/f64 [739688,78423] 0 2024-05-15T15:54:13.736 INFO:tasks.workunit.client.0.smithi049.stdout:5/487: write dc/d29/f87 [69652,18233] 0 2024-05-15T15:54:13.738 INFO:tasks.workunit.client.1.smithi144.stdout:1/532: symlink d7/d1d/d33/d29/d6b/da4/lb2 0 2024-05-15T15:54:13.740 INFO:tasks.workunit.client.0.smithi049.stdout:7/422: symlink d0/d6/da/d3d/d80/l81 0 2024-05-15T15:54:13.741 INFO:tasks.workunit.client.1.smithi144.stdout:5/558: read d0/dc/d24/d57/f77 [228641,86078] 0 2024-05-15T15:54:13.742 INFO:tasks.workunit.client.0.smithi049.stdout:5/488: dwrite dc/dd/d10/f1e [0,4194304] 0 2024-05-15T15:54:13.742 INFO:tasks.workunit.client.0.smithi049.stdout:1/626: fsync d0/d4/dd/de/d48/d5c/fa3 0 2024-05-15T15:54:13.747 INFO:tasks.workunit.client.1.smithi144.stdout:7/543: dread d4/df/d79/f9b [0,4194304] 0 2024-05-15T15:54:13.749 INFO:tasks.workunit.client.0.smithi049.stdout:7/423: fsync d0/f11 0 2024-05-15T15:54:13.754 INFO:tasks.workunit.client.1.smithi144.stdout:4/531: rmdir d6/d42 39 2024-05-15T15:54:13.755 INFO:tasks.workunit.client.1.smithi144.stdout:3/550: sync 2024-05-15T15:54:13.756 INFO:tasks.workunit.client.1.smithi144.stdout:3/551: chown d0/d1/d56/f58 2402330 1 2024-05-15T15:54:13.756 INFO:tasks.workunit.client.1.smithi144.stdout:4/532: truncate d6/dd/d34/d3b/d30/d59/d66/f9b 107696 0 2024-05-15T15:54:13.758 INFO:tasks.workunit.client.1.smithi144.stdout:3/552: truncate d0/d1/d17/d39/d40/d4a/d77/fb0 643133 0 2024-05-15T15:54:13.758 INFO:tasks.workunit.client.0.smithi049.stdout:1/627: fdatasync d0/d4/f2d 0 2024-05-15T15:54:13.760 INFO:tasks.workunit.client.1.smithi144.stdout:2/520: mknod d3/d10/cab 0 2024-05-15T15:54:13.761 INFO:tasks.workunit.client.0.smithi049.stdout:3/549: link d12/d67/c94 d12/d67/ca5 0 2024-05-15T15:54:13.763 INFO:tasks.workunit.client.1.smithi144.stdout:1/533: rmdir d7/d1d/d33/d41/d78 39 2024-05-15T15:54:13.765 INFO:tasks.workunit.client.0.smithi049.stdout:9/379: truncate d0/d8/d1b/f32 1166664 0 2024-05-15T15:54:13.767 INFO:tasks.workunit.client.1.smithi144.stdout:6/479: dwrite d7/fd [0,4194304] 0 2024-05-15T15:54:13.767 INFO:tasks.workunit.client.1.smithi144.stdout:6/480: read - d7/f41 zero size 2024-05-15T15:54:13.771 INFO:tasks.workunit.client.0.smithi049.stdout:5/489: creat dc/d4a/d9b/f9f x:0 0 0 2024-05-15T15:54:13.772 INFO:tasks.workunit.client.1.smithi144.stdout:0/541: getdents d4/d6/d4a 0 2024-05-15T15:54:13.777 INFO:tasks.workunit.client.1.smithi144.stdout:9/564: creat d0/d8/d1b/d62/fc5 x:0 0 0 2024-05-15T15:54:13.777 INFO:tasks.workunit.client.1.smithi144.stdout:9/565: stat d0/d8/d5e/d44/d10/f27 0 2024-05-15T15:54:13.781 INFO:tasks.workunit.client.1.smithi144.stdout:3/553: rmdir d0/d1/d17 39 2024-05-15T15:54:13.782 INFO:tasks.workunit.client.1.smithi144.stdout:3/554: write d0/d1/f23 [4462476,107324] 0 2024-05-15T15:54:13.783 INFO:tasks.workunit.client.1.smithi144.stdout:8/592: sync 2024-05-15T15:54:13.784 INFO:tasks.workunit.client.1.smithi144.stdout:1/534: stat d7/d1d/d33/d29/c69 0 2024-05-15T15:54:13.785 INFO:tasks.workunit.client.1.smithi144.stdout:8/593: write d2/d3a/d7b/d4b/f61 [1012266,111036] 0 2024-05-15T15:54:13.785 INFO:tasks.workunit.client.0.smithi049.stdout:9/380: dwrite d0/d8/d21/f2f [0,4194304] 0 2024-05-15T15:54:13.785 INFO:tasks.workunit.client.1.smithi144.stdout:1/535: chown d7/d1d/d33/d29/d6b/d5e/d74/fab 7264649 1 2024-05-15T15:54:13.788 INFO:tasks.workunit.client.0.smithi049.stdout:7/424: dwrite d0/f47 [4194304,4194304] 0 2024-05-15T15:54:13.788 INFO:tasks.workunit.client.0.smithi049.stdout:9/381: write d0/d8/d9/d45/f4a [418198,113701] 0 2024-05-15T15:54:13.791 INFO:tasks.workunit.client.0.smithi049.stdout:1/628: rename d0/d4/dd/f38 to d0/d4/dd/de/d48/d5c/fe0 0 2024-05-15T15:54:13.797 INFO:tasks.workunit.client.1.smithi144.stdout:6/481: mkdir d7/db/d35/d84/d81/db4 0 2024-05-15T15:54:13.800 INFO:tasks.workunit.client.1.smithi144.stdout:6/482: chown d7/db/d35/d84/d81/f83 379484615 1 2024-05-15T15:54:13.801 INFO:tasks.workunit.client.0.smithi049.stdout:0/488: sync 2024-05-15T15:54:13.802 INFO:tasks.workunit.client.1.smithi144.stdout:9/566: fsync d0/d8/d50/f72 0 2024-05-15T15:54:13.804 INFO:tasks.workunit.client.1.smithi144.stdout:9/567: stat d0/d8/d5e/d44/d77/cc1 0 2024-05-15T15:54:13.804 INFO:tasks.workunit.client.1.smithi144.stdout:3/555: dread d0/d11/d24/f3e [0,4194304] 0 2024-05-15T15:54:13.805 INFO:tasks.workunit.client.1.smithi144.stdout:2/521: mknod d3/d7/d20/d22/d52/d64/d97/cac 0 2024-05-15T15:54:13.806 INFO:tasks.workunit.client.1.smithi144.stdout:2/522: fsync d3/d7/d20/d3b/f77 0 2024-05-15T15:54:13.806 INFO:tasks.workunit.client.1.smithi144.stdout:3/556: truncate d0/d11/d24/f57 4375881 0 2024-05-15T15:54:13.810 INFO:tasks.workunit.client.0.smithi049.stdout:7/425: dwrite d0/d6/da/d3d/f2c [4194304,4194304] 0 2024-05-15T15:54:13.811 INFO:tasks.workunit.client.1.smithi144.stdout:5/559: rmdir d0/dc/d24/d28/d2d/d59/d74/db6 0 2024-05-15T15:54:13.812 INFO:tasks.workunit.client.0.smithi049.stdout:0/489: link dc/d13/d14/d33/l80 dc/d32/d93/l94 0 2024-05-15T15:54:13.814 INFO:tasks.workunit.client.1.smithi144.stdout:7/544: link d4/d3c/d76/d9d/dac/l33 d4/df/d79/lb4 0 2024-05-15T15:54:13.818 INFO:tasks.workunit.client.1.smithi144.stdout:5/560: dwrite d0/f22 [4194304,4194304] 0 2024-05-15T15:54:13.819 INFO:tasks.workunit.client.0.smithi049.stdout:1/629: dwrite d0/d4/dd/de/d36/f40 [0,4194304] 0 2024-05-15T15:54:13.821 INFO:tasks.workunit.client.0.smithi049.stdout:0/490: chown dc/d32/d93/l94 412533 1 2024-05-15T15:54:13.822 INFO:tasks.workunit.client.1.smithi144.stdout:9/568: fdatasync d0/d8/d5e/d44/d10/d19/f67 0 2024-05-15T15:54:13.827 INFO:tasks.workunit.client.1.smithi144.stdout:3/557: symlink d0/d11/d24/da5/lbf 0 2024-05-15T15:54:13.827 INFO:tasks.workunit.client.1.smithi144.stdout:3/558: dread - d0/d11/d3b/da7/fa9 zero size 2024-05-15T15:54:13.829 INFO:tasks.workunit.client.0.smithi049.stdout:0/491: dread dc/d13/d56/f90 [0,4194304] 0 2024-05-15T15:54:13.830 INFO:tasks.workunit.client.0.smithi049.stdout:0/492: chown dc/d32/l92 1857850 1 2024-05-15T15:54:13.833 INFO:tasks.workunit.client.0.smithi049.stdout:1/630: link d0/d4/f39 d0/d4/d81/dab/fe1 0 2024-05-15T15:54:13.834 INFO:tasks.workunit.client.1.smithi144.stdout:2/523: truncate d3/d7/d20/d22/d52/d64/f5f 2285261 0 2024-05-15T15:54:13.835 INFO:tasks.workunit.client.1.smithi144.stdout:1/536: truncate d7/d1d/d33/d41/fe 1710959 0 2024-05-15T15:54:13.835 INFO:tasks.workunit.client.0.smithi049.stdout:7/426: dwrite d0/f11 [4194304,4194304] 0 2024-05-15T15:54:13.837 INFO:tasks.workunit.client.0.smithi049.stdout:7/427: write d0/d6/d1d/f7a [989601,85873] 0 2024-05-15T15:54:13.838 INFO:tasks.workunit.client.1.smithi144.stdout:0/542: link d4/d2d/f8d d4/fc2 0 2024-05-15T15:54:13.841 INFO:tasks.workunit.client.1.smithi144.stdout:7/545: symlink d4/df/d25/lb5 0 2024-05-15T15:54:13.842 INFO:tasks.workunit.client.1.smithi144.stdout:7/546: fdatasync d4/f12 0 2024-05-15T15:54:13.844 INFO:tasks.workunit.client.1.smithi144.stdout:4/533: getdents d6/dd/d34/d3b/d30/d59 0 2024-05-15T15:54:13.845 INFO:tasks.workunit.client.0.smithi049.stdout:0/493: creat dc/d13/d14/f95 x:0 0 0 2024-05-15T15:54:13.845 INFO:tasks.workunit.client.0.smithi049.stdout:1/631: dread d0/d4/dd/de/d36/f40 [0,4194304] 0 2024-05-15T15:54:13.846 INFO:tasks.workunit.client.0.smithi049.stdout:7/428: mkdir d0/d6/da/d3d/d82 0 2024-05-15T15:54:13.846 INFO:tasks.workunit.client.0.smithi049.stdout:0/494: dread - dc/d32/f7b zero size 2024-05-15T15:54:13.848 INFO:tasks.workunit.client.1.smithi144.stdout:7/547: dwrite d4/df/f32 [0,4194304] 0 2024-05-15T15:54:13.849 INFO:tasks.workunit.client.0.smithi049.stdout:1/632: chown d0/d4/dd/de/d4d/d8b/lce 851352 1 2024-05-15T15:54:13.849 INFO:tasks.workunit.client.1.smithi144.stdout:5/561: rename d0/dc/f71 to d0/dc/d2f/d79/fb9 0 2024-05-15T15:54:13.850 INFO:tasks.workunit.client.1.smithi144.stdout:5/562: chown d0/dc/d1f/d50/db3 29334 1 2024-05-15T15:54:13.859 INFO:tasks.workunit.client.1.smithi144.stdout:8/594: creat d2/fbe x:0 0 0 2024-05-15T15:54:13.860 INFO:tasks.workunit.client.1.smithi144.stdout:5/563: dwrite d0/dc/f1b [0,4194304] 0 2024-05-15T15:54:13.860 INFO:tasks.workunit.client.1.smithi144.stdout:9/569: dwrite d0/d8/d5e/d44/dc2/fbd [0,4194304] 0 2024-05-15T15:54:13.862 INFO:tasks.workunit.client.1.smithi144.stdout:5/564: write d0/dc/d24/d28/f82 [797144,71132] 0 2024-05-15T15:54:13.866 INFO:tasks.workunit.client.1.smithi144.stdout:5/565: read d0/dc/f1b [4490950,36596] 0 2024-05-15T15:54:13.870 INFO:tasks.workunit.client.1.smithi144.stdout:9/570: dread d0/d8/d1b/d62/f8e [0,4194304] 0 2024-05-15T15:54:13.873 INFO:tasks.workunit.client.1.smithi144.stdout:2/524: dread d3/d7/d20/d37/f46 [0,4194304] 0 2024-05-15T15:54:13.878 INFO:tasks.workunit.client.1.smithi144.stdout:1/537: rename d7/d1d/d33/d41/d22/f3a to d7/d1d/d33/d41/d22/d45/fb3 0 2024-05-15T15:54:13.878 INFO:tasks.workunit.client.1.smithi144.stdout:9/571: dwrite d0/d8/d5e/d44/dc2/f45 [0,4194304] 0 2024-05-15T15:54:13.881 INFO:tasks.workunit.client.1.smithi144.stdout:3/559: rmdir d0/d1/d17/d39/d40/d4a/d9e 39 2024-05-15T15:54:13.883 INFO:tasks.workunit.client.1.smithi144.stdout:8/595: creat d2/de/d36/d87/fbf x:0 0 0 2024-05-15T15:54:13.888 INFO:tasks.workunit.client.1.smithi144.stdout:8/596: dwrite d2/d3a/d7b/d4b/f61 [0,4194304] 0 2024-05-15T15:54:13.889 INFO:tasks.workunit.client.1.smithi144.stdout:5/566: symlink d0/dc/d24/d28/lba 0 2024-05-15T15:54:13.891 INFO:tasks.workunit.client.1.smithi144.stdout:6/483: sync 2024-05-15T15:54:13.892 INFO:tasks.workunit.client.1.smithi144.stdout:7/548: write d4/df/d25/d40/d73/f98 [152783,12233] 0 2024-05-15T15:54:13.895 INFO:tasks.workunit.client.1.smithi144.stdout:2/525: symlink d3/d7/d20/d22/d52/d64/lad 0 2024-05-15T15:54:13.900 INFO:tasks.workunit.client.1.smithi144.stdout:6/484: dread d7/db/d35/f67 [0,4194304] 0 2024-05-15T15:54:13.904 INFO:tasks.workunit.client.1.smithi144.stdout:0/543: rename d4/df/f43 to d4/d6/d4a/d74/fc3 0 2024-05-15T15:54:13.904 INFO:tasks.workunit.client.1.smithi144.stdout:4/534: rename d6/dd/d34/d3b/d30 to d6/dd/d34/d3b/d30/db/d1d/d4e/da7 22 2024-05-15T15:54:13.904 INFO:tasks.workunit.client.1.smithi144.stdout:0/544: chown d4/df/d14/lb4 243 1 2024-05-15T15:54:13.905 INFO:tasks.workunit.client.1.smithi144.stdout:1/538: mkdir d7/d1d/d33/d41/d22/db4 0 2024-05-15T15:54:13.906 INFO:tasks.workunit.client.1.smithi144.stdout:4/535: write d6/dd/d34/d3b/d30/db/d1d/d40/f79 [160987,8197] 0 2024-05-15T15:54:13.907 INFO:tasks.workunit.client.1.smithi144.stdout:3/560: creat d0/d1/d17/d39/d40/fc0 x:0 0 0 2024-05-15T15:54:13.908 INFO:tasks.workunit.client.1.smithi144.stdout:4/536: dread - d6/dd/d2e/d94/fa3 zero size 2024-05-15T15:54:13.914 INFO:tasks.workunit.client.0.smithi049.stdout:7/429: mkdir d0/d6/d23/d45/d83 0 2024-05-15T15:54:13.915 INFO:tasks.workunit.client.0.smithi049.stdout:0/495: rename dc/d13/d14/f46 to dc/d32/f96 0 2024-05-15T15:54:13.916 INFO:tasks.workunit.client.0.smithi049.stdout:0/496: write dc/d32/f7b [595966,68265] 0 2024-05-15T15:54:13.918 INFO:tasks.workunit.client.1.smithi144.stdout:8/597: creat d2/d24/d2d/da0/d6e/d71/d8c/fc0 x:0 0 0 2024-05-15T15:54:13.919 INFO:tasks.workunit.client.0.smithi049.stdout:7/430: dread - d0/d6/da/d3d/f5f zero size 2024-05-15T15:54:13.919 INFO:tasks.workunit.client.1.smithi144.stdout:6/485: dread d7/db/f29 [0,4194304] 0 2024-05-15T15:54:13.920 INFO:tasks.workunit.client.0.smithi049.stdout:7/431: chown d0/d2f/f3e 15163152 1 2024-05-15T15:54:13.921 INFO:tasks.workunit.client.0.smithi049.stdout:1/633: symlink d0/d4/le2 0 2024-05-15T15:54:13.922 INFO:tasks.workunit.client.1.smithi144.stdout:2/526: creat d3/d4d/fae x:0 0 0 2024-05-15T15:54:13.922 INFO:tasks.workunit.client.0.smithi049.stdout:0/497: mknod dc/d13/d14/d25/c97 0 2024-05-15T15:54:13.924 INFO:tasks.workunit.client.0.smithi049.stdout:7/432: read d0/d6/d23/f30 [2217161,48762] 0 2024-05-15T15:54:13.926 INFO:tasks.workunit.client.1.smithi144.stdout:1/539: dread d7/d1d/d33/d41/f83 [0,4194304] 0 2024-05-15T15:54:13.927 INFO:tasks.workunit.client.1.smithi144.stdout:0/545: rename d4/d6/c33 to d4/d6/d3a/d6a/cc4 0 2024-05-15T15:54:13.928 INFO:tasks.workunit.client.1.smithi144.stdout:2/527: dwrite d3/d7/d54/d98/fa2 [0,4194304] 0 2024-05-15T15:54:13.929 INFO:tasks.workunit.client.1.smithi144.stdout:0/546: write d4/df/d1f/f2c [432718,100425] 0 2024-05-15T15:54:13.931 INFO:tasks.workunit.client.0.smithi049.stdout:0/498: fsync dc/d13/d14/d33/f53 0 2024-05-15T15:54:13.932 INFO:tasks.workunit.client.1.smithi144.stdout:3/561: creat d0/d1/d6c/fc1 x:0 0 0 2024-05-15T15:54:13.933 INFO:tasks.workunit.client.0.smithi049.stdout:0/499: write dc/d13/d48/d57/f65 [362888,92851] 0 2024-05-15T15:54:13.942 INFO:tasks.workunit.client.1.smithi144.stdout:7/549: chown d4/d41/d9c 80778655 1 2024-05-15T15:54:13.943 INFO:tasks.workunit.client.0.smithi049.stdout:0/500: truncate dc/f15 708260 0 2024-05-15T15:54:13.945 INFO:tasks.workunit.client.1.smithi144.stdout:9/572: creat d0/fc6 x:0 0 0 2024-05-15T15:54:13.949 INFO:tasks.workunit.client.0.smithi049.stdout:7/433: getdents d0/d6/d23/d45/d6a 0 2024-05-15T15:54:13.951 INFO:tasks.workunit.client.1.smithi144.stdout:6/486: rename d7/db/d11/d3c/d61/d6f to d7/d1e/d28/db5 0 2024-05-15T15:54:13.951 INFO:tasks.workunit.client.0.smithi049.stdout:0/501: read dc/d13/d14/d25/d31/d71/f67 [3383895,126385] 0 2024-05-15T15:54:13.952 INFO:tasks.workunit.client.0.smithi049.stdout:0/502: chown dc/d32/d93/f89 3437254 1 2024-05-15T15:54:13.954 INFO:tasks.workunit.client.1.smithi144.stdout:2/528: symlink d3/d7/d20/d22/d52/d75/laf 0 2024-05-15T15:54:13.955 INFO:tasks.workunit.client.0.smithi049.stdout:0/503: creat dc/d13/d2b/f98 x:0 0 0 2024-05-15T15:54:13.957 INFO:tasks.workunit.client.0.smithi049.stdout:0/504: write dc/d13/d14/d33/f5b [472924,56916] 0 2024-05-15T15:54:13.957 INFO:tasks.workunit.client.1.smithi144.stdout:4/537: creat d6/d42/d46/d4b/d1e/d44/fa8 x:0 0 0 2024-05-15T15:54:13.957 INFO:tasks.workunit.client.0.smithi049.stdout:1/634: truncate d0/d4/dd/de/d1e/d2e/d3e/d90/fcb 2466018 0 2024-05-15T15:54:13.958 INFO:tasks.workunit.client.1.smithi144.stdout:5/567: write d0/dc/d1f/d50/f58 [319851,20014] 0 2024-05-15T15:54:13.960 INFO:tasks.workunit.client.1.smithi144.stdout:3/562: unlink d0/d11/d3b/da7/fbc 0 2024-05-15T15:54:13.962 INFO:tasks.workunit.client.0.smithi049.stdout:7/434: rename d0/l5d to d0/l84 0 2024-05-15T15:54:13.964 INFO:tasks.workunit.client.1.smithi144.stdout:8/598: creat d2/d24/d2d/da0/d9e/db8/fc1 x:0 0 0 2024-05-15T15:54:13.965 INFO:tasks.workunit.client.1.smithi144.stdout:8/599: chown d2/de/d3f/d6b/lb7 3508 1 2024-05-15T15:54:13.967 INFO:tasks.workunit.client.0.smithi049.stdout:1/635: rename d0/d4/l14 to d0/d4/dd/de/d48/le3 0 2024-05-15T15:54:13.969 INFO:tasks.workunit.client.1.smithi144.stdout:9/573: creat d0/d8/d1b/d79/fc7 x:0 0 0 2024-05-15T15:54:13.976 INFO:tasks.workunit.client.0.smithi049.stdout:1/636: dread d0/d4/dd/de/d1e/d2e/dad/d65/d72/f7a [0,4194304] 0 2024-05-15T15:54:13.976 INFO:tasks.workunit.client.1.smithi144.stdout:0/547: rename d4/dc/c37 to d4/df/d14/cc5 0 2024-05-15T15:54:13.977 INFO:tasks.workunit.client.0.smithi049.stdout:7/435: mknod d0/d6/da/c85 0 2024-05-15T15:54:13.977 INFO:tasks.workunit.client.1.smithi144.stdout:0/548: write d4/d6/d3a/d5d/f8f [788661,19920] 0 2024-05-15T15:54:13.978 INFO:tasks.workunit.client.1.smithi144.stdout:6/487: mkdir d7/db/da2/d70/db6 0 2024-05-15T15:54:13.980 INFO:tasks.workunit.client.1.smithi144.stdout:2/529: fsync d3/d7/d20/d3f/f53 0 2024-05-15T15:54:13.983 INFO:tasks.workunit.client.0.smithi049.stdout:1/637: creat d0/d4/dd/de/d1e/d2e/d3e/fe4 x:0 0 0 2024-05-15T15:54:13.985 INFO:tasks.workunit.client.1.smithi144.stdout:5/568: creat d0/dc/d24/d28/d2d/d59/fbb x:0 0 0 2024-05-15T15:54:13.986 INFO:tasks.workunit.client.0.smithi049.stdout:0/505: getdents dc/d13/d2b/d76/d8a 0 2024-05-15T15:54:13.986 INFO:tasks.workunit.client.1.smithi144.stdout:6/488: dread f6 [0,4194304] 0 2024-05-15T15:54:13.987 INFO:tasks.workunit.client.0.smithi049.stdout:0/506: write dc/d32/d93/f88 [783284,124627] 0 2024-05-15T15:54:13.987 INFO:tasks.workunit.client.1.smithi144.stdout:5/569: write d0/dc/d2f/d68/d5c/f98 [343272,119979] 0 2024-05-15T15:54:13.988 INFO:tasks.workunit.client.1.smithi144.stdout:8/600: mknod d2/de/d36/d58/cc2 0 2024-05-15T15:54:13.989 INFO:tasks.workunit.client.0.smithi049.stdout:1/638: mkdir d0/d4/dd/de/d1e/d2e/d3e/d90/db3/de5 0 2024-05-15T15:54:13.991 INFO:tasks.workunit.client.0.smithi049.stdout:1/639: chown d0/d4/c59 647580020 1 2024-05-15T15:54:13.993 INFO:tasks.workunit.client.1.smithi144.stdout:7/550: rename d4/l2b to d4/df/d25/d38/d68/lb6 0 2024-05-15T15:54:13.993 INFO:tasks.workunit.client.0.smithi049.stdout:0/507: dwrite dc/d32/d93/f88 [0,4194304] 0 2024-05-15T15:54:13.994 INFO:tasks.workunit.client.1.smithi144.stdout:7/551: chown d4/d3c/d76/d9d/dac/d48/d72/d82/fad 2254 1 2024-05-15T15:54:13.994 INFO:tasks.workunit.client.0.smithi049.stdout:1/640: fsync d0/d4/dd/de/d4d/fd5 0 2024-05-15T15:54:14.000 INFO:tasks.workunit.client.0.smithi049.stdout:0/508: mknod dc/d13/d14/d2f/d8f/c99 0 2024-05-15T15:54:14.002 INFO:tasks.workunit.client.1.smithi144.stdout:6/489: truncate d7/ff 340915 0 2024-05-15T15:54:14.002 INFO:tasks.workunit.client.1.smithi144.stdout:5/570: dread - d0/dc/d2f/d34/d3e/f48 zero size 2024-05-15T15:54:14.004 INFO:tasks.workunit.client.0.smithi049.stdout:0/509: fdatasync dc/d32/f39 0 2024-05-15T15:54:14.007 INFO:tasks.workunit.client.1.smithi144.stdout:9/574: link d0/d8/d5e/d44/dc2/f61 d0/d8/d5e/d44/d10/d9c/fc8 0 2024-05-15T15:54:14.008 INFO:tasks.workunit.client.1.smithi144.stdout:9/575: write d0/f18 [2927238,24275] 0 2024-05-15T15:54:14.008 INFO:tasks.workunit.client.1.smithi144.stdout:1/540: link d7/d1d/d33/d41/f19 d7/d1d/d33/d29/d6b/da4/fb5 0 2024-05-15T15:54:14.013 INFO:tasks.workunit.client.1.smithi144.stdout:4/538: creat d6/fa9 x:0 0 0 2024-05-15T15:54:14.016 INFO:tasks.workunit.client.1.smithi144.stdout:8/601: symlink d2/d24/d2d/da0/d6e/d71/db1/lc3 0 2024-05-15T15:54:14.017 INFO:tasks.workunit.client.1.smithi144.stdout:2/530: write d3/f1c [2802249,58906] 0 2024-05-15T15:54:14.019 INFO:tasks.workunit.client.1.smithi144.stdout:2/531: write d3/d7/d20/d3f/f53 [925905,111457] 0 2024-05-15T15:54:14.019 INFO:tasks.workunit.client.1.smithi144.stdout:3/563: truncate d0/f46 198066 0 2024-05-15T15:54:14.021 INFO:tasks.workunit.client.1.smithi144.stdout:4/539: mknod d6/dd/d34/caa 0 2024-05-15T15:54:14.023 INFO:tasks.workunit.client.0.smithi049.stdout:1/641: dwrite d0/d4/dd/de/d4e/f76 [0,4194304] 0 2024-05-15T15:54:14.024 INFO:tasks.workunit.client.1.smithi144.stdout:5/571: rename d0/dc/d24/d28/d2d/d59/fbb to d0/dc/d1f/d50/fbc 0 2024-05-15T15:54:14.026 INFO:tasks.workunit.client.1.smithi144.stdout:5/572: write d0/dc/d1f/d50/f86 [2615461,41526] 0 2024-05-15T15:54:14.026 INFO:tasks.workunit.client.1.smithi144.stdout:5/573: chown d0/dc/d2f/d34/d3e/c72 26 1 2024-05-15T15:54:14.027 INFO:tasks.workunit.client.1.smithi144.stdout:7/552: creat d4/df/fb7 x:0 0 0 2024-05-15T15:54:14.029 INFO:tasks.workunit.client.1.smithi144.stdout:0/549: truncate d4/d2d/f2e 2251546 0 2024-05-15T15:54:14.029 INFO:tasks.workunit.client.1.smithi144.stdout:7/553: read - d4/d3c/d76/d9d/dac/d48/fa7 zero size 2024-05-15T15:54:14.030 INFO:tasks.workunit.client.1.smithi144.stdout:0/550: readlink d4/d6/d3a/d4c/l4f 0 2024-05-15T15:54:14.030 INFO:tasks.workunit.client.0.smithi049.stdout:1/642: fdatasync d0/d4/dd/de/d1e/f4f 0 2024-05-15T15:54:14.035 INFO:tasks.workunit.client.0.smithi049.stdout:1/643: mknod d0/d4/dd/de/d48/d5c/da2/ce6 0 2024-05-15T15:54:14.035 INFO:tasks.workunit.client.1.smithi144.stdout:4/540: dread d6/dd/d34/d3b/d30/f52 [0,4194304] 0 2024-05-15T15:54:14.036 INFO:tasks.workunit.client.1.smithi144.stdout:0/551: dread d4/d6/d4a/d74/f79 [0,4194304] 0 2024-05-15T15:54:14.037 INFO:tasks.workunit.client.1.smithi144.stdout:6/490: dwrite d7/db/da2/f66 [0,4194304] 0 2024-05-15T15:54:14.038 INFO:tasks.workunit.client.1.smithi144.stdout:1/541: rename d7/d4b/f4f to d7/d1d/d33/d1e/d4e/fb6 0 2024-05-15T15:54:14.039 INFO:tasks.workunit.client.1.smithi144.stdout:6/491: chown d7/d1e/d28/fab 2680 1 2024-05-15T15:54:14.050 INFO:tasks.workunit.client.1.smithi144.stdout:9/576: creat d0/d8/d5e/d44/d10/d19/fc9 x:0 0 0 2024-05-15T15:54:14.051 INFO:tasks.workunit.client.1.smithi144.stdout:5/574: symlink d0/dc/d24/d8e/d93/d6b/d88/lbd 0 2024-05-15T15:54:14.051 INFO:tasks.workunit.client.1.smithi144.stdout:3/564: creat d0/d11/d24/d78/fc2 x:0 0 0 2024-05-15T15:54:14.052 INFO:tasks.workunit.client.1.smithi144.stdout:6/492: chown d7/d1e/d28/f8a 0 1 2024-05-15T15:54:14.053 INFO:tasks.workunit.client.1.smithi144.stdout:0/552: creat d4/d6/d3a/d4c/fc6 x:0 0 0 2024-05-15T15:54:14.053 INFO:tasks.workunit.client.1.smithi144.stdout:9/577: dread - d0/d8/d1b/d62/fa3 zero size 2024-05-15T15:54:14.053 INFO:tasks.workunit.client.1.smithi144.stdout:3/565: chown d0/d11/d3b/db5 34964 1 2024-05-15T15:54:14.053 INFO:tasks.workunit.client.1.smithi144.stdout:6/493: write d7/f8 [4548343,55971] 0 2024-05-15T15:54:14.055 INFO:tasks.workunit.client.1.smithi144.stdout:9/578: write d0/d8/d1b/fb9 [739986,5889] 0 2024-05-15T15:54:14.056 INFO:tasks.workunit.client.1.smithi144.stdout:0/553: write d4/d2d/d48/d5b/f75 [785653,85732] 0 2024-05-15T15:54:14.057 INFO:tasks.workunit.client.1.smithi144.stdout:1/542: fdatasync d7/d1d/d33/d29/f2c 0 2024-05-15T15:54:14.058 INFO:tasks.workunit.client.1.smithi144.stdout:8/602: getdents d2/d3a/d7b 0 2024-05-15T15:54:14.059 INFO:tasks.workunit.client.1.smithi144.stdout:1/543: write d7/d1d/d33/f71 [3736980,35105] 0 2024-05-15T15:54:14.060 INFO:tasks.workunit.client.1.smithi144.stdout:5/575: unlink d0/dc/d1f/fa3 0 2024-05-15T15:54:14.060 INFO:tasks.workunit.client.0.smithi049.stdout:1/644: dread d0/d4/dd/de/d1e/f27 [0,4194304] 0 2024-05-15T15:54:14.061 INFO:tasks.workunit.client.1.smithi144.stdout:5/576: read - d0/dc/d12/d31/f4c zero size 2024-05-15T15:54:14.061 INFO:tasks.workunit.client.1.smithi144.stdout:5/577: stat d0/dc/d1f/f39 0 2024-05-15T15:54:14.063 INFO:tasks.workunit.client.0.smithi049.stdout:1/645: write d0/d4/f39 [1611345,102645] 0 2024-05-15T15:54:14.063 INFO:tasks.workunit.client.1.smithi144.stdout:6/494: mkdir d7/db/d9d/db7 0 2024-05-15T15:54:14.065 INFO:tasks.workunit.client.0.smithi049.stdout:1/646: read - d0/d4/dd/de/d1e/d2e/d3e/d90/db3/fd4 zero size 2024-05-15T15:54:14.065 INFO:tasks.workunit.client.1.smithi144.stdout:3/566: creat d0/d11/d3b/da0/d8f/fc3 x:0 0 0 2024-05-15T15:54:14.066 INFO:tasks.workunit.client.1.smithi144.stdout:6/495: truncate d7/db/d9d/f88 554022 0 2024-05-15T15:54:14.067 INFO:tasks.workunit.client.1.smithi144.stdout:0/554: mkdir d4/d6/d4a/d4d/dc7 0 2024-05-15T15:54:14.067 INFO:tasks.workunit.client.1.smithi144.stdout:6/496: readlink d7/db/l1a 0 2024-05-15T15:54:14.067 INFO:tasks.workunit.client.1.smithi144.stdout:5/578: dwrite d0/dc/d1f/d50/f58 [0,4194304] 0 2024-05-15T15:54:14.069 INFO:tasks.workunit.client.0.smithi049.stdout:1/647: symlink d0/d4/dd/de/d1e/d16/le7 0 2024-05-15T15:54:14.071 INFO:tasks.workunit.client.0.smithi049.stdout:1/648: dread d0/d4/dd/de/d1e/d2e/dad/d65/d72/f7a [0,4194304] 0 2024-05-15T15:54:14.072 INFO:tasks.workunit.client.0.smithi049.stdout:1/649: chown d0/d4/dd/f5a 13907 1 2024-05-15T15:54:14.073 INFO:tasks.workunit.client.1.smithi144.stdout:8/603: creat d2/d24/d2d/da0/d6e/fc4 x:0 0 0 2024-05-15T15:54:14.074 INFO:tasks.workunit.client.1.smithi144.stdout:3/567: dwrite d0/d11/d3b/f64 [0,4194304] 0 2024-05-15T15:54:14.074 INFO:tasks.workunit.client.0.smithi049.stdout:1/650: truncate d0/d4/dd/de/d36/f66 1144397 0 2024-05-15T15:54:14.076 INFO:tasks.workunit.client.1.smithi144.stdout:1/544: creat d7/d1d/d33/d41/d22/d45/fb7 x:0 0 0 2024-05-15T15:54:14.076 INFO:tasks.workunit.client.1.smithi144.stdout:1/545: chown d7/d1d/d33/d29/f64 540 1 2024-05-15T15:54:14.077 INFO:tasks.workunit.client.1.smithi144.stdout:1/546: write d7/d1d/d33/d55/f5b [5026912,84830] 0 2024-05-15T15:54:14.078 INFO:tasks.workunit.client.1.smithi144.stdout:2/532: rename d3/d7/d54/f85 to d3/d7/d20/fb0 0 2024-05-15T15:54:14.079 INFO:tasks.workunit.client.0.smithi049.stdout:1/651: mkdir d0/d4/dd/de/d4d/d8b/de8 0 2024-05-15T15:54:14.126 INFO:tasks.workunit.client.1.smithi144.stdout:7/554: sync 2024-05-15T15:54:14.127 INFO:tasks.workunit.client.1.smithi144.stdout:7/555: dread - d4/d3c/d76/d9d/dac/d48/fa7 zero size 2024-05-15T15:54:14.192 INFO:tasks.workunit.client.0.smithi049.stdout:1/652: rename d0/d4/dd/de/d4d/fd5 to d0/d4/dd/de/d4d/d8b/dba/fe9 0 2024-05-15T15:54:14.194 INFO:tasks.workunit.client.0.smithi049.stdout:1/653: write d0/d4/f39 [2991085,57531] 0 2024-05-15T15:54:14.194 INFO:tasks.workunit.client.1.smithi144.stdout:6/497: creat d7/db/d5f/fb8 x:0 0 0 2024-05-15T15:54:14.195 INFO:tasks.workunit.client.1.smithi144.stdout:0/555: symlink d4/d2d/d48/d5b/d8e/lc8 0 2024-05-15T15:54:14.195 INFO:tasks.workunit.client.1.smithi144.stdout:0/556: fsync d4/f22 0 2024-05-15T15:54:14.196 INFO:tasks.workunit.client.1.smithi144.stdout:5/579: creat d0/dc/d1f/d50/fbe x:0 0 0 2024-05-15T15:54:14.197 INFO:tasks.workunit.client.0.smithi049.stdout:1/654: symlink d0/d4/dd/de/d36/lea 0 2024-05-15T15:54:14.197 INFO:tasks.workunit.client.0.smithi049.stdout:1/655: chown d0/d4/dd/de/d36 0 1 2024-05-15T15:54:14.202 INFO:tasks.workunit.client.1.smithi144.stdout:4/541: rename d6/d7e to d6/d42/d46/d4b/d1e/d99/dab 0 2024-05-15T15:54:14.203 INFO:tasks.workunit.client.0.smithi049.stdout:1/656: symlink d0/d4/dd/de/d1e/d2e/dad/d65/leb 0 2024-05-15T15:54:14.204 INFO:tasks.workunit.client.1.smithi144.stdout:2/533: mkdir d3/d7/d20/d89/d86/db1 0 2024-05-15T15:54:14.210 INFO:tasks.workunit.client.0.smithi049.stdout:1/657: dwrite d0/d4/dd/de/d4e/fa7 [0,4194304] 0 2024-05-15T15:54:14.212 INFO:tasks.workunit.client.0.smithi049.stdout:1/658: dread - d0/d4/dd/de/d1e/db7/fd7 zero size 2024-05-15T15:54:14.218 INFO:tasks.workunit.client.1.smithi144.stdout:9/579: truncate d0/d8/d1b/d79/d8a/dae/fbe 1510686 0 2024-05-15T15:54:14.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.216+0000 7f1509345700 1 -- 172.21.15.49:0/2271022124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 msgr2=0x7f150410b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:14.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.216+0000 7f1509345700 1 --2- 172.21.15.49:0/2271022124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 0x7f150410b0c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009910 tx=0x7f14f4009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:14.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.216+0000 7f1509345700 1 -- 172.21.15.49:0/2271022124 shutdown_connections 2024-05-15T15:54:14.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.216+0000 7f1509345700 1 --2- 172.21.15.49:0/2271022124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f150410b600 0x7f150410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:14.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.216+0000 7f1509345700 1 --2- 172.21.15.49:0/2271022124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 0x7f150410b0c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:14.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.216+0000 7f1509345700 1 -- 172.21.15.49:0/2271022124 >> 172.21.15.49:0/2271022124 conn(0x7f150406df00 msgr2=0x7f1504070320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:14.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.217+0000 7f1509345700 1 -- 172.21.15.49:0/2271022124 shutdown_connections 2024-05-15T15:54:14.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.217+0000 7f1509345700 1 -- 172.21.15.49:0/2271022124 wait complete. 2024-05-15T15:54:14.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.217+0000 7f1509345700 1 Processor -- start 2024-05-15T15:54:14.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.218+0000 7f1509345700 1 -- start start 2024-05-15T15:54:14.221 INFO:tasks.workunit.client.0.smithi049.stdout:1/659: mknod d0/d4/dd/de/d1e/db7/cec 0 2024-05-15T15:54:14.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.218+0000 7f1509345700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 0x7f150419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:14.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.218+0000 7f1509345700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f150410b600 0x7f150419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:14.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.218+0000 7f1509345700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419ea80 con 0x7f1504072ca0 2024-05-15T15:54:14.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.218+0000 7f1509345700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419ebc0 con 0x7f150410b600 2024-05-15T15:54:14.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.218+0000 7f1502ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 0x7f150419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:14.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.218+0000 7f1502ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 0x7f150419df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54748/0 (socket says 172.21.15.49:54748) 2024-05-15T15:54:14.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.218+0000 7f1502ffd700 1 -- 172.21.15.49:0/345778123 learned_addr learned my addr 172.21.15.49:0/345778123 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.219+0000 7f15027fc700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f150410b600 0x7f150419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.219+0000 7f1502ffd700 1 -- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f150410b600 msgr2=0x7f150419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.219+0000 7f1502ffd700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f150410b600 0x7f150419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.219+0000 7f1502ffd700 1 -- 172.21.15.49:0/345778123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f40095f0 con 0x7f1504072ca0 2024-05-15T15:54:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.219+0000 7f1502ffd700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 0x7f150419df20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005090 tx=0x7f14f400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.219+0000 7f14ebfff700 1 -- 172.21.15.49:0/345778123 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14f4004ab0 con 0x7f1504072ca0 2024-05-15T15:54:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.219+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15041080e0 con 0x7f1504072ca0 2024-05-15T15:54:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.219+0000 7f14ebfff700 1 -- 172.21.15.49:0/345778123 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14f4004c10 con 0x7f1504072ca0 2024-05-15T15:54:14.224 INFO:tasks.workunit.client.1.smithi144.stdout:9/580: dread d0/d8/d1b/f6f [0,4194304] 0 2024-05-15T15:54:14.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.220+0000 7f14ebfff700 1 -- 172.21.15.49:0/345778123 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14f401d620 con 0x7f1504072ca0 2024-05-15T15:54:14.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.220+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1504108650 con 0x7f1504072ca0 2024-05-15T15:54:14.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.221+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150404ec60 con 0x7f1504072ca0 2024-05-15T15:54:14.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.221+0000 7f14ebfff700 1 -- 172.21.15.49:0/345778123 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f14f401d7c0 con 0x7f1504072ca0 2024-05-15T15:54:14.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.222+0000 7f14ebfff700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f14ec06d6f0 0x7f14ec06fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:14.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.222+0000 7f14ebfff700 1 -- 172.21.15.49:0/345778123 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f14f40928d0 con 0x7f1504072ca0 2024-05-15T15:54:14.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.222+0000 7f15027fc700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f14ec06d6f0 0x7f14ec06fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:14.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.223+0000 7f15027fc700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f14ec06d6f0 0x7f14ec06fbb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14f8006fd0 tx=0x7f14f8009190 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:14.227 INFO:tasks.workunit.client.0.smithi049.stdout:1/660: unlink d0/d4/dd/de/d1e/d2e/d3e/d90/db3/fd4 0 2024-05-15T15:54:14.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.228+0000 7f14ebfff700 1 -- 172.21.15.49:0/345778123 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f14f405fd10 con 0x7f1504072ca0 2024-05-15T15:54:14.231 INFO:tasks.workunit.client.0.smithi049.stdout:1/661: mknod d0/d4/dd/de/d1e/d16/d6e/dbf/ced 0 2024-05-15T15:54:14.234 INFO:tasks.workunit.client.1.smithi144.stdout:6/498: symlink d7/db/d35/d84/lb9 0 2024-05-15T15:54:14.236 INFO:tasks.workunit.client.1.smithi144.stdout:0/557: symlink d4/d6/d4a/lc9 0 2024-05-15T15:54:14.238 INFO:tasks.workunit.client.0.smithi049.stdout:1/662: unlink d0/d4/dd/de/d1e/d2e/d3e/d90/fcb 0 2024-05-15T15:54:14.245 INFO:tasks.workunit.client.1.smithi144.stdout:4/542: mknod d6/dd/d2e/cac 0 2024-05-15T15:54:14.246 INFO:tasks.workunit.client.1.smithi144.stdout:2/534: chown d3/d7/d20/d22/c95 9206 1 2024-05-15T15:54:14.250 INFO:tasks.workunit.client.1.smithi144.stdout:4/543: dwrite d6/f95 [0,4194304] 0 2024-05-15T15:54:14.255 INFO:tasks.workunit.client.1.smithi144.stdout:8/604: rename d2/d24/d2d/da0/d6e/d71/l80 to d2/d3a/d7b/d4b/lc5 0 2024-05-15T15:54:14.260 INFO:tasks.workunit.client.0.smithi049.stdout:8/516: sync 2024-05-15T15:54:14.260 INFO:tasks.workunit.client.1.smithi144.stdout:7/556: link d4/df/f11 d4/df/d25/d40/d62/fb8 0 2024-05-15T15:54:14.262 INFO:tasks.workunit.client.1.smithi144.stdout:7/557: read d4/df/d25/d40/f4d [2028976,80363] 0 2024-05-15T15:54:14.262 INFO:tasks.workunit.client.1.smithi144.stdout:4/544: mkdir d6/d42/d46/d4b/d1e/d99/dad 0 2024-05-15T15:54:14.264 INFO:tasks.workunit.client.1.smithi144.stdout:0/558: write d4/df/d1f/d24/fbc [311829,89906] 0 2024-05-15T15:54:14.265 INFO:tasks.workunit.client.1.smithi144.stdout:1/547: rename d7/df/ca9 to d7/d1d/d33/d29/d6b/d8e/cb8 0 2024-05-15T15:54:14.266 INFO:tasks.workunit.client.1.smithi144.stdout:3/568: truncate d0/d11/d24/f49 517327 0 2024-05-15T15:54:14.266 INFO:tasks.workunit.client.0.smithi049.stdout:8/517: mkdir d2/de/d27/d5b/da6 0 2024-05-15T15:54:14.268 INFO:tasks.workunit.client.1.smithi144.stdout:8/605: mknod d2/d24/d2d/da0/d6e/d71/d8c/cc6 0 2024-05-15T15:54:14.269 INFO:tasks.workunit.client.1.smithi144.stdout:1/548: stat d7/d1d/d33/d1e/d34 0 2024-05-15T15:54:14.269 INFO:tasks.workunit.client.0.smithi049.stdout:1/663: write d0/d4/dd/de/d48/f4a [264453,104573] 0 2024-05-15T15:54:14.270 INFO:tasks.workunit.client.1.smithi144.stdout:5/580: dwrite d0/dc/d24/d8e/f8a [0,4194304] 0 2024-05-15T15:54:14.270 INFO:tasks.workunit.client.1.smithi144.stdout:8/606: dread - d2/de/d36/d87/fbf zero size 2024-05-15T15:54:14.274 INFO:tasks.workunit.client.1.smithi144.stdout:4/545: mkdir d6/dd/dae 0 2024-05-15T15:54:14.276 INFO:tasks.workunit.client.1.smithi144.stdout:7/558: dread - d4/df/d25/f85 zero size 2024-05-15T15:54:14.280 INFO:tasks.workunit.client.0.smithi049.stdout:8/518: link d2/de/d27/d5b/d60/l77 d2/de/d95/la7 0 2024-05-15T15:54:14.281 INFO:tasks.workunit.client.0.smithi049.stdout:8/519: stat d2/de/d71/d86/d5d/d7d/d1b/f2b 0 2024-05-15T15:54:14.289 INFO:tasks.workunit.client.0.smithi049.stdout:8/520: dwrite d2/de/d71/d86/f8d [4194304,4194304] 0 2024-05-15T15:54:14.291 INFO:tasks.workunit.client.0.smithi049.stdout:8/521: fdatasync d2/de/d71/d86/d5d/d7d/d1b/d68/f8e 0 2024-05-15T15:54:14.292 INFO:tasks.workunit.client.1.smithi144.stdout:2/535: creat d3/d7/d20/d22/fb2 x:0 0 0 2024-05-15T15:54:14.292 INFO:tasks.workunit.client.1.smithi144.stdout:2/536: stat d3/d7/d54/d98 0 2024-05-15T15:54:14.293 INFO:tasks.workunit.client.1.smithi144.stdout:3/569: creat d0/d1/d17/d39/d40/d4a/d77/fc4 x:0 0 0 2024-05-15T15:54:14.295 INFO:tasks.workunit.client.1.smithi144.stdout:1/549: write d7/d1d/d33/d1e/d4e/fb6 [431324,23313] 0 2024-05-15T15:54:14.296 INFO:tasks.workunit.client.0.smithi049.stdout:1/664: truncate d0/d4/dd/de/d1e/d2e/dad/d65/d72/f8d 219875 0 2024-05-15T15:54:14.299 INFO:tasks.workunit.client.0.smithi049.stdout:1/665: truncate d0/d4/dd/de/d4d/d8b/dba/fe9 947025 0 2024-05-15T15:54:14.300 INFO:tasks.workunit.client.0.smithi049.stdout:8/522: mknod d2/de/d71/ca8 0 2024-05-15T15:54:14.302 INFO:tasks.workunit.client.1.smithi144.stdout:5/581: chown d0/dc/d24/d28/cb7 7697 1 2024-05-15T15:54:14.303 INFO:tasks.workunit.client.1.smithi144.stdout:8/607: fsync d2/d24/d2d/d30/f44 0 2024-05-15T15:54:14.306 INFO:tasks.workunit.client.0.smithi049.stdout:1/666: link d0/d4/dd/de/d48/d5c/f94 d0/d4/dd/de/d36/fee 0 2024-05-15T15:54:14.307 INFO:tasks.workunit.client.1.smithi144.stdout:2/537: write d3/d7/d20/d37/f46 [4715776,28514] 0 2024-05-15T15:54:14.307 INFO:tasks.workunit.client.0.smithi049.stdout:1/667: truncate d0/d4/dd/de/d1e/f27 5126770 0 2024-05-15T15:54:14.310 INFO:tasks.workunit.client.1.smithi144.stdout:1/550: rename d7/d1d/d33/d1e/d4e/l7e to d7/d1d/d33/d1e/d4e/lb9 0 2024-05-15T15:54:14.311 INFO:tasks.workunit.client.1.smithi144.stdout:1/551: chown d7/d1d/d33/d29/d6b/d5e/f9d 7818 1 2024-05-15T15:54:14.314 INFO:tasks.workunit.client.0.smithi049.stdout:1/668: fdatasync d0/d4/f2d 0 2024-05-15T15:54:14.314 INFO:tasks.workunit.client.1.smithi144.stdout:1/552: dwrite d7/d1d/d33/d29/d6b/d5e/f95 [0,4194304] 0 2024-05-15T15:54:14.316 INFO:tasks.workunit.client.1.smithi144.stdout:9/581: rename d0/d8/d5e/d44/dc2/fbd to d0/d8/d1b/d62/fca 0 2024-05-15T15:54:14.318 INFO:tasks.workunit.client.1.smithi144.stdout:8/608: creat d2/d24/d2d/da0/d9e/db8/fc7 x:0 0 0 2024-05-15T15:54:14.322 INFO:tasks.workunit.client.0.smithi049.stdout:1/669: dwrite d0/d4/d81/dab/fe1 [0,4194304] 0 2024-05-15T15:54:14.323 INFO:tasks.workunit.client.1.smithi144.stdout:3/570: creat d0/d1/d17/da1/fc5 x:0 0 0 2024-05-15T15:54:14.325 INFO:tasks.workunit.client.1.smithi144.stdout:8/609: dwrite d2/d24/d2d/da0/d6e/d8d/fb4 [0,4194304] 0 2024-05-15T15:54:14.326 INFO:tasks.workunit.client.1.smithi144.stdout:6/499: sync 2024-05-15T15:54:14.327 INFO:tasks.workunit.client.0.smithi049.stdout:1/670: creat d0/d4/dd/de/d36/fef x:0 0 0 2024-05-15T15:54:14.328 INFO:tasks.workunit.client.1.smithi144.stdout:8/610: write d2/d24/d2d/da0/d9e/db8/fc1 [868411,94363] 0 2024-05-15T15:54:14.328 INFO:tasks.workunit.client.0.smithi049.stdout:1/671: stat d0/d4/dd/f96 0 2024-05-15T15:54:14.328 INFO:tasks.workunit.client.1.smithi144.stdout:4/546: truncate d6/dd/d2d/f54 237790 0 2024-05-15T15:54:14.329 INFO:tasks.workunit.client.1.smithi144.stdout:4/547: fsync d6/dd/d2e/f70 0 2024-05-15T15:54:14.330 INFO:tasks.workunit.client.0.smithi049.stdout:1/672: fsync d0/d4/dd/de/d36/f66 0 2024-05-15T15:54:14.332 INFO:tasks.workunit.client.1.smithi144.stdout:4/548: chown d6/d42/d46/d4b/d1e/d44/f43 1 1 2024-05-15T15:54:14.334 INFO:tasks.workunit.client.1.smithi144.stdout:7/559: rename d4/df/d25/d38/d5d/f84 to d4/d3c/d76/d9d/dac/d48/fb9 0 2024-05-15T15:54:14.335 INFO:tasks.workunit.client.1.smithi144.stdout:0/559: getdents d4/d2d 0 2024-05-15T15:54:14.335 INFO:tasks.workunit.client.1.smithi144.stdout:1/553: write d7/d1d/d33/d41/d78/fb1 [822819,100964] 0 2024-05-15T15:54:14.336 INFO:tasks.workunit.client.0.smithi049.stdout:1/673: symlink d0/d4/dd/de/d1e/d2e/d3e/d90/db3/lf0 0 2024-05-15T15:54:14.338 INFO:tasks.workunit.client.1.smithi144.stdout:0/560: dread d4/df/d1f/d24/fbc [0,4194304] 0 2024-05-15T15:54:14.344 INFO:tasks.workunit.client.1.smithi144.stdout:6/500: mkdir d7/db/d9d/d97/dba 0 2024-05-15T15:54:14.344 INFO:tasks.workunit.client.1.smithi144.stdout:6/501: fdatasync d7/d1e/d28/f8a 0 2024-05-15T15:54:14.345 INFO:tasks.workunit.client.1.smithi144.stdout:6/502: chown d7/da6/faa 1574 1 2024-05-15T15:54:14.345 INFO:tasks.workunit.client.1.smithi144.stdout:8/611: creat d2/d1a/fc8 x:0 0 0 2024-05-15T15:54:14.346 INFO:tasks.workunit.client.1.smithi144.stdout:6/503: write d7/db/d11/f42 [1977330,16365] 0 2024-05-15T15:54:14.347 INFO:tasks.workunit.client.1.smithi144.stdout:4/549: mknod d6/dd/d2d/caf 0 2024-05-15T15:54:14.350 INFO:tasks.workunit.client.1.smithi144.stdout:9/582: symlink d0/d8/d5e/d44/dc2/lcb 0 2024-05-15T15:54:14.355 INFO:tasks.workunit.client.1.smithi144.stdout:5/582: rename d0/dc/d12/d31 to d0/dc/d2f/d34/d63/d6e/dbf 0 2024-05-15T15:54:14.356 INFO:tasks.workunit.client.1.smithi144.stdout:5/583: chown d0/dc/d12/d4a/f60 2642024 1 2024-05-15T15:54:14.357 INFO:tasks.workunit.client.1.smithi144.stdout:7/560: truncate d4/d41/d5b/f91 77881 0 2024-05-15T15:54:14.357 INFO:tasks.workunit.client.1.smithi144.stdout:5/584: write d0/dc/d24/d28/f82 [1499790,53334] 0 2024-05-15T15:54:14.358 INFO:tasks.workunit.client.1.smithi144.stdout:7/561: chown d4/df/f34 378633352 1 2024-05-15T15:54:14.361 INFO:tasks.workunit.client.1.smithi144.stdout:1/554: unlink d7/d1d/d33/d41/l18 0 2024-05-15T15:54:14.361 INFO:tasks.workunit.client.1.smithi144.stdout:1/555: stat d7/d1d/c2e 0 2024-05-15T15:54:14.367 INFO:tasks.workunit.client.1.smithi144.stdout:2/538: truncate d3/d7/d20/d3f/f53 1068986 0 2024-05-15T15:54:14.372 INFO:tasks.workunit.client.1.smithi144.stdout:8/612: rename d2/de/d36/d58/f94 to d2/d24/d2d/da0/d6e/d71/d8c/fc9 0 2024-05-15T15:54:14.372 INFO:tasks.workunit.client.1.smithi144.stdout:4/550: rename d6/d42/d46/d4b/d1e to d6/d42/d46/d4b/d1e/d99/dad/db0 22 2024-05-15T15:54:14.374 INFO:tasks.workunit.client.0.smithi049.stdout:1/674: dwrite d0/fd1 [0,4194304] 0 2024-05-15T15:54:14.374 INFO:tasks.workunit.client.1.smithi144.stdout:7/562: truncate d4/d41/f5a 387852 0 2024-05-15T15:54:14.376 INFO:tasks.workunit.client.1.smithi144.stdout:5/585: creat d0/dc/d12/d4a/d5d/fc0 x:0 0 0 2024-05-15T15:54:14.378 INFO:tasks.workunit.client.1.smithi144.stdout:2/539: dread d3/d7/d54/fa0 [0,4194304] 0 2024-05-15T15:54:14.385 INFO:tasks.workunit.client.1.smithi144.stdout:6/504: link d7/db/f2c d7/db/da2/d70/fbb 0 2024-05-15T15:54:14.386 INFO:tasks.workunit.client.1.smithi144.stdout:9/583: symlink d0/d8/d5e/d44/d99/lcc 0 2024-05-15T15:54:14.387 INFO:tasks.workunit.client.1.smithi144.stdout:3/571: write d0/d1/d17/d39/d83/f9d [1412874,31952] 0 2024-05-15T15:54:14.388 INFO:tasks.workunit.client.1.smithi144.stdout:8/613: rename d2/d24/d2d/da0/d6e/f56 to d2/de/d49/fca 0 2024-05-15T15:54:14.390 INFO:tasks.workunit.client.1.smithi144.stdout:7/563: dread d4/df/f34 [4194304,4194304] 0 2024-05-15T15:54:14.392 INFO:tasks.workunit.client.1.smithi144.stdout:2/540: mknod d3/d7/d20/d22/d52/cb3 0 2024-05-15T15:54:14.395 INFO:tasks.workunit.client.1.smithi144.stdout:6/505: rmdir d7/db/d35/d84 39 2024-05-15T15:54:14.401 INFO:tasks.workunit.client.0.smithi049.stdout:6/473: sync 2024-05-15T15:54:14.401 INFO:tasks.workunit.client.0.smithi049.stdout:2/523: sync 2024-05-15T15:54:14.401 INFO:tasks.workunit.client.1.smithi144.stdout:4/551: rename d6/dd/d34/d3b/d30/ce to d6/dd/d2e/d75/cb1 0 2024-05-15T15:54:14.402 INFO:tasks.workunit.client.0.smithi049.stdout:1/675: dwrite d0/d4/dd/de/d1e/d16/d8a/f8f [0,4194304] 0 2024-05-15T15:54:14.402 INFO:tasks.workunit.client.1.smithi144.stdout:3/572: dwrite d0/f2e [0,4194304] 0 2024-05-15T15:54:14.402 INFO:tasks.workunit.client.1.smithi144.stdout:5/586: mkdir d0/dc/d2f/d34/dc1 0 2024-05-15T15:54:14.403 INFO:tasks.workunit.client.1.smithi144.stdout:3/573: stat d0/d1/f43 0 2024-05-15T15:54:14.403 INFO:tasks.workunit.client.1.smithi144.stdout:5/587: fdatasync d0/dc/d1f/f75 0 2024-05-15T15:54:14.404 INFO:tasks.workunit.client.0.smithi049.stdout:1/676: stat d0/d4/dd/de/d48/d5c/fc6 0 2024-05-15T15:54:14.404 INFO:tasks.workunit.client.1.smithi144.stdout:3/574: write d0/d1/d56/f7f [716800,93908] 0 2024-05-15T15:54:14.406 INFO:tasks.workunit.client.1.smithi144.stdout:7/564: dread d4/df/f34 [0,4194304] 0 2024-05-15T15:54:14.407 INFO:tasks.workunit.client.1.smithi144.stdout:2/541: mkdir d3/d4d/db4 0 2024-05-15T15:54:14.412 INFO:tasks.workunit.client.0.smithi049.stdout:2/524: truncate d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d 155348 0 2024-05-15T15:54:14.414 INFO:tasks.workunit.client.0.smithi049.stdout:1/677: dwrite d0/d4/dd/de/d48/d5c/fc6 [0,4194304] 0 2024-05-15T15:54:14.415 INFO:tasks.workunit.client.1.smithi144.stdout:1/556: write d7/d1d/d33/d41/f19 [2287538,44815] 0 2024-05-15T15:54:14.415 INFO:tasks.workunit.client.1.smithi144.stdout:2/542: dwrite d3/d7/d20/d22/f7a [0,4194304] 0 2024-05-15T15:54:14.415 INFO:tasks.workunit.client.0.smithi049.stdout:2/525: fdatasync d3/db/d1d/d1e/d2b/d3b/d42/f69 0 2024-05-15T15:54:14.416 INFO:tasks.workunit.client.1.smithi144.stdout:1/557: write d7/d1d/d33/f3f [744004,35272] 0 2024-05-15T15:54:14.417 INFO:tasks.workunit.client.1.smithi144.stdout:9/584: mkdir d0/d8/d5e/d44/d16/dcd 0 2024-05-15T15:54:14.419 INFO:tasks.workunit.client.1.smithi144.stdout:9/585: stat d0/d8/d1b/d62/c92 0 2024-05-15T15:54:14.421 INFO:tasks.workunit.client.0.smithi049.stdout:2/526: mkdir d3/d8/daa 0 2024-05-15T15:54:14.423 INFO:tasks.workunit.client.1.smithi144.stdout:8/614: symlink d2/d24/d2d/da0/d6e/d71/d73/lcb 0 2024-05-15T15:54:14.423 INFO:tasks.workunit.client.0.smithi049.stdout:1/678: mkdir d0/d4/dd/de/d1e/d16/df1 0 2024-05-15T15:54:14.424 INFO:tasks.workunit.client.1.smithi144.stdout:4/552: creat d6/d42/d46/d4b/fb2 x:0 0 0 2024-05-15T15:54:14.425 INFO:tasks.workunit.client.0.smithi049.stdout:1/679: read d0/d4/dd/de/d1e/db7/fd7 [333818,30022] 0 2024-05-15T15:54:14.425 INFO:tasks.workunit.client.1.smithi144.stdout:5/588: creat d0/d7e/fc2 x:0 0 0 2024-05-15T15:54:14.426 INFO:tasks.workunit.client.1.smithi144.stdout:1/558: dwrite d7/d1d/d33/d41/d78/fb1 [0,4194304] 0 2024-05-15T15:54:14.427 INFO:tasks.workunit.client.0.smithi049.stdout:1/680: write d0/d4/dd/de/d4d/d8b/fc2 [803717,60309] 0 2024-05-15T15:54:14.427 INFO:tasks.workunit.client.0.smithi049.stdout:2/527: mknod d3/d52/cab 0 2024-05-15T15:54:14.428 INFO:tasks.workunit.client.1.smithi144.stdout:3/575: chown d0/d1/d56/d71/c82 11 1 2024-05-15T15:54:14.429 INFO:tasks.workunit.client.1.smithi144.stdout:3/576: stat d0/d1/d17/f48 0 2024-05-15T15:54:14.429 INFO:tasks.workunit.client.1.smithi144.stdout:3/577: stat d0/d11/d24/d78/fc2 0 2024-05-15T15:54:14.430 INFO:tasks.workunit.client.0.smithi049.stdout:1/681: chown d0/d4/dd/de/d1e/d16/d6e 7172416 1 2024-05-15T15:54:14.432 INFO:tasks.workunit.client.1.smithi144.stdout:7/565: mkdir d4/d3c/d76/d9d/dba 0 2024-05-15T15:54:14.432 INFO:tasks.workunit.client.0.smithi049.stdout:1/682: stat d0/d4/dd/de/d1e/d16/le7 0 2024-05-15T15:54:14.436 INFO:tasks.workunit.client.0.smithi049.stdout:1/683: stat d0/d4/dd/de/d1e/d2e/f41 0 2024-05-15T15:54:14.439 INFO:tasks.workunit.client.0.smithi049.stdout:0/510: sync 2024-05-15T15:54:14.439 INFO:tasks.workunit.client.0.smithi049.stdout:8/523: sync 2024-05-15T15:54:14.439 INFO:tasks.workunit.client.0.smithi049.stdout:3/550: sync 2024-05-15T15:54:14.439 INFO:tasks.workunit.client.0.smithi049.stdout:9/382: sync 2024-05-15T15:54:14.439 INFO:tasks.workunit.client.0.smithi049.stdout:5/490: sync 2024-05-15T15:54:14.439 INFO:tasks.workunit.client.0.smithi049.stdout:4/513: sync 2024-05-15T15:54:14.439 INFO:tasks.workunit.client.0.smithi049.stdout:7/436: sync 2024-05-15T15:54:14.440 INFO:tasks.workunit.client.0.smithi049.stdout:2/528: dwrite d3/db/f1f [0,4194304] 0 2024-05-15T15:54:14.442 INFO:tasks.workunit.client.1.smithi144.stdout:6/506: truncate d7/f4b 3021754 0 2024-05-15T15:54:14.444 INFO:tasks.workunit.client.1.smithi144.stdout:6/507: write d7/db/d9d/d97/f9a [389556,78846] 0 2024-05-15T15:54:14.447 INFO:tasks.workunit.client.0.smithi049.stdout:6/474: dwrite dd/f57 [0,4194304] 0 2024-05-15T15:54:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:14 smithi144 ceph-mon[36389]: pgmap v27: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 70 MiB/s rd, 256 MiB/s wr, 487 op/s 2024-05-15T15:54:14.449 INFO:tasks.workunit.client.0.smithi049.stdout:2/529: fdatasync d3/db/d1d/d1e/d2b/d3b/d42/f92 0 2024-05-15T15:54:14.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:14 smithi049 ceph-mon[28399]: pgmap v27: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 70 MiB/s rd, 256 MiB/s wr, 487 op/s 2024-05-15T15:54:14.455 INFO:tasks.workunit.client.0.smithi049.stdout:5/491: creat dc/dd/d39/fa0 x:0 0 0 2024-05-15T15:54:14.463 INFO:tasks.workunit.client.0.smithi049.stdout:7/437: mkdir d0/d6/d23/d45/d86 0 2024-05-15T15:54:14.464 INFO:tasks.workunit.client.0.smithi049.stdout:7/438: readlink d0/d2f/d5c/l66 0 2024-05-15T15:54:14.465 INFO:tasks.workunit.client.1.smithi144.stdout:1/559: symlink d7/df/d89/lba 0 2024-05-15T15:54:14.466 INFO:tasks.workunit.client.1.smithi144.stdout:3/578: creat d0/d11/d3b/da7/fc6 x:0 0 0 2024-05-15T15:54:14.468 INFO:tasks.workunit.client.0.smithi049.stdout:0/511: write dc/d13/d3d/f3f [1149842,74851] 0 2024-05-15T15:54:14.474 INFO:tasks.workunit.client.0.smithi049.stdout:9/383: write d0/d8/d9/fa [2678228,13264] 0 2024-05-15T15:54:14.477 INFO:tasks.workunit.client.1.smithi144.stdout:2/543: mkdir d3/d7/d20/d37/d76/db5 0 2024-05-15T15:54:14.478 INFO:tasks.workunit.client.0.smithi049.stdout:4/514: dwrite d18/d1e/d2a/d36/f58 [0,4194304] 0 2024-05-15T15:54:14.481 INFO:tasks.workunit.client.1.smithi144.stdout:2/544: dwrite d3/f6c [0,4194304] 0 2024-05-15T15:54:14.482 INFO:tasks.workunit.client.0.smithi049.stdout:1/684: fdatasync d0/d4/dd/de/d1e/d2e/dad/d65/d72/f8d 0 2024-05-15T15:54:14.482 INFO:tasks.workunit.client.1.smithi144.stdout:9/586: mkdir d0/d8/d1b/d62/d7d/d8d/dbb/dce 0 2024-05-15T15:54:14.485 INFO:tasks.workunit.client.1.smithi144.stdout:0/561: sync 2024-05-15T15:54:14.485 INFO:tasks.workunit.client.1.smithi144.stdout:9/587: read d0/d8/d5e/d44/d10/f24 [410258,83742] 0 2024-05-15T15:54:14.486 INFO:tasks.workunit.client.1.smithi144.stdout:4/553: getdents d6/dd/dae 0 2024-05-15T15:54:14.486 INFO:tasks.workunit.client.1.smithi144.stdout:6/508: write d7/db/f2c [669100,83968] 0 2024-05-15T15:54:14.488 INFO:tasks.workunit.client.1.smithi144.stdout:6/509: chown d7/d62/fa9 57 1 2024-05-15T15:54:14.489 INFO:tasks.workunit.client.1.smithi144.stdout:1/560: creat d7/d1d/d33/d1e/d4e/fbb x:0 0 0 2024-05-15T15:54:14.491 INFO:tasks.workunit.client.1.smithi144.stdout:6/510: chown d7/db/d11/d3c/l49 22 1 2024-05-15T15:54:14.495 INFO:tasks.workunit.client.1.smithi144.stdout:1/561: truncate d7/d1d/d33/d41/f19 3126401 0 2024-05-15T15:54:14.496 INFO:tasks.workunit.client.1.smithi144.stdout:9/588: dwrite d0/d8/d5e/d44/d77/f89 [0,4194304] 0 2024-05-15T15:54:14.496 INFO:tasks.workunit.client.0.smithi049.stdout:8/524: creat d2/de/d27/d5b/d60/da5/fa9 x:0 0 0 2024-05-15T15:54:14.496 INFO:tasks.workunit.client.0.smithi049.stdout:5/492: write dc/dd/d47/f5d [4882635,15340] 0 2024-05-15T15:54:14.499 INFO:tasks.workunit.client.1.smithi144.stdout:9/589: chown d0/d8/d5e/d44/dc2/f61 43 1 2024-05-15T15:54:14.500 INFO:tasks.workunit.client.1.smithi144.stdout:6/511: dwrite d7/d62/fa9 [0,4194304] 0 2024-05-15T15:54:14.501 INFO:tasks.workunit.client.0.smithi049.stdout:8/525: read d2/de/d71/d86/d5d/d63/f6e [123810,18066] 0 2024-05-15T15:54:14.501 INFO:tasks.workunit.client.1.smithi144.stdout:6/512: stat d7/db/d11/d3c/cad 0 2024-05-15T15:54:14.507 INFO:tasks.workunit.client.1.smithi144.stdout:9/590: dwrite d0/d8/d5e/d44/dc2/f42 [0,4194304] 0 2024-05-15T15:54:14.510 INFO:tasks.workunit.client.0.smithi049.stdout:5/493: dwrite dc/dd/d7b/d8d/f94 [0,4194304] 0 2024-05-15T15:54:14.511 INFO:tasks.workunit.client.0.smithi049.stdout:2/530: dread d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d [0,4194304] 0 2024-05-15T15:54:14.512 INFO:tasks.workunit.client.0.smithi049.stdout:5/494: write dc/d4a/f8c [195324,96364] 0 2024-05-15T15:54:14.514 INFO:tasks.workunit.client.0.smithi049.stdout:9/384: truncate d0/d8/d1b/f28 585349 0 2024-05-15T15:54:14.514 INFO:tasks.workunit.client.0.smithi049.stdout:5/495: chown dc/dd/d47/d83 228966 1 2024-05-15T15:54:14.515 INFO:tasks.workunit.client.1.smithi144.stdout:4/554: mknod d6/d42/d46/d4b/d1e/cb3 0 2024-05-15T15:54:14.516 INFO:tasks.workunit.client.1.smithi144.stdout:3/579: dwrite d0/d11/d24/f4f [0,4194304] 0 2024-05-15T15:54:14.516 INFO:tasks.workunit.client.0.smithi049.stdout:0/512: write dc/d13/d14/f1c [1302304,102417] 0 2024-05-15T15:54:14.517 INFO:tasks.workunit.client.0.smithi049.stdout:0/513: chown dc/d13/d14/d25/c97 100 1 2024-05-15T15:54:14.520 INFO:tasks.workunit.client.0.smithi049.stdout:0/514: fdatasync dc/d13/d14/d25/d31/f58 0 2024-05-15T15:54:14.521 INFO:tasks.workunit.client.1.smithi144.stdout:0/562: truncate d4/d6/d3a/d4c/f9a 389386 0 2024-05-15T15:54:14.526 INFO:tasks.workunit.client.0.smithi049.stdout:6/475: creat dd/d17/d24/d67/f84 x:0 0 0 2024-05-15T15:54:14.526 INFO:tasks.workunit.client.1.smithi144.stdout:7/566: link d4/d41/d5b/l5c d4/df/d81/lbb 0 2024-05-15T15:54:14.528 INFO:tasks.workunit.client.1.smithi144.stdout:8/615: sync 2024-05-15T15:54:14.528 INFO:tasks.workunit.client.1.smithi144.stdout:2/545: mkdir d3/d7/d20/d37/d76/db5/db6 0 2024-05-15T15:54:14.530 INFO:tasks.workunit.client.0.smithi049.stdout:7/439: dread d0/d2f/f51 [0,4194304] 0 2024-05-15T15:54:14.532 INFO:tasks.workunit.client.0.smithi049.stdout:3/551: rename d12/d26/d73/d95 to d12/d30/d42/da6 0 2024-05-15T15:54:14.533 INFO:tasks.workunit.client.1.smithi144.stdout:0/563: dwrite d4/d6/f11 [4194304,4194304] 0 2024-05-15T15:54:14.533 INFO:tasks.workunit.client.0.smithi049.stdout:8/526: dread - d2/de/f6c zero size 2024-05-15T15:54:14.533 INFO:tasks.workunit.client.1.smithi144.stdout:6/513: truncate d7/db/f4a 21580 0 2024-05-15T15:54:14.537 INFO:tasks.workunit.client.0.smithi049.stdout:3/552: dread - d12/d30/d63/d68/f91 zero size 2024-05-15T15:54:14.538 INFO:tasks.workunit.client.1.smithi144.stdout:5/589: getdents d0/dc/d12 0 2024-05-15T15:54:14.538 INFO:tasks.workunit.client.0.smithi049.stdout:5/496: chown dc/dd/f3b 22381 1 2024-05-15T15:54:14.541 INFO:tasks.workunit.client.0.smithi049.stdout:9/385: creat d0/d8/d53/f84 x:0 0 0 2024-05-15T15:54:14.542 INFO:tasks.workunit.client.0.smithi049.stdout:7/440: dwrite d0/d6/d23/f4c [8388608,4194304] 0 2024-05-15T15:54:14.544 INFO:tasks.workunit.client.1.smithi144.stdout:3/580: mkdir d0/d11/d3b/dc7 0 2024-05-15T15:54:14.549 INFO:tasks.workunit.client.1.smithi144.stdout:7/567: mkdir d4/df/d25/d40/d62/dbc 0 2024-05-15T15:54:14.550 INFO:tasks.workunit.client.0.smithi049.stdout:8/527: dwrite d2/de/d71/d86/f96 [0,4194304] 0 2024-05-15T15:54:14.552 INFO:tasks.workunit.client.0.smithi049.stdout:1/685: mkdir d0/d4/dd/de/d1e/d16/d8a/dd2/df2 0 2024-05-15T15:54:14.554 INFO:tasks.workunit.client.1.smithi144.stdout:1/562: rename d7/d1d/d33/d1e/d4e/l67 to d7/d1d/d33/d29/d6b/da4/lbc 0 2024-05-15T15:54:14.555 INFO:tasks.workunit.client.1.smithi144.stdout:2/546: creat d3/d4d/fb7 x:0 0 0 2024-05-15T15:54:14.558 INFO:tasks.workunit.client.1.smithi144.stdout:8/616: creat d2/d24/d2d/da0/d6e/d71/d8c/fcc x:0 0 0 2024-05-15T15:54:14.559 INFO:tasks.workunit.client.0.smithi049.stdout:6/476: rename dd/l1b to dd/d17/l85 0 2024-05-15T15:54:14.560 INFO:tasks.workunit.client.1.smithi144.stdout:8/617: write d2/d3a/d7b/d4b/f61 [2531903,54551] 0 2024-05-15T15:54:14.564 INFO:tasks.workunit.client.1.smithi144.stdout:0/564: truncate d4/df/d14/f5f 3768153 0 2024-05-15T15:54:14.566 INFO:tasks.workunit.client.1.smithi144.stdout:9/591: mknod d0/d8/d1b/d62/d7d/d8d/d94/ccf 0 2024-05-15T15:54:14.567 INFO:tasks.workunit.client.1.smithi144.stdout:9/592: chown d0/d8/d5e/d44/d16/d5a/cab 107 1 2024-05-15T15:54:14.568 INFO:tasks.workunit.client.1.smithi144.stdout:5/590: mknod d0/dc/d24/d28/d2d/d59/cc3 0 2024-05-15T15:54:14.569 INFO:tasks.workunit.client.1.smithi144.stdout:9/593: truncate d0/d8/d1b/d79/fc7 613482 0 2024-05-15T15:54:14.570 INFO:tasks.workunit.client.0.smithi049.stdout:5/497: symlink dc/dd/d47/la1 0 2024-05-15T15:54:14.572 INFO:tasks.workunit.client.0.smithi049.stdout:4/515: creat d18/d21/d2b/fac x:0 0 0 2024-05-15T15:54:14.572 INFO:tasks.workunit.client.1.smithi144.stdout:9/594: dread d0/d8/d1b/d62/fca [0,4194304] 0 2024-05-15T15:54:14.575 INFO:tasks.workunit.client.1.smithi144.stdout:1/563: mkdir d7/d1d/d33/d41/d22/dbd 0 2024-05-15T15:54:14.575 INFO:tasks.workunit.client.1.smithi144.stdout:1/564: stat d7/d1d/d58/l6d 0 2024-05-15T15:54:14.576 INFO:tasks.workunit.client.0.smithi049.stdout:9/386: dread - d0/f54 zero size 2024-05-15T15:54:14.577 INFO:tasks.workunit.client.1.smithi144.stdout:2/547: mkdir d3/d7/d20/d22/d52/d64/d97/db8 0 2024-05-15T15:54:14.577 INFO:tasks.workunit.client.1.smithi144.stdout:4/555: write d6/d7c/f8c [530703,89404] 0 2024-05-15T15:54:14.577 INFO:tasks.workunit.client.0.smithi049.stdout:7/441: creat d0/d6/d23/d4a/f87 x:0 0 0 2024-05-15T15:54:14.586 INFO:tasks.workunit.client.1.smithi144.stdout:5/591: mknod d0/dc/d12/d4a/d73/cc4 0 2024-05-15T15:54:14.590 INFO:tasks.workunit.client.0.smithi049.stdout:6/477: write dd/d17/d24/f27 [2597390,94371] 0 2024-05-15T15:54:14.590 INFO:tasks.workunit.client.0.smithi049.stdout:3/553: dread d12/d30/d42/f8e [4194304,4194304] 0 2024-05-15T15:54:14.591 INFO:tasks.workunit.client.1.smithi144.stdout:7/568: mkdir d4/df/d25/d40/d62/dbc/dbd 0 2024-05-15T15:54:14.591 INFO:tasks.workunit.client.1.smithi144.stdout:6/514: write d7/d1e/f44 [4685140,107951] 0 2024-05-15T15:54:14.594 INFO:tasks.workunit.client.0.smithi049.stdout:8/528: dread d2/de/d71/d86/d5d/d7d/d1b/f2b [0,4194304] 0 2024-05-15T15:54:14.594 INFO:tasks.workunit.client.1.smithi144.stdout:9/595: creat d0/d8/d7a/fd0 x:0 0 0 2024-05-15T15:54:14.596 INFO:tasks.workunit.client.1.smithi144.stdout:1/565: readlink d7/d1d/d33/d29/d6b/da4/lbc 0 2024-05-15T15:54:14.596 INFO:tasks.workunit.client.0.smithi049.stdout:8/529: read d2/de/d71/d86/d5d/f89 [1632134,27040] 0 2024-05-15T15:54:14.597 INFO:tasks.workunit.client.0.smithi049.stdout:8/530: chown d2/d18/l29 229380002 1 2024-05-15T15:54:14.598 INFO:tasks.workunit.client.0.smithi049.stdout:8/531: dread - d2/f5a zero size 2024-05-15T15:54:14.598 INFO:tasks.workunit.client.0.smithi049.stdout:4/516: mkdir d18/d1e/d2a/d50/d54/dad 0 2024-05-15T15:54:14.600 INFO:tasks.workunit.client.1.smithi144.stdout:4/556: fsync d6/f29 0 2024-05-15T15:54:14.601 INFO:tasks.workunit.client.0.smithi049.stdout:7/442: mkdir d0/d34/d88 0 2024-05-15T15:54:14.602 INFO:tasks.workunit.client.1.smithi144.stdout:4/557: write d6/d42/d46/d4b/f31 [4958900,124722] 0 2024-05-15T15:54:14.603 INFO:tasks.workunit.client.0.smithi049.stdout:7/443: truncate d0/d2f/d5c/f6e 898182 0 2024-05-15T15:54:14.606 INFO:tasks.workunit.client.1.smithi144.stdout:0/565: link d4/df/d1f/f58 d4/df/d1f/d24/d96/fca 0 2024-05-15T15:54:14.607 INFO:tasks.workunit.client.1.smithi144.stdout:0/566: stat d4/d6/d4a/ca9 0 2024-05-15T15:54:14.608 INFO:tasks.workunit.client.1.smithi144.stdout:0/567: read d4/d6/d3a/d4c/fa6 [682354,82973] 0 2024-05-15T15:54:14.608 INFO:tasks.workunit.client.1.smithi144.stdout:5/592: chown d0/dc/d24/d8e/d93/cac 810 1 2024-05-15T15:54:14.609 INFO:tasks.workunit.client.0.smithi049.stdout:6/478: read f0 [596988,33608] 0 2024-05-15T15:54:14.610 INFO:tasks.workunit.client.0.smithi049.stdout:6/479: dread - dd/d17/d24/d58/d5f/f79 zero size 2024-05-15T15:54:14.611 INFO:tasks.workunit.client.0.smithi049.stdout:3/554: chown d12/d14/d52/f7b 1 1 2024-05-15T15:54:14.613 INFO:tasks.workunit.client.0.smithi049.stdout:2/531: getdents d3/d8 0 2024-05-15T15:54:14.614 INFO:tasks.workunit.client.1.smithi144.stdout:7/569: creat d4/d3c/d76/d9d/dac/d48/d72/da2/fbe x:0 0 0 2024-05-15T15:54:14.614 INFO:tasks.workunit.client.1.smithi144.stdout:7/570: chown d4/d3c/d76 21 1 2024-05-15T15:54:14.616 INFO:tasks.workunit.client.1.smithi144.stdout:8/618: truncate d2/d3a/f7f 3130329 0 2024-05-15T15:54:14.620 INFO:tasks.workunit.client.0.smithi049.stdout:4/517: rename d18/d76/d81 to d18/d85/dae 0 2024-05-15T15:54:14.624 INFO:tasks.workunit.client.0.smithi049.stdout:9/387: creat d0/d8/d42/d75/f85 x:0 0 0 2024-05-15T15:54:14.626 INFO:tasks.workunit.client.1.smithi144.stdout:9/596: write d0/d8/fb8 [932060,51493] 0 2024-05-15T15:54:14.628 INFO:tasks.workunit.client.0.smithi049.stdout:0/515: getdents dc/d13 0 2024-05-15T15:54:14.632 INFO:tasks.workunit.client.0.smithi049.stdout:5/498: dwrite dc/d4a/f62 [0,4194304] 0 2024-05-15T15:54:14.633 INFO:tasks.workunit.client.0.smithi049.stdout:4/518: dwrite d18/d1e/d63/d99/fa5 [0,4194304] 0 2024-05-15T15:54:14.633 INFO:tasks.workunit.client.1.smithi144.stdout:0/568: creat d4/d6/fcb x:0 0 0 2024-05-15T15:54:14.635 INFO:tasks.workunit.client.1.smithi144.stdout:5/593: rename d0/dc/d1f/l2c to d0/dc/d24/d8e/lc5 0 2024-05-15T15:54:14.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.634+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 --> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1504108ab0 con 0x7f14ec06d6f0 2024-05-15T15:54:14.637 INFO:tasks.workunit.client.0.smithi049.stdout:6/480: mkdir dd/d17/d24/d2b/d36/d86 0 2024-05-15T15:54:14.638 INFO:tasks.workunit.client.0.smithi049.stdout:9/388: dwrite d0/d8/d9/d52/d5e/d5f/f63 [0,4194304] 0 2024-05-15T15:54:14.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.637+0000 7f14ebfff700 1 -- 172.21.15.49:0/345778123 <== mgr.24413 v2:172.21.15.144:6828/177983287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f1504108ab0 con 0x7f14ec06d6f0 2024-05-15T15:54:14.640 INFO:tasks.workunit.client.0.smithi049.stdout:6/481: readlink dd/d17/d24/d58/l77 0 2024-05-15T15:54:14.641 INFO:tasks.workunit.client.0.smithi049.stdout:2/532: unlink d3/db/d1d/d1e/d2b/l44 0 2024-05-15T15:54:14.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.641+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f14ec06d6f0 msgr2=0x7f14ec06fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:14.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.641+0000 7f1509345700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f14ec06d6f0 0x7f14ec06fbb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14f8006fd0 tx=0x7f14f8009190 comp rx=0 tx=0).stop 2024-05-15T15:54:14.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.641+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 msgr2=0x7f150419df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:14.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.641+0000 7f1509345700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 0x7f150419df20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005090 tx=0x7f14f400bd70 comp rx=0 tx=0).stop 2024-05-15T15:54:14.644 INFO:tasks.workunit.client.1.smithi144.stdout:7/571: creat d4/d3c/d76/d9d/dac/d48/d72/d82/fbf x:0 0 0 2024-05-15T15:54:14.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.642+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 shutdown_connections 2024-05-15T15:54:14.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.642+0000 7f1509345700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f14ec06d6f0 0x7f14ec06fbb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:14.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.642+0000 7f1509345700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1504072ca0 0x7f150419df20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:14.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.642+0000 7f1509345700 1 --2- 172.21.15.49:0/345778123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f150410b600 0x7f150419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:14.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.642+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 >> 172.21.15.49:0/345778123 conn(0x7f150406df00 msgr2=0x7f15040702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:14.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.642+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 shutdown_connections 2024-05-15T15:54:14.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.642+0000 7f1509345700 1 -- 172.21.15.49:0/345778123 wait complete. 2024-05-15T15:54:14.648 INFO:tasks.workunit.client.1.smithi144.stdout:8/619: mknod d2/d24/ccd 0 2024-05-15T15:54:14.649 INFO:tasks.workunit.client.1.smithi144.stdout:6/515: truncate d7/f68 3677844 0 2024-05-15T15:54:14.651 INFO:tasks.workunit.client.1.smithi144.stdout:3/581: sync 2024-05-15T15:54:14.652 INFO:tasks.workunit.client.0.smithi049.stdout:3/555: rename d12/d30/d63/d68/f77 to d12/d14/d32/fa7 0 2024-05-15T15:54:14.653 INFO:tasks.workunit.client.1.smithi144.stdout:3/582: write d0/d11/f26 [2052471,120021] 0 2024-05-15T15:54:14.653 INFO:tasks.workunit.client.0.smithi049.stdout:8/532: symlink d2/de/d27/d5b/da6/laa 0 2024-05-15T15:54:14.654 INFO:tasks.workunit.client.1.smithi144.stdout:3/583: readlink d0/d11/d3b/da7/lbd 0 2024-05-15T15:54:14.655 INFO:tasks.workunit.client.1.smithi144.stdout:2/548: creat d3/d7/d20/d22/fb9 x:0 0 0 2024-05-15T15:54:14.655 INFO:tasks.workunit.client.1.smithi144.stdout:8/620: dwrite d2/d24/d2d/da0/d9e/fae [0,4194304] 0 2024-05-15T15:54:14.656 INFO:tasks.workunit.client.1.smithi144.stdout:2/549: stat d3/d7/d20/l71 0 2024-05-15T15:54:14.657 INFO:tasks.workunit.client.1.smithi144.stdout:8/621: write d2/de/d3f/f9f [195574,12905] 0 2024-05-15T15:54:14.658 INFO:tasks.workunit.client.1.smithi144.stdout:2/550: write d3/d7/d20/d37/f3c [2066305,27568] 0 2024-05-15T15:54:14.659 INFO:tasks.workunit.client.0.smithi049.stdout:7/444: mknod d0/d6/c89 0 2024-05-15T15:54:14.661 INFO:tasks.workunit.client.1.smithi144.stdout:2/551: truncate d3/d7/d20/d22/d52/d75/d65/f93 693273 0 2024-05-15T15:54:14.661 INFO:tasks.workunit.client.0.smithi049.stdout:1/686: getdents d0/d4/d81 0 2024-05-15T15:54:14.662 INFO:tasks.workunit.client.1.smithi144.stdout:2/552: write d3/d7/d20/d37/f79 [270239,99911] 0 2024-05-15T15:54:14.665 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:54:14.666 INFO:tasks.workunit.client.0.smithi049.stdout:0/516: creat dc/d13/d2b/d76/f9a x:0 0 0 2024-05-15T15:54:14.668 INFO:tasks.workunit.client.0.smithi049.stdout:0/517: write dc/d13/d14/d25/d31/f7e [177680,107473] 0 2024-05-15T15:54:14.668 INFO:tasks.workunit.client.0.smithi049.stdout:5/499: symlink dc/dd/d7b/la2 0 2024-05-15T15:54:14.669 INFO:tasks.workunit.client.1.smithi144.stdout:7/572: dread d4/df/d25/d38/f71 [0,4194304] 0 2024-05-15T15:54:14.670 INFO:tasks.workunit.client.0.smithi049.stdout:5/500: fsync dc/d4a/f8c 0 2024-05-15T15:54:14.671 INFO:tasks.workunit.client.1.smithi144.stdout:7/573: read d4/d41/d5b/f4f [2685475,92264] 0 2024-05-15T15:54:14.672 INFO:tasks.workunit.client.1.smithi144.stdout:7/574: write d4/df/fb7 [228810,95616] 0 2024-05-15T15:54:14.673 INFO:tasks.workunit.client.1.smithi144.stdout:7/575: chown d4/df/d79 1035 1 2024-05-15T15:54:14.675 INFO:tasks.workunit.client.1.smithi144.stdout:5/594: creat d0/dc/d24/d8e/d93/d6b/fc6 x:0 0 0 2024-05-15T15:54:14.688 INFO:tasks.workunit.client.0.smithi049.stdout:2/533: dread d3/da/d15/f26 [0,4194304] 0 2024-05-15T15:54:14.688 INFO:tasks.workunit.client.0.smithi049.stdout:3/556: mkdir d12/d67/da8 0 2024-05-15T15:54:14.693 INFO:tasks.workunit.client.1.smithi144.stdout:6/516: dread d7/db/d35/d84/f98 [0,4194304] 0 2024-05-15T15:54:14.694 INFO:tasks.workunit.client.1.smithi144.stdout:9/597: unlink d0/d8/d1b/d79/c8f 0 2024-05-15T15:54:14.695 INFO:tasks.workunit.client.1.smithi144.stdout:9/598: dread - d0/d8/d50/f72 zero size 2024-05-15T15:54:14.696 INFO:tasks.workunit.client.0.smithi049.stdout:6/482: write dd/f53 [3571390,38094] 0 2024-05-15T15:54:14.697 INFO:tasks.workunit.client.1.smithi144.stdout:9/599: write d0/d8/d1b/d62/fc5 [502351,116258] 0 2024-05-15T15:54:14.697 INFO:tasks.workunit.client.0.smithi049.stdout:6/483: readlink la 0 2024-05-15T15:54:14.697 INFO:tasks.workunit.client.1.smithi144.stdout:6/517: dread d7/db/f29 [0,4194304] 0 2024-05-15T15:54:14.698 INFO:tasks.workunit.client.1.smithi144.stdout:9/600: chown d0/d8/d1b/d79/c9d 2 1 2024-05-15T15:54:14.698 INFO:tasks.workunit.client.1.smithi144.stdout:6/518: chown d7/d62/f95 1161109 1 2024-05-15T15:54:14.700 INFO:tasks.workunit.client.1.smithi144.stdout:4/558: getdents d6/dd/d34/d3b/d30/d59 0 2024-05-15T15:54:14.705 INFO:tasks.workunit.client.0.smithi049.stdout:7/445: write d0/d6/da/d3d/d39/f5b [5221754,38213] 0 2024-05-15T15:54:14.708 INFO:tasks.workunit.client.1.smithi144.stdout:8/622: write d2/de/d49/f76 [3458058,86691] 0 2024-05-15T15:54:14.710 INFO:tasks.workunit.client.0.smithi049.stdout:0/518: symlink dc/d13/d14/d2f/d8f/l9b 0 2024-05-15T15:54:14.711 INFO:tasks.workunit.client.1.smithi144.stdout:0/569: mknod d4/df/d1f/d24/d2b/d55/da5/ccc 0 2024-05-15T15:54:14.712 INFO:tasks.workunit.client.0.smithi049.stdout:0/519: write dc/d13/d14/f73 [525885,25989] 0 2024-05-15T15:54:14.713 INFO:tasks.workunit.client.0.smithi049.stdout:5/501: rmdir dc/dd/d10 39 2024-05-15T15:54:14.713 INFO:tasks.workunit.client.0.smithi049.stdout:7/446: dwrite d0/d6/d1d/f4d [0,4194304] 0 2024-05-15T15:54:14.716 INFO:tasks.workunit.client.1.smithi144.stdout:7/576: rmdir d4/df/d25/d38/d68 39 2024-05-15T15:54:14.717 INFO:tasks.workunit.client.0.smithi049.stdout:7/447: read d0/d34/f3f [7883428,25201] 0 2024-05-15T15:54:14.724 INFO:tasks.workunit.client.1.smithi144.stdout:1/566: getdents d7/d1d/d33/d41/d78 0 2024-05-15T15:54:14.728 INFO:tasks.workunit.client.0.smithi049.stdout:2/534: rename d3/d78/d8c/f96 to d3/db/d1d/d1e/d2b/d2e/d67/fac 0 2024-05-15T15:54:14.732 INFO:tasks.workunit.client.1.smithi144.stdout:3/584: mkdir d0/d1/d17/d39/dc8 0 2024-05-15T15:54:14.732 INFO:tasks.workunit.client.0.smithi049.stdout:9/389: dwrite d0/d8/d9/d22/f2e [0,4194304] 0 2024-05-15T15:54:14.735 INFO:tasks.workunit.client.0.smithi049.stdout:1/687: getdents d0/d4/dd/de/d1e/d16/d8a/dd2/df2 0 2024-05-15T15:54:14.738 INFO:tasks.workunit.client.1.smithi144.stdout:4/559: mkdir d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4 0 2024-05-15T15:54:14.739 INFO:tasks.workunit.client.1.smithi144.stdout:3/585: dwrite d0/d1/d17/d39/d40/d4a/d77/fc4 [0,4194304] 0 2024-05-15T15:54:14.742 INFO:tasks.workunit.client.1.smithi144.stdout:4/560: truncate d6/dd/d34/d3b/d30/db/d1d/d4e/f8b 1626880 0 2024-05-15T15:54:14.743 INFO:tasks.workunit.client.0.smithi049.stdout:6/484: write dd/d17/f21 [2587252,44625] 0 2024-05-15T15:54:14.747 INFO:tasks.workunit.client.0.smithi049.stdout:4/519: creat d18/d21/d2b/faf x:0 0 0 2024-05-15T15:54:14.748 INFO:tasks.workunit.client.1.smithi144.stdout:8/623: creat d2/d3a/fce x:0 0 0 2024-05-15T15:54:14.748 INFO:tasks.workunit.client.0.smithi049.stdout:5/502: mknod dc/dd/d47/ca3 0 2024-05-15T15:54:14.749 INFO:tasks.workunit.client.0.smithi049.stdout:8/533: dwrite d2/de/d71/d86/d5d/d7d/d45/f8a [0,4194304] 0 2024-05-15T15:54:14.749 INFO:tasks.workunit.client.1.smithi144.stdout:1/567: dread d7/d1d/d33/d29/d6b/d5e/f9d [0,4194304] 0 2024-05-15T15:54:14.750 INFO:tasks.workunit.client.1.smithi144.stdout:0/570: fsync d4/d6/d4a/d74/f79 0 2024-05-15T15:54:14.751 INFO:tasks.workunit.client.1.smithi144.stdout:1/568: read - d7/d1d/d33/d29/d6b/d5e/d74/fab zero size 2024-05-15T15:54:14.754 INFO:tasks.workunit.client.0.smithi049.stdout:8/534: stat d2/de/d71/d86/d5d/l91 0 2024-05-15T15:54:14.756 INFO:tasks.workunit.client.1.smithi144.stdout:5/595: mknod d0/dc/d1f/d50/db3/cc7 0 2024-05-15T15:54:14.760 INFO:tasks.workunit.client.0.smithi049.stdout:2/535: mkdir d3/d8/dad 0 2024-05-15T15:54:14.766 INFO:tasks.workunit.client.1.smithi144.stdout:4/561: dread d6/d42/f58 [0,4194304] 0 2024-05-15T15:54:14.767 INFO:tasks.workunit.client.0.smithi049.stdout:9/390: rename d0/d8/d53/d6b to d0/d8/d1b/d71/d86 0 2024-05-15T15:54:14.767 INFO:tasks.workunit.client.1.smithi144.stdout:4/562: chown d6/dd/d34/c3c 1 1 2024-05-15T15:54:14.768 INFO:tasks.workunit.client.1.smithi144.stdout:4/563: fdatasync d6/dd/d34/d3b/d30/db/d1d/d40/f97 0 2024-05-15T15:54:14.771 INFO:tasks.workunit.client.0.smithi049.stdout:0/520: dwrite dc/d13/d14/d33/f47 [0,4194304] 0 2024-05-15T15:54:14.773 INFO:tasks.workunit.client.0.smithi049.stdout:1/688: mkdir d0/d4/dd/de/d1e/d2e/dad/d65/df3 0 2024-05-15T15:54:14.774 INFO:tasks.workunit.client.1.smithi144.stdout:2/553: link d3/d7/d20/d22/d52/d64/l41 d3/d10/lba 0 2024-05-15T15:54:14.775 INFO:tasks.workunit.client.0.smithi049.stdout:6/485: fdatasync dd/d26/f2e 0 2024-05-15T15:54:14.777 INFO:tasks.workunit.client.0.smithi049.stdout:6/486: chown dd/d17/d24/d2b/d36/d50/c7d 49294 1 2024-05-15T15:54:14.778 INFO:tasks.workunit.client.1.smithi144.stdout:9/601: truncate d0/d8/d5e/d44/d10/d9c/f54 326709 0 2024-05-15T15:54:14.781 INFO:tasks.workunit.client.1.smithi144.stdout:8/624: mknod d2/d24/d2d/da0/ccf 0 2024-05-15T15:54:14.787 INFO:tasks.workunit.client.1.smithi144.stdout:7/577: creat d4/d3c/d76/d9d/dba/fc0 x:0 0 0 2024-05-15T15:54:14.788 INFO:tasks.workunit.client.0.smithi049.stdout:7/448: getdents d0/d34/d88 0 2024-05-15T15:54:14.790 INFO:tasks.workunit.client.0.smithi049.stdout:4/520: truncate d18/d21/d2b/d4e/f87 905802 0 2024-05-15T15:54:14.790 INFO:tasks.workunit.client.0.smithi049.stdout:7/449: write d0/d2f/d5c/f6e [515869,3316] 0 2024-05-15T15:54:14.790 INFO:tasks.workunit.client.0.smithi049.stdout:8/535: dread d2/de/d71/d86/f8d [0,4194304] 0 2024-05-15T15:54:14.794 INFO:tasks.workunit.client.0.smithi049.stdout:2/536: unlink d3/db/d1d/d1e/d2b/d2e/f6c 0 2024-05-15T15:54:14.796 INFO:tasks.workunit.client.0.smithi049.stdout:9/391: mkdir d0/d8/d1b/d71/d87 0 2024-05-15T15:54:14.797 INFO:tasks.workunit.client.1.smithi144.stdout:0/571: dwrite d4/df/d1f/d24/d2b/d55/f60 [0,4194304] 0 2024-05-15T15:54:14.797 INFO:tasks.workunit.client.1.smithi144.stdout:0/572: readlink d4/d2d/l51 0 2024-05-15T15:54:14.798 INFO:tasks.workunit.client.1.smithi144.stdout:0/573: chown d4/l1e 3 1 2024-05-15T15:54:14.801 INFO:tasks.workunit.client.1.smithi144.stdout:9/602: rmdir d0/d8/d5e/d44/d10/d19 39 2024-05-15T15:54:14.802 INFO:tasks.workunit.client.0.smithi049.stdout:6/487: creat dd/d26/d5b/f87 x:0 0 0 2024-05-15T15:54:14.808 INFO:tasks.workunit.client.1.smithi144.stdout:7/578: symlink d4/d41/d5b/da0/lc1 0 2024-05-15T15:54:14.814 INFO:tasks.workunit.client.1.smithi144.stdout:8/625: dread d2/d24/d2d/da0/d6e/d71/d8c/fc9 [0,4194304] 0 2024-05-15T15:54:14.814 INFO:tasks.workunit.client.1.smithi144.stdout:7/579: dwrite d4/d3c/fa3 [0,4194304] 0 2024-05-15T15:54:14.815 INFO:tasks.workunit.client.0.smithi049.stdout:4/521: dread - d18/d1e/d2a/d50/f73 zero size 2024-05-15T15:54:14.815 INFO:tasks.workunit.client.0.smithi049.stdout:7/450: rmdir d0/d6/d23/d45/d6a 39 2024-05-15T15:54:14.816 INFO:tasks.workunit.client.1.smithi144.stdout:8/626: chown d2/de/d36/f1e 15 1 2024-05-15T15:54:14.818 INFO:tasks.workunit.client.0.smithi049.stdout:4/522: write d18/d76/f92 [303053,100433] 0 2024-05-15T15:54:14.818 INFO:tasks.workunit.client.0.smithi049.stdout:0/521: dwrite dc/d13/d56/f90 [0,4194304] 0 2024-05-15T15:54:14.819 INFO:tasks.workunit.client.1.smithi144.stdout:8/627: write d2/d24/d2d/da0/d6e/d71/d8c/fc0 [286296,6314] 0 2024-05-15T15:54:14.819 INFO:tasks.workunit.client.0.smithi049.stdout:5/503: truncate dc/d4a/d31/d5c/f7d 1896862 0 2024-05-15T15:54:14.820 INFO:tasks.workunit.client.1.smithi144.stdout:2/554: dwrite d3/d7/d54/d98/f9c [0,4194304] 0 2024-05-15T15:54:14.821 INFO:tasks.workunit.client.0.smithi049.stdout:7/451: read d0/d6/f1b [4689844,24198] 0 2024-05-15T15:54:14.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.818+0000 7faa2585d700 1 -- 172.21.15.49:0/345328186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 msgr2=0x7faa201010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:14.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.818+0000 7faa2585d700 1 --2- 172.21.15.49:0/345328186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 0x7faa201010f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faa10009910 tx=0x7faa10009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.818+0000 7faa2585d700 1 -- 172.21.15.49:0/345328186 shutdown_connections 2024-05-15T15:54:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.819+0000 7faa2585d700 1 --2- 172.21.15.49:0/345328186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa20101ed0 0x7faa20102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.819+0000 7faa2585d700 1 --2- 172.21.15.49:0/345328186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 0x7faa201010f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:14.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.819+0000 7faa2585d700 1 -- 172.21.15.49:0/345328186 >> 172.21.15.49:0/345328186 conn(0x7faa200fc2b0 msgr2=0x7faa200fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:14.822 INFO:tasks.workunit.client.1.smithi144.stdout:8/628: fdatasync d2/d24/d2d/da0/d6e/d71/d8c/fcc 0 2024-05-15T15:54:14.822 INFO:tasks.workunit.client.1.smithi144.stdout:2/555: truncate d3/d7/d20/fb0 1807845 0 2024-05-15T15:54:14.823 INFO:tasks.workunit.client.1.smithi144.stdout:8/629: truncate d2/de/d49/d83/fa7 456152 0 2024-05-15T15:54:14.824 INFO:tasks.workunit.client.1.smithi144.stdout:3/586: link d0/d1/c55 d0/d1/d17/d39/d40/d97/cc9 0 2024-05-15T15:54:14.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.821+0000 7faa2585d700 1 -- 172.21.15.49:0/345328186 shutdown_connections 2024-05-15T15:54:14.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.822+0000 7faa2585d700 1 -- 172.21.15.49:0/345328186 wait complete. 2024-05-15T15:54:14.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.823+0000 7faa2585d700 1 Processor -- start 2024-05-15T15:54:14.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.823+0000 7faa2585d700 1 -- start start 2024-05-15T15:54:14.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.823+0000 7faa2585d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 0x7faa20074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:14.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.823+0000 7faa2585d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa20101ed0 0x7faa20073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:14.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.823+0000 7faa2585d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2019a350 con 0x7faa20100cd0 2024-05-15T15:54:14.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.823+0000 7faa2585d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2019a490 con 0x7faa20101ed0 2024-05-15T15:54:14.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.824+0000 7faa1effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 0x7faa20074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:14.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.824+0000 7faa1effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 0x7faa20074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54768/0 (socket says 172.21.15.49:54768) 2024-05-15T15:54:14.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.824+0000 7faa1effd700 1 -- 172.21.15.49:0/1480696721 learned_addr learned my addr 172.21.15.49:0/1480696721 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:14.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.824+0000 7faa1effd700 1 -- 172.21.15.49:0/1480696721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa20101ed0 msgr2=0x7faa20073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:14.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.824+0000 7faa1effd700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa20101ed0 0x7faa20073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:14.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.824+0000 7faa1effd700 1 -- 172.21.15.49:0/1480696721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa100095f0 con 0x7faa20100cd0 2024-05-15T15:54:14.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.824+0000 7faa1effd700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 0x7faa20074ca0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa10009fd0 tx=0x7faa10005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:14.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.825+0000 7faa2485b700 1 -- 172.21.15.49:0/1480696721 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa1001d070 con 0x7faa20100cd0 2024-05-15T15:54:14.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.825+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2019a710 con 0x7faa20100cd0 2024-05-15T15:54:14.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.825+0000 7faa2485b700 1 -- 172.21.15.49:0/1480696721 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa1000bc50 con 0x7faa20100cd0 2024-05-15T15:54:14.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.825+0000 7faa2485b700 1 -- 172.21.15.49:0/1480696721 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa10004a00 con 0x7faa20100cd0 2024-05-15T15:54:14.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.825+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2019ac80 con 0x7faa20100cd0 2024-05-15T15:54:14.828 INFO:tasks.workunit.client.1.smithi144.stdout:5/596: truncate d0/dc/d12/f18 4630266 0 2024-05-15T15:54:14.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.826+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2004ec60 con 0x7faa20100cd0 2024-05-15T15:54:14.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.827+0000 7faa2485b700 1 -- 172.21.15.49:0/1480696721 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7faa10022470 con 0x7faa20100cd0 2024-05-15T15:54:14.831 INFO:tasks.workunit.client.0.smithi049.stdout:2/537: fdatasync d3/da/f45 0 2024-05-15T15:54:14.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.828+0000 7faa2485b700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7faa0806d6f0 0x7faa0806fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:14.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.829+0000 7faa1e7fc700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7faa0806d6f0 0x7faa0806fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.829+0000 7faa2485b700 1 -- 172.21.15.49:0/1480696721 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7faa1008eb00 con 0x7faa20100cd0 2024-05-15T15:54:14.832 INFO:tasks.workunit.client.0.smithi049.stdout:2/538: fdatasync d3/db/f1f 0 2024-05-15T15:54:14.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.830+0000 7faa1e7fc700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7faa0806d6f0 0x7faa0806fbb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa14005950 tx=0x7faa140058e0 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:14.834 INFO:tasks.workunit.client.1.smithi144.stdout:4/564: truncate d6/dd/d34/d3b/d30/db/d1d/f41 1644186 0 2024-05-15T15:54:14.835 INFO:tasks.workunit.client.1.smithi144.stdout:4/565: read - d6/d42/d46/d4b/d1e/d99/dab/f9a zero size 2024-05-15T15:54:14.836 INFO:tasks.workunit.client.0.smithi049.stdout:9/392: mkdir d0/d8/d9/d52/d5e/d5f/d88 0 2024-05-15T15:54:14.837 INFO:tasks.workunit.client.1.smithi144.stdout:0/574: mknod d4/dc/ccd 0 2024-05-15T15:54:14.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:14.837+0000 7faa2485b700 1 -- 172.21.15.49:0/1480696721 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7faa1005c060 con 0x7faa20100cd0 2024-05-15T15:54:14.840 INFO:tasks.workunit.client.1.smithi144.stdout:6/519: link d7/db/ce d7/db/cbc 0 2024-05-15T15:54:14.841 INFO:tasks.workunit.client.0.smithi049.stdout:8/536: write d2/de/d25/f32 [2518583,124227] 0 2024-05-15T15:54:14.841 INFO:tasks.workunit.client.1.smithi144.stdout:6/520: dread - d7/f41 zero size 2024-05-15T15:54:14.841 INFO:tasks.workunit.client.1.smithi144.stdout:6/521: dread - d7/db/d11/fa5 zero size 2024-05-15T15:54:14.845 INFO:tasks.workunit.client.1.smithi144.stdout:7/580: mknod d4/df/d25/d40/d62/dbc/cc2 0 2024-05-15T15:54:14.847 INFO:tasks.workunit.client.1.smithi144.stdout:6/522: dwrite d7/f78 [0,4194304] 0 2024-05-15T15:54:14.849 INFO:tasks.workunit.client.1.smithi144.stdout:6/523: chown d7/db/d35/d84/d81/f83 54 1 2024-05-15T15:54:14.851 INFO:tasks.workunit.client.0.smithi049.stdout:3/557: getdents d12/d30/d63/d68/d70 0 2024-05-15T15:54:14.852 INFO:tasks.workunit.client.0.smithi049.stdout:8/537: dwrite d2/de/d27/d5b/f88 [0,4194304] 0 2024-05-15T15:54:14.852 INFO:tasks.workunit.client.1.smithi144.stdout:1/569: sync 2024-05-15T15:54:14.853 INFO:tasks.workunit.client.1.smithi144.stdout:1/570: truncate d7/df/f9f 259258 0 2024-05-15T15:54:14.854 INFO:tasks.workunit.client.1.smithi144.stdout:6/524: dwrite d7/d62/fa9 [0,4194304] 0 2024-05-15T15:54:14.856 INFO:tasks.workunit.client.1.smithi144.stdout:8/630: readlink d2/d3a/d7b/d4b/lc5 0 2024-05-15T15:54:14.859 INFO:tasks.workunit.client.0.smithi049.stdout:5/504: fsync dc/d4a/d42/f48 0 2024-05-15T15:54:14.862 INFO:tasks.workunit.client.1.smithi144.stdout:2/556: dread d3/d7/f1f [0,4194304] 0 2024-05-15T15:54:14.866 INFO:tasks.workunit.client.0.smithi049.stdout:5/505: dwrite dc/d4a/d31/f82 [0,4194304] 0 2024-05-15T15:54:14.867 INFO:tasks.workunit.client.1.smithi144.stdout:5/597: mknod d0/dc/d24/d8e/d95/cc8 0 2024-05-15T15:54:14.868 INFO:tasks.workunit.client.0.smithi049.stdout:0/522: mknod dc/d13/d14/d25/c9c 0 2024-05-15T15:54:14.868 INFO:tasks.workunit.client.1.smithi144.stdout:5/598: chown d0/dc/d24/d57/f7f 6821 1 2024-05-15T15:54:14.868 INFO:tasks.workunit.client.0.smithi049.stdout:0/523: chown dc/d13/d14/f95 110 1 2024-05-15T15:54:14.877 INFO:tasks.workunit.client.1.smithi144.stdout:0/575: creat d4/d2d/fce x:0 0 0 2024-05-15T15:54:14.879 INFO:tasks.workunit.client.0.smithi049.stdout:8/538: fdatasync d2/de/f55 0 2024-05-15T15:54:14.881 INFO:tasks.workunit.client.1.smithi144.stdout:9/603: truncate d0/d8/d5e/d44/d10/d19/f67 470691 0 2024-05-15T15:54:14.882 INFO:tasks.workunit.client.1.smithi144.stdout:9/604: chown d0/f14 433992911 1 2024-05-15T15:54:14.887 INFO:tasks.workunit.client.0.smithi049.stdout:1/689: link d0/d4/dd/de/d1e/l2a d0/d4/d81/dab/lf4 0 2024-05-15T15:54:14.888 INFO:tasks.workunit.client.0.smithi049.stdout:1/690: chown d0/d4/f2d 74192724 1 2024-05-15T15:54:14.892 INFO:tasks.workunit.client.1.smithi144.stdout:1/571: unlink d7/df/l7d 0 2024-05-15T15:54:14.893 INFO:tasks.workunit.client.1.smithi144.stdout:8/631: write d2/d24/f68 [386841,106307] 0 2024-05-15T15:54:14.900 INFO:tasks.workunit.client.1.smithi144.stdout:5/599: mknod d0/dc/d2f/d79/cc9 0 2024-05-15T15:54:14.900 INFO:tasks.workunit.client.1.smithi144.stdout:5/600: chown d0/dc/d2f/d34/d63/d6e/c6f 1997 1 2024-05-15T15:54:14.902 INFO:tasks.workunit.client.1.smithi144.stdout:4/566: rename d6/dd/d34/d3b/d30/db/f7b to d6/dd/d2e/d75/fb5 0 2024-05-15T15:54:14.903 INFO:tasks.workunit.client.1.smithi144.stdout:0/576: fdatasync d4/d6/d3a/f91 0 2024-05-15T15:54:14.904 INFO:tasks.workunit.client.1.smithi144.stdout:4/567: truncate d6/dd/d34/d3b/d30/db/d1d/d4e/f8b 1888444 0 2024-05-15T15:54:14.905 INFO:tasks.workunit.client.1.smithi144.stdout:0/577: dread - d4/df/d1f/d24/d2b/f86 zero size 2024-05-15T15:54:14.906 INFO:tasks.workunit.client.0.smithi049.stdout:3/558: mknod d12/d30/d42/d7a/ca9 0 2024-05-15T15:54:14.908 INFO:tasks.workunit.client.1.smithi144.stdout:6/525: mkdir d7/db/d9d/d72/dbd 0 2024-05-15T15:54:14.908 INFO:tasks.workunit.client.0.smithi049.stdout:4/523: getdents d18/d21/d2b/d4e 0 2024-05-15T15:54:14.909 INFO:tasks.workunit.client.1.smithi144.stdout:6/526: read d7/db/d11/d3c/f77 [292054,112541] 0 2024-05-15T15:54:14.910 INFO:tasks.workunit.client.0.smithi049.stdout:4/524: fdatasync d18/d1e/f8e 0 2024-05-15T15:54:14.910 INFO:tasks.workunit.client.1.smithi144.stdout:6/527: fsync d7/f78 0 2024-05-15T15:54:14.910 INFO:tasks.workunit.client.1.smithi144.stdout:6/528: stat d7/db/d35/f67 0 2024-05-15T15:54:14.911 INFO:tasks.workunit.client.1.smithi144.stdout:8/632: dread d2/de/d36/f4d [0,4194304] 0 2024-05-15T15:54:14.912 INFO:tasks.workunit.client.0.smithi049.stdout:2/539: write d3/da/d15/d24/f64 [210540,5112] 0 2024-05-15T15:54:14.913 INFO:tasks.workunit.client.1.smithi144.stdout:5/601: creat d0/dc/d24/d28/d2d/d59/fca x:0 0 0 2024-05-15T15:54:14.914 INFO:tasks.workunit.client.1.smithi144.stdout:5/602: dread - d0/d7e/d9b/fa2 zero size 2024-05-15T15:54:14.914 INFO:tasks.workunit.client.0.smithi049.stdout:3/559: dwrite d12/d14/d32/d64/f93 [0,4194304] 0 2024-05-15T15:54:14.914 INFO:tasks.workunit.client.0.smithi049.stdout:2/540: chown d3/db/d1d/d1e/d2b/d3b/d42 1503664 1 2024-05-15T15:54:14.917 INFO:tasks.workunit.client.1.smithi144.stdout:7/581: rename d4/df/f34 to d4/df/d25/d40/d62/dbc/fc3 0 2024-05-15T15:54:14.923 INFO:tasks.workunit.client.0.smithi049.stdout:0/524: rename dc/d6d/c64 to dc/d13/d14/d33/c9d 0 2024-05-15T15:54:14.924 INFO:tasks.workunit.client.1.smithi144.stdout:9/605: mkdir d0/d8/d1b/d62/d7d/d8d/dbb/dce/dd1 0 2024-05-15T15:54:14.925 INFO:tasks.workunit.client.1.smithi144.stdout:3/587: truncate d0/d1/f23 319621 0 2024-05-15T15:54:14.925 INFO:tasks.workunit.client.1.smithi144.stdout:9/606: write d0/d8/d5e/d44/d10/d9c/f98 [705433,103867] 0 2024-05-15T15:54:14.927 INFO:tasks.workunit.client.1.smithi144.stdout:2/557: write d3/d7/d20/d3b/f57 [573552,87109] 0 2024-05-15T15:54:14.932 INFO:tasks.workunit.client.0.smithi049.stdout:8/539: creat d2/de/d71/d86/d5d/d7d/d1b/d68/fab x:0 0 0 2024-05-15T15:54:14.933 INFO:tasks.workunit.client.0.smithi049.stdout:8/540: chown d2/de/d27/d5b/d93 147030 1 2024-05-15T15:54:14.938 INFO:tasks.workunit.client.1.smithi144.stdout:7/582: dread d4/df/f11 [0,4194304] 0 2024-05-15T15:54:14.938 INFO:tasks.workunit.client.1.smithi144.stdout:7/583: write d4/d3c/d76/d9d/dba/fc0 [1048457,109522] 0 2024-05-15T15:54:14.939 INFO:tasks.workunit.client.0.smithi049.stdout:9/393: getdents d0/d8/d1b/d40/d5c/d69 0 2024-05-15T15:54:14.939 INFO:tasks.workunit.client.0.smithi049.stdout:6/488: link dd/l2a dd/d17/d24/d2b/l88 0 2024-05-15T15:54:14.941 INFO:tasks.workunit.client.1.smithi144.stdout:7/584: dwrite d4/f1c [0,4194304] 0 2024-05-15T15:54:14.942 INFO:tasks.workunit.client.0.smithi049.stdout:9/394: dwrite d0/d8/d42/f62 [0,4194304] 0 2024-05-15T15:54:14.942 INFO:tasks.workunit.client.0.smithi049.stdout:4/525: fsync d18/f7c 0 2024-05-15T15:54:14.943 INFO:tasks.workunit.client.1.smithi144.stdout:8/633: fdatasync d2/d24/d2d/f54 0 2024-05-15T15:54:14.944 INFO:tasks.workunit.client.0.smithi049.stdout:9/395: chown d0/d25/d5d/c6c 0 1 2024-05-15T15:54:14.946 INFO:tasks.workunit.client.0.smithi049.stdout:9/396: truncate d0/d8/d53/f68 210974 0 2024-05-15T15:54:14.953 INFO:tasks.workunit.client.0.smithi049.stdout:2/541: mknod d3/d8/cae 0 2024-05-15T15:54:14.954 INFO:tasks.workunit.client.0.smithi049.stdout:5/506: dwrite dc/d4a/d31/d5c/f7e [0,4194304] 0 2024-05-15T15:54:14.956 INFO:tasks.workunit.client.1.smithi144.stdout:4/568: fdatasync d6/dd/d34/d3b/d30/db/f33 0 2024-05-15T15:54:14.960 INFO:tasks.workunit.client.0.smithi049.stdout:0/525: symlink dc/d13/d2b/l9e 0 2024-05-15T15:54:14.961 INFO:tasks.workunit.client.0.smithi049.stdout:0/526: write dc/d13/f91 [404321,117995] 0 2024-05-15T15:54:14.961 INFO:tasks.workunit.client.0.smithi049.stdout:7/452: creat d0/d6/d23/f8a x:0 0 0 2024-05-15T15:54:14.964 INFO:tasks.workunit.client.0.smithi049.stdout:0/527: write dc/d13/d14/f73 [730765,128252] 0 2024-05-15T15:54:14.964 INFO:tasks.workunit.client.1.smithi144.stdout:1/572: creat d7/d1d/d33/d1e/fbe x:0 0 0 2024-05-15T15:54:14.966 INFO:tasks.workunit.client.1.smithi144.stdout:0/578: dwrite d4/d2d/f8d [0,4194304] 0 2024-05-15T15:54:14.967 INFO:tasks.workunit.client.0.smithi049.stdout:1/691: symlink d0/d4/dd/de/d48/d5c/lf5 0 2024-05-15T15:54:14.968 INFO:tasks.workunit.client.1.smithi144.stdout:4/569: dwrite d6/dd/d34/d3b/d30/d59/d66/f9e [0,4194304] 0 2024-05-15T15:54:14.971 INFO:tasks.workunit.client.0.smithi049.stdout:7/453: dwrite d0/d6/d23/f4c [0,4194304] 0 2024-05-15T15:54:14.974 INFO:tasks.workunit.client.1.smithi144.stdout:0/579: write d4/d6/d3a/d4c/f70 [497014,9628] 0 2024-05-15T15:54:14.974 INFO:tasks.workunit.client.0.smithi049.stdout:4/526: rmdir d18/d1e/d2a/d36/d5d 39 2024-05-15T15:54:14.975 INFO:tasks.workunit.client.1.smithi144.stdout:6/529: dwrite d7/db/d11/f1f [8388608,4194304] 0 2024-05-15T15:54:14.975 INFO:tasks.workunit.client.1.smithi144.stdout:7/585: truncate d4/df/f1a 73359 0 2024-05-15T15:54:14.977 INFO:tasks.workunit.client.1.smithi144.stdout:6/530: fsync d7/db/d11/f60 0 2024-05-15T15:54:14.979 INFO:tasks.workunit.client.0.smithi049.stdout:9/397: creat d0/d8/d9/d52/d5e/d5f/f89 x:0 0 0 2024-05-15T15:54:14.980 INFO:tasks.workunit.client.1.smithi144.stdout:8/634: write d2/d24/d2d/da0/d6e/d71/d8c/fc9 [2661404,10469] 0 2024-05-15T15:54:14.982 INFO:tasks.workunit.client.1.smithi144.stdout:8/635: stat d2/de/d49/d83/c84 0 2024-05-15T15:54:14.983 INFO:tasks.workunit.client.1.smithi144.stdout:3/588: mknod d0/d1/d17/d39/cca 0 2024-05-15T15:54:14.984 INFO:tasks.workunit.client.1.smithi144.stdout:9/607: write d0/f78 [73300,58506] 0 2024-05-15T15:54:14.985 INFO:tasks.workunit.client.0.smithi049.stdout:2/542: fdatasync d3/d8/f50 0 2024-05-15T15:54:14.986 INFO:tasks.workunit.client.1.smithi144.stdout:9/608: chown d0/d8/d21/c70 720356 1 2024-05-15T15:54:14.988 INFO:tasks.workunit.client.0.smithi049.stdout:5/507: mknod dc/d4a/d42/ca4 0 2024-05-15T15:54:14.990 INFO:tasks.workunit.client.1.smithi144.stdout:2/558: symlink d3/d7/d20/d22/lbb 0 2024-05-15T15:54:14.995 INFO:tasks.workunit.client.1.smithi144.stdout:1/573: dread d7/d1d/d33/d41/d22/f90 [0,4194304] 0 2024-05-15T15:54:14.996 INFO:tasks.workunit.client.0.smithi049.stdout:1/692: creat d0/d4/dd/de/d48/ff6 x:0 0 0 2024-05-15T15:54:14.997 INFO:tasks.workunit.client.1.smithi144.stdout:4/570: symlink d6/d42/d46/d4b/d1e/d99/dab/lb6 0 2024-05-15T15:54:14.998 INFO:tasks.workunit.client.1.smithi144.stdout:1/574: chown d7/df/c16 10 1 2024-05-15T15:54:14.998 INFO:tasks.workunit.client.1.smithi144.stdout:2/559: dwrite d3/d4d/fae [0,4194304] 0 2024-05-15T15:54:15.000 INFO:tasks.workunit.client.1.smithi144.stdout:3/589: dread d0/d11/f14 [0,4194304] 0 2024-05-15T15:54:15.003 INFO:tasks.workunit.client.1.smithi144.stdout:3/590: write d0/d1/d17/d39/d40/d4a/d77/fb0 [1293111,65202] 0 2024-05-15T15:54:15.004 INFO:tasks.workunit.client.1.smithi144.stdout:3/591: chown d0/l7 4997 1 2024-05-15T15:54:15.004 INFO:tasks.workunit.client.1.smithi144.stdout:3/592: readlink d0/d11/d24/da5/lbf 0 2024-05-15T15:54:15.005 INFO:tasks.workunit.client.0.smithi049.stdout:4/527: fdatasync d18/d21/f45 0 2024-05-15T15:54:15.010 INFO:tasks.workunit.client.0.smithi049.stdout:2/543: creat d3/db/d1d/d1e/d2b/d3b/d47/d9a/faf x:0 0 0 2024-05-15T15:54:15.012 INFO:tasks.workunit.client.1.smithi144.stdout:8/636: mkdir d2/d24/d2d/d30/dd0 0 2024-05-15T15:54:15.012 INFO:tasks.workunit.client.0.smithi049.stdout:0/528: dread dc/f34 [0,4194304] 0 2024-05-15T15:54:15.013 INFO:tasks.workunit.client.0.smithi049.stdout:5/508: fdatasync dc/f7f 0 2024-05-15T15:54:15.015 INFO:tasks.workunit.client.1.smithi144.stdout:9/609: mkdir d0/d8/d1b/d62/d7d/dd2 0 2024-05-15T15:54:15.018 INFO:tasks.workunit.client.0.smithi049.stdout:1/693: unlink d0/d4/d81/f9f 0 2024-05-15T15:54:15.020 INFO:tasks.workunit.client.0.smithi049.stdout:6/489: rename f8 to dd/f89 0 2024-05-15T15:54:15.025 INFO:tasks.workunit.client.0.smithi049.stdout:0/529: dread dc/d13/f17 [0,4194304] 0 2024-05-15T15:54:15.026 INFO:tasks.workunit.client.0.smithi049.stdout:7/454: unlink d0/d6/d23/c28 0 2024-05-15T15:54:15.027 INFO:tasks.workunit.client.1.smithi144.stdout:5/603: sync 2024-05-15T15:54:15.027 INFO:tasks.workunit.client.0.smithi049.stdout:7/455: readlink d0/d6/d1d/l2b 0 2024-05-15T15:54:15.028 INFO:tasks.workunit.client.0.smithi049.stdout:3/560: write d12/f4d [440999,80256] 0 2024-05-15T15:54:15.028 INFO:tasks.workunit.client.1.smithi144.stdout:5/604: write d0/dc/d24/d57/fa6 [421231,44909] 0 2024-05-15T15:54:15.029 INFO:tasks.workunit.client.0.smithi049.stdout:6/490: dwrite dd/d17/f21 [0,4194304] 0 2024-05-15T15:54:15.029 INFO:tasks.workunit.client.1.smithi144.stdout:5/605: chown d0/dc/d24/d8e/d93/fa7 61854 1 2024-05-15T15:54:15.029 INFO:tasks.workunit.client.0.smithi049.stdout:4/528: stat d18/d21/d2b/c35 0 2024-05-15T15:54:15.030 INFO:tasks.workunit.client.1.smithi144.stdout:5/606: read d0/f49 [242452,47206] 0 2024-05-15T15:54:15.031 INFO:tasks.workunit.client.0.smithi049.stdout:6/491: write dd/d17/d64/f6a [3622847,73416] 0 2024-05-15T15:54:15.034 INFO:tasks.workunit.client.1.smithi144.stdout:7/586: dread d4/d41/f49 [4194304,4194304] 0 2024-05-15T15:54:15.037 INFO:tasks.workunit.client.0.smithi049.stdout:5/509: chown dc/d4a/f3a 270309 1 2024-05-15T15:54:15.042 INFO:tasks.workunit.client.1.smithi144.stdout:3/593: creat d0/d11/d3b/da0/d8f/fcb x:0 0 0 2024-05-15T15:54:15.042 INFO:tasks.workunit.client.0.smithi049.stdout:5/510: dread dc/d4a/d31/f82 [0,4194304] 0 2024-05-15T15:54:15.043 INFO:tasks.workunit.client.0.smithi049.stdout:5/511: chown dc/dd/d7b/f8f 1 1 2024-05-15T15:54:15.044 INFO:tasks.workunit.client.0.smithi049.stdout:5/512: readlink dc/dd/d7b/l86 0 2024-05-15T15:54:15.045 INFO:tasks.workunit.client.0.smithi049.stdout:5/513: readlink dc/l24 0 2024-05-15T15:54:15.046 INFO:tasks.workunit.client.0.smithi049.stdout:5/514: write dc/dd/d7b/d8d/f94 [704089,58343] 0 2024-05-15T15:54:15.047 INFO:tasks.workunit.client.0.smithi049.stdout:9/398: rename d0/d8/d11 to d0/d8/d9/d52/d5e/d8a 0 2024-05-15T15:54:15.048 INFO:tasks.workunit.client.1.smithi144.stdout:6/531: write d7/db/d35/f47 [4685982,43403] 0 2024-05-15T15:54:15.051 INFO:tasks.workunit.client.1.smithi144.stdout:8/637: creat d2/d24/d2d/da0/d6e/d71/db1/fd1 x:0 0 0 2024-05-15T15:54:15.053 INFO:tasks.workunit.client.0.smithi049.stdout:0/530: creat dc/d13/d14/d2f/d8f/f9f x:0 0 0 2024-05-15T15:54:15.058 INFO:tasks.workunit.client.1.smithi144.stdout:2/560: mknod d3/d7/d20/d89/cbc 0 2024-05-15T15:54:15.059 INFO:tasks.workunit.client.0.smithi049.stdout:7/456: fsync d0/d2f/f48 0 2024-05-15T15:54:15.060 INFO:tasks.workunit.client.0.smithi049.stdout:7/457: chown d0/d6/d23/d76 58392 1 2024-05-15T15:54:15.061 INFO:tasks.workunit.client.0.smithi049.stdout:7/458: write d0/d6/d1d/f4d [4529241,58586] 0 2024-05-15T15:54:15.063 INFO:tasks.workunit.client.0.smithi049.stdout:7/459: truncate d0/d6/d23/d4a/f87 939514 0 2024-05-15T15:54:15.069 INFO:tasks.workunit.client.1.smithi144.stdout:7/587: truncate d4/d3c/d76/d9d/dac/f3b 815118 0 2024-05-15T15:54:15.070 INFO:tasks.workunit.client.1.smithi144.stdout:6/532: mkdir d7/db/d35/d84/dbe 0 2024-05-15T15:54:15.071 INFO:tasks.workunit.client.1.smithi144.stdout:6/533: chown d7/db/d11/fa5 2554 1 2024-05-15T15:54:15.075 INFO:tasks.workunit.client.0.smithi049.stdout:9/399: mknod d0/d8/d9/d52/d5e/d8a/c8b 0 2024-05-15T15:54:15.076 INFO:tasks.workunit.client.0.smithi049.stdout:1/694: dwrite d0/f58 [0,4194304] 0 2024-05-15T15:54:15.078 INFO:tasks.workunit.client.0.smithi049.stdout:1/695: stat d0/d4/dd/de/d4d/d8b/fc2 0 2024-05-15T15:54:15.080 INFO:tasks.workunit.client.1.smithi144.stdout:1/575: rename d7/d1d/d33/d29/f2c to d7/fbf 0 2024-05-15T15:54:15.081 INFO:tasks.workunit.client.1.smithi144.stdout:9/610: write d0/d8/d5e/d44/d99/fa1 [1432717,2528] 0 2024-05-15T15:54:15.083 INFO:tasks.workunit.client.1.smithi144.stdout:5/607: creat d0/dc/d24/d8e/d93/d9e/fcb x:0 0 0 2024-05-15T15:54:15.084 INFO:tasks.workunit.client.1.smithi144.stdout:6/534: dread d7/db/d11/f1f [4194304,4194304] 0 2024-05-15T15:54:15.084 INFO:tasks.workunit.client.1.smithi144.stdout:5/608: dread - d0/d7e/fc2 zero size 2024-05-15T15:54:15.085 INFO:tasks.workunit.client.0.smithi049.stdout:0/531: dwrite f3 [0,4194304] 0 2024-05-15T15:54:15.089 INFO:tasks.workunit.client.0.smithi049.stdout:3/561: dwrite ff [0,4194304] 0 2024-05-15T15:54:15.095 INFO:tasks.workunit.client.0.smithi049.stdout:7/460: mknod d0/d6/da/d3d/c8b 0 2024-05-15T15:54:15.095 INFO:tasks.workunit.client.1.smithi144.stdout:4/571: dwrite d6/dd/d34/d3b/d30/db/d1d/d40/f97 [0,4194304] 0 2024-05-15T15:54:15.096 INFO:tasks.workunit.client.0.smithi049.stdout:7/461: readlink d0/d6/da/d3d/l49 0 2024-05-15T15:54:15.097 INFO:tasks.workunit.client.1.smithi144.stdout:4/572: write d6/dd/d2e/f70 [356266,126565] 0 2024-05-15T15:54:15.098 INFO:tasks.workunit.client.0.smithi049.stdout:6/492: mkdir dd/d17/d24/d8a 0 2024-05-15T15:54:15.100 INFO:tasks.workunit.client.1.smithi144.stdout:2/561: truncate d3/d7/d20/d3f/f53 49236 0 2024-05-15T15:54:15.101 INFO:tasks.workunit.client.0.smithi049.stdout:5/515: creat dc/dd/d47/d83/fa5 x:0 0 0 2024-05-15T15:54:15.103 INFO:tasks.workunit.client.0.smithi049.stdout:9/400: creat d0/d8/d9/d45/f8c x:0 0 0 2024-05-15T15:54:15.105 INFO:tasks.workunit.client.1.smithi144.stdout:6/535: stat d7/db/d35/c6d 0 2024-05-15T15:54:15.105 INFO:tasks.workunit.client.1.smithi144.stdout:6/536: fsync d7/d62/fa9 0 2024-05-15T15:54:15.106 INFO:tasks.workunit.client.1.smithi144.stdout:6/537: stat d7/d1e/l2d 0 2024-05-15T15:54:15.110 INFO:tasks.workunit.client.1.smithi144.stdout:8/638: creat d2/d24/d2d/da0/d6e/d71/d73/fd2 x:0 0 0 2024-05-15T15:54:15.111 INFO:tasks.workunit.client.1.smithi144.stdout:8/639: fdatasync d2/fbe 0 2024-05-15T15:54:15.111 INFO:tasks.workunit.client.0.smithi049.stdout:7/462: stat d0/d6/c7 0 2024-05-15T15:54:15.113 INFO:tasks.workunit.client.0.smithi049.stdout:6/493: creat dd/d26/f8b x:0 0 0 2024-05-15T15:54:15.114 INFO:tasks.workunit.client.0.smithi049.stdout:5/516: dwrite dc/f9a [0,4194304] 0 2024-05-15T15:54:15.115 INFO:tasks.workunit.client.1.smithi144.stdout:4/573: unlink d6/d42/c5b 0 2024-05-15T15:54:15.117 INFO:tasks.workunit.client.0.smithi049.stdout:9/401: dwrite d0/d8/d9/d45/f8c [0,4194304] 0 2024-05-15T15:54:15.118 INFO:tasks.workunit.client.0.smithi049.stdout:6/494: write dd/d17/d64/f6a [4552798,4391] 0 2024-05-15T15:54:15.119 INFO:tasks.workunit.client.0.smithi049.stdout:9/402: readlink d0/d8/d21/l3d 0 2024-05-15T15:54:15.122 INFO:tasks.workunit.client.1.smithi144.stdout:0/580: sync 2024-05-15T15:54:15.124 INFO:tasks.workunit.client.1.smithi144.stdout:1/576: creat d7/d1d/d33/d41/d22/dbd/fc0 x:0 0 0 2024-05-15T15:54:15.124 INFO:tasks.workunit.client.0.smithi049.stdout:3/562: mknod d12/d30/d42/caa 0 2024-05-15T15:54:15.126 INFO:tasks.workunit.client.1.smithi144.stdout:9/611: creat d0/d8/d5e/d44/d16/dcd/fd3 x:0 0 0 2024-05-15T15:54:15.126 INFO:tasks.workunit.client.1.smithi144.stdout:9/612: stat d0/d8/d5e/d44/f41 0 2024-05-15T15:54:15.127 INFO:tasks.workunit.client.1.smithi144.stdout:9/613: readlink d0/d8/d1b/d62/d7d/laa 0 2024-05-15T15:54:15.128 INFO:tasks.workunit.client.0.smithi049.stdout:6/495: dwrite dd/d17/d64/f6a [0,4194304] 0 2024-05-15T15:54:15.130 INFO:tasks.workunit.client.1.smithi144.stdout:2/562: dread d3/d7/d20/d89/d6e/f8f [0,4194304] 0 2024-05-15T15:54:15.131 INFO:tasks.workunit.client.1.smithi144.stdout:2/563: stat d3/d7/d20/d22/d52/d64 0 2024-05-15T15:54:15.133 INFO:tasks.workunit.client.1.smithi144.stdout:9/614: dwrite d0/d8/d5e/d44/d10/f2c [0,4194304] 0 2024-05-15T15:54:15.136 INFO:tasks.workunit.client.0.smithi049.stdout:7/463: symlink d0/d6/da/d3d/d39/l8c 0 2024-05-15T15:54:15.136 INFO:tasks.workunit.client.0.smithi049.stdout:4/529: dwrite d18/d1e/d2a/d36/d5d/f67 [0,4194304] 0 2024-05-15T15:54:15.138 INFO:tasks.workunit.client.1.smithi144.stdout:9/615: dread d0/d8/d5e/d44/d77/f89 [0,4194304] 0 2024-05-15T15:54:15.146 INFO:tasks.workunit.client.1.smithi144.stdout:3/594: sync 2024-05-15T15:54:15.147 INFO:tasks.workunit.client.1.smithi144.stdout:0/581: mkdir d4/d2d/dcf 0 2024-05-15T15:54:15.149 INFO:tasks.workunit.client.0.smithi049.stdout:0/532: dwrite dc/d13/d14/d2f/f51 [4194304,4194304] 0 2024-05-15T15:54:15.149 INFO:tasks.workunit.client.1.smithi144.stdout:0/582: write d4/d6/d4a/d74/f99 [344465,58259] 0 2024-05-15T15:54:15.151 INFO:tasks.workunit.client.0.smithi049.stdout:5/517: dwrite dc/dd/f88 [0,4194304] 0 2024-05-15T15:54:15.151 INFO:tasks.workunit.client.0.smithi049.stdout:9/403: creat d0/d25/d5d/f8d x:0 0 0 2024-05-15T15:54:15.155 INFO:tasks.workunit.client.1.smithi144.stdout:1/577: creat d7/d1d/dac/fc1 x:0 0 0 2024-05-15T15:54:15.160 INFO:tasks.workunit.client.0.smithi049.stdout:6/496: mkdir dd/d17/d64/d8c 0 2024-05-15T15:54:15.162 INFO:tasks.workunit.client.1.smithi144.stdout:5/609: creat d0/dc/d2f/d34/fcc x:0 0 0 2024-05-15T15:54:15.164 INFO:tasks.workunit.client.1.smithi144.stdout:2/564: creat d3/d7/d20/d37/d76/db5/fbd x:0 0 0 2024-05-15T15:54:15.169 INFO:tasks.workunit.client.0.smithi049.stdout:1/696: getdents d0/d4/dd/de/d1e/d2e 0 2024-05-15T15:54:15.172 INFO:tasks.workunit.client.0.smithi049.stdout:1/697: chown d0/d4/dd/de/d1e/d2e/dad/d65/d72/l86 244928170 1 2024-05-15T15:54:15.173 INFO:tasks.workunit.client.0.smithi049.stdout:5/518: mkdir dc/dd/d47/da6 0 2024-05-15T15:54:15.174 INFO:tasks.workunit.client.0.smithi049.stdout:0/533: creat dc/d13/d56/fa0 x:0 0 0 2024-05-15T15:54:15.180 INFO:tasks.workunit.client.0.smithi049.stdout:9/404: fsync d0/d25/d5d/f8d 0 2024-05-15T15:54:15.181 INFO:tasks.workunit.client.0.smithi049.stdout:1/698: rename d0/d4/dd/de/d1e/d2e/d3e/d90/db3/lbc to d0/d4/dd/de/d4d/d8b/dba/lf7 0 2024-05-15T15:54:15.202 INFO:tasks.workunit.client.1.smithi144.stdout:7/588: rename d4/fc to d4/d41/fc4 0 2024-05-15T15:54:15.203 INFO:tasks.workunit.client.1.smithi144.stdout:7/589: read d4/df/f22 [477772,58728] 0 2024-05-15T15:54:15.204 INFO:tasks.workunit.client.1.smithi144.stdout:4/574: chown d6/d42/d46/d4b/c63 1585 1 2024-05-15T15:54:15.204 INFO:tasks.workunit.client.1.smithi144.stdout:1/578: write d7/d1d/d33/d41/d22/d45/fb3 [2715256,6946] 0 2024-05-15T15:54:15.208 INFO:tasks.workunit.client.1.smithi144.stdout:6/538: rmdir d7/db/d35/d84/d81/db4 0 2024-05-15T15:54:15.208 INFO:tasks.workunit.client.1.smithi144.stdout:1/579: write d7/d1d/d33/d29/f64 [396547,86075] 0 2024-05-15T15:54:15.210 INFO:tasks.workunit.client.1.smithi144.stdout:0/583: dwrite d4/df/d1f/d24/d2b/d55/f72 [0,4194304] 0 2024-05-15T15:54:15.211 INFO:tasks.workunit.client.1.smithi144.stdout:3/595: rename d0/d1/d17/f48 to d0/d1/d17/d39/d83/fcc 0 2024-05-15T15:54:15.212 INFO:tasks.workunit.client.1.smithi144.stdout:7/590: mkdir d4/d3c/d76/d9d/dc5 0 2024-05-15T15:54:15.213 INFO:tasks.workunit.client.1.smithi144.stdout:4/575: dread d6/dd/d34/d3b/d30/db/d1d/d40/d57/f5e [0,4194304] 0 2024-05-15T15:54:15.214 INFO:tasks.workunit.client.1.smithi144.stdout:7/591: chown d4/d3c/d76/f9e 829090 1 2024-05-15T15:54:15.217 INFO:tasks.workunit.client.1.smithi144.stdout:8/640: getdents d2/de/d36 0 2024-05-15T15:54:15.218 INFO:tasks.workunit.client.1.smithi144.stdout:6/539: symlink d7/db/d35/d84/d81/lbf 0 2024-05-15T15:54:15.220 INFO:tasks.workunit.client.1.smithi144.stdout:3/596: dwrite d0/f69 [0,4194304] 0 2024-05-15T15:54:15.221 INFO:tasks.workunit.client.0.smithi049.stdout:3/563: write d12/d30/d42/f8e [5889429,68739] 0 2024-05-15T15:54:15.222 INFO:tasks.workunit.client.1.smithi144.stdout:4/576: dwrite d6/f95 [0,4194304] 0 2024-05-15T15:54:15.222 INFO:tasks.workunit.client.0.smithi049.stdout:5/519: write dc/f1f [2648884,26618] 0 2024-05-15T15:54:15.224 INFO:tasks.workunit.client.0.smithi049.stdout:5/520: chown dc/f64 7546 1 2024-05-15T15:54:15.226 INFO:tasks.workunit.client.1.smithi144.stdout:0/584: write d4/d6/d3a/d40/f7d [4664281,46608] 0 2024-05-15T15:54:15.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.225+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 --> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa200738d0 con 0x7faa0806d6f0 2024-05-15T15:54:15.228 INFO:tasks.workunit.client.0.smithi049.stdout:1/699: mknod d0/d4/dd/de/d1e/d2e/d3e/cf8 0 2024-05-15T15:54:15.228 INFO:tasks.workunit.client.1.smithi144.stdout:0/585: fsync d4/d2d/fce 0 2024-05-15T15:54:15.229 INFO:tasks.workunit.client.0.smithi049.stdout:7/464: dwrite d0/d6/d1d/f59 [0,4194304] 0 2024-05-15T15:54:15.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.228+0000 7faa2485b700 1 -- 172.21.15.49:0/1480696721 <== mgr.24413 v2:172.21.15.144:6828/177983287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7faa200738d0 con 0x7faa0806d6f0 2024-05-15T15:54:15.233 INFO:tasks.workunit.client.0.smithi049.stdout:4/530: getdents d18/d1e/d2a/d36 0 2024-05-15T15:54:15.233 INFO:tasks.workunit.client.0.smithi049.stdout:6/497: dread dd/f32 [0,4194304] 0 2024-05-15T15:54:15.234 INFO:tasks.workunit.client.0.smithi049.stdout:9/405: mkdir d0/d8e 0 2024-05-15T15:54:15.234 INFO:tasks.workunit.client.0.smithi049.stdout:6/498: chown dd/d17 616093 1 2024-05-15T15:54:15.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.232+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7faa0806d6f0 msgr2=0x7faa0806fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:15.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.232+0000 7faa2585d700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7faa0806d6f0 0x7faa0806fbb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa14005950 tx=0x7faa140058e0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.232+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 msgr2=0x7faa20074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:15.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.232+0000 7faa2585d700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 0x7faa20074ca0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa10009fd0 tx=0x7faa10005e70 comp rx=0 tx=0).stop 2024-05-15T15:54:15.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.232+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 shutdown_connections 2024-05-15T15:54:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.232+0000 7faa2585d700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7faa0806d6f0 0x7faa0806fbb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.233+0000 7faa2585d700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa20100cd0 0x7faa20074ca0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.233+0000 7faa2585d700 1 --2- 172.21.15.49:0/1480696721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa20101ed0 0x7faa20073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.233+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 >> 172.21.15.49:0/1480696721 conn(0x7faa200fc2b0 msgr2=0x7faa20105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.233+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 shutdown_connections 2024-05-15T15:54:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.233+0000 7faa2585d700 1 -- 172.21.15.49:0/1480696721 wait complete. 2024-05-15T15:54:15.245 INFO:tasks.workunit.client.1.smithi144.stdout:6/540: dread d7/db/d11/d3c/f77 [0,4194304] 0 2024-05-15T15:54:15.252 INFO:tasks.workunit.client.1.smithi144.stdout:7/592: unlink d4/df/d25/d40/cb3 0 2024-05-15T15:54:15.253 INFO:tasks.workunit.client.0.smithi049.stdout:5/521: stat dc/dd/d7b/c81 0 2024-05-15T15:54:15.254 INFO:tasks.workunit.client.0.smithi049.stdout:5/522: chown dc/dd/d47/f4d 3 1 2024-05-15T15:54:15.261 INFO:tasks.workunit.client.1.smithi144.stdout:2/565: getdents d3/d10 0 2024-05-15T15:54:15.263 INFO:tasks.workunit.client.1.smithi144.stdout:4/577: dread d6/d49/f5c [0,4194304] 0 2024-05-15T15:54:15.265 INFO:tasks.workunit.client.1.smithi144.stdout:6/541: dread d7/db/f4a [0,4194304] 0 2024-05-15T15:54:15.266 INFO:tasks.workunit.client.0.smithi049.stdout:6/499: creat dd/d17/f8d x:0 0 0 2024-05-15T15:54:15.270 INFO:tasks.workunit.client.1.smithi144.stdout:2/566: dread d3/d7/d20/d89/d6e/f8f [0,4194304] 0 2024-05-15T15:54:15.270 INFO:tasks.workunit.client.0.smithi049.stdout:5/523: chown dc/dd/d7b/f93 3639744 1 2024-05-15T15:54:15.270 INFO:tasks.workunit.client.1.smithi144.stdout:5/610: dwrite d0/dc/d2f/d34/d3e/f48 [0,4194304] 0 2024-05-15T15:54:15.271 INFO:tasks.workunit.client.0.smithi049.stdout:1/700: mkdir d0/d4/dd/de/d1e/d2e/dad/df9 0 2024-05-15T15:54:15.272 INFO:tasks.workunit.client.1.smithi144.stdout:1/580: write d7/d1d/d33/d41/d22/f90 [2856095,31960] 0 2024-05-15T15:54:15.272 INFO:tasks.workunit.client.0.smithi049.stdout:1/701: chown d0/d4/dd/de/d36/f4c 181410 1 2024-05-15T15:54:15.275 INFO:tasks.workunit.client.1.smithi144.stdout:9/616: dwrite d0/d8/d5e/d44/d10/d9c/f3f [0,4194304] 0 2024-05-15T15:54:15.276 INFO:tasks.workunit.client.1.smithi144.stdout:1/581: write d7/d31/f73 [1972006,104948] 0 2024-05-15T15:54:15.277 INFO:tasks.workunit.client.1.smithi144.stdout:5/611: dread d0/dc/d24/d57/f77 [0,4194304] 0 2024-05-15T15:54:15.283 INFO:tasks.workunit.client.0.smithi049.stdout:3/564: rename d12/d30/c7e to d12/d14/cab 0 2024-05-15T15:54:15.283 INFO:tasks.workunit.client.1.smithi144.stdout:6/542: creat d7/db/d9d/d97/fc0 x:0 0 0 2024-05-15T15:54:15.283 INFO:tasks.workunit.client.1.smithi144.stdout:3/597: dwrite d0/f2c [0,4194304] 0 2024-05-15T15:54:15.288 INFO:tasks.workunit.client.0.smithi049.stdout:5/524: mkdir dc/dd/d47/d83/da7 0 2024-05-15T15:54:15.291 INFO:tasks.workunit.client.1.smithi144.stdout:2/567: creat d3/d7/d20/d22/d52/d64/d97/fbe x:0 0 0 2024-05-15T15:54:15.292 INFO:tasks.workunit.client.1.smithi144.stdout:0/586: truncate d4/d2d/d48/d5b/f75 731694 0 2024-05-15T15:54:15.293 INFO:tasks.workunit.client.1.smithi144.stdout:4/578: dread f0 [0,4194304] 0 2024-05-15T15:54:15.294 INFO:tasks.workunit.client.0.smithi049.stdout:7/465: link d0/d2f/d5c/f6e d0/d6/da/f8d 0 2024-05-15T15:54:15.295 INFO:tasks.workunit.client.0.smithi049.stdout:7/466: fdatasync d0/d34/f46 0 2024-05-15T15:54:15.295 INFO:tasks.workunit.client.0.smithi049.stdout:6/500: write dd/f4c [1205860,34484] 0 2024-05-15T15:54:15.295 INFO:tasks.workunit.client.1.smithi144.stdout:0/587: dread d4/f1c [4194304,4194304] 0 2024-05-15T15:54:15.296 INFO:tasks.workunit.client.0.smithi049.stdout:4/531: creat d18/d1e/d2a/fb0 x:0 0 0 2024-05-15T15:54:15.296 INFO:tasks.workunit.client.1.smithi144.stdout:0/588: readlink d4/d6/d3a/d4c/l57 0 2024-05-15T15:54:15.300 INFO:tasks.workunit.client.1.smithi144.stdout:1/582: symlink d7/d1d/d33/d41/d78/da3/lc2 0 2024-05-15T15:54:15.301 INFO:tasks.workunit.client.1.smithi144.stdout:4/579: dwrite d6/dd/d34/f5f [0,4194304] 0 2024-05-15T15:54:15.302 INFO:tasks.workunit.client.1.smithi144.stdout:9/617: dwrite d0/f4 [0,4194304] 0 2024-05-15T15:54:15.302 INFO:tasks.workunit.client.1.smithi144.stdout:1/583: dread - d7/d1d/f8b zero size 2024-05-15T15:54:15.303 INFO:tasks.workunit.client.1.smithi144.stdout:5/612: unlink d0/dc/d24/d8e/d93/fa7 0 2024-05-15T15:54:15.306 INFO:tasks.workunit.client.0.smithi049.stdout:1/702: creat d0/d4/dd/de/d1e/d2e/dad/d65/df3/ffa x:0 0 0 2024-05-15T15:54:15.307 INFO:tasks.workunit.client.0.smithi049.stdout:1/703: readlink d0/d4/dd/de/d36/lea 0 2024-05-15T15:54:15.307 INFO:tasks.workunit.client.1.smithi144.stdout:6/543: mknod d7/db/d3a/cc1 0 2024-05-15T15:54:15.308 INFO:tasks.workunit.client.0.smithi049.stdout:1/704: dread - d0/d4/dd/de/d48/ff6 zero size 2024-05-15T15:54:15.309 INFO:tasks.workunit.client.0.smithi049.stdout:5/525: unlink dc/dd/d47/la1 0 2024-05-15T15:54:15.313 INFO:tasks.workunit.client.1.smithi144.stdout:8/641: sync 2024-05-15T15:54:15.316 INFO:tasks.workunit.client.1.smithi144.stdout:2/568: dwrite d3/d10/f2e [0,4194304] 0 2024-05-15T15:54:15.317 INFO:tasks.workunit.client.1.smithi144.stdout:7/593: rename d4/d41/l59 to d4/d41/lc6 0 2024-05-15T15:54:15.318 INFO:tasks.workunit.client.1.smithi144.stdout:7/594: fdatasync d4/df/f32 0 2024-05-15T15:54:15.319 INFO:tasks.workunit.client.0.smithi049.stdout:4/532: creat d18/d1e/d2a/d50/d54/fb1 x:0 0 0 2024-05-15T15:54:15.322 INFO:tasks.workunit.client.1.smithi144.stdout:0/589: creat d4/df/d1f/d24/d85/da1/fd0 x:0 0 0 2024-05-15T15:54:15.323 INFO:tasks.workunit.client.1.smithi144.stdout:0/590: chown d4/d6/c38 2116814 1 2024-05-15T15:54:15.324 INFO:tasks.workunit.client.0.smithi049.stdout:9/406: getdents d0/d8/d42/d75 0 2024-05-15T15:54:15.328 INFO:tasks.workunit.client.0.smithi049.stdout:3/565: write d12/d14/d32/f62 [403296,28226] 0 2024-05-15T15:54:15.329 INFO:tasks.workunit.client.1.smithi144.stdout:9/618: mkdir d0/d8/d5e/dd4 0 2024-05-15T15:54:15.334 INFO:tasks.workunit.client.0.smithi049.stdout:7/467: link d0/d6/da/d3d/d39/f5b d0/d2f/f8e 0 2024-05-15T15:54:15.335 INFO:tasks.workunit.client.0.smithi049.stdout:4/533: unlink d18/d1e/d2a/d3b/c65 0 2024-05-15T15:54:15.337 INFO:tasks.workunit.client.1.smithi144.stdout:9/619: dwrite d0/f4 [0,4194304] 0 2024-05-15T15:54:15.337 INFO:tasks.workunit.client.1.smithi144.stdout:3/598: dwrite d0/d11/f14 [4194304,4194304] 0 2024-05-15T15:54:15.338 INFO:tasks.workunit.client.0.smithi049.stdout:1/705: symlink d0/d4/lfb 0 2024-05-15T15:54:15.339 INFO:tasks.workunit.client.0.smithi049.stdout:1/706: chown d0/d4/dd/de/d1e/f4f 12 1 2024-05-15T15:54:15.340 INFO:tasks.workunit.client.1.smithi144.stdout:8/642: creat d2/d1a/fd3 x:0 0 0 2024-05-15T15:54:15.342 INFO:tasks.workunit.client.0.smithi049.stdout:3/566: chown d12/d67/ca5 76 1 2024-05-15T15:54:15.349 INFO:tasks.workunit.client.0.smithi049.stdout:6/501: creat dd/f8e x:0 0 0 2024-05-15T15:54:15.350 INFO:tasks.workunit.client.0.smithi049.stdout:6/502: stat dd/d17/d64/d8c 0 2024-05-15T15:54:15.350 INFO:tasks.workunit.client.0.smithi049.stdout:1/707: dread d0/f5b [0,4194304] 0 2024-05-15T15:54:15.351 INFO:tasks.workunit.client.0.smithi049.stdout:6/503: write dd/d17/d24/f27 [1542454,48959] 0 2024-05-15T15:54:15.352 INFO:tasks.workunit.client.0.smithi049.stdout:1/708: stat d0/d4/dd/de/d1e/d2e/d3e/d90/db3/lf0 0 2024-05-15T15:54:15.353 INFO:tasks.workunit.client.0.smithi049.stdout:7/468: creat d0/d2f/f8f x:0 0 0 2024-05-15T15:54:15.354 INFO:tasks.workunit.client.0.smithi049.stdout:1/709: write d0/d4/dd/fcc [676516,29041] 0 2024-05-15T15:54:15.355 INFO:tasks.workunit.client.0.smithi049.stdout:9/407: symlink d0/d8/d9/d52/d5e/d5f/d88/l8f 0 2024-05-15T15:54:15.355 INFO:tasks.workunit.client.1.smithi144.stdout:4/580: dwrite d6/dd/d34/d3b/d30/d59/f74 [0,4194304] 0 2024-05-15T15:54:15.357 INFO:tasks.workunit.client.0.smithi049.stdout:9/408: read d0/d8/d9/d22/f2e [2663604,11670] 0 2024-05-15T15:54:15.358 INFO:tasks.workunit.client.1.smithi144.stdout:4/581: chown d6/d42/d46/d4b/c63 130317992 1 2024-05-15T15:54:15.358 INFO:tasks.workunit.client.1.smithi144.stdout:6/544: symlink d7/db/d35/d84/dbe/lc2 0 2024-05-15T15:54:15.359 INFO:tasks.workunit.client.1.smithi144.stdout:5/613: dwrite d0/dc/d1f/f39 [0,4194304] 0 2024-05-15T15:54:15.360 INFO:tasks.workunit.client.1.smithi144.stdout:4/582: read d6/dd/d34/d3b/d30/db/d1d/f41 [4046137,79023] 0 2024-05-15T15:54:15.362 INFO:tasks.workunit.client.1.smithi144.stdout:3/599: fsync d0/d11/f14 0 2024-05-15T15:54:15.366 INFO:tasks.workunit.client.1.smithi144.stdout:3/600: dread d0/f2c [0,4194304] 0 2024-05-15T15:54:15.367 INFO:tasks.workunit.client.1.smithi144.stdout:3/601: write d0/d1/d6c/fc1 [307206,103130] 0 2024-05-15T15:54:15.374 INFO:tasks.workunit.client.1.smithi144.stdout:8/643: rename d2/de/d3f/f5f to d2/d24/fd4 0 2024-05-15T15:54:15.376 INFO:tasks.workunit.client.1.smithi144.stdout:8/644: truncate d2/d24/d2d/d30/f81 1509286 0 2024-05-15T15:54:15.376 INFO:tasks.workunit.client.1.smithi144.stdout:2/569: write d3/d10/f35 [1008660,46822] 0 2024-05-15T15:54:15.378 INFO:tasks.workunit.client.0.smithi049.stdout:5/526: write dc/dd/d10/f2a [896066,103298] 0 2024-05-15T15:54:15.379 INFO:tasks.workunit.client.0.smithi049.stdout:9/409: mknod d0/d8/d9/d52/d5e/d5f/c90 0 2024-05-15T15:54:15.379 INFO:tasks.workunit.client.0.smithi049.stdout:5/527: chown dc/dd/d39/f6e 1036 1 2024-05-15T15:54:15.380 INFO:tasks.workunit.client.0.smithi049.stdout:3/567: dread d12/d14/d32/fa7 [0,4194304] 0 2024-05-15T15:54:15.386 INFO:tasks.workunit.client.0.smithi049.stdout:4/534: creat d18/d21/d2b/fb2 x:0 0 0 2024-05-15T15:54:15.387 INFO:tasks.workunit.client.0.smithi049.stdout:6/504: write dd/d17/d24/d2b/f63 [1095442,26969] 0 2024-05-15T15:54:15.387 INFO:tasks.workunit.client.0.smithi049.stdout:1/710: mknod d0/d4/dd/cfc 0 2024-05-15T15:54:15.391 INFO:tasks.workunit.client.1.smithi144.stdout:2/570: dread d3/f9 [0,4194304] 0 2024-05-15T15:54:15.391 INFO:tasks.workunit.client.0.smithi049.stdout:6/505: truncate dd/d17/d24/d2b/d36/d50/d54/f5c 531056 0 2024-05-15T15:54:15.392 INFO:tasks.workunit.client.0.smithi049.stdout:6/506: chown dd/d17/f21 218225827 1 2024-05-15T15:54:15.394 INFO:tasks.workunit.client.0.smithi049.stdout:9/410: creat d0/d8/d42/f91 x:0 0 0 2024-05-15T15:54:15.394 INFO:tasks.workunit.client.1.smithi144.stdout:5/614: symlink d0/dc/d24/d8e/d93/lcd 0 2024-05-15T15:54:15.395 INFO:tasks.workunit.client.0.smithi049.stdout:9/411: fdatasync d0/d8/d42/f62 0 2024-05-15T15:54:15.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.396+0000 7f1f8801a700 1 -- 172.21.15.49:0/3390759499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f80100d60 msgr2=0x7f1f801011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:15.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.396+0000 7f1f8801a700 1 --2- 172.21.15.49:0/3390759499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f80100d60 0x7f1f801011c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009910 tx=0x7f1f70009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:15.399 INFO:tasks.workunit.client.0.smithi049.stdout:3/568: mkdir d12/d30/d63/d68/d70/d75/dac 0 2024-05-15T15:54:15.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.397+0000 7f1f8801a700 1 -- 172.21.15.49:0/3390759499 shutdown_connections 2024-05-15T15:54:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.397+0000 7f1f8801a700 1 --2- 172.21.15.49:0/3390759499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f80100d60 0x7f1f801011c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.397+0000 7f1f8801a700 1 --2- 172.21.15.49:0/3390759499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800ffb60 0x7f1f800fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.397+0000 7f1f8801a700 1 -- 172.21.15.49:0/3390759499 >> 172.21.15.49:0/3390759499 conn(0x7f1f800fb0e0 msgr2=0x7f1f800fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.397+0000 7f1f8801a700 1 -- 172.21.15.49:0/3390759499 shutdown_connections 2024-05-15T15:54:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.397+0000 7f1f8801a700 1 -- 172.21.15.49:0/3390759499 wait complete. 2024-05-15T15:54:15.400 INFO:tasks.workunit.client.0.smithi049.stdout:6/507: dwrite dd/d17/d64/f6a [0,4194304] 0 2024-05-15T15:54:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.398+0000 7f1f8801a700 1 Processor -- start 2024-05-15T15:54:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.398+0000 7f1f8801a700 1 -- start start 2024-05-15T15:54:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.398+0000 7f1f8801a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800ffb60 0x7f1f8019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.398+0000 7f1f8801a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f80100d60 0x7f1f8019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.398+0000 7f1f8801a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8019d5e0 con 0x7f1f80100d60 2024-05-15T15:54:15.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.398+0000 7f1f8801a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8019d720 con 0x7f1f800ffb60 2024-05-15T15:54:15.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.399+0000 7f1f85db6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800ffb60 0x7f1f8019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:15.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.399+0000 7f1f85db6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800ffb60 0x7f1f8019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:57530/0 (socket says 172.21.15.49:57530) 2024-05-15T15:54:15.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.399+0000 7f1f85db6700 1 -- 172.21.15.49:0/3041064269 learned_addr learned my addr 172.21.15.49:0/3041064269 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:15.404 INFO:tasks.workunit.client.1.smithi144.stdout:5/615: dwrite d0/dc/d2f/d34/d63/d6e/dbf/f9f [0,4194304] 0 2024-05-15T15:54:15.404 INFO:tasks.workunit.client.1.smithi144.stdout:5/616: truncate d0/dc/d24/d28/d2d/d59/fca 56331 0 2024-05-15T15:54:15.404 INFO:tasks.workunit.client.1.smithi144.stdout:5/617: stat d0/dc/f1b 0 2024-05-15T15:54:15.404 INFO:tasks.workunit.client.1.smithi144.stdout:9/620: mkdir d0/d8/d5e/d44/d10/d19/dd5 0 2024-05-15T15:54:15.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.399+0000 7f1f855b5700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f80100d60 0x7f1f8019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:15.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.399+0000 7f1f85db6700 1 -- 172.21.15.49:0/3041064269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f80100d60 msgr2=0x7f1f8019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:15.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.399+0000 7f1f85db6700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f80100d60 0x7f1f8019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.399+0000 7f1f85db6700 1 -- 172.21.15.49:0/3041064269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f700095f0 con 0x7f1f800ffb60 2024-05-15T15:54:15.404 INFO:tasks.workunit.client.0.smithi049.stdout:7/469: creat d0/f90 x:0 0 0 2024-05-15T15:54:15.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.400+0000 7f1f85db6700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800ffb60 0x7f1f8019ca80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e8f0 tx=0x7f1f7c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:15.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.400+0000 7f1f76ffd700 1 -- 172.21.15.49:0/3041064269 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f7c009790 con 0x7f1f800ffb60 2024-05-15T15:54:15.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.400+0000 7f1f76ffd700 1 -- 172.21.15.49:0/3041064269 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f7c0044e0 con 0x7f1f800ffb60 2024-05-15T15:54:15.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.400+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f801a2130 con 0x7f1f800ffb60 2024-05-15T15:54:15.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.400+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f801a2700 con 0x7f1f800ffb60 2024-05-15T15:54:15.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.401+0000 7f1f76ffd700 1 -- 172.21.15.49:0/3041064269 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f7c003e00 con 0x7f1f800ffb60 2024-05-15T15:54:15.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.402+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f64005320 con 0x7f1f800ffb60 2024-05-15T15:54:15.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.403+0000 7f1f76ffd700 1 -- 172.21.15.49:0/3041064269 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f1f7c010480 con 0x7f1f800ffb60 2024-05-15T15:54:15.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.404+0000 7f1f76ffd700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f1f6c06d6f0 0x7f1f6c06fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:15.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.404+0000 7f1f76ffd700 1 -- 172.21.15.49:0/3041064269 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1f7c014070 con 0x7f1f800ffb60 2024-05-15T15:54:15.408 INFO:tasks.workunit.client.0.smithi049.stdout:4/535: rmdir d18/d1e 39 2024-05-15T15:54:15.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.405+0000 7f1f855b5700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f1f6c06d6f0 0x7f1f6c06fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:15.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.406+0000 7f1f855b5700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f1f6c06d6f0 0x7f1f6c06fbb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009fd0 tx=0x7f1f70005fb0 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:15.412 INFO:tasks.workunit.client.0.smithi049.stdout:1/711: rename d0/d4/dd/de/d1e/d16/d6e/dbf/ldb to d0/d4/dd/de/d1e/d16/d8a/lfd 0 2024-05-15T15:54:15.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.412+0000 7f1f76ffd700 1 -- 172.21.15.49:0/3041064269 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f1f7c05a750 con 0x7f1f800ffb60 2024-05-15T15:54:15.416 INFO:tasks.workunit.client.1.smithi144.stdout:8/645: mkdir d2/d24/d2d/da0/dd5 0 2024-05-15T15:54:15.417 INFO:tasks.workunit.client.1.smithi144.stdout:8/646: write d2/d3a/fce [539826,77004] 0 2024-05-15T15:54:15.418 INFO:tasks.workunit.client.1.smithi144.stdout:8/647: chown d2/de/d3f/f9f 17422 1 2024-05-15T15:54:15.420 INFO:tasks.workunit.client.0.smithi049.stdout:5/528: link dc/d4a/f91 dc/dd/d7b/d8d/fa8 0 2024-05-15T15:54:15.422 INFO:tasks.workunit.client.0.smithi049.stdout:9/412: mknod d0/d8/d9/d52/d5e/d5f/d88/c92 0 2024-05-15T15:54:15.423 INFO:tasks.workunit.client.0.smithi049.stdout:9/413: write d0/d8/d12/f6e [9198,107104] 0 2024-05-15T15:54:15.424 INFO:tasks.workunit.client.1.smithi144.stdout:1/584: getdents d7 0 2024-05-15T15:54:15.426 INFO:tasks.workunit.client.0.smithi049.stdout:3/569: creat d12/d30/d42/da6/fad x:0 0 0 2024-05-15T15:54:15.429 INFO:tasks.workunit.client.0.smithi049.stdout:6/508: creat dd/d17/d24/d2b/d36/d50/d54/f8f x:0 0 0 2024-05-15T15:54:15.435 INFO:tasks.workunit.client.0.smithi049.stdout:4/536: chown d18/d1e/d2a/d36/f8f 4925599 1 2024-05-15T15:54:15.436 INFO:tasks.workunit.client.0.smithi049.stdout:1/712: truncate d0/d4/dd/de/d1e/db7/fd7 2922194 0 2024-05-15T15:54:15.436 INFO:tasks.workunit.client.0.smithi049.stdout:4/537: write d18/d1e/d63/d99/fa5 [3105083,127109] 0 2024-05-15T15:54:15.438 INFO:tasks.workunit.client.0.smithi049.stdout:4/538: chown d18/l1d 31003565 1 2024-05-15T15:54:15.439 INFO:tasks.workunit.client.0.smithi049.stdout:4/539: chown d18/f22 1 1 2024-05-15T15:54:15.440 INFO:tasks.workunit.client.0.smithi049.stdout:5/529: mkdir dc/d4a/d31/da9 0 2024-05-15T15:54:15.442 INFO:tasks.workunit.client.0.smithi049.stdout:9/414: creat d0/d8/d1b/d71/d86/f93 x:0 0 0 2024-05-15T15:54:15.443 INFO:tasks.workunit.client.1.smithi144.stdout:4/583: creat d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/fb7 x:0 0 0 2024-05-15T15:54:15.444 INFO:tasks.workunit.client.0.smithi049.stdout:7/470: dwrite d0/d2f/f51 [0,4194304] 0 2024-05-15T15:54:15.445 INFO:tasks.workunit.client.0.smithi049.stdout:7/471: fsync d0/d6/d23/d4a/f75 0 2024-05-15T15:54:15.447 INFO:tasks.workunit.client.0.smithi049.stdout:3/570: fdatasync d12/f22 0 2024-05-15T15:54:15.447 INFO:tasks.workunit.client.0.smithi049.stdout:3/571: chown d12/d30/d42/f46 3411 1 2024-05-15T15:54:15.448 INFO:tasks.workunit.client.1.smithi144.stdout:5/618: creat d0/dc/d24/d8e/d93/d6b/d88/fce x:0 0 0 2024-05-15T15:54:15.454 INFO:tasks.workunit.client.0.smithi049.stdout:1/713: symlink d0/d4/dd/de/d1e/d2e/d3e/d90/db3/lfe 0 2024-05-15T15:54:15.455 INFO:tasks.workunit.client.1.smithi144.stdout:7/595: getdents d4/d3c/d76/d9d/dac 0 2024-05-15T15:54:15.456 INFO:tasks.workunit.client.0.smithi049.stdout:4/540: creat d18/d1e/d2a/d36/d86/fb3 x:0 0 0 2024-05-15T15:54:15.458 INFO:tasks.workunit.client.0.smithi049.stdout:9/415: creat d0/d8/d1b/f94 x:0 0 0 2024-05-15T15:54:15.459 INFO:tasks.workunit.client.1.smithi144.stdout:0/591: getdents d4/df/d1f 0 2024-05-15T15:54:15.459 INFO:tasks.workunit.client.0.smithi049.stdout:1/714: dread d0/d4/dd/de/d4d/d8b/fc2 [0,4194304] 0 2024-05-15T15:54:15.460 INFO:tasks.workunit.client.0.smithi049.stdout:5/530: truncate dc/d4a/d42/f53 910065 0 2024-05-15T15:54:15.461 INFO:tasks.workunit.client.1.smithi144.stdout:1/585: truncate d7/d1d/d33/d1e/f25 995658 0 2024-05-15T15:54:15.462 INFO:tasks.workunit.client.1.smithi144.stdout:6/545: rmdir d7/db/d3a/db0 0 2024-05-15T15:54:15.464 INFO:tasks.workunit.client.1.smithi144.stdout:1/586: write d7/d1d/d33/d29/d6b/d5e/f95 [2808162,28850] 0 2024-05-15T15:54:15.464 INFO:tasks.workunit.client.1.smithi144.stdout:2/571: mkdir d3/d7/d20/d37/d76/db5/db6/dbf 0 2024-05-15T15:54:15.464 INFO:tasks.workunit.client.1.smithi144.stdout:0/592: dwrite d4/df/d14/f42 [4194304,4194304] 0 2024-05-15T15:54:15.466 INFO:tasks.workunit.client.0.smithi049.stdout:6/509: creat dd/d17/d24/d8a/f90 x:0 0 0 2024-05-15T15:54:15.466 INFO:tasks.workunit.client.0.smithi049.stdout:3/572: dread d12/d30/d42/d6b/f80 [0,4194304] 0 2024-05-15T15:54:15.466 INFO:tasks.workunit.client.1.smithi144.stdout:4/584: read d6/dd/d2d/f54 [224810,5900] 0 2024-05-15T15:54:15.467 INFO:tasks.workunit.client.1.smithi144.stdout:5/619: unlink d0/l21 0 2024-05-15T15:54:15.469 INFO:tasks.workunit.client.1.smithi144.stdout:4/585: read d6/d7c/f8c [280805,78274] 0 2024-05-15T15:54:15.469 INFO:tasks.workunit.client.1.smithi144.stdout:5/620: chown d0/c55 362361 1 2024-05-15T15:54:15.472 INFO:tasks.workunit.client.0.smithi049.stdout:9/416: creat d0/d8/d1b/d40/f95 x:0 0 0 2024-05-15T15:54:15.473 INFO:tasks.workunit.client.0.smithi049.stdout:5/531: rmdir dc/d4a/d31/d5c 39 2024-05-15T15:54:15.475 INFO:tasks.workunit.client.0.smithi049.stdout:1/715: creat d0/d4/dd/de/d1e/d2e/dad/d65/df3/fff x:0 0 0 2024-05-15T15:54:15.476 INFO:tasks.workunit.client.0.smithi049.stdout:1/716: chown d0/d4/dd/de/d48/d5c/da2 446042 1 2024-05-15T15:54:15.483 INFO:tasks.workunit.client.0.smithi049.stdout:3/573: dread d12/d14/f17 [4194304,4194304] 0 2024-05-15T15:54:15.484 INFO:tasks.workunit.client.0.smithi049.stdout:3/574: stat d12/d30/d66 0 2024-05-15T15:54:15.490 INFO:tasks.workunit.client.0.smithi049.stdout:9/417: creat d0/d8/d9/d22/f96 x:0 0 0 2024-05-15T15:54:15.490 INFO:tasks.workunit.client.0.smithi049.stdout:9/418: dread - d0/f54 zero size 2024-05-15T15:54:15.491 INFO:tasks.workunit.client.0.smithi049.stdout:4/541: write d18/d1e/d2a/d36/f4a [3523121,86948] 0 2024-05-15T15:54:15.492 INFO:tasks.workunit.client.0.smithi049.stdout:5/532: rmdir dc/d4a/d42 39 2024-05-15T15:54:15.495 INFO:tasks.workunit.client.0.smithi049.stdout:7/472: creat d0/d6/d23/f91 x:0 0 0 2024-05-15T15:54:15.496 INFO:tasks.workunit.client.0.smithi049.stdout:1/717: creat d0/d4/dd/de/d1e/d16/d6e/f100 x:0 0 0 2024-05-15T15:54:15.499 INFO:tasks.workunit.client.0.smithi049.stdout:9/419: mknod d0/d8/d1b/d71/c97 0 2024-05-15T15:54:15.500 INFO:tasks.workunit.client.1.smithi144.stdout:7/596: symlink d4/d3c/d76/d9d/lc7 0 2024-05-15T15:54:15.500 INFO:tasks.workunit.client.0.smithi049.stdout:4/542: dread - d18/d85/f8b zero size 2024-05-15T15:54:15.501 INFO:tasks.workunit.client.0.smithi049.stdout:5/533: truncate dc/dd/d7b/f93 4465055 0 2024-05-15T15:54:15.502 INFO:tasks.workunit.client.1.smithi144.stdout:9/621: creat d0/d8/d5e/d44/fd6 x:0 0 0 2024-05-15T15:54:15.502 INFO:tasks.workunit.client.1.smithi144.stdout:3/602: getdents d0/d1/d56/d98 0 2024-05-15T15:54:15.503 INFO:tasks.workunit.client.1.smithi144.stdout:5/621: symlink d0/d7e/d9b/lcf 0 2024-05-15T15:54:15.503 INFO:tasks.workunit.client.0.smithi049.stdout:9/420: chown d0/d8/d9/d52/d5e/d5f/f89 384373 1 2024-05-15T15:54:15.503 INFO:tasks.workunit.client.0.smithi049.stdout:1/718: dread d0/f58 [0,4194304] 0 2024-05-15T15:54:15.503 INFO:tasks.workunit.client.1.smithi144.stdout:1/587: dwrite d7/d1d/d33/d1e/d87/f92 [0,4194304] 0 2024-05-15T15:54:15.504 INFO:tasks.workunit.client.1.smithi144.stdout:4/586: creat d6/dd/d34/d3b/d30/d59/d66/fb8 x:0 0 0 2024-05-15T15:54:15.504 INFO:tasks.workunit.client.0.smithi049.stdout:9/421: chown d0/d8/d9/d22 5110 1 2024-05-15T15:54:15.506 INFO:tasks.workunit.client.1.smithi144.stdout:3/603: truncate d0/d11/d3b/d93/f86 627262 0 2024-05-15T15:54:15.510 INFO:tasks.workunit.client.1.smithi144.stdout:1/588: dwrite d7/d1d/d33/f71 [0,4194304] 0 2024-05-15T15:54:15.511 INFO:tasks.workunit.client.1.smithi144.stdout:1/589: write d7/d1d/d33/d1e/fbe [5669,94630] 0 2024-05-15T15:54:15.567 INFO:tasks.workunit.client.0.smithi049.stdout:9/422: chown d0/d8/d9/d52/d5e/d5f/d56/c6a 5202999 1 2024-05-15T15:54:15.568 INFO:tasks.workunit.client.1.smithi144.stdout:0/593: mkdir d4/df/d14/db5/dd1 0 2024-05-15T15:54:15.568 INFO:tasks.workunit.client.0.smithi049.stdout:6/510: dwrite dd/d17/d24/d2b/d36/d50/f52 [0,4194304] 0 2024-05-15T15:54:15.569 INFO:tasks.workunit.client.0.smithi049.stdout:9/423: chown d0/d8/d12/c7 4142260 1 2024-05-15T15:54:15.570 INFO:tasks.workunit.client.0.smithi049.stdout:4/543: mknod d18/d85/d98/cb4 0 2024-05-15T15:54:15.572 INFO:tasks.workunit.client.0.smithi049.stdout:5/534: fsync dc/dd/d10/f1a 0 2024-05-15T15:54:15.575 INFO:tasks.workunit.client.1.smithi144.stdout:7/597: creat d4/df/d25/d40/d62/dbc/fc8 x:0 0 0 2024-05-15T15:54:15.578 INFO:tasks.workunit.client.1.smithi144.stdout:2/572: sync 2024-05-15T15:54:15.587 INFO:tasks.workunit.client.0.smithi049.stdout:1/719: dread d0/d4/dd/de/d9e/fa0 [0,4194304] 0 2024-05-15T15:54:15.597 INFO:tasks.workunit.client.0.smithi049.stdout:3/575: rename d12/d30/d42/d7a/ca9 to d12/d26/cae 0 2024-05-15T15:54:15.598 INFO:tasks.workunit.client.1.smithi144.stdout:8/648: getdents d2/de/d49 0 2024-05-15T15:54:15.599 INFO:tasks.workunit.client.1.smithi144.stdout:8/649: chown d2/de/d36/f2f 13152523 1 2024-05-15T15:54:15.601 INFO:tasks.workunit.client.1.smithi144.stdout:5/622: mkdir d0/dc/d2f/d34/d3e/dd0 0 2024-05-15T15:54:15.601 INFO:tasks.workunit.client.1.smithi144.stdout:5/623: chown d0/f1a 30759 1 2024-05-15T15:54:15.605 INFO:tasks.workunit.client.0.smithi049.stdout:4/544: creat d18/d85/fb5 x:0 0 0 2024-05-15T15:54:15.610 INFO:tasks.workunit.client.0.smithi049.stdout:1/720: creat d0/d4/dd/de/d36/f101 x:0 0 0 2024-05-15T15:54:15.610 INFO:tasks.workunit.client.1.smithi144.stdout:9/622: dwrite d0/d8/d5e/d44/f74 [0,4194304] 0 2024-05-15T15:54:15.611 INFO:tasks.workunit.client.1.smithi144.stdout:9/623: chown d0/d8/d5e/d44/d16/dcd/fd3 124950123 1 2024-05-15T15:54:15.611 INFO:tasks.workunit.client.0.smithi049.stdout:1/721: write d0/d4/dd/de/d1e/f15 [1547185,103217] 0 2024-05-15T15:54:15.611 INFO:tasks.workunit.client.1.smithi144.stdout:1/590: mkdir d7/d1d/d33/d41/d78/dc3 0 2024-05-15T15:54:15.612 INFO:tasks.workunit.client.1.smithi144.stdout:9/624: chown d0/d8/d5e/d44/dc2/lcb 120595061 1 2024-05-15T15:54:15.612 INFO:tasks.workunit.client.0.smithi049.stdout:9/424: write d0/d8/d1b/d40/d5c/f60 [684718,30763] 0 2024-05-15T15:54:15.612 INFO:tasks.workunit.client.0.smithi049.stdout:9/425: readlink d0/d8/d9/l18 0 2024-05-15T15:54:15.614 INFO:tasks.workunit.client.0.smithi049.stdout:5/535: rename dc/d29/f92 to dc/d4a/d31/da9/faa 0 2024-05-15T15:54:15.614 INFO:tasks.workunit.client.0.smithi049.stdout:9/426: chown d0/d8/d1b/d71/f79 3633 1 2024-05-15T15:54:15.615 INFO:tasks.workunit.client.0.smithi049.stdout:9/427: stat d0/d8/d21/c47 0 2024-05-15T15:54:15.617 INFO:tasks.workunit.client.0.smithi049.stdout:7/473: getdents d0/d6/da/d3d/d80 0 2024-05-15T15:54:15.618 INFO:tasks.workunit.client.0.smithi049.stdout:7/474: fsync d0/d6/d1d/f7a 0 2024-05-15T15:54:15.619 INFO:tasks.workunit.client.0.smithi049.stdout:6/511: mknod dd/c91 0 2024-05-15T15:54:15.621 INFO:tasks.workunit.client.1.smithi144.stdout:6/546: getdents d7/db/d35/d84/dbe 0 2024-05-15T15:54:15.628 INFO:tasks.workunit.client.1.smithi144.stdout:2/573: rename d3/d7/d20/c84 to d3/d7/d20/d22/d52/d75/cc0 0 2024-05-15T15:54:15.629 INFO:tasks.workunit.client.1.smithi144.stdout:8/650: symlink d2/d24/d2d/ld6 0 2024-05-15T15:54:15.629 INFO:tasks.workunit.client.0.smithi049.stdout:3/576: mknod d12/d67/da8/caf 0 2024-05-15T15:54:15.630 INFO:tasks.workunit.client.1.smithi144.stdout:9/625: dwrite d0/d8/d1b/d62/fca [4194304,4194304] 0 2024-05-15T15:54:15.630 INFO:tasks.workunit.client.1.smithi144.stdout:6/547: symlink d7/db/d5f/lc3 0 2024-05-15T15:54:15.631 INFO:tasks.workunit.client.0.smithi049.stdout:9/428: creat d0/d8/d1b/f98 x:0 0 0 2024-05-15T15:54:15.632 INFO:tasks.workunit.client.0.smithi049.stdout:9/429: readlink d0/d8/d9/l18 0 2024-05-15T15:54:15.632 INFO:tasks.workunit.client.0.smithi049.stdout:9/430: stat d0/d8/f39 0 2024-05-15T15:54:15.633 INFO:tasks.workunit.client.0.smithi049.stdout:9/431: stat d0/d8/d21/f82 0 2024-05-15T15:54:15.633 INFO:tasks.workunit.client.0.smithi049.stdout:9/432: fdatasync d0/d8/f61 0 2024-05-15T15:54:15.635 INFO:tasks.workunit.client.0.smithi049.stdout:7/475: truncate d0/d34/f3f 941149 0 2024-05-15T15:54:15.639 INFO:tasks.workunit.client.1.smithi144.stdout:0/594: write d4/d6/d3a/f91 [31137,46691] 0 2024-05-15T15:54:15.640 INFO:tasks.workunit.client.0.smithi049.stdout:6/512: mkdir dd/d17/d24/d2b/d36/d50/d54/d92 0 2024-05-15T15:54:15.640 INFO:tasks.workunit.client.1.smithi144.stdout:7/598: write d4/d3c/f57 [858797,130079] 0 2024-05-15T15:54:15.641 INFO:tasks.workunit.client.1.smithi144.stdout:8/651: mknod d2/d1a/cd7 0 2024-05-15T15:54:15.642 INFO:tasks.workunit.client.1.smithi144.stdout:7/599: chown d4/d3c/d76/d9d/dac/d48/l56 72995 1 2024-05-15T15:54:15.645 INFO:tasks.workunit.client.0.smithi049.stdout:1/722: mknod d0/d4/dd/de/d1e/d16/d8a/dd2/df2/c102 0 2024-05-15T15:54:15.647 INFO:tasks.workunit.client.1.smithi144.stdout:3/604: write d0/d1/f23 [1349276,120137] 0 2024-05-15T15:54:15.649 INFO:tasks.workunit.client.0.smithi049.stdout:4/545: dwrite d18/d21/f26 [0,4194304] 0 2024-05-15T15:54:15.650 INFO:tasks.workunit.client.1.smithi144.stdout:9/626: dread d0/d8/d1b/d62/fca [0,4194304] 0 2024-05-15T15:54:15.651 INFO:tasks.workunit.client.1.smithi144.stdout:4/587: link d6/dd/d34/d3b/d30/f2f d6/dd/d34/d3b/d30/db/d1d/d40/d6c/d82/fb9 0 2024-05-15T15:54:15.652 INFO:tasks.workunit.client.1.smithi144.stdout:9/627: chown d0/d8/d5e/d44/d10/d9c/f3f 534160 1 2024-05-15T15:54:15.652 INFO:tasks.workunit.client.0.smithi049.stdout:9/433: rmdir d0/d8/d9/d22 39 2024-05-15T15:54:15.653 INFO:tasks.workunit.client.1.smithi144.stdout:9/628: write d0/f96 [521999,88676] 0 2024-05-15T15:54:15.653 INFO:tasks.workunit.client.0.smithi049.stdout:9/434: readlink d0/d8/d9/l18 0 2024-05-15T15:54:15.655 INFO:tasks.workunit.client.0.smithi049.stdout:4/546: dread f10 [4194304,4194304] 0 2024-05-15T15:54:15.656 INFO:tasks.workunit.client.1.smithi144.stdout:5/624: rename d0/dc/d2f/d34/d63/d6e/dbf/c41 to d0/dc/d24/d28/d2d/d59/d74/cd1 0 2024-05-15T15:54:15.657 INFO:tasks.workunit.client.0.smithi049.stdout:7/476: mknod d0/d6/da/d3d/d80/c92 0 2024-05-15T15:54:15.658 INFO:tasks.workunit.client.1.smithi144.stdout:5/625: write d0/dc/d24/d28/f82 [2319072,26952] 0 2024-05-15T15:54:15.659 INFO:tasks.workunit.client.1.smithi144.stdout:0/595: creat d4/df/d1f/d24/d85/da1/da3/fd2 x:0 0 0 2024-05-15T15:54:15.661 INFO:tasks.workunit.client.0.smithi049.stdout:6/513: mkdir dd/d17/d24/d2b/d36/d93 0 2024-05-15T15:54:15.663 INFO:tasks.workunit.client.1.smithi144.stdout:8/652: symlink d2/de/d3f/ld8 0 2024-05-15T15:54:15.665 INFO:tasks.workunit.client.1.smithi144.stdout:7/600: fdatasync d4/d3c/d76/d9d/dac/f23 0 2024-05-15T15:54:15.669 INFO:tasks.workunit.client.1.smithi144.stdout:3/605: mknod d0/d1/d17/d39/d40/ccd 0 2024-05-15T15:54:15.673 INFO:tasks.workunit.client.0.smithi049.stdout:9/435: creat d0/d8/d9/d52/d5e/d8a/f99 x:0 0 0 2024-05-15T15:54:15.675 INFO:tasks.workunit.client.1.smithi144.stdout:4/588: mkdir d6/dd/d2d/dba 0 2024-05-15T15:54:15.677 INFO:tasks.workunit.client.0.smithi049.stdout:5/536: dwrite dc/d4a/d42/f48 [4194304,4194304] 0 2024-05-15T15:54:15.678 INFO:tasks.workunit.client.0.smithi049.stdout:5/537: fsync dc/dd/d47/f9c 0 2024-05-15T15:54:15.679 INFO:tasks.workunit.client.0.smithi049.stdout:5/538: write dc/d4a/f8c [824352,35840] 0 2024-05-15T15:54:15.680 INFO:tasks.workunit.client.1.smithi144.stdout:9/629: mknod d0/d8/d21/cd7 0 2024-05-15T15:54:15.681 INFO:tasks.workunit.client.1.smithi144.stdout:9/630: write d0/f96 [684140,32142] 0 2024-05-15T15:54:15.683 INFO:tasks.workunit.client.0.smithi049.stdout:7/477: dread d0/d6/f29 [0,4194304] 0 2024-05-15T15:54:15.684 INFO:tasks.workunit.client.0.smithi049.stdout:1/723: mkdir d0/d4/dd/de/d1e/d16/df1/d103 0 2024-05-15T15:54:15.685 INFO:tasks.workunit.client.0.smithi049.stdout:7/478: write d0/d2f/f8f [108000,117410] 0 2024-05-15T15:54:15.686 INFO:tasks.workunit.client.1.smithi144.stdout:8/653: rmdir d2/de/d3f/d6b 39 2024-05-15T15:54:15.689 INFO:tasks.workunit.client.0.smithi049.stdout:4/547: unlink d18/d1e/d2a/f41 0 2024-05-15T15:54:15.692 INFO:tasks.workunit.client.0.smithi049.stdout:3/577: dwrite d12/f25 [4194304,4194304] 0 2024-05-15T15:54:15.694 INFO:tasks.workunit.client.1.smithi144.stdout:4/589: fdatasync d6/dd/d34/d3b/f4c 0 2024-05-15T15:54:15.695 INFO:tasks.workunit.client.1.smithi144.stdout:3/606: dwrite d0/d1/d6c/f99 [0,4194304] 0 2024-05-15T15:54:15.697 INFO:tasks.workunit.client.1.smithi144.stdout:9/631: readlink d0/d8/d5e/d44/l37 0 2024-05-15T15:54:15.701 INFO:tasks.workunit.client.0.smithi049.stdout:1/724: rmdir d0/d4/dd/de/d36 39 2024-05-15T15:54:15.702 INFO:tasks.workunit.client.0.smithi049.stdout:1/725: chown d0/d4/dd/de/d48/d5c 17680811 1 2024-05-15T15:54:15.703 INFO:tasks.workunit.client.0.smithi049.stdout:1/726: readlink d0/d4/dd/de/d1e/d2e/l3a 0 2024-05-15T15:54:15.706 INFO:tasks.workunit.client.0.smithi049.stdout:9/436: write d0/d8/f39 [565804,225] 0 2024-05-15T15:54:15.707 INFO:tasks.workunit.client.0.smithi049.stdout:9/437: stat d0/d8/f7e 0 2024-05-15T15:54:15.708 INFO:tasks.workunit.client.0.smithi049.stdout:4/548: mkdir d18/d85/db6 0 2024-05-15T15:54:15.709 INFO:tasks.workunit.client.1.smithi144.stdout:6/548: link d7/db/d35/f67 d7/fc4 0 2024-05-15T15:54:15.710 INFO:tasks.workunit.client.1.smithi144.stdout:6/549: chown d7/f8 814302 1 2024-05-15T15:54:15.714 INFO:tasks.workunit.client.0.smithi049.stdout:3/578: chown d12/d14/l31 80685 1 2024-05-15T15:54:15.714 INFO:tasks.workunit.client.0.smithi049.stdout:4/549: dwrite d18/f28 [0,4194304] 0 2024-05-15T15:54:15.715 INFO:tasks.workunit.client.0.smithi049.stdout:4/550: chown d18/f1b 454 1 2024-05-15T15:54:15.715 INFO:tasks.workunit.client.1.smithi144.stdout:4/590: write d6/dd/d34/d3b/f4c [547397,92881] 0 2024-05-15T15:54:15.716 INFO:tasks.workunit.client.0.smithi049.stdout:5/539: mknod dc/d4a/d9e/cab 0 2024-05-15T15:54:15.716 INFO:tasks.workunit.client.1.smithi144.stdout:5/626: truncate d0/dc/d12/d4a/f60 2984847 0 2024-05-15T15:54:15.718 INFO:tasks.workunit.client.1.smithi144.stdout:3/607: fsync d0/d1/d17/f8b 0 2024-05-15T15:54:15.720 INFO:tasks.workunit.client.1.smithi144.stdout:3/608: chown d0/d1/d56/d71/d73/ca2 35905 1 2024-05-15T15:54:15.722 INFO:tasks.workunit.client.0.smithi049.stdout:7/479: symlink d0/d6/da/l93 0 2024-05-15T15:54:15.723 INFO:tasks.workunit.client.1.smithi144.stdout:5/627: dwrite d0/f11 [4194304,4194304] 0 2024-05-15T15:54:15.724 INFO:tasks.workunit.client.1.smithi144.stdout:8/654: truncate d2/f96 2164371 0 2024-05-15T15:54:15.725 INFO:tasks.workunit.client.0.smithi049.stdout:9/438: mkdir d0/d8/d12/d9a 0 2024-05-15T15:54:15.727 INFO:tasks.workunit.client.1.smithi144.stdout:1/591: sync 2024-05-15T15:54:15.728 INFO:tasks.workunit.client.1.smithi144.stdout:4/591: creat d6/dd/fbb x:0 0 0 2024-05-15T15:54:15.731 INFO:tasks.workunit.client.1.smithi144.stdout:3/609: mkdir d0/d1/d17/dce 0 2024-05-15T15:54:15.731 INFO:tasks.workunit.client.0.smithi049.stdout:6/514: link dd/d17/d24/d2b/d36/d50/c7d dd/c94 0 2024-05-15T15:54:15.731 INFO:tasks.workunit.client.0.smithi049.stdout:6/515: chown la 14493 1 2024-05-15T15:54:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.730+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 --> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f64000bf0 con 0x7f1f6c06d6f0 2024-05-15T15:54:15.733 INFO:tasks.workunit.client.1.smithi144.stdout:5/628: creat d0/dc/d24/fd2 x:0 0 0 2024-05-15T15:54:15.733 INFO:tasks.workunit.client.0.smithi049.stdout:9/439: creat d0/d8/d1b/d71/d7d/f9b x:0 0 0 2024-05-15T15:54:15.734 INFO:tasks.workunit.client.0.smithi049.stdout:9/440: readlink d0/d8/l81 0 2024-05-15T15:54:15.734 INFO:tasks.workunit.client.1.smithi144.stdout:3/610: write d0/d11/d3b/da7/fa9 [958861,105966] 0 2024-05-15T15:54:15.735 INFO:tasks.workunit.client.1.smithi144.stdout:6/550: truncate d7/ff 3536 0 2024-05-15T15:54:15.736 INFO:tasks.workunit.client.1.smithi144.stdout:1/592: dwrite d7/d1d/d33/d1e/f77 [0,4194304] 0 2024-05-15T15:54:15.737 INFO:tasks.workunit.client.1.smithi144.stdout:3/611: stat d0/d1/d56/d71/c82 0 2024-05-15T15:54:15.738 INFO:tasks.workunit.client.1.smithi144.stdout:9/632: link d0/f78 d0/fd8 0 2024-05-15T15:54:15.738 INFO:tasks.workunit.client.1.smithi144.stdout:6/551: fsync d7/db/d11/d3c/f79 0 2024-05-15T15:54:15.739 INFO:tasks.workunit.client.1.smithi144.stdout:4/592: dwrite d6/d42/d46/d4b/fb2 [0,4194304] 0 2024-05-15T15:54:15.740 INFO:tasks.workunit.client.1.smithi144.stdout:6/552: fsync d7/d62/fa9 0 2024-05-15T15:54:15.741 INFO:tasks.workunit.client.0.smithi049.stdout:5/540: rename dc/dd/d39/l8e to dc/lac 0 2024-05-15T15:54:15.742 INFO:tasks.workunit.client.0.smithi049.stdout:7/480: link d0/d6/da/d3d/l20 d0/d6/d23/d45/d83/l94 0 2024-05-15T15:54:15.743 INFO:tasks.workunit.client.0.smithi049.stdout:7/481: write d0/d6/d23/f8a [8751,8085] 0 2024-05-15T15:54:15.744 INFO:tasks.workunit.client.1.smithi144.stdout:3/612: dread - d0/d11/d3b/da0/d8f/f9b zero size 2024-05-15T15:54:15.744 INFO:tasks.workunit.client.1.smithi144.stdout:4/593: read - d6/d42/d46/f90 zero size 2024-05-15T15:54:15.744 INFO:tasks.workunit.client.0.smithi049.stdout:6/516: creat dd/d17/d24/d2b/d36/d86/f95 x:0 0 0 2024-05-15T15:54:15.745 INFO:tasks.workunit.client.1.smithi144.stdout:4/594: write d6/dd/d34/d3b/d30/db/d1d/d4e/f8b [2476921,51366] 0 2024-05-15T15:54:15.745 INFO:tasks.workunit.client.0.smithi049.stdout:7/482: write d0/d2f/d5c/f7d [172899,41869] 0 2024-05-15T15:54:15.746 INFO:tasks.workunit.client.0.smithi049.stdout:3/579: getdents d12/d14/d32/d64 0 2024-05-15T15:54:15.746 INFO:tasks.workunit.client.0.smithi049.stdout:6/517: chown dd/d17/d24/d2b/d36/l49 921 1 2024-05-15T15:54:15.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.750+0000 7f1f76ffd700 1 -- 172.21.15.49:0/3041064269 <== mgr.24413 v2:172.21.15.144:6828/177983287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1f64000bf0 con 0x7f1f6c06d6f0 2024-05-15T15:54:15.753 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:54:15.753 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (7m) 36s ago 9m 24.2M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:54:15.753 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (9m) 36s ago 9m 7864k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (8m) 41s ago 8m 7671k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (9m) 36s ago 9m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (8m) 41s ago 8m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (7m) 36s ago 8m 86.1M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (3m) 36s ago 3m 140M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (3m) 36s ago 3m 153M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (3m) 41s ago 3m 21.4M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (3m) 41s ago 3m 24.4M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:9283,8765,8443 running (10m) 36s ago 10m 314M - 18.2.2-1343-g2965081f a04a73ffe079 ea7d1a4b9c55 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (68s) 41s ago 7m 490M - 19.0.0-2576-gde5210d5 48bc18eb22d8 f3b66ea0d25c 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (10m) 36s ago 10m 48.7M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (7m) 41s ago 7m 46.2M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (9m) 36s ago 9m 15.5M - 1.5.0 0da6a335fe13 171e0a34f565 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (8m) 41s ago 8m 15.5M - 1.5.0 0da6a335fe13 be3a2837bc85 2024-05-15T15:54:15.754 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (7m) 36s ago 7m 86.3M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:54:15.755 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (6m) 36s ago 6m 84.0M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:54:15.755 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (6m) 36s ago 6m 76.9M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:54:15.755 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (5m) 41s ago 5m 88.8M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:54:15.755 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (5m) 41s ago 5m 79.8M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:54:15.755 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (4m) 41s ago 4m 87.0M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:54:15.755 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 starting - - - - 2024-05-15T15:54:15.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.753+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f1f6c06d6f0 msgr2=0x7f1f6c06fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:15.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.753+0000 7f1f8801a700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f1f6c06d6f0 0x7f1f6c06fbb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009fd0 tx=0x7f1f70005fb0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.754+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800ffb60 msgr2=0x7f1f8019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:15.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.754+0000 7f1f8801a700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800ffb60 0x7f1f8019ca80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e8f0 tx=0x7f1f7c00ecb0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.754+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 shutdown_connections 2024-05-15T15:54:15.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.754+0000 7f1f8801a700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f1f6c06d6f0 0x7f1f6c06fbb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.754+0000 7f1f8801a700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800ffb60 0x7f1f8019ca80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.754+0000 7f1f8801a700 1 --2- 172.21.15.49:0/3041064269 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f80100d60 0x7f1f8019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.754+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 >> 172.21.15.49:0/3041064269 conn(0x7f1f800fb0e0 msgr2=0x7f1f80068610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:15.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.754+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 shutdown_connections 2024-05-15T15:54:15.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.754+0000 7f1f8801a700 1 -- 172.21.15.49:0/3041064269 wait complete. 2024-05-15T15:54:15.758 INFO:tasks.workunit.client.0.smithi049.stdout:3/580: creat d12/d67/fb0 x:0 0 0 2024-05-15T15:54:15.761 INFO:tasks.workunit.client.0.smithi049.stdout:7/483: rmdir d0/d6/da/d3d/d54 39 2024-05-15T15:54:15.762 INFO:tasks.workunit.client.0.smithi049.stdout:4/551: dwrite d18/f22 [0,4194304] 0 2024-05-15T15:54:15.763 INFO:tasks.workunit.client.0.smithi049.stdout:6/518: mkdir dd/d17/d24/d2b/d36/d86/d96 0 2024-05-15T15:54:15.763 INFO:tasks.workunit.client.0.smithi049.stdout:7/484: write d0/d6/d1d/f59 [2662560,93064] 0 2024-05-15T15:54:15.768 INFO:tasks.workunit.client.1.smithi144.stdout:8/655: getdents d2/d24/d2d 0 2024-05-15T15:54:15.768 INFO:tasks.workunit.client.1.smithi144.stdout:1/593: rename d7/d1d/d33/d41/d78/da3/l30 to d7/d1d/d33/d29/d6b/d5e/lc4 0 2024-05-15T15:54:15.774 INFO:tasks.workunit.client.1.smithi144.stdout:1/594: dread d7/d1d/d33/d55/f8d [0,4194304] 0 2024-05-15T15:54:15.775 INFO:tasks.workunit.client.0.smithi049.stdout:9/441: rename d0/d8/d1b/d71/d86 to d0/d8/d9c 0 2024-05-15T15:54:15.776 INFO:tasks.workunit.client.0.smithi049.stdout:9/442: chown d0/d25/d5d/f8d 1405 1 2024-05-15T15:54:15.778 INFO:tasks.workunit.client.1.smithi144.stdout:8/656: dread d2/d24/f68 [0,4194304] 0 2024-05-15T15:54:15.778 INFO:tasks.workunit.client.1.smithi144.stdout:8/657: dread - d2/de/d36/faf zero size 2024-05-15T15:54:15.783 INFO:tasks.workunit.client.1.smithi144.stdout:5/629: dread - d0/dc/d24/d8e/d93/fb2 zero size 2024-05-15T15:54:15.783 INFO:tasks.workunit.client.1.smithi144.stdout:4/595: mknod d6/d42/d46/d4b/d1e/d99/dad/cbc 0 2024-05-15T15:54:15.786 INFO:tasks.workunit.client.1.smithi144.stdout:1/595: mknod d7/d1d/d33/d29/d6b/da4/cc5 0 2024-05-15T15:54:15.787 INFO:tasks.workunit.client.1.smithi144.stdout:6/553: dwrite d7/f71 [0,4194304] 0 2024-05-15T15:54:15.788 INFO:tasks.workunit.client.1.smithi144.stdout:3/613: dwrite d0/d1/f4 [4194304,4194304] 0 2024-05-15T15:54:15.788 INFO:tasks.workunit.client.1.smithi144.stdout:3/614: chown d0/l2b 0 1 2024-05-15T15:54:15.789 INFO:tasks.workunit.client.1.smithi144.stdout:8/658: mkdir d2/de/d49/d83/dd9 0 2024-05-15T15:54:15.791 INFO:tasks.workunit.client.1.smithi144.stdout:8/659: stat d2/d24/d2d/da0/d6e/d71/d8c/fc9 0 2024-05-15T15:54:15.794 INFO:tasks.workunit.client.1.smithi144.stdout:4/596: mkdir d6/dd/d34/d3b/d30/d59/d66/dbd 0 2024-05-15T15:54:15.795 INFO:tasks.workunit.client.1.smithi144.stdout:6/554: dwrite d7/db/d11/f43 [0,4194304] 0 2024-05-15T15:54:15.797 INFO:tasks.workunit.client.1.smithi144.stdout:3/615: mkdir d0/d11/d3b/dcf 0 2024-05-15T15:54:15.806 INFO:tasks.workunit.client.1.smithi144.stdout:1/596: mknod d7/d1d/d33/cc6 0 2024-05-15T15:54:15.807 INFO:tasks.workunit.client.1.smithi144.stdout:1/597: dread - d7/d1d/f8b zero size 2024-05-15T15:54:15.810 INFO:tasks.workunit.client.1.smithi144.stdout:3/616: chown d0/d11/d24/fb8 111484234 1 2024-05-15T15:54:15.815 INFO:tasks.workunit.client.1.smithi144.stdout:6/555: link d7/db/d9d/f88 d7/db/d11/d3c/d61/fc5 0 2024-05-15T15:54:15.816 INFO:tasks.workunit.client.1.smithi144.stdout:2/574: sync 2024-05-15T15:54:15.818 INFO:tasks.workunit.client.1.smithi144.stdout:8/660: getdents d2/d24 0 2024-05-15T15:54:15.822 INFO:tasks.workunit.client.1.smithi144.stdout:8/661: unlink d2/d24/d2d/da0/d6e/d71/d8c/cc6 0 2024-05-15T15:54:15.823 INFO:tasks.workunit.client.1.smithi144.stdout:4/597: write d6/d49/f69 [839191,124569] 0 2024-05-15T15:54:15.823 INFO:tasks.workunit.client.1.smithi144.stdout:8/662: chown d2/de/fa1 3 1 2024-05-15T15:54:15.830 INFO:tasks.workunit.client.1.smithi144.stdout:1/598: dwrite d7/d1d/f32 [0,4194304] 0 2024-05-15T15:54:15.830 INFO:tasks.workunit.client.1.smithi144.stdout:3/617: dread d0/d11/f2a [0,4194304] 0 2024-05-15T15:54:15.832 INFO:tasks.workunit.client.1.smithi144.stdout:6/556: dread d7/d1e/f44 [0,4194304] 0 2024-05-15T15:54:15.832 INFO:tasks.workunit.client.1.smithi144.stdout:8/663: rename d2/de/fa1 to d2/de/d36/d87/fda 0 2024-05-15T15:54:15.834 INFO:tasks.workunit.client.1.smithi144.stdout:8/664: readlink d2/de/d36/l63 0 2024-05-15T15:54:15.834 INFO:tasks.workunit.client.1.smithi144.stdout:6/557: stat d7/db/d3a/cc1 0 2024-05-15T15:54:15.834 INFO:tasks.workunit.client.1.smithi144.stdout:2/575: dread d3/d7/d20/d22/d52/d64/f26 [4194304,4194304] 0 2024-05-15T15:54:15.834 INFO:tasks.workunit.client.1.smithi144.stdout:1/599: mknod d7/d1d/d33/d1e/d4e/cc7 0 2024-05-15T15:54:15.836 INFO:tasks.workunit.client.1.smithi144.stdout:1/600: stat d7/d1d/d33/d29/d6b/l5f 0 2024-05-15T15:54:15.836 INFO:tasks.workunit.client.1.smithi144.stdout:4/598: link d6/dd/d34/d3b/d30/d59/d66/f9e d6/dd/d2d/dba/fbe 0 2024-05-15T15:54:15.837 INFO:tasks.workunit.client.1.smithi144.stdout:3/618: creat d0/d1/d17/dce/fd0 x:0 0 0 2024-05-15T15:54:15.838 INFO:tasks.workunit.client.1.smithi144.stdout:1/601: truncate d7/d1d/d33/d1e/d4e/fbb 496173 0 2024-05-15T15:54:15.841 INFO:tasks.workunit.client.1.smithi144.stdout:6/558: rename d7/db/da2/d70/fbb to d7/db/d9d/fc6 0 2024-05-15T15:54:15.843 INFO:tasks.workunit.client.1.smithi144.stdout:8/665: fdatasync d2/f96 0 2024-05-15T15:54:15.843 INFO:tasks.workunit.client.1.smithi144.stdout:4/599: dwrite d6/d42/d46/d4b/d1e/d99/dab/f9a [0,4194304] 0 2024-05-15T15:54:15.846 INFO:tasks.workunit.client.1.smithi144.stdout:3/619: unlink d0/d11/d3b/da0/d8f/fcb 0 2024-05-15T15:54:15.847 INFO:tasks.workunit.client.1.smithi144.stdout:3/620: chown d0/d11/d24/f4f 36 1 2024-05-15T15:54:15.848 INFO:tasks.workunit.client.1.smithi144.stdout:1/602: mkdir d7/d1d/d33/d41/d22/dbd/dc8 0 2024-05-15T15:54:15.852 INFO:tasks.workunit.client.1.smithi144.stdout:3/621: dwrite d0/d1/d17/d39/d40/f92 [0,4194304] 0 2024-05-15T15:54:15.854 INFO:tasks.workunit.client.1.smithi144.stdout:3/622: dread - d0/d1/d17/d39/d40/fc0 zero size 2024-05-15T15:54:15.860 INFO:tasks.workunit.client.1.smithi144.stdout:4/600: rename d6/dd/d2e/f70 to d6/d42/d46/d4b/d1e/d44/fbf 0 2024-05-15T15:54:15.862 INFO:tasks.workunit.client.1.smithi144.stdout:1/603: unlink d7/df/c14 0 2024-05-15T15:54:15.863 INFO:tasks.workunit.client.1.smithi144.stdout:1/604: write d7/d1d/f8b [920837,102498] 0 2024-05-15T15:54:15.864 INFO:tasks.workunit.client.1.smithi144.stdout:1/605: readlink d7/d1d/d33/d41/d78/da3/lc2 0 2024-05-15T15:54:15.872 INFO:tasks.workunit.client.1.smithi144.stdout:1/606: dwrite d7/d1d/d33/d41/d22/fa8 [0,4194304] 0 2024-05-15T15:54:15.873 INFO:tasks.workunit.client.1.smithi144.stdout:3/623: dread d0/d1/d17/d39/d40/d4a/d77/fc4 [0,4194304] 0 2024-05-15T15:54:15.876 INFO:tasks.workunit.client.1.smithi144.stdout:6/559: link d7/db/da2/d70/c87 d7/da6/cc7 0 2024-05-15T15:54:15.877 INFO:tasks.workunit.client.1.smithi144.stdout:6/560: chown d7/db/d5f/f65 0 1 2024-05-15T15:54:15.880 INFO:tasks.workunit.client.1.smithi144.stdout:3/624: dwrite d0/d1/d17/d39/d40/d7e/f81 [0,4194304] 0 2024-05-15T15:54:15.882 INFO:tasks.workunit.client.1.smithi144.stdout:6/561: symlink d7/db/d5f/lc8 0 2024-05-15T15:54:15.882 INFO:tasks.workunit.client.1.smithi144.stdout:4/601: link d6/dd/d34/d3b/l24 d6/dd/d34/d3b/d30/d59/lc0 0 2024-05-15T15:54:15.885 INFO:tasks.workunit.client.1.smithi144.stdout:4/602: stat d6/dd/d34/d3b/d30/db/d1d/d40/f80 0 2024-05-15T15:54:15.890 INFO:tasks.workunit.client.1.smithi144.stdout:6/562: rename d7/d1e/f69 to d7/d1e/fc9 0 2024-05-15T15:54:15.890 INFO:tasks.workunit.client.1.smithi144.stdout:8/666: dwrite d2/de/d36/f1e [0,4194304] 0 2024-05-15T15:54:15.892 INFO:tasks.workunit.client.1.smithi144.stdout:6/563: rmdir d7/db/d9d 39 2024-05-15T15:54:15.895 INFO:tasks.workunit.client.1.smithi144.stdout:4/603: rename d6/d42/d46/d4b/d1e/d99/dab/lb6 to d6/d42/lc1 0 2024-05-15T15:54:15.896 INFO:tasks.workunit.client.1.smithi144.stdout:6/564: write d7/d1e/d28/f3d [5576193,106600] 0 2024-05-15T15:54:15.898 INFO:tasks.workunit.client.1.smithi144.stdout:8/667: rename d2/d24/d2d/da0/d9e/db8/fc7 to d2/d3a/d7b/fdb 0 2024-05-15T15:54:15.899 INFO:tasks.workunit.client.1.smithi144.stdout:6/565: creat d7/db/d35/d84/d81/fca x:0 0 0 2024-05-15T15:54:15.901 INFO:tasks.workunit.client.1.smithi144.stdout:4/604: dwrite d6/f95 [0,4194304] 0 2024-05-15T15:54:15.901 INFO:tasks.workunit.client.1.smithi144.stdout:8/668: mknod d2/d3a/d7b/d4b/cdc 0 2024-05-15T15:54:15.903 INFO:tasks.workunit.client.1.smithi144.stdout:6/566: write d7/db/f2c [1901072,13410] 0 2024-05-15T15:54:15.903 INFO:tasks.workunit.client.1.smithi144.stdout:8/669: write d2/de/d36/f46 [1685775,109612] 0 2024-05-15T15:54:15.904 INFO:tasks.workunit.client.1.smithi144.stdout:6/567: dread - d7/f41 zero size 2024-05-15T15:54:15.905 INFO:tasks.workunit.client.1.smithi144.stdout:4/605: rmdir d6/dd/d2e 39 2024-05-15T15:54:15.907 INFO:tasks.workunit.client.0.smithi049.stdout:6/519: creat dd/d17/d24/d8a/f97 x:0 0 0 2024-05-15T15:54:15.907 INFO:tasks.workunit.client.0.smithi049.stdout:5/541: write dc/f30 [758591,72120] 0 2024-05-15T15:54:15.910 INFO:tasks.workunit.client.1.smithi144.stdout:8/670: chown d2/l34 726640079 1 2024-05-15T15:54:15.911 INFO:tasks.workunit.client.1.smithi144.stdout:3/625: truncate d0/d1/d17/f84 1191018 0 2024-05-15T15:54:15.914 INFO:tasks.workunit.client.0.smithi049.stdout:9/443: dwrite d0/d8/d9/f57 [0,4194304] 0 2024-05-15T15:54:15.920 INFO:tasks.workunit.client.1.smithi144.stdout:0/596: sync 2024-05-15T15:54:15.920 INFO:tasks.workunit.client.1.smithi144.stdout:7/601: sync 2024-05-15T15:54:15.928 INFO:tasks.workunit.client.1.smithi144.stdout:6/568: truncate d7/db/d11/d3c/fac 4111630 0 2024-05-15T15:54:15.931 INFO:tasks.workunit.client.1.smithi144.stdout:7/602: symlink d4/d3c/d76/lc9 0 2024-05-15T15:54:15.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.931+0000 7fd30f5e3700 1 -- 172.21.15.49:0/3849539536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 msgr2=0x7fd308101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:15.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.931+0000 7fd30f5e3700 1 --2- 172.21.15.49:0/3849539536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 0x7fd308101230 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009910 tx=0x7fd2f8009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:15.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.931+0000 7fd30f5e3700 1 -- 172.21.15.49:0/3849539536 shutdown_connections 2024-05-15T15:54:15.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.931+0000 7fd30f5e3700 1 --2- 172.21.15.49:0/3849539536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 0x7fd308101230 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.931+0000 7fd30f5e3700 1 --2- 172.21.15.49:0/3849539536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3080ffbd0 0x7fd3080ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.931+0000 7fd30f5e3700 1 -- 172.21.15.49:0/3849539536 >> 172.21.15.49:0/3849539536 conn(0x7fd3080fb170 msgr2=0x7fd3080fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:15.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.932+0000 7fd30f5e3700 1 -- 172.21.15.49:0/3849539536 shutdown_connections 2024-05-15T15:54:15.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.932+0000 7fd30f5e3700 1 -- 172.21.15.49:0/3849539536 wait complete. 2024-05-15T15:54:15.935 INFO:tasks.workunit.client.0.smithi049.stdout:4/552: dread d18/d1e/d63/f96 [0,4194304] 0 2024-05-15T15:54:15.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.932+0000 7fd30f5e3700 1 Processor -- start 2024-05-15T15:54:15.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.932+0000 7fd30f5e3700 1 -- start start 2024-05-15T15:54:15.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30f5e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3080ffbd0 0x7fd3081944d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:15.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30f5e3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 0x7fd308194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:15.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30f5e3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd308195030 con 0x7fd308100dd0 2024-05-15T15:54:15.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30f5e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd308195170 con 0x7fd3080ffbd0 2024-05-15T15:54:15.936 INFO:tasks.workunit.client.0.smithi049.stdout:4/553: chown d18/d1e/f8e 1 1 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30cb7e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 0x7fd308194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30cb7e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 0x7fd308194a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54804/0 (socket says 172.21.15.49:54804) 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30cb7e700 1 -- 172.21.15.49:0/193574544 learned_addr learned my addr 172.21.15.49:0/193574544 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30d37f700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3080ffbd0 0x7fd3081944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30cb7e700 1 -- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3080ffbd0 msgr2=0x7fd3081944d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30cb7e700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3080ffbd0 0x7fd3081944d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30cb7e700 1 -- 172.21.15.49:0/193574544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f80095f0 con 0x7fd308100dd0 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.933+0000 7fd30d37f700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3080ffbd0 0x7fd3081944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.934+0000 7fd30cb7e700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 0x7fd308194a10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80092e0 tx=0x7fd2f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:15.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.934+0000 7fd2fe7fc700 1 -- 172.21.15.49:0/193574544 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f801d070 con 0x7fd308100dd0 2024-05-15T15:54:15.938 INFO:tasks.workunit.client.0.smithi049.stdout:4/554: write d18/d1e/fa6 [870067,9677] 0 2024-05-15T15:54:15.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.934+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd308199b20 con 0x7fd308100dd0 2024-05-15T15:54:15.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.934+0000 7fd2fe7fc700 1 -- 172.21.15.49:0/193574544 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2f8005dc0 con 0x7fd308100dd0 2024-05-15T15:54:15.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.934+0000 7fd2fe7fc700 1 -- 172.21.15.49:0/193574544 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f8021410 con 0x7fd308100dd0 2024-05-15T15:54:15.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.934+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30819a090 con 0x7fd308100dd0 2024-05-15T15:54:15.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.936+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30818e640 con 0x7fd308100dd0 2024-05-15T15:54:15.939 INFO:tasks.workunit.client.1.smithi144.stdout:4/606: unlink d6/d42/d46/d4b/d1e/c23 0 2024-05-15T15:54:15.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.936+0000 7fd2fe7fc700 1 -- 172.21.15.49:0/193574544 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7fd2f800f460 con 0x7fd308100dd0 2024-05-15T15:54:15.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.937+0000 7fd2fe7fc700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fd2f406f8f0 0x7fd2f4071db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:15.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.937+0000 7fd2fe7fc700 1 -- 172.21.15.49:0/193574544 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd2f808e340 con 0x7fd308100dd0 2024-05-15T15:54:15.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.937+0000 7fd30d37f700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fd2f406f8f0 0x7fd2f4071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:15.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.938+0000 7fd30d37f700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fd2f406f8f0 0x7fd2f4071db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd308100c30 tx=0x7fd30400b410 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:15.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:15.943+0000 7fd2fe7fc700 1 -- 172.21.15.49:0/193574544 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fd2f805b5b0 con 0x7fd308100dd0 2024-05-15T15:54:15.947 INFO:tasks.workunit.client.1.smithi144.stdout:0/597: dread d4/d6/f11 [0,4194304] 0 2024-05-15T15:54:15.954 INFO:tasks.workunit.client.1.smithi144.stdout:6/569: rmdir d7/db/d5f 39 2024-05-15T15:54:15.958 INFO:tasks.workunit.client.1.smithi144.stdout:4/607: creat d6/dd/d34/fc2 x:0 0 0 2024-05-15T15:54:15.962 INFO:tasks.workunit.client.1.smithi144.stdout:3/626: dwrite d0/d11/d24/f41 [0,4194304] 0 2024-05-15T15:54:15.962 INFO:tasks.workunit.client.1.smithi144.stdout:3/627: read d0/d1/d17/f33 [461098,5843] 0 2024-05-15T15:54:15.965 INFO:tasks.workunit.client.1.smithi144.stdout:0/598: dread d4/d2d/d48/d5b/f75 [0,4194304] 0 2024-05-15T15:54:15.969 INFO:tasks.workunit.client.1.smithi144.stdout:4/608: dwrite d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/fb7 [0,4194304] 0 2024-05-15T15:54:15.969 INFO:tasks.workunit.client.1.smithi144.stdout:0/599: dread d4/d6/d3a/f91 [0,4194304] 0 2024-05-15T15:54:15.973 INFO:tasks.workunit.client.1.smithi144.stdout:8/671: dwrite d2/d1a/f3d [0,4194304] 0 2024-05-15T15:54:15.974 INFO:tasks.workunit.client.1.smithi144.stdout:3/628: mkdir d0/d1/d56/d71/d73/dd1 0 2024-05-15T15:54:15.982 INFO:tasks.workunit.client.0.smithi049.stdout:8/541: sync 2024-05-15T15:54:15.984 INFO:tasks.workunit.client.0.smithi049.stdout:7/485: symlink d0/d6/d23/d45/d6a/l95 0 2024-05-15T15:54:15.985 INFO:tasks.workunit.client.0.smithi049.stdout:8/542: write d2/de/d71/d86/d5d/d7d/d1b/d68/f8e [314182,42473] 0 2024-05-15T15:54:15.989 INFO:tasks.workunit.client.0.smithi049.stdout:6/520: creat dd/d17/d24/d2b/d36/f98 x:0 0 0 2024-05-15T15:54:15.992 INFO:tasks.workunit.client.0.smithi049.stdout:4/555: unlink d18/d1e/d2a/d50/d54/f6d 0 2024-05-15T15:54:15.993 INFO:tasks.workunit.client.1.smithi144.stdout:6/570: dwrite d7/d1e/f82 [0,4194304] 0 2024-05-15T15:54:15.993 INFO:tasks.workunit.client.1.smithi144.stdout:3/629: creat d0/d1/d56/d98/fd2 x:0 0 0 2024-05-15T15:54:15.995 INFO:tasks.workunit.client.0.smithi049.stdout:8/543: dwrite d2/de/d27/d5b/d60/da5/fa9 [0,4194304] 0 2024-05-15T15:54:15.997 INFO:tasks.workunit.client.1.smithi144.stdout:0/600: dread d4/dc/f17 [0,4194304] 0 2024-05-15T15:54:15.998 INFO:tasks.workunit.client.0.smithi049.stdout:5/542: fsync dc/dd/f14 0 2024-05-15T15:54:16.000 INFO:tasks.workunit.client.1.smithi144.stdout:4/609: mkdir d6/dd/dc3 0 2024-05-15T15:54:16.000 INFO:tasks.workunit.client.0.smithi049.stdout:6/521: dwrite dd/d17/d24/d2b/f63 [0,4194304] 0 2024-05-15T15:54:16.001 INFO:tasks.workunit.client.0.smithi049.stdout:5/543: write dc/f1b [8196443,128728] 0 2024-05-15T15:54:16.001 INFO:tasks.workunit.client.1.smithi144.stdout:4/610: readlink d6/dd/d34/d3b/d30/db/d1d/l28 0 2024-05-15T15:54:16.002 INFO:tasks.workunit.client.1.smithi144.stdout:4/611: fsync d6/dd/d34/fc2 0 2024-05-15T15:54:16.008 INFO:tasks.workunit.client.1.smithi144.stdout:8/672: creat d2/d24/d2d/d30/dd0/fdd x:0 0 0 2024-05-15T15:54:16.010 INFO:tasks.workunit.client.1.smithi144.stdout:6/571: write d7/db/d9d/fc6 [2121221,29087] 0 2024-05-15T15:54:16.014 INFO:tasks.workunit.client.1.smithi144.stdout:0/601: mkdir d4/d2d/d48/d5b/dd3 0 2024-05-15T15:54:16.015 INFO:tasks.workunit.client.0.smithi049.stdout:8/544: mkdir d2/de/d71/d86/d5d/d7d/d1b/d69/dac 0 2024-05-15T15:54:16.018 INFO:tasks.workunit.client.1.smithi144.stdout:4/612: rename d6/dd/d34/d3b/d30/db/d1d/l28 to d6/dd/d34/d3b/d30/db/d1d/d40/d57/lc4 0 2024-05-15T15:54:16.019 INFO:tasks.workunit.client.1.smithi144.stdout:8/673: mkdir d2/d24/d2d/da0/d9e/db8/dde 0 2024-05-15T15:54:16.021 INFO:tasks.workunit.client.0.smithi049.stdout:6/522: unlink dd/f60 0 2024-05-15T15:54:16.021 INFO:tasks.workunit.client.0.smithi049.stdout:8/545: dwrite d2/de/d25/f32 [0,4194304] 0 2024-05-15T15:54:16.023 INFO:tasks.workunit.client.0.smithi049.stdout:4/556: mknod d18/d85/db6/cb7 0 2024-05-15T15:54:16.024 INFO:tasks.workunit.client.1.smithi144.stdout:6/572: truncate d7/db/d9d/f88 136624 0 2024-05-15T15:54:16.026 INFO:tasks.workunit.client.1.smithi144.stdout:5/630: sync 2024-05-15T15:54:16.026 INFO:tasks.workunit.client.1.smithi144.stdout:9/633: sync 2024-05-15T15:54:16.028 INFO:tasks.workunit.client.0.smithi049.stdout:6/523: mknod dd/d17/d64/c99 0 2024-05-15T15:54:16.029 INFO:tasks.workunit.client.1.smithi144.stdout:8/674: symlink d2/d24/d2d/da0/d6e/d71/ldf 0 2024-05-15T15:54:16.030 INFO:tasks.workunit.client.1.smithi144.stdout:8/675: chown d2/d3a/c52 11896062 1 2024-05-15T15:54:16.031 INFO:tasks.workunit.client.0.smithi049.stdout:8/546: fdatasync d2/de/d71/d86/d5d/f89 0 2024-05-15T15:54:16.032 INFO:tasks.workunit.client.0.smithi049.stdout:4/557: dwrite d18/d1e/d2a/f43 [0,4194304] 0 2024-05-15T15:54:16.033 INFO:tasks.workunit.client.0.smithi049.stdout:4/558: chown d18/d1e 28027 1 2024-05-15T15:54:16.034 INFO:tasks.workunit.client.1.smithi144.stdout:4/613: mknod d6/dd/dc3/cc5 0 2024-05-15T15:54:16.036 INFO:tasks.workunit.client.0.smithi049.stdout:7/486: getdents d0/d6/d23/d45 0 2024-05-15T15:54:16.036 INFO:tasks.workunit.client.0.smithi049.stdout:4/559: truncate d18/d1e/d2a/d36/f58 4672115 0 2024-05-15T15:54:16.037 INFO:tasks.workunit.client.1.smithi144.stdout:4/614: truncate d6/dd/d34/d3b/d30/d59/d66/f9e 5144976 0 2024-05-15T15:54:16.038 INFO:tasks.workunit.client.0.smithi049.stdout:5/544: rename dc/dd/d47/c97 to dc/d4a/d31/cad 0 2024-05-15T15:54:16.043 INFO:tasks.workunit.client.1.smithi144.stdout:5/631: unlink d0/dc/d2f/d68/d5c/l64 0 2024-05-15T15:54:16.043 INFO:tasks.workunit.client.1.smithi144.stdout:4/615: dread d6/f95 [0,4194304] 0 2024-05-15T15:54:16.044 INFO:tasks.workunit.client.1.smithi144.stdout:5/632: dread - d0/dc/d1f/d50/fbc zero size 2024-05-15T15:54:16.044 INFO:tasks.workunit.client.1.smithi144.stdout:3/630: getdents d0/d11/d3b/da0 0 2024-05-15T15:54:16.046 INFO:tasks.workunit.client.0.smithi049.stdout:8/547: fsync d2/de/d71/d86/d5d/d7d/d45/f7c 0 2024-05-15T15:54:16.048 INFO:tasks.workunit.client.1.smithi144.stdout:9/634: dread d0/d8/d5e/d44/f26 [0,4194304] 0 2024-05-15T15:54:16.049 INFO:tasks.workunit.client.1.smithi144.stdout:9/635: dread d0/d8/d5e/d44/d10/d9c/fa9 [0,4194304] 0 2024-05-15T15:54:16.050 INFO:tasks.workunit.client.0.smithi049.stdout:4/560: rename d18/d1e/d2a/d36/f8f to d18/d1e/d63/fb8 0 2024-05-15T15:54:16.050 INFO:tasks.workunit.client.1.smithi144.stdout:9/636: write d0/d8/d5e/d44/d10/f58 [321809,116086] 0 2024-05-15T15:54:16.053 INFO:tasks.workunit.client.0.smithi049.stdout:5/545: readlink dc/d4a/l68 0 2024-05-15T15:54:16.061 INFO:tasks.workunit.client.1.smithi144.stdout:4/616: creat d6/dd/d34/d3b/d30/fc6 x:0 0 0 2024-05-15T15:54:16.062 INFO:tasks.workunit.client.0.smithi049.stdout:5/546: creat dc/d4a/d9b/fae x:0 0 0 2024-05-15T15:54:16.063 INFO:tasks.workunit.client.0.smithi049.stdout:5/547: chown dc/dd/d10/f1a 115 1 2024-05-15T15:54:16.064 INFO:tasks.workunit.client.0.smithi049.stdout:5/548: chown dc/d29/f7c 8 1 2024-05-15T15:54:16.065 INFO:tasks.workunit.client.0.smithi049.stdout:5/549: chown dc/dd/d7b/la2 2100923 1 2024-05-15T15:54:16.065 INFO:tasks.workunit.client.1.smithi144.stdout:0/602: dwrite d4/df/d1f/d24/f4b [0,4194304] 0 2024-05-15T15:54:16.066 INFO:tasks.workunit.client.0.smithi049.stdout:6/524: creat dd/d17/d24/d2b/d36/d50/f9a x:0 0 0 2024-05-15T15:54:16.067 INFO:tasks.workunit.client.1.smithi144.stdout:3/631: mknod d0/d11/d3b/cd3 0 2024-05-15T15:54:16.069 INFO:tasks.workunit.client.0.smithi049.stdout:7/487: creat d0/d6/da/d3d/d54/d5e/f96 x:0 0 0 2024-05-15T15:54:16.073 INFO:tasks.workunit.client.1.smithi144.stdout:2/576: sync 2024-05-15T15:54:16.073 INFO:tasks.workunit.client.1.smithi144.stdout:7/603: sync 2024-05-15T15:54:16.073 INFO:tasks.workunit.client.1.smithi144.stdout:1/607: sync 2024-05-15T15:54:16.074 INFO:tasks.workunit.client.1.smithi144.stdout:6/573: dwrite f6 [0,4194304] 0 2024-05-15T15:54:16.078 INFO:tasks.workunit.client.0.smithi049.stdout:6/525: mknod dd/c9b 0 2024-05-15T15:54:16.083 INFO:tasks.workunit.client.1.smithi144.stdout:9/637: rename d0/d8/d5e/d44/d10/d19/lad to d0/d8/d21/ld9 0 2024-05-15T15:54:16.086 INFO:tasks.workunit.client.1.smithi144.stdout:4/617: truncate d6/dd/d2e/d94/fa3 632248 0 2024-05-15T15:54:16.087 INFO:tasks.workunit.client.0.smithi049.stdout:4/561: write d18/d1e/d63/f96 [3160963,89070] 0 2024-05-15T15:54:16.087 INFO:tasks.workunit.client.0.smithi049.stdout:6/526: dwrite dd/f6b [0,4194304] 0 2024-05-15T15:54:16.088 INFO:tasks.workunit.client.1.smithi144.stdout:4/618: chown d6/d42/d46/d4b/f4d 0 1 2024-05-15T15:54:16.088 INFO:tasks.workunit.client.0.smithi049.stdout:6/527: chown dd/f32 0 1 2024-05-15T15:54:16.090 INFO:tasks.workunit.client.0.smithi049.stdout:6/528: read dd/d26/f4b [2648164,114047] 0 2024-05-15T15:54:16.091 INFO:tasks.workunit.client.1.smithi144.stdout:0/603: creat d4/df/d14/db5/fd4 x:0 0 0 2024-05-15T15:54:16.092 INFO:tasks.workunit.client.0.smithi049.stdout:5/550: dread dc/dd/d7b/f93 [0,4194304] 0 2024-05-15T15:54:16.093 INFO:tasks.workunit.client.0.smithi049.stdout:6/529: write dd/d17/d24/d2b/d36/d86/f95 [368758,120856] 0 2024-05-15T15:54:16.093 INFO:tasks.workunit.client.0.smithi049.stdout:4/562: mkdir d18/d1e/d63/d99/db9 0 2024-05-15T15:54:16.097 INFO:tasks.workunit.client.1.smithi144.stdout:1/608: creat d7/fc9 x:0 0 0 2024-05-15T15:54:16.097 INFO:tasks.workunit.client.0.smithi049.stdout:5/551: readlink dc/d4a/l4e 0 2024-05-15T15:54:16.098 INFO:tasks.workunit.client.1.smithi144.stdout:8/676: write d2/d3a/f7f [2916979,29137] 0 2024-05-15T15:54:16.099 INFO:tasks.workunit.client.0.smithi049.stdout:5/552: write dc/dd/d7b/f8f [46919,50724] 0 2024-05-15T15:54:16.099 INFO:tasks.workunit.client.1.smithi144.stdout:8/677: dread - d2/d24/d2d/da0/d6e/fc4 zero size 2024-05-15T15:54:16.099 INFO:tasks.workunit.client.1.smithi144.stdout:7/604: symlink d4/d3c/lca 0 2024-05-15T15:54:16.100 INFO:tasks.workunit.client.0.smithi049.stdout:5/553: stat dc/f1b 0 2024-05-15T15:54:16.103 INFO:tasks.workunit.client.0.smithi049.stdout:7/488: write d0/f15 [1336292,2919] 0 2024-05-15T15:54:16.104 INFO:tasks.workunit.client.0.smithi049.stdout:4/563: truncate d18/d21/d2b/f3f 106887 0 2024-05-15T15:54:16.105 INFO:tasks.workunit.client.0.smithi049.stdout:7/489: read d0/d6/d23/d45/f5a [53438,78230] 0 2024-05-15T15:54:16.106 INFO:tasks.workunit.client.1.smithi144.stdout:3/632: rename d0/l2b to d0/d11/d3b/dc7/ld4 0 2024-05-15T15:54:16.107 INFO:tasks.workunit.client.0.smithi049.stdout:5/554: creat dc/dd/d10/faf x:0 0 0 2024-05-15T15:54:16.107 INFO:tasks.workunit.client.1.smithi144.stdout:3/633: read d0/d11/d3b/d93/f86 [134316,90322] 0 2024-05-15T15:54:16.111 INFO:tasks.workunit.client.0.smithi049.stdout:7/490: creat d0/d2f/f97 x:0 0 0 2024-05-15T15:54:16.114 INFO:tasks.workunit.client.1.smithi144.stdout:8/678: unlink d2/d24/d2d/da0/d6e/d8d/l93 0 2024-05-15T15:54:16.116 INFO:tasks.workunit.client.1.smithi144.stdout:6/574: fdatasync d7/f68 0 2024-05-15T15:54:16.117 INFO:tasks.workunit.client.1.smithi144.stdout:6/575: chown d7/db/d9d/d97 43437 1 2024-05-15T15:54:16.118 INFO:tasks.workunit.client.0.smithi049.stdout:5/555: mkdir dc/db0 0 2024-05-15T15:54:16.119 INFO:tasks.workunit.client.1.smithi144.stdout:2/577: mkdir d3/d7/d20/d22/d52/d75/da6/dc1 0 2024-05-15T15:54:16.119 INFO:tasks.workunit.client.1.smithi144.stdout:0/604: dread d4/d2d/f44 [0,4194304] 0 2024-05-15T15:54:16.120 INFO:tasks.workunit.client.1.smithi144.stdout:2/578: truncate d3/d7/d20/f7c 1455367 0 2024-05-15T15:54:16.120 INFO:tasks.workunit.client.0.smithi049.stdout:7/491: unlink d0/d6/da/d3d/f5f 0 2024-05-15T15:54:16.122 INFO:tasks.workunit.client.0.smithi049.stdout:7/492: dread d0/d2f/d5c/f64 [0,4194304] 0 2024-05-15T15:54:16.122 INFO:tasks.workunit.client.0.smithi049.stdout:4/564: fsync d18/d1e/d2a/d36/f83 0 2024-05-15T15:54:16.124 INFO:tasks.workunit.client.1.smithi144.stdout:7/605: rename d4/d3c/d76/d9d/dac/d48/d72/d82/c97 to d4/d3c/d76/d9d/dac/ccb 0 2024-05-15T15:54:16.125 INFO:tasks.workunit.client.0.smithi049.stdout:5/556: creat dc/d4a/d9e/fb1 x:0 0 0 2024-05-15T15:54:16.133 INFO:tasks.workunit.client.0.smithi049.stdout:7/493: dread d0/f15 [0,4194304] 0 2024-05-15T15:54:16.189 INFO:tasks.workunit.client.1.smithi144.stdout:3/634: creat d0/d1/d17/d39/d40/d7e/fd5 x:0 0 0 2024-05-15T15:54:16.189 INFO:tasks.workunit.client.1.smithi144.stdout:3/635: readlink d0/d11/l47 0 2024-05-15T15:54:16.190 INFO:tasks.workunit.client.0.smithi049.stdout:2/544: sync 2024-05-15T15:54:16.192 INFO:tasks.workunit.client.1.smithi144.stdout:4/619: creat d6/dd/d34/d3b/d30/db/d1d/d40/d6c/d82/fc7 x:0 0 0 2024-05-15T15:54:16.193 INFO:tasks.workunit.client.0.smithi049.stdout:6/530: dwrite dd/f19 [4194304,4194304] 0 2024-05-15T15:54:16.194 INFO:tasks.workunit.client.1.smithi144.stdout:5/633: truncate d0/dc/d24/d28/f82 1306628 0 2024-05-15T15:54:16.195 INFO:tasks.workunit.client.1.smithi144.stdout:5/634: truncate d0/dc/d1f/f75 4003 0 2024-05-15T15:54:16.196 INFO:tasks.workunit.client.1.smithi144.stdout:5/635: fdatasync d0/dc/d1f/d50/f86 0 2024-05-15T15:54:16.198 INFO:tasks.workunit.client.1.smithi144.stdout:8/679: write d2/de/d36/d87/fda [4622784,25748] 0 2024-05-15T15:54:16.200 INFO:tasks.workunit.client.0.smithi049.stdout:7/494: rmdir d0/d34 39 2024-05-15T15:54:16.203 INFO:tasks.workunit.client.0.smithi049.stdout:4/565: mknod d18/cba 0 2024-05-15T15:54:16.207 INFO:tasks.workunit.client.1.smithi144.stdout:0/605: creat d4/d2d/d48/fd5 x:0 0 0 2024-05-15T15:54:16.208 INFO:tasks.workunit.client.1.smithi144.stdout:2/579: fsync d3/d7/d20/d22/d52/d64/f26 0 2024-05-15T15:54:16.209 INFO:tasks.workunit.client.1.smithi144.stdout:2/580: readlink d3/d7/d20/d22/d52/d64/lad 0 2024-05-15T15:54:16.210 INFO:tasks.workunit.client.1.smithi144.stdout:7/606: truncate d4/d41/d5b/f80 388619 0 2024-05-15T15:54:16.211 INFO:tasks.workunit.client.1.smithi144.stdout:9/638: sync 2024-05-15T15:54:16.211 INFO:tasks.workunit.client.0.smithi049.stdout:7/495: symlink d0/d2f/d5c/l98 0 2024-05-15T15:54:16.216 INFO:tasks.workunit.client.0.smithi049.stdout:4/566: symlink d18/d1e/lbb 0 2024-05-15T15:54:16.219 INFO:tasks.workunit.client.0.smithi049.stdout:7/496: mknod d0/d6/c99 0 2024-05-15T15:54:16.219 INFO:tasks.workunit.client.0.smithi049.stdout:5/557: write dc/dd/d39/f6e [3751916,76313] 0 2024-05-15T15:54:16.220 INFO:tasks.workunit.client.0.smithi049.stdout:7/497: fdatasync d0/d2f/f8e 0 2024-05-15T15:54:16.222 INFO:tasks.workunit.client.0.smithi049.stdout:7/498: truncate d0/d6/d1d/f7a 1404805 0 2024-05-15T15:54:16.223 INFO:tasks.workunit.client.0.smithi049.stdout:4/567: rmdir d18/d85/db6 39 2024-05-15T15:54:16.224 INFO:tasks.workunit.client.0.smithi049.stdout:4/568: write d18/d1e/d2a/f4f [3527552,49790] 0 2024-05-15T15:54:16.226 INFO:tasks.workunit.client.0.smithi049.stdout:7/499: dread d0/d2f/f51 [0,4194304] 0 2024-05-15T15:54:16.227 INFO:tasks.workunit.client.0.smithi049.stdout:5/558: fdatasync dc/d4a/d42/f74 0 2024-05-15T15:54:16.229 INFO:tasks.workunit.client.0.smithi049.stdout:5/559: fdatasync dc/d29/f87 0 2024-05-15T15:54:16.236 INFO:tasks.workunit.client.0.smithi049.stdout:7/500: unlink d0/d6/d23/l62 0 2024-05-15T15:54:16.244 INFO:tasks.workunit.client.0.smithi049.stdout:5/560: dwrite dc/d4a/d31/f82 [4194304,4194304] 0 2024-05-15T15:54:16.249 INFO:tasks.workunit.client.0.smithi049.stdout:7/501: creat d0/d6/da/f9a x:0 0 0 2024-05-15T15:54:16.251 INFO:tasks.workunit.client.0.smithi049.stdout:5/561: mknod dc/dd/d7b/cb2 0 2024-05-15T15:54:16.256 INFO:tasks.workunit.client.0.smithi049.stdout:5/562: mkdir dc/dd/d10/db3 0 2024-05-15T15:54:16.260 INFO:tasks.workunit.client.0.smithi049.stdout:7/502: dwrite d0/d2f/f97 [0,4194304] 0 2024-05-15T15:54:16.264 INFO:tasks.workunit.client.0.smithi049.stdout:4/569: dwrite d18/d1e/d2a/d50/f5a [0,4194304] 0 2024-05-15T15:54:16.264 INFO:tasks.workunit.client.0.smithi049.stdout:7/503: truncate d0/d6/f14 4865282 0 2024-05-15T15:54:16.266 INFO:tasks.workunit.client.1.smithi144.stdout:3/636: mknod d0/d1/d17/d39/d40/d4a/d77/cd6 0 2024-05-15T15:54:16.266 INFO:tasks.workunit.client.1.smithi144.stdout:3/637: chown d0/d1/c8 561125 1 2024-05-15T15:54:16.267 INFO:tasks.workunit.client.0.smithi049.stdout:5/563: symlink dc/d4a/d31/lb4 0 2024-05-15T15:54:16.272 INFO:tasks.workunit.client.1.smithi144.stdout:3/638: dwrite d0/d1/d17/d39/d40/f92 [0,4194304] 0 2024-05-15T15:54:16.273 INFO:tasks.workunit.client.0.smithi049.stdout:7/504: mknod d0/d6/d23/d4a/d7b/c9b 0 2024-05-15T15:54:16.275 INFO:tasks.workunit.client.0.smithi049.stdout:4/570: truncate d18/d1e/d63/f7f 963623 0 2024-05-15T15:54:16.276 INFO:tasks.workunit.client.1.smithi144.stdout:7/607: truncate d4/f46 343047 0 2024-05-15T15:54:16.278 INFO:tasks.workunit.client.0.smithi049.stdout:7/505: mknod d0/d2f/c9c 0 2024-05-15T15:54:16.279 INFO:tasks.workunit.client.0.smithi049.stdout:7/506: dread - d0/d6/d23/f91 zero size 2024-05-15T15:54:16.282 INFO:tasks.workunit.client.0.smithi049.stdout:4/571: mkdir d18/d1e/d2a/d50/dbc 0 2024-05-15T15:54:16.285 INFO:tasks.workunit.client.0.smithi049.stdout:7/507: fsync d0/d6/d1d/f4e 0 2024-05-15T15:54:16.288 INFO:tasks.workunit.client.1.smithi144.stdout:4/620: mkdir d6/d42/dc8 0 2024-05-15T15:54:16.289 INFO:tasks.workunit.client.1.smithi144.stdout:4/621: dread - d6/dd/d34/d3b/d30/fc6 zero size 2024-05-15T15:54:16.292 INFO:tasks.workunit.client.0.smithi049.stdout:4/572: dwrite d18/d85/fb5 [0,4194304] 0 2024-05-15T15:54:16.292 INFO:tasks.workunit.client.1.smithi144.stdout:1/609: dread d7/d1d/d33/d41/d78/da3/f54 [0,4194304] 0 2024-05-15T15:54:16.296 INFO:tasks.workunit.client.1.smithi144.stdout:5/636: sync 2024-05-15T15:54:16.301 INFO:tasks.workunit.client.1.smithi144.stdout:9/639: symlink d0/d8/d5e/d44/d10/d19/lda 0 2024-05-15T15:54:16.303 INFO:tasks.workunit.client.1.smithi144.stdout:5/637: dwrite d0/dc/d24/d8e/f8a [4194304,4194304] 0 2024-05-15T15:54:16.305 INFO:tasks.workunit.client.0.smithi049.stdout:7/508: rename d0/d6/d23/f30 to d0/d6/d23/d45/d6a/f9d 0 2024-05-15T15:54:16.307 INFO:tasks.workunit.client.1.smithi144.stdout:9/640: dwrite d0/d8/d5e/d44/d10/f2c [0,4194304] 0 2024-05-15T15:54:16.309 INFO:tasks.workunit.client.0.smithi049.stdout:4/573: mkdir d18/d1e/d2a/d36/d86/dbd 0 2024-05-15T15:54:16.314 INFO:tasks.workunit.client.1.smithi144.stdout:9/641: dwrite d0/d8/d7a/fd0 [0,4194304] 0 2024-05-15T15:54:16.314 INFO:tasks.workunit.client.1.smithi144.stdout:6/576: dwrite d7/db/d35/d84/d81/f83 [0,4194304] 0 2024-05-15T15:54:16.315 INFO:tasks.workunit.client.1.smithi144.stdout:6/577: chown d7/db/d11/l16 1 1 2024-05-15T15:54:16.320 INFO:tasks.workunit.client.0.smithi049.stdout:5/564: dwrite dc/d4a/f91 [0,4194304] 0 2024-05-15T15:54:16.322 INFO:tasks.workunit.client.0.smithi049.stdout:5/565: write dc/d4a/f57 [1266914,39256] 0 2024-05-15T15:54:16.323 INFO:tasks.workunit.client.0.smithi049.stdout:5/566: truncate dc/dd/f88 4357912 0 2024-05-15T15:54:16.324 INFO:tasks.workunit.client.0.smithi049.stdout:5/567: chown dc/f7f 12 1 2024-05-15T15:54:16.326 INFO:tasks.workunit.client.1.smithi144.stdout:1/610: read d7/d1d/d33/d1e/d34/f38 [523638,69112] 0 2024-05-15T15:54:16.329 INFO:tasks.workunit.client.1.smithi144.stdout:3/639: mkdir d0/d1/d17/dae/dd7 0 2024-05-15T15:54:16.330 INFO:tasks.workunit.client.1.smithi144.stdout:6/578: dwrite d7/d1e/d28/fab [0,4194304] 0 2024-05-15T15:54:16.333 INFO:tasks.workunit.client.1.smithi144.stdout:0/606: dwrite d4/d6/d3a/f91 [0,4194304] 0 2024-05-15T15:54:16.334 INFO:tasks.workunit.client.1.smithi144.stdout:3/640: write d0/d11/d24/f4f [428343,53204] 0 2024-05-15T15:54:16.337 INFO:tasks.workunit.client.0.smithi049.stdout:5/568: mknod dc/d4a/d42/cb5 0 2024-05-15T15:54:16.342 INFO:tasks.workunit.client.0.smithi049.stdout:4/574: link d18/d1e/d2a/d36/d86/la1 d18/d1e/d2a/d50/dbc/lbe 0 2024-05-15T15:54:16.343 INFO:tasks.workunit.client.0.smithi049.stdout:4/575: write d18/d85/fab [596529,24065] 0 2024-05-15T15:54:16.344 INFO:tasks.workunit.client.1.smithi144.stdout:9/642: mkdir d0/d8/d1b/d79/d8a/dae/ddb 0 2024-05-15T15:54:16.345 INFO:tasks.workunit.client.1.smithi144.stdout:7/608: dwrite d4/df/d25/f29 [0,4194304] 0 2024-05-15T15:54:16.346 INFO:tasks.workunit.client.0.smithi049.stdout:5/569: unlink dc/dd/d39/f56 0 2024-05-15T15:54:16.347 INFO:tasks.workunit.client.1.smithi144.stdout:7/609: write d4/d41/d5b/f88 [1499534,130567] 0 2024-05-15T15:54:16.349 INFO:tasks.workunit.client.1.smithi144.stdout:8/680: sync 2024-05-15T15:54:16.349 INFO:tasks.workunit.client.1.smithi144.stdout:8/681: chown d2/de/d36 5464210 1 2024-05-15T15:54:16.350 INFO:tasks.workunit.client.0.smithi049.stdout:5/570: creat dc/d29/fb6 x:0 0 0 2024-05-15T15:54:16.351 INFO:tasks.workunit.client.0.smithi049.stdout:4/576: mknod d18/d1e/d2a/d50/d54/dad/cbf 0 2024-05-15T15:54:16.352 INFO:tasks.workunit.client.1.smithi144.stdout:3/641: dread d0/d1/f43 [0,4194304] 0 2024-05-15T15:54:16.354 INFO:tasks.workunit.client.1.smithi144.stdout:1/611: write d7/d1d/d33/d29/d6b/d5e/f9d [231761,88200] 0 2024-05-15T15:54:16.355 INFO:tasks.workunit.client.1.smithi144.stdout:7/610: dwrite d4/fa [0,4194304] 0 2024-05-15T15:54:16.359 INFO:tasks.workunit.client.0.smithi049.stdout:7/509: write d0/f15 [3057542,109759] 0 2024-05-15T15:54:16.360 INFO:tasks.workunit.client.1.smithi144.stdout:6/579: symlink d7/db/d9d/d72/lcb 0 2024-05-15T15:54:16.360 INFO:tasks.workunit.client.0.smithi049.stdout:7/510: chown d0/d2f/d5c/f7d 5 1 2024-05-15T15:54:16.361 INFO:tasks.workunit.client.1.smithi144.stdout:6/580: write d7/d62/f6b [1330571,114264] 0 2024-05-15T15:54:16.362 INFO:tasks.workunit.client.1.smithi144.stdout:6/581: write d7/db/d3a/f91 [1912694,26034] 0 2024-05-15T15:54:16.363 INFO:tasks.workunit.client.1.smithi144.stdout:6/582: readlink d7/d1e/l89 0 2024-05-15T15:54:16.364 INFO:tasks.workunit.client.1.smithi144.stdout:0/607: write d4/d6/d4a/d74/fc3 [3838621,42789] 0 2024-05-15T15:54:16.366 INFO:tasks.workunit.client.0.smithi049.stdout:5/571: creat dc/fb7 x:0 0 0 2024-05-15T15:54:16.367 INFO:tasks.workunit.client.1.smithi144.stdout:2/581: link d3/d7/l4b d3/d7/d20/d40/lc2 0 2024-05-15T15:54:16.367 INFO:tasks.workunit.client.1.smithi144.stdout:4/622: dwrite d6/dd/d34/d3b/d30/d59/d66/f9b [0,4194304] 0 2024-05-15T15:54:16.368 INFO:tasks.workunit.client.0.smithi049.stdout:5/572: chown dc/d4a/d31/l37 17976386 1 2024-05-15T15:54:16.368 INFO:tasks.workunit.client.0.smithi049.stdout:7/511: creat d0/d6/da/d3d/d54/d5e/f9e x:0 0 0 2024-05-15T15:54:16.372 INFO:tasks.workunit.client.0.smithi049.stdout:7/512: readlink d0/d6/l43 0 2024-05-15T15:54:16.377 INFO:tasks.workunit.client.1.smithi144.stdout:3/642: creat d0/d1/d17/fd8 x:0 0 0 2024-05-15T15:54:16.382 INFO:tasks.workunit.client.1.smithi144.stdout:1/612: rmdir d7/d1d/d33/d1e 39 2024-05-15T15:54:16.382 INFO:tasks.workunit.client.1.smithi144.stdout:1/613: readlink d7/l27 0 2024-05-15T15:54:16.383 INFO:tasks.workunit.client.0.smithi049.stdout:5/573: dread dc/d4a/f50 [0,4194304] 0 2024-05-15T15:54:16.384 INFO:tasks.workunit.client.0.smithi049.stdout:7/513: symlink d0/d6/d23/d45/d86/l9f 0 2024-05-15T15:54:16.390 INFO:tasks.workunit.client.0.smithi049.stdout:7/514: dwrite d0/f2e [0,4194304] 0 2024-05-15T15:54:16.391 INFO:tasks.workunit.client.1.smithi144.stdout:6/583: chown d7/c7f 487319279 1 2024-05-15T15:54:16.392 INFO:tasks.workunit.client.0.smithi049.stdout:0/534: sync 2024-05-15T15:54:16.392 INFO:tasks.workunit.client.0.smithi049.stdout:5/574: symlink dc/d4a/lb8 0 2024-05-15T15:54:16.392 INFO:tasks.workunit.client.1.smithi144.stdout:6/584: write d7/db/d35/d84/d81/fca [825084,9928] 0 2024-05-15T15:54:16.392 INFO:tasks.workunit.client.1.smithi144.stdout:6/585: chown d7/da6/faa 128262586 1 2024-05-15T15:54:16.396 INFO:tasks.workunit.client.0.smithi049.stdout:7/515: symlink d0/d6/da/d3d/d80/la0 0 2024-05-15T15:54:16.397 INFO:tasks.workunit.client.0.smithi049.stdout:4/577: write d18/d1e/d2a/d3b/f47 [938111,103640] 0 2024-05-15T15:54:16.402 INFO:tasks.workunit.client.1.smithi144.stdout:0/608: rename d4/df/d1f/d24/d2b to d4/d6/d3a/d4c/dd6 0 2024-05-15T15:54:16.404 INFO:tasks.workunit.client.1.smithi144.stdout:4/623: mkdir d6/dd/d34/d3b/d30/dc9 0 2024-05-15T15:54:16.406 INFO:tasks.workunit.client.0.smithi049.stdout:0/535: dread dc/d13/d14/d25/d31/f7e [0,4194304] 0 2024-05-15T15:54:16.409 INFO:tasks.workunit.client.0.smithi049.stdout:0/536: creat dc/fa1 x:0 0 0 2024-05-15T15:54:16.411 INFO:tasks.workunit.client.1.smithi144.stdout:9/643: dwrite d0/f78 [0,4194304] 0 2024-05-15T15:54:16.411 INFO:tasks.workunit.client.1.smithi144.stdout:3/643: write d0/d1/d17/d39/d40/d7e/f81 [559073,50524] 0 2024-05-15T15:54:16.412 INFO:tasks.workunit.client.1.smithi144.stdout:5/638: getdents d0/d7e 0 2024-05-15T15:54:16.414 INFO:tasks.workunit.client.1.smithi144.stdout:6/586: mkdir d7/db/d11/d3c/d61/dcc 0 2024-05-15T15:54:16.414 INFO:tasks.workunit.client.1.smithi144.stdout:8/682: dwrite d2/de/d36/f4d [0,4194304] 0 2024-05-15T15:54:16.416 INFO:tasks.workunit.client.1.smithi144.stdout:5/639: chown d0/dc/d24/d8e/d95/cc8 7 1 2024-05-15T15:54:16.417 INFO:tasks.workunit.client.1.smithi144.stdout:2/582: rename d3/l44 to d3/d10/lc3 0 2024-05-15T15:54:16.417 INFO:tasks.workunit.client.1.smithi144.stdout:6/587: chown d7/db/d11 334196 1 2024-05-15T15:54:16.419 INFO:tasks.workunit.client.0.smithi049.stdout:7/516: rename d0/d6/da/d3d to d0/d6/d23/d45/d6a/da1 0 2024-05-15T15:54:16.420 INFO:tasks.workunit.client.1.smithi144.stdout:0/609: mkdir d4/d6/dd7 0 2024-05-15T15:54:16.420 INFO:tasks.workunit.client.1.smithi144.stdout:5/640: dread - d0/dc/d2f/d34/d63/d6e/dbf/f4c zero size 2024-05-15T15:54:16.421 INFO:tasks.workunit.client.1.smithi144.stdout:5/641: write d0/dc/d24/d8e/d93/f89 [401825,6628] 0 2024-05-15T15:54:16.421 INFO:tasks.workunit.client.1.smithi144.stdout:4/624: chown d6/dd/d2e/c37 6 1 2024-05-15T15:54:16.422 INFO:tasks.workunit.client.0.smithi049.stdout:7/517: write d0/d6/d23/f91 [730717,94471] 0 2024-05-15T15:54:16.423 INFO:tasks.workunit.client.0.smithi049.stdout:5/575: write dc/dd/d39/f4c [561016,74310] 0 2024-05-15T15:54:16.427 INFO:tasks.workunit.client.0.smithi049.stdout:0/537: dread dc/d13/f22 [0,4194304] 0 2024-05-15T15:54:16.429 INFO:tasks.workunit.client.0.smithi049.stdout:4/578: rename d18/d1e/f8e to d18/d69/fc0 0 2024-05-15T15:54:16.434 INFO:tasks.workunit.client.1.smithi144.stdout:6/588: dread d7/db/f2c [0,4194304] 0 2024-05-15T15:54:16.434 INFO:tasks.workunit.client.1.smithi144.stdout:8/683: creat d2/d24/d2d/d30/fe0 x:0 0 0 2024-05-15T15:54:16.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.436+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd30804ec60 con 0x7fd308100dd0 2024-05-15T15:54:16.439 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:16 smithi049 ceph-mon[28399]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:16.439 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:16 smithi049 ceph-mon[28399]: pgmap v28: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 64 MiB/s rd, 188 MiB/s wr, 368 op/s 2024-05-15T15:54:16.439 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:16 smithi049 ceph-mon[28399]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:16.440 INFO:tasks.workunit.client.0.smithi049.stdout:0/538: rename dc/d13/d14/d82 to dc/d13/d2b/d76/d8a/da2 0 2024-05-15T15:54:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.437+0000 7fd2fe7fc700 1 -- 172.21.15.49:0/193574544 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+841 (secure 0 0 0) 0x7fd2f805b140 con 0x7fd308100dd0 2024-05-15T15:54:16.440 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:54:16.440 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:54:16.440 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:54:16.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 1 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 13, 2024-05-15T15:54:16.441 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 1 2024-05-15T15:54:16.442 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:54:16.442 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:54:16.442 INFO:tasks.workunit.client.1.smithi144.stdout:0/610: chown d4/d6/c1b 3482249 1 2024-05-15T15:54:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.440+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fd2f406f8f0 msgr2=0x7fd2f4071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.440+0000 7fd30f5e3700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fd2f406f8f0 0x7fd2f4071db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd308100c30 tx=0x7fd30400b410 comp rx=0 tx=0).stop 2024-05-15T15:54:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.440+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 msgr2=0x7fd308194a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.440+0000 7fd30f5e3700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 0x7fd308194a10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80092e0 tx=0x7fd2f800bd70 comp rx=0 tx=0).stop 2024-05-15T15:54:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.440+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 shutdown_connections 2024-05-15T15:54:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.440+0000 7fd30f5e3700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fd2f406f8f0 0x7fd2f4071db0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:16.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.440+0000 7fd30f5e3700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3080ffbd0 0x7fd3081944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:16.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.440+0000 7fd30f5e3700 1 --2- 172.21.15.49:0/193574544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd308100dd0 0x7fd308194a10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:16.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.440+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 >> 172.21.15.49:0/193574544 conn(0x7fd3080fb170 msgr2=0x7fd3081043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:16.444 INFO:tasks.workunit.client.0.smithi049.stdout:7/518: mknod d0/d34/ca2 0 2024-05-15T15:54:16.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.441+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 shutdown_connections 2024-05-15T15:54:16.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.441+0000 7fd30f5e3700 1 -- 172.21.15.49:0/193574544 wait complete. 2024-05-15T15:54:16.445 INFO:tasks.workunit.client.0.smithi049.stdout:4/579: dread fa [0,4194304] 0 2024-05-15T15:54:16.447 INFO:tasks.workunit.client.0.smithi049.stdout:5/576: mknod dc/d4a/cb9 0 2024-05-15T15:54:16.447 INFO:tasks.workunit.client.0.smithi049.stdout:5/577: fdatasync dc/f80 0 2024-05-15T15:54:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:16 smithi144 ceph-mon[36389]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:16.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:16 smithi144 ceph-mon[36389]: pgmap v28: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 64 MiB/s rd, 188 MiB/s wr, 368 op/s 2024-05-15T15:54:16.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:16 smithi144 ceph-mon[36389]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:16.449 INFO:tasks.workunit.client.0.smithi049.stdout:5/578: chown dc/d4a/f91 2933262 1 2024-05-15T15:54:16.449 INFO:tasks.workunit.client.0.smithi049.stdout:7/519: dwrite d0/f2e [0,4194304] 0 2024-05-15T15:54:16.451 INFO:tasks.workunit.client.1.smithi144.stdout:7/611: getdents d4/d45 0 2024-05-15T15:54:16.452 INFO:tasks.workunit.client.1.smithi144.stdout:1/614: sync 2024-05-15T15:54:16.454 INFO:tasks.workunit.client.0.smithi049.stdout:0/539: symlink dc/d13/d2b/d76/la3 0 2024-05-15T15:54:16.455 INFO:tasks.workunit.client.0.smithi049.stdout:0/540: chown dc/d13/d2b/c35 17252026 1 2024-05-15T15:54:16.457 INFO:tasks.workunit.client.1.smithi144.stdout:6/589: symlink d7/d62/lcd 0 2024-05-15T15:54:16.459 INFO:tasks.workunit.client.1.smithi144.stdout:9/644: dwrite d0/d8/d5e/d44/d10/d19/f63 [0,4194304] 0 2024-05-15T15:54:16.460 INFO:tasks.workunit.client.0.smithi049.stdout:5/579: unlink dc/d4a/d42/f74 0 2024-05-15T15:54:16.461 INFO:tasks.workunit.client.1.smithi144.stdout:9/645: chown d0/d8/d5e/d44/d10/d19/c7b 1116832 1 2024-05-15T15:54:16.463 INFO:tasks.workunit.client.1.smithi144.stdout:8/684: rmdir d2/d3a/d7b 39 2024-05-15T15:54:16.465 INFO:tasks.workunit.client.1.smithi144.stdout:2/583: write d3/d7/d20/d37/f70 [804268,97905] 0 2024-05-15T15:54:16.468 INFO:tasks.workunit.client.0.smithi049.stdout:0/541: mkdir dc/da4 0 2024-05-15T15:54:16.469 INFO:tasks.workunit.client.0.smithi049.stdout:5/580: mkdir dc/d4a/d31/da9/dba 0 2024-05-15T15:54:16.471 INFO:tasks.workunit.client.1.smithi144.stdout:7/612: creat d4/df/d79/fcc x:0 0 0 2024-05-15T15:54:16.472 INFO:tasks.workunit.client.1.smithi144.stdout:7/613: write d4/df/fb7 [877332,24946] 0 2024-05-15T15:54:16.473 INFO:tasks.workunit.client.0.smithi049.stdout:0/542: symlink dc/d13/d14/d33/la5 0 2024-05-15T15:54:16.475 INFO:tasks.workunit.client.1.smithi144.stdout:1/615: creat d7/d1d/d33/d29/d6b/d8e/fca x:0 0 0 2024-05-15T15:54:16.476 INFO:tasks.workunit.client.0.smithi049.stdout:5/581: creat dc/dd/d7b/d8d/fbb x:0 0 0 2024-05-15T15:54:16.478 INFO:tasks.workunit.client.0.smithi049.stdout:5/582: fsync dc/d4a/d31/f95 0 2024-05-15T15:54:16.478 INFO:tasks.workunit.client.1.smithi144.stdout:7/614: dwrite d4/f12 [4194304,4194304] 0 2024-05-15T15:54:16.478 INFO:tasks.workunit.client.1.smithi144.stdout:7/615: chown d4/d3c/d76/d9d/dac/f23 10716401 1 2024-05-15T15:54:16.484 INFO:tasks.workunit.client.1.smithi144.stdout:3/644: rename d0/d1/d17/f84 to d0/d1/fd9 0 2024-05-15T15:54:16.484 INFO:tasks.workunit.client.0.smithi049.stdout:7/520: write d0/f9 [7041142,47108] 0 2024-05-15T15:54:16.487 INFO:tasks.workunit.client.1.smithi144.stdout:9/646: creat d0/d8/d5e/d9b/fdc x:0 0 0 2024-05-15T15:54:16.488 INFO:tasks.workunit.client.1.smithi144.stdout:9/647: dread - d0/d8/d1b/d62/d7d/d8d/faf zero size 2024-05-15T15:54:16.488 INFO:tasks.workunit.client.1.smithi144.stdout:9/648: chown d0/d8/d1b/d62/f8e 0 1 2024-05-15T15:54:16.492 INFO:tasks.workunit.client.0.smithi049.stdout:7/521: dread d0/d2f/d5c/f70 [0,4194304] 0 2024-05-15T15:54:16.492 INFO:tasks.workunit.client.1.smithi144.stdout:8/685: read d2/de/f14 [381951,21101] 0 2024-05-15T15:54:16.493 INFO:tasks.workunit.client.1.smithi144.stdout:8/686: write d2/d24/d2d/da0/d6e/f62 [2544052,4413] 0 2024-05-15T15:54:16.494 INFO:tasks.workunit.client.0.smithi049.stdout:7/522: write d0/d6/d23/d45/d6a/da1/d39/f5b [3768560,67485] 0 2024-05-15T15:54:16.494 INFO:tasks.workunit.client.0.smithi049.stdout:0/543: write dc/d13/d3d/f43 [2142002,15684] 0 2024-05-15T15:54:16.496 INFO:tasks.workunit.client.0.smithi049.stdout:5/583: mknod dc/dd/d10/db3/cbc 0 2024-05-15T15:54:16.497 INFO:tasks.workunit.client.1.smithi144.stdout:1/616: dread d7/df/f9f [0,4194304] 0 2024-05-15T15:54:16.497 INFO:tasks.workunit.client.0.smithi049.stdout:5/584: readlink dc/dd/d10/l25 0 2024-05-15T15:54:16.500 INFO:tasks.workunit.client.0.smithi049.stdout:5/585: readlink dc/dd/l28 0 2024-05-15T15:54:16.501 INFO:tasks.workunit.client.0.smithi049.stdout:5/586: write dc/f9a [3231384,82306] 0 2024-05-15T15:54:16.504 INFO:tasks.workunit.client.1.smithi144.stdout:4/625: rename d6/d42/d46/d4b/d1e/cb3 to d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/cca 0 2024-05-15T15:54:16.504 INFO:tasks.workunit.client.1.smithi144.stdout:4/626: readlink d6/dd/l16 0 2024-05-15T15:54:16.505 INFO:tasks.workunit.client.1.smithi144.stdout:4/627: write d6/dd/d34/d3b/d30/d59/d66/f9b [3251112,102502] 0 2024-05-15T15:54:16.506 INFO:tasks.workunit.client.1.smithi144.stdout:4/628: read d6/d7c/f8c [3062753,10899] 0 2024-05-15T15:54:16.509 INFO:tasks.workunit.client.1.smithi144.stdout:9/649: rmdir d0/d8/d5e/d44/d16/db0 39 2024-05-15T15:54:16.509 INFO:tasks.workunit.client.0.smithi049.stdout:7/523: dread d0/d6/d1d/f27 [0,4194304] 0 2024-05-15T15:54:16.510 INFO:tasks.workunit.client.0.smithi049.stdout:5/587: fsync dc/dd/d10/f1a 0 2024-05-15T15:54:16.514 INFO:tasks.workunit.client.1.smithi144.stdout:5/642: link d0/c55 d0/dc/d24/d57/cd3 0 2024-05-15T15:54:16.516 INFO:tasks.workunit.client.1.smithi144.stdout:7/616: dwrite d4/df/fb1 [0,4194304] 0 2024-05-15T15:54:16.518 INFO:tasks.workunit.client.0.smithi049.stdout:7/524: link d0/f2e d0/d6/d23/d45/d6a/da1/d39/fa3 0 2024-05-15T15:54:16.521 INFO:tasks.workunit.client.0.smithi049.stdout:7/525: dread d0/d2f/d5c/f70 [0,4194304] 0 2024-05-15T15:54:16.523 INFO:tasks.workunit.client.0.smithi049.stdout:7/526: chown d0/d6/d23/d45/d6a/da1/d54/d5e/f96 842 1 2024-05-15T15:54:16.526 INFO:tasks.workunit.client.0.smithi049.stdout:5/588: getdents dc/d4a/d31/da9 0 2024-05-15T15:54:16.529 INFO:tasks.workunit.client.0.smithi049.stdout:0/544: dread dc/d13/d3d/f43 [0,4194304] 0 2024-05-15T15:54:16.531 INFO:tasks.workunit.client.1.smithi144.stdout:7/617: unlink d4/d3c/d76/d9d/fa4 0 2024-05-15T15:54:16.533 INFO:tasks.workunit.client.0.smithi049.stdout:5/589: rename dc/d4a/d42/ca4 to dc/db0/cbd 0 2024-05-15T15:54:16.536 INFO:tasks.workunit.client.1.smithi144.stdout:7/618: dwrite d4/f12 [4194304,4194304] 0 2024-05-15T15:54:16.542 INFO:tasks.workunit.client.1.smithi144.stdout:5/643: dread d0/dc/d2f/d34/d63/d6e/f7c [0,4194304] 0 2024-05-15T15:54:16.543 INFO:tasks.workunit.client.1.smithi144.stdout:7/619: dwrite d4/d3c/d76/d9d/fa5 [0,4194304] 0 2024-05-15T15:54:16.547 INFO:tasks.workunit.client.0.smithi049.stdout:7/527: dread d0/d34/f46 [0,4194304] 0 2024-05-15T15:54:16.555 INFO:tasks.workunit.client.0.smithi049.stdout:7/528: dwrite d0/f90 [0,4194304] 0 2024-05-15T15:54:16.556 INFO:tasks.workunit.client.1.smithi144.stdout:0/611: sync 2024-05-15T15:54:16.557 INFO:tasks.workunit.client.0.smithi049.stdout:7/529: write d0/d2f/f8e [3996697,76266] 0 2024-05-15T15:54:16.574 INFO:tasks.workunit.client.1.smithi144.stdout:7/620: dread d4/d3c/d76/d9d/dac/d48/d72/d82/f89 [0,4194304] 0 2024-05-15T15:54:16.574 INFO:tasks.workunit.client.1.smithi144.stdout:6/590: sync 2024-05-15T15:54:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.599+0000 7fbb40f72700 1 -- 172.21.15.49:0/2314362039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 msgr2=0x7fbb3c106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.599+0000 7fbb40f72700 1 --2- 172.21.15.49:0/2314362039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 0x7fbb3c106500 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009910 tx=0x7fbb30009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.599+0000 7fbb40f72700 1 -- 172.21.15.49:0/2314362039 shutdown_connections 2024-05-15T15:54:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.599+0000 7fbb40f72700 1 --2- 172.21.15.49:0/2314362039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 0x7fbb3c106500 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.599+0000 7fbb40f72700 1 --2- 172.21.15.49:0/2314362039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb3c1017e0 0x7fbb3c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.599+0000 7fbb40f72700 1 -- 172.21.15.49:0/2314362039 >> 172.21.15.49:0/2314362039 conn(0x7fbb3c0fb1d0 msgr2=0x7fbb3c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.599+0000 7fbb40f72700 1 -- 172.21.15.49:0/2314362039 shutdown_connections 2024-05-15T15:54:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.599+0000 7fbb40f72700 1 -- 172.21.15.49:0/2314362039 wait complete. 2024-05-15T15:54:16.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb40f72700 1 Processor -- start 2024-05-15T15:54:16.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb40f72700 1 -- start start 2024-05-15T15:54:16.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb40f72700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb3c1017e0 0x7fbb3c1010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:16.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb40f72700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 0x7fbb3c0ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:16.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb40f72700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0ffcd0 con 0x7fbb3c104110 2024-05-15T15:54:16.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb40f72700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0ffe10 con 0x7fbb3c1017e0 2024-05-15T15:54:16.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb39d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 0x7fbb3c0ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb39d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 0x7fbb3c0ff790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54826/0 (socket says 172.21.15.49:54826) 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb39d9b700 1 -- 172.21.15.49:0/1237110665 learned_addr learned my addr 172.21.15.49:0/1237110665 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb3a59c700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb3c1017e0 0x7fbb3c1010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb39d9b700 1 -- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb3c1017e0 msgr2=0x7fbb3c1010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb39d9b700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb3c1017e0 0x7fbb3c1010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb39d9b700 1 -- 172.21.15.49:0/1237110665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb300095f0 con 0x7fbb3c104110 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.600+0000 7fbb3a59c700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb3c1017e0 0x7fbb3c1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.601+0000 7fbb39d9b700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 0x7fbb3c0ff790 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb30006010 tx=0x7fbb3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.601+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/1237110665 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb3001d070 con 0x7fbb3c104110 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.601+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/1237110665 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb3000f460 con 0x7fbb3c104110 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.601+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c100090 con 0x7fbb3c104110 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.601+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/1237110665 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb30021640 con 0x7fbb3c104110 2024-05-15T15:54:16.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.601+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c100600 con 0x7fbb3c104110 2024-05-15T15:54:16.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.602+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c04ec60 con 0x7fbb3c104110 2024-05-15T15:54:16.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.602+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/1237110665 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7fbb3000fab0 con 0x7fbb3c104110 2024-05-15T15:54:16.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.603+0000 7fbb2b7fe700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fbb2406d6f0 0x7fbb2406fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:16.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.603+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/1237110665 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbb3008e3d0 con 0x7fbb3c104110 2024-05-15T15:54:16.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.603+0000 7fbb3a59c700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fbb2406d6f0 0x7fbb2406fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:16.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.604+0000 7fbb3a59c700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fbb2406d6f0 0x7fbb2406fbb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007900 tx=0x7fbb2c008040 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:16.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.608+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/1237110665 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fbb3005b640 con 0x7fbb3c104110 2024-05-15T15:54:16.614 INFO:tasks.workunit.client.1.smithi144.stdout:1/617: unlink d7/d4b/cae 0 2024-05-15T15:54:16.621 INFO:tasks.workunit.client.1.smithi144.stdout:4/629: dwrite d6/d42/d46/d4b/f4d [0,4194304] 0 2024-05-15T15:54:16.625 INFO:tasks.workunit.client.1.smithi144.stdout:9/650: truncate d0/d8/d5e/fbc 889665 0 2024-05-15T15:54:16.626 INFO:tasks.workunit.client.1.smithi144.stdout:2/584: getdents d3/d7/d20/d37 0 2024-05-15T15:54:16.627 INFO:tasks.workunit.client.1.smithi144.stdout:8/687: link d2/d24/d2d/c4f d2/d24/d2d/da0/d6e/d8d/ce1 0 2024-05-15T15:54:16.629 INFO:tasks.workunit.client.0.smithi049.stdout:0/545: dwrite dc/d13/f26 [0,4194304] 0 2024-05-15T15:54:16.629 INFO:tasks.workunit.client.0.smithi049.stdout:5/590: mkdir dc/d4a/d31/da9/dba/dbe 0 2024-05-15T15:54:16.631 INFO:tasks.workunit.client.0.smithi049.stdout:5/591: truncate dc/dd/d39/fa0 495830 0 2024-05-15T15:54:16.632 INFO:tasks.workunit.client.1.smithi144.stdout:9/651: dwrite d0/d8/d5e/d9b/f9f [0,4194304] 0 2024-05-15T15:54:16.634 INFO:tasks.workunit.client.0.smithi049.stdout:0/546: mknod dc/d13/d2b/d76/ca6 0 2024-05-15T15:54:16.636 INFO:tasks.workunit.client.1.smithi144.stdout:9/652: read d0/d8/d1b/d62/d6b/fa7 [3201574,12361] 0 2024-05-15T15:54:16.636 INFO:tasks.workunit.client.1.smithi144.stdout:9/653: chown d0/d8/d5e 63348 1 2024-05-15T15:54:16.638 INFO:tasks.workunit.client.0.smithi049.stdout:5/592: unlink dc/f3e 0 2024-05-15T15:54:16.640 INFO:tasks.workunit.client.1.smithi144.stdout:0/612: creat d4/d6/d3a/d4c/dd6/fd8 x:0 0 0 2024-05-15T15:54:16.642 INFO:tasks.workunit.client.0.smithi049.stdout:0/547: creat dc/d13/d14/d2f/d8f/fa7 x:0 0 0 2024-05-15T15:54:16.645 INFO:tasks.workunit.client.0.smithi049.stdout:7/530: write d0/d6/d23/f40 [51475,21243] 0 2024-05-15T15:54:16.648 INFO:tasks.workunit.client.0.smithi049.stdout:5/593: mkdir dc/db0/dbf 0 2024-05-15T15:54:16.649 INFO:tasks.workunit.client.1.smithi144.stdout:3/645: getdents d0/d1/d17/d39 0 2024-05-15T15:54:16.650 INFO:tasks.workunit.client.1.smithi144.stdout:4/630: dread - d6/d42/d46/d4b/d1e/d44/f81 zero size 2024-05-15T15:54:16.651 INFO:tasks.workunit.client.1.smithi144.stdout:4/631: fdatasync d6/dd/d34/d3b/d30/db/d1d/d40/f79 0 2024-05-15T15:54:16.654 INFO:tasks.workunit.client.0.smithi049.stdout:5/594: creat dc/dd/d47/d83/fc0 x:0 0 0 2024-05-15T15:54:16.655 INFO:tasks.workunit.client.0.smithi049.stdout:7/531: dread d0/d34/f3f [0,4194304] 0 2024-05-15T15:54:16.656 INFO:tasks.workunit.client.0.smithi049.stdout:7/532: chown d0/d2f/f97 51 1 2024-05-15T15:54:16.657 INFO:tasks.workunit.client.0.smithi049.stdout:7/533: fdatasync d0/d6/d23/d45/d6a/da1/d54/d5e/f9e 0 2024-05-15T15:54:16.658 INFO:tasks.workunit.client.0.smithi049.stdout:0/548: symlink dc/d13/d3d/d84/la8 0 2024-05-15T15:54:16.662 INFO:tasks.workunit.client.0.smithi049.stdout:0/549: symlink dc/d13/d2b/d76/d8a/la9 0 2024-05-15T15:54:16.665 INFO:tasks.workunit.client.0.smithi049.stdout:5/595: creat dc/d4a/d31/da9/dba/dbe/fc1 x:0 0 0 2024-05-15T15:54:16.667 INFO:tasks.workunit.client.1.smithi144.stdout:2/585: dwrite d3/d10/f7e [0,4194304] 0 2024-05-15T15:54:16.669 INFO:tasks.workunit.client.0.smithi049.stdout:0/550: rename dc/d13/d3d/f3f to dc/d32/faa 0 2024-05-15T15:54:16.670 INFO:tasks.workunit.client.0.smithi049.stdout:0/551: fsync dc/d13/d2b/f98 0 2024-05-15T15:54:16.671 INFO:tasks.workunit.client.1.smithi144.stdout:9/654: truncate d0/d8/d1b/f6f 3482712 0 2024-05-15T15:54:16.673 INFO:tasks.workunit.client.1.smithi144.stdout:9/655: chown d0/d8/d5e/d44/dc2/f45 0 1 2024-05-15T15:54:16.675 INFO:tasks.workunit.client.1.smithi144.stdout:2/586: dwrite d3/d7/d20/d37/f70 [0,4194304] 0 2024-05-15T15:54:16.677 INFO:tasks.workunit.client.1.smithi144.stdout:2/587: write d3/d7/d20/d22/d52/d75/d65/f9f [428018,4519] 0 2024-05-15T15:54:16.677 INFO:tasks.workunit.client.0.smithi049.stdout:7/534: dread d0/f4 [4194304,4194304] 0 2024-05-15T15:54:16.680 INFO:tasks.workunit.client.1.smithi144.stdout:0/613: rename d4/df/d1f/d24/f5e to d4/d6/d3a/d40/fd9 0 2024-05-15T15:54:16.684 INFO:tasks.workunit.client.1.smithi144.stdout:6/591: fsync d7/db/d35/f67 0 2024-05-15T15:54:16.684 INFO:tasks.workunit.client.1.smithi144.stdout:1/618: sync 2024-05-15T15:54:16.685 INFO:tasks.workunit.client.1.smithi144.stdout:6/592: truncate d7/db/d11/d3c/f79 558011 0 2024-05-15T15:54:16.686 INFO:tasks.workunit.client.0.smithi049.stdout:7/535: rename d0/d2f/d5c/l66 to d0/d6/d23/d45/d6a/da1/d54/la4 0 2024-05-15T15:54:16.687 INFO:tasks.workunit.client.1.smithi144.stdout:3/646: chown d0/d1/c28 36970288 1 2024-05-15T15:54:16.688 INFO:tasks.workunit.client.0.smithi049.stdout:5/596: rmdir dc 39 2024-05-15T15:54:16.688 INFO:tasks.workunit.client.0.smithi049.stdout:7/536: write d0/d6/d1d/f7a [611989,60975] 0 2024-05-15T15:54:16.690 INFO:tasks.workunit.client.0.smithi049.stdout:0/552: creat dc/da4/fab x:0 0 0 2024-05-15T15:54:16.691 INFO:tasks.workunit.client.0.smithi049.stdout:7/537: truncate d0/d6/d23/d45/d6a/da1/d54/d5e/f96 433796 0 2024-05-15T15:54:16.691 INFO:tasks.workunit.client.0.smithi049.stdout:0/553: write dc/d13/d14/f1c [3681298,103595] 0 2024-05-15T15:54:16.693 INFO:tasks.workunit.client.0.smithi049.stdout:7/538: write d0/d6/d1d/f4d [3248809,28615] 0 2024-05-15T15:54:16.701 INFO:tasks.workunit.client.0.smithi049.stdout:0/554: dwrite dc/d13/d48/d57/f65 [0,4194304] 0 2024-05-15T15:54:16.701 INFO:tasks.workunit.client.1.smithi144.stdout:4/632: rmdir d6/dd 39 2024-05-15T15:54:16.702 INFO:tasks.workunit.client.0.smithi049.stdout:7/539: dwrite d0/d6/d23/f4c [8388608,4194304] 0 2024-05-15T15:54:16.710 INFO:tasks.workunit.client.0.smithi049.stdout:7/540: dread d0/d6/d23/f91 [0,4194304] 0 2024-05-15T15:54:16.711 INFO:tasks.workunit.client.1.smithi144.stdout:5/644: link d0/dc/d24/d28/lba d0/dc/d12/d4a/d73/ld4 0 2024-05-15T15:54:16.712 INFO:tasks.workunit.client.0.smithi049.stdout:7/541: write d0/d6/d23/d45/d6a/da1/f2c [8858830,120277] 0 2024-05-15T15:54:16.717 INFO:tasks.workunit.client.0.smithi049.stdout:0/555: creat dc/d13/d2b/fac x:0 0 0 2024-05-15T15:54:16.717 INFO:tasks.workunit.client.1.smithi144.stdout:5/645: dread d0/dc/d2f/d68/d5c/f98 [0,4194304] 0 2024-05-15T15:54:16.723 INFO:tasks.workunit.client.1.smithi144.stdout:2/588: creat d3/d7/d20/d89/d6e/fc4 x:0 0 0 2024-05-15T15:54:16.723 INFO:tasks.workunit.client.1.smithi144.stdout:2/589: chown d3/d7/d20/d40/l59 63 1 2024-05-15T15:54:16.724 INFO:tasks.workunit.client.1.smithi144.stdout:2/590: write d3/d4d/fb7 [768152,69412] 0 2024-05-15T15:54:16.724 INFO:tasks.workunit.client.1.smithi144.stdout:8/688: rename d2/d24/d2d/d30/f44 to d2/de/d3f/d6b/fe2 0 2024-05-15T15:54:16.725 INFO:tasks.workunit.client.1.smithi144.stdout:2/591: readlink d3/d7/d20/d22/l4e 0 2024-05-15T15:54:16.728 INFO:tasks.workunit.client.1.smithi144.stdout:0/614: mkdir d4/d2d/d48/d5b/dda 0 2024-05-15T15:54:16.728 INFO:tasks.workunit.client.1.smithi144.stdout:0/615: dread - d4/d2d/d48/fd5 zero size 2024-05-15T15:54:16.731 INFO:tasks.workunit.client.1.smithi144.stdout:1/619: dread - d7/d31/f7b zero size 2024-05-15T15:54:16.733 INFO:tasks.workunit.client.0.smithi049.stdout:0/556: mknod dc/d13/d14/cad 0 2024-05-15T15:54:16.733 INFO:tasks.workunit.client.0.smithi049.stdout:1/727: sync 2024-05-15T15:54:16.734 INFO:tasks.workunit.client.0.smithi049.stdout:7/542: link d0/d6/c7 d0/d6/d23/d45/d6a/da1/d39/ca5 0 2024-05-15T15:54:16.736 INFO:tasks.workunit.client.1.smithi144.stdout:3/647: truncate d0/d11/d3b/f96 724241 0 2024-05-15T15:54:16.737 INFO:tasks.workunit.client.0.smithi049.stdout:0/557: mkdir dc/d13/d14/d25/d31/dae 0 2024-05-15T15:54:16.741 INFO:tasks.workunit.client.0.smithi049.stdout:7/543: unlink d0/d6/d1d/c52 0 2024-05-15T15:54:16.742 INFO:tasks.workunit.client.0.smithi049.stdout:0/558: symlink dc/d32/d93/laf 0 2024-05-15T15:54:16.743 INFO:tasks.workunit.client.0.smithi049.stdout:7/544: fdatasync d0/f15 0 2024-05-15T15:54:16.745 INFO:tasks.workunit.client.1.smithi144.stdout:7/621: getdents d4/d3c/d76/d9d/dac/d48/d72 0 2024-05-15T15:54:16.745 INFO:tasks.workunit.client.0.smithi049.stdout:1/728: dwrite d0/d4/dd/de/d48/d5c/f87 [0,4194304] 0 2024-05-15T15:54:16.745 INFO:tasks.workunit.client.0.smithi049.stdout:5/597: dwrite dc/dd/f14 [4194304,4194304] 0 2024-05-15T15:54:16.746 INFO:tasks.workunit.client.1.smithi144.stdout:8/689: creat d2/d24/d2d/d30/fe3 x:0 0 0 2024-05-15T15:54:16.747 INFO:tasks.workunit.client.0.smithi049.stdout:7/545: fdatasync d0/d6/d1d/f7a 0 2024-05-15T15:54:16.749 INFO:tasks.workunit.client.0.smithi049.stdout:5/598: stat dc/d4a/d31/c49 0 2024-05-15T15:54:16.750 INFO:tasks.workunit.client.0.smithi049.stdout:0/559: creat dc/d13/d2b/d76/fb0 x:0 0 0 2024-05-15T15:54:16.751 INFO:tasks.workunit.client.0.smithi049.stdout:7/546: write d0/d6/d23/d45/d6a/da1/d54/d5e/f9e [212971,128014] 0 2024-05-15T15:54:16.754 INFO:tasks.workunit.client.1.smithi144.stdout:1/620: mkdir d7/d1d/d33/d29/d6b/d8e/dcb 0 2024-05-15T15:54:16.755 INFO:tasks.workunit.client.1.smithi144.stdout:6/593: mknod d7/d1e/cce 0 2024-05-15T15:54:16.756 INFO:tasks.workunit.client.0.smithi049.stdout:5/599: creat dc/dd/d47/d83/fc2 x:0 0 0 2024-05-15T15:54:16.758 INFO:tasks.workunit.client.1.smithi144.stdout:3/648: unlink d0/d1/f4 0 2024-05-15T15:54:16.760 INFO:tasks.workunit.client.0.smithi049.stdout:7/547: rename d0/d6/d1d/l2b to d0/d34/la6 0 2024-05-15T15:54:16.762 INFO:tasks.workunit.client.0.smithi049.stdout:7/548: write d0/d6/d23/d45/d6a/da1/d54/d5e/f9e [112325,92286] 0 2024-05-15T15:54:16.762 INFO:tasks.workunit.client.1.smithi144.stdout:4/633: rename d6/dd/dae to d6/d42/d46/dcb 0 2024-05-15T15:54:16.763 INFO:tasks.workunit.client.0.smithi049.stdout:1/729: creat d0/d4/dd/de/d4d/d8b/de8/f104 x:0 0 0 2024-05-15T15:54:16.765 INFO:tasks.workunit.client.0.smithi049.stdout:5/600: fsync dc/f7f 0 2024-05-15T15:54:16.770 INFO:tasks.workunit.client.1.smithi144.stdout:8/690: dread - d2/d3a/d7b/d4b/fa9 zero size 2024-05-15T15:54:16.771 INFO:tasks.workunit.client.1.smithi144.stdout:0/616: creat d4/d2d/dcf/fdb x:0 0 0 2024-05-15T15:54:16.772 INFO:tasks.workunit.client.1.smithi144.stdout:1/621: mkdir d7/df/d89/dcc 0 2024-05-15T15:54:16.778 INFO:tasks.workunit.client.1.smithi144.stdout:1/622: dwrite d7/d1d/d33/d29/d6b/f65 [4194304,4194304] 0 2024-05-15T15:54:16.783 INFO:tasks.workunit.client.1.smithi144.stdout:6/594: creat d7/db/d35/d84/fcf x:0 0 0 2024-05-15T15:54:16.783 INFO:tasks.workunit.client.1.smithi144.stdout:9/656: getdents d0/d8/d1b 0 2024-05-15T15:54:16.783 INFO:tasks.workunit.client.1.smithi144.stdout:8/691: rmdir d2/d24/d2d/da0/d6e/d71/d73 39 2024-05-15T15:54:16.783 INFO:tasks.workunit.client.1.smithi144.stdout:0/617: symlink d4/d6/d3a/d4c/dd6/d55/da5/ldc 0 2024-05-15T15:54:16.786 INFO:tasks.workunit.client.1.smithi144.stdout:6/595: creat d7/db/d9d/d72/fd0 x:0 0 0 2024-05-15T15:54:16.786 INFO:tasks.workunit.client.1.smithi144.stdout:5/646: getdents d0/dc/d1f 0 2024-05-15T15:54:16.789 INFO:tasks.workunit.client.1.smithi144.stdout:3/649: dread d0/d1/d6c/f99 [0,4194304] 0 2024-05-15T15:54:16.789 INFO:tasks.workunit.client.0.smithi049.stdout:5/601: readlink dc/dd/d7b/la2 0 2024-05-15T15:54:16.791 INFO:tasks.workunit.client.0.smithi049.stdout:7/549: dwrite d0/d34/f35 [4194304,4194304] 0 2024-05-15T15:54:16.792 INFO:tasks.workunit.client.1.smithi144.stdout:3/650: fdatasync d0/d1/d17/d39/d40/f92 0 2024-05-15T15:54:16.793 INFO:tasks.workunit.client.0.smithi049.stdout:7/550: chown d0/d6/d23/d45/d6a/c72 3640 1 2024-05-15T15:54:16.794 INFO:tasks.workunit.client.1.smithi144.stdout:9/657: unlink d0/d8/d5e/d44/d77/c5f 0 2024-05-15T15:54:16.794 INFO:tasks.workunit.client.0.smithi049.stdout:7/551: chown d0/d2f/f3e 247084 1 2024-05-15T15:54:16.795 INFO:tasks.workunit.client.1.smithi144.stdout:5/647: dread d0/dc/d2f/d79/fb9 [0,4194304] 0 2024-05-15T15:54:16.796 INFO:tasks.workunit.client.1.smithi144.stdout:0/618: dwrite d4/d2d/dcf/fdb [0,4194304] 0 2024-05-15T15:54:16.797 INFO:tasks.workunit.client.1.smithi144.stdout:1/623: dwrite d7/d1d/d33/d1e/fbe [0,4194304] 0 2024-05-15T15:54:16.797 INFO:tasks.workunit.client.0.smithi049.stdout:5/602: symlink dc/d29/lc3 0 2024-05-15T15:54:16.797 INFO:tasks.workunit.client.1.smithi144.stdout:6/596: mknod d7/db/d35/d84/d81/cd1 0 2024-05-15T15:54:16.800 INFO:tasks.workunit.client.0.smithi049.stdout:7/552: mknod d0/d6/d23/d45/d6a/ca7 0 2024-05-15T15:54:16.803 INFO:tasks.workunit.client.1.smithi144.stdout:1/624: chown d7/d1d/d33/d41/f8c 507 1 2024-05-15T15:54:16.804 INFO:tasks.workunit.client.1.smithi144.stdout:5/648: dread d0/dc/d24/d8e/d93/fa1 [0,4194304] 0 2024-05-15T15:54:16.809 INFO:tasks.workunit.client.0.smithi049.stdout:5/603: fdatasync dc/d4a/f6a 0 2024-05-15T15:54:16.812 INFO:tasks.workunit.client.1.smithi144.stdout:0/619: mknod d4/df/d1f/d24/cdd 0 2024-05-15T15:54:16.816 INFO:tasks.workunit.client.0.smithi049.stdout:7/553: rmdir d0/d6/d1d 39 2024-05-15T15:54:16.817 INFO:tasks.workunit.client.1.smithi144.stdout:0/620: dwrite d4/df/d1f/d24/f4b [0,4194304] 0 2024-05-15T15:54:16.817 INFO:tasks.workunit.client.1.smithi144.stdout:8/692: link d2/de/d36/f17 d2/d24/fe4 0 2024-05-15T15:54:16.818 INFO:tasks.workunit.client.0.smithi049.stdout:7/554: write d0/d6/d23/d45/d6a/da1/d39/fa3 [1784822,88747] 0 2024-05-15T15:54:16.818 INFO:tasks.workunit.client.1.smithi144.stdout:8/693: write d2/d24/d2d/d30/fe0 [396381,97352] 0 2024-05-15T15:54:16.821 INFO:tasks.workunit.client.1.smithi144.stdout:8/694: chown d2/d24/d2d/da0/d9e/db8/fc1 7552509 1 2024-05-15T15:54:16.824 INFO:tasks.workunit.client.1.smithi144.stdout:9/658: creat d0/d8/d1b/d62/d7d/d8d/dbb/dce/fdd x:0 0 0 2024-05-15T15:54:16.826 INFO:tasks.workunit.client.0.smithi049.stdout:1/730: write d0/d4/f9 [91530,49963] 0 2024-05-15T15:54:16.826 INFO:tasks.workunit.client.1.smithi144.stdout:9/659: truncate d0/d8/d5e/d44/d16/dcd/fd3 242330 0 2024-05-15T15:54:16.827 INFO:tasks.workunit.client.1.smithi144.stdout:1/625: fsync d7/d1d/d33/d41/f83 0 2024-05-15T15:54:16.828 INFO:tasks.workunit.client.1.smithi144.stdout:2/592: write d3/d7/d20/d3f/f53 [417493,22704] 0 2024-05-15T15:54:16.830 INFO:tasks.workunit.client.1.smithi144.stdout:2/593: write d3/d7/d20/d3b/f77 [762307,106566] 0 2024-05-15T15:54:16.831 INFO:tasks.workunit.client.0.smithi049.stdout:1/731: creat d0/d4/dd/de/d9e/f105 x:0 0 0 2024-05-15T15:54:16.835 INFO:tasks.workunit.client.1.smithi144.stdout:0/621: mkdir d4/dc/dde 0 2024-05-15T15:54:16.837 INFO:tasks.workunit.client.0.smithi049.stdout:1/732: creat d0/d4/dd/de/d1e/d16/d8a/dd2/df2/f106 x:0 0 0 2024-05-15T15:54:16.839 INFO:tasks.workunit.client.1.smithi144.stdout:7/622: dwrite d4/f27 [0,4194304] 0 2024-05-15T15:54:16.841 INFO:tasks.workunit.client.0.smithi049.stdout:1/733: mknod d0/d4/dd/de/d4d/c107 0 2024-05-15T15:54:16.842 INFO:tasks.workunit.client.1.smithi144.stdout:8/695: creat d2/d24/d2d/da0/fe5 x:0 0 0 2024-05-15T15:54:16.847 INFO:tasks.workunit.client.1.smithi144.stdout:3/651: creat d0/d1/fda x:0 0 0 2024-05-15T15:54:16.850 INFO:tasks.workunit.client.0.smithi049.stdout:1/734: truncate d0/d4/dd/de/d36/f40 988565 0 2024-05-15T15:54:16.851 INFO:tasks.workunit.client.1.smithi144.stdout:3/652: dread d0/d11/d24/f74 [0,4194304] 0 2024-05-15T15:54:16.854 INFO:tasks.workunit.client.1.smithi144.stdout:0/622: rename d4/d6/d3a/d4c/f70 to d4/d6/fdf 0 2024-05-15T15:54:16.857 INFO:tasks.workunit.client.0.smithi049.stdout:1/735: creat d0/d4/dd/de/d36/f108 x:0 0 0 2024-05-15T15:54:16.858 INFO:tasks.workunit.client.0.smithi049.stdout:5/604: write dc/dd/f17 [5061673,84534] 0 2024-05-15T15:54:16.859 INFO:tasks.workunit.client.1.smithi144.stdout:7/623: unlink d4/d3c/d76/d9d/dac/f3b 0 2024-05-15T15:54:16.860 INFO:tasks.workunit.client.1.smithi144.stdout:7/624: dread - d4/d3c/d76/d9d/dac/d48/d72/d82/fbf zero size 2024-05-15T15:54:16.869 INFO:tasks.workunit.client.1.smithi144.stdout:5/649: creat d0/dc/d2f/d34/d63/fd5 x:0 0 0 2024-05-15T15:54:16.870 INFO:tasks.workunit.client.1.smithi144.stdout:5/650: write d0/f1a [2141036,96984] 0 2024-05-15T15:54:16.870 INFO:tasks.workunit.client.0.smithi049.stdout:7/555: dwrite d0/d6/d23/d45/f5a [0,4194304] 0 2024-05-15T15:54:16.871 INFO:tasks.workunit.client.1.smithi144.stdout:5/651: chown d0/dc/d24/d57/cd3 101000 1 2024-05-15T15:54:16.872 INFO:tasks.workunit.client.1.smithi144.stdout:5/652: dread - d0/dc/d24/d8e/d93/d9e/fcb zero size 2024-05-15T15:54:16.878 INFO:tasks.workunit.client.0.smithi049.stdout:5/605: rename dc/d4a/d42/l51 to dc/db0/lc4 0 2024-05-15T15:54:16.880 INFO:tasks.workunit.client.1.smithi144.stdout:6/597: dwrite d7/ff [0,4194304] 0 2024-05-15T15:54:16.882 INFO:tasks.workunit.client.1.smithi144.stdout:2/594: symlink d3/d7/da4/lc5 0 2024-05-15T15:54:16.882 INFO:tasks.workunit.client.0.smithi049.stdout:7/556: creat d0/d6/d23/d45/d6a/da1/d39/fa8 x:0 0 0 2024-05-15T15:54:16.885 INFO:tasks.workunit.client.1.smithi144.stdout:3/653: symlink d0/d11/d3b/dc7/ldb 0 2024-05-15T15:54:16.886 INFO:tasks.workunit.client.0.smithi049.stdout:5/606: dwrite dc/dd/f14 [0,4194304] 0 2024-05-15T15:54:16.894 INFO:tasks.workunit.client.0.smithi049.stdout:5/607: rmdir dc/dd/d47/d83 39 2024-05-15T15:54:16.895 INFO:tasks.workunit.client.1.smithi144.stdout:7/625: fdatasync d4/d45/f96 0 2024-05-15T15:54:16.896 INFO:tasks.workunit.client.1.smithi144.stdout:7/626: readlink d4/df/l66 0 2024-05-15T15:54:16.897 INFO:tasks.workunit.client.0.smithi049.stdout:7/557: write d0/d6/d1d/f27 [2672681,115283] 0 2024-05-15T15:54:16.899 INFO:tasks.workunit.client.1.smithi144.stdout:8/696: write d2/d3a/d7b/f97 [84362,50444] 0 2024-05-15T15:54:16.903 INFO:tasks.workunit.client.1.smithi144.stdout:5/653: creat d0/dc/d1f/d50/fd6 x:0 0 0 2024-05-15T15:54:16.903 INFO:tasks.workunit.client.1.smithi144.stdout:1/626: write d7/d1d/d33/d1e/f25 [216641,33262] 0 2024-05-15T15:54:16.904 INFO:tasks.workunit.client.1.smithi144.stdout:5/654: write d0/dc/d24/fd2 [492451,110264] 0 2024-05-15T15:54:16.906 INFO:tasks.workunit.client.1.smithi144.stdout:4/634: sync 2024-05-15T15:54:16.907 INFO:tasks.workunit.client.1.smithi144.stdout:2/595: fsync d3/d7/f1f 0 2024-05-15T15:54:16.908 INFO:tasks.workunit.client.1.smithi144.stdout:2/596: chown d3/d7/d54/f5a 28434 1 2024-05-15T15:54:16.911 INFO:tasks.workunit.client.1.smithi144.stdout:0/623: mknod d4/d2d/d48/d5b/dda/ce0 0 2024-05-15T15:54:16.912 INFO:tasks.workunit.client.1.smithi144.stdout:6/598: dwrite d7/d1e/f44 [0,4194304] 0 2024-05-15T15:54:16.913 INFO:tasks.workunit.client.0.smithi049.stdout:1/736: write d0/d4/dd/de/d1e/d2e/dad/d65/fdf [149659,55162] 0 2024-05-15T15:54:16.915 INFO:tasks.workunit.client.1.smithi144.stdout:2/597: dwrite d3/d10/f35 [0,4194304] 0 2024-05-15T15:54:16.916 INFO:tasks.workunit.client.1.smithi144.stdout:8/697: chown d2/l4 4297996 1 2024-05-15T15:54:16.919 INFO:tasks.workunit.client.0.smithi049.stdout:3/581: sync 2024-05-15T15:54:16.921 INFO:tasks.workunit.client.1.smithi144.stdout:5/655: creat d0/dc/d12/fd7 x:0 0 0 2024-05-15T15:54:16.928 INFO:tasks.workunit.client.1.smithi144.stdout:6/599: unlink d7/d1e/l3b 0 2024-05-15T15:54:16.928 INFO:tasks.workunit.client.1.smithi144.stdout:0/624: rmdir d4/df/d1f/d24/d96 39 2024-05-15T15:54:16.928 INFO:tasks.workunit.client.1.smithi144.stdout:5/656: dwrite d0/dc/d2f/d34/d63/d6e/dbf/f9f [0,4194304] 0 2024-05-15T15:54:16.943 INFO:tasks.workunit.client.1.smithi144.stdout:6/600: mkdir d7/db/d3a/dd2 0 2024-05-15T15:54:16.946 INFO:tasks.workunit.client.1.smithi144.stdout:2/598: creat d3/d7/d20/d22/d52/d75/da6/dc1/fc6 x:0 0 0 2024-05-15T15:54:16.948 INFO:tasks.workunit.client.1.smithi144.stdout:0/625: symlink d4/d6/d3a/d4c/dd6/le1 0 2024-05-15T15:54:16.950 INFO:tasks.workunit.client.1.smithi144.stdout:5/657: rename d0/dc/d24/d8e/d95/cc8 to d0/dc/d1f/d50/db3/cd8 0 2024-05-15T15:54:16.951 INFO:tasks.workunit.client.0.smithi049.stdout:5/608: write dc/d4a/f3a [994072,32125] 0 2024-05-15T15:54:16.960 INFO:tasks.workunit.client.1.smithi144.stdout:2/599: rename d3/d10/l82 to d3/d10/lc7 0 2024-05-15T15:54:16.960 INFO:tasks.workunit.client.1.smithi144.stdout:2/600: chown d3/d7/d20/d22/d5e/f96 6 1 2024-05-15T15:54:16.962 INFO:tasks.workunit.client.1.smithi144.stdout:5/658: rmdir d0/dc/d2f/d34/d63/d6e 39 2024-05-15T15:54:16.964 INFO:tasks.workunit.client.1.smithi144.stdout:1/627: getdents d7/d1d/d33/d41/d22/dbd 0 2024-05-15T15:54:16.965 INFO:tasks.workunit.client.1.smithi144.stdout:1/628: read d7/d31/f73 [327030,40807] 0 2024-05-15T15:54:16.966 INFO:tasks.workunit.client.0.smithi049.stdout:5/609: dread dc/d4a/f3a [0,4194304] 0 2024-05-15T15:54:16.966 INFO:tasks.workunit.client.1.smithi144.stdout:6/601: getdents d7/db/d9d/d97/dba 0 2024-05-15T15:54:16.966 INFO:tasks.workunit.client.1.smithi144.stdout:8/698: dread d2/d3a/f59 [0,4194304] 0 2024-05-15T15:54:16.967 INFO:tasks.workunit.client.1.smithi144.stdout:8/699: write d2/d3a/f3e [4251609,23581] 0 2024-05-15T15:54:16.968 INFO:tasks.workunit.client.1.smithi144.stdout:0/626: symlink d4/d2d/d48/d5b/dd3/le2 0 2024-05-15T15:54:16.969 INFO:tasks.workunit.client.1.smithi144.stdout:7/627: write d4/d41/d5b/f53 [4879467,46530] 0 2024-05-15T15:54:16.970 INFO:tasks.workunit.client.1.smithi144.stdout:2/601: creat d3/d7/d20/d22/d52/d75/d65/fc8 x:0 0 0 2024-05-15T15:54:16.972 INFO:tasks.workunit.client.1.smithi144.stdout:5/659: fdatasync d0/dc/d24/d28/d2d/f3d 0 2024-05-15T15:54:16.974 INFO:tasks.workunit.client.1.smithi144.stdout:6/602: dwrite d7/db/d11/f60 [4194304,4194304] 0 2024-05-15T15:54:16.983 INFO:tasks.workunit.client.1.smithi144.stdout:0/627: creat d4/d6/d3a/d4c/dd6/d55/da5/fe3 x:0 0 0 2024-05-15T15:54:16.986 INFO:tasks.workunit.client.1.smithi144.stdout:8/700: creat d2/d24/d2d/da0/d6e/d71/d8c/fe6 x:0 0 0 2024-05-15T15:54:16.990 INFO:tasks.workunit.client.1.smithi144.stdout:4/635: write d6/dd/d2d/f48 [1336107,50142] 0 2024-05-15T15:54:16.991 INFO:tasks.workunit.client.1.smithi144.stdout:2/602: rename d3/d7/d20/f3e to d3/d7/d20/d40/fc9 0 2024-05-15T15:54:16.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.993+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 --> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb3c100600 con 0x7fbb2406d6f0 2024-05-15T15:54:16.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.994+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/1237110665 <== mgr.24413 v2:172.21.15.144:6828/177983287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fbb3c100600 con 0x7fbb2406d6f0 2024-05-15T15:54:16.997 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:54:16.997 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:54:16.997 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:54:16.997 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:54:16.997 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [], 2024-05-15T15:54:16.997 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "1/2 daemons upgraded", 2024-05-15T15:54:16.997 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mgr daemons", 2024-05-15T15:54:16.997 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:54:16.997 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:54:16.998 INFO:tasks.workunit.client.1.smithi144.stdout:1/629: mkdir d7/d1d/d33/d29/dcd 0 2024-05-15T15:54:16.999 INFO:tasks.workunit.client.1.smithi144.stdout:1/630: write d7/d1d/d33/d29/f64 [1223726,97257] 0 2024-05-15T15:54:17.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.997+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fbb2406d6f0 msgr2=0x7fbb2406fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.997+0000 7fbb40f72700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fbb2406d6f0 0x7fbb2406fbb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007900 tx=0x7fbb2c008040 comp rx=0 tx=0).stop 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.997+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 msgr2=0x7fbb3c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.997+0000 7fbb40f72700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 0x7fbb3c0ff790 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb30006010 tx=0x7fbb3000be20 comp rx=0 tx=0).stop 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.998+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 shutdown_connections 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.998+0000 7fbb40f72700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7fbb2406d6f0 0x7fbb2406fbb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.998+0000 7fbb40f72700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb3c1017e0 0x7fbb3c1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.998+0000 7fbb40f72700 1 --2- 172.21.15.49:0/1237110665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c104110 0x7fbb3c0ff790 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.998+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 >> 172.21.15.49:0/1237110665 conn(0x7fbb3c0fb1d0 msgr2=0x7fbb3c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.998+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 shutdown_connections 2024-05-15T15:54:17.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:16.998+0000 7fbb40f72700 1 -- 172.21.15.49:0/1237110665 wait complete. 2024-05-15T15:54:17.002 INFO:tasks.workunit.client.1.smithi144.stdout:0/628: mkdir d4/d6/d4a/d74/de4 0 2024-05-15T15:54:17.004 INFO:tasks.workunit.client.1.smithi144.stdout:8/701: readlink d2/d24/d2d/da0/d6e/d71/d73/lcb 0 2024-05-15T15:54:17.005 INFO:tasks.workunit.client.1.smithi144.stdout:8/702: truncate d2/de/d36/d87/fda 5188755 0 2024-05-15T15:54:17.005 INFO:tasks.workunit.client.0.smithi049.stdout:7/558: link d0/lf d0/d2f/la9 0 2024-05-15T15:54:17.006 INFO:tasks.workunit.client.0.smithi049.stdout:7/559: chown d0/d2f/d5c/f64 3991 1 2024-05-15T15:54:17.006 INFO:tasks.workunit.client.1.smithi144.stdout:8/703: read d2/d24/d2d/d30/f81 [739268,30975] 0 2024-05-15T15:54:17.007 INFO:tasks.workunit.client.1.smithi144.stdout:6/603: dread d7/d1e/fc9 [0,4194304] 0 2024-05-15T15:54:17.008 INFO:tasks.workunit.client.0.smithi049.stdout:6/531: sync 2024-05-15T15:54:17.009 INFO:tasks.workunit.client.0.smithi049.stdout:9/444: sync 2024-05-15T15:54:17.009 INFO:tasks.workunit.client.0.smithi049.stdout:4/580: sync 2024-05-15T15:54:17.009 INFO:tasks.workunit.client.0.smithi049.stdout:8/548: sync 2024-05-15T15:54:17.009 INFO:tasks.workunit.client.0.smithi049.stdout:2/545: sync 2024-05-15T15:54:17.009 INFO:tasks.workunit.client.0.smithi049.stdout:1/737: mknod d0/d4/dd/de/d4e/c109 0 2024-05-15T15:54:17.010 INFO:tasks.workunit.client.1.smithi144.stdout:4/636: mkdir d6/dd/dc3/dcc 0 2024-05-15T15:54:17.011 INFO:tasks.workunit.client.0.smithi049.stdout:3/582: dread - d12/d14/d32/f58 zero size 2024-05-15T15:54:17.012 INFO:tasks.workunit.client.1.smithi144.stdout:2/603: creat d3/d4d/fca x:0 0 0 2024-05-15T15:54:17.013 INFO:tasks.workunit.client.1.smithi144.stdout:6/604: dwrite d7/d1e/d28/f3d [4194304,4194304] 0 2024-05-15T15:54:17.013 INFO:tasks.workunit.client.1.smithi144.stdout:6/605: stat d7/db/d35 0 2024-05-15T15:54:17.016 INFO:tasks.workunit.client.0.smithi049.stdout:5/610: rename dc/dd/d7b/d8d/f94 to dc/db0/dbf/fc5 0 2024-05-15T15:54:17.017 INFO:tasks.workunit.client.0.smithi049.stdout:4/581: dread - d18/d1e/d63/fb8 zero size 2024-05-15T15:54:17.018 INFO:tasks.workunit.client.1.smithi144.stdout:1/631: dread d7/df/f9f [0,4194304] 0 2024-05-15T15:54:17.020 INFO:tasks.workunit.client.0.smithi049.stdout:9/445: mkdir d0/d8/d9/d52/d5e/d5f/d88/d9d 0 2024-05-15T15:54:17.023 INFO:tasks.workunit.client.0.smithi049.stdout:2/546: rename d3/db/d1d/c32 to d3/db/d1d/cb0 0 2024-05-15T15:54:17.024 INFO:tasks.workunit.client.0.smithi049.stdout:1/738: rmdir d0/d4/dd/de/d1e/d16/d8a/dd2 39 2024-05-15T15:54:17.024 INFO:tasks.workunit.client.1.smithi144.stdout:4/637: mknod d6/d42/d46/d4b/ccd 0 2024-05-15T15:54:17.024 INFO:tasks.workunit.client.1.smithi144.stdout:1/632: dwrite d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:17.025 INFO:tasks.workunit.client.1.smithi144.stdout:2/604: creat d3/d7/d20/d22/d52/d64/fcb x:0 0 0 2024-05-15T15:54:17.026 INFO:tasks.workunit.client.1.smithi144.stdout:9/660: sync 2024-05-15T15:54:17.027 INFO:tasks.workunit.client.1.smithi144.stdout:2/605: write d3/d7/d20/d37/d76/faa [3921455,63738] 0 2024-05-15T15:54:17.031 INFO:tasks.workunit.client.1.smithi144.stdout:8/704: getdents d2/d24/d2d/da0/d9e/db8/dde 0 2024-05-15T15:54:17.032 INFO:tasks.workunit.client.0.smithi049.stdout:5/611: rmdir dc 39 2024-05-15T15:54:17.032 INFO:tasks.workunit.client.1.smithi144.stdout:4/638: fsync d6/d7c/f8c 0 2024-05-15T15:54:17.034 INFO:tasks.workunit.client.0.smithi049.stdout:8/549: mknod d2/de/d71/d86/d5d/d7d/d1b/cad 0 2024-05-15T15:54:17.034 INFO:tasks.workunit.client.1.smithi144.stdout:9/661: dwrite d0/d8/d5e/d44/d10/d9c/f98 [0,4194304] 0 2024-05-15T15:54:17.037 INFO:tasks.workunit.client.0.smithi049.stdout:2/547: creat d3/db/d82/da1/fb1 x:0 0 0 2024-05-15T15:54:17.037 INFO:tasks.workunit.client.1.smithi144.stdout:6/606: link d7/db/d11/f43 d7/db/d35/d84/dbe/fd3 0 2024-05-15T15:54:17.038 INFO:tasks.workunit.client.0.smithi049.stdout:2/548: stat d3/d8/cae 0 2024-05-15T15:54:17.038 INFO:tasks.workunit.client.0.smithi049.stdout:4/582: dread d18/f7c [0,4194304] 0 2024-05-15T15:54:17.040 INFO:tasks.workunit.client.1.smithi144.stdout:5/660: getdents d0/dc/d24/d28/d2d 0 2024-05-15T15:54:17.040 INFO:tasks.workunit.client.0.smithi049.stdout:3/583: mknod d12/d30/d42/cb1 0 2024-05-15T15:54:17.043 INFO:tasks.workunit.client.0.smithi049.stdout:7/560: rename d0/d6/d23/d45/d83/l94 to d0/d6/d23/d76/laa 0 2024-05-15T15:54:17.046 INFO:tasks.workunit.client.1.smithi144.stdout:1/633: creat d7/d1d/d33/d41/d78/dc3/fce x:0 0 0 2024-05-15T15:54:17.046 INFO:tasks.workunit.client.0.smithi049.stdout:7/561: dread d0/d6/d23/d45/d6a/da1/d54/d5e/f96 [0,4194304] 0 2024-05-15T15:54:17.047 INFO:tasks.workunit.client.1.smithi144.stdout:8/705: creat d2/d24/d2d/da0/d9e/db8/fe7 x:0 0 0 2024-05-15T15:54:17.048 INFO:tasks.workunit.client.1.smithi144.stdout:8/706: fdatasync d2/de/d36/f1e 0 2024-05-15T15:54:17.049 INFO:tasks.workunit.client.1.smithi144.stdout:8/707: read - d2/d3a/d7b/fdb zero size 2024-05-15T15:54:17.050 INFO:tasks.workunit.client.0.smithi049.stdout:2/549: rmdir d3/da/d15/d24 39 2024-05-15T15:54:17.050 INFO:tasks.workunit.client.0.smithi049.stdout:8/550: dread d2/de/d71/d86/d5d/d7d/d1b/d68/f66 [0,4194304] 0 2024-05-15T15:54:17.052 INFO:tasks.workunit.client.1.smithi144.stdout:6/607: symlink d7/db/da2/ld4 0 2024-05-15T15:54:17.052 INFO:tasks.workunit.client.1.smithi144.stdout:6/608: read d7/db/d11/d3c/d61/f9b [583837,6222] 0 2024-05-15T15:54:17.053 INFO:tasks.workunit.client.0.smithi049.stdout:3/584: creat d12/d30/d63/fb2 x:0 0 0 2024-05-15T15:54:17.053 INFO:tasks.workunit.client.0.smithi049.stdout:3/585: stat d12/d14/d32/l5a 0 2024-05-15T15:54:17.055 INFO:tasks.workunit.client.1.smithi144.stdout:1/634: unlink d7/d1d/f8f 0 2024-05-15T15:54:17.057 INFO:tasks.workunit.client.0.smithi049.stdout:6/532: rename dd/d17/d24/d58/d5f/f7c to dd/d17/d24/d2b/d36/d86/d96/f9c 0 2024-05-15T15:54:17.059 INFO:tasks.workunit.client.1.smithi144.stdout:4/639: mknod d6/dd/d34/d3b/d30/dc9/cce 0 2024-05-15T15:54:17.060 INFO:tasks.workunit.client.0.smithi049.stdout:6/533: dread - dd/d17/d24/d2b/d36/d50/f83 zero size 2024-05-15T15:54:17.061 INFO:tasks.workunit.client.0.smithi049.stdout:3/586: dwrite d12/d30/d42/da6/f97 [0,4194304] 0 2024-05-15T15:54:17.061 INFO:tasks.workunit.client.1.smithi144.stdout:8/708: symlink d2/de/d36/d87/le8 0 2024-05-15T15:54:17.062 INFO:tasks.workunit.client.0.smithi049.stdout:6/534: stat dd/d17/d24/d2b/d36/d50/d7e 0 2024-05-15T15:54:17.069 INFO:tasks.workunit.client.1.smithi144.stdout:6/609: write d7/db/d5f/fb8 [821095,83341] 0 2024-05-15T15:54:17.070 INFO:tasks.workunit.client.1.smithi144.stdout:5/661: creat d0/dc/d2f/d34/d63/d6e/fd9 x:0 0 0 2024-05-15T15:54:17.075 INFO:tasks.workunit.client.0.smithi049.stdout:2/550: creat d3/db/d1d/d1e/d2b/d3b/d47/d59/fb2 x:0 0 0 2024-05-15T15:54:17.075 INFO:tasks.workunit.client.0.smithi049.stdout:0/560: sync 2024-05-15T15:54:17.077 INFO:tasks.workunit.client.1.smithi144.stdout:4/640: rename d6/d42/d46/dcb to d6/dd/d34/d3b/d30/d59/d66/dcf 0 2024-05-15T15:54:17.079 INFO:tasks.workunit.client.1.smithi144.stdout:9/662: link d0/d8/d5e/d44/d77/l22 d0/d8/d1b/d62/d7d/d8d/dbb/dce/lde 0 2024-05-15T15:54:17.080 INFO:tasks.workunit.client.1.smithi144.stdout:9/663: chown d0/d8/d5e/d44 16299 1 2024-05-15T15:54:17.084 INFO:tasks.workunit.client.1.smithi144.stdout:1/635: dread d7/d1d/d33/d29/d6b/d5e/f95 [0,4194304] 0 2024-05-15T15:54:17.084 INFO:tasks.workunit.client.1.smithi144.stdout:6/610: dread d7/db/d35/d84/f98 [0,4194304] 0 2024-05-15T15:54:17.086 INFO:tasks.workunit.client.1.smithi144.stdout:1/636: chown d7/d1d/d33/d29/d6b/d5e/f95 395881735 1 2024-05-15T15:54:17.103 INFO:tasks.workunit.client.1.smithi144.stdout:3/654: sync 2024-05-15T15:54:17.135 INFO:tasks.workunit.client.1.smithi144.stdout:7/628: sync 2024-05-15T15:54:17.135 INFO:tasks.workunit.client.1.smithi144.stdout:0/629: sync 2024-05-15T15:54:17.137 INFO:tasks.workunit.client.0.smithi049.stdout:5/612: mknod dc/dd/d47/da6/cc6 0 2024-05-15T15:54:17.141 INFO:tasks.workunit.client.0.smithi049.stdout:9/446: rename d0/d8/d1b/d71/c97 to d0/d8/d1b/d71/d87/c9e 0 2024-05-15T15:54:17.142 INFO:tasks.workunit.client.0.smithi049.stdout:9/447: chown d0/d8/d42/f91 40898 1 2024-05-15T15:54:17.143 INFO:tasks.workunit.client.0.smithi049.stdout:9/448: fsync d0/d8/d9/d52/d5e/d5f/f55 0 2024-05-15T15:54:17.148 INFO:tasks.workunit.client.1.smithi144.stdout:8/709: dwrite d2/f16 [0,4194304] 0 2024-05-15T15:54:17.149 INFO:tasks.workunit.client.0.smithi049.stdout:7/562: link d0/d6/d23/f40 d0/d6/d23/d45/d83/fab 0 2024-05-15T15:54:17.159 INFO:tasks.workunit.client.1.smithi144.stdout:6/611: write d7/db/d35/d84/dbe/fd3 [3381174,129393] 0 2024-05-15T15:54:17.159 INFO:tasks.workunit.client.1.smithi144.stdout:6/612: chown d7/d1e/fc9 41 1 2024-05-15T15:54:17.161 INFO:tasks.workunit.client.0.smithi049.stdout:0/561: creat dc/d13/d48/fb1 x:0 0 0 2024-05-15T15:54:17.163 INFO:tasks.workunit.client.0.smithi049.stdout:2/551: mknod d3/d78/d8c/d91/cb3 0 2024-05-15T15:54:17.165 INFO:tasks.workunit.client.0.smithi049.stdout:5/613: mkdir dc/db0/dc7 0 2024-05-15T15:54:17.165 INFO:tasks.workunit.client.0.smithi049.stdout:5/614: readlink dc/dd/d10/l33 0 2024-05-15T15:54:17.168 INFO:tasks.workunit.client.1.smithi144.stdout:5/662: link d0/d7e/fc2 d0/dc/d2f/d34/d63/d6e/db8/fda 0 2024-05-15T15:54:17.171 INFO:tasks.workunit.client.1.smithi144.stdout:8/710: creat d2/d3a/d7b/d4b/fe9 x:0 0 0 2024-05-15T15:54:17.176 INFO:tasks.workunit.client.0.smithi049.stdout:1/739: getdents d0/d4/dd/de/d1e/d16/d8a/dd2 0 2024-05-15T15:54:17.177 INFO:tasks.workunit.client.0.smithi049.stdout:4/583: write d18/f1c [378043,69607] 0 2024-05-15T15:54:17.177 INFO:tasks.workunit.client.1.smithi144.stdout:2/606: dwrite d3/f9 [4194304,4194304] 0 2024-05-15T15:54:17.191 INFO:tasks.workunit.client.0.smithi049.stdout:9/449: link d0/d8/d9c/f93 d0/d8/d9c/f9f 0 2024-05-15T15:54:17.199 INFO:tasks.workunit.client.0.smithi049.stdout:8/551: write d2/de/d71/d86/d5d/d63/f6e [1039971,45873] 0 2024-05-15T15:54:17.199 INFO:tasks.workunit.client.0.smithi049.stdout:8/552: chown d2/f3d 888 1 2024-05-15T15:54:17.213 INFO:tasks.workunit.client.0.smithi049.stdout:8/553: dread d2/fc [0,4194304] 0 2024-05-15T15:54:17.213 INFO:tasks.workunit.client.0.smithi049.stdout:9/450: getdents d0/d8/d9 0 2024-05-15T15:54:17.231 INFO:tasks.workunit.client.1.smithi144.stdout:1/637: creat d7/d1d/d33/d1e/d34/d9c/fcf x:0 0 0 2024-05-15T15:54:17.231 INFO:tasks.workunit.client.1.smithi144.stdout:6/613: mknod d7/db/d9d/d72/cd5 0 2024-05-15T15:54:17.232 INFO:tasks.workunit.client.1.smithi144.stdout:6/614: chown d7/db/d3a/f91 417 1 2024-05-15T15:54:17.232 INFO:tasks.workunit.client.1.smithi144.stdout:6/615: chown d7/d32 1 1 2024-05-15T15:54:17.242 INFO:tasks.workunit.client.1.smithi144.stdout:6/616: rename d7/d1e/l2d to d7/db/d9d/d72/ld6 0 2024-05-15T15:54:17.242 INFO:tasks.workunit.client.1.smithi144.stdout:0/630: link f2 d4/d6/d3a/d4c/dd6/fe5 0 2024-05-15T15:54:17.243 INFO:tasks.workunit.client.1.smithi144.stdout:0/631: chown d4/df/d1f/d24/f69 1 1 2024-05-15T15:54:17.243 INFO:tasks.workunit.client.1.smithi144.stdout:0/632: dread - d4/d6/fcb zero size 2024-05-15T15:54:17.244 INFO:tasks.workunit.client.1.smithi144.stdout:9/664: getdents d0/d8/d5e/d44 0 2024-05-15T15:54:17.246 INFO:tasks.workunit.client.1.smithi144.stdout:9/665: dread d0/d8/d5e/fa4 [0,4194304] 0 2024-05-15T15:54:17.246 INFO:tasks.workunit.client.1.smithi144.stdout:1/638: link d7/d1d/fa5 d7/d1d/d33/d41/d22/dbd/fd0 0 2024-05-15T15:54:17.246 INFO:tasks.workunit.client.1.smithi144.stdout:1/639: chown d7/d31/l99 0 1 2024-05-15T15:54:17.247 INFO:tasks.workunit.client.1.smithi144.stdout:9/666: chown d0/d8/d5e/d44/d16/d5a/cab 0 1 2024-05-15T15:54:17.250 INFO:tasks.workunit.client.1.smithi144.stdout:0/633: rmdir d4/df/d1f/d24/d85/da1/da3 39 2024-05-15T15:54:17.251 INFO:tasks.workunit.client.1.smithi144.stdout:0/634: write d4/d6/d3a/d4c/dd6/d55/da5/fe3 [608219,114605] 0 2024-05-15T15:54:17.251 INFO:tasks.workunit.client.1.smithi144.stdout:6/617: symlink d7/db/d9d/d97/dba/ld7 0 2024-05-15T15:54:17.253 INFO:tasks.workunit.client.1.smithi144.stdout:1/640: creat d7/df/d89/fd1 x:0 0 0 2024-05-15T15:54:17.254 INFO:tasks.workunit.client.1.smithi144.stdout:9/667: symlink d0/d8/d1b/d62/d7d/d8d/d94/ldf 0 2024-05-15T15:54:17.256 INFO:tasks.workunit.client.1.smithi144.stdout:0/635: fdatasync d4/df/d1f/d24/fbc 0 2024-05-15T15:54:17.257 INFO:tasks.workunit.client.1.smithi144.stdout:0/636: write d4/d6/d3a/f91 [4993088,20017] 0 2024-05-15T15:54:17.260 INFO:tasks.workunit.client.1.smithi144.stdout:9/668: write d0/d8/d1b/d62/fca [6409735,12964] 0 2024-05-15T15:54:17.261 INFO:tasks.workunit.client.1.smithi144.stdout:9/669: readlink d0/d8/d1b/d62/d7d/d8d/lb2 0 2024-05-15T15:54:17.262 INFO:tasks.workunit.client.1.smithi144.stdout:9/670: chown d0/d8/d5e/d44/d16/f4e 54 1 2024-05-15T15:54:17.263 INFO:tasks.workunit.client.1.smithi144.stdout:6/618: link d7/ff d7/d1e/d28/db5/fd8 0 2024-05-15T15:54:17.264 INFO:tasks.workunit.client.1.smithi144.stdout:6/619: fdatasync d7/d1e/d28/f8a 0 2024-05-15T15:54:17.273 INFO:tasks.workunit.client.1.smithi144.stdout:6/620: link d7/db/d11/f42 d7/d62/fd9 0 2024-05-15T15:54:17.274 INFO:tasks.workunit.client.1.smithi144.stdout:6/621: fdatasync d7/db/f29 0 2024-05-15T15:54:17.274 INFO:tasks.workunit.client.1.smithi144.stdout:9/671: link d0/d8/d5e/d44/d16/d5a/c68 d0/d8/d1b/d62/d7d/d8d/dbb/ce0 0 2024-05-15T15:54:17.275 INFO:tasks.workunit.client.1.smithi144.stdout:6/622: read d7/db/d11/d3c/d61/f9b [512998,2220] 0 2024-05-15T15:54:17.277 INFO:tasks.workunit.client.1.smithi144.stdout:9/672: symlink d0/d8/d50/le1 0 2024-05-15T15:54:17.280 INFO:tasks.workunit.client.1.smithi144.stdout:9/673: dread d0/f18 [0,4194304] 0 2024-05-15T15:54:17.282 INFO:tasks.workunit.client.1.smithi144.stdout:9/674: creat d0/d8/d5e/d44/d16/d5a/fe2 x:0 0 0 2024-05-15T15:54:17.285 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:17 smithi049 ceph-mon[28399]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:17.285 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:17 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/193574544' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:54:17.286 INFO:tasks.workunit.client.1.smithi144.stdout:9/675: creat d0/d8/d5e/d44/da2/fe3 x:0 0 0 2024-05-15T15:54:17.302 INFO:tasks.workunit.client.0.smithi049.stdout:8/554: creat d2/de/d25/d74/fae x:0 0 0 2024-05-15T15:54:17.302 INFO:tasks.workunit.client.0.smithi049.stdout:0/562: creat dc/d13/d3d/fb2 x:0 0 0 2024-05-15T15:54:17.304 INFO:tasks.workunit.client.0.smithi049.stdout:9/451: creat d0/d8/d9/d52/d5e/d5f/fa0 x:0 0 0 2024-05-15T15:54:17.306 INFO:tasks.workunit.client.0.smithi049.stdout:0/563: unlink dc/d13/d48/fb1 0 2024-05-15T15:54:17.309 INFO:tasks.workunit.client.0.smithi049.stdout:6/535: write dd/d17/d24/d2b/f5d [1334633,84098] 0 2024-05-15T15:54:17.311 INFO:tasks.workunit.client.0.smithi049.stdout:0/564: mkdir dc/d13/d2b/d76/d8a/da2/db3 0 2024-05-15T15:54:17.312 INFO:tasks.workunit.client.1.smithi144.stdout:4/641: write d6/dd/d34/d3b/d30/db/f33 [599518,35384] 0 2024-05-15T15:54:17.312 INFO:tasks.workunit.client.1.smithi144.stdout:4/642: readlink d6/dd/d34/d3b/l2c 0 2024-05-15T15:54:17.316 INFO:tasks.workunit.client.0.smithi049.stdout:0/565: dread fa [0,4194304] 0 2024-05-15T15:54:17.316 INFO:tasks.workunit.client.1.smithi144.stdout:4/643: mknod d6/dd/d34/d3b/d30/db/d1d/d40/d57/cd0 0 2024-05-15T15:54:17.318 INFO:tasks.workunit.client.1.smithi144.stdout:4/644: read d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/fb7 [2550258,83203] 0 2024-05-15T15:54:17.319 INFO:tasks.workunit.client.0.smithi049.stdout:0/566: mkdir dc/da4/db4 0 2024-05-15T15:54:17.319 INFO:tasks.workunit.client.1.smithi144.stdout:4/645: chown d6/d42/d46/c73 2 1 2024-05-15T15:54:17.319 INFO:tasks.workunit.client.0.smithi049.stdout:0/567: write f3 [200658,78051] 0 2024-05-15T15:54:17.320 INFO:tasks.workunit.client.0.smithi049.stdout:9/452: symlink d0/d8/d9/d52/la1 0 2024-05-15T15:54:17.321 INFO:tasks.workunit.client.0.smithi049.stdout:6/536: getdents dd/d17/d64 0 2024-05-15T15:54:17.321 INFO:tasks.workunit.client.0.smithi049.stdout:0/568: stat dc/d13/d2b 0 2024-05-15T15:54:17.322 INFO:tasks.workunit.client.0.smithi049.stdout:9/453: chown d0/d8/d9/d52/d5e/d5f/f89 138495 1 2024-05-15T15:54:17.325 INFO:tasks.workunit.client.1.smithi144.stdout:4/646: mknod d6/d42/d46/d4b/d1e/d44/cd1 0 2024-05-15T15:54:17.329 INFO:tasks.workunit.client.1.smithi144.stdout:3/655: write d0/d11/f61 [346666,99895] 0 2024-05-15T15:54:17.330 INFO:tasks.workunit.client.0.smithi049.stdout:0/569: symlink dc/d13/d14/d25/d31/dae/lb5 0 2024-05-15T15:54:17.334 INFO:tasks.workunit.client.0.smithi049.stdout:0/570: readlink l9 0 2024-05-15T15:54:17.334 INFO:tasks.workunit.client.1.smithi144.stdout:3/656: dwrite d0/d11/f26 [0,4194304] 0 2024-05-15T15:54:17.340 INFO:tasks.workunit.client.0.smithi049.stdout:0/571: dwrite dc/d13/d48/d57/f65 [0,4194304] 0 2024-05-15T15:54:17.356 INFO:tasks.workunit.client.0.smithi049.stdout:1/740: write d0/d4/dd/de/d1e/d2e/dad/d65/d72/f8d [848606,112260] 0 2024-05-15T15:54:17.362 INFO:tasks.workunit.client.0.smithi049.stdout:1/741: link d0/d4/dd/de/d4d/d8b/lce d0/d4/dd/de/d4d/d8b/l10a 0 2024-05-15T15:54:17.366 INFO:tasks.workunit.client.0.smithi049.stdout:1/742: dread d0/d4/dd/de/d9e/fa0 [0,4194304] 0 2024-05-15T15:54:17.369 INFO:tasks.workunit.client.0.smithi049.stdout:1/743: creat d0/d4/dd/de/d1e/d16/df1/f10b x:0 0 0 2024-05-15T15:54:17.370 INFO:tasks.workunit.client.0.smithi049.stdout:4/584: link d18/d1e/d2a/d50/d54/f5c d18/d1e/fc1 0 2024-05-15T15:54:17.371 INFO:tasks.workunit.client.0.smithi049.stdout:1/744: dread - d0/d4/dd/de/d1e/d16/d6e/f100 zero size 2024-05-15T15:54:17.378 INFO:tasks.workunit.client.0.smithi049.stdout:1/745: dwrite d0/d4/dd/de/d48/d5c/fe0 [0,4194304] 0 2024-05-15T15:54:17.380 INFO:tasks.workunit.client.0.smithi049.stdout:1/746: fsync d0/d4/dd/de/d9e/f105 0 2024-05-15T15:54:17.381 INFO:tasks.workunit.client.0.smithi049.stdout:2/552: creat d3/da/fb4 x:0 0 0 2024-05-15T15:54:17.384 INFO:tasks.workunit.client.0.smithi049.stdout:1/747: symlink d0/d4/dd/de/d1e/d16/l10c 0 2024-05-15T15:54:17.385 INFO:tasks.workunit.client.0.smithi049.stdout:2/553: fsync d3/da/d15/d24/f64 0 2024-05-15T15:54:17.387 INFO:tasks.workunit.client.0.smithi049.stdout:2/554: write d3/db/f30 [1601806,85781] 0 2024-05-15T15:54:17.389 INFO:tasks.workunit.client.1.smithi144.stdout:7/629: truncate d4/d3c/fa3 2318341 0 2024-05-15T15:54:17.389 INFO:tasks.workunit.client.1.smithi144.stdout:7/630: chown d4/f27 15562658 1 2024-05-15T15:54:17.390 INFO:tasks.workunit.client.1.smithi144.stdout:7/631: chown c0 42 1 2024-05-15T15:54:17.390 INFO:tasks.workunit.client.0.smithi049.stdout:2/555: mknod d3/db/d1d/d1e/d2b/d3b/cb5 0 2024-05-15T15:54:17.393 INFO:tasks.workunit.client.1.smithi144.stdout:7/632: mknod d4/d3c/d76/d9d/dac/d48/d72/d82/ccd 0 2024-05-15T15:54:17.393 INFO:tasks.workunit.client.0.smithi049.stdout:2/556: read d3/f11 [1829692,5177] 0 2024-05-15T15:54:17.394 INFO:tasks.workunit.client.1.smithi144.stdout:6/623: creat d7/db/da2/fda x:0 0 0 2024-05-15T15:54:17.402 INFO:tasks.workunit.client.0.smithi049.stdout:2/557: symlink d3/db/d1d/d1e/d2b/d3b/lb6 0 2024-05-15T15:54:17.403 INFO:tasks.workunit.client.0.smithi049.stdout:1/748: link d0/d4/dd/de/d4d/lbd d0/d4/dd/de/d48/d5c/l10d 0 2024-05-15T15:54:17.403 INFO:tasks.workunit.client.1.smithi144.stdout:7/633: creat d4/d3c/d76/d9d/dac/d48/d72/da2/daf/fce x:0 0 0 2024-05-15T15:54:17.405 INFO:tasks.workunit.client.1.smithi144.stdout:7/634: fdatasync d4/d3c/d76/d9d/dac/d48/fb9 0 2024-05-15T15:54:17.406 INFO:tasks.workunit.client.0.smithi049.stdout:1/749: getdents d0/d4/d81 0 2024-05-15T15:54:17.406 INFO:tasks.workunit.client.1.smithi144.stdout:6/624: dwrite d7/db/d11/d3c/f77 [4194304,4194304] 0 2024-05-15T15:54:17.406 INFO:tasks.workunit.client.0.smithi049.stdout:1/750: read d0/d4/f9 [120063,54787] 0 2024-05-15T15:54:17.409 INFO:tasks.workunit.client.1.smithi144.stdout:6/625: mknod d7/da6/cdb 0 2024-05-15T15:54:17.411 INFO:tasks.workunit.client.1.smithi144.stdout:7/635: fsync d4/df/d25/d40/d62/dbc/fc3 0 2024-05-15T15:54:17.415 INFO:tasks.workunit.client.1.smithi144.stdout:6/626: mknod d7/db/d11/cdc 0 2024-05-15T15:54:17.416 INFO:tasks.workunit.client.1.smithi144.stdout:7/636: dwrite d4/f12 [4194304,4194304] 0 2024-05-15T15:54:17.417 INFO:tasks.workunit.client.1.smithi144.stdout:6/627: write d7/d62/f95 [900657,117191] 0 2024-05-15T15:54:17.417 INFO:tasks.workunit.client.1.smithi144.stdout:7/637: readlink d4/df/d81/l90 0 2024-05-15T15:54:17.421 INFO:tasks.workunit.client.0.smithi049.stdout:1/751: rmdir d0/d4/dd/de/d1e/d2e/d3e/d90/db3/de5 0 2024-05-15T15:54:17.424 INFO:tasks.workunit.client.1.smithi144.stdout:6/628: unlink d7/d1e/d28/f96 0 2024-05-15T15:54:17.425 INFO:tasks.workunit.client.1.smithi144.stdout:7/638: fdatasync d4/d41/f49 0 2024-05-15T15:54:17.427 INFO:tasks.workunit.client.1.smithi144.stdout:0/637: write d4/d2d/f2e [1145804,115602] 0 2024-05-15T15:54:17.431 INFO:tasks.workunit.client.1.smithi144.stdout:7/639: creat d4/df/d79/fcf x:0 0 0 2024-05-15T15:54:17.432 INFO:tasks.workunit.client.1.smithi144.stdout:6/629: symlink d7/d1e/d28/da8/ldd 0 2024-05-15T15:54:17.436 INFO:tasks.workunit.client.1.smithi144.stdout:7/640: link d4/d3c/d76/d9d/dac/c7d d4/d3c/d76/d9d/dac/d48/d72/da2/cd0 0 2024-05-15T15:54:17.438 INFO:tasks.workunit.client.0.smithi049.stdout:1/752: dread d0/d4/dd/de/d48/d5c/fc6 [0,4194304] 0 2024-05-15T15:54:17.439 INFO:tasks.workunit.client.1.smithi144.stdout:7/641: mknod d4/df/d25/d40/d62/dbc/cd1 0 2024-05-15T15:54:17.439 INFO:tasks.workunit.client.1.smithi144.stdout:6/630: dread d7/db/d35/f67 [0,4194304] 0 2024-05-15T15:54:17.440 INFO:tasks.workunit.client.1.smithi144.stdout:7/642: write d4/d3c/d76/d9d/dac/d48/d72/da2/fbe [353455,72152] 0 2024-05-15T15:54:17.441 INFO:tasks.workunit.client.1.smithi144.stdout:6/631: write d7/f41 [403461,43793] 0 2024-05-15T15:54:17.441 INFO:tasks.workunit.client.1.smithi144.stdout:5/663: mkdir d0/dc/d24/d8e/d93/ddb 0 2024-05-15T15:54:17.443 INFO:tasks.workunit.client.0.smithi049.stdout:0/572: mknod dc/d13/d2b/cb6 0 2024-05-15T15:54:17.444 INFO:tasks.workunit.client.1.smithi144.stdout:7/643: chown d4/df/d25/d38/d68/f74 120248921 1 2024-05-15T15:54:17.445 INFO:tasks.workunit.client.1.smithi144.stdout:6/632: creat d7/d32/fde x:0 0 0 2024-05-15T15:54:17.445 INFO:tasks.workunit.client.1.smithi144.stdout:9/676: dwrite d0/d8/d1b/d79/d8a/f97 [0,4194304] 0 2024-05-15T15:54:17.447 INFO:tasks.workunit.client.1.smithi144.stdout:2/607: unlink d3/d7/d20/l5c 0 2024-05-15T15:54:17.448 INFO:tasks.workunit.client.0.smithi049.stdout:0/573: creat dc/d13/d3d/d84/fb7 x:0 0 0 2024-05-15T15:54:17.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:17 smithi144 ceph-mon[36389]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:17.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:17 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/193574544' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:54:17.452 INFO:tasks.workunit.client.1.smithi144.stdout:4/647: creat d6/dd/fd2 x:0 0 0 2024-05-15T15:54:17.454 INFO:tasks.workunit.client.0.smithi049.stdout:0/574: rmdir dc/d13/d2b 39 2024-05-15T15:54:17.456 INFO:tasks.workunit.client.1.smithi144.stdout:6/633: stat d7/db/d3a/l93 0 2024-05-15T15:54:17.461 INFO:tasks.workunit.client.0.smithi049.stdout:8/555: dwrite d2/de/d25/d74/f9d [0,4194304] 0 2024-05-15T15:54:17.461 INFO:tasks.workunit.client.0.smithi049.stdout:8/556: chown d2/c75 7 1 2024-05-15T15:54:17.462 INFO:tasks.workunit.client.0.smithi049.stdout:8/557: stat d2/de/d71/d86/d5d/d7d/d1b/d69/dac 0 2024-05-15T15:54:17.463 INFO:tasks.workunit.client.1.smithi144.stdout:6/634: mknod d7/d1e/d28/da8/cdf 0 2024-05-15T15:54:17.464 INFO:tasks.workunit.client.1.smithi144.stdout:6/635: truncate d7/f41 959156 0 2024-05-15T15:54:17.465 INFO:tasks.workunit.client.1.smithi144.stdout:6/636: chown d7/db/d35/d84 291 1 2024-05-15T15:54:17.465 INFO:tasks.workunit.client.0.smithi049.stdout:0/575: creat dc/d13/d3d/fb8 x:0 0 0 2024-05-15T15:54:17.466 INFO:tasks.workunit.client.1.smithi144.stdout:4/648: dwrite d6/dd/d34/d3b/d30/f52 [0,4194304] 0 2024-05-15T15:54:17.466 INFO:tasks.workunit.client.1.smithi144.stdout:9/677: getdents d0/d8/d1b/d79/d8a/dae/ddb 0 2024-05-15T15:54:17.467 INFO:tasks.workunit.client.1.smithi144.stdout:9/678: chown d0/d8/d5e/d9b/fdc 99562 1 2024-05-15T15:54:17.469 INFO:tasks.workunit.client.1.smithi144.stdout:2/608: symlink d3/d7/d20/d89/d86/db1/lcc 0 2024-05-15T15:54:17.472 INFO:tasks.workunit.client.1.smithi144.stdout:6/637: dread d7/db/d11/d3c/d61/fc5 [0,4194304] 0 2024-05-15T15:54:17.473 INFO:tasks.workunit.client.1.smithi144.stdout:6/638: read - d7/db/da2/fda zero size 2024-05-15T15:54:17.474 INFO:tasks.workunit.client.0.smithi049.stdout:0/576: fdatasync dc/d13/d2b/f81 0 2024-05-15T15:54:17.474 INFO:tasks.workunit.client.1.smithi144.stdout:6/639: fdatasync d7/db/d11/d3c/f79 0 2024-05-15T15:54:17.475 INFO:tasks.workunit.client.1.smithi144.stdout:6/640: dread - d7/db/d35/fb1 zero size 2024-05-15T15:54:17.478 INFO:tasks.workunit.client.0.smithi049.stdout:0/577: truncate fa 450390 0 2024-05-15T15:54:17.480 INFO:tasks.workunit.client.1.smithi144.stdout:9/679: mkdir d0/d8/d1b/d79/d8a/dae/ddb/de4 0 2024-05-15T15:54:17.481 INFO:tasks.workunit.client.1.smithi144.stdout:9/680: stat d0/d8/d5e/d44/d10/d19/c7b 0 2024-05-15T15:54:17.484 INFO:tasks.workunit.client.1.smithi144.stdout:6/641: creat d7/db/da2/fe0 x:0 0 0 2024-05-15T15:54:17.485 INFO:tasks.workunit.client.1.smithi144.stdout:2/609: link d3/d7/la9 d3/d7/d20/d22/d52/d75/lcd 0 2024-05-15T15:54:17.486 INFO:tasks.workunit.client.0.smithi049.stdout:3/587: rename d12/d30/f90 to d12/d14/d32/d64/d88/fb3 0 2024-05-15T15:54:17.488 INFO:tasks.workunit.client.1.smithi144.stdout:3/657: dwrite d0/d11/d24/f49 [0,4194304] 0 2024-05-15T15:54:17.488 INFO:tasks.workunit.client.1.smithi144.stdout:9/681: mkdir d0/d8/d5e/d44/d10/d19/dd5/de5 0 2024-05-15T15:54:17.489 INFO:tasks.workunit.client.1.smithi144.stdout:1/641: rename d7/d31 to d7/d1d/d33/d29/dd2 0 2024-05-15T15:54:17.490 INFO:tasks.workunit.client.0.smithi049.stdout:6/537: rmdir dd/d17/d24/d2b/d36/d50 39 2024-05-15T15:54:17.494 INFO:tasks.workunit.client.1.smithi144.stdout:8/711: creat d2/de/fea x:0 0 0 2024-05-15T15:54:17.495 INFO:tasks.workunit.client.1.smithi144.stdout:6/642: mknod d7/db/d11/d3c/ce1 0 2024-05-15T15:54:17.496 INFO:tasks.workunit.client.0.smithi049.stdout:8/558: rmdir d2/de/d71/d86/d5d/d7d 39 2024-05-15T15:54:17.497 INFO:tasks.workunit.client.0.smithi049.stdout:8/559: chown d2/de/d25/d74/f76 116268910 1 2024-05-15T15:54:17.498 INFO:tasks.workunit.client.0.smithi049.stdout:6/538: mkdir dd/d17/d9d 0 2024-05-15T15:54:17.498 INFO:tasks.workunit.client.0.smithi049.stdout:8/560: readlink d2/de/d71/d86/d5d/l91 0 2024-05-15T15:54:17.499 INFO:tasks.workunit.client.1.smithi144.stdout:3/658: fdatasync d0/d1/d6c/f99 0 2024-05-15T15:54:17.501 INFO:tasks.workunit.client.0.smithi049.stdout:9/454: rename d0/d8/d9/d52/d5e/d8a/f16 to d0/fa2 0 2024-05-15T15:54:17.504 INFO:tasks.workunit.client.0.smithi049.stdout:4/585: write d18/d1e/d2a/d50/f73 [163079,51990] 0 2024-05-15T15:54:17.506 INFO:tasks.workunit.client.0.smithi049.stdout:8/561: chown d2/de/d71/d86/d5d/d7d/d1b/f2b 16 1 2024-05-15T15:54:17.507 INFO:tasks.workunit.client.0.smithi049.stdout:8/562: dread - d2/de/d71/d86/d5d/d7d/d1b/d68/fab zero size 2024-05-15T15:54:17.508 INFO:tasks.workunit.client.1.smithi144.stdout:0/638: rename d4/d6/d3a/d4c/c73 to d4/d6/d4a/d4d/dc7/ce6 0 2024-05-15T15:54:17.509 INFO:tasks.workunit.client.1.smithi144.stdout:6/643: dread d7/db/d11/f43 [0,4194304] 0 2024-05-15T15:54:17.509 INFO:tasks.workunit.client.0.smithi049.stdout:2/558: rename d3/db/d1d/d1e/d2b/d3b/d42/f92 to d3/db/d1d/d1e/d2b/d2e/fb7 0 2024-05-15T15:54:17.515 INFO:tasks.workunit.client.0.smithi049.stdout:6/539: mknod dd/d17/d24/d2b/d36/d50/d7e/c9e 0 2024-05-15T15:54:17.516 INFO:tasks.workunit.client.0.smithi049.stdout:2/559: dwrite d3/db/d1d/d1e/d2b/d3b/fa7 [0,4194304] 0 2024-05-15T15:54:17.517 INFO:tasks.workunit.client.0.smithi049.stdout:6/540: chown dd/d17/d24/d2b/d36/d50/f9a 5392 1 2024-05-15T15:54:17.519 INFO:tasks.workunit.client.0.smithi049.stdout:2/560: write d3/db/d82/da1/fb1 [370936,126585] 0 2024-05-15T15:54:17.519 INFO:tasks.workunit.client.0.smithi049.stdout:4/586: chown d18/d1e/d2a/d36/d86/la1 0 1 2024-05-15T15:54:17.524 INFO:tasks.workunit.client.0.smithi049.stdout:8/563: symlink d2/de/d25/laf 0 2024-05-15T15:54:17.525 INFO:tasks.workunit.client.1.smithi144.stdout:9/682: symlink d0/d8/d5e/d44/d16/db0/le6 0 2024-05-15T15:54:17.526 INFO:tasks.workunit.client.0.smithi049.stdout:8/564: write d2/de/d71/d86/d5d/d7d/d45/f8a [1583327,41571] 0 2024-05-15T15:54:17.528 INFO:tasks.workunit.client.1.smithi144.stdout:7/644: creat d4/d3c/d76/d9d/dac/d48/fd2 x:0 0 0 2024-05-15T15:54:17.529 INFO:tasks.workunit.client.0.smithi049.stdout:1/753: rename d0/d4/le2 to d0/d4/dd/de/d1e/d16/l10e 0 2024-05-15T15:54:17.530 INFO:tasks.workunit.client.1.smithi144.stdout:7/645: write d4/d3c/d76/d9d/dac/d48/d72/d82/f92 [1186100,59852] 0 2024-05-15T15:54:17.532 INFO:tasks.workunit.client.1.smithi144.stdout:6/644: mkdir d7/d1e/d28/db5/de2 0 2024-05-15T15:54:17.532 INFO:tasks.workunit.client.1.smithi144.stdout:6/645: readlink d7/d1e/d28/l34 0 2024-05-15T15:54:17.533 INFO:tasks.workunit.client.1.smithi144.stdout:2/610: link d3/d7/c9e d3/d7/d20/d89/d6e/cce 0 2024-05-15T15:54:17.534 INFO:tasks.workunit.client.1.smithi144.stdout:2/611: write d3/d7/d20/d37/f46 [2606775,60449] 0 2024-05-15T15:54:17.536 INFO:tasks.workunit.client.1.smithi144.stdout:8/712: unlink d2/d24/d2d/d30/l55 0 2024-05-15T15:54:17.540 INFO:tasks.workunit.client.0.smithi049.stdout:3/588: rename d12/f22 to d12/d14/d32/d64/d88/d9c/fb4 0 2024-05-15T15:54:17.542 INFO:tasks.workunit.client.0.smithi049.stdout:2/561: creat d3/d8/daa/fb8 x:0 0 0 2024-05-15T15:54:17.544 INFO:tasks.workunit.client.0.smithi049.stdout:1/754: mkdir d0/d4/d10f 0 2024-05-15T15:54:17.546 INFO:tasks.workunit.client.1.smithi144.stdout:9/683: dwrite d0/d8/d1b/d62/d6b/fa7 [0,4194304] 0 2024-05-15T15:54:17.546 INFO:tasks.workunit.client.0.smithi049.stdout:1/755: write d0/d4/dd/de/d4e/f76 [1859308,90337] 0 2024-05-15T15:54:17.546 INFO:tasks.workunit.client.1.smithi144.stdout:9/684: stat d0/d8/d5e/d44/d99/lcc 0 2024-05-15T15:54:17.547 INFO:tasks.workunit.client.0.smithi049.stdout:1/756: chown d0/d4/dd/de/d48/laf 4818 1 2024-05-15T15:54:17.548 INFO:tasks.workunit.client.1.smithi144.stdout:7/646: mknod d4/df/d25/d40/d62/dbc/cd3 0 2024-05-15T15:54:17.548 INFO:tasks.workunit.client.1.smithi144.stdout:7/647: chown d4/f67 5133023 1 2024-05-15T15:54:17.550 INFO:tasks.workunit.client.1.smithi144.stdout:6/646: mkdir d7/db/d3a/de3 0 2024-05-15T15:54:17.553 INFO:tasks.workunit.client.0.smithi049.stdout:6/541: rename fb to dd/d17/d24/d58/d5f/f9f 0 2024-05-15T15:54:17.554 INFO:tasks.workunit.client.0.smithi049.stdout:6/542: readlink dd/l25 0 2024-05-15T15:54:17.556 INFO:tasks.workunit.client.1.smithi144.stdout:8/713: mkdir d2/d24/d2d/da0/d6e/deb 0 2024-05-15T15:54:17.557 INFO:tasks.workunit.client.1.smithi144.stdout:8/714: chown d2/de/d36/f95 17 1 2024-05-15T15:54:17.557 INFO:tasks.workunit.client.1.smithi144.stdout:8/715: stat d2/de/l50 0 2024-05-15T15:54:17.558 INFO:tasks.workunit.client.1.smithi144.stdout:5/664: truncate d0/dc/d1f/f38 3391000 0 2024-05-15T15:54:17.558 INFO:tasks.workunit.client.1.smithi144.stdout:0/639: creat d4/fe7 x:0 0 0 2024-05-15T15:54:17.560 INFO:tasks.workunit.client.0.smithi049.stdout:1/757: mkdir d0/d4/dd/de/d1e/d16/d8a/dd2/d110 0 2024-05-15T15:54:17.561 INFO:tasks.workunit.client.1.smithi144.stdout:4/649: write d6/d42/f58 [1858114,52578] 0 2024-05-15T15:54:17.562 INFO:tasks.workunit.client.0.smithi049.stdout:2/562: mkdir d3/db/d1d/d9f/db9 0 2024-05-15T15:54:17.563 INFO:tasks.workunit.client.1.smithi144.stdout:5/665: dwrite d0/f49 [0,4194304] 0 2024-05-15T15:54:17.564 INFO:tasks.workunit.client.1.smithi144.stdout:4/650: write d6/dd/d34/d3b/d30/db/f33 [37932,64765] 0 2024-05-15T15:54:17.567 INFO:tasks.workunit.client.1.smithi144.stdout:2/612: mkdir d3/d7/d20/dcf 0 2024-05-15T15:54:17.567 INFO:tasks.workunit.client.0.smithi049.stdout:0/578: write dc/f1d [516106,18932] 0 2024-05-15T15:54:17.570 INFO:tasks.workunit.client.0.smithi049.stdout:6/543: creat dd/d17/d64/d8c/fa0 x:0 0 0 2024-05-15T15:54:17.575 INFO:tasks.workunit.client.0.smithi049.stdout:1/758: dread d0/f5b [0,4194304] 0 2024-05-15T15:54:17.582 INFO:tasks.workunit.client.0.smithi049.stdout:2/563: mknod d3/d8/dad/cba 0 2024-05-15T15:54:17.584 INFO:tasks.workunit.client.1.smithi144.stdout:7/648: creat d4/df/d25/d40/d62/dbc/dbd/fd4 x:0 0 0 2024-05-15T15:54:17.591 INFO:tasks.workunit.client.0.smithi049.stdout:2/564: rename d3/da/d15/f5c to d3/db/d1d/d1e/d2b/d2e/fbb 0 2024-05-15T15:54:17.595 INFO:tasks.workunit.client.1.smithi144.stdout:4/651: creat d6/d42/d46/d4b/d1e/d99/fd3 x:0 0 0 2024-05-15T15:54:17.595 INFO:tasks.workunit.client.0.smithi049.stdout:9/455: dwrite d0/d8/d1b/d40/f58 [0,4194304] 0 2024-05-15T15:54:17.596 INFO:tasks.workunit.client.0.smithi049.stdout:0/579: dread dc/d13/f17 [4194304,4194304] 0 2024-05-15T15:54:17.599 INFO:tasks.workunit.client.0.smithi049.stdout:1/759: dread d0/d4/dd/de/f13 [0,4194304] 0 2024-05-15T15:54:17.601 INFO:tasks.workunit.client.1.smithi144.stdout:3/659: dwrite d0/d1/d17/d39/f8c [0,4194304] 0 2024-05-15T15:54:17.601 INFO:tasks.workunit.client.0.smithi049.stdout:2/565: rename d3/d8/f14 to d3/db/d82/fbc 0 2024-05-15T15:54:17.614 INFO:tasks.workunit.client.0.smithi049.stdout:0/580: mkdir dc/d13/d14/d25/db9 0 2024-05-15T15:54:17.616 INFO:tasks.workunit.client.0.smithi049.stdout:0/581: read dc/d13/d14/d33/f47 [3134268,100551] 0 2024-05-15T15:54:17.618 INFO:tasks.workunit.client.1.smithi144.stdout:4/652: creat d6/dd/d34/d3b/d30/db/d1d/d4e/fd4 x:0 0 0 2024-05-15T15:54:17.619 INFO:tasks.workunit.client.1.smithi144.stdout:4/653: write d6/dd/d34/d3b/d30/d59/f74 [2724409,12217] 0 2024-05-15T15:54:17.621 INFO:tasks.workunit.client.1.smithi144.stdout:2/613: link d3/d7/d20/d22/d52/fa3 d3/d7/d20/d37/d76/db5/db6/dbf/fd0 0 2024-05-15T15:54:17.627 INFO:tasks.workunit.client.1.smithi144.stdout:8/716: rename d2/d24/d2d/d30/f60 to d2/d24/d2d/da0/d6e/d71/fec 0 2024-05-15T15:54:17.630 INFO:tasks.workunit.client.1.smithi144.stdout:1/642: truncate d7/d1d/d33/d41/d22/f70 302704 0 2024-05-15T15:54:17.631 INFO:tasks.workunit.client.1.smithi144.stdout:7/649: mknod d4/df/d25/cd5 0 2024-05-15T15:54:17.634 INFO:tasks.workunit.client.1.smithi144.stdout:9/685: truncate d0/f14 459300 0 2024-05-15T15:54:17.636 INFO:tasks.workunit.client.0.smithi049.stdout:6/544: write dd/f3a [2792451,24007] 0 2024-05-15T15:54:17.637 INFO:tasks.workunit.client.1.smithi144.stdout:6/647: getdents d7/db/d3a 0 2024-05-15T15:54:17.637 INFO:tasks.workunit.client.1.smithi144.stdout:0/640: write d4/df/d1f/d56/fa7 [432766,116935] 0 2024-05-15T15:54:17.639 INFO:tasks.workunit.client.1.smithi144.stdout:0/641: fsync d4/d6/d4a/d74/fc3 0 2024-05-15T15:54:17.639 INFO:tasks.workunit.client.0.smithi049.stdout:0/582: getdents dc/d13/d14/d25/db9 0 2024-05-15T15:54:17.642 INFO:tasks.workunit.client.0.smithi049.stdout:3/589: rename d12/d14/f4e to d12/d14/d32/d64/d88/fb5 0 2024-05-15T15:54:17.644 INFO:tasks.workunit.client.1.smithi144.stdout:5/666: write d0/dc/d24/d8e/d93/fa1 [996386,91993] 0 2024-05-15T15:54:17.645 INFO:tasks.workunit.client.1.smithi144.stdout:8/717: mknod d2/de/d49/d83/ced 0 2024-05-15T15:54:17.645 INFO:tasks.workunit.client.1.smithi144.stdout:7/650: dread d4/df/f32 [0,4194304] 0 2024-05-15T15:54:17.647 INFO:tasks.workunit.client.1.smithi144.stdout:3/660: creat d0/d1/d17/d39/dc8/fdc x:0 0 0 2024-05-15T15:54:17.651 INFO:tasks.workunit.client.1.smithi144.stdout:5/667: dwrite d0/dc/d24/d8e/d95/fb0 [0,4194304] 0 2024-05-15T15:54:17.769 INFO:tasks.workunit.client.0.smithi049.stdout:6/545: mkdir dd/d17/d9d/da1 0 2024-05-15T15:54:17.770 INFO:tasks.workunit.client.1.smithi144.stdout:9/686: mkdir d0/d8/d1b/d79/de7 0 2024-05-15T15:54:17.770 INFO:tasks.workunit.client.1.smithi144.stdout:9/687: fsync d0/f78 0 2024-05-15T15:54:17.771 INFO:tasks.workunit.client.1.smithi144.stdout:9/688: chown d0/d8/d5e/d44/d10/d9c/fc8 3 1 2024-05-15T15:54:17.772 INFO:tasks.workunit.client.0.smithi049.stdout:1/760: write d0/d4/dd/de/d1e/f37 [5200032,93095] 0 2024-05-15T15:54:17.773 INFO:tasks.workunit.client.0.smithi049.stdout:3/590: symlink d12/d30/d42/d7a/lb6 0 2024-05-15T15:54:17.774 INFO:tasks.workunit.client.0.smithi049.stdout:2/566: dwrite d3/db/f12 [0,4194304] 0 2024-05-15T15:54:17.776 INFO:tasks.workunit.client.0.smithi049.stdout:6/546: symlink dd/d26/la2 0 2024-05-15T15:54:17.778 INFO:tasks.workunit.client.0.smithi049.stdout:9/456: dwrite d0/d8/d1b/f32 [0,4194304] 0 2024-05-15T15:54:17.779 INFO:tasks.workunit.client.1.smithi144.stdout:0/642: creat d4/d2d/dae/fe8 x:0 0 0 2024-05-15T15:54:17.784 INFO:tasks.workunit.client.0.smithi049.stdout:3/591: write d12/d30/d42/f8e [1240644,113033] 0 2024-05-15T15:54:17.787 INFO:tasks.workunit.client.1.smithi144.stdout:2/614: link d3/d7/d20/d22/d52/d75/da6/dc1/fc6 d3/d7/d20/dcf/fd1 0 2024-05-15T15:54:17.794 INFO:tasks.workunit.client.1.smithi144.stdout:7/651: creat d4/df/d25/d38/d68/fd6 x:0 0 0 2024-05-15T15:54:17.795 INFO:tasks.workunit.client.1.smithi144.stdout:7/652: readlink d4/d45/l6c 0 2024-05-15T15:54:17.796 INFO:tasks.workunit.client.1.smithi144.stdout:7/653: truncate d4/df/d25/d40/d73/f9a 699763 0 2024-05-15T15:54:17.797 INFO:tasks.workunit.client.0.smithi049.stdout:9/457: rmdir d0/d8/d42/d75 39 2024-05-15T15:54:17.798 INFO:tasks.workunit.client.1.smithi144.stdout:4/654: write d6/dd/d34/d3b/d30/f1b [2066283,14428] 0 2024-05-15T15:54:17.798 INFO:tasks.workunit.client.0.smithi049.stdout:9/458: write d0/f65 [3965035,119776] 0 2024-05-15T15:54:17.799 INFO:tasks.workunit.client.1.smithi144.stdout:4/655: write d6/dd/d34/f9d [134985,122113] 0 2024-05-15T15:54:17.800 INFO:tasks.workunit.client.0.smithi049.stdout:9/459: chown d0/d8/d1b/f3a 16421747 1 2024-05-15T15:54:17.804 INFO:tasks.workunit.client.1.smithi144.stdout:1/643: write d7/d1d/d33/d55/f8d [236483,22037] 0 2024-05-15T15:54:17.805 INFO:tasks.workunit.client.1.smithi144.stdout:1/644: fsync d7/d1d/d33/d1e/fbe 0 2024-05-15T15:54:17.806 INFO:tasks.workunit.client.0.smithi049.stdout:0/583: dwrite dc/d13/d14/d25/d31/d71/f2c [0,4194304] 0 2024-05-15T15:54:17.811 INFO:tasks.workunit.client.1.smithi144.stdout:9/689: symlink d0/d8/d5e/le8 0 2024-05-15T15:54:17.811 INFO:tasks.workunit.client.1.smithi144.stdout:8/718: dread d2/d1a/f31 [0,4194304] 0 2024-05-15T15:54:17.812 INFO:tasks.workunit.client.1.smithi144.stdout:0/643: mknod d4/df/d1f/ce9 0 2024-05-15T15:54:17.813 INFO:tasks.workunit.client.1.smithi144.stdout:1/645: dwrite d7/d1d/d33/d55/f8d [0,4194304] 0 2024-05-15T15:54:17.815 INFO:tasks.workunit.client.1.smithi144.stdout:1/646: chown d7/d1d/d33/d29/d6b/c97 1869 1 2024-05-15T15:54:17.815 INFO:tasks.workunit.client.1.smithi144.stdout:6/648: dwrite d7/db/d11/d3c/f4d [0,4194304] 0 2024-05-15T15:54:17.817 INFO:tasks.workunit.client.1.smithi144.stdout:2/615: readlink d3/d7/l21 0 2024-05-15T15:54:17.818 INFO:tasks.workunit.client.0.smithi049.stdout:2/567: link d3/da/d15/d73/c81 d3/da/d15/d73/cbd 0 2024-05-15T15:54:17.819 INFO:tasks.workunit.client.1.smithi144.stdout:2/616: truncate d3/d7/d20/d37/f3c 4372695 0 2024-05-15T15:54:17.821 INFO:tasks.workunit.client.0.smithi049.stdout:6/547: getdents dd/d17/d64/d8c 0 2024-05-15T15:54:17.824 INFO:tasks.workunit.client.1.smithi144.stdout:7/654: unlink d4/d3c/d76/d9d/fa5 0 2024-05-15T15:54:17.826 INFO:tasks.workunit.client.0.smithi049.stdout:1/761: getdents d0/d4/dd/de/d1e/d16/d8a/dd2/df2 0 2024-05-15T15:54:17.828 INFO:tasks.workunit.client.1.smithi144.stdout:3/661: write d0/d11/d3b/f6b [432956,6918] 0 2024-05-15T15:54:17.830 INFO:tasks.workunit.client.1.smithi144.stdout:5/668: mknod d0/dc/d2f/d79/da5/cdc 0 2024-05-15T15:54:17.831 INFO:tasks.workunit.client.1.smithi144.stdout:5/669: stat d0/dc/d24/d8e/d93/d6b/d88 0 2024-05-15T15:54:17.832 INFO:tasks.workunit.client.0.smithi049.stdout:2/568: mknod d3/db/d1d/d1e/d2b/d2e/cbe 0 2024-05-15T15:54:17.832 INFO:tasks.workunit.client.0.smithi049.stdout:2/569: chown d3/d8/f36 2 1 2024-05-15T15:54:17.836 INFO:tasks.workunit.client.0.smithi049.stdout:1/762: rmdir d0/d4/dd/de/d1e/d2e/d3e/d90 39 2024-05-15T15:54:17.836 INFO:tasks.workunit.client.0.smithi049.stdout:1/763: dread - d0/d4/dd/de/d36/f108 zero size 2024-05-15T15:54:17.839 INFO:tasks.workunit.client.1.smithi144.stdout:8/719: mknod d2/d3a/d7b/d4b/cee 0 2024-05-15T15:54:17.840 INFO:tasks.workunit.client.0.smithi049.stdout:6/548: getdents dd/d17/d64/d8c 0 2024-05-15T15:54:17.846 INFO:tasks.workunit.client.1.smithi144.stdout:8/720: dwrite d2/d3a/d7b/d4b/fa9 [0,4194304] 0 2024-05-15T15:54:17.846 INFO:tasks.workunit.client.1.smithi144.stdout:6/649: creat d7/db/d3a/fe4 x:0 0 0 2024-05-15T15:54:17.846 INFO:tasks.workunit.client.1.smithi144.stdout:9/690: write d0/f7 [1994212,21189] 0 2024-05-15T15:54:17.848 INFO:tasks.workunit.client.0.smithi049.stdout:6/549: creat dd/d17/d24/d2b/fa3 x:0 0 0 2024-05-15T15:54:17.849 INFO:tasks.workunit.client.1.smithi144.stdout:6/650: dread d7/db/d11/d3c/f79 [0,4194304] 0 2024-05-15T15:54:17.853 INFO:tasks.workunit.client.0.smithi049.stdout:0/584: getdents dc/d13/d56/d6f 0 2024-05-15T15:54:17.855 INFO:tasks.workunit.client.1.smithi144.stdout:9/691: dwrite d0/d8/d1b/d62/d6b/fa7 [0,4194304] 0 2024-05-15T15:54:17.856 INFO:tasks.workunit.client.1.smithi144.stdout:7/655: mknod d4/df/d25/d38/d5d/cd7 0 2024-05-15T15:54:17.856 INFO:tasks.workunit.client.1.smithi144.stdout:9/692: stat d0/d8/d5e/d44/d16 0 2024-05-15T15:54:17.857 INFO:tasks.workunit.client.0.smithi049.stdout:1/764: dread d0/fd1 [0,4194304] 0 2024-05-15T15:54:17.857 INFO:tasks.workunit.client.1.smithi144.stdout:7/656: stat d4/f21 0 2024-05-15T15:54:17.858 INFO:tasks.workunit.client.0.smithi049.stdout:2/570: dwrite d3/db/d1d/d1e/f3c [4194304,4194304] 0 2024-05-15T15:54:17.859 INFO:tasks.workunit.client.1.smithi144.stdout:7/657: chown d4/d41/d9c/caa 39561 1 2024-05-15T15:54:17.860 INFO:tasks.workunit.client.1.smithi144.stdout:3/662: creat d0/d1/d6c/fdd x:0 0 0 2024-05-15T15:54:17.860 INFO:tasks.workunit.client.1.smithi144.stdout:2/617: dwrite d3/d7/f3d [0,4194304] 0 2024-05-15T15:54:17.861 INFO:tasks.workunit.client.0.smithi049.stdout:0/585: symlink dc/d13/d56/lba 0 2024-05-15T15:54:17.861 INFO:tasks.workunit.client.0.smithi049.stdout:1/765: chown d0/d4/dd/de/d1e/d16/d8a/dd2 32873339 1 2024-05-15T15:54:17.862 INFO:tasks.workunit.client.1.smithi144.stdout:5/670: mknod d0/dc/d1f/d50/db3/cdd 0 2024-05-15T15:54:17.862 INFO:tasks.workunit.client.0.smithi049.stdout:6/550: dwrite dd/d17/f8d [0,4194304] 0 2024-05-15T15:54:17.865 INFO:tasks.workunit.client.0.smithi049.stdout:0/586: write dc/d13/d14/d2f/d8f/fa7 [907256,127045] 0 2024-05-15T15:54:17.865 INFO:tasks.workunit.client.0.smithi049.stdout:6/551: read f0 [3365834,74048] 0 2024-05-15T15:54:17.870 INFO:tasks.workunit.client.0.smithi049.stdout:2/571: mkdir d3/db/d5b/dbf 0 2024-05-15T15:54:17.872 INFO:tasks.workunit.client.1.smithi144.stdout:3/663: dwrite d0/d1/d17/d39/f8c [0,4194304] 0 2024-05-15T15:54:17.878 INFO:tasks.workunit.client.1.smithi144.stdout:4/656: fsync d6/dd/d2d/dba/fbe 0 2024-05-15T15:54:17.881 INFO:tasks.workunit.client.0.smithi049.stdout:6/552: unlink dd/d17/d24/d58/l77 0 2024-05-15T15:54:17.885 INFO:tasks.workunit.client.1.smithi144.stdout:7/658: mkdir d4/d3c/d76/d9d/dd8 0 2024-05-15T15:54:17.887 INFO:tasks.workunit.client.1.smithi144.stdout:4/657: dwrite d6/dd/d2d/f48 [0,4194304] 0 2024-05-15T15:54:17.887 INFO:tasks.workunit.client.1.smithi144.stdout:7/659: readlink d4/d41/d5b/l5c 0 2024-05-15T15:54:17.887 INFO:tasks.workunit.client.1.smithi144.stdout:2/618: creat d3/d7/d20/d3b/fd2 x:0 0 0 2024-05-15T15:54:17.887 INFO:tasks.workunit.client.1.smithi144.stdout:1/647: dread f1 [4194304,4194304] 0 2024-05-15T15:54:17.888 INFO:tasks.workunit.client.0.smithi049.stdout:0/587: creat dc/d13/fbb x:0 0 0 2024-05-15T15:54:17.889 INFO:tasks.workunit.client.0.smithi049.stdout:0/588: chown dc/d13/d3d/d84/la8 120727 1 2024-05-15T15:54:17.890 INFO:tasks.workunit.client.1.smithi144.stdout:7/660: dread - d4/d3c/d76/d9d/dac/d48/d72/d82/fad zero size 2024-05-15T15:54:17.890 INFO:tasks.workunit.client.0.smithi049.stdout:6/553: symlink dd/d17/d24/d2b/d36/d50/d7e/la4 0 2024-05-15T15:54:17.893 INFO:tasks.workunit.client.1.smithi144.stdout:5/671: rename d0/dc/d1f/c7b to d0/dc/d24/d57/d96/cde 0 2024-05-15T15:54:17.894 INFO:tasks.workunit.client.0.smithi049.stdout:2/572: creat d3/da/d15/fc0 x:0 0 0 2024-05-15T15:54:17.896 INFO:tasks.workunit.client.0.smithi049.stdout:2/573: dread - d3/db/d1d/d1e/d2b/d2e/f72 zero size 2024-05-15T15:54:17.902 INFO:tasks.workunit.client.1.smithi144.stdout:8/721: creat d2/d24/d2d/da0/d9e/db8/dde/fef x:0 0 0 2024-05-15T15:54:17.904 INFO:tasks.workunit.client.1.smithi144.stdout:8/722: write d2/d24/d2d/da0/d6e/d71/d8c/fc0 [390665,92118] 0 2024-05-15T15:54:17.906 INFO:tasks.workunit.client.0.smithi049.stdout:2/574: rename d3/db/d82/da1 to d3/db/d5b/dbf/dc1 0 2024-05-15T15:54:17.911 INFO:tasks.workunit.client.1.smithi144.stdout:0/644: dwrite f2 [0,4194304] 0 2024-05-15T15:54:17.912 INFO:tasks.workunit.client.1.smithi144.stdout:2/619: creat d3/d7/d20/d22/d52/d64/fd3 x:0 0 0 2024-05-15T15:54:17.915 INFO:tasks.workunit.client.0.smithi049.stdout:6/554: link dd/d17/d24/c34 dd/d17/d24/d2b/ca5 0 2024-05-15T15:54:17.916 INFO:tasks.workunit.client.0.smithi049.stdout:6/555: fdatasync f4 0 2024-05-15T15:54:17.918 INFO:tasks.workunit.client.0.smithi049.stdout:1/766: dwrite d0/d4/dd/de/d36/f6a [0,4194304] 0 2024-05-15T15:54:17.919 INFO:tasks.workunit.client.1.smithi144.stdout:3/664: fsync d0/f46 0 2024-05-15T15:54:17.920 INFO:tasks.workunit.client.0.smithi049.stdout:1/767: chown d0/d4/dd/de/d1e/d2e/dad/d65/d72/f7a 61 1 2024-05-15T15:54:17.924 INFO:tasks.workunit.client.1.smithi144.stdout:6/651: creat d7/fe5 x:0 0 0 2024-05-15T15:54:17.926 INFO:tasks.workunit.client.1.smithi144.stdout:9/693: creat d0/d8/fe9 x:0 0 0 2024-05-15T15:54:17.926 INFO:tasks.workunit.client.1.smithi144.stdout:6/652: write d7/db/d35/f47 [723290,7415] 0 2024-05-15T15:54:17.926 INFO:tasks.workunit.client.1.smithi144.stdout:0/645: dread d4/d6/d3a/d40/f7d [0,4194304] 0 2024-05-15T15:54:17.927 INFO:tasks.workunit.client.1.smithi144.stdout:1/648: mkdir d7/d4b/d53/dd3 0 2024-05-15T15:54:17.932 INFO:tasks.workunit.client.1.smithi144.stdout:3/665: mknod d0/d11/d3b/d93/cde 0 2024-05-15T15:54:17.933 INFO:tasks.workunit.client.1.smithi144.stdout:3/666: write d0/d11/f26 [3867540,43483] 0 2024-05-15T15:54:17.937 INFO:tasks.workunit.client.1.smithi144.stdout:2/620: dread d3/d7/f27 [0,4194304] 0 2024-05-15T15:54:17.940 INFO:tasks.workunit.client.1.smithi144.stdout:1/649: dread d7/d1d/d33/d55/f5b [0,4194304] 0 2024-05-15T15:54:17.943 INFO:tasks.workunit.client.1.smithi144.stdout:3/667: creat d0/d1/d17/d39/d4d/fdf x:0 0 0 2024-05-15T15:54:17.944 INFO:tasks.workunit.client.1.smithi144.stdout:6/653: mknod d7/db/d11/d3c/ce6 0 2024-05-15T15:54:17.944 INFO:tasks.workunit.client.1.smithi144.stdout:3/668: chown d0/d1/d17/d39/d40/fc0 967269674 1 2024-05-15T15:54:17.944 INFO:tasks.workunit.client.1.smithi144.stdout:9/694: getdents d0/d8/d1b/d62/d7d/d8d/dbb/dce/dd1 0 2024-05-15T15:54:17.946 INFO:tasks.workunit.client.0.smithi049.stdout:6/556: creat dd/d17/d9d/fa6 x:0 0 0 2024-05-15T15:54:17.947 INFO:tasks.workunit.client.1.smithi144.stdout:6/654: dread d7/db/d35/f67 [0,4194304] 0 2024-05-15T15:54:17.947 INFO:tasks.workunit.client.0.smithi049.stdout:1/768: rename d0/d4/dd/de/d48/d5c/f94 to d0/d4/d63/f111 0 2024-05-15T15:54:17.948 INFO:tasks.workunit.client.1.smithi144.stdout:7/661: link d4/f13 d4/d3c/d76/d9d/dac/d48/d72/d82/fd9 0 2024-05-15T15:54:17.948 INFO:tasks.workunit.client.1.smithi144.stdout:2/621: dread d3/d7/d20/d22/d52/d64/f31 [0,4194304] 0 2024-05-15T15:54:17.949 INFO:tasks.workunit.client.0.smithi049.stdout:6/557: creat dd/d17/d64/fa7 x:0 0 0 2024-05-15T15:54:17.950 INFO:tasks.workunit.client.0.smithi049.stdout:1/769: truncate d0/d4/dd/de/d1e/d16/d6e/f100 36615 0 2024-05-15T15:54:17.950 INFO:tasks.workunit.client.1.smithi144.stdout:1/650: rename d7/d1d/d58/c62 to d7/d1d/d33/d29/d6b/d8e/cd4 0 2024-05-15T15:54:17.951 INFO:tasks.workunit.client.1.smithi144.stdout:1/651: read f1 [11218973,103789] 0 2024-05-15T15:54:17.951 INFO:tasks.workunit.client.1.smithi144.stdout:3/669: dwrite d0/d1/d17/d39/d40/d4a/d77/fb0 [0,4194304] 0 2024-05-15T15:54:17.951 INFO:tasks.workunit.client.0.smithi049.stdout:7/563: sync 2024-05-15T15:54:17.952 INFO:tasks.workunit.client.0.smithi049.stdout:6/558: rmdir dd/d17/d24/d2b/d36/d86/d96 39 2024-05-15T15:54:17.953 INFO:tasks.workunit.client.0.smithi049.stdout:6/559: write dd/d17/d24/d2b/f46 [68373,100029] 0 2024-05-15T15:54:17.955 INFO:tasks.workunit.client.1.smithi144.stdout:7/662: rmdir d4/df/d25/d40/d73 39 2024-05-15T15:54:17.957 INFO:tasks.workunit.client.1.smithi144.stdout:1/652: dwrite d7/d1d/d33/d29/dd2/f7b [0,4194304] 0 2024-05-15T15:54:17.959 INFO:tasks.workunit.client.0.smithi049.stdout:7/564: mknod d0/d6/d23/d76/cac 0 2024-05-15T15:54:17.960 INFO:tasks.workunit.client.1.smithi144.stdout:9/695: mknod d0/d8/d1b/d62/d7d/d8d/dbb/dce/dd1/cea 0 2024-05-15T15:54:17.966 INFO:tasks.workunit.client.0.smithi049.stdout:6/560: getdents dd/d17/d24/d58 0 2024-05-15T15:54:17.967 INFO:tasks.workunit.client.1.smithi144.stdout:6/655: truncate d7/db/f29 4164159 0 2024-05-15T15:54:17.968 INFO:tasks.workunit.client.0.smithi049.stdout:6/561: mkdir dd/d26/d5b/da8 0 2024-05-15T15:54:17.971 INFO:tasks.workunit.client.0.smithi049.stdout:6/562: creat dd/d17/d24/d2b/d36/d93/fa9 x:0 0 0 2024-05-15T15:54:17.971 INFO:tasks.workunit.client.1.smithi144.stdout:7/663: creat d4/d3c/d76/d9d/fda x:0 0 0 2024-05-15T15:54:17.972 INFO:tasks.workunit.client.1.smithi144.stdout:3/670: truncate d0/d11/d3b/f96 1224494 0 2024-05-15T15:54:17.972 INFO:tasks.workunit.client.1.smithi144.stdout:7/664: fsync d4/df/f32 0 2024-05-15T15:54:17.973 INFO:tasks.workunit.client.0.smithi049.stdout:6/563: fsync dd/d26/f4f 0 2024-05-15T15:54:17.974 INFO:tasks.workunit.client.1.smithi144.stdout:3/671: write d0/d1/f23 [298454,38381] 0 2024-05-15T15:54:17.974 INFO:tasks.workunit.client.1.smithi144.stdout:7/665: dread - d4/df/d79/fcc zero size 2024-05-15T15:54:17.976 INFO:tasks.workunit.client.1.smithi144.stdout:3/672: write d0/d1/d17/d39/dc8/fdc [941794,16935] 0 2024-05-15T15:54:17.977 INFO:tasks.workunit.client.1.smithi144.stdout:9/696: link d0/d8/d5e/d44/d10/d9c/f98 d0/d8/d1b/d79/d8a/dae/feb 0 2024-05-15T15:54:17.981 INFO:tasks.workunit.client.0.smithi049.stdout:6/564: dwrite dd/f32 [0,4194304] 0 2024-05-15T15:54:17.981 INFO:tasks.workunit.client.1.smithi144.stdout:7/666: mknod d4/d41/d5b/cdb 0 2024-05-15T15:54:17.982 INFO:tasks.workunit.client.1.smithi144.stdout:6/656: creat d7/fe7 x:0 0 0 2024-05-15T15:54:17.984 INFO:tasks.workunit.client.1.smithi144.stdout:1/653: symlink d7/d1d/d33/d1e/d87/ld5 0 2024-05-15T15:54:17.988 INFO:tasks.workunit.client.1.smithi144.stdout:3/673: creat d0/d11/d3b/da0/d8f/fe0 x:0 0 0 2024-05-15T15:54:17.989 INFO:tasks.workunit.client.1.smithi144.stdout:6/657: symlink d7/da6/le8 0 2024-05-15T15:54:17.989 INFO:tasks.workunit.client.1.smithi144.stdout:3/674: chown d0/d1/c5d 4054 1 2024-05-15T15:54:17.991 INFO:tasks.workunit.client.1.smithi144.stdout:1/654: dread d7/d1d/d33/d1e/fbe [0,4194304] 0 2024-05-15T15:54:17.991 INFO:tasks.workunit.client.1.smithi144.stdout:7/667: symlink d4/df/d25/d40/d62/dbc/dbd/ldc 0 2024-05-15T15:54:17.994 INFO:tasks.workunit.client.0.smithi049.stdout:0/589: dwrite dc/d13/d14/d25/d31/d71/f3a [0,4194304] 0 2024-05-15T15:54:17.995 INFO:tasks.workunit.client.1.smithi144.stdout:1/655: mknod d7/d1d/d58/cd6 0 2024-05-15T15:54:17.995 INFO:tasks.workunit.client.1.smithi144.stdout:1/656: stat d7/d1d/d33/d55/f8d 0 2024-05-15T15:54:17.996 INFO:tasks.workunit.client.1.smithi144.stdout:1/657: chown d7/d1d/d33/d29/f42 51 1 2024-05-15T15:54:17.999 INFO:tasks.workunit.client.0.smithi049.stdout:6/565: dread f2 [0,4194304] 0 2024-05-15T15:54:18.001 INFO:tasks.workunit.client.0.smithi049.stdout:6/566: write dd/d17/d24/d2b/d36/f98 [1007211,9579] 0 2024-05-15T15:54:18.005 INFO:tasks.workunit.client.1.smithi144.stdout:4/658: write d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/fb7 [2236439,19918] 0 2024-05-15T15:54:18.006 INFO:tasks.workunit.client.1.smithi144.stdout:7/668: dread d4/d3c/d76/d9d/dba/fc0 [0,4194304] 0 2024-05-15T15:54:18.007 INFO:tasks.workunit.client.0.smithi049.stdout:6/567: rename dd/l2a to dd/d26/d5b/laa 0 2024-05-15T15:54:18.008 INFO:tasks.workunit.client.0.smithi049.stdout:2/575: dwrite d3/d8/f36 [0,4194304] 0 2024-05-15T15:54:18.010 INFO:tasks.workunit.client.1.smithi144.stdout:7/669: symlink d4/d3c/d76/d9d/dac/d48/ldd 0 2024-05-15T15:54:18.012 INFO:tasks.workunit.client.0.smithi049.stdout:5/615: sync 2024-05-15T15:54:18.012 INFO:tasks.workunit.client.0.smithi049.stdout:3/592: sync 2024-05-15T15:54:18.012 INFO:tasks.workunit.client.0.smithi049.stdout:8/565: sync 2024-05-15T15:54:18.012 INFO:tasks.workunit.client.0.smithi049.stdout:4/587: sync 2024-05-15T15:54:18.012 INFO:tasks.workunit.client.0.smithi049.stdout:9/460: sync 2024-05-15T15:54:18.013 INFO:tasks.workunit.client.1.smithi144.stdout:8/723: write d2/d24/d2d/d30/f81 [56034,107046] 0 2024-05-15T15:54:18.013 INFO:tasks.workunit.client.1.smithi144.stdout:8/724: dread - d2/d3a/d7b/fdb zero size 2024-05-15T15:54:18.024 INFO:tasks.workunit.client.1.smithi144.stdout:6/658: dread d7/db/d11/f60 [0,4194304] 0 2024-05-15T15:54:18.024 INFO:tasks.workunit.client.1.smithi144.stdout:6/659: chown d7/db/d35/d84/lb9 402 1 2024-05-15T15:54:18.025 INFO:tasks.workunit.client.1.smithi144.stdout:7/670: rename d4/d41/lc6 to d4/d3c/d76/d9d/dba/lde 0 2024-05-15T15:54:18.025 INFO:tasks.workunit.client.1.smithi144.stdout:6/660: fdatasync d7/d1e/f44 0 2024-05-15T15:54:18.026 INFO:tasks.workunit.client.0.smithi049.stdout:3/593: dwrite d12/d67/fb0 [0,4194304] 0 2024-05-15T15:54:18.026 INFO:tasks.workunit.client.1.smithi144.stdout:6/661: stat d7/db/d11/d3c/c5c 0 2024-05-15T15:54:18.029 INFO:tasks.workunit.client.0.smithi049.stdout:4/588: chown d18/d1e/d2a/d50/l6f 1791 1 2024-05-15T15:54:18.032 INFO:tasks.workunit.client.0.smithi049.stdout:0/590: dread f3 [0,4194304] 0 2024-05-15T15:54:18.032 INFO:tasks.workunit.client.0.smithi049.stdout:0/591: stat dc/d13/d56/lba 0 2024-05-15T15:54:18.034 INFO:tasks.workunit.client.0.smithi049.stdout:8/566: symlink d2/de/d71/d86/d5d/d63/lb0 0 2024-05-15T15:54:18.038 INFO:tasks.workunit.client.0.smithi049.stdout:5/616: mkdir dc/dd/dc8 0 2024-05-15T15:54:18.040 INFO:tasks.workunit.client.1.smithi144.stdout:6/662: symlink d7/db/d3a/dd2/le9 0 2024-05-15T15:54:18.045 INFO:tasks.workunit.client.1.smithi144.stdout:6/663: dwrite d7/d32/f50 [0,4194304] 0 2024-05-15T15:54:18.045 INFO:tasks.workunit.client.1.smithi144.stdout:8/725: dread d2/d3a/f3e [0,4194304] 0 2024-05-15T15:54:18.048 INFO:tasks.workunit.client.0.smithi049.stdout:0/592: creat dc/d32/d93/fbc x:0 0 0 2024-05-15T15:54:18.057 INFO:tasks.workunit.client.1.smithi144.stdout:6/664: rename d7/db/d11/d3c/f77 to d7/db/d9d/fea 0 2024-05-15T15:54:18.059 INFO:tasks.workunit.client.1.smithi144.stdout:0/646: write d4/df/f16 [1936345,57432] 0 2024-05-15T15:54:18.066 INFO:tasks.workunit.client.0.smithi049.stdout:3/594: truncate d12/d26/f40 3631379 0 2024-05-15T15:54:18.066 INFO:tasks.workunit.client.0.smithi049.stdout:5/617: dwrite dc/d4a/d31/d5c/f7e [0,4194304] 0 2024-05-15T15:54:18.069 INFO:tasks.workunit.client.0.smithi049.stdout:5/618: fsync dc/dd/d39/fa0 0 2024-05-15T15:54:18.069 INFO:tasks.workunit.client.0.smithi049.stdout:3/595: dread d12/d67/fb0 [0,4194304] 0 2024-05-15T15:54:18.069 INFO:tasks.workunit.client.0.smithi049.stdout:9/461: getdents d0/d8/d42/d75 0 2024-05-15T15:54:18.071 INFO:tasks.workunit.client.0.smithi049.stdout:5/619: dread dc/d29/f87 [0,4194304] 0 2024-05-15T15:54:18.072 INFO:tasks.workunit.client.1.smithi144.stdout:2/622: write d3/f34 [2996193,63642] 0 2024-05-15T15:54:18.075 INFO:tasks.workunit.client.1.smithi144.stdout:2/623: truncate d3/d7/d20/d22/d52/d75/d65/fc8 989118 0 2024-05-15T15:54:18.075 INFO:tasks.workunit.client.0.smithi049.stdout:1/770: dwrite d0/d4/dd/de/d1e/d2e/dad/d65/fc7 [0,4194304] 0 2024-05-15T15:54:18.080 INFO:tasks.workunit.client.0.smithi049.stdout:9/462: chown d0/d8/d12/l3f 53177358 1 2024-05-15T15:54:18.081 INFO:tasks.workunit.client.1.smithi144.stdout:0/647: rmdir d4/df 39 2024-05-15T15:54:18.081 INFO:tasks.workunit.client.0.smithi049.stdout:0/593: creat dc/d13/d2b/d76/d8a/da2/fbd x:0 0 0 2024-05-15T15:54:18.081 INFO:tasks.workunit.client.1.smithi144.stdout:6/665: mknod d7/d1e/ceb 0 2024-05-15T15:54:18.082 INFO:tasks.workunit.client.1.smithi144.stdout:0/648: stat d4/d6/d4a/d4d/db1 0 2024-05-15T15:54:18.083 INFO:tasks.workunit.client.1.smithi144.stdout:6/666: write d7/db/d35/d84/d81/fca [1787927,14771] 0 2024-05-15T15:54:18.087 INFO:tasks.workunit.client.0.smithi049.stdout:7/565: write d0/d2f/f48 [325655,27220] 0 2024-05-15T15:54:18.091 INFO:tasks.workunit.client.0.smithi049.stdout:2/576: rename d3/da/d15/d73/cbd to d3/da/d15/cc2 0 2024-05-15T15:54:18.091 INFO:tasks.workunit.client.0.smithi049.stdout:2/577: truncate d3/da/fb4 575076 0 2024-05-15T15:54:18.104 INFO:tasks.workunit.client.1.smithi144.stdout:6/667: symlink d7/db/d3a/lec 0 2024-05-15T15:54:18.106 INFO:tasks.workunit.client.1.smithi144.stdout:1/658: readlink d7/d1d/d33/d1e/d87/ld5 0 2024-05-15T15:54:18.107 INFO:tasks.workunit.client.1.smithi144.stdout:3/675: fsync d0/d11/d3b/da0/d8f/fe0 0 2024-05-15T15:54:18.110 INFO:tasks.workunit.client.1.smithi144.stdout:2/624: rename d3/d7/d20/d22/d52/d64/l41 to d3/d7/d20/d22/ld4 0 2024-05-15T15:54:18.110 INFO:tasks.workunit.client.1.smithi144.stdout:1/659: dwrite d7/d1d/f32 [0,4194304] 0 2024-05-15T15:54:18.112 INFO:tasks.workunit.client.1.smithi144.stdout:1/660: write d7/d1d/f32 [974346,94899] 0 2024-05-15T15:54:18.113 INFO:tasks.workunit.client.1.smithi144.stdout:1/661: chown d7/d1d/d33/d29/dd2/l39 0 1 2024-05-15T15:54:18.114 INFO:tasks.workunit.client.1.smithi144.stdout:1/662: readlink d7/d1d/d33/d29/l8a 0 2024-05-15T15:54:18.116 INFO:tasks.workunit.client.1.smithi144.stdout:6/668: unlink d7/db/d35/d84/f98 0 2024-05-15T15:54:18.116 INFO:tasks.workunit.client.0.smithi049.stdout:5/620: dwrite dc/f6c [0,4194304] 0 2024-05-15T15:54:18.117 INFO:tasks.workunit.client.0.smithi049.stdout:5/621: stat dc/d4a/f62 0 2024-05-15T15:54:18.118 INFO:tasks.workunit.client.0.smithi049.stdout:0/594: creat dc/d13/d14/d25/fbe x:0 0 0 2024-05-15T15:54:18.119 INFO:tasks.workunit.client.0.smithi049.stdout:5/622: write dc/f11 [5245245,26234] 0 2024-05-15T15:54:18.122 INFO:tasks.workunit.client.0.smithi049.stdout:9/463: rmdir d0/d8/d9/d52/d5e/d8a 39 2024-05-15T15:54:18.123 INFO:tasks.workunit.client.0.smithi049.stdout:9/464: stat d0/f65 0 2024-05-15T15:54:18.124 INFO:tasks.workunit.client.1.smithi144.stdout:4/659: write d6/dd/d34/d3b/d30/db/d1d/d40/d57/f85 [448058,89662] 0 2024-05-15T15:54:18.124 INFO:tasks.workunit.client.0.smithi049.stdout:9/465: stat d0/d8/d1b/d40/f77 0 2024-05-15T15:54:18.125 INFO:tasks.workunit.client.1.smithi144.stdout:3/676: creat d0/d11/d3b/da7/fe1 x:0 0 0 2024-05-15T15:54:18.126 INFO:tasks.workunit.client.1.smithi144.stdout:3/677: write d0/d1/d17/f91 [4861756,123287] 0 2024-05-15T15:54:18.127 INFO:tasks.workunit.client.0.smithi049.stdout:7/566: creat d0/d34/fad x:0 0 0 2024-05-15T15:54:18.128 INFO:tasks.workunit.client.1.smithi144.stdout:2/625: stat d3/d7/d20/d22/c95 0 2024-05-15T15:54:18.131 INFO:tasks.workunit.client.0.smithi049.stdout:6/568: dwrite dd/d17/d24/d2b/d36/f41 [0,4194304] 0 2024-05-15T15:54:18.131 INFO:tasks.workunit.client.1.smithi144.stdout:7/671: write d4/d41/d5b/f99 [473392,26305] 0 2024-05-15T15:54:18.134 INFO:tasks.workunit.client.1.smithi144.stdout:1/663: fsync d7/d1d/d33/d1e/d34/f76 0 2024-05-15T15:54:18.134 INFO:tasks.workunit.client.1.smithi144.stdout:2/626: dwrite d3/d7/d20/f7c [0,4194304] 0 2024-05-15T15:54:18.135 INFO:tasks.workunit.client.0.smithi049.stdout:2/578: dread d3/da/d15/f39 [0,4194304] 0 2024-05-15T15:54:18.139 INFO:tasks.workunit.client.1.smithi144.stdout:6/669: rename d7/db/d11/l16 to d7/db/d11/d7e/led 0 2024-05-15T15:54:18.142 INFO:tasks.workunit.client.1.smithi144.stdout:4/660: mkdir d6/d42/d46/d4b/d1e/d99/dab/dd5 0 2024-05-15T15:54:18.143 INFO:tasks.workunit.client.0.smithi049.stdout:4/589: write d18/f49 [1510894,42849] 0 2024-05-15T15:54:18.146 INFO:tasks.workunit.client.0.smithi049.stdout:0/595: mkdir dc/d13/d14/d25/d31/d71/dbf 0 2024-05-15T15:54:18.147 INFO:tasks.workunit.client.1.smithi144.stdout:6/670: dwrite d7/da6/faa [0,4194304] 0 2024-05-15T15:54:18.148 INFO:tasks.workunit.client.1.smithi144.stdout:3/678: mkdir d0/d1/d56/d98/de2 0 2024-05-15T15:54:18.150 INFO:tasks.workunit.client.1.smithi144.stdout:3/679: stat d0/d1/d56/d98/fd2 0 2024-05-15T15:54:18.151 INFO:tasks.workunit.client.0.smithi049.stdout:8/567: dwrite d2/f2c [0,4194304] 0 2024-05-15T15:54:18.152 INFO:tasks.workunit.client.0.smithi049.stdout:8/568: chown d2/de/d71 1470919626 1 2024-05-15T15:54:18.153 INFO:tasks.workunit.client.1.smithi144.stdout:8/726: dwrite d2/d1a/f31 [0,4194304] 0 2024-05-15T15:54:18.153 INFO:tasks.workunit.client.1.smithi144.stdout:7/672: symlink d4/df/d25/d40/d62/ldf 0 2024-05-15T15:54:18.155 INFO:tasks.workunit.client.1.smithi144.stdout:6/671: dread d7/d1e/d28/f3d [4194304,4194304] 0 2024-05-15T15:54:18.158 INFO:tasks.workunit.client.0.smithi049.stdout:6/569: mknod dd/d17/d24/d67/cab 0 2024-05-15T15:54:18.160 INFO:tasks.workunit.client.0.smithi049.stdout:3/596: getdents d12/d30/d63/d68/d70 0 2024-05-15T15:54:18.167 INFO:tasks.workunit.client.0.smithi049.stdout:5/623: creat dc/db0/dc7/fc9 x:0 0 0 2024-05-15T15:54:18.168 INFO:tasks.workunit.client.0.smithi049.stdout:5/624: read dc/d4a/f91 [1644449,24890] 0 2024-05-15T15:54:18.168 INFO:tasks.workunit.client.0.smithi049.stdout:5/625: chown dc/d4a/f8b 7 1 2024-05-15T15:54:18.178 INFO:tasks.workunit.client.1.smithi144.stdout:7/673: creat d4/df/d25/d38/d68/fe0 x:0 0 0 2024-05-15T15:54:18.182 INFO:tasks.workunit.client.0.smithi049.stdout:7/567: mknod d0/d6/d23/d45/d6a/da1/cae 0 2024-05-15T15:54:18.183 INFO:tasks.workunit.client.0.smithi049.stdout:7/568: fsync d0/d34/f46 0 2024-05-15T15:54:18.184 INFO:tasks.workunit.client.0.smithi049.stdout:7/569: chown d0/d6/d23/d45/d6a/c72 201692 1 2024-05-15T15:54:18.185 INFO:tasks.workunit.client.0.smithi049.stdout:8/569: write d2/de/d27/f85 [1069528,4121] 0 2024-05-15T15:54:18.187 INFO:tasks.workunit.client.0.smithi049.stdout:6/570: symlink dd/d17/d24/d2b/d36/d93/lac 0 2024-05-15T15:54:18.189 INFO:tasks.workunit.client.0.smithi049.stdout:7/570: dread d0/f90 [0,4194304] 0 2024-05-15T15:54:18.193 INFO:tasks.workunit.client.0.smithi049.stdout:1/771: getdents d0/d4/dd/de/d1e/d2e/dad/d65/df3 0 2024-05-15T15:54:18.193 INFO:tasks.workunit.client.0.smithi049.stdout:5/626: dread dc/d4a/f62 [0,4194304] 0 2024-05-15T15:54:18.197 INFO:tasks.workunit.client.1.smithi144.stdout:2/627: dwrite d3/d7/d20/d22/d52/d75/d65/f68 [0,4194304] 0 2024-05-15T15:54:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:18 smithi144 ceph-mon[36389]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 103 MiB/s rd, 283 MiB/s wr, 568 op/s 2024-05-15T15:54:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:18 smithi144 ceph-mon[36389]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:18.201 INFO:tasks.workunit.client.0.smithi049.stdout:9/466: truncate d0/f65 308770 0 2024-05-15T15:54:18.202 INFO:tasks.workunit.client.0.smithi049.stdout:0/596: mknod dc/d13/d2b/d76/d8a/da2/db3/cc0 0 2024-05-15T15:54:18.202 INFO:tasks.workunit.client.0.smithi049.stdout:9/467: dread - d0/d8/d53/f84 zero size 2024-05-15T15:54:18.204 INFO:tasks.workunit.client.0.smithi049.stdout:9/468: write d0/d8/f61 [237670,75790] 0 2024-05-15T15:54:18.206 INFO:tasks.workunit.client.1.smithi144.stdout:6/672: dwrite d7/db/d11/f1b [0,4194304] 0 2024-05-15T15:54:18.210 INFO:tasks.workunit.client.0.smithi049.stdout:8/570: symlink d2/de/d71/lb1 0 2024-05-15T15:54:18.211 INFO:tasks.workunit.client.1.smithi144.stdout:6/673: dread d7/da6/faa [0,4194304] 0 2024-05-15T15:54:18.214 INFO:tasks.workunit.client.1.smithi144.stdout:6/674: dwrite f6 [0,4194304] 0 2024-05-15T15:54:18.217 INFO:tasks.workunit.client.0.smithi049.stdout:3/597: dwrite d12/d30/d63/d68/f91 [0,4194304] 0 2024-05-15T15:54:18.219 INFO:tasks.workunit.client.0.smithi049.stdout:4/590: write d18/d1e/d2a/f93 [632599,42538] 0 2024-05-15T15:54:18.220 INFO:tasks.workunit.client.0.smithi049.stdout:1/772: symlink d0/d4/dd/de/d48/d84/l112 0 2024-05-15T15:54:18.230 INFO:tasks.workunit.client.0.smithi049.stdout:0/597: creat dc/d13/d14/d25/d31/d71/fc1 x:0 0 0 2024-05-15T15:54:18.232 INFO:tasks.workunit.client.1.smithi144.stdout:5/672: sync 2024-05-15T15:54:18.232 INFO:tasks.workunit.client.1.smithi144.stdout:9/697: sync 2024-05-15T15:54:18.233 INFO:tasks.workunit.client.0.smithi049.stdout:9/469: creat d0/d8/d1b/d71/d7d/fa3 x:0 0 0 2024-05-15T15:54:18.233 INFO:tasks.workunit.client.1.smithi144.stdout:5/673: read - d0/dc/d1f/d50/fbe zero size 2024-05-15T15:54:18.234 INFO:tasks.workunit.client.0.smithi049.stdout:9/470: write d0/f41 [1610327,52962] 0 2024-05-15T15:54:18.237 INFO:tasks.workunit.client.0.smithi049.stdout:6/571: mknod dd/d17/d24/d2b/d36/d50/cad 0 2024-05-15T15:54:18.240 INFO:tasks.workunit.client.0.smithi049.stdout:0/598: dwrite dc/d13/d14/d25/d31/d71/fc1 [0,4194304] 0 2024-05-15T15:54:18.244 INFO:tasks.workunit.client.0.smithi049.stdout:5/627: unlink dc/d4a/c3f 0 2024-05-15T15:54:18.245 INFO:tasks.workunit.client.0.smithi049.stdout:6/572: dread f0 [0,4194304] 0 2024-05-15T15:54:18.247 INFO:tasks.workunit.client.0.smithi049.stdout:6/573: write dd/d17/d24/d2b/d36/d50/f52 [800101,94904] 0 2024-05-15T15:54:18.248 INFO:tasks.workunit.client.0.smithi049.stdout:6/574: write dd/f19 [2250545,19147] 0 2024-05-15T15:54:18.249 INFO:tasks.workunit.client.0.smithi049.stdout:6/575: dread - dd/f8e zero size 2024-05-15T15:54:18.250 INFO:tasks.workunit.client.0.smithi049.stdout:1/773: fsync d0/d4/dd/de/d1e/d2e/d3e/d90/db3/fc4 0 2024-05-15T15:54:18.256 INFO:tasks.workunit.client.0.smithi049.stdout:9/471: creat d0/d8/d21/fa4 x:0 0 0 2024-05-15T15:54:18.257 INFO:tasks.workunit.client.1.smithi144.stdout:4/661: rename d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/cca to d6/d42/d46/d4b/d1e/cd6 0 2024-05-15T15:54:18.258 INFO:tasks.workunit.client.0.smithi049.stdout:3/598: fsync d12/d26/f40 0 2024-05-15T15:54:18.260 INFO:tasks.workunit.client.0.smithi049.stdout:0/599: mknod dc/d13/d56/d6f/cc2 0 2024-05-15T15:54:18.263 INFO:tasks.workunit.client.0.smithi049.stdout:5/628: mkdir dc/d4a/d31/da9/dba/dbe/dca 0 2024-05-15T15:54:18.264 INFO:tasks.workunit.client.0.smithi049.stdout:4/591: symlink d18/d1e/d2a/d3b/da8/lc2 0 2024-05-15T15:54:18.266 INFO:tasks.workunit.client.0.smithi049.stdout:8/571: dwrite d2/de/d71/d86/d5d/f80 [0,4194304] 0 2024-05-15T15:54:18.267 INFO:tasks.workunit.client.0.smithi049.stdout:8/572: chown d2/de/la1 106 1 2024-05-15T15:54:18.272 INFO:tasks.workunit.client.1.smithi144.stdout:2/628: mknod d3/d7/d20/d22/d52/cd5 0 2024-05-15T15:54:18.280 INFO:tasks.workunit.client.1.smithi144.stdout:6/675: mkdir d7/db/d9d/d72/dee 0 2024-05-15T15:54:18.281 INFO:tasks.workunit.client.1.smithi144.stdout:6/676: write d7/db/da2/fe0 [879321,63121] 0 2024-05-15T15:54:18.282 INFO:tasks.workunit.client.0.smithi049.stdout:9/472: creat d0/d8/d42/d75/fa5 x:0 0 0 2024-05-15T15:54:18.283 INFO:tasks.workunit.client.1.smithi144.stdout:9/698: symlink d0/d8/d5e/d44/d77/lec 0 2024-05-15T15:54:18.287 INFO:tasks.workunit.client.1.smithi144.stdout:4/662: truncate d6/f95 2862125 0 2024-05-15T15:54:18.288 INFO:tasks.workunit.client.0.smithi049.stdout:4/592: dread d18/f3e [0,4194304] 0 2024-05-15T15:54:18.288 INFO:tasks.workunit.client.0.smithi049.stdout:0/600: mkdir dc/d6d/dc3 0 2024-05-15T15:54:18.292 INFO:tasks.workunit.client.1.smithi144.stdout:8/727: rename d2/de/f14 to d2/d24/d2d/da0/d6e/d71/d73/d98/ff0 0 2024-05-15T15:54:18.294 INFO:tasks.workunit.client.1.smithi144.stdout:7/674: symlink d4/d3c/d76/d9d/dc5/le1 0 2024-05-15T15:54:18.295 INFO:tasks.workunit.client.1.smithi144.stdout:7/675: stat d4/d3c/d76/d9d/dac/d48/d72/d82/f92 0 2024-05-15T15:54:18.295 INFO:tasks.workunit.client.1.smithi144.stdout:7/676: readlink d4/d41/d5b/da0/lb0 0 2024-05-15T15:54:18.300 INFO:tasks.workunit.client.0.smithi049.stdout:1/774: mknod d0/d4/c113 0 2024-05-15T15:54:18.301 INFO:tasks.workunit.client.0.smithi049.stdout:1/775: chown d0/d4/dd/de/d1e/d2e/dad/d65/fdf 9868714 1 2024-05-15T15:54:18.301 INFO:tasks.workunit.client.1.smithi144.stdout:9/699: chown d0/f14 0 1 2024-05-15T15:54:18.302 INFO:tasks.workunit.client.1.smithi144.stdout:9/700: chown d0/f59 125667 1 2024-05-15T15:54:18.303 INFO:tasks.workunit.client.1.smithi144.stdout:5/674: readlink d0/dc/d12/d4a/d5d/l97 0 2024-05-15T15:54:18.304 INFO:tasks.workunit.client.1.smithi144.stdout:5/675: truncate d0/dc/d24/d8e/d93/fb2 999058 0 2024-05-15T15:54:18.305 INFO:tasks.workunit.client.1.smithi144.stdout:5/676: chown d0/c4f 1015049184 1 2024-05-15T15:54:18.305 INFO:tasks.workunit.client.1.smithi144.stdout:4/663: creat d6/dd/d34/d3b/d30/db/d1d/d40/d6c/d82/fd7 x:0 0 0 2024-05-15T15:54:18.305 INFO:tasks.workunit.client.0.smithi049.stdout:5/629: write dc/d4a/d42/f5e [242229,57702] 0 2024-05-15T15:54:18.309 INFO:tasks.workunit.client.0.smithi049.stdout:8/573: write d2/d18/f47 [4607267,3932] 0 2024-05-15T15:54:18.309 INFO:tasks.workunit.client.0.smithi049.stdout:0/601: readlink dc/d13/d48/d57/l86 0 2024-05-15T15:54:18.310 INFO:tasks.workunit.client.1.smithi144.stdout:0/649: sync 2024-05-15T15:54:18.311 INFO:tasks.workunit.client.0.smithi049.stdout:8/574: chown d2/de/d27/d5b/da6/laa 1 1 2024-05-15T15:54:18.311 INFO:tasks.workunit.client.1.smithi144.stdout:6/677: rename d7/db/d11/d3c/d61/c6c to d7/db/d3a/de3/cef 0 2024-05-15T15:54:18.314 INFO:tasks.workunit.client.1.smithi144.stdout:8/728: fdatasync d2/de/d49/f69 0 2024-05-15T15:54:18.315 INFO:tasks.workunit.client.1.smithi144.stdout:7/677: write d4/df/d25/d40/f4d [3976415,42690] 0 2024-05-15T15:54:18.317 INFO:tasks.workunit.client.1.smithi144.stdout:2/629: mkdir d3/d7/d20/d22/d52/d64/d97/db8/dd6 0 2024-05-15T15:54:18.319 INFO:tasks.workunit.client.0.smithi049.stdout:6/576: creat dd/d17/d24/d2b/d36/d86/fae x:0 0 0 2024-05-15T15:54:18.319 INFO:tasks.workunit.client.0.smithi049.stdout:3/599: write d12/f47 [3626522,79286] 0 2024-05-15T15:54:18.322 INFO:tasks.workunit.client.0.smithi049.stdout:1/776: unlink d0/f19 0 2024-05-15T15:54:18.324 INFO:tasks.workunit.client.1.smithi144.stdout:5/677: chown d0/dc/d24/d28/f82 1225174 1 2024-05-15T15:54:18.324 INFO:tasks.workunit.client.1.smithi144.stdout:5/678: chown d0/dc/d24/d28/c2e 220849 1 2024-05-15T15:54:18.324 INFO:tasks.workunit.client.1.smithi144.stdout:4/664: dread - d6/dd/d34/d3b/d30/db/d1d/d40/f80 zero size 2024-05-15T15:54:18.325 INFO:tasks.workunit.client.1.smithi144.stdout:5/679: dread d0/dc/d1f/f75 [0,4194304] 0 2024-05-15T15:54:18.328 INFO:tasks.workunit.client.1.smithi144.stdout:0/650: fsync d4/d6/d4a/d74/f79 0 2024-05-15T15:54:18.329 INFO:tasks.workunit.client.0.smithi049.stdout:4/593: dwrite d18/d1e/d63/fa2 [0,4194304] 0 2024-05-15T15:54:18.331 INFO:tasks.workunit.client.1.smithi144.stdout:6/678: symlink d7/d1e/d28/lf0 0 2024-05-15T15:54:18.333 INFO:tasks.workunit.client.0.smithi049.stdout:5/630: write dc/d4a/f36 [1514236,24215] 0 2024-05-15T15:54:18.336 INFO:tasks.workunit.client.0.smithi049.stdout:6/577: creat dd/d17/d24/d2b/d36/d50/d54/faf x:0 0 0 2024-05-15T15:54:18.336 INFO:tasks.workunit.client.1.smithi144.stdout:8/729: dread - d2/d24/d2d/da0/d6e/d71/d73/fd2 zero size 2024-05-15T15:54:18.338 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:18 smithi049 ceph-mon[28399]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 103 MiB/s rd, 283 MiB/s wr, 568 op/s 2024-05-15T15:54:18.338 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:18 smithi049 ceph-mon[28399]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:18.339 INFO:tasks.workunit.client.0.smithi049.stdout:1/777: symlink d0/d4/dd/de/d4e/l114 0 2024-05-15T15:54:18.342 INFO:tasks.workunit.client.0.smithi049.stdout:3/600: dwrite d12/d30/d42/d7a/f83 [0,4194304] 0 2024-05-15T15:54:18.342 INFO:tasks.workunit.client.1.smithi144.stdout:9/701: mknod d0/d8/d5e/d44/dc2/ced 0 2024-05-15T15:54:18.342 INFO:tasks.workunit.client.1.smithi144.stdout:8/730: dwrite d2/d24/d2d/d30/fe0 [0,4194304] 0 2024-05-15T15:54:18.346 INFO:tasks.workunit.client.0.smithi049.stdout:4/594: mkdir d18/d85/d98/dc3 0 2024-05-15T15:54:18.347 INFO:tasks.workunit.client.1.smithi144.stdout:3/680: getdents d0/d1/d17/d39 0 2024-05-15T15:54:18.350 INFO:tasks.workunit.client.1.smithi144.stdout:3/681: dread d0/d1/d17/d39/d40/d4a/d77/fb0 [0,4194304] 0 2024-05-15T15:54:18.352 INFO:tasks.workunit.client.1.smithi144.stdout:0/651: dread - d4/d2d/fab zero size 2024-05-15T15:54:18.354 INFO:tasks.workunit.client.0.smithi049.stdout:9/473: getdents d0/d8 0 2024-05-15T15:54:18.356 INFO:tasks.workunit.client.0.smithi049.stdout:1/778: mknod d0/d4/dd/de/d1e/d2e/c115 0 2024-05-15T15:54:18.357 INFO:tasks.workunit.client.0.smithi049.stdout:1/779: chown d0/d4/f9 49297 1 2024-05-15T15:54:18.362 INFO:tasks.workunit.client.1.smithi144.stdout:2/630: dwrite d3/d7/d20/d22/d52/d75/da6/dc1/fc6 [0,4194304] 0 2024-05-15T15:54:18.363 INFO:tasks.workunit.client.1.smithi144.stdout:2/631: stat d3/d7/d54/c7b 0 2024-05-15T15:54:18.363 INFO:tasks.workunit.client.0.smithi049.stdout:5/631: mknod dc/d4a/d31/ccb 0 2024-05-15T15:54:18.365 INFO:tasks.workunit.client.1.smithi144.stdout:8/731: unlink d2/de/d36/d87/le8 0 2024-05-15T15:54:18.367 INFO:tasks.workunit.client.1.smithi144.stdout:9/702: symlink d0/d8/d1b/d62/d7d/d8d/d94/lee 0 2024-05-15T15:54:18.372 INFO:tasks.workunit.client.0.smithi049.stdout:0/602: getdents dc/d13/d14/d25/d31 0 2024-05-15T15:54:18.373 INFO:tasks.workunit.client.0.smithi049.stdout:3/601: mknod d12/d26/cb7 0 2024-05-15T15:54:18.375 INFO:tasks.workunit.client.0.smithi049.stdout:4/595: mkdir d18/d1e/d2a/d50/dc4 0 2024-05-15T15:54:18.376 INFO:tasks.workunit.client.0.smithi049.stdout:8/575: link d2/de/d71/d86/d5d/d7d/d1b/f36 d2/de/d27/d5b/fb2 0 2024-05-15T15:54:18.380 INFO:tasks.workunit.client.0.smithi049.stdout:8/576: chown d2/de/d71/d86/d5d/f80 1475 1 2024-05-15T15:54:18.380 INFO:tasks.workunit.client.1.smithi144.stdout:7/678: link d4/df/d79/fcc d4/d41/fe2 0 2024-05-15T15:54:18.381 INFO:tasks.workunit.client.0.smithi049.stdout:0/603: dwrite dc/d13/d3d/fb2 [0,4194304] 0 2024-05-15T15:54:18.382 INFO:tasks.workunit.client.1.smithi144.stdout:7/679: write d4/df/d25/d38/d68/fe0 [120966,112871] 0 2024-05-15T15:54:18.384 INFO:tasks.workunit.client.0.smithi049.stdout:3/602: dread d12/d30/d42/f48 [0,4194304] 0 2024-05-15T15:54:18.386 INFO:tasks.workunit.client.1.smithi144.stdout:2/632: creat d3/d7/d54/fd7 x:0 0 0 2024-05-15T15:54:18.386 INFO:tasks.workunit.client.1.smithi144.stdout:2/633: stat d3/d7/d20/d89/d86 0 2024-05-15T15:54:18.387 INFO:tasks.workunit.client.1.smithi144.stdout:2/634: write d3/f1c [3689933,96137] 0 2024-05-15T15:54:18.389 INFO:tasks.workunit.client.1.smithi144.stdout:5/680: write d0/dc/d24/d28/d2d/f2b [1234900,93463] 0 2024-05-15T15:54:18.390 INFO:tasks.workunit.client.1.smithi144.stdout:5/681: write d0/dc/d2f/d34/d63/fd5 [859431,66528] 0 2024-05-15T15:54:18.394 INFO:tasks.workunit.client.1.smithi144.stdout:4/665: rename d6/dd/d34/d3b/d30/fc6 to d6/dd/fd8 0 2024-05-15T15:54:18.395 INFO:tasks.workunit.client.1.smithi144.stdout:4/666: stat d6/d42/d46/d4b/d1e/d99/la4 0 2024-05-15T15:54:18.396 INFO:tasks.workunit.client.0.smithi049.stdout:6/578: dwrite dd/d26/f2e [0,4194304] 0 2024-05-15T15:54:18.402 INFO:tasks.workunit.client.1.smithi144.stdout:0/652: link d4/d2d/dcf/fdb d4/d6/d3a/d4c/fea 0 2024-05-15T15:54:18.405 INFO:tasks.workunit.client.1.smithi144.stdout:6/679: creat d7/db/d35/ff1 x:0 0 0 2024-05-15T15:54:18.406 INFO:tasks.workunit.client.0.smithi049.stdout:0/604: dread dc/d13/f17 [0,4194304] 0 2024-05-15T15:54:18.407 INFO:tasks.workunit.client.1.smithi144.stdout:0/653: dwrite d4/d6/d3a/d4c/dd6/d55/f72 [0,4194304] 0 2024-05-15T15:54:18.409 INFO:tasks.workunit.client.1.smithi144.stdout:8/732: dread d2/d24/d2d/da0/d6e/d71/fec [0,4194304] 0 2024-05-15T15:54:18.412 INFO:tasks.workunit.client.0.smithi049.stdout:4/596: link d18/d1e/d2a/f43 d18/fc5 0 2024-05-15T15:54:18.414 INFO:tasks.workunit.client.0.smithi049.stdout:4/597: chown d18/d1e/d2a/d36/f83 10413 1 2024-05-15T15:54:18.416 INFO:tasks.workunit.client.0.smithi049.stdout:9/474: dwrite d0/d8/d9/d52/d5e/d5f/f3e [0,4194304] 0 2024-05-15T15:54:18.418 INFO:tasks.workunit.client.0.smithi049.stdout:5/632: write dc/d4a/f6a [366509,30376] 0 2024-05-15T15:54:18.422 INFO:tasks.workunit.client.0.smithi049.stdout:3/603: creat d12/d14/fb8 x:0 0 0 2024-05-15T15:54:18.426 INFO:tasks.workunit.client.1.smithi144.stdout:7/680: rename d4/df/f11 to d4/d41/d5b/da0/fe3 0 2024-05-15T15:54:18.429 INFO:tasks.workunit.client.1.smithi144.stdout:4/667: write d6/dd/d34/d3b/d30/db/d1d/f41 [36131,18444] 0 2024-05-15T15:54:18.431 INFO:tasks.workunit.client.1.smithi144.stdout:4/668: write d6/dd/d34/d3b/d30/db/d1d/f41 [3965517,114241] 0 2024-05-15T15:54:18.432 INFO:tasks.workunit.client.1.smithi144.stdout:4/669: chown d6/dd/d2e/d75/l76 3480701 1 2024-05-15T15:54:18.434 INFO:tasks.workunit.client.1.smithi144.stdout:6/680: mkdir d7/db/d3a/de3/df2 0 2024-05-15T15:54:18.437 INFO:tasks.workunit.client.1.smithi144.stdout:2/635: write d3/d7/d20/d37/d76/db5/db6/dbf/fd0 [46630,45627] 0 2024-05-15T15:54:18.437 INFO:tasks.workunit.client.1.smithi144.stdout:1/664: sync 2024-05-15T15:54:18.438 INFO:tasks.workunit.client.1.smithi144.stdout:1/665: chown d7/d1d/d33/d29/l8a 61281269 1 2024-05-15T15:54:18.439 INFO:tasks.workunit.client.0.smithi049.stdout:5/633: unlink dc/d29/lc3 0 2024-05-15T15:54:18.440 INFO:tasks.workunit.client.1.smithi144.stdout:8/733: symlink d2/d24/d2d/da0/d9e/db8/dde/lf1 0 2024-05-15T15:54:18.441 INFO:tasks.workunit.client.1.smithi144.stdout:2/636: dread d3/d7/d20/f7c [0,4194304] 0 2024-05-15T15:54:18.445 INFO:tasks.workunit.client.0.smithi049.stdout:0/605: dwrite dc/d13/d14/f79 [0,4194304] 0 2024-05-15T15:54:18.447 INFO:tasks.workunit.client.0.smithi049.stdout:3/604: rename d12/d30/d42 to d12/d14/d52/db9 0 2024-05-15T15:54:18.451 INFO:tasks.workunit.client.1.smithi144.stdout:5/682: mknod d0/dc/d2f/d34/d63/d6e/dbf/cdf 0 2024-05-15T15:54:18.454 INFO:tasks.workunit.client.0.smithi049.stdout:6/579: link dd/d17/d24/d2b/d36/d50/d7e/la4 dd/d17/d24/d2b/lb0 0 2024-05-15T15:54:18.458 INFO:tasks.workunit.client.0.smithi049.stdout:9/475: dwrite d0/d8/f7e [0,4194304] 0 2024-05-15T15:54:18.466 INFO:tasks.workunit.client.1.smithi144.stdout:4/670: readlink d6/d42/lc1 0 2024-05-15T15:54:18.467 INFO:tasks.workunit.client.1.smithi144.stdout:4/671: write d6/d42/d46/d4b/fb2 [2851717,116788] 0 2024-05-15T15:54:18.468 INFO:tasks.workunit.client.0.smithi049.stdout:4/598: write d18/d1e/d63/f7f [716962,56350] 0 2024-05-15T15:54:18.469 INFO:tasks.workunit.client.0.smithi049.stdout:8/577: link d2/de/d71/d86/d5d/d7d/d1b/d68/f66 d2/de/d71/d86/d5d/d7d/fb3 0 2024-05-15T15:54:18.470 INFO:tasks.workunit.client.0.smithi049.stdout:8/578: dread - d2/de/fa3 zero size 2024-05-15T15:54:18.473 INFO:tasks.workunit.client.0.smithi049.stdout:0/606: creat dc/d13/d2b/d76/d8a/da2/fc4 x:0 0 0 2024-05-15T15:54:18.476 INFO:tasks.workunit.client.1.smithi144.stdout:8/734: truncate d2/de/d3f/f4e 2149241 0 2024-05-15T15:54:18.477 INFO:tasks.workunit.client.0.smithi049.stdout:6/580: symlink dd/d26/lb1 0 2024-05-15T15:54:18.479 INFO:tasks.workunit.client.1.smithi144.stdout:9/703: link d0/d8/d5e/d44/d10/d19/f67 d0/d8/d5e/d44/d4a/fef 0 2024-05-15T15:54:18.481 INFO:tasks.workunit.client.1.smithi144.stdout:7/681: mkdir d4/df/d25/d38/de4 0 2024-05-15T15:54:18.484 INFO:tasks.workunit.client.0.smithi049.stdout:5/634: rename dc/d4a/d31 to dc/dd/d47/d83/dcc 0 2024-05-15T15:54:18.486 INFO:tasks.workunit.client.1.smithi144.stdout:4/672: rmdir d6/d42/d46/d4b/d1e/d99/dab 39 2024-05-15T15:54:18.487 INFO:tasks.workunit.client.1.smithi144.stdout:4/673: dread - d6/dd/d34/d3b/d30/d59/d66/fb8 zero size 2024-05-15T15:54:18.493 INFO:tasks.workunit.client.0.smithi049.stdout:9/476: fsync d0/d8/d9/d52/d5e/d5f/f38 0 2024-05-15T15:54:18.494 INFO:tasks.workunit.client.0.smithi049.stdout:9/477: fdatasync d0/d8/d12/f6e 0 2024-05-15T15:54:18.494 INFO:tasks.workunit.client.0.smithi049.stdout:9/478: fsync d0/d8/d42/f91 0 2024-05-15T15:54:18.495 INFO:tasks.workunit.client.1.smithi144.stdout:1/666: unlink d7/d1d/d33/d41/d78/da3/l2f 0 2024-05-15T15:54:18.498 INFO:tasks.workunit.client.1.smithi144.stdout:4/674: dread d6/d42/d46/d4b/f31 [0,4194304] 0 2024-05-15T15:54:18.500 INFO:tasks.workunit.client.0.smithi049.stdout:0/607: rename dc/d13/d48 to dc/d13/d14/d25/d31/dae/dc5 0 2024-05-15T15:54:18.500 INFO:tasks.workunit.client.0.smithi049.stdout:0/608: chown dc 49 1 2024-05-15T15:54:18.501 INFO:tasks.workunit.client.1.smithi144.stdout:8/735: mknod d2/d24/d2d/da0/d9e/cf2 0 2024-05-15T15:54:18.502 INFO:tasks.workunit.client.0.smithi049.stdout:3/605: truncate d12/d14/d52/db9/f8e 5619382 0 2024-05-15T15:54:18.503 INFO:tasks.workunit.client.0.smithi049.stdout:5/635: mknod dc/dd/d7b/d8d/ccd 0 2024-05-15T15:54:18.504 INFO:tasks.workunit.client.1.smithi144.stdout:5/683: mknod d0/dc/d24/ce0 0 2024-05-15T15:54:18.508 INFO:tasks.workunit.client.0.smithi049.stdout:2/579: sync 2024-05-15T15:54:18.510 INFO:tasks.workunit.client.0.smithi049.stdout:9/479: dread d0/d8/d9/d52/d5e/d5f/f55 [0,4194304] 0 2024-05-15T15:54:18.511 INFO:tasks.workunit.client.0.smithi049.stdout:8/579: creat d2/de/d27/d5b/d93/fb4 x:0 0 0 2024-05-15T15:54:18.512 INFO:tasks.workunit.client.1.smithi144.stdout:3/682: sync 2024-05-15T15:54:18.513 INFO:tasks.workunit.client.1.smithi144.stdout:7/682: mkdir d4/d3c/d76/d9d/dac/de5 0 2024-05-15T15:54:18.513 INFO:tasks.workunit.client.1.smithi144.stdout:3/683: write d0/d11/f14 [2667625,126951] 0 2024-05-15T15:54:18.514 INFO:tasks.workunit.client.1.smithi144.stdout:7/683: write d4/d3c/f57 [155961,122556] 0 2024-05-15T15:54:18.519 INFO:tasks.workunit.client.0.smithi049.stdout:8/580: dwrite d2/de/d27/d5b/f88 [0,4194304] 0 2024-05-15T15:54:18.519 INFO:tasks.workunit.client.1.smithi144.stdout:6/681: link d7/d32/c36 d7/db/d35/d84/cf3 0 2024-05-15T15:54:18.522 INFO:tasks.workunit.client.0.smithi049.stdout:6/581: rename dd/d26/c78 to dd/d17/d24/d58/cb2 0 2024-05-15T15:54:18.524 INFO:tasks.workunit.client.1.smithi144.stdout:9/704: dread d0/d8/d5e/d44/d10/d19/fc0 [4194304,4194304] 0 2024-05-15T15:54:18.525 INFO:tasks.workunit.client.0.smithi049.stdout:4/599: dwrite d18/d1e/d2a/d3b/f57 [0,4194304] 0 2024-05-15T15:54:18.526 INFO:tasks.workunit.client.1.smithi144.stdout:6/682: dwrite d7/db/d11/d3c/f79 [0,4194304] 0 2024-05-15T15:54:18.526 INFO:tasks.workunit.client.1.smithi144.stdout:0/654: read d4/df/d1f/d24/d96/fca [3036074,42066] 0 2024-05-15T15:54:18.527 INFO:tasks.workunit.client.1.smithi144.stdout:6/683: stat d7/db/l1a 0 2024-05-15T15:54:18.529 INFO:tasks.workunit.client.1.smithi144.stdout:4/675: truncate d6/dd/d34/d3b/d30/db/d1d/d40/f80 978056 0 2024-05-15T15:54:18.529 INFO:tasks.workunit.client.1.smithi144.stdout:6/684: chown d7/db/d11/f1b 9994238 1 2024-05-15T15:54:18.529 INFO:tasks.workunit.client.1.smithi144.stdout:0/655: dread - d4/df/d14/db5/fd4 zero size 2024-05-15T15:54:18.529 INFO:tasks.workunit.client.1.smithi144.stdout:6/685: chown d7/db/d3a 351 1 2024-05-15T15:54:18.530 INFO:tasks.workunit.client.0.smithi049.stdout:5/636: mkdir dc/dd/d47/da6/dce 0 2024-05-15T15:54:18.533 INFO:tasks.workunit.client.0.smithi049.stdout:5/637: dread - dc/dd/d7b/d8d/fbb zero size 2024-05-15T15:54:18.533 INFO:tasks.workunit.client.1.smithi144.stdout:8/736: symlink d2/de/d36/lf3 0 2024-05-15T15:54:18.534 INFO:tasks.workunit.client.0.smithi049.stdout:2/580: mknod d3/db/d1d/d1e/d2b/d6b/cc3 0 2024-05-15T15:54:18.535 INFO:tasks.workunit.client.1.smithi144.stdout:5/684: write d0/dc/d24/d8e/d93/fa1 [1367795,59444] 0 2024-05-15T15:54:18.537 INFO:tasks.workunit.client.0.smithi049.stdout:6/582: dwrite dd/d17/d64/fa7 [0,4194304] 0 2024-05-15T15:54:18.539 INFO:tasks.workunit.client.0.smithi049.stdout:0/609: write dc/d32/d93/f88 [4685043,116063] 0 2024-05-15T15:54:18.539 INFO:tasks.workunit.client.1.smithi144.stdout:4/676: dwrite d6/d42/d46/d4b/fb2 [0,4194304] 0 2024-05-15T15:54:18.541 INFO:tasks.workunit.client.0.smithi049.stdout:9/480: write d0/d8/d9/d52/d5e/d8a/f1f [241620,14155] 0 2024-05-15T15:54:18.545 INFO:tasks.workunit.client.0.smithi049.stdout:0/610: chown dc/d13/d14/d2f/d8f/f9f 2742534 1 2024-05-15T15:54:18.545 INFO:tasks.workunit.client.0.smithi049.stdout:3/606: dwrite d12/d14/d52/f99 [0,4194304] 0 2024-05-15T15:54:18.545 INFO:tasks.workunit.client.0.smithi049.stdout:3/607: fsync d12/d14/d52/db9/d6b/fa2 0 2024-05-15T15:54:18.546 INFO:tasks.workunit.client.0.smithi049.stdout:0/611: fsync dc/d13/d56/fa0 0 2024-05-15T15:54:18.548 INFO:tasks.workunit.client.0.smithi049.stdout:6/583: truncate dd/d17/d24/d2b/d36/d50/d54/faf 119141 0 2024-05-15T15:54:18.549 INFO:tasks.workunit.client.0.smithi049.stdout:6/584: stat dd/d17 0 2024-05-15T15:54:18.554 INFO:tasks.workunit.client.1.smithi144.stdout:2/637: sync 2024-05-15T15:54:18.555 INFO:tasks.workunit.client.0.smithi049.stdout:2/581: unlink d3/db/f1f 0 2024-05-15T15:54:18.555 INFO:tasks.workunit.client.0.smithi049.stdout:5/638: mknod dc/dd/d47/d83/dcc/d5c/ccf 0 2024-05-15T15:54:18.555 INFO:tasks.workunit.client.1.smithi144.stdout:1/667: getdents d7/d1d/d33/d41/d22/dbd/dc8 0 2024-05-15T15:54:18.558 INFO:tasks.workunit.client.0.smithi049.stdout:5/639: chown dc/dd/d47/d83/dcc/da9 748849470 1 2024-05-15T15:54:18.561 INFO:tasks.workunit.client.1.smithi144.stdout:9/705: dread d0/d8/d1b/d79/d8a/dae/feb [0,4194304] 0 2024-05-15T15:54:18.562 INFO:tasks.workunit.client.1.smithi144.stdout:0/656: fsync d4/d2d/dcf/fdb 0 2024-05-15T15:54:18.568 INFO:tasks.workunit.client.0.smithi049.stdout:5/640: dwrite dc/d4a/d42/f5e [0,4194304] 0 2024-05-15T15:54:18.570 INFO:tasks.workunit.client.1.smithi144.stdout:6/686: symlink d7/d1e/d28/db5/lf4 0 2024-05-15T15:54:18.574 INFO:tasks.workunit.client.1.smithi144.stdout:8/737: fdatasync d2/d24/f2b 0 2024-05-15T15:54:18.579 INFO:tasks.workunit.client.0.smithi049.stdout:2/582: creat d3/db/d1d/d1e/d2b/d2e/d67/fc4 x:0 0 0 2024-05-15T15:54:18.580 INFO:tasks.workunit.client.1.smithi144.stdout:5/685: symlink d0/dc/d24/d8e/d93/le1 0 2024-05-15T15:54:18.581 INFO:tasks.workunit.client.1.smithi144.stdout:7/684: mknod d4/d3c/d76/d9d/dac/de5/ce6 0 2024-05-15T15:54:18.581 INFO:tasks.workunit.client.1.smithi144.stdout:8/738: dwrite d2/d24/d2d/da0/d6e/d71/db1/fd1 [0,4194304] 0 2024-05-15T15:54:18.583 INFO:tasks.workunit.client.1.smithi144.stdout:3/684: link d0/d1/d17/f91 d0/d1/d56/d98/d9c/fe3 0 2024-05-15T15:54:18.584 INFO:tasks.workunit.client.1.smithi144.stdout:3/685: read - d0/d11/d3b/da0/d8f/fe0 zero size 2024-05-15T15:54:18.586 INFO:tasks.workunit.client.0.smithi049.stdout:5/641: truncate dc/d29/f32 923899 0 2024-05-15T15:54:18.589 INFO:tasks.workunit.client.0.smithi049.stdout:6/585: dread dd/f19 [4194304,4194304] 0 2024-05-15T15:54:18.595 INFO:tasks.workunit.client.0.smithi049.stdout:2/583: creat d3/da/fc5 x:0 0 0 2024-05-15T15:54:18.599 INFO:tasks.workunit.client.0.smithi049.stdout:9/481: dwrite d0/f54 [0,4194304] 0 2024-05-15T15:54:18.604 INFO:tasks.workunit.client.0.smithi049.stdout:8/581: getdents d2/de/d27/d5b/da6 0 2024-05-15T15:54:18.613 INFO:tasks.workunit.client.1.smithi144.stdout:8/739: unlink d2/d24/d2d/da0/d6e/d71/d8c/fc9 0 2024-05-15T15:54:18.614 INFO:tasks.workunit.client.0.smithi049.stdout:3/608: dwrite d12/d14/f4f [0,4194304] 0 2024-05-15T15:54:18.621 INFO:tasks.workunit.client.0.smithi049.stdout:0/612: truncate dc/d13/f26 216218 0 2024-05-15T15:54:18.628 INFO:tasks.workunit.client.0.smithi049.stdout:6/586: write dd/d26/f3f [3829520,91642] 0 2024-05-15T15:54:18.630 INFO:tasks.workunit.client.0.smithi049.stdout:5/642: dwrite dc/dd/d10/f6f [0,4194304] 0 2024-05-15T15:54:18.630 INFO:tasks.workunit.client.0.smithi049.stdout:8/582: mknod d2/de/d27/d5b/d60/cb5 0 2024-05-15T15:54:18.633 INFO:tasks.workunit.client.1.smithi144.stdout:2/638: write d3/d7/d20/d3f/f56 [1639539,64863] 0 2024-05-15T15:54:18.634 INFO:tasks.workunit.client.1.smithi144.stdout:5/686: symlink d0/dc/d24/d28/d2d/d59/le2 0 2024-05-15T15:54:18.639 INFO:tasks.workunit.client.1.smithi144.stdout:7/685: truncate d4/d3c/fa3 2714942 0 2024-05-15T15:54:18.640 INFO:tasks.workunit.client.0.smithi049.stdout:9/482: write d0/d8/d1b/f3a [311387,71325] 0 2024-05-15T15:54:18.641 INFO:tasks.workunit.client.0.smithi049.stdout:9/483: dread - d0/d8/d1b/d71/d7d/fa3 zero size 2024-05-15T15:54:18.643 INFO:tasks.workunit.client.0.smithi049.stdout:3/609: read d12/f3f [2557338,122450] 0 2024-05-15T15:54:18.643 INFO:tasks.workunit.client.0.smithi049.stdout:0/613: symlink dc/d13/d56/d6f/lc6 0 2024-05-15T15:54:18.646 INFO:tasks.workunit.client.1.smithi144.stdout:1/668: readlink d7/d1d/d33/d29/l37 0 2024-05-15T15:54:18.648 INFO:tasks.workunit.client.1.smithi144.stdout:9/706: link d0/d8/d1b/d62/d7d/d8d/dbb/dce/dd1/cea d0/d8/d5e/d44/da2/cf0 0 2024-05-15T15:54:18.650 INFO:tasks.workunit.client.1.smithi144.stdout:6/687: creat d7/db/d11/d3c/ff5 x:0 0 0 2024-05-15T15:54:18.655 INFO:tasks.workunit.client.1.smithi144.stdout:5/687: creat d0/dc/d24/d28/d2d/d59/d74/fe3 x:0 0 0 2024-05-15T15:54:18.655 INFO:tasks.workunit.client.0.smithi049.stdout:8/583: truncate d2/f22 1008768 0 2024-05-15T15:54:18.655 INFO:tasks.workunit.client.1.smithi144.stdout:4/677: sync 2024-05-15T15:54:18.656 INFO:tasks.workunit.client.1.smithi144.stdout:6/688: dwrite d7/db/d11/d3c/ff5 [0,4194304] 0 2024-05-15T15:54:18.657 INFO:tasks.workunit.client.1.smithi144.stdout:7/686: rename d4/df/d25/d40/f4d to d4/df/d81/fe7 0 2024-05-15T15:54:18.660 INFO:tasks.workunit.client.1.smithi144.stdout:1/669: symlink d7/d1d/d58/d81/ld7 0 2024-05-15T15:54:18.661 INFO:tasks.workunit.client.1.smithi144.stdout:9/707: symlink d0/d8/d1b/d62/d7d/d8d/dbb/dce/lf1 0 2024-05-15T15:54:18.663 INFO:tasks.workunit.client.1.smithi144.stdout:2/639: unlink d3/l6 0 2024-05-15T15:54:18.667 INFO:tasks.workunit.client.1.smithi144.stdout:5/688: rmdir d0/dc/d24 39 2024-05-15T15:54:18.669 INFO:tasks.workunit.client.0.smithi049.stdout:0/614: link dc/d13/d56/fa0 dc/d13/d14/d25/d31/d71/fc7 0 2024-05-15T15:54:18.670 INFO:tasks.workunit.client.1.smithi144.stdout:6/689: mkdir d7/d1e/d28/db5/df6 0 2024-05-15T15:54:18.671 INFO:tasks.workunit.client.1.smithi144.stdout:8/740: creat d2/de/ff4 x:0 0 0 2024-05-15T15:54:18.672 INFO:tasks.workunit.client.1.smithi144.stdout:4/678: rename d6/dd/d34/d3b/d30/db/d1d/d40/d6c/d82/fc7 to d6/dd/d34/d3b/fd9 0 2024-05-15T15:54:18.674 INFO:tasks.workunit.client.1.smithi144.stdout:4/679: write d6/dd/d34/d3b/d30/db/d1d/d40/d6c/d82/fd7 [726424,6422] 0 2024-05-15T15:54:18.675 INFO:tasks.workunit.client.0.smithi049.stdout:5/643: dread f0 [0,4194304] 0 2024-05-15T15:54:18.677 INFO:tasks.workunit.client.0.smithi049.stdout:3/610: creat d12/d30/d63/d68/fba x:0 0 0 2024-05-15T15:54:18.681 INFO:tasks.workunit.client.0.smithi049.stdout:0/615: rmdir dc/d13/d3d 39 2024-05-15T15:54:18.681 INFO:tasks.workunit.client.0.smithi049.stdout:5/644: dwrite dc/dd/d47/f5d [0,4194304] 0 2024-05-15T15:54:18.682 INFO:tasks.workunit.client.0.smithi049.stdout:0/616: read - dc/d13/d14/d25/d31/d71/fc7 zero size 2024-05-15T15:54:18.683 INFO:tasks.workunit.client.0.smithi049.stdout:0/617: stat dc/d32/faa 0 2024-05-15T15:54:18.684 INFO:tasks.workunit.client.1.smithi144.stdout:2/640: unlink d3/d7/d20/d22/d52/d64/lad 0 2024-05-15T15:54:18.685 INFO:tasks.workunit.client.1.smithi144.stdout:5/689: dread d0/dc/d1f/d50/f58 [0,4194304] 0 2024-05-15T15:54:18.686 INFO:tasks.workunit.client.0.smithi049.stdout:2/584: getdents d3/db/d1d/d1e 0 2024-05-15T15:54:18.687 INFO:tasks.workunit.client.1.smithi144.stdout:5/690: chown d0/dc/d2f/d34/d63/d6e/dbf/f9f 2 1 2024-05-15T15:54:18.688 INFO:tasks.workunit.client.0.smithi049.stdout:5/645: dread dc/dd/d7b/d8d/fa8 [0,4194304] 0 2024-05-15T15:54:18.690 INFO:tasks.workunit.client.0.smithi049.stdout:8/584: write d2/de/d71/d86/d5d/d7d/f43 [2259799,96746] 0 2024-05-15T15:54:18.691 INFO:tasks.workunit.client.0.smithi049.stdout:8/585: stat d2/de/d27/c5f 0 2024-05-15T15:54:18.692 INFO:tasks.workunit.client.0.smithi049.stdout:3/611: readlink d12/d14/l53 0 2024-05-15T15:54:18.692 INFO:tasks.workunit.client.1.smithi144.stdout:5/691: dwrite d0/f22 [4194304,4194304] 0 2024-05-15T15:54:18.695 INFO:tasks.workunit.client.0.smithi049.stdout:6/587: dwrite dd/ff [0,4194304] 0 2024-05-15T15:54:18.697 INFO:tasks.workunit.client.0.smithi049.stdout:6/588: chown dd/d17/d24/d2b/d36/l49 377548314 1 2024-05-15T15:54:18.697 INFO:tasks.workunit.client.0.smithi049.stdout:6/589: chown dd/l69 129630862 1 2024-05-15T15:54:18.699 INFO:tasks.workunit.client.1.smithi144.stdout:1/670: mkdir d7/d1d/d33/d1e/d34/dd8 0 2024-05-15T15:54:18.701 INFO:tasks.workunit.client.0.smithi049.stdout:0/618: creat dc/d13/d2b/d76/fc8 x:0 0 0 2024-05-15T15:54:18.704 INFO:tasks.workunit.client.0.smithi049.stdout:2/585: rename d3/d8/ld to d3/db/d1d/d1e/d2b/d6b/lc6 0 2024-05-15T15:54:18.705 INFO:tasks.workunit.client.1.smithi144.stdout:9/708: write d0/d8/d5e/d44/d10/d9c/f65 [2892284,50188] 0 2024-05-15T15:54:18.707 INFO:tasks.workunit.client.0.smithi049.stdout:5/646: unlink dc/dd/d47/c5f 0 2024-05-15T15:54:18.710 INFO:tasks.workunit.client.0.smithi049.stdout:3/612: creat d12/d14/d52/db9/d6b/fbb x:0 0 0 2024-05-15T15:54:18.717 INFO:tasks.workunit.client.1.smithi144.stdout:6/690: creat d7/db/d9d/d72/dee/ff7 x:0 0 0 2024-05-15T15:54:18.719 INFO:tasks.workunit.client.0.smithi049.stdout:6/590: rename dd/d17/l22 to dd/d17/d64/lb3 0 2024-05-15T15:54:18.721 INFO:tasks.workunit.client.0.smithi049.stdout:2/586: creat d3/d78/d8c/fc7 x:0 0 0 2024-05-15T15:54:18.725 INFO:tasks.workunit.client.1.smithi144.stdout:1/671: chown d7/d1d/d33/d55/f8d 45 1 2024-05-15T15:54:18.726 INFO:tasks.workunit.client.0.smithi049.stdout:3/613: symlink d12/d14/d52/db9/d6b/lbc 0 2024-05-15T15:54:18.728 INFO:tasks.workunit.client.1.smithi144.stdout:9/709: readlink d0/d8/d21/ld9 0 2024-05-15T15:54:18.730 INFO:tasks.workunit.client.1.smithi144.stdout:0/657: sync 2024-05-15T15:54:18.730 INFO:tasks.workunit.client.1.smithi144.stdout:3/686: sync 2024-05-15T15:54:18.731 INFO:tasks.workunit.client.0.smithi049.stdout:6/591: write dd/f19 [1491034,100883] 0 2024-05-15T15:54:18.732 INFO:tasks.workunit.client.0.smithi049.stdout:6/592: chown dd/d17/d24/d2b/d36/d50/d54 947 1 2024-05-15T15:54:18.732 INFO:tasks.workunit.client.0.smithi049.stdout:2/587: creat d3/db/d1d/d1e/fc8 x:0 0 0 2024-05-15T15:54:18.733 INFO:tasks.workunit.client.0.smithi049.stdout:6/593: stat dd/d17/d24/d8a 0 2024-05-15T15:54:18.734 INFO:tasks.workunit.client.1.smithi144.stdout:8/741: dwrite d2/d24/d2d/da0/d6e/d71/d73/d98/ff0 [0,4194304] 0 2024-05-15T15:54:18.735 INFO:tasks.workunit.client.1.smithi144.stdout:8/742: read - d2/de/fea zero size 2024-05-15T15:54:18.738 INFO:tasks.workunit.client.1.smithi144.stdout:4/680: rename d6/dd/d34/d3b/d30/db/d1d/d40/d6c/d82 to d6/d42/d46/d4b/d1e/d99/dda 0 2024-05-15T15:54:18.739 INFO:tasks.workunit.client.1.smithi144.stdout:2/641: dwrite d3/d7/d20/d22/d52/f69 [0,4194304] 0 2024-05-15T15:54:18.740 INFO:tasks.workunit.client.0.smithi049.stdout:6/594: dwrite dd/d17/d24/d2b/d36/f41 [0,4194304] 0 2024-05-15T15:54:18.741 INFO:tasks.workunit.client.0.smithi049.stdout:3/614: dread - d12/d14/f54 zero size 2024-05-15T15:54:18.742 INFO:tasks.workunit.client.0.smithi049.stdout:3/615: read - d12/d14/fb8 zero size 2024-05-15T15:54:18.743 INFO:tasks.workunit.client.0.smithi049.stdout:5/647: truncate dc/dd/d47/d83/dcc/f95 5800195 0 2024-05-15T15:54:18.743 INFO:tasks.workunit.client.1.smithi144.stdout:1/672: dwrite d7/d1d/d33/f71 [0,4194304] 0 2024-05-15T15:54:18.746 INFO:tasks.workunit.client.1.smithi144.stdout:9/710: rmdir d0/d8/d1b/d62/d7d/d8d 39 2024-05-15T15:54:18.747 INFO:tasks.workunit.client.0.smithi049.stdout:7/571: sync 2024-05-15T15:54:18.747 INFO:tasks.workunit.client.0.smithi049.stdout:1/780: sync 2024-05-15T15:54:18.747 INFO:tasks.workunit.client.1.smithi144.stdout:6/691: mknod d7/db/d9d/db7/cf8 0 2024-05-15T15:54:18.747 INFO:tasks.workunit.client.0.smithi049.stdout:7/572: chown d0/l84 492 1 2024-05-15T15:54:18.749 INFO:tasks.workunit.client.1.smithi144.stdout:6/692: write d7/db/da2/f66 [1450488,62458] 0 2024-05-15T15:54:18.753 INFO:tasks.workunit.client.0.smithi049.stdout:2/588: read d3/db/d1d/d1e/d2b/d2e/f41 [2424172,48549] 0 2024-05-15T15:54:18.755 INFO:tasks.workunit.client.1.smithi144.stdout:5/692: rename d0/dc/d24/d28/d2d/l66 to d0/dc/d24/d8e/d95/le4 0 2024-05-15T15:54:18.756 INFO:tasks.workunit.client.0.smithi049.stdout:8/586: getdents d2/de/d71/d86/d5d 0 2024-05-15T15:54:18.757 INFO:tasks.workunit.client.1.smithi144.stdout:4/681: creat d6/d42/d46/d4b/fdb x:0 0 0 2024-05-15T15:54:18.758 INFO:tasks.workunit.client.0.smithi049.stdout:0/619: getdents dc 0 2024-05-15T15:54:18.759 INFO:tasks.workunit.client.0.smithi049.stdout:0/620: stat dc/d13/d14/d33/f53 0 2024-05-15T15:54:18.760 INFO:tasks.workunit.client.0.smithi049.stdout:0/621: fdatasync dc/d13/d14/d25/d31/dae/dc5/f6c 0 2024-05-15T15:54:18.760 INFO:tasks.workunit.client.1.smithi144.stdout:9/711: unlink d0/d8/d1b/d62/fc5 0 2024-05-15T15:54:18.762 INFO:tasks.workunit.client.1.smithi144.stdout:9/712: write d0/d8/d5e/d44/da2/fe3 [271647,36189] 0 2024-05-15T15:54:18.763 INFO:tasks.workunit.client.0.smithi049.stdout:6/595: rename dd/d17/d24/d2b/d36/d50/c7d to dd/d26/cb4 0 2024-05-15T15:54:18.764 INFO:tasks.workunit.client.1.smithi144.stdout:1/673: dwrite d7/d1d/d33/d1e/d87/f92 [4194304,4194304] 0 2024-05-15T15:54:18.764 INFO:tasks.workunit.client.0.smithi049.stdout:3/616: link d12/d14/f1e d12/d30/d63/fbd 0 2024-05-15T15:54:18.765 INFO:tasks.workunit.client.1.smithi144.stdout:1/674: write d7/d1d/f32 [173106,125687] 0 2024-05-15T15:54:18.766 INFO:tasks.workunit.client.1.smithi144.stdout:5/693: creat d0/dc/d2f/d68/fe5 x:0 0 0 2024-05-15T15:54:18.767 INFO:tasks.workunit.client.1.smithi144.stdout:4/682: write d6/d42/d46/d4b/d1e/d44/fbf [912835,119882] 0 2024-05-15T15:54:18.767 INFO:tasks.workunit.client.0.smithi049.stdout:8/587: unlink d2/de/d71/ca8 0 2024-05-15T15:54:18.768 INFO:tasks.workunit.client.0.smithi049.stdout:8/588: write d2/f2c [1169007,58455] 0 2024-05-15T15:54:18.772 INFO:tasks.workunit.client.1.smithi144.stdout:9/713: creat d0/d8/d1b/d79/d8a/ff2 x:0 0 0 2024-05-15T15:54:18.774 INFO:tasks.workunit.client.0.smithi049.stdout:1/781: dread d0/d4/dd/de/d1e/f37 [0,4194304] 0 2024-05-15T15:54:18.775 INFO:tasks.workunit.client.0.smithi049.stdout:1/782: stat d0/d4/dd/de/d36/fef 0 2024-05-15T15:54:18.776 INFO:tasks.workunit.client.1.smithi144.stdout:5/694: creat d0/dc/d12/d4a/d73/fe6 x:0 0 0 2024-05-15T15:54:18.776 INFO:tasks.workunit.client.1.smithi144.stdout:5/695: chown d0/dc/d2f/d34/d63 512601823 1 2024-05-15T15:54:18.777 INFO:tasks.workunit.client.1.smithi144.stdout:4/683: rmdir d6/dd/d34/d3b/d30/d59/d66 39 2024-05-15T15:54:18.777 INFO:tasks.workunit.client.1.smithi144.stdout:4/684: chown d6/dd/d2d/caf 62 1 2024-05-15T15:54:18.778 INFO:tasks.workunit.client.0.smithi049.stdout:2/589: mkdir d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/dc9 0 2024-05-15T15:54:18.783 INFO:tasks.workunit.client.0.smithi049.stdout:0/622: mkdir dc/d13/d14/dc9 0 2024-05-15T15:54:18.784 INFO:tasks.workunit.client.0.smithi049.stdout:7/573: creat d0/d6/d23/faf x:0 0 0 2024-05-15T15:54:18.785 INFO:tasks.workunit.client.1.smithi144.stdout:5/696: dread d0/dc/d24/d8e/d93/f62 [0,4194304] 0 2024-05-15T15:54:18.786 INFO:tasks.workunit.client.0.smithi049.stdout:7/574: write d0/d2f/f97 [634069,42225] 0 2024-05-15T15:54:18.788 INFO:tasks.workunit.client.1.smithi144.stdout:9/714: readlink d0/d8/d1b/d62/d7d/d8d/dbb/dce/lde 0 2024-05-15T15:54:18.791 INFO:tasks.workunit.client.0.smithi049.stdout:1/783: write d0/d4/dd/de/d4d/d8b/dba/fe9 [1593360,12714] 0 2024-05-15T15:54:18.794 INFO:tasks.workunit.client.1.smithi144.stdout:5/697: mkdir d0/dc/d2f/d34/d63/d6e/de7 0 2024-05-15T15:54:18.794 INFO:tasks.workunit.client.1.smithi144.stdout:6/693: getdents d7/db/d9d/db7 0 2024-05-15T15:54:18.795 INFO:tasks.workunit.client.0.smithi049.stdout:2/590: readlink d3/db/d82/la2 0 2024-05-15T15:54:18.796 INFO:tasks.workunit.client.0.smithi049.stdout:8/589: dread d2/de/d71/d86/d5d/d63/f6e [0,4194304] 0 2024-05-15T15:54:18.796 INFO:tasks.workunit.client.1.smithi144.stdout:9/715: write d0/d8/d5e/d44/d10/d19/f49 [4270108,124891] 0 2024-05-15T15:54:18.800 INFO:tasks.workunit.client.1.smithi144.stdout:6/694: dread d7/f41 [0,4194304] 0 2024-05-15T15:54:18.800 INFO:tasks.workunit.client.1.smithi144.stdout:1/675: dread d7/d1d/d33/d41/d22/fa7 [0,4194304] 0 2024-05-15T15:54:18.800 INFO:tasks.workunit.client.1.smithi144.stdout:6/695: chown d7/db/d35/d84 3802 1 2024-05-15T15:54:18.801 INFO:tasks.workunit.client.1.smithi144.stdout:6/696: chown d7/db/d11/d3c/fac 124445250 1 2024-05-15T15:54:18.802 INFO:tasks.workunit.client.1.smithi144.stdout:7/687: sync 2024-05-15T15:54:18.803 INFO:tasks.workunit.client.1.smithi144.stdout:1/676: truncate d7/d1d/d33/d29/d6b/d5e/d74/fab 1043188 0 2024-05-15T15:54:18.803 INFO:tasks.workunit.client.0.smithi049.stdout:5/648: rename dc/d4a/d9e/cab to dc/dd/d47/d83/dcc/cd0 0 2024-05-15T15:54:18.805 INFO:tasks.workunit.client.1.smithi144.stdout:6/697: dread d7/d32/f50 [0,4194304] 0 2024-05-15T15:54:18.806 INFO:tasks.workunit.client.0.smithi049.stdout:7/575: mknod d0/d2f/cb0 0 2024-05-15T15:54:18.807 INFO:tasks.workunit.client.1.smithi144.stdout:9/716: mkdir d0/d8/d1b/d62/d7d/d8d/df3 0 2024-05-15T15:54:18.809 INFO:tasks.workunit.client.0.smithi049.stdout:1/784: creat d0/d4/dd/de/d1e/d16/df1/f116 x:0 0 0 2024-05-15T15:54:18.810 INFO:tasks.workunit.client.1.smithi144.stdout:1/677: dwrite d7/d1d/d33/d41/d22/fa8 [4194304,4194304] 0 2024-05-15T15:54:18.811 INFO:tasks.workunit.client.0.smithi049.stdout:1/785: fsync d0/d4/dd/de/d36/f6a 0 2024-05-15T15:54:18.811 INFO:tasks.workunit.client.1.smithi144.stdout:9/717: read d0/f78 [1901877,111233] 0 2024-05-15T15:54:18.812 INFO:tasks.workunit.client.0.smithi049.stdout:2/591: creat d3/d85/fca x:0 0 0 2024-05-15T15:54:18.815 INFO:tasks.workunit.client.0.smithi049.stdout:8/590: unlink d2/de/d25/d74/f76 0 2024-05-15T15:54:18.818 INFO:tasks.workunit.client.0.smithi049.stdout:0/623: mknod dc/d13/d14/d25/db9/cca 0 2024-05-15T15:54:18.819 INFO:tasks.workunit.client.0.smithi049.stdout:0/624: chown dc/c1a 123698261 1 2024-05-15T15:54:18.821 INFO:tasks.workunit.client.1.smithi144.stdout:4/685: link d6/dd/d34/d3b/d30/d59/d66/l6a d6/dd/d34/d3b/d30/db/d1d/d40/ldc 0 2024-05-15T15:54:18.822 INFO:tasks.workunit.client.1.smithi144.stdout:4/686: chown d6/dd/d34/d3b/f4c 1139374 1 2024-05-15T15:54:18.823 INFO:tasks.workunit.client.0.smithi049.stdout:6/596: dwrite dd/d17/d24/d2b/f76 [0,4194304] 0 2024-05-15T15:54:18.825 INFO:tasks.workunit.client.1.smithi144.stdout:6/698: creat d7/db/d11/d7e/ff9 x:0 0 0 2024-05-15T15:54:18.825 INFO:tasks.workunit.client.0.smithi049.stdout:7/576: mkdir d0/d6/d23/d45/db1 0 2024-05-15T15:54:18.826 INFO:tasks.workunit.client.1.smithi144.stdout:5/698: link d0/dc/d24/d8e/d93/cac d0/dc/d2f/d34/d63/d6e/ce8 0 2024-05-15T15:54:18.828 INFO:tasks.workunit.client.0.smithi049.stdout:3/617: write d12/d14/d32/d64/d88/fb3 [383372,20253] 0 2024-05-15T15:54:18.830 INFO:tasks.workunit.client.1.smithi144.stdout:4/687: dwrite d6/dd/d34/d3b/d30/db/d1d/d40/d57/f85 [0,4194304] 0 2024-05-15T15:54:18.831 INFO:tasks.workunit.client.1.smithi144.stdout:4/688: readlink d6/d42/d46/d4b/l1a 0 2024-05-15T15:54:18.832 INFO:tasks.workunit.client.1.smithi144.stdout:4/689: readlink d6/dd/d2d/l72 0 2024-05-15T15:54:18.832 INFO:tasks.workunit.client.1.smithi144.stdout:9/718: dread - d0/d8/d1b/d62/f84 zero size 2024-05-15T15:54:18.833 INFO:tasks.workunit.client.0.smithi049.stdout:0/625: dwrite dc/fa1 [0,4194304] 0 2024-05-15T15:54:18.835 INFO:tasks.workunit.client.0.smithi049.stdout:0/626: chown dc/l16 51762921 1 2024-05-15T15:54:18.835 INFO:tasks.workunit.client.0.smithi049.stdout:6/597: dwrite dd/f4c [0,4194304] 0 2024-05-15T15:54:18.836 INFO:tasks.workunit.client.0.smithi049.stdout:0/627: chown dc/d13/d14/f19 5353222 1 2024-05-15T15:54:18.837 INFO:tasks.workunit.client.0.smithi049.stdout:5/649: creat dc/dd/d47/da6/dce/fd1 x:0 0 0 2024-05-15T15:54:18.840 INFO:tasks.workunit.client.0.smithi049.stdout:0/628: write dc/da4/fab [454040,72081] 0 2024-05-15T15:54:18.841 INFO:tasks.workunit.client.0.smithi049.stdout:5/650: write dc/db0/dc7/fc9 [101058,130994] 0 2024-05-15T15:54:18.842 INFO:tasks.workunit.client.1.smithi144.stdout:7/688: mkdir d4/d41/de8 0 2024-05-15T15:54:18.843 INFO:tasks.workunit.client.0.smithi049.stdout:5/651: fsync dc/d4a/d9e/fb1 0 2024-05-15T15:54:18.844 INFO:tasks.workunit.client.0.smithi049.stdout:0/629: chown dc/d13/d2b/d76/d8a/da2/db3/cc0 333081 1 2024-05-15T15:54:18.848 INFO:tasks.workunit.client.1.smithi144.stdout:6/699: fsync d7/d1e/fa3 0 2024-05-15T15:54:18.849 INFO:tasks.workunit.client.0.smithi049.stdout:0/630: dread f3 [0,4194304] 0 2024-05-15T15:54:18.858 INFO:tasks.workunit.client.0.smithi049.stdout:2/592: mkdir d3/db/d5b/dcb 0 2024-05-15T15:54:18.860 INFO:tasks.workunit.client.1.smithi144.stdout:4/690: creat d6/d49/fdd x:0 0 0 2024-05-15T15:54:18.861 INFO:tasks.workunit.client.0.smithi049.stdout:8/591: write d2/de/d71/d86/d5d/d7d/d1b/f2b [1629688,118788] 0 2024-05-15T15:54:18.862 INFO:tasks.workunit.client.0.smithi049.stdout:3/618: creat d12/d30/d63/d68/d70/d75/fbe x:0 0 0 2024-05-15T15:54:18.866 INFO:tasks.workunit.client.0.smithi049.stdout:6/598: creat dd/d17/d64/fb5 x:0 0 0 2024-05-15T15:54:18.867 INFO:tasks.workunit.client.1.smithi144.stdout:0/658: sync 2024-05-15T15:54:18.868 INFO:tasks.workunit.client.0.smithi049.stdout:5/652: mkdir dc/dd/d7b/dd2 0 2024-05-15T15:54:18.869 INFO:tasks.workunit.client.1.smithi144.stdout:6/700: creat d7/db/d9d/db7/ffa x:0 0 0 2024-05-15T15:54:18.870 INFO:tasks.workunit.client.1.smithi144.stdout:6/701: truncate d7/db/da2/f5e 644634 0 2024-05-15T15:54:18.872 INFO:tasks.workunit.client.1.smithi144.stdout:1/678: rename d7/d4b/d53 to d7/d4b/dd9 0 2024-05-15T15:54:18.873 INFO:tasks.workunit.client.0.smithi049.stdout:4/600: sync 2024-05-15T15:54:18.873 INFO:tasks.workunit.client.0.smithi049.stdout:9/484: sync 2024-05-15T15:54:18.873 INFO:tasks.workunit.client.1.smithi144.stdout:1/679: write d7/d1d/d33/d29/f64 [2107950,106351] 0 2024-05-15T15:54:18.880 INFO:tasks.workunit.client.0.smithi049.stdout:0/631: dwrite dc/d13/d14/d25/d31/f7e [0,4194304] 0 2024-05-15T15:54:18.888 INFO:tasks.workunit.client.0.smithi049.stdout:1/786: getdents d0/d4/dd/de/d4d/d8b/de8 0 2024-05-15T15:54:18.892 INFO:tasks.workunit.client.0.smithi049.stdout:0/632: mkdir dc/d13/d14/d2f/d8f/dcb 0 2024-05-15T15:54:18.893 INFO:tasks.workunit.client.0.smithi049.stdout:8/592: symlink d2/de/d25/d74/d94/lb6 0 2024-05-15T15:54:18.894 INFO:tasks.workunit.client.0.smithi049.stdout:0/633: fdatasync dc/d13/d14/d25/d31/d71/f3a 0 2024-05-15T15:54:18.894 INFO:tasks.workunit.client.0.smithi049.stdout:5/653: creat dc/dd/d7b/dd2/fd3 x:0 0 0 2024-05-15T15:54:18.894 INFO:tasks.workunit.client.0.smithi049.stdout:8/593: fsync d2/de/d25/f32 0 2024-05-15T15:54:18.895 INFO:tasks.workunit.client.0.smithi049.stdout:9/485: symlink d0/d8e/la6 0 2024-05-15T15:54:18.895 INFO:tasks.workunit.client.0.smithi049.stdout:1/787: readlink d0/d4/dd/de/d1e/d16/d8a/lfd 0 2024-05-15T15:54:18.896 INFO:tasks.workunit.client.0.smithi049.stdout:4/601: getdents d18/d85/d98/dc3 0 2024-05-15T15:54:18.897 INFO:tasks.workunit.client.0.smithi049.stdout:3/619: dread d12/d14/d32/d64/d88/d9c/fb4 [0,4194304] 0 2024-05-15T15:54:18.900 INFO:tasks.workunit.client.0.smithi049.stdout:8/594: fdatasync d2/f3d 0 2024-05-15T15:54:18.903 INFO:tasks.workunit.client.0.smithi049.stdout:1/788: read d0/d4/dd/f28 [6125659,64219] 0 2024-05-15T15:54:18.904 INFO:tasks.workunit.client.0.smithi049.stdout:3/620: dwrite d12/d14/fb8 [0,4194304] 0 2024-05-15T15:54:18.904 INFO:tasks.workunit.client.0.smithi049.stdout:4/602: unlink d18/d21/d2b/f2c 0 2024-05-15T15:54:18.905 INFO:tasks.workunit.client.0.smithi049.stdout:4/603: chown d18/d69/c97 4348 1 2024-05-15T15:54:18.906 INFO:tasks.workunit.client.0.smithi049.stdout:4/604: fsync d18/d85/fab 0 2024-05-15T15:54:18.906 INFO:tasks.workunit.client.0.smithi049.stdout:9/486: creat d0/d8/d9/d52/d5e/d5f/d56/fa7 x:0 0 0 2024-05-15T15:54:18.906 INFO:tasks.workunit.client.0.smithi049.stdout:7/577: link d0/d2f/f3e d0/d6/d23/d45/d6a/da1/d80/fb2 0 2024-05-15T15:54:18.908 INFO:tasks.workunit.client.0.smithi049.stdout:9/487: dread - d0/d8/d1b/d71/d7d/fa3 zero size 2024-05-15T15:54:18.908 INFO:tasks.workunit.client.0.smithi049.stdout:5/654: symlink dc/dd/d47/ld4 0 2024-05-15T15:54:18.909 INFO:tasks.workunit.client.0.smithi049.stdout:5/655: fdatasync dc/d4a/f8c 0 2024-05-15T15:54:18.910 INFO:tasks.workunit.client.0.smithi049.stdout:8/595: stat d2/de/d71/d86/d5d/d7d/f4c 0 2024-05-15T15:54:18.911 INFO:tasks.workunit.client.0.smithi049.stdout:0/634: rename dc/d6d/l59 to dc/d13/d3d/lcc 0 2024-05-15T15:54:18.920 INFO:tasks.workunit.client.0.smithi049.stdout:7/578: unlink d0/d6/da/l71 0 2024-05-15T15:54:18.920 INFO:tasks.workunit.client.0.smithi049.stdout:5/656: mkdir dc/db0/dd5 0 2024-05-15T15:54:18.922 INFO:tasks.workunit.client.0.smithi049.stdout:1/789: symlink d0/d4/dd/de/d4d/db8/l117 0 2024-05-15T15:54:18.923 INFO:tasks.workunit.client.1.smithi144.stdout:7/689: creat d4/df/d25/d40/d73/fe9 x:0 0 0 2024-05-15T15:54:18.923 INFO:tasks.workunit.client.1.smithi144.stdout:0/659: truncate d4/d2d/fab 652503 0 2024-05-15T15:54:18.924 INFO:tasks.workunit.client.0.smithi049.stdout:4/605: dwrite d18/d1e/d63/f96 [0,4194304] 0 2024-05-15T15:54:18.926 INFO:tasks.workunit.client.1.smithi144.stdout:5/699: getdents d0/dc/d2f/d34 0 2024-05-15T15:54:18.927 INFO:tasks.workunit.client.1.smithi144.stdout:9/719: dwrite d0/f5 [0,4194304] 0 2024-05-15T15:54:18.928 INFO:tasks.workunit.client.0.smithi049.stdout:5/657: chown dc/d4a/lb8 4086361 1 2024-05-15T15:54:18.928 INFO:tasks.workunit.client.1.smithi144.stdout:5/700: stat d0/d7e/d9b 0 2024-05-15T15:54:18.931 INFO:tasks.workunit.client.1.smithi144.stdout:0/660: write d4/d2d/f44 [191432,86761] 0 2024-05-15T15:54:18.932 INFO:tasks.workunit.client.0.smithi049.stdout:9/488: dwrite d0/d8/d42/f91 [0,4194304] 0 2024-05-15T15:54:18.936 INFO:tasks.workunit.client.0.smithi049.stdout:3/621: dread d12/d67/fb0 [0,4194304] 0 2024-05-15T15:54:18.939 INFO:tasks.workunit.client.0.smithi049.stdout:8/596: dwrite d2/de/d25/f32 [0,4194304] 0 2024-05-15T15:54:18.940 INFO:tasks.workunit.client.1.smithi144.stdout:7/690: creat d4/d3c/d76/d9d/dac/de5/fea x:0 0 0 2024-05-15T15:54:18.942 INFO:tasks.workunit.client.0.smithi049.stdout:0/635: dwrite dc/da4/fab [0,4194304] 0 2024-05-15T15:54:18.943 INFO:tasks.workunit.client.1.smithi144.stdout:1/680: link d7/d1d/d33/d41/d22/fa7 d7/d1d/d58/fda 0 2024-05-15T15:54:18.944 INFO:tasks.workunit.client.0.smithi049.stdout:9/489: chown d0/d8/d21/c47 1 1 2024-05-15T15:54:18.944 INFO:tasks.workunit.client.0.smithi049.stdout:3/622: readlink d12/d14/d52/db9/d7a/lb6 0 2024-05-15T15:54:18.947 INFO:tasks.workunit.client.1.smithi144.stdout:9/720: symlink d0/d8/d5e/d44/d99/lf4 0 2024-05-15T15:54:18.949 INFO:tasks.workunit.client.0.smithi049.stdout:2/593: truncate d3/da/fb4 570560 0 2024-05-15T15:54:18.954 INFO:tasks.workunit.client.0.smithi049.stdout:6/599: dwrite dd/f6c [0,4194304] 0 2024-05-15T15:54:18.956 INFO:tasks.workunit.client.1.smithi144.stdout:5/701: truncate d0/dc/d2f/d68/d5c/f81 809824 0 2024-05-15T15:54:18.961 INFO:tasks.workunit.client.1.smithi144.stdout:8/743: sync 2024-05-15T15:54:18.961 INFO:tasks.workunit.client.1.smithi144.stdout:3/687: sync 2024-05-15T15:54:18.962 INFO:tasks.workunit.client.1.smithi144.stdout:2/642: sync 2024-05-15T15:54:18.963 INFO:tasks.workunit.client.1.smithi144.stdout:3/688: chown d0/d1/d17/d39/d40/fc0 32992994 1 2024-05-15T15:54:18.964 INFO:tasks.workunit.client.1.smithi144.stdout:3/689: stat d0/d1/d56/d71/d73/dd1 0 2024-05-15T15:54:18.969 INFO:tasks.workunit.client.1.smithi144.stdout:6/702: link d7/d1e/l9f d7/db/lfb 0 2024-05-15T15:54:18.970 INFO:tasks.workunit.client.1.smithi144.stdout:0/661: fdatasync d4/d2d/fab 0 2024-05-15T15:54:18.971 INFO:tasks.workunit.client.1.smithi144.stdout:1/681: dread d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:18.972 INFO:tasks.workunit.client.1.smithi144.stdout:1/682: chown d7/d1d/f32 10065 1 2024-05-15T15:54:18.972 INFO:tasks.workunit.client.1.smithi144.stdout:5/702: creat d0/dc/d2f/d34/d3e/fe9 x:0 0 0 2024-05-15T15:54:18.973 INFO:tasks.workunit.client.1.smithi144.stdout:7/691: symlink d4/leb 0 2024-05-15T15:54:18.974 INFO:tasks.workunit.client.1.smithi144.stdout:5/703: readlink d0/dc/d24/d28/l53 0 2024-05-15T15:54:18.977 INFO:tasks.workunit.client.1.smithi144.stdout:0/662: dwrite d4/d2d/d48/fd5 [0,4194304] 0 2024-05-15T15:54:18.979 INFO:tasks.workunit.client.1.smithi144.stdout:6/703: fsync d7/db/f2c 0 2024-05-15T15:54:18.980 INFO:tasks.workunit.client.0.smithi049.stdout:7/579: mkdir d0/d34/db3 0 2024-05-15T15:54:18.982 INFO:tasks.workunit.client.0.smithi049.stdout:5/658: mkdir dc/dd/d10/db3/dd6 0 2024-05-15T15:54:18.982 INFO:tasks.workunit.client.0.smithi049.stdout:7/580: write d0/d2f/f48 [1127912,99881] 0 2024-05-15T15:54:18.986 INFO:tasks.workunit.client.0.smithi049.stdout:4/606: dread d18/d21/d2b/d4e/f87 [0,4194304] 0 2024-05-15T15:54:18.988 INFO:tasks.workunit.client.1.smithi144.stdout:5/704: mkdir d0/dc/d24/d8e/d93/d6b/dea 0 2024-05-15T15:54:18.992 INFO:tasks.workunit.client.0.smithi049.stdout:9/490: fsync d0/d8/d9/d52/d5e/d5f/f38 0 2024-05-15T15:54:18.994 INFO:tasks.workunit.client.1.smithi144.stdout:5/705: dread d0/dc/d24/d28/f82 [0,4194304] 0 2024-05-15T15:54:18.997 INFO:tasks.workunit.client.0.smithi049.stdout:2/594: unlink d3/db/d1d/d1e/d2b/d2e/f48 0 2024-05-15T15:54:18.998 INFO:tasks.workunit.client.0.smithi049.stdout:6/600: chown dd/d26/cb4 56670 1 2024-05-15T15:54:19.000 INFO:tasks.workunit.client.0.smithi049.stdout:0/636: dread dc/d13/d14/d25/d31/dae/dc5/f4f [0,4194304] 0 2024-05-15T15:54:19.005 INFO:tasks.workunit.client.0.smithi049.stdout:7/581: creat d0/d6/d23/d4a/fb4 x:0 0 0 2024-05-15T15:54:19.005 INFO:tasks.workunit.client.1.smithi144.stdout:9/721: getdents d0/d8/d1b/d62 0 2024-05-15T15:54:19.007 INFO:tasks.workunit.client.0.smithi049.stdout:0/637: dwrite dc/d13/d14/d2f/f51 [4194304,4194304] 0 2024-05-15T15:54:19.008 INFO:tasks.workunit.client.1.smithi144.stdout:0/663: link d4/f22 d4/d2d/dae/feb 0 2024-05-15T15:54:19.013 INFO:tasks.workunit.client.0.smithi049.stdout:3/623: mknod d12/d26/cbf 0 2024-05-15T15:54:19.014 INFO:tasks.workunit.client.1.smithi144.stdout:5/706: creat d0/dc/d24/d8e/d95/feb x:0 0 0 2024-05-15T15:54:19.015 INFO:tasks.workunit.client.0.smithi049.stdout:7/582: dwrite d0/d2f/d5c/f64 [0,4194304] 0 2024-05-15T15:54:19.015 INFO:tasks.workunit.client.1.smithi144.stdout:5/707: dread - d0/dc/d24/d8e/d93/d9e/fcb zero size 2024-05-15T15:54:19.016 INFO:tasks.workunit.client.1.smithi144.stdout:5/708: stat d0/dc/d24/d28/d2d/d59 0 2024-05-15T15:54:19.017 INFO:tasks.workunit.client.1.smithi144.stdout:1/683: creat d7/d1d/d33/d1e/d87/fdb x:0 0 0 2024-05-15T15:54:19.017 INFO:tasks.workunit.client.1.smithi144.stdout:5/709: write d0/dc/d2f/d34/d63/d6e/fd9 [303001,41169] 0 2024-05-15T15:54:19.018 INFO:tasks.workunit.client.1.smithi144.stdout:5/710: dread - d0/dc/d12/d4a/d5d/fc0 zero size 2024-05-15T15:54:19.019 INFO:tasks.workunit.client.1.smithi144.stdout:2/643: symlink d3/d7/d20/dcf/ld8 0 2024-05-15T15:54:19.022 INFO:tasks.workunit.client.0.smithi049.stdout:7/583: write d0/d6/da/f9a [1041600,83390] 0 2024-05-15T15:54:19.025 INFO:tasks.workunit.client.0.smithi049.stdout:9/491: creat d0/d8/d9/d52/d5e/d8a/fa8 x:0 0 0 2024-05-15T15:54:19.027 INFO:tasks.workunit.client.0.smithi049.stdout:2/595: fsync d3/db/d1d/d1e/d2b/d2e/f87 0 2024-05-15T15:54:19.029 INFO:tasks.workunit.client.0.smithi049.stdout:6/601: mkdir dd/d17/d24/d2b/d36/d50/d54/db6 0 2024-05-15T15:54:19.034 INFO:tasks.workunit.client.1.smithi144.stdout:4/691: sync 2024-05-15T15:54:19.035 INFO:tasks.workunit.client.0.smithi049.stdout:9/492: dwrite d0/d8/d9/d22/f2e [0,4194304] 0 2024-05-15T15:54:19.035 INFO:tasks.workunit.client.0.smithi049.stdout:1/790: rename d0/d4/dd/de/d48/laf to d0/d4/dd/l118 0 2024-05-15T15:54:19.037 INFO:tasks.workunit.client.1.smithi144.stdout:4/692: dwrite d6/dd/d34/d3b/d30/d59/d66/f9e [0,4194304] 0 2024-05-15T15:54:19.037 INFO:tasks.workunit.client.0.smithi049.stdout:6/602: dread dd/ff [0,4194304] 0 2024-05-15T15:54:19.039 INFO:tasks.workunit.client.0.smithi049.stdout:1/791: read - d0/d4/dd/de/d1e/d2e/dad/d65/df3/ffa zero size 2024-05-15T15:54:19.040 INFO:tasks.workunit.client.0.smithi049.stdout:6/603: dread - dd/d17/d24/d2b/d36/d93/fa9 zero size 2024-05-15T15:54:19.042 INFO:tasks.workunit.client.0.smithi049.stdout:6/604: chown dd/d17/c68 19262525 1 2024-05-15T15:54:19.044 INFO:tasks.workunit.client.1.smithi144.stdout:5/711: creat d0/dc/d2f/d34/d63/fec x:0 0 0 2024-05-15T15:54:19.045 INFO:tasks.workunit.client.0.smithi049.stdout:7/584: truncate d0/d6/f3c 756561 0 2024-05-15T15:54:19.050 INFO:tasks.workunit.client.1.smithi144.stdout:5/712: dwrite d0/dc/d24/d8e/d95/fb0 [0,4194304] 0 2024-05-15T15:54:19.051 INFO:tasks.workunit.client.1.smithi144.stdout:5/713: readlink d0/dc/d2f/d79/l92 0 2024-05-15T15:54:19.052 INFO:tasks.workunit.client.1.smithi144.stdout:0/664: getdents d4/df/d1f/d24/d85 0 2024-05-15T15:54:19.052 INFO:tasks.workunit.client.0.smithi049.stdout:2/596: read - d3/db/d1d/d1e/d2b/d3b/f5f zero size 2024-05-15T15:54:19.056 INFO:tasks.workunit.client.1.smithi144.stdout:2/644: getdents d3/d7/d20/d22/d52/d75 0 2024-05-15T15:54:19.057 INFO:tasks.workunit.client.1.smithi144.stdout:2/645: dread - d3/d7/d20/d89/d6e/fc4 zero size 2024-05-15T15:54:19.059 INFO:tasks.workunit.client.0.smithi049.stdout:5/659: creat dc/dd/fd7 x:0 0 0 2024-05-15T15:54:19.061 INFO:tasks.workunit.client.0.smithi049.stdout:1/792: fdatasync d0/f58 0 2024-05-15T15:54:19.064 INFO:tasks.workunit.client.0.smithi049.stdout:5/660: dread f0 [0,4194304] 0 2024-05-15T15:54:19.065 INFO:tasks.workunit.client.0.smithi049.stdout:6/605: mknod dd/d17/d24/d58/d5f/cb7 0 2024-05-15T15:54:19.066 INFO:tasks.workunit.client.0.smithi049.stdout:6/606: readlink dd/l25 0 2024-05-15T15:54:19.068 INFO:tasks.workunit.client.1.smithi144.stdout:2/646: rename d3/d7/d20/d40/fc9 to d3/d7/d20/d3b/fd9 0 2024-05-15T15:54:19.068 INFO:tasks.workunit.client.1.smithi144.stdout:5/714: dread d0/dc/f8c [0,4194304] 0 2024-05-15T15:54:19.069 INFO:tasks.workunit.client.1.smithi144.stdout:5/715: readlink d0/dc/d12/d4a/d73/ld4 0 2024-05-15T15:54:19.070 INFO:tasks.workunit.client.0.smithi049.stdout:2/597: creat d3/db/d1d/d1e/d2b/d3b/fcc x:0 0 0 2024-05-15T15:54:19.071 INFO:tasks.workunit.client.0.smithi049.stdout:2/598: chown d3/db/d1d/d1e/d2b/d3b/f5f 658157 1 2024-05-15T15:54:19.073 INFO:tasks.workunit.client.1.smithi144.stdout:2/647: creat d3/d7/d20/d89/d86/fda x:0 0 0 2024-05-15T15:54:19.074 INFO:tasks.workunit.client.1.smithi144.stdout:3/690: dwrite d0/d1/d56/f87 [0,4194304] 0 2024-05-15T15:54:19.075 INFO:tasks.workunit.client.1.smithi144.stdout:5/716: creat d0/dc/d2f/d34/d3e/fed x:0 0 0 2024-05-15T15:54:19.076 INFO:tasks.workunit.client.1.smithi144.stdout:3/691: write d0/d11/f61 [611260,122021] 0 2024-05-15T15:54:19.080 INFO:tasks.workunit.client.0.smithi049.stdout:8/597: write d2/de/d71/d86/d5d/d7d/f72 [3440953,90093] 0 2024-05-15T15:54:19.080 INFO:tasks.workunit.client.0.smithi049.stdout:8/598: readlink d2/d18/l79 0 2024-05-15T15:54:19.084 INFO:tasks.workunit.client.1.smithi144.stdout:8/744: dwrite d2/d24/fe4 [0,4194304] 0 2024-05-15T15:54:19.084 INFO:tasks.workunit.client.1.smithi144.stdout:5/717: mknod d0/dc/d24/d8e/d93/d9e/cee 0 2024-05-15T15:54:19.087 INFO:tasks.workunit.client.1.smithi144.stdout:5/718: chown d0/dc/d24/fd2 0 1 2024-05-15T15:54:19.087 INFO:tasks.workunit.client.1.smithi144.stdout:0/665: dread d4/df/d14/f5f [0,4194304] 0 2024-05-15T15:54:19.089 INFO:tasks.workunit.client.0.smithi049.stdout:2/599: rename d3/da/l8d to d3/d78/lcd 0 2024-05-15T15:54:19.091 INFO:tasks.workunit.client.1.smithi144.stdout:6/704: dwrite d7/db/d11/f42 [0,4194304] 0 2024-05-15T15:54:19.093 INFO:tasks.workunit.client.1.smithi144.stdout:0/666: dread d4/d6/fdf [0,4194304] 0 2024-05-15T15:54:19.094 INFO:tasks.workunit.client.1.smithi144.stdout:8/745: mknod d2/d1a/cf5 0 2024-05-15T15:54:19.095 INFO:tasks.workunit.client.0.smithi049.stdout:5/661: mkdir dc/db0/dd5/dd8 0 2024-05-15T15:54:19.099 INFO:tasks.workunit.client.0.smithi049.stdout:6/607: mknod dd/d17/d24/d2b/d36/d50/d54/db6/cb8 0 2024-05-15T15:54:19.100 INFO:tasks.workunit.client.0.smithi049.stdout:6/608: chown dd/d17/d24/d2b/f76 854 1 2024-05-15T15:54:19.103 INFO:tasks.workunit.client.0.smithi049.stdout:4/607: write d18/f1c [302624,77815] 0 2024-05-15T15:54:19.105 INFO:tasks.workunit.client.1.smithi144.stdout:8/746: unlink d2/d24/d2d/d30/fe3 0 2024-05-15T15:54:19.105 INFO:tasks.workunit.client.1.smithi144.stdout:8/747: dread - d2/fbe zero size 2024-05-15T15:54:19.105 INFO:tasks.workunit.client.1.smithi144.stdout:8/748: fsync d2/fbe 0 2024-05-15T15:54:19.106 INFO:tasks.workunit.client.0.smithi049.stdout:9/493: getdents d0/d8/d9/d52/d5e/d5f/d88 0 2024-05-15T15:54:19.106 INFO:tasks.workunit.client.1.smithi144.stdout:8/749: stat d2/d24/d2d/da0/d6e/d8d/fb4 0 2024-05-15T15:54:19.107 INFO:tasks.workunit.client.0.smithi049.stdout:9/494: fsync d0/d8/d1b/d71/d7d/fa3 0 2024-05-15T15:54:19.107 INFO:tasks.workunit.client.1.smithi144.stdout:5/719: mknod d0/dc/d2f/d34/d63/d6e/db8/cef 0 2024-05-15T15:54:19.110 INFO:tasks.workunit.client.0.smithi049.stdout:2/600: rename d3/db/d1d/d1e/d2b/d2e/f6a to d3/db/d1d/d1e/d2b/d3b/d47/d9a/fce 0 2024-05-15T15:54:19.111 INFO:tasks.workunit.client.0.smithi049.stdout:3/624: write d12/d14/d52/f5f [2243866,14197] 0 2024-05-15T15:54:19.112 INFO:tasks.workunit.client.1.smithi144.stdout:0/667: truncate d4/d6/d3a/d4c/fa6 327289 0 2024-05-15T15:54:19.113 INFO:tasks.workunit.client.1.smithi144.stdout:2/648: dread d3/d10/f7e [0,4194304] 0 2024-05-15T15:54:19.113 INFO:tasks.workunit.client.0.smithi049.stdout:8/599: mknod d2/d6a/cb7 0 2024-05-15T15:54:19.115 INFO:tasks.workunit.client.1.smithi144.stdout:2/649: read d3/d7/d20/d3f/f53 [351184,4399] 0 2024-05-15T15:54:19.116 INFO:tasks.workunit.client.0.smithi049.stdout:5/662: truncate dc/dd/d10/f1e 599380 0 2024-05-15T15:54:19.121 INFO:tasks.workunit.client.1.smithi144.stdout:4/693: dwrite d6/dd/d34/d3b/d30/db/d1d/d40/d57/f5e [0,4194304] 0 2024-05-15T15:54:19.121 INFO:tasks.workunit.client.0.smithi049.stdout:4/608: write d18/d85/fb5 [4184026,90451] 0 2024-05-15T15:54:19.122 INFO:tasks.workunit.client.0.smithi049.stdout:9/495: read - d0/d25/f76 zero size 2024-05-15T15:54:19.123 INFO:tasks.workunit.client.1.smithi144.stdout:5/720: rmdir d0/dc/d1f/d50/db3 39 2024-05-15T15:54:19.125 INFO:tasks.workunit.client.1.smithi144.stdout:0/668: mknod d4/df/da0/cec 0 2024-05-15T15:54:19.128 INFO:tasks.workunit.client.1.smithi144.stdout:7/692: sync 2024-05-15T15:54:19.129 INFO:tasks.workunit.client.0.smithi049.stdout:1/793: dwrite d0/f58 [0,4194304] 0 2024-05-15T15:54:19.137 INFO:tasks.workunit.client.0.smithi049.stdout:7/585: dwrite d0/d34/f3f [0,4194304] 0 2024-05-15T15:54:19.138 INFO:tasks.workunit.client.1.smithi144.stdout:4/694: creat d6/dd/d34/d3b/d30/db/d1d/d40/d6c/fde x:0 0 0 2024-05-15T15:54:19.139 INFO:tasks.workunit.client.0.smithi049.stdout:7/586: write d0/d2f/d5c/f7d [130500,112700] 0 2024-05-15T15:54:19.140 INFO:tasks.workunit.client.0.smithi049.stdout:8/600: unlink d2/de/d71/d86/d5d/d7d/d1b/cad 0 2024-05-15T15:54:19.143 INFO:tasks.workunit.client.0.smithi049.stdout:5/663: rmdir dc/d4a 39 2024-05-15T15:54:19.147 INFO:tasks.workunit.client.0.smithi049.stdout:7/587: dread d0/d2f/d5c/f70 [0,4194304] 0 2024-05-15T15:54:19.148 INFO:tasks.workunit.client.0.smithi049.stdout:5/664: dwrite dc/dd/f88 [0,4194304] 0 2024-05-15T15:54:19.163 INFO:tasks.workunit.client.1.smithi144.stdout:1/684: sync 2024-05-15T15:54:19.163 INFO:tasks.workunit.client.1.smithi144.stdout:9/722: sync 2024-05-15T15:54:19.168 INFO:tasks.workunit.client.1.smithi144.stdout:9/723: chown d0/d8/d1b/d62/d7d/d8d/dbb/dce/lf1 6123898 1 2024-05-15T15:54:19.173 INFO:tasks.workunit.client.1.smithi144.stdout:9/724: dwrite d0/d8/d1b/d62/fca [8388608,4194304] 0 2024-05-15T15:54:19.197 INFO:tasks.workunit.client.1.smithi144.stdout:0/669: symlink d4/d6/d4a/d74/led 0 2024-05-15T15:54:19.200 INFO:tasks.workunit.client.1.smithi144.stdout:3/692: truncate d0/d1/d17/d39/d40/d7e/f81 681997 0 2024-05-15T15:54:19.203 INFO:tasks.workunit.client.1.smithi144.stdout:6/705: write d7/db/d11/d3c/d61/fc5 [1016422,52224] 0 2024-05-15T15:54:19.204 INFO:tasks.workunit.client.0.smithi049.stdout:9/496: unlink d0/d8/cf 0 2024-05-15T15:54:19.206 INFO:tasks.workunit.client.1.smithi144.stdout:3/693: dwrite d0/d1/f23 [0,4194304] 0 2024-05-15T15:54:19.206 INFO:tasks.workunit.client.1.smithi144.stdout:3/694: chown d0/d11/caf 262 1 2024-05-15T15:54:19.213 INFO:tasks.workunit.client.1.smithi144.stdout:3/695: dwrite d0/d1/d56/f7f [0,4194304] 0 2024-05-15T15:54:19.215 INFO:tasks.workunit.client.1.smithi144.stdout:8/750: creat d2/d24/d2d/da0/d6e/d71/ff6 x:0 0 0 2024-05-15T15:54:19.220 INFO:tasks.workunit.client.0.smithi049.stdout:3/625: symlink d12/d14/d52/db9/lc0 0 2024-05-15T15:54:19.221 INFO:tasks.workunit.client.1.smithi144.stdout:4/695: fdatasync d6/d42/d46/d4b/d1e/d44/f93 0 2024-05-15T15:54:19.222 INFO:tasks.workunit.client.1.smithi144.stdout:8/751: dwrite d2/d1a/f3d [0,4194304] 0 2024-05-15T15:54:19.223 INFO:tasks.workunit.client.1.smithi144.stdout:5/721: creat d0/dc/d2f/d34/d3e/dd0/ff0 x:0 0 0 2024-05-15T15:54:19.224 INFO:tasks.workunit.client.1.smithi144.stdout:5/722: stat d0/dc/d2f/d34/l47 0 2024-05-15T15:54:19.224 INFO:tasks.workunit.client.1.smithi144.stdout:8/752: chown d2/d24/d2d/da0/d6e/d71/d8c 919115 1 2024-05-15T15:54:19.228 INFO:tasks.workunit.client.0.smithi049.stdout:8/601: rmdir d2/de/d71/d86 39 2024-05-15T15:54:19.229 INFO:tasks.workunit.client.1.smithi144.stdout:4/696: dwrite d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/fb7 [0,4194304] 0 2024-05-15T15:54:19.229 INFO:tasks.workunit.client.1.smithi144.stdout:4/697: chown d6/dd/d2d 213 1 2024-05-15T15:54:19.241 INFO:tasks.workunit.client.1.smithi144.stdout:7/693: write d4/f21 [139041,44725] 0 2024-05-15T15:54:19.244 INFO:tasks.workunit.client.1.smithi144.stdout:2/650: truncate d3/d7/d20/d37/f70 3732789 0 2024-05-15T15:54:19.250 INFO:tasks.workunit.client.0.smithi049.stdout:6/609: rename dd/d26/la2 to dd/d17/d24/d2b/d36/lb9 0 2024-05-15T15:54:19.251 INFO:tasks.workunit.client.0.smithi049.stdout:2/601: dwrite d3/db/f43 [0,4194304] 0 2024-05-15T15:54:19.252 INFO:tasks.workunit.client.0.smithi049.stdout:6/610: chown dd/d17/d24/d67/cab 11382993 1 2024-05-15T15:54:19.253 INFO:tasks.workunit.client.0.smithi049.stdout:6/611: write dd/d17/d24/d2b/d36/d50/f83 [633580,88685] 0 2024-05-15T15:54:19.254 INFO:tasks.workunit.client.0.smithi049.stdout:1/794: write d0/f5e [802350,46605] 0 2024-05-15T15:54:19.257 INFO:tasks.workunit.client.1.smithi144.stdout:7/694: dread d4/df/d25/d38/d68/f8d [0,4194304] 0 2024-05-15T15:54:19.258 INFO:tasks.workunit.client.0.smithi049.stdout:8/602: readlink d2/de/d71/d86/d5d/l91 0 2024-05-15T15:54:19.267 INFO:tasks.workunit.client.0.smithi049.stdout:6/612: dread dd/f57 [0,4194304] 0 2024-05-15T15:54:19.267 INFO:tasks.workunit.client.0.smithi049.stdout:2/602: fdatasync d3/d8/fe 0 2024-05-15T15:54:19.270 INFO:tasks.workunit.client.0.smithi049.stdout:7/588: dwrite d0/f11 [4194304,4194304] 0 2024-05-15T15:54:19.275 INFO:tasks.workunit.client.0.smithi049.stdout:3/626: mkdir d12/d30/dc1 0 2024-05-15T15:54:19.288 INFO:tasks.workunit.client.0.smithi049.stdout:2/603: unlink d3/f10 0 2024-05-15T15:54:19.289 INFO:tasks.workunit.client.0.smithi049.stdout:2/604: chown d3/db/d1d/d1e/d2b/d3b/cb5 384617 1 2024-05-15T15:54:19.292 INFO:tasks.workunit.client.0.smithi049.stdout:0/638: sync 2024-05-15T15:54:19.297 INFO:tasks.workunit.client.0.smithi049.stdout:5/665: link dc/dd/f13 dc/dd/d47/da6/fd9 0 2024-05-15T15:54:19.298 INFO:tasks.workunit.client.0.smithi049.stdout:4/609: getdents d18/d85/db6 0 2024-05-15T15:54:19.300 INFO:tasks.workunit.client.0.smithi049.stdout:6/613: dread dd/d17/d24/d58/d5f/f9f [4194304,4194304] 0 2024-05-15T15:54:19.302 INFO:tasks.workunit.client.0.smithi049.stdout:6/614: dread - dd/d26/f8b zero size 2024-05-15T15:54:19.303 INFO:tasks.workunit.client.0.smithi049.stdout:6/615: read - dd/d17/d24/d8a/f90 zero size 2024-05-15T15:54:19.305 INFO:tasks.workunit.client.0.smithi049.stdout:3/627: unlink d12/f92 0 2024-05-15T15:54:19.306 INFO:tasks.workunit.client.0.smithi049.stdout:9/497: dwrite d0/d8/f5a [0,4194304] 0 2024-05-15T15:54:19.312 INFO:tasks.workunit.client.0.smithi049.stdout:8/603: rename d2/de/d27 to d2/de/d71/d86/d5d/d7d/db8 0 2024-05-15T15:54:19.313 INFO:tasks.workunit.client.0.smithi049.stdout:8/604: chown d2/c13 0 1 2024-05-15T15:54:19.313 INFO:tasks.workunit.client.0.smithi049.stdout:5/666: rmdir dc/dd/d47/d83/dcc/d5c 39 2024-05-15T15:54:19.316 INFO:tasks.workunit.client.0.smithi049.stdout:2/605: dread d3/db/f30 [0,4194304] 0 2024-05-15T15:54:19.319 INFO:tasks.workunit.client.0.smithi049.stdout:1/795: getdents d0/d4/dd/de/d1e 0 2024-05-15T15:54:19.324 INFO:tasks.workunit.client.1.smithi144.stdout:1/685: rename d7/d1d/c93 to d7/d1d/d33/d41/d78/dc3/cdc 0 2024-05-15T15:54:19.326 INFO:tasks.workunit.client.0.smithi049.stdout:9/498: write d0/d8/d1b/f32 [1153774,60938] 0 2024-05-15T15:54:19.331 INFO:tasks.workunit.client.0.smithi049.stdout:8/605: mknod d2/de/d71/d86/d5d/d63/cb9 0 2024-05-15T15:54:19.336 INFO:tasks.workunit.client.1.smithi144.stdout:6/706: write d7/db/d9d/fea [5156151,21938] 0 2024-05-15T15:54:19.336 INFO:tasks.workunit.client.0.smithi049.stdout:1/796: creat d0/d4/dd/de/d48/d84/f119 x:0 0 0 2024-05-15T15:54:19.339 INFO:tasks.workunit.client.0.smithi049.stdout:3/628: mkdir d12/d14/d52/db9/da6/dc2 0 2024-05-15T15:54:19.341 INFO:tasks.workunit.client.1.smithi144.stdout:3/696: unlink d0/d1/d17/dce/fd0 0 2024-05-15T15:54:19.341 INFO:tasks.workunit.client.0.smithi049.stdout:0/639: link dc/d13/d14/d25/d31/dae/dc5/d57/c77 dc/d13/d14/d25/ccd 0 2024-05-15T15:54:19.342 INFO:tasks.workunit.client.0.smithi049.stdout:0/640: readlink dc/d13/d14/d33/l54 0 2024-05-15T15:54:19.345 INFO:tasks.workunit.client.0.smithi049.stdout:7/589: rename d0/d6/d23/d45/d6a/da1/d54/la4 to d0/d2f/lb5 0 2024-05-15T15:54:19.346 INFO:tasks.workunit.client.1.smithi144.stdout:9/725: dwrite d0/d8/d5e/d44/f41 [0,4194304] 0 2024-05-15T15:54:19.350 INFO:tasks.workunit.client.1.smithi144.stdout:4/698: symlink d6/dd/d34/d3b/d30/db/d1d/d40/d57/ldf 0 2024-05-15T15:54:19.355 INFO:tasks.workunit.client.0.smithi049.stdout:7/590: dwrite d0/d6/d23/d4a/f87 [0,4194304] 0 2024-05-15T15:54:19.355 INFO:tasks.workunit.client.0.smithi049.stdout:4/610: dwrite d18/d21/f7a [0,4194304] 0 2024-05-15T15:54:19.355 INFO:tasks.workunit.client.1.smithi144.stdout:2/651: rmdir d3/d7/d20/d89/d6e 39 2024-05-15T15:54:19.356 INFO:tasks.workunit.client.0.smithi049.stdout:7/591: readlink d0/d2f/la9 0 2024-05-15T15:54:19.359 INFO:tasks.workunit.client.0.smithi049.stdout:1/797: truncate d0/d4/d63/f83 2555232 0 2024-05-15T15:54:19.361 INFO:tasks.workunit.client.0.smithi049.stdout:0/641: mkdir dc/d13/d56/d6f/dce 0 2024-05-15T15:54:19.365 INFO:tasks.workunit.client.1.smithi144.stdout:8/753: rename d2/de/d36 to d2/d24/d2d/d30/dd0/df7 0 2024-05-15T15:54:19.372 INFO:tasks.workunit.client.1.smithi144.stdout:8/754: dread d2/d3a/fce [0,4194304] 0 2024-05-15T15:54:19.373 INFO:tasks.workunit.client.1.smithi144.stdout:8/755: dread - d2/d24/d2d/da0/d9e/db8/fe7 zero size 2024-05-15T15:54:19.373 INFO:tasks.workunit.client.1.smithi144.stdout:6/707: truncate d7/db/d11/f43 3141746 0 2024-05-15T15:54:19.374 INFO:tasks.workunit.client.1.smithi144.stdout:6/708: chown d7/db/da2/fb2 128174 1 2024-05-15T15:54:19.378 INFO:tasks.workunit.client.1.smithi144.stdout:5/723: mknod d0/dc/d24/cf1 0 2024-05-15T15:54:19.379 INFO:tasks.workunit.client.1.smithi144.stdout:5/724: read - d0/dc/d2f/d34/d63/d6e/dbf/f4c zero size 2024-05-15T15:54:19.380 INFO:tasks.workunit.client.1.smithi144.stdout:5/725: fdatasync d0/dc/d24/d28/d2d/d59/fca 0 2024-05-15T15:54:19.380 INFO:tasks.workunit.client.0.smithi049.stdout:7/592: creat d0/d6/d23/d45/d6a/da1/d54/d5e/fb6 x:0 0 0 2024-05-15T15:54:19.382 INFO:tasks.workunit.client.0.smithi049.stdout:1/798: creat d0/d4/d63/f11a x:0 0 0 2024-05-15T15:54:19.383 INFO:tasks.workunit.client.1.smithi144.stdout:2/652: creat d3/d7/d20/d22/d52/d64/d97/db8/fdb x:0 0 0 2024-05-15T15:54:19.385 INFO:tasks.workunit.client.0.smithi049.stdout:9/499: truncate d0/d8/d1b/f32 360309 0 2024-05-15T15:54:19.388 INFO:tasks.workunit.client.1.smithi144.stdout:7/695: rename d4/df/d25/d38/f3e to d4/d3c/d76/d9d/dac/d48/d72/fec 0 2024-05-15T15:54:19.389 INFO:tasks.workunit.client.1.smithi144.stdout:7/696: readlink d4/df/d25/d40/d62/ldf 0 2024-05-15T15:54:19.389 INFO:tasks.workunit.client.1.smithi144.stdout:0/670: truncate d4/df/f16 1700008 0 2024-05-15T15:54:19.392 INFO:tasks.workunit.client.1.smithi144.stdout:8/756: mkdir d2/d24/d2d/da0/df8 0 2024-05-15T15:54:19.393 INFO:tasks.workunit.client.0.smithi049.stdout:8/606: creat d2/de/d71/fba x:0 0 0 2024-05-15T15:54:19.394 INFO:tasks.workunit.client.0.smithi049.stdout:8/607: stat d2/de/c19 0 2024-05-15T15:54:19.396 INFO:tasks.workunit.client.1.smithi144.stdout:9/726: mknod d0/d8/cf5 0 2024-05-15T15:54:19.397 INFO:tasks.workunit.client.0.smithi049.stdout:2/606: link d3/da/d15/d24/c7c d3/db/d5b/ccf 0 2024-05-15T15:54:19.399 INFO:tasks.workunit.client.1.smithi144.stdout:4/699: mkdir d6/dd/d2d/dba/de0 0 2024-05-15T15:54:19.400 INFO:tasks.workunit.client.0.smithi049.stdout:4/611: dwrite d18/d1e/d2a/d36/f4a [4194304,4194304] 0 2024-05-15T15:54:19.402 INFO:tasks.workunit.client.0.smithi049.stdout:7/593: unlink d0/d6/d23/d45/d6a/da1/d39/l8c 0 2024-05-15T15:54:19.403 INFO:tasks.workunit.client.1.smithi144.stdout:5/726: creat d0/dc/d24/d28/d2d/d59/d74/ff2 x:0 0 0 2024-05-15T15:54:19.407 INFO:tasks.workunit.client.0.smithi049.stdout:9/500: truncate d0/d8/d9/d22/f96 81866 0 2024-05-15T15:54:19.408 INFO:tasks.workunit.client.0.smithi049.stdout:3/629: dwrite d12/d26/d73/f7f [0,4194304] 0 2024-05-15T15:54:19.499 INFO:tasks.workunit.client.1.smithi144.stdout:2/653: rmdir d3/d7 39 2024-05-15T15:54:19.501 INFO:tasks.workunit.client.1.smithi144.stdout:1/686: rename d7/d1d/d33/d29/dd2/l61 to d7/d1d/ldd 0 2024-05-15T15:54:19.506 INFO:tasks.workunit.client.0.smithi049.stdout:0/642: rename dc/d13/f22 to dc/d13/d3d/d84/fcf 0 2024-05-15T15:54:19.507 INFO:tasks.workunit.client.1.smithi144.stdout:7/697: rmdir d4/df/d25/d38/d68 39 2024-05-15T15:54:19.507 INFO:tasks.workunit.client.1.smithi144.stdout:7/698: chown d4/df/d25/d38/c39 2 1 2024-05-15T15:54:19.508 INFO:tasks.workunit.client.0.smithi049.stdout:5/667: getdents dc/d4a/d9b 0 2024-05-15T15:54:19.513 INFO:tasks.workunit.client.1.smithi144.stdout:8/757: creat d2/d3a/d7b/d4b/ff9 x:0 0 0 2024-05-15T15:54:19.514 INFO:tasks.workunit.client.1.smithi144.stdout:6/709: mkdir d7/db/d11/d3c/dfc 0 2024-05-15T15:54:19.517 INFO:tasks.workunit.client.1.smithi144.stdout:9/727: creat d0/d8/d5e/d44/d99/ff6 x:0 0 0 2024-05-15T15:54:19.518 INFO:tasks.workunit.client.1.smithi144.stdout:8/758: dwrite d2/d1a/f31 [4194304,4194304] 0 2024-05-15T15:54:19.519 INFO:tasks.workunit.client.1.smithi144.stdout:4/700: write d6/dd/d34/d3b/fd9 [12643,59681] 0 2024-05-15T15:54:19.523 INFO:tasks.workunit.client.0.smithi049.stdout:9/501: mknod d0/d8/d9/d52/d5e/d5f/d56/ca9 0 2024-05-15T15:54:19.525 INFO:tasks.workunit.client.1.smithi144.stdout:3/697: rename d0/d11/d24/c65 to d0/d11/d3b/ce4 0 2024-05-15T15:54:19.529 INFO:tasks.workunit.client.1.smithi144.stdout:7/699: mkdir d4/d3c/d76/d9d/dac/d48/ded 0 2024-05-15T15:54:19.529 INFO:tasks.workunit.client.1.smithi144.stdout:0/671: dwrite d4/f1c [0,4194304] 0 2024-05-15T15:54:19.534 INFO:tasks.workunit.client.0.smithi049.stdout:7/594: link d0/d34/f3f d0/d6/d1d/fb7 0 2024-05-15T15:54:19.535 INFO:tasks.workunit.client.1.smithi144.stdout:0/672: dread d4/d6/d4a/d74/f99 [0,4194304] 0 2024-05-15T15:54:19.536 INFO:tasks.workunit.client.0.smithi049.stdout:7/595: read d0/d2f/f51 [4555152,124876] 0 2024-05-15T15:54:19.536 INFO:tasks.workunit.client.1.smithi144.stdout:0/673: read d4/d2d/f8d [3876860,43418] 0 2024-05-15T15:54:19.540 INFO:tasks.workunit.client.1.smithi144.stdout:8/759: truncate d2/d24/d2d/da0/d6e/d8d/fbb 253455 0 2024-05-15T15:54:19.541 INFO:tasks.workunit.client.1.smithi144.stdout:2/654: symlink d3/d7/da4/ldc 0 2024-05-15T15:54:19.542 INFO:tasks.workunit.client.0.smithi049.stdout:5/668: truncate dc/dd/d47/da6/fd9 964006 0 2024-05-15T15:54:19.543 INFO:tasks.workunit.client.1.smithi144.stdout:1/687: mknod d7/d1d/d33/d29/cde 0 2024-05-15T15:54:19.544 INFO:tasks.workunit.client.0.smithi049.stdout:2/607: link d3/fa6 d3/db/d1d/d1e/d2b/d6b/fd0 0 2024-05-15T15:54:19.544 INFO:tasks.workunit.client.0.smithi049.stdout:5/669: truncate dc/dd/d47/d83/dcc/da9/dba/dbe/fc1 901601 0 2024-05-15T15:54:19.546 INFO:tasks.workunit.client.1.smithi144.stdout:7/700: symlink d4/d41/d5b/da0/lee 0 2024-05-15T15:54:19.553 INFO:tasks.workunit.client.0.smithi049.stdout:9/502: creat d0/d8/d53/faa x:0 0 0 2024-05-15T15:54:19.554 INFO:tasks.workunit.client.1.smithi144.stdout:9/728: mknod d0/d8/d5e/d44/d10/cf7 0 2024-05-15T15:54:19.556 INFO:tasks.workunit.client.0.smithi049.stdout:7/596: dwrite d0/d6/f1b [0,4194304] 0 2024-05-15T15:54:19.557 INFO:tasks.workunit.client.0.smithi049.stdout:1/799: rename d0/d4/dd/de/d1e/d2e/d3e/d90/db3/lf0 to d0/d4/l11b 0 2024-05-15T15:54:19.558 INFO:tasks.workunit.client.1.smithi144.stdout:6/710: dwrite d7/db/f3f [4194304,4194304] 0 2024-05-15T15:54:19.560 INFO:tasks.workunit.client.1.smithi144.stdout:6/711: write d7/d62/fd9 [1619997,17708] 0 2024-05-15T15:54:19.560 INFO:tasks.workunit.client.1.smithi144.stdout:6/712: readlink d7/d62/l76 0 2024-05-15T15:54:19.560 INFO:tasks.workunit.client.0.smithi049.stdout:8/608: getdents d2/de/d71/d86/d5d/d7d/d45 0 2024-05-15T15:54:19.563 INFO:tasks.workunit.client.1.smithi144.stdout:5/727: creat d0/dc/ff3 x:0 0 0 2024-05-15T15:54:19.564 INFO:tasks.workunit.client.0.smithi049.stdout:3/630: write d12/d30/d63/d68/d70/d75/f85 [1744456,14162] 0 2024-05-15T15:54:19.565 INFO:tasks.workunit.client.0.smithi049.stdout:5/670: creat dc/d29/fda x:0 0 0 2024-05-15T15:54:19.566 INFO:tasks.workunit.client.0.smithi049.stdout:0/643: truncate dc/da4/fab 3564536 0 2024-05-15T15:54:19.566 INFO:tasks.workunit.client.0.smithi049.stdout:3/631: readlink d12/d14/d52/db9/d7a/lb6 0 2024-05-15T15:54:19.569 INFO:tasks.workunit.client.0.smithi049.stdout:1/800: dwrite d0/d4/dd/de/d36/f101 [0,4194304] 0 2024-05-15T15:54:19.569 INFO:tasks.workunit.client.1.smithi144.stdout:8/760: rmdir d2/d3a/d7b/d4b 39 2024-05-15T15:54:19.570 INFO:tasks.workunit.client.1.smithi144.stdout:4/701: dwrite d6/dd/f22 [4194304,4194304] 0 2024-05-15T15:54:19.573 INFO:tasks.workunit.client.1.smithi144.stdout:2/655: creat d3/d7/d20/d37/d76/fdd x:0 0 0 2024-05-15T15:54:19.574 INFO:tasks.workunit.client.1.smithi144.stdout:2/656: chown d3/d7/d54/l8a 7 1 2024-05-15T15:54:19.576 INFO:tasks.workunit.client.1.smithi144.stdout:3/698: symlink d0/d1/le5 0 2024-05-15T15:54:19.577 INFO:tasks.workunit.client.0.smithi049.stdout:7/597: rename d0/d6/d1d/l67 to d0/d6/d1d/lb8 0 2024-05-15T15:54:19.586 INFO:tasks.workunit.client.0.smithi049.stdout:2/608: fdatasync d3/db/d1d/d1e/f21 0 2024-05-15T15:54:19.588 INFO:tasks.workunit.client.1.smithi144.stdout:9/729: mkdir d0/d8/d5e/d9b/df8 0 2024-05-15T15:54:19.590 INFO:tasks.workunit.client.0.smithi049.stdout:0/644: mkdir dc/d13/d14/d25/db9/dd0 0 2024-05-15T15:54:19.592 INFO:tasks.workunit.client.0.smithi049.stdout:5/671: truncate dc/dd/d47/f4d 643008 0 2024-05-15T15:54:19.597 INFO:tasks.workunit.client.0.smithi049.stdout:1/801: mknod d0/d4/dd/de/d4e/c11c 0 2024-05-15T15:54:19.598 INFO:tasks.workunit.client.0.smithi049.stdout:0/645: dwrite dc/d13/d2b/d76/d8a/da2/fc4 [0,4194304] 0 2024-05-15T15:54:19.599 INFO:tasks.workunit.client.1.smithi144.stdout:6/713: symlink d7/db/d35/d84/d81/lfd 0 2024-05-15T15:54:19.600 INFO:tasks.workunit.client.1.smithi144.stdout:6/714: chown d7/d62/lcd 12 1 2024-05-15T15:54:19.602 INFO:tasks.workunit.client.0.smithi049.stdout:9/503: mknod d0/d8/cab 0 2024-05-15T15:54:19.602 INFO:tasks.workunit.client.0.smithi049.stdout:1/802: dread d0/d4/dd/de/d4e/f76 [4194304,4194304] 0 2024-05-15T15:54:19.605 INFO:tasks.workunit.client.1.smithi144.stdout:5/728: rmdir d0/dc/d2f/d68/d5c 39 2024-05-15T15:54:19.608 INFO:tasks.workunit.client.0.smithi049.stdout:8/609: write d2/de/d71/d86/d5d/d7d/d1d/f61 [3427372,36318] 0 2024-05-15T15:54:19.609 INFO:tasks.workunit.client.1.smithi144.stdout:7/701: dwrite d4/df/d25/d40/d73/f98 [0,4194304] 0 2024-05-15T15:54:19.610 INFO:tasks.workunit.client.0.smithi049.stdout:8/610: fdatasync d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fb4 0 2024-05-15T15:54:19.611 INFO:tasks.workunit.client.0.smithi049.stdout:8/611: readlink d2/de/d71/d86/d5d/d7d/d1b/l49 0 2024-05-15T15:54:19.613 INFO:tasks.workunit.client.1.smithi144.stdout:4/702: rmdir d6/dd/d2e/d75 39 2024-05-15T15:54:19.614 INFO:tasks.workunit.client.0.smithi049.stdout:5/672: creat dc/d4a/d9b/fdb x:0 0 0 2024-05-15T15:54:19.616 INFO:tasks.workunit.client.1.smithi144.stdout:3/699: rmdir d0/d1/d56/d71/d73 39 2024-05-15T15:54:19.617 INFO:tasks.workunit.client.1.smithi144.stdout:7/702: dwrite d4/d3c/f57 [0,4194304] 0 2024-05-15T15:54:19.618 INFO:tasks.workunit.client.0.smithi049.stdout:3/632: creat d12/d14/d52/db9/da6/dc2/fc3 x:0 0 0 2024-05-15T15:54:19.621 INFO:tasks.workunit.client.1.smithi144.stdout:1/688: symlink d7/ldf 0 2024-05-15T15:54:19.626 INFO:tasks.workunit.client.1.smithi144.stdout:0/674: link d4/d6/d3a/d4c/c68 d4/d6/d4a/cee 0 2024-05-15T15:54:19.629 INFO:tasks.workunit.client.0.smithi049.stdout:7/598: dwrite d0/f2e [4194304,4194304] 0 2024-05-15T15:54:19.631 INFO:tasks.workunit.client.0.smithi049.stdout:4/612: rename d18/d69/fc0 to d18/d1e/d2a/d36/fc6 0 2024-05-15T15:54:19.634 INFO:tasks.workunit.client.1.smithi144.stdout:8/761: symlink d2/de/d3f/lfa 0 2024-05-15T15:54:19.636 INFO:tasks.workunit.client.0.smithi049.stdout:8/612: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d60/fbb x:0 0 0 2024-05-15T15:54:19.637 INFO:tasks.workunit.client.1.smithi144.stdout:4/703: fsync d6/dd/d2d/f54 0 2024-05-15T15:54:19.639 INFO:tasks.workunit.client.1.smithi144.stdout:4/704: write d6/dd/d34/d3b/d30/db/d1d/d40/f79 [917752,36703] 0 2024-05-15T15:54:19.639 INFO:tasks.workunit.client.1.smithi144.stdout:3/700: truncate d0/d1/f43 1712859 0 2024-05-15T15:54:19.640 INFO:tasks.workunit.client.1.smithi144.stdout:3/701: readlink d0/d1/d17/l37 0 2024-05-15T15:54:19.642 INFO:tasks.workunit.client.1.smithi144.stdout:7/703: creat d4/d3c/d76/d9d/dac/d48/fef x:0 0 0 2024-05-15T15:54:19.643 INFO:tasks.workunit.client.1.smithi144.stdout:7/704: write d4/d3c/d76/d9d/dac/de5/fea [237721,71222] 0 2024-05-15T15:54:19.643 INFO:tasks.workunit.client.0.smithi049.stdout:9/504: mkdir d0/d8/d9/d52/d5e/d8a/dac 0 2024-05-15T15:54:19.644 INFO:tasks.workunit.client.0.smithi049.stdout:9/505: fdatasync d0/d8/d9/d52/d5e/d8a/f99 0 2024-05-15T15:54:19.645 INFO:tasks.workunit.client.0.smithi049.stdout:1/803: symlink d0/d4/dd/de/d1e/d2e/d3e/l11d 0 2024-05-15T15:54:19.647 INFO:tasks.workunit.client.1.smithi144.stdout:2/657: rename d3/d7/d20/d37/d76/fdd to d3/d7/d20/d3f/fde 0 2024-05-15T15:54:19.650 INFO:tasks.workunit.client.1.smithi144.stdout:9/730: mkdir d0/d8/d1b/d79/de7/df9 0 2024-05-15T15:54:19.653 INFO:tasks.workunit.client.0.smithi049.stdout:5/673: rename dc/dd/d47/d83/dcc/ccb to dc/db0/dbf/cdc 0 2024-05-15T15:54:19.654 INFO:tasks.workunit.client.1.smithi144.stdout:6/715: creat d7/db/da2/d70/db6/ffe x:0 0 0 2024-05-15T15:54:19.657 INFO:tasks.workunit.client.0.smithi049.stdout:8/613: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fbc x:0 0 0 2024-05-15T15:54:19.658 INFO:tasks.workunit.client.1.smithi144.stdout:8/762: rmdir d2 39 2024-05-15T15:54:19.664 INFO:tasks.workunit.client.0.smithi049.stdout:4/613: write d18/f28 [8514013,7191] 0 2024-05-15T15:54:19.664 INFO:tasks.workunit.client.0.smithi049.stdout:3/633: write d12/f25 [2103616,49649] 0 2024-05-15T15:54:19.667 INFO:tasks.workunit.client.1.smithi144.stdout:3/702: rename d0/d1/d17/d39/d40/l42 to d0/d1/d17/d39/le6 0 2024-05-15T15:54:19.667 INFO:tasks.workunit.client.1.smithi144.stdout:5/729: dwrite d0/dc/d24/d57/f77 [0,4194304] 0 2024-05-15T15:54:19.668 INFO:tasks.workunit.client.0.smithi049.stdout:5/674: stat dc/d29/f32 0 2024-05-15T15:54:19.668 INFO:tasks.workunit.client.1.smithi144.stdout:2/658: readlink d3/d7/d20/d22/d52/d75/lcd 0 2024-05-15T15:54:19.669 INFO:tasks.workunit.client.0.smithi049.stdout:8/614: creat d2/de/d25/fbd x:0 0 0 2024-05-15T15:54:19.669 INFO:tasks.workunit.client.0.smithi049.stdout:8/615: stat d2/de 0 2024-05-15T15:54:19.671 INFO:tasks.workunit.client.0.smithi049.stdout:7/599: creat d0/fb9 x:0 0 0 2024-05-15T15:54:19.672 INFO:tasks.workunit.client.1.smithi144.stdout:0/675: chown d4/df/d1f/d56/l6f 5 1 2024-05-15T15:54:19.673 INFO:tasks.workunit.client.1.smithi144.stdout:6/716: creat d7/d32/fff x:0 0 0 2024-05-15T15:54:19.674 INFO:tasks.workunit.client.1.smithi144.stdout:6/717: write d7/d62/f6b [3814620,33379] 0 2024-05-15T15:54:19.675 INFO:tasks.workunit.client.0.smithi049.stdout:3/634: fsync d12/d30/f3d 0 2024-05-15T15:54:19.675 INFO:tasks.workunit.client.1.smithi144.stdout:6/718: chown d7/db/d5f/f7a 0 1 2024-05-15T15:54:19.676 INFO:tasks.workunit.client.0.smithi049.stdout:3/635: read - d12/d26/f3b zero size 2024-05-15T15:54:19.679 INFO:tasks.workunit.client.1.smithi144.stdout:7/705: mkdir d4/d3c/d76/d9d/dac/d48/ded/df0 0 2024-05-15T15:54:19.682 INFO:tasks.workunit.client.1.smithi144.stdout:5/730: symlink d0/d7e/lf4 0 2024-05-15T15:54:19.682 INFO:tasks.workunit.client.1.smithi144.stdout:8/763: dwrite d2/d24/d2d/d30/dd0/df7/faf [0,4194304] 0 2024-05-15T15:54:19.684 INFO:tasks.workunit.client.0.smithi049.stdout:5/675: mkdir dc/d4a/ddd 0 2024-05-15T15:54:19.685 INFO:tasks.workunit.client.0.smithi049.stdout:5/676: dread - dc/d29/fda zero size 2024-05-15T15:54:19.686 INFO:tasks.workunit.client.1.smithi144.stdout:3/703: creat d0/d1/d17/d39/d40/fe7 x:0 0 0 2024-05-15T15:54:19.686 INFO:tasks.workunit.client.1.smithi144.stdout:3/704: chown d0/d11 99 1 2024-05-15T15:54:19.687 INFO:tasks.workunit.client.1.smithi144.stdout:3/705: fdatasync d0/d1/d56/f7f 0 2024-05-15T15:54:19.688 INFO:tasks.workunit.client.0.smithi049.stdout:5/677: mkdir dc/d29/dde 0 2024-05-15T15:54:19.690 INFO:tasks.workunit.client.1.smithi144.stdout:1/689: unlink d7/d1d/d33/d1e/d87/caf 0 2024-05-15T15:54:19.691 INFO:tasks.workunit.client.0.smithi049.stdout:9/506: truncate d0/d8/d42/f91 1154653 0 2024-05-15T15:54:19.692 INFO:tasks.workunit.client.0.smithi049.stdout:4/614: dwrite d18/d85/fa3 [0,4194304] 0 2024-05-15T15:54:19.693 INFO:tasks.workunit.client.0.smithi049.stdout:7/600: creat d0/d6/d23/d45/fba x:0 0 0 2024-05-15T15:54:19.695 INFO:tasks.workunit.client.0.smithi049.stdout:7/601: dread - d0/d6/d23/d4a/f75 zero size 2024-05-15T15:54:19.695 INFO:tasks.workunit.client.1.smithi144.stdout:9/731: link d0/f18 d0/d8/d5e/d9b/ffa 0 2024-05-15T15:54:19.696 INFO:tasks.workunit.client.1.smithi144.stdout:9/732: readlink d0/d8/d50/l6e 0 2024-05-15T15:54:19.698 INFO:tasks.workunit.client.1.smithi144.stdout:0/676: mkdir d4/d2d/d48/d5b/dda/def 0 2024-05-15T15:54:19.700 INFO:tasks.workunit.client.1.smithi144.stdout:6/719: symlink d7/db/d35/d84/d81/l100 0 2024-05-15T15:54:19.704 INFO:tasks.workunit.client.1.smithi144.stdout:4/705: link d6/dd/d34/d3b/d30/db/d1d/d40/d6c/fde d6/dd/d34/d3b/d30/d59/fe1 0 2024-05-15T15:54:19.705 INFO:tasks.workunit.client.0.smithi049.stdout:7/602: unlink d0/d34/l79 0 2024-05-15T15:54:19.708 INFO:tasks.workunit.client.0.smithi049.stdout:9/507: symlink d0/d8/d1b/lad 0 2024-05-15T15:54:19.708 INFO:tasks.workunit.client.1.smithi144.stdout:4/706: dread d6/dd/d34/f9d [0,4194304] 0 2024-05-15T15:54:19.710 INFO:tasks.workunit.client.1.smithi144.stdout:7/706: symlink d4/df/d79/lf1 0 2024-05-15T15:54:19.712 INFO:tasks.workunit.client.0.smithi049.stdout:4/615: rename d18/faa to d18/d1e/d63/d99/db9/fc7 0 2024-05-15T15:54:19.714 INFO:tasks.workunit.client.0.smithi049.stdout:7/603: stat d0/d6/d23/d45/d6a/f9d 0 2024-05-15T15:54:19.722 INFO:tasks.workunit.client.0.smithi049.stdout:7/604: creat d0/d6/d23/d45/d6a/da1/d80/fbb x:0 0 0 2024-05-15T15:54:19.724 INFO:tasks.workunit.client.1.smithi144.stdout:1/690: rmdir d7/d1d/d33/d41 39 2024-05-15T15:54:19.727 INFO:tasks.workunit.client.0.smithi049.stdout:5/678: rename dc/dd/d47/d83/dcc/d5c/f7d to dc/dd/d7b/fdf 0 2024-05-15T15:54:19.796 INFO:tasks.workunit.client.0.smithi049.stdout:7/605: rmdir d0/d2f/d5c 39 2024-05-15T15:54:19.798 INFO:tasks.workunit.client.0.smithi049.stdout:9/508: link d0/d25/d5d/f8d d0/d8/d21/fae 0 2024-05-15T15:54:19.798 INFO:tasks.workunit.client.1.smithi144.stdout:9/733: write d0/d8/d50/f72 [73208,74000] 0 2024-05-15T15:54:19.847 INFO:tasks.workunit.client.1.smithi144.stdout:7/707: sync 2024-05-15T15:54:19.874 INFO:tasks.workunit.client.0.smithi049.stdout:6/616: sync 2024-05-15T15:54:19.878 INFO:tasks.workunit.client.0.smithi049.stdout:5/679: symlink dc/d4a/d9e/le0 0 2024-05-15T15:54:19.879 INFO:tasks.workunit.client.1.smithi144.stdout:5/731: creat d0/dc/d2f/d34/dc1/ff5 x:0 0 0 2024-05-15T15:54:19.880 INFO:tasks.workunit.client.0.smithi049.stdout:7/606: rmdir d0/d34 39 2024-05-15T15:54:19.885 INFO:tasks.workunit.client.1.smithi144.stdout:2/659: rename d3/d7/d20/d22/d52/d75/f55 to d3/d7/d20/d89/fdf 0 2024-05-15T15:54:19.885 INFO:tasks.workunit.client.1.smithi144.stdout:5/732: dread d0/dc/d24/d28/d2d/f3d [0,4194304] 0 2024-05-15T15:54:19.886 INFO:tasks.workunit.client.0.smithi049.stdout:6/617: unlink dd/d17/c68 0 2024-05-15T15:54:19.891 INFO:tasks.workunit.client.0.smithi049.stdout:5/680: rename dc/dd/d10/f6f to dc/db0/dd5/fe1 0 2024-05-15T15:54:19.893 INFO:tasks.workunit.client.0.smithi049.stdout:5/681: write dc/dd/f14 [2032194,101470] 0 2024-05-15T15:54:19.894 INFO:tasks.workunit.client.0.smithi049.stdout:9/509: dread - d0/d8/d12/f59 zero size 2024-05-15T15:54:19.895 INFO:tasks.workunit.client.1.smithi144.stdout:0/677: rmdir d4/df/d1f/d24/d85/da1/da3 39 2024-05-15T15:54:19.899 INFO:tasks.workunit.client.1.smithi144.stdout:6/720: truncate d7/db/f2c 1413991 0 2024-05-15T15:54:19.902 INFO:tasks.workunit.client.0.smithi049.stdout:6/618: rename dd/ff to dd/d17/d64/d8c/fba 0 2024-05-15T15:54:19.904 INFO:tasks.workunit.client.1.smithi144.stdout:3/706: write d0/d1/fd9 [1496905,78961] 0 2024-05-15T15:54:19.904 INFO:tasks.workunit.client.0.smithi049.stdout:7/607: dread d0/d6/d23/d45/d6a/da1/f33 [0,4194304] 0 2024-05-15T15:54:19.907 INFO:tasks.workunit.client.0.smithi049.stdout:7/608: truncate d0/d6/d23/d45/d6a/da1/d39/fa8 1022993 0 2024-05-15T15:54:19.908 INFO:tasks.workunit.client.1.smithi144.stdout:9/734: mkdir d0/d8/d1b/d62/d7d/d8d/dbb/dce/dfb 0 2024-05-15T15:54:19.916 INFO:tasks.workunit.client.0.smithi049.stdout:9/510: dwrite d0/d8/d53/f68 [0,4194304] 0 2024-05-15T15:54:19.918 INFO:tasks.workunit.client.0.smithi049.stdout:6/619: dread dd/d17/d24/d2b/d36/f42 [0,4194304] 0 2024-05-15T15:54:19.918 INFO:tasks.workunit.client.1.smithi144.stdout:9/735: dread d0/f7 [0,4194304] 0 2024-05-15T15:54:19.924 INFO:tasks.workunit.client.1.smithi144.stdout:4/707: dwrite d6/f95 [0,4194304] 0 2024-05-15T15:54:19.925 INFO:tasks.workunit.client.0.smithi049.stdout:6/620: fsync dd/d17/d64/f6a 0 2024-05-15T15:54:19.926 INFO:tasks.workunit.client.1.smithi144.stdout:8/764: truncate d2/d3a/d7b/d4b/f61 3525859 0 2024-05-15T15:54:19.927 INFO:tasks.workunit.client.0.smithi049.stdout:6/621: read dd/f19 [4343129,32323] 0 2024-05-15T15:54:19.927 INFO:tasks.workunit.client.0.smithi049.stdout:5/682: dwrite dc/dd/d10/f71 [0,4194304] 0 2024-05-15T15:54:19.929 INFO:tasks.workunit.client.0.smithi049.stdout:7/609: creat d0/d34/fbc x:0 0 0 2024-05-15T15:54:19.931 INFO:tasks.workunit.client.0.smithi049.stdout:9/511: fdatasync d0/d8/d1b/d40/f49 0 2024-05-15T15:54:19.934 INFO:tasks.workunit.client.1.smithi144.stdout:1/691: creat d7/d1d/d33/d1e/d34/dd8/fe0 x:0 0 0 2024-05-15T15:54:19.936 INFO:tasks.workunit.client.1.smithi144.stdout:0/678: fdatasync d4/d2d/d48/d5b/f75 0 2024-05-15T15:54:19.938 INFO:tasks.workunit.client.1.smithi144.stdout:6/721: symlink d7/da6/l101 0 2024-05-15T15:54:19.940 INFO:tasks.workunit.client.1.smithi144.stdout:3/707: rmdir d0/d11/d24/d78 39 2024-05-15T15:54:19.949 INFO:tasks.workunit.client.1.smithi144.stdout:9/736: dread d0/d8/d5e/fa4 [0,4194304] 0 2024-05-15T15:54:19.950 INFO:tasks.workunit.client.1.smithi144.stdout:4/708: mkdir d6/dd/d34/de2 0 2024-05-15T15:54:19.952 INFO:tasks.workunit.client.1.smithi144.stdout:8/765: fdatasync d2/d24/f89 0 2024-05-15T15:54:19.953 INFO:tasks.workunit.client.1.smithi144.stdout:8/766: chown d2/d24/d2d/d30/c41 4 1 2024-05-15T15:54:19.953 INFO:tasks.workunit.client.1.smithi144.stdout:5/733: dread d0/dc/d12/d4a/f60 [0,4194304] 0 2024-05-15T15:54:19.955 INFO:tasks.workunit.client.1.smithi144.stdout:5/734: readlink d0/dc/d24/d8e/d93/lcd 0 2024-05-15T15:54:19.955 INFO:tasks.workunit.client.1.smithi144.stdout:5/735: readlink d0/dc/d2f/laf 0 2024-05-15T15:54:19.956 INFO:tasks.workunit.client.1.smithi144.stdout:0/679: rename d4/d6/d4a/d74/led to d4/dc/dde/lf0 0 2024-05-15T15:54:19.956 INFO:tasks.workunit.client.1.smithi144.stdout:0/680: chown d4/d6/d4a/cee 6179677 1 2024-05-15T15:54:19.957 INFO:tasks.workunit.client.1.smithi144.stdout:6/722: creat d7/db/d35/d84/d81/f102 x:0 0 0 2024-05-15T15:54:19.962 INFO:tasks.workunit.client.1.smithi144.stdout:7/708: link d4/d41/d5b/f70 d4/d3c/d76/d9d/dac/d48/d72/d82/ff2 0 2024-05-15T15:54:19.962 INFO:tasks.workunit.client.1.smithi144.stdout:9/737: dread d0/d8/d5e/d44/f26 [0,4194304] 0 2024-05-15T15:54:19.967 INFO:tasks.workunit.client.1.smithi144.stdout:2/660: link d3/d7/d20/d37/f79 d3/d4d/db4/fe0 0 2024-05-15T15:54:19.968 INFO:tasks.workunit.client.1.smithi144.stdout:2/661: readlink d3/d7/d20/d37/d76/l8b 0 2024-05-15T15:54:19.971 INFO:tasks.workunit.client.1.smithi144.stdout:4/709: creat d6/d42/d46/fe3 x:0 0 0 2024-05-15T15:54:19.974 INFO:tasks.workunit.client.1.smithi144.stdout:1/692: truncate d7/df/f10 4016430 0 2024-05-15T15:54:19.977 INFO:tasks.workunit.client.1.smithi144.stdout:5/736: rename d0/dc/d2f/d79/fb9 to d0/dc/d24/d8e/ff6 0 2024-05-15T15:54:19.979 INFO:tasks.workunit.client.0.smithi049.stdout:5/683: rename dc/dd/d47/d83/dcc/f82 to dc/dd/d47/da6/dce/fe2 0 2024-05-15T15:54:19.980 INFO:tasks.workunit.client.1.smithi144.stdout:0/681: creat d4/d6/d3a/ff1 x:0 0 0 2024-05-15T15:54:19.982 INFO:tasks.workunit.client.0.smithi049.stdout:9/512: creat d0/d25/faf x:0 0 0 2024-05-15T15:54:19.984 INFO:tasks.workunit.client.0.smithi049.stdout:7/610: mkdir d0/d6/d23/d45/d6a/da1/d54/d5e/dbd 0 2024-05-15T15:54:19.986 INFO:tasks.workunit.client.1.smithi144.stdout:7/709: mknod d4/d41/d9c/cf3 0 2024-05-15T15:54:19.988 INFO:tasks.workunit.client.1.smithi144.stdout:9/738: mkdir d0/d8/d1b/d79/d8a/dae/ddb/dfc 0 2024-05-15T15:54:19.991 INFO:tasks.workunit.client.0.smithi049.stdout:5/684: rename dc/dd/d47/d83/dcc/l37 to dc/d29/dde/le3 0 2024-05-15T15:54:19.994 INFO:tasks.workunit.client.0.smithi049.stdout:9/513: creat d0/d8/d12/fb0 x:0 0 0 2024-05-15T15:54:19.995 INFO:tasks.workunit.client.0.smithi049.stdout:9/514: read d0/f41 [313082,102510] 0 2024-05-15T15:54:19.997 INFO:tasks.workunit.client.0.smithi049.stdout:2/609: sync 2024-05-15T15:54:20.002 INFO:tasks.workunit.client.1.smithi144.stdout:5/737: mkdir d0/dc/d24/d28/d2d/df7 0 2024-05-15T15:54:20.006 INFO:tasks.workunit.client.0.smithi049.stdout:6/622: dwrite dd/f16 [0,4194304] 0 2024-05-15T15:54:20.007 INFO:tasks.workunit.client.0.smithi049.stdout:6/623: readlink dd/d17/d24/d2b/d36/d93/lac 0 2024-05-15T15:54:20.007 INFO:tasks.workunit.client.1.smithi144.stdout:6/723: link d7/db/d11/f42 d7/db/d9d/f103 0 2024-05-15T15:54:20.009 INFO:tasks.workunit.client.0.smithi049.stdout:6/624: write dd/d17/d24/d2b/d36/d50/f83 [534998,652] 0 2024-05-15T15:54:20.011 INFO:tasks.workunit.client.0.smithi049.stdout:9/515: creat d0/d8/d9/d22/fb1 x:0 0 0 2024-05-15T15:54:20.013 INFO:tasks.workunit.client.1.smithi144.stdout:6/724: dwrite d7/db/d11/fa5 [0,4194304] 0 2024-05-15T15:54:20.013 INFO:tasks.workunit.client.1.smithi144.stdout:3/708: creat d0/d1/d56/fe8 x:0 0 0 2024-05-15T15:54:20.014 INFO:tasks.workunit.client.1.smithi144.stdout:6/725: chown d7/da6 2071353 1 2024-05-15T15:54:20.016 INFO:tasks.workunit.client.0.smithi049.stdout:2/610: rename d3/db/f30 to d3/db/d1d/d1e/d2b/d3b/d47/d59/fd1 0 2024-05-15T15:54:20.018 INFO:tasks.workunit.client.0.smithi049.stdout:5/685: creat dc/dd/d47/d83/da7/fe4 x:0 0 0 2024-05-15T15:54:20.018 INFO:tasks.workunit.client.0.smithi049.stdout:5/686: chown dc/l24 594 1 2024-05-15T15:54:20.020 INFO:tasks.workunit.client.1.smithi144.stdout:9/739: unlink d0/d8/d5e/d44/d16/db0/le6 0 2024-05-15T15:54:20.021 INFO:tasks.workunit.client.1.smithi144.stdout:6/726: dwrite d7/db/d9d/fea [4194304,4194304] 0 2024-05-15T15:54:20.022 INFO:tasks.workunit.client.1.smithi144.stdout:6/727: read - d7/db/d11/d7e/ff9 zero size 2024-05-15T15:54:20.024 INFO:tasks.workunit.client.1.smithi144.stdout:2/662: write d3/d7/d20/d3f/fde [1027238,41934] 0 2024-05-15T15:54:20.028 INFO:tasks.workunit.client.1.smithi144.stdout:6/728: dwrite d7/fe5 [0,4194304] 0 2024-05-15T15:54:20.028 INFO:tasks.workunit.client.0.smithi049.stdout:6/625: readlink dd/d17/d24/d58/d5f/l7f 0 2024-05-15T15:54:20.029 INFO:tasks.workunit.client.1.smithi144.stdout:6/729: chown d7/db/d3a/f91 106328 1 2024-05-15T15:54:20.029 INFO:tasks.workunit.client.0.smithi049.stdout:6/626: chown c3 21180 1 2024-05-15T15:54:20.030 INFO:tasks.workunit.client.0.smithi049.stdout:9/516: truncate d0/d8/d53/f80 3777831 0 2024-05-15T15:54:20.034 INFO:tasks.workunit.client.1.smithi144.stdout:0/682: mknod d4/df/d1f/d24/d85/da1/da3/cf2 0 2024-05-15T15:54:20.036 INFO:tasks.workunit.client.1.smithi144.stdout:8/767: dwrite d2/de/d3f/d6b/fe2 [0,4194304] 0 2024-05-15T15:54:20.037 INFO:tasks.workunit.client.1.smithi144.stdout:0/683: dread - d4/df/d1f/d24/d85/da1/da3/fd2 zero size 2024-05-15T15:54:20.037 INFO:tasks.workunit.client.0.smithi049.stdout:7/611: link d0/d6/d23/d45/d6a/da1/c8b d0/d6/cbe 0 2024-05-15T15:54:20.038 INFO:tasks.workunit.client.1.smithi144.stdout:8/768: chown d2/d24/d2d/d30/fe0 436821 1 2024-05-15T15:54:20.041 INFO:tasks.workunit.client.0.smithi049.stdout:9/517: truncate d0/d8/d9c/f9f 195722 0 2024-05-15T15:54:20.045 INFO:tasks.workunit.client.0.smithi049.stdout:6/627: mknod dd/d17/d24/d2b/d36/d50/cbb 0 2024-05-15T15:54:20.049 INFO:tasks.workunit.client.0.smithi049.stdout:9/518: creat d0/d8/d21/fb2 x:0 0 0 2024-05-15T15:54:20.052 INFO:tasks.workunit.client.0.smithi049.stdout:5/687: dread dc/dd/d47/d83/dcc/d5c/f7e [0,4194304] 0 2024-05-15T15:54:20.053 INFO:tasks.workunit.client.0.smithi049.stdout:5/688: chown dc/c66 1093363 1 2024-05-15T15:54:20.054 INFO:tasks.workunit.client.0.smithi049.stdout:6/628: read dd/d17/d64/d8c/fba [332857,109367] 0 2024-05-15T15:54:20.058 INFO:tasks.workunit.client.1.smithi144.stdout:6/730: mknod d7/db/d9d/d97/c104 0 2024-05-15T15:54:20.058 INFO:tasks.workunit.client.0.smithi049.stdout:9/519: mkdir d0/d8/d1b/db3 0 2024-05-15T15:54:20.058 INFO:tasks.workunit.client.0.smithi049.stdout:7/612: creat d0/d6/d23/d45/d6a/da1/fbf x:0 0 0 2024-05-15T15:54:20.059 INFO:tasks.workunit.client.0.smithi049.stdout:7/613: fsync d0/d6/d23/d45/d6a/f9d 0 2024-05-15T15:54:20.064 INFO:tasks.workunit.client.0.smithi049.stdout:5/689: dread - dc/d4a/f8b zero size 2024-05-15T15:54:20.067 INFO:tasks.workunit.client.1.smithi144.stdout:3/709: creat d0/d11/d3b/db5/fe9 x:0 0 0 2024-05-15T15:54:20.068 INFO:tasks.workunit.client.0.smithi049.stdout:9/520: rmdir d0/d8/d12 39 2024-05-15T15:54:20.070 INFO:tasks.workunit.client.1.smithi144.stdout:9/740: dwrite d0/d8/d1b/fb9 [0,4194304] 0 2024-05-15T15:54:20.071 INFO:tasks.workunit.client.1.smithi144.stdout:9/741: chown d0/d8/d5e/d44/d99/lcc 414175 1 2024-05-15T15:54:20.075 INFO:tasks.workunit.client.1.smithi144.stdout:7/710: rename d4/fa to d4/df/ff4 0 2024-05-15T15:54:20.076 INFO:tasks.workunit.client.1.smithi144.stdout:7/711: chown d4/d3c/d76/d9d/dac/d48/ldd 3 1 2024-05-15T15:54:20.078 INFO:tasks.workunit.client.1.smithi144.stdout:2/663: fsync d3/d7/d20/d89/d6e/f8f 0 2024-05-15T15:54:20.081 INFO:tasks.workunit.client.0.smithi049.stdout:6/629: mkdir dd/dbc 0 2024-05-15T15:54:20.082 INFO:tasks.workunit.client.1.smithi144.stdout:0/684: dwrite d4/df/d1f/d24/d96/fca [0,4194304] 0 2024-05-15T15:54:20.082 INFO:tasks.workunit.client.0.smithi049.stdout:6/630: chown dd/d26/f3f 17 1 2024-05-15T15:54:20.083 INFO:tasks.workunit.client.1.smithi144.stdout:1/693: getdents d7/d1d/d58/d81 0 2024-05-15T15:54:20.084 INFO:tasks.workunit.client.0.smithi049.stdout:9/521: write d0/d8/d9/f7f [993683,80294] 0 2024-05-15T15:54:20.086 INFO:tasks.workunit.client.1.smithi144.stdout:6/731: fsync d7/f8 0 2024-05-15T15:54:20.086 INFO:tasks.workunit.client.0.smithi049.stdout:1/804: sync 2024-05-15T15:54:20.086 INFO:tasks.workunit.client.0.smithi049.stdout:0/646: sync 2024-05-15T15:54:20.088 INFO:tasks.workunit.client.0.smithi049.stdout:0/647: chown dc/d13/d56/d6f/dce 53 1 2024-05-15T15:54:20.088 INFO:tasks.workunit.client.1.smithi144.stdout:5/738: creat d0/ff8 x:0 0 0 2024-05-15T15:54:20.090 INFO:tasks.workunit.client.1.smithi144.stdout:8/769: dwrite d2/de/d49/d83/f99 [0,4194304] 0 2024-05-15T15:54:20.096 INFO:tasks.workunit.client.1.smithi144.stdout:9/742: dread - d0/d8/d5e/f9a zero size 2024-05-15T15:54:20.097 INFO:tasks.workunit.client.1.smithi144.stdout:9/743: chown d0/d8/d5e/d44/d99/lcc 44197 1 2024-05-15T15:54:20.098 INFO:tasks.workunit.client.1.smithi144.stdout:9/744: write d0/d8/d1b/d79/d8a/f97 [646502,48206] 0 2024-05-15T15:54:20.099 INFO:tasks.workunit.client.0.smithi049.stdout:9/522: unlink d0/l5b 0 2024-05-15T15:54:20.099 INFO:tasks.workunit.client.1.smithi144.stdout:9/745: write d0/d8/d5e/d44/d16/d5a/fe2 [245620,55816] 0 2024-05-15T15:54:20.102 INFO:tasks.workunit.client.0.smithi049.stdout:7/614: dwrite d0/d6/d23/d45/d6a/da1/d80/fb2 [0,4194304] 0 2024-05-15T15:54:20.102 INFO:tasks.workunit.client.1.smithi144.stdout:3/710: rename d0/d11/d3b/cd3 to d0/d1/d17/d39/d40/cea 0 2024-05-15T15:54:20.104 INFO:tasks.workunit.client.0.smithi049.stdout:1/805: unlink d0/d4/dd/de/d36/f3f 0 2024-05-15T15:54:20.106 INFO:tasks.workunit.client.0.smithi049.stdout:6/631: write dd/d17/d24/d2b/d36/f56 [1400828,34892] 0 2024-05-15T15:54:20.108 INFO:tasks.workunit.client.1.smithi144.stdout:4/710: link d6/dd/d2e/d75/fb5 d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/fe4 0 2024-05-15T15:54:20.109 INFO:tasks.workunit.client.0.smithi049.stdout:2/611: rmdir d3/db/d1d/d9f/db9 0 2024-05-15T15:54:20.110 INFO:tasks.workunit.client.1.smithi144.stdout:4/711: truncate d6/dd/fd2 308176 0 2024-05-15T15:54:20.110 INFO:tasks.workunit.client.0.smithi049.stdout:5/690: link dc/dd/d7b/la2 dc/dd/d7b/le5 0 2024-05-15T15:54:20.111 INFO:tasks.workunit.client.1.smithi144.stdout:4/712: chown d6/d42/d46/d4b/d1e/d44/f81 2178570 1 2024-05-15T15:54:20.113 INFO:tasks.workunit.client.0.smithi049.stdout:9/523: creat d0/d8/d9/d52/d5e/d5f/d56/fb4 x:0 0 0 2024-05-15T15:54:20.114 INFO:tasks.workunit.client.0.smithi049.stdout:6/632: dwrite f4 [0,4194304] 0 2024-05-15T15:54:20.114 INFO:tasks.workunit.client.1.smithi144.stdout:5/739: creat d0/dc/d24/d8e/d95/ff9 x:0 0 0 2024-05-15T15:54:20.114 INFO:tasks.workunit.client.1.smithi144.stdout:5/740: dread - d0/dc/d2f/d34/d3e/fed zero size 2024-05-15T15:54:20.118 INFO:tasks.workunit.client.1.smithi144.stdout:3/711: mkdir d0/deb 0 2024-05-15T15:54:20.119 INFO:tasks.workunit.client.1.smithi144.stdout:3/712: readlink d0/d1/l36 0 2024-05-15T15:54:20.119 INFO:tasks.workunit.client.1.smithi144.stdout:3/713: dread - d0/d1/d17/f8b zero size 2024-05-15T15:54:20.122 INFO:tasks.workunit.client.1.smithi144.stdout:0/685: rename d4/dc/ccd to d4/d2d/d48/d5b/cf3 0 2024-05-15T15:54:20.123 INFO:tasks.workunit.client.0.smithi049.stdout:2/612: rename d3/da/d15/d24/f62 to d3/db/d1d/d1e/d2b/d6b/fd2 0 2024-05-15T15:54:20.126 INFO:tasks.workunit.client.1.smithi144.stdout:3/714: dwrite d0/d1/d17/d39/d83/f9d [0,4194304] 0 2024-05-15T15:54:20.129 INFO:tasks.workunit.client.0.smithi049.stdout:1/806: symlink d0/d4/dd/de/d1e/d2e/dad/l11e 0 2024-05-15T15:54:20.130 INFO:tasks.workunit.client.0.smithi049.stdout:0/648: link dc/d13/f17 dc/da4/db4/fd1 0 2024-05-15T15:54:20.132 INFO:tasks.workunit.client.0.smithi049.stdout:5/691: symlink dc/d4a/le6 0 2024-05-15T15:54:20.136 INFO:tasks.workunit.client.1.smithi144.stdout:6/732: creat d7/d1e/d28/db5/df6/f105 x:0 0 0 2024-05-15T15:54:20.137 INFO:tasks.workunit.client.1.smithi144.stdout:6/733: chown d7/db/d11/f15 14 1 2024-05-15T15:54:20.138 INFO:tasks.workunit.client.0.smithi049.stdout:2/613: symlink d3/db/d1d/d1e/d2b/d3b/ld3 0 2024-05-15T15:54:20.139 INFO:tasks.workunit.client.1.smithi144.stdout:4/713: mknod d6/dd/d2e/d71/ce5 0 2024-05-15T15:54:20.140 INFO:tasks.workunit.client.1.smithi144.stdout:4/714: stat d6/d42/d46/d4b/fdb 0 2024-05-15T15:54:20.141 INFO:tasks.workunit.client.0.smithi049.stdout:7/615: creat d0/d6/da/fc0 x:0 0 0 2024-05-15T15:54:20.141 INFO:tasks.workunit.client.1.smithi144.stdout:9/746: mknod d0/d8/d1b/d79/d8a/dae/ddb/dfc/cfd 0 2024-05-15T15:54:20.143 INFO:tasks.workunit.client.1.smithi144.stdout:9/747: stat d0/d8/d5e/d44/d10/d19/dd5/de5 0 2024-05-15T15:54:20.144 INFO:tasks.workunit.client.0.smithi049.stdout:0/649: creat dc/d13/d56/d6f/fd2 x:0 0 0 2024-05-15T15:54:20.144 INFO:tasks.workunit.client.1.smithi144.stdout:2/664: dwrite d3/f5 [0,4194304] 0 2024-05-15T15:54:20.149 INFO:tasks.workunit.client.1.smithi144.stdout:5/741: dwrite d0/dc/d2f/d34/d63/d6e/f7c [0,4194304] 0 2024-05-15T15:54:20.149 INFO:tasks.workunit.client.1.smithi144.stdout:5/742: chown d0/dc/d2f/d34/dc1 215 1 2024-05-15T15:54:20.150 INFO:tasks.workunit.client.0.smithi049.stdout:9/524: rename d0/d8/d1b/d71/d87/c9e to d0/d8e/cb5 0 2024-05-15T15:54:20.150 INFO:tasks.workunit.client.1.smithi144.stdout:5/743: dread - d0/dc/d1f/d50/fbe zero size 2024-05-15T15:54:20.154 INFO:tasks.workunit.client.1.smithi144.stdout:7/712: creat d4/df/d25/ff5 x:0 0 0 2024-05-15T15:54:20.155 INFO:tasks.workunit.client.0.smithi049.stdout:2/614: symlink d3/db/d1d/d1e/d2b/d3b/d42/d8f/ld4 0 2024-05-15T15:54:20.156 INFO:tasks.workunit.client.1.smithi144.stdout:3/715: rmdir d0/d11/d3b 39 2024-05-15T15:54:20.156 INFO:tasks.workunit.client.1.smithi144.stdout:3/716: fdatasync d0/d1/d6c/fc1 0 2024-05-15T15:54:20.162 INFO:tasks.workunit.client.1.smithi144.stdout:1/694: rmdir d7/d1d/d33/d29/d6b/d8e/dcb 0 2024-05-15T15:54:20.162 INFO:tasks.workunit.client.0.smithi049.stdout:0/650: creat dc/d13/d14/d33/fd3 x:0 0 0 2024-05-15T15:54:20.164 INFO:tasks.workunit.client.0.smithi049.stdout:1/807: dwrite d0/d4/dd/de/d1e/f1d [0,4194304] 0 2024-05-15T15:54:20.164 INFO:tasks.workunit.client.0.smithi049.stdout:0/651: dread - dc/d13/d2b/d76/fb0 zero size 2024-05-15T15:54:20.165 INFO:tasks.workunit.client.1.smithi144.stdout:2/665: mkdir d3/d7/d20/d37/de1 0 2024-05-15T15:54:20.166 INFO:tasks.workunit.client.0.smithi049.stdout:0/652: chown dc/d13/d14/d25/d31/d71/c7d 1040673 1 2024-05-15T15:54:20.167 INFO:tasks.workunit.client.0.smithi049.stdout:2/615: creat d3/db/d1d/d1e/d2b/d6b/fd5 x:0 0 0 2024-05-15T15:54:20.173 INFO:tasks.workunit.client.0.smithi049.stdout:0/653: creat dc/d13/d14/d25/d31/dae/fd4 x:0 0 0 2024-05-15T15:54:20.173 INFO:tasks.workunit.client.1.smithi144.stdout:0/686: rename d4/df/d14/cc5 to d4/df/d14/db5/cf4 0 2024-05-15T15:54:20.175 INFO:tasks.workunit.client.1.smithi144.stdout:2/666: mkdir d3/d7/da4/de2 0 2024-05-15T15:54:20.177 INFO:tasks.workunit.client.0.smithi049.stdout:7/616: creat d0/d6/da/fc1 x:0 0 0 2024-05-15T15:54:20.177 INFO:tasks.workunit.client.1.smithi144.stdout:4/715: dread d6/dd/d2e/d75/fb5 [0,4194304] 0 2024-05-15T15:54:20.178 INFO:tasks.workunit.client.0.smithi049.stdout:5/692: link dc/dd/l73 dc/dd/d47/d83/dcc/da9/dba/dbe/le7 0 2024-05-15T15:54:20.179 INFO:tasks.workunit.client.0.smithi049.stdout:0/654: dwrite dc/d13/d14/f79 [0,4194304] 0 2024-05-15T15:54:20.180 INFO:tasks.workunit.client.0.smithi049.stdout:9/525: creat d0/d8/fb6 x:0 0 0 2024-05-15T15:54:20.180 INFO:tasks.workunit.client.0.smithi049.stdout:0/655: stat dc/d13/d14/cad 0 2024-05-15T15:54:20.182 INFO:tasks.workunit.client.1.smithi144.stdout:0/687: read d4/df/d1f/d24/f69 [4031587,61844] 0 2024-05-15T15:54:20.187 INFO:tasks.workunit.client.0.smithi049.stdout:5/693: truncate dc/d4a/f3a 1228533 0 2024-05-15T15:54:20.197 INFO:tasks.workunit.client.0.smithi049.stdout:1/808: getdents d0/d4/dd/de/d4d/d8b 0 2024-05-15T15:54:20.200 INFO:tasks.workunit.client.0.smithi049.stdout:0/656: rename dc/d13/d3d/d84/fcf to dc/fd5 0 2024-05-15T15:54:20.201 INFO:tasks.workunit.client.0.smithi049.stdout:5/694: rename dc to dc/dd/d10/db3/de8 22 2024-05-15T15:54:20.205 INFO:tasks.workunit.client.0.smithi049.stdout:1/809: rename d0/d4/dd/de/d1e/d2e/d3e/f62 to d0/d4/dd/de/d1e/d16/d8a/dd2/df2/f11f 0 2024-05-15T15:54:20.206 INFO:tasks.workunit.client.0.smithi049.stdout:1/810: stat d0/d4/dd/de/d48/d5c/fa3 0 2024-05-15T15:54:20.208 INFO:tasks.workunit.client.0.smithi049.stdout:5/695: rmdir dc/d4a/d42 39 2024-05-15T15:54:20.209 INFO:tasks.workunit.client.0.smithi049.stdout:5/696: truncate dc/d4a/d9b/fdb 137838 0 2024-05-15T15:54:20.210 INFO:tasks.workunit.client.0.smithi049.stdout:5/697: fdatasync dc/dd/d47/d83/da7/fe4 0 2024-05-15T15:54:20.210 INFO:tasks.workunit.client.0.smithi049.stdout:2/616: truncate d3/d8/f36 623942 0 2024-05-15T15:54:20.211 INFO:tasks.workunit.client.0.smithi049.stdout:5/698: truncate dc/dd/d47/d83/da7/fe4 829343 0 2024-05-15T15:54:20.215 INFO:tasks.workunit.client.1.smithi144.stdout:8/770: sync 2024-05-15T15:54:20.216 INFO:tasks.workunit.client.0.smithi049.stdout:5/699: dread dc/dd/d47/f5d [0,4194304] 0 2024-05-15T15:54:20.218 INFO:tasks.workunit.client.0.smithi049.stdout:5/700: chown dc/dd/d47/d83/dcc/da9/dba/dbe 798860128 1 2024-05-15T15:54:20.220 INFO:tasks.workunit.client.0.smithi049.stdout:9/526: dwrite d0/d8/d9c/f9f [0,4194304] 0 2024-05-15T15:54:20.221 INFO:tasks.workunit.client.0.smithi049.stdout:0/657: getdents dc/d13/d14/d33 0 2024-05-15T15:54:20.222 INFO:tasks.workunit.client.0.smithi049.stdout:9/527: write d0/d8/d9c/f93 [941556,121671] 0 2024-05-15T15:54:20.223 INFO:tasks.workunit.client.0.smithi049.stdout:9/528: dread - d0/d8/d9/d22/fb1 zero size 2024-05-15T15:54:20.228 INFO:tasks.workunit.client.0.smithi049.stdout:0/658: dwrite dc/d13/d14/d25/d31/d71/fc1 [0,4194304] 0 2024-05-15T15:54:20.229 INFO:tasks.workunit.client.0.smithi049.stdout:0/659: chown dc/d13/d14/d25/d31/dae/dc5/d57 952 1 2024-05-15T15:54:20.235 INFO:tasks.workunit.client.0.smithi049.stdout:1/811: write d0/d4/dd/de/d1e/d2e/d3e/d90/f91 [342459,52431] 0 2024-05-15T15:54:20.236 INFO:tasks.workunit.client.0.smithi049.stdout:3/636: sync 2024-05-15T15:54:20.237 INFO:tasks.workunit.client.0.smithi049.stdout:4/616: sync 2024-05-15T15:54:20.237 INFO:tasks.workunit.client.0.smithi049.stdout:8/616: sync 2024-05-15T15:54:20.239 INFO:tasks.workunit.client.0.smithi049.stdout:4/617: chown fa 3 1 2024-05-15T15:54:20.241 INFO:tasks.workunit.client.0.smithi049.stdout:5/701: rename dc/d4a/l96 to dc/dd/d7b/le9 0 2024-05-15T15:54:20.249 INFO:tasks.workunit.client.0.smithi049.stdout:8/617: mkdir d2/de/d25/d74/dbe 0 2024-05-15T15:54:20.250 INFO:tasks.workunit.client.0.smithi049.stdout:3/637: fdatasync d12/d14/d32/d64/d88/f9f 0 2024-05-15T15:54:20.253 INFO:tasks.workunit.client.0.smithi049.stdout:0/660: symlink dc/d13/d14/d25/d31/d71/dbf/ld6 0 2024-05-15T15:54:20.253 INFO:tasks.workunit.client.0.smithi049.stdout:0/661: stat dc/c10 0 2024-05-15T15:54:20.257 INFO:tasks.workunit.client.0.smithi049.stdout:3/638: dwrite f9 [4194304,4194304] 0 2024-05-15T15:54:20.262 INFO:tasks.workunit.client.0.smithi049.stdout:9/529: dwrite d0/f41 [0,4194304] 0 2024-05-15T15:54:20.267 INFO:tasks.workunit.client.0.smithi049.stdout:8/618: mkdir d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf 0 2024-05-15T15:54:20.270 INFO:tasks.workunit.client.0.smithi049.stdout:1/812: dwrite d0/d4/dd/de/d1e/d2e/dad/fa1 [0,4194304] 0 2024-05-15T15:54:20.274 INFO:tasks.workunit.client.1.smithi144.stdout:6/734: creat d7/f106 x:0 0 0 2024-05-15T15:54:20.274 INFO:tasks.workunit.client.1.smithi144.stdout:6/735: chown d7/d1e 27925 1 2024-05-15T15:54:20.278 INFO:tasks.workunit.client.1.smithi144.stdout:7/713: rename d4/d41/d5b/l5c to d4/d3c/d76/d9d/dac/lf6 0 2024-05-15T15:54:20.278 INFO:tasks.workunit.client.1.smithi144.stdout:6/736: dread d7/d1e/d28/f8a [0,4194304] 0 2024-05-15T15:54:20.279 INFO:tasks.workunit.client.0.smithi049.stdout:0/662: symlink dc/d13/d3d/d84/ld7 0 2024-05-15T15:54:20.281 INFO:tasks.workunit.client.0.smithi049.stdout:0/663: truncate dc/d13/d56/d6f/fd2 429808 0 2024-05-15T15:54:20.284 INFO:tasks.workunit.client.0.smithi049.stdout:4/618: chown d18/d1e/d2a/d50/l6c 19532 1 2024-05-15T15:54:20.287 INFO:tasks.workunit.client.1.smithi144.stdout:2/667: creat d3/d7/d20/d3f/fe3 x:0 0 0 2024-05-15T15:54:20.289 INFO:tasks.workunit.client.1.smithi144.stdout:9/748: link d0/d8/d5e/d44/d10/d19/cb7 d0/d8/d5e/d44/dc2/d75/cfe 0 2024-05-15T15:54:20.289 INFO:tasks.workunit.client.1.smithi144.stdout:5/744: write d0/dc/d12/fab [257566,117847] 0 2024-05-15T15:54:20.289 INFO:tasks.workunit.client.1.smithi144.stdout:9/749: readlink d0/d8/d21/ld9 0 2024-05-15T15:54:20.289 INFO:tasks.workunit.client.0.smithi049.stdout:9/530: dread d0/d8/d42/f91 [0,4194304] 0 2024-05-15T15:54:20.290 INFO:tasks.workunit.client.1.smithi144.stdout:5/745: dread - d0/dc/d2f/d34/d63/d6e/dbf/f4c zero size 2024-05-15T15:54:20.291 INFO:tasks.workunit.client.1.smithi144.stdout:4/716: fdatasync d6/d42/d46/d4b/f31 0 2024-05-15T15:54:20.292 INFO:tasks.workunit.client.1.smithi144.stdout:4/717: readlink d6/d42/d46/d4b/d1e/d44/l89 0 2024-05-15T15:54:20.293 INFO:tasks.workunit.client.1.smithi144.stdout:4/718: dread d6/dd/d34/d3b/fd9 [0,4194304] 0 2024-05-15T15:54:20.298 INFO:tasks.workunit.client.0.smithi049.stdout:5/702: link dc/dd/d10/c65 dc/db0/dd5/dd8/cea 0 2024-05-15T15:54:20.300 INFO:tasks.workunit.client.1.smithi144.stdout:1/695: write d7/d1d/d33/d41/d78/da3/f88 [3371003,105311] 0 2024-05-15T15:54:20.300 INFO:tasks.workunit.client.0.smithi049.stdout:0/664: mknod dc/cd8 0 2024-05-15T15:54:20.301 INFO:tasks.workunit.client.1.smithi144.stdout:6/737: rmdir d7/d1e/d28/da8 39 2024-05-15T15:54:20.304 INFO:tasks.workunit.client.0.smithi049.stdout:9/531: creat d0/d8/d53/fb7 x:0 0 0 2024-05-15T15:54:20.306 INFO:tasks.workunit.client.1.smithi144.stdout:3/717: link d0/d1/d17/d39/dc8/fdc d0/d11/d3b/da7/fec 0 2024-05-15T15:54:20.308 INFO:tasks.workunit.client.1.smithi144.stdout:8/771: dwrite d2/f96 [0,4194304] 0 2024-05-15T15:54:20.309 INFO:tasks.workunit.client.0.smithi049.stdout:5/703: fdatasync dc/d4a/f8b 0 2024-05-15T15:54:20.310 INFO:tasks.workunit.client.0.smithi049.stdout:8/619: dwrite d2/f7b [0,4194304] 0 2024-05-15T15:54:20.312 INFO:tasks.workunit.client.1.smithi144.stdout:9/750: mknod d0/d8/d50/cff 0 2024-05-15T15:54:20.313 INFO:tasks.workunit.client.1.smithi144.stdout:8/772: dread d2/f96 [0,4194304] 0 2024-05-15T15:54:20.313 INFO:tasks.workunit.client.0.smithi049.stdout:8/620: chown d2/de/d71/d86/d5d/d7d/db8/l9f 817489 1 2024-05-15T15:54:20.313 INFO:tasks.workunit.client.0.smithi049.stdout:1/813: dwrite d0/d4/fa [0,4194304] 0 2024-05-15T15:54:20.315 INFO:tasks.workunit.client.1.smithi144.stdout:8/773: chown d2/d24/f68 15091 1 2024-05-15T15:54:20.316 INFO:tasks.workunit.client.1.smithi144.stdout:8/774: stat d2/d24/d2d/d30/dd0/df7/d87/fda 0 2024-05-15T15:54:20.317 INFO:tasks.workunit.client.0.smithi049.stdout:3/639: rename d12/d30/d63 to d12/d14/d52/db9/dc4 0 2024-05-15T15:54:20.319 INFO:tasks.workunit.client.1.smithi144.stdout:5/746: rmdir d0/dc/d2f/d34/d63/d6e/dbf 39 2024-05-15T15:54:20.325 INFO:tasks.workunit.client.1.smithi144.stdout:4/719: rmdir d6/d42/d46/d4b/d1e/d99/dad 39 2024-05-15T15:54:20.327 INFO:tasks.workunit.client.1.smithi144.stdout:5/747: dwrite d0/dc/d2f/d34/dc1/ff5 [0,4194304] 0 2024-05-15T15:54:20.327 INFO:tasks.workunit.client.1.smithi144.stdout:4/720: chown d6/dd/d34/d3b/d30/d59/d66/fb8 288039979 1 2024-05-15T15:54:20.328 INFO:tasks.workunit.client.0.smithi049.stdout:5/704: mknod dc/dd/d10/ceb 0 2024-05-15T15:54:20.328 INFO:tasks.workunit.client.1.smithi144.stdout:4/721: truncate d6/dd/fd2 707235 0 2024-05-15T15:54:20.331 INFO:tasks.workunit.client.1.smithi144.stdout:0/688: rename d4/d6/d4a/d4d/c88 to d4/d6/d4a/cf5 0 2024-05-15T15:54:20.332 INFO:tasks.workunit.client.0.smithi049.stdout:1/814: read - d0/d4/d63/f64 zero size 2024-05-15T15:54:20.333 INFO:tasks.workunit.client.0.smithi049.stdout:1/815: chown d0/d4/dd/de/d1e/d16/cc8 445966150 1 2024-05-15T15:54:20.335 INFO:tasks.workunit.client.0.smithi049.stdout:4/619: rename f5 to d18/d1e/d2a/d36/d86/fc8 0 2024-05-15T15:54:20.336 INFO:tasks.workunit.client.1.smithi144.stdout:0/689: dwrite d4/d2d/fce [0,4194304] 0 2024-05-15T15:54:20.338 INFO:tasks.workunit.client.0.smithi049.stdout:9/532: symlink d0/d8/d12/lb8 0 2024-05-15T15:54:20.340 INFO:tasks.workunit.client.0.smithi049.stdout:9/533: dread d0/d8/d9/d22/f96 [0,4194304] 0 2024-05-15T15:54:20.341 INFO:tasks.workunit.client.1.smithi144.stdout:7/714: link d4/df/d25/d40/d62/dbc/fc8 d4/d41/d5b/da0/ff7 0 2024-05-15T15:54:20.342 INFO:tasks.workunit.client.0.smithi049.stdout:9/534: fdatasync d0/d8/d9/f57 0 2024-05-15T15:54:20.347 INFO:tasks.workunit.client.0.smithi049.stdout:1/816: dread d0/f5b [0,4194304] 0 2024-05-15T15:54:20.347 INFO:tasks.workunit.client.1.smithi144.stdout:2/668: rmdir d3/d7/d20/d37 39 2024-05-15T15:54:20.348 INFO:tasks.workunit.client.1.smithi144.stdout:2/669: readlink d3/d7/d20/d22/d52/d75/lcd 0 2024-05-15T15:54:20.349 INFO:tasks.workunit.client.1.smithi144.stdout:9/751: symlink d0/d8/d1b/d62/d7d/d8d/d94/l100 0 2024-05-15T15:54:20.351 INFO:tasks.workunit.client.1.smithi144.stdout:8/775: write d2/fb6 [2542500,102470] 0 2024-05-15T15:54:20.354 INFO:tasks.workunit.client.0.smithi049.stdout:3/640: symlink d12/d14/d52/db9/dc4/d68/lc5 0 2024-05-15T15:54:20.356 INFO:tasks.workunit.client.0.smithi049.stdout:9/535: stat d0/d8/d9/fb 0 2024-05-15T15:54:20.358 INFO:tasks.workunit.client.0.smithi049.stdout:0/665: getdents dc/d13/d14/d25/d31/dae/dc5/d57 0 2024-05-15T15:54:20.360 INFO:tasks.workunit.client.1.smithi144.stdout:1/696: dread d7/d1d/d33/d1e/d34/f4a [4194304,4194304] 0 2024-05-15T15:54:20.362 INFO:tasks.workunit.client.1.smithi144.stdout:1/697: fsync d7/d1d/d33/d41/d22/d45/fb3 0 2024-05-15T15:54:20.363 INFO:tasks.workunit.client.0.smithi049.stdout:0/666: dread dc/d13/d14/d25/d31/d71/fc1 [0,4194304] 0 2024-05-15T15:54:20.365 INFO:tasks.workunit.client.1.smithi144.stdout:6/738: dwrite d7/ff [0,4194304] 0 2024-05-15T15:54:20.366 INFO:tasks.workunit.client.0.smithi049.stdout:4/620: fdatasync d18/f80 0 2024-05-15T15:54:20.372 INFO:tasks.workunit.client.0.smithi049.stdout:9/536: creat d0/d8e/fb9 x:0 0 0 2024-05-15T15:54:20.373 INFO:tasks.workunit.client.0.smithi049.stdout:5/705: write dc/dd/d7b/fdf [2574351,64106] 0 2024-05-15T15:54:20.375 INFO:tasks.workunit.client.0.smithi049.stdout:9/537: write d0/d8/d1b/d71/d7d/f9b [376852,112269] 0 2024-05-15T15:54:20.375 INFO:tasks.workunit.client.1.smithi144.stdout:2/670: write d3/d7/d20/f7c [5173928,120119] 0 2024-05-15T15:54:20.376 INFO:tasks.workunit.client.0.smithi049.stdout:5/706: write dc/d4a/f36 [3471440,29622] 0 2024-05-15T15:54:20.376 INFO:tasks.workunit.client.0.smithi049.stdout:3/641: dread d12/d14/d32/fa7 [0,4194304] 0 2024-05-15T15:54:20.378 INFO:tasks.workunit.client.1.smithi144.stdout:9/752: creat d0/d8/d1b/d62/d7d/d8d/d94/f101 x:0 0 0 2024-05-15T15:54:20.379 INFO:tasks.workunit.client.0.smithi049.stdout:8/621: rename d2/de/d71/d86/d5d/d7d/db8/d5b/da6/laa to d2/de/d71/d86/lc0 0 2024-05-15T15:54:20.380 INFO:tasks.workunit.client.0.smithi049.stdout:0/667: creat dc/d13/d2b/d76/d8a/da2/fd9 x:0 0 0 2024-05-15T15:54:20.381 INFO:tasks.workunit.client.1.smithi144.stdout:5/748: mkdir d0/dc/d24/d8e/d93/d6b/dea/dfa 0 2024-05-15T15:54:20.383 INFO:tasks.workunit.client.1.smithi144.stdout:9/753: chown d0/d8/d5e/d44/d10/d9c/fc8 45570 1 2024-05-15T15:54:20.383 INFO:tasks.workunit.client.0.smithi049.stdout:4/621: stat d18/d85/d98/la9 0 2024-05-15T15:54:20.383 INFO:tasks.workunit.client.0.smithi049.stdout:4/622: chown d18/d85/d98/ca0 1060120250 1 2024-05-15T15:54:20.384 INFO:tasks.workunit.client.1.smithi144.stdout:7/715: dwrite d4/df/d25/f85 [0,4194304] 0 2024-05-15T15:54:20.384 INFO:tasks.workunit.client.1.smithi144.stdout:5/749: fsync d0/dc/d24/d8e/d93/fa1 0 2024-05-15T15:54:20.385 INFO:tasks.workunit.client.1.smithi144.stdout:1/698: mkdir d7/d1d/dac/de1 0 2024-05-15T15:54:20.385 INFO:tasks.workunit.client.1.smithi144.stdout:2/671: dwrite d3/d4d/fca [0,4194304] 0 2024-05-15T15:54:20.386 INFO:tasks.workunit.client.1.smithi144.stdout:7/716: chown d4/d3c/d76/d9d/dc5/le1 430 1 2024-05-15T15:54:20.389 INFO:tasks.workunit.client.1.smithi144.stdout:3/718: creat d0/d1/d17/d39/d40/d4a/fed x:0 0 0 2024-05-15T15:54:20.393 INFO:tasks.workunit.client.1.smithi144.stdout:8/776: creat d2/da8/ffb x:0 0 0 2024-05-15T15:54:20.401 INFO:tasks.workunit.client.1.smithi144.stdout:4/722: link d6/d42/d46/d4b/d1e/d44/f43 d6/dd/d34/d3b/d30/db/d1d/d40/d57/fe6 0 2024-05-15T15:54:20.401 INFO:tasks.workunit.client.1.smithi144.stdout:4/723: stat d6/d42/d46/d4b/d1e/d44/ca1 0 2024-05-15T15:54:20.402 INFO:tasks.workunit.client.1.smithi144.stdout:4/724: dread - d6/d49/fdd zero size 2024-05-15T15:54:20.402 INFO:tasks.workunit.client.1.smithi144.stdout:4/725: chown d6/dd/fa2 188160769 1 2024-05-15T15:54:20.403 INFO:tasks.workunit.client.0.smithi049.stdout:1/817: truncate d0/d4/dd/de/d1e/d2e/dad/d65/d72/f8d 943900 0 2024-05-15T15:54:20.405 INFO:tasks.workunit.client.0.smithi049.stdout:1/818: chown d0/d4/dd/de/d1e/d2e/dad/d65/d72/ldc 9 1 2024-05-15T15:54:20.406 INFO:tasks.workunit.client.1.smithi144.stdout:0/690: write d4/df/d1f/f2c [1569832,30438] 0 2024-05-15T15:54:20.407 INFO:tasks.workunit.client.0.smithi049.stdout:1/819: stat d0/d4/dd/de/d1e/d2e/dad/d65/d72/dae/ccd 0 2024-05-15T15:54:20.408 INFO:tasks.workunit.client.0.smithi049.stdout:0/668: dread dc/d32/f96 [0,4194304] 0 2024-05-15T15:54:20.408 INFO:tasks.workunit.client.1.smithi144.stdout:6/739: symlink d7/db/d11/d3c/d61/dcc/l107 0 2024-05-15T15:54:20.409 INFO:tasks.workunit.client.0.smithi049.stdout:3/642: creat d12/d67/da8/fc6 x:0 0 0 2024-05-15T15:54:20.411 INFO:tasks.workunit.client.1.smithi144.stdout:7/717: readlink d4/d3c/d76/d9d/dba/lde 0 2024-05-15T15:54:20.412 INFO:tasks.workunit.client.0.smithi049.stdout:9/538: write d0/d8/d21/fae [80917,51594] 0 2024-05-15T15:54:20.413 INFO:tasks.workunit.client.1.smithi144.stdout:6/740: dwrite d7/db/d9d/d97/fc0 [0,4194304] 0 2024-05-15T15:54:20.414 INFO:tasks.workunit.client.1.smithi144.stdout:6/741: truncate d7/db/d5f/fb8 1090512 0 2024-05-15T15:54:20.415 INFO:tasks.workunit.client.1.smithi144.stdout:2/672: dread - d3/d7/d20/d22/d52/f6f zero size 2024-05-15T15:54:20.416 INFO:tasks.workunit.client.1.smithi144.stdout:2/673: chown d3/d7/d20/d89 0 1 2024-05-15T15:54:20.420 INFO:tasks.workunit.client.1.smithi144.stdout:3/719: creat d0/d1/d56/d98/d9c/fee x:0 0 0 2024-05-15T15:54:20.421 INFO:tasks.workunit.client.1.smithi144.stdout:8/777: mknod d2/d24/d2d/d30/dd0/df7/d58/cfc 0 2024-05-15T15:54:20.422 INFO:tasks.workunit.client.0.smithi049.stdout:0/669: creat dc/fda x:0 0 0 2024-05-15T15:54:20.423 INFO:tasks.workunit.client.1.smithi144.stdout:9/754: dwrite d0/d8/d5e/d44/dc2/d75/fb1 [0,4194304] 0 2024-05-15T15:54:20.423 INFO:tasks.workunit.client.0.smithi049.stdout:1/820: mknod d0/d4/dd/de/d4d/d8b/dba/c120 0 2024-05-15T15:54:20.424 INFO:tasks.workunit.client.0.smithi049.stdout:8/622: write d2/de/d71/d86/d5d/d7d/db8/d5b/fb2 [555279,130184] 0 2024-05-15T15:54:20.425 INFO:tasks.workunit.client.0.smithi049.stdout:5/707: creat dc/fec x:0 0 0 2024-05-15T15:54:20.430 INFO:tasks.workunit.client.1.smithi144.stdout:1/699: write d7/d1d/d33/d29/d6b/d5e/fa1 [224125,34420] 0 2024-05-15T15:54:20.432 INFO:tasks.workunit.client.1.smithi144.stdout:5/750: rmdir d0/dc/d2f/d34/d63/d6e/dbf 39 2024-05-15T15:54:20.439 INFO:tasks.workunit.client.0.smithi049.stdout:8/623: symlink d2/de/d71/d86/d5d/d7d/d45/lc1 0 2024-05-15T15:54:20.441 INFO:tasks.workunit.client.0.smithi049.stdout:1/821: truncate d0/d4/dd/de/d4e/fa7 1972188 0 2024-05-15T15:54:20.442 INFO:tasks.workunit.client.0.smithi049.stdout:5/708: dread dc/dd/f13 [0,4194304] 0 2024-05-15T15:54:20.444 INFO:tasks.workunit.client.0.smithi049.stdout:5/709: fsync dc/dd/d10/f71 0 2024-05-15T15:54:20.447 INFO:tasks.workunit.client.1.smithi144.stdout:8/778: unlink d2/de/d3f/d6b/fe2 0 2024-05-15T15:54:20.448 INFO:tasks.workunit.client.0.smithi049.stdout:5/710: read dc/db0/dd5/fe1 [2315031,52385] 0 2024-05-15T15:54:20.449 INFO:tasks.workunit.client.1.smithi144.stdout:6/742: write d7/db/d11/d3c/d61/f9b [206105,82066] 0 2024-05-15T15:54:20.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:20 smithi144 ceph-mon[36389]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 74 MiB/s rd, 190 MiB/s wr, 399 op/s 2024-05-15T15:54:20.449 INFO:tasks.workunit.client.0.smithi049.stdout:8/624: dwrite d2/de/d25/d74/f9d [0,4194304] 0 2024-05-15T15:54:20.449 INFO:tasks.workunit.client.0.smithi049.stdout:4/623: dread d18/d21/f56 [0,4194304] 0 2024-05-15T15:54:20.450 INFO:tasks.workunit.client.1.smithi144.stdout:6/743: stat d7/db/d35/d84/dbe/lc2 0 2024-05-15T15:54:20.451 INFO:tasks.workunit.client.0.smithi049.stdout:1/822: symlink d0/d4/dd/de/d36/l121 0 2024-05-15T15:54:20.453 INFO:tasks.workunit.client.0.smithi049.stdout:9/539: dwrite d0/d8/d9/fa [0,4194304] 0 2024-05-15T15:54:20.453 INFO:tasks.workunit.client.0.smithi049.stdout:5/711: truncate dc/d4a/f91 4731451 0 2024-05-15T15:54:20.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:20 smithi049 ceph-mon[28399]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 74 MiB/s rd, 190 MiB/s wr, 399 op/s 2024-05-15T15:54:20.456 INFO:tasks.workunit.client.1.smithi144.stdout:9/755: mknod d0/d8/d1b/d79/de7/c102 0 2024-05-15T15:54:20.456 INFO:tasks.workunit.client.0.smithi049.stdout:1/823: chown d0/d4/dd/de/d4d 33 1 2024-05-15T15:54:20.457 INFO:tasks.workunit.client.0.smithi049.stdout:4/624: write d18/d85/fb5 [4187861,56972] 0 2024-05-15T15:54:20.459 INFO:tasks.workunit.client.1.smithi144.stdout:1/700: symlink d7/d1d/d33/d29/d6b/d5e/le2 0 2024-05-15T15:54:20.460 INFO:tasks.workunit.client.0.smithi049.stdout:5/712: truncate dc/dd/f3b 5133794 0 2024-05-15T15:54:20.463 INFO:tasks.workunit.client.1.smithi144.stdout:5/751: chown d0/dc/d24/d8e/d93/cac 0 1 2024-05-15T15:54:20.465 INFO:tasks.workunit.client.0.smithi049.stdout:8/625: fsync d2/de/d71/d86/f8d 0 2024-05-15T15:54:20.466 INFO:tasks.workunit.client.1.smithi144.stdout:7/718: fsync d4/d3c/d76/d9d/dac/d48/d72/d82/ff2 0 2024-05-15T15:54:20.468 INFO:tasks.workunit.client.0.smithi049.stdout:9/540: dread d0/d25/f4c [0,4194304] 0 2024-05-15T15:54:20.469 INFO:tasks.workunit.client.0.smithi049.stdout:0/670: getdents dc/d13/d2b/d76 0 2024-05-15T15:54:20.472 INFO:tasks.workunit.client.0.smithi049.stdout:4/625: chown d18/d1e/d2a/d36/l5f 764845 1 2024-05-15T15:54:20.473 INFO:tasks.workunit.client.1.smithi144.stdout:3/720: fdatasync d0/d11/d3b/f96 0 2024-05-15T15:54:20.476 INFO:tasks.workunit.client.0.smithi049.stdout:1/824: dread d0/d4/dd/de/d1e/f15 [0,4194304] 0 2024-05-15T15:54:20.477 INFO:tasks.workunit.client.1.smithi144.stdout:8/779: truncate d2/d24/d2d/f54 311551 0 2024-05-15T15:54:20.478 INFO:tasks.workunit.client.0.smithi049.stdout:1/825: chown d0/d4/dd/de/d36/c5f 3152 1 2024-05-15T15:54:20.478 INFO:tasks.workunit.client.0.smithi049.stdout:5/713: write dc/d4a/f50 [1871293,108543] 0 2024-05-15T15:54:20.479 INFO:tasks.workunit.client.1.smithi144.stdout:4/726: creat d6/fe7 x:0 0 0 2024-05-15T15:54:20.481 INFO:tasks.workunit.client.0.smithi049.stdout:8/626: mkdir d2/de/d71/d86/d5d/d7d/db8/d5b/d93/dc2 0 2024-05-15T15:54:20.482 INFO:tasks.workunit.client.1.smithi144.stdout:6/744: mknod d7/db/d11/d7e/c108 0 2024-05-15T15:54:20.484 INFO:tasks.workunit.client.1.smithi144.stdout:0/691: creat d4/ff6 x:0 0 0 2024-05-15T15:54:20.485 INFO:tasks.workunit.client.0.smithi049.stdout:1/826: dwrite d0/d4/dd/de/d36/f6a [4194304,4194304] 0 2024-05-15T15:54:20.485 INFO:tasks.workunit.client.1.smithi144.stdout:0/692: read - d4/d6/d3a/d4c/dd6/fd8 zero size 2024-05-15T15:54:20.486 INFO:tasks.workunit.client.1.smithi144.stdout:9/756: mknod d0/d8/d1b/d79/d8a/dae/c103 0 2024-05-15T15:54:20.487 INFO:tasks.workunit.client.0.smithi049.stdout:0/671: creat dc/d13/d2b/fdb x:0 0 0 2024-05-15T15:54:20.487 INFO:tasks.workunit.client.1.smithi144.stdout:2/674: dread d3/d7/d20/d37/f70 [0,4194304] 0 2024-05-15T15:54:20.489 INFO:tasks.workunit.client.0.smithi049.stdout:0/672: rename dc/d13/d14 to dc/d13/d14/d2f/d8f/ddc 22 2024-05-15T15:54:20.489 INFO:tasks.workunit.client.1.smithi144.stdout:3/721: dread d0/d1/d17/d39/f8c [0,4194304] 0 2024-05-15T15:54:20.489 INFO:tasks.workunit.client.1.smithi144.stdout:5/752: mknod d0/dc/d1f/cfb 0 2024-05-15T15:54:20.490 INFO:tasks.workunit.client.1.smithi144.stdout:5/753: dread - d0/dc/d1f/d50/fd6 zero size 2024-05-15T15:54:20.490 INFO:tasks.workunit.client.0.smithi049.stdout:0/673: chown dc/d13/d14/d25/d31/dae/dc5/d57/c77 4178454 1 2024-05-15T15:54:20.490 INFO:tasks.workunit.client.1.smithi144.stdout:5/754: dread - d0/dc/ff3 zero size 2024-05-15T15:54:20.491 INFO:tasks.workunit.client.1.smithi144.stdout:7/719: rename d4/d41/d9c/cf3 to d4/df/d81/cf8 0 2024-05-15T15:54:20.496 INFO:tasks.workunit.client.0.smithi049.stdout:5/714: dwrite dc/dd/d7b/d8d/fa8 [0,4194304] 0 2024-05-15T15:54:20.498 INFO:tasks.workunit.client.1.smithi144.stdout:7/720: dwrite d4/d3c/d76/d9d/dac/d48/d72/da2/fbe [0,4194304] 0 2024-05-15T15:54:20.588 INFO:tasks.workunit.client.1.smithi144.stdout:8/780: creat d2/d24/d2d/da0/d6e/d8d/ffd x:0 0 0 2024-05-15T15:54:20.590 INFO:tasks.workunit.client.1.smithi144.stdout:4/727: readlink d6/dd/d34/d3b/d30/db/d1d/d40/ldc 0 2024-05-15T15:54:20.631 INFO:tasks.workunit.client.1.smithi144.stdout:3/722: sync 2024-05-15T15:54:20.631 INFO:tasks.workunit.client.1.smithi144.stdout:9/757: sync 2024-05-15T15:54:20.631 INFO:tasks.workunit.client.1.smithi144.stdout:3/723: fsync d0/d1/d6c/f99 0 2024-05-15T15:54:20.632 INFO:tasks.workunit.client.1.smithi144.stdout:9/758: write d0/d8/d5e/d44/dc2/d75/fb1 [5102920,50345] 0 2024-05-15T15:54:20.636 INFO:tasks.workunit.client.1.smithi144.stdout:3/724: dread d0/d1/d17/f33 [0,4194304] 0 2024-05-15T15:54:20.683 INFO:tasks.workunit.client.0.smithi049.stdout:5/715: symlink dc/db0/led 0 2024-05-15T15:54:20.684 INFO:tasks.workunit.client.1.smithi144.stdout:1/701: symlink d7/d1d/d33/d29/le3 0 2024-05-15T15:54:20.685 INFO:tasks.workunit.client.1.smithi144.stdout:1/702: chown d7/d1d/d33/d1e/d4e/fb6 4932963 1 2024-05-15T15:54:20.685 INFO:tasks.workunit.client.1.smithi144.stdout:2/675: symlink d3/d7/da4/le4 0 2024-05-15T15:54:20.691 INFO:tasks.workunit.client.0.smithi049.stdout:8/627: dwrite d2/fc [0,4194304] 0 2024-05-15T15:54:20.692 INFO:tasks.workunit.client.0.smithi049.stdout:4/626: getdents d18/d1e/d63/d99/db9 0 2024-05-15T15:54:20.693 INFO:tasks.workunit.client.0.smithi049.stdout:1/827: write d0/d4/dd/de/d1e/db7/fd7 [3699316,129702] 0 2024-05-15T15:54:20.696 INFO:tasks.workunit.client.0.smithi049.stdout:1/828: write d0/d4/f24 [373721,72990] 0 2024-05-15T15:54:20.697 INFO:tasks.workunit.client.0.smithi049.stdout:5/716: link dc/dd/d47/d83/dcc/da9/dba/dbe/fc1 dc/dd/d47/d83/dcc/da9/dba/dbe/dca/fee 0 2024-05-15T15:54:20.697 INFO:tasks.workunit.client.0.smithi049.stdout:9/541: dwrite d0/d8/d1b/f32 [0,4194304] 0 2024-05-15T15:54:20.698 INFO:tasks.workunit.client.0.smithi049.stdout:1/829: stat d0/d4/dd/de/d1e/d16/d8a/lb1 0 2024-05-15T15:54:20.699 INFO:tasks.workunit.client.0.smithi049.stdout:5/717: write dc/f6c [4263418,120495] 0 2024-05-15T15:54:20.699 INFO:tasks.workunit.client.0.smithi049.stdout:8/628: mkdir d2/de/dc3 0 2024-05-15T15:54:20.701 INFO:tasks.workunit.client.1.smithi144.stdout:7/721: creat d4/d3c/ff9 x:0 0 0 2024-05-15T15:54:20.705 INFO:tasks.workunit.client.0.smithi049.stdout:9/542: dread d0/d8/d1b/d71/d7d/f9b [0,4194304] 0 2024-05-15T15:54:20.712 INFO:tasks.workunit.client.1.smithi144.stdout:9/759: creat d0/d8/d5e/d44/f104 x:0 0 0 2024-05-15T15:54:20.714 INFO:tasks.workunit.client.0.smithi049.stdout:5/718: symlink dc/dd/d47/d83/da7/lef 0 2024-05-15T15:54:20.715 INFO:tasks.workunit.client.1.smithi144.stdout:0/693: mknod d4/df/d1f/d24/d85/da1/cf7 0 2024-05-15T15:54:20.717 INFO:tasks.workunit.client.0.smithi049.stdout:4/627: symlink d18/d1e/d2a/d50/lc9 0 2024-05-15T15:54:20.717 INFO:tasks.workunit.client.0.smithi049.stdout:4/628: stat d18/d1e/d2a/d3b 0 2024-05-15T15:54:20.720 INFO:tasks.workunit.client.0.smithi049.stdout:9/543: mkdir d0/d8/d1b/d71/d7d/dba 0 2024-05-15T15:54:20.721 INFO:tasks.workunit.client.0.smithi049.stdout:9/544: stat d0/d8/d1b/f3a 0 2024-05-15T15:54:20.722 INFO:tasks.workunit.client.0.smithi049.stdout:1/830: fsync d0/d4/f2d 0 2024-05-15T15:54:20.725 INFO:tasks.workunit.client.1.smithi144.stdout:3/725: sync 2024-05-15T15:54:20.726 INFO:tasks.workunit.client.0.smithi049.stdout:8/629: creat d2/de/dc3/fc4 x:0 0 0 2024-05-15T15:54:20.727 INFO:tasks.workunit.client.1.smithi144.stdout:4/728: write d6/dd/d2e/d75/fb5 [2472834,62250] 0 2024-05-15T15:54:20.728 INFO:tasks.workunit.client.0.smithi049.stdout:5/719: chown dc/d4a/d42/f48 55226 1 2024-05-15T15:54:20.733 INFO:tasks.workunit.client.0.smithi049.stdout:8/630: dread d2/de/d71/d86/d5d/d63/f6e [0,4194304] 0 2024-05-15T15:54:20.733 INFO:tasks.workunit.client.0.smithi049.stdout:9/545: mkdir d0/d8/d9/d22/dbb 0 2024-05-15T15:54:20.736 INFO:tasks.workunit.client.0.smithi049.stdout:5/720: dwrite dc/d4a/f36 [0,4194304] 0 2024-05-15T15:54:20.737 INFO:tasks.workunit.client.1.smithi144.stdout:1/703: fdatasync d7/d1d/d33/f3f 0 2024-05-15T15:54:20.738 INFO:tasks.workunit.client.1.smithi144.stdout:0/694: dwrite d4/d2d/f8d [0,4194304] 0 2024-05-15T15:54:20.739 INFO:tasks.workunit.client.1.smithi144.stdout:0/695: readlink d4/d6/d3a/d4c/l57 0 2024-05-15T15:54:20.743 INFO:tasks.workunit.client.1.smithi144.stdout:7/722: getdents d4/d3c/d76/d9d/dac/d48/ded/df0 0 2024-05-15T15:54:20.743 INFO:tasks.workunit.client.0.smithi049.stdout:5/721: dwrite dc/d29/f63 [4194304,4194304] 0 2024-05-15T15:54:20.745 INFO:tasks.workunit.client.0.smithi049.stdout:4/629: symlink d18/d69/lca 0 2024-05-15T15:54:20.746 INFO:tasks.workunit.client.1.smithi144.stdout:3/726: truncate d0/d11/d24/f74 585702 0 2024-05-15T15:54:20.747 INFO:tasks.workunit.client.0.smithi049.stdout:9/546: creat d0/d8/d1b/d40/fbc x:0 0 0 2024-05-15T15:54:20.748 INFO:tasks.workunit.client.1.smithi144.stdout:4/729: dread d6/dd/d34/d3b/fd9 [0,4194304] 0 2024-05-15T15:54:20.749 INFO:tasks.workunit.client.1.smithi144.stdout:6/745: getdents d7/db/d11/d7e 0 2024-05-15T15:54:20.750 INFO:tasks.workunit.client.1.smithi144.stdout:6/746: chown d7/db/da2/f5e 341964 1 2024-05-15T15:54:20.751 INFO:tasks.workunit.client.0.smithi049.stdout:5/722: mknod dc/dd/d47/d83/dcc/da9/cf0 0 2024-05-15T15:54:20.751 INFO:tasks.workunit.client.1.smithi144.stdout:6/747: chown d7/d62/f6b 23 1 2024-05-15T15:54:20.755 INFO:tasks.workunit.client.0.smithi049.stdout:9/547: symlink d0/d8/d1b/d40/d5c/lbd 0 2024-05-15T15:54:20.757 INFO:tasks.workunit.client.1.smithi144.stdout:5/755: link d0/dc/d24/d8e/lc5 d0/dc/d24/d57/d96/lfc 0 2024-05-15T15:54:20.757 INFO:tasks.workunit.client.1.smithi144.stdout:4/730: dread d6/dd/d34/d3b/d30/db/d1d/d40/f79 [0,4194304] 0 2024-05-15T15:54:20.758 INFO:tasks.workunit.client.1.smithi144.stdout:7/723: creat d4/df/d25/d38/d5d/ffa x:0 0 0 2024-05-15T15:54:20.759 INFO:tasks.workunit.client.1.smithi144.stdout:8/781: getdents d2/d24/d2d/d30/dd0 0 2024-05-15T15:54:20.759 INFO:tasks.workunit.client.1.smithi144.stdout:7/724: stat d4 0 2024-05-15T15:54:20.759 INFO:tasks.workunit.client.1.smithi144.stdout:5/756: chown d0/dc/d12/d4a/lb4 0 1 2024-05-15T15:54:20.760 INFO:tasks.workunit.client.1.smithi144.stdout:2/676: sync 2024-05-15T15:54:20.760 INFO:tasks.workunit.client.0.smithi049.stdout:8/631: creat d2/fc5 x:0 0 0 2024-05-15T15:54:20.760 INFO:tasks.workunit.client.0.smithi049.stdout:1/831: getdents d0/d4/dd/de/d4d/db8 0 2024-05-15T15:54:20.761 INFO:tasks.workunit.client.1.smithi144.stdout:9/760: link d0/fc6 d0/d8/d1b/d62/d7d/d8d/dbb/dce/dfb/f105 0 2024-05-15T15:54:20.762 INFO:tasks.workunit.client.1.smithi144.stdout:9/761: chown d0/d8/d1b/d79/de7/df9 5198 1 2024-05-15T15:54:20.762 INFO:tasks.workunit.client.1.smithi144.stdout:0/696: dread d4/d6/d4a/d74/f79 [0,4194304] 0 2024-05-15T15:54:20.762 INFO:tasks.workunit.client.0.smithi049.stdout:4/630: creat d18/d85/fcb x:0 0 0 2024-05-15T15:54:20.763 INFO:tasks.workunit.client.0.smithi049.stdout:1/832: dread d0/d4/dd/f5a [0,4194304] 0 2024-05-15T15:54:20.763 INFO:tasks.workunit.client.0.smithi049.stdout:8/632: symlink d2/d6a/lc6 0 2024-05-15T15:54:20.764 INFO:tasks.workunit.client.0.smithi049.stdout:1/833: chown d0/d4/dd/de/d36/dd3 112317 1 2024-05-15T15:54:20.765 INFO:tasks.workunit.client.0.smithi049.stdout:6/633: sync 2024-05-15T15:54:20.765 INFO:tasks.workunit.client.0.smithi049.stdout:7/617: sync 2024-05-15T15:54:20.766 INFO:tasks.workunit.client.1.smithi144.stdout:4/731: creat d6/d49/fe8 x:0 0 0 2024-05-15T15:54:20.767 INFO:tasks.workunit.client.0.smithi049.stdout:1/834: symlink d0/d4/dd/de/d1e/d16/l122 0 2024-05-15T15:54:20.767 INFO:tasks.workunit.client.0.smithi049.stdout:6/634: read dd/d17/d24/d2b/d36/f42 [1273965,56847] 0 2024-05-15T15:54:20.769 INFO:tasks.workunit.client.0.smithi049.stdout:8/633: mkdir d2/de/d71/d86/d5d/d7d/db8/d5b/da6/dc7 0 2024-05-15T15:54:20.770 INFO:tasks.workunit.client.1.smithi144.stdout:7/725: dread - d4/df/d25/d40/d62/f95 zero size 2024-05-15T15:54:20.771 INFO:tasks.workunit.client.0.smithi049.stdout:7/618: fdatasync d0/f4 0 2024-05-15T15:54:20.772 INFO:tasks.workunit.client.0.smithi049.stdout:7/619: stat d0/f9 0 2024-05-15T15:54:20.777 INFO:tasks.workunit.client.0.smithi049.stdout:6/635: creat dd/d17/d64/fbd x:0 0 0 2024-05-15T15:54:20.782 INFO:tasks.workunit.client.0.smithi049.stdout:1/835: rename d0/d4/dd/fcc to d0/d4/dd/de/d4d/d8b/dba/f123 0 2024-05-15T15:54:20.783 INFO:tasks.workunit.client.0.smithi049.stdout:5/723: dread dc/db0/dbf/fc5 [0,4194304] 0 2024-05-15T15:54:20.784 INFO:tasks.workunit.client.0.smithi049.stdout:7/620: dread d0/f9 [4194304,4194304] 0 2024-05-15T15:54:20.790 INFO:tasks.workunit.client.1.smithi144.stdout:1/704: rename d7/d1d/d33/d41/d22/f90 to d7/d1d/d33/d1e/fe4 0 2024-05-15T15:54:20.793 INFO:tasks.workunit.client.1.smithi144.stdout:6/748: link d7/d62/f6b d7/db/d35/d84/d81/f109 0 2024-05-15T15:54:20.793 INFO:tasks.workunit.client.0.smithi049.stdout:7/621: dwrite d0/d34/fad [0,4194304] 0 2024-05-15T15:54:20.795 INFO:tasks.workunit.client.1.smithi144.stdout:5/757: write d0/d7e/d9b/fa2 [399081,3490] 0 2024-05-15T15:54:20.795 INFO:tasks.workunit.client.1.smithi144.stdout:1/705: dwrite d7/d1d/f8b [0,4194304] 0 2024-05-15T15:54:20.796 INFO:tasks.workunit.client.1.smithi144.stdout:5/758: dread - d0/dc/d2f/d34/d3e/dd0/ff0 zero size 2024-05-15T15:54:20.797 INFO:tasks.workunit.client.0.smithi049.stdout:5/724: dread f9 [0,4194304] 0 2024-05-15T15:54:20.797 INFO:tasks.workunit.client.0.smithi049.stdout:4/631: dwrite d18/d1e/d2a/d50/f5e [0,4194304] 0 2024-05-15T15:54:20.801 INFO:tasks.workunit.client.0.smithi049.stdout:8/634: mkdir d2/de/d71/d86/d5d/d7d/d1b/dc8 0 2024-05-15T15:54:20.806 INFO:tasks.workunit.client.1.smithi144.stdout:1/706: dwrite d7/d1d/d33/d29/d6b/d5e/fa1 [0,4194304] 0 2024-05-15T15:54:20.811 INFO:tasks.workunit.client.0.smithi049.stdout:4/632: mknod d18/d1e/d63/ccc 0 2024-05-15T15:54:20.811 INFO:tasks.workunit.client.1.smithi144.stdout:1/707: chown d7/d1d/d33/d1e 3960 1 2024-05-15T15:54:20.812 INFO:tasks.workunit.client.1.smithi144.stdout:8/782: dwrite d2/d3a/d7b/d4b/fb5 [0,4194304] 0 2024-05-15T15:54:20.815 INFO:tasks.workunit.client.1.smithi144.stdout:2/677: dwrite d3/d7/d20/d22/d52/d64/d80/f9b [0,4194304] 0 2024-05-15T15:54:20.817 INFO:tasks.workunit.client.1.smithi144.stdout:0/697: dwrite d4/d2d/dcf/fdb [0,4194304] 0 2024-05-15T15:54:20.820 INFO:tasks.workunit.client.0.smithi049.stdout:8/635: dread d2/de/d71/d86/d5d/f89 [0,4194304] 0 2024-05-15T15:54:20.823 INFO:tasks.workunit.client.1.smithi144.stdout:7/726: rename d4/df/d25/d40/d62/dbc to d4/df/d81/dfb 0 2024-05-15T15:54:20.827 INFO:tasks.workunit.client.1.smithi144.stdout:6/749: mkdir d7/db/d35/d84/dbe/d10a 0 2024-05-15T15:54:20.829 INFO:tasks.workunit.client.0.smithi049.stdout:4/633: creat d18/d1e/d63/fcd x:0 0 0 2024-05-15T15:54:20.834 INFO:tasks.workunit.client.0.smithi049.stdout:6/636: dread dd/f14 [0,4194304] 0 2024-05-15T15:54:20.837 INFO:tasks.workunit.client.0.smithi049.stdout:6/637: truncate dd/d17/d24/d2b/f5d 1978410 0 2024-05-15T15:54:20.838 INFO:tasks.workunit.client.0.smithi049.stdout:1/836: rename d0/d4/dd/de/d1e/d2e/dad/d65/d72/ldc to d0/d4/dd/de/d1e/d16/d8a/l124 0 2024-05-15T15:54:20.838 INFO:tasks.workunit.client.0.smithi049.stdout:5/725: write dc/dd/d47/d83/dcc/f44 [1643656,109494] 0 2024-05-15T15:54:20.840 INFO:tasks.workunit.client.0.smithi049.stdout:7/622: creat d0/d6/da/fc2 x:0 0 0 2024-05-15T15:54:20.843 INFO:tasks.workunit.client.0.smithi049.stdout:6/638: creat dd/d17/fbe x:0 0 0 2024-05-15T15:54:20.849 INFO:tasks.workunit.client.0.smithi049.stdout:1/837: rename d0/d4/dd/f5a to d0/d4/dd/de/d4d/d8b/de8/f125 0 2024-05-15T15:54:20.850 INFO:tasks.workunit.client.1.smithi144.stdout:3/727: sync 2024-05-15T15:54:20.850 INFO:tasks.workunit.client.1.smithi144.stdout:4/732: truncate d6/d42/d46/d4b/f4d 1938200 0 2024-05-15T15:54:20.851 INFO:tasks.workunit.client.0.smithi049.stdout:4/634: unlink d18/d1e/d2a/c46 0 2024-05-15T15:54:20.856 INFO:tasks.workunit.client.1.smithi144.stdout:2/678: rmdir d3/d7/d20 39 2024-05-15T15:54:20.857 INFO:tasks.workunit.client.1.smithi144.stdout:2/679: readlink d3/d7/la 0 2024-05-15T15:54:20.859 INFO:tasks.workunit.client.1.smithi144.stdout:0/698: mknod d4/d2d/d48/d5b/dd3/cf8 0 2024-05-15T15:54:20.861 INFO:tasks.workunit.client.1.smithi144.stdout:7/727: symlink d4/d3c/d76/d9d/lfc 0 2024-05-15T15:54:20.863 INFO:tasks.workunit.client.1.smithi144.stdout:5/759: dwrite d0/dc/d24/d57/f7f [0,4194304] 0 2024-05-15T15:54:20.864 INFO:tasks.workunit.client.0.smithi049.stdout:1/838: mknod d0/d4/dd/de/d1e/d2e/dad/d65/df3/c126 0 2024-05-15T15:54:20.865 INFO:tasks.workunit.client.0.smithi049.stdout:1/839: chown d0/d4/dd/de/d36/l121 1 1 2024-05-15T15:54:20.868 INFO:tasks.workunit.client.0.smithi049.stdout:7/623: dread d0/d6/d23/f4c [0,4194304] 0 2024-05-15T15:54:20.869 INFO:tasks.workunit.client.0.smithi049.stdout:4/635: rename d18/d1e/d2a/d36/fc6 to d18/d1e/d2a/d50/d54/dad/fce 0 2024-05-15T15:54:20.874 INFO:tasks.workunit.client.1.smithi144.stdout:9/762: creat d0/d8/d5e/d44/d10/d19/dd5/de5/f106 x:0 0 0 2024-05-15T15:54:20.876 INFO:tasks.workunit.client.0.smithi049.stdout:6/639: dwrite dd/d17/d24/d58/f75 [0,4194304] 0 2024-05-15T15:54:20.879 INFO:tasks.workunit.client.0.smithi049.stdout:6/640: dread dd/d17/d24/d2b/d36/d50/d54/f5c [0,4194304] 0 2024-05-15T15:54:20.883 INFO:tasks.workunit.client.1.smithi144.stdout:3/728: fsync d0/f2c 0 2024-05-15T15:54:20.885 INFO:tasks.workunit.client.0.smithi049.stdout:1/840: rmdir d0/d4/dd/de/d9e 39 2024-05-15T15:54:20.887 INFO:tasks.workunit.client.1.smithi144.stdout:4/733: mknod d6/dd/d34/d3b/d30/db/d1d/d40/ce9 0 2024-05-15T15:54:20.890 INFO:tasks.workunit.client.1.smithi144.stdout:0/699: symlink d4/d6/d3a/d4c/lf9 0 2024-05-15T15:54:20.891 INFO:tasks.workunit.client.1.smithi144.stdout:1/708: dwrite d7/d1d/d33/d1e/d4e/f50 [0,4194304] 0 2024-05-15T15:54:20.892 INFO:tasks.workunit.client.1.smithi144.stdout:1/709: chown d7/d1d/f8b 74 1 2024-05-15T15:54:20.892 INFO:tasks.workunit.client.0.smithi049.stdout:1/841: dwrite d0/d4/f24 [0,4194304] 0 2024-05-15T15:54:20.897 INFO:tasks.workunit.client.1.smithi144.stdout:7/728: mknod d4/d3c/d76/cfd 0 2024-05-15T15:54:20.897 INFO:tasks.workunit.client.1.smithi144.stdout:7/729: stat d4/f37 0 2024-05-15T15:54:20.899 INFO:tasks.workunit.client.0.smithi049.stdout:5/726: getdents dc/dd 0 2024-05-15T15:54:20.900 INFO:tasks.workunit.client.1.smithi144.stdout:5/760: rmdir d0/dc/d2f/d34/d3e/dd0 39 2024-05-15T15:54:20.902 INFO:tasks.workunit.client.0.smithi049.stdout:6/641: creat dd/d17/d24/d2b/d36/d93/fbf x:0 0 0 2024-05-15T15:54:20.906 INFO:tasks.workunit.client.0.smithi049.stdout:7/624: unlink d0/d6/d23/d76/laa 0 2024-05-15T15:54:20.906 INFO:tasks.workunit.client.1.smithi144.stdout:8/783: link d2/d1a/cd7 d2/da8/cfe 0 2024-05-15T15:54:20.907 INFO:tasks.workunit.client.1.smithi144.stdout:8/784: stat d2/d24/d2d/da0/d6e/d71/fec 0 2024-05-15T15:54:20.907 INFO:tasks.workunit.client.0.smithi049.stdout:4/636: symlink d18/d1e/d2a/d36/d5d/lcf 0 2024-05-15T15:54:20.909 INFO:tasks.workunit.client.0.smithi049.stdout:1/842: unlink d0/d4/dd/de/d1e/d2e/d3e/d90/f91 0 2024-05-15T15:54:20.910 INFO:tasks.workunit.client.1.smithi144.stdout:3/729: mkdir d0/d11/d3b/da7/def 0 2024-05-15T15:54:20.911 INFO:tasks.workunit.client.1.smithi144.stdout:3/730: fsync d0/d11/d3b/da0/d8f/f9b 0 2024-05-15T15:54:20.912 INFO:tasks.workunit.client.0.smithi049.stdout:6/642: symlink dd/d17/d24/d2b/d36/d50/d54/db6/lc0 0 2024-05-15T15:54:20.914 INFO:tasks.workunit.client.1.smithi144.stdout:0/700: rename d4/d6/d3a/d4c/dd6/d55/da5/ldc to d4/d6/lfa 0 2024-05-15T15:54:20.915 INFO:tasks.workunit.client.1.smithi144.stdout:6/750: link d7/db/da2/ld4 d7/db/d5f/l10b 0 2024-05-15T15:54:20.918 INFO:tasks.workunit.client.1.smithi144.stdout:0/701: truncate d4/d2d/f2e 2573376 0 2024-05-15T15:54:20.918 INFO:tasks.workunit.client.1.smithi144.stdout:6/751: chown d7/d1e/d28/cb3 74 1 2024-05-15T15:54:20.918 INFO:tasks.workunit.client.1.smithi144.stdout:7/730: symlink d4/d3c/d76/d9d/dac/d48/lfe 0 2024-05-15T15:54:20.919 INFO:tasks.workunit.client.1.smithi144.stdout:6/752: chown f6 2072707481 1 2024-05-15T15:54:20.919 INFO:tasks.workunit.client.1.smithi144.stdout:3/731: dwrite d0/d1/f23 [0,4194304] 0 2024-05-15T15:54:20.920 INFO:tasks.workunit.client.1.smithi144.stdout:5/761: creat d0/dc/d24/d8e/d93/d6b/dea/ffd x:0 0 0 2024-05-15T15:54:20.921 INFO:tasks.workunit.client.1.smithi144.stdout:3/732: chown d0/d1/d56/d98/d9c/fee 8337045 1 2024-05-15T15:54:20.923 INFO:tasks.workunit.client.1.smithi144.stdout:1/710: dread f1 [4194304,4194304] 0 2024-05-15T15:54:20.923 INFO:tasks.workunit.client.0.smithi049.stdout:5/727: fdatasync dc/dd/d47/d83/dcc/f95 0 2024-05-15T15:54:20.926 INFO:tasks.workunit.client.0.smithi049.stdout:3/643: sync 2024-05-15T15:54:20.926 INFO:tasks.workunit.client.0.smithi049.stdout:0/674: sync 2024-05-15T15:54:20.926 INFO:tasks.workunit.client.0.smithi049.stdout:2/617: sync 2024-05-15T15:54:20.927 INFO:tasks.workunit.client.1.smithi144.stdout:8/785: unlink d2/d24/d2d/da0/d6e/d8d/fb4 0 2024-05-15T15:54:20.927 INFO:tasks.workunit.client.1.smithi144.stdout:8/786: stat d2/da8/cfe 0 2024-05-15T15:54:20.930 INFO:tasks.workunit.client.0.smithi049.stdout:6/643: mknod dd/d26/d5b/cc1 0 2024-05-15T15:54:20.932 INFO:tasks.workunit.client.0.smithi049.stdout:5/728: write dc/dd/d7b/fdf [2714108,82435] 0 2024-05-15T15:54:20.932 INFO:tasks.workunit.client.0.smithi049.stdout:0/675: fsync dc/d13/d2b/f98 0 2024-05-15T15:54:20.933 INFO:tasks.workunit.client.0.smithi049.stdout:2/618: dread - d3/db/d1d/d1e/d2b/d6b/fd5 zero size 2024-05-15T15:54:20.933 INFO:tasks.workunit.client.0.smithi049.stdout:7/625: creat d0/d34/db3/fc3 x:0 0 0 2024-05-15T15:54:20.934 INFO:tasks.workunit.client.0.smithi049.stdout:6/644: read dd/f6b [3362931,69965] 0 2024-05-15T15:54:20.938 INFO:tasks.workunit.client.0.smithi049.stdout:1/843: unlink d0/d4/c61 0 2024-05-15T15:54:20.939 INFO:tasks.workunit.client.1.smithi144.stdout:0/702: symlink d4/df/d1f/d24/d96/lfb 0 2024-05-15T15:54:20.941 INFO:tasks.workunit.client.1.smithi144.stdout:7/731: fsync d4/d3c/d76/d9d/dba/fc0 0 2024-05-15T15:54:20.941 INFO:tasks.workunit.client.1.smithi144.stdout:7/732: fdatasync d4/d3c/d76/d9d/dac/d48/fb9 0 2024-05-15T15:54:20.942 INFO:tasks.workunit.client.1.smithi144.stdout:6/753: mknod d7/db/da2/d70/c10c 0 2024-05-15T15:54:20.947 INFO:tasks.workunit.client.1.smithi144.stdout:3/733: rmdir d0/d1/d17 39 2024-05-15T15:54:20.949 INFO:tasks.workunit.client.1.smithi144.stdout:2/680: sync 2024-05-15T15:54:20.953 INFO:tasks.workunit.client.1.smithi144.stdout:9/763: dwrite d0/d8/d5e/d44/dc2/f45 [4194304,4194304] 0 2024-05-15T15:54:20.953 INFO:tasks.workunit.client.1.smithi144.stdout:4/734: link d6/d42/d46/d4b/f31 d6/dd/d2e/d71/fea 0 2024-05-15T15:54:20.957 INFO:tasks.workunit.client.1.smithi144.stdout:0/703: rename d4/d6/d3a/d4c/lbd to d4/d2d/d48/d5b/dda/lfc 0 2024-05-15T15:54:20.958 INFO:tasks.workunit.client.1.smithi144.stdout:0/704: write d4/d6/d3a/ff1 [644291,72720] 0 2024-05-15T15:54:20.958 INFO:tasks.workunit.client.0.smithi049.stdout:4/637: dwrite d18/f31 [0,4194304] 0 2024-05-15T15:54:20.965 INFO:tasks.workunit.client.1.smithi144.stdout:5/762: mkdir d0/dc/d1f/d50/dfe 0 2024-05-15T15:54:20.969 INFO:tasks.workunit.client.1.smithi144.stdout:8/787: mkdir d2/de/d3f/dff 0 2024-05-15T15:54:20.970 INFO:tasks.workunit.client.1.smithi144.stdout:8/788: fsync d2/d24/d2d/da0/d6e/d71/ff6 0 2024-05-15T15:54:20.974 INFO:tasks.workunit.client.1.smithi144.stdout:0/705: rename d4/df/d1f/d24/d85/da1/da3/cf2 to d4/d6/d4a/d4d/db1/cfd 0 2024-05-15T15:54:20.977 INFO:tasks.workunit.client.1.smithi144.stdout:6/754: truncate d7/db/d11/f42 1638841 0 2024-05-15T15:54:20.978 INFO:tasks.workunit.client.1.smithi144.stdout:6/755: chown d7/db/d9d/d72/dbd 1 1 2024-05-15T15:54:20.979 INFO:tasks.workunit.client.1.smithi144.stdout:5/763: chown d0/dc/d24/d28/d2d/d59/d74/cd1 24753623 1 2024-05-15T15:54:20.980 INFO:tasks.workunit.client.1.smithi144.stdout:5/764: write d0/dc/d12/fd7 [796438,97942] 0 2024-05-15T15:54:20.981 INFO:tasks.workunit.client.1.smithi144.stdout:3/734: dread - d0/d1/d17/da1/fc5 zero size 2024-05-15T15:54:20.987 INFO:tasks.workunit.client.1.smithi144.stdout:9/764: mknod d0/d8/d1b/d62/d7d/dd2/c107 0 2024-05-15T15:54:20.990 INFO:tasks.workunit.client.1.smithi144.stdout:0/706: rmdir d4/d6/d3a/d40 39 2024-05-15T15:54:20.992 INFO:tasks.workunit.client.1.smithi144.stdout:8/789: dread d2/d24/d2d/d30/dd0/df7/f95 [0,4194304] 0 2024-05-15T15:54:20.993 INFO:tasks.workunit.client.1.smithi144.stdout:6/756: mknod d7/db/d11/d3c/d61/dcc/c10d 0 2024-05-15T15:54:20.995 INFO:tasks.workunit.client.1.smithi144.stdout:5/765: symlink d0/dc/d24/d28/d2d/d59/lff 0 2024-05-15T15:54:20.998 INFO:tasks.workunit.client.1.smithi144.stdout:7/733: dwrite d4/d3c/fa3 [0,4194304] 0 2024-05-15T15:54:20.999 INFO:tasks.workunit.client.1.smithi144.stdout:3/735: fsync d0/d1/d17/f33 0 2024-05-15T15:54:21.000 INFO:tasks.workunit.client.1.smithi144.stdout:3/736: truncate d0/f15 4950118 0 2024-05-15T15:54:21.000 INFO:tasks.workunit.client.1.smithi144.stdout:3/737: fdatasync d0/d11/d3b/da0/d8f/fe0 0 2024-05-15T15:54:21.001 INFO:tasks.workunit.client.1.smithi144.stdout:2/681: rmdir d3/d7/d20/d37/d76/db5/db6 39 2024-05-15T15:54:21.001 INFO:tasks.workunit.client.1.smithi144.stdout:3/738: chown d0/d1/fd9 10139996 1 2024-05-15T15:54:21.003 INFO:tasks.workunit.client.1.smithi144.stdout:1/711: link d7/d1d/d33/cc6 d7/df/d89/dcc/ce5 0 2024-05-15T15:54:21.012 INFO:tasks.workunit.client.0.smithi049.stdout:9/548: sync 2024-05-15T15:54:21.013 INFO:tasks.workunit.client.1.smithi144.stdout:8/790: creat d2/d24/d2d/d30/dd0/df7/d58/f100 x:0 0 0 2024-05-15T15:54:21.015 INFO:tasks.workunit.client.1.smithi144.stdout:6/757: truncate d7/db/d5f/f65 1245314 0 2024-05-15T15:54:21.018 INFO:tasks.workunit.client.1.smithi144.stdout:5/766: rmdir d0/dc/d24/d28/d2d/d59/d74 39 2024-05-15T15:54:21.024 INFO:tasks.workunit.client.0.smithi049.stdout:5/729: mkdir dc/dd/d10/df1 0 2024-05-15T15:54:21.026 INFO:tasks.workunit.client.0.smithi049.stdout:7/626: rmdir d0/d34 39 2024-05-15T15:54:21.028 INFO:tasks.workunit.client.1.smithi144.stdout:1/712: symlink d7/d1d/d33/d41/d78/le6 0 2024-05-15T15:54:21.029 INFO:tasks.workunit.client.0.smithi049.stdout:0/676: truncate dc/d13/d14/d25/d31/f58 401800 0 2024-05-15T15:54:21.030 INFO:tasks.workunit.client.1.smithi144.stdout:4/735: getdents d6/dd/d2e 0 2024-05-15T15:54:21.031 INFO:tasks.workunit.client.1.smithi144.stdout:4/736: write d6/dd/d2d/f48 [5174603,79670] 0 2024-05-15T15:54:21.031 INFO:tasks.workunit.client.0.smithi049.stdout:6/645: unlink dd/d17/d24/d8a/f97 0 2024-05-15T15:54:21.032 INFO:tasks.workunit.client.1.smithi144.stdout:9/765: mknod d0/d8/d5e/d44/d77/c108 0 2024-05-15T15:54:21.035 INFO:tasks.workunit.client.1.smithi144.stdout:0/707: creat d4/d6/d4a/d74/de4/ffe x:0 0 0 2024-05-15T15:54:21.036 INFO:tasks.workunit.client.0.smithi049.stdout:1/844: truncate d0/d4/dd/de/d4e/f76 1095668 0 2024-05-15T15:54:21.037 INFO:tasks.workunit.client.1.smithi144.stdout:4/737: dwrite d6/fe7 [0,4194304] 0 2024-05-15T15:54:21.038 INFO:tasks.workunit.client.1.smithi144.stdout:4/738: dread - d6/d49/fe8 zero size 2024-05-15T15:54:21.043 INFO:tasks.workunit.client.1.smithi144.stdout:6/758: rename d7/db/da2/d70/c10c to d7/db/d9d/d72/c10e 0 2024-05-15T15:54:21.043 INFO:tasks.workunit.client.1.smithi144.stdout:6/759: stat d7/db/l4e 0 2024-05-15T15:54:21.044 INFO:tasks.workunit.client.1.smithi144.stdout:6/760: chown d7/db/c52 959 1 2024-05-15T15:54:21.049 INFO:tasks.workunit.client.0.smithi049.stdout:2/619: dwrite d3/db/d1d/d1e/d2b/d2e/f72 [0,4194304] 0 2024-05-15T15:54:21.050 INFO:tasks.workunit.client.0.smithi049.stdout:9/549: rename d0/d8/d53/f80 to d0/d8/d1b/fbe 0 2024-05-15T15:54:21.053 INFO:tasks.workunit.client.0.smithi049.stdout:9/550: write d0/d8/d9/f57 [681270,101123] 0 2024-05-15T15:54:21.054 INFO:tasks.workunit.client.0.smithi049.stdout:9/551: fdatasync d0/d8e/fb9 0 2024-05-15T15:54:21.061 INFO:tasks.workunit.client.0.smithi049.stdout:5/730: fsync dc/dd/d47/d83/dcc/d5c/f7e 0 2024-05-15T15:54:21.063 INFO:tasks.workunit.client.0.smithi049.stdout:4/638: dwrite d18/d1e/d2a/d36/d5d/f67 [0,4194304] 0 2024-05-15T15:54:21.064 INFO:tasks.workunit.client.0.smithi049.stdout:7/627: mkdir d0/d6/d23/d4a/d7b/dc4 0 2024-05-15T15:54:21.065 INFO:tasks.workunit.client.0.smithi049.stdout:3/644: truncate f9 3707221 0 2024-05-15T15:54:21.068 INFO:tasks.workunit.client.1.smithi144.stdout:2/682: truncate d3/d7/d20/f7c 3633186 0 2024-05-15T15:54:21.069 INFO:tasks.workunit.client.0.smithi049.stdout:8/636: sync 2024-05-15T15:54:21.070 INFO:tasks.workunit.client.1.smithi144.stdout:9/766: rmdir d0/d8/d1b/d79/de7 39 2024-05-15T15:54:21.073 INFO:tasks.workunit.client.1.smithi144.stdout:1/713: dwrite d7/d1d/d33/d1e/fe4 [0,4194304] 0 2024-05-15T15:54:21.073 INFO:tasks.workunit.client.0.smithi049.stdout:7/628: dwrite d0/fb9 [0,4194304] 0 2024-05-15T15:54:21.079 INFO:tasks.workunit.client.0.smithi049.stdout:7/629: dread d0/d6/d23/d45/d6a/f9d [0,4194304] 0 2024-05-15T15:54:21.083 INFO:tasks.workunit.client.0.smithi049.stdout:0/677: rename dc/d13/d2b/c35 to dc/d13/d14/d2f/d8f/cdd 0 2024-05-15T15:54:21.087 INFO:tasks.workunit.client.1.smithi144.stdout:4/739: rename d6/dd/d2d to d6/d7c/deb 0 2024-05-15T15:54:21.088 INFO:tasks.workunit.client.1.smithi144.stdout:4/740: write d6/d42/d46/d4b/fdb [1009075,17110] 0 2024-05-15T15:54:21.092 INFO:tasks.workunit.client.0.smithi049.stdout:9/552: dread - d0/d8/d1b/f94 zero size 2024-05-15T15:54:21.093 INFO:tasks.workunit.client.1.smithi144.stdout:9/767: dread d0/d8/d5e/d44/dc2/f61 [0,4194304] 0 2024-05-15T15:54:21.094 INFO:tasks.workunit.client.1.smithi144.stdout:5/767: symlink d0/dc/d2f/d68/d5c/l100 0 2024-05-15T15:54:21.094 INFO:tasks.workunit.client.0.smithi049.stdout:9/553: dread - d0/d8/d1b/d40/fbc zero size 2024-05-15T15:54:21.095 INFO:tasks.workunit.client.0.smithi049.stdout:9/554: chown d0/d8/d21/fae 1 1 2024-05-15T15:54:21.097 INFO:tasks.workunit.client.1.smithi144.stdout:7/734: creat d4/fff x:0 0 0 2024-05-15T15:54:21.099 INFO:tasks.workunit.client.1.smithi144.stdout:0/708: dwrite d4/f22 [0,4194304] 0 2024-05-15T15:54:21.104 INFO:tasks.workunit.client.0.smithi049.stdout:4/639: mknod d18/d85/db6/cd0 0 2024-05-15T15:54:21.105 INFO:tasks.workunit.client.0.smithi049.stdout:4/640: readlink d18/d1e/d2a/d36/d86/la1 0 2024-05-15T15:54:21.106 INFO:tasks.workunit.client.1.smithi144.stdout:2/683: creat d3/d7/d20/d37/d76/fe5 x:0 0 0 2024-05-15T15:54:21.115 INFO:tasks.workunit.client.0.smithi049.stdout:1/845: creat d0/d4/dd/de/d1e/d16/df1/d103/f127 x:0 0 0 2024-05-15T15:54:21.117 INFO:tasks.workunit.client.0.smithi049.stdout:1/846: read d0/d4/dd/f18 [2135003,83717] 0 2024-05-15T15:54:21.122 INFO:tasks.workunit.client.1.smithi144.stdout:6/761: getdents d7/db/d11/d7e 0 2024-05-15T15:54:21.123 INFO:tasks.workunit.client.0.smithi049.stdout:5/731: dwrite dc/dd/d47/d83/dcc/f7a [0,4194304] 0 2024-05-15T15:54:21.124 INFO:tasks.workunit.client.1.smithi144.stdout:4/741: creat d6/dd/d34/d3b/d30/db/d1d/d40/fec x:0 0 0 2024-05-15T15:54:21.126 INFO:tasks.workunit.client.0.smithi049.stdout:0/678: symlink dc/d13/d3d/lde 0 2024-05-15T15:54:21.127 INFO:tasks.workunit.client.0.smithi049.stdout:0/679: dread - dc/d13/d2b/fdb zero size 2024-05-15T15:54:21.129 INFO:tasks.workunit.client.1.smithi144.stdout:4/742: dread d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/fe4 [0,4194304] 0 2024-05-15T15:54:21.129 INFO:tasks.workunit.client.0.smithi049.stdout:3/645: dwrite d12/d14/d32/f6d [0,4194304] 0 2024-05-15T15:54:21.130 INFO:tasks.workunit.client.1.smithi144.stdout:9/768: chown d0/d8/d50/c55 14 1 2024-05-15T15:54:21.131 INFO:tasks.workunit.client.0.smithi049.stdout:0/680: fsync dc/d13/d14/d25/d31/d71/f3a 0 2024-05-15T15:54:21.135 INFO:tasks.workunit.client.1.smithi144.stdout:7/735: fdatasync d4/d41/f5a 0 2024-05-15T15:54:21.138 INFO:tasks.workunit.client.1.smithi144.stdout:3/739: link d0/d1/d6c/ca4 d0/d11/d24/cf0 0 2024-05-15T15:54:21.138 INFO:tasks.workunit.client.0.smithi049.stdout:4/641: symlink d18/d1e/d63/ld1 0 2024-05-15T15:54:21.138 INFO:tasks.workunit.client.0.smithi049.stdout:4/642: chown fc 6081 1 2024-05-15T15:54:21.139 INFO:tasks.workunit.client.1.smithi144.stdout:3/740: fsync d0/d1/d17/d39/d83/f9d 0 2024-05-15T15:54:21.140 INFO:tasks.workunit.client.1.smithi144.stdout:0/709: rename d4/df/d1f/d24/d85/da1/cf7 to d4/d6/d3a/d4c/dd6/d55/cff 0 2024-05-15T15:54:21.143 INFO:tasks.workunit.client.1.smithi144.stdout:1/714: truncate d7/d1d/d58/fda 1174975 0 2024-05-15T15:54:21.144 INFO:tasks.workunit.client.0.smithi049.stdout:1/847: dread - d0/d4/dd/de/d48/d5c/f9c zero size 2024-05-15T15:54:21.145 INFO:tasks.workunit.client.1.smithi144.stdout:8/791: link d2/d24/d2d/da0/d9e/db8/fc1 d2/d3a/d7b/f101 0 2024-05-15T15:54:21.146 INFO:tasks.workunit.client.0.smithi049.stdout:8/637: fdatasync d2/de/d71/d86/f51 0 2024-05-15T15:54:21.148 INFO:tasks.workunit.client.1.smithi144.stdout:6/762: symlink d7/db/d9d/l10f 0 2024-05-15T15:54:21.150 INFO:tasks.workunit.client.0.smithi049.stdout:2/620: creat d3/db/d1d/d1e/d2b/fd6 x:0 0 0 2024-05-15T15:54:21.152 INFO:tasks.workunit.client.0.smithi049.stdout:5/732: chown dc/dd/d10/l26 6 1 2024-05-15T15:54:21.154 INFO:tasks.workunit.client.0.smithi049.stdout:5/733: fdatasync dc/d4a/f8c 0 2024-05-15T15:54:21.154 INFO:tasks.workunit.client.1.smithi144.stdout:4/743: creat d6/dd/d34/fed x:0 0 0 2024-05-15T15:54:21.155 INFO:tasks.workunit.client.0.smithi049.stdout:5/734: stat dc/dd/d10/f2a 0 2024-05-15T15:54:21.158 INFO:tasks.workunit.client.0.smithi049.stdout:2/621: dwrite d3/db/d1d/d1e/d2b/d3b/fa7 [0,4194304] 0 2024-05-15T15:54:21.158 INFO:tasks.workunit.client.1.smithi144.stdout:2/684: dread d3/d4d/fae [0,4194304] 0 2024-05-15T15:54:21.161 INFO:tasks.workunit.client.0.smithi049.stdout:9/555: mknod d0/d8/d1b/db3/cbf 0 2024-05-15T15:54:21.165 INFO:tasks.workunit.client.0.smithi049.stdout:6/646: getdents dd/d17/d64 0 2024-05-15T15:54:21.167 INFO:tasks.workunit.client.0.smithi049.stdout:6/647: readlink dd/d17/d24/d2b/d36/d93/lac 0 2024-05-15T15:54:21.167 INFO:tasks.workunit.client.0.smithi049.stdout:6/648: chown dd/d17/f21 13 1 2024-05-15T15:54:21.170 INFO:tasks.workunit.client.0.smithi049.stdout:4/643: mkdir d18/d21/d2b/d4e/dd2 0 2024-05-15T15:54:21.177 INFO:tasks.workunit.client.1.smithi144.stdout:3/741: fsync d0/d11/d3b/f5b 0 2024-05-15T15:54:21.177 INFO:tasks.workunit.client.1.smithi144.stdout:0/710: mkdir d4/d2d/d100 0 2024-05-15T15:54:21.177 INFO:tasks.workunit.client.1.smithi144.stdout:8/792: mknod d2/de/d3f/d6b/c102 0 2024-05-15T15:54:21.179 INFO:tasks.workunit.client.1.smithi144.stdout:6/763: mknod d7/db/d35/d84/c110 0 2024-05-15T15:54:21.181 INFO:tasks.workunit.client.0.smithi049.stdout:5/735: rmdir dc/dd/d7b/d8d 39 2024-05-15T15:54:21.182 INFO:tasks.workunit.client.0.smithi049.stdout:2/622: mknod d3/db/d1d/d1e/d2b/d3b/cd7 0 2024-05-15T15:54:21.183 INFO:tasks.workunit.client.0.smithi049.stdout:2/623: chown d3/f11 24728 1 2024-05-15T15:54:21.184 INFO:tasks.workunit.client.1.smithi144.stdout:9/769: mkdir d0/d8/d5e/dd4/d109 0 2024-05-15T15:54:21.184 INFO:tasks.workunit.client.1.smithi144.stdout:8/793: dwrite d2/d3a/d7b/d4b/fe9 [0,4194304] 0 2024-05-15T15:54:21.186 INFO:tasks.workunit.client.0.smithi049.stdout:3/646: mkdir d12/d30/dc1/dc7 0 2024-05-15T15:54:21.188 INFO:tasks.workunit.client.1.smithi144.stdout:2/685: mknod d3/d7/d20/d22/d52/d64/ce6 0 2024-05-15T15:54:21.189 INFO:tasks.workunit.client.0.smithi049.stdout:9/556: symlink d0/d8/d1b/d40/lc0 0 2024-05-15T15:54:21.190 INFO:tasks.workunit.client.0.smithi049.stdout:9/557: write d0/d25/d5d/f8d [627876,116696] 0 2024-05-15T15:54:21.191 INFO:tasks.workunit.client.0.smithi049.stdout:0/681: link dc/d13/d2b/f98 dc/d13/d56/fdf 0 2024-05-15T15:54:21.191 INFO:tasks.workunit.client.1.smithi144.stdout:8/794: dwrite d2/d24/d2d/da0/d9e/db8/dde/fef [0,4194304] 0 2024-05-15T15:54:21.199 INFO:tasks.workunit.client.0.smithi049.stdout:8/638: dread d2/de/d71/d86/d5d/d7d/db8/f40 [0,4194304] 0 2024-05-15T15:54:21.207 INFO:tasks.workunit.client.1.smithi144.stdout:4/744: dread d6/dd/d34/d3b/f21 [0,4194304] 0 2024-05-15T15:54:21.288 INFO:tasks.workunit.client.0.smithi049.stdout:6/649: mkdir dd/d17/d24/d8a/dc2 0 2024-05-15T15:54:21.294 INFO:tasks.workunit.client.0.smithi049.stdout:2/624: creat d3/db/d1d/d1e/d2b/d2e/d67/fd8 x:0 0 0 2024-05-15T15:54:21.298 INFO:tasks.workunit.client.0.smithi049.stdout:9/558: dread - d0/d8/d21/f82 zero size 2024-05-15T15:54:21.299 INFO:tasks.workunit.client.0.smithi049.stdout:4/644: dread d18/f1b [0,4194304] 0 2024-05-15T15:54:21.301 INFO:tasks.workunit.client.0.smithi049.stdout:0/682: creat dc/d13/d3d/fe0 x:0 0 0 2024-05-15T15:54:21.307 INFO:tasks.workunit.client.1.smithi144.stdout:7/736: dwrite d4/d41/d5b/da0/fe3 [0,4194304] 0 2024-05-15T15:54:21.319 INFO:tasks.workunit.client.0.smithi049.stdout:1/848: write d0/d4/dd/de/d1e/d2e/dad/d65/d72/f8d [252172,68925] 0 2024-05-15T15:54:21.319 INFO:tasks.workunit.client.0.smithi049.stdout:5/736: getdents dc/dd/d10/df1 0 2024-05-15T15:54:21.321 INFO:tasks.workunit.client.0.smithi049.stdout:5/737: chown dc/dd/d7b/le5 10 1 2024-05-15T15:54:21.322 INFO:tasks.workunit.client.0.smithi049.stdout:8/639: write d2/de/d71/d86/d5d/f89 [2969950,119449] 0 2024-05-15T15:54:21.322 INFO:tasks.workunit.client.0.smithi049.stdout:0/683: rename dc/d13/d14/d33/l5c to dc/d13/d3d/le1 0 2024-05-15T15:54:21.324 INFO:tasks.workunit.client.0.smithi049.stdout:6/650: creat dd/d17/d9d/da1/fc3 x:0 0 0 2024-05-15T15:54:21.326 INFO:tasks.workunit.client.0.smithi049.stdout:2/625: dread - d3/db/d5b/dbf/dc1/fa5 zero size 2024-05-15T15:54:21.327 INFO:tasks.workunit.client.0.smithi049.stdout:4/645: symlink d18/d1e/d2a/d36/ld3 0 2024-05-15T15:54:21.328 INFO:tasks.workunit.client.0.smithi049.stdout:4/646: truncate d18/d1e/fa6 1699277 0 2024-05-15T15:54:21.332 INFO:tasks.workunit.client.0.smithi049.stdout:5/738: rename dc/dd/d7b/dd2 to dc/db0/df2 0 2024-05-15T15:54:21.333 INFO:tasks.workunit.client.0.smithi049.stdout:1/849: mknod d0/d4/dd/de/d1e/d2e/d3e/d90/db3/c128 0 2024-05-15T15:54:21.339 INFO:tasks.workunit.client.0.smithi049.stdout:2/626: truncate d3/db/d5b/f74 341613 0 2024-05-15T15:54:21.345 INFO:tasks.workunit.client.0.smithi049.stdout:0/684: mkdir dc/d13/d14/d2f/d8f/dcb/de2 0 2024-05-15T15:54:21.347 INFO:tasks.workunit.client.1.smithi144.stdout:1/715: mkdir d7/d1d/d33/de7 0 2024-05-15T15:54:21.347 INFO:tasks.workunit.client.0.smithi049.stdout:6/651: mknod dd/d26/d5b/da8/cc4 0 2024-05-15T15:54:21.348 INFO:tasks.workunit.client.1.smithi144.stdout:1/716: write d7/d1d/d33/d29/d6b/d5e/fa1 [3744559,130209] 0 2024-05-15T15:54:21.349 INFO:tasks.workunit.client.0.smithi049.stdout:6/652: fsync dd/d17/d24/d2b/f46 0 2024-05-15T15:54:21.349 INFO:tasks.workunit.client.1.smithi144.stdout:3/742: fdatasync d0/d11/d24/f88 0 2024-05-15T15:54:21.352 INFO:tasks.workunit.client.0.smithi049.stdout:9/559: getdents d0/d8 0 2024-05-15T15:54:21.352 INFO:tasks.workunit.client.1.smithi144.stdout:0/711: rmdir d4/d2d/d48/d5b/dd3 39 2024-05-15T15:54:21.355 INFO:tasks.workunit.client.1.smithi144.stdout:6/764: write d7/db/d3a/f91 [5242803,30096] 0 2024-05-15T15:54:21.355 INFO:tasks.workunit.client.0.smithi049.stdout:6/653: dread f4 [0,4194304] 0 2024-05-15T15:54:21.355 INFO:tasks.workunit.client.0.smithi049.stdout:1/850: dread d0/d4/dd/de/d1e/d16/d8a/f9b [0,4194304] 0 2024-05-15T15:54:21.357 INFO:tasks.workunit.client.0.smithi049.stdout:6/654: chown dd/l1a 120 1 2024-05-15T15:54:21.359 INFO:tasks.workunit.client.0.smithi049.stdout:6/655: readlink dd/d17/d24/d2b/lb0 0 2024-05-15T15:54:21.359 INFO:tasks.workunit.client.0.smithi049.stdout:0/685: write f3 [977372,129118] 0 2024-05-15T15:54:21.359 INFO:tasks.workunit.client.1.smithi144.stdout:9/770: creat d0/d8/d1b/d79/d8a/dae/f10a x:0 0 0 2024-05-15T15:54:21.361 INFO:tasks.workunit.client.0.smithi049.stdout:2/627: dwrite d3/db/f12 [0,4194304] 0 2024-05-15T15:54:21.362 INFO:tasks.workunit.client.1.smithi144.stdout:2/686: stat d3/f9 0 2024-05-15T15:54:21.363 INFO:tasks.workunit.client.1.smithi144.stdout:1/717: dread d7/d1d/d33/d41/d78/da3/f88 [0,4194304] 0 2024-05-15T15:54:21.363 INFO:tasks.workunit.client.1.smithi144.stdout:1/718: readlink d7/d1d/d33/d1e/l2d 0 2024-05-15T15:54:21.365 INFO:tasks.workunit.client.1.smithi144.stdout:5/768: getdents d0/dc/d1f 0 2024-05-15T15:54:21.368 INFO:tasks.workunit.client.0.smithi049.stdout:8/640: dwrite d2/fb [0,4194304] 0 2024-05-15T15:54:21.369 INFO:tasks.workunit.client.0.smithi049.stdout:9/560: creat d0/d8e/fc1 x:0 0 0 2024-05-15T15:54:21.369 INFO:tasks.workunit.client.0.smithi049.stdout:8/641: chown d2/de/d25/d8c 66116338 1 2024-05-15T15:54:21.375 INFO:tasks.workunit.client.0.smithi049.stdout:1/851: mkdir d0/d4/dd/de/d48/d129 0 2024-05-15T15:54:21.375 INFO:tasks.workunit.client.0.smithi049.stdout:1/852: chown d0/fd1 0 1 2024-05-15T15:54:21.376 INFO:tasks.workunit.client.0.smithi049.stdout:0/686: symlink dc/d6d/le3 0 2024-05-15T15:54:21.378 INFO:tasks.workunit.client.1.smithi144.stdout:7/737: fsync d4/d41/d5b/f70 0 2024-05-15T15:54:21.378 INFO:tasks.workunit.client.0.smithi049.stdout:1/853: write d0/d4/dd/de/d1e/d2e/dad/d65/fc7 [3067267,89574] 0 2024-05-15T15:54:21.379 INFO:tasks.workunit.client.1.smithi144.stdout:7/738: fsync d4/df/f32 0 2024-05-15T15:54:21.379 INFO:tasks.workunit.client.0.smithi049.stdout:6/656: creat dd/d17/d24/d2b/fc5 x:0 0 0 2024-05-15T15:54:21.381 INFO:tasks.workunit.client.0.smithi049.stdout:6/657: dread - dd/d17/d64/fb5 zero size 2024-05-15T15:54:21.382 INFO:tasks.workunit.client.0.smithi049.stdout:2/628: mkdir d3/db/d1d/d1e/dd9 0 2024-05-15T15:54:21.388 INFO:tasks.workunit.client.0.smithi049.stdout:9/561: mknod d0/d8/d1b/d71/d87/cc2 0 2024-05-15T15:54:21.390 INFO:tasks.workunit.client.1.smithi144.stdout:3/743: rmdir d0/d11/d3b 39 2024-05-15T15:54:21.393 INFO:tasks.workunit.client.1.smithi144.stdout:0/712: unlink d4/df/d1f/d24/f69 0 2024-05-15T15:54:21.396 INFO:tasks.workunit.client.1.smithi144.stdout:6/765: read d7/db/d11/f43 [1076282,129466] 0 2024-05-15T15:54:21.397 INFO:tasks.workunit.client.0.smithi049.stdout:0/687: creat dc/d13/d14/d25/d31/d71/dbf/fe4 x:0 0 0 2024-05-15T15:54:21.401 INFO:tasks.workunit.client.1.smithi144.stdout:1/719: rmdir d7/d1d/d33/d29 39 2024-05-15T15:54:21.401 INFO:tasks.workunit.client.1.smithi144.stdout:6/766: dwrite d7/db/d11/fa5 [0,4194304] 0 2024-05-15T15:54:21.402 INFO:tasks.workunit.client.1.smithi144.stdout:1/720: chown d7/d1d/d33/d41/d78/da3/c47 2574 1 2024-05-15T15:54:21.410 INFO:tasks.workunit.client.0.smithi049.stdout:8/642: mkdir d2/de/d71/d86/d5d/d7d/d1b/dc8/dc9 0 2024-05-15T15:54:21.411 INFO:tasks.workunit.client.0.smithi049.stdout:4/647: link d18/d1e/d2a/d50/l75 d18/d21/ld4 0 2024-05-15T15:54:21.413 INFO:tasks.workunit.client.1.smithi144.stdout:7/739: mkdir d4/df/d79/d100 0 2024-05-15T15:54:21.413 INFO:tasks.workunit.client.1.smithi144.stdout:2/687: dwrite d3/d7/f61 [0,4194304] 0 2024-05-15T15:54:21.414 INFO:tasks.workunit.client.0.smithi049.stdout:0/688: truncate dc/d13/d56/f90 5143015 0 2024-05-15T15:54:21.418 INFO:tasks.workunit.client.0.smithi049.stdout:6/658: mknod dd/d17/d24/d2b/d36/d86/cc6 0 2024-05-15T15:54:21.421 INFO:tasks.workunit.client.0.smithi049.stdout:2/629: creat d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/dc9/fda x:0 0 0 2024-05-15T15:54:21.422 INFO:tasks.workunit.client.1.smithi144.stdout:0/713: mknod d4/d6/d3a/d4c/dd6/d55/da5/c101 0 2024-05-15T15:54:21.424 INFO:tasks.workunit.client.1.smithi144.stdout:9/771: creat d0/d8/d5e/d44/d99/f10b x:0 0 0 2024-05-15T15:54:21.425 INFO:tasks.workunit.client.1.smithi144.stdout:3/744: dread d0/d1/d17/d39/d40/d4a/f8e [0,4194304] 0 2024-05-15T15:54:21.427 INFO:tasks.workunit.client.1.smithi144.stdout:3/745: write d0/d1/d17/fd8 [856085,97494] 0 2024-05-15T15:54:21.427 INFO:tasks.workunit.client.0.smithi049.stdout:1/854: link d0/d4/d63/f70 d0/d4/dd/de/d1e/f12a 0 2024-05-15T15:54:21.428 INFO:tasks.workunit.client.0.smithi049.stdout:1/855: chown d0/d4/dd/de/d48/d5c 598078 1 2024-05-15T15:54:21.430 INFO:tasks.workunit.client.1.smithi144.stdout:9/772: dread d0/d8/d5e/d44/dc2/d75/fb1 [0,4194304] 0 2024-05-15T15:54:21.430 INFO:tasks.workunit.client.1.smithi144.stdout:6/767: unlink d7/db/d35/f47 0 2024-05-15T15:54:21.431 INFO:tasks.workunit.client.0.smithi049.stdout:4/648: dread d18/f7c [0,4194304] 0 2024-05-15T15:54:21.431 INFO:tasks.workunit.client.0.smithi049.stdout:6/659: mknod dd/d17/d24/d58/cc7 0 2024-05-15T15:54:21.431 INFO:tasks.workunit.client.0.smithi049.stdout:9/562: dread d0/d8/d1b/fbe [0,4194304] 0 2024-05-15T15:54:21.432 INFO:tasks.workunit.client.0.smithi049.stdout:4/649: write d18/d21/f7a [5142879,43833] 0 2024-05-15T15:54:21.433 INFO:tasks.workunit.client.0.smithi049.stdout:8/643: symlink d2/de/d71/d86/d5d/d7d/db8/d5b/da6/dc7/lca 0 2024-05-15T15:54:21.434 INFO:tasks.workunit.client.0.smithi049.stdout:8/644: stat d2/f2c 0 2024-05-15T15:54:21.435 INFO:tasks.workunit.client.1.smithi144.stdout:5/769: truncate d0/dc/d12/f18 4995244 0 2024-05-15T15:54:21.435 INFO:tasks.workunit.client.1.smithi144.stdout:9/773: dwrite d0/f5 [0,4194304] 0 2024-05-15T15:54:21.438 INFO:tasks.workunit.client.0.smithi049.stdout:1/856: creat d0/d4/dd/de/d4d/db8/f12b x:0 0 0 2024-05-15T15:54:21.439 INFO:tasks.workunit.client.1.smithi144.stdout:1/721: dwrite d7/d1d/d33/d29/d6b/d5e/f9d [0,4194304] 0 2024-05-15T15:54:21.439 INFO:tasks.workunit.client.1.smithi144.stdout:4/745: link d6/dd/d2e/c37 d6/dd/dc3/dcc/cee 0 2024-05-15T15:54:21.440 INFO:tasks.workunit.client.1.smithi144.stdout:4/746: fsync d6/dd/d34/d3b/d30/db/d1d/d40/fec 0 2024-05-15T15:54:21.446 INFO:tasks.workunit.client.0.smithi049.stdout:9/563: fsync d0/d8/f39 0 2024-05-15T15:54:21.454 INFO:tasks.workunit.client.1.smithi144.stdout:2/688: unlink d3/d7/f47 0 2024-05-15T15:54:21.456 INFO:tasks.workunit.client.0.smithi049.stdout:0/689: dwrite dc/d32/f96 [0,4194304] 0 2024-05-15T15:54:21.456 INFO:tasks.workunit.client.1.smithi144.stdout:1/722: dread d7/d1d/d33/d41/d78/da3/f54 [0,4194304] 0 2024-05-15T15:54:21.458 INFO:tasks.workunit.client.0.smithi049.stdout:8/645: stat d2/c9 0 2024-05-15T15:54:21.459 INFO:tasks.workunit.client.0.smithi049.stdout:8/646: write d2/f2c [1392384,104906] 0 2024-05-15T15:54:21.462 INFO:tasks.workunit.client.0.smithi049.stdout:8/647: truncate d2/de/d71/d86/d5d/d7d/d1b/d68/f97 715603 0 2024-05-15T15:54:21.466 INFO:tasks.workunit.client.0.smithi049.stdout:6/660: symlink dd/d17/d24/d2b/d36/d86/lc8 0 2024-05-15T15:54:21.468 INFO:tasks.workunit.client.0.smithi049.stdout:6/661: truncate dd/d17/d9d/fa6 356594 0 2024-05-15T15:54:21.469 INFO:tasks.workunit.client.0.smithi049.stdout:6/662: truncate dd/f8e 997242 0 2024-05-15T15:54:21.470 INFO:tasks.workunit.client.0.smithi049.stdout:0/690: rmdir dc/d13/d14/d25/d31/d71/dbf 39 2024-05-15T15:54:21.473 INFO:tasks.workunit.client.1.smithi144.stdout:8/795: rename d2/d3a/d7b/d4b/f61 to d2/de/f103 0 2024-05-15T15:54:21.474 INFO:tasks.workunit.client.1.smithi144.stdout:8/796: fdatasync d2/d24/d2d/da0/d6e/f66 0 2024-05-15T15:54:21.474 INFO:tasks.workunit.client.1.smithi144.stdout:5/770: creat d0/dc/d24/d8e/d95/f101 x:0 0 0 2024-05-15T15:54:21.475 INFO:tasks.workunit.client.1.smithi144.stdout:8/797: chown d2/d24/d2d/d30/dd0/df7/f48 369 1 2024-05-15T15:54:21.475 INFO:tasks.workunit.client.1.smithi144.stdout:8/798: readlink d2/l34 0 2024-05-15T15:54:21.476 INFO:tasks.workunit.client.1.smithi144.stdout:4/747: mknod d6/d7c/deb/cef 0 2024-05-15T15:54:21.477 INFO:tasks.workunit.client.1.smithi144.stdout:4/748: write d6/dd/d34/fed [307768,45679] 0 2024-05-15T15:54:21.480 INFO:tasks.workunit.client.0.smithi049.stdout:8/648: unlink d2/de/d71/d86/d5d/d7d/d1b/l49 0 2024-05-15T15:54:21.481 INFO:tasks.workunit.client.1.smithi144.stdout:2/689: symlink d3/d7/d54/d98/le7 0 2024-05-15T15:54:21.481 INFO:tasks.workunit.client.0.smithi049.stdout:8/649: read - d2/de/d25/d74/fae zero size 2024-05-15T15:54:21.483 INFO:tasks.workunit.client.1.smithi144.stdout:9/774: write d0/d8/f11 [4840936,69603] 0 2024-05-15T15:54:21.483 INFO:tasks.workunit.client.0.smithi049.stdout:4/650: rmdir d18/d85/dae 0 2024-05-15T15:54:21.487 INFO:tasks.workunit.client.1.smithi144.stdout:5/771: dread d0/dc/d12/d4a/f60 [0,4194304] 0 2024-05-15T15:54:21.488 INFO:tasks.workunit.client.1.smithi144.stdout:1/723: truncate d7/d1d/d33/d1e/d4e/fb6 150818 0 2024-05-15T15:54:21.488 INFO:tasks.workunit.client.0.smithi049.stdout:1/857: fsync d0/d4/dd/de/d1e/d2e/f88 0 2024-05-15T15:54:21.493 INFO:tasks.workunit.client.1.smithi144.stdout:0/714: truncate d4/d6/d4a/d74/f99 563386 0 2024-05-15T15:54:21.498 INFO:tasks.workunit.client.1.smithi144.stdout:8/799: rename d2/d1a/fc8 to d2/d24/d2d/d30/f104 0 2024-05-15T15:54:21.506 INFO:tasks.workunit.client.1.smithi144.stdout:3/746: symlink d0/d1/d56/d71/d73/dd1/lf1 0 2024-05-15T15:54:21.506 INFO:tasks.workunit.client.1.smithi144.stdout:5/772: read d0/dc/d12/f18 [4044566,23214] 0 2024-05-15T15:54:21.507 INFO:tasks.workunit.client.1.smithi144.stdout:8/800: symlink d2/d24/d2d/d30/l105 0 2024-05-15T15:54:21.509 INFO:tasks.workunit.client.1.smithi144.stdout:7/740: getdents d4/d41/d5b 0 2024-05-15T15:54:21.511 INFO:tasks.workunit.client.1.smithi144.stdout:7/741: write d4/d3c/f57 [2711073,120676] 0 2024-05-15T15:54:21.512 INFO:tasks.workunit.client.1.smithi144.stdout:2/690: creat d3/d4d/db4/fe8 x:0 0 0 2024-05-15T15:54:21.514 INFO:tasks.workunit.client.1.smithi144.stdout:0/715: mkdir d4/df/d102 0 2024-05-15T15:54:21.516 INFO:tasks.workunit.client.1.smithi144.stdout:4/749: getdents d6/d42/d46/d4b/d1e/d99/dab/dd5 0 2024-05-15T15:54:21.516 INFO:tasks.workunit.client.1.smithi144.stdout:8/801: dwrite d2/d1a/f3d [0,4194304] 0 2024-05-15T15:54:21.518 INFO:tasks.workunit.client.1.smithi144.stdout:1/724: creat d7/d4b/dd9/dd3/fe8 x:0 0 0 2024-05-15T15:54:21.518 INFO:tasks.workunit.client.0.smithi049.stdout:9/564: write d0/d8/d1b/f28 [1004139,19244] 0 2024-05-15T15:54:21.519 INFO:tasks.workunit.client.1.smithi144.stdout:1/725: chown d7/df/d89 12 1 2024-05-15T15:54:21.520 INFO:tasks.workunit.client.0.smithi049.stdout:4/651: truncate d18/d1e/d2a/d36/f83 411697 0 2024-05-15T15:54:21.520 INFO:tasks.workunit.client.1.smithi144.stdout:2/691: truncate d3/d7/d20/d22/d52/d75/d65/fc8 1390581 0 2024-05-15T15:54:21.520 INFO:tasks.workunit.client.1.smithi144.stdout:9/775: dread d0/f7 [0,4194304] 0 2024-05-15T15:54:21.521 INFO:tasks.workunit.client.0.smithi049.stdout:1/858: rename d0/d4/dd/de/d36/f53 to d0/d4/dd/de/d1e/d2e/d3e/d90/db3/f12c 0 2024-05-15T15:54:21.521 INFO:tasks.workunit.client.0.smithi049.stdout:8/650: dread d2/f3 [0,4194304] 0 2024-05-15T15:54:21.523 INFO:tasks.workunit.client.1.smithi144.stdout:7/742: truncate d4/df/d25/d38/f7f 237099 0 2024-05-15T15:54:21.524 INFO:tasks.workunit.client.0.smithi049.stdout:8/651: stat d2/de/d71/d86/d5d/d7d/d45/lc1 0 2024-05-15T15:54:21.525 INFO:tasks.workunit.client.1.smithi144.stdout:3/747: dread d0/d11/f14 [4194304,4194304] 0 2024-05-15T15:54:21.525 INFO:tasks.workunit.client.1.smithi144.stdout:0/716: fsync d4/df/f16 0 2024-05-15T15:54:21.526 INFO:tasks.workunit.client.1.smithi144.stdout:0/717: chown d4/d2d/f44 135046 1 2024-05-15T15:54:21.526 INFO:tasks.workunit.client.0.smithi049.stdout:9/565: symlink d0/d8/d9/lc3 0 2024-05-15T15:54:21.527 INFO:tasks.workunit.client.0.smithi049.stdout:8/652: truncate d2/de/d71/d86/d5d/f80 4420501 0 2024-05-15T15:54:21.528 INFO:tasks.workunit.client.0.smithi049.stdout:9/566: dread - d0/d8/d1b/d71/d7d/fa3 zero size 2024-05-15T15:54:21.532 INFO:tasks.workunit.client.1.smithi144.stdout:5/773: getdents d0/dc/d2f/d68/d5c 0 2024-05-15T15:54:21.535 INFO:tasks.workunit.client.1.smithi144.stdout:2/692: rename d3/d7/d20/d40/l59 to d3/d7/d20/d37/de1/le9 0 2024-05-15T15:54:21.537 INFO:tasks.workunit.client.1.smithi144.stdout:7/743: unlink d4/d3c/d76/f9e 0 2024-05-15T15:54:21.537 INFO:tasks.workunit.client.1.smithi144.stdout:1/726: dread d7/d1d/d33/d1e/d34/f4a [0,4194304] 0 2024-05-15T15:54:21.540 INFO:tasks.workunit.client.0.smithi049.stdout:1/859: mkdir d0/d4/dd/de/d1e/d2e/dad/d65/d72/d12d 0 2024-05-15T15:54:21.542 INFO:tasks.workunit.client.0.smithi049.stdout:1/860: write d0/d4/dd/de/d4d/d8b/de8/f104 [289920,22457] 0 2024-05-15T15:54:21.542 INFO:tasks.workunit.client.0.smithi049.stdout:0/691: getdents dc/d13/d14/d25/d31/dae 0 2024-05-15T15:54:21.545 INFO:tasks.workunit.client.1.smithi144.stdout:8/802: dread d2/de/d49/f69 [0,4194304] 0 2024-05-15T15:54:21.545 INFO:tasks.workunit.client.1.smithi144.stdout:1/727: dwrite d7/d1d/d33/d41/d22/d45/fb3 [0,4194304] 0 2024-05-15T15:54:21.547 INFO:tasks.workunit.client.0.smithi049.stdout:9/567: rename d0/d8/d53/f68 to d0/d8/d9/d52/d5e/d8a/dac/fc4 0 2024-05-15T15:54:21.548 INFO:tasks.workunit.client.0.smithi049.stdout:9/568: fsync d0/d8/d9/f57 0 2024-05-15T15:54:21.549 INFO:tasks.workunit.client.0.smithi049.stdout:9/569: chown d0/d8/d9/d52/d5e 20 1 2024-05-15T15:54:21.551 INFO:tasks.workunit.client.1.smithi144.stdout:4/750: dread d6/dd/d2e/d71/f8e [0,4194304] 0 2024-05-15T15:54:21.552 INFO:tasks.workunit.client.0.smithi049.stdout:4/652: symlink d18/d1e/d2a/d50/dc4/ld5 0 2024-05-15T15:54:21.552 INFO:tasks.workunit.client.1.smithi144.stdout:1/728: dwrite d7/d1d/d33/d41/d22/d45/fb3 [4194304,4194304] 0 2024-05-15T15:54:21.553 INFO:tasks.workunit.client.0.smithi049.stdout:4/653: truncate d18/f31 5216426 0 2024-05-15T15:54:21.553 INFO:tasks.workunit.client.1.smithi144.stdout:1/729: write d7/d1d/d33/d29/f64 [1899925,17319] 0 2024-05-15T15:54:21.557 INFO:tasks.workunit.client.0.smithi049.stdout:0/692: mknod dc/d13/d2b/d76/d8a/da2/db3/ce5 0 2024-05-15T15:54:21.558 INFO:tasks.workunit.client.1.smithi144.stdout:6/768: sync 2024-05-15T15:54:21.559 INFO:tasks.workunit.client.0.smithi049.stdout:8/653: symlink d2/de/d25/d74/dbe/lcb 0 2024-05-15T15:54:21.560 INFO:tasks.workunit.client.1.smithi144.stdout:6/769: chown d7/d1e/l89 209973 1 2024-05-15T15:54:21.561 INFO:tasks.workunit.client.1.smithi144.stdout:4/751: dwrite d6/dd/d34/d3b/d30/d59/d66/f9e [4194304,4194304] 0 2024-05-15T15:54:21.563 INFO:tasks.workunit.client.1.smithi144.stdout:4/752: dread d6/dd/d34/d3b/fd9 [0,4194304] 0 2024-05-15T15:54:21.565 INFO:tasks.workunit.client.1.smithi144.stdout:3/748: unlink d0/d11/d3b/f6b 0 2024-05-15T15:54:21.567 INFO:tasks.workunit.client.1.smithi144.stdout:6/770: read d7/d62/f95 [2774918,72679] 0 2024-05-15T15:54:21.568 INFO:tasks.workunit.client.1.smithi144.stdout:9/776: creat d0/d8/d1b/d79/de7/df9/f10c x:0 0 0 2024-05-15T15:54:21.568 INFO:tasks.workunit.client.1.smithi144.stdout:8/803: dread - d2/d3a/f91 zero size 2024-05-15T15:54:21.570 INFO:tasks.workunit.client.0.smithi049.stdout:4/654: unlink d18/d1e/d2a/d50/f73 0 2024-05-15T15:54:21.572 INFO:tasks.workunit.client.1.smithi144.stdout:9/777: read d0/d8/d1b/d79/d8a/f97 [783323,44287] 0 2024-05-15T15:54:21.572 INFO:tasks.workunit.client.1.smithi144.stdout:5/774: dwrite d0/dc/d12/d4a/d5d/fa4 [0,4194304] 0 2024-05-15T15:54:21.574 INFO:tasks.workunit.client.0.smithi049.stdout:1/861: write d0/d4/dd/de/d1e/f37 [4697859,9046] 0 2024-05-15T15:54:21.576 INFO:tasks.workunit.client.0.smithi049.stdout:0/693: creat dc/d6d/dc3/fe6 x:0 0 0 2024-05-15T15:54:21.577 INFO:tasks.workunit.client.0.smithi049.stdout:9/570: dwrite d0/d8/d1b/d40/f77 [4194304,4194304] 0 2024-05-15T15:54:21.578 INFO:tasks.workunit.client.0.smithi049.stdout:8/654: dread d2/de/d71/d86/d5d/d7d/d45/f7c [0,4194304] 0 2024-05-15T15:54:21.579 INFO:tasks.workunit.client.0.smithi049.stdout:4/655: unlink d18/d1e/d2a/d36/d86/la1 0 2024-05-15T15:54:21.580 INFO:tasks.workunit.client.0.smithi049.stdout:8/655: write d2/fc5 [240160,91588] 0 2024-05-15T15:54:21.583 INFO:tasks.workunit.client.0.smithi049.stdout:0/694: rmdir dc/d13/d2b/d76 39 2024-05-15T15:54:21.588 INFO:tasks.workunit.client.0.smithi049.stdout:1/862: rename d0/d4/dd/de/d1e/d2e/dad/fbb to d0/d4/dd/de/d48/d84/f12e 0 2024-05-15T15:54:21.590 INFO:tasks.workunit.client.0.smithi049.stdout:4/656: creat d18/d21/d2b/d4e/fd6 x:0 0 0 2024-05-15T15:54:21.595 INFO:tasks.workunit.client.1.smithi144.stdout:3/749: dread - d0/d11/d3b/da7/fc6 zero size 2024-05-15T15:54:21.597 INFO:tasks.workunit.client.0.smithi049.stdout:4/657: dread - d18/d1e/d2a/fb0 zero size 2024-05-15T15:54:21.599 INFO:tasks.workunit.client.0.smithi049.stdout:4/658: chown d18/d1e/d2a/d50/f5e 12571 1 2024-05-15T15:54:21.599 INFO:tasks.workunit.client.1.smithi144.stdout:6/771: symlink d7/d1e/d28/db5/l111 0 2024-05-15T15:54:21.600 INFO:tasks.workunit.client.1.smithi144.stdout:2/693: write d3/d7/d20/d89/fdf [2801263,11352] 0 2024-05-15T15:54:21.600 INFO:tasks.workunit.client.0.smithi049.stdout:8/656: getdents d2/de/d71/d86/d5d/d7d/db8/d5b/d93/dc2 0 2024-05-15T15:54:21.600 INFO:tasks.workunit.client.1.smithi144.stdout:6/772: fsync d7/db/d9d/db7/ffa 0 2024-05-15T15:54:21.603 INFO:tasks.workunit.client.1.smithi144.stdout:0/718: rename d4/d6/d3a/d4c/dd6/d55/l94 to d4/df/d1f/d24/d96/l103 0 2024-05-15T15:54:21.604 INFO:tasks.workunit.client.1.smithi144.stdout:7/744: write d4/df/d25/d38/d5d/f8c [1097684,21466] 0 2024-05-15T15:54:21.605 INFO:tasks.workunit.client.0.smithi049.stdout:9/571: dread d0/d8/d9/d22/f2e [0,4194304] 0 2024-05-15T15:54:21.607 INFO:tasks.workunit.client.0.smithi049.stdout:1/863: mkdir d0/d4/dd/de/d1e/d12f 0 2024-05-15T15:54:21.611 INFO:tasks.workunit.client.1.smithi144.stdout:5/775: creat d0/d7e/d9b/f102 x:0 0 0 2024-05-15T15:54:21.611 INFO:tasks.workunit.client.0.smithi049.stdout:0/695: link dc/d13/d14/l42 dc/d13/d14/dc9/le7 0 2024-05-15T15:54:21.613 INFO:tasks.workunit.client.0.smithi049.stdout:9/572: creat d0/d25/d5d/fc5 x:0 0 0 2024-05-15T15:54:21.613 INFO:tasks.workunit.client.1.smithi144.stdout:5/776: dread d0/d7e/d9b/fa2 [0,4194304] 0 2024-05-15T15:54:21.615 INFO:tasks.workunit.client.1.smithi144.stdout:1/730: fsync d7/d1d/d33/d1e/d4e/fb6 0 2024-05-15T15:54:21.618 INFO:tasks.workunit.client.1.smithi144.stdout:9/778: dread d0/d8/d1b/d79/d8a/dae/fbe [0,4194304] 0 2024-05-15T15:54:21.618 INFO:tasks.workunit.client.0.smithi049.stdout:0/696: creat dc/d13/d56/d6f/fe8 x:0 0 0 2024-05-15T15:54:21.620 INFO:tasks.workunit.client.0.smithi049.stdout:9/573: unlink d0/d25/c78 0 2024-05-15T15:54:21.623 INFO:tasks.workunit.client.0.smithi049.stdout:1/864: unlink d0/d4/dd/de/d1e/d16/df1/f10b 0 2024-05-15T15:54:21.626 INFO:tasks.workunit.client.0.smithi049.stdout:0/697: mkdir dc/de9 0 2024-05-15T15:54:21.627 INFO:tasks.workunit.client.1.smithi144.stdout:3/750: write d0/d11/d24/f3e [3513906,93474] 0 2024-05-15T15:54:21.627 INFO:tasks.workunit.client.0.smithi049.stdout:0/698: write dc/d13/d14/f79 [3329459,59007] 0 2024-05-15T15:54:21.629 INFO:tasks.workunit.client.1.smithi144.stdout:2/694: chown d3/d7/d20/c7f 17007 1 2024-05-15T15:54:21.631 INFO:tasks.workunit.client.1.smithi144.stdout:4/753: write d6/d42/d46/d4b/d1e/d99/dda/fb9 [3538145,19319] 0 2024-05-15T15:54:21.632 INFO:tasks.workunit.client.0.smithi049.stdout:9/574: write d0/d8/d1b/fbe [2000393,68324] 0 2024-05-15T15:54:21.634 INFO:tasks.workunit.client.0.smithi049.stdout:0/699: symlink dc/d13/d3d/lea 0 2024-05-15T15:54:21.635 INFO:tasks.workunit.client.1.smithi144.stdout:8/804: symlink d2/de/d49/d83/dd9/l106 0 2024-05-15T15:54:21.636 INFO:tasks.workunit.client.0.smithi049.stdout:1/865: mknod d0/d4/dd/de/d36/dd3/c130 0 2024-05-15T15:54:21.636 INFO:tasks.workunit.client.1.smithi144.stdout:6/773: dread d7/db/d9d/d97/f9a [0,4194304] 0 2024-05-15T15:54:21.636 INFO:tasks.workunit.client.1.smithi144.stdout:6/774: fdatasync d7/f106 0 2024-05-15T15:54:21.637 INFO:tasks.workunit.client.1.smithi144.stdout:6/775: readlink d7/d1e/d28/db5/lf4 0 2024-05-15T15:54:21.637 INFO:tasks.workunit.client.0.smithi049.stdout:1/866: chown d0/d4/dd/de/d1e/d2e/d3e/cf8 133112291 1 2024-05-15T15:54:21.638 INFO:tasks.workunit.client.0.smithi049.stdout:9/575: read d0/d8/d9/d52/d5e/d5f/f63 [2042909,1802] 0 2024-05-15T15:54:21.644 INFO:tasks.workunit.client.1.smithi144.stdout:0/719: symlink d4/d2d/d100/l104 0 2024-05-15T15:54:21.644 INFO:tasks.workunit.client.1.smithi144.stdout:0/720: read - d4/d6/d3a/d4c/dd6/fd8 zero size 2024-05-15T15:54:21.645 INFO:tasks.workunit.client.1.smithi144.stdout:7/745: creat d4/df/d79/d100/f101 x:0 0 0 2024-05-15T15:54:21.645 INFO:tasks.workunit.client.1.smithi144.stdout:5/777: write d0/dc/d2f/d34/fcc [312568,44162] 0 2024-05-15T15:54:21.647 INFO:tasks.workunit.client.1.smithi144.stdout:6/776: mkdir d7/d62/d112 0 2024-05-15T15:54:21.649 INFO:tasks.workunit.client.1.smithi144.stdout:9/779: rename d0/d8/d5e/d44/d10/d19/f33 to d0/d8/d1b/d79/d8a/f10d 0 2024-05-15T15:54:21.649 INFO:tasks.workunit.client.1.smithi144.stdout:3/751: creat d0/d11/d24/d78/ff2 x:0 0 0 2024-05-15T15:54:21.649 INFO:tasks.workunit.client.1.smithi144.stdout:1/731: dread d7/d1d/d33/d41/f19 [0,4194304] 0 2024-05-15T15:54:21.653 INFO:tasks.workunit.client.1.smithi144.stdout:1/732: readlink l5 0 2024-05-15T15:54:21.653 INFO:tasks.workunit.client.1.smithi144.stdout:9/780: mknod d0/d8/d1b/d79/de7/c10e 0 2024-05-15T15:54:21.653 INFO:tasks.workunit.client.1.smithi144.stdout:1/733: chown d7/d1d/d33/d41/l84 62207 1 2024-05-15T15:54:21.654 INFO:tasks.workunit.client.1.smithi144.stdout:9/781: chown d0/d8/d5e/d44/dc2/lcb 0 1 2024-05-15T15:54:21.655 INFO:tasks.workunit.client.1.smithi144.stdout:0/721: fsync d4/d6/d3a/d4c/fa6 0 2024-05-15T15:54:21.655 INFO:tasks.workunit.client.1.smithi144.stdout:2/695: getdents d3/d7/d20/d22/d52/d75/da6/dc1 0 2024-05-15T15:54:21.657 INFO:tasks.workunit.client.1.smithi144.stdout:4/754: rename d6/d42/c83 to d6/dd/d34/d3b/cf0 0 2024-05-15T15:54:21.658 INFO:tasks.workunit.client.1.smithi144.stdout:3/752: mkdir d0/d1/d56/d71/df3 0 2024-05-15T15:54:21.659 INFO:tasks.workunit.client.1.smithi144.stdout:1/734: creat d7/d1d/d33/d1e/d34/fe9 x:0 0 0 2024-05-15T15:54:21.659 INFO:tasks.workunit.client.1.smithi144.stdout:3/753: dread - d0/d11/d3b/da7/fe1 zero size 2024-05-15T15:54:21.661 INFO:tasks.workunit.client.1.smithi144.stdout:8/805: truncate d2/d24/d2d/d30/fe0 1629641 0 2024-05-15T15:54:21.662 INFO:tasks.workunit.client.1.smithi144.stdout:2/696: symlink d3/d7/da4/lea 0 2024-05-15T15:54:21.663 INFO:tasks.workunit.client.1.smithi144.stdout:7/746: dread d4/df/f16 [0,4194304] 0 2024-05-15T15:54:21.663 INFO:tasks.workunit.client.1.smithi144.stdout:5/778: rename d0/dc/d2f/d34/d63/d6e/db8/cef to d0/dc/d24/d8e/d95/c103 0 2024-05-15T15:54:21.664 INFO:tasks.workunit.client.1.smithi144.stdout:1/735: fdatasync d7/d1d/d33/d1e/d4e/fb6 0 2024-05-15T15:54:21.665 INFO:tasks.workunit.client.1.smithi144.stdout:2/697: truncate d3/d7/d20/d22/d52/d64/fcb 959287 0 2024-05-15T15:54:21.666 INFO:tasks.workunit.client.1.smithi144.stdout:3/754: creat d0/d1/d17/d39/d40/d7e/ff4 x:0 0 0 2024-05-15T15:54:21.666 INFO:tasks.workunit.client.1.smithi144.stdout:0/722: truncate d4/d6/d3a/d5d/f8f 411540 0 2024-05-15T15:54:21.667 INFO:tasks.workunit.client.1.smithi144.stdout:4/755: mkdir d6/d42/dc8/df1 0 2024-05-15T15:54:21.670 INFO:tasks.workunit.client.1.smithi144.stdout:7/747: truncate d4/d3c/d76/d9d/dac/f1d 1057090 0 2024-05-15T15:54:21.671 INFO:tasks.workunit.client.1.smithi144.stdout:8/806: mkdir d2/de/d3f/d107 0 2024-05-15T15:54:21.673 INFO:tasks.workunit.client.1.smithi144.stdout:2/698: dwrite d3/d7/d20/d3b/f77 [0,4194304] 0 2024-05-15T15:54:21.673 INFO:tasks.workunit.client.1.smithi144.stdout:1/736: mkdir d7/d1d/d33/d29/dd2/dea 0 2024-05-15T15:54:21.674 INFO:tasks.workunit.client.1.smithi144.stdout:1/737: chown d7/d1d/d33/d29/d6b/d8e 5880839 1 2024-05-15T15:54:21.677 INFO:tasks.workunit.client.1.smithi144.stdout:3/755: dread - d0/d1/d56/d98/fd2 zero size 2024-05-15T15:54:21.677 INFO:tasks.workunit.client.1.smithi144.stdout:4/756: mknod d6/d42/d46/cf2 0 2024-05-15T15:54:21.682 INFO:tasks.workunit.client.1.smithi144.stdout:1/738: mkdir d7/d1d/d33/d41/d78/da3/deb 0 2024-05-15T15:54:21.682 INFO:tasks.workunit.client.1.smithi144.stdout:8/807: dwrite d2/d24/d2d/da0/d6e/d71/db1/fd1 [4194304,4194304] 0 2024-05-15T15:54:21.687 INFO:tasks.workunit.client.1.smithi144.stdout:2/699: truncate d3/d7/d20/d3f/fe3 66949 0 2024-05-15T15:54:21.691 INFO:tasks.workunit.client.1.smithi144.stdout:5/779: dwrite d0/dc/f8c [0,4194304] 0 2024-05-15T15:54:21.691 INFO:tasks.workunit.client.1.smithi144.stdout:8/808: mknod d2/d24/d2d/da0/dd5/c108 0 2024-05-15T15:54:21.693 INFO:tasks.workunit.client.1.smithi144.stdout:3/756: link d0/d11/cf d0/d11/d24/da5/cf5 0 2024-05-15T15:54:21.698 INFO:tasks.workunit.client.1.smithi144.stdout:1/739: creat d7/df/fec x:0 0 0 2024-05-15T15:54:21.710 INFO:tasks.workunit.client.1.smithi144.stdout:4/757: write d6/d42/d46/d4b/d1e/d44/f81 [780164,104202] 0 2024-05-15T15:54:21.711 INFO:tasks.workunit.client.1.smithi144.stdout:8/809: mkdir d2/de/d49/d83/d109 0 2024-05-15T15:54:21.711 INFO:tasks.workunit.client.1.smithi144.stdout:3/757: mkdir d0/d11/d3b/da0/df6 0 2024-05-15T15:54:21.712 INFO:tasks.workunit.client.1.smithi144.stdout:1/740: chown d7/d1d/d33/d41/d22/dbd/fd0 91 1 2024-05-15T15:54:21.713 INFO:tasks.workunit.client.1.smithi144.stdout:2/700: mkdir d3/d7/d20/d37/d76/db5/db6/dbf/deb 0 2024-05-15T15:54:21.713 INFO:tasks.workunit.client.1.smithi144.stdout:3/758: stat d0/d1/fe 0 2024-05-15T15:54:21.715 INFO:tasks.workunit.client.1.smithi144.stdout:2/701: fdatasync d3/d7/d20/d89/d6e/fc4 0 2024-05-15T15:54:21.717 INFO:tasks.workunit.client.1.smithi144.stdout:5/780: dwrite d0/dc/d24/d28/f51 [0,4194304] 0 2024-05-15T15:54:21.756 INFO:tasks.workunit.client.0.smithi049.stdout:5/739: sync 2024-05-15T15:54:21.757 INFO:tasks.workunit.client.0.smithi049.stdout:7/630: sync 2024-05-15T15:54:21.757 INFO:tasks.workunit.client.0.smithi049.stdout:3/647: sync 2024-05-15T15:54:21.757 INFO:tasks.workunit.client.0.smithi049.stdout:9/576: symlink d0/d8/d9/d22/lc6 0 2024-05-15T15:54:21.758 INFO:tasks.workunit.client.0.smithi049.stdout:3/648: readlink d12/d14/d32/l76 0 2024-05-15T15:54:21.759 INFO:tasks.workunit.client.0.smithi049.stdout:0/700: write dc/d13/d3d/f5d [318412,29839] 0 2024-05-15T15:54:21.761 INFO:tasks.workunit.client.0.smithi049.stdout:7/631: fdatasync d0/d6/d23/d45/d83/fab 0 2024-05-15T15:54:21.763 INFO:tasks.workunit.client.0.smithi049.stdout:3/649: chown d12/d14/f43 0 1 2024-05-15T15:54:21.764 INFO:tasks.workunit.client.0.smithi049.stdout:3/650: stat d12/d14/d52/db9/dc4 0 2024-05-15T15:54:21.764 INFO:tasks.workunit.client.0.smithi049.stdout:9/577: mkdir d0/d8/d9/d52/dc7 0 2024-05-15T15:54:21.768 INFO:tasks.workunit.client.0.smithi049.stdout:0/701: rmdir dc 39 2024-05-15T15:54:21.769 INFO:tasks.workunit.client.0.smithi049.stdout:5/740: mknod dc/dd/d47/d83/dcc/da9/dba/dbe/cf3 0 2024-05-15T15:54:21.770 INFO:tasks.workunit.client.0.smithi049.stdout:5/741: chown dc/dd/d10/df1 14528178 1 2024-05-15T15:54:21.773 INFO:tasks.workunit.client.0.smithi049.stdout:9/578: rmdir d0/d8/d21 39 2024-05-15T15:54:21.776 INFO:tasks.workunit.client.1.smithi144.stdout:8/810: mkdir d2/d24/d2d/da0/df8/d10a 0 2024-05-15T15:54:21.776 INFO:tasks.workunit.client.1.smithi144.stdout:4/758: rename d6/dd/d34/d3b/d30/dc9/cce to d6/d42/d46/d4b/d1e/d44/cf3 0 2024-05-15T15:54:21.777 INFO:tasks.workunit.client.0.smithi049.stdout:7/632: creat d0/d6/d23/d45/d86/fc5 x:0 0 0 2024-05-15T15:54:21.777 INFO:tasks.workunit.client.1.smithi144.stdout:2/702: dread d3/d10/f1d [0,4194304] 0 2024-05-15T15:54:21.780 INFO:tasks.workunit.client.0.smithi049.stdout:0/702: write dc/d32/faa [638323,92742] 0 2024-05-15T15:54:21.781 INFO:tasks.workunit.client.1.smithi144.stdout:4/759: creat d6/dd/d34/d3b/d30/ff4 x:0 0 0 2024-05-15T15:54:21.782 INFO:tasks.workunit.client.0.smithi049.stdout:5/742: rename dc/dd/d10/f71 to dc/dd/d7b/d8d/ff4 0 2024-05-15T15:54:21.783 INFO:tasks.workunit.client.0.smithi049.stdout:5/743: stat dc/d4a/d9b/fae 0 2024-05-15T15:54:21.784 INFO:tasks.workunit.client.1.smithi144.stdout:8/811: rename d2/de/d49/fca to d2/d24/d2d/da0/f10b 0 2024-05-15T15:54:21.785 INFO:tasks.workunit.client.1.smithi144.stdout:5/781: symlink d0/dc/d24/l104 0 2024-05-15T15:54:21.786 INFO:tasks.workunit.client.1.smithi144.stdout:2/703: mknod d3/d7/d20/d22/d52/d75/d65/cec 0 2024-05-15T15:54:21.787 INFO:tasks.workunit.client.0.smithi049.stdout:0/703: fsync dc/f34 0 2024-05-15T15:54:21.788 INFO:tasks.workunit.client.1.smithi144.stdout:1/741: link d7/d1d/d33/d41/c9b d7/d1d/d33/d41/d22/dbd/ced 0 2024-05-15T15:54:21.789 INFO:tasks.workunit.client.1.smithi144.stdout:1/742: fsync d7/d1d/d33/d29/d6b/d8e/fca 0 2024-05-15T15:54:21.790 INFO:tasks.workunit.client.1.smithi144.stdout:4/760: truncate d6/dd/d34/f9d 437427 0 2024-05-15T15:54:21.792 INFO:tasks.workunit.client.0.smithi049.stdout:5/744: creat dc/dd/d7b/d8d/ff5 x:0 0 0 2024-05-15T15:54:21.796 INFO:tasks.workunit.client.1.smithi144.stdout:5/782: dread - d0/dc/d24/d28/d2d/d59/d74/fe3 zero size 2024-05-15T15:54:21.799 INFO:tasks.workunit.client.0.smithi049.stdout:3/651: getdents d12/d14 0 2024-05-15T15:54:21.799 INFO:tasks.workunit.client.0.smithi049.stdout:3/652: stat d12/d67 0 2024-05-15T15:54:21.800 INFO:tasks.workunit.client.1.smithi144.stdout:2/704: dread d3/d7/d20/d22/d52/d75/d65/f9f [0,4194304] 0 2024-05-15T15:54:21.801 INFO:tasks.workunit.client.0.smithi049.stdout:5/745: dwrite dc/dd/d39/fa0 [0,4194304] 0 2024-05-15T15:54:21.801 INFO:tasks.workunit.client.1.smithi144.stdout:8/812: write d2/d24/d2d/d30/dd0/df7/f1b [935268,7525] 0 2024-05-15T15:54:21.802 INFO:tasks.workunit.client.1.smithi144.stdout:2/705: write d3/d7/d20/d37/d76/faa [493757,123668] 0 2024-05-15T15:54:21.808 INFO:tasks.workunit.client.1.smithi144.stdout:6/777: sync 2024-05-15T15:54:21.809 INFO:tasks.workunit.client.0.smithi049.stdout:0/704: symlink dc/d13/d14/d25/db9/dd0/leb 0 2024-05-15T15:54:21.809 INFO:tasks.workunit.client.1.smithi144.stdout:6/778: fsync d7/fe5 0 2024-05-15T15:54:21.810 INFO:tasks.workunit.client.0.smithi049.stdout:7/633: link d0/d6/f29 d0/d6/fc6 0 2024-05-15T15:54:21.811 INFO:tasks.workunit.client.0.smithi049.stdout:0/705: write dc/d13/d56/d6f/fe8 [917987,108158] 0 2024-05-15T15:54:21.812 INFO:tasks.workunit.client.1.smithi144.stdout:1/743: dwrite d7/d1d/d33/d1e/d4e/fbb [0,4194304] 0 2024-05-15T15:54:21.812 INFO:tasks.workunit.client.1.smithi144.stdout:1/744: dread - d7/d1d/fa5 zero size 2024-05-15T15:54:21.813 INFO:tasks.workunit.client.1.smithi144.stdout:4/761: mknod d6/d42/dc8/df1/cf5 0 2024-05-15T15:54:21.814 INFO:tasks.workunit.client.0.smithi049.stdout:5/746: symlink dc/dd/d7b/lf6 0 2024-05-15T15:54:21.816 INFO:tasks.workunit.client.1.smithi144.stdout:2/706: rename d3/d4d/db4/fe8 to d3/d4d/db4/fed 0 2024-05-15T15:54:21.819 INFO:tasks.workunit.client.0.smithi049.stdout:0/706: creat dc/d13/d14/d25/d31/dae/fec x:0 0 0 2024-05-15T15:54:21.819 INFO:tasks.workunit.client.0.smithi049.stdout:3/653: write d12/d14/f17 [5215540,12643] 0 2024-05-15T15:54:21.820 INFO:tasks.workunit.client.0.smithi049.stdout:3/654: chown d12/d26/c60 446 1 2024-05-15T15:54:21.821 INFO:tasks.workunit.client.0.smithi049.stdout:7/634: mkdir d0/d6/d23/d45/d6a/da1/d54/dc7 0 2024-05-15T15:54:21.821 INFO:tasks.workunit.client.0.smithi049.stdout:3/655: write d12/d14/d52/db9/d6b/fbb [353958,59558] 0 2024-05-15T15:54:21.823 INFO:tasks.workunit.client.1.smithi144.stdout:8/813: write d2/d24/f68 [57693,157] 0 2024-05-15T15:54:21.827 INFO:tasks.workunit.client.0.smithi049.stdout:3/656: fdatasync d12/d14/d32/fa7 0 2024-05-15T15:54:21.828 INFO:tasks.workunit.client.0.smithi049.stdout:5/747: symlink dc/dd/d10/df1/lf7 0 2024-05-15T15:54:21.829 INFO:tasks.workunit.client.0.smithi049.stdout:5/748: readlink dc/dd/d10/df1/lf7 0 2024-05-15T15:54:21.830 INFO:tasks.workunit.client.0.smithi049.stdout:5/749: write dc/dd/f14 [7647139,62480] 0 2024-05-15T15:54:21.830 INFO:tasks.workunit.client.0.smithi049.stdout:0/707: creat dc/d13/d2b/d76/fed x:0 0 0 2024-05-15T15:54:21.835 INFO:tasks.workunit.client.1.smithi144.stdout:4/762: read d6/d7c/deb/f77 [5206800,45302] 0 2024-05-15T15:54:21.836 INFO:tasks.workunit.client.1.smithi144.stdout:1/745: write d7/d1d/fa5 [790599,17570] 0 2024-05-15T15:54:21.837 INFO:tasks.workunit.client.0.smithi049.stdout:5/750: mknod dc/db0/dd5/cf8 0 2024-05-15T15:54:21.838 INFO:tasks.workunit.client.0.smithi049.stdout:5/751: stat dc/dd/d10/ceb 0 2024-05-15T15:54:21.839 INFO:tasks.workunit.client.1.smithi144.stdout:5/783: creat d0/dc/d24/f105 x:0 0 0 2024-05-15T15:54:21.840 INFO:tasks.workunit.client.0.smithi049.stdout:0/708: unlink dc/d13/d14/d25/d31/c87 0 2024-05-15T15:54:21.840 INFO:tasks.workunit.client.1.smithi144.stdout:6/779: dwrite d7/db/d35/d84/d81/f109 [4194304,4194304] 0 2024-05-15T15:54:21.843 INFO:tasks.workunit.client.1.smithi144.stdout:7/748: sync 2024-05-15T15:54:21.843 INFO:tasks.workunit.client.1.smithi144.stdout:0/723: sync 2024-05-15T15:54:21.843 INFO:tasks.workunit.client.1.smithi144.stdout:9/782: sync 2024-05-15T15:54:21.843 INFO:tasks.workunit.client.0.smithi049.stdout:7/635: rename d0/fb9 to d0/fc8 0 2024-05-15T15:54:21.845 INFO:tasks.workunit.client.0.smithi049.stdout:3/657: symlink d12/d14/d52/db9/dc4/d68/d70/d75/d9d/lc8 0 2024-05-15T15:54:21.846 INFO:tasks.workunit.client.0.smithi049.stdout:5/752: write dc/dd/d47/da6/dce/fe2 [5424315,85286] 0 2024-05-15T15:54:21.849 INFO:tasks.workunit.client.0.smithi049.stdout:0/709: mknod dc/d13/d14/d2f/d8f/cee 0 2024-05-15T15:54:21.851 INFO:tasks.workunit.client.1.smithi144.stdout:2/707: rename d3/d7/d20/d89/d6e/l90 to d3/d7/d20/d22/d52/d64/lee 0 2024-05-15T15:54:21.852 INFO:tasks.workunit.client.0.smithi049.stdout:0/710: readlink dc/d13/d14/d25/d31/dae/dc5/l4a 0 2024-05-15T15:54:21.853 INFO:tasks.workunit.client.1.smithi144.stdout:1/746: symlink d7/d1d/d33/d1e/d34/lee 0 2024-05-15T15:54:21.854 INFO:tasks.workunit.client.0.smithi049.stdout:7/636: dwrite d0/d6/d23/d45/d6a/da1/d39/fa8 [0,4194304] 0 2024-05-15T15:54:21.855 INFO:tasks.workunit.client.1.smithi144.stdout:1/747: write d7/d1d/d33/d41/d22/dbd/fd0 [880899,114482] 0 2024-05-15T15:54:21.855 INFO:tasks.workunit.client.1.smithi144.stdout:1/748: read f1 [4476163,93491] 0 2024-05-15T15:54:21.856 INFO:tasks.workunit.client.1.smithi144.stdout:6/780: creat d7/db/d5f/f113 x:0 0 0 2024-05-15T15:54:21.860 INFO:tasks.workunit.client.1.smithi144.stdout:8/814: truncate d2/d24/d2d/f54 124859 0 2024-05-15T15:54:21.862 INFO:tasks.workunit.client.0.smithi049.stdout:5/753: rename dc/db0/df2/fd3 to dc/dd/d47/d83/dcc/da9/ff9 0 2024-05-15T15:54:21.863 INFO:tasks.workunit.client.1.smithi144.stdout:9/783: unlink d0/f59 0 2024-05-15T15:54:21.864 INFO:tasks.workunit.client.1.smithi144.stdout:9/784: chown d0/d8/d5e/d44/d10/d9c/f98 6 1 2024-05-15T15:54:21.865 INFO:tasks.workunit.client.1.smithi144.stdout:9/785: readlink d0/d8/d1b/d62/d7d/d8d/dbb/dce/lf1 0 2024-05-15T15:54:21.866 INFO:tasks.workunit.client.1.smithi144.stdout:7/749: creat d4/d3c/d76/d9d/f102 x:0 0 0 2024-05-15T15:54:21.869 INFO:tasks.workunit.client.1.smithi144.stdout:0/724: fsync d4/d6/d3a/d4c/dd6/d55/f60 0 2024-05-15T15:54:21.870 INFO:tasks.workunit.client.1.smithi144.stdout:4/763: dwrite d6/fa9 [0,4194304] 0 2024-05-15T15:54:21.876 INFO:tasks.workunit.client.0.smithi049.stdout:3/658: dwrite d12/d14/d52/f7b [0,4194304] 0 2024-05-15T15:54:21.879 INFO:tasks.workunit.client.1.smithi144.stdout:1/749: unlink d7/d1d/d33/d1e/l2d 0 2024-05-15T15:54:21.879 INFO:tasks.workunit.client.0.smithi049.stdout:0/711: mkdir dc/d13/d56/d6f/dce/def 0 2024-05-15T15:54:21.881 INFO:tasks.workunit.client.1.smithi144.stdout:6/781: mknod d7/db/da2/c114 0 2024-05-15T15:54:21.883 INFO:tasks.workunit.client.0.smithi049.stdout:7/637: symlink d0/d2f/lc9 0 2024-05-15T15:54:21.887 INFO:tasks.workunit.client.0.smithi049.stdout:5/754: write dc/dd/d47/d83/dcc/da9/dba/dbe/dca/fee [996978,103352] 0 2024-05-15T15:54:21.888 INFO:tasks.workunit.client.1.smithi144.stdout:7/750: creat d4/df/d25/d40/d73/f103 x:0 0 0 2024-05-15T15:54:21.902 INFO:tasks.workunit.client.1.smithi144.stdout:2/708: mkdir d3/d7/d20/def 0 2024-05-15T15:54:21.906 INFO:tasks.workunit.client.0.smithi049.stdout:7/638: dread d0/d6/d1d/f4d [0,4194304] 0 2024-05-15T15:54:21.908 INFO:tasks.workunit.client.0.smithi049.stdout:7/639: chown d0/d6/d23/f8a 2124 1 2024-05-15T15:54:21.908 INFO:tasks.workunit.client.0.smithi049.stdout:7/640: dread - d0/d6/d23/d45/d6a/da1/fbf zero size 2024-05-15T15:54:21.941 INFO:tasks.workunit.client.1.smithi144.stdout:8/815: mknod d2/d24/d2d/da0/d6e/d71/c10c 0 2024-05-15T15:54:21.943 INFO:tasks.workunit.client.0.smithi049.stdout:0/712: link dc/d13/d56/d6f/lc6 dc/d6d/lf0 0 2024-05-15T15:54:21.944 INFO:tasks.workunit.client.0.smithi049.stdout:3/659: write d12/d14/f43 [1045249,45447] 0 2024-05-15T15:54:21.945 INFO:tasks.workunit.client.0.smithi049.stdout:3/660: chown d12/d14/d52/db9/dc4/d68/d70 264760772 1 2024-05-15T15:54:21.947 INFO:tasks.workunit.client.1.smithi144.stdout:7/751: unlink d4/df/d25/d40/d62/ldf 0 2024-05-15T15:54:21.948 INFO:tasks.workunit.client.1.smithi144.stdout:7/752: fsync d4/f27 0 2024-05-15T15:54:21.951 INFO:tasks.workunit.client.1.smithi144.stdout:3/759: sync 2024-05-15T15:54:21.951 INFO:tasks.workunit.client.1.smithi144.stdout:0/725: write d4/d6/d4a/d74/f79 [1833285,106616] 0 2024-05-15T15:54:21.952 INFO:tasks.workunit.client.0.smithi049.stdout:3/661: dwrite d12/d14/d52/db9/dc4/d68/fba [0,4194304] 0 2024-05-15T15:54:21.952 INFO:tasks.workunit.client.1.smithi144.stdout:4/764: write d6/f29 [2298720,3153] 0 2024-05-15T15:54:21.953 INFO:tasks.workunit.client.1.smithi144.stdout:2/709: truncate d3/d7/d54/fa0 2851207 0 2024-05-15T15:54:21.957 INFO:tasks.workunit.client.1.smithi144.stdout:6/782: mknod d7/c115 0 2024-05-15T15:54:21.959 INFO:tasks.workunit.client.1.smithi144.stdout:8/816: creat d2/da8/f10d x:0 0 0 2024-05-15T15:54:21.961 INFO:tasks.workunit.client.1.smithi144.stdout:9/786: creat d0/d8/d5e/d44/d10/f10f x:0 0 0 2024-05-15T15:54:21.961 INFO:tasks.workunit.client.1.smithi144.stdout:9/787: readlink d0/le 0 2024-05-15T15:54:21.964 INFO:tasks.workunit.client.0.smithi049.stdout:7/641: dwrite d0/f6b [0,4194304] 0 2024-05-15T15:54:21.968 INFO:tasks.workunit.client.1.smithi144.stdout:4/765: mknod d6/d42/d46/d4b/d1e/d99/dda/cf6 0 2024-05-15T15:54:21.971 INFO:tasks.workunit.client.1.smithi144.stdout:2/710: rmdir d3/d7/d54 39 2024-05-15T15:54:21.974 INFO:tasks.workunit.client.1.smithi144.stdout:6/783: fdatasync d7/db/d35/d84/d81/f83 0 2024-05-15T15:54:21.976 INFO:tasks.workunit.client.1.smithi144.stdout:4/766: dwrite d6/dd/d34/d3b/d30/db/d1d/d40/d57/f5e [4194304,4194304] 0 2024-05-15T15:54:21.977 INFO:tasks.workunit.client.1.smithi144.stdout:1/750: sync 2024-05-15T15:54:21.977 INFO:tasks.workunit.client.1.smithi144.stdout:5/784: sync 2024-05-15T15:54:21.978 INFO:tasks.workunit.client.1.smithi144.stdout:5/785: write d0/f22 [2435293,88233] 0 2024-05-15T15:54:21.984 INFO:tasks.workunit.client.1.smithi144.stdout:5/786: dwrite d0/dc/d24/d8e/d93/f89 [0,4194304] 0 2024-05-15T15:54:21.989 INFO:tasks.workunit.client.1.smithi144.stdout:3/760: write d0/d11/d3b/da7/fc6 [759759,128611] 0 2024-05-15T15:54:21.990 INFO:tasks.workunit.client.1.smithi144.stdout:2/711: chown d3/d7/d54/d98/le7 32636381 1 2024-05-15T15:54:21.992 INFO:tasks.workunit.client.1.smithi144.stdout:0/726: write d4/df/d14/fc0 [360097,8127] 0 2024-05-15T15:54:21.993 INFO:tasks.workunit.client.1.smithi144.stdout:6/784: creat d7/db/d3a/dd2/f116 x:0 0 0 2024-05-15T15:54:21.995 INFO:tasks.workunit.client.1.smithi144.stdout:8/817: symlink d2/de/d49/d83/d109/l10e 0 2024-05-15T15:54:22.001 INFO:tasks.workunit.client.1.smithi144.stdout:4/767: rename d6/dd/d34/d3b/c3a to d6/dd/d34/d3b/d30/db/d1d/d40/d6c/cf7 0 2024-05-15T15:54:22.003 INFO:tasks.workunit.client.1.smithi144.stdout:4/768: dread - d6/dd/d34/d3b/d30/db/d1d/d40/fec zero size 2024-05-15T15:54:22.003 INFO:tasks.workunit.client.1.smithi144.stdout:1/751: dread d7/d1d/d33/d41/d22/fa7 [0,4194304] 0 2024-05-15T15:54:22.004 INFO:tasks.workunit.client.1.smithi144.stdout:9/788: creat d0/d8/d5e/d9b/df8/f110 x:0 0 0 2024-05-15T15:54:22.005 INFO:tasks.workunit.client.1.smithi144.stdout:7/753: creat d4/df/d25/d38/f104 x:0 0 0 2024-05-15T15:54:22.009 INFO:tasks.workunit.client.1.smithi144.stdout:2/712: fdatasync d3/f34 0 2024-05-15T15:54:22.009 INFO:tasks.workunit.client.1.smithi144.stdout:2/713: chown d3/d4d/fae 337 1 2024-05-15T15:54:22.010 INFO:tasks.workunit.client.1.smithi144.stdout:0/727: truncate d4/d6/d3a/d5d/f65 445181 0 2024-05-15T15:54:22.010 INFO:tasks.workunit.client.1.smithi144.stdout:8/818: mkdir d2/d24/d2d/d30/dd0/df7/d87/d10f 0 2024-05-15T15:54:22.013 INFO:tasks.workunit.client.1.smithi144.stdout:6/785: rename d7/db/d11/d3c/cad to d7/db/d35/d84/dbe/c117 0 2024-05-15T15:54:22.014 INFO:tasks.workunit.client.1.smithi144.stdout:4/769: creat d6/d42/dc8/ff8 x:0 0 0 2024-05-15T15:54:22.015 INFO:tasks.workunit.client.1.smithi144.stdout:7/754: symlink d4/d3c/d76/d9d/dac/d48/l105 0 2024-05-15T15:54:22.016 INFO:tasks.workunit.client.1.smithi144.stdout:4/770: readlink d6/d42/d46/l6f 0 2024-05-15T15:54:22.017 INFO:tasks.workunit.client.1.smithi144.stdout:5/787: mkdir d0/dc/d1f/d50/db3/d106 0 2024-05-15T15:54:22.019 INFO:tasks.workunit.client.1.smithi144.stdout:6/786: dwrite d7/db/d9d/fea [0,4194304] 0 2024-05-15T15:54:22.019 INFO:tasks.workunit.client.0.smithi049.stdout:5/755: getdents dc/dd/d39 0 2024-05-15T15:54:22.020 INFO:tasks.workunit.client.0.smithi049.stdout:5/756: chown dc/dd/d39/f6e 2288554 1 2024-05-15T15:54:22.020 INFO:tasks.workunit.client.0.smithi049.stdout:5/757: dread - dc/f7f zero size 2024-05-15T15:54:22.021 INFO:tasks.workunit.client.0.smithi049.stdout:3/662: mknod d12/d30/dc1/dc7/cc9 0 2024-05-15T15:54:22.022 INFO:tasks.workunit.client.1.smithi144.stdout:6/787: read d7/db/da2/f66 [1807117,129265] 0 2024-05-15T15:54:22.022 INFO:tasks.workunit.client.0.smithi049.stdout:3/663: write d12/d14/fb8 [2190489,16374] 0 2024-05-15T15:54:22.023 INFO:tasks.workunit.client.1.smithi144.stdout:2/714: creat d3/d7/d54/d98/ff0 x:0 0 0 2024-05-15T15:54:22.025 INFO:tasks.workunit.client.0.smithi049.stdout:3/664: truncate d12/d14/d52/db9/dc4/d68/f9b 46351 0 2024-05-15T15:54:22.026 INFO:tasks.workunit.client.0.smithi049.stdout:7/642: rename d0/d6/d23/d45/d6a/da1/d54 to d0/d34/d88/dca 0 2024-05-15T15:54:22.029 INFO:tasks.workunit.client.0.smithi049.stdout:5/758: mknod dc/db0/dc7/cfa 0 2024-05-15T15:54:22.030 INFO:tasks.workunit.client.1.smithi144.stdout:8/819: symlink d2/d3a/da5/l110 0 2024-05-15T15:54:22.033 INFO:tasks.workunit.client.1.smithi144.stdout:1/752: mkdir d7/d1d/d33/def 0 2024-05-15T15:54:22.035 INFO:tasks.workunit.client.0.smithi049.stdout:3/665: creat d12/d30/d66/fca x:0 0 0 2024-05-15T15:54:22.036 INFO:tasks.workunit.client.1.smithi144.stdout:4/771: creat d6/d7c/ff9 x:0 0 0 2024-05-15T15:54:22.039 INFO:tasks.workunit.client.1.smithi144.stdout:7/755: fsync d4/df/d25/d40/d62/f95 0 2024-05-15T15:54:22.041 INFO:tasks.workunit.client.0.smithi049.stdout:5/759: rename dc/dd/d47/d83/dcc/da9/dba/dbe/dca to dc/d4a/ddd/dfb 0 2024-05-15T15:54:22.044 INFO:tasks.workunit.client.1.smithi144.stdout:3/761: rename d0/d11/f26 to d0/d11/ff7 0 2024-05-15T15:54:22.044 INFO:tasks.workunit.client.1.smithi144.stdout:3/762: chown d0/d1/d17/d39 333939 1 2024-05-15T15:54:22.045 INFO:tasks.workunit.client.0.smithi049.stdout:3/666: mknod d12/d14/d52/db9/dc4/d68/d70/d75/ccb 0 2024-05-15T15:54:22.048 INFO:tasks.workunit.client.1.smithi144.stdout:6/788: mkdir d7/db/d35/d84/dbe/d118 0 2024-05-15T15:54:22.049 INFO:tasks.workunit.client.1.smithi144.stdout:0/728: symlink d4/d2d/d48/d5b/dd3/l105 0 2024-05-15T15:54:22.054 INFO:tasks.workunit.client.0.smithi049.stdout:5/760: creat dc/dd/d10/db3/dd6/ffc x:0 0 0 2024-05-15T15:54:22.055 INFO:tasks.workunit.client.1.smithi144.stdout:4/772: creat d6/dd/d2e/d94/ffa x:0 0 0 2024-05-15T15:54:22.058 INFO:tasks.workunit.client.0.smithi049.stdout:5/761: fdatasync dc/dd/d47/f9c 0 2024-05-15T15:54:22.060 INFO:tasks.workunit.client.1.smithi144.stdout:7/756: mkdir d4/d41/d9c/d106 0 2024-05-15T15:54:22.063 INFO:tasks.workunit.client.1.smithi144.stdout:3/763: creat d0/d1/d17/d39/dc8/ff8 x:0 0 0 2024-05-15T15:54:22.064 INFO:tasks.workunit.client.0.smithi049.stdout:5/762: fsync dc/dd/d47/f9c 0 2024-05-15T15:54:22.065 INFO:tasks.workunit.client.0.smithi049.stdout:3/667: dread f7 [0,4194304] 0 2024-05-15T15:54:22.070 INFO:tasks.workunit.client.0.smithi049.stdout:3/668: dread d12/d14/d52/db9/d6b/fbb [0,4194304] 0 2024-05-15T15:54:22.071 INFO:tasks.workunit.client.0.smithi049.stdout:7/643: dwrite d0/d6/d1d/f7a [0,4194304] 0 2024-05-15T15:54:22.074 INFO:tasks.workunit.client.0.smithi049.stdout:2/630: sync 2024-05-15T15:54:22.074 INFO:tasks.workunit.client.0.smithi049.stdout:6/663: sync 2024-05-15T15:54:22.084 INFO:tasks.workunit.client.1.smithi144.stdout:8/820: dwrite d2/d3a/f7f [0,4194304] 0 2024-05-15T15:54:22.084 INFO:tasks.workunit.client.1.smithi144.stdout:8/821: readlink d2/l34 0 2024-05-15T15:54:22.086 INFO:tasks.workunit.client.1.smithi144.stdout:6/789: dread d7/f8 [0,4194304] 0 2024-05-15T15:54:22.089 INFO:tasks.workunit.client.1.smithi144.stdout:0/729: dread d4/df/d1f/d24/f47 [0,4194304] 0 2024-05-15T15:54:22.093 INFO:tasks.workunit.client.1.smithi144.stdout:9/789: sync 2024-05-15T15:54:22.096 INFO:tasks.workunit.client.1.smithi144.stdout:6/790: dread d7/db/d5f/fb8 [0,4194304] 0 2024-05-15T15:54:22.102 INFO:tasks.workunit.client.1.smithi144.stdout:9/790: dwrite d0/d8/d5e/d44/d10/f10f [0,4194304] 0 2024-05-15T15:54:22.109 INFO:tasks.workunit.client.1.smithi144.stdout:2/715: sync 2024-05-15T15:54:22.193 INFO:tasks.workunit.client.1.smithi144.stdout:7/757: write d4/df/d81/fe7 [5039558,72845] 0 2024-05-15T15:54:22.200 INFO:tasks.workunit.client.1.smithi144.stdout:5/788: creat d0/dc/d2f/d34/d63/f107 x:0 0 0 2024-05-15T15:54:22.201 INFO:tasks.workunit.client.0.smithi049.stdout:7/644: creat d0/d6/d23/d45/db1/fcb x:0 0 0 2024-05-15T15:54:22.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:22 smithi049 ceph-mon[28399]: pgmap v31: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 79 MiB/s rd, 201 MiB/s wr, 418 op/s 2024-05-15T15:54:22.205 INFO:tasks.workunit.client.1.smithi144.stdout:1/753: rmdir d7/d1d/d33/def 0 2024-05-15T15:54:22.210 INFO:tasks.workunit.client.0.smithi049.stdout:5/763: dwrite dc/dd/d39/f4c [0,4194304] 0 2024-05-15T15:54:22.210 INFO:tasks.workunit.client.0.smithi049.stdout:5/764: dread - dc/f7f zero size 2024-05-15T15:54:22.210 INFO:tasks.workunit.client.1.smithi144.stdout:4/773: link d6/dd/d34/d3b/d30/db/d1d/d4e/fd4 d6/dd/d34/d3b/d30/db/d1d/d4e/ffb 0 2024-05-15T15:54:22.221 INFO:tasks.workunit.client.1.smithi144.stdout:0/730: creat d4/df/d14/db5/f106 x:0 0 0 2024-05-15T15:54:22.222 INFO:tasks.workunit.client.0.smithi049.stdout:6/664: link dd/d17/d24/d2b/d36/d50/d54/db6/lc0 dd/d17/d24/d67/lc9 0 2024-05-15T15:54:22.222 INFO:tasks.workunit.client.0.smithi049.stdout:6/665: fsync dd/d26/f3f 0 2024-05-15T15:54:22.223 INFO:tasks.workunit.client.0.smithi049.stdout:6/666: readlink dd/d26/l39 0 2024-05-15T15:54:22.223 INFO:tasks.workunit.client.1.smithi144.stdout:6/791: fdatasync d7/db/d35/d84/d81/fa0 0 2024-05-15T15:54:22.228 INFO:tasks.workunit.client.0.smithi049.stdout:6/667: dwrite dd/d17/d24/d2b/d36/d50/f83 [0,4194304] 0 2024-05-15T15:54:22.236 INFO:tasks.workunit.client.0.smithi049.stdout:4/659: sync 2024-05-15T15:54:22.256 INFO:tasks.workunit.client.0.smithi049.stdout:6/668: fsync dd/f80 0 2024-05-15T15:54:22.272 INFO:tasks.workunit.client.0.smithi049.stdout:4/660: creat d18/d85/fd7 x:0 0 0 2024-05-15T15:54:22.276 INFO:tasks.workunit.client.0.smithi049.stdout:4/661: dread - d18/d1e/fa7 zero size 2024-05-15T15:54:22.315 INFO:tasks.workunit.client.1.smithi144.stdout:7/758: write d4/df/d25/f29 [3185740,13639] 0 2024-05-15T15:54:22.318 INFO:tasks.workunit.client.1.smithi144.stdout:4/774: mkdir d6/dd/d34/d3b/d30/db/d1d/d40/d57/dfc 0 2024-05-15T15:54:22.319 INFO:tasks.workunit.client.1.smithi144.stdout:8/822: unlink d2/d24/d2d/d30/dd0/df7/d58/f72 0 2024-05-15T15:54:22.330 INFO:tasks.workunit.client.1.smithi144.stdout:4/775: truncate d6/d42/d46/d4b/f4d 727338 0 2024-05-15T15:54:22.331 INFO:tasks.workunit.client.1.smithi144.stdout:4/776: fsync d6/f29 0 2024-05-15T15:54:22.332 INFO:tasks.workunit.client.1.smithi144.stdout:4/777: chown d6/d42/lc1 528 1 2024-05-15T15:54:22.350 INFO:tasks.workunit.client.0.smithi049.stdout:7/645: rename d0/d6/d23/d45/d6a/da1/d39 to d0/d34/d88/dcc 0 2024-05-15T15:54:22.351 INFO:tasks.workunit.client.0.smithi049.stdout:7/646: chown d0/d6/d23/d45/d6a/da1/fbf 10 1 2024-05-15T15:54:22.364 INFO:tasks.workunit.client.0.smithi049.stdout:3/669: truncate d12/d14/d32/d64/d88/d9c/fb4 2933018 0 2024-05-15T15:54:22.364 INFO:tasks.workunit.client.1.smithi144.stdout:3/764: dwrite d0/f2c [0,4194304] 0 2024-05-15T15:54:22.365 INFO:tasks.workunit.client.1.smithi144.stdout:3/765: chown d0/d1/d17/d39/d83/f9d 0 1 2024-05-15T15:54:22.381 INFO:tasks.workunit.client.1.smithi144.stdout:3/766: dread d0/d1/d17/d39/d40/f50 [0,4194304] 0 2024-05-15T15:54:22.381 INFO:tasks.workunit.client.1.smithi144.stdout:3/767: chown d0/d1/d17/f19 380 1 2024-05-15T15:54:22.383 INFO:tasks.workunit.client.0.smithi049.stdout:6/669: dwrite dd/d17/d24/d2b/d36/d50/d54/f8f [0,4194304] 0 2024-05-15T15:54:22.388 INFO:tasks.workunit.client.0.smithi049.stdout:4/662: unlink d18/f1b 0 2024-05-15T15:54:22.388 INFO:tasks.workunit.client.0.smithi049.stdout:4/663: chown d18 179985462 1 2024-05-15T15:54:22.390 INFO:tasks.workunit.client.0.smithi049.stdout:4/664: creat d18/d85/d98/fd8 x:0 0 0 2024-05-15T15:54:22.398 INFO:tasks.workunit.client.1.smithi144.stdout:2/716: unlink d3/d7/d20/d22/l4e 0 2024-05-15T15:54:22.405 INFO:tasks.workunit.client.1.smithi144.stdout:2/717: rmdir d3/d7/d20/d37/d76 39 2024-05-15T15:54:22.406 INFO:tasks.workunit.client.1.smithi144.stdout:2/718: write d3/d7/d20/d22/d52/fa3 [794996,115140] 0 2024-05-15T15:54:22.408 INFO:tasks.workunit.client.1.smithi144.stdout:2/719: fdatasync d3/d7/d20/d40/fa5 0 2024-05-15T15:54:22.410 INFO:tasks.workunit.client.0.smithi049.stdout:5/765: rename l6 to dc/dd/d47/d83/dcc/da9/lfd 0 2024-05-15T15:54:22.415 INFO:tasks.workunit.client.0.smithi049.stdout:3/670: rename d12/d14/d52/db9/d6b/fa2 to d12/fcc 0 2024-05-15T15:54:22.417 INFO:tasks.workunit.client.1.smithi144.stdout:7/759: write d4/f13 [4206368,28333] 0 2024-05-15T15:54:22.417 INFO:tasks.workunit.client.1.smithi144.stdout:6/792: dwrite d7/db/d11/d7e/f8c [0,4194304] 0 2024-05-15T15:54:22.420 INFO:tasks.workunit.client.1.smithi144.stdout:9/791: dwrite d0/d8/d1b/d79/d8a/f10d [0,4194304] 0 2024-05-15T15:54:22.425 INFO:tasks.workunit.client.0.smithi049.stdout:6/670: rename f0 to dd/d17/d24/d2b/d36/d50/fca 0 2024-05-15T15:54:22.426 INFO:tasks.workunit.client.1.smithi144.stdout:2/720: dread d3/d7/d20/d22/f2b [0,4194304] 0 2024-05-15T15:54:22.427 INFO:tasks.workunit.client.1.smithi144.stdout:6/793: symlink d7/db/d9d/l119 0 2024-05-15T15:54:22.427 INFO:tasks.workunit.client.1.smithi144.stdout:2/721: fdatasync d3/d7/d20/d40/fa5 0 2024-05-15T15:54:22.428 INFO:tasks.workunit.client.0.smithi049.stdout:6/671: write dd/d17/d24/d2b/d36/d50/f83 [538670,106148] 0 2024-05-15T15:54:22.428 INFO:tasks.workunit.client.0.smithi049.stdout:3/671: link d12/d14/d52/db9/cb1 d12/d30/dc1/dc7/ccd 0 2024-05-15T15:54:22.428 INFO:tasks.workunit.client.0.smithi049.stdout:6/672: read dd/f14 [285522,66960] 0 2024-05-15T15:54:22.429 INFO:tasks.workunit.client.0.smithi049.stdout:3/672: write d12/d14/d52/db9/dc4/fb2 [860568,16832] 0 2024-05-15T15:54:22.430 INFO:tasks.workunit.client.1.smithi144.stdout:7/760: chown d4/df/d25/d38/d68/fd6 47 1 2024-05-15T15:54:22.434 INFO:tasks.workunit.client.1.smithi144.stdout:6/794: mknod d7/db/d9d/c11a 0 2024-05-15T15:54:22.436 INFO:tasks.workunit.client.0.smithi049.stdout:3/673: creat d12/d14/d52/fce x:0 0 0 2024-05-15T15:54:22.438 INFO:tasks.workunit.client.0.smithi049.stdout:6/673: symlink dd/dbc/lcb 0 2024-05-15T15:54:22.438 INFO:tasks.workunit.client.0.smithi049.stdout:6/674: fsync dd/d17/d64/fb5 0 2024-05-15T15:54:22.439 INFO:tasks.workunit.client.1.smithi144.stdout:5/789: rename d0/dc/d24/d28/d2d/d59/fca to d0/dc/d24/d28/d2d/df7/f108 0 2024-05-15T15:54:22.441 INFO:tasks.workunit.client.0.smithi049.stdout:3/674: dread d12/d14/d52/db9/d6b/fbb [0,4194304] 0 2024-05-15T15:54:22.442 INFO:tasks.workunit.client.1.smithi144.stdout:5/790: write d0/dc/d1f/d50/fbe [114652,120420] 0 2024-05-15T15:54:22.442 INFO:tasks.workunit.client.1.smithi144.stdout:7/761: dwrite d4/df/fb1 [0,4194304] 0 2024-05-15T15:54:22.442 INFO:tasks.workunit.client.0.smithi049.stdout:6/675: creat dd/d17/fcc x:0 0 0 2024-05-15T15:54:22.443 INFO:tasks.workunit.client.0.smithi049.stdout:6/676: write dd/d26/f3f [913927,35163] 0 2024-05-15T15:54:22.444 INFO:tasks.workunit.client.1.smithi144.stdout:5/791: stat d0/dc/d2f/d68/d5c/f81 0 2024-05-15T15:54:22.445 INFO:tasks.workunit.client.1.smithi144.stdout:3/768: symlink d0/d11/d24/lf9 0 2024-05-15T15:54:22.447 INFO:tasks.workunit.client.1.smithi144.stdout:0/731: rename d4/df/d1f/d24/d85/da1/da3 to d4/d2d/d48/d5b/dd3/d107 0 2024-05-15T15:54:22.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:22 smithi144 ceph-mon[36389]: pgmap v31: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 79 MiB/s rd, 201 MiB/s wr, 418 op/s 2024-05-15T15:54:22.452 INFO:tasks.workunit.client.1.smithi144.stdout:6/795: truncate d7/d62/fd9 820034 0 2024-05-15T15:54:22.454 INFO:tasks.workunit.client.0.smithi049.stdout:3/675: dwrite d12/fcc [0,4194304] 0 2024-05-15T15:54:22.455 INFO:tasks.workunit.client.1.smithi144.stdout:7/762: creat d4/df/d81/f107 x:0 0 0 2024-05-15T15:54:22.458 INFO:tasks.workunit.client.1.smithi144.stdout:5/792: fdatasync d0/dc/d2f/d68/d5c/f81 0 2024-05-15T15:54:22.458 INFO:tasks.workunit.client.1.smithi144.stdout:7/763: dread d4/df/d25/d40/d73/f9a [0,4194304] 0 2024-05-15T15:54:22.459 INFO:tasks.workunit.client.1.smithi144.stdout:6/796: dwrite d7/db/d9d/d97/fc0 [0,4194304] 0 2024-05-15T15:54:22.461 INFO:tasks.workunit.client.0.smithi049.stdout:3/676: mkdir d12/d14/d52/db9/d6b/dcf 0 2024-05-15T15:54:22.462 INFO:tasks.workunit.client.1.smithi144.stdout:7/764: write d4/df/d25/d40/d73/f98 [4179981,79238] 0 2024-05-15T15:54:22.463 INFO:tasks.workunit.client.1.smithi144.stdout:7/765: read d4/df/f16 [1521974,9923] 0 2024-05-15T15:54:22.464 INFO:tasks.workunit.client.1.smithi144.stdout:8/823: rename d2/de/d3f/lfa to d2/de/d49/d83/dd9/l111 0 2024-05-15T15:54:22.464 INFO:tasks.workunit.client.1.smithi144.stdout:7/766: write d4/d3c/d76/d9d/f102 [211062,25970] 0 2024-05-15T15:54:22.470 INFO:tasks.workunit.client.0.smithi049.stdout:6/677: dread dd/d17/d24/f74 [0,4194304] 0 2024-05-15T15:54:22.471 INFO:tasks.workunit.client.1.smithi144.stdout:5/793: dread d0/dc/d2f/d68/d5c/f98 [0,4194304] 0 2024-05-15T15:54:22.472 INFO:tasks.workunit.client.1.smithi144.stdout:5/794: fsync d0/dc/d24/d28/d2d/d59/d74/ff2 0 2024-05-15T15:54:22.474 INFO:tasks.workunit.client.0.smithi049.stdout:3/677: rename d12/f1d to d12/d14/d32/fd0 0 2024-05-15T15:54:22.475 INFO:tasks.workunit.client.0.smithi049.stdout:3/678: stat d12/d14/d32 0 2024-05-15T15:54:22.478 INFO:tasks.workunit.client.0.smithi049.stdout:7/647: creat d0/d6/fcd x:0 0 0 2024-05-15T15:54:22.482 INFO:tasks.workunit.client.0.smithi049.stdout:3/679: fsync ff 0 2024-05-15T15:54:22.483 INFO:tasks.workunit.client.0.smithi049.stdout:3/680: truncate d12/d14/f4f 4985861 0 2024-05-15T15:54:22.484 INFO:tasks.workunit.client.0.smithi049.stdout:7/648: stat d0/d6/d23/d45/d6a/da1/c8b 0 2024-05-15T15:54:22.486 INFO:tasks.workunit.client.0.smithi049.stdout:6/678: dread dd/d17/d24/d2b/d36/d50/fca [0,4194304] 0 2024-05-15T15:54:22.490 INFO:tasks.workunit.client.0.smithi049.stdout:7/649: creat d0/d6/d23/d45/d83/fce x:0 0 0 2024-05-15T15:54:22.492 INFO:tasks.workunit.client.0.smithi049.stdout:6/679: dread dd/d17/d24/d2b/d36/f98 [0,4194304] 0 2024-05-15T15:54:22.492 INFO:tasks.workunit.client.1.smithi144.stdout:4/778: rename d6/dd/l16 to d6/d7c/deb/lfd 0 2024-05-15T15:54:22.493 INFO:tasks.workunit.client.1.smithi144.stdout:4/779: chown d6/d42/f58 19495 1 2024-05-15T15:54:22.495 INFO:tasks.workunit.client.1.smithi144.stdout:1/754: symlink d7/d1d/lf0 0 2024-05-15T15:54:22.497 INFO:tasks.workunit.client.0.smithi049.stdout:4/665: write d18/f22 [3153946,86810] 0 2024-05-15T15:54:22.500 INFO:tasks.workunit.client.1.smithi144.stdout:8/824: mknod d2/d24/d2d/da0/d9e/db8/c112 0 2024-05-15T15:54:22.501 INFO:tasks.workunit.client.1.smithi144.stdout:8/825: dread d2/d24/d2d/da0/f10b [0,4194304] 0 2024-05-15T15:54:22.503 INFO:tasks.workunit.client.1.smithi144.stdout:7/767: mkdir d4/df/d81/dfb/d108 0 2024-05-15T15:54:22.505 INFO:tasks.workunit.client.1.smithi144.stdout:7/768: write d4/d3c/d76/d9d/dac/de5/fea [814688,7427] 0 2024-05-15T15:54:22.506 INFO:tasks.workunit.client.0.smithi049.stdout:6/680: symlink dd/d17/d24/d8a/lcd 0 2024-05-15T15:54:22.507 INFO:tasks.workunit.client.1.smithi144.stdout:8/826: dwrite d2/de/fea [0,4194304] 0 2024-05-15T15:54:22.508 INFO:tasks.workunit.client.0.smithi049.stdout:7/650: dread d0/d6/d23/d4a/f87 [0,4194304] 0 2024-05-15T15:54:22.515 INFO:tasks.workunit.client.0.smithi049.stdout:5/766: dwrite dc/f64 [0,4194304] 0 2024-05-15T15:54:22.516 INFO:tasks.workunit.client.0.smithi049.stdout:5/767: write dc/dd/f88 [2359773,31608] 0 2024-05-15T15:54:22.517 INFO:tasks.workunit.client.0.smithi049.stdout:4/666: fdatasync d18/d1e/d2a/d50/f5a 0 2024-05-15T15:54:22.518 INFO:tasks.workunit.client.0.smithi049.stdout:5/768: readlink dc/dd/d47/d83/da7/lef 0 2024-05-15T15:54:22.521 INFO:tasks.workunit.client.1.smithi144.stdout:4/780: truncate d6/dd/d34/d3b/d30/db/d1d/d40/f79 278026 0 2024-05-15T15:54:22.525 INFO:tasks.workunit.client.1.smithi144.stdout:1/755: symlink d7/d1d/d33/d29/d6b/lf1 0 2024-05-15T15:54:22.526 INFO:tasks.workunit.client.0.smithi049.stdout:6/681: rmdir dd/d17/d24/d2b/d36 39 2024-05-15T15:54:22.527 INFO:tasks.workunit.client.1.smithi144.stdout:4/781: dwrite d6/dd/d34/d3b/d30/d59/d66/f9e [0,4194304] 0 2024-05-15T15:54:22.529 INFO:tasks.workunit.client.0.smithi049.stdout:3/681: rename d12/d14/d52/db9/d6b/ca1 to d12/d14/d52/db9/cd1 0 2024-05-15T15:54:22.531 INFO:tasks.workunit.client.0.smithi049.stdout:7/651: fsync d0/fc8 0 2024-05-15T15:54:22.534 INFO:tasks.workunit.client.1.smithi144.stdout:2/722: dwrite d3/d7/d20/d22/d52/f6f [0,4194304] 0 2024-05-15T15:54:22.538 INFO:tasks.workunit.client.1.smithi144.stdout:5/795: mkdir d0/dc/d2f/d109 0 2024-05-15T15:54:22.542 INFO:tasks.workunit.client.0.smithi049.stdout:4/667: chown d18/fc5 19 1 2024-05-15T15:54:22.543 INFO:tasks.workunit.client.1.smithi144.stdout:9/792: dwrite d0/d8/d5e/d44/fd6 [0,4194304] 0 2024-05-15T15:54:22.544 INFO:tasks.workunit.client.0.smithi049.stdout:5/769: rmdir dc/db0/dd5/dd8 39 2024-05-15T15:54:22.545 INFO:tasks.workunit.client.1.smithi144.stdout:0/732: getdents d4/d2d/d48/d5b/dda 0 2024-05-15T15:54:22.546 INFO:tasks.workunit.client.0.smithi049.stdout:6/682: creat dd/d17/d64/d8c/fce x:0 0 0 2024-05-15T15:54:22.548 INFO:tasks.workunit.client.0.smithi049.stdout:6/683: write dd/d17/d64/d8c/fa0 [1047223,7209] 0 2024-05-15T15:54:22.549 INFO:tasks.workunit.client.1.smithi144.stdout:1/756: rmdir d7/d1d/d33/d41/d22/dbd 39 2024-05-15T15:54:22.551 INFO:tasks.workunit.client.1.smithi144.stdout:1/757: read d7/d1d/d33/d29/d6b/d5e/d74/fab [637352,3510] 0 2024-05-15T15:54:22.554 INFO:tasks.workunit.client.1.smithi144.stdout:1/758: dread d7/d1d/d33/d29/d6b/d5e/f9d [0,4194304] 0 2024-05-15T15:54:22.556 INFO:tasks.workunit.client.0.smithi049.stdout:7/652: dread d0/d6/f29 [0,4194304] 0 2024-05-15T15:54:22.559 INFO:tasks.workunit.client.1.smithi144.stdout:3/769: truncate d0/d11/d24/f3e 1616737 0 2024-05-15T15:54:22.561 INFO:tasks.workunit.client.0.smithi049.stdout:4/668: dwrite d18/d1e/d2a/d50/d54/dad/fce [4194304,4194304] 0 2024-05-15T15:54:22.563 INFO:tasks.workunit.client.0.smithi049.stdout:3/682: symlink d12/d14/d52/db9/d6b/dcf/ld2 0 2024-05-15T15:54:22.570 INFO:tasks.workunit.client.1.smithi144.stdout:6/797: rename d7/db/d9d/l10f to d7/d1e/d28/l11b 0 2024-05-15T15:54:22.572 INFO:tasks.workunit.client.1.smithi144.stdout:5/796: dread d0/dc/d24/d8e/d93/d6b/d88/f8d [0,4194304] 0 2024-05-15T15:54:22.574 INFO:tasks.workunit.client.1.smithi144.stdout:9/793: truncate d0/d8/d5e/d44/d10/d19/f63 4742072 0 2024-05-15T15:54:22.578 INFO:tasks.workunit.client.1.smithi144.stdout:0/733: symlink d4/d2d/d48/l108 0 2024-05-15T15:54:22.585 INFO:tasks.workunit.client.1.smithi144.stdout:4/782: link d6/dd/d2e/d94/ffa d6/d7c/ffe 0 2024-05-15T15:54:22.587 INFO:tasks.workunit.client.1.smithi144.stdout:1/759: mkdir d7/d1d/d33/d29/d6b/d5e/d74/df2 0 2024-05-15T15:54:22.587 INFO:tasks.workunit.client.0.smithi049.stdout:7/653: fdatasync d0/d2f/d5c/f70 0 2024-05-15T15:54:22.589 INFO:tasks.workunit.client.1.smithi144.stdout:3/770: rmdir d0/d1/d56/d98/d9c 39 2024-05-15T15:54:22.592 INFO:tasks.workunit.client.0.smithi049.stdout:6/684: mkdir dd/d17/d24/d2b/d36/d50/dcf 0 2024-05-15T15:54:22.593 INFO:tasks.workunit.client.1.smithi144.stdout:7/769: link d4/d3c/d76/d9d/lc7 d4/d3c/d76/d9d/dac/de5/l109 0 2024-05-15T15:54:22.594 INFO:tasks.workunit.client.0.smithi049.stdout:7/654: mknod d0/d6/d23/d4a/ccf 0 2024-05-15T15:54:22.597 INFO:tasks.workunit.client.0.smithi049.stdout:5/770: write dc/dd/d7b/f90 [1008929,96025] 0 2024-05-15T15:54:22.597 INFO:tasks.workunit.client.1.smithi144.stdout:8/827: dwrite d2/de/f103 [0,4194304] 0 2024-05-15T15:54:22.598 INFO:tasks.workunit.client.0.smithi049.stdout:4/669: creat d18/fd9 x:0 0 0 2024-05-15T15:54:22.598 INFO:tasks.workunit.client.1.smithi144.stdout:8/828: readlink d2/d24/l8a 0 2024-05-15T15:54:22.598 INFO:tasks.workunit.client.1.smithi144.stdout:2/723: write d3/d10/f29 [658720,99563] 0 2024-05-15T15:54:22.602 INFO:tasks.workunit.client.0.smithi049.stdout:7/655: dwrite d0/d6/d23/d45/d6a/f9d [0,4194304] 0 2024-05-15T15:54:22.606 INFO:tasks.workunit.client.0.smithi049.stdout:5/771: dread - dc/dd/d10/faf zero size 2024-05-15T15:54:22.609 INFO:tasks.workunit.client.0.smithi049.stdout:4/670: mkdir d18/d1e/d2a/d3b/da8/dda 0 2024-05-15T15:54:22.613 INFO:tasks.workunit.client.1.smithi144.stdout:4/783: write d6/dd/d2e/d94/ffa [664984,88555] 0 2024-05-15T15:54:22.615 INFO:tasks.workunit.client.0.smithi049.stdout:5/772: fsync dc/db0/dd5/fe1 0 2024-05-15T15:54:22.616 INFO:tasks.workunit.client.0.smithi049.stdout:5/773: chown dc/f1b 666 1 2024-05-15T15:54:22.620 INFO:tasks.workunit.client.1.smithi144.stdout:1/760: rename d7/d1d/d33/d1e/d87 to d7/d1d/d33/d29/d6b/d5e/d74/df3 0 2024-05-15T15:54:22.623 INFO:tasks.workunit.client.0.smithi049.stdout:3/683: write d12/d14/d32/d64/d88/fb5 [5030650,106109] 0 2024-05-15T15:54:22.624 INFO:tasks.workunit.client.0.smithi049.stdout:1/867: sync 2024-05-15T15:54:22.625 INFO:tasks.workunit.client.0.smithi049.stdout:9/579: sync 2024-05-15T15:54:22.625 INFO:tasks.workunit.client.0.smithi049.stdout:8/657: sync 2024-05-15T15:54:22.625 INFO:tasks.workunit.client.0.smithi049.stdout:6/685: rename dd/f3a to dd/d17/d24/fd0 0 2024-05-15T15:54:22.630 INFO:tasks.workunit.client.0.smithi049.stdout:5/774: symlink dc/d4a/ddd/dfb/lfe 0 2024-05-15T15:54:22.631 INFO:tasks.workunit.client.0.smithi049.stdout:8/658: write d2/de/d71/d86/d5d/d7d/d1b/d68/fab [785226,54616] 0 2024-05-15T15:54:22.632 INFO:tasks.workunit.client.0.smithi049.stdout:5/775: fsync dc/db0/dd5/fe1 0 2024-05-15T15:54:22.635 INFO:tasks.workunit.client.0.smithi049.stdout:3/684: dwrite d12/d14/d52/db9/da6/dc2/fc3 [0,4194304] 0 2024-05-15T15:54:22.635 INFO:tasks.workunit.client.1.smithi144.stdout:8/829: truncate d2/d24/d2d/d30/dd0/fdd 507825 0 2024-05-15T15:54:22.635 INFO:tasks.workunit.client.0.smithi049.stdout:4/671: dread d18/d1e/d2a/d36/f4b [0,4194304] 0 2024-05-15T15:54:22.637 INFO:tasks.workunit.client.0.smithi049.stdout:9/580: symlink d0/d25/lc8 0 2024-05-15T15:54:22.637 INFO:tasks.workunit.client.0.smithi049.stdout:7/656: write d0/d2f/f8e [949929,9181] 0 2024-05-15T15:54:22.640 INFO:tasks.workunit.client.0.smithi049.stdout:1/868: creat d0/d4/dd/de/d48/d5c/da2/f131 x:0 0 0 2024-05-15T15:54:22.641 INFO:tasks.workunit.client.0.smithi049.stdout:6/686: dwrite f2 [0,4194304] 0 2024-05-15T15:54:22.643 INFO:tasks.workunit.client.0.smithi049.stdout:6/687: fdatasync dd/d17/d24/d2b/fc5 0 2024-05-15T15:54:22.643 INFO:tasks.workunit.client.1.smithi144.stdout:3/771: write d0/d11/d24/f57 [1944742,66122] 0 2024-05-15T15:54:22.648 INFO:tasks.workunit.client.0.smithi049.stdout:5/776: unlink dc/dd/d47/d83/dcc/da9/dba/dbe/cf3 0 2024-05-15T15:54:22.649 INFO:tasks.workunit.client.0.smithi049.stdout:5/777: chown dc/dd/d47/ca3 1498248394 1 2024-05-15T15:54:22.649 INFO:tasks.workunit.client.1.smithi144.stdout:4/784: unlink d6/d42/d46/d4b/d1e/d44/fa8 0 2024-05-15T15:54:22.650 INFO:tasks.workunit.client.0.smithi049.stdout:8/659: creat d2/de/d71/d86/d5d/d7d/db8/d5b/da6/dc7/fcc x:0 0 0 2024-05-15T15:54:22.651 INFO:tasks.workunit.client.0.smithi049.stdout:8/660: chown d2/de/d25/f32 9904798 1 2024-05-15T15:54:22.652 INFO:tasks.workunit.client.1.smithi144.stdout:1/761: truncate d7/d1d/d33/d1e/d34/f4a 7053797 0 2024-05-15T15:54:22.658 INFO:tasks.workunit.client.0.smithi049.stdout:1/869: mknod d0/d4/dd/de/d1e/d16/d6e/c132 0 2024-05-15T15:54:22.659 INFO:tasks.workunit.client.1.smithi144.stdout:6/798: creat d7/f11c x:0 0 0 2024-05-15T15:54:22.662 INFO:tasks.workunit.client.0.smithi049.stdout:5/778: mkdir dc/d4a/d42/dff 0 2024-05-15T15:54:22.663 INFO:tasks.workunit.client.1.smithi144.stdout:8/830: chown d2/d24/d2d/d30/fe0 216971206 1 2024-05-15T15:54:22.664 INFO:tasks.workunit.client.0.smithi049.stdout:9/581: chown d0/d8/d1b/d71/d7d/dba 0 1 2024-05-15T15:54:22.667 INFO:tasks.workunit.client.1.smithi144.stdout:9/794: link d0/d8/d1b/fb9 d0/d8/d5e/d44/d4a/f111 0 2024-05-15T15:54:22.668 INFO:tasks.workunit.client.0.smithi049.stdout:8/661: mknod d2/de/d71/d86/d5d/d7d/db8/d5b/d93/ccd 0 2024-05-15T15:54:22.670 INFO:tasks.workunit.client.0.smithi049.stdout:4/672: creat d18/d1e/d2a/d36/d86/dbd/fdb x:0 0 0 2024-05-15T15:54:22.671 INFO:tasks.workunit.client.1.smithi144.stdout:3/772: symlink d0/d1/d17/d39/d4d/lfa 0 2024-05-15T15:54:22.673 INFO:tasks.workunit.client.0.smithi049.stdout:2/631: sync 2024-05-15T15:54:22.673 INFO:tasks.workunit.client.0.smithi049.stdout:0/713: sync 2024-05-15T15:54:22.674 INFO:tasks.workunit.client.0.smithi049.stdout:0/714: chown dc/d13/d3d/fe0 31 1 2024-05-15T15:54:22.675 INFO:tasks.workunit.client.1.smithi144.stdout:4/785: rename d6/d7c to d6/dd/d2e/d75/dff 0 2024-05-15T15:54:22.676 INFO:tasks.workunit.client.0.smithi049.stdout:3/685: write d12/f3f [2183100,99949] 0 2024-05-15T15:54:22.676 INFO:tasks.workunit.client.0.smithi049.stdout:6/688: symlink dd/d17/d24/d2b/ld1 0 2024-05-15T15:54:22.677 INFO:tasks.workunit.client.1.smithi144.stdout:4/786: write d6/dd/d34/d3b/d30/db/f33 [1139487,30430] 0 2024-05-15T15:54:22.677 INFO:tasks.workunit.client.1.smithi144.stdout:4/787: write d6/dd/fd2 [519485,13251] 0 2024-05-15T15:54:22.679 INFO:tasks.workunit.client.1.smithi144.stdout:1/762: mkdir d7/d1d/d33/d1e/d34/dd8/df4 0 2024-05-15T15:54:22.681 INFO:tasks.workunit.client.1.smithi144.stdout:7/770: link d4/d3c/d76/d9d/dac/f6e d4/d41/d5b/f10a 0 2024-05-15T15:54:22.683 INFO:tasks.workunit.client.1.smithi144.stdout:5/797: getdents d0/dc/d2f/d34/d63/d6e/db8 0 2024-05-15T15:54:22.687 INFO:tasks.workunit.client.0.smithi049.stdout:4/673: rename d18/d1e/d2a/d3b/c40 to d18/d1e/d2a/d50/dc4/cdc 0 2024-05-15T15:54:22.690 INFO:tasks.workunit.client.0.smithi049.stdout:7/657: dwrite d0/d6/f3c [0,4194304] 0 2024-05-15T15:54:22.690 INFO:tasks.workunit.client.1.smithi144.stdout:2/724: link d3/d7/d20/d89/d6e/fc4 d3/d7/d20/d37/d76/db5/db6/dbf/deb/ff1 0 2024-05-15T15:54:22.690 INFO:tasks.workunit.client.1.smithi144.stdout:2/725: chown d3/f34 75458498 1 2024-05-15T15:54:22.691 INFO:tasks.workunit.client.1.smithi144.stdout:8/831: dread - d2/d24/d2d/faa zero size 2024-05-15T15:54:22.691 INFO:tasks.workunit.client.1.smithi144.stdout:2/726: write d3/d7/d20/d22/d52/fa3 [1626559,80953] 0 2024-05-15T15:54:22.692 INFO:tasks.workunit.client.0.smithi049.stdout:7/658: fsync d0/d6/d23/d45/d6a/da1/d80/fbb 0 2024-05-15T15:54:22.693 INFO:tasks.workunit.client.0.smithi049.stdout:9/582: write d0/d8/d1b/f94 [840821,46422] 0 2024-05-15T15:54:22.694 INFO:tasks.workunit.client.1.smithi144.stdout:9/795: mkdir d0/d8/d5e/d112 0 2024-05-15T15:54:22.695 INFO:tasks.workunit.client.0.smithi049.stdout:7/659: read d0/d34/fad [1690975,24078] 0 2024-05-15T15:54:22.697 INFO:tasks.workunit.client.1.smithi144.stdout:0/734: link d4/df/d1f/la2 d4/d2d/d48/d5b/dd3/d107/l109 0 2024-05-15T15:54:22.699 INFO:tasks.workunit.client.1.smithi144.stdout:6/799: dwrite d7/db/d11/fa1 [0,4194304] 0 2024-05-15T15:54:22.700 INFO:tasks.workunit.client.1.smithi144.stdout:6/800: write d7/fe5 [4559111,79462] 0 2024-05-15T15:54:22.701 INFO:tasks.workunit.client.0.smithi049.stdout:5/779: mknod dc/c100 0 2024-05-15T15:54:22.702 INFO:tasks.workunit.client.0.smithi049.stdout:3/686: dread d12/f4d [0,4194304] 0 2024-05-15T15:54:22.703 INFO:tasks.workunit.client.1.smithi144.stdout:1/763: symlink d7/d1d/d33/d29/d6b/d5e/lf5 0 2024-05-15T15:54:22.706 INFO:tasks.workunit.client.1.smithi144.stdout:5/798: mknod d0/dc/d2f/d34/c10a 0 2024-05-15T15:54:22.707 INFO:tasks.workunit.client.1.smithi144.stdout:1/764: dread d7/d1d/d33/d41/f19 [0,4194304] 0 2024-05-15T15:54:22.708 INFO:tasks.workunit.client.0.smithi049.stdout:2/632: creat d3/db/d1d/d1e/dd9/fdb x:0 0 0 2024-05-15T15:54:22.712 INFO:tasks.workunit.client.0.smithi049.stdout:0/715: mkdir dc/da4/db4/df1 0 2024-05-15T15:54:22.712 INFO:tasks.workunit.client.1.smithi144.stdout:1/765: dwrite d7/d1d/d33/d29/d6b/d5e/fa1 [0,4194304] 0 2024-05-15T15:54:22.713 INFO:tasks.workunit.client.0.smithi049.stdout:2/633: dread d3/db/f12 [0,4194304] 0 2024-05-15T15:54:22.716 INFO:tasks.workunit.client.0.smithi049.stdout:6/689: symlink dd/d17/d24/d2b/ld2 0 2024-05-15T15:54:22.717 INFO:tasks.workunit.client.0.smithi049.stdout:6/690: chown dd/d17/d64/f6a 7 1 2024-05-15T15:54:22.722 INFO:tasks.workunit.client.0.smithi049.stdout:1/870: rmdir d0/d4/dd/de/d1e/d2e/dad/df9 0 2024-05-15T15:54:22.722 INFO:tasks.workunit.client.0.smithi049.stdout:4/674: dread d18/d1e/d2a/d36/f58 [0,4194304] 0 2024-05-15T15:54:22.724 INFO:tasks.workunit.client.1.smithi144.stdout:4/788: unlink d6/dd/d2e/d75/dff/deb/lfd 0 2024-05-15T15:54:22.727 INFO:tasks.workunit.client.0.smithi049.stdout:3/687: fdatasync d12/d14/d32/fd0 0 2024-05-15T15:54:22.730 INFO:tasks.workunit.client.1.smithi144.stdout:4/789: dwrite d6/d42/d46/fe3 [0,4194304] 0 2024-05-15T15:54:22.732 INFO:tasks.workunit.client.0.smithi049.stdout:3/688: dread d12/fcc [0,4194304] 0 2024-05-15T15:54:22.732 INFO:tasks.workunit.client.0.smithi049.stdout:5/780: dread dc/dd/d39/f6e [0,4194304] 0 2024-05-15T15:54:22.734 INFO:tasks.workunit.client.0.smithi049.stdout:9/583: link d0/d25/d5d/f8d d0/d25/d5d/fc9 0 2024-05-15T15:54:22.740 INFO:tasks.workunit.client.1.smithi144.stdout:1/766: symlink d7/d1d/d33/d1e/d4e/lf6 0 2024-05-15T15:54:22.742 INFO:tasks.workunit.client.0.smithi049.stdout:6/691: dread - dd/d17/f6d zero size 2024-05-15T15:54:22.742 INFO:tasks.workunit.client.0.smithi049.stdout:2/634: read d3/db/d1d/d1e/f21 [1569272,97709] 0 2024-05-15T15:54:22.743 INFO:tasks.workunit.client.1.smithi144.stdout:1/767: dread - d7/d1d/d33/d1e/d34/d9c/fcf zero size 2024-05-15T15:54:22.743 INFO:tasks.workunit.client.1.smithi144.stdout:3/773: link d0/d1/c2d d0/d1/d17/d39/d83/cfb 0 2024-05-15T15:54:22.743 INFO:tasks.workunit.client.1.smithi144.stdout:2/727: dwrite d3/d7/f36 [0,4194304] 0 2024-05-15T15:54:22.743 INFO:tasks.workunit.client.1.smithi144.stdout:0/735: write d4/d6/d3a/d4c/fa6 [96049,103203] 0 2024-05-15T15:54:22.752 INFO:tasks.workunit.client.0.smithi049.stdout:4/675: dread d18/d1e/d2a/d3b/f47 [0,4194304] 0 2024-05-15T15:54:22.754 INFO:tasks.workunit.client.0.smithi049.stdout:5/781: mknod dc/dd/d47/da6/dce/c101 0 2024-05-15T15:54:22.755 INFO:tasks.workunit.client.0.smithi049.stdout:0/716: write dc/f15 [1256810,89276] 0 2024-05-15T15:54:22.756 INFO:tasks.workunit.client.0.smithi049.stdout:5/782: read dc/dd/f3b [4703644,70860] 0 2024-05-15T15:54:22.757 INFO:tasks.workunit.client.1.smithi144.stdout:8/832: dwrite d2/f35 [0,4194304] 0 2024-05-15T15:54:22.759 INFO:tasks.workunit.client.1.smithi144.stdout:8/833: fsync d2/d24/d2d/da0/d6e/d8d/ffd 0 2024-05-15T15:54:22.760 INFO:tasks.workunit.client.0.smithi049.stdout:7/660: rename d0/d6/da/c85 to d0/d6/d23/cd0 0 2024-05-15T15:54:22.761 INFO:tasks.workunit.client.1.smithi144.stdout:9/796: creat d0/d8/d5e/d44/d10/d19/f113 x:0 0 0 2024-05-15T15:54:22.764 INFO:tasks.workunit.client.0.smithi049.stdout:2/635: creat d3/db/d5b/fdc x:0 0 0 2024-05-15T15:54:22.767 INFO:tasks.workunit.client.0.smithi049.stdout:2/636: chown d3/db/d1d/d1e/d2b/d2e/d67/fac 31993 1 2024-05-15T15:54:22.768 INFO:tasks.workunit.client.1.smithi144.stdout:3/774: chown d0/d11/d3b/dc7/ld4 2 1 2024-05-15T15:54:22.769 INFO:tasks.workunit.client.1.smithi144.stdout:2/728: dread d3/d7/d20/d22/d52/d75/da6/dc1/fc6 [0,4194304] 0 2024-05-15T15:54:22.769 INFO:tasks.workunit.client.1.smithi144.stdout:5/799: truncate d0/dc/d24/d57/f77 2251672 0 2024-05-15T15:54:22.770 INFO:tasks.workunit.client.1.smithi144.stdout:2/729: chown d3/d7/f1f 58 1 2024-05-15T15:54:22.771 INFO:tasks.workunit.client.1.smithi144.stdout:2/730: chown d3/d7/d54/d98/ff0 446195234 1 2024-05-15T15:54:22.771 INFO:tasks.workunit.client.1.smithi144.stdout:0/736: mkdir d4/d2d/d48/d10a 0 2024-05-15T15:54:22.776 INFO:tasks.workunit.client.1.smithi144.stdout:6/801: rename d7/d32/f50 to d7/db/d11/d3c/f11d 0 2024-05-15T15:54:22.777 INFO:tasks.workunit.client.1.smithi144.stdout:6/802: readlink d7/db/d35/d84/d81/lfd 0 2024-05-15T15:54:22.779 INFO:tasks.workunit.client.1.smithi144.stdout:7/771: getdents d4/d3c/d76/d9d/dac/d48/ded 0 2024-05-15T15:54:22.780 INFO:tasks.workunit.client.0.smithi049.stdout:0/717: fsync dc/d13/d14/d25/d31/f8c 0 2024-05-15T15:54:22.782 INFO:tasks.workunit.client.0.smithi049.stdout:0/718: chown dc/d13/d14/d33 53 1 2024-05-15T15:54:22.782 INFO:tasks.workunit.client.0.smithi049.stdout:5/783: rmdir dc/dd/d47/d83/dcc/da9 39 2024-05-15T15:54:22.783 INFO:tasks.workunit.client.1.smithi144.stdout:3/775: symlink d0/d1/d56/d71/d73/lfc 0 2024-05-15T15:54:22.785 INFO:tasks.workunit.client.0.smithi049.stdout:4/676: dwrite d18/d1e/d2a/d50/f79 [0,4194304] 0 2024-05-15T15:54:22.786 INFO:tasks.workunit.client.0.smithi049.stdout:9/584: unlink d0/d8/d9/d52/d5e/d5f/d56/ca9 0 2024-05-15T15:54:22.787 INFO:tasks.workunit.client.0.smithi049.stdout:4/677: chown d18/d1e/d2a/d3b/da8 8 1 2024-05-15T15:54:22.788 INFO:tasks.workunit.client.0.smithi049.stdout:7/661: creat d0/d34/d88/dcc/fd1 x:0 0 0 2024-05-15T15:54:22.789 INFO:tasks.workunit.client.0.smithi049.stdout:7/662: write d0/d34/f46 [1631557,54133] 0 2024-05-15T15:54:22.790 INFO:tasks.workunit.client.1.smithi144.stdout:2/731: creat d3/d7/d20/d37/d76/db5/db6/ff2 x:0 0 0 2024-05-15T15:54:22.795 INFO:tasks.workunit.client.1.smithi144.stdout:4/790: rename d6/f95 to d6/d42/d46/d4b/d1e/d99/dda/f100 0 2024-05-15T15:54:22.799 INFO:tasks.workunit.client.1.smithi144.stdout:8/834: mknod d2/d24/d2d/da0/d6e/deb/c113 0 2024-05-15T15:54:22.801 INFO:tasks.workunit.client.1.smithi144.stdout:3/776: mkdir d0/d1/d17/d39/d4d/dfd 0 2024-05-15T15:54:22.802 INFO:tasks.workunit.client.1.smithi144.stdout:3/777: dread - d0/d11/d3b/da0/d8f/f9b zero size 2024-05-15T15:54:22.802 INFO:tasks.workunit.client.0.smithi049.stdout:4/678: mkdir d18/d85/d98/ddd 0 2024-05-15T15:54:22.805 INFO:tasks.workunit.client.0.smithi049.stdout:6/692: link dd/d17/d24/d8a/f90 dd/d17/d24/d2b/d36/fd3 0 2024-05-15T15:54:22.806 INFO:tasks.workunit.client.0.smithi049.stdout:6/693: read - dd/d17/d24/d2b/fc5 zero size 2024-05-15T15:54:22.809 INFO:tasks.workunit.client.1.smithi144.stdout:4/791: dread d6/dd/d34/d3b/d30/db/d1d/d40/d57/f85 [0,4194304] 0 2024-05-15T15:54:22.813 INFO:tasks.workunit.client.0.smithi049.stdout:1/871: rename d0/d4/dd/de/d9e/f105 to d0/f133 0 2024-05-15T15:54:22.816 INFO:tasks.workunit.client.0.smithi049.stdout:6/694: fsync dd/f14 0 2024-05-15T15:54:22.817 INFO:tasks.workunit.client.1.smithi144.stdout:5/800: truncate d0/dc/d12/fd7 27978 0 2024-05-15T15:54:22.818 INFO:tasks.workunit.client.0.smithi049.stdout:6/695: chown dd/d17/d24/d2b/d36/d50/d54/f5c 5287 1 2024-05-15T15:54:22.819 INFO:tasks.workunit.client.1.smithi144.stdout:1/768: rename d7/d4b/dd9/dd3/fe8 to d7/d1d/dac/de1/ff7 0 2024-05-15T15:54:22.819 INFO:tasks.workunit.client.1.smithi144.stdout:1/769: chown d7/d4b/c57 34 1 2024-05-15T15:54:22.821 INFO:tasks.workunit.client.1.smithi144.stdout:6/803: mkdir d7/db/d3a/de3/df2/d11e 0 2024-05-15T15:54:22.825 INFO:tasks.workunit.client.0.smithi049.stdout:6/696: dwrite dd/d17/d24/d2b/d36/d86/fae [0,4194304] 0 2024-05-15T15:54:22.826 INFO:tasks.workunit.client.0.smithi049.stdout:2/637: rename d3/db/d1d/d1e/d2b/d3b/l89 to d3/db/d1d/d1e/d2b/d2e/d67/ldd 0 2024-05-15T15:54:22.826 INFO:tasks.workunit.client.1.smithi144.stdout:6/804: dwrite d7/db/da2/fe0 [0,4194304] 0 2024-05-15T15:54:22.829 INFO:tasks.workunit.client.1.smithi144.stdout:7/772: truncate d4/df/d25/d38/f54 467291 0 2024-05-15T15:54:22.829 INFO:tasks.workunit.client.0.smithi049.stdout:0/719: dwrite dc/d13/d56/f90 [0,4194304] 0 2024-05-15T15:54:22.830 INFO:tasks.workunit.client.1.smithi144.stdout:7/773: chown d4/df/d79/c93 2550 1 2024-05-15T15:54:22.834 INFO:tasks.workunit.client.0.smithi049.stdout:6/697: dread dd/f16 [0,4194304] 0 2024-05-15T15:54:22.834 INFO:tasks.workunit.client.0.smithi049.stdout:1/872: mkdir d0/d4/dd/de/d48/d84/d134 0 2024-05-15T15:54:22.836 INFO:tasks.workunit.client.1.smithi144.stdout:8/835: creat d2/d24/d2d/d30/dd0/df7/f114 x:0 0 0 2024-05-15T15:54:22.837 INFO:tasks.workunit.client.0.smithi049.stdout:4/679: symlink d18/d21/lde 0 2024-05-15T15:54:22.842 INFO:tasks.workunit.client.1.smithi144.stdout:0/737: creat d4/df/f10b x:0 0 0 2024-05-15T15:54:22.856 INFO:tasks.workunit.client.1.smithi144.stdout:9/797: rename d0/d8/d21 to d0/d8/d1b/d114 0 2024-05-15T15:54:22.858 INFO:tasks.workunit.client.1.smithi144.stdout:5/801: dwrite d0/dc/d24/d28/d2d/f3d [0,4194304] 0 2024-05-15T15:54:22.862 INFO:tasks.workunit.client.0.smithi049.stdout:9/585: rename d0/d8/f7e to d0/d8/d1b/d71/d7d/dba/fca 0 2024-05-15T15:54:22.864 INFO:tasks.workunit.client.1.smithi144.stdout:4/792: write d6/dd/fbb [101803,120870] 0 2024-05-15T15:54:22.865 INFO:tasks.workunit.client.1.smithi144.stdout:6/805: mknod d7/db/d11/d7e/c11f 0 2024-05-15T15:54:22.867 INFO:tasks.workunit.client.0.smithi049.stdout:2/638: dread d3/db/d1d/d1e/f21 [0,4194304] 0 2024-05-15T15:54:22.867 INFO:tasks.workunit.client.0.smithi049.stdout:1/873: dread d0/d4/dd/de/d36/fee [0,4194304] 0 2024-05-15T15:54:22.873 INFO:tasks.workunit.client.1.smithi144.stdout:3/778: mknod d0/d1/d17/d39/d40/d4a/d9e/cfe 0 2024-05-15T15:54:22.874 INFO:tasks.workunit.client.1.smithi144.stdout:3/779: write d0/d11/d3b/da7/fc6 [1792302,15270] 0 2024-05-15T15:54:22.875 INFO:tasks.workunit.client.0.smithi049.stdout:4/680: rename d18/d85/d98/c9d to d18/d85/d98/dc3/cdf 0 2024-05-15T15:54:22.876 INFO:tasks.workunit.client.0.smithi049.stdout:2/639: dwrite d3/da/fc5 [0,4194304] 0 2024-05-15T15:54:22.877 INFO:tasks.workunit.client.1.smithi144.stdout:9/798: dread d0/f5 [0,4194304] 0 2024-05-15T15:54:22.877 INFO:tasks.workunit.client.0.smithi049.stdout:6/698: dread dd/d26/f40 [0,4194304] 0 2024-05-15T15:54:22.878 INFO:tasks.workunit.client.1.smithi144.stdout:0/738: fsync d4/d6/f32 0 2024-05-15T15:54:22.891 INFO:tasks.workunit.client.0.smithi049.stdout:0/720: dwrite dc/d13/f17 [0,4194304] 0 2024-05-15T15:54:22.892 INFO:tasks.workunit.client.1.smithi144.stdout:8/836: dwrite d2/de/d49/d83/fa7 [0,4194304] 0 2024-05-15T15:54:22.892 INFO:tasks.workunit.client.0.smithi049.stdout:9/586: rename d0/d25/d5d/fc5 to d0/d8/d9/d52/d5e/d8a/dac/fcb 0 2024-05-15T15:54:22.898 INFO:tasks.workunit.client.0.smithi049.stdout:1/874: dread d0/d4/dd/de/d36/f4c [0,4194304] 0 2024-05-15T15:54:22.899 INFO:tasks.workunit.client.1.smithi144.stdout:5/802: creat d0/dc/d24/d8e/d93/f10b x:0 0 0 2024-05-15T15:54:22.901 INFO:tasks.workunit.client.0.smithi049.stdout:2/640: dread d3/db/d1d/d1e/d2b/d6b/f80 [0,4194304] 0 2024-05-15T15:54:22.904 INFO:tasks.workunit.client.0.smithi049.stdout:6/699: creat dd/d17/d24/d2b/d36/d93/fd4 x:0 0 0 2024-05-15T15:54:22.907 INFO:tasks.workunit.client.0.smithi049.stdout:9/587: symlink d0/d8/d9/lcc 0 2024-05-15T15:54:23.013 INFO:tasks.workunit.client.0.smithi049.stdout:4/681: unlink d18/d1e/d2a/d50/f90 0 2024-05-15T15:54:23.018 INFO:tasks.workunit.client.0.smithi049.stdout:2/641: fdatasync d3/db/d5b/dbf/dc1/fa5 0 2024-05-15T15:54:23.021 INFO:tasks.workunit.client.1.smithi144.stdout:2/732: getdents d3/d7/d20/d37/d76/db5/db6/dbf 0 2024-05-15T15:54:23.024 INFO:tasks.workunit.client.1.smithi144.stdout:1/770: rename d7/d1d/d33/d29/d6b/d5e/d74/df3/f92 to d7/d1d/d33/d1e/d34/ff8 0 2024-05-15T15:54:23.025 INFO:tasks.workunit.client.0.smithi049.stdout:0/721: write dc/d13/d14/d33/f53 [1372436,60372] 0 2024-05-15T15:54:23.029 INFO:tasks.workunit.client.1.smithi144.stdout:2/733: dwrite d3/d7/d20/d89/d6e/f8f [0,4194304] 0 2024-05-15T15:54:23.031 INFO:tasks.workunit.client.1.smithi144.stdout:7/774: sync 2024-05-15T15:54:23.033 INFO:tasks.workunit.client.1.smithi144.stdout:6/806: dwrite d7/db/f48 [0,4194304] 0 2024-05-15T15:54:23.035 INFO:tasks.workunit.client.0.smithi049.stdout:9/588: dwrite d0/d8/d9/d45/f4a [0,4194304] 0 2024-05-15T15:54:23.036 INFO:tasks.workunit.client.1.smithi144.stdout:3/780: mknod d0/d1/d56/d71/cff 0 2024-05-15T15:54:23.039 INFO:tasks.workunit.client.1.smithi144.stdout:0/739: creat d4/d6/d3a/d40/f10c x:0 0 0 2024-05-15T15:54:23.039 INFO:tasks.workunit.client.1.smithi144.stdout:9/799: dwrite d0/d8/d1b/d79/f93 [0,4194304] 0 2024-05-15T15:54:23.042 INFO:tasks.workunit.client.1.smithi144.stdout:1/771: rmdir d7/d1d/d33/d29/dd2 39 2024-05-15T15:54:23.045 INFO:tasks.workunit.client.0.smithi049.stdout:6/700: dread dd/f89 [0,4194304] 0 2024-05-15T15:54:23.050 INFO:tasks.workunit.client.1.smithi144.stdout:6/807: symlink d7/db/da2/d70/db6/l120 0 2024-05-15T15:54:23.050 INFO:tasks.workunit.client.1.smithi144.stdout:6/808: write d7/db/d11/fa5 [1098313,48646] 0 2024-05-15T15:54:23.051 INFO:tasks.workunit.client.1.smithi144.stdout:3/781: truncate d0/d1/d17/f33 1410236 0 2024-05-15T15:54:23.055 INFO:tasks.workunit.client.1.smithi144.stdout:4/793: getdents d6/dd/d34 0 2024-05-15T15:54:23.056 INFO:tasks.workunit.client.1.smithi144.stdout:4/794: write d6/dd/fd2 [1342052,9605] 0 2024-05-15T15:54:23.061 INFO:tasks.workunit.client.1.smithi144.stdout:5/803: rename d0/dc/d12/d4a to d0/d10c 0 2024-05-15T15:54:23.062 INFO:tasks.workunit.client.0.smithi049.stdout:4/682: creat d18/d1e/d63/d99/db9/fe0 x:0 0 0 2024-05-15T15:54:23.063 INFO:tasks.workunit.client.0.smithi049.stdout:1/875: truncate d0/d4/dd/de/d48/d84/f12e 969153 0 2024-05-15T15:54:23.063 INFO:tasks.workunit.client.1.smithi144.stdout:8/837: getdents d2/d24/d2d/da0/df8 0 2024-05-15T15:54:23.064 INFO:tasks.workunit.client.0.smithi049.stdout:2/642: fsync d3/db/d1d/d1e/f3c 0 2024-05-15T15:54:23.066 INFO:tasks.workunit.client.1.smithi144.stdout:4/795: mkdir d6/dd/d34/d3b/d30/db/d1d/d101 0 2024-05-15T15:54:23.066 INFO:tasks.workunit.client.0.smithi049.stdout:2/643: chown d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d 431 1 2024-05-15T15:54:23.067 INFO:tasks.workunit.client.0.smithi049.stdout:0/722: dread - dc/d13/d14/f95 zero size 2024-05-15T15:54:23.069 INFO:tasks.workunit.client.1.smithi144.stdout:5/804: creat d0/dc/d24/d8e/d93/d9e/f10d x:0 0 0 2024-05-15T15:54:23.069 INFO:tasks.workunit.client.0.smithi049.stdout:9/589: fsync d0/d8/d9/d52/d5e/d5f/fa0 0 2024-05-15T15:54:23.070 INFO:tasks.workunit.client.1.smithi144.stdout:5/805: write d0/dc/ff3 [685433,101028] 0 2024-05-15T15:54:23.070 INFO:tasks.workunit.client.1.smithi144.stdout:6/809: symlink d7/db/d35/d84/dbe/d118/l121 0 2024-05-15T15:54:23.071 INFO:tasks.workunit.client.0.smithi049.stdout:6/701: read dd/f16 [3179956,111302] 0 2024-05-15T15:54:23.076 INFO:tasks.workunit.client.1.smithi144.stdout:7/775: rename d4/d3c/d76/d9d/dac/l33 to d4/d3c/l10b 0 2024-05-15T15:54:23.079 INFO:tasks.workunit.client.1.smithi144.stdout:5/806: dwrite d0/dc/d24/d8e/d95/f101 [0,4194304] 0 2024-05-15T15:54:23.080 INFO:tasks.workunit.client.0.smithi049.stdout:2/644: write d3/db/d1d/d1e/d2b/d2e/d67/fac [638068,61857] 0 2024-05-15T15:54:23.082 INFO:tasks.workunit.client.0.smithi049.stdout:0/723: fsync dc/d13/d14/d2f/f52 0 2024-05-15T15:54:23.083 INFO:tasks.workunit.client.0.smithi049.stdout:0/724: chown dc/d13/d14/d2f/d8f/c99 132360 1 2024-05-15T15:54:23.084 INFO:tasks.workunit.client.0.smithi049.stdout:4/683: dread d18/f4d [0,4194304] 0 2024-05-15T15:54:23.085 INFO:tasks.workunit.client.1.smithi144.stdout:8/838: dread d2/d1a/f31 [0,4194304] 0 2024-05-15T15:54:23.086 INFO:tasks.workunit.client.0.smithi049.stdout:9/590: symlink d0/d8/d1b/d71/d7d/dba/lcd 0 2024-05-15T15:54:23.088 INFO:tasks.workunit.client.1.smithi144.stdout:3/782: getdents d0/d1 0 2024-05-15T15:54:23.095 INFO:tasks.workunit.client.1.smithi144.stdout:7/776: creat d4/d3c/d76/d9d/dba/f10c x:0 0 0 2024-05-15T15:54:23.096 INFO:tasks.workunit.client.1.smithi144.stdout:4/796: dread d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4/fb7 [0,4194304] 0 2024-05-15T15:54:23.097 INFO:tasks.workunit.client.1.smithi144.stdout:7/777: read d4/d3c/d76/d9d/f102 [207338,102010] 0 2024-05-15T15:54:23.098 INFO:tasks.workunit.client.0.smithi049.stdout:4/684: rename d18/d1e/d2a/d3b to d18/d1e/d63/d99/de1 0 2024-05-15T15:54:23.100 INFO:tasks.workunit.client.0.smithi049.stdout:4/685: read d18/d1e/d63/d99/de1/f57 [2385444,30654] 0 2024-05-15T15:54:23.100 INFO:tasks.workunit.client.0.smithi049.stdout:9/591: read d0/d8/d9c/f9f [2549659,63097] 0 2024-05-15T15:54:23.102 INFO:tasks.workunit.client.0.smithi049.stdout:4/686: truncate d18/d1e/d63/d99/db9/fe0 878114 0 2024-05-15T15:54:23.103 INFO:tasks.workunit.client.1.smithi144.stdout:9/800: rename d0/d8/d5e/d44/f87 to d0/d8/d5e/f115 0 2024-05-15T15:54:23.103 INFO:tasks.workunit.client.0.smithi049.stdout:4/687: stat d18/d69/l77 0 2024-05-15T15:54:23.104 INFO:tasks.workunit.client.0.smithi049.stdout:2/645: unlink d3/db/d82/la2 0 2024-05-15T15:54:23.104 INFO:tasks.workunit.client.0.smithi049.stdout:4/688: chown d18/d1e/d2a/d36 5 1 2024-05-15T15:54:23.107 INFO:tasks.workunit.client.0.smithi049.stdout:0/725: mknod dc/d13/d14/d25/d31/d71/dbf/cf2 0 2024-05-15T15:54:23.109 INFO:tasks.workunit.client.1.smithi144.stdout:5/807: write d0/dc/d2f/d34/dc1/ff5 [383487,74546] 0 2024-05-15T15:54:23.109 INFO:tasks.workunit.client.0.smithi049.stdout:9/592: dwrite d0/d8/d9/d22/fb1 [0,4194304] 0 2024-05-15T15:54:23.111 INFO:tasks.workunit.client.1.smithi144.stdout:4/797: mkdir d6/dd/d2e/d75/dff/deb/d102 0 2024-05-15T15:54:23.115 INFO:tasks.workunit.client.1.smithi144.stdout:7/778: truncate d4/df/f5f 254124 0 2024-05-15T15:54:23.115 INFO:tasks.workunit.client.0.smithi049.stdout:4/689: dwrite d18/d85/fab [0,4194304] 0 2024-05-15T15:54:23.115 INFO:tasks.workunit.client.1.smithi144.stdout:7/779: chown d4/df/d25/d40/d73/fe9 1847972470 1 2024-05-15T15:54:23.116 INFO:tasks.workunit.client.1.smithi144.stdout:7/780: chown d4/d41/d5b/ca9 25 1 2024-05-15T15:54:23.121 INFO:tasks.workunit.client.0.smithi049.stdout:2/646: creat d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/fde x:0 0 0 2024-05-15T15:54:23.129 INFO:tasks.workunit.client.0.smithi049.stdout:0/726: mknod dc/d13/d2b/d76/d8a/da2/db3/cf3 0 2024-05-15T15:54:23.132 INFO:tasks.workunit.client.0.smithi049.stdout:4/690: rmdir d18/d1e/d63/d99 39 2024-05-15T15:54:23.133 INFO:tasks.workunit.client.1.smithi144.stdout:4/798: creat d6/dd/d2e/d71/f103 x:0 0 0 2024-05-15T15:54:23.137 INFO:tasks.workunit.client.0.smithi049.stdout:1/876: dwrite d0/d4/f39 [4194304,4194304] 0 2024-05-15T15:54:23.159 INFO:tasks.workunit.client.1.smithi144.stdout:2/734: sync 2024-05-15T15:54:23.166 INFO:tasks.workunit.client.0.smithi049.stdout:8/662: sync 2024-05-15T15:54:23.187 INFO:tasks.workunit.client.1.smithi144.stdout:7/781: symlink d4/d3c/d76/d9d/dac/l10d 0 2024-05-15T15:54:23.187 INFO:tasks.workunit.client.1.smithi144.stdout:0/740: sync 2024-05-15T15:54:23.187 INFO:tasks.workunit.client.1.smithi144.stdout:6/810: write d7/db/da2/f66 [822289,80909] 0 2024-05-15T15:54:23.187 INFO:tasks.workunit.client.0.smithi049.stdout:2/647: rename d3/db/d1d/d1e/d2b/d3b/d42/d8f to d3/d8/daa/ddf 0 2024-05-15T15:54:23.190 INFO:tasks.workunit.client.1.smithi144.stdout:1/772: rename d7/d1d/d33/d29/d6b/d8e/cd4 to d7/d1d/d33/d41/d22/d45/cf9 0 2024-05-15T15:54:23.190 INFO:tasks.workunit.client.0.smithi049.stdout:6/702: write dd/d26/f4b [3276981,118737] 0 2024-05-15T15:54:23.191 INFO:tasks.workunit.client.0.smithi049.stdout:9/593: symlink d0/d8/d53/lce 0 2024-05-15T15:54:23.192 INFO:tasks.workunit.client.0.smithi049.stdout:9/594: readlink d0/d8e/la6 0 2024-05-15T15:54:23.195 INFO:tasks.workunit.client.0.smithi049.stdout:1/877: creat d0/d4/dd/de/d1e/d16/d8a/dd2/df2/f135 x:0 0 0 2024-05-15T15:54:23.196 INFO:tasks.workunit.client.1.smithi144.stdout:8/839: write d2/d24/d2d/d30/dd0/df7/f4d [2468386,14727] 0 2024-05-15T15:54:23.196 INFO:tasks.workunit.client.1.smithi144.stdout:2/735: chown d3/d7/d20/d22/d52/d64/d80/l99 1891153 1 2024-05-15T15:54:23.198 INFO:tasks.workunit.client.0.smithi049.stdout:8/663: mknod d2/de/d25/cce 0 2024-05-15T15:54:23.201 INFO:tasks.workunit.client.0.smithi049.stdout:2/648: creat d3/db/d1d/fe0 x:0 0 0 2024-05-15T15:54:23.202 INFO:tasks.workunit.client.0.smithi049.stdout:1/878: dwrite d0/fd1 [4194304,4194304] 0 2024-05-15T15:54:23.213 INFO:tasks.workunit.client.1.smithi144.stdout:9/801: dwrite d0/d8/d5e/d9b/f9f [0,4194304] 0 2024-05-15T15:54:23.213 INFO:tasks.workunit.client.0.smithi049.stdout:2/649: creat d3/d85/fe1 x:0 0 0 2024-05-15T15:54:23.215 INFO:tasks.workunit.client.0.smithi049.stdout:6/703: creat dd/d17/d24/d2b/d36/d50/dcf/fd5 x:0 0 0 2024-05-15T15:54:23.217 INFO:tasks.workunit.client.0.smithi049.stdout:1/879: read - d0/d4/dd/de/d1e/d2e/d3e/fe4 zero size 2024-05-15T15:54:23.220 INFO:tasks.workunit.client.1.smithi144.stdout:5/808: dwrite d0/dc/d2f/d68/d5c/f80 [0,4194304] 0 2024-05-15T15:54:23.220 INFO:tasks.workunit.client.0.smithi049.stdout:9/595: creat d0/d8/d42/d75/fcf x:0 0 0 2024-05-15T15:54:23.232 INFO:tasks.workunit.client.0.smithi049.stdout:6/704: creat dd/dbc/fd6 x:0 0 0 2024-05-15T15:54:23.232 INFO:tasks.workunit.client.0.smithi049.stdout:4/691: dwrite d18/d1e/d2a/d50/f5a [0,4194304] 0 2024-05-15T15:54:23.239 INFO:tasks.workunit.client.0.smithi049.stdout:8/664: write d2/de/f16 [813829,121547] 0 2024-05-15T15:54:23.242 INFO:tasks.workunit.client.0.smithi049.stdout:6/705: write dd/d17/d24/fd0 [4590273,57335] 0 2024-05-15T15:54:23.247 INFO:tasks.workunit.client.0.smithi049.stdout:4/692: symlink d18/d1e/d63/d99/le2 0 2024-05-15T15:54:23.250 INFO:tasks.workunit.client.0.smithi049.stdout:2/650: write d3/d8/f50 [208032,74077] 0 2024-05-15T15:54:23.252 INFO:tasks.workunit.client.0.smithi049.stdout:2/651: fsync d3/db/d1d/d1e/d2b/d6b/fd5 0 2024-05-15T15:54:23.253 INFO:tasks.workunit.client.0.smithi049.stdout:1/880: creat d0/d4/dd/de/d1e/d2e/dad/f136 x:0 0 0 2024-05-15T15:54:23.253 INFO:tasks.workunit.client.0.smithi049.stdout:2/652: chown d3/db/d1d/d1e/d2b/fd6 42697 1 2024-05-15T15:54:23.256 INFO:tasks.workunit.client.0.smithi049.stdout:4/693: symlink d18/d85/d98/le3 0 2024-05-15T15:54:23.263 INFO:tasks.workunit.client.0.smithi049.stdout:2/653: dread - d3/db/d5b/dbf/dc1/fa5 zero size 2024-05-15T15:54:23.264 INFO:tasks.workunit.client.0.smithi049.stdout:1/881: readlink d0/d4/dd/de/d4d/d8b/lce 0 2024-05-15T15:54:23.266 INFO:tasks.workunit.client.0.smithi049.stdout:6/706: dwrite dd/f57 [0,4194304] 0 2024-05-15T15:54:23.266 INFO:tasks.workunit.client.0.smithi049.stdout:1/882: dread - d0/d4/dd/de/d1e/d2e/dad/d65/d72/f78 zero size 2024-05-15T15:54:23.267 INFO:tasks.workunit.client.0.smithi049.stdout:9/596: link d0/d8e/cb5 d0/d8e/cd0 0 2024-05-15T15:54:23.268 INFO:tasks.workunit.client.0.smithi049.stdout:6/707: fsync dd/d17/d24/d2b/d36/f56 0 2024-05-15T15:54:23.269 INFO:tasks.workunit.client.0.smithi049.stdout:9/597: fdatasync d0/d8/d1b/d40/fbc 0 2024-05-15T15:54:23.271 INFO:tasks.workunit.client.0.smithi049.stdout:6/708: write dd/d26/f8b [318788,91055] 0 2024-05-15T15:54:23.273 INFO:tasks.workunit.client.0.smithi049.stdout:8/665: truncate d2/de/d71/d86/d5d/d7d/d1b/d68/fab 112775 0 2024-05-15T15:54:23.276 INFO:tasks.workunit.client.0.smithi049.stdout:1/883: creat d0/d4/dd/de/d1e/d16/f137 x:0 0 0 2024-05-15T15:54:23.279 INFO:tasks.workunit.client.0.smithi049.stdout:2/654: truncate d3/d8/f36 262073 0 2024-05-15T15:54:23.282 INFO:tasks.workunit.client.0.smithi049.stdout:6/709: rename dd/d17/d24/d2b/d36/d93 to dd/d17/d24/d58/d5f/dd7 0 2024-05-15T15:54:23.284 INFO:tasks.workunit.client.0.smithi049.stdout:9/598: dwrite d0/d8/d12/f6e [0,4194304] 0 2024-05-15T15:54:23.285 INFO:tasks.workunit.client.0.smithi049.stdout:8/666: fdatasync d2/de/d71/f82 0 2024-05-15T15:54:23.285 INFO:tasks.workunit.client.0.smithi049.stdout:6/710: fdatasync dd/d17/d24/d2b/d36/d50/d54/faf 0 2024-05-15T15:54:23.286 INFO:tasks.workunit.client.0.smithi049.stdout:9/599: read d0/d8/d9/d52/d5e/d5f/f38 [11708,127903] 0 2024-05-15T15:54:23.288 INFO:tasks.workunit.client.0.smithi049.stdout:8/667: truncate d2/de/d71/fba 353490 0 2024-05-15T15:54:23.288 INFO:tasks.workunit.client.0.smithi049.stdout:9/600: dread - d0/d25/f76 zero size 2024-05-15T15:54:23.290 INFO:tasks.workunit.client.0.smithi049.stdout:8/668: chown d2/de/d71/d86/d5d/d7d/db8/d5b/d60/f9b 4 1 2024-05-15T15:54:23.293 INFO:tasks.workunit.client.0.smithi049.stdout:2/655: chown d3/db/d1d/d1e/d2b/d2e/d67/c90 19 1 2024-05-15T15:54:23.301 INFO:tasks.workunit.client.0.smithi049.stdout:6/711: creat dd/d17/d24/d2b/d36/d50/d54/db6/fd8 x:0 0 0 2024-05-15T15:54:23.306 INFO:tasks.workunit.client.0.smithi049.stdout:2/656: mkdir d3/db/d1d/d1e/de2 0 2024-05-15T15:54:23.307 INFO:tasks.workunit.client.0.smithi049.stdout:2/657: fsync d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/dc9/fda 0 2024-05-15T15:54:23.312 INFO:tasks.workunit.client.1.smithi144.stdout:0/741: rmdir d4/d6/d3a 39 2024-05-15T15:54:23.314 INFO:tasks.workunit.client.0.smithi049.stdout:1/884: getdents d0/d4/dd/de/d1e/d16/df1 0 2024-05-15T15:54:23.315 INFO:tasks.workunit.client.0.smithi049.stdout:1/885: fsync d0/d4/dd/de/d1e/d2e/dad/d65/fc7 0 2024-05-15T15:54:23.317 INFO:tasks.workunit.client.1.smithi144.stdout:1/773: truncate d7/d1d/d33/d41/d78/fb1 2425419 0 2024-05-15T15:54:23.319 INFO:tasks.workunit.client.1.smithi144.stdout:8/840: read d2/d24/d2d/d30/dd0/df7/f1e [1422648,38615] 0 2024-05-15T15:54:23.323 INFO:tasks.workunit.client.0.smithi049.stdout:2/658: write d3/db/d1d/d1e/d2b/d3b/d47/f86 [2308722,115271] 0 2024-05-15T15:54:23.330 INFO:tasks.workunit.client.1.smithi144.stdout:7/782: mknod d4/d41/c10e 0 2024-05-15T15:54:23.331 INFO:tasks.workunit.client.1.smithi144.stdout:6/811: symlink d7/db/d11/d3c/dfc/l122 0 2024-05-15T15:54:23.332 INFO:tasks.workunit.client.1.smithi144.stdout:6/812: chown d7/d1e/d28/db5/de2 0 1 2024-05-15T15:54:23.332 INFO:tasks.workunit.client.0.smithi049.stdout:9/601: getdents d0/d8/d1b/d40/d5c 0 2024-05-15T15:54:23.334 INFO:tasks.workunit.client.0.smithi049.stdout:2/659: fsync d3/db/f43 0 2024-05-15T15:54:23.335 INFO:tasks.workunit.client.1.smithi144.stdout:1/774: mkdir d7/d1d/d33/d29/d6b/d5e/d74/dfa 0 2024-05-15T15:54:23.335 INFO:tasks.workunit.client.0.smithi049.stdout:8/669: link d2/de/d71/d86/f6d d2/de/d71/d86/fcf 0 2024-05-15T15:54:23.336 INFO:tasks.workunit.client.0.smithi049.stdout:8/670: chown d2/l37 663884 1 2024-05-15T15:54:23.336 INFO:tasks.workunit.client.1.smithi144.stdout:8/841: fdatasync d2/d24/d2d/d30/dd0/df7/f1e 0 2024-05-15T15:54:23.338 INFO:tasks.workunit.client.0.smithi049.stdout:6/712: getdents dd/d17/d24/d2b/d36/d86 0 2024-05-15T15:54:23.342 INFO:tasks.workunit.client.1.smithi144.stdout:9/802: stat d0/d8/d5e/d44/dc2/d75/fb1 0 2024-05-15T15:54:23.342 INFO:tasks.workunit.client.0.smithi049.stdout:1/886: creat d0/d4/dd/de/d1e/d2e/dad/f138 x:0 0 0 2024-05-15T15:54:23.343 INFO:tasks.workunit.client.1.smithi144.stdout:7/783: creat d4/df/d25/d38/d68/f10f x:0 0 0 2024-05-15T15:54:23.344 INFO:tasks.workunit.client.0.smithi049.stdout:6/713: fsync dd/d17/d24/f27 0 2024-05-15T15:54:23.345 INFO:tasks.workunit.client.0.smithi049.stdout:9/602: getdents d0/d8/d9/d22/dbb 0 2024-05-15T15:54:23.348 INFO:tasks.workunit.client.1.smithi144.stdout:8/842: rmdir d2/d3a 39 2024-05-15T15:54:23.350 INFO:tasks.workunit.client.1.smithi144.stdout:2/736: creat d3/d7/d20/ff3 x:0 0 0 2024-05-15T15:54:23.351 INFO:tasks.workunit.client.1.smithi144.stdout:9/803: creat d0/d8/d1b/d62/d6b/f116 x:0 0 0 2024-05-15T15:54:23.351 INFO:tasks.workunit.client.0.smithi049.stdout:8/671: fsync d2/de/d71/d86/d5d/d7d/d1b/d68/f97 0 2024-05-15T15:54:23.352 INFO:tasks.workunit.client.1.smithi144.stdout:9/804: readlink d0/d8/d1b/d62/d7d/d8d/d94/ldf 0 2024-05-15T15:54:23.353 INFO:tasks.workunit.client.1.smithi144.stdout:5/809: creat d0/dc/d2f/f10e x:0 0 0 2024-05-15T15:54:23.354 INFO:tasks.workunit.client.1.smithi144.stdout:5/810: dread d0/dc/d1f/f75 [0,4194304] 0 2024-05-15T15:54:23.355 INFO:tasks.workunit.client.1.smithi144.stdout:0/742: creat d4/df/d1f/d56/f10d x:0 0 0 2024-05-15T15:54:23.355 INFO:tasks.workunit.client.1.smithi144.stdout:5/811: fsync d0/dc/d24/d28/d2d/d59/d74/ff2 0 2024-05-15T15:54:23.356 INFO:tasks.workunit.client.1.smithi144.stdout:0/743: chown d4/d6/d4a/d74/de4 4301 1 2024-05-15T15:54:23.359 INFO:tasks.workunit.client.0.smithi049.stdout:1/887: dread d0/d4/dd/de/f17 [4194304,4194304] 0 2024-05-15T15:54:23.360 INFO:tasks.workunit.client.0.smithi049.stdout:1/888: dread - d0/d4/dd/de/d4d/fcf zero size 2024-05-15T15:54:23.360 INFO:tasks.workunit.client.1.smithi144.stdout:3/783: rename d0/d1/d56/l60 to d0/d1/d17/d39/d40/d4a/l100 0 2024-05-15T15:54:23.363 INFO:tasks.workunit.client.0.smithi049.stdout:6/714: truncate dd/f6c 2394654 0 2024-05-15T15:54:23.364 INFO:tasks.workunit.client.0.smithi049.stdout:2/660: getdents d3/d8/daa 0 2024-05-15T15:54:23.364 INFO:tasks.workunit.client.0.smithi049.stdout:6/715: chown dd/d17/d64/d8c/fce 693472 1 2024-05-15T15:54:23.367 INFO:tasks.workunit.client.1.smithi144.stdout:1/775: truncate d7/d1d/f32 1514754 0 2024-05-15T15:54:23.368 INFO:tasks.workunit.client.1.smithi144.stdout:8/843: creat d2/d24/d2d/d30/dd0/df7/f115 x:0 0 0 2024-05-15T15:54:23.369 INFO:tasks.workunit.client.1.smithi144.stdout:2/737: readlink d3/l66 0 2024-05-15T15:54:23.370 INFO:tasks.workunit.client.0.smithi049.stdout:8/672: dwrite d2/f5a [0,4194304] 0 2024-05-15T15:54:23.371 INFO:tasks.workunit.client.0.smithi049.stdout:2/661: dwrite d3/db/d5b/f70 [0,4194304] 0 2024-05-15T15:54:23.380 INFO:tasks.workunit.client.0.smithi049.stdout:8/673: dread d2/de/d71/d86/d5d/d7d/d1b/f36 [0,4194304] 0 2024-05-15T15:54:23.381 INFO:tasks.workunit.client.0.smithi049.stdout:8/674: read - d2/de/d71/d86/f6d zero size 2024-05-15T15:54:23.384 INFO:tasks.workunit.client.0.smithi049.stdout:3/689: sync 2024-05-15T15:54:23.385 INFO:tasks.workunit.client.0.smithi049.stdout:6/716: rmdir dd/d17/d24/d2b/d36 39 2024-05-15T15:54:23.386 INFO:tasks.workunit.client.1.smithi144.stdout:9/805: fsync d0/d8/d1b/d62/d6b/f7c 0 2024-05-15T15:54:23.387 INFO:tasks.workunit.client.0.smithi049.stdout:1/889: mkdir d0/d4/dd/de/d1e/d16/d8a/dd2/d110/d139 0 2024-05-15T15:54:23.390 INFO:tasks.workunit.client.1.smithi144.stdout:7/784: mknod d4/df/c110 0 2024-05-15T15:54:23.391 INFO:tasks.workunit.client.1.smithi144.stdout:0/744: dread - d4/d2d/dae/fe8 zero size 2024-05-15T15:54:23.393 INFO:tasks.workunit.client.0.smithi049.stdout:2/662: rename d3/db/d1d/d1e/d2b/d2e/d67/fc4 to d3/da/d15/d73/fe3 0 2024-05-15T15:54:23.394 INFO:tasks.workunit.client.1.smithi144.stdout:6/813: creat d7/f123 x:0 0 0 2024-05-15T15:54:23.394 INFO:tasks.workunit.client.1.smithi144.stdout:6/814: chown d7/d62 4016 1 2024-05-15T15:54:23.396 INFO:tasks.workunit.client.0.smithi049.stdout:2/663: read d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d [70521,103897] 0 2024-05-15T15:54:23.396 INFO:tasks.workunit.client.1.smithi144.stdout:3/784: symlink d0/d11/d3b/da0/l101 0 2024-05-15T15:54:23.399 INFO:tasks.workunit.client.0.smithi049.stdout:1/890: symlink d0/d4/dd/de/d1e/d2e/d3e/d90/l13a 0 2024-05-15T15:54:23.401 INFO:tasks.workunit.client.0.smithi049.stdout:1/891: fsync d0/d4/dd/de/d1e/d16/d6e/f100 0 2024-05-15T15:54:23.403 INFO:tasks.workunit.client.1.smithi144.stdout:5/812: dread d0/dc/d24/f37 [4194304,4194304] 0 2024-05-15T15:54:23.405 INFO:tasks.workunit.client.1.smithi144.stdout:8/844: symlink d2/d24/l116 0 2024-05-15T15:54:23.408 INFO:tasks.workunit.client.0.smithi049.stdout:6/717: rename dd/d26/d5b/f87 to dd/d26/d5b/da8/fd9 0 2024-05-15T15:54:23.409 INFO:tasks.workunit.client.0.smithi049.stdout:1/892: dread d0/f5b [0,4194304] 0 2024-05-15T15:54:23.410 INFO:tasks.workunit.client.0.smithi049.stdout:8/675: mkdir d2/de/d71/d86/d5d/d7d/db8/d5b/dd0 0 2024-05-15T15:54:23.410 INFO:tasks.workunit.client.0.smithi049.stdout:1/893: readlink d0/d4/dd/de/d1e/d2e/dad/d65/d72/l86 0 2024-05-15T15:54:23.411 INFO:tasks.workunit.client.0.smithi049.stdout:8/676: fdatasync d2/de/d71/d86/d5d/d7d/d45/f7c 0 2024-05-15T15:54:23.414 INFO:tasks.workunit.client.0.smithi049.stdout:3/690: link d12/fcc d12/d14/d52/db9/dc4/fd3 0 2024-05-15T15:54:23.415 INFO:tasks.workunit.client.1.smithi144.stdout:0/745: creat d4/d2d/f10e x:0 0 0 2024-05-15T15:54:23.418 INFO:tasks.workunit.client.1.smithi144.stdout:4/799: rename d6/dd/d34/d3b/d30/db/d1d/d40/f79 to d6/dd/d34/d3b/d30/db/d1d/f104 0 2024-05-15T15:54:23.418 INFO:tasks.workunit.client.0.smithi049.stdout:2/664: dread d3/db/d5b/dbf/dc1/fb1 [0,4194304] 0 2024-05-15T15:54:23.422 INFO:tasks.workunit.client.0.smithi049.stdout:3/691: mknod d12/d14/d32/d64/cd4 0 2024-05-15T15:54:23.422 INFO:tasks.workunit.client.0.smithi049.stdout:3/692: readlink d12/d14/d52/db9/d6b/lbc 0 2024-05-15T15:54:23.423 INFO:tasks.workunit.client.1.smithi144.stdout:2/738: write d3/d7/d20/d3b/fd2 [122571,53344] 0 2024-05-15T15:54:23.426 INFO:tasks.workunit.client.0.smithi049.stdout:6/718: symlink dd/d17/d24/d8a/dc2/lda 0 2024-05-15T15:54:23.426 INFO:tasks.workunit.client.1.smithi144.stdout:9/806: write d0/d8/d1b/d114/f2a [493603,71758] 0 2024-05-15T15:54:23.427 INFO:tasks.workunit.client.0.smithi049.stdout:2/665: readlink d3/db/d1d/d1e/d2b/d3b/l9d 0 2024-05-15T15:54:23.429 INFO:tasks.workunit.client.0.smithi049.stdout:3/693: mkdir d12/d14/dd5 0 2024-05-15T15:54:23.430 INFO:tasks.workunit.client.1.smithi144.stdout:7/785: rename d4/d3c/d76/d9d/dac/d48/ded to d4/d3c/d76/d9d/dac/d111 0 2024-05-15T15:54:23.431 INFO:tasks.workunit.client.0.smithi049.stdout:2/666: dread d3/f11 [0,4194304] 0 2024-05-15T15:54:23.433 INFO:tasks.workunit.client.0.smithi049.stdout:2/667: write d3/db/d1d/d1e/d2b/d2e/f72 [3407346,112435] 0 2024-05-15T15:54:23.440 INFO:tasks.workunit.client.1.smithi144.stdout:8/845: creat d2/d3a/d7b/d4b/f117 x:0 0 0 2024-05-15T15:54:23.444 INFO:tasks.workunit.client.0.smithi049.stdout:2/668: mkdir d3/db/d1d/d1e/d2b/d3b/d42/de4 0 2024-05-15T15:54:23.444 INFO:tasks.workunit.client.0.smithi049.stdout:6/719: write dd/d17/f4a [400854,68919] 0 2024-05-15T15:54:23.445 INFO:tasks.workunit.client.1.smithi144.stdout:0/746: mkdir d4/d6/d3a/d4c/dd6/d55/da5/d10f 0 2024-05-15T15:54:23.447 INFO:tasks.workunit.client.0.smithi049.stdout:3/694: creat d12/d30/fd6 x:0 0 0 2024-05-15T15:54:23.449 INFO:tasks.workunit.client.0.smithi049.stdout:6/720: mkdir dd/dbc/ddb 0 2024-05-15T15:54:23.451 INFO:tasks.workunit.client.0.smithi049.stdout:2/669: rename d3/db/d5b/dbf/dc1/fb1 to d3/d8/daa/ddf/fe5 0 2024-05-15T15:54:23.456 INFO:tasks.workunit.client.1.smithi144.stdout:9/807: dwrite d0/d8/d1b/d62/d6b/f7c [0,4194304] 0 2024-05-15T15:54:23.457 INFO:tasks.workunit.client.1.smithi144.stdout:8/846: rename d2/de/d3f/d107 to d2/d3a/da5/d118 0 2024-05-15T15:54:23.457 INFO:tasks.workunit.client.1.smithi144.stdout:4/800: truncate d6/dd/d34/d3b/d30/db/d1d/f41 1502672 0 2024-05-15T15:54:23.460 INFO:tasks.workunit.client.1.smithi144.stdout:5/813: write d0/dc/d12/fd7 [309615,42820] 0 2024-05-15T15:54:23.461 INFO:tasks.workunit.client.1.smithi144.stdout:1/776: dwrite d7/d1d/f32 [0,4194304] 0 2024-05-15T15:54:23.461 INFO:tasks.workunit.client.0.smithi049.stdout:2/670: dread d3/f11 [0,4194304] 0 2024-05-15T15:54:23.463 INFO:tasks.workunit.client.0.smithi049.stdout:3/695: creat d12/d14/d52/db9/d6b/fd7 x:0 0 0 2024-05-15T15:54:23.466 INFO:tasks.workunit.client.1.smithi144.stdout:2/739: creat d3/ff4 x:0 0 0 2024-05-15T15:54:23.467 INFO:tasks.workunit.client.1.smithi144.stdout:2/740: fsync d3/f5 0 2024-05-15T15:54:23.468 INFO:tasks.workunit.client.0.smithi049.stdout:2/671: dread d3/db/d5b/f70 [0,4194304] 0 2024-05-15T15:54:23.468 INFO:tasks.workunit.client.1.smithi144.stdout:2/741: readlink d3/d7/l4b 0 2024-05-15T15:54:23.474 INFO:tasks.workunit.client.1.smithi144.stdout:8/847: truncate d2/d24/d2d/da0/d6e/d71/d8c/fc0 230066 0 2024-05-15T15:54:23.475 INFO:tasks.workunit.client.1.smithi144.stdout:4/801: dread d6/dd/d34/d3b/d30/db/d1d/f104 [0,4194304] 0 2024-05-15T15:54:23.476 INFO:tasks.workunit.client.0.smithi049.stdout:2/672: unlink d3/db/d1d/d1e/d2b/d3b/ld3 0 2024-05-15T15:54:23.476 INFO:tasks.workunit.client.0.smithi049.stdout:2/673: readlink d3/db/d1d/l53 0 2024-05-15T15:54:23.477 INFO:tasks.workunit.client.0.smithi049.stdout:3/696: dread d12/d14/d32/f62 [0,4194304] 0 2024-05-15T15:54:23.477 INFO:tasks.workunit.client.0.smithi049.stdout:6/721: getdents dd/d17/d24/d2b/d36/d86/d96 0 2024-05-15T15:54:23.478 INFO:tasks.workunit.client.0.smithi049.stdout:6/722: stat dd/d17/d64/fbd 0 2024-05-15T15:54:23.479 INFO:tasks.workunit.client.1.smithi144.stdout:5/814: symlink d0/dc/d24/d8e/d93/d6b/l10f 0 2024-05-15T15:54:23.487 INFO:tasks.workunit.client.0.smithi049.stdout:6/723: rename dd/d17/d9d/fa6 to dd/d17/d24/d2b/d36/d50/d7e/fdc 0 2024-05-15T15:54:23.489 INFO:tasks.workunit.client.1.smithi144.stdout:2/742: rmdir d3/d7/d20/d22 39 2024-05-15T15:54:23.490 INFO:tasks.workunit.client.0.smithi049.stdout:2/674: write d3/db/d1d/d1e/d2b/d3b/d42/f4f [1499500,120172] 0 2024-05-15T15:54:23.490 INFO:tasks.workunit.client.0.smithi049.stdout:2/675: readlink d3/l88 0 2024-05-15T15:54:23.491 INFO:tasks.workunit.client.1.smithi144.stdout:8/848: mkdir d2/da8/d119 0 2024-05-15T15:54:23.491 INFO:tasks.workunit.client.0.smithi049.stdout:2/676: chown d3/db/d1d/d1e/d2b/d3b 2239130 1 2024-05-15T15:54:23.497 INFO:tasks.workunit.client.0.smithi049.stdout:3/697: dwrite d12/d14/d52/db9/d6b/f80 [0,4194304] 0 2024-05-15T15:54:23.498 INFO:tasks.workunit.client.0.smithi049.stdout:3/698: chown d12/d14/d52/fce 470277831 1 2024-05-15T15:54:23.499 INFO:tasks.workunit.client.0.smithi049.stdout:3/699: dread - d12/d30/d66/fca zero size 2024-05-15T15:54:23.500 INFO:tasks.workunit.client.1.smithi144.stdout:5/815: rename d0/dc/d24/l104 to d0/dc/d24/d8e/d93/d6b/dea/dfa/l110 0 2024-05-15T15:54:23.501 INFO:tasks.workunit.client.0.smithi049.stdout:2/677: readlink d3/d78/lcd 0 2024-05-15T15:54:23.502 INFO:tasks.workunit.client.0.smithi049.stdout:2/678: chown d3/db/d1d/d1e/de2 4 1 2024-05-15T15:54:23.503 INFO:tasks.workunit.client.1.smithi144.stdout:0/747: creat d4/df/f110 x:0 0 0 2024-05-15T15:54:23.504 INFO:tasks.workunit.client.0.smithi049.stdout:6/724: write dd/d17/d24/d2b/f63 [779996,52569] 0 2024-05-15T15:54:23.508 INFO:tasks.workunit.client.0.smithi049.stdout:6/725: fdatasync dd/d17/d24/d2b/d36/d50/dcf/fd5 0 2024-05-15T15:54:23.508 INFO:tasks.workunit.client.1.smithi144.stdout:1/777: dwrite d7/d1d/d33/d41/d22/dbd/fc0 [0,4194304] 0 2024-05-15T15:54:23.508 INFO:tasks.workunit.client.1.smithi144.stdout:9/808: creat d0/d8/d5e/d44/dc2/f117 x:0 0 0 2024-05-15T15:54:23.511 INFO:tasks.workunit.client.1.smithi144.stdout:6/815: sync 2024-05-15T15:54:23.511 INFO:tasks.workunit.client.1.smithi144.stdout:8/849: read - d2/d24/d2d/da0/d6e/d71/fba zero size 2024-05-15T15:54:23.512 INFO:tasks.workunit.client.0.smithi049.stdout:6/726: fsync dd/d26/f4f 0 2024-05-15T15:54:23.513 INFO:tasks.workunit.client.1.smithi144.stdout:8/850: chown d2/d24/d2d/da0/d6e/c70 2599693 1 2024-05-15T15:54:23.513 INFO:tasks.workunit.client.1.smithi144.stdout:4/802: symlink d6/l105 0 2024-05-15T15:54:23.514 INFO:tasks.workunit.client.0.smithi049.stdout:3/700: mknod d12/d14/cd8 0 2024-05-15T15:54:23.515 INFO:tasks.workunit.client.1.smithi144.stdout:8/851: readlink d2/d24/d2d/d30/dd0/df7/l63 0 2024-05-15T15:54:23.516 INFO:tasks.workunit.client.0.smithi049.stdout:2/679: write d3/db/d1d/d1e/d2b/d3b/d47/d59/f8e [943423,14718] 0 2024-05-15T15:54:23.516 INFO:tasks.workunit.client.1.smithi144.stdout:4/803: fsync d6/dd/d34/d3b/d30/d59/d66/f9e 0 2024-05-15T15:54:23.518 INFO:tasks.workunit.client.1.smithi144.stdout:6/816: dwrite d7/d1e/d28/db5/df6/f105 [0,4194304] 0 2024-05-15T15:54:23.518 INFO:tasks.workunit.client.0.smithi049.stdout:3/701: read d12/d14/d32/fd0 [6627883,116442] 0 2024-05-15T15:54:23.520 INFO:tasks.workunit.client.0.smithi049.stdout:2/680: write d3/db/d82/fbc [3884526,80254] 0 2024-05-15T15:54:23.523 INFO:tasks.workunit.client.0.smithi049.stdout:2/681: fsync d3/da/d15/d73/fe3 0 2024-05-15T15:54:23.527 INFO:tasks.workunit.client.1.smithi144.stdout:1/778: rename d7/d1d/d33/d41/d22/c6c to d7/d1d/d33/d41/d78/da3/cfb 0 2024-05-15T15:54:23.529 INFO:tasks.workunit.client.0.smithi049.stdout:3/702: unlink d12/d14/d32/c87 0 2024-05-15T15:54:23.531 INFO:tasks.workunit.client.0.smithi049.stdout:2/682: dwrite d3/db/d1d/d1e/d2b/d2e/f72 [0,4194304] 0 2024-05-15T15:54:23.535 INFO:tasks.workunit.client.1.smithi144.stdout:8/852: fdatasync d2/d24/f2b 0 2024-05-15T15:54:23.536 INFO:tasks.workunit.client.0.smithi049.stdout:2/683: creat d3/db/d1d/d1e/d2b/d3b/d42/da8/fe6 x:0 0 0 2024-05-15T15:54:23.538 INFO:tasks.workunit.client.0.smithi049.stdout:6/727: dread dd/d17/f8d [0,4194304] 0 2024-05-15T15:54:23.539 INFO:tasks.workunit.client.1.smithi144.stdout:0/748: dwrite d4/d6/f32 [0,4194304] 0 2024-05-15T15:54:23.539 INFO:tasks.workunit.client.1.smithi144.stdout:9/809: mknod d0/c118 0 2024-05-15T15:54:23.541 INFO:tasks.workunit.client.0.smithi049.stdout:6/728: mknod dd/d26/d5b/cdd 0 2024-05-15T15:54:23.541 INFO:tasks.workunit.client.0.smithi049.stdout:6/729: chown dd/dbc/fd6 9837 1 2024-05-15T15:54:23.544 INFO:tasks.workunit.client.0.smithi049.stdout:2/684: truncate d3/d78/d8c/d91/f9c 746429 0 2024-05-15T15:54:23.545 INFO:tasks.workunit.client.0.smithi049.stdout:6/730: read dd/d26/f4b [98239,80452] 0 2024-05-15T15:54:23.549 INFO:tasks.workunit.client.1.smithi144.stdout:4/804: link d6/fe7 d6/dd/d2e/d94/f106 0 2024-05-15T15:54:23.551 INFO:tasks.workunit.client.1.smithi144.stdout:4/805: write d6/dd/fbb [418332,128089] 0 2024-05-15T15:54:23.553 INFO:tasks.workunit.client.1.smithi144.stdout:8/853: creat d2/d24/d2d/da0/df8/f11a x:0 0 0 2024-05-15T15:54:23.553 INFO:tasks.workunit.client.1.smithi144.stdout:5/816: truncate d0/f22 409236 0 2024-05-15T15:54:23.553 INFO:tasks.workunit.client.1.smithi144.stdout:8/854: fdatasync d2/da8/ffb 0 2024-05-15T15:54:23.554 INFO:tasks.workunit.client.1.smithi144.stdout:8/855: read - d2/de/ff4 zero size 2024-05-15T15:54:23.555 INFO:tasks.workunit.client.0.smithi049.stdout:6/731: rename dd/d17/d64 to dd/d17/d24/d2b/d36/d50/d54/dde 0 2024-05-15T15:54:23.556 INFO:tasks.workunit.client.0.smithi049.stdout:6/732: chown dd/d17/d24/d2b/d36/f41 10072352 1 2024-05-15T15:54:23.568 INFO:tasks.workunit.client.1.smithi144.stdout:7/786: sync 2024-05-15T15:54:23.568 INFO:tasks.workunit.client.1.smithi144.stdout:3/785: sync 2024-05-15T15:54:23.592 INFO:tasks.workunit.client.0.smithi049.stdout:5/784: sync 2024-05-15T15:54:23.592 INFO:tasks.workunit.client.0.smithi049.stdout:7/663: sync 2024-05-15T15:54:23.595 INFO:tasks.workunit.client.1.smithi144.stdout:0/749: fdatasync d4/d6/d3a/d40/fd9 0 2024-05-15T15:54:23.596 INFO:tasks.workunit.client.0.smithi049.stdout:7/664: dread d0/d6/d1d/f7a [0,4194304] 0 2024-05-15T15:54:23.598 INFO:tasks.workunit.client.1.smithi144.stdout:1/779: dwrite d7/d1d/d58/fa6 [0,4194304] 0 2024-05-15T15:54:23.601 INFO:tasks.workunit.client.0.smithi049.stdout:2/685: creat d3/d78/d8c/d91/d7a/fe7 x:0 0 0 2024-05-15T15:54:23.602 INFO:tasks.workunit.client.1.smithi144.stdout:2/743: link d3/d10/c39 d3/d7/d20/d22/d52/d64/cf5 0 2024-05-15T15:54:23.604 INFO:tasks.workunit.client.0.smithi049.stdout:6/733: creat dd/dbc/fdf x:0 0 0 2024-05-15T15:54:23.611 INFO:tasks.workunit.client.1.smithi144.stdout:0/750: dread d4/d2d/f44 [0,4194304] 0 2024-05-15T15:54:23.615 INFO:tasks.workunit.client.0.smithi049.stdout:7/665: symlink d0/d34/db3/ld2 0 2024-05-15T15:54:23.616 INFO:tasks.workunit.client.1.smithi144.stdout:5/817: rmdir d0/dc/d24/d8e/d95 39 2024-05-15T15:54:23.617 INFO:tasks.workunit.client.1.smithi144.stdout:4/806: dread d6/d42/d46/d4b/d1e/f51 [0,4194304] 0 2024-05-15T15:54:23.622 INFO:tasks.workunit.client.0.smithi049.stdout:2/686: dread d3/db/d1d/d1e/d2b/d3b/d42/f6d [0,4194304] 0 2024-05-15T15:54:23.624 INFO:tasks.workunit.client.0.smithi049.stdout:5/785: mkdir dc/db0/dd5/d102 0 2024-05-15T15:54:23.625 INFO:tasks.workunit.client.1.smithi144.stdout:7/787: mkdir d4/df/d25/d40/d73/d112 0 2024-05-15T15:54:23.626 INFO:tasks.workunit.client.1.smithi144.stdout:6/817: link d7/db/da2/caf d7/d32/c124 0 2024-05-15T15:54:23.634 INFO:tasks.workunit.client.0.smithi049.stdout:6/734: creat dd/dbc/ddb/fe0 x:0 0 0 2024-05-15T15:54:23.634 INFO:tasks.workunit.client.1.smithi144.stdout:3/786: dwrite d0/d11/d24/fb8 [0,4194304] 0 2024-05-15T15:54:23.635 INFO:tasks.workunit.client.1.smithi144.stdout:3/787: stat d0/d1/d17/d39/dc8/fdc 0 2024-05-15T15:54:23.638 INFO:tasks.workunit.client.1.smithi144.stdout:7/788: dread d4/df/d81/dfb/fc3 [0,4194304] 0 2024-05-15T15:54:23.638 INFO:tasks.workunit.client.0.smithi049.stdout:2/687: creat d3/db/d1d/d1e/dd9/fe8 x:0 0 0 2024-05-15T15:54:23.640 INFO:tasks.workunit.client.1.smithi144.stdout:0/751: creat d4/df/d14/f111 x:0 0 0 2024-05-15T15:54:23.641 INFO:tasks.workunit.client.0.smithi049.stdout:5/786: symlink dc/db0/df2/l103 0 2024-05-15T15:54:23.641 INFO:tasks.workunit.client.0.smithi049.stdout:2/688: chown d3/db/d5b/dbf/dc1/c7b 29686 1 2024-05-15T15:54:23.643 INFO:tasks.workunit.client.0.smithi049.stdout:7/666: dwrite d0/d2f/f8f [0,4194304] 0 2024-05-15T15:54:23.645 INFO:tasks.workunit.client.1.smithi144.stdout:3/788: dwrite d0/d11/d24/d78/ff2 [0,4194304] 0 2024-05-15T15:54:23.647 INFO:tasks.workunit.client.0.smithi049.stdout:6/735: fdatasync dd/f80 0 2024-05-15T15:54:23.650 INFO:tasks.workunit.client.1.smithi144.stdout:4/807: symlink d6/dd/d34/d3b/d30/l107 0 2024-05-15T15:54:23.653 INFO:tasks.workunit.client.1.smithi144.stdout:9/810: dwrite d0/d8/d5e/d44/d10/f2c [0,4194304] 0 2024-05-15T15:54:23.653 INFO:tasks.workunit.client.1.smithi144.stdout:5/818: write d0/dc/d24/d57/fa6 [1163067,22095] 0 2024-05-15T15:54:23.654 INFO:tasks.workunit.client.1.smithi144.stdout:5/819: dread - d0/dc/d24/d8e/d93/f10b zero size 2024-05-15T15:54:23.659 INFO:tasks.workunit.client.0.smithi049.stdout:2/689: fsync d3/da/f45 0 2024-05-15T15:54:23.667 INFO:tasks.workunit.client.1.smithi144.stdout:7/789: rmdir d4/d3c/d76/d9d/dac/de5 39 2024-05-15T15:54:23.670 INFO:tasks.workunit.client.0.smithi049.stdout:2/690: fsync d3/db/d1d/d1e/d2b/f75 0 2024-05-15T15:54:23.671 INFO:tasks.workunit.client.1.smithi144.stdout:0/752: rmdir d4/d6/d3a/d4c 39 2024-05-15T15:54:23.673 INFO:tasks.workunit.client.0.smithi049.stdout:7/667: mkdir d0/d6/d23/d45/d6a/da1/d82/dd3 0 2024-05-15T15:54:23.675 INFO:tasks.workunit.client.0.smithi049.stdout:7/668: fdatasync d0/d6/da/fc0 0 2024-05-15T15:54:23.677 INFO:tasks.workunit.client.0.smithi049.stdout:6/736: link dd/d17/d24/d2b/d36/d86/d96/f9c dd/d17/d24/d2b/d36/d86/d96/fe1 0 2024-05-15T15:54:23.677 INFO:tasks.workunit.client.0.smithi049.stdout:7/669: fsync d0/d6/da/fc2 0 2024-05-15T15:54:23.677 INFO:tasks.workunit.client.0.smithi049.stdout:5/787: dwrite dc/d29/f32 [0,4194304] 0 2024-05-15T15:54:23.677 INFO:tasks.workunit.client.1.smithi144.stdout:6/818: write d7/fd [152983,105309] 0 2024-05-15T15:54:23.677 INFO:tasks.workunit.client.1.smithi144.stdout:2/744: sync 2024-05-15T15:54:23.678 INFO:tasks.workunit.client.0.smithi049.stdout:6/737: write dd/f4c [1712514,41138] 0 2024-05-15T15:54:23.679 INFO:tasks.workunit.client.0.smithi049.stdout:5/788: readlink dc/dd/d39/l52 0 2024-05-15T15:54:23.681 INFO:tasks.workunit.client.0.smithi049.stdout:2/691: symlink d3/db/d5b/dbf/dc1/le9 0 2024-05-15T15:54:23.682 INFO:tasks.workunit.client.1.smithi144.stdout:6/819: dwrite d7/db/d9d/f88 [0,4194304] 0 2024-05-15T15:54:23.683 INFO:tasks.workunit.client.0.smithi049.stdout:2/692: truncate d3/db/d1d/fe0 696180 0 2024-05-15T15:54:23.683 INFO:tasks.workunit.client.1.smithi144.stdout:6/820: stat d7/db/da2/d70/db6/l120 0 2024-05-15T15:54:23.687 INFO:tasks.workunit.client.1.smithi144.stdout:9/811: read - d0/d8/d5e/d9b/fdc zero size 2024-05-15T15:54:23.688 INFO:tasks.workunit.client.1.smithi144.stdout:6/821: dread d7/d1e/d28/f8a [0,4194304] 0 2024-05-15T15:54:23.688 INFO:tasks.workunit.client.0.smithi049.stdout:5/789: symlink dc/dd/d47/d83/l104 0 2024-05-15T15:54:23.689 INFO:tasks.workunit.client.1.smithi144.stdout:5/820: symlink d0/dc/d1f/l111 0 2024-05-15T15:54:23.699 INFO:tasks.workunit.client.0.smithi049.stdout:2/693: unlink d3/da/d15/d73/c81 0 2024-05-15T15:54:23.701 INFO:tasks.workunit.client.0.smithi049.stdout:7/670: write d0/d6/f14 [1039830,90864] 0 2024-05-15T15:54:23.702 INFO:tasks.workunit.client.0.smithi049.stdout:5/790: write dc/f80 [1613615,119491] 0 2024-05-15T15:54:23.703 INFO:tasks.workunit.client.1.smithi144.stdout:4/808: creat d6/d42/d46/d4b/d1e/d99/dad/f108 x:0 0 0 2024-05-15T15:54:23.704 INFO:tasks.workunit.client.1.smithi144.stdout:4/809: truncate d6/dd/d2e/d75/dff/ff9 185089 0 2024-05-15T15:54:23.706 INFO:tasks.workunit.client.1.smithi144.stdout:8/856: getdents d2/d24/d2d/da0/d6e/d71 0 2024-05-15T15:54:23.706 INFO:tasks.workunit.client.1.smithi144.stdout:2/745: write d3/d10/f1d [647308,82071] 0 2024-05-15T15:54:23.706 INFO:tasks.workunit.client.0.smithi049.stdout:5/791: dread dc/f64 [0,4194304] 0 2024-05-15T15:54:23.708 INFO:tasks.workunit.client.1.smithi144.stdout:9/812: symlink d0/d8/d1b/d79/d8a/dae/ddb/l119 0 2024-05-15T15:54:23.712 INFO:tasks.workunit.client.1.smithi144.stdout:6/822: rename d7/db/cbc to d7/db/da2/d70/c125 0 2024-05-15T15:54:23.713 INFO:tasks.workunit.client.0.smithi049.stdout:6/738: link dd/d26/lb1 dd/d17/d24/d2b/le2 0 2024-05-15T15:54:23.714 INFO:tasks.workunit.client.0.smithi049.stdout:6/739: write dd/f32 [467933,108898] 0 2024-05-15T15:54:23.716 INFO:tasks.workunit.client.1.smithi144.stdout:5/821: rmdir d0/dc/d24 39 2024-05-15T15:54:23.717 INFO:tasks.workunit.client.0.smithi049.stdout:7/671: write d0/d34/d88/dca/d5e/f9e [158381,1517] 0 2024-05-15T15:54:23.718 INFO:tasks.workunit.client.1.smithi144.stdout:6/823: dwrite d7/db/d11/d7e/ff9 [0,4194304] 0 2024-05-15T15:54:23.720 INFO:tasks.workunit.client.1.smithi144.stdout:1/780: rename d7/d1d/d33/c1b to d7/d1d/d33/cfc 0 2024-05-15T15:54:23.721 INFO:tasks.workunit.client.0.smithi049.stdout:6/740: dread dd/d17/d24/d2b/f5d [0,4194304] 0 2024-05-15T15:54:23.723 INFO:tasks.workunit.client.0.smithi049.stdout:7/672: dread d0/d6/d23/d45/d6a/f9d [0,4194304] 0 2024-05-15T15:54:23.723 INFO:tasks.workunit.client.0.smithi049.stdout:6/741: chown dd/d17/d24/d2b/d36/f42 40 1 2024-05-15T15:54:23.723 INFO:tasks.workunit.client.0.smithi049.stdout:7/673: readlink d0/d34/l65 0 2024-05-15T15:54:23.724 INFO:tasks.workunit.client.0.smithi049.stdout:7/674: chown d0/d6 149181 1 2024-05-15T15:54:23.725 INFO:tasks.workunit.client.0.smithi049.stdout:7/675: dread - d0/d6/d23/d45/d6a/da1/d80/fbb zero size 2024-05-15T15:54:23.729 INFO:tasks.workunit.client.1.smithi144.stdout:7/790: dwrite d4/d3c/d76/d9d/dac/f1d [0,4194304] 0 2024-05-15T15:54:23.731 INFO:tasks.workunit.client.1.smithi144.stdout:4/810: creat d6/dd/d2e/d75/dff/deb/f109 x:0 0 0 2024-05-15T15:54:23.732 INFO:tasks.workunit.client.1.smithi144.stdout:8/857: fsync d2/d3a/f7c 0 2024-05-15T15:54:23.732 INFO:tasks.workunit.client.0.smithi049.stdout:5/792: symlink dc/dd/d47/d83/dcc/l105 0 2024-05-15T15:54:23.734 INFO:tasks.workunit.client.1.smithi144.stdout:2/746: dread - d3/d7/d20/d22/d52/d64/fd3 zero size 2024-05-15T15:54:23.735 INFO:tasks.workunit.client.0.smithi049.stdout:6/742: unlink dd/d17/d24/d2b/fc5 0 2024-05-15T15:54:23.735 INFO:tasks.workunit.client.0.smithi049.stdout:6/743: read - dd/d17/fbe zero size 2024-05-15T15:54:23.737 INFO:tasks.workunit.client.0.smithi049.stdout:6/744: fdatasync dd/d17/fbe 0 2024-05-15T15:54:23.737 INFO:tasks.workunit.client.1.smithi144.stdout:9/813: truncate d0/d8/d1b/d79/d8a/dae/feb 5041843 0 2024-05-15T15:54:23.738 INFO:tasks.workunit.client.1.smithi144.stdout:0/753: write d4/d6/d3a/d5d/f8f [768766,29677] 0 2024-05-15T15:54:23.738 INFO:tasks.workunit.client.0.smithi049.stdout:6/745: fsync dd/d17/d24/d2b/d36/d50/d54/dde/fa7 0 2024-05-15T15:54:23.740 INFO:tasks.workunit.client.0.smithi049.stdout:6/746: stat dd/d26/d5b/da8/cc4 0 2024-05-15T15:54:23.740 INFO:tasks.workunit.client.0.smithi049.stdout:0/727: sync 2024-05-15T15:54:23.741 INFO:tasks.workunit.client.0.smithi049.stdout:0/728: read dc/d13/d56/d6f/fd2 [410368,71188] 0 2024-05-15T15:54:23.742 INFO:tasks.workunit.client.0.smithi049.stdout:0/729: stat dc/d13/d2b/d76/d8a/da2/db3 0 2024-05-15T15:54:23.747 INFO:tasks.workunit.client.0.smithi049.stdout:5/793: symlink dc/d4a/ddd/l106 0 2024-05-15T15:54:23.749 INFO:tasks.workunit.client.0.smithi049.stdout:7/676: creat d0/d6/d23/d4a/d7b/dc4/fd4 x:0 0 0 2024-05-15T15:54:23.755 INFO:tasks.workunit.client.1.smithi144.stdout:7/791: rename d4/d3c/d76/d9d/dac/d48/d72/d82/f89 to d4/df/d81/f113 0 2024-05-15T15:54:23.757 INFO:tasks.workunit.client.1.smithi144.stdout:6/824: dread d7/d62/f95 [4194304,4194304] 0 2024-05-15T15:54:23.758 INFO:tasks.workunit.client.0.smithi049.stdout:0/730: fsync dc/d13/d14/f95 0 2024-05-15T15:54:23.759 INFO:tasks.workunit.client.1.smithi144.stdout:1/781: dread d7/d1d/f91 [0,4194304] 0 2024-05-15T15:54:23.759 INFO:tasks.workunit.client.1.smithi144.stdout:8/858: dread - d2/d24/d2d/d30/dd0/df7/d87/fbf zero size 2024-05-15T15:54:23.761 INFO:tasks.workunit.client.1.smithi144.stdout:4/811: truncate d6/d42/d46/d4b/d1e/d44/f93 623921 0 2024-05-15T15:54:23.761 INFO:tasks.workunit.client.1.smithi144.stdout:8/859: read d2/de/d49/f69 [3836101,24339] 0 2024-05-15T15:54:23.762 INFO:tasks.workunit.client.0.smithi049.stdout:7/677: write d0/d6/f6f [54082,5762] 0 2024-05-15T15:54:23.766 INFO:tasks.workunit.client.1.smithi144.stdout:9/814: symlink d0/d8/d1b/d79/de7/df9/l11a 0 2024-05-15T15:54:23.767 INFO:tasks.workunit.client.1.smithi144.stdout:1/782: dread d7/d1d/d33/d29/f64 [0,4194304] 0 2024-05-15T15:54:23.767 INFO:tasks.workunit.client.0.smithi049.stdout:6/747: mknod dd/d17/d24/d2b/d36/d50/d54/d92/ce3 0 2024-05-15T15:54:23.768 INFO:tasks.workunit.client.1.smithi144.stdout:0/754: symlink d4/d6/d4a/d4d/l112 0 2024-05-15T15:54:23.768 INFO:tasks.workunit.client.1.smithi144.stdout:1/783: write d7/d1d/d33/d1e/d34/dd8/fe0 [425392,71699] 0 2024-05-15T15:54:23.771 INFO:tasks.workunit.client.0.smithi049.stdout:0/731: truncate dc/d13/d14/d25/d31/f75 2090780 0 2024-05-15T15:54:23.772 INFO:tasks.workunit.client.1.smithi144.stdout:3/789: link d0/d1/d17/d39/le6 d0/d1/d56/d71/l102 0 2024-05-15T15:54:23.774 INFO:tasks.workunit.client.0.smithi049.stdout:6/748: dwrite dd/d17/d24/d2b/f63 [0,4194304] 0 2024-05-15T15:54:23.774 INFO:tasks.workunit.client.0.smithi049.stdout:7/678: mknod d0/d6/d23/d4a/cd5 0 2024-05-15T15:54:23.775 INFO:tasks.workunit.client.1.smithi144.stdout:2/747: dwrite d3/d7/d20/d22/d52/d64/fd3 [0,4194304] 0 2024-05-15T15:54:23.776 INFO:tasks.workunit.client.0.smithi049.stdout:5/794: dread dc/d4a/f36 [0,4194304] 0 2024-05-15T15:54:23.777 INFO:tasks.workunit.client.0.smithi049.stdout:6/749: chown dd/d17/d24/d58/f75 52 1 2024-05-15T15:54:23.777 INFO:tasks.workunit.client.0.smithi049.stdout:5/795: chown dc/dd/d7b/la2 1271 1 2024-05-15T15:54:23.779 INFO:tasks.workunit.client.1.smithi144.stdout:6/825: creat d7/d1e/d28/db5/df6/f126 x:0 0 0 2024-05-15T15:54:23.780 INFO:tasks.workunit.client.0.smithi049.stdout:0/732: symlink dc/d13/d14/d25/db9/lf4 0 2024-05-15T15:54:23.781 INFO:tasks.workunit.client.0.smithi049.stdout:0/733: chown dc/d13/d2b/d76/fb0 4627484 1 2024-05-15T15:54:23.782 INFO:tasks.workunit.client.0.smithi049.stdout:7/679: unlink d0/f90 0 2024-05-15T15:54:23.783 INFO:tasks.workunit.client.1.smithi144.stdout:5/822: dread d0/dc/d24/d57/f77 [0,4194304] 0 2024-05-15T15:54:23.784 INFO:tasks.workunit.client.1.smithi144.stdout:2/748: dwrite d3/d7/d20/d37/d76/db5/db6/ff2 [0,4194304] 0 2024-05-15T15:54:23.784 INFO:tasks.workunit.client.1.smithi144.stdout:5/823: chown d0/dc/d2f/f10e 424 1 2024-05-15T15:54:23.786 INFO:tasks.workunit.client.1.smithi144.stdout:8/860: mkdir d2/d3a/da5/d11b 0 2024-05-15T15:54:23.786 INFO:tasks.workunit.client.0.smithi049.stdout:5/796: fdatasync dc/d4a/d9b/fae 0 2024-05-15T15:54:23.791 INFO:tasks.workunit.client.0.smithi049.stdout:7/680: mknod d0/d34/cd6 0 2024-05-15T15:54:23.792 INFO:tasks.workunit.client.0.smithi049.stdout:7/681: write d0/d34/f46 [953262,70507] 0 2024-05-15T15:54:23.794 INFO:tasks.workunit.client.0.smithi049.stdout:7/682: write d0/d6/d23/d4a/d7b/dc4/fd4 [321676,30545] 0 2024-05-15T15:54:23.798 INFO:tasks.workunit.client.1.smithi144.stdout:0/755: rename d4/d6/d3a/d40/l87 to d4/d2d/d48/d5b/dd3/d107/l113 0 2024-05-15T15:54:23.802 INFO:tasks.workunit.client.1.smithi144.stdout:3/790: symlink d0/d1/d6c/l103 0 2024-05-15T15:54:23.806 INFO:tasks.workunit.client.0.smithi049.stdout:6/750: dread dd/d26/f3f [0,4194304] 0 2024-05-15T15:54:23.808 INFO:tasks.workunit.client.0.smithi049.stdout:5/797: creat dc/dd/dc8/f107 x:0 0 0 2024-05-15T15:54:23.812 INFO:tasks.workunit.client.1.smithi144.stdout:4/812: symlink d6/dd/d34/d3b/d30/d59/d66/dbd/l10a 0 2024-05-15T15:54:23.812 INFO:tasks.workunit.client.0.smithi049.stdout:0/734: write dc/d13/d14/d25/d31/d71/f67 [1829997,106755] 0 2024-05-15T15:54:23.812 INFO:tasks.workunit.client.0.smithi049.stdout:6/751: write dd/d17/d24/d2b/d36/d50/d7e/fdc [48843,92422] 0 2024-05-15T15:54:23.813 INFO:tasks.workunit.client.1.smithi144.stdout:4/813: write d6/dd/d2e/d75/dff/ff9 [227327,26137] 0 2024-05-15T15:54:23.816 INFO:tasks.workunit.client.0.smithi049.stdout:5/798: rename dc/dd/f1d to dc/dd/d47/d83/da7/f108 0 2024-05-15T15:54:23.818 INFO:tasks.workunit.client.0.smithi049.stdout:6/752: fsync dd/d17/d24/d67/f84 0 2024-05-15T15:54:23.819 INFO:tasks.workunit.client.0.smithi049.stdout:6/753: chown dd/d26/lb1 4965 1 2024-05-15T15:54:23.821 INFO:tasks.workunit.client.0.smithi049.stdout:7/683: write d0/d2f/d5c/f6e [1325197,60174] 0 2024-05-15T15:54:23.821 INFO:tasks.workunit.client.0.smithi049.stdout:7/684: dread - d0/d34/fbc zero size 2024-05-15T15:54:23.823 INFO:tasks.workunit.client.1.smithi144.stdout:1/784: symlink d7/d1d/d33/d29/d6b/d5e/d74/df3/lfd 0 2024-05-15T15:54:23.823 INFO:tasks.workunit.client.1.smithi144.stdout:1/785: fdatasync d7/d1d/d33/d1e/d34/fe9 0 2024-05-15T15:54:23.824 INFO:tasks.workunit.client.1.smithi144.stdout:1/786: write d7/df/fec [172361,50874] 0 2024-05-15T15:54:23.827 INFO:tasks.workunit.client.0.smithi049.stdout:6/754: symlink dd/d17/d24/d2b/d36/d50/d54/d92/le4 0 2024-05-15T15:54:23.829 INFO:tasks.workunit.client.1.smithi144.stdout:3/791: read d0/d11/d24/f88 [1936347,123319] 0 2024-05-15T15:54:23.829 INFO:tasks.workunit.client.1.smithi144.stdout:7/792: creat d4/df/f114 x:0 0 0 2024-05-15T15:54:23.832 INFO:tasks.workunit.client.1.smithi144.stdout:2/749: creat d3/d7/d20/d37/d76/db5/db6/dbf/deb/ff6 x:0 0 0 2024-05-15T15:54:23.833 INFO:tasks.workunit.client.1.smithi144.stdout:2/750: dread - d3/d4d/db4/fed zero size 2024-05-15T15:54:23.833 INFO:tasks.workunit.client.1.smithi144.stdout:4/814: mkdir d6/d42/d46/d10b 0 2024-05-15T15:54:23.834 INFO:tasks.workunit.client.1.smithi144.stdout:4/815: fsync d6/dd/d2e/d75/dff/ffe 0 2024-05-15T15:54:23.838 INFO:tasks.workunit.client.1.smithi144.stdout:0/756: creat d4/d6/d3a/d4c/f114 x:0 0 0 2024-05-15T15:54:23.838 INFO:tasks.workunit.client.1.smithi144.stdout:1/787: rmdir d7/d1d/d33/d41/d78/dc3 39 2024-05-15T15:54:23.839 INFO:tasks.workunit.client.1.smithi144.stdout:7/793: mknod d4/d3c/d76/d9d/dc5/c115 0 2024-05-15T15:54:23.840 INFO:tasks.workunit.client.1.smithi144.stdout:6/826: dwrite d7/db/d5f/f65 [0,4194304] 0 2024-05-15T15:54:23.840 INFO:tasks.workunit.client.1.smithi144.stdout:3/792: truncate d0/d11/d24/f88 2648151 0 2024-05-15T15:54:23.841 INFO:tasks.workunit.client.1.smithi144.stdout:8/861: write d2/d24/d2d/da0/d6e/d71/d8c/fc0 [554957,64247] 0 2024-05-15T15:54:23.842 INFO:tasks.workunit.client.1.smithi144.stdout:2/751: mknod d3/d7/d20/d3b/cf7 0 2024-05-15T15:54:23.844 INFO:tasks.workunit.client.1.smithi144.stdout:8/862: write d2/d3a/d7b/d4b/fb5 [2278543,21713] 0 2024-05-15T15:54:23.844 INFO:tasks.workunit.client.1.smithi144.stdout:4/816: mknod d6/dd/dc3/c10c 0 2024-05-15T15:54:23.845 INFO:tasks.workunit.client.1.smithi144.stdout:9/815: getdents d0/d8 0 2024-05-15T15:54:23.846 INFO:tasks.workunit.client.1.smithi144.stdout:4/817: chown d6/dd/d2e/d75/dff/deb/f48 4040604 1 2024-05-15T15:54:23.847 INFO:tasks.workunit.client.1.smithi144.stdout:0/757: mkdir d4/d6/d3a/d4c/dd6/d55/da5/d115 0 2024-05-15T15:54:23.850 INFO:tasks.workunit.client.1.smithi144.stdout:3/793: creat d0/d11/d24/d78/f104 x:0 0 0 2024-05-15T15:54:23.851 INFO:tasks.workunit.client.1.smithi144.stdout:5/824: link d0/d10c/l94 d0/dc/d24/d8e/d95/l112 0 2024-05-15T15:54:23.855 INFO:tasks.workunit.client.1.smithi144.stdout:9/816: dwrite d0/d8/d1b/d79/de7/df9/f10c [0,4194304] 0 2024-05-15T15:54:23.856 INFO:tasks.workunit.client.1.smithi144.stdout:9/817: readlink d0/d8/d1b/d62/d7d/laa 0 2024-05-15T15:54:23.857 INFO:tasks.workunit.client.1.smithi144.stdout:5/825: rmdir d0/dc/d2f/d34/dc1 39 2024-05-15T15:54:23.858 INFO:tasks.workunit.client.1.smithi144.stdout:9/818: chown d0/d8/d1b/d62/f8e 22440802 1 2024-05-15T15:54:23.858 INFO:tasks.workunit.client.1.smithi144.stdout:5/826: chown d0/dc/d24/d8e/d93/fa1 40 1 2024-05-15T15:54:23.859 INFO:tasks.workunit.client.1.smithi144.stdout:8/863: fsync d2/fd 0 2024-05-15T15:54:23.860 INFO:tasks.workunit.client.1.smithi144.stdout:4/818: symlink d6/dd/d34/d3b/d30/d59/l10d 0 2024-05-15T15:54:23.862 INFO:tasks.workunit.client.1.smithi144.stdout:3/794: truncate d0/d1/d17/d39/f8c 2709650 0 2024-05-15T15:54:23.863 INFO:tasks.workunit.client.1.smithi144.stdout:6/827: dread d7/fd [0,4194304] 0 2024-05-15T15:54:23.864 INFO:tasks.workunit.client.0.smithi049.stdout:5/799: truncate dc/dd/d7b/f90 760829 0 2024-05-15T15:54:23.866 INFO:tasks.workunit.client.0.smithi049.stdout:7/685: symlink d0/d6/d23/ld7 0 2024-05-15T15:54:23.870 INFO:tasks.workunit.client.0.smithi049.stdout:5/800: mknod dc/dd/d39/c109 0 2024-05-15T15:54:23.874 INFO:tasks.workunit.client.1.smithi144.stdout:0/758: write d4/d6/d3a/d4c/fc6 [745129,26403] 0 2024-05-15T15:54:23.874 INFO:tasks.workunit.client.1.smithi144.stdout:1/788: dwrite d7/d1d/d33/d41/d78/da3/f54 [0,4194304] 0 2024-05-15T15:54:23.875 INFO:tasks.workunit.client.1.smithi144.stdout:3/795: read d0/d1/d17/f20 [1811045,21055] 0 2024-05-15T15:54:23.876 INFO:tasks.workunit.client.1.smithi144.stdout:2/752: getdents d3/d7/d20/d22/d52/d75/d65 0 2024-05-15T15:54:23.877 INFO:tasks.workunit.client.1.smithi144.stdout:7/794: truncate d4/df/d25/d38/d68/f8d 844656 0 2024-05-15T15:54:23.884 INFO:tasks.workunit.client.0.smithi049.stdout:6/755: truncate dd/d17/d24/d2b/d36/d50/d54/dde/fa7 1453531 0 2024-05-15T15:54:23.888 INFO:tasks.workunit.client.0.smithi049.stdout:6/756: dread f2 [0,4194304] 0 2024-05-15T15:54:23.892 INFO:tasks.workunit.client.1.smithi144.stdout:3/796: dread - d0/d11/d24/da5/fbe zero size 2024-05-15T15:54:23.894 INFO:tasks.workunit.client.1.smithi144.stdout:4/819: dwrite d6/fe7 [0,4194304] 0 2024-05-15T15:54:23.895 INFO:tasks.workunit.client.0.smithi049.stdout:6/757: mkdir dd/d17/d24/d2b/d36/d50/d54/d92/de5 0 2024-05-15T15:54:23.896 INFO:tasks.workunit.client.1.smithi144.stdout:8/864: dwrite d2/d24/d2d/d30/dd0/df7/f1e [0,4194304] 0 2024-05-15T15:54:23.897 INFO:tasks.workunit.client.0.smithi049.stdout:6/758: write dd/d17/d24/fd0 [1806811,16412] 0 2024-05-15T15:54:23.898 INFO:tasks.workunit.client.1.smithi144.stdout:0/759: dread d4/d2d/f44 [0,4194304] 0 2024-05-15T15:54:23.899 INFO:tasks.workunit.client.0.smithi049.stdout:7/686: rename d0/d6/d1d/lb8 to d0/d6/d23/d45/d6a/da1/ld8 0 2024-05-15T15:54:23.900 INFO:tasks.workunit.client.0.smithi049.stdout:6/759: fdatasync dd/d17/d24/d58/d5f/dd7/fd4 0 2024-05-15T15:54:23.903 INFO:tasks.workunit.client.0.smithi049.stdout:5/801: dwrite f0 [4194304,4194304] 0 2024-05-15T15:54:23.912 INFO:tasks.workunit.client.0.smithi049.stdout:6/760: dwrite dd/f8e [0,4194304] 0 2024-05-15T15:54:23.913 INFO:tasks.workunit.client.0.smithi049.stdout:6/761: readlink dd/d17/d24/d2b/ld2 0 2024-05-15T15:54:23.917 INFO:tasks.workunit.client.1.smithi144.stdout:5/827: creat d0/d10c/f113 x:0 0 0 2024-05-15T15:54:23.918 INFO:tasks.workunit.client.0.smithi049.stdout:7/687: mknod d0/d6/d23/d45/d86/cd9 0 2024-05-15T15:54:23.925 INFO:tasks.workunit.client.0.smithi049.stdout:5/802: dread dc/dd/d7b/f90 [0,4194304] 0 2024-05-15T15:54:23.926 INFO:tasks.workunit.client.1.smithi144.stdout:2/753: dwrite d3/d7/d20/d22/f7a [4194304,4194304] 0 2024-05-15T15:54:23.927 INFO:tasks.workunit.client.0.smithi049.stdout:6/762: rename dd/d26/lb1 to dd/d17/le6 0 2024-05-15T15:54:23.927 INFO:tasks.workunit.client.0.smithi049.stdout:7/688: dread d0/d2f/f48 [0,4194304] 0 2024-05-15T15:54:23.931 INFO:tasks.workunit.client.1.smithi144.stdout:7/795: symlink d4/df/d25/d38/l116 0 2024-05-15T15:54:23.935 INFO:tasks.workunit.client.1.smithi144.stdout:6/828: link d7/db/d3a/fe4 d7/db/d9d/d97/f127 0 2024-05-15T15:54:23.938 INFO:tasks.workunit.client.1.smithi144.stdout:5/828: dread - d0/dc/d2f/d34/d3e/fed zero size 2024-05-15T15:54:23.940 INFO:tasks.workunit.client.1.smithi144.stdout:3/797: chown d0/d11/d24/f88 126818917 1 2024-05-15T15:54:23.942 INFO:tasks.workunit.client.0.smithi049.stdout:5/803: creat dc/dd/d47/d83/dcc/da9/f10a x:0 0 0 2024-05-15T15:54:23.946 INFO:tasks.workunit.client.1.smithi144.stdout:0/760: symlink d4/df/d1f/d24/d85/da1/l116 0 2024-05-15T15:54:23.948 INFO:tasks.workunit.client.1.smithi144.stdout:8/865: dread d2/de/f33 [0,4194304] 0 2024-05-15T15:54:23.950 INFO:tasks.workunit.client.0.smithi049.stdout:5/804: mkdir dc/d4a/d9e/d10b 0 2024-05-15T15:54:23.951 INFO:tasks.workunit.client.0.smithi049.stdout:7/689: creat d0/d6/d23/fda x:0 0 0 2024-05-15T15:54:23.951 INFO:tasks.workunit.client.1.smithi144.stdout:1/789: link d7/d1d/d58/l7a d7/d1d/d33/d1e/d34/dd8/lfe 0 2024-05-15T15:54:23.953 INFO:tasks.workunit.client.1.smithi144.stdout:6/829: creat d7/d1e/d28/db5/df6/f128 x:0 0 0 2024-05-15T15:54:23.953 INFO:tasks.workunit.client.1.smithi144.stdout:6/830: read - d7/db/d9d/db7/ffa zero size 2024-05-15T15:54:23.958 INFO:tasks.workunit.client.1.smithi144.stdout:4/820: dwrite d6/d42/d46/d4b/f4d [0,4194304] 0 2024-05-15T15:54:23.960 INFO:tasks.workunit.client.1.smithi144.stdout:4/821: readlink d6/dd/d34/d3b/d30/d59/d66/l6a 0 2024-05-15T15:54:23.961 INFO:tasks.workunit.client.0.smithi049.stdout:4/694: sync 2024-05-15T15:54:23.962 INFO:tasks.workunit.client.0.smithi049.stdout:4/695: chown d18/d21/ld4 8900953 1 2024-05-15T15:54:23.962 INFO:tasks.workunit.client.1.smithi144.stdout:0/761: creat d4/d2d/d100/f117 x:0 0 0 2024-05-15T15:54:23.965 INFO:tasks.workunit.client.1.smithi144.stdout:7/796: dwrite d4/f67 [0,4194304] 0 2024-05-15T15:54:23.965 INFO:tasks.workunit.client.1.smithi144.stdout:8/866: creat d2/d24/d2d/da0/d6e/d71/d8c/f11c x:0 0 0 2024-05-15T15:54:23.966 INFO:tasks.workunit.client.1.smithi144.stdout:4/822: write d6/dd/d34/d3b/d30/d59/d66/fb8 [502615,107875] 0 2024-05-15T15:54:23.969 INFO:tasks.workunit.client.1.smithi144.stdout:7/797: write d4/d41/fc4 [870085,23578] 0 2024-05-15T15:54:23.972 INFO:tasks.workunit.client.1.smithi144.stdout:9/819: sync 2024-05-15T15:54:23.973 INFO:tasks.workunit.client.1.smithi144.stdout:3/798: symlink d0/d1/d17/d39/d40/d4a/l105 0 2024-05-15T15:54:23.974 INFO:tasks.workunit.client.1.smithi144.stdout:7/798: stat d4/d3c/d76/d9d/dc5/le1 0 2024-05-15T15:54:23.974 INFO:tasks.workunit.client.1.smithi144.stdout:5/829: dwrite d0/dc/d24/f37 [0,4194304] 0 2024-05-15T15:54:23.975 INFO:tasks.workunit.client.1.smithi144.stdout:4/823: chown d6/d42/d46/d4b/d1e/d44/cd1 3448133 1 2024-05-15T15:54:23.977 INFO:tasks.workunit.client.1.smithi144.stdout:2/754: dwrite d3/d7/d20/f7c [0,4194304] 0 2024-05-15T15:54:23.977 INFO:tasks.workunit.client.1.smithi144.stdout:5/830: fsync d0/d10c/f113 0 2024-05-15T15:54:23.978 INFO:tasks.workunit.client.1.smithi144.stdout:7/799: chown d4/d3c/d76/d9d/dac/f23 1320 1 2024-05-15T15:54:23.979 INFO:tasks.workunit.client.1.smithi144.stdout:4/824: fdatasync d6/dd/d2e/d94/ffa 0 2024-05-15T15:54:23.979 INFO:tasks.workunit.client.1.smithi144.stdout:0/762: rename d4/dc/l67 to d4/d6/d3a/d4c/dd6/d55/da5/d115/l118 0 2024-05-15T15:54:23.982 INFO:tasks.workunit.client.1.smithi144.stdout:8/867: mknod d2/d3a/c11d 0 2024-05-15T15:54:23.984 INFO:tasks.workunit.client.1.smithi144.stdout:3/799: mkdir d0/d11/d24/da5/d106 0 2024-05-15T15:54:23.987 INFO:tasks.workunit.client.1.smithi144.stdout:1/790: write d7/d1d/d33/d1e/f77 [5070805,55961] 0 2024-05-15T15:54:23.988 INFO:tasks.workunit.client.1.smithi144.stdout:8/868: dwrite d2/de/d49/d83/f99 [0,4194304] 0 2024-05-15T15:54:23.990 INFO:tasks.workunit.client.1.smithi144.stdout:9/820: symlink d0/d8/d1b/d62/d7d/d8d/dbb/l11b 0 2024-05-15T15:54:24.000 INFO:tasks.workunit.client.1.smithi144.stdout:0/763: rename d4/d2d/f44 to d4/d2d/d100/f119 0 2024-05-15T15:54:24.003 INFO:tasks.workunit.client.1.smithi144.stdout:6/831: truncate d7/db/d35/d84/d81/f109 3534602 0 2024-05-15T15:54:24.004 INFO:tasks.workunit.client.1.smithi144.stdout:6/832: read d7/db/f4a [4821,9333] 0 2024-05-15T15:54:24.010 INFO:tasks.workunit.client.1.smithi144.stdout:8/869: creat d2/d24/d2d/da0/d9e/db8/dde/f11e x:0 0 0 2024-05-15T15:54:24.013 INFO:tasks.workunit.client.1.smithi144.stdout:9/821: write d0/d8/d5e/d44/f74 [1574781,59225] 0 2024-05-15T15:54:24.017 INFO:tasks.workunit.client.1.smithi144.stdout:2/755: write d3/d4d/fae [4207788,114119] 0 2024-05-15T15:54:24.018 INFO:tasks.workunit.client.1.smithi144.stdout:7/800: write d4/f1c [3409831,17002] 0 2024-05-15T15:54:24.020 INFO:tasks.workunit.client.1.smithi144.stdout:0/764: creat d4/d6/d3a/d4c/f11a x:0 0 0 2024-05-15T15:54:24.024 INFO:tasks.workunit.client.1.smithi144.stdout:3/800: mknod d0/d1/d17/d39/d4d/dfd/c107 0 2024-05-15T15:54:24.025 INFO:tasks.workunit.client.1.smithi144.stdout:3/801: stat d0/d1/d56/c59 0 2024-05-15T15:54:24.026 INFO:tasks.workunit.client.1.smithi144.stdout:3/802: truncate d0/d1/d17/d39/d40/d7e/ff4 58381 0 2024-05-15T15:54:24.027 INFO:tasks.workunit.client.1.smithi144.stdout:8/870: creat d2/d24/d2d/da0/d6e/d8d/f11f x:0 0 0 2024-05-15T15:54:24.028 INFO:tasks.workunit.client.1.smithi144.stdout:3/803: read d0/d11/d3b/da7/fec [80242,51424] 0 2024-05-15T15:54:24.029 INFO:tasks.workunit.client.1.smithi144.stdout:8/871: chown d2/d24/d2d/d30/dd0 15322363 1 2024-05-15T15:54:24.030 INFO:tasks.workunit.client.1.smithi144.stdout:1/791: dwrite d7/d1d/d33/d41/d78/da3/f88 [0,4194304] 0 2024-05-15T15:54:24.030 INFO:tasks.workunit.client.1.smithi144.stdout:9/822: truncate d0/f14 399965 0 2024-05-15T15:54:24.031 INFO:tasks.workunit.client.1.smithi144.stdout:8/872: chown d2/d24/d2d/d30/dd0/df7/lf3 37 1 2024-05-15T15:54:24.041 INFO:tasks.workunit.client.1.smithi144.stdout:5/831: rmdir d0/dc/d2f/d34/d63/d6e/de7 0 2024-05-15T15:54:24.043 INFO:tasks.workunit.client.1.smithi144.stdout:8/873: creat d2/d24/d2d/da0/d6e/d71/d73/d98/f120 x:0 0 0 2024-05-15T15:54:24.044 INFO:tasks.workunit.client.1.smithi144.stdout:4/825: rename d6/dd/d34/d3b/d30/db/d1d/d40/d57/db4 to d6/d42/d46/d4b/d1e/d99/d10e 0 2024-05-15T15:54:24.047 INFO:tasks.workunit.client.0.smithi049.stdout:6/763: getdents dd/d26/d5b/da8 0 2024-05-15T15:54:24.048 INFO:tasks.workunit.client.1.smithi144.stdout:6/833: rmdir d7/db/d9d/d72/dbd 0 2024-05-15T15:54:24.049 INFO:tasks.workunit.client.0.smithi049.stdout:7/690: mkdir d0/ddb 0 2024-05-15T15:54:24.050 INFO:tasks.workunit.client.1.smithi144.stdout:3/804: mknod d0/d11/d24/da5/d106/c108 0 2024-05-15T15:54:24.051 INFO:tasks.workunit.client.1.smithi144.stdout:3/805: chown d0/d1/d56/d71/d73 31911 1 2024-05-15T15:54:24.051 INFO:tasks.workunit.client.0.smithi049.stdout:4/696: creat d18/d85/d98/ddd/fe4 x:0 0 0 2024-05-15T15:54:24.052 INFO:tasks.workunit.client.0.smithi049.stdout:7/691: truncate d0/d6/d23/d45/d6a/da1/d80/fbb 49660 0 2024-05-15T15:54:24.053 INFO:tasks.workunit.client.1.smithi144.stdout:1/792: mkdir d7/d1d/d33/d29/dcd/dff 0 2024-05-15T15:54:24.059 INFO:tasks.workunit.client.0.smithi049.stdout:7/692: unlink d0/d6/da/fc1 0 2024-05-15T15:54:24.060 INFO:tasks.workunit.client.1.smithi144.stdout:8/874: symlink d2/d24/d2d/da0/d9e/db8/l121 0 2024-05-15T15:54:24.061 INFO:tasks.workunit.client.0.smithi049.stdout:7/693: dread - d0/d34/fbc zero size 2024-05-15T15:54:24.061 INFO:tasks.workunit.client.0.smithi049.stdout:4/697: dwrite d18/d1e/d63/f96 [0,4194304] 0 2024-05-15T15:54:24.063 INFO:tasks.workunit.client.0.smithi049.stdout:7/694: dread d0/d6/f29 [0,4194304] 0 2024-05-15T15:54:24.069 INFO:tasks.workunit.client.1.smithi144.stdout:6/834: truncate d7/db/d35/d84/fcf 27706 0 2024-05-15T15:54:24.071 INFO:tasks.workunit.client.0.smithi049.stdout:7/695: dwrite d0/d6/d23/d45/d6a/da1/fbf [0,4194304] 0 2024-05-15T15:54:24.072 INFO:tasks.workunit.client.0.smithi049.stdout:4/698: unlink d18/d1e/d2a/d36/d86/dbd/fdb 0 2024-05-15T15:54:24.075 INFO:tasks.workunit.client.1.smithi144.stdout:6/835: dwrite d7/d1e/d28/db5/df6/f128 [0,4194304] 0 2024-05-15T15:54:24.078 INFO:tasks.workunit.client.1.smithi144.stdout:1/793: chown d7/d1d/d33/d41/c9b 62779930 1 2024-05-15T15:54:24.083 INFO:tasks.workunit.client.0.smithi049.stdout:7/696: stat d0/d6/d23/d4a/l57 0 2024-05-15T15:54:24.089 INFO:tasks.workunit.client.1.smithi144.stdout:7/801: getdents d4/d3c/d76/d9d/dba 0 2024-05-15T15:54:24.092 INFO:tasks.workunit.client.0.smithi049.stdout:6/764: truncate dd/d17/d24/d2b/d36/d50/d7e/fdc 263855 0 2024-05-15T15:54:24.092 INFO:tasks.workunit.client.0.smithi049.stdout:6/765: readlink dd/l25 0 2024-05-15T15:54:24.094 INFO:tasks.workunit.client.0.smithi049.stdout:6/766: chown dd/d17/d24/d2b/d36/f42 434 1 2024-05-15T15:54:24.094 INFO:tasks.workunit.client.1.smithi144.stdout:3/806: fdatasync d0/d11/d24/f74 0 2024-05-15T15:54:24.094 INFO:tasks.workunit.client.1.smithi144.stdout:5/832: dwrite d0/dc/d2f/d34/d63/d6e/db8/fda [0,4194304] 0 2024-05-15T15:54:24.094 INFO:tasks.workunit.client.0.smithi049.stdout:6/767: read - dd/dbc/ddb/fe0 zero size 2024-05-15T15:54:24.095 INFO:tasks.workunit.client.0.smithi049.stdout:4/699: rmdir d18/d1e/d63/d99/de1/da8/dda 0 2024-05-15T15:54:24.097 INFO:tasks.workunit.client.1.smithi144.stdout:6/836: symlink d7/db/d3a/de3/df2/l129 0 2024-05-15T15:54:24.100 INFO:tasks.workunit.client.1.smithi144.stdout:9/823: getdents d0/d8/d1b/d62/d6b 0 2024-05-15T15:54:24.100 INFO:tasks.workunit.client.0.smithi049.stdout:6/768: truncate dd/d17/d24/d2b/d36/f98 22166 0 2024-05-15T15:54:24.101 INFO:tasks.workunit.client.1.smithi144.stdout:8/875: write d2/f3 [2101017,50595] 0 2024-05-15T15:54:24.102 INFO:tasks.workunit.client.1.smithi144.stdout:4/826: dwrite d6/dd/d34/d3b/d30/db/d1d/d40/d6c/fde [0,4194304] 0 2024-05-15T15:54:24.102 INFO:tasks.workunit.client.0.smithi049.stdout:6/769: readlink dd/d17/d24/d2b/ld1 0 2024-05-15T15:54:24.105 INFO:tasks.workunit.client.1.smithi144.stdout:4/827: write d6/dd/f22 [2323577,52068] 0 2024-05-15T15:54:24.106 INFO:tasks.workunit.client.0.smithi049.stdout:7/697: dwrite d0/d34/f35 [8388608,4194304] 0 2024-05-15T15:54:24.107 INFO:tasks.workunit.client.1.smithi144.stdout:2/756: rename d3/d7/d20/d22/d52/d75 to d3/d7/df8 0 2024-05-15T15:54:24.108 INFO:tasks.workunit.client.1.smithi144.stdout:4/828: stat d6/dd/d34/d3b/d30/db/d1d/d101 0 2024-05-15T15:54:24.109 INFO:tasks.workunit.client.1.smithi144.stdout:7/802: truncate d4/df/d25/d40/d62/f6b 255805 0 2024-05-15T15:54:24.110 INFO:tasks.workunit.client.0.smithi049.stdout:4/700: getdents d18/d85/d98 0 2024-05-15T15:54:24.110 INFO:tasks.workunit.client.1.smithi144.stdout:1/794: dwrite d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:24.110 INFO:tasks.workunit.client.1.smithi144.stdout:2/757: chown d3/d7/l21 8860 1 2024-05-15T15:54:24.112 INFO:tasks.workunit.client.1.smithi144.stdout:4/829: write d6/dd/d2e/d75/dff/deb/dba/fbe [6470149,108046] 0 2024-05-15T15:54:24.112 INFO:tasks.workunit.client.0.smithi049.stdout:7/698: creat d0/d6/d23/d4a/fdc x:0 0 0 2024-05-15T15:54:24.113 INFO:tasks.workunit.client.0.smithi049.stdout:7/699: readlink d0/d6/d1d/l63 0 2024-05-15T15:54:24.115 INFO:tasks.workunit.client.1.smithi144.stdout:3/807: unlink d0/d1/f23 0 2024-05-15T15:54:24.116 INFO:tasks.workunit.client.0.smithi049.stdout:6/770: symlink dd/d17/d24/d2b/d36/d50/d54/d92/de5/le7 0 2024-05-15T15:54:24.118 INFO:tasks.workunit.client.1.smithi144.stdout:3/808: dread d0/d11/d3b/da7/fec [0,4194304] 0 2024-05-15T15:54:24.123 INFO:tasks.workunit.client.0.smithi049.stdout:4/701: truncate d18/d21/d2b/fac 704254 0 2024-05-15T15:54:24.123 INFO:tasks.workunit.client.0.smithi049.stdout:7/700: dwrite d0/d6/d23/d4a/fb4 [0,4194304] 0 2024-05-15T15:54:24.124 INFO:tasks.workunit.client.0.smithi049.stdout:9/603: sync 2024-05-15T15:54:24.125 INFO:tasks.workunit.client.1.smithi144.stdout:9/824: rmdir d0/d8/d5e/d44/dc2 39 2024-05-15T15:54:24.126 INFO:tasks.workunit.client.0.smithi049.stdout:7/701: fdatasync d0/d6/d23/d4a/f75 0 2024-05-15T15:54:24.130 INFO:tasks.workunit.client.1.smithi144.stdout:8/876: unlink d2/d24/d2d/d30/dd0/df7/f2c 0 2024-05-15T15:54:24.133 INFO:tasks.workunit.client.1.smithi144.stdout:0/765: rename d4/df/d14/db5 to d4/df/d102/d11b 0 2024-05-15T15:54:24.135 INFO:tasks.workunit.client.0.smithi049.stdout:9/604: stat d0/fa2 0 2024-05-15T15:54:24.137 INFO:tasks.workunit.client.0.smithi049.stdout:4/702: symlink d18/d1e/d2a/d36/d5d/le5 0 2024-05-15T15:54:24.138 INFO:tasks.workunit.client.1.smithi144.stdout:1/795: dread - d7/df/d89/fd1 zero size 2024-05-15T15:54:24.140 INFO:tasks.workunit.client.1.smithi144.stdout:8/877: dwrite d2/d24/d2d/da0/d6e/d8d/ffd [0,4194304] 0 2024-05-15T15:54:24.142 INFO:tasks.workunit.client.0.smithi049.stdout:9/605: mkdir d0/d8/d1b/d71/d87/dd1 0 2024-05-15T15:54:24.146 INFO:tasks.workunit.client.1.smithi144.stdout:9/825: fsync d0/d8/d1b/d62/fa3 0 2024-05-15T15:54:24.147 INFO:tasks.workunit.client.0.smithi049.stdout:4/703: creat d18/d1e/d63/d99/de1/da8/fe6 x:0 0 0 2024-05-15T15:54:24.149 INFO:tasks.workunit.client.1.smithi144.stdout:3/809: dread d0/d11/d24/f74 [0,4194304] 0 2024-05-15T15:54:24.150 INFO:tasks.workunit.client.0.smithi049.stdout:2/694: sync 2024-05-15T15:54:24.150 INFO:tasks.workunit.client.0.smithi049.stdout:0/735: sync 2024-05-15T15:54:24.150 INFO:tasks.workunit.client.0.smithi049.stdout:5/805: sync 2024-05-15T15:54:24.150 INFO:tasks.workunit.client.0.smithi049.stdout:1/894: sync 2024-05-15T15:54:24.150 INFO:tasks.workunit.client.0.smithi049.stdout:3/703: sync 2024-05-15T15:54:24.150 INFO:tasks.workunit.client.0.smithi049.stdout:8/677: sync 2024-05-15T15:54:24.151 INFO:tasks.workunit.client.1.smithi144.stdout:3/810: write d0/d11/d24/fb8 [1829403,77449] 0 2024-05-15T15:54:24.152 INFO:tasks.workunit.client.0.smithi049.stdout:2/695: chown d3/f11 7 1 2024-05-15T15:54:24.153 INFO:tasks.workunit.client.1.smithi144.stdout:6/837: rename d7/db/da2/d70/db6/ffe to d7/db/d35/d84/dbe/d10a/f12a 0 2024-05-15T15:54:24.155 INFO:tasks.workunit.client.1.smithi144.stdout:1/796: dread d7/d1d/d33/d29/d6b/d5e/fa1 [0,4194304] 0 2024-05-15T15:54:24.156 INFO:tasks.workunit.client.0.smithi049.stdout:9/606: rename d0/d8/d9c to d0/d8/d53/dd2 0 2024-05-15T15:54:24.157 INFO:tasks.workunit.client.0.smithi049.stdout:9/607: stat d0/d8/d1b/c37 0 2024-05-15T15:54:24.158 INFO:tasks.workunit.client.1.smithi144.stdout:8/878: dread d2/d3a/d7b/d4b/fe9 [0,4194304] 0 2024-05-15T15:54:24.159 INFO:tasks.workunit.client.1.smithi144.stdout:7/803: creat d4/d41/de8/f117 x:0 0 0 2024-05-15T15:54:24.159 INFO:tasks.workunit.client.0.smithi049.stdout:9/608: read d0/d8/d9/d52/d5e/d5f/f63 [1824424,103398] 0 2024-05-15T15:54:24.159 INFO:tasks.workunit.client.0.smithi049.stdout:7/702: link d0/d6/d23/d45/d6a/ca7 d0/d34/d88/cdd 0 2024-05-15T15:54:24.160 INFO:tasks.workunit.client.1.smithi144.stdout:7/804: fdatasync d4/df/d25/f85 0 2024-05-15T15:54:24.161 INFO:tasks.workunit.client.1.smithi144.stdout:7/805: write d4/df/f20 [4402989,95554] 0 2024-05-15T15:54:24.161 INFO:tasks.workunit.client.0.smithi049.stdout:6/771: link dd/d17/d24/d2b/d36/lb9 dd/d17/d9d/le8 0 2024-05-15T15:54:24.164 INFO:tasks.workunit.client.1.smithi144.stdout:8/879: dwrite d2/de/f103 [0,4194304] 0 2024-05-15T15:54:24.165 INFO:tasks.workunit.client.0.smithi049.stdout:2/696: dread d3/da/fb4 [0,4194304] 0 2024-05-15T15:54:24.175 INFO:tasks.workunit.client.0.smithi049.stdout:0/736: truncate dc/d13/d14/f1c 1697874 0 2024-05-15T15:54:24.177 INFO:tasks.workunit.client.0.smithi049.stdout:8/678: unlink d2/de/d71/d86/d5d/d7d/d1b/d68/f66 0 2024-05-15T15:54:24.178 INFO:tasks.workunit.client.0.smithi049.stdout:8/679: read d2/de/d71/fba [152397,62411] 0 2024-05-15T15:54:24.180 INFO:tasks.workunit.client.0.smithi049.stdout:5/806: rename dc/l34 to dc/dd/d47/d83/l10c 0 2024-05-15T15:54:24.182 INFO:tasks.workunit.client.0.smithi049.stdout:4/704: mkdir d18/d1e/d2a/de7 0 2024-05-15T15:54:24.186 INFO:tasks.workunit.client.0.smithi049.stdout:1/895: dwrite d0/d4/dd/de/d36/fef [0,4194304] 0 2024-05-15T15:54:24.191 INFO:tasks.workunit.client.1.smithi144.stdout:2/758: rename d3/d7/f36 to d3/d7/d20/d89/d86/ff9 0 2024-05-15T15:54:24.194 INFO:tasks.workunit.client.0.smithi049.stdout:3/704: symlink d12/d14/d52/db9/dc4/d68/d70/d75/dac/ld9 0 2024-05-15T15:54:24.194 INFO:tasks.workunit.client.0.smithi049.stdout:7/703: dwrite d0/f47 [0,4194304] 0 2024-05-15T15:54:24.196 INFO:tasks.workunit.client.1.smithi144.stdout:1/797: fdatasync d7/d1d/d33/d29/d6b/f65 0 2024-05-15T15:54:24.200 INFO:tasks.workunit.client.0.smithi049.stdout:2/697: rename d3/db/d1d/d1e/d2b/d2e/d67 to d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/dc9/dea 0 2024-05-15T15:54:24.204 INFO:tasks.workunit.client.0.smithi049.stdout:5/807: creat dc/dd/d10/db3/dd6/f10d x:0 0 0 2024-05-15T15:54:24.208 INFO:tasks.workunit.client.0.smithi049.stdout:4/705: rmdir d18/d1e/d2a/d50/dc4 39 2024-05-15T15:54:24.211 INFO:tasks.workunit.client.0.smithi049.stdout:1/896: truncate d0/d4/f1b 6703571 0 2024-05-15T15:54:24.212 INFO:tasks.workunit.client.1.smithi144.stdout:0/766: dwrite d4/d6/d3a/d4c/f9a [0,4194304] 0 2024-05-15T15:54:24.215 INFO:tasks.workunit.client.1.smithi144.stdout:3/811: symlink d0/d11/d3b/dcf/l109 0 2024-05-15T15:54:24.218 INFO:tasks.workunit.client.0.smithi049.stdout:7/704: truncate d0/d2f/f97 4092428 0 2024-05-15T15:54:24.221 INFO:tasks.workunit.client.0.smithi049.stdout:9/609: write d0/d8/d21/f82 [928952,5182] 0 2024-05-15T15:54:24.226 INFO:tasks.workunit.client.0.smithi049.stdout:8/680: rename d2/de/d71/d86/d5d/f89 to d2/de/d95/fd1 0 2024-05-15T15:54:24.228 INFO:tasks.workunit.client.0.smithi049.stdout:2/698: fsync d3/db/d1d/f28 0 2024-05-15T15:54:24.231 INFO:tasks.workunit.client.0.smithi049.stdout:5/808: write dc/dd/d47/d83/dcc/da9/ff9 [868377,112907] 0 2024-05-15T15:54:24.234 INFO:tasks.workunit.client.0.smithi049.stdout:7/705: truncate d0/d6/d1d/f27 2734123 0 2024-05-15T15:54:24.236 INFO:tasks.workunit.client.0.smithi049.stdout:9/610: unlink d0/d8/d1b/f36 0 2024-05-15T15:54:24.243 INFO:tasks.workunit.client.0.smithi049.stdout:4/706: rename d18/d1e/d2a/f43 to d18/d1e/d63/d99/de1/fe8 0 2024-05-15T15:54:24.244 INFO:tasks.workunit.client.0.smithi049.stdout:4/707: write d18/d21/d2b/d4e/fd6 [775588,83007] 0 2024-05-15T15:54:24.245 INFO:tasks.workunit.client.1.smithi144.stdout:5/833: sync 2024-05-15T15:54:24.253 INFO:tasks.workunit.client.0.smithi049.stdout:7/706: rename d0/d34/d88 to d0/ddb/dde 0 2024-05-15T15:54:24.254 INFO:tasks.workunit.client.0.smithi049.stdout:7/707: fsync d0/d6/da/fc2 0 2024-05-15T15:54:24.255 INFO:tasks.workunit.client.0.smithi049.stdout:7/708: write d0/d2f/d5c/f70 [1713812,86280] 0 2024-05-15T15:54:24.257 INFO:tasks.workunit.client.0.smithi049.stdout:7/709: write d0/d6/d23/d4a/fb4 [3831456,124609] 0 2024-05-15T15:54:24.263 INFO:tasks.workunit.client.0.smithi049.stdout:8/681: write d2/d18/f47 [1679394,121566] 0 2024-05-15T15:54:24.268 INFO:tasks.workunit.client.0.smithi049.stdout:9/611: rename d0/d8/d1b/d71 to d0/d8/d9/d52/d5e/d8a/dd3 0 2024-05-15T15:54:24.269 INFO:tasks.workunit.client.0.smithi049.stdout:9/612: chown d0/d8/d9/d52/d5e/d5f/c6d 418653340 1 2024-05-15T15:54:24.275 INFO:tasks.workunit.client.0.smithi049.stdout:7/710: readlink d0/d6/d23/d45/d6a/da1/l78 0 2024-05-15T15:54:24.279 INFO:tasks.workunit.client.0.smithi049.stdout:5/809: write dc/d4a/d42/f53 [878369,37176] 0 2024-05-15T15:54:24.287 INFO:tasks.workunit.client.0.smithi049.stdout:2/699: getdents d3/db/d82 0 2024-05-15T15:54:24.287 INFO:tasks.workunit.client.1.smithi144.stdout:8/880: sync 2024-05-15T15:54:24.294 INFO:tasks.workunit.client.0.smithi049.stdout:4/708: link d18/d1e/l6e d18/d1e/d63/d99/de1/da8/le9 0 2024-05-15T15:54:24.295 INFO:tasks.workunit.client.0.smithi049.stdout:7/711: write d0/d34/fad [1142187,58564] 0 2024-05-15T15:54:24.297 INFO:tasks.workunit.client.0.smithi049.stdout:2/700: mknod d3/d52/ceb 0 2024-05-15T15:54:24.299 INFO:tasks.workunit.client.0.smithi049.stdout:2/701: stat d3/db/d5b/dbf/dc1/fa5 0 2024-05-15T15:54:24.303 INFO:tasks.workunit.client.0.smithi049.stdout:9/613: creat d0/d8/d42/fd4 x:0 0 0 2024-05-15T15:54:24.303 INFO:tasks.workunit.client.0.smithi049.stdout:7/712: dwrite d0/d6/d23/d45/fba [0,4194304] 0 2024-05-15T15:54:24.307 INFO:tasks.workunit.client.0.smithi049.stdout:8/682: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fd2 x:0 0 0 2024-05-15T15:54:24.310 INFO:tasks.workunit.client.0.smithi049.stdout:8/683: write d2/de/d71/d86/d5d/d7d/db8/d5b/d60/fbb [208329,19782] 0 2024-05-15T15:54:24.311 INFO:tasks.workunit.client.1.smithi144.stdout:4/830: rename d6/d42/d46 to d6/dd/d34/d3b/d30/dc9/d10f 0 2024-05-15T15:54:24.317 INFO:tasks.workunit.client.0.smithi049.stdout:7/713: read d0/d6/d1d/f27 [1156909,90354] 0 2024-05-15T15:54:24.318 INFO:tasks.workunit.client.0.smithi049.stdout:9/614: mknod d0/d8/d9/d52/cd5 0 2024-05-15T15:54:24.319 INFO:tasks.workunit.client.1.smithi144.stdout:1/798: fsync d7/d1d/d33/d29/d6b/da4/fb5 0 2024-05-15T15:54:24.320 INFO:tasks.workunit.client.0.smithi049.stdout:5/810: getdents dc/db0/dc7 0 2024-05-15T15:54:24.326 INFO:tasks.workunit.client.1.smithi144.stdout:0/767: mknod d4/d6/d3a/d4c/dd6/c11c 0 2024-05-15T15:54:24.330 INFO:tasks.workunit.client.0.smithi049.stdout:7/714: fdatasync d0/d6/d1d/f4d 0 2024-05-15T15:54:24.331 INFO:tasks.workunit.client.1.smithi144.stdout:2/759: write d3/d7/d20/d22/d52/f69 [1463417,92423] 0 2024-05-15T15:54:24.332 INFO:tasks.workunit.client.1.smithi144.stdout:2/760: chown d3/d7/d20/c7f 1 1 2024-05-15T15:54:24.332 INFO:tasks.workunit.client.1.smithi144.stdout:8/881: creat d2/d24/d2d/da0/f122 x:0 0 0 2024-05-15T15:54:24.332 INFO:tasks.workunit.client.1.smithi144.stdout:8/882: readlink d2/d24/d2d/ld6 0 2024-05-15T15:54:24.334 INFO:tasks.workunit.client.0.smithi049.stdout:5/811: unlink dc/d4a/f50 0 2024-05-15T15:54:24.334 INFO:tasks.workunit.client.1.smithi144.stdout:2/761: dread d3/d7/df8/d65/f93 [0,4194304] 0 2024-05-15T15:54:24.336 INFO:tasks.workunit.client.1.smithi144.stdout:9/826: rename d0/d8/d5e/d9b to d0/d8/d1b/d11c 0 2024-05-15T15:54:24.337 INFO:tasks.workunit.client.0.smithi049.stdout:8/684: creat d2/de/d25/d8c/d9c/fd3 x:0 0 0 2024-05-15T15:54:24.338 INFO:tasks.workunit.client.0.smithi049.stdout:2/702: write d3/db/d1d/d1e/d2b/d3b/d47/d9a/fce [201305,5244] 0 2024-05-15T15:54:24.339 INFO:tasks.workunit.client.0.smithi049.stdout:4/709: link d18/d69/c82 d18/d1e/d63/cea 0 2024-05-15T15:54:24.341 INFO:tasks.workunit.client.1.smithi144.stdout:6/838: creat d7/db/d35/f12b x:0 0 0 2024-05-15T15:54:24.342 INFO:tasks.workunit.client.0.smithi049.stdout:9/615: mkdir d0/d8/dd6 0 2024-05-15T15:54:24.346 INFO:tasks.workunit.client.0.smithi049.stdout:5/812: unlink dc/d4a/l68 0 2024-05-15T15:54:24.348 INFO:tasks.workunit.client.0.smithi049.stdout:8/685: dwrite d2/de/d71/d86/d5d/d7d/db8/d5b/d60/fbb [0,4194304] 0 2024-05-15T15:54:24.349 INFO:tasks.workunit.client.0.smithi049.stdout:2/703: dwrite d3/db/d1d/d1e/d2b/d3b/fa7 [0,4194304] 0 2024-05-15T15:54:24.351 INFO:tasks.workunit.client.0.smithi049.stdout:4/710: creat d18/d85/d98/ddd/feb x:0 0 0 2024-05-15T15:54:24.356 INFO:tasks.workunit.client.1.smithi144.stdout:2/762: truncate d3/d10/f35 4934810 0 2024-05-15T15:54:24.357 INFO:tasks.workunit.client.1.smithi144.stdout:7/806: rename d4/d3c/d76/d9d/lb2 to d4/df/d81/dfb/l118 0 2024-05-15T15:54:24.358 INFO:tasks.workunit.client.1.smithi144.stdout:7/807: fdatasync d4/df/d25/d38/d5d/ffa 0 2024-05-15T15:54:24.360 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:24 smithi049 ceph-mon[28399]: pgmap v32: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 100 MiB/s rd, 266 MiB/s wr, 514 op/s 2024-05-15T15:54:24.361 INFO:tasks.workunit.client.1.smithi144.stdout:4/831: symlink d6/dd/l110 0 2024-05-15T15:54:24.361 INFO:tasks.workunit.client.1.smithi144.stdout:6/839: dread d7/fe5 [0,4194304] 0 2024-05-15T15:54:24.364 INFO:tasks.workunit.client.0.smithi049.stdout:2/704: dread d3/db/d1d/d1e/d2b/d3b/d42/f4f [0,4194304] 0 2024-05-15T15:54:24.365 INFO:tasks.workunit.client.0.smithi049.stdout:5/813: rename dc/dd/d47/da6/cc6 to dc/dd/d47/d83/da7/c10e 0 2024-05-15T15:54:24.365 INFO:tasks.workunit.client.1.smithi144.stdout:1/799: mknod d7/d1d/d33/d41/d78/da3/deb/c100 0 2024-05-15T15:54:24.366 INFO:tasks.workunit.client.0.smithi049.stdout:9/616: symlink d0/d8/dd6/ld7 0 2024-05-15T15:54:24.373 INFO:tasks.workunit.client.0.smithi049.stdout:7/715: getdents d0/d2f 0 2024-05-15T15:54:24.378 INFO:tasks.workunit.client.1.smithi144.stdout:7/808: read - d4/df/d79/fcf zero size 2024-05-15T15:54:24.380 INFO:tasks.workunit.client.1.smithi144.stdout:4/832: fdatasync f0 0 2024-05-15T15:54:24.380 INFO:tasks.workunit.client.1.smithi144.stdout:9/827: write d0/d8/d5e/fb6 [330143,69828] 0 2024-05-15T15:54:24.382 INFO:tasks.workunit.client.1.smithi144.stdout:4/833: dread - d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/dad/f108 zero size 2024-05-15T15:54:24.386 INFO:tasks.workunit.client.0.smithi049.stdout:7/716: unlink d0/d6/d23/d4a/f87 0 2024-05-15T15:54:24.387 INFO:tasks.workunit.client.1.smithi144.stdout:0/768: link f2 d4/d2d/d100/f11d 0 2024-05-15T15:54:24.390 INFO:tasks.workunit.client.1.smithi144.stdout:3/812: symlink d0/d1/d17/dae/dd7/l10a 0 2024-05-15T15:54:24.390 INFO:tasks.workunit.client.0.smithi049.stdout:4/711: write d18/f3e [742865,56734] 0 2024-05-15T15:54:24.391 INFO:tasks.workunit.client.1.smithi144.stdout:5/834: link d0/dc/d24/d8e/d93/d9e/cee d0/dc/d2f/c114 0 2024-05-15T15:54:24.392 INFO:tasks.workunit.client.0.smithi049.stdout:4/712: write d18/d85/d98/fd8 [647339,62496] 0 2024-05-15T15:54:24.394 INFO:tasks.workunit.client.0.smithi049.stdout:9/617: symlink d0/d8/ld8 0 2024-05-15T15:54:24.394 INFO:tasks.workunit.client.1.smithi144.stdout:2/763: symlink d3/d7/d20/d22/lfa 0 2024-05-15T15:54:24.396 INFO:tasks.workunit.client.0.smithi049.stdout:5/814: dwrite dc/dd/d47/f5d [0,4194304] 0 2024-05-15T15:54:24.396 INFO:tasks.workunit.client.0.smithi049.stdout:8/686: rmdir d2/de/d71/d86/d5d/d7d/d1b/dc8/dc9 0 2024-05-15T15:54:24.399 INFO:tasks.workunit.client.0.smithi049.stdout:7/717: symlink d0/d34/db3/ldf 0 2024-05-15T15:54:24.401 INFO:tasks.workunit.client.1.smithi144.stdout:7/809: mknod d4/df/d25/d40/d73/c119 0 2024-05-15T15:54:24.402 INFO:tasks.workunit.client.0.smithi049.stdout:4/713: symlink d18/d1e/d63/d99/de1/da8/lec 0 2024-05-15T15:54:24.405 INFO:tasks.workunit.client.0.smithi049.stdout:5/815: rmdir dc/dd/d7b/d8d 39 2024-05-15T15:54:24.407 INFO:tasks.workunit.client.0.smithi049.stdout:8/687: dwrite d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fd2 [0,4194304] 0 2024-05-15T15:54:24.408 INFO:tasks.workunit.client.1.smithi144.stdout:6/840: unlink d7/l22 0 2024-05-15T15:54:24.410 INFO:tasks.workunit.client.0.smithi049.stdout:2/705: rename d3/d8/daa to d3/da/d15/dec 0 2024-05-15T15:54:24.415 INFO:tasks.workunit.client.1.smithi144.stdout:1/800: mknod d7/d4b/c101 0 2024-05-15T15:54:24.418 INFO:tasks.workunit.client.0.smithi049.stdout:8/688: dwrite d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fb4 [0,4194304] 0 2024-05-15T15:54:24.424 INFO:tasks.workunit.client.0.smithi049.stdout:5/816: creat dc/dd/d47/da6/dce/f10f x:0 0 0 2024-05-15T15:54:24.424 INFO:tasks.workunit.client.0.smithi049.stdout:5/817: readlink dc/dd/d39/l52 0 2024-05-15T15:54:24.428 INFO:tasks.workunit.client.0.smithi049.stdout:2/706: symlink d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/dc9/dea/led 0 2024-05-15T15:54:24.429 INFO:tasks.workunit.client.1.smithi144.stdout:4/834: dwrite d6/dd/d34/f5f [0,4194304] 0 2024-05-15T15:54:24.431 INFO:tasks.workunit.client.0.smithi049.stdout:5/818: dwrite f9 [4194304,4194304] 0 2024-05-15T15:54:24.433 INFO:tasks.workunit.client.0.smithi049.stdout:9/618: creat d0/d8/fd9 x:0 0 0 2024-05-15T15:54:24.435 INFO:tasks.workunit.client.0.smithi049.stdout:9/619: dread d0/d25/f4c [0,4194304] 0 2024-05-15T15:54:24.437 INFO:tasks.workunit.client.0.smithi049.stdout:8/689: creat d2/de/d95/fd4 x:0 0 0 2024-05-15T15:54:24.438 INFO:tasks.workunit.client.0.smithi049.stdout:8/690: fsync d2/fc 0 2024-05-15T15:54:24.439 INFO:tasks.workunit.client.1.smithi144.stdout:0/769: getdents d4/d2d/d48/d5b/dda/def 0 2024-05-15T15:54:24.445 INFO:tasks.workunit.client.1.smithi144.stdout:0/770: dwrite d4/df/d1f/f2c [0,4194304] 0 2024-05-15T15:54:24.447 INFO:tasks.workunit.client.1.smithi144.stdout:7/810: write d4/f37 [348841,130778] 0 2024-05-15T15:54:24.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:24 smithi144 ceph-mon[36389]: pgmap v32: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 100 MiB/s rd, 266 MiB/s wr, 514 op/s 2024-05-15T15:54:24.449 INFO:tasks.workunit.client.1.smithi144.stdout:8/883: rename d2/c6f to d2/d24/d2d/da0/d6e/d71/c123 0 2024-05-15T15:54:24.451 INFO:tasks.workunit.client.1.smithi144.stdout:4/835: symlink d6/dd/d34/d3b/d30/l111 0 2024-05-15T15:54:24.452 INFO:tasks.workunit.client.1.smithi144.stdout:4/836: stat d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d44 0 2024-05-15T15:54:24.453 INFO:tasks.workunit.client.1.smithi144.stdout:9/828: write d0/d8/d5e/d44/dc2/d75/fb1 [940309,22455] 0 2024-05-15T15:54:24.454 INFO:tasks.workunit.client.1.smithi144.stdout:9/829: dread - d0/d8/d5e/f9a zero size 2024-05-15T15:54:24.455 INFO:tasks.workunit.client.0.smithi049.stdout:7/718: getdents d0/ddb/dde 0 2024-05-15T15:54:24.457 INFO:tasks.workunit.client.0.smithi049.stdout:2/707: rename d3/db/d1d/d1e/d2b/d2e/fbb to d3/db/d1d/d1e/d2b/d6b/fee 0 2024-05-15T15:54:24.458 INFO:tasks.workunit.client.0.smithi049.stdout:9/620: write d0/d8/d42/f91 [107096,66515] 0 2024-05-15T15:54:24.458 INFO:tasks.workunit.client.0.smithi049.stdout:9/621: stat d0/d8/d12 0 2024-05-15T15:54:24.461 INFO:tasks.workunit.client.0.smithi049.stdout:6/772: sync 2024-05-15T15:54:24.462 INFO:tasks.workunit.client.1.smithi144.stdout:1/801: dwrite d7/fc9 [0,4194304] 0 2024-05-15T15:54:24.462 INFO:tasks.workunit.client.0.smithi049.stdout:4/714: getdents d18/d1e/d2a/d50/d54 0 2024-05-15T15:54:24.463 INFO:tasks.workunit.client.0.smithi049.stdout:4/715: stat d18/d1e 0 2024-05-15T15:54:24.463 INFO:tasks.workunit.client.0.smithi049.stdout:8/691: mknod d2/cd5 0 2024-05-15T15:54:24.464 INFO:tasks.workunit.client.1.smithi144.stdout:3/813: dwrite d0/d1/d17/d39/f8c [0,4194304] 0 2024-05-15T15:54:24.465 INFO:tasks.workunit.client.0.smithi049.stdout:8/692: chown d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fb4 1042 1 2024-05-15T15:54:24.465 INFO:tasks.workunit.client.0.smithi049.stdout:2/708: symlink d3/db/d82/lef 0 2024-05-15T15:54:24.466 INFO:tasks.workunit.client.1.smithi144.stdout:1/802: write d7/d1d/d33/d1e/d4e/fbb [426160,28020] 0 2024-05-15T15:54:24.467 INFO:tasks.workunit.client.1.smithi144.stdout:2/764: creat d3/ffb x:0 0 0 2024-05-15T15:54:24.468 INFO:tasks.workunit.client.1.smithi144.stdout:5/835: truncate d0/dc/d12/fab 4154 0 2024-05-15T15:54:24.469 INFO:tasks.workunit.client.1.smithi144.stdout:0/771: dread d4/d2d/f8d [0,4194304] 0 2024-05-15T15:54:24.469 INFO:tasks.workunit.client.0.smithi049.stdout:9/622: creat d0/d8/d21/fda x:0 0 0 2024-05-15T15:54:24.471 INFO:tasks.workunit.client.1.smithi144.stdout:7/811: creat d4/d3c/d76/d9d/dc5/f11a x:0 0 0 2024-05-15T15:54:24.473 INFO:tasks.workunit.client.1.smithi144.stdout:7/812: write d4/df/d25/d38/d68/f10f [602224,22270] 0 2024-05-15T15:54:24.477 INFO:tasks.workunit.client.1.smithi144.stdout:9/830: dread - d0/d8/d1b/d62/d7d/d8d/dbb/dce/dfb/f105 zero size 2024-05-15T15:54:24.477 INFO:tasks.workunit.client.0.smithi049.stdout:8/693: dread d2/fc5 [0,4194304] 0 2024-05-15T15:54:24.480 INFO:tasks.workunit.client.1.smithi144.stdout:6/841: sync 2024-05-15T15:54:24.480 INFO:tasks.workunit.client.0.smithi049.stdout:9/623: symlink d0/d8/d9/d52/d5e/d8a/ldb 0 2024-05-15T15:54:24.482 INFO:tasks.workunit.client.0.smithi049.stdout:5/819: getdents dc/dd/d47/da6/dce 0 2024-05-15T15:54:24.484 INFO:tasks.workunit.client.1.smithi144.stdout:2/765: mknod d3/d7/d54/d98/cfc 0 2024-05-15T15:54:24.486 INFO:tasks.workunit.client.1.smithi144.stdout:5/836: creat d0/d7e/f115 x:0 0 0 2024-05-15T15:54:24.487 INFO:tasks.workunit.client.0.smithi049.stdout:8/694: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d60/da5/fd6 x:0 0 0 2024-05-15T15:54:24.488 INFO:tasks.workunit.client.1.smithi144.stdout:5/837: stat d0/dc/d2f/d34/d3e/fed 0 2024-05-15T15:54:24.489 INFO:tasks.workunit.client.1.smithi144.stdout:5/838: fdatasync d0/dc/d12/fd7 0 2024-05-15T15:54:24.492 INFO:tasks.workunit.client.1.smithi144.stdout:2/766: dwrite d3/d7/d20/d40/fa5 [0,4194304] 0 2024-05-15T15:54:24.492 INFO:tasks.workunit.client.0.smithi049.stdout:9/624: mknod d0/d8/d1b/d40/d5c/cdc 0 2024-05-15T15:54:24.495 INFO:tasks.workunit.client.0.smithi049.stdout:8/695: dwrite d2/de/d71/d86/d5d/d7d/db8/d5b/d60/fbb [0,4194304] 0 2024-05-15T15:54:24.496 INFO:tasks.workunit.client.1.smithi144.stdout:9/831: dread d0/d8/d1b/d79/d8a/f10d [0,4194304] 0 2024-05-15T15:54:24.496 INFO:tasks.workunit.client.0.smithi049.stdout:9/625: write d0/d8/d42/fd4 [798844,41236] 0 2024-05-15T15:54:24.497 INFO:tasks.workunit.client.0.smithi049.stdout:4/716: creat d18/d1e/d2a/fed x:0 0 0 2024-05-15T15:54:24.503 INFO:tasks.workunit.client.0.smithi049.stdout:6/773: write dd/d17/d24/d2b/d36/d86/f95 [520522,31150] 0 2024-05-15T15:54:24.506 INFO:tasks.workunit.client.0.smithi049.stdout:8/696: creat d2/de/d95/fd7 x:0 0 0 2024-05-15T15:54:24.506 INFO:tasks.workunit.client.1.smithi144.stdout:0/772: dread d4/d6/d3a/d5d/f82 [0,4194304] 0 2024-05-15T15:54:24.507 INFO:tasks.workunit.client.1.smithi144.stdout:1/803: getdents d7/d1d/dac/dad 0 2024-05-15T15:54:24.507 INFO:tasks.workunit.client.1.smithi144.stdout:8/884: dwrite d2/d3a/f91 [0,4194304] 0 2024-05-15T15:54:24.511 INFO:tasks.workunit.client.0.smithi049.stdout:2/709: dwrite d3/db/d1d/d1e/d2b/d2e/f87 [0,4194304] 0 2024-05-15T15:54:24.515 INFO:tasks.workunit.client.1.smithi144.stdout:7/813: mknod d4/df/c11b 0 2024-05-15T15:54:24.519 INFO:tasks.workunit.client.1.smithi144.stdout:5/839: mkdir d0/d10c/d73/d116 0 2024-05-15T15:54:24.520 INFO:tasks.workunit.client.1.smithi144.stdout:5/840: fdatasync d0/dc/d24/d8e/d93/d6b/dea/ffd 0 2024-05-15T15:54:24.520 INFO:tasks.workunit.client.0.smithi049.stdout:8/697: fdatasync d2/de/d71/d86/f96 0 2024-05-15T15:54:24.521 INFO:tasks.workunit.client.0.smithi049.stdout:8/698: readlink d2/de/d25/l92 0 2024-05-15T15:54:24.522 INFO:tasks.workunit.client.1.smithi144.stdout:2/767: write d3/d7/df8/d65/f9f [2097506,110673] 0 2024-05-15T15:54:24.523 INFO:tasks.workunit.client.0.smithi049.stdout:9/626: mknod d0/d8/cdd 0 2024-05-15T15:54:24.524 INFO:tasks.workunit.client.0.smithi049.stdout:9/627: readlink d0/d8/d1b/l6f 0 2024-05-15T15:54:24.524 INFO:tasks.workunit.client.0.smithi049.stdout:9/628: chown d0/d8/d12 23970373 1 2024-05-15T15:54:24.525 INFO:tasks.workunit.client.1.smithi144.stdout:4/837: creat d6/d42/f112 x:0 0 0 2024-05-15T15:54:24.530 INFO:tasks.workunit.client.0.smithi049.stdout:5/820: link dc/dd/d7b/lf6 dc/db0/dd5/dd8/l110 0 2024-05-15T15:54:24.535 INFO:tasks.workunit.client.0.smithi049.stdout:8/699: unlink d2/de/d71/f82 0 2024-05-15T15:54:24.535 INFO:tasks.workunit.client.0.smithi049.stdout:6/774: dwrite f2 [0,4194304] 0 2024-05-15T15:54:24.537 INFO:tasks.workunit.client.1.smithi144.stdout:8/885: creat d2/d24/d2d/da0/d6e/deb/f124 x:0 0 0 2024-05-15T15:54:24.538 INFO:tasks.workunit.client.0.smithi049.stdout:2/710: write d3/db/d1d/d1e/d2b/f54 [285807,50748] 0 2024-05-15T15:54:24.540 INFO:tasks.workunit.client.0.smithi049.stdout:9/629: rename d0/d8/d12/c50 to d0/d8/d9/d52/d5e/d8a/dd3/cde 0 2024-05-15T15:54:24.542 INFO:tasks.workunit.client.0.smithi049.stdout:4/717: link d18/d69/c82 d18/d1e/d2a/d36/d86/cee 0 2024-05-15T15:54:24.543 INFO:tasks.workunit.client.1.smithi144.stdout:5/841: mknod d0/dc/d24/d28/c117 0 2024-05-15T15:54:24.547 INFO:tasks.workunit.client.0.smithi049.stdout:8/700: mkdir d2/de/d25/d74/dbe/dd8 0 2024-05-15T15:54:24.548 INFO:tasks.workunit.client.1.smithi144.stdout:9/832: rename d0/d8/d5e/d44/dc2/f117 to d0/d8/d5e/d44/d16/db0/f11d 0 2024-05-15T15:54:24.550 INFO:tasks.workunit.client.1.smithi144.stdout:3/814: getdents d0/d1/d17/d39/d40/d4a/d9e 0 2024-05-15T15:54:24.552 INFO:tasks.workunit.client.1.smithi144.stdout:0/773: creat d4/d2d/d48/d10a/f11e x:0 0 0 2024-05-15T15:54:24.554 INFO:tasks.workunit.client.0.smithi049.stdout:9/630: mknod d0/d8/d9/d52/d5e/d8a/dd3/d87/cdf 0 2024-05-15T15:54:24.554 INFO:tasks.workunit.client.0.smithi049.stdout:9/631: readlink d0/d8/d53/lce 0 2024-05-15T15:54:24.556 INFO:tasks.workunit.client.1.smithi144.stdout:7/814: creat d4/d3c/d76/d9d/dac/d111/df0/f11c x:0 0 0 2024-05-15T15:54:24.557 INFO:tasks.workunit.client.0.smithi049.stdout:8/701: creat d2/de/d71/d86/d5d/d7d/db8/d5b/da6/fd9 x:0 0 0 2024-05-15T15:54:24.558 INFO:tasks.workunit.client.1.smithi144.stdout:5/842: symlink d0/dc/d2f/d79/da5/l118 0 2024-05-15T15:54:24.559 INFO:tasks.workunit.client.0.smithi049.stdout:5/821: dwrite dc/d4a/f36 [4194304,4194304] 0 2024-05-15T15:54:24.563 INFO:tasks.workunit.client.0.smithi049.stdout:4/718: fdatasync d18/d76/f92 0 2024-05-15T15:54:24.565 INFO:tasks.workunit.client.1.smithi144.stdout:9/833: creat d0/d8/d1b/d79/de7/df9/f11e x:0 0 0 2024-05-15T15:54:24.566 INFO:tasks.workunit.client.0.smithi049.stdout:8/702: truncate d2/de/d71/d86/d5d/d63/f6e 1220561 0 2024-05-15T15:54:24.567 INFO:tasks.workunit.client.1.smithi144.stdout:0/774: dread d4/df/d14/f5f [0,4194304] 0 2024-05-15T15:54:24.567 INFO:tasks.workunit.client.1.smithi144.stdout:3/815: creat d0/d1/d17/d39/d83/f10b x:0 0 0 2024-05-15T15:54:24.568 INFO:tasks.workunit.client.1.smithi144.stdout:0/775: chown d4/d6/d4a/lc9 6 1 2024-05-15T15:54:24.574 INFO:tasks.workunit.client.0.smithi049.stdout:6/775: write dd/d17/d24/d2b/d36/d50/d54/dde/fa7 [1798094,102200] 0 2024-05-15T15:54:24.575 INFO:tasks.workunit.client.0.smithi049.stdout:2/711: readlink d3/da/d15/dec/ddf/l9b 0 2024-05-15T15:54:24.575 INFO:tasks.workunit.client.0.smithi049.stdout:2/712: readlink d3/da/d15/dec/ddf/ld4 0 2024-05-15T15:54:24.575 INFO:tasks.workunit.client.0.smithi049.stdout:2/713: chown d3/db/d1d/d1e/d2b/d3b/d42/f69 2 1 2024-05-15T15:54:24.576 INFO:tasks.workunit.client.1.smithi144.stdout:7/815: dread - d4/df/d81/dfb/dbd/fd4 zero size 2024-05-15T15:54:24.576 INFO:tasks.workunit.client.1.smithi144.stdout:0/776: dwrite d4/df/d14/f111 [0,4194304] 0 2024-05-15T15:54:24.576 INFO:tasks.workunit.client.1.smithi144.stdout:7/816: stat d4/d3c/d76/d9d/dac/d48/d72/da2 0 2024-05-15T15:54:24.576 INFO:tasks.workunit.client.1.smithi144.stdout:0/777: write d4/d2d/f10e [321169,10490] 0 2024-05-15T15:54:24.577 INFO:tasks.workunit.client.0.smithi049.stdout:2/714: readlink d3/da/d15/dec/ddf/ld4 0 2024-05-15T15:54:24.577 INFO:tasks.workunit.client.0.smithi049.stdout:4/719: rename d18/d21/c52 to d18/d1e/d2a/d50/d54/cef 0 2024-05-15T15:54:24.577 INFO:tasks.workunit.client.1.smithi144.stdout:0/778: readlink d4/d2d/d48/d5b/l89 0 2024-05-15T15:54:24.577 INFO:tasks.workunit.client.1.smithi144.stdout:0/779: fdatasync d4/d6/d3a/d4c/fea 0 2024-05-15T15:54:24.577 INFO:tasks.workunit.client.1.smithi144.stdout:4/838: link d6/dd/d34/d3b/d30/dc9/d10f/cf2 d6/dd/d34/d3b/d30/d59/d66/dcf/c113 0 2024-05-15T15:54:24.578 INFO:tasks.workunit.client.0.smithi049.stdout:8/703: unlink d2/de/d95/fd4 0 2024-05-15T15:54:24.581 INFO:tasks.workunit.client.0.smithi049.stdout:5/822: mkdir dc/d4a/d42/dff/d111 0 2024-05-15T15:54:24.583 INFO:tasks.workunit.client.0.smithi049.stdout:5/823: readlink dc/dd/d47/d83/dcc/l105 0 2024-05-15T15:54:24.584 INFO:tasks.workunit.client.0.smithi049.stdout:6/776: fsync dd/d17/d24/d2b/d36/d50/d7e/f81 0 2024-05-15T15:54:24.585 INFO:tasks.workunit.client.1.smithi144.stdout:2/768: getdents d3/d7/d20/d22/d5e 0 2024-05-15T15:54:24.589 INFO:tasks.workunit.client.1.smithi144.stdout:7/817: mknod d4/df/d25/d40/d62/c11d 0 2024-05-15T15:54:24.589 INFO:tasks.workunit.client.1.smithi144.stdout:7/818: readlink d4/d3c/d76/d9d/dac/d48/l56 0 2024-05-15T15:54:24.593 INFO:tasks.workunit.client.0.smithi049.stdout:5/824: creat dc/dd/d10/db3/dd6/f112 x:0 0 0 2024-05-15T15:54:24.595 INFO:tasks.workunit.client.0.smithi049.stdout:6/777: symlink dd/d17/d24/d2b/d36/d50/d54/dde/d8c/le9 0 2024-05-15T15:54:24.597 INFO:tasks.workunit.client.0.smithi049.stdout:2/715: mkdir d3/db/df0 0 2024-05-15T15:54:24.597 INFO:tasks.workunit.client.1.smithi144.stdout:9/834: dwrite d0/d8/d5e/f115 [0,4194304] 0 2024-05-15T15:54:24.597 INFO:tasks.workunit.client.0.smithi049.stdout:8/704: write d2/f3 [7622589,230] 0 2024-05-15T15:54:24.598 INFO:tasks.workunit.client.1.smithi144.stdout:8/886: dwrite d2/d24/d2d/da0/d6e/d71/d73/d98/ff0 [0,4194304] 0 2024-05-15T15:54:24.599 INFO:tasks.workunit.client.1.smithi144.stdout:5/843: creat d0/dc/d2f/d68/f119 x:0 0 0 2024-05-15T15:54:24.601 INFO:tasks.workunit.client.1.smithi144.stdout:7/819: dread - d4/d3c/d76/d9d/dac/d48/fd2 zero size 2024-05-15T15:54:24.603 INFO:tasks.workunit.client.1.smithi144.stdout:9/835: mkdir d0/d8/d1b/d79/d8a/dae/ddb/d11f 0 2024-05-15T15:54:24.605 INFO:tasks.workunit.client.1.smithi144.stdout:4/839: creat d6/dd/f114 x:0 0 0 2024-05-15T15:54:24.605 INFO:tasks.workunit.client.1.smithi144.stdout:9/836: chown d0/d8/d5e/d44/d16/f3d 0 1 2024-05-15T15:54:24.606 INFO:tasks.workunit.client.0.smithi049.stdout:5/825: dread dc/dd/d10/f2a [0,4194304] 0 2024-05-15T15:54:24.608 INFO:tasks.workunit.client.0.smithi049.stdout:6/778: getdents dd/d17/d24/d2b/d36/d50/d54/db6 0 2024-05-15T15:54:24.609 INFO:tasks.workunit.client.0.smithi049.stdout:5/826: rename dc/d4a/d42/dff/d111 to dc/d4a/d42/dff/d111/d113 22 2024-05-15T15:54:24.610 INFO:tasks.workunit.client.1.smithi144.stdout:9/837: fdatasync d0/d8/d5e/fa4 0 2024-05-15T15:54:24.610 INFO:tasks.workunit.client.1.smithi144.stdout:5/844: read d0/dc/d1f/f39 [3539877,12089] 0 2024-05-15T15:54:24.610 INFO:tasks.workunit.client.0.smithi049.stdout:5/827: chown dc/dd/d10/db3/dd6/f10d 5399236 1 2024-05-15T15:54:24.611 INFO:tasks.workunit.client.1.smithi144.stdout:4/840: symlink d6/dd/d34/de2/l115 0 2024-05-15T15:54:24.613 INFO:tasks.workunit.client.1.smithi144.stdout:0/780: dread d4/df/d1f/fb3 [0,4194304] 0 2024-05-15T15:54:24.613 INFO:tasks.workunit.client.0.smithi049.stdout:6/779: creat dd/d17/d24/d2b/d36/d86/fea x:0 0 0 2024-05-15T15:54:24.614 INFO:tasks.workunit.client.1.smithi144.stdout:9/838: chown d0/d8/d5e/d44/l37 31689999 1 2024-05-15T15:54:24.615 INFO:tasks.workunit.client.0.smithi049.stdout:5/828: symlink dc/db0/dbf/l114 0 2024-05-15T15:54:24.615 INFO:tasks.workunit.client.0.smithi049.stdout:5/829: write dc/dd/f3b [7715753,18748] 0 2024-05-15T15:54:24.616 INFO:tasks.workunit.client.1.smithi144.stdout:6/842: sync 2024-05-15T15:54:24.616 INFO:tasks.workunit.client.1.smithi144.stdout:5/845: unlink d0/dc/d12/fd7 0 2024-05-15T15:54:24.618 INFO:tasks.workunit.client.0.smithi049.stdout:6/780: creat dd/d17/d24/d58/d5f/dd7/feb x:0 0 0 2024-05-15T15:54:24.619 INFO:tasks.workunit.client.1.smithi144.stdout:5/846: readlink d0/dc/d24/d28/l53 0 2024-05-15T15:54:24.622 INFO:tasks.workunit.client.1.smithi144.stdout:0/781: rename d4/dc/dde to d4/d6/d3a/d4c/d11f 0 2024-05-15T15:54:24.623 INFO:tasks.workunit.client.1.smithi144.stdout:7/820: dread d4/d41/d5b/f4c [0,4194304] 0 2024-05-15T15:54:24.624 INFO:tasks.workunit.client.1.smithi144.stdout:7/821: chown d4/d3c/d76/d9d/dac/d48 522491127 1 2024-05-15T15:54:24.625 INFO:tasks.workunit.client.1.smithi144.stdout:9/839: read d0/d8/fb8 [108817,48841] 0 2024-05-15T15:54:24.625 INFO:tasks.workunit.client.0.smithi049.stdout:6/781: fdatasync dd/d26/f2e 0 2024-05-15T15:54:24.630 INFO:tasks.workunit.client.0.smithi049.stdout:8/705: dwrite d2/de/f55 [0,4194304] 0 2024-05-15T15:54:24.630 INFO:tasks.workunit.client.1.smithi144.stdout:8/887: truncate d2/d3a/d7b/d4b/fb5 3586959 0 2024-05-15T15:54:24.632 INFO:tasks.workunit.client.0.smithi049.stdout:6/782: creat dd/dbc/fec x:0 0 0 2024-05-15T15:54:24.634 INFO:tasks.workunit.client.1.smithi144.stdout:6/843: dread d7/db/d35/d84/dbe/fd3 [0,4194304] 0 2024-05-15T15:54:24.634 INFO:tasks.workunit.client.1.smithi144.stdout:0/782: mknod d4/d6/d3a/d4c/dd6/d55/da5/c120 0 2024-05-15T15:54:24.635 INFO:tasks.workunit.client.1.smithi144.stdout:0/783: stat d4/df/d1f/d24/c8a 0 2024-05-15T15:54:24.636 INFO:tasks.workunit.client.0.smithi049.stdout:5/830: creat dc/dd/d47/d83/dcc/f115 x:0 0 0 2024-05-15T15:54:24.639 INFO:tasks.workunit.client.1.smithi144.stdout:7/822: creat d4/d41/d5b/da0/f11e x:0 0 0 2024-05-15T15:54:24.641 INFO:tasks.workunit.client.0.smithi049.stdout:5/831: chown dc/dd/d10/l43 19921 1 2024-05-15T15:54:24.641 INFO:tasks.workunit.client.0.smithi049.stdout:5/832: chown dc/l24 1 1 2024-05-15T15:54:24.643 INFO:tasks.workunit.client.1.smithi144.stdout:4/841: dwrite f0 [0,4194304] 0 2024-05-15T15:54:24.644 INFO:tasks.workunit.client.1.smithi144.stdout:4/842: dread - d6/d42/f112 zero size 2024-05-15T15:54:24.650 INFO:tasks.workunit.client.1.smithi144.stdout:6/844: dread d7/d1e/d28/db5/df6/f105 [0,4194304] 0 2024-05-15T15:54:24.680 INFO:tasks.workunit.client.1.smithi144.stdout:1/804: sync 2024-05-15T15:54:24.686 INFO:tasks.workunit.client.0.smithi049.stdout:5/833: creat dc/dd/d10/df1/f116 x:0 0 0 2024-05-15T15:54:24.686 INFO:tasks.workunit.client.1.smithi144.stdout:8/888: read d2/d24/d2d/d30/dd0/df7/f17 [224626,54534] 0 2024-05-15T15:54:24.686 INFO:tasks.workunit.client.1.smithi144.stdout:5/847: dwrite d0/dc/d2f/d34/d3e/f48 [0,4194304] 0 2024-05-15T15:54:24.687 INFO:tasks.workunit.client.0.smithi049.stdout:5/834: dread - dc/dd/d47/d83/dcc/f115 zero size 2024-05-15T15:54:24.689 INFO:tasks.workunit.client.0.smithi049.stdout:8/706: link d2/de/c41 d2/de/d71/d86/d5d/d7d/d1b/d68/cda 0 2024-05-15T15:54:24.692 INFO:tasks.workunit.client.0.smithi049.stdout:6/783: link dd/c91 dd/d17/d24/d58/d5f/ced 0 2024-05-15T15:54:24.695 INFO:tasks.workunit.client.0.smithi049.stdout:6/784: readlink dd/d26/l31 0 2024-05-15T15:54:24.695 INFO:tasks.workunit.client.1.smithi144.stdout:7/823: creat d4/d3c/f11f x:0 0 0 2024-05-15T15:54:24.698 INFO:tasks.workunit.client.0.smithi049.stdout:5/835: dwrite f0 [0,4194304] 0 2024-05-15T15:54:24.702 INFO:tasks.workunit.client.0.smithi049.stdout:5/836: fdatasync dc/dd/d47/d83/dcc/da9/ff9 0 2024-05-15T15:54:24.703 INFO:tasks.workunit.client.0.smithi049.stdout:6/785: creat dd/d17/d9d/da1/fee x:0 0 0 2024-05-15T15:54:24.704 INFO:tasks.workunit.client.0.smithi049.stdout:8/707: dwrite d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fd2 [4194304,4194304] 0 2024-05-15T15:54:24.708 INFO:tasks.workunit.client.0.smithi049.stdout:6/786: symlink dd/d26/d5b/lef 0 2024-05-15T15:54:24.712 INFO:tasks.workunit.client.0.smithi049.stdout:6/787: write dd/d17/d24/d2b/d36/f56 [2305130,52601] 0 2024-05-15T15:54:24.715 INFO:tasks.workunit.client.0.smithi049.stdout:8/708: dwrite d2/de/d71/d86/d5d/d7d/d1b/d68/f97 [0,4194304] 0 2024-05-15T15:54:24.723 INFO:tasks.workunit.client.1.smithi144.stdout:6/845: dread - d7/db/d3a/fe4 zero size 2024-05-15T15:54:24.723 INFO:tasks.workunit.client.1.smithi144.stdout:8/889: truncate d2/d24/fa2 214308 0 2024-05-15T15:54:24.724 INFO:tasks.workunit.client.1.smithi144.stdout:7/824: creat d4/d3c/f120 x:0 0 0 2024-05-15T15:54:24.726 INFO:tasks.workunit.client.1.smithi144.stdout:9/840: dwrite d0/d8/d5e/d44/d10/f81 [0,4194304] 0 2024-05-15T15:54:24.726 INFO:tasks.workunit.client.0.smithi049.stdout:5/837: dread dc/dd/d47/d83/dcc/da9/faa [0,4194304] 0 2024-05-15T15:54:24.730 INFO:tasks.workunit.client.1.smithi144.stdout:5/848: mknod d0/dc/d2f/d68/c11a 0 2024-05-15T15:54:24.732 INFO:tasks.workunit.client.0.smithi049.stdout:8/709: write d2/de/d25/f73 [547047,3777] 0 2024-05-15T15:54:24.733 INFO:tasks.workunit.client.1.smithi144.stdout:6/846: rmdir d7/d1e/d28 39 2024-05-15T15:54:24.736 INFO:tasks.workunit.client.1.smithi144.stdout:4/843: truncate d6/dd/d34/d3b/d30/dc9/d10f/fe3 2138746 0 2024-05-15T15:54:24.737 INFO:tasks.workunit.client.1.smithi144.stdout:4/844: chown d6/dd/d34/d3b/d30/db/d1d/d4e 536 1 2024-05-15T15:54:24.740 INFO:tasks.workunit.client.0.smithi049.stdout:5/838: dwrite dc/d4a/f62 [0,4194304] 0 2024-05-15T15:54:24.740 INFO:tasks.workunit.client.1.smithi144.stdout:9/841: dread d0/d8/d1b/d79/d8a/f10d [0,4194304] 0 2024-05-15T15:54:24.743 INFO:tasks.workunit.client.1.smithi144.stdout:0/784: link d4/df/d1f/f58 d4/df/d1f/d24/f121 0 2024-05-15T15:54:24.743 INFO:tasks.workunit.client.0.smithi049.stdout:8/710: creat d2/de/d25/fdb x:0 0 0 2024-05-15T15:54:24.744 INFO:tasks.workunit.client.1.smithi144.stdout:8/890: mkdir d2/de/d3f/dff/d125 0 2024-05-15T15:54:24.744 INFO:tasks.workunit.client.0.smithi049.stdout:8/711: readlink d2/de/d71/d86/d5d/d7d/d1b/d68/l99 0 2024-05-15T15:54:24.746 INFO:tasks.workunit.client.0.smithi049.stdout:8/712: stat d2/de/d71/d86/d5d/d7d/db8/f85 0 2024-05-15T15:54:24.748 INFO:tasks.workunit.client.0.smithi049.stdout:5/839: creat dc/dd/d47/d83/dcc/da9/f117 x:0 0 0 2024-05-15T15:54:24.749 INFO:tasks.workunit.client.1.smithi144.stdout:5/849: write d0/dc/d24/d8e/d93/d6b/d88/f8d [817021,4213] 0 2024-05-15T15:54:24.750 INFO:tasks.workunit.client.1.smithi144.stdout:4/845: mkdir d6/dd/d34/d116 0 2024-05-15T15:54:24.753 INFO:tasks.workunit.client.0.smithi049.stdout:8/713: dwrite d2/fc [0,4194304] 0 2024-05-15T15:54:24.757 INFO:tasks.workunit.client.0.smithi049.stdout:5/840: readlink dc/dd/d47/d83/dcc/da9/dba/dbe/le7 0 2024-05-15T15:54:24.757 INFO:tasks.workunit.client.1.smithi144.stdout:2/769: sync 2024-05-15T15:54:24.757 INFO:tasks.workunit.client.1.smithi144.stdout:3/816: sync 2024-05-15T15:54:24.757 INFO:tasks.workunit.client.0.smithi049.stdout:6/788: dread dd/d17/d24/d2b/d36/d50/d54/f8f [0,4194304] 0 2024-05-15T15:54:24.763 INFO:tasks.workunit.client.0.smithi049.stdout:6/789: unlink dd/d17/d24/d2b/d36/d50/d54/dde/d8c/fce 0 2024-05-15T15:54:24.765 INFO:tasks.workunit.client.1.smithi144.stdout:6/847: dwrite d7/d62/fa9 [4194304,4194304] 0 2024-05-15T15:54:24.765 INFO:tasks.workunit.client.0.smithi049.stdout:8/714: write d2/de/d71/d86/d5d/d7d/db8/f85 [187350,59249] 0 2024-05-15T15:54:24.766 INFO:tasks.workunit.client.1.smithi144.stdout:9/842: truncate d0/d8/d5e/d44/d10/d9c/f54 443485 0 2024-05-15T15:54:24.767 INFO:tasks.workunit.client.0.smithi049.stdout:5/841: mknod dc/dd/d47/c118 0 2024-05-15T15:54:24.769 INFO:tasks.workunit.client.1.smithi144.stdout:0/785: unlink d4/f1c 0 2024-05-15T15:54:24.771 INFO:tasks.workunit.client.1.smithi144.stdout:3/817: chown d0/d1/c5d 7730402 1 2024-05-15T15:54:24.771 INFO:tasks.workunit.client.0.smithi049.stdout:6/790: mknod dd/d17/d9d/cf0 0 2024-05-15T15:54:24.771 INFO:tasks.workunit.client.1.smithi144.stdout:0/786: write d4/d6/f32 [2089910,126996] 0 2024-05-15T15:54:24.775 INFO:tasks.workunit.client.1.smithi144.stdout:0/787: chown d4/df/d1f/d24/fbc 90397583 1 2024-05-15T15:54:24.775 INFO:tasks.workunit.client.1.smithi144.stdout:6/848: dwrite d7/db/d11/d7e/ff9 [0,4194304] 0 2024-05-15T15:54:24.778 INFO:tasks.workunit.client.1.smithi144.stdout:9/843: fsync d0/d8/d1b/d62/d7d/d8d/dbb/dce/fdd 0 2024-05-15T15:54:24.780 INFO:tasks.workunit.client.1.smithi144.stdout:6/849: mkdir d7/db/d5f/d12c 0 2024-05-15T15:54:24.780 INFO:tasks.workunit.client.0.smithi049.stdout:0/737: sync 2024-05-15T15:54:24.780 INFO:tasks.workunit.client.0.smithi049.stdout:8/715: dwrite d2/fc5 [0,4194304] 0 2024-05-15T15:54:24.785 INFO:tasks.workunit.client.1.smithi144.stdout:3/818: dwrite d0/d11/d3b/da0/d8f/f9b [0,4194304] 0 2024-05-15T15:54:24.787 INFO:tasks.workunit.client.1.smithi144.stdout:8/891: dread d2/d24/d2d/da0/d6e/f78 [0,4194304] 0 2024-05-15T15:54:24.801 INFO:tasks.workunit.client.1.smithi144.stdout:1/805: sync 2024-05-15T15:54:24.805 INFO:tasks.workunit.client.0.smithi049.stdout:5/842: dread dc/db0/dd5/fe1 [0,4194304] 0 2024-05-15T15:54:24.827 INFO:tasks.workunit.client.1.smithi144.stdout:4/846: rename d6/dd/d34/d3b/d30/db/d1d/d40/d57/l91 to d6/d42/l117 0 2024-05-15T15:54:24.831 INFO:tasks.workunit.client.0.smithi049.stdout:0/738: mkdir dc/d13/d14/d2f/df5 0 2024-05-15T15:54:24.831 INFO:tasks.workunit.client.1.smithi144.stdout:7/825: sync 2024-05-15T15:54:24.832 INFO:tasks.workunit.client.1.smithi144.stdout:7/826: fsync d4/d3c/d76/d9d/dba/f10c 0 2024-05-15T15:54:24.832 INFO:tasks.workunit.client.1.smithi144.stdout:2/770: creat d3/d7/d20/d22/ffd x:0 0 0 2024-05-15T15:54:24.835 INFO:tasks.workunit.client.0.smithi049.stdout:8/716: unlink d2/de/d25/l92 0 2024-05-15T15:54:24.838 INFO:tasks.workunit.client.0.smithi049.stdout:5/843: creat dc/dd/d10/f119 x:0 0 0 2024-05-15T15:54:24.839 INFO:tasks.workunit.client.1.smithi144.stdout:0/788: mkdir d4/d6/d3a/d4c/dd6/d55/da5/d115/d122 0 2024-05-15T15:54:24.840 INFO:tasks.workunit.client.0.smithi049.stdout:5/844: truncate dc/dd/d47/d83/dcc/da9/f10a 685004 0 2024-05-15T15:54:24.844 INFO:tasks.workunit.client.0.smithi049.stdout:0/739: mknod dc/d13/d56/d6f/dce/def/cf6 0 2024-05-15T15:54:24.845 INFO:tasks.workunit.client.1.smithi144.stdout:8/892: stat d2/d24/d2d/da0/d6e/d71/d8c/fcc 0 2024-05-15T15:54:24.845 INFO:tasks.workunit.client.1.smithi144.stdout:8/893: readlink d2/de/l50 0 2024-05-15T15:54:24.846 INFO:tasks.workunit.client.0.smithi049.stdout:5/845: rmdir dc/dd/d47/d83/dcc/da9/dba/dbe 39 2024-05-15T15:54:24.849 INFO:tasks.workunit.client.0.smithi049.stdout:8/717: symlink d2/de/d71/d86/d5d/d7d/db8/d5b/dd0/ldc 0 2024-05-15T15:54:24.851 INFO:tasks.workunit.client.1.smithi144.stdout:5/850: rename d0/dc/d2f/d34/dc1/ff5 to d0/dc/d2f/d79/f11b 0 2024-05-15T15:54:24.852 INFO:tasks.workunit.client.0.smithi049.stdout:0/740: truncate dc/d13/d2b/f4d 89321 0 2024-05-15T15:54:24.852 INFO:tasks.workunit.client.1.smithi144.stdout:5/851: write d0/dc/d24/d57/f7f [2040902,126011] 0 2024-05-15T15:54:24.853 INFO:tasks.workunit.client.1.smithi144.stdout:4/847: truncate d6/dd/d34/fc2 328533 0 2024-05-15T15:54:24.853 INFO:tasks.workunit.client.1.smithi144.stdout:4/848: readlink d6/dd/l110 0 2024-05-15T15:54:24.853 INFO:tasks.workunit.client.1.smithi144.stdout:5/852: chown d0/dc/d2f/d79 459188321 1 2024-05-15T15:54:24.854 INFO:tasks.workunit.client.0.smithi049.stdout:5/846: dwrite dc/dd/d47/d83/dcc/da9/f117 [0,4194304] 0 2024-05-15T15:54:24.855 INFO:tasks.workunit.client.0.smithi049.stdout:0/741: dread - dc/d13/d14/d25/d31/d71/dbf/fe4 zero size 2024-05-15T15:54:24.857 INFO:tasks.workunit.client.1.smithi144.stdout:1/806: write d7/d1d/d33/d29/f42 [490636,68758] 0 2024-05-15T15:54:24.860 INFO:tasks.workunit.client.1.smithi144.stdout:2/771: fdatasync d3/d7/d20/d22/d52/d64/f31 0 2024-05-15T15:54:24.862 INFO:tasks.workunit.client.0.smithi049.stdout:5/847: dwrite dc/dd/d47/d83/dcc/d5c/f7e [0,4194304] 0 2024-05-15T15:54:24.865 INFO:tasks.workunit.client.0.smithi049.stdout:8/718: rename d2/f5a to d2/de/d71/d86/fdd 0 2024-05-15T15:54:24.869 INFO:tasks.workunit.client.1.smithi144.stdout:0/789: creat d4/d6/d4a/f123 x:0 0 0 2024-05-15T15:54:24.872 INFO:tasks.workunit.client.0.smithi049.stdout:0/742: creat dc/d13/d14/dc9/ff7 x:0 0 0 2024-05-15T15:54:24.873 INFO:tasks.workunit.client.0.smithi049.stdout:5/848: rmdir dc/d4a/ddd 39 2024-05-15T15:54:24.878 INFO:tasks.workunit.client.1.smithi144.stdout:8/894: dread d2/d24/d2d/d30/dd0/fdd [0,4194304] 0 2024-05-15T15:54:24.879 INFO:tasks.workunit.client.1.smithi144.stdout:8/895: fsync d2/d24/d2d/d30/dd0/df7/f1b 0 2024-05-15T15:54:24.880 INFO:tasks.workunit.client.0.smithi049.stdout:8/719: rename d2/de/d71/d86/d5d/d7d/db8/d5b/d60/fbb to d2/de/d71/d86/d5d/d7d/d1b/d69/fde 0 2024-05-15T15:54:24.886 INFO:tasks.workunit.client.1.smithi144.stdout:6/850: rename d7/db/d9d/d97/f9a to d7/db/d3a/de3/df2/d11e/f12d 0 2024-05-15T15:54:24.892 INFO:tasks.workunit.client.0.smithi049.stdout:8/720: mknod d2/de/dc3/cdf 0 2024-05-15T15:54:24.895 INFO:tasks.workunit.client.0.smithi049.stdout:0/743: dwrite dc/d13/d14/d25/d31/dae/dc5/d57/f65 [0,4194304] 0 2024-05-15T15:54:24.896 INFO:tasks.workunit.client.0.smithi049.stdout:5/849: rmdir dc/dd/d47/d83/dcc/da9/dba/dbe 39 2024-05-15T15:54:24.897 INFO:tasks.workunit.client.1.smithi144.stdout:6/851: dread d7/db/d11/f60 [4194304,4194304] 0 2024-05-15T15:54:24.898 INFO:tasks.workunit.client.1.smithi144.stdout:7/827: mknod d4/df/d25/c121 0 2024-05-15T15:54:24.898 INFO:tasks.workunit.client.0.smithi049.stdout:8/721: truncate d2/de/d71/d86/fcf 786963 0 2024-05-15T15:54:24.901 INFO:tasks.workunit.client.0.smithi049.stdout:8/722: write d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fd2 [2187987,17986] 0 2024-05-15T15:54:24.905 INFO:tasks.workunit.client.0.smithi049.stdout:5/850: creat dc/d4a/d42/dff/f11a x:0 0 0 2024-05-15T15:54:24.908 INFO:tasks.workunit.client.1.smithi144.stdout:1/807: write d7/d1d/dac/fc1 [775055,80682] 0 2024-05-15T15:54:24.910 INFO:tasks.workunit.client.0.smithi049.stdout:8/723: write d2/de/d71/d86/d5d/d7d/d45/f7c [4780620,56502] 0 2024-05-15T15:54:24.912 INFO:tasks.workunit.client.1.smithi144.stdout:3/819: mkdir d0/d1/d56/d98/d10c 0 2024-05-15T15:54:24.916 INFO:tasks.workunit.client.1.smithi144.stdout:5/853: getdents d0/dc/d24/d8e/d93/ddb 0 2024-05-15T15:54:24.917 INFO:tasks.workunit.client.1.smithi144.stdout:2/772: write d3/d7/d20/d89/d6e/fc4 [578947,28199] 0 2024-05-15T15:54:24.918 INFO:tasks.workunit.client.0.smithi049.stdout:5/851: rmdir dc/d4a/d42/dff 39 2024-05-15T15:54:24.919 INFO:tasks.workunit.client.1.smithi144.stdout:6/852: read - d7/d32/fde zero size 2024-05-15T15:54:24.922 INFO:tasks.workunit.client.0.smithi049.stdout:5/852: symlink dc/dd/d39/l11b 0 2024-05-15T15:54:24.923 INFO:tasks.workunit.client.0.smithi049.stdout:5/853: dread - dc/dd/d10/db3/dd6/ffc zero size 2024-05-15T15:54:24.926 INFO:tasks.workunit.client.1.smithi144.stdout:4/849: creat d6/f118 x:0 0 0 2024-05-15T15:54:24.926 INFO:tasks.workunit.client.1.smithi144.stdout:8/896: dwrite d2/d24/f89 [0,4194304] 0 2024-05-15T15:54:24.927 INFO:tasks.workunit.client.1.smithi144.stdout:2/773: chown d3/d10/lba 15043 1 2024-05-15T15:54:24.928 INFO:tasks.workunit.client.0.smithi049.stdout:0/744: dread dc/d13/d14/d25/d31/dae/dc5/f50 [0,4194304] 0 2024-05-15T15:54:24.932 INFO:tasks.workunit.client.0.smithi049.stdout:8/724: dread d2/fb [0,4194304] 0 2024-05-15T15:54:24.932 INFO:tasks.workunit.client.0.smithi049.stdout:0/745: rmdir dc/d13/d14/d25/db9/dd0 39 2024-05-15T15:54:24.933 INFO:tasks.workunit.client.1.smithi144.stdout:1/808: mknod d7/d1d/d33/d41/d22/db4/c102 0 2024-05-15T15:54:24.933 INFO:tasks.workunit.client.0.smithi049.stdout:8/725: stat d2/de/d71/d86/d5d/d63/c6b 0 2024-05-15T15:54:24.934 INFO:tasks.workunit.client.1.smithi144.stdout:0/790: fsync d4/df/d1f/d24/f121 0 2024-05-15T15:54:24.935 INFO:tasks.workunit.client.1.smithi144.stdout:9/844: getdents d0/d8/d5e/d44/da2 0 2024-05-15T15:54:24.936 INFO:tasks.workunit.client.1.smithi144.stdout:0/791: dread d4/d6/d3a/d4c/dd6/f3b [0,4194304] 0 2024-05-15T15:54:24.936 INFO:tasks.workunit.client.1.smithi144.stdout:0/792: fsync d4/d6/d3a/d5d/f8f 0 2024-05-15T15:54:24.939 INFO:tasks.workunit.client.0.smithi049.stdout:0/746: rename dc/d13/d56/d6f/lc6 to dc/de9/lf8 0 2024-05-15T15:54:24.940 INFO:tasks.workunit.client.1.smithi144.stdout:5/854: dwrite d0/dc/d1f/f39 [0,4194304] 0 2024-05-15T15:54:24.941 INFO:tasks.workunit.client.1.smithi144.stdout:5/855: fdatasync d0/dc/f8c 0 2024-05-15T15:54:24.942 INFO:tasks.workunit.client.1.smithi144.stdout:5/856: chown d0/dc/d24/d57/f7f 15 1 2024-05-15T15:54:24.943 INFO:tasks.workunit.client.1.smithi144.stdout:8/897: readlink d2/de/d49/d83/dd9/l111 0 2024-05-15T15:54:24.944 INFO:tasks.workunit.client.0.smithi049.stdout:5/854: truncate dc/db0/dbf/fc5 3068012 0 2024-05-15T15:54:24.947 INFO:tasks.workunit.client.0.smithi049.stdout:0/747: unlink dc/d13/d14/d33/f47 0 2024-05-15T15:54:24.952 INFO:tasks.workunit.client.1.smithi144.stdout:4/850: write d6/dd/d34/d3b/d30/d59/f74 [3898140,100690] 0 2024-05-15T15:54:24.955 INFO:tasks.workunit.client.0.smithi049.stdout:8/726: write d2/de/d71/d86/d5d/d7d/f43 [5065494,96232] 0 2024-05-15T15:54:24.955 INFO:tasks.workunit.client.0.smithi049.stdout:0/748: rename dc/d6d/c66 to dc/da4/db4/cf9 0 2024-05-15T15:54:24.957 INFO:tasks.workunit.client.1.smithi144.stdout:1/809: creat d7/d1d/d33/d29/d6b/f103 x:0 0 0 2024-05-15T15:54:24.968 INFO:tasks.workunit.client.1.smithi144.stdout:3/820: rmdir d0/d1/d56/d71/df3 0 2024-05-15T15:54:24.974 INFO:tasks.workunit.client.1.smithi144.stdout:5/857: mknod d0/dc/d2f/d34/dc1/c11c 0 2024-05-15T15:54:24.974 INFO:tasks.workunit.client.1.smithi144.stdout:5/858: fdatasync d0/ff8 0 2024-05-15T15:54:24.979 INFO:tasks.workunit.client.1.smithi144.stdout:6/853: link d7/d1e/d28/db5/df6/f126 d7/db/d3a/dd2/f12e 0 2024-05-15T15:54:24.986 INFO:tasks.workunit.client.1.smithi144.stdout:9/845: mkdir d0/d8/d5e/d44/d99/d120 0 2024-05-15T15:54:24.988 INFO:tasks.workunit.client.1.smithi144.stdout:3/821: fsync d0/d1/d17/d39/d40/d4a/f8e 0 2024-05-15T15:54:24.992 INFO:tasks.workunit.client.1.smithi144.stdout:4/851: getdents d6/dd/d34/d116 0 2024-05-15T15:54:24.994 INFO:tasks.workunit.client.1.smithi144.stdout:1/810: fdatasync d7/d1d/d33/d41/d78/dc3/fce 0 2024-05-15T15:54:24.994 INFO:tasks.workunit.client.1.smithi144.stdout:9/846: rmdir d0/d8/d5e/d44/dc2 39 2024-05-15T15:54:24.994 INFO:tasks.workunit.client.1.smithi144.stdout:4/852: dread d6/dd/d2e/d75/dff/ff9 [0,4194304] 0 2024-05-15T15:54:24.995 INFO:tasks.workunit.client.1.smithi144.stdout:4/853: read d6/dd/d2e/d94/f106 [2789731,109059] 0 2024-05-15T15:54:24.996 INFO:tasks.workunit.client.1.smithi144.stdout:9/847: write d0/d8/d5e/d44/d99/ff6 [725859,11740] 0 2024-05-15T15:54:24.997 INFO:tasks.workunit.client.1.smithi144.stdout:3/822: fdatasync d0/d11/d3b/f96 0 2024-05-15T15:54:24.998 INFO:tasks.workunit.client.1.smithi144.stdout:8/898: creat d2/f126 x:0 0 0 2024-05-15T15:54:24.999 INFO:tasks.workunit.client.1.smithi144.stdout:6/854: creat d7/db/d5f/d12c/f12f x:0 0 0 2024-05-15T15:54:25.002 INFO:tasks.workunit.client.0.smithi049.stdout:5/855: rmdir dc/d4a/d42/dff 39 2024-05-15T15:54:25.003 INFO:tasks.workunit.client.0.smithi049.stdout:0/749: mknod dc/d13/d14/d25/d31/dae/dc5/d57/cfa 0 2024-05-15T15:54:25.004 INFO:tasks.workunit.client.0.smithi049.stdout:0/750: chown dc/d6d 1701893076 1 2024-05-15T15:54:25.005 INFO:tasks.workunit.client.0.smithi049.stdout:8/727: rmdir d2/de/d71/d86/d5d/d7d/db8/d5b/d60/da5 39 2024-05-15T15:54:25.007 INFO:tasks.workunit.client.0.smithi049.stdout:5/856: rmdir dc/db0 39 2024-05-15T15:54:25.008 INFO:tasks.workunit.client.0.smithi049.stdout:0/751: creat dc/d6d/dc3/ffb x:0 0 0 2024-05-15T15:54:25.011 INFO:tasks.workunit.client.0.smithi049.stdout:3/705: sync 2024-05-15T15:54:25.012 INFO:tasks.workunit.client.1.smithi144.stdout:1/811: mknod d7/d1d/d33/d29/dcd/dff/c104 0 2024-05-15T15:54:25.015 INFO:tasks.workunit.client.0.smithi049.stdout:5/857: mknod dc/db0/dd5/d102/c11c 0 2024-05-15T15:54:25.015 INFO:tasks.workunit.client.0.smithi049.stdout:8/728: dread d2/de/d71/d86/f51 [0,4194304] 0 2024-05-15T15:54:25.017 INFO:tasks.workunit.client.1.smithi144.stdout:9/848: unlink d0/d8/d5e/d44/dc2/f61 0 2024-05-15T15:54:25.019 INFO:tasks.workunit.client.0.smithi049.stdout:0/752: link dc/d13/d2b/fac dc/d13/d3d/d84/ffc 0 2024-05-15T15:54:25.019 INFO:tasks.workunit.client.1.smithi144.stdout:2/774: write d3/d7/d20/d22/d52/d64/f31 [1657567,72158] 0 2024-05-15T15:54:25.020 INFO:tasks.workunit.client.1.smithi144.stdout:5/859: getdents d0/dc/d24/d8e/d93/d9e 0 2024-05-15T15:54:25.024 INFO:tasks.workunit.client.1.smithi144.stdout:8/899: creat d2/da8/d119/f127 x:0 0 0 2024-05-15T15:54:25.025 INFO:tasks.workunit.client.1.smithi144.stdout:7/828: truncate d4/d3c/d76/d9d/dac/de5/fea 94152 0 2024-05-15T15:54:25.025 INFO:tasks.workunit.client.0.smithi049.stdout:5/858: truncate dc/dd/d47/f9c 1047086 0 2024-05-15T15:54:25.028 INFO:tasks.workunit.client.1.smithi144.stdout:4/854: dread d6/dd/d34/d3b/d30/db/d1d/f9f [0,4194304] 0 2024-05-15T15:54:25.030 INFO:tasks.workunit.client.1.smithi144.stdout:4/855: stat d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/dab 0 2024-05-15T15:54:25.030 INFO:tasks.workunit.client.0.smithi049.stdout:0/753: mkdir dc/d13/d14/d25/d31/dfd 0 2024-05-15T15:54:25.032 INFO:tasks.workunit.client.0.smithi049.stdout:5/859: dwrite dc/dd/d7b/fdf [0,4194304] 0 2024-05-15T15:54:25.032 INFO:tasks.workunit.client.1.smithi144.stdout:9/849: creat d0/d8/d1b/d62/d7d/dd2/f121 x:0 0 0 2024-05-15T15:54:25.033 INFO:tasks.workunit.client.0.smithi049.stdout:8/729: mkdir d2/de/d71/d86/de0 0 2024-05-15T15:54:25.036 INFO:tasks.workunit.client.1.smithi144.stdout:4/856: dwrite d6/dd/d2e/d75/dff/deb/f48 [0,4194304] 0 2024-05-15T15:54:25.042 INFO:tasks.workunit.client.1.smithi144.stdout:7/829: creat d4/d41/de8/f122 x:0 0 0 2024-05-15T15:54:25.043 INFO:tasks.workunit.client.1.smithi144.stdout:0/793: dwrite d4/d6/fcb [0,4194304] 0 2024-05-15T15:54:25.043 INFO:tasks.workunit.client.0.smithi049.stdout:5/860: dread dc/d4a/ddd/dfb/fee [0,4194304] 0 2024-05-15T15:54:25.049 INFO:tasks.workunit.client.1.smithi144.stdout:9/850: truncate d0/d8/d1b/d62/d6b/fa7 1868571 0 2024-05-15T15:54:25.050 INFO:tasks.workunit.client.0.smithi049.stdout:8/730: unlink d2/de/d71/d86/d5d/d7d/db8/d5b/d60/da5/fa9 0 2024-05-15T15:54:25.050 INFO:tasks.workunit.client.1.smithi144.stdout:4/857: dwrite d6/dd/fbb [0,4194304] 0 2024-05-15T15:54:25.051 INFO:tasks.workunit.client.0.smithi049.stdout:8/731: write d2/de/d71/d86/d5d/d7d/d45/f7c [5079611,114731] 0 2024-05-15T15:54:25.052 INFO:tasks.workunit.client.0.smithi049.stdout:5/861: dread - dc/fb7 zero size 2024-05-15T15:54:25.055 INFO:tasks.workunit.client.1.smithi144.stdout:8/900: creat d2/d24/d2d/da0/df8/d10a/f128 x:0 0 0 2024-05-15T15:54:25.061 INFO:tasks.workunit.client.0.smithi049.stdout:5/862: rename dc/db0/df2 to dc/dd/d10/d11d 0 2024-05-15T15:54:25.063 INFO:tasks.workunit.client.1.smithi144.stdout:6/855: getdents d7/d32 0 2024-05-15T15:54:25.064 INFO:tasks.workunit.client.1.smithi144.stdout:7/830: creat d4/df/d25/d38/d5d/f123 x:0 0 0 2024-05-15T15:54:25.066 INFO:tasks.workunit.client.0.smithi049.stdout:8/732: rename d2/de/d71/d86/d5d/d7d/db8/f40 to d2/de/d25/d8c/fe1 0 2024-05-15T15:54:25.068 INFO:tasks.workunit.client.1.smithi144.stdout:0/794: creat d4/d6/d3a/d4c/d11f/f124 x:0 0 0 2024-05-15T15:54:25.068 INFO:tasks.workunit.client.0.smithi049.stdout:5/863: mknod dc/db0/dd5/dd8/c11e 0 2024-05-15T15:54:25.072 INFO:tasks.workunit.client.1.smithi144.stdout:5/860: creat d0/f11d x:0 0 0 2024-05-15T15:54:25.072 INFO:tasks.workunit.client.0.smithi049.stdout:5/864: dread f0 [0,4194304] 0 2024-05-15T15:54:25.074 INFO:tasks.workunit.client.1.smithi144.stdout:4/858: truncate d6/dd/d2e/d75/fb5 3245684 0 2024-05-15T15:54:25.075 INFO:tasks.workunit.client.1.smithi144.stdout:8/901: mkdir d2/d3a/da5/d129 0 2024-05-15T15:54:25.076 INFO:tasks.workunit.client.1.smithi144.stdout:8/902: write d2/d24/d2d/da0/df8/d10a/f128 [976749,39979] 0 2024-05-15T15:54:25.077 INFO:tasks.workunit.client.1.smithi144.stdout:8/903: chown d2/d24/d2d/da0/d9e/db8/dde/f11e 2046 1 2024-05-15T15:54:25.077 INFO:tasks.workunit.client.1.smithi144.stdout:6/856: rename d7/c7f to d7/db/da2/c130 0 2024-05-15T15:54:25.078 INFO:tasks.workunit.client.1.smithi144.stdout:8/904: stat d2/d3a/fce 0 2024-05-15T15:54:25.078 INFO:tasks.workunit.client.1.smithi144.stdout:6/857: stat d7/db/d11/d3c/d61/dcc/c10d 0 2024-05-15T15:54:25.078 INFO:tasks.workunit.client.1.smithi144.stdout:8/905: stat d2/d24/d2d/da0/d6e/d8d/f11f 0 2024-05-15T15:54:25.080 INFO:tasks.workunit.client.1.smithi144.stdout:0/795: fdatasync d4/d6/d3a/d4c/f7a 0 2024-05-15T15:54:25.083 INFO:tasks.workunit.client.1.smithi144.stdout:5/861: mknod d0/d7e/d9b/c11e 0 2024-05-15T15:54:25.085 INFO:tasks.workunit.client.1.smithi144.stdout:7/831: symlink d4/df/d81/dfb/d108/l124 0 2024-05-15T15:54:25.088 INFO:tasks.workunit.client.1.smithi144.stdout:3/823: write d0/d11/d3b/f64 [1990342,91733] 0 2024-05-15T15:54:25.096 INFO:tasks.workunit.client.1.smithi144.stdout:3/824: creat d0/d11/d3b/da0/f10d x:0 0 0 2024-05-15T15:54:25.096 INFO:tasks.workunit.client.0.smithi049.stdout:3/706: write d12/d26/f40 [3400046,98537] 0 2024-05-15T15:54:25.098 INFO:tasks.workunit.client.1.smithi144.stdout:6/858: rename d7/db/d11/d3c/ff5 to d7/db/d35/f131 0 2024-05-15T15:54:25.101 INFO:tasks.workunit.client.0.smithi049.stdout:3/707: mkdir d12/d14/d52/db9/dc4/d68/d70/dda 0 2024-05-15T15:54:25.104 INFO:tasks.workunit.client.1.smithi144.stdout:5/862: dread d0/dc/d12/f18 [0,4194304] 0 2024-05-15T15:54:25.104 INFO:tasks.workunit.client.1.smithi144.stdout:6/859: dwrite d7/d1e/d28/f3d [4194304,4194304] 0 2024-05-15T15:54:25.106 INFO:tasks.workunit.client.0.smithi049.stdout:3/708: dread - d12/d14/d52/db9/dc4/d68/d70/f8a zero size 2024-05-15T15:54:25.107 INFO:tasks.workunit.client.0.smithi049.stdout:3/709: dread - d12/d14/d52/db9/dc4/d68/d70/f8a zero size 2024-05-15T15:54:25.109 INFO:tasks.workunit.client.0.smithi049.stdout:3/710: stat d12/d14/f17 0 2024-05-15T15:54:25.110 INFO:tasks.workunit.client.0.smithi049.stdout:3/711: chown d12/d26 903 1 2024-05-15T15:54:25.110 INFO:tasks.workunit.client.1.smithi144.stdout:1/812: dwrite d7/d1d/d33/d41/f8c [0,4194304] 0 2024-05-15T15:54:25.114 INFO:tasks.workunit.client.0.smithi049.stdout:0/754: dwrite dc/d32/f7b [0,4194304] 0 2024-05-15T15:54:25.115 INFO:tasks.workunit.client.1.smithi144.stdout:8/906: getdents d2/d24/d2d/da0/d6e/deb 0 2024-05-15T15:54:25.117 INFO:tasks.workunit.client.1.smithi144.stdout:0/796: getdents d4 0 2024-05-15T15:54:25.126 INFO:tasks.workunit.client.1.smithi144.stdout:6/860: truncate d7/db/d11/f1f 9946753 0 2024-05-15T15:54:25.128 INFO:tasks.workunit.client.1.smithi144.stdout:1/813: unlink d7/d1d/d33/d1e/d34/d9c/fcf 0 2024-05-15T15:54:25.129 INFO:tasks.workunit.client.0.smithi049.stdout:5/865: rmdir dc/dd/d10/d11d 39 2024-05-15T15:54:25.133 INFO:tasks.workunit.client.1.smithi144.stdout:6/861: dread d7/db/d35/f67 [0,4194304] 0 2024-05-15T15:54:25.133 INFO:tasks.workunit.client.0.smithi049.stdout:5/866: chown dc/d4a/d42/c9d 1 1 2024-05-15T15:54:25.133 INFO:tasks.workunit.client.1.smithi144.stdout:3/825: link d0/d11/d3b/da7/fa9 d0/d11/d3b/da7/def/f10e 0 2024-05-15T15:54:25.135 INFO:tasks.workunit.client.1.smithi144.stdout:2/775: sync 2024-05-15T15:54:25.135 INFO:tasks.workunit.client.0.smithi049.stdout:8/733: write d2/de/d71/d86/f96 [4162996,91983] 0 2024-05-15T15:54:25.138 INFO:tasks.workunit.client.1.smithi144.stdout:1/814: rename d7/d1d/d33/d41/d78/da3/deb to d7/d1d/d58/d81/d105 0 2024-05-15T15:54:25.139 INFO:tasks.workunit.client.0.smithi049.stdout:5/867: truncate dc/d4a/d42/f60 8686 0 2024-05-15T15:54:25.141 INFO:tasks.workunit.client.0.smithi049.stdout:8/734: dread d2/de/d71/d86/d5d/d7d/d1b/d69/fde [0,4194304] 0 2024-05-15T15:54:25.143 INFO:tasks.workunit.client.1.smithi144.stdout:6/862: fsync d7/f41 0 2024-05-15T15:54:25.145 INFO:tasks.workunit.client.1.smithi144.stdout:1/815: dwrite d7/d1d/d33/d1e/d4e/fbb [4194304,4194304] 0 2024-05-15T15:54:25.145 INFO:tasks.workunit.client.0.smithi049.stdout:5/868: creat dc/dd/d47/d83/dcc/d5c/f11f x:0 0 0 2024-05-15T15:54:25.146 INFO:tasks.workunit.client.1.smithi144.stdout:1/816: chown d7/d1d/d58/d81 2629607 1 2024-05-15T15:54:25.146 INFO:tasks.workunit.client.0.smithi049.stdout:5/869: stat dc/d29/f87 0 2024-05-15T15:54:25.146 INFO:tasks.workunit.client.0.smithi049.stdout:3/712: dread d12/d14/d52/db9/dc4/fbd [0,4194304] 0 2024-05-15T15:54:25.148 INFO:tasks.workunit.client.0.smithi049.stdout:0/755: dread dc/d13/d3d/f5d [0,4194304] 0 2024-05-15T15:54:25.148 INFO:tasks.workunit.client.0.smithi049.stdout:8/735: symlink d2/de/d71/d86/d5d/d7d/d1b/d69/le2 0 2024-05-15T15:54:25.151 INFO:tasks.workunit.client.1.smithi144.stdout:5/863: getdents d0/dc/d24/d28/d2d/df7 0 2024-05-15T15:54:25.152 INFO:tasks.workunit.client.1.smithi144.stdout:7/832: write d4/df/d25/d38/d68/fe0 [1160390,39328] 0 2024-05-15T15:54:25.153 INFO:tasks.workunit.client.1.smithi144.stdout:5/864: write d0/dc/d24/d28/f51 [4152525,129107] 0 2024-05-15T15:54:25.153 INFO:tasks.workunit.client.1.smithi144.stdout:9/851: dwrite d0/d8/d5e/d44/d4a/f111 [0,4194304] 0 2024-05-15T15:54:25.160 INFO:tasks.workunit.client.1.smithi144.stdout:6/863: rename d7/db/d35/d84/dbe/d10a to d7/db/d11/d7e/d132 0 2024-05-15T15:54:25.162 INFO:tasks.workunit.client.0.smithi049.stdout:3/713: dread - d12/d14/d32/f58 zero size 2024-05-15T15:54:25.162 INFO:tasks.workunit.client.1.smithi144.stdout:7/833: dwrite d4/d3c/d76/d9d/dac/d48/d72/da2/fbe [0,4194304] 0 2024-05-15T15:54:25.164 INFO:tasks.workunit.client.0.smithi049.stdout:8/736: mkdir d2/de/d71/d86/d5d/de3 0 2024-05-15T15:54:25.166 INFO:tasks.workunit.client.1.smithi144.stdout:2/776: symlink d3/lfe 0 2024-05-15T15:54:25.168 INFO:tasks.workunit.client.0.smithi049.stdout:0/756: symlink dc/d13/d14/d2f/d8f/dcb/de2/lfe 0 2024-05-15T15:54:25.172 INFO:tasks.workunit.client.0.smithi049.stdout:5/870: creat dc/dd/d47/f120 x:0 0 0 2024-05-15T15:54:25.178 INFO:tasks.workunit.client.0.smithi049.stdout:8/737: creat d2/de/d25/d74/dbe/fe4 x:0 0 0 2024-05-15T15:54:25.179 INFO:tasks.workunit.client.1.smithi144.stdout:0/797: write d4/d6/d3a/d40/f7d [3272741,87072] 0 2024-05-15T15:54:25.179 INFO:tasks.workunit.client.0.smithi049.stdout:8/738: dread - d2/de/d25/d74/dbe/fe4 zero size 2024-05-15T15:54:25.182 INFO:tasks.workunit.client.0.smithi049.stdout:0/757: rename dc/fda to dc/d13/d2b/d76/d8a/da2/fff 0 2024-05-15T15:54:25.186 INFO:tasks.workunit.client.1.smithi144.stdout:6/864: symlink d7/db/d11/d3c/dfc/l133 0 2024-05-15T15:54:25.187 INFO:tasks.workunit.client.0.smithi049.stdout:8/739: rmdir d2/de/d71/d86 39 2024-05-15T15:54:25.187 INFO:tasks.workunit.client.0.smithi049.stdout:8/740: readlink d2/de/l31 0 2024-05-15T15:54:25.189 INFO:tasks.workunit.client.1.smithi144.stdout:4/859: dwrite d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/d10e/fe4 [0,4194304] 0 2024-05-15T15:54:25.189 INFO:tasks.workunit.client.1.smithi144.stdout:1/817: mknod d7/d1d/d33/d29/dd2/c106 0 2024-05-15T15:54:25.189 INFO:tasks.workunit.client.0.smithi049.stdout:7/719: sync 2024-05-15T15:54:25.189 INFO:tasks.workunit.client.0.smithi049.stdout:1/897: sync 2024-05-15T15:54:25.189 INFO:tasks.workunit.client.0.smithi049.stdout:9/632: sync 2024-05-15T15:54:25.190 INFO:tasks.workunit.client.0.smithi049.stdout:0/758: fsync dc/d13/d14/d2f/d8f/fa7 0 2024-05-15T15:54:25.190 INFO:tasks.workunit.client.0.smithi049.stdout:9/633: stat d0/d8/d1b/d40/f49 0 2024-05-15T15:54:25.192 INFO:tasks.workunit.client.0.smithi049.stdout:9/634: readlink d0/d8e/la6 0 2024-05-15T15:54:25.195 INFO:tasks.workunit.client.1.smithi144.stdout:2/777: creat d3/d7/d20/d22/d52/d64/d97/db8/fff x:0 0 0 2024-05-15T15:54:25.195 INFO:tasks.workunit.client.1.smithi144.stdout:0/798: dread d4/dc/f18 [0,4194304] 0 2024-05-15T15:54:25.197 INFO:tasks.workunit.client.0.smithi049.stdout:3/714: dread d12/d30/f38 [0,4194304] 0 2024-05-15T15:54:25.198 INFO:tasks.workunit.client.0.smithi049.stdout:5/871: dread dc/d4a/f3a [0,4194304] 0 2024-05-15T15:54:25.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:25 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:54:25.199 INFO:tasks.workunit.client.1.smithi144.stdout:8/907: sync 2024-05-15T15:54:25.199 INFO:tasks.workunit.client.0.smithi049.stdout:3/715: write d12/d14/d52/fce [1026236,76894] 0 2024-05-15T15:54:25.200 INFO:tasks.workunit.client.1.smithi144.stdout:3/826: dwrite d0/d1/d6c/fdd [0,4194304] 0 2024-05-15T15:54:25.201 INFO:tasks.workunit.client.1.smithi144.stdout:5/865: mknod d0/dc/d24/d8e/c11f 0 2024-05-15T15:54:25.204 INFO:tasks.workunit.client.0.smithi049.stdout:7/720: mkdir d0/d6/d23/d4a/d7b/dc4/de0 0 2024-05-15T15:54:25.207 INFO:tasks.workunit.client.0.smithi049.stdout:7/721: write d0/d6/da/fc0 [208286,53395] 0 2024-05-15T15:54:25.207 INFO:tasks.workunit.client.1.smithi144.stdout:9/852: read d0/d8/d5e/d44/d10/d9c/f65 [3815195,27070] 0 2024-05-15T15:54:25.207 INFO:tasks.workunit.client.1.smithi144.stdout:3/827: dwrite d0/d1/d17/d39/d83/f10b [0,4194304] 0 2024-05-15T15:54:25.211 INFO:tasks.workunit.client.0.smithi049.stdout:5/872: dwrite dc/dd/d39/fa0 [0,4194304] 0 2024-05-15T15:54:25.211 INFO:tasks.workunit.client.1.smithi144.stdout:4/860: readlink d6/dd/d34/d3b/d30/db/l8f 0 2024-05-15T15:54:25.222 INFO:tasks.workunit.client.1.smithi144.stdout:2/778: dread d3/d7/d20/d89/d86/ff9 [0,4194304] 0 2024-05-15T15:54:25.224 INFO:tasks.workunit.client.0.smithi049.stdout:1/898: creat d0/d4/dd/de/d1e/d16/d8a/dd2/d110/d139/f13b x:0 0 0 2024-05-15T15:54:25.226 INFO:tasks.workunit.client.0.smithi049.stdout:3/716: dwrite d12/d14/d32/d64/d88/fb5 [0,4194304] 0 2024-05-15T15:54:25.228 INFO:tasks.workunit.client.0.smithi049.stdout:8/741: mknod d2/de/d71/d86/de0/ce5 0 2024-05-15T15:54:25.233 INFO:tasks.workunit.client.0.smithi049.stdout:9/635: truncate d0/d8/f61 852666 0 2024-05-15T15:54:25.234 INFO:tasks.workunit.client.0.smithi049.stdout:9/636: fdatasync d0/d8/d42/f91 0 2024-05-15T15:54:25.238 INFO:tasks.workunit.client.0.smithi049.stdout:3/717: mknod d12/d14/d52/cdb 0 2024-05-15T15:54:25.244 INFO:tasks.workunit.client.0.smithi049.stdout:8/742: dwrite d2/de/d95/fd1 [0,4194304] 0 2024-05-15T15:54:25.245 INFO:tasks.workunit.client.0.smithi049.stdout:7/722: creat d0/d6/da/fe1 x:0 0 0 2024-05-15T15:54:25.245 INFO:tasks.workunit.client.0.smithi049.stdout:1/899: dread d0/d4/d63/f83 [0,4194304] 0 2024-05-15T15:54:25.246 INFO:tasks.workunit.client.0.smithi049.stdout:3/718: mkdir d12/d30/dc1/dc7/ddc 0 2024-05-15T15:54:25.247 INFO:tasks.workunit.client.0.smithi049.stdout:7/723: readlink d0/d6/d23/d45/d6a/da1/d80/la0 0 2024-05-15T15:54:25.247 INFO:tasks.workunit.client.0.smithi049.stdout:1/900: write d0/d4/dd/de/d4d/d8b/de8/f104 [1283915,106395] 0 2024-05-15T15:54:25.251 INFO:tasks.workunit.client.0.smithi049.stdout:9/637: mkdir d0/d8/d9/d52/d5e/d5f/d88/d9d/de0 0 2024-05-15T15:54:25.256 INFO:tasks.workunit.client.0.smithi049.stdout:5/873: getdents dc/dd/d47 0 2024-05-15T15:54:25.258 INFO:tasks.workunit.client.0.smithi049.stdout:7/724: mkdir d0/d6/d23/d4a/d7b/de2 0 2024-05-15T15:54:25.263 INFO:tasks.workunit.client.0.smithi049.stdout:1/901: dread d0/d4/dd/de/d4d/d8b/de8/f125 [0,4194304] 0 2024-05-15T15:54:25.269 INFO:tasks.workunit.client.0.smithi049.stdout:7/725: symlink d0/ddb/le3 0 2024-05-15T15:54:25.276 INFO:tasks.workunit.client.1.smithi144.stdout:3/828: rmdir d0/d11/d3b/dc7 39 2024-05-15T15:54:25.279 INFO:tasks.workunit.client.1.smithi144.stdout:9/853: unlink d0/d8/d5e/d44/d16/c39 0 2024-05-15T15:54:25.284 INFO:tasks.workunit.client.0.smithi049.stdout:7/726: dread d0/d6/d1d/f4d [0,4194304] 0 2024-05-15T15:54:25.285 INFO:tasks.workunit.client.1.smithi144.stdout:4/861: mknod d6/dd/d34/d3b/d30/db/d1d/d4e/c119 0 2024-05-15T15:54:25.286 INFO:tasks.workunit.client.0.smithi049.stdout:3/719: write d12/d14/d32/f58 [819975,80499] 0 2024-05-15T15:54:25.286 INFO:tasks.workunit.client.0.smithi049.stdout:8/743: link d2/de/d25/c38 d2/de/d71/d86/d5d/d7d/db8/d5b/dd0/ce6 0 2024-05-15T15:54:25.287 INFO:tasks.workunit.client.0.smithi049.stdout:3/720: chown d12/d14/d52/f7b 33557 1 2024-05-15T15:54:25.287 INFO:tasks.workunit.client.0.smithi049.stdout:8/744: chown d2/f2c 0 1 2024-05-15T15:54:25.289 INFO:tasks.workunit.client.1.smithi144.stdout:2/779: symlink d3/d7/d20/d22/d52/d64/d97/l100 0 2024-05-15T15:54:25.289 INFO:tasks.workunit.client.1.smithi144.stdout:7/834: write d4/d3c/d76/d9d/dac/d48/d72/fec [828895,126373] 0 2024-05-15T15:54:25.289 INFO:tasks.workunit.client.0.smithi049.stdout:7/727: truncate d0/f11 3932196 0 2024-05-15T15:54:25.290 INFO:tasks.workunit.client.0.smithi049.stdout:5/874: write dc/dd/d39/f6e [1447256,100329] 0 2024-05-15T15:54:25.290 INFO:tasks.workunit.client.1.smithi144.stdout:4/862: dwrite d6/dd/d34/d3b/d30/d59/d66/fb8 [0,4194304] 0 2024-05-15T15:54:25.292 INFO:tasks.workunit.client.1.smithi144.stdout:4/863: chown d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d44/fbf 162383363 1 2024-05-15T15:54:25.294 INFO:tasks.workunit.client.0.smithi049.stdout:8/745: mkdir d2/de/d71/d86/d5d/d7d/db8/d5b/da6/de7 0 2024-05-15T15:54:25.295 INFO:tasks.workunit.client.0.smithi049.stdout:9/638: truncate d0/d8/d9/f7f 24226 0 2024-05-15T15:54:25.295 INFO:tasks.workunit.client.0.smithi049.stdout:1/902: write d0/d4/dd/de/d1e/d16/d8a/f9b [776666,7112] 0 2024-05-15T15:54:25.297 INFO:tasks.workunit.client.0.smithi049.stdout:5/875: fdatasync dc/f9a 0 2024-05-15T15:54:25.298 INFO:tasks.workunit.client.1.smithi144.stdout:8/908: mknod d2/de/d3f/dff/d125/c12a 0 2024-05-15T15:54:25.299 INFO:tasks.workunit.client.0.smithi049.stdout:1/903: chown d0/d4/dd/de/d1e/d2e/dad/d65/leb 70 1 2024-05-15T15:54:25.300 INFO:tasks.workunit.client.0.smithi049.stdout:7/728: fsync d0/d6/d23/d45/d6a/da1/f33 0 2024-05-15T15:54:25.300 INFO:tasks.workunit.client.0.smithi049.stdout:5/876: dread - dc/dd/d10/db3/dd6/ffc zero size 2024-05-15T15:54:25.300 INFO:tasks.workunit.client.1.smithi144.stdout:5/866: mknod d0/d10c/c120 0 2024-05-15T15:54:25.301 INFO:tasks.workunit.client.0.smithi049.stdout:5/877: fsync dc/d4a/f57 0 2024-05-15T15:54:25.302 INFO:tasks.workunit.client.1.smithi144.stdout:6/865: creat d7/db/f134 x:0 0 0 2024-05-15T15:54:25.302 INFO:tasks.workunit.client.0.smithi049.stdout:5/878: dread - dc/dd/d47/d83/dcc/f115 zero size 2024-05-15T15:54:25.304 INFO:tasks.workunit.client.1.smithi144.stdout:3/829: rmdir d0/d1/d17/dae 39 2024-05-15T15:54:25.307 INFO:tasks.workunit.client.1.smithi144.stdout:1/818: rename d7/d1d/d58/cd6 to d7/d4b/c107 0 2024-05-15T15:54:25.311 INFO:tasks.workunit.client.1.smithi144.stdout:2/780: creat d3/d7/d20/d37/de1/f101 x:0 0 0 2024-05-15T15:54:25.314 INFO:tasks.workunit.client.1.smithi144.stdout:7/835: mkdir d4/d3c/d76/d9d/dac/d48/d125 0 2024-05-15T15:54:25.315 INFO:tasks.workunit.client.1.smithi144.stdout:7/836: fdatasync d4/df/d25/d38/d5d/ffa 0 2024-05-15T15:54:25.315 INFO:tasks.workunit.client.0.smithi049.stdout:3/721: dread d12/d14/d32/d64/f93 [0,4194304] 0 2024-05-15T15:54:25.317 INFO:tasks.workunit.client.0.smithi049.stdout:7/729: symlink d0/ddb/dde/dcc/le4 0 2024-05-15T15:54:25.320 INFO:tasks.workunit.client.1.smithi144.stdout:0/799: rmdir d4/d6/dd7 0 2024-05-15T15:54:25.322 INFO:tasks.workunit.client.0.smithi049.stdout:9/639: write d0/d8/d9/d52/d5e/d5f/d56/fa7 [123067,108846] 0 2024-05-15T15:54:25.323 INFO:tasks.workunit.client.1.smithi144.stdout:5/867: fsync d0/dc/d24/d8e/d93/d6b/d88/fce 0 2024-05-15T15:54:25.325 INFO:tasks.workunit.client.1.smithi144.stdout:6/866: symlink d7/db/d3a/dd2/l135 0 2024-05-15T15:54:25.326 INFO:tasks.workunit.client.0.smithi049.stdout:3/722: readlink d12/d14/d32/l76 0 2024-05-15T15:54:25.327 INFO:tasks.workunit.client.0.smithi049.stdout:3/723: truncate d12/d14/d52/db9/d6b/fd7 1047189 0 2024-05-15T15:54:25.328 INFO:tasks.workunit.client.0.smithi049.stdout:7/730: dread - d0/ddb/dde/dca/d5e/fb6 zero size 2024-05-15T15:54:25.330 INFO:tasks.workunit.client.0.smithi049.stdout:7/731: read d0/d34/f35 [7594810,2585] 0 2024-05-15T15:54:25.332 INFO:tasks.workunit.client.1.smithi144.stdout:8/909: dread d2/de/d49/f76 [0,4194304] 0 2024-05-15T15:54:25.333 INFO:tasks.workunit.client.0.smithi049.stdout:8/746: getdents d2/de/d71/d86/d5d/d7d 0 2024-05-15T15:54:25.335 INFO:tasks.workunit.client.1.smithi144.stdout:2/781: mkdir d3/d7/d20/d89/d86/d102 0 2024-05-15T15:54:25.336 INFO:tasks.workunit.client.0.smithi049.stdout:9/640: symlink d0/d8/d9/d52/d5e/d8a/dd3/d87/le1 0 2024-05-15T15:54:25.336 INFO:tasks.workunit.client.1.smithi144.stdout:4/864: write d6/dd/d34/d3b/d30/f1b [2618193,12969] 0 2024-05-15T15:54:25.337 INFO:tasks.workunit.client.1.smithi144.stdout:7/837: chown d4/df/c19 27 1 2024-05-15T15:54:25.338 INFO:tasks.workunit.client.0.smithi049.stdout:1/904: creat d0/d4/dd/de/d1e/f13c x:0 0 0 2024-05-15T15:54:25.340 INFO:tasks.workunit.client.0.smithi049.stdout:1/905: fdatasync d0/d4/dd/de/d1e/d16/d8a/dd2/d110/d139/f13b 0 2024-05-15T15:54:25.340 INFO:tasks.workunit.client.0.smithi049.stdout:3/724: mkdir d12/d67/da8/ddd 0 2024-05-15T15:54:25.341 INFO:tasks.workunit.client.1.smithi144.stdout:1/819: dread d7/d1d/d33/d41/d78/fb1 [0,4194304] 0 2024-05-15T15:54:25.341 INFO:tasks.workunit.client.1.smithi144.stdout:1/820: write d7/d1d/d33/d1e/f25 [708161,70399] 0 2024-05-15T15:54:25.342 INFO:tasks.workunit.client.1.smithi144.stdout:3/830: write d0/d1/d17/f91 [4986542,88912] 0 2024-05-15T15:54:25.347 INFO:tasks.workunit.client.1.smithi144.stdout:1/821: dwrite d7/d1d/fa5 [0,4194304] 0 2024-05-15T15:54:25.372 INFO:tasks.workunit.client.0.smithi049.stdout:4/720: sync 2024-05-15T15:54:25.372 INFO:tasks.workunit.client.0.smithi049.stdout:6/791: sync 2024-05-15T15:54:25.373 INFO:tasks.workunit.client.0.smithi049.stdout:2/716: sync 2024-05-15T15:54:25.433 INFO:tasks.workunit.client.0.smithi049.stdout:5/879: link dc/db0/dd5/dd8/l110 dc/dd/l121 0 2024-05-15T15:54:25.435 INFO:tasks.workunit.client.0.smithi049.stdout:7/732: rmdir d0/d6/d23/d45/d86 39 2024-05-15T15:54:25.440 INFO:tasks.workunit.client.1.smithi144.stdout:9/854: creat d0/d8/d5e/d44/f122 x:0 0 0 2024-05-15T15:54:25.443 INFO:tasks.workunit.client.1.smithi144.stdout:8/910: unlink d2/d24/d2d/da0/d6e/d8d/f11f 0 2024-05-15T15:54:25.444 INFO:tasks.workunit.client.1.smithi144.stdout:6/867: write d7/db/d9d/d97/f127 [236378,19845] 0 2024-05-15T15:54:25.444 INFO:tasks.workunit.client.0.smithi049.stdout:1/906: symlink d0/d4/dd/de/d4d/db8/l13d 0 2024-05-15T15:54:25.448 INFO:tasks.workunit.client.1.smithi144.stdout:7/838: rmdir d4/d3c/d76/d9d/dba 39 2024-05-15T15:54:25.450 INFO:tasks.workunit.client.0.smithi049.stdout:4/721: fdatasync d18/f80 0 2024-05-15T15:54:25.451 INFO:tasks.workunit.client.0.smithi049.stdout:4/722: readlink d18/d1e/l7b 0 2024-05-15T15:54:25.453 INFO:tasks.workunit.client.0.smithi049.stdout:3/725: dwrite d12/d26/f3b [0,4194304] 0 2024-05-15T15:54:25.454 INFO:tasks.workunit.client.1.smithi144.stdout:4/865: dwrite d6/dd/d34/d3b/f5d [0,4194304] 0 2024-05-15T15:54:25.454 INFO:tasks.workunit.client.0.smithi049.stdout:6/792: mknod dd/d17/d9d/cf1 0 2024-05-15T15:54:25.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:25 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:54:25.459 INFO:tasks.workunit.client.0.smithi049.stdout:7/733: truncate d0/d2f/f97 1555233 0 2024-05-15T15:54:25.462 INFO:tasks.workunit.client.1.smithi144.stdout:1/822: creat d7/d1d/d33/d55/f108 x:0 0 0 2024-05-15T15:54:25.462 INFO:tasks.workunit.client.0.smithi049.stdout:1/907: rmdir d0/d4/dd/de/d1e/d2e/dad/d65/d72/dae 39 2024-05-15T15:54:25.463 INFO:tasks.workunit.client.0.smithi049.stdout:2/717: dwrite d3/da/fb4 [0,4194304] 0 2024-05-15T15:54:25.466 INFO:tasks.workunit.client.1.smithi144.stdout:8/911: dread - d2/d24/d2d/da0/d9e/db8/fe7 zero size 2024-05-15T15:54:25.468 INFO:tasks.workunit.client.0.smithi049.stdout:6/793: mkdir dd/d17/d24/d2b/d36/d50/d54/dde/df2 0 2024-05-15T15:54:25.470 INFO:tasks.workunit.client.1.smithi144.stdout:7/839: creat d4/d3c/d76/d9d/dac/d48/f126 x:0 0 0 2024-05-15T15:54:25.471 INFO:tasks.workunit.client.1.smithi144.stdout:0/800: rename d4/dc/f18 to d4/df/d1f/d24/d85/da1/f125 0 2024-05-15T15:54:25.472 INFO:tasks.workunit.client.1.smithi144.stdout:7/840: read - d4/d3c/d76/d9d/dac/d48/fd2 zero size 2024-05-15T15:54:25.473 INFO:tasks.workunit.client.1.smithi144.stdout:3/831: link d0/d11/d3b/da0/f10d d0/d1/d17/d39/d4d/dfd/f10f 0 2024-05-15T15:54:25.475 INFO:tasks.workunit.client.1.smithi144.stdout:5/868: creat d0/dc/d2f/d34/f121 x:0 0 0 2024-05-15T15:54:25.476 INFO:tasks.workunit.client.0.smithi049.stdout:5/880: write dc/f16 [7557572,112035] 0 2024-05-15T15:54:25.477 INFO:tasks.workunit.client.0.smithi049.stdout:1/908: symlink d0/d4/d63/l13e 0 2024-05-15T15:54:25.482 INFO:tasks.workunit.client.1.smithi144.stdout:4/866: fsync d6/dd/d34/d3b/d30/db/d1d/f41 0 2024-05-15T15:54:25.484 INFO:tasks.workunit.client.1.smithi144.stdout:2/782: rename d3/d7/d20/dcf to d3/d4d/d103 0 2024-05-15T15:54:25.485 INFO:tasks.workunit.client.0.smithi049.stdout:6/794: truncate dd/d17/f8d 3731219 0 2024-05-15T15:54:25.488 INFO:tasks.workunit.client.0.smithi049.stdout:8/747: getdents d2/de/d25/d74/d94 0 2024-05-15T15:54:25.488 INFO:tasks.workunit.client.1.smithi144.stdout:3/832: dread d0/d11/d24/d78/ff2 [0,4194304] 0 2024-05-15T15:54:25.489 INFO:tasks.workunit.client.0.smithi049.stdout:8/748: truncate d2/de/d95/fd7 723528 0 2024-05-15T15:54:25.491 INFO:tasks.workunit.client.1.smithi144.stdout:7/841: fdatasync d4/d3c/d76/d9d/dac/d48/d72/da2/daf/fce 0 2024-05-15T15:54:25.493 INFO:tasks.workunit.client.0.smithi049.stdout:5/881: creat dc/d29/f122 x:0 0 0 2024-05-15T15:54:25.493 INFO:tasks.workunit.client.0.smithi049.stdout:3/726: write d12/d14/d52/db9/dc4/fbd [3792557,96873] 0 2024-05-15T15:54:25.493 INFO:tasks.workunit.client.1.smithi144.stdout:5/869: truncate d0/dc/d24/d57/f77 2589551 0 2024-05-15T15:54:25.498 INFO:tasks.workunit.client.0.smithi049.stdout:7/734: creat d0/d6/d23/fe5 x:0 0 0 2024-05-15T15:54:25.501 INFO:tasks.workunit.client.1.smithi144.stdout:1/823: write d7/d1d/d33/d29/dd2/f7b [381422,10654] 0 2024-05-15T15:54:25.501 INFO:tasks.workunit.client.0.smithi049.stdout:5/882: symlink dc/dd/d10/df1/l123 0 2024-05-15T15:54:25.502 INFO:tasks.workunit.client.1.smithi144.stdout:1/824: readlink d7/d1d/d33/d29/l8a 0 2024-05-15T15:54:25.502 INFO:tasks.workunit.client.0.smithi049.stdout:5/883: chown dc/d29/c38 1807415287 1 2024-05-15T15:54:25.504 INFO:tasks.workunit.client.0.smithi049.stdout:4/723: truncate d18/f22 3559271 0 2024-05-15T15:54:25.517 INFO:tasks.workunit.client.0.smithi049.stdout:4/724: creat d18/d85/d98/ff0 x:0 0 0 2024-05-15T15:54:25.518 INFO:tasks.workunit.client.0.smithi049.stdout:7/735: dread d0/ddb/dde/dcc/fa3 [4194304,4194304] 0 2024-05-15T15:54:25.518 INFO:tasks.workunit.client.0.smithi049.stdout:4/725: write d18/d85/d98/ddd/fe4 [74482,86563] 0 2024-05-15T15:54:25.522 INFO:tasks.workunit.client.0.smithi049.stdout:8/749: creat d2/de/d71/d86/d5d/fe8 x:0 0 0 2024-05-15T15:54:25.523 INFO:tasks.workunit.client.1.smithi144.stdout:2/783: fdatasync d3/d7/d20/d3b/fd9 0 2024-05-15T15:54:25.523 INFO:tasks.workunit.client.1.smithi144.stdout:6/868: write d7/db/da2/fb2 [3949983,70892] 0 2024-05-15T15:54:25.523 INFO:tasks.workunit.client.0.smithi049.stdout:1/909: getdents d0/d4/dd/de/d1e/d16/d8a/dd2/df2 0 2024-05-15T15:54:25.525 INFO:tasks.workunit.client.0.smithi049.stdout:3/727: link d12/d67/c84 d12/d26/d73/cde 0 2024-05-15T15:54:25.526 INFO:tasks.workunit.client.1.smithi144.stdout:1/825: read d7/d1d/d33/d41/d78/da3/f88 [1939390,118190] 0 2024-05-15T15:54:25.528 INFO:tasks.workunit.client.0.smithi049.stdout:4/726: fdatasync d18/d1e/d63/d99/db9/fc7 0 2024-05-15T15:54:25.529 INFO:tasks.workunit.client.0.smithi049.stdout:6/795: dwrite dd/d17/d24/d2b/d36/f98 [0,4194304] 0 2024-05-15T15:54:25.531 INFO:tasks.workunit.client.1.smithi144.stdout:0/801: dwrite d4/d6/fb [0,4194304] 0 2024-05-15T15:54:25.533 INFO:tasks.workunit.client.1.smithi144.stdout:3/833: dwrite d0/d1/d56/f58 [0,4194304] 0 2024-05-15T15:54:25.534 INFO:tasks.workunit.client.1.smithi144.stdout:0/802: write d4/df/d14/fc0 [155746,86573] 0 2024-05-15T15:54:25.534 INFO:tasks.workunit.client.0.smithi049.stdout:5/884: link dc/dd/d47/d83/dcc/d5c/f7e dc/dd/d7b/f124 0 2024-05-15T15:54:25.535 INFO:tasks.workunit.client.1.smithi144.stdout:2/784: rmdir d3/d7/d20/d37 39 2024-05-15T15:54:25.538 INFO:tasks.workunit.client.0.smithi049.stdout:3/728: chown d12/f96 606887 1 2024-05-15T15:54:25.538 INFO:tasks.workunit.client.1.smithi144.stdout:2/785: chown d3/d7/da4 249 1 2024-05-15T15:54:25.538 INFO:tasks.workunit.client.1.smithi144.stdout:8/912: getdents d2/d24/d2d/d30 0 2024-05-15T15:54:25.541 INFO:tasks.workunit.client.1.smithi144.stdout:7/842: truncate d4/df/f1a 903008 0 2024-05-15T15:54:25.542 INFO:tasks.workunit.client.1.smithi144.stdout:8/913: fdatasync d2/d24/d2d/d30/dd0/df7/f115 0 2024-05-15T15:54:25.547 INFO:tasks.workunit.client.1.smithi144.stdout:6/869: dread d7/fd [0,4194304] 0 2024-05-15T15:54:25.549 INFO:tasks.workunit.client.1.smithi144.stdout:5/870: truncate d0/dc/d2f/d79/f11b 3830368 0 2024-05-15T15:54:25.550 INFO:tasks.workunit.client.1.smithi144.stdout:0/803: rename d4/d6/d4a/d4d/f6e to d4/d2d/d100/f126 0 2024-05-15T15:54:25.550 INFO:tasks.workunit.client.0.smithi049.stdout:1/910: getdents d0/d4/d10f 0 2024-05-15T15:54:25.552 INFO:tasks.workunit.client.0.smithi049.stdout:1/911: dread - d0/d4/dd/de/d4d/fcf zero size 2024-05-15T15:54:25.553 INFO:tasks.workunit.client.1.smithi144.stdout:2/786: read d3/d7/d20/d37/d76/db5/db6/dbf/deb/ff1 [465139,101922] 0 2024-05-15T15:54:25.555 INFO:tasks.workunit.client.1.smithi144.stdout:8/914: truncate d2/d1a/fd3 467384 0 2024-05-15T15:54:25.557 INFO:tasks.workunit.client.1.smithi144.stdout:5/871: read d0/dc/d2f/d34/d63/d6e/f7c [379075,23624] 0 2024-05-15T15:54:25.557 INFO:tasks.workunit.client.0.smithi049.stdout:1/912: mknod d0/d4/dd/de/d1e/d2e/dad/d65/c13f 0 2024-05-15T15:54:25.559 INFO:tasks.workunit.client.1.smithi144.stdout:6/870: rename d7/d1e/f82 to d7/d32/f136 0 2024-05-15T15:54:25.559 INFO:tasks.workunit.client.1.smithi144.stdout:0/804: dread - d4/d2d/d48/d5b/d8e/f97 zero size 2024-05-15T15:54:25.559 INFO:tasks.workunit.client.1.smithi144.stdout:2/787: rename d3/d7/d20/d37/d76/db5/db6 to d3/d7/d20/d37/d76/db5/db6/dbf/deb/d104 22 2024-05-15T15:54:25.560 INFO:tasks.workunit.client.1.smithi144.stdout:9/855: sync 2024-05-15T15:54:25.561 INFO:tasks.workunit.client.0.smithi049.stdout:3/729: rename d12/d67/ca5 to d12/d30/cdf 0 2024-05-15T15:54:25.562 INFO:tasks.workunit.client.1.smithi144.stdout:7/843: mkdir d4/df/d127 0 2024-05-15T15:54:25.563 INFO:tasks.workunit.client.0.smithi049.stdout:6/796: getdents dd/d17/d24/d2b/d36/d50/dcf 0 2024-05-15T15:54:25.564 INFO:tasks.workunit.client.1.smithi144.stdout:5/872: stat d0/d10c/l94 0 2024-05-15T15:54:25.565 INFO:tasks.workunit.client.0.smithi049.stdout:0/759: sync 2024-05-15T15:54:25.566 INFO:tasks.workunit.client.0.smithi049.stdout:6/797: read dd/d17/d24/d2b/f46 [3847022,7959] 0 2024-05-15T15:54:25.569 INFO:tasks.workunit.client.1.smithi144.stdout:5/873: dwrite d0/dc/d1f/f39 [0,4194304] 0 2024-05-15T15:54:25.571 INFO:tasks.workunit.client.0.smithi049.stdout:8/750: dwrite d2/de/d71/f84 [0,4194304] 0 2024-05-15T15:54:25.574 INFO:tasks.workunit.client.1.smithi144.stdout:9/856: unlink d0/d8/d5e/d44/d4a/f111 0 2024-05-15T15:54:25.575 INFO:tasks.workunit.client.1.smithi144.stdout:3/834: write d0/d11/d24/d78/fc2 [76958,51209] 0 2024-05-15T15:54:25.575 INFO:tasks.workunit.client.0.smithi049.stdout:0/760: rmdir dc/d13/d14/d25/d31 39 2024-05-15T15:54:25.580 INFO:tasks.workunit.client.1.smithi144.stdout:1/826: getdents d7/d1d/d33/d29/d6b/d8e 0 2024-05-15T15:54:25.582 INFO:tasks.workunit.client.0.smithi049.stdout:1/913: creat d0/d4/dd/de/d1e/d2e/dad/f140 x:0 0 0 2024-05-15T15:54:25.584 INFO:tasks.workunit.client.1.smithi144.stdout:7/844: unlink d4/df/d25/d40/d73/f98 0 2024-05-15T15:54:25.585 INFO:tasks.workunit.client.0.smithi049.stdout:4/727: write fa [2401083,96441] 0 2024-05-15T15:54:25.585 INFO:tasks.workunit.client.1.smithi144.stdout:7/845: chown d4/df/d81/dfb 44 1 2024-05-15T15:54:25.588 INFO:tasks.workunit.client.0.smithi049.stdout:6/798: truncate dd/d26/f4f 285258 0 2024-05-15T15:54:25.589 INFO:tasks.workunit.client.0.smithi049.stdout:6/799: read - dd/d17/d24/d2b/d36/fd3 zero size 2024-05-15T15:54:25.591 INFO:tasks.workunit.client.1.smithi144.stdout:8/915: rename d2/d24/d2d/da0/d6e/d71/ff6 to d2/d24/d2d/d30/f12b 0 2024-05-15T15:54:25.594 INFO:tasks.workunit.client.0.smithi049.stdout:3/730: creat d12/d26/fe0 x:0 0 0 2024-05-15T15:54:25.595 INFO:tasks.workunit.client.1.smithi144.stdout:0/805: write d4/df/d14/f5f [3917708,130403] 0 2024-05-15T15:54:25.600 INFO:tasks.workunit.client.1.smithi144.stdout:0/806: dread - d4/df/f110 zero size 2024-05-15T15:54:25.600 INFO:tasks.workunit.client.1.smithi144.stdout:0/807: truncate d4/d2d/d48/d10a/f11e 315821 0 2024-05-15T15:54:25.601 INFO:tasks.workunit.client.0.smithi049.stdout:1/914: rename d0/d4/dd/de/d1e/d2e/l3a to d0/d4/dd/de/d1e/d12f/l141 0 2024-05-15T15:54:25.601 INFO:tasks.workunit.client.1.smithi144.stdout:9/857: stat d0/d8/d5e/d44/d4a/lba 0 2024-05-15T15:54:25.604 INFO:tasks.workunit.client.1.smithi144.stdout:3/835: mknod d0/d11/d3b/d93/c110 0 2024-05-15T15:54:25.605 INFO:tasks.workunit.client.1.smithi144.stdout:1/827: truncate d7/d1d/d58/fda 1394615 0 2024-05-15T15:54:25.606 INFO:tasks.workunit.client.1.smithi144.stdout:1/828: stat d7/d4b/dd9/c7c 0 2024-05-15T15:54:25.608 INFO:tasks.workunit.client.0.smithi049.stdout:6/800: creat dd/d17/d24/d2b/d36/d50/d54/d92/de5/ff3 x:0 0 0 2024-05-15T15:54:25.609 INFO:tasks.workunit.client.0.smithi049.stdout:0/761: dread dc/d13/d14/f19 [0,4194304] 0 2024-05-15T15:54:25.609 INFO:tasks.workunit.client.1.smithi144.stdout:3/836: dwrite d0/d1/d17/f91 [0,4194304] 0 2024-05-15T15:54:25.609 INFO:tasks.workunit.client.0.smithi049.stdout:6/801: chown dd/d17/d24/f27 347 1 2024-05-15T15:54:25.611 INFO:tasks.workunit.client.1.smithi144.stdout:3/837: dread d0/d1/d17/d39/d40/d7e/ff4 [0,4194304] 0 2024-05-15T15:54:25.612 INFO:tasks.workunit.client.1.smithi144.stdout:7/846: rmdir d4/df/d25/d38/d68 39 2024-05-15T15:54:25.614 INFO:tasks.workunit.client.0.smithi049.stdout:8/751: mknod d2/de/d71/d86/d5d/ce9 0 2024-05-15T15:54:25.620 INFO:tasks.workunit.client.1.smithi144.stdout:1/829: dread f1 [4194304,4194304] 0 2024-05-15T15:54:25.629 INFO:tasks.workunit.client.0.smithi049.stdout:1/915: write d0/d4/dd/de/d1e/d2e/dad/d65/df3/fff [471370,53508] 0 2024-05-15T15:54:25.629 INFO:tasks.workunit.client.0.smithi049.stdout:0/762: write dc/f34 [4632321,86627] 0 2024-05-15T15:54:25.631 INFO:tasks.workunit.client.0.smithi049.stdout:6/802: mknod dd/d17/d24/d2b/d36/d50/d54/d92/de5/cf4 0 2024-05-15T15:54:25.631 INFO:tasks.workunit.client.0.smithi049.stdout:0/763: stat dc/d13/d3d/d84/ld7 0 2024-05-15T15:54:25.637 INFO:tasks.workunit.client.1.smithi144.stdout:9/858: dwrite d0/d8/d1b/d79/fc7 [0,4194304] 0 2024-05-15T15:54:25.638 INFO:tasks.workunit.client.0.smithi049.stdout:3/731: creat d12/d14/d52/db9/fe1 x:0 0 0 2024-05-15T15:54:25.639 INFO:tasks.workunit.client.0.smithi049.stdout:8/752: dwrite d2/de/d71/d86/d5d/d7d/db8/f35 [0,4194304] 0 2024-05-15T15:54:25.640 INFO:tasks.workunit.client.1.smithi144.stdout:8/916: symlink d2/l12c 0 2024-05-15T15:54:25.641 INFO:tasks.workunit.client.1.smithi144.stdout:8/917: dread - d2/d24/d2d/da0/d6e/d71/d8c/f11c zero size 2024-05-15T15:54:25.642 INFO:tasks.workunit.client.0.smithi049.stdout:8/753: write d2/de/d25/d74/fae [400449,15527] 0 2024-05-15T15:54:25.643 INFO:tasks.workunit.client.1.smithi144.stdout:5/874: creat d0/dc/d2f/f122 x:0 0 0 2024-05-15T15:54:25.643 INFO:tasks.workunit.client.1.smithi144.stdout:7/847: write d4/d41/d5b/f4c [1994622,35373] 0 2024-05-15T15:54:25.645 INFO:tasks.workunit.client.1.smithi144.stdout:5/875: stat d0/dc/d24/d8e/d93/cac 0 2024-05-15T15:54:25.649 INFO:tasks.workunit.client.0.smithi049.stdout:4/728: rmdir d18/d88 0 2024-05-15T15:54:25.649 INFO:tasks.workunit.client.0.smithi049.stdout:0/764: dread dc/d13/d14/f79 [0,4194304] 0 2024-05-15T15:54:25.651 INFO:tasks.workunit.client.0.smithi049.stdout:3/732: truncate d12/d14/d52/db9/dc4/d68/d70/d75/f85 421443 0 2024-05-15T15:54:25.653 INFO:tasks.workunit.client.0.smithi049.stdout:3/733: read ff [767967,109718] 0 2024-05-15T15:54:25.653 INFO:tasks.workunit.client.1.smithi144.stdout:3/838: symlink d0/d1/d17/l111 0 2024-05-15T15:54:25.655 INFO:tasks.workunit.client.0.smithi049.stdout:6/803: rename dd/d17/d24/d2b/d36/d50/d7e/fdc to dd/d17/d24/d2b/d36/d50/d54/d92/ff5 0 2024-05-15T15:54:25.655 INFO:tasks.workunit.client.1.smithi144.stdout:4/867: sync 2024-05-15T15:54:25.656 INFO:tasks.workunit.client.1.smithi144.stdout:4/868: read - d6/dd/d2e/d71/f103 zero size 2024-05-15T15:54:25.657 INFO:tasks.workunit.client.0.smithi049.stdout:6/804: read - dd/d17/d24/d58/d5f/dd7/fd4 zero size 2024-05-15T15:54:25.657 INFO:tasks.workunit.client.0.smithi049.stdout:1/916: dread d0/f58 [0,4194304] 0 2024-05-15T15:54:25.658 INFO:tasks.workunit.client.0.smithi049.stdout:6/805: read - dd/d17/d24/d2b/d36/d50/d54/db6/fd8 zero size 2024-05-15T15:54:25.660 INFO:tasks.workunit.client.1.smithi144.stdout:2/788: rename d3/d7/d20/d37/d76/db5/db6/dbf/deb to d3/d7/d20/d37/d76/db5/d105 0 2024-05-15T15:54:25.660 INFO:tasks.workunit.client.1.smithi144.stdout:9/859: rename d0/d8/d5e to d0/d8/d5e/dd4/d109/d123 22 2024-05-15T15:54:25.662 INFO:tasks.workunit.client.0.smithi049.stdout:0/765: mkdir dc/d13/d2b/d76/d8a/da2/db3/d100 0 2024-05-15T15:54:25.663 INFO:tasks.workunit.client.0.smithi049.stdout:3/734: readlink d12/d14/l31 0 2024-05-15T15:54:25.664 INFO:tasks.workunit.client.0.smithi049.stdout:3/735: readlink d12/d14/d52/db9/l51 0 2024-05-15T15:54:25.664 INFO:tasks.workunit.client.1.smithi144.stdout:8/918: truncate d2/d24/d2d/d30/dd0/df7/f46 2450421 0 2024-05-15T15:54:25.666 INFO:tasks.workunit.client.0.smithi049.stdout:4/729: unlink d18/f1c 0 2024-05-15T15:54:25.669 INFO:tasks.workunit.client.0.smithi049.stdout:1/917: fdatasync d0/d4/d63/f111 0 2024-05-15T15:54:25.670 INFO:tasks.workunit.client.1.smithi144.stdout:0/808: rmdir d4/d2d/d48/d5b/dda/def 0 2024-05-15T15:54:25.672 INFO:tasks.workunit.client.0.smithi049.stdout:6/806: fdatasync dd/d17/d24/f74 0 2024-05-15T15:54:25.673 INFO:tasks.workunit.client.1.smithi144.stdout:3/839: symlink d0/d11/d3b/da0/d8f/l112 0 2024-05-15T15:54:25.674 INFO:tasks.workunit.client.1.smithi144.stdout:3/840: stat d0/d11/d3b/da7/fec 0 2024-05-15T15:54:25.675 INFO:tasks.workunit.client.0.smithi049.stdout:8/754: truncate d2/de/d25/d74/f9d 247206 0 2024-05-15T15:54:25.677 INFO:tasks.workunit.client.1.smithi144.stdout:4/869: unlink d6/dd/d34/f9d 0 2024-05-15T15:54:25.678 INFO:tasks.workunit.client.1.smithi144.stdout:1/830: write d7/d1d/d58/fda [135994,34127] 0 2024-05-15T15:54:25.681 INFO:tasks.workunit.client.1.smithi144.stdout:4/870: dread d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/d10e/fe4 [0,4194304] 0 2024-05-15T15:54:25.681 INFO:tasks.workunit.client.0.smithi049.stdout:4/730: creat d18/d85/db6/ff1 x:0 0 0 2024-05-15T15:54:25.682 INFO:tasks.workunit.client.1.smithi144.stdout:2/789: rmdir d3/d7/d20/d89 39 2024-05-15T15:54:25.684 INFO:tasks.workunit.client.0.smithi049.stdout:1/918: mknod d0/d4/dd/de/d1e/d2e/d3e/d90/db3/c142 0 2024-05-15T15:54:25.685 INFO:tasks.workunit.client.0.smithi049.stdout:8/755: dwrite d2/de/d71/d86/d5d/d7d/db8/d5b/da6/dc7/fcc [0,4194304] 0 2024-05-15T15:54:25.685 INFO:tasks.workunit.client.0.smithi049.stdout:9/641: sync 2024-05-15T15:54:25.687 INFO:tasks.workunit.client.0.smithi049.stdout:3/736: dwrite d12/d14/d32/f6d [0,4194304] 0 2024-05-15T15:54:25.688 INFO:tasks.workunit.client.1.smithi144.stdout:5/876: dwrite d0/dc/d2f/d34/d63/fd5 [0,4194304] 0 2024-05-15T15:54:25.688 INFO:tasks.workunit.client.0.smithi049.stdout:9/642: chown d0/d8/d9/d52/d5e/d8a/dac/fc4 23910964 1 2024-05-15T15:54:25.693 INFO:tasks.workunit.client.1.smithi144.stdout:0/809: symlink d4/d6/d3a/d5d/l127 0 2024-05-15T15:54:25.695 INFO:tasks.workunit.client.1.smithi144.stdout:3/841: creat d0/d11/d3b/da7/f113 x:0 0 0 2024-05-15T15:54:25.695 INFO:tasks.workunit.client.0.smithi049.stdout:3/737: dread d12/d14/d52/db9/dc4/d68/f9b [0,4194304] 0 2024-05-15T15:54:25.696 INFO:tasks.workunit.client.1.smithi144.stdout:3/842: fsync d0/d1/d6c/fdd 0 2024-05-15T15:54:25.697 INFO:tasks.workunit.client.0.smithi049.stdout:1/919: creat d0/d4/d63/f143 x:0 0 0 2024-05-15T15:54:25.699 INFO:tasks.workunit.client.0.smithi049.stdout:6/807: symlink dd/d17/d24/d2b/d36/lf6 0 2024-05-15T15:54:25.700 INFO:tasks.workunit.client.1.smithi144.stdout:1/831: chown d7/d1d/d33/cfc 1 1 2024-05-15T15:54:25.701 INFO:tasks.workunit.client.0.smithi049.stdout:9/643: unlink d0/d8e/cb5 0 2024-05-15T15:54:25.706 INFO:tasks.workunit.client.1.smithi144.stdout:6/871: sync 2024-05-15T15:54:25.712 INFO:tasks.workunit.client.0.smithi049.stdout:5/885: sync 2024-05-15T15:54:25.712 INFO:tasks.workunit.client.0.smithi049.stdout:7/736: sync 2024-05-15T15:54:25.712 INFO:tasks.workunit.client.0.smithi049.stdout:2/718: sync 2024-05-15T15:54:25.713 INFO:tasks.workunit.client.0.smithi049.stdout:1/920: creat d0/d4/dd/de/d4d/d8b/dba/f144 x:0 0 0 2024-05-15T15:54:25.713 INFO:tasks.workunit.client.0.smithi049.stdout:3/738: dread d12/d14/d32/d64/d88/fb3 [0,4194304] 0 2024-05-15T15:54:25.714 INFO:tasks.workunit.client.0.smithi049.stdout:7/737: stat d0/d34/fbc 0 2024-05-15T15:54:25.714 INFO:tasks.workunit.client.0.smithi049.stdout:2/719: chown d3/d52/cab 172379 1 2024-05-15T15:54:25.714 INFO:tasks.workunit.client.1.smithi144.stdout:8/919: dwrite d2/d24/d2d/d30/dd0/df7/f48 [0,4194304] 0 2024-05-15T15:54:25.715 INFO:tasks.workunit.client.1.smithi144.stdout:0/810: creat d4/d2d/dae/f128 x:0 0 0 2024-05-15T15:54:25.715 INFO:tasks.workunit.client.0.smithi049.stdout:3/739: stat d12/d14/d52/db9/l57 0 2024-05-15T15:54:25.717 INFO:tasks.workunit.client.0.smithi049.stdout:8/756: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/fea x:0 0 0 2024-05-15T15:54:25.719 INFO:tasks.workunit.client.0.smithi049.stdout:9/644: unlink d0/d8e/fb9 0 2024-05-15T15:54:25.723 INFO:tasks.workunit.client.1.smithi144.stdout:3/843: rmdir d0/d11/d3b/da0 39 2024-05-15T15:54:25.724 INFO:tasks.workunit.client.0.smithi049.stdout:0/766: link dc/d13/d14/d25/d31/d71/fc1 dc/d13/d14/d2f/df5/f101 0 2024-05-15T15:54:25.726 INFO:tasks.workunit.client.1.smithi144.stdout:7/848: rename d4/d3c/c55 to d4/d3c/d76/d9d/c128 0 2024-05-15T15:54:25.727 INFO:tasks.workunit.client.1.smithi144.stdout:7/849: write d4/f27 [2171206,108406] 0 2024-05-15T15:54:25.728 INFO:tasks.workunit.client.1.smithi144.stdout:4/871: mknod d6/dd/d34/d116/c11a 0 2024-05-15T15:54:25.732 INFO:tasks.workunit.client.0.smithi049.stdout:7/738: creat d0/ddb/dde/dca/fe6 x:0 0 0 2024-05-15T15:54:25.732 INFO:tasks.workunit.client.0.smithi049.stdout:7/739: stat d0/d34/f46 0 2024-05-15T15:54:25.733 INFO:tasks.workunit.client.1.smithi144.stdout:1/832: creat d7/d1d/d33/d29/dcd/f109 x:0 0 0 2024-05-15T15:54:25.734 INFO:tasks.workunit.client.1.smithi144.stdout:4/872: dwrite d6/d42/f112 [0,4194304] 0 2024-05-15T15:54:25.735 INFO:tasks.workunit.client.0.smithi049.stdout:6/808: fdatasync dd/d26/f4f 0 2024-05-15T15:54:25.735 INFO:tasks.workunit.client.1.smithi144.stdout:5/877: write d0/d10c/d73/fe6 [796911,6580] 0 2024-05-15T15:54:25.737 INFO:tasks.workunit.client.0.smithi049.stdout:1/921: dread - d0/d4/dd/de/d36/f108 zero size 2024-05-15T15:54:25.738 INFO:tasks.workunit.client.0.smithi049.stdout:6/809: fsync dd/d17/d24/d2b/d36/f56 0 2024-05-15T15:54:25.738 INFO:tasks.workunit.client.1.smithi144.stdout:9/860: link d0/d8/d5e/d44/d99/lcc d0/d8/d1b/d79/d8a/dae/ddb/dfc/l124 0 2024-05-15T15:54:25.745 INFO:tasks.workunit.client.1.smithi144.stdout:0/811: creat d4/df/d14/f129 x:0 0 0 2024-05-15T15:54:25.747 INFO:tasks.workunit.client.1.smithi144.stdout:3/844: creat d0/d11/d3b/dcf/f114 x:0 0 0 2024-05-15T15:54:25.748 INFO:tasks.workunit.client.0.smithi049.stdout:9/645: rmdir d0/d8/d9/d52/d5e 39 2024-05-15T15:54:25.749 INFO:tasks.workunit.client.1.smithi144.stdout:7/850: creat d4/df/d81/dfb/d108/f129 x:0 0 0 2024-05-15T15:54:25.751 INFO:tasks.workunit.client.0.smithi049.stdout:4/731: getdents d18/d21 0 2024-05-15T15:54:25.752 INFO:tasks.workunit.client.0.smithi049.stdout:4/732: chown d18/d1e/d2a/d36/d86 1780 1 2024-05-15T15:54:25.755 INFO:tasks.workunit.client.1.smithi144.stdout:1/833: mknod d7/d1d/d33/d41/d22/db4/c10a 0 2024-05-15T15:54:25.756 INFO:tasks.workunit.client.0.smithi049.stdout:4/733: dread d18/d21/d2b/d4e/fd6 [0,4194304] 0 2024-05-15T15:54:25.756 INFO:tasks.workunit.client.1.smithi144.stdout:1/834: stat d7/d1d/d33/d29/dcd/dff 0 2024-05-15T15:54:25.758 INFO:tasks.workunit.client.0.smithi049.stdout:7/740: truncate d0/d6/d23/f91 915155 0 2024-05-15T15:54:25.758 INFO:tasks.workunit.client.1.smithi144.stdout:2/790: link d3/d4d/fb7 d3/d7/d20/d22/d5e/f106 0 2024-05-15T15:54:25.762 INFO:tasks.workunit.client.1.smithi144.stdout:6/872: mkdir d7/d1e/d137 0 2024-05-15T15:54:25.763 INFO:tasks.workunit.client.1.smithi144.stdout:4/873: dwrite d6/fe7 [4194304,4194304] 0 2024-05-15T15:54:25.763 INFO:tasks.workunit.client.0.smithi049.stdout:1/922: mkdir d0/d4/dd/de/d36/dd3/d145 0 2024-05-15T15:54:25.766 INFO:tasks.workunit.client.0.smithi049.stdout:6/810: mkdir dd/d17/d24/d2b/d36/d86/df7 0 2024-05-15T15:54:25.767 INFO:tasks.workunit.client.0.smithi049.stdout:6/811: chown dd/d26/d5b/cdd 310430 1 2024-05-15T15:54:25.772 INFO:tasks.workunit.client.0.smithi049.stdout:3/740: symlink d12/d26/le2 0 2024-05-15T15:54:25.772 INFO:tasks.workunit.client.1.smithi144.stdout:8/920: truncate d2/f35 3770657 0 2024-05-15T15:54:25.773 INFO:tasks.workunit.client.1.smithi144.stdout:8/921: chown d2/da8/f10d 7506 1 2024-05-15T15:54:25.774 INFO:tasks.workunit.client.0.smithi049.stdout:2/720: symlink d3/db/df0/lf1 0 2024-05-15T15:54:25.775 INFO:tasks.workunit.client.0.smithi049.stdout:8/757: mknod d2/de/d71/d86/d5d/d7d/d1d/ceb 0 2024-05-15T15:54:25.779 INFO:tasks.workunit.client.1.smithi144.stdout:1/835: creat d7/d4b/dd9/f10b x:0 0 0 2024-05-15T15:54:25.780 INFO:tasks.workunit.client.1.smithi144.stdout:1/836: write d7/d4b/dd9/f10b [516591,16066] 0 2024-05-15T15:54:25.781 INFO:tasks.workunit.client.1.smithi144.stdout:3/845: write d0/d11/d24/f49 [3640594,112803] 0 2024-05-15T15:54:25.782 INFO:tasks.workunit.client.1.smithi144.stdout:5/878: creat d0/dc/d1f/d50/db3/d106/f123 x:0 0 0 2024-05-15T15:54:25.783 INFO:tasks.workunit.client.0.smithi049.stdout:0/767: truncate dc/d13/d14/d25/d31/d71/f3a 3416499 0 2024-05-15T15:54:25.786 INFO:tasks.workunit.client.1.smithi144.stdout:9/861: mknod d0/d8/d5e/d44/d99/d120/c125 0 2024-05-15T15:54:25.786 INFO:tasks.workunit.client.0.smithi049.stdout:0/768: dread dc/d32/f7b [0,4194304] 0 2024-05-15T15:54:25.787 INFO:tasks.workunit.client.0.smithi049.stdout:4/734: truncate d18/d1e/d63/fa2 3669900 0 2024-05-15T15:54:25.789 INFO:tasks.workunit.client.0.smithi049.stdout:7/741: creat d0/d6/d23/d4a/d7b/dc4/fe7 x:0 0 0 2024-05-15T15:54:25.791 INFO:tasks.workunit.client.1.smithi144.stdout:6/873: rename d7/c23 to d7/db/d5f/c138 0 2024-05-15T15:54:25.792 INFO:tasks.workunit.client.1.smithi144.stdout:4/874: chown d6/dd/d34/d3b/d30/db/d1d/d40/f80 1695 1 2024-05-15T15:54:25.798 INFO:tasks.workunit.client.0.smithi049.stdout:6/812: mkdir dd/d17/d24/d2b/d36/d50/d54/d92/de5/df8 0 2024-05-15T15:54:25.799 INFO:tasks.workunit.client.1.smithi144.stdout:8/922: dread - d2/d24/d2d/da0/d6e/d71/d8c/fcc zero size 2024-05-15T15:54:25.802 INFO:tasks.workunit.client.0.smithi049.stdout:5/886: link dc/dd/d47/ld4 dc/dd/d47/d83/da7/l125 0 2024-05-15T15:54:25.802 INFO:tasks.workunit.client.0.smithi049.stdout:6/813: dread f2 [0,4194304] 0 2024-05-15T15:54:25.803 INFO:tasks.workunit.client.0.smithi049.stdout:6/814: chown dd/d17/d24/d2b/ld1 135 1 2024-05-15T15:54:25.804 INFO:tasks.workunit.client.1.smithi144.stdout:7/851: mknod d4/df/d25/c12a 0 2024-05-15T15:54:25.805 INFO:tasks.workunit.client.0.smithi049.stdout:3/741: mkdir d12/d14/d52/db9/da6/de3 0 2024-05-15T15:54:25.807 INFO:tasks.workunit.client.1.smithi144.stdout:1/837: mknod d7/d1d/d58/d81/c10c 0 2024-05-15T15:54:25.807 INFO:tasks.workunit.client.1.smithi144.stdout:1/838: stat d7/d1d/d33/d29/dcd/dff/c104 0 2024-05-15T15:54:25.808 INFO:tasks.workunit.client.1.smithi144.stdout:1/839: write d7/df/fec [138001,4953] 0 2024-05-15T15:54:25.809 INFO:tasks.workunit.client.0.smithi049.stdout:8/758: mknod d2/d6a/cec 0 2024-05-15T15:54:25.810 INFO:tasks.workunit.client.0.smithi049.stdout:9/646: fsync d0/d8/d9/d52/d5e/d8a/dd3/d7d/dba/fca 0 2024-05-15T15:54:25.812 INFO:tasks.workunit.client.1.smithi144.stdout:9/862: fdatasync d0/f5 0 2024-05-15T15:54:25.816 INFO:tasks.workunit.client.1.smithi144.stdout:6/874: creat d7/db/d11/d7e/f139 x:0 0 0 2024-05-15T15:54:25.818 INFO:tasks.workunit.client.0.smithi049.stdout:0/769: rename dc/f15 to dc/d13/d2b/d76/d8a/f102 0 2024-05-15T15:54:25.821 INFO:tasks.workunit.client.1.smithi144.stdout:0/812: link d4/df/d1f/d24/d85/lb0 d4/d6/d4a/d4d/db1/l12a 0 2024-05-15T15:54:25.823 INFO:tasks.workunit.client.0.smithi049.stdout:7/742: rmdir d0/d6/d23/d45 39 2024-05-15T15:54:25.824 INFO:tasks.workunit.client.1.smithi144.stdout:2/791: dwrite d3/d7/d20/d3f/fde [0,4194304] 0 2024-05-15T15:54:25.826 INFO:tasks.workunit.client.1.smithi144.stdout:8/923: creat d2/d24/d2d/da0/d6e/f12d x:0 0 0 2024-05-15T15:54:25.826 INFO:tasks.workunit.client.1.smithi144.stdout:8/924: read - d2/d24/d2d/da0/d9e/db8/fe7 zero size 2024-05-15T15:54:25.834 INFO:tasks.workunit.client.0.smithi049.stdout:5/887: fsync dc/dd/d7b/f8f 0 2024-05-15T15:54:25.835 INFO:tasks.workunit.client.1.smithi144.stdout:1/840: symlink d7/d1d/d33/d29/d6b/d8e/l10d 0 2024-05-15T15:54:25.838 INFO:tasks.workunit.client.1.smithi144.stdout:7/852: dread d4/f21 [0,4194304] 0 2024-05-15T15:54:25.841 INFO:tasks.workunit.client.0.smithi049.stdout:2/721: read d3/da/d15/d73/f7f [6658,20259] 0 2024-05-15T15:54:25.842 INFO:tasks.workunit.client.0.smithi049.stdout:8/759: fsync d2/de/d71/d86/d5d/d7d/d1b/d68/f8e 0 2024-05-15T15:54:25.845 INFO:tasks.workunit.client.1.smithi144.stdout:6/875: creat d7/db/d9d/f13a x:0 0 0 2024-05-15T15:54:25.846 INFO:tasks.workunit.client.0.smithi049.stdout:9/647: mkdir d0/d8/dd6/de2 0 2024-05-15T15:54:25.847 INFO:tasks.workunit.client.1.smithi144.stdout:4/875: symlink d6/dd/d34/d3b/d30/d59/l11b 0 2024-05-15T15:54:25.849 INFO:tasks.workunit.client.1.smithi144.stdout:0/813: write d4/d6/d3a/ff1 [388256,56898] 0 2024-05-15T15:54:25.852 INFO:tasks.workunit.client.0.smithi049.stdout:0/770: mkdir dc/d13/d14/d2f/d8f/dcb/de2/d103 0 2024-05-15T15:54:25.853 INFO:tasks.workunit.client.1.smithi144.stdout:2/792: mknod d3/d7/d54/c107 0 2024-05-15T15:54:25.854 INFO:tasks.workunit.client.1.smithi144.stdout:9/863: dwrite d0/d8/d1b/d79/d8a/f10d [0,4194304] 0 2024-05-15T15:54:25.854 INFO:tasks.workunit.client.1.smithi144.stdout:2/793: chown d3/d7/d20/d22/c28 5 1 2024-05-15T15:54:25.856 INFO:tasks.workunit.client.0.smithi049.stdout:3/742: dwrite d12/d14/d32/d64/d88/fb3 [0,4194304] 0 2024-05-15T15:54:25.860 INFO:tasks.workunit.client.1.smithi144.stdout:3/846: creat d0/d1/f115 x:0 0 0 2024-05-15T15:54:25.862 INFO:tasks.workunit.client.1.smithi144.stdout:1/841: creat d7/d4b/dd9/f10e x:0 0 0 2024-05-15T15:54:25.867 INFO:tasks.workunit.client.0.smithi049.stdout:5/888: fdatasync dc/dd/fd7 0 2024-05-15T15:54:25.868 INFO:tasks.workunit.client.1.smithi144.stdout:5/879: rename d0/dc/d1f/d50/f86 to d0/d10c/f124 0 2024-05-15T15:54:25.869 INFO:tasks.workunit.client.1.smithi144.stdout:5/880: dread - d0/dc/d1f/d50/fbc zero size 2024-05-15T15:54:25.874 INFO:tasks.workunit.client.1.smithi144.stdout:4/876: dread d6/dd/d34/d3b/f4c [0,4194304] 0 2024-05-15T15:54:25.875 INFO:tasks.workunit.client.1.smithi144.stdout:0/814: creat d4/d2d/f12b x:0 0 0 2024-05-15T15:54:25.876 INFO:tasks.workunit.client.1.smithi144.stdout:0/815: chown d4/d6/f32 1 1 2024-05-15T15:54:25.881 INFO:tasks.workunit.client.1.smithi144.stdout:9/864: mkdir d0/d8/d1b/d62/d7d/d8d/dbb/dce/dd1/d126 0 2024-05-15T15:54:25.882 INFO:tasks.workunit.client.0.smithi049.stdout:9/648: creat d0/d8/d9/d52/d5e/d8a/fe3 x:0 0 0 2024-05-15T15:54:25.887 INFO:tasks.workunit.client.0.smithi049.stdout:2/722: dwrite d3/da/f2c [0,4194304] 0 2024-05-15T15:54:25.889 INFO:tasks.workunit.client.1.smithi144.stdout:2/794: dwrite d3/d7/d20/d3b/f4f [0,4194304] 0 2024-05-15T15:54:25.889 INFO:tasks.workunit.client.0.smithi049.stdout:0/771: creat dc/d13/d14/d25/d31/dae/dc5/d57/f104 x:0 0 0 2024-05-15T15:54:25.889 INFO:tasks.workunit.client.0.smithi049.stdout:2/723: truncate d3/d78/d8c/d91/d7a/fe7 948688 0 2024-05-15T15:54:25.894 INFO:tasks.workunit.client.0.smithi049.stdout:4/735: link d18/d1e/d63/d99/fa5 d18/d21/ff2 0 2024-05-15T15:54:25.895 INFO:tasks.workunit.client.1.smithi144.stdout:1/842: write d7/d1d/d33/d55/f5b [2595861,25697] 0 2024-05-15T15:54:25.897 INFO:tasks.workunit.client.1.smithi144.stdout:7/853: symlink d4/d41/d9c/d106/l12b 0 2024-05-15T15:54:25.898 INFO:tasks.workunit.client.0.smithi049.stdout:3/743: chown d12/d14/f27 24469962 1 2024-05-15T15:54:25.899 INFO:tasks.workunit.client.1.smithi144.stdout:2/795: dwrite d3/ffb [0,4194304] 0 2024-05-15T15:54:25.899 INFO:tasks.workunit.client.1.smithi144.stdout:6/876: mknod d7/c13b 0 2024-05-15T15:54:25.900 INFO:tasks.workunit.client.0.smithi049.stdout:7/743: rmdir d0/d6/d23/d45/d6a/da1/d82 39 2024-05-15T15:54:25.902 INFO:tasks.workunit.client.0.smithi049.stdout:0/772: dread dc/da4/db4/fd1 [0,4194304] 0 2024-05-15T15:54:25.903 INFO:tasks.workunit.client.1.smithi144.stdout:5/881: creat d0/dc/d24/d28/f125 x:0 0 0 2024-05-15T15:54:25.903 INFO:tasks.workunit.client.1.smithi144.stdout:8/925: dwrite d2/d24/d2d/d30/dd0/df7/f46 [0,4194304] 0 2024-05-15T15:54:25.915 INFO:tasks.workunit.client.0.smithi049.stdout:5/889: truncate dc/dd/d7b/d8d/ff5 701624 0 2024-05-15T15:54:25.923 INFO:tasks.workunit.client.0.smithi049.stdout:3/744: write d12/d14/fb8 [778793,18137] 0 2024-05-15T15:54:25.924 INFO:tasks.workunit.client.0.smithi049.stdout:0/773: truncate dc/d13/d2b/d76/f9a 187223 0 2024-05-15T15:54:25.924 INFO:tasks.workunit.client.0.smithi049.stdout:7/744: read d0/f15 [3717663,58587] 0 2024-05-15T15:54:25.924 INFO:tasks.workunit.client.0.smithi049.stdout:1/923: link d0/d4/dd/de/d36/c51 d0/d4/dd/de/d1e/d2e/d3e/c146 0 2024-05-15T15:54:25.925 INFO:tasks.workunit.client.1.smithi144.stdout:8/926: write d2/da8/ffb [355859,110560] 0 2024-05-15T15:54:25.925 INFO:tasks.workunit.client.1.smithi144.stdout:0/816: unlink d4/d2d/d48/cb9 0 2024-05-15T15:54:25.925 INFO:tasks.workunit.client.1.smithi144.stdout:9/865: creat d0/d8/d5e/dd4/f127 x:0 0 0 2024-05-15T15:54:25.926 INFO:tasks.workunit.client.0.smithi049.stdout:6/815: link dd/f53 dd/ff9 0 2024-05-15T15:54:25.926 INFO:tasks.workunit.client.1.smithi144.stdout:3/847: readlink d0/d1/d6c/lb6 0 2024-05-15T15:54:25.927 INFO:tasks.workunit.client.0.smithi049.stdout:1/924: dread - d0/d4/dd/de/d36/f108 zero size 2024-05-15T15:54:25.927 INFO:tasks.workunit.client.1.smithi144.stdout:6/877: mknod d7/db/d11/d7e/c13c 0 2024-05-15T15:54:25.930 INFO:tasks.workunit.client.1.smithi144.stdout:1/843: getdents d7/d1d/d33/d29/dd2/dea 0 2024-05-15T15:54:25.933 INFO:tasks.workunit.client.1.smithi144.stdout:4/877: dwrite d6/dd/d2e/f61 [0,4194304] 0 2024-05-15T15:54:25.934 INFO:tasks.workunit.client.0.smithi049.stdout:2/724: symlink d3/lf2 0 2024-05-15T15:54:25.934 INFO:tasks.workunit.client.0.smithi049.stdout:7/745: dwrite d0/d6/da/f8d [0,4194304] 0 2024-05-15T15:54:25.937 INFO:tasks.workunit.client.1.smithi144.stdout:3/848: read d0/d11/d24/d78/ff2 [817485,22909] 0 2024-05-15T15:54:25.938 INFO:tasks.workunit.client.1.smithi144.stdout:8/927: creat d2/d24/d2d/da0/d6e/d71/d8c/f12e x:0 0 0 2024-05-15T15:54:25.941 INFO:tasks.workunit.client.1.smithi144.stdout:6/878: mknod d7/db/da2/d70/db6/c13d 0 2024-05-15T15:54:25.943 INFO:tasks.workunit.client.1.smithi144.stdout:6/879: write d7/db/d35/d84/d81/f83 [6469573,108612] 0 2024-05-15T15:54:25.943 INFO:tasks.workunit.client.1.smithi144.stdout:9/866: truncate d0/d8/d1b/d62/fa3 459653 0 2024-05-15T15:54:25.945 INFO:tasks.workunit.client.1.smithi144.stdout:9/867: write d0/d8/d1b/d11c/ffa [3681901,70207] 0 2024-05-15T15:54:25.947 INFO:tasks.workunit.client.0.smithi049.stdout:8/760: write d2/de/d25/d8c/fe1 [2305732,51378] 0 2024-05-15T15:54:25.948 INFO:tasks.workunit.client.0.smithi049.stdout:9/649: write d0/d8/d1b/d40/d5c/f70 [451635,62991] 0 2024-05-15T15:54:25.948 INFO:tasks.workunit.client.1.smithi144.stdout:1/844: mknod d7/d1d/d33/d1e/d34/dd8/c10f 0 2024-05-15T15:54:25.949 INFO:tasks.workunit.client.1.smithi144.stdout:1/845: chown d7/d1d/d58/fda 33709084 1 2024-05-15T15:54:25.950 INFO:tasks.workunit.client.0.smithi049.stdout:9/650: chown d0/d8/d21/fa4 22 1 2024-05-15T15:54:25.950 INFO:tasks.workunit.client.1.smithi144.stdout:0/817: rename d4/d6/d3a/d6a/c9b to d4/d2d/d48/c12c 0 2024-05-15T15:54:25.950 INFO:tasks.workunit.client.0.smithi049.stdout:9/651: read - d0/d8/fd9 zero size 2024-05-15T15:54:25.952 INFO:tasks.workunit.client.0.smithi049.stdout:6/816: creat dd/d17/d24/d58/d5f/dd7/ffa x:0 0 0 2024-05-15T15:54:25.953 INFO:tasks.workunit.client.1.smithi144.stdout:2/796: link d3/d7/c9e d3/d7/d20/d37/de1/c108 0 2024-05-15T15:54:25.954 INFO:tasks.workunit.client.0.smithi049.stdout:0/774: dread dc/d32/d93/f88 [0,4194304] 0 2024-05-15T15:54:25.957 INFO:tasks.workunit.client.1.smithi144.stdout:4/878: write d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/d10e/fb7 [635797,62447] 0 2024-05-15T15:54:25.959 INFO:tasks.workunit.client.0.smithi049.stdout:1/925: rename d0/d4/dd/de/d1e/d2e/f2c to d0/d4/dd/de/d48/d84/d134/f147 0 2024-05-15T15:54:25.959 INFO:tasks.workunit.client.0.smithi049.stdout:0/775: chown dc/d13/d3d/d84 11444 1 2024-05-15T15:54:25.960 INFO:tasks.workunit.client.0.smithi049.stdout:8/761: dread d2/de/d71/d86/d5d/d63/f6e [0,4194304] 0 2024-05-15T15:54:25.962 INFO:tasks.workunit.client.0.smithi049.stdout:4/736: unlink d18/d21/d2b/faf 0 2024-05-15T15:54:25.962 INFO:tasks.workunit.client.0.smithi049.stdout:7/746: dread d0/d6/d23/f8a [0,4194304] 0 2024-05-15T15:54:25.963 INFO:tasks.workunit.client.0.smithi049.stdout:8/762: chown d2/de/c19 1 1 2024-05-15T15:54:25.963 INFO:tasks.workunit.client.1.smithi144.stdout:6/880: dread d7/db/d35/d84/dbe/fd3 [0,4194304] 0 2024-05-15T15:54:25.964 INFO:tasks.workunit.client.1.smithi144.stdout:8/928: creat d2/de/d3f/dff/d125/f12f x:0 0 0 2024-05-15T15:54:25.964 INFO:tasks.workunit.client.0.smithi049.stdout:8/763: chown d2/d6a/cb7 3 1 2024-05-15T15:54:25.965 INFO:tasks.workunit.client.0.smithi049.stdout:8/764: stat d2/d18/f47 0 2024-05-15T15:54:25.966 INFO:tasks.workunit.client.0.smithi049.stdout:5/890: dwrite dc/f7f [0,4194304] 0 2024-05-15T15:54:25.966 INFO:tasks.workunit.client.0.smithi049.stdout:5/891: chown f9 31852 1 2024-05-15T15:54:25.967 INFO:tasks.workunit.client.1.smithi144.stdout:9/868: truncate d0/d8/d1b/d62/d7d/d8d/faf 648018 0 2024-05-15T15:54:25.969 INFO:tasks.workunit.client.1.smithi144.stdout:1/846: rmdir d7/d1d/d33/d41/d22/dbd 39 2024-05-15T15:54:25.972 INFO:tasks.workunit.client.1.smithi144.stdout:0/818: rename d4/df/d1f/d24/c8a to d4/c12d 0 2024-05-15T15:54:25.974 INFO:tasks.workunit.client.1.smithi144.stdout:4/879: stat d6/dd/d34/d3b/l24 0 2024-05-15T15:54:25.975 INFO:tasks.workunit.client.0.smithi049.stdout:6/817: dwrite dd/d26/f3f [0,4194304] 0 2024-05-15T15:54:25.975 INFO:tasks.workunit.client.1.smithi144.stdout:5/882: link d0/dc/lf d0/d7e/l126 0 2024-05-15T15:54:25.978 INFO:tasks.workunit.client.1.smithi144.stdout:6/881: dread d7/db/d35/d84/fcf [0,4194304] 0 2024-05-15T15:54:25.979 INFO:tasks.workunit.client.1.smithi144.stdout:9/869: mkdir d0/d8/d1b/d62/d6b/d128 0 2024-05-15T15:54:25.980 INFO:tasks.workunit.client.0.smithi049.stdout:6/818: read dd/d17/d24/d2b/f4d [301279,88662] 0 2024-05-15T15:54:25.982 INFO:tasks.workunit.client.1.smithi144.stdout:8/929: rename d2/d24/d2d/da0/d6e/d8d/ce1 to d2/d24/d2d/c130 0 2024-05-15T15:54:25.993 INFO:tasks.workunit.client.1.smithi144.stdout:4/880: creat d6/dd/d2e/d75/dff/deb/dba/f11c x:0 0 0 2024-05-15T15:54:25.993 INFO:tasks.workunit.client.1.smithi144.stdout:5/883: mkdir d0/dc/d24/d8e/d93/d9e/d127 0 2024-05-15T15:54:25.993 INFO:tasks.workunit.client.1.smithi144.stdout:6/882: mknod d7/db/d35/d84/dbe/c13e 0 2024-05-15T15:54:25.994 INFO:tasks.workunit.client.1.smithi144.stdout:9/870: rmdir d0/d8/d1b/d79/d8a/dae/ddb 39 2024-05-15T15:54:25.996 INFO:tasks.workunit.client.1.smithi144.stdout:2/797: getdents d3/d7/d20/d22/d52/d64/d97 0 2024-05-15T15:54:26.001 INFO:tasks.workunit.client.1.smithi144.stdout:4/881: dwrite d6/dd/d34/fed [0,4194304] 0 2024-05-15T15:54:26.007 INFO:tasks.workunit.client.1.smithi144.stdout:2/798: dwrite d3/d7/d20/d37/d76/db5/d105/ff6 [0,4194304] 0 2024-05-15T15:54:26.012 INFO:tasks.workunit.client.1.smithi144.stdout:8/930: dread d2/d24/d2d/da0/d6e/f66 [0,4194304] 0 2024-05-15T15:54:26.014 INFO:tasks.workunit.client.1.smithi144.stdout:6/883: mkdir d7/d62/d112/d13f 0 2024-05-15T15:54:26.019 INFO:tasks.workunit.client.1.smithi144.stdout:8/931: dwrite d2/d24/d2d/da0/d6e/d8d/ffd [0,4194304] 0 2024-05-15T15:54:26.020 INFO:tasks.workunit.client.1.smithi144.stdout:8/932: chown d2/d24/d2d/da0/d6e/d71/d73/d98 1124695 1 2024-05-15T15:54:26.021 INFO:tasks.workunit.client.1.smithi144.stdout:9/871: creat d0/d8/d1b/d114/dbf/f129 x:0 0 0 2024-05-15T15:54:26.024 INFO:tasks.workunit.client.1.smithi144.stdout:3/849: truncate d0/d11/d3b/da0/d8f/f9b 3560178 0 2024-05-15T15:54:26.027 INFO:tasks.workunit.client.1.smithi144.stdout:4/882: symlink d6/dd/d34/d3b/d30/db/l11d 0 2024-05-15T15:54:26.032 INFO:tasks.workunit.client.1.smithi144.stdout:0/819: dwrite f2 [0,4194304] 0 2024-05-15T15:54:26.033 INFO:tasks.workunit.client.1.smithi144.stdout:7/854: sync 2024-05-15T15:54:26.035 INFO:tasks.workunit.client.0.smithi049.stdout:2/725: symlink d3/db/d1d/d1e/dd9/lf3 0 2024-05-15T15:54:26.039 INFO:tasks.workunit.client.1.smithi144.stdout:4/883: mknod d6/dd/d34/d3b/d30/dc9/d10f/c11e 0 2024-05-15T15:54:26.045 INFO:tasks.workunit.client.1.smithi144.stdout:9/872: fsync d0/d8/d1b/d62/d7d/d8d/faf 0 2024-05-15T15:54:26.046 INFO:tasks.workunit.client.1.smithi144.stdout:9/873: chown d0/d8/d1b/d79/d8a 13964 1 2024-05-15T15:54:26.047 INFO:tasks.workunit.client.0.smithi049.stdout:7/747: mknod d0/d6/d23/d76/ce8 0 2024-05-15T15:54:26.048 INFO:tasks.workunit.client.1.smithi144.stdout:5/884: write d0/dc/d24/d28/d2d/f2b [1157009,42894] 0 2024-05-15T15:54:26.049 INFO:tasks.workunit.client.0.smithi049.stdout:3/745: link d12/d14/d32/l76 d12/d30/dc1/le4 0 2024-05-15T15:54:26.050 INFO:tasks.workunit.client.1.smithi144.stdout:3/850: chown d0/d1/d17/dae 2678 1 2024-05-15T15:54:26.051 INFO:tasks.workunit.client.1.smithi144.stdout:3/851: chown d0/d11/d3b/da0/d8f/fc3 16404 1 2024-05-15T15:54:26.051 INFO:tasks.workunit.client.0.smithi049.stdout:8/765: symlink d2/de/d25/led 0 2024-05-15T15:54:26.052 INFO:tasks.workunit.client.1.smithi144.stdout:5/885: dread d0/dc/d1f/f39 [0,4194304] 0 2024-05-15T15:54:26.052 INFO:tasks.workunit.client.1.smithi144.stdout:5/886: readlink d0/la 0 2024-05-15T15:54:26.053 INFO:tasks.workunit.client.0.smithi049.stdout:5/892: rename dc/c66 to dc/db0/dbf/c126 0 2024-05-15T15:54:26.055 INFO:tasks.workunit.client.1.smithi144.stdout:4/884: creat d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/dda/f11f x:0 0 0 2024-05-15T15:54:26.056 INFO:tasks.workunit.client.1.smithi144.stdout:4/885: write d6/dd/d34/d3b/d30/dc9/d10f/d4b/fdb [738011,19955] 0 2024-05-15T15:54:26.057 INFO:tasks.workunit.client.0.smithi049.stdout:6/819: symlink dd/d17/d24/d58/d5f/dd7/lfb 0 2024-05-15T15:54:26.058 INFO:tasks.workunit.client.1.smithi144.stdout:8/933: link d2/d24/d2d/d30/c41 d2/d24/d2d/da0/df8/c131 0 2024-05-15T15:54:26.059 INFO:tasks.workunit.client.0.smithi049.stdout:0/776: mknod dc/d13/c105 0 2024-05-15T15:54:26.060 INFO:tasks.workunit.client.0.smithi049.stdout:5/893: dwrite dc/dd/d47/f5d [0,4194304] 0 2024-05-15T15:54:26.062 INFO:tasks.workunit.client.0.smithi049.stdout:1/926: mkdir d0/d4/d10f/d148 0 2024-05-15T15:54:26.064 INFO:tasks.workunit.client.1.smithi144.stdout:9/874: mknod d0/d8/d5e/d44/d16/db0/c12a 0 2024-05-15T15:54:26.067 INFO:tasks.workunit.client.1.smithi144.stdout:0/820: chown d4/d6/f32 0 1 2024-05-15T15:54:26.073 INFO:tasks.workunit.client.0.smithi049.stdout:3/746: mknod d12/d14/d52/db9/da6/dc2/ce5 0 2024-05-15T15:54:26.074 INFO:tasks.workunit.client.1.smithi144.stdout:5/887: mkdir d0/d7e/d128 0 2024-05-15T15:54:26.075 INFO:tasks.workunit.client.1.smithi144.stdout:6/884: write d7/db/d9d/fc6 [981307,72147] 0 2024-05-15T15:54:26.075 INFO:tasks.workunit.client.0.smithi049.stdout:8/766: truncate d2/de/f6c 46060 0 2024-05-15T15:54:26.076 INFO:tasks.workunit.client.1.smithi144.stdout:8/934: dread d2/de/f103 [0,4194304] 0 2024-05-15T15:54:26.076 INFO:tasks.workunit.client.1.smithi144.stdout:5/888: chown d0/dc/d24/d57/d96/lfc 47915581 1 2024-05-15T15:54:26.080 INFO:tasks.workunit.client.1.smithi144.stdout:4/886: mkdir d6/dd/d34/d120 0 2024-05-15T15:54:26.083 INFO:tasks.workunit.client.1.smithi144.stdout:4/887: dread d6/dd/fbb [0,4194304] 0 2024-05-15T15:54:26.085 INFO:tasks.workunit.client.0.smithi049.stdout:6/820: rmdir dd/d17/d24/d2b/d36/d86 39 2024-05-15T15:54:26.085 INFO:tasks.workunit.client.1.smithi144.stdout:4/888: truncate d6/dd/d34/d3b/d30/db/f33 2075921 0 2024-05-15T15:54:26.085 INFO:tasks.workunit.client.0.smithi049.stdout:4/737: dwrite d18/fc5 [0,4194304] 0 2024-05-15T15:54:26.086 INFO:tasks.workunit.client.1.smithi144.stdout:4/889: chown d6/dd/d34/d3b/d30/db/d1d/d4e/fd4 6590104 1 2024-05-15T15:54:26.088 INFO:tasks.workunit.client.0.smithi049.stdout:1/927: unlink d0/d4/dd/de/d4d/db8/l117 0 2024-05-15T15:54:26.090 INFO:tasks.workunit.client.0.smithi049.stdout:1/928: chown d0/d4/dd/de/d1e/d16/d8a/lb1 62323 1 2024-05-15T15:54:26.091 INFO:tasks.workunit.client.0.smithi049.stdout:8/767: truncate d2/de/d71/d86/f51 274387 0 2024-05-15T15:54:26.094 INFO:tasks.workunit.client.0.smithi049.stdout:9/652: rename d0/d8e/la6 to d0/d8/d9/le4 0 2024-05-15T15:54:26.095 INFO:tasks.workunit.client.0.smithi049.stdout:9/653: fsync d0/d8/d9/d52/d5e/d8a/fe3 0 2024-05-15T15:54:26.097 INFO:tasks.workunit.client.1.smithi144.stdout:6/885: symlink d7/db/d9d/d97/l140 0 2024-05-15T15:54:26.099 INFO:tasks.workunit.client.1.smithi144.stdout:6/886: chown d7/db/d35/d84/dbe/d118/l121 1654 1 2024-05-15T15:54:26.099 INFO:tasks.workunit.client.0.smithi049.stdout:7/748: read d0/d6/d23/d45/d6a/da1/d80/fb2 [1758511,2799] 0 2024-05-15T15:54:26.100 INFO:tasks.workunit.client.1.smithi144.stdout:6/887: chown d7/d32/l80 455200 1 2024-05-15T15:54:26.101 INFO:tasks.workunit.client.0.smithi049.stdout:6/821: unlink dd/d17/d24/d2b/d36/d50/d54/f8f 0 2024-05-15T15:54:26.103 INFO:tasks.workunit.client.0.smithi049.stdout:1/929: truncate d0/d4/fa 3746243 0 2024-05-15T15:54:26.105 INFO:tasks.workunit.client.1.smithi144.stdout:8/935: mkdir d2/de/d3f/dff/d125/d132 0 2024-05-15T15:54:26.107 INFO:tasks.workunit.client.0.smithi049.stdout:8/768: mkdir d2/de/d71/d86/d5d/d7d/d1b/d68/dee 0 2024-05-15T15:54:26.107 INFO:tasks.workunit.client.0.smithi049.stdout:8/769: readlink d2/d18/l79 0 2024-05-15T15:54:26.109 INFO:tasks.workunit.client.1.smithi144.stdout:8/936: chown d2/d24/d2d/da0/d6e/c70 215350 1 2024-05-15T15:54:26.109 INFO:tasks.workunit.client.1.smithi144.stdout:3/852: dwrite d0/d1/d17/f19 [0,4194304] 0 2024-05-15T15:54:26.110 INFO:tasks.workunit.client.1.smithi144.stdout:3/853: stat d0/d1/d17/d39/d40/d7e/f81 0 2024-05-15T15:54:26.114 INFO:tasks.workunit.client.0.smithi049.stdout:9/654: mknod d0/d8/d9/d52/d5e/d5f/ce5 0 2024-05-15T15:54:26.115 INFO:tasks.workunit.client.0.smithi049.stdout:7/749: readlink d0/l3 0 2024-05-15T15:54:26.116 INFO:tasks.workunit.client.0.smithi049.stdout:7/750: chown d0/d6/da/f9a 0 1 2024-05-15T15:54:26.118 INFO:tasks.workunit.client.0.smithi049.stdout:6/822: unlink dd/f14 0 2024-05-15T15:54:26.119 INFO:tasks.workunit.client.1.smithi144.stdout:6/888: read d7/db/d5f/fb8 [513874,39820] 0 2024-05-15T15:54:26.120 INFO:tasks.workunit.client.0.smithi049.stdout:6/823: dread - dd/d17/d24/d2b/d36/d50/d54/db6/fd8 zero size 2024-05-15T15:54:26.123 INFO:tasks.workunit.client.1.smithi144.stdout:5/889: unlink d0/dc/d1f/d65/lb5 0 2024-05-15T15:54:26.124 INFO:tasks.workunit.client.1.smithi144.stdout:0/821: dread d4/d2d/f8d [0,4194304] 0 2024-05-15T15:54:26.124 INFO:tasks.workunit.client.0.smithi049.stdout:8/770: mknod d2/de/d95/cef 0 2024-05-15T15:54:26.125 INFO:tasks.workunit.client.0.smithi049.stdout:8/771: write d2/de/d25/d74/dbe/fe4 [463160,57209] 0 2024-05-15T15:54:26.129 INFO:tasks.workunit.client.0.smithi049.stdout:9/655: unlink d0/d8/d9/d52/d5e/d8a/f99 0 2024-05-15T15:54:26.130 INFO:tasks.workunit.client.1.smithi144.stdout:1/847: sync 2024-05-15T15:54:26.130 INFO:tasks.workunit.client.1.smithi144.stdout:1/848: chown d7/d1d/d58/c68 9031 1 2024-05-15T15:54:26.131 INFO:tasks.workunit.client.0.smithi049.stdout:7/751: readlink d0/l2 0 2024-05-15T15:54:26.132 INFO:tasks.workunit.client.0.smithi049.stdout:7/752: readlink d0/ddb/le3 0 2024-05-15T15:54:26.132 INFO:tasks.workunit.client.0.smithi049.stdout:8/772: dwrite d2/de/d71/d86/f96 [0,4194304] 0 2024-05-15T15:54:26.134 INFO:tasks.workunit.client.0.smithi049.stdout:7/753: write d0/d6/d23/d45/fba [4773581,117847] 0 2024-05-15T15:54:26.136 INFO:tasks.workunit.client.0.smithi049.stdout:6/824: rmdir dd/d17/d24/d2b/d36/d50/d54/d92/de5 39 2024-05-15T15:54:26.139 INFO:tasks.workunit.client.1.smithi144.stdout:6/889: mkdir d7/d1e/d28/db5/d141 0 2024-05-15T15:54:26.140 INFO:tasks.workunit.client.1.smithi144.stdout:6/890: fdatasync d7/db/d5f/f113 0 2024-05-15T15:54:26.141 INFO:tasks.workunit.client.0.smithi049.stdout:5/894: rename dc/dd/d47/d83/dcc/c49 to dc/c127 0 2024-05-15T15:54:26.141 INFO:tasks.workunit.client.1.smithi144.stdout:6/891: dread - d7/db/d3a/dd2/f116 zero size 2024-05-15T15:54:26.142 INFO:tasks.workunit.client.1.smithi144.stdout:7/855: sync 2024-05-15T15:54:26.142 INFO:tasks.workunit.client.1.smithi144.stdout:2/799: sync 2024-05-15T15:54:26.142 INFO:tasks.workunit.client.1.smithi144.stdout:5/890: creat d0/d7e/f129 x:0 0 0 2024-05-15T15:54:26.144 INFO:tasks.workunit.client.1.smithi144.stdout:7/856: chown d4/df/c110 10 1 2024-05-15T15:54:26.149 INFO:tasks.workunit.client.1.smithi144.stdout:8/937: mkdir d2/d3a/d133 0 2024-05-15T15:54:26.150 INFO:tasks.workunit.client.1.smithi144.stdout:8/938: fsync d2/d24/d2d/da0/d9e/db8/dde/f11e 0 2024-05-15T15:54:26.152 INFO:tasks.workunit.client.1.smithi144.stdout:3/854: stat d0/d1/d56/d98/d9c/lab 0 2024-05-15T15:54:26.154 INFO:tasks.workunit.client.1.smithi144.stdout:1/849: creat d7/d1d/d33/d41/d22/d45/f110 x:0 0 0 2024-05-15T15:54:26.156 INFO:tasks.workunit.client.0.smithi049.stdout:8/773: readlink d2/de/d71/d86/d5d/d7d/db8/d5b/d60/l77 0 2024-05-15T15:54:26.157 INFO:tasks.workunit.client.1.smithi144.stdout:9/875: dwrite d0/d8/d5e/d44/d10/d9c/f3f [0,4194304] 0 2024-05-15T15:54:26.157 INFO:tasks.workunit.client.1.smithi144.stdout:4/890: link d6/dd/d34/d3b/d30/db/d1d/f104 d6/dd/d34/d3b/f121 0 2024-05-15T15:54:26.159 INFO:tasks.workunit.client.1.smithi144.stdout:5/891: dread d0/dc/d2f/d34/d63/d6e/f7c [0,4194304] 0 2024-05-15T15:54:26.166 INFO:tasks.workunit.client.0.smithi049.stdout:9/656: dread d0/d8/f5a [0,4194304] 0 2024-05-15T15:54:26.167 INFO:tasks.workunit.client.0.smithi049.stdout:1/930: dwrite d0/d4/dd/de/d1e/d16/d8a/dd2/df2/f11f [0,4194304] 0 2024-05-15T15:54:26.169 INFO:tasks.workunit.client.1.smithi144.stdout:7/857: chown d4/d3c/d76/d9d/dac/de5/fea 0 1 2024-05-15T15:54:26.169 INFO:tasks.workunit.client.0.smithi049.stdout:5/895: truncate dc/dd/d10/f2a 1052005 0 2024-05-15T15:54:26.170 INFO:tasks.workunit.client.0.smithi049.stdout:4/738: link d18/d21/f30 d18/d1e/d2a/d36/ff3 0 2024-05-15T15:54:26.176 INFO:tasks.workunit.client.1.smithi144.stdout:8/939: creat d2/de/d49/d83/d109/f134 x:0 0 0 2024-05-15T15:54:26.179 INFO:tasks.workunit.client.1.smithi144.stdout:8/940: dread d2/de/d49/f76 [0,4194304] 0 2024-05-15T15:54:26.182 INFO:tasks.workunit.client.1.smithi144.stdout:1/850: write d7/d1d/d58/faa [610030,96165] 0 2024-05-15T15:54:26.183 INFO:tasks.workunit.client.1.smithi144.stdout:1/851: readlink d7/d1d/d33/d29/l8a 0 2024-05-15T15:54:26.183 INFO:tasks.workunit.client.1.smithi144.stdout:1/852: fdatasync d7/d4b/dd9/f10b 0 2024-05-15T15:54:26.185 INFO:tasks.workunit.client.1.smithi144.stdout:9/876: mkdir d0/d8/d5e/d44/d10/d19/dd5/d12b 0 2024-05-15T15:54:26.187 INFO:tasks.workunit.client.0.smithi049.stdout:9/657: creat d0/d8/d9/d52/d5e/d8a/dd3/d87/fe6 x:0 0 0 2024-05-15T15:54:26.190 INFO:tasks.workunit.client.1.smithi144.stdout:4/891: rename d6/dd/d2e/d94/fa3 to d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/d10e/f122 0 2024-05-15T15:54:26.190 INFO:tasks.workunit.client.1.smithi144.stdout:4/892: stat d6/d49 0 2024-05-15T15:54:26.190 INFO:tasks.workunit.client.1.smithi144.stdout:4/893: chown d6/dd/dc3/dcc 96211 1 2024-05-15T15:54:26.192 INFO:tasks.workunit.client.0.smithi049.stdout:1/931: creat d0/d4/dd/de/d1e/d2e/dad/d65/f149 x:0 0 0 2024-05-15T15:54:26.197 INFO:tasks.workunit.client.0.smithi049.stdout:8/774: mknod d2/de/d71/d86/d5d/d7d/d1b/d69/dac/cf0 0 2024-05-15T15:54:26.198 INFO:tasks.workunit.client.0.smithi049.stdout:7/754: dwrite d0/d6/d23/d45/d6a/da1/d80/fb2 [0,4194304] 0 2024-05-15T15:54:26.199 INFO:tasks.workunit.client.1.smithi144.stdout:2/800: symlink d3/d7/d20/d22/d52/d64/d97/db8/dd6/l109 0 2024-05-15T15:54:26.201 INFO:tasks.workunit.client.1.smithi144.stdout:3/855: write d0/d1/d17/d39/dc8/fdc [1215892,66696] 0 2024-05-15T15:54:26.201 INFO:tasks.workunit.client.1.smithi144.stdout:6/892: dwrite d7/f71 [0,4194304] 0 2024-05-15T15:54:26.201 INFO:tasks.workunit.client.0.smithi049.stdout:7/755: read d0/d6/d23/d45/d6a/da1/fbf [3754498,57223] 0 2024-05-15T15:54:26.204 INFO:tasks.workunit.client.0.smithi049.stdout:2/726: rename d3/db/d5b/dbf/dc1/c7b to d3/db/d1d/d1e/d2b/cf4 0 2024-05-15T15:54:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:26 smithi049 ceph-mon[28399]: pgmap v33: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 69 MiB/s rd, 182 MiB/s wr, 374 op/s 2024-05-15T15:54:26.206 INFO:tasks.workunit.client.1.smithi144.stdout:0/822: creat d4/d2d/d48/d5b/f12e x:0 0 0 2024-05-15T15:54:26.211 INFO:tasks.workunit.client.0.smithi049.stdout:9/658: fdatasync d0/d8/d1b/f32 0 2024-05-15T15:54:26.212 INFO:tasks.workunit.client.1.smithi144.stdout:8/941: creat d2/da8/d119/f135 x:0 0 0 2024-05-15T15:54:26.213 INFO:tasks.workunit.client.0.smithi049.stdout:1/932: mkdir d0/d4/dd/de/d1e/d2e/d3e/d90/d14a 0 2024-05-15T15:54:26.216 INFO:tasks.workunit.client.0.smithi049.stdout:5/896: truncate dc/db0/dbf/fc5 2060920 0 2024-05-15T15:54:26.219 INFO:tasks.workunit.client.1.smithi144.stdout:1/853: unlink d7/d1d/d58/d81/ld7 0 2024-05-15T15:54:26.224 INFO:tasks.workunit.client.1.smithi144.stdout:7/858: dwrite d4/df/d25/d40/d62/f95 [0,4194304] 0 2024-05-15T15:54:26.226 INFO:tasks.workunit.client.0.smithi049.stdout:6/825: rename dd/d17/d24 to dd/d17/d24/d2b/d36/d50/dfc 22 2024-05-15T15:54:26.227 INFO:tasks.workunit.client.0.smithi049.stdout:2/727: mknod d3/d8/dad/cf5 0 2024-05-15T15:54:26.228 INFO:tasks.workunit.client.0.smithi049.stdout:2/728: stat d3/db/d5b/fdc 0 2024-05-15T15:54:26.229 INFO:tasks.workunit.client.0.smithi049.stdout:2/729: chown d3/db/d1d/f28 50 1 2024-05-15T15:54:26.235 INFO:tasks.workunit.client.0.smithi049.stdout:8/775: dread d2/de/d71/d86/d5d/d7d/db8/d5b/fb2 [0,4194304] 0 2024-05-15T15:54:26.238 INFO:tasks.workunit.client.0.smithi049.stdout:9/659: mkdir d0/d8/d42/d75/de7 0 2024-05-15T15:54:26.242 INFO:tasks.workunit.client.1.smithi144.stdout:5/892: truncate d0/dc/d2f/d68/d5c/f80 910167 0 2024-05-15T15:54:26.244 INFO:tasks.workunit.client.0.smithi049.stdout:5/897: creat dc/d29/f128 x:0 0 0 2024-05-15T15:54:26.244 INFO:tasks.workunit.client.0.smithi049.stdout:8/776: dwrite d2/de/d25/d8c/d9c/fd3 [0,4194304] 0 2024-05-15T15:54:26.251 INFO:tasks.workunit.client.0.smithi049.stdout:7/756: symlink d0/le9 0 2024-05-15T15:54:26.252 INFO:tasks.workunit.client.0.smithi049.stdout:7/757: write d0/d2f/f8e [3423712,67680] 0 2024-05-15T15:54:26.253 INFO:tasks.workunit.client.0.smithi049.stdout:7/758: stat d0/d6/da/fc2 0 2024-05-15T15:54:26.254 INFO:tasks.workunit.client.0.smithi049.stdout:6/826: creat dd/d17/d24/d2b/d36/d50/d54/dde/ffd x:0 0 0 2024-05-15T15:54:26.256 INFO:tasks.workunit.client.0.smithi049.stdout:9/660: mknod d0/d8/d9/d22/ce8 0 2024-05-15T15:54:26.261 INFO:tasks.workunit.client.0.smithi049.stdout:5/898: creat dc/dd/f129 x:0 0 0 2024-05-15T15:54:26.270 INFO:tasks.workunit.client.0.smithi049.stdout:4/739: rename d18/d21/f45 to d18/d1e/d63/d99/ff4 0 2024-05-15T15:54:26.281 INFO:tasks.workunit.client.0.smithi049.stdout:5/899: unlink dc/dd/d47/da6/dce/fe2 0 2024-05-15T15:54:26.283 INFO:tasks.workunit.client.0.smithi049.stdout:5/900: write dc/dd/d7b/fdf [114945,80993] 0 2024-05-15T15:54:26.285 INFO:tasks.workunit.client.0.smithi049.stdout:1/933: dwrite d0/d4/dd/de/d48/d84/f12e [0,4194304] 0 2024-05-15T15:54:26.293 INFO:tasks.workunit.client.0.smithi049.stdout:9/661: mkdir d0/d8/d9/d52/dc7/de9 0 2024-05-15T15:54:26.300 INFO:tasks.workunit.client.0.smithi049.stdout:6/827: dwrite dd/d26/d5b/da8/fd9 [0,4194304] 0 2024-05-15T15:54:26.305 INFO:tasks.workunit.client.0.smithi049.stdout:8/777: truncate d2/de/d25/d8c/d9c/fd3 2360021 0 2024-05-15T15:54:26.307 INFO:tasks.workunit.client.0.smithi049.stdout:1/934: chown d0/d4/dd/de/d4d/l52 6354 1 2024-05-15T15:54:26.309 INFO:tasks.workunit.client.0.smithi049.stdout:4/740: creat d18/d1e/d2a/d50/dc4/ff5 x:0 0 0 2024-05-15T15:54:26.311 INFO:tasks.workunit.client.0.smithi049.stdout:7/759: creat d0/d6/da/fea x:0 0 0 2024-05-15T15:54:26.312 INFO:tasks.workunit.client.0.smithi049.stdout:7/760: chown d0/ddb/dde/dcc/f5b 0 1 2024-05-15T15:54:26.319 INFO:tasks.workunit.client.0.smithi049.stdout:5/901: dread dc/db0/dbf/fc5 [0,4194304] 0 2024-05-15T15:54:26.319 INFO:tasks.workunit.client.1.smithi144.stdout:6/893: chown d7/d1e/d28/da8/ldd 799256701 1 2024-05-15T15:54:26.320 INFO:tasks.workunit.client.1.smithi144.stdout:6/894: readlink d7/db/d9d/la4 0 2024-05-15T15:54:26.320 INFO:tasks.workunit.client.1.smithi144.stdout:6/895: fsync d7/db/d11/f1f 0 2024-05-15T15:54:26.321 INFO:tasks.workunit.client.1.smithi144.stdout:6/896: write d7/db/d3a/dd2/f116 [990369,58423] 0 2024-05-15T15:54:26.322 INFO:tasks.workunit.client.1.smithi144.stdout:6/897: fsync d7/d1e/d28/f3d 0 2024-05-15T15:54:26.322 INFO:tasks.workunit.client.0.smithi049.stdout:4/741: chown d18/d21/l25 12 1 2024-05-15T15:54:26.325 INFO:tasks.workunit.client.1.smithi144.stdout:0/823: mknod d4/d6/d4a/d4d/db1/c12f 0 2024-05-15T15:54:26.326 INFO:tasks.workunit.client.1.smithi144.stdout:8/942: dread - d2/fbe zero size 2024-05-15T15:54:26.327 INFO:tasks.workunit.client.1.smithi144.stdout:8/943: chown d2/d1a/cd7 3 1 2024-05-15T15:54:26.327 INFO:tasks.workunit.client.0.smithi049.stdout:6/828: mknod dd/d17/d24/d2b/d36/d86/d96/cfe 0 2024-05-15T15:54:26.330 INFO:tasks.workunit.client.0.smithi049.stdout:5/902: rename dc/d4a/ddd/dfb to dc/d4a/d9e/d12a 0 2024-05-15T15:54:26.334 INFO:tasks.workunit.client.0.smithi049.stdout:4/742: readlink d18/l1d 0 2024-05-15T15:54:26.335 INFO:tasks.workunit.client.0.smithi049.stdout:4/743: chown d18/d85/d98/ca0 0 1 2024-05-15T15:54:26.339 INFO:tasks.workunit.client.1.smithi144.stdout:9/877: write d0/d8/d5e/d44/d10/d9c/f98 [1996770,85486] 0 2024-05-15T15:54:26.340 INFO:tasks.workunit.client.1.smithi144.stdout:9/878: dread - d0/d8/d1b/d79/d8a/dae/f10a zero size 2024-05-15T15:54:26.344 INFO:tasks.workunit.client.0.smithi049.stdout:8/778: dwrite d2/de/d71/d86/f6d [0,4194304] 0 2024-05-15T15:54:26.347 INFO:tasks.workunit.client.0.smithi049.stdout:4/744: dread - d18/d1e/d2a/d36/f7e zero size 2024-05-15T15:54:26.348 INFO:tasks.workunit.client.0.smithi049.stdout:9/662: getdents d0/d8/d9/d52/d5e/d5f/d88/d9d 0 2024-05-15T15:54:26.348 INFO:tasks.workunit.client.0.smithi049.stdout:4/745: read - d18/d85/db6/ff1 zero size 2024-05-15T15:54:26.348 INFO:tasks.workunit.client.1.smithi144.stdout:2/801: dwrite d3/d7/d20/d37/f3c [0,4194304] 0 2024-05-15T15:54:26.349 INFO:tasks.workunit.client.0.smithi049.stdout:4/746: stat d18/d21/d2b/d4e/f87 0 2024-05-15T15:54:26.350 INFO:tasks.workunit.client.1.smithi144.stdout:2/802: write d3/d7/d20/d3b/f77 [4092655,36648] 0 2024-05-15T15:54:26.350 INFO:tasks.workunit.client.0.smithi049.stdout:7/761: mknod d0/d6/d23/d45/d6a/da1/d82/dd3/ceb 0 2024-05-15T15:54:26.352 INFO:tasks.workunit.client.1.smithi144.stdout:3/856: truncate d0/d1/d17/d39/dc8/fdc 1255979 0 2024-05-15T15:54:26.354 INFO:tasks.workunit.client.1.smithi144.stdout:0/824: creat d4/d2d/d100/f130 x:0 0 0 2024-05-15T15:54:26.359 INFO:tasks.workunit.client.1.smithi144.stdout:8/944: unlink d2/d24/d2d/da0/cb0 0 2024-05-15T15:54:26.359 INFO:tasks.workunit.client.1.smithi144.stdout:1/854: symlink d7/d1d/d33/d29/d6b/d5e/d74/df2/l111 0 2024-05-15T15:54:26.361 INFO:tasks.workunit.client.0.smithi049.stdout:7/762: dwrite d0/ddb/dde/dcc/fd1 [0,4194304] 0 2024-05-15T15:54:26.365 INFO:tasks.workunit.client.1.smithi144.stdout:8/945: dwrite d2/d1a/f3d [0,4194304] 0 2024-05-15T15:54:26.365 INFO:tasks.workunit.client.0.smithi049.stdout:9/663: rmdir d0/d8 39 2024-05-15T15:54:26.367 INFO:tasks.workunit.client.1.smithi144.stdout:4/894: creat d6/dd/d34/d3b/d30/db/f123 x:0 0 0 2024-05-15T15:54:26.369 INFO:tasks.workunit.client.1.smithi144.stdout:7/859: getdents d4/df/d25/d40/d73/d112 0 2024-05-15T15:54:26.371 INFO:tasks.workunit.client.0.smithi049.stdout:5/903: creat dc/f12b x:0 0 0 2024-05-15T15:54:26.380 INFO:tasks.workunit.client.1.smithi144.stdout:9/879: rmdir d0/d8/d1b/d79/d8a/dae 39 2024-05-15T15:54:26.382 INFO:tasks.workunit.client.0.smithi049.stdout:4/747: dread d18/f1a [0,4194304] 0 2024-05-15T15:54:26.384 INFO:tasks.workunit.client.0.smithi049.stdout:6/829: rename dd/d17/f21 to dd/d17/d24/fff 0 2024-05-15T15:54:26.386 INFO:tasks.workunit.client.0.smithi049.stdout:8/779: mknod d2/de/d71/d86/d5d/d7d/d1b/cf1 0 2024-05-15T15:54:26.392 INFO:tasks.workunit.client.1.smithi144.stdout:3/857: readlink d0/d1/le5 0 2024-05-15T15:54:26.392 INFO:tasks.workunit.client.0.smithi049.stdout:5/904: rmdir dc/db0/dbf 39 2024-05-15T15:54:26.394 INFO:tasks.workunit.client.1.smithi144.stdout:6/898: unlink d7/db/d11/f42 0 2024-05-15T15:54:26.395 INFO:tasks.workunit.client.1.smithi144.stdout:0/825: stat d4/df/d14/f42 0 2024-05-15T15:54:26.395 INFO:tasks.workunit.client.0.smithi049.stdout:6/830: rmdir dd/d17/d24/d8a/dc2 39 2024-05-15T15:54:26.396 INFO:tasks.workunit.client.0.smithi049.stdout:6/831: chown dd/d17/d24/d2b/f76 4840883 1 2024-05-15T15:54:26.398 INFO:tasks.workunit.client.0.smithi049.stdout:7/763: mknod d0/ddb/dde/dca/d5e/dbd/cec 0 2024-05-15T15:54:26.399 INFO:tasks.workunit.client.1.smithi144.stdout:8/946: creat d2/d24/d2d/da0/f136 x:0 0 0 2024-05-15T15:54:26.400 INFO:tasks.workunit.client.0.smithi049.stdout:9/664: creat d0/d8/d9/d52/d5e/d8a/dd3/d7d/fea x:0 0 0 2024-05-15T15:54:26.403 INFO:tasks.workunit.client.1.smithi144.stdout:7/860: mkdir d4/d3c/d76/d9d/dac/d111/d12c 0 2024-05-15T15:54:26.404 INFO:tasks.workunit.client.0.smithi049.stdout:4/748: mknod d18/d21/d2b/cf6 0 2024-05-15T15:54:26.407 INFO:tasks.workunit.client.0.smithi049.stdout:8/780: symlink d2/de/d25/d74/dbe/dd8/lf2 0 2024-05-15T15:54:26.411 INFO:tasks.workunit.client.1.smithi144.stdout:1/855: dread d7/d1d/d33/d55/f8d [0,4194304] 0 2024-05-15T15:54:26.412 INFO:tasks.workunit.client.0.smithi049.stdout:7/764: creat d0/d6/d23/d4a/d7b/fed x:0 0 0 2024-05-15T15:54:26.412 INFO:tasks.workunit.client.0.smithi049.stdout:5/905: write dc/d4a/d42/f5e [1802886,106694] 0 2024-05-15T15:54:26.413 INFO:tasks.workunit.client.1.smithi144.stdout:8/947: dread d2/de/fea [0,4194304] 0 2024-05-15T15:54:26.414 INFO:tasks.workunit.client.1.smithi144.stdout:8/948: fdatasync d2/da8/ffb 0 2024-05-15T15:54:26.419 INFO:tasks.workunit.client.1.smithi144.stdout:8/949: dread - d2/de/d3f/dff/d125/f12f zero size 2024-05-15T15:54:26.419 INFO:tasks.workunit.client.1.smithi144.stdout:2/803: mknod d3/d7/d20/c10a 0 2024-05-15T15:54:26.419 INFO:tasks.workunit.client.1.smithi144.stdout:2/804: stat d3/d7/df8/d65 0 2024-05-15T15:54:26.419 INFO:tasks.workunit.client.0.smithi049.stdout:5/906: stat dc/dd/d47/da6/dce/fd1 0 2024-05-15T15:54:26.420 INFO:tasks.workunit.client.0.smithi049.stdout:6/832: truncate dd/d17/d24/d2b/d36/d50/d54/dde/d8c/fba 2180623 0 2024-05-15T15:54:26.420 INFO:tasks.workunit.client.0.smithi049.stdout:6/833: dread - dd/f80 zero size 2024-05-15T15:54:26.420 INFO:tasks.workunit.client.0.smithi049.stdout:5/907: truncate dc/dd/d47/d83/dcc/d5c/f11f 560845 0 2024-05-15T15:54:26.420 INFO:tasks.workunit.client.0.smithi049.stdout:9/665: creat d0/d8/d9/d45/feb x:0 0 0 2024-05-15T15:54:26.421 INFO:tasks.workunit.client.1.smithi144.stdout:3/858: rename d0/d11/d3b/db5 to d0/d11/d3b/d93/d116 0 2024-05-15T15:54:26.426 INFO:tasks.workunit.client.0.smithi049.stdout:4/749: unlink d18/d1e/d2a/d50/l6f 0 2024-05-15T15:54:26.427 INFO:tasks.workunit.client.0.smithi049.stdout:8/781: dread d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fb4 [0,4194304] 0 2024-05-15T15:54:26.428 INFO:tasks.workunit.client.1.smithi144.stdout:4/895: symlink d6/dd/d34/d120/l124 0 2024-05-15T15:54:26.432 INFO:tasks.workunit.client.1.smithi144.stdout:7/861: mkdir d4/d3c/d76/d9d/dac/d48/d72/d82/d12d 0 2024-05-15T15:54:26.432 INFO:tasks.workunit.client.1.smithi144.stdout:7/862: chown d4/df/d25/d40/d62/la1 516 1 2024-05-15T15:54:26.434 INFO:tasks.workunit.client.1.smithi144.stdout:4/896: dread d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d44/f43 [0,4194304] 0 2024-05-15T15:54:26.434 INFO:tasks.workunit.client.0.smithi049.stdout:7/765: unlink d0/d6/d23/d45/d6a/da1/c8b 0 2024-05-15T15:54:26.435 INFO:tasks.workunit.client.1.smithi144.stdout:4/897: chown d6/dd/d34/d3b/d30/db/d1d/d40/fec 6 1 2024-05-15T15:54:26.435 INFO:tasks.workunit.client.1.smithi144.stdout:5/893: link d0/dc/d24/d8e/d93/d9e/fcb d0/dc/d2f/d34/f12a 0 2024-05-15T15:54:26.436 INFO:tasks.workunit.client.1.smithi144.stdout:5/894: write d0/dc/f1b [1014602,25139] 0 2024-05-15T15:54:26.437 INFO:tasks.workunit.client.0.smithi049.stdout:6/834: mknod dd/d17/d24/d2b/d36/d86/d96/c100 0 2024-05-15T15:54:26.440 INFO:tasks.workunit.client.1.smithi144.stdout:4/898: dwrite d6/dd/d34/d3b/d30/d59/f74 [0,4194304] 0 2024-05-15T15:54:26.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:26 smithi144 ceph-mon[36389]: pgmap v33: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 69 MiB/s rd, 182 MiB/s wr, 374 op/s 2024-05-15T15:54:26.456 INFO:tasks.workunit.client.1.smithi144.stdout:2/805: symlink d3/d4d/db4/l10b 0 2024-05-15T15:54:26.459 INFO:tasks.workunit.client.0.smithi049.stdout:6/835: symlink dd/d17/l101 0 2024-05-15T15:54:26.460 INFO:tasks.workunit.client.1.smithi144.stdout:3/859: fsync d0/f15 0 2024-05-15T15:54:26.463 INFO:tasks.workunit.client.1.smithi144.stdout:6/899: rename d7/db/f134 to d7/d32/f142 0 2024-05-15T15:54:26.466 INFO:tasks.workunit.client.1.smithi144.stdout:3/860: dwrite d0/d1/d56/d98/d9c/fe3 [4194304,4194304] 0 2024-05-15T15:54:26.469 INFO:tasks.workunit.client.0.smithi049.stdout:4/750: dread d18/f49 [0,4194304] 0 2024-05-15T15:54:26.470 INFO:tasks.workunit.client.1.smithi144.stdout:1/856: write d7/d1d/d33/d29/d6b/d5e/d74/df3/fdb [838745,18164] 0 2024-05-15T15:54:26.474 INFO:tasks.workunit.client.0.smithi049.stdout:7/766: write d0/f15 [581671,113140] 0 2024-05-15T15:54:26.474 INFO:tasks.workunit.client.0.smithi049.stdout:9/666: dwrite d0/d25/f4c [0,4194304] 0 2024-05-15T15:54:26.478 INFO:tasks.workunit.client.0.smithi049.stdout:6/836: creat dd/d17/d24/d2b/d36/d50/d54/f102 x:0 0 0 2024-05-15T15:54:26.479 INFO:tasks.workunit.client.0.smithi049.stdout:8/782: truncate d2/de/d71/d86/d5d/d7d/d45/f7c 2226116 0 2024-05-15T15:54:26.483 INFO:tasks.workunit.client.1.smithi144.stdout:5/895: rmdir d0/d10c 39 2024-05-15T15:54:26.488 INFO:tasks.workunit.client.0.smithi049.stdout:6/837: dread dd/d17/d24/d2b/d36/d50/d54/f5c [0,4194304] 0 2024-05-15T15:54:26.491 INFO:tasks.workunit.client.1.smithi144.stdout:6/900: creat d7/d1e/d28/da8/f143 x:0 0 0 2024-05-15T15:54:26.499 INFO:tasks.workunit.client.0.smithi049.stdout:6/838: truncate dd/f53 2328822 0 2024-05-15T15:54:26.500 INFO:tasks.workunit.client.1.smithi144.stdout:5/896: fsync d0/dc/d2f/d68/fe5 0 2024-05-15T15:54:26.501 INFO:tasks.workunit.client.0.smithi049.stdout:4/751: dread d18/f3e [0,4194304] 0 2024-05-15T15:54:26.503 INFO:tasks.workunit.client.0.smithi049.stdout:9/667: getdents d0/d8/d1b 0 2024-05-15T15:54:26.504 INFO:tasks.workunit.client.1.smithi144.stdout:9/880: creat d0/d8/d1b/d79/d8a/dae/f12c x:0 0 0 2024-05-15T15:54:26.506 INFO:tasks.workunit.client.1.smithi144.stdout:3/861: write d0/d11/d3b/d93/d116/fe9 [187493,30641] 0 2024-05-15T15:54:26.507 INFO:tasks.workunit.client.0.smithi049.stdout:8/783: dwrite d2/de/d71/d86/d5d/d7d/d1b/d68/f8e [0,4194304] 0 2024-05-15T15:54:26.510 INFO:tasks.workunit.client.0.smithi049.stdout:4/752: creat d18/d1e/d63/d99/de1/ff7 x:0 0 0 2024-05-15T15:54:26.510 INFO:tasks.workunit.client.0.smithi049.stdout:9/668: dread d0/d25/f4c [0,4194304] 0 2024-05-15T15:54:26.515 INFO:tasks.workunit.client.1.smithi144.stdout:4/899: truncate d6/fe7 8204618 0 2024-05-15T15:54:26.518 INFO:tasks.workunit.client.0.smithi049.stdout:6/839: mkdir dd/d17/d24/d2b/d36/d50/d54/dde/df2/d103 0 2024-05-15T15:54:26.518 INFO:tasks.workunit.client.1.smithi144.stdout:2/806: dwrite d3/d7/d20/d37/f46 [0,4194304] 0 2024-05-15T15:54:26.520 INFO:tasks.workunit.client.0.smithi049.stdout:6/840: read dd/d17/d24/d2b/d36/f42 [556509,33232] 0 2024-05-15T15:54:26.522 INFO:tasks.workunit.client.0.smithi049.stdout:8/784: creat d2/d6a/ff3 x:0 0 0 2024-05-15T15:54:26.528 INFO:tasks.workunit.client.1.smithi144.stdout:5/897: mknod d0/dc/d2f/d34/dc1/c12b 0 2024-05-15T15:54:26.528 INFO:tasks.workunit.client.1.smithi144.stdout:6/901: dwrite d7/db/d11/d3c/f4d [0,4194304] 0 2024-05-15T15:54:26.529 INFO:tasks.workunit.client.0.smithi049.stdout:7/767: creat d0/d6/d23/d4a/d7b/de2/fee x:0 0 0 2024-05-15T15:54:26.530 INFO:tasks.workunit.client.1.smithi144.stdout:3/862: stat d0/d11/d3b/da0/d8f/f9b 0 2024-05-15T15:54:26.530 INFO:tasks.workunit.client.1.smithi144.stdout:6/902: chown d7/db/d35/f12b 30538 1 2024-05-15T15:54:26.531 INFO:tasks.workunit.client.0.smithi049.stdout:6/841: creat dd/d26/d5b/da8/f104 x:0 0 0 2024-05-15T15:54:26.532 INFO:tasks.workunit.client.1.smithi144.stdout:4/900: symlink d6/dd/d34/d3b/d30/dc9/l125 0 2024-05-15T15:54:26.533 INFO:tasks.workunit.client.0.smithi049.stdout:6/842: write dd/d17/d9d/da1/fee [941593,51780] 0 2024-05-15T15:54:26.533 INFO:tasks.workunit.client.0.smithi049.stdout:8/785: rmdir d2/de/d71/d86/d5d/d7d/db8 39 2024-05-15T15:54:26.537 INFO:tasks.workunit.client.1.smithi144.stdout:7/863: link d4/df/d25/d40/l50 d4/d3c/d76/d9d/dac/de5/l12e 0 2024-05-15T15:54:26.539 INFO:tasks.workunit.client.1.smithi144.stdout:1/857: truncate d7/d1d/d33/d41/d78/da3/f88 635428 0 2024-05-15T15:54:26.542 INFO:tasks.workunit.client.0.smithi049.stdout:7/768: unlink d0/d6/d23/f4c 0 2024-05-15T15:54:26.544 INFO:tasks.workunit.client.0.smithi049.stdout:4/753: write d18/d1e/d63/d99/de1/f47 [1285559,116948] 0 2024-05-15T15:54:26.546 INFO:tasks.workunit.client.0.smithi049.stdout:4/754: write d18/fd9 [912204,80046] 0 2024-05-15T15:54:26.546 INFO:tasks.workunit.client.0.smithi049.stdout:9/669: write d0/d8/d21/fa4 [13227,107731] 0 2024-05-15T15:54:26.547 INFO:tasks.workunit.client.0.smithi049.stdout:9/670: readlink d0/d8/d12/l3f 0 2024-05-15T15:54:26.551 INFO:tasks.workunit.client.1.smithi144.stdout:6/903: rename d7/db/d11/d3c/c5c to d7/d1e/d28/db5/df6/c144 0 2024-05-15T15:54:26.552 INFO:tasks.workunit.client.1.smithi144.stdout:4/901: dread d6/dd/d34/d3b/d30/d59/fe1 [0,4194304] 0 2024-05-15T15:54:26.554 INFO:tasks.workunit.client.1.smithi144.stdout:4/902: chown d6/dd/d34/d3b/d30/d59/fe1 3218 1 2024-05-15T15:54:26.554 INFO:tasks.workunit.client.1.smithi144.stdout:2/807: mknod d3/c10c 0 2024-05-15T15:54:26.557 INFO:tasks.workunit.client.1.smithi144.stdout:6/904: dwrite d7/d1e/d28/da8/f143 [0,4194304] 0 2024-05-15T15:54:26.558 INFO:tasks.workunit.client.1.smithi144.stdout:6/905: fdatasync d7/db/d9d/d97/f127 0 2024-05-15T15:54:26.559 INFO:tasks.workunit.client.0.smithi049.stdout:7/769: symlink d0/d6/lef 0 2024-05-15T15:54:26.560 INFO:tasks.workunit.client.0.smithi049.stdout:8/786: write d2/de/fa3 [761750,50031] 0 2024-05-15T15:54:26.562 INFO:tasks.workunit.client.1.smithi144.stdout:0/826: sync 2024-05-15T15:54:26.562 INFO:tasks.workunit.client.1.smithi144.stdout:8/950: sync 2024-05-15T15:54:26.562 INFO:tasks.workunit.client.0.smithi049.stdout:6/843: write dd/d17/d24/d2b/d36/d50/d54/f7b [292227,38720] 0 2024-05-15T15:54:26.564 INFO:tasks.workunit.client.1.smithi144.stdout:9/881: creat d0/d8/d5e/d44/d16/f12d x:0 0 0 2024-05-15T15:54:26.567 INFO:tasks.workunit.client.1.smithi144.stdout:5/898: dwrite d0/dc/d24/d8e/d93/d9e/fcb [0,4194304] 0 2024-05-15T15:54:26.569 INFO:tasks.workunit.client.1.smithi144.stdout:5/899: write d0/dc/d24/d28/d2d/f2b [1562943,103041] 0 2024-05-15T15:54:26.571 INFO:tasks.workunit.client.1.smithi144.stdout:1/858: write d7/d1d/d33/d1e/d34/f76 [90070,41360] 0 2024-05-15T15:54:26.572 INFO:tasks.workunit.client.0.smithi049.stdout:4/755: creat d18/d1e/ff8 x:0 0 0 2024-05-15T15:54:26.572 INFO:tasks.workunit.client.1.smithi144.stdout:1/859: write d7/d1d/d33/d1e/f25 [1610773,126254] 0 2024-05-15T15:54:26.576 INFO:tasks.workunit.client.1.smithi144.stdout:2/808: creat d3/d7/d54/f10d x:0 0 0 2024-05-15T15:54:26.577 INFO:tasks.workunit.client.0.smithi049.stdout:7/770: readlink d0/d6/d23/d45/d6a/da1/l20 0 2024-05-15T15:54:26.578 INFO:tasks.workunit.client.1.smithi144.stdout:7/864: creat d4/d3c/d76/d9d/dac/d111/d12c/f12f x:0 0 0 2024-05-15T15:54:26.581 INFO:tasks.workunit.client.0.smithi049.stdout:8/787: write d2/de/d71/fba [1297665,99718] 0 2024-05-15T15:54:26.585 INFO:tasks.workunit.client.1.smithi144.stdout:3/863: dwrite d0/d1/fe [4194304,4194304] 0 2024-05-15T15:54:26.587 INFO:tasks.workunit.client.0.smithi049.stdout:6/844: creat dd/f105 x:0 0 0 2024-05-15T15:54:26.589 INFO:tasks.workunit.client.1.smithi144.stdout:0/827: creat d4/df/d1f/d24/d96/f131 x:0 0 0 2024-05-15T15:54:26.589 INFO:tasks.workunit.client.0.smithi049.stdout:6/845: write dd/d17/d24/d2b/d36/d50/d54/f102 [187061,34921] 0 2024-05-15T15:54:26.592 INFO:tasks.workunit.client.1.smithi144.stdout:9/882: write d0/d8/d5e/d44/d16/db0/f11d [988932,44529] 0 2024-05-15T15:54:26.595 INFO:tasks.workunit.client.0.smithi049.stdout:6/846: mkdir dd/d26/d106 0 2024-05-15T15:54:26.596 INFO:tasks.workunit.client.1.smithi144.stdout:6/906: dwrite d7/d1e/d28/fab [0,4194304] 0 2024-05-15T15:54:26.597 INFO:tasks.workunit.client.0.smithi049.stdout:6/847: stat dd/d17/d24/d2b/ld1 0 2024-05-15T15:54:26.598 INFO:tasks.workunit.client.0.smithi049.stdout:4/756: creat d18/d1e/d2a/d36/ff9 x:0 0 0 2024-05-15T15:54:26.599 INFO:tasks.workunit.client.0.smithi049.stdout:7/771: mknod d0/d6/d23/d45/d6a/da1/d80/cf0 0 2024-05-15T15:54:26.601 INFO:tasks.workunit.client.0.smithi049.stdout:6/848: rename dd/d17/d24/d67/f84 to dd/d17/d9d/f107 0 2024-05-15T15:54:26.602 INFO:tasks.workunit.client.1.smithi144.stdout:1/860: mknod d7/d1d/d33/d1e/d34/d9c/c112 0 2024-05-15T15:54:26.602 INFO:tasks.workunit.client.0.smithi049.stdout:6/849: write dd/d17/d24/d58/d5f/f79 [1152272,54995] 0 2024-05-15T15:54:26.605 INFO:tasks.workunit.client.0.smithi049.stdout:7/772: fdatasync d0/d2f/d5c/f6e 0 2024-05-15T15:54:26.606 INFO:tasks.workunit.client.1.smithi144.stdout:7/865: mkdir d4/df/d25/d38/d5d/d130 0 2024-05-15T15:54:26.609 INFO:tasks.workunit.client.1.smithi144.stdout:8/951: unlink d2/d24/d2d/da0/d6e/d71/d73/c85 0 2024-05-15T15:54:26.609 INFO:tasks.workunit.client.1.smithi144.stdout:8/952: chown d2/d3a/d133 83 1 2024-05-15T15:54:26.613 INFO:tasks.workunit.client.1.smithi144.stdout:3/864: rename d0/d1/d17/d39/d83 to d0/d11/d3b/da7/def/d117 0 2024-05-15T15:54:26.614 INFO:tasks.workunit.client.1.smithi144.stdout:0/828: mkdir d4/df/da0/d132 0 2024-05-15T15:54:26.615 INFO:tasks.workunit.client.0.smithi049.stdout:6/850: symlink dd/d17/d24/d2b/d36/d50/d54/d92/l108 0 2024-05-15T15:54:26.615 INFO:tasks.workunit.client.1.smithi144.stdout:8/953: dwrite d2/d24/fd4 [0,4194304] 0 2024-05-15T15:54:26.616 INFO:tasks.workunit.client.0.smithi049.stdout:6/851: write dd/d26/d5b/da8/f104 [837986,22782] 0 2024-05-15T15:54:26.622 INFO:tasks.workunit.client.0.smithi049.stdout:6/852: truncate dd/d17/d24/d2b/d36/d86/d96/f9c 798963 0 2024-05-15T15:54:26.622 INFO:tasks.workunit.client.0.smithi049.stdout:4/757: getdents d18/d1e/d63/d99/de1/da8 0 2024-05-15T15:54:26.623 INFO:tasks.workunit.client.1.smithi144.stdout:1/861: unlink d7/d1d/d33/d1e/d4e/fb6 0 2024-05-15T15:54:26.623 INFO:tasks.workunit.client.1.smithi144.stdout:6/907: dread d7/db/d9d/f103 [0,4194304] 0 2024-05-15T15:54:26.625 INFO:tasks.workunit.client.1.smithi144.stdout:1/862: write d7/d1d/d33/d1e/d34/fe9 [536489,107239] 0 2024-05-15T15:54:26.625 INFO:tasks.workunit.client.1.smithi144.stdout:7/866: mkdir d4/df/d81/dfb/dbd/d131 0 2024-05-15T15:54:26.625 INFO:tasks.workunit.client.0.smithi049.stdout:6/853: dread dd/d17/f4a [0,4194304] 0 2024-05-15T15:54:26.626 INFO:tasks.workunit.client.1.smithi144.stdout:1/863: chown d7/d1d/d33/d29/d6b/l5f 1 1 2024-05-15T15:54:26.633 INFO:tasks.workunit.client.1.smithi144.stdout:1/864: dwrite d7/d1d/d58/fa6 [0,4194304] 0 2024-05-15T15:54:26.642 INFO:tasks.workunit.client.1.smithi144.stdout:6/908: rmdir d7/db/d9d/d97 39 2024-05-15T15:54:26.644 INFO:tasks.workunit.client.1.smithi144.stdout:7/867: creat d4/d3c/d76/d9d/dc5/f132 x:0 0 0 2024-05-15T15:54:26.648 INFO:tasks.workunit.client.1.smithi144.stdout:8/954: creat d2/d24/d2d/da0/d6e/d71/db1/f137 x:0 0 0 2024-05-15T15:54:26.649 INFO:tasks.workunit.client.1.smithi144.stdout:6/909: mknod d7/db/d11/d7e/c145 0 2024-05-15T15:54:26.650 INFO:tasks.workunit.client.0.smithi049.stdout:6/854: write dd/d17/d24/d2b/d36/f42 [1075719,83328] 0 2024-05-15T15:54:26.652 INFO:tasks.workunit.client.1.smithi144.stdout:9/883: truncate d0/d8/d1b/d11c/ffa 1554289 0 2024-05-15T15:54:26.654 INFO:tasks.workunit.client.1.smithi144.stdout:1/865: mkdir d7/d1d/d33/d29/dd2/dea/d113 0 2024-05-15T15:54:26.654 INFO:tasks.workunit.client.0.smithi049.stdout:4/758: write d18/d21/d2b/d4e/f87 [702769,82980] 0 2024-05-15T15:54:26.654 INFO:tasks.workunit.client.1.smithi144.stdout:1/866: fdatasync d7/d1d/d33/d55/f108 0 2024-05-15T15:54:26.655 INFO:tasks.workunit.client.1.smithi144.stdout:7/868: mkdir d4/d133 0 2024-05-15T15:54:26.658 INFO:tasks.workunit.client.1.smithi144.stdout:3/865: link d0/d11/d3b/da7/fec d0/d1/d17/da1/f118 0 2024-05-15T15:54:26.659 INFO:tasks.workunit.client.1.smithi144.stdout:9/884: dwrite d0/d8/d1b/d79/de7/df9/f10c [0,4194304] 0 2024-05-15T15:54:26.660 INFO:tasks.workunit.client.1.smithi144.stdout:9/885: fdatasync d0/d8/d1b/d79/de7/df9/f11e 0 2024-05-15T15:54:26.661 INFO:tasks.workunit.client.1.smithi144.stdout:1/867: creat d7/d1d/d33/d1e/d34/f114 x:0 0 0 2024-05-15T15:54:26.663 INFO:tasks.workunit.client.1.smithi144.stdout:7/869: creat d4/df/d25/d40/d73/f134 x:0 0 0 2024-05-15T15:54:26.664 INFO:tasks.workunit.client.0.smithi049.stdout:6/855: getdents dd/d17/d24/d2b/d36/d50/d54/dde 0 2024-05-15T15:54:26.665 INFO:tasks.workunit.client.1.smithi144.stdout:8/955: dread d2/de/d49/f69 [0,4194304] 0 2024-05-15T15:54:26.666 INFO:tasks.workunit.client.1.smithi144.stdout:9/886: mkdir d0/d8/d1b/d62/d7d/dd2/d12e 0 2024-05-15T15:54:26.669 INFO:tasks.workunit.client.0.smithi049.stdout:4/759: getdents d18/d1e/d2a/d50/dbc 0 2024-05-15T15:54:26.669 INFO:tasks.workunit.client.1.smithi144.stdout:1/868: dwrite d7/d1d/d58/fa6 [0,4194304] 0 2024-05-15T15:54:26.671 INFO:tasks.workunit.client.0.smithi049.stdout:6/856: symlink dd/d17/d24/d2b/d36/d86/d96/l109 0 2024-05-15T15:54:26.673 INFO:tasks.workunit.client.0.smithi049.stdout:6/857: read - dd/d17/d24/d58/d5f/dd7/fd4 zero size 2024-05-15T15:54:26.679 INFO:tasks.workunit.client.1.smithi144.stdout:6/910: getdents d7/d1e 0 2024-05-15T15:54:26.681 INFO:tasks.workunit.client.1.smithi144.stdout:7/870: mknod d4/df/d25/d38/d68/c135 0 2024-05-15T15:54:26.682 INFO:tasks.workunit.client.1.smithi144.stdout:7/871: fsync d4/d3c/d76/d9d/dac/d48/f126 0 2024-05-15T15:54:26.682 INFO:tasks.workunit.client.0.smithi049.stdout:6/858: symlink dd/d17/d24/d2b/d36/d50/d54/d92/de5/df8/l10a 0 2024-05-15T15:54:26.684 INFO:tasks.workunit.client.1.smithi144.stdout:1/869: mkdir d7/d115 0 2024-05-15T15:54:26.686 INFO:tasks.workunit.client.0.smithi049.stdout:6/859: creat dd/dbc/f10b x:0 0 0 2024-05-15T15:54:26.686 INFO:tasks.workunit.client.1.smithi144.stdout:6/911: dread d7/db/d35/f67 [0,4194304] 0 2024-05-15T15:54:26.689 INFO:tasks.workunit.client.1.smithi144.stdout:1/870: dread d7/d1d/d33/d29/d6b/d5e/d74/df3/fdb [0,4194304] 0 2024-05-15T15:54:26.690 INFO:tasks.workunit.client.1.smithi144.stdout:1/871: read - d7/d1d/d33/d29/d6b/d8e/fca zero size 2024-05-15T15:54:26.690 INFO:tasks.workunit.client.0.smithi049.stdout:4/760: dwrite d18/d1e/d2a/f4f [0,4194304] 0 2024-05-15T15:54:26.691 INFO:tasks.workunit.client.1.smithi144.stdout:9/887: getdents d0/d8/d7a 0 2024-05-15T15:54:26.692 INFO:tasks.workunit.client.1.smithi144.stdout:8/956: dread d2/d24/d2d/d30/dd0/df7/f17 [0,4194304] 0 2024-05-15T15:54:26.692 INFO:tasks.workunit.client.1.smithi144.stdout:8/957: readlink d2/l12c 0 2024-05-15T15:54:26.697 INFO:tasks.workunit.client.1.smithi144.stdout:7/872: write d4/d41/d5b/f80 [886359,66464] 0 2024-05-15T15:54:26.698 INFO:tasks.workunit.client.0.smithi049.stdout:6/860: write dd/d26/f40 [1855524,41465] 0 2024-05-15T15:54:26.701 INFO:tasks.workunit.client.1.smithi144.stdout:6/912: write d7/db/d35/d84/d81/fa0 [573266,101068] 0 2024-05-15T15:54:26.704 INFO:tasks.workunit.client.1.smithi144.stdout:9/888: write d0/d8/d1b/d62/fca [10021481,83838] 0 2024-05-15T15:54:26.706 INFO:tasks.workunit.client.1.smithi144.stdout:9/889: dread - d0/d8/d5e/d44/d10/d19/f113 zero size 2024-05-15T15:54:26.707 INFO:tasks.workunit.client.1.smithi144.stdout:8/958: getdents d2/d24/d2d/da0/df8 0 2024-05-15T15:54:26.709 INFO:tasks.workunit.client.0.smithi049.stdout:6/861: dread dd/d17/d24/d2b/d36/f42 [0,4194304] 0 2024-05-15T15:54:26.709 INFO:tasks.workunit.client.1.smithi144.stdout:6/913: symlink d7/d62/l146 0 2024-05-15T15:54:26.711 INFO:tasks.workunit.client.1.smithi144.stdout:6/914: unlink d7/db/d11/c24 0 2024-05-15T15:54:26.711 INFO:tasks.workunit.client.1.smithi144.stdout:4/903: sync 2024-05-15T15:54:26.711 INFO:tasks.workunit.client.1.smithi144.stdout:5/900: sync 2024-05-15T15:54:26.712 INFO:tasks.workunit.client.0.smithi049.stdout:6/862: fsync dd/d17/d24/d2b/d36/d50/fca 0 2024-05-15T15:54:26.713 INFO:tasks.workunit.client.1.smithi144.stdout:8/959: getdents d2/d24/d2d/da0/d6e/d71/d73 0 2024-05-15T15:54:26.714 INFO:tasks.workunit.client.1.smithi144.stdout:8/960: chown d2/d24/d2d/d30 29 1 2024-05-15T15:54:26.717 INFO:tasks.workunit.client.1.smithi144.stdout:7/873: write d4/df/d79/fcc [1028778,58801] 0 2024-05-15T15:54:26.718 INFO:tasks.workunit.client.0.smithi049.stdout:6/863: truncate dd/d17/d24/d2b/d36/d50/d54/dde/fbd 403975 0 2024-05-15T15:54:26.718 INFO:tasks.workunit.client.1.smithi144.stdout:9/890: write d0/f5 [1196559,111443] 0 2024-05-15T15:54:26.719 INFO:tasks.workunit.client.0.smithi049.stdout:6/864: chown dd/ce 134 1 2024-05-15T15:54:26.719 INFO:tasks.workunit.client.1.smithi144.stdout:6/915: dwrite d7/db/da2/f5e [0,4194304] 0 2024-05-15T15:54:26.719 INFO:tasks.workunit.client.0.smithi049.stdout:6/865: dread - dd/f105 zero size 2024-05-15T15:54:26.720 INFO:tasks.workunit.client.1.smithi144.stdout:7/874: stat d4/df/d25/d38/de4 0 2024-05-15T15:54:26.724 INFO:tasks.workunit.client.1.smithi144.stdout:4/904: creat d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/dab/dd5/f126 x:0 0 0 2024-05-15T15:54:26.724 INFO:tasks.workunit.client.1.smithi144.stdout:1/872: dwrite d7/d1d/d33/d41/d78/da3/f88 [0,4194304] 0 2024-05-15T15:54:26.726 INFO:tasks.workunit.client.0.smithi049.stdout:6/866: creat dd/d17/d24/d2b/d36/d50/d7e/f10c x:0 0 0 2024-05-15T15:54:26.727 INFO:tasks.workunit.client.0.smithi049.stdout:6/867: dread - dd/d17/d24/d2b/d36/d86/fea zero size 2024-05-15T15:54:26.728 INFO:tasks.workunit.client.1.smithi144.stdout:8/961: write d2/d24/d2d/d30/f7d [3698117,86597] 0 2024-05-15T15:54:26.729 INFO:tasks.workunit.client.1.smithi144.stdout:6/916: mknod d7/db/d3a/dd2/c147 0 2024-05-15T15:54:26.730 INFO:tasks.workunit.client.1.smithi144.stdout:6/917: readlink d7/d1e/l46 0 2024-05-15T15:54:26.731 INFO:tasks.workunit.client.1.smithi144.stdout:7/875: rmdir d4/df/d79/d100 39 2024-05-15T15:54:26.732 INFO:tasks.workunit.client.1.smithi144.stdout:6/918: dread - d7/d32/fde zero size 2024-05-15T15:54:26.733 INFO:tasks.workunit.client.1.smithi144.stdout:4/905: mknod d6/dd/d34/d120/c127 0 2024-05-15T15:54:26.734 INFO:tasks.workunit.client.1.smithi144.stdout:7/876: stat d4/d3c/c58 0 2024-05-15T15:54:26.735 INFO:tasks.workunit.client.1.smithi144.stdout:1/873: mknod d7/d4b/c116 0 2024-05-15T15:54:26.736 INFO:tasks.workunit.client.1.smithi144.stdout:8/962: symlink d2/d24/d2d/da0/d9e/db8/l138 0 2024-05-15T15:54:26.737 INFO:tasks.workunit.client.1.smithi144.stdout:6/919: dread - d7/db/d9d/db7/ffa zero size 2024-05-15T15:54:26.737 INFO:tasks.workunit.client.1.smithi144.stdout:5/901: dwrite d0/dc/d12/fab [0,4194304] 0 2024-05-15T15:54:26.739 INFO:tasks.workunit.client.1.smithi144.stdout:8/963: rmdir d2/de/d49 39 2024-05-15T15:54:26.743 INFO:tasks.workunit.client.1.smithi144.stdout:7/877: creat d4/df/d25/d40/d73/d112/f136 x:0 0 0 2024-05-15T15:54:26.745 INFO:tasks.workunit.client.0.smithi049.stdout:0/777: sync 2024-05-15T15:54:26.747 INFO:tasks.workunit.client.1.smithi144.stdout:4/906: dwrite d6/d49/f5c [8388608,4194304] 0 2024-05-15T15:54:26.747 INFO:tasks.workunit.client.1.smithi144.stdout:1/874: rename d7/d1d/dac/de1/ff7 to d7/d1d/d33/d29/d6b/d5e/f117 0 2024-05-15T15:54:26.749 INFO:tasks.workunit.client.0.smithi049.stdout:0/778: mknod dc/d13/c106 0 2024-05-15T15:54:26.750 INFO:tasks.workunit.client.1.smithi144.stdout:1/875: chown d7 625 1 2024-05-15T15:54:26.751 INFO:tasks.workunit.client.1.smithi144.stdout:6/920: dwrite d7/db/d11/fa1 [0,4194304] 0 2024-05-15T15:54:26.752 INFO:tasks.workunit.client.1.smithi144.stdout:1/876: readlink d7/d1d/d33/d41/l84 0 2024-05-15T15:54:26.753 INFO:tasks.workunit.client.1.smithi144.stdout:5/902: rename d0/dc/d24/d8e/d93/d9e/d127 to d0/dc/d24/d8e/d93/d6b/d12c 0 2024-05-15T15:54:26.753 INFO:tasks.workunit.client.1.smithi144.stdout:1/877: stat d7/d1d/lf0 0 2024-05-15T15:54:26.755 INFO:tasks.workunit.client.1.smithi144.stdout:1/878: truncate d7/d1d/d33/d29/dcd/f109 1004298 0 2024-05-15T15:54:26.764 INFO:tasks.workunit.client.1.smithi144.stdout:9/891: dwrite d0/d8/d5e/d44/dc2/f45 [8388608,4194304] 0 2024-05-15T15:54:26.766 INFO:tasks.workunit.client.1.smithi144.stdout:6/921: fsync d7/db/d11/d3c/f11d 0 2024-05-15T15:54:26.768 INFO:tasks.workunit.client.0.smithi049.stdout:6/868: dread dd/f4c [0,4194304] 0 2024-05-15T15:54:26.770 INFO:tasks.workunit.client.1.smithi144.stdout:8/964: write d2/de/f103 [4325,30768] 0 2024-05-15T15:54:26.770 INFO:tasks.workunit.client.0.smithi049.stdout:6/869: write dd/d17/d24/d2b/d36/d50/d54/faf [593671,86102] 0 2024-05-15T15:54:26.772 INFO:tasks.workunit.client.1.smithi144.stdout:5/903: mknod d0/dc/d24/d28/d2d/d59/c12d 0 2024-05-15T15:54:26.779 INFO:tasks.workunit.client.0.smithi049.stdout:6/870: rmdir dd/d17/d24/d2b/d36/d50/d54/dde/df2 39 2024-05-15T15:54:26.785 INFO:tasks.workunit.client.1.smithi144.stdout:4/907: dwrite d6/dd/d34/d3b/d30/db/d1d/d40/f97 [0,4194304] 0 2024-05-15T15:54:26.786 INFO:tasks.workunit.client.1.smithi144.stdout:7/878: truncate d4/df/d25/f35 1189651 0 2024-05-15T15:54:26.786 INFO:tasks.workunit.client.0.smithi049.stdout:0/779: dwrite dc/d13/d14/d25/f69 [0,4194304] 0 2024-05-15T15:54:26.786 INFO:tasks.workunit.client.1.smithi144.stdout:7/879: readlink d4/l83 0 2024-05-15T15:54:26.787 INFO:tasks.workunit.client.0.smithi049.stdout:6/871: dwrite dd/f57 [0,4194304] 0 2024-05-15T15:54:26.787 INFO:tasks.workunit.client.1.smithi144.stdout:5/904: mknod d0/dc/d24/d8e/d93/d6b/dea/dfa/c12e 0 2024-05-15T15:54:26.788 INFO:tasks.workunit.client.0.smithi049.stdout:0/780: dread - dc/d13/d14/d25/d31/dae/dc5/d57/f104 zero size 2024-05-15T15:54:26.788 INFO:tasks.workunit.client.1.smithi144.stdout:9/892: write d0/d8/d1b/d79/d8a/f97 [4837615,2871] 0 2024-05-15T15:54:26.789 INFO:tasks.workunit.client.1.smithi144.stdout:5/905: dread - d0/dc/d24/d8e/d93/d6b/dea/ffd zero size 2024-05-15T15:54:26.790 INFO:tasks.workunit.client.1.smithi144.stdout:4/908: read d6/dd/d34/d3b/f21 [3454094,76783] 0 2024-05-15T15:54:26.790 INFO:tasks.workunit.client.1.smithi144.stdout:1/879: dread d7/df/f10 [0,4194304] 0 2024-05-15T15:54:26.791 INFO:tasks.workunit.client.1.smithi144.stdout:8/965: write d2/d24/d2d/d30/dd0/df7/d87/fda [6229146,62538] 0 2024-05-15T15:54:26.793 INFO:tasks.workunit.client.1.smithi144.stdout:5/906: chown d0/dc/d2f/laf 105929660 1 2024-05-15T15:54:26.794 INFO:tasks.workunit.client.1.smithi144.stdout:4/909: chown d6/dd/d34/d3b/d30/dc9/d10f/d4b 2 1 2024-05-15T15:54:26.796 INFO:tasks.workunit.client.0.smithi049.stdout:6/872: dwrite dd/d17/d24/d2b/d36/f98 [0,4194304] 0 2024-05-15T15:54:26.798 INFO:tasks.workunit.client.1.smithi144.stdout:4/910: write d6/dd/d34/d3b/d30/dc9/d10f/d4b/fdb [1136545,103679] 0 2024-05-15T15:54:26.799 INFO:tasks.workunit.client.0.smithi049.stdout:6/873: chown dd/d17/d24/d2b/d36/d50/d54/d92 132922884 1 2024-05-15T15:54:26.799 INFO:tasks.workunit.client.1.smithi144.stdout:8/966: symlink d2/de/d49/d83/l139 0 2024-05-15T15:54:26.800 INFO:tasks.workunit.client.1.smithi144.stdout:4/911: dread - d6/dd/d2e/d75/dff/deb/dba/f11c zero size 2024-05-15T15:54:26.801 INFO:tasks.workunit.client.1.smithi144.stdout:9/893: creat d0/d8/d5e/d44/dc2/f12f x:0 0 0 2024-05-15T15:54:26.803 INFO:tasks.workunit.client.1.smithi144.stdout:5/907: fsync d0/dc/d24/d28/d2d/df7/f108 0 2024-05-15T15:54:26.804 INFO:tasks.workunit.client.1.smithi144.stdout:8/967: dread - d2/d24/d2d/da0/d6e/d71/d73/d98/f120 zero size 2024-05-15T15:54:26.805 INFO:tasks.workunit.client.0.smithi049.stdout:0/781: mknod dc/d13/c107 0 2024-05-15T15:54:26.805 INFO:tasks.workunit.client.1.smithi144.stdout:4/912: readlink d6/dd/d34/d3b/d30/d59/d66/l6a 0 2024-05-15T15:54:26.806 INFO:tasks.workunit.client.1.smithi144.stdout:8/968: fsync d2/d24/d2d/da0/d6e/deb/f124 0 2024-05-15T15:54:26.807 INFO:tasks.workunit.client.1.smithi144.stdout:8/969: fsync d2/d24/d2d/da0/df8/d10a/f128 0 2024-05-15T15:54:26.808 INFO:tasks.workunit.client.1.smithi144.stdout:9/894: creat d0/d8/f130 x:0 0 0 2024-05-15T15:54:26.809 INFO:tasks.workunit.client.1.smithi144.stdout:8/970: fsync d2/d24/d2d/da0/d6e/d71/db1/fd1 0 2024-05-15T15:54:26.810 INFO:tasks.workunit.client.1.smithi144.stdout:5/908: creat d0/dc/d2f/d79/da5/f12f x:0 0 0 2024-05-15T15:54:26.810 INFO:tasks.workunit.client.1.smithi144.stdout:8/971: chown d2/f96 170 1 2024-05-15T15:54:26.811 INFO:tasks.workunit.client.1.smithi144.stdout:8/972: dread - d2/d24/d2d/da0/d9e/db8/dde/f11e zero size 2024-05-15T15:54:26.812 INFO:tasks.workunit.client.1.smithi144.stdout:8/973: chown d2/d24/d2d/d30/dd0/df7/cab 4945 1 2024-05-15T15:54:26.812 INFO:tasks.workunit.client.0.smithi049.stdout:0/782: rename dc/d13/d14/d33 to dc/d13/d14/d25/d31/dae/dc5/d108 0 2024-05-15T15:54:26.813 INFO:tasks.workunit.client.1.smithi144.stdout:6/922: dread d7/f4b [0,4194304] 0 2024-05-15T15:54:26.813 INFO:tasks.workunit.client.1.smithi144.stdout:8/974: read d2/de/d49/d83/f99 [2176857,122012] 0 2024-05-15T15:54:26.814 INFO:tasks.workunit.client.1.smithi144.stdout:6/923: chown d7/d32/c36 924700919 1 2024-05-15T15:54:26.814 INFO:tasks.workunit.client.0.smithi049.stdout:6/874: mknod dd/d17/d24/d2b/d36/d50/d54/dde/df2/d103/c10d 0 2024-05-15T15:54:26.815 INFO:tasks.workunit.client.0.smithi049.stdout:6/875: stat dd/d17/d24/d2b/d36/l49 0 2024-05-15T15:54:26.816 INFO:tasks.workunit.client.0.smithi049.stdout:6/876: write dd/d26/f40 [4272702,60286] 0 2024-05-15T15:54:26.817 INFO:tasks.workunit.client.0.smithi049.stdout:6/877: readlink dd/d17/d24/d2b/ld2 0 2024-05-15T15:54:26.819 INFO:tasks.workunit.client.0.smithi049.stdout:6/878: chown dd/d17/d24/d2b/d36/d50/d54/d92/de5/ff3 17562 1 2024-05-15T15:54:26.820 INFO:tasks.workunit.client.0.smithi049.stdout:0/783: mkdir dc/de9/d109 0 2024-05-15T15:54:26.823 INFO:tasks.workunit.client.1.smithi144.stdout:9/895: fsync d0/d8/d5e/d44/d4a/fef 0 2024-05-15T15:54:26.825 INFO:tasks.workunit.client.1.smithi144.stdout:7/880: truncate d4/df/d25/d40/d73/f9a 478539 0 2024-05-15T15:54:26.826 INFO:tasks.workunit.client.0.smithi049.stdout:6/879: symlink dd/d17/d24/d2b/d36/d50/d7e/l10e 0 2024-05-15T15:54:26.827 INFO:tasks.workunit.client.0.smithi049.stdout:6/880: chown dd/d17/d24/d2b/d36/d50/dcf 4 1 2024-05-15T15:54:26.827 INFO:tasks.workunit.client.1.smithi144.stdout:4/913: symlink d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/l128 0 2024-05-15T15:54:26.828 INFO:tasks.workunit.client.0.smithi049.stdout:6/881: read - dd/d17/d24/d58/d5f/dd7/feb zero size 2024-05-15T15:54:26.828 INFO:tasks.workunit.client.0.smithi049.stdout:0/784: symlink dc/de9/d109/l10a 0 2024-05-15T15:54:26.829 INFO:tasks.workunit.client.0.smithi049.stdout:0/785: readlink dc/d13/d14/d25/db9/lf4 0 2024-05-15T15:54:26.830 INFO:tasks.workunit.client.0.smithi049.stdout:0/786: write dc/d13/d14/dc9/ff7 [920566,86243] 0 2024-05-15T15:54:26.833 INFO:tasks.workunit.client.0.smithi049.stdout:6/882: symlink dd/d17/d24/d2b/d36/d50/d54/dde/df2/l10f 0 2024-05-15T15:54:26.837 INFO:tasks.workunit.client.0.smithi049.stdout:0/787: getdents dc/da4/db4/df1 0 2024-05-15T15:54:26.840 INFO:tasks.workunit.client.1.smithi144.stdout:8/975: creat d2/d3a/d133/f13a x:0 0 0 2024-05-15T15:54:26.841 INFO:tasks.workunit.client.1.smithi144.stdout:7/881: getdents d4/d3c/d76/d9d/dac/d48/d125 0 2024-05-15T15:54:26.841 INFO:tasks.workunit.client.0.smithi049.stdout:0/788: dread dc/d32/f7b [0,4194304] 0 2024-05-15T15:54:26.842 INFO:tasks.workunit.client.1.smithi144.stdout:7/882: chown d4/d3c/d76/d9d/dac/d48/l56 32068515 1 2024-05-15T15:54:26.844 INFO:tasks.workunit.client.1.smithi144.stdout:4/914: symlink d6/l129 0 2024-05-15T15:54:26.846 INFO:tasks.workunit.client.1.smithi144.stdout:8/976: mknod d2/d3a/da5/d118/c13b 0 2024-05-15T15:54:26.847 INFO:tasks.workunit.client.1.smithi144.stdout:8/977: fdatasync d2/de/d3f/dff/d125/f12f 0 2024-05-15T15:54:26.848 INFO:tasks.workunit.client.0.smithi049.stdout:0/789: read - dc/d13/d3d/d84/fb7 zero size 2024-05-15T15:54:26.849 INFO:tasks.workunit.client.1.smithi144.stdout:4/915: mkdir d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d44/d12a 0 2024-05-15T15:54:26.851 INFO:tasks.workunit.client.1.smithi144.stdout:8/978: creat d2/d24/d2d/da0/d6e/f13c x:0 0 0 2024-05-15T15:54:26.852 INFO:tasks.workunit.client.0.smithi049.stdout:0/790: creat dc/d13/d56/d6f/dce/def/f10b x:0 0 0 2024-05-15T15:54:26.854 INFO:tasks.workunit.client.1.smithi144.stdout:9/896: write d0/d8/d5e/d44/d16/f46 [108171,130455] 0 2024-05-15T15:54:26.855 INFO:tasks.workunit.client.1.smithi144.stdout:6/924: dwrite d7/fe7 [0,4194304] 0 2024-05-15T15:54:26.857 INFO:tasks.workunit.client.1.smithi144.stdout:9/897: write d0/d8/d1b/d79/de7/df9/f11e [410231,118304] 0 2024-05-15T15:54:26.861 INFO:tasks.workunit.client.1.smithi144.stdout:4/916: rename d6/d42/dc8/ff8 to d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/f12b 0 2024-05-15T15:54:26.861 INFO:tasks.workunit.client.1.smithi144.stdout:4/917: fsync d6/dd/d2e/d75/dff/ffe 0 2024-05-15T15:54:26.865 INFO:tasks.workunit.client.1.smithi144.stdout:6/925: mkdir d7/db/d5f/d12c/d148 0 2024-05-15T15:54:26.866 INFO:tasks.workunit.client.1.smithi144.stdout:6/926: write d7/db/d11/d3c/d61/fc5 [1742115,81299] 0 2024-05-15T15:54:26.870 INFO:tasks.workunit.client.1.smithi144.stdout:4/918: creat d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/d10e/f12c x:0 0 0 2024-05-15T15:54:26.872 INFO:tasks.workunit.client.1.smithi144.stdout:9/898: symlink d0/d8/d5e/d44/d10/d9c/l131 0 2024-05-15T15:54:26.873 INFO:tasks.workunit.client.1.smithi144.stdout:7/883: dwrite d4/df/f1a [0,4194304] 0 2024-05-15T15:54:26.874 INFO:tasks.workunit.client.0.smithi049.stdout:0/791: write dc/d13/d14/f1c [2709820,13968] 0 2024-05-15T15:54:26.874 INFO:tasks.workunit.client.1.smithi144.stdout:8/979: dwrite d2/d24/d2d/d30/fe0 [0,4194304] 0 2024-05-15T15:54:26.876 INFO:tasks.workunit.client.1.smithi144.stdout:6/927: creat d7/d1e/d28/db5/f149 x:0 0 0 2024-05-15T15:54:26.881 INFO:tasks.workunit.client.1.smithi144.stdout:7/884: fdatasync d4/f2f 0 2024-05-15T15:54:26.884 INFO:tasks.workunit.client.0.smithi049.stdout:0/792: mknod dc/d13/d2b/d76/d8a/da2/db3/d100/c10c 0 2024-05-15T15:54:26.886 INFO:tasks.workunit.client.1.smithi144.stdout:8/980: unlink d2/d24/d2d/da0/f122 0 2024-05-15T15:54:26.888 INFO:tasks.workunit.client.1.smithi144.stdout:6/928: read - d7/db/da2/fda zero size 2024-05-15T15:54:26.888 INFO:tasks.workunit.client.1.smithi144.stdout:9/899: write d0/f78 [4467299,46174] 0 2024-05-15T15:54:26.889 INFO:tasks.workunit.client.1.smithi144.stdout:6/929: fsync d7/d1e/d28/db5/f149 0 2024-05-15T15:54:26.892 INFO:tasks.workunit.client.1.smithi144.stdout:7/885: rename d4/df/d81/f107 to d4/d41/d5b/f137 0 2024-05-15T15:54:26.893 INFO:tasks.workunit.client.1.smithi144.stdout:7/886: dread - d4/df/d81/dfb/d108/f129 zero size 2024-05-15T15:54:26.894 INFO:tasks.workunit.client.1.smithi144.stdout:6/930: mknod d7/db/da2/d70/c14a 0 2024-05-15T15:54:26.898 INFO:tasks.workunit.client.1.smithi144.stdout:9/900: dwrite d0/d8/d5e/d44/d16/db0/f11d [0,4194304] 0 2024-05-15T15:54:26.898 INFO:tasks.workunit.client.1.smithi144.stdout:9/901: readlink d0/le 0 2024-05-15T15:54:26.900 INFO:tasks.workunit.client.1.smithi144.stdout:6/931: unlink d7/db/f2c 0 2024-05-15T15:54:26.903 INFO:tasks.workunit.client.1.smithi144.stdout:9/902: dread d0/d8/d5e/d44/d10/d9c/f3f [0,4194304] 0 2024-05-15T15:54:26.904 INFO:tasks.workunit.client.1.smithi144.stdout:4/919: dread d6/fe7 [4194304,4194304] 0 2024-05-15T15:54:26.904 INFO:tasks.workunit.client.1.smithi144.stdout:9/903: chown d0/d8/d1b/d62/d7d/d8d/d94 30873945 1 2024-05-15T15:54:26.904 INFO:tasks.workunit.client.0.smithi049.stdout:0/793: dread dc/fd5 [0,4194304] 0 2024-05-15T15:54:26.905 INFO:tasks.workunit.client.1.smithi144.stdout:9/904: fdatasync d0/d8/d1b/d62/fca 0 2024-05-15T15:54:26.908 INFO:tasks.workunit.client.1.smithi144.stdout:7/887: mkdir d4/df/d127/d138 0 2024-05-15T15:54:26.911 INFO:tasks.workunit.client.1.smithi144.stdout:8/981: rename d2/de/d49/f76 to d2/de/f13d 0 2024-05-15T15:54:26.912 INFO:tasks.workunit.client.0.smithi049.stdout:0/794: mknod dc/da4/db4/c10d 0 2024-05-15T15:54:26.913 INFO:tasks.workunit.client.1.smithi144.stdout:4/920: mkdir d6/d42/dc8/d12d 0 2024-05-15T15:54:26.915 INFO:tasks.workunit.client.1.smithi144.stdout:9/905: creat d0/d8/d1b/d79/de7/df9/f132 x:0 0 0 2024-05-15T15:54:26.917 INFO:tasks.workunit.client.1.smithi144.stdout:7/888: truncate d4/df/f65 466766 0 2024-05-15T15:54:26.920 INFO:tasks.workunit.client.0.smithi049.stdout:0/795: dwrite dc/d13/d56/d6f/dce/def/f10b [0,4194304] 0 2024-05-15T15:54:26.920 INFO:tasks.workunit.client.1.smithi144.stdout:6/932: rename d7/db/d11/d7e/c108 to d7/db/d11/c14b 0 2024-05-15T15:54:26.923 INFO:tasks.workunit.client.1.smithi144.stdout:4/921: fsync d6/dd/d34/d3b/d30/d59/fe1 0 2024-05-15T15:54:26.925 INFO:tasks.workunit.client.0.smithi049.stdout:0/796: creat dc/d13/d3d/f10e x:0 0 0 2024-05-15T15:54:26.930 INFO:tasks.workunit.client.1.smithi144.stdout:9/906: unlink d0/d8/d5e/d44/d4a/lba 0 2024-05-15T15:54:26.931 INFO:tasks.workunit.client.1.smithi144.stdout:7/889: rmdir d4/df/d25/d38 39 2024-05-15T15:54:26.938 INFO:tasks.workunit.client.1.smithi144.stdout:4/922: fsync d6/dd/d34/d3b/d30/dc9/d10f/d4b/fb2 0 2024-05-15T15:54:26.938 INFO:tasks.workunit.client.1.smithi144.stdout:6/933: read d7/db/d9d/d97/fc0 [2033877,28325] 0 2024-05-15T15:54:26.939 INFO:tasks.workunit.client.0.smithi049.stdout:0/797: dread dc/d13/d14/d25/d31/dae/dc5/d108/f53 [0,4194304] 0 2024-05-15T15:54:26.940 INFO:tasks.workunit.client.1.smithi144.stdout:6/934: write d7/db/d11/d3c/fac [8012340,109844] 0 2024-05-15T15:54:26.941 INFO:tasks.workunit.client.1.smithi144.stdout:9/907: mknod d0/d8/d1b/d114/dbf/c133 0 2024-05-15T15:54:26.942 INFO:tasks.workunit.client.1.smithi144.stdout:6/935: write d7/db/d11/d7e/f139 [704004,4883] 0 2024-05-15T15:54:26.948 INFO:tasks.workunit.client.1.smithi144.stdout:4/923: creat d6/dd/d2e/d75/f12e x:0 0 0 2024-05-15T15:54:26.955 INFO:tasks.workunit.client.1.smithi144.stdout:0/829: sync 2024-05-15T15:54:26.955 INFO:tasks.workunit.client.1.smithi144.stdout:3/866: sync 2024-05-15T15:54:26.955 INFO:tasks.workunit.client.1.smithi144.stdout:2/809: sync 2024-05-15T15:54:26.965 INFO:tasks.workunit.client.1.smithi144.stdout:3/867: dread d0/d1/d17/d39/f8c [0,4194304] 0 2024-05-15T15:54:26.980 INFO:tasks.workunit.client.0.smithi049.stdout:3/747: sync 2024-05-15T15:54:26.994 INFO:tasks.workunit.client.1.smithi144.stdout:2/810: creat d3/d4d/db4/f10e x:0 0 0 2024-05-15T15:54:26.994 INFO:tasks.workunit.client.1.smithi144.stdout:2/811: write d3/f34 [1514176,48524] 0 2024-05-15T15:54:26.997 INFO:tasks.workunit.client.1.smithi144.stdout:0/830: rmdir d4/d6/d4a/d4d 39 2024-05-15T15:54:27.001 INFO:tasks.workunit.client.1.smithi144.stdout:8/982: dwrite d2/d24/d2d/da0/d9e/fbc [0,4194304] 0 2024-05-15T15:54:27.005 INFO:tasks.workunit.client.1.smithi144.stdout:3/868: dread d0/d1/d17/da1/f118 [0,4194304] 0 2024-05-15T15:54:27.005 INFO:tasks.workunit.client.1.smithi144.stdout:3/869: write d0/d1/d6c/fdd [4515540,23078] 0 2024-05-15T15:54:27.007 INFO:tasks.workunit.client.1.smithi144.stdout:6/936: creat d7/db/d5f/d12c/d148/f14c x:0 0 0 2024-05-15T15:54:27.007 INFO:tasks.workunit.client.1.smithi144.stdout:6/937: fdatasync d7/fe7 0 2024-05-15T15:54:27.008 INFO:tasks.workunit.client.1.smithi144.stdout:6/938: write d7/db/d5f/d12c/d148/f14c [329284,94463] 0 2024-05-15T15:54:27.011 INFO:tasks.workunit.client.0.smithi049.stdout:0/798: dwrite dc/da4/db4/fd1 [0,4194304] 0 2024-05-15T15:54:27.013 INFO:tasks.workunit.client.0.smithi049.stdout:0/799: chown dc/d13/d14/d25/d31/dae/dc5 7 1 2024-05-15T15:54:27.014 INFO:tasks.workunit.client.1.smithi144.stdout:7/890: dwrite d4/df/d25/d38/d68/f74 [0,4194304] 0 2024-05-15T15:54:27.015 INFO:tasks.workunit.client.1.smithi144.stdout:2/812: mknod d3/d7/df8/da6/dc1/c10f 0 2024-05-15T15:54:27.016 INFO:tasks.workunit.client.1.smithi144.stdout:9/908: write d0/d8/d1b/d62/d7d/d8d/dbb/dce/fdd [69304,97775] 0 2024-05-15T15:54:27.016 INFO:tasks.workunit.client.1.smithi144.stdout:9/909: chown d0/l6 111137893 1 2024-05-15T15:54:27.028 INFO:tasks.workunit.client.1.smithi144.stdout:4/924: write d6/d42/f58 [1983880,91529] 0 2024-05-15T15:54:27.035 INFO:tasks.workunit.client.1.smithi144.stdout:3/870: read d0/d11/d3b/da7/def/f10e [718331,130263] 0 2024-05-15T15:54:27.052 INFO:tasks.workunit.client.0.smithi049.stdout:0/800: stat dc/d13/d2b/f4d 0 2024-05-15T15:54:27.053 INFO:tasks.workunit.client.0.smithi049.stdout:0/801: fdatasync dc/d13/d56/d6f/dce/def/f10b 0 2024-05-15T15:54:27.063 INFO:tasks.workunit.client.0.smithi049.stdout:3/748: write ff [446983,72276] 0 2024-05-15T15:54:27.067 INFO:tasks.workunit.client.0.smithi049.stdout:0/802: dread dc/d13/d2b/d76/d8a/da2/fc4 [0,4194304] 0 2024-05-15T15:54:27.067 INFO:tasks.workunit.client.0.smithi049.stdout:3/749: creat d12/d14/d52/db9/dc4/d68/d70/dda/fe6 x:0 0 0 2024-05-15T15:54:27.069 INFO:tasks.workunit.client.0.smithi049.stdout:0/803: fsync dc/d13/d56/fdf 0 2024-05-15T15:54:27.070 INFO:tasks.workunit.client.0.smithi049.stdout:3/750: mknod d12/d14/d52/ce7 0 2024-05-15T15:54:27.074 INFO:tasks.workunit.client.0.smithi049.stdout:3/751: creat d12/d14/d52/db9/dc4/d68/d70/d75/dac/fe8 x:0 0 0 2024-05-15T15:54:27.075 INFO:tasks.workunit.client.0.smithi049.stdout:0/804: write dc/d13/d14/d25/fbe [313880,63447] 0 2024-05-15T15:54:27.078 INFO:tasks.workunit.client.0.smithi049.stdout:3/752: mknod d12/d14/d32/d64/d88/ce9 0 2024-05-15T15:54:27.079 INFO:tasks.workunit.client.0.smithi049.stdout:0/805: symlink dc/d13/d2b/d76/d8a/da2/l10f 0 2024-05-15T15:54:27.080 INFO:tasks.workunit.client.0.smithi049.stdout:3/753: chown d12/l82 78807858 1 2024-05-15T15:54:27.082 INFO:tasks.workunit.client.0.smithi049.stdout:0/806: creat dc/d13/d2b/d76/d8a/da2/db3/f110 x:0 0 0 2024-05-15T15:54:27.091 INFO:tasks.workunit.client.1.smithi144.stdout:9/910: mknod d0/d8/d50/c134 0 2024-05-15T15:54:27.092 INFO:tasks.workunit.client.1.smithi144.stdout:4/925: truncate d6/dd/d2e/d71/f8e 1856528 0 2024-05-15T15:54:27.092 INFO:tasks.workunit.client.1.smithi144.stdout:2/813: chown d3/d7/d20/d3b/l72 3971522 1 2024-05-15T15:54:27.092 INFO:tasks.workunit.client.1.smithi144.stdout:1/880: sync 2024-05-15T15:54:27.092 INFO:tasks.workunit.client.1.smithi144.stdout:5/909: sync 2024-05-15T15:54:27.093 INFO:tasks.workunit.client.1.smithi144.stdout:9/911: readlink d0/d8/d5e/le8 0 2024-05-15T15:54:27.154 INFO:tasks.workunit.client.1.smithi144.stdout:8/983: sync 2024-05-15T15:54:27.210 INFO:tasks.workunit.client.1.smithi144.stdout:6/939: sync 2024-05-15T15:54:27.210 INFO:tasks.workunit.client.1.smithi144.stdout:3/871: sync 2024-05-15T15:54:27.244 INFO:tasks.workunit.client.0.smithi049.stdout:2/730: sync 2024-05-15T15:54:27.304 INFO:tasks.workunit.client.0.smithi049.stdout:0/807: mknod dc/d13/d56/d6f/dce/c111 0 2024-05-15T15:54:27.304 INFO:tasks.workunit.client.0.smithi049.stdout:0/808: read dc/d13/d14/d25/d31/dae/dc5/d108/f5b [626225,92028] 0 2024-05-15T15:54:27.304 INFO:tasks.workunit.client.0.smithi049.stdout:0/809: dread dc/d13/f26 [0,4194304] 0 2024-05-15T15:54:27.304 INFO:tasks.workunit.client.1.smithi144.stdout:7/891: creat d4/d3c/d76/d9d/dd8/f139 x:0 0 0 2024-05-15T15:54:27.304 INFO:tasks.workunit.client.1.smithi144.stdout:0/831: link d4/d6/d3a/d5d/l127 d4/d6/d4a/d74/de4/l133 0 2024-05-15T15:54:27.304 INFO:tasks.workunit.client.1.smithi144.stdout:0/832: readlink d4/d6/d4a/d74/lbf 0 2024-05-15T15:54:27.304 INFO:tasks.workunit.client.1.smithi144.stdout:4/926: fsync d6/dd/fd8 0 2024-05-15T15:54:27.309 INFO:tasks.workunit.client.0.smithi049.stdout:2/731: creat d3/da/d15/d9e/ff6 x:0 0 0 2024-05-15T15:54:27.321 INFO:tasks.workunit.client.0.smithi049.stdout:2/732: dwrite d3/db/f43 [0,4194304] 0 2024-05-15T15:54:27.327 INFO:tasks.workunit.client.0.smithi049.stdout:2/733: creat d3/db/d1d/d1e/d2b/d3b/ff7 x:0 0 0 2024-05-15T15:54:27.362 INFO:tasks.workunit.client.1.smithi144.stdout:0/833: sync 2024-05-15T15:54:27.364 INFO:tasks.workunit.client.1.smithi144.stdout:3/872: mkdir d0/d11/d3b/da7/def/d117/d119 0 2024-05-15T15:54:27.364 INFO:tasks.workunit.client.1.smithi144.stdout:7/892: creat d4/d41/d5b/da0/f13a x:0 0 0 2024-05-15T15:54:27.370 INFO:tasks.workunit.client.1.smithi144.stdout:4/927: mkdir d6/dd/d34/d3b/d30/dc9/d10f/d10b/d12f 0 2024-05-15T15:54:27.372 INFO:tasks.workunit.client.1.smithi144.stdout:2/814: creat d3/d7/d20/d22/f110 x:0 0 0 2024-05-15T15:54:27.372 INFO:tasks.workunit.client.1.smithi144.stdout:2/815: fdatasync d3/d4d/db4/f10e 0 2024-05-15T15:54:27.373 INFO:tasks.workunit.client.1.smithi144.stdout:2/816: read d3/d7/d20/d22/f2b [3357388,79365] 0 2024-05-15T15:54:27.374 INFO:tasks.workunit.client.1.smithi144.stdout:2/817: chown d3/d10/l13 117 1 2024-05-15T15:54:27.375 INFO:tasks.workunit.client.1.smithi144.stdout:3/873: mkdir d0/d1/d17/d39/d40/d4a/d11a 0 2024-05-15T15:54:27.375 INFO:tasks.workunit.client.1.smithi144.stdout:3/874: readlink d0/d11/d3b/da7/lbd 0 2024-05-15T15:54:27.376 INFO:tasks.workunit.client.1.smithi144.stdout:7/893: getdents d4/df/d25/d38/de4 0 2024-05-15T15:54:27.378 INFO:tasks.workunit.client.1.smithi144.stdout:4/928: mkdir d6/dd/dc3/d130 0 2024-05-15T15:54:27.387 INFO:tasks.workunit.client.1.smithi144.stdout:0/834: getdents d4/d6/d3a/d4c/dd6/d55 0 2024-05-15T15:54:27.389 INFO:tasks.workunit.client.1.smithi144.stdout:4/929: creat d6/dd/d34/d3b/d30/db/d1d/d40/f131 x:0 0 0 2024-05-15T15:54:27.395 INFO:tasks.workunit.client.1.smithi144.stdout:7/894: mknod d4/df/d25/d38/de4/c13b 0 2024-05-15T15:54:27.397 INFO:tasks.workunit.client.1.smithi144.stdout:4/930: mkdir d6/dd/d2e/d71/d132 0 2024-05-15T15:54:27.399 INFO:tasks.workunit.client.1.smithi144.stdout:4/931: truncate d6/dd/d34/d3b/d30/db/d1d/d4e/ffb 944555 0 2024-05-15T15:54:27.400 INFO:tasks.workunit.client.1.smithi144.stdout:4/932: truncate d6/f118 543790 0 2024-05-15T15:54:27.402 INFO:tasks.workunit.client.1.smithi144.stdout:7/895: chown d4/d3c/d76/d9d/dac/d48/d72/da2/cd0 121758 1 2024-05-15T15:54:27.404 INFO:tasks.workunit.client.1.smithi144.stdout:0/835: symlink d4/d6/l134 0 2024-05-15T15:54:27.405 INFO:tasks.workunit.client.1.smithi144.stdout:0/836: chown d4/d6/d3a/d4c/d11f/f124 17159 1 2024-05-15T15:54:27.405 INFO:tasks.workunit.client.1.smithi144.stdout:2/818: getdents d3/d7/d20/d37/d76/db5/db6/dbf 0 2024-05-15T15:54:27.406 INFO:tasks.workunit.client.1.smithi144.stdout:0/837: write d4/d2d/d48/d5b/f12e [1010748,16284] 0 2024-05-15T15:54:27.407 INFO:tasks.workunit.client.1.smithi144.stdout:3/875: getdents d0/d1/d56/d98/d9c 0 2024-05-15T15:54:27.408 INFO:tasks.workunit.client.1.smithi144.stdout:0/838: write d4/df/f110 [950659,94563] 0 2024-05-15T15:54:27.409 INFO:tasks.workunit.client.1.smithi144.stdout:0/839: chown d4/d6/d3a/d5d/f82 97 1 2024-05-15T15:54:27.410 INFO:tasks.workunit.client.1.smithi144.stdout:0/840: chown d4/d2d/d100/l104 18 1 2024-05-15T15:54:27.440 INFO:tasks.workunit.client.0.smithi049.stdout:3/754: dwrite d12/d14/f43 [4194304,4194304] 0 2024-05-15T15:54:27.446 INFO:tasks.workunit.client.0.smithi049.stdout:3/755: symlink d12/d14/d52/db9/da6/dc2/lea 0 2024-05-15T15:54:27.448 INFO:tasks.workunit.client.0.smithi049.stdout:3/756: creat d12/d67/da8/ddd/feb x:0 0 0 2024-05-15T15:54:27.449 INFO:tasks.workunit.client.0.smithi049.stdout:3/757: write f7 [3647940,116283] 0 2024-05-15T15:54:27.452 INFO:tasks.workunit.client.0.smithi049.stdout:3/758: dread d12/d14/d52/db9/dc4/d68/f9b [0,4194304] 0 2024-05-15T15:54:27.452 INFO:tasks.workunit.client.1.smithi144.stdout:1/881: write d7/d1d/d33/d55/f8d [5108076,89855] 0 2024-05-15T15:54:27.452 INFO:tasks.workunit.client.0.smithi049.stdout:0/810: write dc/d13/d2b/d76/d8a/da2/fbd [337157,94521] 0 2024-05-15T15:54:27.453 INFO:tasks.workunit.client.1.smithi144.stdout:1/882: chown d7/d1d/d33/d41/d22/d45/fb7 242 1 2024-05-15T15:54:27.453 INFO:tasks.workunit.client.1.smithi144.stdout:9/912: write d0/d8/d5e/fbc [593193,107694] 0 2024-05-15T15:54:27.456 INFO:tasks.workunit.client.0.smithi049.stdout:3/759: dread - d12/d14/d52/db9/dc4/d68/fa0 zero size 2024-05-15T15:54:27.457 INFO:tasks.workunit.client.1.smithi144.stdout:8/984: write d2/de/d49/f69 [899116,22652] 0 2024-05-15T15:54:27.457 INFO:tasks.workunit.client.0.smithi049.stdout:3/760: chown d12/d14/d32/d64 462772167 1 2024-05-15T15:54:27.459 INFO:tasks.workunit.client.0.smithi049.stdout:0/811: fdatasync dc/d13/d14/d25/d31/f58 0 2024-05-15T15:54:27.461 INFO:tasks.workunit.client.1.smithi144.stdout:1/883: creat d7/d1d/d33/de7/f118 x:0 0 0 2024-05-15T15:54:27.462 INFO:tasks.workunit.client.1.smithi144.stdout:1/884: write d7/d1d/d33/d29/d6b/d8e/fca [188840,125160] 0 2024-05-15T15:54:27.464 INFO:tasks.workunit.client.1.smithi144.stdout:9/913: mknod d0/d8/d5e/d112/c135 0 2024-05-15T15:54:27.489 INFO:tasks.workunit.client.0.smithi049.stdout:1/935: sync 2024-05-15T15:54:27.492 INFO:tasks.workunit.client.0.smithi049.stdout:0/812: mknod dc/d13/d14/d2f/d8f/dcb/de2/d103/c112 0 2024-05-15T15:54:27.644 INFO:tasks.workunit.client.0.smithi049.stdout:1/936: unlink d0/d4/dd/de/d4e/c11c 0 2024-05-15T15:54:27.646 INFO:tasks.workunit.client.1.smithi144.stdout:9/914: rename d0/d8/d1b/d62/d7d/d8d/d94/l100 to d0/d8/d1b/d79/d8a/dae/ddb/l136 0 2024-05-15T15:54:27.649 INFO:tasks.workunit.client.1.smithi144.stdout:1/885: link d7/d1d/d33/d29/dd2/l39 d7/d1d/d33/d41/d22/db4/l119 0 2024-05-15T15:54:27.652 INFO:tasks.workunit.client.1.smithi144.stdout:9/915: symlink d0/d8/d1b/d114/dbf/l137 0 2024-05-15T15:54:27.653 INFO:tasks.workunit.client.1.smithi144.stdout:1/886: dread d7/d1d/d33/d29/f42 [0,4194304] 0 2024-05-15T15:54:27.665 INFO:tasks.workunit.client.1.smithi144.stdout:9/916: rename d0/d8/d1b/d62/d7d/laa to d0/d8/d5e/d44/d77/l138 0 2024-05-15T15:54:27.665 INFO:tasks.workunit.client.1.smithi144.stdout:9/917: dread - d0/d8/f130 zero size 2024-05-15T15:54:27.667 INFO:tasks.workunit.client.1.smithi144.stdout:9/918: creat d0/d8/d1b/d79/d8a/dae/ddb/f139 x:0 0 0 2024-05-15T15:54:27.670 INFO:tasks.workunit.client.1.smithi144.stdout:9/919: creat d0/d8/d1b/d79/d8a/dae/f13a x:0 0 0 2024-05-15T15:54:27.675 INFO:tasks.workunit.client.1.smithi144.stdout:9/920: dwrite d0/d8/d1b/d114/dbf/f129 [0,4194304] 0 2024-05-15T15:54:27.677 INFO:tasks.workunit.client.1.smithi144.stdout:9/921: chown d0/d8/d5e/d44/d10/d19/dd5/de5 118 1 2024-05-15T15:54:27.680 INFO:tasks.workunit.client.0.smithi049.stdout:5/908: sync 2024-05-15T15:54:27.682 INFO:tasks.workunit.client.0.smithi049.stdout:5/909: write dc/dd/f88 [5267284,55923] 0 2024-05-15T15:54:27.685 INFO:tasks.workunit.client.1.smithi144.stdout:6/940: dwrite d7/db/d35/d84/d81/f102 [0,4194304] 0 2024-05-15T15:54:27.686 INFO:tasks.workunit.client.1.smithi144.stdout:2/819: rmdir d3/d7/df8 39 2024-05-15T15:54:27.690 INFO:tasks.workunit.client.0.smithi049.stdout:5/910: dwrite dc/f7f [0,4194304] 0 2024-05-15T15:54:27.694 INFO:tasks.workunit.client.1.smithi144.stdout:2/820: mkdir d3/d7/d20/d22/d52/d64/d97/d111 0 2024-05-15T15:54:27.696 INFO:tasks.workunit.client.0.smithi049.stdout:1/937: rename d0/d4/dd/de/d1e/c77 to d0/d4/dd/de/d1e/d16/d8a/dd2/df2/c14b 0 2024-05-15T15:54:27.698 INFO:tasks.workunit.client.0.smithi049.stdout:5/911: rename dc/d4a/d42 to dc/d4a/d42/d12c 22 2024-05-15T15:54:27.700 INFO:tasks.workunit.client.1.smithi144.stdout:6/941: mknod d7/d62/d112/d13f/c14d 0 2024-05-15T15:54:27.701 INFO:tasks.workunit.client.1.smithi144.stdout:2/821: dwrite d3/d7/d20/f7c [0,4194304] 0 2024-05-15T15:54:27.701 INFO:tasks.workunit.client.0.smithi049.stdout:1/938: symlink d0/d4/dd/de/d1e/d16/d6e/dbf/l14c 0 2024-05-15T15:54:27.704 INFO:tasks.workunit.client.0.smithi049.stdout:5/912: creat dc/db0/dc7/f12d x:0 0 0 2024-05-15T15:54:27.705 INFO:tasks.workunit.client.1.smithi144.stdout:6/942: mknod d7/d62/d112/c14e 0 2024-05-15T15:54:27.708 INFO:tasks.workunit.client.1.smithi144.stdout:8/985: mknod d2/d24/d2d/da0/c13e 0 2024-05-15T15:54:27.711 INFO:tasks.workunit.client.1.smithi144.stdout:2/822: mkdir d3/d7/d20/def/d112 0 2024-05-15T15:54:27.714 INFO:tasks.workunit.client.1.smithi144.stdout:8/986: creat d2/d24/d2d/d30/dd0/df7/d87/f13f x:0 0 0 2024-05-15T15:54:27.715 INFO:tasks.workunit.client.1.smithi144.stdout:4/933: truncate d6/dd/d34/d3b/d30/d59/f74 2311334 0 2024-05-15T15:54:27.717 INFO:tasks.workunit.client.1.smithi144.stdout:6/943: link d7/f123 d7/db/d35/d84/dbe/f14f 0 2024-05-15T15:54:27.717 INFO:tasks.workunit.client.1.smithi144.stdout:4/934: readlink d6/dd/d34/d3b/d30/d59/l10d 0 2024-05-15T15:54:27.718 INFO:tasks.workunit.client.1.smithi144.stdout:3/876: dwrite d0/d11/d3b/da7/fec [0,4194304] 0 2024-05-15T15:54:27.720 INFO:tasks.workunit.client.1.smithi144.stdout:0/841: truncate d4/d2d/f10e 211264 0 2024-05-15T15:54:27.723 INFO:tasks.workunit.client.0.smithi049.stdout:1/939: dread d0/d4/dd/de/d1e/d16/d8a/f8f [0,4194304] 0 2024-05-15T15:54:27.723 INFO:tasks.workunit.client.1.smithi144.stdout:6/944: symlink d7/d62/d112/l150 0 2024-05-15T15:54:27.727 INFO:tasks.workunit.client.1.smithi144.stdout:3/877: mkdir d0/d1/d56/d11b 0 2024-05-15T15:54:27.728 INFO:tasks.workunit.client.1.smithi144.stdout:4/935: dread d6/dd/d34/d3b/d30/db/d1d/f9f [4194304,4194304] 0 2024-05-15T15:54:27.728 INFO:tasks.workunit.client.1.smithi144.stdout:8/987: dwrite d2/d1a/f31 [4194304,4194304] 0 2024-05-15T15:54:27.729 INFO:tasks.workunit.client.1.smithi144.stdout:7/896: creat d4/df/d79/f13c x:0 0 0 2024-05-15T15:54:27.731 INFO:tasks.workunit.client.1.smithi144.stdout:8/988: write d2/d24/d2d/da0/f136 [319968,13512] 0 2024-05-15T15:54:27.734 INFO:tasks.workunit.client.1.smithi144.stdout:3/878: write d0/d11/d24/f74 [1420764,53718] 0 2024-05-15T15:54:27.735 INFO:tasks.workunit.client.1.smithi144.stdout:0/842: creat d4/df/d1f/d24/d85/da1/f135 x:0 0 0 2024-05-15T15:54:27.738 INFO:tasks.workunit.client.1.smithi144.stdout:2/823: dread d3/d7/df8/da6/dc1/fc6 [0,4194304] 0 2024-05-15T15:54:27.738 INFO:tasks.workunit.client.1.smithi144.stdout:5/910: rmdir d0/dc/d24 39 2024-05-15T15:54:27.739 INFO:tasks.workunit.client.0.smithi049.stdout:5/913: dread dc/f1b [0,4194304] 0 2024-05-15T15:54:27.740 INFO:tasks.workunit.client.1.smithi144.stdout:2/824: write d3/ffb [3395909,40876] 0 2024-05-15T15:54:27.741 INFO:tasks.workunit.client.1.smithi144.stdout:7/897: creat d4/d3c/d76/d9d/dac/d48/d72/da2/f13d x:0 0 0 2024-05-15T15:54:27.745 INFO:tasks.workunit.client.1.smithi144.stdout:8/989: rename d2/f96 to d2/de/d3f/dff/d125/f140 0 2024-05-15T15:54:27.745 INFO:tasks.workunit.client.0.smithi049.stdout:5/914: creat dc/d4a/d9e/d10b/f12e x:0 0 0 2024-05-15T15:54:27.748 INFO:tasks.workunit.client.1.smithi144.stdout:2/825: fdatasync d3/d7/f3d 0 2024-05-15T15:54:27.749 INFO:tasks.workunit.client.1.smithi144.stdout:0/843: dread d4/df/f16 [0,4194304] 0 2024-05-15T15:54:27.752 INFO:tasks.workunit.client.1.smithi144.stdout:6/945: link d7/db/d9d/f88 d7/f151 0 2024-05-15T15:54:27.752 INFO:tasks.workunit.client.1.smithi144.stdout:0/844: dread d4/d6/d3a/d4c/dd6/f3b [0,4194304] 0 2024-05-15T15:54:27.752 INFO:tasks.workunit.client.1.smithi144.stdout:8/990: dwrite d2/d24/f89 [4194304,4194304] 0 2024-05-15T15:54:27.753 INFO:tasks.workunit.client.1.smithi144.stdout:2/826: truncate d3/d7/d20/d22/ffd 988422 0 2024-05-15T15:54:27.754 INFO:tasks.workunit.client.0.smithi049.stdout:3/761: dwrite d12/d14/d52/db9/d7a/f83 [0,4194304] 0 2024-05-15T15:54:27.754 INFO:tasks.workunit.client.1.smithi144.stdout:2/827: chown d3/d7/d20/d22/d52/f6f 10726012 1 2024-05-15T15:54:27.756 INFO:tasks.workunit.client.0.smithi049.stdout:3/762: write d12/d14/d52/db9/d7a/f83 [2706318,39705] 0 2024-05-15T15:54:27.758 INFO:tasks.workunit.client.0.smithi049.stdout:5/915: truncate dc/db0/dd5/fe1 827387 0 2024-05-15T15:54:27.759 INFO:tasks.workunit.client.0.smithi049.stdout:5/916: fdatasync dc/dd/d10/f119 0 2024-05-15T15:54:27.761 INFO:tasks.workunit.client.0.smithi049.stdout:5/917: chown dc/db0/dd5/cf8 7669 1 2024-05-15T15:54:27.762 INFO:tasks.workunit.client.0.smithi049.stdout:1/940: dread d0/d4/dd/de/d1e/d2e/f41 [0,4194304] 0 2024-05-15T15:54:27.762 INFO:tasks.workunit.client.0.smithi049.stdout:0/813: write dc/d32/d93/f89 [564058,70313] 0 2024-05-15T15:54:27.763 INFO:tasks.workunit.client.0.smithi049.stdout:1/941: readlink d0/lb2 0 2024-05-15T15:54:27.767 INFO:tasks.workunit.client.1.smithi144.stdout:6/946: creat d7/db/d11/d3c/d61/f152 x:0 0 0 2024-05-15T15:54:27.768 INFO:tasks.workunit.client.1.smithi144.stdout:6/947: readlink d7/db/d11/d3c/d61/dcc/l107 0 2024-05-15T15:54:27.768 INFO:tasks.workunit.client.0.smithi049.stdout:3/763: fsync d12/d14/d52/db9/dc4/d68/d70/f8a 0 2024-05-15T15:54:27.769 INFO:tasks.workunit.client.1.smithi144.stdout:1/887: write d7/d1d/d33/f71 [41206,62127] 0 2024-05-15T15:54:27.770 INFO:tasks.workunit.client.1.smithi144.stdout:6/948: chown d7/db/d11/d3c/f11d 8505 1 2024-05-15T15:54:27.772 INFO:tasks.workunit.client.0.smithi049.stdout:5/918: fdatasync dc/d4a/f8c 0 2024-05-15T15:54:27.774 INFO:tasks.workunit.client.1.smithi144.stdout:8/991: mkdir d2/d3a/da5/d11b/d141 0 2024-05-15T15:54:27.774 INFO:tasks.workunit.client.1.smithi144.stdout:9/922: write d0/d8/d5e/d44/d10/f27 [42644,103396] 0 2024-05-15T15:54:27.776 INFO:tasks.workunit.client.0.smithi049.stdout:0/814: rename dc/d13/d2b/d76/d8a/da2/fc4 to dc/f113 0 2024-05-15T15:54:27.777 INFO:tasks.workunit.client.1.smithi144.stdout:1/888: mknod d7/d1d/d33/d1e/d34/c11a 0 2024-05-15T15:54:27.778 INFO:tasks.workunit.client.1.smithi144.stdout:1/889: chown d7/d1d/d33/d29/d6b/d5e/d74/df3/lfd 423892701 1 2024-05-15T15:54:27.779 INFO:tasks.workunit.client.0.smithi049.stdout:3/764: fdatasync d12/d14/d52/db9/dc4/d68/f9b 0 2024-05-15T15:54:27.781 INFO:tasks.workunit.client.1.smithi144.stdout:0/845: dread d4/d6/fb [0,4194304] 0 2024-05-15T15:54:27.783 INFO:tasks.workunit.client.0.smithi049.stdout:5/919: symlink dc/dd/d10/df1/l12f 0 2024-05-15T15:54:27.784 INFO:tasks.workunit.client.0.smithi049.stdout:5/920: chown dc/db0/dc7/cfa 922 1 2024-05-15T15:54:27.784 INFO:tasks.workunit.client.1.smithi144.stdout:9/923: truncate d0/d8/d5e/d44/d10/d19/fc0 7348588 0 2024-05-15T15:54:27.785 INFO:tasks.workunit.client.0.smithi049.stdout:1/942: mknod d0/d4/dd/de/d1e/c14d 0 2024-05-15T15:54:27.787 INFO:tasks.workunit.client.1.smithi144.stdout:1/890: dread d7/d1d/d33/d29/d6b/d5e/d74/df3/fdb [0,4194304] 0 2024-05-15T15:54:27.788 INFO:tasks.workunit.client.0.smithi049.stdout:0/815: mkdir dc/da4/db4/d114 0 2024-05-15T15:54:27.790 INFO:tasks.workunit.client.1.smithi144.stdout:6/949: truncate d7/d62/f95 2665243 0 2024-05-15T15:54:27.793 INFO:tasks.workunit.client.1.smithi144.stdout:0/846: read d4/d6/d3a/d5d/f65 [354561,29188] 0 2024-05-15T15:54:27.793 INFO:tasks.workunit.client.1.smithi144.stdout:8/992: dread d2/f35 [0,4194304] 0 2024-05-15T15:54:27.797 INFO:tasks.workunit.client.0.smithi049.stdout:1/943: fsync d0/d4/dd/de/f13 0 2024-05-15T15:54:27.798 INFO:tasks.workunit.client.0.smithi049.stdout:1/944: chown d0/d4/dd/de/d1e/d12f 0 1 2024-05-15T15:54:27.799 INFO:tasks.workunit.client.0.smithi049.stdout:0/816: symlink dc/d6d/dc3/l115 0 2024-05-15T15:54:27.800 INFO:tasks.workunit.client.1.smithi144.stdout:1/891: readlink d7/d1d/ldd 0 2024-05-15T15:54:27.803 INFO:tasks.workunit.client.0.smithi049.stdout:5/921: mknod dc/dd/d47/d83/dcc/da9/dba/dbe/c130 0 2024-05-15T15:54:27.804 INFO:tasks.workunit.client.0.smithi049.stdout:5/922: chown dc/dd/d47 39 1 2024-05-15T15:54:27.805 INFO:tasks.workunit.client.1.smithi144.stdout:0/847: readlink d4/d6/d3a/d4c/dd6/d55/da5/d115/l118 0 2024-05-15T15:54:27.808 INFO:tasks.workunit.client.0.smithi049.stdout:1/945: mknod d0/d4/dd/de/d1e/d16/d8a/dd2/c14e 0 2024-05-15T15:54:27.809 INFO:tasks.workunit.client.1.smithi144.stdout:1/892: dwrite d7/d1d/d33/d1e/f25 [0,4194304] 0 2024-05-15T15:54:27.809 INFO:tasks.workunit.client.1.smithi144.stdout:4/936: dwrite d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/dda/fb9 [0,4194304] 0 2024-05-15T15:54:27.812 INFO:tasks.workunit.client.1.smithi144.stdout:3/879: dwrite d0/f15 [0,4194304] 0 2024-05-15T15:54:27.813 INFO:tasks.workunit.client.1.smithi144.stdout:1/893: readlink d7/d1d/d33/d41/d78/da3/lc2 0 2024-05-15T15:54:27.814 INFO:tasks.workunit.client.1.smithi144.stdout:1/894: stat d7/d1d/dac/de1 0 2024-05-15T15:54:27.815 INFO:tasks.workunit.client.1.smithi144.stdout:1/895: chown d7/d1d/d33/d1e/d34/f114 32394701 1 2024-05-15T15:54:27.816 INFO:tasks.workunit.client.0.smithi049.stdout:1/946: mknod d0/d4/d81/c14f 0 2024-05-15T15:54:27.818 INFO:tasks.workunit.client.1.smithi144.stdout:6/950: link d7/db/d11/d3c/dfc/l133 d7/d32/l153 0 2024-05-15T15:54:27.819 INFO:tasks.workunit.client.1.smithi144.stdout:6/951: stat d7/db/d11/d3c 0 2024-05-15T15:54:27.821 INFO:tasks.workunit.client.1.smithi144.stdout:4/937: readlink d6/dd/d34/d3b/d30/db/d1d/d40/d57/lc4 0 2024-05-15T15:54:27.824 INFO:tasks.workunit.client.1.smithi144.stdout:3/880: fdatasync d0/d11/d3b/da7/fe1 0 2024-05-15T15:54:27.827 INFO:tasks.workunit.client.1.smithi144.stdout:2/828: dwrite d3/d7/df8/d65/f68 [0,4194304] 0 2024-05-15T15:54:27.829 INFO:tasks.workunit.client.1.smithi144.stdout:1/896: unlink d7/d1d/d33/d1e/d34/f76 0 2024-05-15T15:54:27.830 INFO:tasks.workunit.client.0.smithi049.stdout:0/817: creat dc/d13/d14/d25/d31/d71/f116 x:0 0 0 2024-05-15T15:54:27.830 INFO:tasks.workunit.client.1.smithi144.stdout:1/897: chown d7/d1d/d33/d41/d22/d45 3 1 2024-05-15T15:54:27.830 INFO:tasks.workunit.client.1.smithi144.stdout:1/898: chown d7/d4b/dd9 376114519 1 2024-05-15T15:54:27.832 INFO:tasks.workunit.client.1.smithi144.stdout:6/952: symlink d7/d32/l154 0 2024-05-15T15:54:27.838 INFO:tasks.workunit.client.1.smithi144.stdout:1/899: dwrite d7/d1d/d33/d1e/d34/f114 [0,4194304] 0 2024-05-15T15:54:27.838 INFO:tasks.workunit.client.1.smithi144.stdout:3/881: rename d0/d1/d17/l37 to d0/d11/d3b/da0/df6/l11c 0 2024-05-15T15:54:27.840 INFO:tasks.workunit.client.0.smithi049.stdout:0/818: mknod dc/d13/d14/d2f/d8f/dcb/de2/c117 0 2024-05-15T15:54:27.841 INFO:tasks.workunit.client.0.smithi049.stdout:0/819: stat dc/d13/d14/d25/d31/dae/dc5/d108/fd3 0 2024-05-15T15:54:27.843 INFO:tasks.workunit.client.0.smithi049.stdout:5/923: getdents dc/d4a/ddd 0 2024-05-15T15:54:27.846 INFO:tasks.workunit.client.1.smithi144.stdout:2/829: unlink d3/d7/df8/lcd 0 2024-05-15T15:54:27.847 INFO:tasks.workunit.client.1.smithi144.stdout:1/900: dwrite d7/d1d/d58/fda [0,4194304] 0 2024-05-15T15:54:27.851 INFO:tasks.workunit.client.1.smithi144.stdout:4/938: unlink d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/d10e/f122 0 2024-05-15T15:54:27.852 INFO:tasks.workunit.client.1.smithi144.stdout:6/953: dwrite d7/d62/fa9 [4194304,4194304] 0 2024-05-15T15:54:27.853 INFO:tasks.workunit.client.1.smithi144.stdout:9/924: write d0/d8/d1b/f6f [2852043,36878] 0 2024-05-15T15:54:27.855 INFO:tasks.workunit.client.0.smithi049.stdout:3/765: dwrite d12/d14/f54 [0,4194304] 0 2024-05-15T15:54:27.858 INFO:tasks.workunit.client.0.smithi049.stdout:3/766: chown d12/d14/d52/db9/dc4/d68/d70/dda 293 1 2024-05-15T15:54:27.870 INFO:tasks.workunit.client.1.smithi144.stdout:0/848: dwrite d4/d6/d3a/d4c/dd6/d55/da5/fe3 [0,4194304] 0 2024-05-15T15:54:27.871 INFO:tasks.workunit.client.0.smithi049.stdout:3/767: rename d12/c28 to d12/d30/dc1/dc7/ddc/cec 0 2024-05-15T15:54:27.872 INFO:tasks.workunit.client.0.smithi049.stdout:1/947: write d0/d4/dd/de/d36/f108 [819512,58580] 0 2024-05-15T15:54:27.873 INFO:tasks.workunit.client.0.smithi049.stdout:1/948: dread - d0/d4/d63/f64 zero size 2024-05-15T15:54:27.878 INFO:tasks.workunit.client.1.smithi144.stdout:4/939: dread d6/dd/d34/d3b/d30/dc9/d10f/d4b/f4d [0,4194304] 0 2024-05-15T15:54:27.879 INFO:tasks.workunit.client.1.smithi144.stdout:5/911: sync 2024-05-15T15:54:27.882 INFO:tasks.workunit.client.1.smithi144.stdout:7/898: sync 2024-05-15T15:54:27.884 INFO:tasks.workunit.client.0.smithi049.stdout:0/820: dread dc/d13/d3d/f43 [0,4194304] 0 2024-05-15T15:54:27.886 INFO:tasks.workunit.client.0.smithi049.stdout:5/924: dread dc/dd/f14 [4194304,4194304] 0 2024-05-15T15:54:27.887 INFO:tasks.workunit.client.0.smithi049.stdout:5/925: chown dc/d4a/l89 6017438 1 2024-05-15T15:54:27.889 INFO:tasks.workunit.client.0.smithi049.stdout:0/821: rmdir dc/d13/d14/d25/d31/d71 39 2024-05-15T15:54:27.891 INFO:tasks.workunit.client.0.smithi049.stdout:5/926: creat dc/db0/dd5/d102/f131 x:0 0 0 2024-05-15T15:54:27.892 INFO:tasks.workunit.client.0.smithi049.stdout:5/927: chown dc/dd/d7b/cb2 630996 1 2024-05-15T15:54:27.894 INFO:tasks.workunit.client.1.smithi144.stdout:3/882: write d0/d1/d17/d39/d40/d7e/fd5 [926517,109676] 0 2024-05-15T15:54:27.895 INFO:tasks.workunit.client.1.smithi144.stdout:6/954: creat d7/d1e/d137/f155 x:0 0 0 2024-05-15T15:54:27.896 INFO:tasks.workunit.client.1.smithi144.stdout:3/883: chown d0/d11/d3b/da7/def/d117/f9d 437272181 1 2024-05-15T15:54:27.901 INFO:tasks.workunit.client.0.smithi049.stdout:1/949: write d0/f5e [3308298,8934] 0 2024-05-15T15:54:27.902 INFO:tasks.workunit.client.1.smithi144.stdout:9/925: creat d0/d8/d5e/d44/d10/d19/dd5/d12b/f13b x:0 0 0 2024-05-15T15:54:27.904 INFO:tasks.workunit.client.1.smithi144.stdout:5/912: unlink d0/dc/d1f/d50/l30 0 2024-05-15T15:54:27.904 INFO:tasks.workunit.client.0.smithi049.stdout:3/768: write d12/f29 [353028,36001] 0 2024-05-15T15:54:27.906 INFO:tasks.workunit.client.0.smithi049.stdout:1/950: mknod d0/d4/dd/de/d1e/d16/d8a/dd2/df2/c150 0 2024-05-15T15:54:27.907 INFO:tasks.workunit.client.1.smithi144.stdout:7/899: rename d4/df/d81/dfb/cd1 to d4/df/d81/dfb/c13e 0 2024-05-15T15:54:27.908 INFO:tasks.workunit.client.1.smithi144.stdout:7/900: fsync d4/d3c/d76/d9d/dd8/f139 0 2024-05-15T15:54:27.909 INFO:tasks.workunit.client.0.smithi049.stdout:3/769: unlink d12/d14/d32/l76 0 2024-05-15T15:54:27.912 INFO:tasks.workunit.client.1.smithi144.stdout:8/993: sync 2024-05-15T15:54:27.912 INFO:tasks.workunit.client.0.smithi049.stdout:1/951: mknod d0/d4/dd/de/d1e/d16/d8a/dd2/df2/c151 0 2024-05-15T15:54:27.913 INFO:tasks.workunit.client.1.smithi144.stdout:2/830: creat d3/d7/d20/f113 x:0 0 0 2024-05-15T15:54:27.914 INFO:tasks.workunit.client.0.smithi049.stdout:0/822: dwrite dc/d13/d14/d25/d31/dae/dc5/d108/f5b [0,4194304] 0 2024-05-15T15:54:27.914 INFO:tasks.workunit.client.0.smithi049.stdout:3/770: dread d12/f4d [0,4194304] 0 2024-05-15T15:54:27.916 INFO:tasks.workunit.client.1.smithi144.stdout:1/901: mkdir d7/d1d/d33/d41/d22/dbd/dc8/d11b 0 2024-05-15T15:54:27.918 INFO:tasks.workunit.client.1.smithi144.stdout:1/902: write d7/d1d/d33/d29/dd2/f7b [1055063,356] 0 2024-05-15T15:54:27.918 INFO:tasks.workunit.client.1.smithi144.stdout:0/849: creat d4/df/d102/d11b/dd1/f136 x:0 0 0 2024-05-15T15:54:27.918 INFO:tasks.workunit.client.1.smithi144.stdout:6/955: chown d7/d1e/d28/db5/df6/c144 1047646 1 2024-05-15T15:54:27.920 INFO:tasks.workunit.client.1.smithi144.stdout:3/884: fsync d0/d11/d24/da5/fbe 0 2024-05-15T15:54:27.922 INFO:tasks.workunit.client.0.smithi049.stdout:5/928: dwrite dc/dd/d47/d83/fc2 [0,4194304] 0 2024-05-15T15:54:27.922 INFO:tasks.workunit.client.0.smithi049.stdout:3/771: chown d12/d14/d32/c33 0 1 2024-05-15T15:54:27.926 INFO:tasks.workunit.client.0.smithi049.stdout:3/772: write d12/d26/fe0 [323458,75487] 0 2024-05-15T15:54:27.927 INFO:tasks.workunit.client.1.smithi144.stdout:4/940: symlink d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d44/d12a/l133 0 2024-05-15T15:54:27.929 INFO:tasks.workunit.client.1.smithi144.stdout:4/941: dread d6/dd/d34/d3b/d30/db/d1d/d40/f80 [0,4194304] 0 2024-05-15T15:54:27.930 INFO:tasks.workunit.client.0.smithi049.stdout:5/929: mkdir dc/dd/d47/d132 0 2024-05-15T15:54:27.933 INFO:tasks.workunit.client.1.smithi144.stdout:8/994: rename d2/d24/d2d/da0/d6e/d71/d8c/fe6 to d2/d3a/da5/d11b/d141/f142 0 2024-05-15T15:54:27.937 INFO:tasks.workunit.client.1.smithi144.stdout:1/903: stat d7/d1d/d33/d41/fe 0 2024-05-15T15:54:27.939 INFO:tasks.workunit.client.1.smithi144.stdout:1/904: truncate d7/d1d/d33/d41/d78/da3/f54 5139438 0 2024-05-15T15:54:27.942 INFO:tasks.workunit.client.0.smithi049.stdout:1/952: dread d0/d4/dd/de/f13 [0,4194304] 0 2024-05-15T15:54:27.943 INFO:tasks.workunit.client.0.smithi049.stdout:1/953: chown d0/d4/dd/de/d1e/d2e/dad/f136 251043470 1 2024-05-15T15:54:27.945 INFO:tasks.workunit.client.1.smithi144.stdout:2/831: symlink d3/d7/d20/d37/l114 0 2024-05-15T15:54:27.945 INFO:tasks.workunit.client.1.smithi144.stdout:2/832: chown d3/d7/d20/d22/d52/cd5 61524 1 2024-05-15T15:54:27.946 INFO:tasks.workunit.client.1.smithi144.stdout:4/942: mkdir d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/dda/d134 0 2024-05-15T15:54:27.946 INFO:tasks.workunit.client.1.smithi144.stdout:2/833: chown d3/d7/d20/d37/d76/db5/fbd 1050128 1 2024-05-15T15:54:27.948 INFO:tasks.workunit.client.1.smithi144.stdout:2/834: truncate d3/d4d/db4/f10e 22842 0 2024-05-15T15:54:27.951 INFO:tasks.workunit.client.1.smithi144.stdout:6/956: creat d7/d1e/d28/db5/d141/f156 x:0 0 0 2024-05-15T15:54:27.951 INFO:tasks.workunit.client.0.smithi049.stdout:1/954: symlink d0/d4/dd/de/d36/dd3/d145/l152 0 2024-05-15T15:54:27.953 INFO:tasks.workunit.client.1.smithi144.stdout:3/885: creat d0/d1/d56/d98/d10c/f11d x:0 0 0 2024-05-15T15:54:27.954 INFO:tasks.workunit.client.0.smithi049.stdout:1/955: chown d0/d4/dd/de/d1e/d16/df1/d103/f127 0 1 2024-05-15T15:54:27.954 INFO:tasks.workunit.client.1.smithi144.stdout:3/886: stat d0/d1/d6c/fdd 0 2024-05-15T15:54:27.954 INFO:tasks.workunit.client.1.smithi144.stdout:0/850: dread d4/d2d/fce [0,4194304] 0 2024-05-15T15:54:27.954 INFO:tasks.workunit.client.1.smithi144.stdout:1/905: fsync d7/d1d/d33/d29/d6b/d5e/d74/df3/fdb 0 2024-05-15T15:54:27.957 INFO:tasks.workunit.client.0.smithi049.stdout:9/671: sync 2024-05-15T15:54:27.958 INFO:tasks.workunit.client.1.smithi144.stdout:9/926: write d0/d8/d5e/d44/d10/d19/f49 [782292,112864] 0 2024-05-15T15:54:27.960 INFO:tasks.workunit.client.0.smithi049.stdout:1/956: dwrite d0/d4/dd/de/d4d/d8b/dba/f144 [0,4194304] 0 2024-05-15T15:54:27.960 INFO:tasks.workunit.client.0.smithi049.stdout:0/823: write dc/d13/d3d/fb8 [693458,8180] 0 2024-05-15T15:54:27.962 INFO:tasks.workunit.client.1.smithi144.stdout:8/995: mkdir d2/d24/d2d/da0/d6e/d71/d8c/db9/d143 0 2024-05-15T15:54:27.966 INFO:tasks.workunit.client.0.smithi049.stdout:0/824: write dc/d13/d14/d25/d31/dae/dc5/d57/f65 [297050,85901] 0 2024-05-15T15:54:27.967 INFO:tasks.workunit.client.1.smithi144.stdout:9/927: dwrite d0/d8/d1b/d79/d8a/dae/ddb/f139 [0,4194304] 0 2024-05-15T15:54:27.968 INFO:tasks.workunit.client.0.smithi049.stdout:9/672: rename d0/d8/d9/d52/d5e/d5f/d88 to d0/d8/dd6/de2/dec 0 2024-05-15T15:54:27.968 INFO:tasks.workunit.client.0.smithi049.stdout:9/673: readlink d0/d8/d1b/l6f 0 2024-05-15T15:54:27.970 INFO:tasks.workunit.client.0.smithi049.stdout:1/957: dread d0/d4/dd/de/d48/d84/f12e [0,4194304] 0 2024-05-15T15:54:27.970 INFO:tasks.workunit.client.0.smithi049.stdout:3/773: write d12/d14/d52/db9/da6/f97 [5018251,56615] 0 2024-05-15T15:54:27.973 INFO:tasks.workunit.client.1.smithi144.stdout:7/901: dwrite d4/df/d81/dfb/dbd/fd4 [0,4194304] 0 2024-05-15T15:54:27.974 INFO:tasks.workunit.client.1.smithi144.stdout:9/928: write d0/d8/d1b/d79/de7/df9/f132 [62979,48483] 0 2024-05-15T15:54:27.975 INFO:tasks.workunit.client.1.smithi144.stdout:2/835: dwrite d3/d7/d20/d22/d5e/f96 [0,4194304] 0 2024-05-15T15:54:27.975 INFO:tasks.workunit.client.1.smithi144.stdout:5/913: write d0/dc/d24/d8e/d93/d6b/d88/fce [160081,15910] 0 2024-05-15T15:54:27.980 INFO:tasks.workunit.client.1.smithi144.stdout:4/943: getdents d6/dd/d2e/d75/dff/deb/dba/de0 0 2024-05-15T15:54:27.984 INFO:tasks.workunit.client.0.smithi049.stdout:2/734: sync 2024-05-15T15:54:27.984 INFO:tasks.workunit.client.0.smithi049.stdout:8/788: sync 2024-05-15T15:54:27.984 INFO:tasks.workunit.client.0.smithi049.stdout:7/773: sync 2024-05-15T15:54:27.985 INFO:tasks.workunit.client.0.smithi049.stdout:4/761: sync 2024-05-15T15:54:27.985 INFO:tasks.workunit.client.0.smithi049.stdout:6/883: sync 2024-05-15T15:54:27.986 INFO:tasks.workunit.client.1.smithi144.stdout:8/996: mknod d2/d24/d2d/da0/d9e/db8/dde/c144 0 2024-05-15T15:54:27.998 INFO:tasks.workunit.client.1.smithi144.stdout:0/851: rename d4/d6/d3a/d40/c9c to d4/d6/d4a/d4d/db1/c137 0 2024-05-15T15:54:27.998 INFO:tasks.workunit.client.1.smithi144.stdout:8/997: rename d2/d24 to d2/d24/d2d/d30/dd0/df7/d87/d145 22 2024-05-15T15:54:28.001 INFO:tasks.workunit.client.1.smithi144.stdout:2/836: mknod d3/d7/da4/c115 0 2024-05-15T15:54:28.002 INFO:tasks.workunit.client.1.smithi144.stdout:3/887: dread d0/d1/d17/d39/d40/f92 [0,4194304] 0 2024-05-15T15:54:28.006 INFO:tasks.workunit.client.1.smithi144.stdout:5/914: dread - d0/dc/d2f/f122 zero size 2024-05-15T15:54:28.017 INFO:tasks.workunit.client.0.smithi049.stdout:9/674: mkdir d0/d8/d21/ded 0 2024-05-15T15:54:28.019 INFO:tasks.workunit.client.0.smithi049.stdout:3/774: creat d12/d67/da8/fed x:0 0 0 2024-05-15T15:54:28.020 INFO:tasks.workunit.client.0.smithi049.stdout:1/958: creat d0/d4/dd/de/d1e/d2e/dad/d65/f153 x:0 0 0 2024-05-15T15:54:28.022 INFO:tasks.workunit.client.0.smithi049.stdout:7/774: readlink d0/d6/d1d/l60 0 2024-05-15T15:54:28.022 INFO:tasks.workunit.client.0.smithi049.stdout:4/762: write d18/d1e/d2a/fed [273845,5235] 0 2024-05-15T15:54:28.024 INFO:tasks.workunit.client.0.smithi049.stdout:8/789: read d2/de/d25/f32 [2045542,49722] 0 2024-05-15T15:54:28.026 INFO:tasks.workunit.client.0.smithi049.stdout:2/735: mkdir d3/db/d1d/d1e/d2b/d3b/df8 0 2024-05-15T15:54:28.027 INFO:tasks.workunit.client.0.smithi049.stdout:5/930: rename dc/dd/d7b/cb2 to dc/dd/d47/d83/c133 0 2024-05-15T15:54:28.028 INFO:tasks.workunit.client.0.smithi049.stdout:5/931: stat dc/dd/d47/d83/dcc 0 2024-05-15T15:54:28.034 INFO:tasks.workunit.client.0.smithi049.stdout:7/775: symlink d0/d6/d23/d4a/d7b/lf1 0 2024-05-15T15:54:28.035 INFO:tasks.workunit.client.0.smithi049.stdout:5/932: dwrite dc/d29/f32 [4194304,4194304] 0 2024-05-15T15:54:28.038 INFO:tasks.workunit.client.0.smithi049.stdout:4/763: rename d18/d1e/d63/d99/le2 to d18/d85/db6/lfa 0 2024-05-15T15:54:28.041 INFO:tasks.workunit.client.0.smithi049.stdout:3/775: mknod d12/d14/d32/cee 0 2024-05-15T15:54:28.049 INFO:tasks.workunit.client.0.smithi049.stdout:2/736: mknod d3/cf9 0 2024-05-15T15:54:28.055 INFO:tasks.workunit.client.0.smithi049.stdout:6/884: dwrite dd/d17/d24/f74 [0,4194304] 0 2024-05-15T15:54:28.055 INFO:tasks.workunit.client.0.smithi049.stdout:1/959: rmdir d0/d4/dd/de/d48/d129 0 2024-05-15T15:54:28.064 INFO:tasks.workunit.client.1.smithi144.stdout:7/902: dwrite d4/df/d81/dfb/fc3 [0,4194304] 0 2024-05-15T15:54:28.064 INFO:tasks.workunit.client.1.smithi144.stdout:8/998: fdatasync d2/de/d3f/f4e 0 2024-05-15T15:54:28.065 INFO:tasks.workunit.client.0.smithi049.stdout:7/776: dwrite d0/d6/da/fc0 [0,4194304] 0 2024-05-15T15:54:28.065 INFO:tasks.workunit.client.0.smithi049.stdout:5/933: link dc/db0/dd5/fe1 dc/dd/dc8/f134 0 2024-05-15T15:54:28.065 INFO:tasks.workunit.client.0.smithi049.stdout:2/737: chown d3/db/d1d/d1e/d2b/d6b/l98 45 1 2024-05-15T15:54:28.066 INFO:tasks.workunit.client.1.smithi144.stdout:6/957: getdents d7/db/d9d/d72/dee 0 2024-05-15T15:54:28.068 INFO:tasks.workunit.client.1.smithi144.stdout:7/903: dread - d4/d3c/d76/d9d/dac/d111/d12c/f12f zero size 2024-05-15T15:54:28.068 INFO:tasks.workunit.client.1.smithi144.stdout:4/944: dread d6/dd/d2e/f61 [0,4194304] 0 2024-05-15T15:54:28.069 INFO:tasks.workunit.client.0.smithi049.stdout:6/885: chown dd/d17/d24/d2b/c61 1074511 1 2024-05-15T15:54:28.072 INFO:tasks.workunit.client.0.smithi049.stdout:9/675: dwrite d0/d8/d9/d52/d5e/d5f/f38 [0,4194304] 0 2024-05-15T15:54:28.073 INFO:tasks.workunit.client.1.smithi144.stdout:9/929: dread d0/d8/d5e/d44/d16/f3d [0,4194304] 0 2024-05-15T15:54:28.074 INFO:tasks.workunit.client.1.smithi144.stdout:2/837: mkdir d3/d7/d20/d89/d86/db1/d116 0 2024-05-15T15:54:28.074 INFO:tasks.workunit.client.0.smithi049.stdout:9/676: chown d0/d8/d21/fb2 0 1 2024-05-15T15:54:28.075 INFO:tasks.workunit.client.1.smithi144.stdout:8/999: creat d2/d24/d2d/da0/d6e/f146 x:0 0 0 2024-05-15T15:54:28.079 INFO:tasks.workunit.client.0.smithi049.stdout:5/934: unlink dc/dd/d10/db3/dd6/f112 0 2024-05-15T15:54:28.081 INFO:tasks.workunit.client.0.smithi049.stdout:5/935: stat dc/dd/d7b/fdf 0 2024-05-15T15:54:28.081 INFO:tasks.workunit.client.0.smithi049.stdout:4/764: link d18/f70 d18/d1e/d2a/d36/d86/dbd/ffb 0 2024-05-15T15:54:28.084 INFO:tasks.workunit.client.0.smithi049.stdout:8/790: dread d2/de/d71/d86/d5d/d7d/d1b/d68/f97 [0,4194304] 0 2024-05-15T15:54:28.085 INFO:tasks.workunit.client.0.smithi049.stdout:7/777: mkdir d0/d6/d1d/df2 0 2024-05-15T15:54:28.087 INFO:tasks.workunit.client.1.smithi144.stdout:7/904: mkdir d4/d3c/d76/d9d/dac/d48/d13f 0 2024-05-15T15:54:28.087 INFO:tasks.workunit.client.0.smithi049.stdout:2/738: mknod d3/da/d15/dec/cfa 0 2024-05-15T15:54:28.089 INFO:tasks.workunit.client.1.smithi144.stdout:4/945: creat d6/dd/d2e/d75/dff/deb/dba/f135 x:0 0 0 2024-05-15T15:54:28.090 INFO:tasks.workunit.client.1.smithi144.stdout:4/946: write d6/f29 [1966271,42896] 0 2024-05-15T15:54:28.092 INFO:tasks.workunit.client.1.smithi144.stdout:9/930: creat d0/d8/d5e/d44/d77/f13c x:0 0 0 2024-05-15T15:54:28.093 INFO:tasks.workunit.client.1.smithi144.stdout:3/888: dread d0/f2e [4194304,4194304] 0 2024-05-15T15:54:28.093 INFO:tasks.workunit.client.0.smithi049.stdout:9/677: symlink d0/d8/d21/lee 0 2024-05-15T15:54:28.094 INFO:tasks.workunit.client.1.smithi144.stdout:2/838: symlink d3/d7/d20/d37/d76/db5/db6/l117 0 2024-05-15T15:54:28.099 INFO:tasks.workunit.client.1.smithi144.stdout:2/839: dwrite d3/d7/d20/d3b/f77 [0,4194304] 0 2024-05-15T15:54:28.100 INFO:tasks.workunit.client.1.smithi144.stdout:1/906: sync 2024-05-15T15:54:28.101 INFO:tasks.workunit.client.1.smithi144.stdout:2/840: stat d3/d10/f29 0 2024-05-15T15:54:28.101 INFO:tasks.workunit.client.1.smithi144.stdout:5/915: dread d0/dc/d24/d8e/d93/d9e/fcb [0,4194304] 0 2024-05-15T15:54:28.101 INFO:tasks.workunit.client.1.smithi144.stdout:6/958: truncate d7/db/d3a/f91 3295273 0 2024-05-15T15:54:28.104 INFO:tasks.workunit.client.1.smithi144.stdout:6/959: write d7/f11c [362575,20863] 0 2024-05-15T15:54:28.104 INFO:tasks.workunit.client.1.smithi144.stdout:1/907: read d7/d1d/d33/d1e/d34/dd8/fe0 [42411,102158] 0 2024-05-15T15:54:28.105 INFO:tasks.workunit.client.1.smithi144.stdout:5/916: read d0/dc/d24/d8e/d93/d6b/d88/fce [127296,122386] 0 2024-05-15T15:54:28.109 INFO:tasks.workunit.client.1.smithi144.stdout:7/905: rmdir d4/df/d81/dfb/dbd 39 2024-05-15T15:54:28.110 INFO:tasks.workunit.client.0.smithi049.stdout:3/776: dread d12/d14/d32/f59 [0,4194304] 0 2024-05-15T15:54:28.111 INFO:tasks.workunit.client.1.smithi144.stdout:7/906: write d4/d3c/d76/d9d/dac/d48/f126 [208623,78189] 0 2024-05-15T15:54:28.111 INFO:tasks.workunit.client.0.smithi049.stdout:5/936: truncate dc/dd/d47/d83/dcc/f7a 4891215 0 2024-05-15T15:54:28.112 INFO:tasks.workunit.client.0.smithi049.stdout:9/678: mknod d0/d8/d9/d52/d5e/d8a/dd3/d7d/cef 0 2024-05-15T15:54:28.117 INFO:tasks.workunit.client.0.smithi049.stdout:9/679: dwrite d0/d8/d9/d52/d5e/d5f/f38 [0,4194304] 0 2024-05-15T15:54:28.147 INFO:tasks.workunit.client.1.smithi144.stdout:0/852: sync 2024-05-15T15:54:28.153 INFO:tasks.workunit.client.0.smithi049.stdout:9/680: truncate d0/d8/d9/d52/d5e/d8a/dac/fcb 169649 0 2024-05-15T15:54:28.157 INFO:tasks.workunit.client.0.smithi049.stdout:4/765: mknod d18/d1e/d2a/d50/cfc 0 2024-05-15T15:54:28.161 INFO:tasks.workunit.client.0.smithi049.stdout:4/766: chown d18/d1e/d2a/d36/f4b 1 1 2024-05-15T15:54:28.161 INFO:tasks.workunit.client.0.smithi049.stdout:2/739: rmdir d3/db/d82 39 2024-05-15T15:54:28.162 INFO:tasks.workunit.client.0.smithi049.stdout:8/791: dwrite d2/f7b [0,4194304] 0 2024-05-15T15:54:28.165 INFO:tasks.workunit.client.0.smithi049.stdout:2/740: read - d3/db/d1d/f5e zero size 2024-05-15T15:54:28.166 INFO:tasks.workunit.client.1.smithi144.stdout:6/960: rmdir d7/d1e/d137 39 2024-05-15T15:54:28.167 INFO:tasks.workunit.client.0.smithi049.stdout:7/778: dwrite d0/d6/d23/faf [0,4194304] 0 2024-05-15T15:54:28.167 INFO:tasks.workunit.client.0.smithi049.stdout:8/792: write d2/de/f16 [3043446,99631] 0 2024-05-15T15:54:28.169 INFO:tasks.workunit.client.0.smithi049.stdout:4/767: write d18/d1e/d63/d99/db9/fe0 [449653,2849] 0 2024-05-15T15:54:28.171 INFO:tasks.workunit.client.1.smithi144.stdout:1/908: rename d7/d1d/d33/d29/d6b/d5e/d74/df3/fdb to d7/d1d/d58/d81/f11c 0 2024-05-15T15:54:28.175 INFO:tasks.workunit.client.0.smithi049.stdout:1/960: getdents d0/d4/dd/de/d48/d84/d134 0 2024-05-15T15:54:28.177 INFO:tasks.workunit.client.1.smithi144.stdout:4/947: symlink d6/dd/d34/d3b/d30/db/d1d/d101/l136 0 2024-05-15T15:54:28.182 INFO:tasks.workunit.client.1.smithi144.stdout:5/917: dwrite d0/d10c/f113 [0,4194304] 0 2024-05-15T15:54:28.183 INFO:tasks.workunit.client.1.smithi144.stdout:3/889: mkdir d0/d11/d11e 0 2024-05-15T15:54:28.187 INFO:tasks.workunit.client.0.smithi049.stdout:3/777: dwrite d12/d14/d32/d64/d88/d9c/fb4 [0,4194304] 0 2024-05-15T15:54:28.188 INFO:tasks.workunit.client.1.smithi144.stdout:2/841: dwrite d3/d7/d20/d37/d76/db5/fbd [0,4194304] 0 2024-05-15T15:54:28.190 INFO:tasks.workunit.client.1.smithi144.stdout:7/907: dread d4/df/d81/dfb/dbd/fd4 [0,4194304] 0 2024-05-15T15:54:28.191 INFO:tasks.workunit.client.0.smithi049.stdout:5/937: dwrite dc/dd/d7b/f93 [4194304,4194304] 0 2024-05-15T15:54:28.198 INFO:tasks.workunit.client.0.smithi049.stdout:9/681: write d0/d8/d1b/d40/d5c/f60 [8017,24168] 0 2024-05-15T15:54:28.202 INFO:tasks.workunit.client.1.smithi144.stdout:1/909: symlink d7/d1d/dac/l11d 0 2024-05-15T15:54:28.203 INFO:tasks.workunit.client.0.smithi049.stdout:5/938: dwrite dc/dd/d47/d83/dcc/f115 [0,4194304] 0 2024-05-15T15:54:28.210 INFO:tasks.workunit.client.1.smithi144.stdout:4/948: unlink d6/dd/d34/d3b/d30/dc9/d10f/d4b/l1a 0 2024-05-15T15:54:28.211 INFO:tasks.workunit.client.1.smithi144.stdout:0/853: dread d4/df/d1f/f58 [0,4194304] 0 2024-05-15T15:54:28.213 INFO:tasks.workunit.client.0.smithi049.stdout:8/793: unlink d2/d18/l20 0 2024-05-15T15:54:28.217 INFO:tasks.workunit.client.1.smithi144.stdout:5/918: creat d0/dc/d24/d8e/d93/d6b/f130 x:0 0 0 2024-05-15T15:54:28.218 INFO:tasks.workunit.client.0.smithi049.stdout:4/768: mknod d18/d1e/cfd 0 2024-05-15T15:54:28.219 INFO:tasks.workunit.client.0.smithi049.stdout:1/961: mkdir d0/d4/dd/de/d1e/d2e/dad/d65/d72/dae/d154 0 2024-05-15T15:54:28.220 INFO:tasks.workunit.client.1.smithi144.stdout:7/908: mknod d4/d3c/d76/d9d/dac/d48/c140 0 2024-05-15T15:54:28.221 INFO:tasks.workunit.client.1.smithi144.stdout:7/909: chown d4/d3c/d76/d9d/dac/d48/d72/d82/f92 69 1 2024-05-15T15:54:28.221 INFO:tasks.workunit.client.0.smithi049.stdout:5/939: mkdir dc/dd/d7b/d8d/d135 0 2024-05-15T15:54:28.221 INFO:tasks.workunit.client.0.smithi049.stdout:5/940: stat dc/l78 0 2024-05-15T15:54:28.223 INFO:tasks.workunit.client.0.smithi049.stdout:8/794: unlink d2/de/d71/d86/f96 0 2024-05-15T15:54:28.226 INFO:tasks.workunit.client.0.smithi049.stdout:4/769: creat d18/d1e/d2a/d50/dc4/ffe x:0 0 0 2024-05-15T15:54:28.227 INFO:tasks.workunit.client.0.smithi049.stdout:1/962: creat d0/d4/dd/de/d1e/d16/d8a/dd2/f155 x:0 0 0 2024-05-15T15:54:28.228 INFO:tasks.workunit.client.0.smithi049.stdout:1/963: chown d0/d4/d81/f97 82782 1 2024-05-15T15:54:28.231 INFO:tasks.workunit.client.0.smithi049.stdout:8/795: mknod d2/de/d25/d74/dbe/dd8/cf4 0 2024-05-15T15:54:28.239 INFO:tasks.workunit.client.0.smithi049.stdout:3/778: dread d12/f47 [0,4194304] 0 2024-05-15T15:54:28.240 INFO:tasks.workunit.client.0.smithi049.stdout:5/941: link dc/f64 dc/db0/dbf/f136 0 2024-05-15T15:54:28.242 INFO:tasks.workunit.client.0.smithi049.stdout:5/942: chown dc/f9a 22708359 1 2024-05-15T15:54:28.244 INFO:tasks.workunit.client.0.smithi049.stdout:3/779: mkdir d12/d14/d52/def 0 2024-05-15T15:54:28.245 INFO:tasks.workunit.client.1.smithi144.stdout:3/890: write d0/f46 [1134507,119711] 0 2024-05-15T15:54:28.246 INFO:tasks.workunit.client.0.smithi049.stdout:7/779: dwrite d0/d2f/f97 [0,4194304] 0 2024-05-15T15:54:28.246 INFO:tasks.workunit.client.1.smithi144.stdout:3/891: fdatasync d0/d1/d17/d39/d40/d7e/fd5 0 2024-05-15T15:54:28.249 INFO:tasks.workunit.client.0.smithi049.stdout:4/770: dread d18/d1e/d63/d99/fa5 [0,4194304] 0 2024-05-15T15:54:28.254 INFO:tasks.workunit.client.0.smithi049.stdout:1/964: dread d0/d4/dd/de/d1e/d2e/dad/d65/d72/f9d [0,4194304] 0 2024-05-15T15:54:28.257 INFO:tasks.workunit.client.0.smithi049.stdout:5/943: fdatasync dc/dd/dc8/f134 0 2024-05-15T15:54:28.263 INFO:tasks.workunit.client.0.smithi049.stdout:9/682: dwrite d0/f65 [0,4194304] 0 2024-05-15T15:54:28.266 INFO:tasks.workunit.client.0.smithi049.stdout:4/771: dread d18/d1e/d2a/d36/f83 [0,4194304] 0 2024-05-15T15:54:28.268 INFO:tasks.workunit.client.0.smithi049.stdout:3/780: symlink d12/lf0 0 2024-05-15T15:54:28.274 INFO:tasks.workunit.client.0.smithi049.stdout:5/944: truncate dc/f30 1406479 0 2024-05-15T15:54:28.274 INFO:tasks.workunit.client.0.smithi049.stdout:5/945: readlink dc/d4a/l89 0 2024-05-15T15:54:28.276 INFO:tasks.workunit.client.0.smithi049.stdout:7/780: mknod d0/d34/cf3 0 2024-05-15T15:54:28.281 INFO:tasks.workunit.client.0.smithi049.stdout:9/683: mkdir d0/d8/d1b/db3/df0 0 2024-05-15T15:54:28.284 INFO:tasks.workunit.client.0.smithi049.stdout:5/946: dread dc/dd/f13 [0,4194304] 0 2024-05-15T15:54:28.285 INFO:tasks.workunit.client.0.smithi049.stdout:3/781: mkdir d12/d67/da8/df1 0 2024-05-15T15:54:28.287 INFO:tasks.workunit.client.0.smithi049.stdout:7/781: readlink d0/l58 0 2024-05-15T15:54:28.290 INFO:tasks.workunit.client.0.smithi049.stdout:4/772: mkdir d18/d85/dff 0 2024-05-15T15:54:28.296 INFO:tasks.workunit.client.0.smithi049.stdout:7/782: creat d0/d6/d23/d76/ff4 x:0 0 0 2024-05-15T15:54:28.299 INFO:tasks.workunit.client.0.smithi049.stdout:9/684: unlink d0/d8/fb6 0 2024-05-15T15:54:28.300 INFO:tasks.workunit.client.1.smithi144.stdout:9/931: getdents d0/d8/d5e/d44/d10/d19/dd5/d12b 0 2024-05-15T15:54:28.302 INFO:tasks.workunit.client.1.smithi144.stdout:5/919: chown d0/dc/d1f/d50/db3 410253 1 2024-05-15T15:54:28.302 INFO:tasks.workunit.client.1.smithi144.stdout:4/949: dread d6/dd/d34/d3b/d30/db/d1d/d40/d57/f5e [0,4194304] 0 2024-05-15T15:54:28.303 INFO:tasks.workunit.client.0.smithi049.stdout:5/947: dread dc/d4a/d42/f48 [0,4194304] 0 2024-05-15T15:54:28.305 INFO:tasks.workunit.client.0.smithi049.stdout:3/782: mkdir d12/d14/d52/db9/df2 0 2024-05-15T15:54:28.307 INFO:tasks.workunit.client.1.smithi144.stdout:6/961: rename d7/db/d35/d84/d81/fca to d7/d62/f157 0 2024-05-15T15:54:28.307 INFO:tasks.workunit.client.1.smithi144.stdout:2/842: rename d3/d7 to d3/d7/d20/d37/de1/d118 22 2024-05-15T15:54:28.309 INFO:tasks.workunit.client.0.smithi049.stdout:7/783: creat d0/d6/d23/d76/ff5 x:0 0 0 2024-05-15T15:54:28.310 INFO:tasks.workunit.client.1.smithi144.stdout:1/910: unlink d7/d1d/d33/d41/d78/da3/cfb 0 2024-05-15T15:54:28.313 INFO:tasks.workunit.client.0.smithi049.stdout:9/685: dread d0/d8/d12/f6e [0,4194304] 0 2024-05-15T15:54:28.316 INFO:tasks.workunit.client.1.smithi144.stdout:9/932: symlink d0/d8/d1b/d79/d8a/dae/ddb/dfc/l13d 0 2024-05-15T15:54:28.318 INFO:tasks.workunit.client.0.smithi049.stdout:3/783: chown d12/d14/d32/l35 1114459 1 2024-05-15T15:54:28.319 INFO:tasks.workunit.client.0.smithi049.stdout:3/784: readlink d12/d14/d52/db9/d6b/dcf/ld2 0 2024-05-15T15:54:28.320 INFO:tasks.workunit.client.1.smithi144.stdout:5/920: creat d0/dc/d24/d28/f131 x:0 0 0 2024-05-15T15:54:28.320 INFO:tasks.workunit.client.0.smithi049.stdout:7/784: stat d0/f9 0 2024-05-15T15:54:28.321 INFO:tasks.workunit.client.0.smithi049.stdout:3/785: chown d12/d14/d52/db9/f48 52 1 2024-05-15T15:54:28.321 INFO:tasks.workunit.client.1.smithi144.stdout:7/910: sync 2024-05-15T15:54:28.323 INFO:tasks.workunit.client.1.smithi144.stdout:4/950: unlink d6/dd/d34/d3b/d30/db/l8f 0 2024-05-15T15:54:28.324 INFO:tasks.workunit.client.1.smithi144.stdout:2/843: dread d3/d7/d20/d22/d52/d64/fd3 [0,4194304] 0 2024-05-15T15:54:28.327 INFO:tasks.workunit.client.1.smithi144.stdout:6/962: rename d7/db/d11/d7e/f8c to d7/db/d11/d7e/d132/f158 0 2024-05-15T15:54:28.328 INFO:tasks.workunit.client.1.smithi144.stdout:6/963: chown d7/db/d11/d3c/ce6 13749410 1 2024-05-15T15:54:28.329 INFO:tasks.workunit.client.1.smithi144.stdout:1/911: chown d7/d1d/d33/c63 160 1 2024-05-15T15:54:28.332 INFO:tasks.workunit.client.0.smithi049.stdout:7/785: truncate d0/d34/db3/fc3 200927 0 2024-05-15T15:54:28.333 INFO:tasks.workunit.client.1.smithi144.stdout:0/854: mknod d4/df/d1f/db7/c138 0 2024-05-15T15:54:28.335 INFO:tasks.workunit.client.0.smithi049.stdout:3/786: rename d12/d14/d32/d64/d88/fb5 to d12/d14/d52/db9/dc4/d68/d70/d75/dac/ff3 0 2024-05-15T15:54:28.337 INFO:tasks.workunit.client.1.smithi144.stdout:9/933: creat d0/d8/d1b/d62/d7d/dd2/f13e x:0 0 0 2024-05-15T15:54:28.337 INFO:tasks.workunit.client.0.smithi049.stdout:4/773: getdents d18/d85/db6 0 2024-05-15T15:54:28.339 INFO:tasks.workunit.client.1.smithi144.stdout:3/892: write d0/f69 [3566580,50150] 0 2024-05-15T15:54:28.340 INFO:tasks.workunit.client.1.smithi144.stdout:3/893: chown d0/d11/d3b/c54 18351901 1 2024-05-15T15:54:28.341 INFO:tasks.workunit.client.1.smithi144.stdout:7/911: symlink d4/d3c/d76/d9d/dac/d111/d12c/l141 0 2024-05-15T15:54:28.343 INFO:tasks.workunit.client.1.smithi144.stdout:4/951: mkdir d6/dd/d2e/d94/d137 0 2024-05-15T15:54:28.344 INFO:tasks.workunit.client.1.smithi144.stdout:4/952: chown d6/dd/fd2 0 1 2024-05-15T15:54:28.345 INFO:tasks.workunit.client.0.smithi049.stdout:9/686: write d0/d8/d42/d75/fa5 [33597,127741] 0 2024-05-15T15:54:28.349 INFO:tasks.workunit.client.0.smithi049.stdout:3/787: creat d12/d67/da8/ddd/ff4 x:0 0 0 2024-05-15T15:54:28.351 INFO:tasks.workunit.client.0.smithi049.stdout:3/788: write d12/d14/d32/f6d [4883966,15571] 0 2024-05-15T15:54:28.352 INFO:tasks.workunit.client.0.smithi049.stdout:4/774: creat d18/d1e/d2a/d50/dbc/f100 x:0 0 0 2024-05-15T15:54:28.354 INFO:tasks.workunit.client.1.smithi144.stdout:6/964: unlink d7/f4b 0 2024-05-15T15:54:28.355 INFO:tasks.workunit.client.0.smithi049.stdout:5/948: getdents dc/db0/dc7 0 2024-05-15T15:54:28.356 INFO:tasks.workunit.client.1.smithi144.stdout:2/844: write d3/d7/d20/d37/d76/db5/db6/dbf/fd0 [704998,93167] 0 2024-05-15T15:54:28.357 INFO:tasks.workunit.client.1.smithi144.stdout:1/912: dread d7/d1d/d33/d29/d6b/d5e/d74/fab [0,4194304] 0 2024-05-15T15:54:28.358 INFO:tasks.workunit.client.0.smithi049.stdout:7/786: mknod d0/cf6 0 2024-05-15T15:54:28.360 INFO:tasks.workunit.client.1.smithi144.stdout:0/855: unlink d4/d6/d3a/d4c/dd6/d55/l8c 0 2024-05-15T15:54:28.362 INFO:tasks.workunit.client.0.smithi049.stdout:5/949: dwrite dc/d4a/d9e/d10b/f12e [0,4194304] 0 2024-05-15T15:54:28.369 INFO:tasks.workunit.client.1.smithi144.stdout:6/965: creat d7/db/d11/d7e/f159 x:0 0 0 2024-05-15T15:54:28.369 INFO:tasks.workunit.client.0.smithi049.stdout:9/687: mknod d0/d8/d1b/d40/d5c/d69/cf1 0 2024-05-15T15:54:28.375 INFO:tasks.workunit.client.1.smithi144.stdout:1/913: unlink d7/d4b/dd9/l9e 0 2024-05-15T15:54:28.377 INFO:tasks.workunit.client.1.smithi144.stdout:9/934: getdents d0/d8/d7a 0 2024-05-15T15:54:28.377 INFO:tasks.workunit.client.1.smithi144.stdout:9/935: chown d0/d8/d5e/d44/d16/db0/c12a 822510 1 2024-05-15T15:54:28.378 INFO:tasks.workunit.client.1.smithi144.stdout:0/856: rename d4 to d4/d2d/d48/d5b/d139 22 2024-05-15T15:54:28.380 INFO:tasks.workunit.client.0.smithi049.stdout:7/787: fsync d0/d6/d1d/f4e 0 2024-05-15T15:54:28.382 INFO:tasks.workunit.client.1.smithi144.stdout:3/894: write d0/d11/d3b/da0/d8f/fe0 [167382,129416] 0 2024-05-15T15:54:28.382 INFO:tasks.workunit.client.0.smithi049.stdout:7/788: write d0/d6/d1d/f4d [5120037,111263] 0 2024-05-15T15:54:28.383 INFO:tasks.workunit.client.1.smithi144.stdout:4/953: write d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/f51 [1162970,74481] 0 2024-05-15T15:54:28.384 INFO:tasks.workunit.client.0.smithi049.stdout:3/789: creat d12/ff5 x:0 0 0 2024-05-15T15:54:28.387 INFO:tasks.workunit.client.1.smithi144.stdout:2/845: write d3/d7/d54/d98/f9c [659202,109030] 0 2024-05-15T15:54:28.392 INFO:tasks.workunit.client.0.smithi049.stdout:5/950: truncate dc/dd/f14 4246502 0 2024-05-15T15:54:28.393 INFO:tasks.workunit.client.1.smithi144.stdout:2/846: dwrite d3/d4d/db4/f10e [0,4194304] 0 2024-05-15T15:54:28.395 INFO:tasks.workunit.client.0.smithi049.stdout:9/688: symlink d0/d8/d12/d9a/lf2 0 2024-05-15T15:54:28.396 INFO:tasks.workunit.client.1.smithi144.stdout:9/936: symlink d0/d8/d5e/d44/d16/db0/l13f 0 2024-05-15T15:54:28.396 INFO:tasks.workunit.client.0.smithi049.stdout:4/775: dwrite f16 [4194304,4194304] 0 2024-05-15T15:54:28.401 INFO:tasks.workunit.client.1.smithi144.stdout:4/954: dread d6/dd/d34/d3b/d30/db/d1d/d40/d57/f85 [0,4194304] 0 2024-05-15T15:54:28.403 INFO:tasks.workunit.client.1.smithi144.stdout:1/914: write d7/d1d/d33/d29/d6b/f65 [215877,47820] 0 2024-05-15T15:54:28.404 INFO:tasks.workunit.client.0.smithi049.stdout:3/790: mkdir d12/d14/d52/db9/dc4/d68/d70/d75/d9d/df6 0 2024-05-15T15:54:28.405 INFO:tasks.workunit.client.1.smithi144.stdout:4/955: stat d6/dd/d34/d3b/d30/d59/d66/dbd/l10a 0 2024-05-15T15:54:28.406 INFO:tasks.workunit.client.1.smithi144.stdout:1/915: fsync d7/d1d/d33/d29/d6b/d8e/fca 0 2024-05-15T15:54:28.406 INFO:tasks.workunit.client.1.smithi144.stdout:9/937: dwrite d0/d8/d1b/f6f [0,4194304] 0 2024-05-15T15:54:28.406 INFO:tasks.workunit.client.1.smithi144.stdout:5/921: creat d0/dc/d12/f132 x:0 0 0 2024-05-15T15:54:28.407 INFO:tasks.workunit.client.1.smithi144.stdout:3/895: rmdir d0/d11/d3b/da0/df6 39 2024-05-15T15:54:28.408 INFO:tasks.workunit.client.1.smithi144.stdout:5/922: chown d0/dc/d24/cf1 115251 1 2024-05-15T15:54:28.411 INFO:tasks.workunit.client.1.smithi144.stdout:7/912: creat d4/d3c/d76/d9d/dac/f142 x:0 0 0 2024-05-15T15:54:28.411 INFO:tasks.workunit.client.0.smithi049.stdout:7/789: write d0/d6/da/f61 [632977,83757] 0 2024-05-15T15:54:28.411 INFO:tasks.workunit.client.1.smithi144.stdout:7/913: stat d4/df/ff4 0 2024-05-15T15:54:28.416 INFO:tasks.workunit.client.1.smithi144.stdout:4/956: dread d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/f51 [0,4194304] 0 2024-05-15T15:54:28.418 INFO:tasks.workunit.client.1.smithi144.stdout:4/957: readlink d6/dd/d34/d3b/d30/d59/d66/dbd/l10a 0 2024-05-15T15:54:28.419 INFO:tasks.workunit.client.1.smithi144.stdout:0/857: dwrite d4/d6/d3a/d4c/dd6/f86 [0,4194304] 0 2024-05-15T15:54:28.420 INFO:tasks.workunit.client.1.smithi144.stdout:0/858: stat d4/d2d/fce 0 2024-05-15T15:54:28.420 INFO:tasks.workunit.client.0.smithi049.stdout:5/951: symlink dc/dd/d7b/l137 0 2024-05-15T15:54:28.423 INFO:tasks.workunit.client.0.smithi049.stdout:7/790: write d0/d2f/f3e [4453653,28918] 0 2024-05-15T15:54:28.426 INFO:tasks.workunit.client.0.smithi049.stdout:9/689: getdents d0/d8/d9/d52/d5e/d8a/dd3/d87/dd1 0 2024-05-15T15:54:28.436 INFO:tasks.workunit.client.0.smithi049.stdout:9/690: dwrite d0/d8/d42/d75/fcf [0,4194304] 0 2024-05-15T15:54:28.436 INFO:tasks.workunit.client.1.smithi144.stdout:7/914: rmdir d4/d41 39 2024-05-15T15:54:28.438 INFO:tasks.workunit.client.0.smithi049.stdout:9/691: chown d0/d8/dd6/de2/dec/d9d 268 1 2024-05-15T15:54:28.438 INFO:tasks.workunit.client.0.smithi049.stdout:9/692: fdatasync d0/d8/d1b/d40/fbc 0 2024-05-15T15:54:28.441 INFO:tasks.workunit.client.1.smithi144.stdout:3/896: dread d0/d1/fd9 [0,4194304] 0 2024-05-15T15:54:28.441 INFO:tasks.workunit.client.0.smithi049.stdout:7/791: truncate d0/f11 2957771 0 2024-05-15T15:54:28.444 INFO:tasks.workunit.client.1.smithi144.stdout:4/958: creat d6/dd/d34/de2/f138 x:0 0 0 2024-05-15T15:54:28.445 INFO:tasks.workunit.client.0.smithi049.stdout:3/791: dread d12/f25 [0,4194304] 0 2024-05-15T15:54:28.445 INFO:tasks.workunit.client.1.smithi144.stdout:9/938: dread d0/d8/f11 [0,4194304] 0 2024-05-15T15:54:28.447 INFO:tasks.workunit.client.0.smithi049.stdout:9/693: mkdir d0/d8/d53/dd2/df3 0 2024-05-15T15:54:28.448 INFO:tasks.workunit.client.0.smithi049.stdout:3/792: readlink d12/d14/d52/db9/l9a 0 2024-05-15T15:54:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:28 smithi144 ceph-mon[36389]: pgmap v34: 65 pgs: 65 active+clean; 3.4 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 108 MiB/s rd, 259 MiB/s wr, 569 op/s 2024-05-15T15:54:28.451 INFO:tasks.workunit.client.1.smithi144.stdout:1/916: write d7/d1d/d33/d41/f19 [329913,110742] 0 2024-05-15T15:54:28.451 INFO:tasks.workunit.client.1.smithi144.stdout:0/859: mkdir d4/d6/d4a/d4d/db1/d13a 0 2024-05-15T15:54:28.452 INFO:tasks.workunit.client.0.smithi049.stdout:9/694: chown d0/d8/d21/fa4 36 1 2024-05-15T15:54:28.453 INFO:tasks.workunit.client.1.smithi144.stdout:2/847: dwrite d3/d4d/fb7 [0,4194304] 0 2024-05-15T15:54:28.453 INFO:tasks.workunit.client.1.smithi144.stdout:0/860: read d4/d6/fb [3887817,30957] 0 2024-05-15T15:54:28.454 INFO:tasks.workunit.client.0.smithi049.stdout:7/792: dwrite d0/d6/da/fea [0,4194304] 0 2024-05-15T15:54:28.454 INFO:tasks.workunit.client.1.smithi144.stdout:1/917: truncate d7/d1d/d33/d29/d6b/d5e/f117 641580 0 2024-05-15T15:54:28.455 INFO:tasks.workunit.client.0.smithi049.stdout:3/793: stat d12/d14/d52/db9/dc4/d68/d70/d75/d9d/df6 0 2024-05-15T15:54:28.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:28 smithi049 ceph-mon[28399]: pgmap v34: 65 pgs: 65 active+clean; 3.4 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 108 MiB/s rd, 259 MiB/s wr, 569 op/s 2024-05-15T15:54:28.457 INFO:tasks.workunit.client.1.smithi144.stdout:5/923: fdatasync d0/d10c/f124 0 2024-05-15T15:54:28.460 INFO:tasks.workunit.client.1.smithi144.stdout:6/966: getdents d7/db/d3a/de3/df2/d11e 0 2024-05-15T15:54:28.461 INFO:tasks.workunit.client.0.smithi049.stdout:3/794: readlink d12/l6c 0 2024-05-15T15:54:28.462 INFO:tasks.workunit.client.0.smithi049.stdout:4/776: dwrite d18/d1e/d63/fb8 [0,4194304] 0 2024-05-15T15:54:28.466 INFO:tasks.workunit.client.1.smithi144.stdout:3/897: symlink d0/d11/d24/da5/l11f 0 2024-05-15T15:54:28.466 INFO:tasks.workunit.client.0.smithi049.stdout:5/952: dread f0 [0,4194304] 0 2024-05-15T15:54:28.471 INFO:tasks.workunit.client.1.smithi144.stdout:4/959: dread d6/d49/f5c [8388608,4194304] 0 2024-05-15T15:54:28.472 INFO:tasks.workunit.client.0.smithi049.stdout:9/695: mknod d0/d8/d9/d52/d5e/d8a/cf4 0 2024-05-15T15:54:28.474 INFO:tasks.workunit.client.0.smithi049.stdout:7/793: mkdir d0/d6/df7 0 2024-05-15T15:54:28.477 INFO:tasks.workunit.client.0.smithi049.stdout:3/795: mkdir d12/d14/d32/d64/df7 0 2024-05-15T15:54:28.481 INFO:tasks.workunit.client.0.smithi049.stdout:5/953: truncate dc/dd/d47/d83/fc0 1012957 0 2024-05-15T15:54:28.481 INFO:tasks.workunit.client.1.smithi144.stdout:0/861: fsync d4/d2d/f8d 0 2024-05-15T15:54:28.482 INFO:tasks.workunit.client.0.smithi049.stdout:9/696: dwrite d0/d8/d1b/d40/d5c/f60 [0,4194304] 0 2024-05-15T15:54:28.491 INFO:tasks.workunit.client.1.smithi144.stdout:4/960: dread d6/dd/d34/d3b/d30/d59/fe1 [0,4194304] 0 2024-05-15T15:54:28.495 INFO:tasks.workunit.client.0.smithi049.stdout:9/697: dwrite d0/d8/d9/d45/feb [0,4194304] 0 2024-05-15T15:54:28.496 INFO:tasks.workunit.client.0.smithi049.stdout:5/954: fsync dc/dd/d47/da6/dce/f10f 0 2024-05-15T15:54:28.497 INFO:tasks.workunit.client.0.smithi049.stdout:3/796: dwrite f7 [0,4194304] 0 2024-05-15T15:54:28.500 INFO:tasks.workunit.client.1.smithi144.stdout:9/939: mkdir d0/d8/d5e/d44/d10/d19/d140 0 2024-05-15T15:54:28.502 INFO:tasks.workunit.client.0.smithi049.stdout:4/777: link d18/d85/d98/ff0 d18/d1e/f101 0 2024-05-15T15:54:28.505 INFO:tasks.workunit.client.0.smithi049.stdout:4/778: read d18/d21/d2b/d4e/f87 [555331,87915] 0 2024-05-15T15:54:28.507 INFO:tasks.workunit.client.0.smithi049.stdout:0/825: sync 2024-05-15T15:54:28.507 INFO:tasks.workunit.client.1.smithi144.stdout:1/918: write d7/d1d/d33/d29/d6b/d5e/fa1 [3375784,85973] 0 2024-05-15T15:54:28.508 INFO:tasks.workunit.client.0.smithi049.stdout:0/826: stat dc/da4/db4 0 2024-05-15T15:54:28.510 INFO:tasks.workunit.client.0.smithi049.stdout:5/955: rename dc/fec to dc/dd/d47/d83/f138 0 2024-05-15T15:54:28.512 INFO:tasks.workunit.client.1.smithi144.stdout:7/915: write d4/df/d25/f35 [540923,48683] 0 2024-05-15T15:54:28.514 INFO:tasks.workunit.client.0.smithi049.stdout:7/794: dwrite d0/d6/d1d/f59 [0,4194304] 0 2024-05-15T15:54:28.517 INFO:tasks.workunit.client.1.smithi144.stdout:6/967: dwrite f6 [4194304,4194304] 0 2024-05-15T15:54:28.519 INFO:tasks.workunit.client.0.smithi049.stdout:9/698: write d0/fa2 [804183,104275] 0 2024-05-15T15:54:28.523 INFO:tasks.workunit.client.0.smithi049.stdout:7/795: dwrite d0/d6/d23/d4a/d7b/de2/fee [0,4194304] 0 2024-05-15T15:54:28.525 INFO:tasks.workunit.client.1.smithi144.stdout:0/862: dwrite d4/df/da0/fa8 [0,4194304] 0 2024-05-15T15:54:28.530 INFO:tasks.workunit.client.0.smithi049.stdout:4/779: rmdir d18 39 2024-05-15T15:54:28.530 INFO:tasks.workunit.client.0.smithi049.stdout:9/699: dwrite d0/d8/d42/d75/fcf [0,4194304] 0 2024-05-15T15:54:28.532 INFO:tasks.workunit.client.1.smithi144.stdout:4/961: chown d6/dd/d34/d3b/d30/db/d1d/d40/l7a 327731500 1 2024-05-15T15:54:28.532 INFO:tasks.workunit.client.1.smithi144.stdout:0/863: dwrite d4/df/d14/f5f [0,4194304] 0 2024-05-15T15:54:28.533 INFO:tasks.workunit.client.1.smithi144.stdout:4/962: chown d6/d42/f112 99 1 2024-05-15T15:54:28.537 INFO:tasks.workunit.client.0.smithi049.stdout:5/956: mknod dc/d29/dde/c139 0 2024-05-15T15:54:28.538 INFO:tasks.workunit.client.0.smithi049.stdout:0/827: fdatasync dc/fa1 0 2024-05-15T15:54:28.539 INFO:tasks.workunit.client.0.smithi049.stdout:5/957: fdatasync dc/d29/f122 0 2024-05-15T15:54:28.539 INFO:tasks.workunit.client.1.smithi144.stdout:3/898: rename d0/d1/c5d to d0/d11/d24/da5/c120 0 2024-05-15T15:54:28.541 INFO:tasks.workunit.client.1.smithi144.stdout:4/963: dread d6/dd/d34/d3b/d30/db/d1d/d40/d57/f85 [0,4194304] 0 2024-05-15T15:54:28.544 INFO:tasks.workunit.client.1.smithi144.stdout:2/848: rmdir d3/d7/da4/de2 0 2024-05-15T15:54:28.548 INFO:tasks.workunit.client.0.smithi049.stdout:4/780: write d18/d1e/d2a/d36/ff9 [976725,113746] 0 2024-05-15T15:54:28.551 INFO:tasks.workunit.client.1.smithi144.stdout:1/919: dwrite d7/d1d/d33/d1e/d34/ff8 [0,4194304] 0 2024-05-15T15:54:28.553 INFO:tasks.workunit.client.1.smithi144.stdout:6/968: truncate d7/db/d11/f1b 4044908 0 2024-05-15T15:54:28.554 INFO:tasks.workunit.client.0.smithi049.stdout:3/797: truncate d12/d14/d32/fd0 4699454 0 2024-05-15T15:54:28.554 INFO:tasks.workunit.client.1.smithi144.stdout:6/969: read - d7/db/d35/d84/dbe/f14f zero size 2024-05-15T15:54:28.557 INFO:tasks.workunit.client.1.smithi144.stdout:1/920: dwrite d7/d1d/d33/d41/f19 [0,4194304] 0 2024-05-15T15:54:28.558 INFO:tasks.workunit.client.1.smithi144.stdout:0/864: creat d4/d2d/d48/d10a/f13b x:0 0 0 2024-05-15T15:54:28.562 INFO:tasks.workunit.client.1.smithi144.stdout:5/924: rmdir d0/dc/d1f/d50/dfe 0 2024-05-15T15:54:28.564 INFO:tasks.workunit.client.1.smithi144.stdout:4/964: symlink d6/dd/d34/d3b/d30/d59/d66/dcf/l139 0 2024-05-15T15:54:28.565 INFO:tasks.workunit.client.0.smithi049.stdout:4/781: truncate d18/d1e/d2a/d36/f58 3022077 0 2024-05-15T15:54:28.567 INFO:tasks.workunit.client.1.smithi144.stdout:2/849: read - d3/d7/d20/d22/fb9 zero size 2024-05-15T15:54:28.568 INFO:tasks.workunit.client.0.smithi049.stdout:9/700: mknod d0/d8/d9/d22/dbb/cf5 0 2024-05-15T15:54:28.576 INFO:tasks.workunit.client.0.smithi049.stdout:7/796: write d0/d6/d23/d45/d6a/f9d [300122,83260] 0 2024-05-15T15:54:28.577 INFO:tasks.workunit.client.0.smithi049.stdout:4/782: creat d18/d85/d98/f102 x:0 0 0 2024-05-15T15:54:28.578 INFO:tasks.workunit.client.1.smithi144.stdout:7/916: write d4/f51 [672555,118650] 0 2024-05-15T15:54:28.578 INFO:tasks.workunit.client.0.smithi049.stdout:3/798: dwrite d12/d14/d52/db9/dc4/d68/d70/f72 [0,4194304] 0 2024-05-15T15:54:28.581 INFO:tasks.workunit.client.1.smithi144.stdout:6/970: mkdir d7/db/d35/d84/d81/d15a 0 2024-05-15T15:54:28.585 INFO:tasks.workunit.client.0.smithi049.stdout:9/701: truncate d0/d8/d1b/f32 956372 0 2024-05-15T15:54:28.587 INFO:tasks.workunit.client.0.smithi049.stdout:5/958: write dc/db0/dc7/fc9 [335922,125099] 0 2024-05-15T15:54:28.590 INFO:tasks.workunit.client.0.smithi049.stdout:0/828: dwrite dc/d13/d2b/d76/fed [0,4194304] 0 2024-05-15T15:54:28.591 INFO:tasks.workunit.client.0.smithi049.stdout:0/829: read - dc/d6d/dc3/ffb zero size 2024-05-15T15:54:28.601 INFO:tasks.workunit.client.1.smithi144.stdout:5/925: creat d0/d7e/d9b/f133 x:0 0 0 2024-05-15T15:54:28.602 INFO:tasks.workunit.client.1.smithi144.stdout:4/965: truncate d6/dd/d2e/d75/dff/deb/f77 8103694 0 2024-05-15T15:54:28.606 INFO:tasks.workunit.client.0.smithi049.stdout:9/702: mknod d0/d8/d42/d75/cf6 0 2024-05-15T15:54:28.608 INFO:tasks.workunit.client.1.smithi144.stdout:7/917: readlink d4/d41/d9c/d106/l12b 0 2024-05-15T15:54:28.608 INFO:tasks.workunit.client.1.smithi144.stdout:2/850: dread d3/d7/d20/d37/f79 [0,4194304] 0 2024-05-15T15:54:28.612 INFO:tasks.workunit.client.0.smithi049.stdout:5/959: unlink dc/d4a/d42/f5e 0 2024-05-15T15:54:28.613 INFO:tasks.workunit.client.1.smithi144.stdout:1/921: dwrite d7/d1d/d33/d29/d6b/d5e/f9d [0,4194304] 0 2024-05-15T15:54:28.613 INFO:tasks.workunit.client.1.smithi144.stdout:0/865: write d4/d6/d4a/d74/fc3 [5001393,41597] 0 2024-05-15T15:54:28.615 INFO:tasks.workunit.client.1.smithi144.stdout:0/866: truncate d4/d6/f32 4497964 0 2024-05-15T15:54:28.617 INFO:tasks.workunit.client.1.smithi144.stdout:9/940: getdents d0/d8/d1b 0 2024-05-15T15:54:28.622 INFO:tasks.workunit.client.1.smithi144.stdout:4/966: mkdir d6/d49/d13a 0 2024-05-15T15:54:28.628 INFO:tasks.workunit.client.0.smithi049.stdout:9/703: rename d0/d25/d5d/c6c to d0/d8/d12/cf7 0 2024-05-15T15:54:28.631 INFO:tasks.workunit.client.0.smithi049.stdout:7/797: write d0/f9 [1817833,121506] 0 2024-05-15T15:54:28.633 INFO:tasks.workunit.client.1.smithi144.stdout:7/918: unlink d4/df/f1a 0 2024-05-15T15:54:28.634 INFO:tasks.workunit.client.0.smithi049.stdout:5/960: chown dc/c127 696707364 1 2024-05-15T15:54:28.637 INFO:tasks.workunit.client.1.smithi144.stdout:2/851: unlink d3/d7/d54/fd7 0 2024-05-15T15:54:28.637 INFO:tasks.workunit.client.1.smithi144.stdout:2/852: chown d3/d7/d20/d22/d52 0 1 2024-05-15T15:54:28.638 INFO:tasks.workunit.client.0.smithi049.stdout:0/830: creat dc/d13/d14/d25/d31/d71/dbf/f118 x:0 0 0 2024-05-15T15:54:28.640 INFO:tasks.workunit.client.0.smithi049.stdout:4/783: mknod d18/d1e/d2a/d50/dbc/c103 0 2024-05-15T15:54:28.640 INFO:tasks.workunit.client.1.smithi144.stdout:0/867: chown d4/ca 16407 1 2024-05-15T15:54:28.641 INFO:tasks.workunit.client.1.smithi144.stdout:0/868: stat d4/d6/d3a/d4c/dd6/d55/da5 0 2024-05-15T15:54:28.644 INFO:tasks.workunit.client.0.smithi049.stdout:3/799: rename d12/d14/d52/db9/d6b to d12/d30/dc1/dc7/df8 0 2024-05-15T15:54:28.646 INFO:tasks.workunit.client.1.smithi144.stdout:5/926: mkdir d0/d10c/d134 0 2024-05-15T15:54:28.646 INFO:tasks.workunit.client.0.smithi049.stdout:0/831: dwrite dc/d13/d14/d25/d31/d71/f67 [0,4194304] 0 2024-05-15T15:54:28.648 INFO:tasks.workunit.client.1.smithi144.stdout:3/899: sync 2024-05-15T15:54:28.648 INFO:tasks.workunit.client.0.smithi049.stdout:7/798: chown d0/d6/fcd 1 1 2024-05-15T15:54:28.648 INFO:tasks.workunit.client.1.smithi144.stdout:4/967: symlink d6/dd/d34/d3b/l13b 0 2024-05-15T15:54:28.650 INFO:tasks.workunit.client.0.smithi049.stdout:4/784: dwrite d18/d1e/d63/f96 [4194304,4194304] 0 2024-05-15T15:54:28.651 INFO:tasks.workunit.client.0.smithi049.stdout:5/961: rmdir dc/d4a/d9e/d10b 39 2024-05-15T15:54:28.658 INFO:tasks.workunit.client.1.smithi144.stdout:7/919: unlink d4/d3c/d76/cfd 0 2024-05-15T15:54:28.664 INFO:tasks.workunit.client.1.smithi144.stdout:7/920: dwrite d4/d3c/d76/d9d/dc5/f132 [0,4194304] 0 2024-05-15T15:54:28.665 INFO:tasks.workunit.client.0.smithi049.stdout:3/800: mknod d12/d26/d73/cf9 0 2024-05-15T15:54:28.671 INFO:tasks.workunit.client.1.smithi144.stdout:0/869: read d4/fc2 [147962,63542] 0 2024-05-15T15:54:28.673 INFO:tasks.workunit.client.0.smithi049.stdout:5/962: symlink dc/db0/dbf/l13a 0 2024-05-15T15:54:28.674 INFO:tasks.workunit.client.0.smithi049.stdout:5/963: dread - dc/d4a/f8b zero size 2024-05-15T15:54:28.675 INFO:tasks.workunit.client.0.smithi049.stdout:5/964: chown dc/d4a/d42/c9d 46610 1 2024-05-15T15:54:28.677 INFO:tasks.workunit.client.0.smithi049.stdout:9/704: rename d0/d8/d9/f7f to d0/d8/d9/d52/d5e/d5f/d56/ff8 0 2024-05-15T15:54:28.679 INFO:tasks.workunit.client.0.smithi049.stdout:3/801: rmdir d12/d30/d66 39 2024-05-15T15:54:28.682 INFO:tasks.workunit.client.0.smithi049.stdout:0/832: mkdir dc/da4/db4/df1/d119 0 2024-05-15T15:54:28.683 INFO:tasks.workunit.client.1.smithi144.stdout:2/853: write d3/d7/d20/d37/f70 [932719,79470] 0 2024-05-15T15:54:28.684 INFO:tasks.workunit.client.0.smithi049.stdout:7/799: mknod d0/d6/d23/d45/d6a/da1/cf8 0 2024-05-15T15:54:28.690 INFO:tasks.workunit.client.0.smithi049.stdout:5/965: mknod dc/dd/d10/db3/c13b 0 2024-05-15T15:54:28.692 INFO:tasks.workunit.client.1.smithi144.stdout:5/927: dwrite d0/f11 [4194304,4194304] 0 2024-05-15T15:54:28.693 INFO:tasks.workunit.client.1.smithi144.stdout:6/971: rename d7/db/d9d/fc6 to d7/d1e/d28/db5/df6/f15b 0 2024-05-15T15:54:28.696 INFO:tasks.workunit.client.0.smithi049.stdout:4/785: write d18/d21/f56 [55278,65894] 0 2024-05-15T15:54:28.697 INFO:tasks.workunit.client.1.smithi144.stdout:9/941: link d0/d8/d5e/d44/d77/l22 d0/d8/d1b/d62/d7d/d8d/d94/l141 0 2024-05-15T15:54:28.698 INFO:tasks.workunit.client.0.smithi049.stdout:0/833: write dc/d13/d14/f79 [3203419,82697] 0 2024-05-15T15:54:28.699 INFO:tasks.workunit.client.1.smithi144.stdout:9/942: dread - d0/d8/d1b/d62/d7d/d8d/dbb/dce/dfb/f105 zero size 2024-05-15T15:54:28.701 INFO:tasks.workunit.client.0.smithi049.stdout:0/834: truncate dc/d13/d14/d25/d31/d71/f116 320269 0 2024-05-15T15:54:28.701 INFO:tasks.workunit.client.0.smithi049.stdout:5/966: fsync dc/dd/d47/d83/dcc/da9/faa 0 2024-05-15T15:54:28.702 INFO:tasks.workunit.client.1.smithi144.stdout:4/968: creat d6/d49/d13a/f13c x:0 0 0 2024-05-15T15:54:28.704 INFO:tasks.workunit.client.0.smithi049.stdout:9/705: rename d0/d8/f61 to d0/d8/d42/d75/de7/ff9 0 2024-05-15T15:54:28.705 INFO:tasks.workunit.client.1.smithi144.stdout:2/854: mknod d3/d7/d20/d89/d6e/c119 0 2024-05-15T15:54:28.706 INFO:tasks.workunit.client.0.smithi049.stdout:3/802: mknod d12/cfa 0 2024-05-15T15:54:28.707 INFO:tasks.workunit.client.1.smithi144.stdout:7/921: mkdir d4/df/d25/d143 0 2024-05-15T15:54:28.708 INFO:tasks.workunit.client.0.smithi049.stdout:5/967: dwrite dc/d4a/f57 [0,4194304] 0 2024-05-15T15:54:28.708 INFO:tasks.workunit.client.0.smithi049.stdout:4/786: write d18/d1e/d2a/f6a [734203,77464] 0 2024-05-15T15:54:28.710 INFO:tasks.workunit.client.1.smithi144.stdout:5/928: rename d0/dc/d24/d28/cb7 to d0/dc/d1f/c135 0 2024-05-15T15:54:28.711 INFO:tasks.workunit.client.0.smithi049.stdout:7/800: mknod d0/d34/db3/cf9 0 2024-05-15T15:54:28.714 INFO:tasks.workunit.client.1.smithi144.stdout:1/922: getdents d7/d1d/d58 0 2024-05-15T15:54:28.717 INFO:tasks.workunit.client.0.smithi049.stdout:9/706: creat d0/d8/d1b/db3/ffa x:0 0 0 2024-05-15T15:54:28.720 INFO:tasks.workunit.client.0.smithi049.stdout:4/787: mknod d18/d1e/d63/d99/c104 0 2024-05-15T15:54:28.721 INFO:tasks.workunit.client.1.smithi144.stdout:2/855: fdatasync d3/d7/df8/d65/f9a 0 2024-05-15T15:54:28.723 INFO:tasks.workunit.client.1.smithi144.stdout:2/856: read d3/d7/d20/d3b/fd2 [78261,94050] 0 2024-05-15T15:54:28.723 INFO:tasks.workunit.client.1.smithi144.stdout:7/922: rmdir d4/d3c/d76/d9d/dac/d48 39 2024-05-15T15:54:28.724 INFO:tasks.workunit.client.1.smithi144.stdout:7/923: chown d4/l9 34690882 1 2024-05-15T15:54:28.728 INFO:tasks.workunit.client.1.smithi144.stdout:9/943: rename d0/d8/d5e/d44/d10/d19/lda to d0/d8/d50/l142 0 2024-05-15T15:54:28.730 INFO:tasks.workunit.client.1.smithi144.stdout:5/929: creat d0/dc/d24/d28/f136 x:0 0 0 2024-05-15T15:54:28.736 INFO:tasks.workunit.client.0.smithi049.stdout:5/968: dread dc/dd/d47/d83/dcc/da9/f117 [0,4194304] 0 2024-05-15T15:54:28.737 INFO:tasks.workunit.client.0.smithi049.stdout:8/796: sync 2024-05-15T15:54:28.737 INFO:tasks.workunit.client.0.smithi049.stdout:6/886: sync 2024-05-15T15:54:28.737 INFO:tasks.workunit.client.0.smithi049.stdout:2/741: sync 2024-05-15T15:54:28.737 INFO:tasks.workunit.client.0.smithi049.stdout:1/965: sync 2024-05-15T15:54:28.739 INFO:tasks.workunit.client.1.smithi144.stdout:1/923: mknod d7/d1d/d33/d41/d22/dbd/dc8/c11e 0 2024-05-15T15:54:28.740 INFO:tasks.workunit.client.1.smithi144.stdout:1/924: write d7/df/fec [554344,23562] 0 2024-05-15T15:54:28.740 INFO:tasks.workunit.client.1.smithi144.stdout:1/925: chown d7/d1d/d58/fa6 1 1 2024-05-15T15:54:28.740 INFO:tasks.workunit.client.1.smithi144.stdout:3/900: getdents d0/d1/d56/d71/d73/dd1 0 2024-05-15T15:54:28.744 INFO:tasks.workunit.client.0.smithi049.stdout:7/801: write d0/d6/d1d/f4e [4269774,41583] 0 2024-05-15T15:54:28.746 INFO:tasks.workunit.client.0.smithi049.stdout:7/802: fsync d0/f9 0 2024-05-15T15:54:28.748 INFO:tasks.workunit.client.0.smithi049.stdout:5/969: mknod dc/dd/d10/df1/c13c 0 2024-05-15T15:54:28.749 INFO:tasks.workunit.client.0.smithi049.stdout:0/835: dwrite dc/d13/d2b/f81 [4194304,4194304] 0 2024-05-15T15:54:28.751 INFO:tasks.workunit.client.1.smithi144.stdout:7/924: creat d4/d3c/d76/d9d/dc5/f144 x:0 0 0 2024-05-15T15:54:28.752 INFO:tasks.workunit.client.1.smithi144.stdout:6/972: dwrite d7/db/d35/fb1 [0,4194304] 0 2024-05-15T15:54:28.754 INFO:tasks.workunit.client.1.smithi144.stdout:6/973: dread - d7/d1e/d28/db5/d141/f156 zero size 2024-05-15T15:54:28.754 INFO:tasks.workunit.client.0.smithi049.stdout:4/788: fsync d18/d21/f30 0 2024-05-15T15:54:28.756 INFO:tasks.workunit.client.1.smithi144.stdout:9/944: creat d0/d8/d50/f143 x:0 0 0 2024-05-15T15:54:28.756 INFO:tasks.workunit.client.0.smithi049.stdout:6/887: rename dd/d17/d24/d2b/d36 to dd/d26/d5b/d110 0 2024-05-15T15:54:28.756 INFO:tasks.workunit.client.0.smithi049.stdout:2/742: dread d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/dc9/dea/fac [0,4194304] 0 2024-05-15T15:54:28.758 INFO:tasks.workunit.client.0.smithi049.stdout:3/803: dwrite d12/d26/d73/f7f [0,4194304] 0 2024-05-15T15:54:28.759 INFO:tasks.workunit.client.1.smithi144.stdout:5/930: creat d0/dc/d2f/d79/f137 x:0 0 0 2024-05-15T15:54:28.761 INFO:tasks.workunit.client.0.smithi049.stdout:8/797: chown d2/de/d95/cef 41226 1 2024-05-15T15:54:28.763 INFO:tasks.workunit.client.0.smithi049.stdout:9/707: dwrite d0/f54 [0,4194304] 0 2024-05-15T15:54:28.767 INFO:tasks.workunit.client.1.smithi144.stdout:0/870: link d4/d2d/d100/f126 d4/df/da0/f13c 0 2024-05-15T15:54:28.768 INFO:tasks.workunit.client.1.smithi144.stdout:2/857: dread d3/d7/d54/d98/fa2 [0,4194304] 0 2024-05-15T15:54:28.771 INFO:tasks.workunit.client.1.smithi144.stdout:0/871: dread d4/df/d1f/f58 [0,4194304] 0 2024-05-15T15:54:28.771 INFO:tasks.workunit.client.1.smithi144.stdout:1/926: rmdir d7/d1d/d33/d29/dcd 39 2024-05-15T15:54:28.776 INFO:tasks.workunit.client.1.smithi144.stdout:1/927: dread d7/d1d/d33/d29/d6b/da4/fb5 [0,4194304] 0 2024-05-15T15:54:28.776 INFO:tasks.workunit.client.1.smithi144.stdout:3/901: rename d0/d1/d17/l111 to d0/d1/d56/d71/d73/dd1/l121 0 2024-05-15T15:54:28.776 INFO:tasks.workunit.client.0.smithi049.stdout:5/970: creat dc/d4a/d9b/f13d x:0 0 0 2024-05-15T15:54:28.777 INFO:tasks.workunit.client.1.smithi144.stdout:3/902: stat d0/d1/d17/dae 0 2024-05-15T15:54:28.778 INFO:tasks.workunit.client.1.smithi144.stdout:4/969: creat d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/f13d x:0 0 0 2024-05-15T15:54:28.779 INFO:tasks.workunit.client.1.smithi144.stdout:4/970: stat d6/dd/d34/d116/c11a 0 2024-05-15T15:54:28.780 INFO:tasks.workunit.client.1.smithi144.stdout:4/971: chown d6/dd/d2e/d75/dff/deb/f54 992437 1 2024-05-15T15:54:28.781 INFO:tasks.workunit.client.0.smithi049.stdout:4/789: unlink d18/d1e/d2a/d50/f5e 0 2024-05-15T15:54:28.786 INFO:tasks.workunit.client.0.smithi049.stdout:6/888: rmdir dd/d26/d5b/d110/d50/d54/d92 39 2024-05-15T15:54:28.788 INFO:tasks.workunit.client.1.smithi144.stdout:5/931: mknod d0/dc/d24/d28/d2d/d59/d74/c138 0 2024-05-15T15:54:28.789 INFO:tasks.workunit.client.1.smithi144.stdout:2/858: chown d3/d7/c8c 364 1 2024-05-15T15:54:28.790 INFO:tasks.workunit.client.0.smithi049.stdout:9/708: creat d0/d8/d42/d75/de7/ffb x:0 0 0 2024-05-15T15:54:28.791 INFO:tasks.workunit.client.1.smithi144.stdout:0/872: unlink d4/df/da0/f13c 0 2024-05-15T15:54:28.791 INFO:tasks.workunit.client.1.smithi144.stdout:0/873: chown d4/d6/l95 1 1 2024-05-15T15:54:28.794 INFO:tasks.workunit.client.1.smithi144.stdout:9/945: rename d0/d8/d5e/d112 to d0/d8/d5e/dd4/d144 0 2024-05-15T15:54:28.794 INFO:tasks.workunit.client.0.smithi049.stdout:0/836: write dc/d13/d3d/fb2 [1281736,109820] 0 2024-05-15T15:54:28.799 INFO:tasks.workunit.client.1.smithi144.stdout:4/972: unlink d6/dd/l98 0 2024-05-15T15:54:28.803 INFO:tasks.workunit.client.1.smithi144.stdout:6/974: dread d7/d32/f136 [0,4194304] 0 2024-05-15T15:54:28.804 INFO:tasks.workunit.client.0.smithi049.stdout:1/966: rename d0/d4/dd/l118 to d0/d4/dd/de/d4d/l156 0 2024-05-15T15:54:28.810 INFO:tasks.workunit.client.1.smithi144.stdout:0/874: symlink d4/d6/d3a/d4c/dd6/d55/l13d 0 2024-05-15T15:54:28.814 INFO:tasks.workunit.client.0.smithi049.stdout:4/790: dwrite d18/f3e [0,4194304] 0 2024-05-15T15:54:28.815 INFO:tasks.workunit.client.1.smithi144.stdout:1/928: rename d7/d1d/d33/d41/d78/le6 to d7/df/d89/dcc/l11f 0 2024-05-15T15:54:28.816 INFO:tasks.workunit.client.1.smithi144.stdout:1/929: chown d7/d1d/d33/d41/d22/f6a 44 1 2024-05-15T15:54:28.818 INFO:tasks.workunit.client.1.smithi144.stdout:9/946: dread - d0/d8/d1b/d11c/df8/f110 zero size 2024-05-15T15:54:28.820 INFO:tasks.workunit.client.0.smithi049.stdout:7/803: creat d0/d6/d23/ffa x:0 0 0 2024-05-15T15:54:28.821 INFO:tasks.workunit.client.1.smithi144.stdout:4/973: unlink d6/dd/fbb 0 2024-05-15T15:54:28.822 INFO:tasks.workunit.client.1.smithi144.stdout:7/925: write d4/df/d25/d38/d68/f8d [604379,109206] 0 2024-05-15T15:54:28.823 INFO:tasks.workunit.client.0.smithi049.stdout:9/709: symlink d0/d8/d9/d52/d5e/d8a/dd3/d87/lfc 0 2024-05-15T15:54:28.824 INFO:tasks.workunit.client.1.smithi144.stdout:2/859: write d3/d4d/fca [4593419,90475] 0 2024-05-15T15:54:28.826 INFO:tasks.workunit.client.0.smithi049.stdout:0/837: symlink dc/d13/d3d/l11a 0 2024-05-15T15:54:28.827 INFO:tasks.workunit.client.0.smithi049.stdout:0/838: write dc/d13/d14/f1c [3190344,19649] 0 2024-05-15T15:54:28.828 INFO:tasks.workunit.client.0.smithi049.stdout:5/971: mkdir dc/dd/d47/d83/dcc/d13e 0 2024-05-15T15:54:28.830 INFO:tasks.workunit.client.1.smithi144.stdout:6/975: read d7/f78 [2671888,114873] 0 2024-05-15T15:54:28.831 INFO:tasks.workunit.client.0.smithi049.stdout:5/972: chown dc/dd/d10/df1/f116 69485701 1 2024-05-15T15:54:28.832 INFO:tasks.workunit.client.1.smithi144.stdout:6/976: stat d7/db/d11/d7e/ff9 0 2024-05-15T15:54:28.832 INFO:tasks.workunit.client.1.smithi144.stdout:0/875: creat d4/d6/d4a/d4d/db1/f13e x:0 0 0 2024-05-15T15:54:28.834 INFO:tasks.workunit.client.0.smithi049.stdout:3/804: creat d12/d14/d52/db9/dc4/d68/ffb x:0 0 0 2024-05-15T15:54:28.835 INFO:tasks.workunit.client.0.smithi049.stdout:6/889: dwrite dd/d26/d5b/d110/d50/d7e/f81 [0,4194304] 0 2024-05-15T15:54:28.836 INFO:tasks.workunit.client.1.smithi144.stdout:3/903: rename d0/d11/d3b/d93/f86 to d0/d1/d56/d71/d73/dd1/f122 0 2024-05-15T15:54:28.837 INFO:tasks.workunit.client.1.smithi144.stdout:1/930: fsync d7/d1d/f91 0 2024-05-15T15:54:28.838 INFO:tasks.workunit.client.0.smithi049.stdout:3/805: stat d12/d30/dc1/dc7/df8/dcf 0 2024-05-15T15:54:28.839 INFO:tasks.workunit.client.1.smithi144.stdout:9/947: fdatasync d0/d8/d1b/d114/f82 0 2024-05-15T15:54:28.840 INFO:tasks.workunit.client.0.smithi049.stdout:8/798: dwrite d2/de/d71/d86/d5d/d7d/fb3 [0,4194304] 0 2024-05-15T15:54:28.843 INFO:tasks.workunit.client.0.smithi049.stdout:9/710: mkdir d0/d8/d12/dfd 0 2024-05-15T15:54:28.846 INFO:tasks.workunit.client.1.smithi144.stdout:6/977: creat d7/db/d11/d7e/d132/f15c x:0 0 0 2024-05-15T15:54:28.846 INFO:tasks.workunit.client.0.smithi049.stdout:9/711: dread - d0/d8/d9/d52/d5e/d5f/fa0 zero size 2024-05-15T15:54:28.848 INFO:tasks.workunit.client.0.smithi049.stdout:0/839: mkdir dc/d13/d2b/d76/d8a/da2/d11b 0 2024-05-15T15:54:28.853 INFO:tasks.workunit.client.1.smithi144.stdout:5/932: rename d0/dc/d2f/d34/d3e/fe9 to d0/dc/d1f/d50/db3/d106/f139 0 2024-05-15T15:54:28.860 INFO:tasks.workunit.client.0.smithi049.stdout:3/806: dread d12/d30/f3d [0,4194304] 0 2024-05-15T15:54:28.862 INFO:tasks.workunit.client.1.smithi144.stdout:1/931: fdatasync d7/d1d/d33/d29/dcd/f109 0 2024-05-15T15:54:28.862 INFO:tasks.workunit.client.1.smithi144.stdout:1/932: chown d7/d1d/d58/d81 9466157 1 2024-05-15T15:54:28.863 INFO:tasks.workunit.client.0.smithi049.stdout:7/804: dread d0/ddb/dde/dcc/fd1 [0,4194304] 0 2024-05-15T15:54:28.863 INFO:tasks.workunit.client.0.smithi049.stdout:4/791: mknod d18/c105 0 2024-05-15T15:54:28.865 INFO:tasks.workunit.client.1.smithi144.stdout:9/948: rmdir d0/d8/d1b/d79/de7 39 2024-05-15T15:54:28.871 INFO:tasks.workunit.client.1.smithi144.stdout:2/860: mkdir d3/d7/d11a 0 2024-05-15T15:54:28.872 INFO:tasks.workunit.client.0.smithi049.stdout:8/799: rename d2/de/d25/d74/dbe/lcb to d2/de/d25/d74/d94/lf5 0 2024-05-15T15:54:28.875 INFO:tasks.workunit.client.1.smithi144.stdout:0/876: mknod d4/d6/d3a/c13f 0 2024-05-15T15:54:28.879 INFO:tasks.workunit.client.1.smithi144.stdout:1/933: symlink d7/d1d/d58/d81/l120 0 2024-05-15T15:54:28.879 INFO:tasks.workunit.client.1.smithi144.stdout:7/926: dwrite d4/df/d25/d38/f54 [0,4194304] 0 2024-05-15T15:54:28.881 INFO:tasks.workunit.client.1.smithi144.stdout:1/934: write d7/d1d/d33/d41/d22/dbd/fc0 [2271804,60822] 0 2024-05-15T15:54:28.882 INFO:tasks.workunit.client.0.smithi049.stdout:3/807: truncate d12/d14/d52/db9/dc4/d68/fa0 916002 0 2024-05-15T15:54:28.882 INFO:tasks.workunit.client.1.smithi144.stdout:1/935: stat d7/d1d/d33/d29/d6b/d5e/d74/df3 0 2024-05-15T15:54:28.884 INFO:tasks.workunit.client.1.smithi144.stdout:9/949: chown d0/d8/d5e/d44/d16/d5a/c68 90463050 1 2024-05-15T15:54:28.889 INFO:tasks.workunit.client.1.smithi144.stdout:3/904: link d0/d11/d3b/da7/def/f10e d0/d11/d3b/da7/f123 0 2024-05-15T15:54:28.890 INFO:tasks.workunit.client.0.smithi049.stdout:8/800: read d2/f7b [1398453,4104] 0 2024-05-15T15:54:28.891 INFO:tasks.workunit.client.1.smithi144.stdout:3/905: read d0/d11/d3b/da7/def/f10e [198271,47578] 0 2024-05-15T15:54:28.892 INFO:tasks.workunit.client.0.smithi049.stdout:9/712: creat d0/d8/d12/dfd/ffe x:0 0 0 2024-05-15T15:54:28.894 INFO:tasks.workunit.client.0.smithi049.stdout:0/840: unlink dc/d6d/lf0 0 2024-05-15T15:54:28.896 INFO:tasks.workunit.client.1.smithi144.stdout:7/927: rename d4/d41/d5b/da0/lc1 to d4/df/d25/d38/d68/l145 0 2024-05-15T15:54:28.897 INFO:tasks.workunit.client.1.smithi144.stdout:7/928: write d4/d41/fc4 [164193,44106] 0 2024-05-15T15:54:28.897 INFO:tasks.workunit.client.0.smithi049.stdout:1/967: getdents d0/d4/dd/de/d1e/d2e/dad/d65 0 2024-05-15T15:54:28.898 INFO:tasks.workunit.client.1.smithi144.stdout:1/936: chown d7/d1d/d33/d41/l1c 373645191 1 2024-05-15T15:54:28.900 INFO:tasks.workunit.client.0.smithi049.stdout:5/973: getdents dc/d4a/d42/dff/d111 0 2024-05-15T15:54:28.906 INFO:tasks.workunit.client.1.smithi144.stdout:6/978: link d7/db/d3a/dd2/c147 d7/da6/c15d 0 2024-05-15T15:54:28.908 INFO:tasks.workunit.client.1.smithi144.stdout:0/877: symlink d4/d6/d3a/l140 0 2024-05-15T15:54:28.911 INFO:tasks.workunit.client.0.smithi049.stdout:7/805: rename d0/f47 to d0/ddb/dde/dca/d5e/dbd/ffb 0 2024-05-15T15:54:28.913 INFO:tasks.workunit.client.1.smithi144.stdout:3/906: mkdir d0/d1/d17/dae/d124 0 2024-05-15T15:54:28.916 INFO:tasks.workunit.client.0.smithi049.stdout:9/713: creat d0/d8/d21/fff x:0 0 0 2024-05-15T15:54:28.916 INFO:tasks.workunit.client.1.smithi144.stdout:7/929: symlink d4/df/d25/d40/d73/d112/l146 0 2024-05-15T15:54:28.918 INFO:tasks.workunit.client.1.smithi144.stdout:1/937: truncate d7/d1d/d33/d29/f64 1114405 0 2024-05-15T15:54:28.920 INFO:tasks.workunit.client.0.smithi049.stdout:1/968: mknod d0/d4/d63/c157 0 2024-05-15T15:54:28.922 INFO:tasks.workunit.client.0.smithi049.stdout:1/969: chown d0/d4/dd/de/d1e/d2e/c3c 12874150 1 2024-05-15T15:54:28.923 INFO:tasks.workunit.client.1.smithi144.stdout:7/930: dwrite d4/d41/d5b/da0/f13a [0,4194304] 0 2024-05-15T15:54:28.925 INFO:tasks.workunit.client.1.smithi144.stdout:1/938: rmdir d7/d1d/d33/d29/dcd/dff 39 2024-05-15T15:54:28.928 INFO:tasks.workunit.client.0.smithi049.stdout:4/792: creat d18/d1e/d63/f106 x:0 0 0 2024-05-15T15:54:28.929 INFO:tasks.workunit.client.0.smithi049.stdout:6/890: link dd/d26/d5b/d110/d50/d54/dde/d8c/le9 dd/d26/d5b/d110/d50/d54/d92/de5/df8/l111 0 2024-05-15T15:54:28.933 INFO:tasks.workunit.client.1.smithi144.stdout:7/931: fdatasync d4/f13 0 2024-05-15T15:54:28.936 INFO:tasks.workunit.client.0.smithi049.stdout:8/801: mknod d2/de/d71/d86/d5d/d7d/d45/cf6 0 2024-05-15T15:54:28.938 INFO:tasks.workunit.client.0.smithi049.stdout:0/841: unlink dc/d13/d2b/d76/f9a 0 2024-05-15T15:54:28.939 INFO:tasks.workunit.client.1.smithi144.stdout:0/878: link d4/df/f16 d4/df/d102/d11b/dd1/f141 0 2024-05-15T15:54:28.940 INFO:tasks.workunit.client.0.smithi049.stdout:9/714: truncate d0/d8/d1b/f28 1455933 0 2024-05-15T15:54:28.940 INFO:tasks.workunit.client.1.smithi144.stdout:0/879: readlink d4/d6/d3a/d4c/dd6/d55/lbe 0 2024-05-15T15:54:28.942 INFO:tasks.workunit.client.0.smithi049.stdout:5/974: mknod dc/d4a/d42/c13f 0 2024-05-15T15:54:28.943 INFO:tasks.workunit.client.1.smithi144.stdout:0/880: dread d4/df/d14/f5f [0,4194304] 0 2024-05-15T15:54:28.944 INFO:tasks.workunit.client.1.smithi144.stdout:7/932: unlink d4/d41/d5b/da0/f11e 0 2024-05-15T15:54:28.945 INFO:tasks.workunit.client.1.smithi144.stdout:7/933: write d4/df/d25/d38/d5d/f8c [1431894,18601] 0 2024-05-15T15:54:28.945 INFO:tasks.workunit.client.0.smithi049.stdout:1/970: write d0/d4/dd/de/d4d/d8b/dba/f123 [489246,94082] 0 2024-05-15T15:54:28.946 INFO:tasks.workunit.client.1.smithi144.stdout:1/939: mknod d7/d1d/d33/d29/d6b/d5e/d74/df3/c121 0 2024-05-15T15:54:28.947 INFO:tasks.workunit.client.0.smithi049.stdout:1/971: write d0/d4/dd/de/d1e/d2e/dad/d65/f149 [845672,23727] 0 2024-05-15T15:54:28.949 INFO:tasks.workunit.client.0.smithi049.stdout:3/808: rmdir d12/d14/d52/db9/df2 0 2024-05-15T15:54:28.950 INFO:tasks.workunit.client.0.smithi049.stdout:3/809: dread d12/f4d [0,4194304] 0 2024-05-15T15:54:28.955 INFO:tasks.workunit.client.0.smithi049.stdout:6/891: creat dd/d17/d9d/f112 x:0 0 0 2024-05-15T15:54:28.955 INFO:tasks.workunit.client.0.smithi049.stdout:6/892: readlink dd/d17/d24/d58/d5f/l65 0 2024-05-15T15:54:28.957 INFO:tasks.workunit.client.1.smithi144.stdout:9/950: write d0/d8/d5e/d44/d10/d19/f63 [5033158,24417] 0 2024-05-15T15:54:28.959 INFO:tasks.workunit.client.0.smithi049.stdout:7/806: rename d0/d6/l25 to d0/ddb/dde/dca/lfc 0 2024-05-15T15:54:28.959 INFO:tasks.workunit.client.1.smithi144.stdout:3/907: write d0/d1/d17/f8b [404934,28733] 0 2024-05-15T15:54:28.962 INFO:tasks.workunit.client.0.smithi049.stdout:6/893: dwrite dd/dbc/f10b [0,4194304] 0 2024-05-15T15:54:28.963 INFO:tasks.workunit.client.1.smithi144.stdout:7/934: rename d4/f37 to d4/d3c/d76/d9d/dac/d111/d12c/f147 0 2024-05-15T15:54:28.964 INFO:tasks.workunit.client.1.smithi144.stdout:4/974: sync 2024-05-15T15:54:28.965 INFO:tasks.workunit.client.1.smithi144.stdout:4/975: write d6/dd/d34/d3b/d30/db/d1d/d40/f97 [3577384,3577] 0 2024-05-15T15:54:28.969 INFO:tasks.workunit.client.1.smithi144.stdout:1/940: fsync d7/d1d/d33/d41/fe 0 2024-05-15T15:54:28.970 INFO:tasks.workunit.client.0.smithi049.stdout:0/842: dread - dc/d6d/dc3/fe6 zero size 2024-05-15T15:54:28.971 INFO:tasks.workunit.client.1.smithi144.stdout:1/941: write d7/d1d/d33/d55/f5b [4422842,6395] 0 2024-05-15T15:54:28.972 INFO:tasks.workunit.client.1.smithi144.stdout:1/942: write d7/d1d/d33/d29/dd2/f7b [575719,36873] 0 2024-05-15T15:54:28.973 INFO:tasks.workunit.client.0.smithi049.stdout:9/715: creat d0/d8/d9/d52/d5e/d8a/dd3/f100 x:0 0 0 2024-05-15T15:54:28.974 INFO:tasks.workunit.client.0.smithi049.stdout:9/716: chown d0/d8/d9/fb 4 1 2024-05-15T15:54:28.974 INFO:tasks.workunit.client.1.smithi144.stdout:1/943: truncate d7/d1d/d33/d29/d6b/d5e/fa1 4350131 0 2024-05-15T15:54:28.977 INFO:tasks.workunit.client.0.smithi049.stdout:5/975: fsync dc/dd/d7b/d8d/ff4 0 2024-05-15T15:54:28.978 INFO:tasks.workunit.client.0.smithi049.stdout:5/976: write dc/dd/d47/d83/dcc/da9/faa [2564929,71457] 0 2024-05-15T15:54:28.979 INFO:tasks.workunit.client.1.smithi144.stdout:1/944: dwrite d7/d1d/d33/d55/f108 [0,4194304] 0 2024-05-15T15:54:28.979 INFO:tasks.workunit.client.0.smithi049.stdout:1/972: fsync d0/d4/dd/de/d1e/d2e/d3e/d90/db3/fc4 0 2024-05-15T15:54:28.982 INFO:tasks.workunit.client.0.smithi049.stdout:1/973: write d0/d4/dd/de/d1e/d2e/dad/d65/f153 [787070,63025] 0 2024-05-15T15:54:28.984 INFO:tasks.workunit.client.0.smithi049.stdout:1/974: chown d0/d4/dd/de/d1e/d16/d8a/dd2/d110/d139 55407586 1 2024-05-15T15:54:28.985 INFO:tasks.workunit.client.0.smithi049.stdout:1/975: read - d0/d4/dd/de/d1e/f13c zero size 2024-05-15T15:54:28.987 INFO:tasks.workunit.client.1.smithi144.stdout:3/908: dread d0/d1/d17/fd8 [0,4194304] 0 2024-05-15T15:54:28.989 INFO:tasks.workunit.client.1.smithi144.stdout:0/881: write d4/d6/d3a/d4c/f7a [611868,78654] 0 2024-05-15T15:54:28.999 INFO:tasks.workunit.client.0.smithi049.stdout:4/793: write d18/d21/f7a [6093973,84180] 0 2024-05-15T15:54:29.000 INFO:tasks.workunit.client.0.smithi049.stdout:4/794: write f16 [2989685,21177] 0 2024-05-15T15:54:29.001 INFO:tasks.workunit.client.0.smithi049.stdout:6/894: mknod dd/dbc/c113 0 2024-05-15T15:54:29.004 INFO:tasks.workunit.client.1.smithi144.stdout:9/951: creat d0/d8/d1b/d62/d7d/d8d/df3/f145 x:0 0 0 2024-05-15T15:54:29.005 INFO:tasks.workunit.client.1.smithi144.stdout:9/952: truncate d0/d8/d50/f143 607951 0 2024-05-15T15:54:29.006 INFO:tasks.workunit.client.0.smithi049.stdout:8/802: truncate d2/de/d25/f73 846597 0 2024-05-15T15:54:29.009 INFO:tasks.workunit.client.1.smithi144.stdout:3/909: fdatasync d0/d1/d56/fe8 0 2024-05-15T15:54:29.011 INFO:tasks.workunit.client.1.smithi144.stdout:0/882: truncate d4/d6/d4a/d74/f79 1773563 0 2024-05-15T15:54:29.013 INFO:tasks.workunit.client.1.smithi144.stdout:1/945: creat d7/d1d/d33/d29/dd2/dea/d113/f122 x:0 0 0 2024-05-15T15:54:29.022 INFO:tasks.workunit.client.1.smithi144.stdout:7/935: rename d4/df/d25/d40/d73/f134 to d4/df/d25/f148 0 2024-05-15T15:54:29.023 INFO:tasks.workunit.client.1.smithi144.stdout:0/883: mkdir d4/df/d102/d11b/d142 0 2024-05-15T15:54:29.027 INFO:tasks.workunit.client.1.smithi144.stdout:9/953: mkdir d0/d8/d1b/d79/d8a/dae/ddb/de4/d146 0 2024-05-15T15:54:29.031 INFO:tasks.workunit.client.1.smithi144.stdout:4/976: rename d6/dd/fa2 to d6/dd/d2e/f13e 0 2024-05-15T15:54:29.031 INFO:tasks.workunit.client.1.smithi144.stdout:3/910: write d0/d11/d24/da5/fbe [281599,2645] 0 2024-05-15T15:54:29.033 INFO:tasks.workunit.client.1.smithi144.stdout:4/977: write d6/d42/f58 [4056496,94915] 0 2024-05-15T15:54:29.034 INFO:tasks.workunit.client.1.smithi144.stdout:9/954: mknod d0/d8/d1b/d62/d7d/d8d/c147 0 2024-05-15T15:54:29.037 INFO:tasks.workunit.client.1.smithi144.stdout:1/946: rename d7/d1d/d33/d29/d6b/d5e/d74/df2/l111 to d7/d1d/d33/d55/l123 0 2024-05-15T15:54:29.038 INFO:tasks.workunit.client.1.smithi144.stdout:1/947: fsync d7/d1d/d33/de7/f118 0 2024-05-15T15:54:29.040 INFO:tasks.workunit.client.1.smithi144.stdout:3/911: stat d0/d1/c8 0 2024-05-15T15:54:29.043 INFO:tasks.workunit.client.1.smithi144.stdout:4/978: fsync d6/dd/fd2 0 2024-05-15T15:54:29.046 INFO:tasks.workunit.client.1.smithi144.stdout:9/955: write d0/d8/d1b/d79/de7/df9/f132 [867416,108663] 0 2024-05-15T15:54:29.048 INFO:tasks.workunit.client.0.smithi049.stdout:2/743: sync 2024-05-15T15:54:29.049 INFO:tasks.workunit.client.1.smithi144.stdout:7/936: dwrite d4/df/d25/d38/d68/fd6 [0,4194304] 0 2024-05-15T15:54:29.050 INFO:tasks.workunit.client.1.smithi144.stdout:1/948: dread d7/d1d/d33/d1e/f77 [0,4194304] 0 2024-05-15T15:54:29.055 INFO:tasks.workunit.client.1.smithi144.stdout:4/979: mkdir d6/dd/d34/d116/d13f 0 2024-05-15T15:54:29.062 INFO:tasks.workunit.client.1.smithi144.stdout:7/937: fdatasync d4/df/f16 0 2024-05-15T15:54:29.066 INFO:tasks.workunit.client.1.smithi144.stdout:4/980: fsync d6/dd/d34/d3b/d30/db/d1d/d40/d57/f5e 0 2024-05-15T15:54:29.073 INFO:tasks.workunit.client.1.smithi144.stdout:7/938: read d4/d3c/d76/d9d/dac/d48/f126 [26265,59240] 0 2024-05-15T15:54:29.078 INFO:tasks.workunit.client.1.smithi144.stdout:3/912: rename d0/d1/d6c to d0/d1/d56/d71/d125 0 2024-05-15T15:54:29.079 INFO:tasks.workunit.client.1.smithi144.stdout:4/981: dread d6/d42/f112 [0,4194304] 0 2024-05-15T15:54:29.086 INFO:tasks.workunit.client.1.smithi144.stdout:9/956: creat d0/d8/d1b/d62/d7d/f148 x:0 0 0 2024-05-15T15:54:29.088 INFO:tasks.workunit.client.1.smithi144.stdout:5/933: sync 2024-05-15T15:54:29.088 INFO:tasks.workunit.client.0.smithi049.stdout:0/843: creat dc/d13/d14/d2f/d8f/dcb/de2/d103/f11c x:0 0 0 2024-05-15T15:54:29.090 INFO:tasks.workunit.client.0.smithi049.stdout:9/717: rename d0/d8/d9/d52/d5e/d8a/dac/fcb to d0/d8/dd6/de2/dec/f101 0 2024-05-15T15:54:29.091 INFO:tasks.workunit.client.1.smithi144.stdout:7/939: readlink d4/df/d25/d38/d68/lb6 0 2024-05-15T15:54:29.093 INFO:tasks.workunit.client.1.smithi144.stdout:3/913: mkdir d0/d11/d3b/d93/d126 0 2024-05-15T15:54:29.093 INFO:tasks.workunit.client.1.smithi144.stdout:3/914: chown d0/d1/d56/d11b 2076291 1 2024-05-15T15:54:29.094 INFO:tasks.workunit.client.0.smithi049.stdout:5/977: readlink dc/d4a/l2d 0 2024-05-15T15:54:29.094 INFO:tasks.workunit.client.1.smithi144.stdout:3/915: readlink d0/d11/l1f 0 2024-05-15T15:54:29.094 INFO:tasks.workunit.client.1.smithi144.stdout:4/982: fdatasync d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/dda/fd7 0 2024-05-15T15:54:29.098 INFO:tasks.workunit.client.0.smithi049.stdout:1/976: unlink d0/d4/d81/dab/lf4 0 2024-05-15T15:54:29.100 INFO:tasks.workunit.client.1.smithi144.stdout:9/957: mknod d0/d8/d5e/d44/d16/db0/c149 0 2024-05-15T15:54:29.101 INFO:tasks.workunit.client.1.smithi144.stdout:9/958: write d0/d8/d1b/d79/d8a/f97 [3950786,83045] 0 2024-05-15T15:54:29.102 INFO:tasks.workunit.client.0.smithi049.stdout:3/810: fsync d12/d14/d52/db9/dc4/d68/d70/d75/f85 0 2024-05-15T15:54:29.103 INFO:tasks.workunit.client.0.smithi049.stdout:3/811: fdatasync d12/d67/da8/ddd/feb 0 2024-05-15T15:54:29.104 INFO:tasks.workunit.client.0.smithi049.stdout:3/812: chown d12/d14/d52/db9/l57 1311205 1 2024-05-15T15:54:29.105 INFO:tasks.workunit.client.1.smithi144.stdout:3/916: dread d0/d1/d56/f58 [0,4194304] 0 2024-05-15T15:54:29.107 INFO:tasks.workunit.client.1.smithi144.stdout:1/949: getdents d7/d1d 0 2024-05-15T15:54:29.108 INFO:tasks.workunit.client.1.smithi144.stdout:7/940: write d4/d3c/d76/d9d/dac/d111/d12c/f147 [407891,45543] 0 2024-05-15T15:54:29.111 INFO:tasks.workunit.client.0.smithi049.stdout:6/895: dread - dd/d26/d5b/d110/d50/f9a zero size 2024-05-15T15:54:29.114 INFO:tasks.workunit.client.1.smithi144.stdout:2/861: sync 2024-05-15T15:54:29.114 INFO:tasks.workunit.client.1.smithi144.stdout:6/979: sync 2024-05-15T15:54:29.114 INFO:tasks.workunit.client.1.smithi144.stdout:0/884: sync 2024-05-15T15:54:29.115 INFO:tasks.workunit.client.1.smithi144.stdout:4/983: read d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/f51 [67693,83476] 0 2024-05-15T15:54:29.115 INFO:tasks.workunit.client.0.smithi049.stdout:8/803: dread d2/de/d71/d86/d5d/d7d/d1b/d68/fab [0,4194304] 0 2024-05-15T15:54:29.116 INFO:tasks.workunit.client.1.smithi144.stdout:5/934: write d0/dc/d1f/d50/fbe [416122,104753] 0 2024-05-15T15:54:29.117 INFO:tasks.workunit.client.1.smithi144.stdout:5/935: chown d0/dc/d24/f37 467172172 1 2024-05-15T15:54:29.120 INFO:tasks.workunit.client.0.smithi049.stdout:0/844: creat dc/d13/d56/d6f/dce/f11d x:0 0 0 2024-05-15T15:54:29.121 INFO:tasks.workunit.client.0.smithi049.stdout:9/718: rmdir d0/d8/d9/d52/d5e/d5f/d56 39 2024-05-15T15:54:29.122 INFO:tasks.workunit.client.0.smithi049.stdout:0/845: chown dc/d13/d14/d25/d31/f58 1 1 2024-05-15T15:54:29.122 INFO:tasks.workunit.client.0.smithi049.stdout:2/744: write d3/db/d1d/d1e/d2b/d3b/f76 [583754,125695] 0 2024-05-15T15:54:29.124 INFO:tasks.workunit.client.1.smithi144.stdout:9/959: write d0/d8/d5e/d44/d10/f58 [1958374,99926] 0 2024-05-15T15:54:29.125 INFO:tasks.workunit.client.1.smithi144.stdout:1/950: symlink d7/d1d/d33/d1e/d4e/l124 0 2024-05-15T15:54:29.127 INFO:tasks.workunit.client.0.smithi049.stdout:5/978: truncate dc/dd/d47/d83/dcc/da9/dba/dbe/fc1 1139857 0 2024-05-15T15:54:29.128 INFO:tasks.workunit.client.1.smithi144.stdout:3/917: write d0/d11/d24/f41 [2894186,108024] 0 2024-05-15T15:54:29.128 INFO:tasks.workunit.client.0.smithi049.stdout:5/979: fdatasync dc/d29/f32 0 2024-05-15T15:54:29.129 INFO:tasks.workunit.client.0.smithi049.stdout:1/977: creat d0/d4/dd/de/d1e/d16/df1/d103/f158 x:0 0 0 2024-05-15T15:54:29.132 INFO:tasks.workunit.client.0.smithi049.stdout:7/807: truncate d0/f2e 620313 0 2024-05-15T15:54:29.133 INFO:tasks.workunit.client.1.smithi144.stdout:0/885: rmdir d4/d6 39 2024-05-15T15:54:29.134 INFO:tasks.workunit.client.0.smithi049.stdout:3/813: rmdir d12/d14/d52/db9/da6/dc2 39 2024-05-15T15:54:29.135 INFO:tasks.workunit.client.0.smithi049.stdout:3/814: stat d12/d26/d73/f7f 0 2024-05-15T15:54:29.136 INFO:tasks.workunit.client.0.smithi049.stdout:4/795: mkdir d18/d21/d107 0 2024-05-15T15:54:29.140 INFO:tasks.workunit.client.1.smithi144.stdout:4/984: creat d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d44/f140 x:0 0 0 2024-05-15T15:54:29.141 INFO:tasks.workunit.client.1.smithi144.stdout:4/985: fsync d6/dd/d34/d3b/d30/d59/d66/f9e 0 2024-05-15T15:54:29.142 INFO:tasks.workunit.client.1.smithi144.stdout:5/936: read d0/f16 [847866,35207] 0 2024-05-15T15:54:29.145 INFO:tasks.workunit.client.0.smithi049.stdout:8/804: unlink d2/de/d25/d74/dbe/dd8/lf2 0 2024-05-15T15:54:29.146 INFO:tasks.workunit.client.1.smithi144.stdout:5/937: dread d0/dc/d2f/d34/f12a [0,4194304] 0 2024-05-15T15:54:29.150 INFO:tasks.workunit.client.0.smithi049.stdout:0/846: creat dc/d13/d14/d2f/d8f/f11e x:0 0 0 2024-05-15T15:54:29.152 INFO:tasks.workunit.client.0.smithi049.stdout:0/847: read dc/d13/d14/d25/f69 [7695162,99747] 0 2024-05-15T15:54:29.153 INFO:tasks.workunit.client.1.smithi144.stdout:2/862: dread d3/f9 [4194304,4194304] 0 2024-05-15T15:54:29.154 INFO:tasks.workunit.client.0.smithi049.stdout:5/980: symlink dc/dd/d10/db3/l140 0 2024-05-15T15:54:29.155 INFO:tasks.workunit.client.1.smithi144.stdout:9/960: dread d0/d8/d1b/d62/d6b/fa7 [0,4194304] 0 2024-05-15T15:54:29.158 INFO:tasks.workunit.client.0.smithi049.stdout:1/978: rmdir d0/d4/dd/de/d36 39 2024-05-15T15:54:29.158 INFO:tasks.workunit.client.1.smithi144.stdout:6/980: dread d7/db/d35/f131 [0,4194304] 0 2024-05-15T15:54:29.159 INFO:tasks.workunit.client.0.smithi049.stdout:1/979: readlink d0/d4/dd/de/d1e/d2e/dad/d65/leb 0 2024-05-15T15:54:29.161 INFO:tasks.workunit.client.0.smithi049.stdout:1/980: write d0/d4/dd/de/d1e/d2e/dad/f140 [286110,16126] 0 2024-05-15T15:54:29.163 INFO:tasks.workunit.client.0.smithi049.stdout:7/808: mkdir d0/d6/d1d/dfd 0 2024-05-15T15:54:29.164 INFO:tasks.workunit.client.1.smithi144.stdout:7/941: truncate d4/df/d25/d38/d68/f8d 461775 0 2024-05-15T15:54:29.165 INFO:tasks.workunit.client.0.smithi049.stdout:7/809: truncate d0/d6/d23/d76/ff5 440658 0 2024-05-15T15:54:29.165 INFO:tasks.workunit.client.1.smithi144.stdout:5/938: mknod d0/d10c/d5d/c13a 0 2024-05-15T15:54:29.167 INFO:tasks.workunit.client.0.smithi049.stdout:3/815: mknod d12/d14/d52/db9/dc4/d68/d70/dda/cfc 0 2024-05-15T15:54:29.167 INFO:tasks.workunit.client.1.smithi144.stdout:7/942: dread d4/df/d25/d38/f54 [0,4194304] 0 2024-05-15T15:54:29.171 INFO:tasks.workunit.client.1.smithi144.stdout:2/863: creat d3/d7/d20/d37/d76/f11b x:0 0 0 2024-05-15T15:54:29.172 INFO:tasks.workunit.client.1.smithi144.stdout:3/918: dwrite d0/d11/d24/f88 [0,4194304] 0 2024-05-15T15:54:29.174 INFO:tasks.workunit.client.1.smithi144.stdout:0/886: fsync d4/d6/d3a/d4c/fa6 0 2024-05-15T15:54:29.176 INFO:tasks.workunit.client.1.smithi144.stdout:3/919: dread d0/d11/d3b/f96 [0,4194304] 0 2024-05-15T15:54:29.176 INFO:tasks.workunit.client.0.smithi049.stdout:9/719: unlink d0/d8/l81 0 2024-05-15T15:54:29.177 INFO:tasks.workunit.client.1.smithi144.stdout:6/981: truncate d7/f68 3588191 0 2024-05-15T15:54:29.183 INFO:tasks.workunit.client.0.smithi049.stdout:5/981: unlink dc/lac 0 2024-05-15T15:54:29.186 INFO:tasks.workunit.client.0.smithi049.stdout:1/981: rename d0 to d0/d4/dd/de/d159 22 2024-05-15T15:54:29.190 INFO:tasks.workunit.client.1.smithi144.stdout:9/961: dread d0/f4 [0,4194304] 0 2024-05-15T15:54:29.191 INFO:tasks.workunit.client.0.smithi049.stdout:7/810: creat d0/d34/ffe x:0 0 0 2024-05-15T15:54:29.191 INFO:tasks.workunit.client.1.smithi144.stdout:1/951: link d7/d1d/d33/d29/le3 d7/d1d/d58/l125 0 2024-05-15T15:54:29.191 INFO:tasks.workunit.client.1.smithi144.stdout:9/962: readlink d0/d8/d5e/d44/d16/db0/l13f 0 2024-05-15T15:54:29.192 INFO:tasks.workunit.client.0.smithi049.stdout:7/811: write d0/d6/d23/d4a/d7b/dc4/fe7 [207421,51212] 0 2024-05-15T15:54:29.195 INFO:tasks.workunit.client.1.smithi144.stdout:2/864: symlink d3/d7/d20/d22/d52/d64/d80/l11c 0 2024-05-15T15:54:29.196 INFO:tasks.workunit.client.1.smithi144.stdout:4/986: dread d6/dd/d2e/d75/fb5 [0,4194304] 0 2024-05-15T15:54:29.197 INFO:tasks.workunit.client.0.smithi049.stdout:6/896: creat dd/d26/d5b/d110/d50/d54/f114 x:0 0 0 2024-05-15T15:54:29.201 INFO:tasks.workunit.client.1.smithi144.stdout:0/887: mkdir d4/d2d/d48/d5b/dd3/d107/d143 0 2024-05-15T15:54:29.204 INFO:tasks.workunit.client.0.smithi049.stdout:8/805: link d2/de/d25/d74/dbe/fe4 d2/de/d25/d74/ff7 0 2024-05-15T15:54:29.204 INFO:tasks.workunit.client.1.smithi144.stdout:6/982: dread d7/db/da2/fb2 [4194304,4194304] 0 2024-05-15T15:54:29.208 INFO:tasks.workunit.client.0.smithi049.stdout:3/816: write d12/d67/fb0 [3271560,26239] 0 2024-05-15T15:54:29.209 INFO:tasks.workunit.client.0.smithi049.stdout:3/817: dread - d12/d14/d52/db9/dc4/d68/ffb zero size 2024-05-15T15:54:29.214 INFO:tasks.workunit.client.1.smithi144.stdout:5/939: mknod d0/dc/d2f/d68/c13b 0 2024-05-15T15:54:29.214 INFO:tasks.workunit.client.1.smithi144.stdout:7/943: mkdir d4/df/d79/d100/d149 0 2024-05-15T15:54:29.215 INFO:tasks.workunit.client.0.smithi049.stdout:5/982: creat dc/dd/d7b/d8d/f141 x:0 0 0 2024-05-15T15:54:29.217 INFO:tasks.workunit.client.1.smithi144.stdout:1/952: mknod d7/d1d/d58/c126 0 2024-05-15T15:54:29.219 INFO:tasks.workunit.client.1.smithi144.stdout:9/963: mkdir d0/d8/d50/d14a 0 2024-05-15T15:54:29.219 INFO:tasks.workunit.client.1.smithi144.stdout:9/964: write d0/f5 [4727533,71427] 0 2024-05-15T15:54:29.221 INFO:tasks.workunit.client.0.smithi049.stdout:9/720: write d0/d8/d9/d52/d5e/d8a/dd3/d7d/fa3 [361152,21635] 0 2024-05-15T15:54:29.227 INFO:tasks.workunit.client.1.smithi144.stdout:0/888: dread - d4/df/d1f/d24/d85/da1/fd0 zero size 2024-05-15T15:54:29.227 INFO:tasks.workunit.client.0.smithi049.stdout:8/806: creat d2/de/d71/d86/d5d/d63/ff8 x:0 0 0 2024-05-15T15:54:29.233 INFO:tasks.workunit.client.1.smithi144.stdout:3/920: rename d0/d1/c7b to d0/d1/d17/d39/d40/d97/c127 0 2024-05-15T15:54:29.234 INFO:tasks.workunit.client.1.smithi144.stdout:3/921: dread - d0/d1/d56/d98/d9c/fee zero size 2024-05-15T15:54:29.234 INFO:tasks.workunit.client.1.smithi144.stdout:3/922: stat d0/d11/d24/d78/ff2 0 2024-05-15T15:54:29.236 INFO:tasks.workunit.client.0.smithi049.stdout:9/721: dwrite d0/d8/d9/d52/d5e/d8a/dd3/d7d/fea [0,4194304] 0 2024-05-15T15:54:29.236 INFO:tasks.workunit.client.0.smithi049.stdout:1/982: dread d0/d4/dd/de/d1e/f37 [0,4194304] 0 2024-05-15T15:54:29.241 INFO:tasks.workunit.client.0.smithi049.stdout:6/897: dwrite dd/f4c [0,4194304] 0 2024-05-15T15:54:29.266 INFO:tasks.workunit.client.0.smithi049.stdout:5/983: dread dc/dd/d10/f2a [0,4194304] 0 2024-05-15T15:54:29.291 INFO:tasks.workunit.client.0.smithi049.stdout:7/812: fdatasync d0/d6/f3c 0 2024-05-15T15:54:29.296 INFO:tasks.workunit.client.0.smithi049.stdout:3/818: mknod d12/d14/d32/cfd 0 2024-05-15T15:54:29.298 INFO:tasks.workunit.client.0.smithi049.stdout:0/848: rename dc/d13/c18 to dc/d13/c11f 0 2024-05-15T15:54:29.300 INFO:tasks.workunit.client.0.smithi049.stdout:9/722: creat d0/d8/d42/d75/f102 x:0 0 0 2024-05-15T15:54:29.301 INFO:tasks.workunit.client.0.smithi049.stdout:1/983: dread - d0/d4/dd/de/d48/d5c/f68 zero size 2024-05-15T15:54:29.303 INFO:tasks.workunit.client.0.smithi049.stdout:9/723: readlink d0/d8/d53/lce 0 2024-05-15T15:54:29.305 INFO:tasks.workunit.client.0.smithi049.stdout:5/984: mkdir dc/dd/d47/d83/da7/d142 0 2024-05-15T15:54:29.308 INFO:tasks.workunit.client.0.smithi049.stdout:5/985: chown dc/dd/d39/f6e 366359994 1 2024-05-15T15:54:29.309 INFO:tasks.workunit.client.0.smithi049.stdout:3/819: dwrite d12/d14/d52/db9/dc4/d68/d70/d75/dac/fe8 [0,4194304] 0 2024-05-15T15:54:29.309 INFO:tasks.workunit.client.0.smithi049.stdout:0/849: dwrite dc/d13/d2b/f81 [0,4194304] 0 2024-05-15T15:54:29.312 INFO:tasks.workunit.client.0.smithi049.stdout:7/813: mkdir d0/d6/d23/d4a/d7b/de2/dff 0 2024-05-15T15:54:29.312 INFO:tasks.workunit.client.0.smithi049.stdout:3/820: truncate d12/d26/fe0 624769 0 2024-05-15T15:54:29.313 INFO:tasks.workunit.client.0.smithi049.stdout:0/850: chown dc/d13/d14/d25/d31/c6a 5738713 1 2024-05-15T15:54:29.313 INFO:tasks.workunit.client.0.smithi049.stdout:3/821: write d12/d14/fb8 [2442660,66041] 0 2024-05-15T15:54:29.315 INFO:tasks.workunit.client.0.smithi049.stdout:8/807: fdatasync d2/f3d 0 2024-05-15T15:54:29.319 INFO:tasks.workunit.client.0.smithi049.stdout:6/898: dread dd/d17/d24/d2b/f72 [0,4194304] 0 2024-05-15T15:54:29.324 INFO:tasks.workunit.client.0.smithi049.stdout:1/984: creat d0/d4/dd/de/d4e/f15a x:0 0 0 2024-05-15T15:54:29.328 INFO:tasks.workunit.client.0.smithi049.stdout:9/724: creat d0/d8/d9/d22/dbb/f103 x:0 0 0 2024-05-15T15:54:29.329 INFO:tasks.workunit.client.0.smithi049.stdout:9/725: chown d0/d8/d42/d75/fa5 4977075 1 2024-05-15T15:54:29.334 INFO:tasks.workunit.client.0.smithi049.stdout:9/726: dread d0/d8/d1b/d40/d5c/f70 [0,4194304] 0 2024-05-15T15:54:29.339 INFO:tasks.workunit.client.0.smithi049.stdout:0/851: mknod dc/d13/d3d/c120 0 2024-05-15T15:54:29.342 INFO:tasks.workunit.client.0.smithi049.stdout:8/808: fsync d2/fc5 0 2024-05-15T15:54:29.344 INFO:tasks.workunit.client.0.smithi049.stdout:8/809: truncate d2/de/d25/d74/f87 565180 0 2024-05-15T15:54:29.346 INFO:tasks.workunit.client.0.smithi049.stdout:1/985: chown d0/d4/dd/de/d36/fee 8378461 1 2024-05-15T15:54:29.348 INFO:tasks.workunit.client.0.smithi049.stdout:5/986: rename dc/dd/d7b/le9 to dc/dd/d7b/d8d/d135/l143 0 2024-05-15T15:54:29.349 INFO:tasks.workunit.client.0.smithi049.stdout:3/822: read - d12/d30/d66/f8f zero size 2024-05-15T15:54:29.351 INFO:tasks.workunit.client.0.smithi049.stdout:9/727: truncate d0/d8/d53/faa 970143 0 2024-05-15T15:54:29.352 INFO:tasks.workunit.client.0.smithi049.stdout:0/852: mknod dc/da4/db4/df1/c121 0 2024-05-15T15:54:29.364 INFO:tasks.workunit.client.0.smithi049.stdout:9/728: creat d0/d8/d1b/f104 x:0 0 0 2024-05-15T15:54:29.365 INFO:tasks.workunit.client.0.smithi049.stdout:9/729: chown d0/d8/d12/d9a 0 1 2024-05-15T15:54:29.368 INFO:tasks.workunit.client.0.smithi049.stdout:3/823: dwrite d12/d14/f1e [0,4194304] 0 2024-05-15T15:54:29.372 INFO:tasks.workunit.client.0.smithi049.stdout:9/730: dread d0/d8/d42/d75/fcf [0,4194304] 0 2024-05-15T15:54:29.373 INFO:tasks.workunit.client.0.smithi049.stdout:9/731: readlink d0/d8/d9/lcc 0 2024-05-15T15:54:29.373 INFO:tasks.workunit.client.0.smithi049.stdout:9/732: readlink d0/d8/d12/lb8 0 2024-05-15T15:54:29.376 INFO:tasks.workunit.client.0.smithi049.stdout:8/810: creat d2/de/d71/d86/d5d/d7d/db8/ff9 x:0 0 0 2024-05-15T15:54:29.378 INFO:tasks.workunit.client.0.smithi049.stdout:9/733: dread d0/d25/d5d/f8d [0,4194304] 0 2024-05-15T15:54:29.380 INFO:tasks.workunit.client.0.smithi049.stdout:9/734: fsync d0/d8/d9/d52/d5e/d5f/f38 0 2024-05-15T15:54:29.381 INFO:tasks.workunit.client.1.smithi144.stdout:7/944: mkdir d4/d45/d14a 0 2024-05-15T15:54:29.382 INFO:tasks.workunit.client.0.smithi049.stdout:5/987: symlink dc/dd/d10/d11d/l144 0 2024-05-15T15:54:29.383 INFO:tasks.workunit.client.1.smithi144.stdout:5/940: mknod d0/dc/d2f/d34/d63/d6e/db8/c13c 0 2024-05-15T15:54:29.387 INFO:tasks.workunit.client.0.smithi049.stdout:7/814: getdents d0/d6/d23/d45/d6a/da1/d80 0 2024-05-15T15:54:29.392 INFO:tasks.workunit.client.0.smithi049.stdout:6/899: getdents dd/d26/d5b/d110/d50/d7e 0 2024-05-15T15:54:29.393 INFO:tasks.workunit.client.1.smithi144.stdout:6/983: mkdir d7/db/d9d/d15e 0 2024-05-15T15:54:29.396 INFO:tasks.workunit.client.0.smithi049.stdout:6/900: dread dd/dbc/f10b [0,4194304] 0 2024-05-15T15:54:29.398 INFO:tasks.workunit.client.0.smithi049.stdout:0/853: write dc/d13/d56/d6f/fd2 [727724,95730] 0 2024-05-15T15:54:29.399 INFO:tasks.workunit.client.0.smithi049.stdout:0/854: stat dc/l12 0 2024-05-15T15:54:29.400 INFO:tasks.workunit.client.1.smithi144.stdout:3/923: rmdir d0/d1/d56/d71/d125 39 2024-05-15T15:54:29.402 INFO:tasks.workunit.client.0.smithi049.stdout:1/986: creat d0/d4/dd/de/d1e/d2e/dad/d65/d72/f15b x:0 0 0 2024-05-15T15:54:29.405 INFO:tasks.workunit.client.1.smithi144.stdout:5/941: mknod d0/dc/d2f/d79/c13d 0 2024-05-15T15:54:29.407 INFO:tasks.workunit.client.0.smithi049.stdout:9/735: truncate d0/d8/d1b/fbe 2449652 0 2024-05-15T15:54:29.407 INFO:tasks.workunit.client.1.smithi144.stdout:6/984: dread d7/f151 [0,4194304] 0 2024-05-15T15:54:29.408 INFO:tasks.workunit.client.1.smithi144.stdout:6/985: stat d7/d1e/d28/da8/cdf 0 2024-05-15T15:54:29.411 INFO:tasks.workunit.client.0.smithi049.stdout:3/824: mknod d12/d14/d52/db9/da6/cfe 0 2024-05-15T15:54:29.414 INFO:tasks.workunit.client.1.smithi144.stdout:2/865: dwrite d3/d7/d20/d3f/f56 [0,4194304] 0 2024-05-15T15:54:29.417 INFO:tasks.workunit.client.0.smithi049.stdout:0/855: rmdir dc/da4/db4 39 2024-05-15T15:54:29.417 INFO:tasks.workunit.client.1.smithi144.stdout:0/889: dwrite d4/d2d/fab [0,4194304] 0 2024-05-15T15:54:29.417 INFO:tasks.workunit.client.1.smithi144.stdout:7/945: dread d4/f67 [0,4194304] 0 2024-05-15T15:54:29.420 INFO:tasks.workunit.client.0.smithi049.stdout:1/987: creat d0/d4/dd/de/d1e/d16/d6e/dbf/f15c x:0 0 0 2024-05-15T15:54:29.426 INFO:tasks.workunit.client.1.smithi144.stdout:7/946: dread d4/df/d25/d38/d68/fe0 [0,4194304] 0 2024-05-15T15:54:29.428 INFO:tasks.workunit.client.1.smithi144.stdout:6/986: symlink d7/db/d35/d84/dbe/l15f 0 2024-05-15T15:54:29.429 INFO:tasks.workunit.client.1.smithi144.stdout:4/987: link d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/c67 d6/c141 0 2024-05-15T15:54:29.430 INFO:tasks.workunit.client.1.smithi144.stdout:6/987: dread - d7/db/d35/d84/dbe/f14f zero size 2024-05-15T15:54:29.432 INFO:tasks.workunit.client.0.smithi049.stdout:6/901: write dd/d17/d24/d2b/f76 [1618379,94955] 0 2024-05-15T15:54:29.434 INFO:tasks.workunit.client.1.smithi144.stdout:7/947: dwrite d4/d41/de8/f122 [0,4194304] 0 2024-05-15T15:54:29.436 INFO:tasks.workunit.client.1.smithi144.stdout:1/953: rename d7/d4b/dd9 to d7/df/d127 0 2024-05-15T15:54:29.437 INFO:tasks.workunit.client.0.smithi049.stdout:8/811: dwrite d2/de/d71/d86/f51 [0,4194304] 0 2024-05-15T15:54:29.437 INFO:tasks.workunit.client.1.smithi144.stdout:1/954: stat d7/df/d127 0 2024-05-15T15:54:29.438 INFO:tasks.workunit.client.1.smithi144.stdout:1/955: chown d7/d1d/d33/d1e/d34/c11a 418402305 1 2024-05-15T15:54:29.439 INFO:tasks.workunit.client.0.smithi049.stdout:5/988: truncate dc/f80 498148 0 2024-05-15T15:54:29.440 INFO:tasks.workunit.client.0.smithi049.stdout:5/989: chown dc/db0/dd5/d102 58624 1 2024-05-15T15:54:29.441 INFO:tasks.workunit.client.1.smithi144.stdout:3/924: symlink d0/d11/d3b/da7/def/d117/d119/l128 0 2024-05-15T15:54:29.443 INFO:tasks.workunit.client.1.smithi144.stdout:9/965: getdents d0/d8/d5e 0 2024-05-15T15:54:29.446 INFO:tasks.workunit.client.1.smithi144.stdout:2/866: symlink d3/d7/d20/d37/d76/db5/db6/dbf/l11d 0 2024-05-15T15:54:29.447 INFO:tasks.workunit.client.0.smithi049.stdout:8/812: dread d2/de/d71/fba [0,4194304] 0 2024-05-15T15:54:29.452 INFO:tasks.workunit.client.0.smithi049.stdout:6/902: creat dd/d17/d9d/f115 x:0 0 0 2024-05-15T15:54:29.457 INFO:tasks.workunit.client.0.smithi049.stdout:1/988: mknod d0/d4/dd/de/d1e/d2e/dad/d65/d72/c15d 0 2024-05-15T15:54:29.458 INFO:tasks.workunit.client.1.smithi144.stdout:2/867: truncate d3/d7/d20/d37/d76/fe5 68949 0 2024-05-15T15:54:29.459 INFO:tasks.workunit.client.1.smithi144.stdout:2/868: write d3/d7/d20/d22/d52/f69 [4634012,28067] 0 2024-05-15T15:54:29.461 INFO:tasks.workunit.client.0.smithi049.stdout:8/813: symlink d2/de/d25/d8c/d9c/lfa 0 2024-05-15T15:54:29.462 INFO:tasks.workunit.client.0.smithi049.stdout:0/856: write dc/d13/d14/d25/d31/f7e [3013942,41695] 0 2024-05-15T15:54:29.464 INFO:tasks.workunit.client.0.smithi049.stdout:0/857: read dc/d13/d3d/fb2 [2430213,59746] 0 2024-05-15T15:54:29.466 INFO:tasks.workunit.client.1.smithi144.stdout:5/942: link d0/dc/d24/d28/d2d/d59/d74/cd1 d0/dc/d24/d8e/d93/d9e/c13e 0 2024-05-15T15:54:29.466 INFO:tasks.workunit.client.1.smithi144.stdout:5/943: read - d0/dc/d2f/d79/f137 zero size 2024-05-15T15:54:29.467 INFO:tasks.workunit.client.0.smithi049.stdout:6/903: mkdir dd/d26/d5b/d110/d86/d116 0 2024-05-15T15:54:29.468 INFO:tasks.workunit.client.1.smithi144.stdout:4/988: mknod d6/dd/d34/d3b/d30/dc9/d10f/d10b/d12f/c142 0 2024-05-15T15:54:29.475 INFO:tasks.workunit.client.0.smithi049.stdout:9/736: getdents d0/d8/d9/d22 0 2024-05-15T15:54:29.476 INFO:tasks.workunit.client.1.smithi144.stdout:3/925: symlink d0/deb/l129 0 2024-05-15T15:54:29.480 INFO:tasks.workunit.client.0.smithi049.stdout:3/825: link d12/lf0 d12/lff 0 2024-05-15T15:54:29.481 INFO:tasks.workunit.client.1.smithi144.stdout:4/989: mknod d6/dd/d34/d3b/d30/db/d1d/d40/d57/c143 0 2024-05-15T15:54:29.485 INFO:tasks.workunit.client.1.smithi144.stdout:1/956: truncate d7/d1d/d33/d29/d6b/d5e/fa1 3003383 0 2024-05-15T15:54:29.485 INFO:tasks.workunit.client.1.smithi144.stdout:7/948: link d4/df/d79/d100/f101 d4/df/d81/dfb/dbd/f14b 0 2024-05-15T15:54:29.485 INFO:tasks.workunit.client.0.smithi049.stdout:0/858: dwrite dc/d13/d56/fa0 [0,4194304] 0 2024-05-15T15:54:29.487 INFO:tasks.workunit.client.0.smithi049.stdout:0/859: fsync dc/d13/d2b/d76/d8a/da2/fbd 0 2024-05-15T15:54:29.488 INFO:tasks.workunit.client.1.smithi144.stdout:6/988: rename d7/db/d11/d7e/d132/f12a to d7/f160 0 2024-05-15T15:54:29.494 INFO:tasks.workunit.client.0.smithi049.stdout:9/737: creat d0/d8/d1b/d40/f105 x:0 0 0 2024-05-15T15:54:29.494 INFO:tasks.workunit.client.1.smithi144.stdout:4/990: mkdir d6/d42/dc8/d144 0 2024-05-15T15:54:29.495 INFO:tasks.workunit.client.0.smithi049.stdout:6/904: dwrite dd/f32 [0,4194304] 0 2024-05-15T15:54:29.501 INFO:tasks.workunit.client.1.smithi144.stdout:4/991: dwrite d6/dd/fd8 [0,4194304] 0 2024-05-15T15:54:29.505 INFO:tasks.workunit.client.1.smithi144.stdout:9/966: rename d0/d8/d5e/d44/d16/db0/cc3 to d0/d8/d1b/d62/d7d/dd2/c14b 0 2024-05-15T15:54:29.505 INFO:tasks.workunit.client.0.smithi049.stdout:3/826: dread d12/d14/d52/db9/f46 [0,4194304] 0 2024-05-15T15:54:29.509 INFO:tasks.workunit.client.0.smithi049.stdout:5/990: dread dc/f80 [0,4194304] 0 2024-05-15T15:54:29.510 INFO:tasks.workunit.client.1.smithi144.stdout:0/890: getdents d4/df/d1f/d56 0 2024-05-15T15:54:29.511 INFO:tasks.workunit.client.0.smithi049.stdout:1/989: creat d0/d4/dd/de/d1e/d2e/dad/f15e x:0 0 0 2024-05-15T15:54:29.514 INFO:tasks.workunit.client.0.smithi049.stdout:0/860: mknod dc/d13/d2b/d76/d8a/da2/db3/d100/c122 0 2024-05-15T15:54:29.515 INFO:tasks.workunit.client.1.smithi144.stdout:1/957: creat d7/d1d/d33/d41/d22/dbd/dc8/d11b/f128 x:0 0 0 2024-05-15T15:54:29.516 INFO:tasks.workunit.client.0.smithi049.stdout:8/814: creat d2/de/d71/d86/ffb x:0 0 0 2024-05-15T15:54:29.518 INFO:tasks.workunit.client.0.smithi049.stdout:9/738: creat d0/d8/d12/d9a/f106 x:0 0 0 2024-05-15T15:54:29.519 INFO:tasks.workunit.client.0.smithi049.stdout:6/905: creat dd/d26/d5b/d110/d86/f117 x:0 0 0 2024-05-15T15:54:29.520 INFO:tasks.workunit.client.1.smithi144.stdout:7/949: dwrite d4/df/d81/dfb/fc8 [0,4194304] 0 2024-05-15T15:54:29.521 INFO:tasks.workunit.client.0.smithi049.stdout:3/827: creat d12/d14/d52/db9/dc4/f100 x:0 0 0 2024-05-15T15:54:29.522 INFO:tasks.workunit.client.1.smithi144.stdout:3/926: link d0/d11/d24/l85 d0/d11/d3b/d93/d126/l12a 0 2024-05-15T15:54:29.524 INFO:tasks.workunit.client.1.smithi144.stdout:0/891: mknod d4/d6/d3a/d5d/c144 0 2024-05-15T15:54:29.525 INFO:tasks.workunit.client.0.smithi049.stdout:8/815: readlink d2/de/d71/d86/d5d/l91 0 2024-05-15T15:54:29.528 INFO:tasks.workunit.client.0.smithi049.stdout:6/906: rmdir dd/d26/d5b/da8 39 2024-05-15T15:54:29.530 INFO:tasks.workunit.client.1.smithi144.stdout:1/958: mkdir d7/d1d/d33/d29/dd2/dea/d129 0 2024-05-15T15:54:29.530 INFO:tasks.workunit.client.1.smithi144.stdout:6/989: write d7/f123 [360286,112389] 0 2024-05-15T15:54:29.533 INFO:tasks.workunit.client.0.smithi049.stdout:5/991: mkdir dc/dd/d47/d83/dcc/d145 0 2024-05-15T15:54:29.537 INFO:tasks.workunit.client.1.smithi144.stdout:9/967: fsync d0/f14 0 2024-05-15T15:54:29.538 INFO:tasks.workunit.client.0.smithi049.stdout:8/816: creat d2/de/d71/d86/d5d/d7d/db8/ffc x:0 0 0 2024-05-15T15:54:29.539 INFO:tasks.workunit.client.1.smithi144.stdout:9/968: dread d0/d8/d50/f143 [0,4194304] 0 2024-05-15T15:54:29.541 INFO:tasks.workunit.client.0.smithi049.stdout:6/907: unlink dd/d17/d24/d58/d5f/dd7/fbf 0 2024-05-15T15:54:29.541 INFO:tasks.workunit.client.0.smithi049.stdout:6/908: chown dd/d17/d24/d2b/l73 22 1 2024-05-15T15:54:29.542 INFO:tasks.workunit.client.1.smithi144.stdout:7/950: stat d4/d3c/d76/d9d/dac/de5/l109 0 2024-05-15T15:54:29.543 INFO:tasks.workunit.client.0.smithi049.stdout:5/992: write dc/f80 [1292451,126267] 0 2024-05-15T15:54:29.543 INFO:tasks.workunit.client.0.smithi049.stdout:1/990: write d0/d4/dd/de/d48/d5c/fe0 [3249009,33029] 0 2024-05-15T15:54:29.544 INFO:tasks.workunit.client.1.smithi144.stdout:3/927: write d0/d11/ff7 [3987605,3002] 0 2024-05-15T15:54:29.545 INFO:tasks.workunit.client.1.smithi144.stdout:9/969: dwrite d0/d8/d5e/d44/d10/d19/dd5/d12b/f13b [0,4194304] 0 2024-05-15T15:54:29.546 INFO:tasks.workunit.client.1.smithi144.stdout:3/928: dread - d0/d11/d3b/da7/fe1 zero size 2024-05-15T15:54:29.547 INFO:tasks.workunit.client.0.smithi049.stdout:3/828: rename d12/d30/l45 to d12/d14/d52/db9/da6/dc2/l101 0 2024-05-15T15:54:29.552 INFO:tasks.workunit.client.1.smithi144.stdout:6/990: fsync d7/fc4 0 2024-05-15T15:54:29.554 INFO:tasks.workunit.client.0.smithi049.stdout:6/909: mknod dd/d17/d24/d67/c118 0 2024-05-15T15:54:29.556 INFO:tasks.workunit.client.0.smithi049.stdout:1/991: mkdir d0/d4/dd/de/d1e/d16/d6e/d15f 0 2024-05-15T15:54:29.558 INFO:tasks.workunit.client.1.smithi144.stdout:6/991: dwrite d7/db/d35/d84/d81/fa0 [0,4194304] 0 2024-05-15T15:54:29.559 INFO:tasks.workunit.client.0.smithi049.stdout:5/993: rename dc/dd/d47/d83/fc2 to dc/dd/d47/d132/f146 0 2024-05-15T15:54:29.562 INFO:tasks.workunit.client.1.smithi144.stdout:2/869: rename d3/d7/d20/d37/de1/f101 to d3/d7/d20/d89/f11e 0 2024-05-15T15:54:29.564 INFO:tasks.workunit.client.0.smithi049.stdout:3/829: dread d12/d14/d52/db9/dc4/d68/d70/d75/f85 [0,4194304] 0 2024-05-15T15:54:29.566 INFO:tasks.workunit.client.0.smithi049.stdout:1/992: mkdir d0/d4/dd/de/d1e/d16/d8a/dd2/d110/d160 0 2024-05-15T15:54:29.567 INFO:tasks.workunit.client.1.smithi144.stdout:9/970: dread - d0/d8/d1b/d62/d6b/f116 zero size 2024-05-15T15:54:29.574 INFO:tasks.workunit.client.0.smithi049.stdout:6/910: mknod dd/c119 0 2024-05-15T15:54:29.576 INFO:tasks.workunit.client.1.smithi144.stdout:0/892: write d4/ff6 [786093,118910] 0 2024-05-15T15:54:29.576 INFO:tasks.workunit.client.0.smithi049.stdout:3/830: chown d12/d14/d52/db9/da6/dc2/l101 664346956 1 2024-05-15T15:54:29.578 INFO:tasks.workunit.client.0.smithi049.stdout:1/993: rename d0/d4/dd/de/d48/d84/l112 to d0/d4/dd/de/d1e/d16/d8a/dd2/d110/d139/l161 0 2024-05-15T15:54:29.579 INFO:tasks.workunit.client.0.smithi049.stdout:8/817: getdents d2/de/d71/d86/d5d/d7d/db8/d5b 0 2024-05-15T15:54:29.580 INFO:tasks.workunit.client.0.smithi049.stdout:5/994: truncate dc/dd/d47/f9c 1337421 0 2024-05-15T15:54:29.581 INFO:tasks.workunit.client.0.smithi049.stdout:6/911: creat dd/d26/d5b/d110/d50/d54/dde/df2/f11a x:0 0 0 2024-05-15T15:54:29.583 INFO:tasks.workunit.client.0.smithi049.stdout:1/994: fsync d0/d4/d63/f95 0 2024-05-15T15:54:29.588 INFO:tasks.workunit.client.0.smithi049.stdout:3/831: fdatasync d12/d14/d32/f62 0 2024-05-15T15:54:29.589 INFO:tasks.workunit.client.1.smithi144.stdout:4/992: link d6/dd/d2e/d75/dff/deb/f77 d6/dd/d2e/f145 0 2024-05-15T15:54:29.594 INFO:tasks.workunit.client.0.smithi049.stdout:8/818: symlink d2/de/d71/d86/d5d/lfd 0 2024-05-15T15:54:29.596 INFO:tasks.workunit.client.1.smithi144.stdout:5/944: rename d0/dc/d24/d57/d96/cde to d0/dc/d1f/d50/db3/d106/c13f 0 2024-05-15T15:54:29.598 INFO:tasks.workunit.client.1.smithi144.stdout:7/951: creat d4/d3c/d76/d9d/dac/f14c x:0 0 0 2024-05-15T15:54:29.600 INFO:tasks.workunit.client.1.smithi144.stdout:7/952: read d4/df/d81/dfb/dbd/fd4 [1097477,49382] 0 2024-05-15T15:54:29.601 INFO:tasks.workunit.client.1.smithi144.stdout:0/893: dwrite d4/d2d/d48/d5b/d8e/f97 [0,4194304] 0 2024-05-15T15:54:29.601 INFO:tasks.workunit.client.1.smithi144.stdout:4/993: creat d6/dd/d34/d3b/d30/db/d1d/d101/f146 x:0 0 0 2024-05-15T15:54:29.604 INFO:tasks.workunit.client.0.smithi049.stdout:1/995: truncate d0/d4/dd/f18 4180554 0 2024-05-15T15:54:29.604 INFO:tasks.workunit.client.1.smithi144.stdout:7/953: write d4/df/d25/d38/d5d/f8c [1735446,111320] 0 2024-05-15T15:54:29.606 INFO:tasks.workunit.client.0.smithi049.stdout:5/995: creat dc/db0/dd5/f147 x:0 0 0 2024-05-15T15:54:29.626 INFO:tasks.workunit.client.0.smithi049.stdout:5/996: dread dc/dd/d7b/fdf [0,4194304] 0 2024-05-15T15:54:29.660 INFO:tasks.workunit.client.1.smithi144.stdout:3/929: getdents d0/d11/d3b/da0 0 2024-05-15T15:54:29.661 INFO:tasks.workunit.client.0.smithi049.stdout:6/912: creat dd/d26/d5b/d110/d50/d54/d92/f11b x:0 0 0 2024-05-15T15:54:29.663 INFO:tasks.workunit.client.1.smithi144.stdout:0/894: rmdir d4/d2d/d48/d10a 39 2024-05-15T15:54:29.665 INFO:tasks.workunit.client.0.smithi049.stdout:3/832: dwrite d12/d30/f6f [4194304,4194304] 0 2024-05-15T15:54:29.666 INFO:tasks.workunit.client.1.smithi144.stdout:6/992: link d7/db/d11/d3c/d61/fc5 d7/db/f161 0 2024-05-15T15:54:29.667 INFO:tasks.workunit.client.0.smithi049.stdout:3/833: chown d12/d30/dc1/dc7/df8/dcf 652 1 2024-05-15T15:54:29.673 INFO:tasks.workunit.client.0.smithi049.stdout:5/997: stat dc/dd/d47/ld4 0 2024-05-15T15:54:29.673 INFO:tasks.workunit.client.0.smithi049.stdout:6/913: dwrite dd/d26/d5b/d110/d50/d54/f114 [0,4194304] 0 2024-05-15T15:54:29.678 INFO:tasks.workunit.client.1.smithi144.stdout:0/895: mknod d4/df/d14/c145 0 2024-05-15T15:54:29.678 INFO:tasks.workunit.client.1.smithi144.stdout:6/993: creat d7/db/d5f/d12c/d148/f162 x:0 0 0 2024-05-15T15:54:29.679 INFO:tasks.workunit.client.1.smithi144.stdout:0/896: chown d4/d6/d3a/d5d/f82 250671 1 2024-05-15T15:54:29.680 INFO:tasks.workunit.client.1.smithi144.stdout:0/897: stat d4/df/d1f/d24/d96/fca 0 2024-05-15T15:54:29.682 INFO:tasks.workunit.client.1.smithi144.stdout:1/959: rename d7/d1d/d33/d41/d78/da3/f88 to d7/d1d/d33/d41/d78/f12a 0 2024-05-15T15:54:29.682 INFO:tasks.workunit.client.0.smithi049.stdout:1/996: write d0/d4/f79 [526961,64069] 0 2024-05-15T15:54:29.690 INFO:tasks.workunit.client.1.smithi144.stdout:1/960: dwrite d7/d1d/d33/d1e/f25 [4194304,4194304] 0 2024-05-15T15:54:29.690 INFO:tasks.workunit.client.1.smithi144.stdout:7/954: dwrite d4/d3c/d76/d9d/dac/d48/fef [0,4194304] 0 2024-05-15T15:54:29.691 INFO:tasks.workunit.client.1.smithi144.stdout:7/955: stat d4/d45/f96 0 2024-05-15T15:54:29.691 INFO:tasks.workunit.client.1.smithi144.stdout:1/961: chown d7/d1d/d58/d81/d105 450869 1 2024-05-15T15:54:29.692 INFO:tasks.workunit.client.1.smithi144.stdout:1/962: read d7/df/f9f [1985,7738] 0 2024-05-15T15:54:29.694 INFO:tasks.workunit.client.0.smithi049.stdout:3/834: dwrite d12/d30/d66/fca [0,4194304] 0 2024-05-15T15:54:29.695 INFO:tasks.workunit.client.0.smithi049.stdout:8/819: link d2/de/d71/d86/d5d/d7d/db8/d5b/d93/ccd d2/de/d71/d86/d5d/d7d/d45/cfe 0 2024-05-15T15:54:29.696 INFO:tasks.workunit.client.1.smithi144.stdout:0/898: dread - d4/df/d102/d11b/f106 zero size 2024-05-15T15:54:29.698 INFO:tasks.workunit.client.1.smithi144.stdout:9/971: rename d0/d8/d5e/d44/da2 to d0/d8/d5e/d44/d99/d120/d14c 0 2024-05-15T15:54:29.699 INFO:tasks.workunit.client.1.smithi144.stdout:9/972: chown d0/d8/d5e/d44/d10/f81 18 1 2024-05-15T15:54:29.699 INFO:tasks.workunit.client.0.smithi049.stdout:1/997: mknod d0/d4/dd/de/d4d/db8/c162 0 2024-05-15T15:54:29.700 INFO:tasks.workunit.client.1.smithi144.stdout:9/973: fsync d0/d8/d1b/d62/d7d/dd2/f13e 0 2024-05-15T15:54:29.704 INFO:tasks.workunit.client.1.smithi144.stdout:6/994: dread d7/db/d11/d3c/f4d [0,4194304] 0 2024-05-15T15:54:29.706 INFO:tasks.workunit.client.1.smithi144.stdout:4/994: link d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/f51 d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/dab/f147 0 2024-05-15T15:54:29.708 INFO:tasks.workunit.client.1.smithi144.stdout:9/974: dwrite d0/d8/d5e/d44/d10/f27 [0,4194304] 0 2024-05-15T15:54:29.709 INFO:tasks.workunit.client.1.smithi144.stdout:3/930: creat d0/d1/d56/d71/f12b x:0 0 0 2024-05-15T15:54:29.711 INFO:tasks.workunit.client.1.smithi144.stdout:3/931: chown d0/d11/d24/da5/lbf 19123 1 2024-05-15T15:54:29.713 INFO:tasks.workunit.client.0.smithi049.stdout:8/820: dread - d2/de/d71/d86/d5d/d7d/db8/d5b/d60/f9b zero size 2024-05-15T15:54:29.716 INFO:tasks.workunit.client.1.smithi144.stdout:0/899: mknod d4/df/d102/d11b/dd1/c146 0 2024-05-15T15:54:29.717 INFO:tasks.workunit.client.1.smithi144.stdout:0/900: chown d4/d6/d3a/d5d/l127 4578094 1 2024-05-15T15:54:29.721 INFO:tasks.workunit.client.1.smithi144.stdout:5/945: rename d0/c4f to d0/d10c/d73/d116/c140 0 2024-05-15T15:54:29.724 INFO:tasks.workunit.client.0.smithi049.stdout:6/914: truncate dd/f57 3242261 0 2024-05-15T15:54:29.725 INFO:tasks.workunit.client.1.smithi144.stdout:6/995: creat d7/db/d5f/d12c/d148/f163 x:0 0 0 2024-05-15T15:54:29.727 INFO:tasks.workunit.client.0.smithi049.stdout:1/998: mkdir d0/d4/dd/de/d1e/d16/d8a/dd2/d110/d160/d163 0 2024-05-15T15:54:29.727 INFO:tasks.workunit.client.0.smithi049.stdout:6/915: read dd/d17/d24/d2b/f72 [2527613,84739] 0 2024-05-15T15:54:29.728 INFO:tasks.workunit.client.1.smithi144.stdout:1/963: write d7/d1d/d33/d1e/d34/f38 [459988,116676] 0 2024-05-15T15:54:29.730 INFO:tasks.workunit.client.0.smithi049.stdout:5/998: dwrite dc/dd/d47/f4d [0,4194304] 0 2024-05-15T15:54:29.730 INFO:tasks.workunit.client.0.smithi049.stdout:3/835: dwrite d12/d30/d66/fca [4194304,4194304] 0 2024-05-15T15:54:29.731 INFO:tasks.workunit.client.1.smithi144.stdout:3/932: fsync d0/d11/d3b/da0/f10d 0 2024-05-15T15:54:29.731 INFO:tasks.workunit.client.1.smithi144.stdout:4/995: dread d6/dd/d2e/d75/dff/ff9 [0,4194304] 0 2024-05-15T15:54:29.734 INFO:tasks.workunit.client.1.smithi144.stdout:2/870: sync 2024-05-15T15:54:29.737 INFO:tasks.workunit.client.0.smithi049.stdout:5/999: rmdir dc/db0/dbf 39 2024-05-15T15:54:29.740 INFO:tasks.workunit.client.1.smithi144.stdout:0/901: mkdir d4/d2d/dcf/d147 0 2024-05-15T15:54:29.742 INFO:tasks.workunit.client.0.smithi049.stdout:6/916: mkdir dd/d26/d5b/d110/d50/dcf/d11c 0 2024-05-15T15:54:29.745 INFO:tasks.workunit.client.1.smithi144.stdout:6/996: dread d7/db/d35/d84/d81/f102 [0,4194304] 0 2024-05-15T15:54:29.754 INFO:tasks.workunit.client.0.smithi049.stdout:3/836: write d12/d30/f3d [231998,98150] 0 2024-05-15T15:54:29.755 INFO:tasks.workunit.client.0.smithi049.stdout:8/821: dwrite d2/de/d71/d86/d5d/f80 [4194304,4194304] 0 2024-05-15T15:54:29.756 INFO:tasks.workunit.client.0.smithi049.stdout:3/837: chown d12/d14/d32/f59 124 1 2024-05-15T15:54:29.762 INFO:tasks.workunit.client.1.smithi144.stdout:4/996: fdatasync d6/dd/d2e/f145 0 2024-05-15T15:54:29.764 INFO:tasks.workunit.client.0.smithi049.stdout:8/822: dwrite d2/de/f16 [0,4194304] 0 2024-05-15T15:54:29.765 INFO:tasks.workunit.client.1.smithi144.stdout:7/956: creat d4/df/d25/d38/f14d x:0 0 0 2024-05-15T15:54:29.766 INFO:tasks.workunit.client.1.smithi144.stdout:7/957: fsync d4/f27 0 2024-05-15T15:54:29.766 INFO:tasks.workunit.client.0.smithi049.stdout:1/999: link d0/d4/dd/de/d4e/f7e d0/d4/dd/de/d36/dd3/d145/f164 0 2024-05-15T15:54:29.768 INFO:tasks.workunit.client.1.smithi144.stdout:2/871: creat d3/d7/da4/f11f x:0 0 0 2024-05-15T15:54:29.770 INFO:tasks.workunit.client.1.smithi144.stdout:7/958: write d4/d41/fab [3212773,55419] 0 2024-05-15T15:54:29.771 INFO:tasks.workunit.client.1.smithi144.stdout:0/902: creat d4/d6/d4a/d74/f148 x:0 0 0 2024-05-15T15:54:29.774 INFO:tasks.workunit.client.1.smithi144.stdout:6/997: creat d7/d1e/d28/db5/f164 x:0 0 0 2024-05-15T15:54:29.774 INFO:tasks.workunit.client.1.smithi144.stdout:1/964: dwrite d7/d1d/f8b [0,4194304] 0 2024-05-15T15:54:29.776 INFO:tasks.workunit.client.1.smithi144.stdout:9/975: write d0/d8/d1b/d11c/ffa [742780,73438] 0 2024-05-15T15:54:29.782 INFO:tasks.workunit.client.1.smithi144.stdout:2/872: dread d3/d7/d20/d22/d5e/f106 [0,4194304] 0 2024-05-15T15:54:29.783 INFO:tasks.workunit.client.1.smithi144.stdout:6/998: dwrite d7/db/d3a/dd2/f116 [0,4194304] 0 2024-05-15T15:54:29.784 INFO:tasks.workunit.client.1.smithi144.stdout:0/903: mknod d4/d6/d3a/d5d/c149 0 2024-05-15T15:54:29.785 INFO:tasks.workunit.client.0.smithi049.stdout:8/823: link d2/de/d71/d86/d5d/d7d/d1b/d68/fab d2/de/d71/d86/d5d/d7d/d1d/fff 0 2024-05-15T15:54:29.790 INFO:tasks.workunit.client.0.smithi049.stdout:8/824: symlink d2/de/d71/d86/d5d/d7d/d1b/d69/l100 0 2024-05-15T15:54:29.798 INFO:tasks.workunit.client.1.smithi144.stdout:4/997: link d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d99/dda/f11f d6/dd/d2e/d75/dff/deb/dba/de0/f148 0 2024-05-15T15:54:29.799 INFO:tasks.workunit.client.0.smithi049.stdout:3/838: dwrite d12/d14/d52/db9/dc4/d68/fa0 [0,4194304] 0 2024-05-15T15:54:29.802 INFO:tasks.workunit.client.1.smithi144.stdout:3/933: dwrite d0/d1/d56/d71/d125/fc1 [0,4194304] 0 2024-05-15T15:54:29.805 INFO:tasks.workunit.client.1.smithi144.stdout:6/999: mkdir d7/db/d9d/d72/dee/d165 0 2024-05-15T15:54:29.812 INFO:tasks.workunit.client.0.smithi049.stdout:3/839: creat d12/d14/d52/db9/dc4/f102 x:0 0 0 2024-05-15T15:54:29.812 INFO:tasks.workunit.client.1.smithi144.stdout:0/904: dread d4/d6/d3a/d4c/dd6/d55/da5/fe3 [0,4194304] 0 2024-05-15T15:54:29.813 INFO:tasks.workunit.client.1.smithi144.stdout:0/905: rename d4/d6/d4a to d4/d6/d4a/d4d/db1/d13a/d14a 22 2024-05-15T15:54:29.814 INFO:tasks.workunit.client.1.smithi144.stdout:0/906: chown d4/d6/d3a/d4c/f11a 125505979 1 2024-05-15T15:54:29.815 INFO:tasks.workunit.client.1.smithi144.stdout:2/873: truncate d3/d7/d20/d22/f7a 2644633 0 2024-05-15T15:54:29.815 INFO:tasks.workunit.client.0.smithi049.stdout:6/917: dwrite f4 [0,4194304] 0 2024-05-15T15:54:29.818 INFO:tasks.workunit.client.0.smithi049.stdout:8/825: write d2/de/d71/d86/d5d/d7d/d1b/d68/fab [206532,121112] 0 2024-05-15T15:54:29.820 INFO:tasks.workunit.client.1.smithi144.stdout:1/965: truncate d7/d1d/d33/d29/d6b/d5e/fa1 1835991 0 2024-05-15T15:54:29.821 INFO:tasks.workunit.client.0.smithi049.stdout:6/918: rmdir dd/d17 39 2024-05-15T15:54:29.822 INFO:tasks.workunit.client.1.smithi144.stdout:5/946: link d0/dc/d2f/d34/d3e/fed d0/d10c/f141 0 2024-05-15T15:54:29.825 INFO:tasks.workunit.client.0.smithi049.stdout:3/840: unlink d12/d14/f4f 0 2024-05-15T15:54:29.826 INFO:tasks.workunit.client.1.smithi144.stdout:7/959: truncate d4/f27 350822 0 2024-05-15T15:54:29.826 INFO:tasks.workunit.client.1.smithi144.stdout:7/960: chown d4/d3c/d76/d9d/dac/f23 5 1 2024-05-15T15:54:29.828 INFO:tasks.workunit.client.0.smithi049.stdout:8/826: symlink d2/de/d25/d74/l101 0 2024-05-15T15:54:29.832 INFO:tasks.workunit.client.0.smithi049.stdout:6/919: chown dd/d17/d24/d2b/f72 535856 1 2024-05-15T15:54:29.835 INFO:tasks.workunit.client.1.smithi144.stdout:4/998: dwrite d6/dd/d34/d3b/d30/dc9/d10f/d4b/f4d [0,4194304] 0 2024-05-15T15:54:29.838 INFO:tasks.workunit.client.1.smithi144.stdout:0/907: mkdir d4/df/d1f/d56/d14b 0 2024-05-15T15:54:29.839 INFO:tasks.workunit.client.1.smithi144.stdout:9/976: write d0/d8/d1b/fb9 [4518224,123135] 0 2024-05-15T15:54:29.841 INFO:tasks.workunit.client.0.smithi049.stdout:6/920: rmdir dd/d17/d24/d58/d5f 39 2024-05-15T15:54:29.843 INFO:tasks.workunit.client.1.smithi144.stdout:2/874: mknod d3/d10/c120 0 2024-05-15T15:54:29.845 INFO:tasks.workunit.client.1.smithi144.stdout:3/934: write d0/d11/d3b/da7/fc6 [2080049,75994] 0 2024-05-15T15:54:29.846 INFO:tasks.workunit.client.1.smithi144.stdout:3/935: write d0/d11/d24/d78/fc2 [99991,36941] 0 2024-05-15T15:54:29.848 INFO:tasks.workunit.client.1.smithi144.stdout:7/961: dread d4/df/f65 [0,4194304] 0 2024-05-15T15:54:29.849 INFO:tasks.workunit.client.0.smithi049.stdout:8/827: creat d2/de/d71/d86/f102 x:0 0 0 2024-05-15T15:54:29.850 INFO:tasks.workunit.client.0.smithi049.stdout:3/841: getdents d12/d30/dc1/dc7/ddc 0 2024-05-15T15:54:29.851 INFO:tasks.workunit.client.0.smithi049.stdout:3/842: chown d12/d14/d52/db9 12064942 1 2024-05-15T15:54:29.852 INFO:tasks.workunit.client.0.smithi049.stdout:6/921: fsync dd/d26/d5b/d110/d50/d54/d92/ff5 0 2024-05-15T15:54:29.855 INFO:tasks.workunit.client.1.smithi144.stdout:4/999: unlink d6/dd/d34/d3b/d30/dc9/d10f/d4b/d1e/d44/f81 0 2024-05-15T15:54:29.858 INFO:tasks.workunit.client.1.smithi144.stdout:0/908: creat d4/d2d/d48/d5b/dd3/f14c x:0 0 0 2024-05-15T15:54:29.861 INFO:tasks.workunit.client.0.smithi049.stdout:8/828: dwrite d2/de/d71/d86/d5d/d7d/d1b/d68/f97 [0,4194304] 0 2024-05-15T15:54:29.869 INFO:tasks.workunit.client.0.smithi049.stdout:6/922: write dd/f89 [1452368,77482] 0 2024-05-15T15:54:29.873 INFO:tasks.workunit.client.0.smithi049.stdout:8/829: mkdir d2/de/d71/d86/d5d/d7d/d1b/d68/dee/d103 0 2024-05-15T15:54:29.873 INFO:tasks.workunit.client.0.smithi049.stdout:3/843: dwrite d12/d14/d52/db9/f46 [0,4194304] 0 2024-05-15T15:54:29.876 INFO:tasks.workunit.client.1.smithi144.stdout:3/936: unlink d0/d1/d17/d39/d40/l8a 0 2024-05-15T15:54:29.876 INFO:tasks.workunit.client.0.smithi049.stdout:6/923: dread dd/d17/d24/d2b/f72 [0,4194304] 0 2024-05-15T15:54:29.877 INFO:tasks.workunit.client.1.smithi144.stdout:5/947: truncate d0/dc/d24/d28/f51 2047748 0 2024-05-15T15:54:29.878 INFO:tasks.workunit.client.0.smithi049.stdout:6/924: read - dd/d26/d5b/d110/d50/f9a zero size 2024-05-15T15:54:29.878 INFO:tasks.workunit.client.1.smithi144.stdout:2/875: dread d3/d7/d20/d22/f2b [0,4194304] 0 2024-05-15T15:54:29.880 INFO:tasks.workunit.client.1.smithi144.stdout:7/962: symlink d4/df/d25/d40/d62/l14e 0 2024-05-15T15:54:29.881 INFO:tasks.workunit.client.0.smithi049.stdout:8/830: fsync d2/f3d 0 2024-05-15T15:54:29.883 INFO:tasks.workunit.client.0.smithi049.stdout:3/844: chown d12/d30/cdf 0 1 2024-05-15T15:54:29.883 INFO:tasks.workunit.client.0.smithi049.stdout:2/745: sync 2024-05-15T15:54:29.883 INFO:tasks.workunit.client.0.smithi049.stdout:7/815: sync 2024-05-15T15:54:29.883 INFO:tasks.workunit.client.0.smithi049.stdout:4/796: sync 2024-05-15T15:54:29.884 INFO:tasks.workunit.client.0.smithi049.stdout:6/925: creat dd/d26/d5b/d110/d50/d54/dde/d8c/f11d x:0 0 0 2024-05-15T15:54:29.885 INFO:tasks.workunit.client.1.smithi144.stdout:1/966: link d7/d1d/d33/d29/d6b/c97 d7/d1d/d58/c12b 0 2024-05-15T15:54:29.885 INFO:tasks.workunit.client.0.smithi049.stdout:3/845: read ff [3485509,24575] 0 2024-05-15T15:54:29.888 INFO:tasks.workunit.client.1.smithi144.stdout:3/937: rename d0/d11/d3b/da0/c79 to d0/d11/d3b/da0/c12c 0 2024-05-15T15:54:29.894 INFO:tasks.workunit.client.0.smithi049.stdout:3/846: dwrite d12/ff5 [0,4194304] 0 2024-05-15T15:54:29.894 INFO:tasks.workunit.client.0.smithi049.stdout:7/816: dwrite d0/d6/d23/faf [0,4194304] 0 2024-05-15T15:54:29.904 INFO:tasks.workunit.client.0.smithi049.stdout:3/847: dwrite d12/d30/dc1/dc7/df8/fd7 [0,4194304] 0 2024-05-15T15:54:29.905 INFO:tasks.workunit.client.1.smithi144.stdout:5/948: write d0/dc/d1f/f38 [437860,98749] 0 2024-05-15T15:54:29.908 INFO:tasks.workunit.client.1.smithi144.stdout:1/967: mknod d7/d1d/d33/de7/c12c 0 2024-05-15T15:54:29.914 INFO:tasks.workunit.client.1.smithi144.stdout:2/876: link d3/d4d/fca d3/d4d/db4/f121 0 2024-05-15T15:54:29.917 INFO:tasks.workunit.client.0.smithi049.stdout:7/817: creat d0/d34/db3/f100 x:0 0 0 2024-05-15T15:54:29.919 INFO:tasks.workunit.client.0.smithi049.stdout:2/746: creat d3/da/d15/d24/ffb x:0 0 0 2024-05-15T15:54:29.920 INFO:tasks.workunit.client.1.smithi144.stdout:0/909: creat d4/df/d1f/f14d x:0 0 0 2024-05-15T15:54:29.921 INFO:tasks.workunit.client.1.smithi144.stdout:3/938: dread d0/d1/f43 [0,4194304] 0 2024-05-15T15:54:29.922 INFO:tasks.workunit.client.1.smithi144.stdout:3/939: write d0/d1/d56/d98/d9c/fe3 [4382205,4755] 0 2024-05-15T15:54:29.923 INFO:tasks.workunit.client.0.smithi049.stdout:4/797: rename c13 to d18/d21/d2b/d4e/dd2/c108 0 2024-05-15T15:54:29.924 INFO:tasks.workunit.client.0.smithi049.stdout:4/798: readlink d18/d1e/d2a/d36/d5d/le5 0 2024-05-15T15:54:29.924 INFO:tasks.workunit.client.1.smithi144.stdout:9/977: getdents d0/d8/d5e/d44/d10/d19 0 2024-05-15T15:54:29.924 INFO:tasks.workunit.client.1.smithi144.stdout:3/940: dread d0/d1/d17/fd8 [0,4194304] 0 2024-05-15T15:54:29.927 INFO:tasks.workunit.client.0.smithi049.stdout:3/848: creat d12/d14/d52/db9/dc4/f103 x:0 0 0 2024-05-15T15:54:29.929 INFO:tasks.workunit.client.0.smithi049.stdout:6/926: dwrite dd/d26/d5b/d110/d86/d96/fe1 [0,4194304] 0 2024-05-15T15:54:29.931 INFO:tasks.workunit.client.0.smithi049.stdout:2/747: mknod d3/db/d1d/d1e/d2b/d3b/d42/da8/cfc 0 2024-05-15T15:54:29.932 INFO:tasks.workunit.client.0.smithi049.stdout:7/818: dread d0/ddb/dde/dca/d5e/f9e [0,4194304] 0 2024-05-15T15:54:29.932 INFO:tasks.workunit.client.1.smithi144.stdout:0/910: truncate d4/d6/d3a/d40/f10c 768332 0 2024-05-15T15:54:29.936 INFO:tasks.workunit.client.1.smithi144.stdout:5/949: mknod d0/dc/d2f/d109/c142 0 2024-05-15T15:54:29.937 INFO:tasks.workunit.client.1.smithi144.stdout:3/941: read - d0/d1/d56/d71/d125/fb7 zero size 2024-05-15T15:54:29.940 INFO:tasks.workunit.client.0.smithi049.stdout:9/739: sync 2024-05-15T15:54:29.940 INFO:tasks.workunit.client.0.smithi049.stdout:0/861: sync 2024-05-15T15:54:29.941 INFO:tasks.workunit.client.1.smithi144.stdout:0/911: dwrite d4/d2d/f8d [4194304,4194304] 0 2024-05-15T15:54:29.941 INFO:tasks.workunit.client.0.smithi049.stdout:8/831: write d2/de/f6c [283834,68568] 0 2024-05-15T15:54:29.943 INFO:tasks.workunit.client.0.smithi049.stdout:3/849: mkdir d12/d14/d32/d104 0 2024-05-15T15:54:29.944 INFO:tasks.workunit.client.0.smithi049.stdout:8/832: chown d2/de/d71/d86/d5d/d7d/db8/f85 1348 1 2024-05-15T15:54:29.947 INFO:tasks.workunit.client.1.smithi144.stdout:2/877: creat d3/d7/d20/def/d112/f122 x:0 0 0 2024-05-15T15:54:29.948 INFO:tasks.workunit.client.0.smithi049.stdout:2/748: creat d3/db/d82/ffd x:0 0 0 2024-05-15T15:54:29.950 INFO:tasks.workunit.client.1.smithi144.stdout:9/978: creat d0/d8/d1b/d62/d7d/dd2/d12e/f14d x:0 0 0 2024-05-15T15:54:29.952 INFO:tasks.workunit.client.1.smithi144.stdout:7/963: truncate d4/df/d79/fcc 170662 0 2024-05-15T15:54:29.953 INFO:tasks.workunit.client.1.smithi144.stdout:3/942: creat d0/d11/d24/d78/f12d x:0 0 0 2024-05-15T15:54:29.957 INFO:tasks.workunit.client.1.smithi144.stdout:0/912: mknod d4/d6/d3a/d40/c14e 0 2024-05-15T15:54:29.958 INFO:tasks.workunit.client.1.smithi144.stdout:0/913: stat d4/d6/d4a/d4d/db1/l12a 0 2024-05-15T15:54:29.959 INFO:tasks.workunit.client.0.smithi049.stdout:3/850: creat d12/d14/d52/f105 x:0 0 0 2024-05-15T15:54:29.960 INFO:tasks.workunit.client.1.smithi144.stdout:2/878: unlink d3/d7/d54/d98/fa2 0 2024-05-15T15:54:29.962 INFO:tasks.workunit.client.1.smithi144.stdout:9/979: stat d0/d8/d5e/d44/l23 0 2024-05-15T15:54:29.965 INFO:tasks.workunit.client.0.smithi049.stdout:2/749: mkdir d3/d8/dad/dfe 0 2024-05-15T15:54:29.966 INFO:tasks.workunit.client.1.smithi144.stdout:7/964: mkdir d4/df/d81/dfb/d108/d14f 0 2024-05-15T15:54:29.966 INFO:tasks.workunit.client.1.smithi144.stdout:7/965: chown d4/df/d127/d138 3762 1 2024-05-15T15:54:29.967 INFO:tasks.workunit.client.0.smithi049.stdout:4/799: write d18/d21/f26 [2922479,92094] 0 2024-05-15T15:54:29.967 INFO:tasks.workunit.client.0.smithi049.stdout:7/819: write d0/d6/fcd [97366,21806] 0 2024-05-15T15:54:29.968 INFO:tasks.workunit.client.1.smithi144.stdout:3/943: mknod d0/d1/d17/dce/c12e 0 2024-05-15T15:54:29.969 INFO:tasks.workunit.client.1.smithi144.stdout:1/968: truncate d7/d1d/d33/d1e/d34/f114 2090572 0 2024-05-15T15:54:29.971 INFO:tasks.workunit.client.1.smithi144.stdout:1/969: dread d7/d1d/d33/d1e/f77 [4194304,4194304] 0 2024-05-15T15:54:29.974 INFO:tasks.workunit.client.1.smithi144.stdout:2/879: dread d3/d7/d20/d89/d86/ff9 [0,4194304] 0 2024-05-15T15:54:29.975 INFO:tasks.workunit.client.0.smithi049.stdout:9/740: mkdir d0/d8/d21/ded/d107 0 2024-05-15T15:54:29.975 INFO:tasks.workunit.client.1.smithi144.stdout:2/880: readlink d3/d7/lc 0 2024-05-15T15:54:29.975 INFO:tasks.workunit.client.1.smithi144.stdout:2/881: chown d3/d7/d20/d22/fb9 3 1 2024-05-15T15:54:29.976 INFO:tasks.workunit.client.0.smithi049.stdout:6/927: rename dd/d17/d24/c47 to dd/d26/d5b/d110/d50/d54/d92/c11e 0 2024-05-15T15:54:29.977 INFO:tasks.workunit.client.0.smithi049.stdout:6/928: chown dd/d26/d5b/d110/d86/f117 8133184 1 2024-05-15T15:54:29.978 INFO:tasks.workunit.client.0.smithi049.stdout:6/929: dread - dd/d26/d5b/d110/d50/d7e/f10c zero size 2024-05-15T15:54:29.979 INFO:tasks.workunit.client.0.smithi049.stdout:8/833: truncate d2/de/d25/d74/dbe/fe4 209552 0 2024-05-15T15:54:29.980 INFO:tasks.workunit.client.1.smithi144.stdout:5/950: dwrite d0/dc/d2f/d34/d63/d6e/db8/fda [0,4194304] 0 2024-05-15T15:54:29.982 INFO:tasks.workunit.client.1.smithi144.stdout:7/966: symlink d4/d41/d5b/l150 0 2024-05-15T15:54:29.982 INFO:tasks.workunit.client.0.smithi049.stdout:2/750: fsync d3/db/d1d/f5e 0 2024-05-15T15:54:29.985 INFO:tasks.workunit.client.1.smithi144.stdout:3/944: unlink d0/f46 0 2024-05-15T15:54:29.986 INFO:tasks.workunit.client.0.smithi049.stdout:0/862: dwrite dc/d13/d2b/d76/fc8 [0,4194304] 0 2024-05-15T15:54:29.986 INFO:tasks.workunit.client.0.smithi049.stdout:7/820: symlink d0/d6/d23/d76/l101 0 2024-05-15T15:54:29.987 INFO:tasks.workunit.client.0.smithi049.stdout:7/821: stat d0/d6/d1d 0 2024-05-15T15:54:29.988 INFO:tasks.workunit.client.1.smithi144.stdout:1/970: readlink d7/d1d/d33/d55/l123 0 2024-05-15T15:54:29.990 INFO:tasks.workunit.client.0.smithi049.stdout:7/822: stat d0/d6/d23/d45/d6a/da1/d80/cf0 0 2024-05-15T15:54:29.995 INFO:tasks.workunit.client.1.smithi144.stdout:9/980: dread d0/d8/d5e/d44/d16/f85 [0,4194304] 0 2024-05-15T15:54:29.996 INFO:tasks.workunit.client.0.smithi049.stdout:7/823: dwrite d0/d6/d23/faf [0,4194304] 0 2024-05-15T15:54:30.003 INFO:tasks.workunit.client.1.smithi144.stdout:7/967: mkdir d4/d3c/d76/d9d/dac/d48/d151 0 2024-05-15T15:54:30.010 INFO:tasks.workunit.client.1.smithi144.stdout:1/971: creat d7/d1d/f12d x:0 0 0 2024-05-15T15:54:30.013 INFO:tasks.workunit.client.1.smithi144.stdout:0/914: creat d4/d2d/d48/d5b/f14f x:0 0 0 2024-05-15T15:54:30.018 INFO:tasks.workunit.client.1.smithi144.stdout:0/915: dwrite d4/d6/d3a/d4c/f9a [0,4194304] 0 2024-05-15T15:54:30.019 INFO:tasks.workunit.client.1.smithi144.stdout:0/916: fdatasync d4/d6/d4a/f123 0 2024-05-15T15:54:30.021 INFO:tasks.workunit.client.1.smithi144.stdout:7/968: truncate d4/d41/f49 2212522 0 2024-05-15T15:54:30.022 INFO:tasks.workunit.client.1.smithi144.stdout:7/969: chown d4/df/d25/d38/f54 1 1 2024-05-15T15:54:30.022 INFO:tasks.workunit.client.1.smithi144.stdout:3/945: write d0/f2e [2353559,59218] 0 2024-05-15T15:54:30.026 INFO:tasks.workunit.client.1.smithi144.stdout:1/972: creat d7/d1d/d33/d41/d22/db4/f12e x:0 0 0 2024-05-15T15:54:30.032 INFO:tasks.workunit.client.1.smithi144.stdout:9/981: write d0/d8/d5e/d44/d10/d9c/f3f [2039572,8393] 0 2024-05-15T15:54:30.037 INFO:tasks.workunit.client.1.smithi144.stdout:0/917: mknod d4/df/d1f/d24/d85/c150 0 2024-05-15T15:54:30.041 INFO:tasks.workunit.client.1.smithi144.stdout:7/970: truncate d4/df/d25/d38/d68/fe0 114522 0 2024-05-15T15:54:30.045 INFO:tasks.workunit.client.1.smithi144.stdout:1/973: rename d7/d1d/d33/d29/dd2/dea/d113 to d7/d1d/d33/d29/dd2/dea/d129/d12f 0 2024-05-15T15:54:30.047 INFO:tasks.workunit.client.1.smithi144.stdout:1/974: readlink d7/d1d/dac/l11d 0 2024-05-15T15:54:30.047 INFO:tasks.workunit.client.1.smithi144.stdout:2/882: getdents d3/d7/d20/d22/d52/d64 0 2024-05-15T15:54:30.048 INFO:tasks.workunit.client.1.smithi144.stdout:9/982: truncate d0/f7 2924884 0 2024-05-15T15:54:30.048 INFO:tasks.workunit.client.1.smithi144.stdout:2/883: chown d3/d7/d20/d37/d76/f11b 496 1 2024-05-15T15:54:30.053 INFO:tasks.workunit.client.1.smithi144.stdout:9/983: dwrite d0/d8/d1b/d62/d7d/d8d/dbb/dce/fdd [0,4194304] 0 2024-05-15T15:54:30.054 INFO:tasks.workunit.client.1.smithi144.stdout:7/971: fsync d4/d3c/d76/d9d/dac/f6e 0 2024-05-15T15:54:30.061 INFO:tasks.workunit.client.1.smithi144.stdout:1/975: creat d7/d1d/f130 x:0 0 0 2024-05-15T15:54:30.063 INFO:tasks.workunit.client.1.smithi144.stdout:2/884: creat d3/d7/d20/def/d112/f123 x:0 0 0 2024-05-15T15:54:30.068 INFO:tasks.workunit.client.1.smithi144.stdout:5/951: getdents d0/dc/d2f/d34/d3e/dd0 0 2024-05-15T15:54:30.069 INFO:tasks.workunit.client.1.smithi144.stdout:5/952: chown d0/dc/d24/d8e/d93/d6b/d88/f8d 2485 1 2024-05-15T15:54:30.072 INFO:tasks.workunit.client.1.smithi144.stdout:3/946: creat d0/d1/d56/d71/f12f x:0 0 0 2024-05-15T15:54:30.077 INFO:tasks.workunit.client.1.smithi144.stdout:7/972: readlink d4/d3c/d76/d9d/dac/de5/l12e 0 2024-05-15T15:54:30.080 INFO:tasks.workunit.client.1.smithi144.stdout:1/976: symlink d7/d1d/d58/d81/d105/l131 0 2024-05-15T15:54:30.083 INFO:tasks.workunit.client.1.smithi144.stdout:5/953: mkdir d0/dc/d24/d8e/d95/d143 0 2024-05-15T15:54:30.085 INFO:tasks.workunit.client.0.smithi049.stdout:3/851: mkdir d12/d14/d32/d106 0 2024-05-15T15:54:30.086 INFO:tasks.workunit.client.1.smithi144.stdout:3/947: mknod d0/d1/d56/d71/d125/c130 0 2024-05-15T15:54:30.090 INFO:tasks.workunit.client.0.smithi049.stdout:4/800: rename d18/d85 to d18/d1e/d2a/d50/d54/d109 0 2024-05-15T15:54:30.092 INFO:tasks.workunit.client.0.smithi049.stdout:6/930: mknod dd/d26/c11f 0 2024-05-15T15:54:30.093 INFO:tasks.workunit.client.1.smithi144.stdout:2/885: dread d3/d7/d20/d3f/f67 [0,4194304] 0 2024-05-15T15:54:30.093 INFO:tasks.workunit.client.1.smithi144.stdout:2/886: chown d3/d7/d20/d89/d6e 400709 1 2024-05-15T15:54:30.094 INFO:tasks.workunit.client.1.smithi144.stdout:9/984: link d0/d8/d1b/d79/fc7 d0/d8/d5e/d44/d99/d120/f14e 0 2024-05-15T15:54:30.097 INFO:tasks.workunit.client.0.smithi049.stdout:8/834: creat d2/de/d25/d74/d94/f104 x:0 0 0 2024-05-15T15:54:30.099 INFO:tasks.workunit.client.1.smithi144.stdout:7/973: symlink d4/d41/d9c/d106/l152 0 2024-05-15T15:54:30.101 INFO:tasks.workunit.client.0.smithi049.stdout:2/751: symlink d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/dc9/dea/lff 0 2024-05-15T15:54:30.102 INFO:tasks.workunit.client.0.smithi049.stdout:2/752: chown d3/db/d1d/d1e/fc8 6 1 2024-05-15T15:54:30.102 INFO:tasks.workunit.client.1.smithi144.stdout:9/985: dread d0/d8/f11 [4194304,4194304] 0 2024-05-15T15:54:30.102 INFO:tasks.workunit.client.1.smithi144.stdout:5/954: creat d0/dc/d2f/d79/f144 x:0 0 0 2024-05-15T15:54:30.104 INFO:tasks.workunit.client.1.smithi144.stdout:0/918: getdents d4/d6/d3a/d40 0 2024-05-15T15:54:30.104 INFO:tasks.workunit.client.1.smithi144.stdout:0/919: readlink d4/d6/d3a/d4c/l57 0 2024-05-15T15:54:30.106 INFO:tasks.workunit.client.0.smithi049.stdout:7/824: mkdir d0/d6/d23/d45/d6a/d102 0 2024-05-15T15:54:30.107 INFO:tasks.workunit.client.0.smithi049.stdout:7/825: write d0/ddb/dde/dca/fe6 [864148,51052] 0 2024-05-15T15:54:30.108 INFO:tasks.workunit.client.1.smithi144.stdout:2/887: mkdir d3/d4d/d103/d124 0 2024-05-15T15:54:30.111 INFO:tasks.workunit.client.0.smithi049.stdout:3/852: dread - d12/d14/d52/db9/da6/fad zero size 2024-05-15T15:54:30.113 INFO:tasks.workunit.client.0.smithi049.stdout:3/853: write d12/d14/f43 [4928725,123583] 0 2024-05-15T15:54:30.116 INFO:tasks.workunit.client.0.smithi049.stdout:9/741: creat d0/d8/d9/d52/d5e/d5f/d56/f108 x:0 0 0 2024-05-15T15:54:30.117 INFO:tasks.workunit.client.1.smithi144.stdout:9/986: symlink d0/d8/d5e/l14f 0 2024-05-15T15:54:30.117 INFO:tasks.workunit.client.0.smithi049.stdout:7/826: dwrite d0/d6/d23/d76/ff4 [0,4194304] 0 2024-05-15T15:54:30.118 INFO:tasks.workunit.client.1.smithi144.stdout:9/987: read - d0/d8/d1b/d62/d7d/dd2/d12e/f14d zero size 2024-05-15T15:54:30.118 INFO:tasks.workunit.client.1.smithi144.stdout:9/988: fdatasync d0/d8/d1b/d62/fca 0 2024-05-15T15:54:30.119 INFO:tasks.workunit.client.0.smithi049.stdout:4/801: dread - d18/d1e/d63/fcd zero size 2024-05-15T15:54:30.121 INFO:tasks.workunit.client.0.smithi049.stdout:6/931: rename dd/d26/d5b/d110/d50/d54/d92/de5 to dd/d26/d5b/d110/d50/dcf/d11c/d120 0 2024-05-15T15:54:30.123 INFO:tasks.workunit.client.1.smithi144.stdout:0/920: mkdir d4/d6/d3a/d4c/d151 0 2024-05-15T15:54:30.123 INFO:tasks.workunit.client.0.smithi049.stdout:6/932: chown dd/d17/d24/d2b/ld1 4296 1 2024-05-15T15:54:30.128 INFO:tasks.workunit.client.1.smithi144.stdout:2/888: creat d3/d7/da4/f125 x:0 0 0 2024-05-15T15:54:30.133 INFO:tasks.workunit.client.1.smithi144.stdout:1/977: creat d7/f132 x:0 0 0 2024-05-15T15:54:30.134 INFO:tasks.workunit.client.0.smithi049.stdout:0/863: mkdir dc/da4/db4/d123 0 2024-05-15T15:54:30.135 INFO:tasks.workunit.client.0.smithi049.stdout:0/864: fsync dc/d32/d93/f89 0 2024-05-15T15:54:30.136 INFO:tasks.workunit.client.1.smithi144.stdout:5/955: dwrite d0/dc/d2f/d34/d3e/fed [0,4194304] 0 2024-05-15T15:54:30.137 INFO:tasks.workunit.client.1.smithi144.stdout:5/956: dread d0/dc/d1f/f75 [0,4194304] 0 2024-05-15T15:54:30.141 INFO:tasks.workunit.client.0.smithi049.stdout:3/854: symlink d12/d14/d52/db9/dc4/d68/d70/dda/l107 0 2024-05-15T15:54:30.143 INFO:tasks.workunit.client.0.smithi049.stdout:2/753: write d3/db/d1d/d1e/d2b/d3b/d42/f69 [592567,861] 0 2024-05-15T15:54:30.147 INFO:tasks.workunit.client.0.smithi049.stdout:7/827: mkdir d0/ddb/dde/dcc/d103 0 2024-05-15T15:54:30.150 INFO:tasks.workunit.client.0.smithi049.stdout:7/828: chown d0/d2f/lc9 179 1 2024-05-15T15:54:30.150 INFO:tasks.workunit.client.1.smithi144.stdout:3/948: creat d0/d1/d56/d98/f131 x:0 0 0 2024-05-15T15:54:30.150 INFO:tasks.workunit.client.1.smithi144.stdout:9/989: write d0/d8/d5e/d44/dc2/f42 [1429173,5716] 0 2024-05-15T15:54:30.150 INFO:tasks.workunit.client.0.smithi049.stdout:3/855: dwrite d12/f29 [0,4194304] 0 2024-05-15T15:54:30.151 INFO:tasks.workunit.client.1.smithi144.stdout:9/990: fdatasync d0/d8/d1b/d79/d8a/dae/f12c 0 2024-05-15T15:54:30.152 INFO:tasks.workunit.client.0.smithi049.stdout:3/856: write d12/d14/f43 [4129915,128703] 0 2024-05-15T15:54:30.153 INFO:tasks.workunit.client.1.smithi144.stdout:9/991: chown d0/d8/d1b/d62/d7d/dd2/f121 100336 1 2024-05-15T15:54:30.154 INFO:tasks.workunit.client.1.smithi144.stdout:7/974: rmdir d4/d3c/d76/d9d/dac/d48/d72/d82/d12d 0 2024-05-15T15:54:30.159 INFO:tasks.workunit.client.1.smithi144.stdout:1/978: rename d7/d1d/d33/d1e/d4e/cc7 to d7/d1d/dac/dad/c133 0 2024-05-15T15:54:30.161 INFO:tasks.workunit.client.1.smithi144.stdout:1/979: fsync d7/d1d/d33/d41/d22/dbd/fc0 0 2024-05-15T15:54:30.162 INFO:tasks.workunit.client.0.smithi049.stdout:4/802: write d18/d1e/d63/f7f [1376331,64060] 0 2024-05-15T15:54:30.164 INFO:tasks.workunit.client.1.smithi144.stdout:0/921: dwrite d4/d6/d3a/d4c/f114 [0,4194304] 0 2024-05-15T15:54:30.168 INFO:tasks.workunit.client.0.smithi049.stdout:0/865: creat dc/d13/d14/d2f/d8f/dcb/de2/f124 x:0 0 0 2024-05-15T15:54:30.172 INFO:tasks.workunit.client.0.smithi049.stdout:8/835: dwrite d2/fb [0,4194304] 0 2024-05-15T15:54:30.176 INFO:tasks.workunit.client.1.smithi144.stdout:5/957: dwrite d0/dc/d24/d28/d2d/d59/d74/ff2 [0,4194304] 0 2024-05-15T15:54:30.177 INFO:tasks.workunit.client.0.smithi049.stdout:0/866: dread dc/f34 [4194304,4194304] 0 2024-05-15T15:54:30.177 INFO:tasks.workunit.client.1.smithi144.stdout:3/949: rename d0/d1/fe to d0/d1/d17/dae/dd7/f132 0 2024-05-15T15:54:30.180 INFO:tasks.workunit.client.0.smithi049.stdout:2/754: symlink d3/da/d15/d9e/l100 0 2024-05-15T15:54:30.182 INFO:tasks.workunit.client.1.smithi144.stdout:1/980: mkdir d7/d1d/d33/d1e/d34/d134 0 2024-05-15T15:54:30.183 INFO:tasks.workunit.client.1.smithi144.stdout:1/981: readlink d7/d1d/d33/d29/d6b/da4/lbc 0 2024-05-15T15:54:30.189 INFO:tasks.workunit.client.0.smithi049.stdout:6/933: mkdir dd/d17/d24/d121 0 2024-05-15T15:54:30.192 INFO:tasks.workunit.client.0.smithi049.stdout:9/742: link d0/d8/d42/fd4 d0/d25/d5d/f109 0 2024-05-15T15:54:30.196 INFO:tasks.workunit.client.1.smithi144.stdout:9/992: rename d0/d8/d1b/d114/dbf/c133 to d0/d8/d1b/d62/d7d/dd2/d12e/c150 0 2024-05-15T15:54:30.199 INFO:tasks.workunit.client.1.smithi144.stdout:3/950: fsync d0/d1/d17/d39/d40/d4a/d77/fc4 0 2024-05-15T15:54:30.200 INFO:tasks.workunit.client.1.smithi144.stdout:3/951: write d0/d1/d56/d71/d125/fdd [5362758,116578] 0 2024-05-15T15:54:30.200 INFO:tasks.workunit.client.0.smithi049.stdout:2/755: creat d3/d78/d8c/d91/d7a/f101 x:0 0 0 2024-05-15T15:54:30.202 INFO:tasks.workunit.client.1.smithi144.stdout:1/982: mknod d7/d1d/d33/d29/d6b/d5e/d74/c135 0 2024-05-15T15:54:30.212 INFO:tasks.workunit.client.1.smithi144.stdout:1/983: dread d7/d1d/d33/d41/d78/fb1 [0,4194304] 0 2024-05-15T15:54:30.212 INFO:tasks.workunit.client.1.smithi144.stdout:7/975: truncate d4/d3c/d76/d9d/dac/d48/fef 3085356 0 2024-05-15T15:54:30.214 INFO:tasks.workunit.client.0.smithi049.stdout:4/803: write d18/d1e/d63/d99/de1/f57 [171913,20237] 0 2024-05-15T15:54:30.221 INFO:tasks.workunit.client.1.smithi144.stdout:3/952: rename d0/d11/f9a to d0/d1/d56/d71/d73/dd1/f133 0 2024-05-15T15:54:30.224 INFO:tasks.workunit.client.0.smithi049.stdout:3/857: write d12/d14/d52/f99 [4805959,45839] 0 2024-05-15T15:54:30.225 INFO:tasks.workunit.client.1.smithi144.stdout:0/922: creat d4/df/d1f/d24/f152 x:0 0 0 2024-05-15T15:54:30.231 INFO:tasks.workunit.client.0.smithi049.stdout:0/867: truncate dc/f34 1555570 0 2024-05-15T15:54:30.232 INFO:tasks.workunit.client.0.smithi049.stdout:6/934: dwrite dd/d17/d24/d2b/f4d [0,4194304] 0 2024-05-15T15:54:30.238 INFO:tasks.workunit.client.0.smithi049.stdout:7/829: link d0/f15 d0/d6/d23/d45/d86/f104 0 2024-05-15T15:54:30.239 INFO:tasks.workunit.client.0.smithi049.stdout:6/935: dwrite dd/d17/d9d/da1/fee [0,4194304] 0 2024-05-15T15:54:30.242 INFO:tasks.workunit.client.1.smithi144.stdout:7/976: creat d4/d3c/d76/d9d/dac/de5/f153 x:0 0 0 2024-05-15T15:54:30.243 INFO:tasks.workunit.client.1.smithi144.stdout:7/977: chown d4/d3c/d76/d9d/dac/d48/d72/da2/fbe 0 1 2024-05-15T15:54:30.244 INFO:tasks.workunit.client.1.smithi144.stdout:7/978: stat d4/d3c/d76/d9d/dac/d48/c64 0 2024-05-15T15:54:30.251 INFO:tasks.workunit.client.0.smithi049.stdout:4/804: dwrite d18/d1e/d2a/d50/d54/d109/fa3 [0,4194304] 0 2024-05-15T15:54:30.252 INFO:tasks.workunit.client.0.smithi049.stdout:4/805: chown d18/d1e/d2a/f4f 35779506 1 2024-05-15T15:54:30.258 INFO:tasks.workunit.client.0.smithi049.stdout:7/830: unlink d0/d6/d23/d76/cac 0 2024-05-15T15:54:30.263 INFO:tasks.workunit.client.0.smithi049.stdout:4/806: dwrite d18/d1e/d63/f96 [8388608,4194304] 0 2024-05-15T15:54:30.265 INFO:tasks.workunit.client.0.smithi049.stdout:2/756: rmdir d3/db/d1d/d1e/d2b/d3b/d42/de4 0 2024-05-15T15:54:30.265 INFO:tasks.workunit.client.0.smithi049.stdout:4/807: write d18/d1e/d63/f96 [7838748,101087] 0 2024-05-15T15:54:30.275 INFO:tasks.workunit.client.0.smithi049.stdout:7/831: creat d0/d6/d1d/f105 x:0 0 0 2024-05-15T15:54:30.278 INFO:tasks.workunit.client.0.smithi049.stdout:3/858: dread d12/d14/d52/f7b [0,4194304] 0 2024-05-15T15:54:30.284 INFO:tasks.workunit.client.1.smithi144.stdout:2/889: sync 2024-05-15T15:54:30.285 INFO:tasks.workunit.client.0.smithi049.stdout:7/832: dread d0/d6/d1d/f27 [0,4194304] 0 2024-05-15T15:54:30.286 INFO:tasks.workunit.client.0.smithi049.stdout:0/868: write dc/f34 [2109743,56078] 0 2024-05-15T15:54:30.289 INFO:tasks.workunit.client.0.smithi049.stdout:4/808: mknod d18/d1e/d2a/d50/d54/d109/db6/c10a 0 2024-05-15T15:54:30.297 INFO:tasks.workunit.client.0.smithi049.stdout:2/757: write d3/db/d1d/f28 [976832,99282] 0 2024-05-15T15:54:30.304 INFO:tasks.workunit.client.0.smithi049.stdout:6/936: getdents dd/d26/d5b/d110/d86 0 2024-05-15T15:54:30.307 INFO:tasks.workunit.client.0.smithi049.stdout:0/869: write dc/d13/d14/d2f/f52 [3499154,20323] 0 2024-05-15T15:54:30.309 INFO:tasks.workunit.client.0.smithi049.stdout:3/859: dwrite d12/f96 [0,4194304] 0 2024-05-15T15:54:30.311 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:30 smithi049 ceph-mon[28399]: pgmap v35: 65 pgs: 65 active+clean; 3.4 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 69 MiB/s rd, 164 MiB/s wr, 369 op/s 2024-05-15T15:54:30.313 INFO:tasks.workunit.client.0.smithi049.stdout:4/809: creat d18/d1e/d2a/d50/d54/d109/dff/f10b x:0 0 0 2024-05-15T15:54:30.316 INFO:tasks.workunit.client.0.smithi049.stdout:6/937: unlink dd/f80 0 2024-05-15T15:54:30.320 INFO:tasks.workunit.client.0.smithi049.stdout:2/758: write d3/d85/fe1 [1005885,38567] 0 2024-05-15T15:54:30.322 INFO:tasks.workunit.client.0.smithi049.stdout:7/833: creat d0/d6/f106 x:0 0 0 2024-05-15T15:54:30.325 INFO:tasks.workunit.client.0.smithi049.stdout:0/870: creat dc/d13/d3d/f125 x:0 0 0 2024-05-15T15:54:30.330 INFO:tasks.workunit.client.0.smithi049.stdout:3/860: mkdir d12/d14/d52/d108 0 2024-05-15T15:54:30.331 INFO:tasks.workunit.client.0.smithi049.stdout:3/861: read d12/d26/fe0 [509657,100507] 0 2024-05-15T15:54:30.332 INFO:tasks.workunit.client.0.smithi049.stdout:4/810: mknod d18/d1e/d2a/d36/d86/c10c 0 2024-05-15T15:54:30.335 INFO:tasks.workunit.client.0.smithi049.stdout:7/834: stat d0/d6/d23/f91 0 2024-05-15T15:54:30.338 INFO:tasks.workunit.client.0.smithi049.stdout:2/759: rename d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/fde to d3/db/d1d/d1e/d2b/d3b/d42/f102 0 2024-05-15T15:54:30.340 INFO:tasks.workunit.client.0.smithi049.stdout:0/871: mknod dc/d13/d14/d2f/d8f/dcb/de2/c126 0 2024-05-15T15:54:30.347 INFO:tasks.workunit.client.0.smithi049.stdout:2/760: symlink d3/d78/d8c/d91/d7a/l103 0 2024-05-15T15:54:30.350 INFO:tasks.workunit.client.0.smithi049.stdout:0/872: mknod dc/da4/db4/c127 0 2024-05-15T15:54:30.357 INFO:tasks.workunit.client.0.smithi049.stdout:0/873: mkdir dc/de9/d109/d128 0 2024-05-15T15:54:30.363 INFO:tasks.workunit.client.0.smithi049.stdout:0/874: write dc/d13/d2b/d76/d8a/f102 [1203979,38977] 0 2024-05-15T15:54:30.366 INFO:tasks.workunit.client.1.smithi144.stdout:1/984: rename d7/d1d/d33/d1e/d34/f38 to d7/df/d89/dcc/f136 0 2024-05-15T15:54:30.366 INFO:tasks.workunit.client.0.smithi049.stdout:0/875: write dc/d13/d14/d25/d31/dae/dc5/d108/f5b [2111258,36987] 0 2024-05-15T15:54:30.366 INFO:tasks.workunit.client.1.smithi144.stdout:1/985: chown d7/d1d/d33/d1e/d34/d9c/c112 24 1 2024-05-15T15:54:30.367 INFO:tasks.workunit.client.0.smithi049.stdout:3/862: getdents d12/d30/dc1/dc7/df8/dcf 0 2024-05-15T15:54:30.369 INFO:tasks.workunit.client.0.smithi049.stdout:6/938: truncate dd/d26/d5b/d110/f98 3998692 0 2024-05-15T15:54:30.370 INFO:tasks.workunit.client.1.smithi144.stdout:3/953: mkdir d0/d1/d56/d71/d125/d134 0 2024-05-15T15:54:30.370 INFO:tasks.workunit.client.1.smithi144.stdout:3/954: dread - d0/d1/d56/d98/f131 zero size 2024-05-15T15:54:30.371 INFO:tasks.workunit.client.1.smithi144.stdout:3/955: readlink d0/d1/d17/d39/d40/d7e/lba 0 2024-05-15T15:54:30.376 INFO:tasks.workunit.client.1.smithi144.stdout:5/958: link d0/d10c/d5d/l97 d0/dc/d2f/d34/d63/d6e/l145 0 2024-05-15T15:54:30.376 INFO:tasks.workunit.client.0.smithi049.stdout:0/876: dwrite dc/d13/f17 [0,4194304] 0 2024-05-15T15:54:30.377 INFO:tasks.workunit.client.1.smithi144.stdout:3/956: dread d0/d11/d24/f74 [0,4194304] 0 2024-05-15T15:54:30.378 INFO:tasks.workunit.client.1.smithi144.stdout:5/959: write d0/dc/d24/d8e/d93/d6b/dea/ffd [228337,84986] 0 2024-05-15T15:54:30.379 INFO:tasks.workunit.client.0.smithi049.stdout:3/863: dwrite d12/d14/d52/fce [0,4194304] 0 2024-05-15T15:54:30.385 INFO:tasks.workunit.client.0.smithi049.stdout:7/835: dwrite d0/d6/d23/d45/d83/fce [0,4194304] 0 2024-05-15T15:54:30.389 INFO:tasks.workunit.client.0.smithi049.stdout:7/836: readlink d0/d6/d23/d45/d6a/da1/d80/l81 0 2024-05-15T15:54:30.392 INFO:tasks.workunit.client.0.smithi049.stdout:2/761: link d3/db/d5b/dbf/dc1/f8b d3/db/d1d/d1e/d2b/f104 0 2024-05-15T15:54:30.393 INFO:tasks.workunit.client.1.smithi144.stdout:2/890: mkdir d3/d7/d20/d40/d126 0 2024-05-15T15:54:30.401 INFO:tasks.workunit.client.0.smithi049.stdout:6/939: unlink dd/d17/d24/d2b/ca5 0 2024-05-15T15:54:30.404 INFO:tasks.workunit.client.0.smithi049.stdout:3/864: mknod d12/d67/c109 0 2024-05-15T15:54:30.406 INFO:tasks.workunit.client.0.smithi049.stdout:0/877: symlink dc/d13/d14/d2f/df5/l129 0 2024-05-15T15:54:30.408 INFO:tasks.workunit.client.0.smithi049.stdout:0/878: fdatasync dc/f34 0 2024-05-15T15:54:30.408 INFO:tasks.workunit.client.1.smithi144.stdout:9/993: getdents d0/d8/d5e/d44/d16 0 2024-05-15T15:54:30.408 INFO:tasks.workunit.client.1.smithi144.stdout:0/923: dwrite d4/df/f10b [0,4194304] 0 2024-05-15T15:54:30.410 INFO:tasks.workunit.client.1.smithi144.stdout:9/994: write d0/d8/d1b/d79/d8a/dae/feb [4838607,120974] 0 2024-05-15T15:54:30.411 INFO:tasks.workunit.client.0.smithi049.stdout:6/940: dwrite dd/d26/f40 [0,4194304] 0 2024-05-15T15:54:30.415 INFO:tasks.workunit.client.0.smithi049.stdout:0/879: dread dc/d13/d56/d6f/fd2 [0,4194304] 0 2024-05-15T15:54:30.417 INFO:tasks.workunit.client.1.smithi144.stdout:5/960: unlink d0/dc/d2f/d34/d63/fec 0 2024-05-15T15:54:30.418 INFO:tasks.workunit.client.1.smithi144.stdout:5/961: chown d0/dc/d2f/f122 74810468 1 2024-05-15T15:54:30.423 INFO:tasks.workunit.client.1.smithi144.stdout:1/986: symlink d7/d1d/d33/d41/d78/l137 0 2024-05-15T15:54:30.425 INFO:tasks.workunit.client.0.smithi049.stdout:2/762: fsync d3/db/d5b/f74 0 2024-05-15T15:54:30.430 INFO:tasks.workunit.client.0.smithi049.stdout:0/880: creat dc/d6d/f12a x:0 0 0 2024-05-15T15:54:30.431 INFO:tasks.workunit.client.1.smithi144.stdout:5/962: dread d0/dc/d12/f18 [4194304,4194304] 0 2024-05-15T15:54:30.432 INFO:tasks.workunit.client.1.smithi144.stdout:3/957: unlink d0/d1/d56/d71/d73/dd1/f133 0 2024-05-15T15:54:30.432 INFO:tasks.workunit.client.0.smithi049.stdout:6/941: dread dd/d17/d24/f74 [0,4194304] 0 2024-05-15T15:54:30.432 INFO:tasks.workunit.client.0.smithi049.stdout:0/881: write dc/d13/d14/d2f/f52 [1621969,9983] 0 2024-05-15T15:54:30.433 INFO:tasks.workunit.client.1.smithi144.stdout:5/963: chown d0/dc/d24/fd2 684610162 1 2024-05-15T15:54:30.434 INFO:tasks.workunit.client.0.smithi049.stdout:0/882: chown dc/d13/d3d 2477427 1 2024-05-15T15:54:30.443 INFO:tasks.workunit.client.1.smithi144.stdout:3/958: rmdir d0/d1/d17/dae 39 2024-05-15T15:54:30.444 INFO:tasks.workunit.client.0.smithi049.stdout:7/837: dwrite d0/d2f/f48 [0,4194304] 0 2024-05-15T15:54:30.444 INFO:tasks.workunit.client.1.smithi144.stdout:7/979: truncate d4/df/d25/f35 1152746 0 2024-05-15T15:54:30.446 INFO:tasks.workunit.client.0.smithi049.stdout:7/838: write d0/d2f/f3e [4505594,116848] 0 2024-05-15T15:54:30.448 INFO:tasks.workunit.client.1.smithi144.stdout:5/964: rename d0/dc/d24/d28 to d0/dc/d24/d8e/d95/d146 0 2024-05-15T15:54:30.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:30 smithi144 ceph-mon[36389]: pgmap v35: 65 pgs: 65 active+clean; 3.4 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 69 MiB/s rd, 164 MiB/s wr, 369 op/s 2024-05-15T15:54:30.454 INFO:tasks.workunit.client.0.smithi049.stdout:2/763: creat d3/db/d1d/d1e/d2b/d3b/d47/d9a/f105 x:0 0 0 2024-05-15T15:54:30.456 INFO:tasks.workunit.client.1.smithi144.stdout:9/995: mknod d0/d8/d5e/d44/d10/d9c/c151 0 2024-05-15T15:54:30.457 INFO:tasks.workunit.client.1.smithi144.stdout:1/987: dread d7/d1d/d33/d41/f19 [0,4194304] 0 2024-05-15T15:54:30.458 INFO:tasks.workunit.client.0.smithi049.stdout:6/942: creat dd/d26/f122 x:0 0 0 2024-05-15T15:54:30.462 INFO:tasks.workunit.client.1.smithi144.stdout:9/996: dwrite d0/fd8 [4194304,4194304] 0 2024-05-15T15:54:30.463 INFO:tasks.workunit.client.1.smithi144.stdout:9/997: dread - d0/d8/d5e/d44/dc2/f12f zero size 2024-05-15T15:54:30.464 INFO:tasks.workunit.client.1.smithi144.stdout:5/965: rename d0/f49 to d0/dc/d24/d8e/d93/d6b/dea/dfa/f147 0 2024-05-15T15:54:30.466 INFO:tasks.workunit.client.1.smithi144.stdout:0/924: getdents d4/df/d1f/d24/d85 0 2024-05-15T15:54:30.471 INFO:tasks.workunit.client.1.smithi144.stdout:0/925: dwrite d4/d2d/d100/f11d [0,4194304] 0 2024-05-15T15:54:30.477 INFO:tasks.workunit.client.1.smithi144.stdout:3/959: write d0/d11/d3b/da7/f123 [1584763,110823] 0 2024-05-15T15:54:30.478 INFO:tasks.workunit.client.1.smithi144.stdout:3/960: dread - d0/d1/d56/d98/f131 zero size 2024-05-15T15:54:30.479 INFO:tasks.workunit.client.0.smithi049.stdout:6/943: symlink dd/d17/d24/d67/l123 0 2024-05-15T15:54:30.479 INFO:tasks.workunit.client.1.smithi144.stdout:3/961: chown d0/d11/d3b/da7/def/f10e 1273674 1 2024-05-15T15:54:30.479 INFO:tasks.workunit.client.0.smithi049.stdout:3/865: truncate d12/f41 4142771 0 2024-05-15T15:54:30.481 INFO:tasks.workunit.client.1.smithi144.stdout:7/980: mkdir d4/d3c/d76/d9d/dac/d48/d151/d154 0 2024-05-15T15:54:30.481 INFO:tasks.workunit.client.0.smithi049.stdout:6/944: fdatasync dd/d26/d5b/d110/d50/d54/dde/df2/f11a 0 2024-05-15T15:54:30.482 INFO:tasks.workunit.client.0.smithi049.stdout:3/866: chown d12/d14/d52/db9/da6/dc2/ce5 1839986777 1 2024-05-15T15:54:30.483 INFO:tasks.workunit.client.0.smithi049.stdout:0/883: fdatasync dc/d13/d14/d25/d31/d71/fc1 0 2024-05-15T15:54:30.485 INFO:tasks.workunit.client.0.smithi049.stdout:3/867: truncate d12/d14/d32/f58 1155168 0 2024-05-15T15:54:30.486 INFO:tasks.workunit.client.1.smithi144.stdout:9/998: creat d0/d8/d5e/d44/d10/d19/f152 x:0 0 0 2024-05-15T15:54:30.490 INFO:tasks.workunit.client.1.smithi144.stdout:5/966: fsync d0/dc/d24/d8e/d95/fb0 0 2024-05-15T15:54:30.490 INFO:tasks.workunit.client.1.smithi144.stdout:5/967: chown d0/d10c/d73 73 1 2024-05-15T15:54:30.490 INFO:tasks.workunit.client.1.smithi144.stdout:2/891: sync 2024-05-15T15:54:30.491 INFO:tasks.workunit.client.0.smithi049.stdout:2/764: mkdir d3/d8/dad/dfe/d106 0 2024-05-15T15:54:30.498 INFO:tasks.workunit.client.1.smithi144.stdout:1/988: truncate d7/d1d/d33/d41/d22/f70 510868 0 2024-05-15T15:54:30.499 INFO:tasks.workunit.client.0.smithi049.stdout:0/884: mkdir dc/d13/d14/d25/d31/dae/dc5/d12b 0 2024-05-15T15:54:30.499 INFO:tasks.workunit.client.1.smithi144.stdout:3/962: read d0/d11/f61 [418151,13204] 0 2024-05-15T15:54:30.501 INFO:tasks.workunit.client.0.smithi049.stdout:3/868: mknod d12/d14/d32/d64/d88/c10a 0 2024-05-15T15:54:30.503 INFO:tasks.workunit.client.0.smithi049.stdout:3/869: write d12/d26/f3b [284413,47598] 0 2024-05-15T15:54:30.504 INFO:tasks.workunit.client.1.smithi144.stdout:7/981: rename d4/d41/cae to d4/df/d25/d143/c155 0 2024-05-15T15:54:30.506 INFO:tasks.workunit.client.0.smithi049.stdout:7/839: dwrite d0/f4 [8388608,4194304] 0 2024-05-15T15:54:30.511 INFO:tasks.workunit.client.0.smithi049.stdout:6/945: mkdir dd/d17/d24/d58/d124 0 2024-05-15T15:54:30.512 INFO:tasks.workunit.client.0.smithi049.stdout:0/885: fsync dc/d13/d56/d6f/fd2 0 2024-05-15T15:54:30.513 INFO:tasks.workunit.client.0.smithi049.stdout:6/946: read dd/d26/f2e [159259,33358] 0 2024-05-15T15:54:30.514 INFO:tasks.workunit.client.0.smithi049.stdout:0/886: chown dc/d13/d14/d25/d31/dae/fd4 1416867 1 2024-05-15T15:54:30.521 INFO:tasks.workunit.client.1.smithi144.stdout:0/926: mknod d4/df/d1f/d24/d85/da1/c153 0 2024-05-15T15:54:30.524 INFO:tasks.workunit.client.1.smithi144.stdout:1/989: chown d7/d1d/d58/l7a 3829 1 2024-05-15T15:54:30.528 INFO:tasks.workunit.client.1.smithi144.stdout:3/963: creat d0/d1/d17/d39/d40/d4a/d9e/f135 x:0 0 0 2024-05-15T15:54:30.528 INFO:tasks.workunit.client.1.smithi144.stdout:2/892: write d3/d7/d20/fb0 [1200243,70075] 0 2024-05-15T15:54:30.528 INFO:tasks.workunit.client.1.smithi144.stdout:3/964: chown d0/d1/d56/d71/cff 2034548 1 2024-05-15T15:54:30.531 INFO:tasks.workunit.client.0.smithi049.stdout:6/947: creat dd/d17/f125 x:0 0 0 2024-05-15T15:54:30.531 INFO:tasks.workunit.client.1.smithi144.stdout:7/982: fdatasync d4/f21 0 2024-05-15T15:54:30.532 INFO:tasks.workunit.client.0.smithi049.stdout:2/765: dwrite d3/db/d1d/d1e/d2b/fd6 [0,4194304] 0 2024-05-15T15:54:30.534 INFO:tasks.workunit.client.0.smithi049.stdout:2/766: chown d3/da/fb4 103878 1 2024-05-15T15:54:30.537 INFO:tasks.workunit.client.0.smithi049.stdout:0/887: unlink dc/d13/d2b/d76/d8a/da2/fff 0 2024-05-15T15:54:30.540 INFO:tasks.workunit.client.0.smithi049.stdout:3/870: dwrite d12/d30/dc1/dc7/df8/fbb [0,4194304] 0 2024-05-15T15:54:30.541 INFO:tasks.workunit.client.0.smithi049.stdout:7/840: dwrite d0/d6/d23/fda [0,4194304] 0 2024-05-15T15:54:30.545 INFO:tasks.workunit.client.1.smithi144.stdout:9/999: dwrite d0/d8/d5e/d44/d10/d19/dd5/d12b/f13b [4194304,4194304] 0 2024-05-15T15:54:30.549 INFO:tasks.workunit.client.1.smithi144.stdout:0/927: mkdir d4/d2d/d48/d5b/dd3/d154 0 2024-05-15T15:54:30.550 INFO:tasks.workunit.client.0.smithi049.stdout:0/888: dread dc/d13/d14/d25/d31/d71/fc7 [0,4194304] 0 2024-05-15T15:54:30.556 INFO:tasks.workunit.client.1.smithi144.stdout:2/893: creat d3/d7/df8/da6/dc1/f127 x:0 0 0 2024-05-15T15:54:30.559 INFO:tasks.workunit.client.1.smithi144.stdout:7/983: mkdir d4/d3c/d76/d9d/dac/d48/d72/da2/d156 0 2024-05-15T15:54:30.561 INFO:tasks.workunit.client.0.smithi049.stdout:0/889: creat dc/d13/d14/d2f/d8f/dcb/f12c x:0 0 0 2024-05-15T15:54:30.565 INFO:tasks.workunit.client.0.smithi049.stdout:7/841: rename d0/d2f/d5c/c7e to d0/d6/d23/d45/d6a/da1/d82/dd3/c107 0 2024-05-15T15:54:30.566 INFO:tasks.workunit.client.1.smithi144.stdout:5/968: dwrite d0/f22 [0,4194304] 0 2024-05-15T15:54:30.568 INFO:tasks.workunit.client.1.smithi144.stdout:1/990: mknod d7/d4b/c138 0 2024-05-15T15:54:30.570 INFO:tasks.workunit.client.0.smithi049.stdout:0/890: dwrite dc/d13/d14/d25/d31/dae/dc5/d57/f65 [4194304,4194304] 0 2024-05-15T15:54:30.572 INFO:tasks.workunit.client.1.smithi144.stdout:3/965: creat d0/d11/d3b/dc7/f136 x:0 0 0 2024-05-15T15:54:30.574 INFO:tasks.workunit.client.1.smithi144.stdout:2/894: creat d3/d7/df8/f128 x:0 0 0 2024-05-15T15:54:30.574 INFO:tasks.workunit.client.1.smithi144.stdout:0/928: dread d4/d6/d4a/d74/fc3 [0,4194304] 0 2024-05-15T15:54:30.576 INFO:tasks.workunit.client.1.smithi144.stdout:7/984: mkdir d4/d3c/d76/d9d/dac/d111/df0/d157 0 2024-05-15T15:54:30.578 INFO:tasks.workunit.client.0.smithi049.stdout:7/842: getdents d0/d6/d1d/df2 0 2024-05-15T15:54:30.580 INFO:tasks.workunit.client.1.smithi144.stdout:1/991: creat d7/df/d89/f139 x:0 0 0 2024-05-15T15:54:30.580 INFO:tasks.workunit.client.0.smithi049.stdout:2/767: write d3/d85/fca [429080,44391] 0 2024-05-15T15:54:30.581 INFO:tasks.workunit.client.0.smithi049.stdout:0/891: read dc/da4/fab [2856019,102545] 0 2024-05-15T15:54:30.581 INFO:tasks.workunit.client.1.smithi144.stdout:3/966: mknod d0/d1/d56/d98/d9c/c137 0 2024-05-15T15:54:30.581 INFO:tasks.workunit.client.1.smithi144.stdout:1/992: chown d7/d1d/d33/d29/d6b/d8e/l10d 9490 1 2024-05-15T15:54:30.584 INFO:tasks.workunit.client.0.smithi049.stdout:7/843: write d0/d6/d23/d4a/d7b/fed [439973,70960] 0 2024-05-15T15:54:30.585 INFO:tasks.workunit.client.1.smithi144.stdout:7/985: truncate d4/d3c/d76/d9d/dac/d48/d72/d82/fbf 686791 0 2024-05-15T15:54:30.586 INFO:tasks.workunit.client.0.smithi049.stdout:7/844: chown d0/d6/d23/d45/db1 2805071 1 2024-05-15T15:54:30.588 INFO:tasks.workunit.client.0.smithi049.stdout:0/892: read - dc/d13/d3d/d84/ffc zero size 2024-05-15T15:54:30.588 INFO:tasks.workunit.client.0.smithi049.stdout:6/948: dwrite dd/d26/d5b/d110/d50/d54/dde/f6a [0,4194304] 0 2024-05-15T15:54:30.589 INFO:tasks.workunit.client.0.smithi049.stdout:7/845: chown d0/d2f/c9c 92295690 1 2024-05-15T15:54:30.592 INFO:tasks.workunit.client.0.smithi049.stdout:2/768: dread - d3/db/d1d/d1e/d2b/d3b/d42/f102 zero size 2024-05-15T15:54:30.596 INFO:tasks.workunit.client.1.smithi144.stdout:1/993: unlink d7/d1d/d33/d1e/l3e 0 2024-05-15T15:54:30.597 INFO:tasks.workunit.client.1.smithi144.stdout:3/967: getdents d0/d1/d56/d98/de2 0 2024-05-15T15:54:30.599 INFO:tasks.workunit.client.1.smithi144.stdout:7/986: creat d4/d3c/d76/d9d/dac/d111/f158 x:0 0 0 2024-05-15T15:54:30.600 INFO:tasks.workunit.client.0.smithi049.stdout:3/871: truncate d12/d14/d52/db9/dc4/d68/fa0 1567205 0 2024-05-15T15:54:30.601 INFO:tasks.workunit.client.1.smithi144.stdout:5/969: getdents d0/dc/d1f/d65 0 2024-05-15T15:54:30.601 INFO:tasks.workunit.client.1.smithi144.stdout:5/970: fsync d0/dc/d2f/f122 0 2024-05-15T15:54:30.604 INFO:tasks.workunit.client.0.smithi049.stdout:6/949: symlink dd/d26/d5b/d110/d50/d54/dde/df2/d103/l126 0 2024-05-15T15:54:30.605 INFO:tasks.workunit.client.0.smithi049.stdout:3/872: dread f7 [0,4194304] 0 2024-05-15T15:54:30.606 INFO:tasks.workunit.client.1.smithi144.stdout:1/994: rename d7/d1d/d33/d41/d78 to d7/d1d/d33/d55/d13a 0 2024-05-15T15:54:30.608 INFO:tasks.workunit.client.0.smithi049.stdout:2/769: mkdir d3/da/d15/d24/d107 0 2024-05-15T15:54:30.611 INFO:tasks.workunit.client.0.smithi049.stdout:7/846: mknod d0/d6/d23/d4a/d7b/dc4/de0/c108 0 2024-05-15T15:54:30.613 INFO:tasks.workunit.client.1.smithi144.stdout:0/929: write d4/df/d14/f42 [1804544,86930] 0 2024-05-15T15:54:30.614 INFO:tasks.workunit.client.1.smithi144.stdout:5/971: mknod d0/dc/d24/d8e/d93/d6b/d12c/c148 0 2024-05-15T15:54:30.617 INFO:tasks.workunit.client.1.smithi144.stdout:0/930: dwrite d4/d6/f32 [0,4194304] 0 2024-05-15T15:54:30.619 INFO:tasks.workunit.client.1.smithi144.stdout:1/995: rename d7/d1d/d33/d29/dd2/f73 to d7/d1d/d33/d55/d13a/da3/f13b 0 2024-05-15T15:54:30.620 INFO:tasks.workunit.client.0.smithi049.stdout:6/950: chown dd/d17/d24/d58/d5f/dd7/fd4 3 1 2024-05-15T15:54:30.626 INFO:tasks.workunit.client.0.smithi049.stdout:2/770: symlink d3/db/d1d/d1e/d2b/d3b/d42/l108 0 2024-05-15T15:54:30.629 INFO:tasks.workunit.client.0.smithi049.stdout:0/893: rename dc/d13/d14/d25/d31/d71/l85 to dc/d13/d14/l12d 0 2024-05-15T15:54:30.634 INFO:tasks.workunit.client.1.smithi144.stdout:5/972: creat d0/dc/d24/d57/f149 x:0 0 0 2024-05-15T15:54:30.639 INFO:tasks.workunit.client.0.smithi049.stdout:3/873: write d12/d30/dc1/dc7/df8/f80 [609913,89803] 0 2024-05-15T15:54:30.641 INFO:tasks.workunit.client.1.smithi144.stdout:3/968: mkdir d0/d1/d17/dae/d124/d138 0 2024-05-15T15:54:30.644 INFO:tasks.workunit.client.0.smithi049.stdout:6/951: rename dd/d26/d5b/d110/fd3 to dd/d26/d5b/d110/d50/d7e/f127 0 2024-05-15T15:54:30.647 INFO:tasks.workunit.client.0.smithi049.stdout:0/894: unlink dc/d13/d3d/lea 0 2024-05-15T15:54:30.648 INFO:tasks.workunit.client.1.smithi144.stdout:7/987: link d4/c17 d4/d133/c159 0 2024-05-15T15:54:30.651 INFO:tasks.workunit.client.1.smithi144.stdout:5/973: dread - d0/dc/d1f/d50/fd6 zero size 2024-05-15T15:54:30.652 INFO:tasks.workunit.client.1.smithi144.stdout:5/974: fsync d0/dc/d2f/f122 0 2024-05-15T15:54:30.654 INFO:tasks.workunit.client.1.smithi144.stdout:0/931: mknod d4/d6/d3a/d4c/dd6/d55/da5/d10f/c155 0 2024-05-15T15:54:30.659 INFO:tasks.workunit.client.0.smithi049.stdout:2/771: dwrite d3/db/d1d/d1e/d2b/d3b/fa7 [0,4194304] 0 2024-05-15T15:54:30.660 INFO:tasks.workunit.client.0.smithi049.stdout:3/874: rename d12/fcc to d12/d14/d52/db9/dc4/d68/d70/dda/f10b 0 2024-05-15T15:54:30.662 INFO:tasks.workunit.client.0.smithi049.stdout:3/875: stat d12/d14/d52/db9/dc4/d68/ffb 0 2024-05-15T15:54:30.665 INFO:tasks.workunit.client.1.smithi144.stdout:3/969: rename d0/d1/f43 to d0/d1/d56/d71/d73/f139 0 2024-05-15T15:54:30.666 INFO:tasks.workunit.client.1.smithi144.stdout:3/970: chown d0/d11/ff7 37 1 2024-05-15T15:54:30.666 INFO:tasks.workunit.client.1.smithi144.stdout:2/895: sync 2024-05-15T15:54:30.667 INFO:tasks.workunit.client.0.smithi049.stdout:6/952: mkdir dd/d26/d5b/d110/d50/dcf/d11c/d120/df8/d128 0 2024-05-15T15:54:30.668 INFO:tasks.workunit.client.1.smithi144.stdout:7/988: read - d4/d3c/d76/d9d/dac/d48/d72/d82/fad zero size 2024-05-15T15:54:30.668 INFO:tasks.workunit.client.0.smithi049.stdout:6/953: stat dd/d17/d24/d58/d5f/l7f 0 2024-05-15T15:54:30.670 INFO:tasks.workunit.client.0.smithi049.stdout:7/847: link d0/d2f/cb0 d0/d6/d23/d4a/d7b/dc4/c109 0 2024-05-15T15:54:30.671 INFO:tasks.workunit.client.1.smithi144.stdout:1/996: link d7/d1d/d33/d29/d6b/d5e/d74/fab d7/d1d/f13c 0 2024-05-15T15:54:30.671 INFO:tasks.workunit.client.1.smithi144.stdout:3/971: dwrite d0/d11/d3b/da7/f123 [0,4194304] 0 2024-05-15T15:54:30.673 INFO:tasks.workunit.client.0.smithi049.stdout:2/772: creat d3/d78/d8c/d91/d7a/f109 x:0 0 0 2024-05-15T15:54:30.675 INFO:tasks.workunit.client.1.smithi144.stdout:5/975: rename d0/dc/d1f/l111 to d0/dc/d24/d57/l14a 0 2024-05-15T15:54:30.678 INFO:tasks.workunit.client.0.smithi049.stdout:7/848: dwrite d0/f4 [8388608,4194304] 0 2024-05-15T15:54:30.682 INFO:tasks.workunit.client.1.smithi144.stdout:1/997: unlink d7/d1d/d33/cfc 0 2024-05-15T15:54:30.684 INFO:tasks.workunit.client.0.smithi049.stdout:2/773: creat d3/db/d1d/d1e/dd9/f10a x:0 0 0 2024-05-15T15:54:30.686 INFO:tasks.workunit.client.0.smithi049.stdout:3/876: creat d12/d14/d32/d64/df7/f10c x:0 0 0 2024-05-15T15:54:30.686 INFO:tasks.workunit.client.0.smithi049.stdout:8/836: sync 2024-05-15T15:54:30.686 INFO:tasks.workunit.client.0.smithi049.stdout:4/811: sync 2024-05-15T15:54:30.687 INFO:tasks.workunit.client.0.smithi049.stdout:9/743: sync 2024-05-15T15:54:30.687 INFO:tasks.workunit.client.1.smithi144.stdout:0/932: rename d4/dc/c20 to d4/d6/d3a/d4c/dd6/d55/da5/d10f/c156 0 2024-05-15T15:54:30.687 INFO:tasks.workunit.client.1.smithi144.stdout:1/998: dread d7/d1d/f8b [0,4194304] 0 2024-05-15T15:54:30.689 INFO:tasks.workunit.client.1.smithi144.stdout:5/976: creat d0/dc/d24/d8e/d93/d9e/f14b x:0 0 0 2024-05-15T15:54:30.691 INFO:tasks.workunit.client.0.smithi049.stdout:2/774: creat d3/d78/d8c/f10b x:0 0 0 2024-05-15T15:54:30.697 INFO:tasks.workunit.client.0.smithi049.stdout:0/895: getdents dc/d13/d2b/d76/d8a/da2 0 2024-05-15T15:54:30.701 INFO:tasks.workunit.client.1.smithi144.stdout:1/999: symlink d7/d1d/d33/d41/d22/dbd/dc8/l13d 0 2024-05-15T15:54:30.706 INFO:tasks.workunit.client.0.smithi049.stdout:4/812: fdatasync d18/f70 0 2024-05-15T15:54:30.707 INFO:tasks.workunit.client.0.smithi049.stdout:4/813: write d18/d1e/d63/f96 [7119355,89322] 0 2024-05-15T15:54:30.708 INFO:tasks.workunit.client.1.smithi144.stdout:7/989: write d4/d3c/d76/d9d/dac/d48/fb9 [1388324,39725] 0 2024-05-15T15:54:30.709 INFO:tasks.workunit.client.0.smithi049.stdout:2/775: truncate d3/da/d15/d73/f7f 129551 0 2024-05-15T15:54:30.709 INFO:tasks.workunit.client.1.smithi144.stdout:7/990: fdatasync d4/d3c/d76/d9d/dac/d111/d12c/f12f 0 2024-05-15T15:54:30.714 INFO:tasks.workunit.client.1.smithi144.stdout:2/896: getdents d3/d4d 0 2024-05-15T15:54:30.715 INFO:tasks.workunit.client.0.smithi049.stdout:6/954: write dd/d26/d5b/d110/f98 [3888623,29293] 0 2024-05-15T15:54:30.719 INFO:tasks.workunit.client.1.smithi144.stdout:3/972: getdents d0/d1/d17/dae/d124 0 2024-05-15T15:54:30.722 INFO:tasks.workunit.client.0.smithi049.stdout:2/776: mknod d3/da/d15/dec/ddf/c10c 0 2024-05-15T15:54:30.722 INFO:tasks.workunit.client.0.smithi049.stdout:2/777: readlink d3/da/l3a 0 2024-05-15T15:54:30.724 INFO:tasks.workunit.client.1.smithi144.stdout:5/977: link d0/dc/d2f/d68/c13b d0/dc/d24/d8e/d95/d146/d2d/d59/d74/c14c 0 2024-05-15T15:54:30.725 INFO:tasks.workunit.client.0.smithi049.stdout:9/744: rmdir d0/d8/dd6/de2/dec/d9d/de0 0 2024-05-15T15:54:30.727 INFO:tasks.workunit.client.0.smithi049.stdout:9/745: readlink d0/d8/d9/d52/d5e/d8a/dd3/d87/le1 0 2024-05-15T15:54:30.728 INFO:tasks.workunit.client.1.smithi144.stdout:7/991: symlink d4/df/l15a 0 2024-05-15T15:54:30.729 INFO:tasks.workunit.client.1.smithi144.stdout:7/992: read - d4/d3c/d76/d9d/dac/f142 zero size 2024-05-15T15:54:30.729 INFO:tasks.workunit.client.0.smithi049.stdout:3/877: rename d12/d14/d52/db9/l57 to d12/d26/l10d 0 2024-05-15T15:54:30.731 INFO:tasks.workunit.client.1.smithi144.stdout:0/933: creat d4/df/d1f/d56/f157 x:0 0 0 2024-05-15T15:54:30.735 INFO:tasks.workunit.client.1.smithi144.stdout:2/897: symlink d3/d7/d20/d37/d76/db5/l129 0 2024-05-15T15:54:30.736 INFO:tasks.workunit.client.1.smithi144.stdout:5/978: read - d0/ff8 zero size 2024-05-15T15:54:30.738 INFO:tasks.workunit.client.0.smithi049.stdout:8/837: link d2/l5 d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/l105 0 2024-05-15T15:54:30.739 INFO:tasks.workunit.client.1.smithi144.stdout:3/973: write d0/d1/d56/d98/d9c/fee [544614,110915] 0 2024-05-15T15:54:30.739 INFO:tasks.workunit.client.0.smithi049.stdout:6/955: write dd/d26/d5b/d110/d86/fae [4571948,82733] 0 2024-05-15T15:54:30.741 INFO:tasks.workunit.client.1.smithi144.stdout:7/993: mkdir d4/d3c/d76/d9d/dac/d48/d72/da2/daf/d15b 0 2024-05-15T15:54:30.742 INFO:tasks.workunit.client.1.smithi144.stdout:7/994: chown d4/d3c/d76/d9d/dac/d48/d151 10 1 2024-05-15T15:54:30.746 INFO:tasks.workunit.client.0.smithi049.stdout:0/896: rename dc/d13/d2b/d76/d8a/c8b to dc/d13/d3d/c12e 0 2024-05-15T15:54:30.747 INFO:tasks.workunit.client.1.smithi144.stdout:2/898: mknod d3/d7/df8/da6/c12a 0 2024-05-15T15:54:30.748 INFO:tasks.workunit.client.0.smithi049.stdout:3/878: unlink d12/d14/d52/db9/da6/dc2/lea 0 2024-05-15T15:54:30.749 INFO:tasks.workunit.client.1.smithi144.stdout:5/979: creat d0/dc/d2f/d79/da5/f14d x:0 0 0 2024-05-15T15:54:30.759 INFO:tasks.workunit.client.0.smithi049.stdout:9/746: dwrite d0/d8/d9/d52/d5e/d5f/f3e [0,4194304] 0 2024-05-15T15:54:30.764 INFO:tasks.workunit.client.0.smithi049.stdout:0/897: dread - dc/d13/d14/f95 zero size 2024-05-15T15:54:30.766 INFO:tasks.workunit.client.0.smithi049.stdout:6/956: write dd/d17/f6d [439107,84163] 0 2024-05-15T15:54:30.768 INFO:tasks.workunit.client.1.smithi144.stdout:0/934: link d4/d6/d3a/d4c/fa6 d4/d6/d3a/d4c/d151/f158 0 2024-05-15T15:54:30.768 INFO:tasks.workunit.client.1.smithi144.stdout:7/995: dwrite d4/d41/d5b/da0/fe3 [0,4194304] 0 2024-05-15T15:54:30.769 INFO:tasks.workunit.client.0.smithi049.stdout:2/778: dwrite d3/db/d1d/d1e/d2b/d3b/d47/d59/fd1 [0,4194304] 0 2024-05-15T15:54:30.770 INFO:tasks.workunit.client.1.smithi144.stdout:0/935: write d4/d2d/d100/f11d [1420755,106119] 0 2024-05-15T15:54:30.774 INFO:tasks.workunit.client.0.smithi049.stdout:3/879: fsync d12/d14/d52/db9/dc4/d68/d70/f8a 0 2024-05-15T15:54:30.776 INFO:tasks.workunit.client.1.smithi144.stdout:3/974: creat d0/d1/d56/d71/f13a x:0 0 0 2024-05-15T15:54:30.779 INFO:tasks.workunit.client.1.smithi144.stdout:7/996: rmdir d4/d3c/d76 39 2024-05-15T15:54:30.783 INFO:tasks.workunit.client.1.smithi144.stdout:3/975: dread d0/d11/d3b/da7/fc6 [0,4194304] 0 2024-05-15T15:54:30.783 INFO:tasks.workunit.client.1.smithi144.stdout:5/980: creat d0/dc/d24/d8e/d95/d146/d2d/d59/f14e x:0 0 0 2024-05-15T15:54:30.783 INFO:tasks.workunit.client.0.smithi049.stdout:9/747: dread d0/d8/d1b/f32 [0,4194304] 0 2024-05-15T15:54:30.784 INFO:tasks.workunit.client.1.smithi144.stdout:3/976: stat d0/d11/d3b/da7/def/d117/lbb 0 2024-05-15T15:54:30.785 INFO:tasks.workunit.client.0.smithi049.stdout:2/779: mknod d3/db/d1d/c10d 0 2024-05-15T15:54:30.786 INFO:tasks.workunit.client.1.smithi144.stdout:2/899: getdents d3/d7/d20/def 0 2024-05-15T15:54:30.787 INFO:tasks.workunit.client.0.smithi049.stdout:6/957: fdatasync dd/d17/d24/d2b/f63 0 2024-05-15T15:54:30.790 INFO:tasks.workunit.client.0.smithi049.stdout:4/814: rename d18/d1e/d2a/d36/d86/dbd to d18/d1e/d63/d10d 0 2024-05-15T15:54:30.792 INFO:tasks.workunit.client.1.smithi144.stdout:3/977: dwrite d0/d1/f115 [0,4194304] 0 2024-05-15T15:54:30.794 INFO:tasks.workunit.client.0.smithi049.stdout:0/898: dread dc/d13/d14/d25/d31/d71/f2c [0,4194304] 0 2024-05-15T15:54:30.795 INFO:tasks.workunit.client.1.smithi144.stdout:0/936: creat d4/d6/d3a/d4c/d151/f159 x:0 0 0 2024-05-15T15:54:30.799 INFO:tasks.workunit.client.0.smithi049.stdout:6/958: mknod dd/d26/d5b/d110/d50/d7e/c129 0 2024-05-15T15:54:30.801 INFO:tasks.workunit.client.0.smithi049.stdout:2/780: truncate d3/db/d1d/d1e/f21 3629450 0 2024-05-15T15:54:30.803 INFO:tasks.workunit.client.0.smithi049.stdout:4/815: fsync d18/d1e/d63/d99/fa5 0 2024-05-15T15:54:30.805 INFO:tasks.workunit.client.0.smithi049.stdout:4/816: write d18/d1e/d63/d99/de1/f47 [1101134,48500] 0 2024-05-15T15:54:30.805 INFO:tasks.workunit.client.0.smithi049.stdout:0/899: mkdir dc/d13/d2b/d76/d8a/d12f 0 2024-05-15T15:54:30.807 INFO:tasks.workunit.client.1.smithi144.stdout:2/900: mknod d3/d7/d20/d89/d86/db1/d116/c12b 0 2024-05-15T15:54:30.810 INFO:tasks.workunit.client.0.smithi049.stdout:6/959: unlink dd/ce 0 2024-05-15T15:54:30.810 INFO:tasks.workunit.client.0.smithi049.stdout:3/880: write d12/d14/d52/db9/da6/dc2/fc3 [2248406,42777] 0 2024-05-15T15:54:30.813 INFO:tasks.workunit.client.1.smithi144.stdout:7/997: dwrite d4/f1c [0,4194304] 0 2024-05-15T15:54:30.813 INFO:tasks.workunit.client.1.smithi144.stdout:2/901: symlink d3/d7/d20/d22/d52/d64/l12c 0 2024-05-15T15:54:30.814 INFO:tasks.workunit.client.0.smithi049.stdout:2/781: creat d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/dc9/f10e x:0 0 0 2024-05-15T15:54:30.814 INFO:tasks.workunit.client.0.smithi049.stdout:2/782: chown d3/d8/fe 3575 1 2024-05-15T15:54:30.816 INFO:tasks.workunit.client.1.smithi144.stdout:3/978: creat d0/d1/f13b x:0 0 0 2024-05-15T15:54:30.817 INFO:tasks.workunit.client.1.smithi144.stdout:3/979: chown d0/d11/d3b/dcf/f114 32418 1 2024-05-15T15:54:30.818 INFO:tasks.workunit.client.1.smithi144.stdout:3/980: chown d0/d1/d17/d39/d40/d4a/d77 253849511 1 2024-05-15T15:54:30.819 INFO:tasks.workunit.client.1.smithi144.stdout:0/937: write d4/df/d14/f5f [971470,116847] 0 2024-05-15T15:54:30.820 INFO:tasks.workunit.client.1.smithi144.stdout:0/938: fsync d4/d2d/d48/d5b/f14f 0 2024-05-15T15:54:30.821 INFO:tasks.workunit.client.1.smithi144.stdout:2/902: mknod d3/d10/c12d 0 2024-05-15T15:54:30.823 INFO:tasks.workunit.client.1.smithi144.stdout:2/903: read d3/d7/d20/d37/d76/db5/fbd [3540156,92825] 0 2024-05-15T15:54:30.824 INFO:tasks.workunit.client.0.smithi049.stdout:9/748: link d0/d8/d1b/d40/f58 d0/d8/d9/d52/dc7/f10a 0 2024-05-15T15:54:30.824 INFO:tasks.workunit.client.0.smithi049.stdout:3/881: dread - d12/d26/f7d zero size 2024-05-15T15:54:30.826 INFO:tasks.workunit.client.1.smithi144.stdout:3/981: rename d0/d1/d17/d39/d40/d4a/d77 to d0/d1/d56/d71/d73/d13c 0 2024-05-15T15:54:30.827 INFO:tasks.workunit.client.0.smithi049.stdout:6/960: mknod dd/d26/d5b/d110/d50/dcf/c12a 0 2024-05-15T15:54:30.832 INFO:tasks.workunit.client.1.smithi144.stdout:2/904: fsync d3/d7/d20/d22/d52/d64/f5f 0 2024-05-15T15:54:30.832 INFO:tasks.workunit.client.1.smithi144.stdout:3/982: dwrite d0/d1/d56/d71/d125/fc1 [4194304,4194304] 0 2024-05-15T15:54:30.834 INFO:tasks.workunit.client.1.smithi144.stdout:3/983: chown d0/f1d 3924470 1 2024-05-15T15:54:30.837 INFO:tasks.workunit.client.0.smithi049.stdout:9/749: chown d0/d8/d42/d75/de7/ff9 60 1 2024-05-15T15:54:30.838 INFO:tasks.workunit.client.1.smithi144.stdout:7/998: write d4/df/f32 [9262695,43619] 0 2024-05-15T15:54:30.839 INFO:tasks.workunit.client.1.smithi144.stdout:0/939: creat d4/f15a x:0 0 0 2024-05-15T15:54:30.841 INFO:tasks.workunit.client.0.smithi049.stdout:9/750: truncate d0/d8/d9/d52/d5e/d8a/fe3 632662 0 2024-05-15T15:54:30.842 INFO:tasks.workunit.client.0.smithi049.stdout:3/882: creat d12/d14/d32/d64/f10e x:0 0 0 2024-05-15T15:54:30.845 INFO:tasks.workunit.client.1.smithi144.stdout:0/940: dwrite d4/d6/d3a/d4c/f9a [0,4194304] 0 2024-05-15T15:54:30.846 INFO:tasks.workunit.client.1.smithi144.stdout:0/941: stat d4/d6/d3a/d5d/l127 0 2024-05-15T15:54:30.847 INFO:tasks.workunit.client.0.smithi049.stdout:3/883: write d12/d30/dc1/dc7/df8/fbb [4467128,69631] 0 2024-05-15T15:54:30.848 INFO:tasks.workunit.client.0.smithi049.stdout:6/961: creat dd/d26/d5b/d110/d50/dcf/d11c/d120/f12b x:0 0 0 2024-05-15T15:54:30.850 INFO:tasks.workunit.client.0.smithi049.stdout:2/783: write d3/db/d1d/d1e/d2b/d3b/d47/d59/fb2 [560487,111529] 0 2024-05-15T15:54:30.852 INFO:tasks.workunit.client.0.smithi049.stdout:4/817: dwrite d18/d1e/d2a/d50/d54/d109/d98/fd8 [0,4194304] 0 2024-05-15T15:54:30.852 INFO:tasks.workunit.client.0.smithi049.stdout:9/751: creat d0/d8/dd6/de2/dec/f10b x:0 0 0 2024-05-15T15:54:30.857 INFO:tasks.workunit.client.1.smithi144.stdout:7/999: mknod d4/df/d25/d40/d73/d112/c15c 0 2024-05-15T15:54:30.859 INFO:tasks.workunit.client.0.smithi049.stdout:4/818: chown d18/d1e 1704 1 2024-05-15T15:54:30.860 INFO:tasks.workunit.client.0.smithi049.stdout:0/900: dwrite dc/d13/d14/d25/d31/f58 [0,4194304] 0 2024-05-15T15:54:30.863 INFO:tasks.workunit.client.0.smithi049.stdout:3/884: creat d12/d30/d66/f10f x:0 0 0 2024-05-15T15:54:30.864 INFO:tasks.workunit.client.0.smithi049.stdout:9/752: rmdir d0 39 2024-05-15T15:54:30.865 INFO:tasks.workunit.client.0.smithi049.stdout:3/885: chown d12/d30/da3 5314 1 2024-05-15T15:54:30.867 INFO:tasks.workunit.client.0.smithi049.stdout:4/819: stat d18/d21/d2b/cf6 0 2024-05-15T15:54:30.871 INFO:tasks.workunit.client.0.smithi049.stdout:2/784: dread d3/db/d1d/d1e/d2b/d3b/d47/d59/fd1 [0,4194304] 0 2024-05-15T15:54:30.871 INFO:tasks.workunit.client.0.smithi049.stdout:4/820: read d18/d1e/d2a/fed [91593,4525] 0 2024-05-15T15:54:30.874 INFO:tasks.workunit.client.1.smithi144.stdout:3/984: rename d0/d11/d3b/da0/f10d to d0/d1/d56/f13d 0 2024-05-15T15:54:30.876 INFO:tasks.workunit.client.0.smithi049.stdout:0/901: symlink dc/d13/d56/l130 0 2024-05-15T15:54:30.877 INFO:tasks.workunit.client.1.smithi144.stdout:5/981: sync 2024-05-15T15:54:30.877 INFO:tasks.workunit.client.1.smithi144.stdout:0/942: getdents d4/dc 0 2024-05-15T15:54:30.879 INFO:tasks.workunit.client.0.smithi049.stdout:6/962: truncate dd/d17/fbe 667162 0 2024-05-15T15:54:30.879 INFO:tasks.workunit.client.0.smithi049.stdout:4/821: dwrite d18/d1e/d63/f96 [4194304,4194304] 0 2024-05-15T15:54:30.880 INFO:tasks.workunit.client.1.smithi144.stdout:3/985: mknod d0/d1/d17/d39/dc8/c13e 0 2024-05-15T15:54:30.882 INFO:tasks.workunit.client.1.smithi144.stdout:2/905: truncate d3/d7/d20/d89/d86/ff9 3432333 0 2024-05-15T15:54:30.888 INFO:tasks.workunit.client.1.smithi144.stdout:2/906: symlink d3/d7/df8/l12e 0 2024-05-15T15:54:30.892 INFO:tasks.workunit.client.0.smithi049.stdout:6/963: mknod dd/d26/d5b/d110/d86/d96/c12c 0 2024-05-15T15:54:30.894 INFO:tasks.workunit.client.0.smithi049.stdout:4/822: unlink d18/d1e/d2a/d36/d86/fb3 0 2024-05-15T15:54:30.897 INFO:tasks.workunit.client.0.smithi049.stdout:4/823: truncate d18/d1e/d2a/d50/d54/d109/dff/f10b 471713 0 2024-05-15T15:54:30.897 INFO:tasks.workunit.client.0.smithi049.stdout:3/886: dread d12/d14/f27 [0,4194304] 0 2024-05-15T15:54:30.906 INFO:tasks.workunit.client.1.smithi144.stdout:0/943: dwrite d4/d2d/d48/d5b/dd3/d107/fd2 [0,4194304] 0 2024-05-15T15:54:30.910 INFO:tasks.workunit.client.1.smithi144.stdout:2/907: rename d3/d7/d20/d37/d76/db5/l129 to d3/d7/d20/d37/l12f 0 2024-05-15T15:54:30.911 INFO:tasks.workunit.client.1.smithi144.stdout:5/982: write d0/dc/d24/d8e/d95/d146/f51 [814876,24014] 0 2024-05-15T15:54:30.912 INFO:tasks.workunit.client.1.smithi144.stdout:3/986: truncate d0/d11/d3b/da7/def/f10e 2392021 0 2024-05-15T15:54:30.916 INFO:tasks.workunit.client.1.smithi144.stdout:0/944: chown d4/l7 848835572 1 2024-05-15T15:54:30.917 INFO:tasks.workunit.client.1.smithi144.stdout:0/945: chown d4/df/d1f/d24/fbc 21568978 1 2024-05-15T15:54:30.918 INFO:tasks.workunit.client.0.smithi049.stdout:9/753: dread d0/d8/fd [0,4194304] 0 2024-05-15T15:54:30.919 INFO:tasks.workunit.client.0.smithi049.stdout:3/887: rmdir d12/d14/d52/db9/dc4/d68 39 2024-05-15T15:54:30.920 INFO:tasks.workunit.client.0.smithi049.stdout:9/754: dread - d0/d8/d1b/d40/fbc zero size 2024-05-15T15:54:30.923 INFO:tasks.workunit.client.0.smithi049.stdout:0/902: creat dc/d13/d14/d25/d31/f131 x:0 0 0 2024-05-15T15:54:30.923 INFO:tasks.workunit.client.1.smithi144.stdout:0/946: dwrite d4/d2d/f12b [0,4194304] 0 2024-05-15T15:54:30.924 INFO:tasks.workunit.client.1.smithi144.stdout:0/947: write d4/fc2 [5375569,60875] 0 2024-05-15T15:54:30.925 INFO:tasks.workunit.client.0.smithi049.stdout:6/964: symlink dd/d17/d24/d8a/l12d 0 2024-05-15T15:54:30.926 INFO:tasks.workunit.client.1.smithi144.stdout:0/948: chown d4/d6/d3a/d4c/dd6/le1 8695714 1 2024-05-15T15:54:30.928 INFO:tasks.workunit.client.0.smithi049.stdout:4/824: symlink d18/l10e 0 2024-05-15T15:54:30.932 INFO:tasks.workunit.client.0.smithi049.stdout:2/785: getdents d3/db/d1d/d1e/d2b 0 2024-05-15T15:54:30.934 INFO:tasks.workunit.client.0.smithi049.stdout:3/888: mknod d12/d30/dc1/dc7/df8/c110 0 2024-05-15T15:54:30.942 INFO:tasks.workunit.client.0.smithi049.stdout:0/903: rmdir dc/d13/d56/d6f/dce 39 2024-05-15T15:54:30.944 INFO:tasks.workunit.client.0.smithi049.stdout:7/849: sync 2024-05-15T15:54:30.944 INFO:tasks.workunit.client.1.smithi144.stdout:5/983: dwrite d0/dc/d1f/f39 [0,4194304] 0 2024-05-15T15:54:30.957 INFO:tasks.workunit.client.0.smithi049.stdout:3/889: symlink d12/d30/d66/l111 0 2024-05-15T15:54:30.957 INFO:tasks.workunit.client.0.smithi049.stdout:2/786: dread d3/da/f45 [0,4194304] 0 2024-05-15T15:54:30.957 INFO:tasks.workunit.client.0.smithi049.stdout:9/755: mkdir d0/d8/d53/dd2/df3/d10c 0 2024-05-15T15:54:30.958 INFO:tasks.workunit.client.1.smithi144.stdout:0/949: mknod d4/d2d/d48/d5b/dd3/c15b 0 2024-05-15T15:54:30.960 INFO:tasks.workunit.client.1.smithi144.stdout:3/987: link d0/d1/d17/d39/d40/fe7 d0/d11/d3b/da7/def/d117/d119/f13f 0 2024-05-15T15:54:30.962 INFO:tasks.workunit.client.0.smithi049.stdout:6/965: write dd/f16 [2156919,5445] 0 2024-05-15T15:54:30.964 INFO:tasks.workunit.client.1.smithi144.stdout:5/984: dread d0/dc/ff3 [0,4194304] 0 2024-05-15T15:54:30.965 INFO:tasks.workunit.client.0.smithi049.stdout:4/825: unlink d18/d1e/d2a/d50/dc4/cdc 0 2024-05-15T15:54:30.966 INFO:tasks.workunit.client.1.smithi144.stdout:2/908: getdents d3/d7/d20/d40 0 2024-05-15T15:54:30.968 INFO:tasks.workunit.client.0.smithi049.stdout:3/890: dwrite d12/d14/d52/db9/f46 [0,4194304] 0 2024-05-15T15:54:30.969 INFO:tasks.workunit.client.0.smithi049.stdout:7/850: dread d0/ddb/dde/dcc/fd1 [0,4194304] 0 2024-05-15T15:54:30.970 INFO:tasks.workunit.client.0.smithi049.stdout:0/904: mkdir dc/d13/d14/d25/d31/d132 0 2024-05-15T15:54:30.972 INFO:tasks.workunit.client.0.smithi049.stdout:2/787: symlink d3/db/d5b/dbf/l10f 0 2024-05-15T15:54:30.980 INFO:tasks.workunit.client.0.smithi049.stdout:6/966: fdatasync dd/dbc/fd6 0 2024-05-15T15:54:30.985 INFO:tasks.workunit.client.0.smithi049.stdout:4/826: rename d18/d1e/d63/d99/de1/da8/le9 to d18/d1e/d2a/d36/d5d/l10f 0 2024-05-15T15:54:30.987 INFO:tasks.workunit.client.1.smithi144.stdout:3/988: mknod d0/d11/d3b/da0/df6/c140 0 2024-05-15T15:54:30.988 INFO:tasks.workunit.client.0.smithi049.stdout:0/905: mknod dc/d13/d14/d25/d31/dae/dc5/d108/c133 0 2024-05-15T15:54:30.992 INFO:tasks.workunit.client.1.smithi144.stdout:0/950: rename d4/d6/c38 to d4/d6/d3a/c15c 0 2024-05-15T15:54:30.993 INFO:tasks.workunit.client.1.smithi144.stdout:5/985: dread d0/dc/d24/d8e/f8a [4194304,4194304] 0 2024-05-15T15:54:30.993 INFO:tasks.workunit.client.1.smithi144.stdout:0/951: write d4/d2d/fab [2876565,98085] 0 2024-05-15T15:54:30.996 INFO:tasks.workunit.client.1.smithi144.stdout:2/909: write d3/d7/d54/fa0 [3287831,116406] 0 2024-05-15T15:54:30.997 INFO:tasks.workunit.client.1.smithi144.stdout:5/986: rmdir d0/d10c/d73 39 2024-05-15T15:54:31.000 INFO:tasks.workunit.client.1.smithi144.stdout:0/952: unlink d4/d6/d3a/d4c/fea 0 2024-05-15T15:54:31.012 INFO:tasks.workunit.client.1.smithi144.stdout:5/987: creat d0/d10c/d134/f14f x:0 0 0 2024-05-15T15:54:31.013 INFO:tasks.workunit.client.1.smithi144.stdout:5/988: dread - d0/d10c/d134/f14f zero size 2024-05-15T15:54:31.014 INFO:tasks.workunit.client.1.smithi144.stdout:2/910: dwrite d3/d7/df8/d65/f9f [0,4194304] 0 2024-05-15T15:54:31.015 INFO:tasks.workunit.client.1.smithi144.stdout:2/911: dread - d3/d7/d20/def/d112/f122 zero size 2024-05-15T15:54:31.016 INFO:tasks.workunit.client.1.smithi144.stdout:0/953: mkdir d4/df/d1f/d24/d15d 0 2024-05-15T15:54:31.021 INFO:tasks.workunit.client.1.smithi144.stdout:2/912: truncate d3/d7/df8/d65/fc8 1914379 0 2024-05-15T15:54:31.022 INFO:tasks.workunit.client.1.smithi144.stdout:2/913: dread - d3/d7/d20/def/d112/f122 zero size 2024-05-15T15:54:31.023 INFO:tasks.workunit.client.1.smithi144.stdout:0/954: symlink d4/df/l15e 0 2024-05-15T15:54:31.030 INFO:tasks.workunit.client.1.smithi144.stdout:0/955: truncate d4/df/d1f/f2c 1688169 0 2024-05-15T15:54:31.036 INFO:tasks.workunit.client.1.smithi144.stdout:0/956: rename d4/d2d/d48 to d4/d6/d4a/d74/d15f 0 2024-05-15T15:54:31.042 INFO:tasks.workunit.client.1.smithi144.stdout:2/914: dread d3/d7/df8/d65/f68 [0,4194304] 0 2024-05-15T15:54:31.042 INFO:tasks.workunit.client.1.smithi144.stdout:2/915: fdatasync d3/d7/d20/d3b/fd9 0 2024-05-15T15:54:31.043 INFO:tasks.workunit.client.1.smithi144.stdout:2/916: write d3/d7/d20/d3b/fd9 [4984392,34908] 0 2024-05-15T15:54:31.048 INFO:tasks.workunit.client.1.smithi144.stdout:2/917: truncate d3/d10/f1d 1301342 0 2024-05-15T15:54:31.054 INFO:tasks.workunit.client.1.smithi144.stdout:2/918: symlink d3/d7/d20/d22/d5e/l130 0 2024-05-15T15:54:31.065 INFO:tasks.workunit.client.0.smithi049.stdout:7/851: fdatasync d0/d6/f29 0 2024-05-15T15:54:31.067 INFO:tasks.workunit.client.0.smithi049.stdout:6/967: truncate dd/d17/d24/d2b/f5d 275211 0 2024-05-15T15:54:31.067 INFO:tasks.workunit.client.0.smithi049.stdout:7/852: write d0/d34/db3/f100 [560822,93937] 0 2024-05-15T15:54:31.069 INFO:tasks.workunit.client.0.smithi049.stdout:0/906: symlink dc/d13/d14/d2f/d8f/dcb/l134 0 2024-05-15T15:54:31.070 INFO:tasks.workunit.client.0.smithi049.stdout:6/968: truncate dd/d17/d9d/f115 472636 0 2024-05-15T15:54:31.070 INFO:tasks.workunit.client.0.smithi049.stdout:2/788: truncate d3/da/d15/f27 3179847 0 2024-05-15T15:54:31.072 INFO:tasks.workunit.client.1.smithi144.stdout:2/919: dread d3/d10/f7e [0,4194304] 0 2024-05-15T15:54:31.072 INFO:tasks.workunit.client.0.smithi049.stdout:9/756: getdents d0/d8/d53/dd2/df3 0 2024-05-15T15:54:31.072 INFO:tasks.workunit.client.1.smithi144.stdout:2/920: chown d3/d7/d20/d89/d6e/c119 1025042 1 2024-05-15T15:54:31.073 INFO:tasks.workunit.client.0.smithi049.stdout:9/757: chown d0/d8/d21/fff 17657 1 2024-05-15T15:54:31.075 INFO:tasks.workunit.client.0.smithi049.stdout:7/853: creat d0/ddb/dde/dca/f10a x:0 0 0 2024-05-15T15:54:31.076 INFO:tasks.workunit.client.0.smithi049.stdout:7/854: stat d0/ddb/dde/dca/d5e/dbd/cec 0 2024-05-15T15:54:31.079 INFO:tasks.workunit.client.1.smithi144.stdout:2/921: creat d3/d7/d11a/f131 x:0 0 0 2024-05-15T15:54:31.080 INFO:tasks.workunit.client.0.smithi049.stdout:2/789: truncate d3/db/d1d/d1e/d2b/f75 103398 0 2024-05-15T15:54:31.084 INFO:tasks.workunit.client.1.smithi144.stdout:2/922: creat d3/d7/df8/d65/f132 x:0 0 0 2024-05-15T15:54:31.086 INFO:tasks.workunit.client.0.smithi049.stdout:7/855: mknod d0/d6/d23/d4a/d7b/dc4/c10b 0 2024-05-15T15:54:31.092 INFO:tasks.workunit.client.0.smithi049.stdout:2/790: dread d3/da/d15/f39 [0,4194304] 0 2024-05-15T15:54:31.096 INFO:tasks.workunit.client.1.smithi144.stdout:2/923: creat d3/d7/d20/d22/f133 x:0 0 0 2024-05-15T15:54:31.096 INFO:tasks.workunit.client.1.smithi144.stdout:2/924: stat d3/d10 0 2024-05-15T15:54:31.098 INFO:tasks.workunit.client.0.smithi049.stdout:0/907: getdents dc/d13/d14/d2f/d8f 0 2024-05-15T15:54:31.099 INFO:tasks.workunit.client.0.smithi049.stdout:6/969: write dd/d17/d24/d58/d5f/dd7/fa9 [946333,115763] 0 2024-05-15T15:54:31.099 INFO:tasks.workunit.client.0.smithi049.stdout:0/908: chown dc/d13/d3d/c120 140 1 2024-05-15T15:54:31.100 INFO:tasks.workunit.client.0.smithi049.stdout:0/909: readlink dc/d13/d14/l27 0 2024-05-15T15:54:31.101 INFO:tasks.workunit.client.1.smithi144.stdout:2/925: creat d3/d10/f134 x:0 0 0 2024-05-15T15:54:31.102 INFO:tasks.workunit.client.0.smithi049.stdout:2/791: read - d3/da/d15/d73/fe3 zero size 2024-05-15T15:54:31.104 INFO:tasks.workunit.client.1.smithi144.stdout:2/926: fsync d3/d7/f27 0 2024-05-15T15:54:31.106 INFO:tasks.workunit.client.0.smithi049.stdout:6/970: mknod dd/d26/d5b/d110/d50/d7e/c12e 0 2024-05-15T15:54:31.110 INFO:tasks.workunit.client.0.smithi049.stdout:0/910: dwrite dc/d13/d2b/d76/fed [0,4194304] 0 2024-05-15T15:54:31.112 INFO:tasks.workunit.client.0.smithi049.stdout:9/758: dwrite d0/d8/d1b/d40/f77 [0,4194304] 0 2024-05-15T15:54:31.112 INFO:tasks.workunit.client.1.smithi144.stdout:2/927: unlink d3/d7/d20/f113 0 2024-05-15T15:54:31.120 INFO:tasks.workunit.client.1.smithi144.stdout:2/928: creat d3/d7/d20/d37/d76/f135 x:0 0 0 2024-05-15T15:54:31.121 INFO:tasks.workunit.client.1.smithi144.stdout:2/929: write d3/d10/f134 [323114,124710] 0 2024-05-15T15:54:31.124 INFO:tasks.workunit.client.0.smithi049.stdout:0/911: rmdir dc/d13/d14/d25/db9 39 2024-05-15T15:54:31.124 INFO:tasks.workunit.client.0.smithi049.stdout:7/856: dwrite d0/d6/d23/d4a/fb4 [0,4194304] 0 2024-05-15T15:54:31.128 INFO:tasks.workunit.client.1.smithi144.stdout:2/930: symlink d3/d7/d20/d22/d52/l136 0 2024-05-15T15:54:31.131 INFO:tasks.workunit.client.1.smithi144.stdout:3/989: sync 2024-05-15T15:54:31.133 INFO:tasks.workunit.client.1.smithi144.stdout:3/990: chown d0/d11/d3b/da7/fe1 582442 1 2024-05-15T15:54:31.134 INFO:tasks.workunit.client.0.smithi049.stdout:9/759: rename d0/d25/d5d/f109 to d0/d8/d9/d52/d5e/d8a/dd3/d7d/dba/f10d 0 2024-05-15T15:54:31.134 INFO:tasks.workunit.client.0.smithi049.stdout:9/760: dread - d0/d8/d42/d75/f102 zero size 2024-05-15T15:54:31.135 INFO:tasks.workunit.client.0.smithi049.stdout:8/838: sync 2024-05-15T15:54:31.135 INFO:tasks.workunit.client.0.smithi049.stdout:7/857: creat d0/d6/d23/d4a/d7b/de2/dff/f10c x:0 0 0 2024-05-15T15:54:31.136 INFO:tasks.workunit.client.1.smithi144.stdout:2/931: dwrite d3/d7/d20/d22/d5e/f96 [0,4194304] 0 2024-05-15T15:54:31.136 INFO:tasks.workunit.client.0.smithi049.stdout:7/858: stat d0/d2f/f8f 0 2024-05-15T15:54:31.137 INFO:tasks.workunit.client.0.smithi049.stdout:0/912: mknod dc/d13/d14/d25/db9/c135 0 2024-05-15T15:54:31.137 INFO:tasks.workunit.client.1.smithi144.stdout:2/932: fsync d3/d7/d20/d3b/fd9 0 2024-05-15T15:54:31.140 INFO:tasks.workunit.client.0.smithi049.stdout:9/761: fsync d0/f1 0 2024-05-15T15:54:31.142 INFO:tasks.workunit.client.0.smithi049.stdout:9/762: chown d0/d8/dd6/de2/dec/c92 1124748 1 2024-05-15T15:54:31.144 INFO:tasks.workunit.client.1.smithi144.stdout:2/933: creat d3/d7/d20/d89/d6e/f137 x:0 0 0 2024-05-15T15:54:31.150 INFO:tasks.workunit.client.1.smithi144.stdout:2/934: symlink d3/d7/d20/d3b/l138 0 2024-05-15T15:54:31.150 INFO:tasks.workunit.client.0.smithi049.stdout:7/859: creat d0/d6/d23/d4a/d7b/de2/dff/f10d x:0 0 0 2024-05-15T15:54:31.153 INFO:tasks.workunit.client.0.smithi049.stdout:9/763: creat d0/d8/d9/d52/d5e/d8a/dd3/d7d/f10e x:0 0 0 2024-05-15T15:54:31.154 INFO:tasks.workunit.client.1.smithi144.stdout:2/935: mkdir d3/d7/df8/da6/dc1/d139 0 2024-05-15T15:54:31.154 INFO:tasks.workunit.client.0.smithi049.stdout:2/792: dwrite d3/da/d15/fc0 [0,4194304] 0 2024-05-15T15:54:31.154 INFO:tasks.workunit.client.1.smithi144.stdout:2/936: chown d3/d7/d20/d89/d6e 9 1 2024-05-15T15:54:31.157 INFO:tasks.workunit.client.1.smithi144.stdout:5/989: sync 2024-05-15T15:54:31.157 INFO:tasks.workunit.client.1.smithi144.stdout:0/957: sync 2024-05-15T15:54:31.158 INFO:tasks.workunit.client.1.smithi144.stdout:2/937: mknod d3/d7/d20/d37/d76/db5/db6/dbf/c13a 0 2024-05-15T15:54:31.158 INFO:tasks.workunit.client.1.smithi144.stdout:5/990: stat d0/d7e/fc2 0 2024-05-15T15:54:31.160 INFO:tasks.workunit.client.0.smithi049.stdout:6/971: dwrite dd/f53 [0,4194304] 0 2024-05-15T15:54:31.160 INFO:tasks.workunit.client.1.smithi144.stdout:0/958: truncate d4/df/d1f/d24/d96/f131 219536 0 2024-05-15T15:54:31.161 INFO:tasks.workunit.client.0.smithi049.stdout:6/972: write dd/ff9 [3867043,27801] 0 2024-05-15T15:54:31.162 INFO:tasks.workunit.client.0.smithi049.stdout:7/860: truncate d0/d34/fad 2520146 0 2024-05-15T15:54:31.166 INFO:tasks.workunit.client.0.smithi049.stdout:8/839: getdents d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf 0 2024-05-15T15:54:31.171 INFO:tasks.workunit.client.1.smithi144.stdout:5/991: rename d0/dc/d2f/d34/d63/d6e/fd9 to d0/dc/d2f/d109/f150 0 2024-05-15T15:54:31.172 INFO:tasks.workunit.client.0.smithi049.stdout:0/913: write dc/d32/d93/f88 [3383584,95896] 0 2024-05-15T15:54:31.176 INFO:tasks.workunit.client.0.smithi049.stdout:9/764: creat d0/d8e/f10f x:0 0 0 2024-05-15T15:54:31.177 INFO:tasks.workunit.client.1.smithi144.stdout:3/991: dwrite d0/d11/d3b/da7/fa9 [0,4194304] 0 2024-05-15T15:54:31.178 INFO:tasks.workunit.client.1.smithi144.stdout:5/992: mknod d0/d10c/d5d/c151 0 2024-05-15T15:54:31.187 INFO:tasks.workunit.client.1.smithi144.stdout:5/993: unlink d0/dc/d24/fd2 0 2024-05-15T15:54:31.191 INFO:tasks.workunit.client.1.smithi144.stdout:2/938: dwrite d3/d4d/fca [0,4194304] 0 2024-05-15T15:54:31.192 INFO:tasks.workunit.client.1.smithi144.stdout:3/992: mkdir d0/d1/d56/d71/d125/d141 0 2024-05-15T15:54:31.195 INFO:tasks.workunit.client.0.smithi049.stdout:0/914: dread dc/d13/d14/d25/d31/f75 [0,4194304] 0 2024-05-15T15:54:31.197 INFO:tasks.workunit.client.0.smithi049.stdout:2/793: rename d3/db/d1d/d1e/d2b/f104 to d3/db/d1d/d1e/d2b/d3b/d47/d9a/f110 0 2024-05-15T15:54:31.198 INFO:tasks.workunit.client.1.smithi144.stdout:3/993: read d0/d1/d17/d39/d40/f92 [2599198,130368] 0 2024-05-15T15:54:31.201 INFO:tasks.workunit.client.0.smithi049.stdout:6/973: creat dd/d26/d106/f12f x:0 0 0 2024-05-15T15:54:31.201 INFO:tasks.workunit.client.1.smithi144.stdout:2/939: truncate d3/d4d/fae 3313179 0 2024-05-15T15:54:31.202 INFO:tasks.workunit.client.1.smithi144.stdout:3/994: dread - d0/d1/d17/d39/d40/fe7 zero size 2024-05-15T15:54:31.202 INFO:tasks.workunit.client.1.smithi144.stdout:0/959: dwrite d4/df/f16 [0,4194304] 0 2024-05-15T15:54:31.203 INFO:tasks.workunit.client.0.smithi049.stdout:0/915: dread dc/d13/d2b/d76/fc8 [0,4194304] 0 2024-05-15T15:54:31.203 INFO:tasks.workunit.client.0.smithi049.stdout:9/765: getdents d0/d8/d9/d52/dc7/de9 0 2024-05-15T15:54:31.207 INFO:tasks.workunit.client.0.smithi049.stdout:8/840: rename d2/de/d71/d86/de0/ce5 to d2/de/d71/d86/d5d/d7d/d1b/d68/c106 0 2024-05-15T15:54:31.208 INFO:tasks.workunit.client.1.smithi144.stdout:3/995: chown d0/d1/d17/d39/d4d/dfd/f10f 129 1 2024-05-15T15:54:31.209 INFO:tasks.workunit.client.1.smithi144.stdout:3/996: stat d0/d1/d17/d39/d40 0 2024-05-15T15:54:31.209 INFO:tasks.workunit.client.0.smithi049.stdout:2/794: unlink d3/da/d15/d24/f2f 0 2024-05-15T15:54:31.210 INFO:tasks.workunit.client.1.smithi144.stdout:0/960: mkdir d4/d6/d4a/d74/d15f/d5b/d8e/d160 0 2024-05-15T15:54:31.211 INFO:tasks.workunit.client.0.smithi049.stdout:2/795: dread - d3/db/d1d/d1e/d2b/d3b/ff7 zero size 2024-05-15T15:54:31.211 INFO:tasks.workunit.client.1.smithi144.stdout:0/961: readlink d4/df/d1f/d24/d85/lb0 0 2024-05-15T15:54:31.212 INFO:tasks.workunit.client.0.smithi049.stdout:2/796: write d3/db/d1d/d1e/d2b/d3b/d47/d9a/da3/dc9/f10e [797083,25761] 0 2024-05-15T15:54:31.213 INFO:tasks.workunit.client.1.smithi144.stdout:2/940: fdatasync d3/f1c 0 2024-05-15T15:54:31.216 INFO:tasks.workunit.client.1.smithi144.stdout:3/997: mknod d0/d11/d24/d78/c142 0 2024-05-15T15:54:31.217 INFO:tasks.workunit.client.0.smithi049.stdout:0/916: unlink dc/d13/d2b/d76/ca6 0 2024-05-15T15:54:31.220 INFO:tasks.workunit.client.0.smithi049.stdout:9/766: symlink d0/d8/dd6/de2/dec/l110 0 2024-05-15T15:54:31.225 INFO:tasks.workunit.client.0.smithi049.stdout:7/861: rename d0/d6/da/f8d to d0/d6/d23/d45/d6a/da1/d82/dd3/f10e 0 2024-05-15T15:54:31.225 INFO:tasks.workunit.client.1.smithi144.stdout:5/994: dwrite d0/dc/d12/f18 [0,4194304] 0 2024-05-15T15:54:31.230 INFO:tasks.workunit.client.0.smithi049.stdout:6/974: mkdir dd/d17/d24/d58/d124/d130 0 2024-05-15T15:54:31.233 INFO:tasks.workunit.client.0.smithi049.stdout:0/917: symlink dc/d6d/l136 0 2024-05-15T15:54:31.240 INFO:tasks.workunit.client.0.smithi049.stdout:7/862: symlink d0/d6/d23/d45/d6a/l10f 0 2024-05-15T15:54:31.242 INFO:tasks.workunit.client.1.smithi144.stdout:3/998: dwrite d0/d11/d3b/da7/def/d117/f9d [0,4194304] 0 2024-05-15T15:54:31.243 INFO:tasks.workunit.client.0.smithi049.stdout:6/975: mkdir dd/dbc/ddb/d131 0 2024-05-15T15:54:31.252 INFO:tasks.workunit.client.0.smithi049.stdout:7/863: dread d0/d6/d23/d4a/d7b/de2/fee [0,4194304] 0 2024-05-15T15:54:31.304 INFO:tasks.workunit.client.0.smithi049.stdout:3/891: sync 2024-05-15T15:54:31.307 INFO:tasks.workunit.client.0.smithi049.stdout:9/767: write d0/d8/d1b/f98 [354157,74598] 0 2024-05-15T15:54:31.314 INFO:tasks.workunit.client.0.smithi049.stdout:8/841: creat d2/de/d71/d86/d5d/d7d/d1b/f107 x:0 0 0 2024-05-15T15:54:31.323 INFO:tasks.workunit.client.0.smithi049.stdout:0/918: dwrite dc/d13/d14/f19 [0,4194304] 0 2024-05-15T15:54:31.324 INFO:tasks.workunit.client.0.smithi049.stdout:9/768: truncate d0/d8/d1b/f3c 187585 0 2024-05-15T15:54:31.328 INFO:tasks.workunit.client.0.smithi049.stdout:6/976: mkdir dd/d26/d5b/d110/d86/df7/d132 0 2024-05-15T15:54:31.331 INFO:tasks.workunit.client.0.smithi049.stdout:7/864: creat d0/d6/d23/d45/d6a/d102/f110 x:0 0 0 2024-05-15T15:54:31.333 INFO:tasks.workunit.client.0.smithi049.stdout:9/769: dread d0/d8/d9/d52/d5e/d8a/dd3/d7d/fa3 [0,4194304] 0 2024-05-15T15:54:31.337 INFO:tasks.workunit.client.0.smithi049.stdout:8/842: chown d2/d18/c1f 118697 1 2024-05-15T15:54:31.343 INFO:tasks.workunit.client.0.smithi049.stdout:3/892: link d12/d26/le2 d12/d67/da8/l112 0 2024-05-15T15:54:31.346 INFO:tasks.workunit.client.0.smithi049.stdout:8/843: dwrite d2/de/d71/d86/d5d/d7d/d1b/f107 [0,4194304] 0 2024-05-15T15:54:31.351 INFO:tasks.workunit.client.0.smithi049.stdout:9/770: truncate d0/d8/d12/f59 735486 0 2024-05-15T15:54:31.353 INFO:tasks.workunit.client.0.smithi049.stdout:6/977: symlink dd/l133 0 2024-05-15T15:54:31.368 INFO:tasks.workunit.client.0.smithi049.stdout:0/919: dwrite dc/d13/d56/fdf [0,4194304] 0 2024-05-15T15:54:31.371 INFO:tasks.workunit.client.0.smithi049.stdout:9/771: rename d0/d8/d9/d52/d5e/d5f/c7c to d0/d8/d9/d52/d5e/d8a/dd3/d87/dd1/c111 0 2024-05-15T15:54:31.383 INFO:tasks.workunit.client.0.smithi049.stdout:3/893: read d12/d14/d52/db9/dc4/d68/fba [3988039,12126] 0 2024-05-15T15:54:31.385 INFO:tasks.workunit.client.0.smithi049.stdout:3/894: truncate d12/d14/d52/db9/dc4/f100 302009 0 2024-05-15T15:54:31.388 INFO:tasks.workunit.client.0.smithi049.stdout:8/844: dread d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fd2 [4194304,4194304] 0 2024-05-15T15:54:31.389 INFO:tasks.workunit.client.0.smithi049.stdout:9/772: mknod d0/d8/d9/d22/dbb/c112 0 2024-05-15T15:54:31.393 INFO:tasks.workunit.client.0.smithi049.stdout:7/865: getdents d0/d6/d23/d4a/d7b 0 2024-05-15T15:54:31.399 INFO:tasks.workunit.client.0.smithi049.stdout:6/978: dwrite dd/d26/d5b/d110/d50/f83 [0,4194304] 0 2024-05-15T15:54:31.400 INFO:tasks.workunit.client.0.smithi049.stdout:0/920: mknod dc/d13/d56/d6f/dce/c137 0 2024-05-15T15:54:31.401 INFO:tasks.workunit.client.0.smithi049.stdout:8/845: mknod d2/de/d71/d86/d5d/d7d/db8/d5b/c108 0 2024-05-15T15:54:31.404 INFO:tasks.workunit.client.0.smithi049.stdout:8/846: dread d2/de/d71/d86/d5d/d7d/db8/f59 [0,4194304] 0 2024-05-15T15:54:31.407 INFO:tasks.workunit.client.1.smithi144.stdout:0/962: rename d4/d6/d3a/d40/c7e to d4/c161 0 2024-05-15T15:54:31.407 INFO:tasks.workunit.client.0.smithi049.stdout:3/895: mkdir d12/d14/d52/db9/da6/de3/d113 0 2024-05-15T15:54:31.408 INFO:tasks.workunit.client.1.smithi144.stdout:5/995: dread - d0/dc/d2f/d68/f119 zero size 2024-05-15T15:54:31.412 INFO:tasks.workunit.client.0.smithi049.stdout:2/797: sync 2024-05-15T15:54:31.412 INFO:tasks.workunit.client.0.smithi049.stdout:4/827: sync 2024-05-15T15:54:31.416 INFO:tasks.workunit.client.0.smithi049.stdout:9/773: link d0/d8/d9/d52/d5e/d5f/fa0 d0/d8/d53/f113 0 2024-05-15T15:54:31.419 INFO:tasks.workunit.client.1.smithi144.stdout:2/941: rename d3/d7/d20/d89/d6e/f137 to d3/d7/df8/da6/f13b 0 2024-05-15T15:54:31.420 INFO:tasks.workunit.client.1.smithi144.stdout:2/942: chown d3/d7/d20/d22/d52/d64/d97/cac 1174 1 2024-05-15T15:54:31.424 INFO:tasks.workunit.client.0.smithi049.stdout:7/866: write d0/d6/d23/f8a [847494,23161] 0 2024-05-15T15:54:31.425 INFO:tasks.workunit.client.0.smithi049.stdout:8/847: symlink d2/de/d71/d86/d5d/d7d/d1b/dc8/l109 0 2024-05-15T15:54:31.425 INFO:tasks.workunit.client.1.smithi144.stdout:2/943: dwrite d3/d7/d20/d22/f110 [0,4194304] 0 2024-05-15T15:54:31.428 INFO:tasks.workunit.client.0.smithi049.stdout:6/979: write dd/d17/d24/fd0 [2151885,99687] 0 2024-05-15T15:54:31.429 INFO:tasks.workunit.client.1.smithi144.stdout:3/999: write d0/d1/d17/d39/d4d/fdf [627137,67033] 0 2024-05-15T15:54:31.430 INFO:tasks.workunit.client.0.smithi049.stdout:0/921: mkdir dc/d13/d2b/d76/d8a/d12f/d138 0 2024-05-15T15:54:31.435 INFO:tasks.workunit.client.0.smithi049.stdout:4/828: rmdir d18/d1e/d2a/d36/d86 39 2024-05-15T15:54:31.436 INFO:tasks.workunit.client.1.smithi144.stdout:2/944: rmdir d3/d7/d20/d89 39 2024-05-15T15:54:31.440 INFO:tasks.workunit.client.1.smithi144.stdout:5/996: write d0/dc/d24/d57/fa6 [163973,119975] 0 2024-05-15T15:54:31.442 INFO:tasks.workunit.client.1.smithi144.stdout:2/945: mknod d3/d7/d20/d22/d5e/c13c 0 2024-05-15T15:54:31.444 INFO:tasks.workunit.client.1.smithi144.stdout:0/963: getdents d4/d6/d3a/d4c/dd6/d55 0 2024-05-15T15:54:31.446 INFO:tasks.workunit.client.0.smithi049.stdout:2/798: write d3/db/d5b/fdc [831858,102514] 0 2024-05-15T15:54:31.446 INFO:tasks.workunit.client.1.smithi144.stdout:5/997: dwrite d0/d7e/fc2 [0,4194304] 0 2024-05-15T15:54:31.447 INFO:tasks.workunit.client.1.smithi144.stdout:5/998: read d0/f22 [2052663,6726] 0 2024-05-15T15:54:31.450 INFO:tasks.workunit.client.1.smithi144.stdout:0/964: creat d4/d6/d3a/d40/f162 x:0 0 0 2024-05-15T15:54:31.452 INFO:tasks.workunit.client.1.smithi144.stdout:0/965: write d4/d6/f32 [4801680,70339] 0 2024-05-15T15:54:31.452 INFO:tasks.workunit.client.0.smithi049.stdout:0/922: mknod dc/da4/db4/c139 0 2024-05-15T15:54:31.459 INFO:tasks.workunit.client.1.smithi144.stdout:2/946: dread d3/d7/d20/d22/d52/d64/fd3 [0,4194304] 0 2024-05-15T15:54:31.463 INFO:tasks.workunit.client.0.smithi049.stdout:2/799: rmdir d3/d78/d8c 39 2024-05-15T15:54:31.464 INFO:tasks.workunit.client.1.smithi144.stdout:0/966: symlink d4/d6/d4a/d74/d15f/d5b/dd3/d107/d143/l163 0 2024-05-15T15:54:31.465 INFO:tasks.workunit.client.1.smithi144.stdout:0/967: truncate d4/d6/d3a/d40/f162 60372 0 2024-05-15T15:54:31.466 INFO:tasks.workunit.client.0.smithi049.stdout:3/896: creat d12/f114 x:0 0 0 2024-05-15T15:54:31.467 INFO:tasks.workunit.client.1.smithi144.stdout:2/947: stat d3/f6c 0 2024-05-15T15:54:31.471 INFO:tasks.workunit.client.1.smithi144.stdout:2/948: creat d3/d7/d54/f13d x:0 0 0 2024-05-15T15:54:31.473 INFO:tasks.workunit.client.1.smithi144.stdout:2/949: truncate d3/d7/d20/d37/d76/db5/db6/dbf/fd0 2030346 0 2024-05-15T15:54:31.474 INFO:tasks.workunit.client.1.smithi144.stdout:2/950: chown d3/d7/d54/d98/f9c 495020 1 2024-05-15T15:54:31.474 INFO:tasks.workunit.client.0.smithi049.stdout:9/774: write d0/d8e/fc1 [134405,60607] 0 2024-05-15T15:54:31.476 INFO:tasks.workunit.client.1.smithi144.stdout:5/999: write d0/dc/d1f/d50/db3/d106/f139 [256256,14376] 0 2024-05-15T15:54:31.476 INFO:tasks.workunit.client.0.smithi049.stdout:6/980: dwrite dd/f19 [0,4194304] 0 2024-05-15T15:54:31.479 INFO:tasks.workunit.client.0.smithi049.stdout:8/848: write d2/fc [154657,104668] 0 2024-05-15T15:54:31.480 INFO:tasks.workunit.client.0.smithi049.stdout:7/867: creat d0/d6/f111 x:0 0 0 2024-05-15T15:54:31.480 INFO:tasks.workunit.client.0.smithi049.stdout:0/923: dread dc/d13/d14/f79 [0,4194304] 0 2024-05-15T15:54:31.482 INFO:tasks.workunit.client.0.smithi049.stdout:0/924: stat dc/d13/d56/d6f/dce 0 2024-05-15T15:54:31.483 INFO:tasks.workunit.client.0.smithi049.stdout:3/897: truncate d12/d14/d32/d64/f93 4783671 0 2024-05-15T15:54:31.484 INFO:tasks.workunit.client.0.smithi049.stdout:2/800: creat d3/da/da9/f111 x:0 0 0 2024-05-15T15:54:31.486 INFO:tasks.workunit.client.0.smithi049.stdout:4/829: dwrite d18/d21/f30 [0,4194304] 0 2024-05-15T15:54:31.487 INFO:tasks.workunit.client.1.smithi144.stdout:0/968: dread d4/d6/f11 [4194304,4194304] 0 2024-05-15T15:54:31.490 INFO:tasks.workunit.client.0.smithi049.stdout:9/775: mknod d0/d8/d12/c114 0 2024-05-15T15:54:31.490 INFO:tasks.workunit.client.0.smithi049.stdout:0/925: creat dc/d13/d14/dc9/f13a x:0 0 0 2024-05-15T15:54:31.495 INFO:tasks.workunit.client.0.smithi049.stdout:2/801: chown d3/da/d15/cc2 8589463 1 2024-05-15T15:54:31.495 INFO:tasks.workunit.client.0.smithi049.stdout:3/898: dread d12/d14/d52/db9/da6/f97 [4194304,4194304] 0 2024-05-15T15:54:31.495 INFO:tasks.workunit.client.0.smithi049.stdout:7/868: creat d0/d6/df7/f112 x:0 0 0 2024-05-15T15:54:31.496 INFO:tasks.workunit.client.1.smithi144.stdout:2/951: link d3/d10/c12d d3/d7/d20/d22/d52/d64/d97/db8/c13e 0 2024-05-15T15:54:31.497 INFO:tasks.workunit.client.0.smithi049.stdout:6/981: mknod dd/d17/d24/d58/d124/d130/c134 0 2024-05-15T15:54:31.499 INFO:tasks.workunit.client.0.smithi049.stdout:8/849: rename d2/de/d71/d86/c8b to d2/d6a/c10a 0 2024-05-15T15:54:31.500 INFO:tasks.workunit.client.1.smithi144.stdout:2/952: truncate d3/f9 8655746 0 2024-05-15T15:54:31.507 INFO:tasks.workunit.client.0.smithi049.stdout:2/802: write d3/db/d1d/d1e/d2b/d2e/fb7 [949106,93547] 0 2024-05-15T15:54:31.508 INFO:tasks.workunit.client.0.smithi049.stdout:8/850: write d2/de/d71/d86/ffb [327390,54484] 0 2024-05-15T15:54:31.508 INFO:tasks.workunit.client.0.smithi049.stdout:7/869: rename d0/d6/d23/d4a/d7b/de2/dff to d0/d6/d23/d4a/d7b/dc4/de0/d113 0 2024-05-15T15:54:31.509 INFO:tasks.workunit.client.1.smithi144.stdout:0/969: getdents d4/d6/d4a/d4d/dc7 0 2024-05-15T15:54:31.511 INFO:tasks.workunit.client.0.smithi049.stdout:6/982: mknod dd/d26/d5b/d110/d50/dcf/c135 0 2024-05-15T15:54:31.512 INFO:tasks.workunit.client.0.smithi049.stdout:3/899: dwrite d12/d14/f54 [0,4194304] 0 2024-05-15T15:54:31.515 INFO:tasks.workunit.client.1.smithi144.stdout:0/970: chown d4/c161 1850 1 2024-05-15T15:54:31.515 INFO:tasks.workunit.client.0.smithi049.stdout:3/900: fdatasync d12/d14/d52/db9/dc4/d68/d70/dda/fe6 0 2024-05-15T15:54:31.516 INFO:tasks.workunit.client.1.smithi144.stdout:0/971: dread d4/d6/d4a/d74/fc3 [4194304,4194304] 0 2024-05-15T15:54:31.517 INFO:tasks.workunit.client.1.smithi144.stdout:0/972: chown d4/d6/d3a/d4c/dd6/d55/da5/c120 0 1 2024-05-15T15:54:31.521 INFO:tasks.workunit.client.0.smithi049.stdout:0/926: dread dc/d32/f7b [0,4194304] 0 2024-05-15T15:54:31.521 INFO:tasks.workunit.client.0.smithi049.stdout:7/870: mkdir d0/d6/d23/d45/d6a/d114 0 2024-05-15T15:54:31.521 INFO:tasks.workunit.client.0.smithi049.stdout:9/776: getdents d0/d8/d1b/d40/d5c/d69 0 2024-05-15T15:54:31.521 INFO:tasks.workunit.client.0.smithi049.stdout:3/901: dread d12/d14/f27 [0,4194304] 0 2024-05-15T15:54:31.524 INFO:tasks.workunit.client.0.smithi049.stdout:6/983: unlink dd/d26/f4f 0 2024-05-15T15:54:31.526 INFO:tasks.workunit.client.1.smithi144.stdout:2/953: creat d3/d7/d20/d89/f13f x:0 0 0 2024-05-15T15:54:31.530 INFO:tasks.workunit.client.0.smithi049.stdout:2/803: mknod d3/da/d15/d24/d107/c112 0 2024-05-15T15:54:31.535 INFO:tasks.workunit.client.1.smithi144.stdout:0/973: creat d4/d6/d3a/f164 x:0 0 0 2024-05-15T15:54:31.536 INFO:tasks.workunit.client.0.smithi049.stdout:3/902: truncate f7 709669 0 2024-05-15T15:54:31.538 INFO:tasks.workunit.client.0.smithi049.stdout:3/903: chown d12/d14/d52/db9 1971 1 2024-05-15T15:54:31.540 INFO:tasks.workunit.client.1.smithi144.stdout:0/974: unlink d4/df/d1f/d24/d96/c98 0 2024-05-15T15:54:31.542 INFO:tasks.workunit.client.1.smithi144.stdout:2/954: dread d3/d7/d54/d98/f9c [0,4194304] 0 2024-05-15T15:54:31.542 INFO:tasks.workunit.client.0.smithi049.stdout:8/851: fsync d2/de/d71/d86/d5d/d7d/db8/d5b/f88 0 2024-05-15T15:54:31.545 INFO:tasks.workunit.client.1.smithi144.stdout:2/955: mknod d3/d7/d20/def/c140 0 2024-05-15T15:54:31.547 INFO:tasks.workunit.client.0.smithi049.stdout:9/777: dread d0/d8/d9/d52/d5e/d5f/f55 [0,4194304] 0 2024-05-15T15:54:31.548 INFO:tasks.workunit.client.0.smithi049.stdout:8/852: dread d2/de/d71/d86/d5d/d7d/d1b/d68/fab [0,4194304] 0 2024-05-15T15:54:31.549 INFO:tasks.workunit.client.0.smithi049.stdout:3/904: creat d12/d67/f115 x:0 0 0 2024-05-15T15:54:31.551 INFO:tasks.workunit.client.1.smithi144.stdout:0/975: unlink d4/d6/d3a/d4c/dd6/c34 0 2024-05-15T15:54:31.554 INFO:tasks.workunit.client.1.smithi144.stdout:2/956: rmdir d3/d7/d20/d37/d76 39 2024-05-15T15:54:31.555 INFO:tasks.workunit.client.1.smithi144.stdout:2/957: chown d3/d7/d20/d22/d52/d64/ce6 695865811 1 2024-05-15T15:54:31.558 INFO:tasks.workunit.client.1.smithi144.stdout:0/976: mknod d4/df/d1f/db7/c165 0 2024-05-15T15:54:31.560 INFO:tasks.workunit.client.1.smithi144.stdout:2/958: mkdir d3/d4d/db4/d141 0 2024-05-15T15:54:31.560 INFO:tasks.workunit.client.1.smithi144.stdout:2/959: stat d3/d7/d20/d22/f2b 0 2024-05-15T15:54:31.562 INFO:tasks.workunit.client.1.smithi144.stdout:2/960: truncate d3/d7/d54/f10d 336520 0 2024-05-15T15:54:31.567 INFO:tasks.workunit.client.0.smithi049.stdout:9/778: creat d0/d8/d12/dfd/f115 x:0 0 0 2024-05-15T15:54:31.569 INFO:tasks.workunit.client.0.smithi049.stdout:9/779: dread d0/d8/d9/d22/f96 [0,4194304] 0 2024-05-15T15:54:31.569 INFO:tasks.workunit.client.1.smithi144.stdout:0/977: creat d4/d6/d4a/d4d/db1/d13a/f166 x:0 0 0 2024-05-15T15:54:31.576 INFO:tasks.workunit.client.1.smithi144.stdout:0/978: creat d4/d6/d3a/d4c/dd6/d55/da5/d10f/f167 x:0 0 0 2024-05-15T15:54:31.577 INFO:tasks.workunit.client.0.smithi049.stdout:2/804: link d3/da/d15/d9e/l100 d3/da/d15/dec/ddf/l113 0 2024-05-15T15:54:31.579 INFO:tasks.workunit.client.0.smithi049.stdout:0/927: getdents dc/d13/d56/d6f/dce/def 0 2024-05-15T15:54:31.584 INFO:tasks.workunit.client.0.smithi049.stdout:6/984: getdents dd/d17/d9d/da1 0 2024-05-15T15:54:31.586 INFO:tasks.workunit.client.0.smithi049.stdout:0/928: dread dc/d13/d2b/d76/d8a/f102 [0,4194304] 0 2024-05-15T15:54:31.587 INFO:tasks.workunit.client.0.smithi049.stdout:9/780: fdatasync d0/d8/d42/f91 0 2024-05-15T15:54:31.589 INFO:tasks.workunit.client.1.smithi144.stdout:2/961: dwrite d3/d7/d20/d89/d86/ff9 [0,4194304] 0 2024-05-15T15:54:31.589 INFO:tasks.workunit.client.0.smithi049.stdout:3/905: creat d12/d14/d52/d108/f116 x:0 0 0 2024-05-15T15:54:31.592 INFO:tasks.workunit.client.0.smithi049.stdout:2/805: symlink d3/db/d1d/d1e/d2b/d3b/l114 0 2024-05-15T15:54:31.595 INFO:tasks.workunit.client.1.smithi144.stdout:0/979: dwrite d4/d6/d4a/d74/f79 [0,4194304] 0 2024-05-15T15:54:31.596 INFO:tasks.workunit.client.1.smithi144.stdout:2/962: mknod d3/d7/d20/d22/c142 0 2024-05-15T15:54:31.597 INFO:tasks.workunit.client.1.smithi144.stdout:0/980: write d4/d6/d3a/d4c/dd6/d55/da5/d10f/f167 [401882,5645] 0 2024-05-15T15:54:31.598 INFO:tasks.workunit.client.1.smithi144.stdout:0/981: write d4/df/d1f/d24/d96/f131 [1261629,106575] 0 2024-05-15T15:54:31.601 INFO:tasks.workunit.client.1.smithi144.stdout:2/963: unlink d3/ff4 0 2024-05-15T15:54:31.602 INFO:tasks.workunit.client.0.smithi049.stdout:8/853: link d2/d6a/cb7 d2/de/d71/d86/d5d/de3/c10b 0 2024-05-15T15:54:31.603 INFO:tasks.workunit.client.1.smithi144.stdout:0/982: dread d4/d2d/f8d [4194304,4194304] 0 2024-05-15T15:54:31.604 INFO:tasks.workunit.client.1.smithi144.stdout:0/983: fsync d4/d6/d3a/d4c/dd6/d55/da5/d10f/f167 0 2024-05-15T15:54:31.606 INFO:tasks.workunit.client.1.smithi144.stdout:0/984: truncate d4/d6/d3a/d4c/dd6/d55/da5/d10f/f167 816920 0 2024-05-15T15:54:31.607 INFO:tasks.workunit.client.0.smithi049.stdout:9/781: truncate d0/f1d 2302655 0 2024-05-15T15:54:31.607 INFO:tasks.workunit.client.0.smithi049.stdout:0/929: creat dc/d13/d14/d25/d31/dae/dc5/f13b x:0 0 0 2024-05-15T15:54:31.608 INFO:tasks.workunit.client.0.smithi049.stdout:9/782: chown d0/d8/d9/d52/d5e/d8a/dd3/d7d 2 1 2024-05-15T15:54:31.611 INFO:tasks.workunit.client.1.smithi144.stdout:2/964: mkdir d3/d7/d20/d22/d52/d64/d97/d143 0 2024-05-15T15:54:31.615 INFO:tasks.workunit.client.1.smithi144.stdout:0/985: rmdir d4/df/d102/d11b 39 2024-05-15T15:54:31.616 INFO:tasks.workunit.client.1.smithi144.stdout:0/986: truncate d4/d6/d3a/d4c/d151/f159 30676 0 2024-05-15T15:54:31.620 INFO:tasks.workunit.client.0.smithi049.stdout:8/854: rmdir d2/de/d71/d86/d5d/d7d/db8 39 2024-05-15T15:54:31.626 INFO:tasks.workunit.client.1.smithi144.stdout:0/987: fsync d4/d2d/f2e 0 2024-05-15T15:54:31.628 INFO:tasks.workunit.client.0.smithi049.stdout:0/930: dread dc/d13/f91 [0,4194304] 0 2024-05-15T15:54:31.629 INFO:tasks.workunit.client.0.smithi049.stdout:0/931: chown dc/d13/d2b/fac 5660 1 2024-05-15T15:54:31.634 INFO:tasks.workunit.client.1.smithi144.stdout:2/965: write d3/d4d/fb7 [4476659,7957] 0 2024-05-15T15:54:31.635 INFO:tasks.workunit.client.0.smithi049.stdout:6/985: dwrite dd/d17/f8d [0,4194304] 0 2024-05-15T15:54:31.638 INFO:tasks.workunit.client.1.smithi144.stdout:2/966: unlink d3/d7/d20/d3b/f4f 0 2024-05-15T15:54:31.639 INFO:tasks.workunit.client.0.smithi049.stdout:9/783: creat d0/d8/d9/d52/dc7/de9/f116 x:0 0 0 2024-05-15T15:54:31.640 INFO:tasks.workunit.client.0.smithi049.stdout:0/932: read - dc/d13/d14/d25/d31/dae/dc5/d108/fd3 zero size 2024-05-15T15:54:31.641 INFO:tasks.workunit.client.1.smithi144.stdout:0/988: dread d4/df/d1f/fb3 [0,4194304] 0 2024-05-15T15:54:31.642 INFO:tasks.workunit.client.1.smithi144.stdout:2/967: unlink d3/d7/d54/c7b 0 2024-05-15T15:54:31.644 INFO:tasks.workunit.client.0.smithi049.stdout:9/784: read - d0/d8/d21/fb2 zero size 2024-05-15T15:54:31.644 INFO:tasks.workunit.client.0.smithi049.stdout:6/986: fsync dd/d26/d5b/da8/fd9 0 2024-05-15T15:54:31.645 INFO:tasks.workunit.client.1.smithi144.stdout:0/989: mknod d4/d6/d4a/d4d/dc7/c168 0 2024-05-15T15:54:31.646 INFO:tasks.workunit.client.0.smithi049.stdout:8/855: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/f10c x:0 0 0 2024-05-15T15:54:31.646 INFO:tasks.workunit.client.0.smithi049.stdout:2/806: dwrite d3/f4d [0,4194304] 0 2024-05-15T15:54:31.647 INFO:tasks.workunit.client.0.smithi049.stdout:9/785: write d0/f54 [3720709,112678] 0 2024-05-15T15:54:31.647 INFO:tasks.workunit.client.1.smithi144.stdout:2/968: fsync d3/d7/d20/d22/d52/d64/f5f 0 2024-05-15T15:54:31.652 INFO:tasks.workunit.client.0.smithi049.stdout:0/933: readlink dc/d13/d14/dc9/le7 0 2024-05-15T15:54:31.658 INFO:tasks.workunit.client.1.smithi144.stdout:0/990: dread d4/d6/d4a/d74/d15f/d10a/f11e [0,4194304] 0 2024-05-15T15:54:31.658 INFO:tasks.workunit.client.1.smithi144.stdout:0/991: stat d4/df/c30 0 2024-05-15T15:54:31.659 INFO:tasks.workunit.client.0.smithi049.stdout:9/786: mkdir d0/d8/d9/d52/d5e/d8a/dd3/d7d/d117 0 2024-05-15T15:54:31.661 INFO:tasks.workunit.client.0.smithi049.stdout:9/787: write d0/d8/d1b/db3/ffa [951056,73140] 0 2024-05-15T15:54:31.661 INFO:tasks.workunit.client.0.smithi049.stdout:2/807: fdatasync d3/da/d15/f39 0 2024-05-15T15:54:31.665 INFO:tasks.workunit.client.0.smithi049.stdout:8/856: write d2/de/d71/d86/f8d [4126001,16359] 0 2024-05-15T15:54:31.666 INFO:tasks.workunit.client.0.smithi049.stdout:8/857: stat d2/de/dc3/fc4 0 2024-05-15T15:54:31.667 INFO:tasks.workunit.client.1.smithi144.stdout:0/992: mknod d4/d6/d4a/d74/d15f/c169 0 2024-05-15T15:54:31.669 INFO:tasks.workunit.client.1.smithi144.stdout:2/969: getdents d3/d7/d20 0 2024-05-15T15:54:31.669 INFO:tasks.workunit.client.1.smithi144.stdout:0/993: stat d4/df/d1f/c3d 0 2024-05-15T15:54:31.671 INFO:tasks.workunit.client.0.smithi049.stdout:9/788: dread d0/d8/d9/d52/d5e/d5f/d56/fa7 [0,4194304] 0 2024-05-15T15:54:31.676 INFO:tasks.workunit.client.0.smithi049.stdout:0/934: creat dc/d13/d14/d25/d31/d71/f13c x:0 0 0 2024-05-15T15:54:31.680 INFO:tasks.workunit.client.0.smithi049.stdout:9/789: truncate d0/d8/d1b/f3a 1474475 0 2024-05-15T15:54:31.681 INFO:tasks.workunit.client.0.smithi049.stdout:0/935: mkdir dc/d13/d14/d2f/df5/d13d 0 2024-05-15T15:54:31.682 INFO:tasks.workunit.client.1.smithi144.stdout:0/994: dwrite d4/d6/fdf [0,4194304] 0 2024-05-15T15:54:31.682 INFO:tasks.workunit.client.1.smithi144.stdout:0/995: fdatasync d4/df/da0/fa8 0 2024-05-15T15:54:31.683 INFO:tasks.workunit.client.1.smithi144.stdout:0/996: chown d4/d2d/dcf 0 1 2024-05-15T15:54:31.683 INFO:tasks.workunit.client.1.smithi144.stdout:0/997: chown d4/df/d1f/f58 25 1 2024-05-15T15:54:31.684 INFO:tasks.workunit.client.0.smithi049.stdout:0/936: stat dc/d13/d14/d2f/d8f 0 2024-05-15T15:54:31.684 INFO:tasks.workunit.client.1.smithi144.stdout:2/970: rename d3/d7/da4/f11f to d3/d7/d20/d37/d76/db5/f144 0 2024-05-15T15:54:31.684 INFO:tasks.workunit.client.0.smithi049.stdout:8/858: dwrite d2/de/d71/d86/d5d/d7d/d1d/f61 [0,4194304] 0 2024-05-15T15:54:31.685 INFO:tasks.workunit.client.0.smithi049.stdout:9/790: write d0/d8/d42/d75/de7/ffb [439502,56782] 0 2024-05-15T15:54:31.687 INFO:tasks.workunit.client.1.smithi144.stdout:0/998: creat d4/d2d/d100/f16a x:0 0 0 2024-05-15T15:54:31.690 INFO:tasks.workunit.client.0.smithi049.stdout:9/791: stat d0/d8/d9/d52/d5e/d8a/dd3/d7d/f10e 0 2024-05-15T15:54:31.693 INFO:tasks.workunit.client.0.smithi049.stdout:8/859: chown d2/de/d71/d86/d5d/d7d/db8/c90 8670020 1 2024-05-15T15:54:31.693 INFO:tasks.workunit.client.0.smithi049.stdout:2/808: dread d3/db/d1d/d1e/d2b/d2e/f87 [0,4194304] 0 2024-05-15T15:54:31.697 INFO:tasks.workunit.client.0.smithi049.stdout:0/937: dread dc/d13/d14/d25/d31/d71/fc7 [0,4194304] 0 2024-05-15T15:54:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:31 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:31 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:31 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:54:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:31 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:31 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:31.702 INFO:tasks.workunit.client.1.smithi144.stdout:2/971: dread d3/d4d/fae [0,4194304] 0 2024-05-15T15:54:31.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:31 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:31.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:31 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:31.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:31 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:54:31.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:31 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:31.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:31 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:31.706 INFO:tasks.workunit.client.0.smithi049.stdout:0/938: symlink dc/d13/d2b/d76/d8a/d12f/l13e 0 2024-05-15T15:54:31.706 INFO:tasks.workunit.client.1.smithi144.stdout:0/999: link c3 d4/d6/d3a/d5d/c16b 0 2024-05-15T15:54:31.707 INFO:tasks.workunit.client.0.smithi049.stdout:0/939: dread - dc/d13/d14/d2f/d8f/dcb/f12c zero size 2024-05-15T15:54:31.708 INFO:tasks.workunit.client.1.smithi144.stdout:2/972: dwrite d3/d10/f134 [0,4194304] 0 2024-05-15T15:54:31.708 INFO:tasks.workunit.client.0.smithi049.stdout:2/809: creat d3/d78/d8c/d91/f115 x:0 0 0 2024-05-15T15:54:31.712 INFO:tasks.workunit.client.1.smithi144.stdout:2/973: rmdir d3/d7/da4 39 2024-05-15T15:54:31.712 INFO:tasks.workunit.client.0.smithi049.stdout:9/792: write d0/d8/d9/fb [29190,21286] 0 2024-05-15T15:54:31.717 INFO:tasks.workunit.client.0.smithi049.stdout:0/940: dwrite dc/d32/d93/f88 [0,4194304] 0 2024-05-15T15:54:31.722 INFO:tasks.workunit.client.0.smithi049.stdout:2/810: rename d3/db/d1d/d1e/d2b/d3b/d47/d9a to d3/d52/d116 0 2024-05-15T15:54:31.725 INFO:tasks.workunit.client.0.smithi049.stdout:9/793: creat d0/d8/d21/f118 x:0 0 0 2024-05-15T15:54:31.727 INFO:tasks.workunit.client.0.smithi049.stdout:0/941: creat dc/da4/db4/f13f x:0 0 0 2024-05-15T15:54:31.729 INFO:tasks.workunit.client.0.smithi049.stdout:0/942: dread - dc/d13/d14/d2f/d8f/dcb/de2/d103/f11c zero size 2024-05-15T15:54:31.729 INFO:tasks.workunit.client.0.smithi049.stdout:2/811: symlink d3/db/d5b/dbf/dc1/l117 0 2024-05-15T15:54:31.729 INFO:tasks.workunit.client.1.smithi144.stdout:2/974: dread d3/d7/d20/d22/d52/d64/f5f [0,4194304] 0 2024-05-15T15:54:31.730 INFO:tasks.workunit.client.1.smithi144.stdout:2/975: fdatasync d3/d7/d20/d22/ffd 0 2024-05-15T15:54:31.730 INFO:tasks.workunit.client.0.smithi049.stdout:8/860: link d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fbc d2/de/d71/d86/d5d/d7d/db8/d5b/d60/f10d 0 2024-05-15T15:54:31.735 INFO:tasks.workunit.client.0.smithi049.stdout:9/794: symlink d0/d8/d9/d52/d5e/l119 0 2024-05-15T15:54:31.737 INFO:tasks.workunit.client.0.smithi049.stdout:8/861: rmdir d2/de/d71 39 2024-05-15T15:54:31.739 INFO:tasks.workunit.client.1.smithi144.stdout:2/976: mkdir d3/d7/d20/d22/d145 0 2024-05-15T15:54:31.741 INFO:tasks.workunit.client.0.smithi049.stdout:9/795: fdatasync d0/d8/dd6/de2/dec/f101 0 2024-05-15T15:54:31.743 INFO:tasks.workunit.client.0.smithi049.stdout:2/812: write d3/db/d1d/d1e/d2b/d6b/f80 [1535927,42307] 0 2024-05-15T15:54:31.746 INFO:tasks.workunit.client.0.smithi049.stdout:2/813: symlink d3/d85/l118 0 2024-05-15T15:54:31.747 INFO:tasks.workunit.client.0.smithi049.stdout:0/943: link dc/d13/c11f dc/d13/d14/d2f/df5/c140 0 2024-05-15T15:54:31.748 INFO:tasks.workunit.client.0.smithi049.stdout:8/862: mkdir d2/de/d71/d86/d5d/de3/d10e 0 2024-05-15T15:54:31.749 INFO:tasks.workunit.client.0.smithi049.stdout:2/814: dread - d3/da/d15/d24/ffb zero size 2024-05-15T15:54:31.750 INFO:tasks.workunit.client.1.smithi144.stdout:2/977: dread d3/f6c [0,4194304] 0 2024-05-15T15:54:31.751 INFO:tasks.workunit.client.1.smithi144.stdout:2/978: chown d3/d4d/d103/ld8 370605996 1 2024-05-15T15:54:31.751 INFO:tasks.workunit.client.0.smithi049.stdout:9/796: dwrite d0/d8/d21/f118 [0,4194304] 0 2024-05-15T15:54:31.752 INFO:tasks.workunit.client.0.smithi049.stdout:8/863: chown d2/de/d95/fd7 25 1 2024-05-15T15:54:31.753 INFO:tasks.workunit.client.0.smithi049.stdout:0/944: creat dc/d13/d56/d6f/dce/f141 x:0 0 0 2024-05-15T15:54:31.754 INFO:tasks.workunit.client.0.smithi049.stdout:0/945: dread - dc/d13/d3d/f125 zero size 2024-05-15T15:54:31.754 INFO:tasks.workunit.client.1.smithi144.stdout:2/979: write d3/d7/df8/da6/f13b [906783,80754] 0 2024-05-15T15:54:31.755 INFO:tasks.workunit.client.0.smithi049.stdout:0/946: chown dc/d13/d2b/f4d 12837 1 2024-05-15T15:54:31.761 INFO:tasks.workunit.client.1.smithi144.stdout:2/980: creat d3/d7/d54/f146 x:0 0 0 2024-05-15T15:54:31.762 INFO:tasks.workunit.client.0.smithi049.stdout:9/797: write d0/d8/d9/d22/f2e [943593,109443] 0 2024-05-15T15:54:31.769 INFO:tasks.workunit.client.0.smithi049.stdout:2/815: getdents d3/db/d5b/dbf 0 2024-05-15T15:54:31.770 INFO:tasks.workunit.client.0.smithi049.stdout:8/864: dwrite d2/de/dc3/fc4 [0,4194304] 0 2024-05-15T15:54:31.773 INFO:tasks.workunit.client.1.smithi144.stdout:2/981: dwrite d3/d7/d20/d22/d52/d64/f31 [0,4194304] 0 2024-05-15T15:54:31.773 INFO:tasks.workunit.client.1.smithi144.stdout:2/982: readlink d3/d4d/db4/l10b 0 2024-05-15T15:54:31.776 INFO:tasks.workunit.client.0.smithi049.stdout:0/947: link dc/d13/d56/d6f/dce/c111 dc/d13/d2b/d76/d8a/d12f/c142 0 2024-05-15T15:54:31.782 INFO:tasks.workunit.client.1.smithi144.stdout:2/983: creat d3/d7/d20/def/d112/f147 x:0 0 0 2024-05-15T15:54:31.783 INFO:tasks.workunit.client.0.smithi049.stdout:2/816: creat d3/d52/f119 x:0 0 0 2024-05-15T15:54:31.785 INFO:tasks.workunit.client.0.smithi049.stdout:8/865: mknod d2/de/d71/d86/d5d/d7d/d1b/d68/dee/c10f 0 2024-05-15T15:54:31.787 INFO:tasks.workunit.client.1.smithi144.stdout:2/984: rename d3/d7/c30 to d3/d7/d11a/c148 0 2024-05-15T15:54:31.788 INFO:tasks.workunit.client.1.smithi144.stdout:2/985: write d3/d4d/db4/f121 [137780,102529] 0 2024-05-15T15:54:31.791 INFO:tasks.workunit.client.0.smithi049.stdout:9/798: getdents d0/d8/d1b/db3 0 2024-05-15T15:54:31.793 INFO:tasks.workunit.client.0.smithi049.stdout:8/866: unlink d2/de/d71/d86/d5d/d7d/db8/d5b/d60/f9b 0 2024-05-15T15:54:31.796 INFO:tasks.workunit.client.1.smithi144.stdout:2/986: symlink d3/d7/d20/d40/d126/l149 0 2024-05-15T15:54:31.799 INFO:tasks.workunit.client.0.smithi049.stdout:9/799: read d0/d8/d1b/f28 [938693,40472] 0 2024-05-15T15:54:31.801 INFO:tasks.workunit.client.0.smithi049.stdout:0/948: write dc/d13/d14/d25/d31/d71/f3a [278755,19215] 0 2024-05-15T15:54:31.802 INFO:tasks.workunit.client.0.smithi049.stdout:0/949: readlink dc/d6d/l44 0 2024-05-15T15:54:31.803 INFO:tasks.workunit.client.1.smithi144.stdout:2/987: mkdir d3/d7/d20/d37/d76/db5/d14a 0 2024-05-15T15:54:31.804 INFO:tasks.workunit.client.1.smithi144.stdout:2/988: write d3/d7/df8/f128 [795744,60825] 0 2024-05-15T15:54:31.805 INFO:tasks.workunit.client.1.smithi144.stdout:2/989: write d3/d4d/db4/f121 [1631101,52175] 0 2024-05-15T15:54:31.808 INFO:tasks.workunit.client.0.smithi049.stdout:2/817: dwrite d3/db/d1d/d1e/d2b/d3b/d42/f6f [0,4194304] 0 2024-05-15T15:54:31.810 INFO:tasks.workunit.client.0.smithi049.stdout:8/867: write d2/f2c [2457573,58137] 0 2024-05-15T15:54:31.811 INFO:tasks.workunit.client.1.smithi144.stdout:2/990: dwrite d3/d7/d54/fa0 [0,4194304] 0 2024-05-15T15:54:31.812 INFO:tasks.workunit.client.1.smithi144.stdout:2/991: chown d3/d7/d54/d98 357108 1 2024-05-15T15:54:31.812 INFO:tasks.workunit.client.1.smithi144.stdout:2/992: fsync d3/d7/d20/d89/f13f 0 2024-05-15T15:54:31.813 INFO:tasks.workunit.client.0.smithi049.stdout:0/950: read - dc/d13/d14/d25/d31/d71/dbf/fe4 zero size 2024-05-15T15:54:31.814 INFO:tasks.workunit.client.0.smithi049.stdout:9/800: link d0/d8/d9/fa d0/d8/d9/f11a 0 2024-05-15T15:54:31.816 INFO:tasks.workunit.client.0.smithi049.stdout:0/951: readlink dc/d13/d14/d25/d31/d71/l5f 0 2024-05-15T15:54:31.817 INFO:tasks.workunit.client.0.smithi049.stdout:8/868: creat d2/d6a/f110 x:0 0 0 2024-05-15T15:54:31.818 INFO:tasks.workunit.client.0.smithi049.stdout:9/801: mkdir d0/d8/d9/d22/dbb/d11b 0 2024-05-15T15:54:31.819 INFO:tasks.workunit.client.0.smithi049.stdout:8/869: chown d2/de/d71/d86/fcf 877 1 2024-05-15T15:54:31.823 INFO:tasks.workunit.client.0.smithi049.stdout:9/802: symlink d0/d8/d9/d22/dbb/l11c 0 2024-05-15T15:54:31.825 INFO:tasks.workunit.client.0.smithi049.stdout:0/952: dread dc/d13/d14/d25/d31/d71/fc7 [0,4194304] 0 2024-05-15T15:54:31.826 INFO:tasks.workunit.client.0.smithi049.stdout:8/870: symlink d2/de/d71/d86/d5d/d7d/db8/d5b/d93/l111 0 2024-05-15T15:54:31.826 INFO:tasks.workunit.client.0.smithi049.stdout:9/803: symlink d0/d25/l11d 0 2024-05-15T15:54:31.828 INFO:tasks.workunit.client.0.smithi049.stdout:0/953: truncate dc/d13/d56/f90 2122021 0 2024-05-15T15:54:31.832 INFO:tasks.workunit.client.0.smithi049.stdout:9/804: creat d0/d8/d9/d52/d5e/f11e x:0 0 0 2024-05-15T15:54:31.832 INFO:tasks.workunit.client.0.smithi049.stdout:8/871: dread d2/de/d71/d86/d5d/d7d/d1d/f61 [0,4194304] 0 2024-05-15T15:54:31.832 INFO:tasks.workunit.client.0.smithi049.stdout:0/954: mknod dc/da4/db4/c143 0 2024-05-15T15:54:31.834 INFO:tasks.workunit.client.0.smithi049.stdout:9/805: dread - d0/d8/d9/d52/d5e/d8a/dd3/d7d/f10e zero size 2024-05-15T15:54:31.835 INFO:tasks.workunit.client.0.smithi049.stdout:8/872: truncate d2/f3 5354591 0 2024-05-15T15:54:31.836 INFO:tasks.workunit.client.0.smithi049.stdout:8/873: chown d2/de/d25/d74/dbe 277698463 1 2024-05-15T15:54:31.837 INFO:tasks.workunit.client.0.smithi049.stdout:0/955: getdents dc/d13/d14/d25/d31/dae/dc5/d12b 0 2024-05-15T15:54:31.838 INFO:tasks.workunit.client.0.smithi049.stdout:0/956: chown dc/d13/d14/d2f/d8f/dcb/de2/c117 77305 1 2024-05-15T15:54:31.841 INFO:tasks.workunit.client.0.smithi049.stdout:9/806: rename d0/d8/dd6/de2/dec/l110 to d0/d8/d9/d52/d5e/d8a/l11f 0 2024-05-15T15:54:31.841 INFO:tasks.workunit.client.0.smithi049.stdout:2/818: write d3/da/d15/d73/fe3 [684933,100637] 0 2024-05-15T15:54:31.843 INFO:tasks.workunit.client.0.smithi049.stdout:8/874: mknod d2/de/d71/d86/d5d/d7d/d1b/d69/dac/c112 0 2024-05-15T15:54:31.845 INFO:tasks.workunit.client.0.smithi049.stdout:8/875: write d2/de/d71/d86/d5d/d7d/d1b/f107 [1667947,76655] 0 2024-05-15T15:54:31.846 INFO:tasks.workunit.client.0.smithi049.stdout:0/957: creat dc/d13/d14/d2f/df5/f144 x:0 0 0 2024-05-15T15:54:31.849 INFO:tasks.workunit.client.1.smithi144.stdout:2/993: dwrite d3/d7/d20/d3f/f53 [0,4194304] 0 2024-05-15T15:54:31.851 INFO:tasks.workunit.client.0.smithi049.stdout:8/876: unlink d2/de/d71/d86/d5d/d63/cb9 0 2024-05-15T15:54:31.859 INFO:tasks.workunit.client.0.smithi049.stdout:9/807: rename d0/d8/d9/d52/d5e/d5f/d56/f108 to d0/d8/f120 0 2024-05-15T15:54:31.861 INFO:tasks.workunit.client.0.smithi049.stdout:9/808: dread - d0/d8/d12/dfd/ffe zero size 2024-05-15T15:54:31.869 INFO:tasks.workunit.client.0.smithi049.stdout:2/819: rename d3/d85 to d3/d78/d8c/d11a 0 2024-05-15T15:54:31.871 INFO:tasks.workunit.client.1.smithi144.stdout:2/994: getdents d3 0 2024-05-15T15:54:31.872 INFO:tasks.workunit.client.1.smithi144.stdout:2/995: chown d3/d7/d20/d89/f13f 50 1 2024-05-15T15:54:31.872 INFO:tasks.workunit.client.0.smithi049.stdout:4/830: sync 2024-05-15T15:54:31.872 INFO:tasks.workunit.client.0.smithi049.stdout:7/871: sync 2024-05-15T15:54:31.872 INFO:tasks.workunit.client.0.smithi049.stdout:9/809: fdatasync d0/d8/d42/d75/fcf 0 2024-05-15T15:54:31.874 INFO:tasks.workunit.client.0.smithi049.stdout:4/831: read d18/d1e/d2a/d36/f4b [979101,88170] 0 2024-05-15T15:54:31.875 INFO:tasks.workunit.client.0.smithi049.stdout:2/820: creat d3/d52/d116/da3/dc9/f11b x:0 0 0 2024-05-15T15:54:31.877 INFO:tasks.workunit.client.0.smithi049.stdout:8/877: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d60/f113 x:0 0 0 2024-05-15T15:54:31.880 INFO:tasks.workunit.client.1.smithi144.stdout:2/996: unlink d3/d7/d54/d98/ff0 0 2024-05-15T15:54:31.881 INFO:tasks.workunit.client.0.smithi049.stdout:2/821: dread d3/db/d1d/d1e/d2b/d2e/fb7 [0,4194304] 0 2024-05-15T15:54:31.884 INFO:tasks.workunit.client.0.smithi049.stdout:9/810: fsync d0/d8/fd9 0 2024-05-15T15:54:31.885 INFO:tasks.workunit.client.1.smithi144.stdout:2/997: creat d3/d7/d20/d22/d52/d64/d97/f14b x:0 0 0 2024-05-15T15:54:31.887 INFO:tasks.workunit.client.0.smithi049.stdout:0/958: dwrite dc/d13/d3d/fe0 [0,4194304] 0 2024-05-15T15:54:31.891 INFO:tasks.workunit.client.1.smithi144.stdout:2/998: creat d3/d7/df8/f14c x:0 0 0 2024-05-15T15:54:31.894 INFO:tasks.workunit.client.0.smithi049.stdout:4/832: dread d18/d1e/d63/d99/de1/f57 [0,4194304] 0 2024-05-15T15:54:31.897 INFO:tasks.workunit.client.0.smithi049.stdout:4/833: write d18/d1e/d2a/d36/ff3 [3609937,87995] 0 2024-05-15T15:54:31.897 INFO:tasks.workunit.client.0.smithi049.stdout:4/834: chown d18/d1e/d2a/d50/f79 775717 1 2024-05-15T15:54:31.898 INFO:tasks.workunit.client.0.smithi049.stdout:7/872: dread d0/d2f/d5c/f70 [0,4194304] 0 2024-05-15T15:54:31.898 INFO:tasks.workunit.client.0.smithi049.stdout:4/835: fsync d18/d21/f56 0 2024-05-15T15:54:31.901 INFO:tasks.workunit.client.0.smithi049.stdout:4/836: write d18/d1e/d2a/d50/d54/d109/db6/ff1 [66166,74332] 0 2024-05-15T15:54:31.911 INFO:tasks.workunit.client.0.smithi049.stdout:8/878: dread d2/de/d71/f84 [0,4194304] 0 2024-05-15T15:54:31.915 INFO:tasks.workunit.client.0.smithi049.stdout:4/837: chown d18/f22 638 1 2024-05-15T15:54:31.916 INFO:tasks.workunit.client.1.smithi144.stdout:2/999: dwrite d3/d7/d20/d89/d86/fda [0,4194304] 0 2024-05-15T15:54:31.918 INFO:tasks.workunit.client.0.smithi049.stdout:8/879: stat d2/c4d 0 2024-05-15T15:54:31.921 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm -rf -- ./tmp.AHccOwf9uB 2024-05-15T15:54:31.923 INFO:tasks.workunit.client.0.smithi049.stdout:0/959: dwrite dc/d32/f96 [4194304,4194304] 0 2024-05-15T15:54:31.924 INFO:tasks.workunit.client.0.smithi049.stdout:2/822: write d3/da/d15/f27 [3368031,104552] 0 2024-05-15T15:54:31.926 INFO:tasks.workunit.client.0.smithi049.stdout:2/823: stat d3/db/d1d/d1e/d2b/d3b/lb6 0 2024-05-15T15:54:31.927 INFO:tasks.workunit.client.0.smithi049.stdout:0/960: dread - dc/d13/d14/d25/d31/dae/dc5/f13b zero size 2024-05-15T15:54:31.928 INFO:tasks.workunit.client.0.smithi049.stdout:0/961: fsync dc/d32/d93/f89 0 2024-05-15T15:54:31.938 INFO:tasks.workunit.client.0.smithi049.stdout:2/824: link d3/db/d82/fbc d3/d78/d8c/d11a/f11c 0 2024-05-15T15:54:31.940 INFO:tasks.workunit.client.0.smithi049.stdout:0/962: creat dc/d13/d14/d2f/df5/d13d/f145 x:0 0 0 2024-05-15T15:54:31.943 INFO:tasks.workunit.client.0.smithi049.stdout:2/825: chown d3/da/d15/d73/ca4 12580 1 2024-05-15T15:54:31.943 INFO:tasks.workunit.client.0.smithi049.stdout:4/838: dwrite d18/d1e/d63/d99/de1/f74 [0,4194304] 0 2024-05-15T15:54:31.944 INFO:tasks.workunit.client.0.smithi049.stdout:0/963: chown dc/d13/d14/d25/d31/dae/dc5/d108 5 1 2024-05-15T15:54:31.945 INFO:tasks.workunit.client.0.smithi049.stdout:2/826: dread - d3/db/d1d/d1e/d2b/d3b/ff7 zero size 2024-05-15T15:54:31.949 INFO:tasks.workunit.client.0.smithi049.stdout:2/827: rmdir d3/d8/dad 39 2024-05-15T15:54:31.950 INFO:tasks.workunit.client.0.smithi049.stdout:0/964: write dc/d13/d2b/fdb [588213,89790] 0 2024-05-15T15:54:31.955 INFO:tasks.workunit.client.0.smithi049.stdout:0/965: rename dc/f34 to dc/d13/d3d/f146 0 2024-05-15T15:54:31.959 INFO:tasks.workunit.client.0.smithi049.stdout:2/828: link d3/db/d1d/d1e/d2b/d3b/l114 d3/db/d1d/d1e/l11d 0 2024-05-15T15:54:31.960 INFO:tasks.workunit.client.0.smithi049.stdout:4/839: dread d18/f1a [0,4194304] 0 2024-05-15T15:54:31.960 INFO:tasks.workunit.client.0.smithi049.stdout:0/966: truncate dc/d13/d14/d25/d31/dae/dc5/f50 2339584 0 2024-05-15T15:54:31.961 INFO:tasks.workunit.client.0.smithi049.stdout:0/967: readlink dc/d6d/l78 0 2024-05-15T15:54:31.962 INFO:tasks.workunit.client.0.smithi049.stdout:2/829: creat d3/da/d15/d73/f11e x:0 0 0 2024-05-15T15:54:31.962 INFO:tasks.workunit.client.0.smithi049.stdout:0/968: fsync dc/d13/d14/d25/d31/dae/dc5/f13b 0 2024-05-15T15:54:31.963 INFO:tasks.workunit.client.0.smithi049.stdout:4/840: creat d18/d76/f110 x:0 0 0 2024-05-15T15:54:31.967 INFO:tasks.workunit.client.0.smithi049.stdout:2/830: fsync d3/db/f23 0 2024-05-15T15:54:31.971 INFO:tasks.workunit.client.0.smithi049.stdout:0/969: creat dc/d13/d14/d25/d31/f147 x:0 0 0 2024-05-15T15:54:31.975 INFO:tasks.workunit.client.0.smithi049.stdout:4/841: creat d18/d1e/d2a/f111 x:0 0 0 2024-05-15T15:54:31.979 INFO:tasks.workunit.client.0.smithi049.stdout:0/970: creat dc/d13/d14/dc9/f148 x:0 0 0 2024-05-15T15:54:31.985 INFO:tasks.workunit.client.0.smithi049.stdout:4/842: symlink d18/d21/d2b/d4e/l112 0 2024-05-15T15:54:31.985 INFO:tasks.workunit.client.0.smithi049.stdout:2/831: fsync d3/db/d1d/d1e/d2b/d6b/fd0 0 2024-05-15T15:54:32.010 INFO:tasks.workunit.client.0.smithi049.stdout:0/971: rmdir dc/d13/d14/d25/d31/dae/dc5/d57 39 2024-05-15T15:54:32.014 INFO:tasks.workunit.client.0.smithi049.stdout:4/843: read - d18/d1e/f78 zero size 2024-05-15T15:54:32.018 INFO:tasks.workunit.client.0.smithi049.stdout:4/844: dread d18/d1e/d63/d99/de1/f74 [0,4194304] 0 2024-05-15T15:54:32.019 INFO:tasks.workunit.client.0.smithi049.stdout:0/972: creat dc/d13/d2b/d76/f149 x:0 0 0 2024-05-15T15:54:32.021 INFO:tasks.workunit.client.0.smithi049.stdout:4/845: truncate d18/d1e/d2a/d50/dc4/ffe 435068 0 2024-05-15T15:54:32.025 INFO:tasks.workunit.client.0.smithi049.stdout:0/973: creat dc/f14a x:0 0 0 2024-05-15T15:54:32.028 INFO:tasks.workunit.client.0.smithi049.stdout:4/846: truncate d18/f22 3045913 0 2024-05-15T15:54:32.033 INFO:tasks.workunit.client.0.smithi049.stdout:0/974: symlink dc/d13/d2b/d76/d8a/da2/l14b 0 2024-05-15T15:54:32.038 INFO:tasks.workunit.client.0.smithi049.stdout:0/975: creat dc/d13/d14/d2f/d8f/dcb/de2/f14c x:0 0 0 2024-05-15T15:54:32.040 INFO:tasks.workunit.client.0.smithi049.stdout:4/847: truncate d18/d1e/f23 1048084 0 2024-05-15T15:54:32.044 INFO:tasks.workunit.client.0.smithi049.stdout:4/848: dread - d18/d1e/d2a/d50/dc4/ff5 zero size 2024-05-15T15:54:32.046 INFO:tasks.workunit.client.0.smithi049.stdout:4/849: chown d18/d1e/d2a/d50/d54/d109/d98/ddd 0 1 2024-05-15T15:54:32.048 INFO:tasks.workunit.client.0.smithi049.stdout:4/850: read d18/d1e/d63/d99/fa5 [2801115,113837] 0 2024-05-15T15:54:32.048 INFO:tasks.workunit.client.0.smithi049.stdout:4/851: chown d18/d21 63187784 1 2024-05-15T15:54:32.051 INFO:tasks.workunit.client.0.smithi049.stdout:4/852: creat d18/d1e/d63/d10d/f113 x:0 0 0 2024-05-15T15:54:32.052 INFO:tasks.workunit.client.0.smithi049.stdout:4/853: chown d18/d1e/d63/d99/de1/f47 7981 1 2024-05-15T15:54:32.054 INFO:tasks.workunit.client.0.smithi049.stdout:4/854: dread - d18/d1e/d2a/d50/d54/d109/fd7 zero size 2024-05-15T15:54:32.057 INFO:tasks.workunit.client.0.smithi049.stdout:4/855: mkdir d18/d21/d114 0 2024-05-15T15:54:32.064 INFO:tasks.workunit.client.0.smithi049.stdout:2/832: dwrite d3/db/d1d/d1e/f21 [0,4194304] 0 2024-05-15T15:54:32.067 INFO:tasks.workunit.client.0.smithi049.stdout:4/856: creat d18/d1e/d2a/d36/d86/f115 x:0 0 0 2024-05-15T15:54:32.067 INFO:tasks.workunit.client.0.smithi049.stdout:2/833: chown d3/db/d1d/d1e/d2b/d3b/d42/f102 0 1 2024-05-15T15:54:32.079 INFO:tasks.workunit.client.0.smithi049.stdout:0/976: dwrite dc/d13/d14/d2f/f24 [4194304,4194304] 0 2024-05-15T15:54:32.088 INFO:tasks.workunit.client.0.smithi049.stdout:0/977: dread dc/d13/d14/f79 [0,4194304] 0 2024-05-15T15:54:32.088 INFO:tasks.workunit.client.0.smithi049.stdout:3/906: sync 2024-05-15T15:54:32.088 INFO:tasks.workunit.client.0.smithi049.stdout:6/987: sync 2024-05-15T15:54:32.089 INFO:tasks.workunit.client.0.smithi049.stdout:6/988: chown dd/d26/d5b/cdd 179878364 1 2024-05-15T15:54:32.091 INFO:tasks.workunit.client.0.smithi049.stdout:6/989: chown dd/d17/d24/d58/d5f 906765 1 2024-05-15T15:54:32.091 INFO:tasks.workunit.client.0.smithi049.stdout:3/907: truncate d12/d67/da8/ddd/feb 926494 0 2024-05-15T15:54:32.093 INFO:tasks.workunit.client.0.smithi049.stdout:6/990: write dd/d26/d5b/d110/d50/d7e/f10c [709307,8026] 0 2024-05-15T15:54:32.250 INFO:tasks.workunit.client.0.smithi049.stdout:0/978: creat dc/d13/d14/d2f/df5/f14d x:0 0 0 2024-05-15T15:54:32.267 INFO:tasks.workunit.client.0.smithi049.stdout:2/834: getdents d3 0 2024-05-15T15:54:32.271 INFO:tasks.workunit.client.0.smithi049.stdout:9/811: sync 2024-05-15T15:54:32.272 INFO:tasks.workunit.client.0.smithi049.stdout:3/908: creat d12/d14/d32/d104/f117 x:0 0 0 2024-05-15T15:54:32.273 INFO:tasks.workunit.client.0.smithi049.stdout:9/812: fsync d0/d8/d9/d52/d5e/d5f/f38 0 2024-05-15T15:54:32.273 INFO:tasks.workunit.client.0.smithi049.stdout:3/909: write d12/d14/d32/f58 [772310,86444] 0 2024-05-15T15:54:32.275 INFO:tasks.workunit.client.0.smithi049.stdout:3/910: chown d12/d30/f3d 35386 1 2024-05-15T15:54:32.275 INFO:tasks.workunit.client.0.smithi049.stdout:9/813: write d0/d8/d1b/f98 [1124582,56269] 0 2024-05-15T15:54:32.276 INFO:tasks.workunit.client.0.smithi049.stdout:6/991: symlink dd/d26/d5b/d110/d50/d54/l136 0 2024-05-15T15:54:32.280 INFO:tasks.workunit.client.0.smithi049.stdout:4/857: getdents d18/d1e/d2a/d50/d54/dad 0 2024-05-15T15:54:32.285 INFO:tasks.workunit.client.0.smithi049.stdout:2/835: mkdir d3/d78/d8c/d91/d7a/d11f 0 2024-05-15T15:54:32.287 INFO:tasks.workunit.client.0.smithi049.stdout:4/858: dwrite d18/d1e/d2a/d36/f4a [0,4194304] 0 2024-05-15T15:54:32.290 INFO:tasks.workunit.client.0.smithi049.stdout:0/979: rename dc/d13/d14/d25/db9/lf4 to dc/d13/d14/d25/d31/dae/dc5/d57/l14e 0 2024-05-15T15:54:32.290 INFO:tasks.workunit.client.0.smithi049.stdout:0/980: dread - dc/d6d/f12a zero size 2024-05-15T15:54:32.292 INFO:tasks.workunit.client.0.smithi049.stdout:0/981: read dc/da4/fab [2451259,4601] 0 2024-05-15T15:54:32.295 INFO:tasks.workunit.client.0.smithi049.stdout:9/814: dread d0/d8/d9/d22/f2e [0,4194304] 0 2024-05-15T15:54:32.318 INFO:tasks.workunit.client.0.smithi049.stdout:4/859: mknod d18/d1e/d63/d99/db9/c116 0 2024-05-15T15:54:32.327 INFO:tasks.workunit.client.0.smithi049.stdout:8/880: sync 2024-05-15T15:54:32.327 INFO:tasks.workunit.client.0.smithi049.stdout:7/873: sync 2024-05-15T15:54:32.335 INFO:tasks.workunit.client.0.smithi049.stdout:2/836: mknod d3/db/d1d/d1e/d2b/d3b/df8/c120 0 2024-05-15T15:54:32.339 INFO:tasks.workunit.client.0.smithi049.stdout:0/982: symlink dc/d13/d56/d6f/l14f 0 2024-05-15T15:54:32.350 INFO:tasks.workunit.client.0.smithi049.stdout:2/837: creat d3/d78/d8c/d91/d7a/d11f/f121 x:0 0 0 2024-05-15T15:54:32.353 INFO:tasks.workunit.client.0.smithi049.stdout:0/983: symlink dc/da4/l150 0 2024-05-15T15:54:32.354 INFO:tasks.workunit.client.0.smithi049.stdout:0/984: dread - dc/d6d/dc3/ffb zero size 2024-05-15T15:54:32.360 INFO:tasks.workunit.client.0.smithi049.stdout:0/985: dwrite dc/d13/d2b/d76/d8a/da2/db3/f110 [0,4194304] 0 2024-05-15T15:54:32.368 INFO:tasks.workunit.client.0.smithi049.stdout:0/986: link dc/d13/d14/d2f/df5/f144 dc/d13/d14/d2f/d8f/dcb/de2/f151 0 2024-05-15T15:54:32.396 INFO:tasks.workunit.client.0.smithi049.stdout:9/815: creat d0/d8/d9/d52/d5e/d8a/dd3/d7d/f121 x:0 0 0 2024-05-15T15:54:32.408 INFO:tasks.workunit.client.0.smithi049.stdout:9/816: dread d0/d8/d12/f6e [0,4194304] 0 2024-05-15T15:54:32.411 INFO:tasks.workunit.client.0.smithi049.stdout:9/817: rmdir d0/d8/d42/d75/de7 39 2024-05-15T15:54:32.411 INFO:tasks.workunit.client.0.smithi049.stdout:9/818: dread - d0/d25/f76 zero size 2024-05-15T15:54:32.417 INFO:tasks.workunit.client.0.smithi049.stdout:9/819: link d0/f41 d0/d8/d53/dd2/df3/d10c/f122 0 2024-05-15T15:54:32.443 INFO:tasks.workunit.client.0.smithi049.stdout:3/911: write d12/d14/d52/db9/dc4/d68/fba [420664,85614] 0 2024-05-15T15:54:32.446 INFO:tasks.workunit.client.0.smithi049.stdout:3/912: mkdir d12/d14/d32/d64/d88/d9c/d118 0 2024-05-15T15:54:32.447 INFO:tasks.workunit.client.0.smithi049.stdout:3/913: fsync d12/d14/d52/db9/dc4/f100 0 2024-05-15T15:54:32.449 INFO:tasks.workunit.client.0.smithi049.stdout:6/992: dwrite dd/d26/d5b/d110/d50/dcf/fd5 [0,4194304] 0 2024-05-15T15:54:32.452 INFO:tasks.workunit.client.0.smithi049.stdout:6/993: mkdir dd/d26/d5b/d110/d86/d96/d137 0 2024-05-15T15:54:32.453 INFO:tasks.workunit.client.0.smithi049.stdout:3/914: truncate d12/d14/d52/db9/dc4/fbd 606479 0 2024-05-15T15:54:32.455 INFO:tasks.workunit.client.0.smithi049.stdout:6/994: chown dd/d17/d24/fff 301960 1 2024-05-15T15:54:32.462 INFO:tasks.workunit.client.0.smithi049.stdout:6/995: dwrite dd/d17/f8d [0,4194304] 0 2024-05-15T15:54:32.466 INFO:tasks.workunit.client.0.smithi049.stdout:6/996: mkdir dd/d17/d24/d58/d5f/dd7/d138 0 2024-05-15T15:54:32.466 INFO:tasks.workunit.client.0.smithi049.stdout:6/997: chown dd/d17/f6d 55 1 2024-05-15T15:54:32.468 INFO:tasks.workunit.client.0.smithi049.stdout:6/998: truncate dd/d17/d9d/f115 953201 0 2024-05-15T15:54:32.469 INFO:tasks.workunit.client.0.smithi049.stdout:6/999: read - dd/d26/d5b/d110/d50/f9a zero size 2024-05-15T15:54:32.475 INFO:tasks.workunit.client.0.smithi049.stdout:7/874: dwrite d0/ddb/dde/dca/d5e/dbd/ffb [8388608,4194304] 0 2024-05-15T15:54:32.477 INFO:tasks.workunit.client.0.smithi049.stdout:7/875: chown d0/d6/d23/d45/d6a/da1/d80 454 1 2024-05-15T15:54:32.484 INFO:tasks.workunit.client.0.smithi049.stdout:8/881: sync 2024-05-15T15:54:32.495 INFO:tasks.workunit.client.0.smithi049.stdout:3/915: unlink d12/f114 0 2024-05-15T15:54:32.497 INFO:tasks.workunit.client.0.smithi049.stdout:8/882: dread d2/de/d71/d86/d5d/d7d/d1b/f2b [0,4194304] 0 2024-05-15T15:54:32.498 INFO:tasks.workunit.client.0.smithi049.stdout:8/883: chown d2/de/d71/d86/d5d/d63/ff8 509019603 1 2024-05-15T15:54:32.499 INFO:tasks.workunit.client.0.smithi049.stdout:8/884: write d2/de/d71/d86/d5d/d7d/db8/ffc [459743,12695] 0 2024-05-15T15:54:32.502 INFO:tasks.workunit.client.0.smithi049.stdout:3/916: creat d12/d14/d52/db9/f119 x:0 0 0 2024-05-15T15:54:32.507 INFO:tasks.workunit.client.0.smithi049.stdout:2/838: symlink d3/db/d1d/d1e/d2b/d3b/l122 0 2024-05-15T15:54:32.511 INFO:tasks.workunit.client.0.smithi049.stdout:2/839: mkdir d3/db/d1d/d1e/d2b/d3b/df8/d123 0 2024-05-15T15:54:32.513 INFO:tasks.workunit.client.0.smithi049.stdout:2/840: rmdir d3/da/d15/dec 39 2024-05-15T15:54:32.521 INFO:tasks.workunit.client.0.smithi049.stdout:0/987: creat dc/d13/d14/d2f/d8f/dcb/f152 x:0 0 0 2024-05-15T15:54:32.526 INFO:tasks.workunit.client.0.smithi049.stdout:7/876: mkdir d0/ddb/dde/dca/d115 0 2024-05-15T15:54:32.527 INFO:tasks.workunit.client.0.smithi049.stdout:9/820: truncate d0/d8/d9/d52/d5e/d8a/dd3/d7d/fea 168839 0 2024-05-15T15:54:32.529 INFO:tasks.workunit.client.0.smithi049.stdout:7/877: mknod d0/d6/d1d/df2/c116 0 2024-05-15T15:54:32.531 INFO:tasks.workunit.client.0.smithi049.stdout:4/860: unlink d18/c105 0 2024-05-15T15:54:32.536 INFO:tasks.workunit.client.0.smithi049.stdout:0/988: dread dc/d13/f17 [4194304,4194304] 0 2024-05-15T15:54:32.536 INFO:tasks.workunit.client.0.smithi049.stdout:4/861: truncate d18/f22 222004 0 2024-05-15T15:54:32.540 INFO:tasks.workunit.client.0.smithi049.stdout:0/989: creat dc/d13/d14/d25/db9/dd0/f153 x:0 0 0 2024-05-15T15:54:32.541 INFO:tasks.workunit.client.0.smithi049.stdout:0/990: stat dc/d6d/dc3/fe6 0 2024-05-15T15:54:32.544 INFO:tasks.workunit.client.0.smithi049.stdout:3/917: truncate d12/d14/f1e 1159549 0 2024-05-15T15:54:32.547 INFO:tasks.workunit.client.0.smithi049.stdout:0/991: dwrite dc/d13/d2b/fdb [0,4194304] 0 2024-05-15T15:54:32.550 INFO:tasks.workunit.client.0.smithi049.stdout:2/841: mkdir d3/d78/d124 0 2024-05-15T15:54:32.551 INFO:tasks.workunit.client.0.smithi049.stdout:3/918: getdents d12/d30 0 2024-05-15T15:54:32.553 INFO:tasks.workunit.client.0.smithi049.stdout:0/992: mknod dc/da4/db4/df1/d119/c154 0 2024-05-15T15:54:32.556 INFO:tasks.workunit.client.0.smithi049.stdout:3/919: symlink d12/d14/d52/db9/d7a/l11a 0 2024-05-15T15:54:32.557 INFO:tasks.workunit.client.0.smithi049.stdout:2/842: dread d3/da/d15/d73/f7f [0,4194304] 0 2024-05-15T15:54:32.557 INFO:tasks.workunit.client.0.smithi049.stdout:3/920: chown d12/d67/da8/caf 13 1 2024-05-15T15:54:32.558 INFO:tasks.workunit.client.0.smithi049.stdout:0/993: truncate dc/fa1 612038 0 2024-05-15T15:54:32.560 INFO:tasks.workunit.client.0.smithi049.stdout:0/994: dread dc/d13/f91 [0,4194304] 0 2024-05-15T15:54:32.563 INFO:tasks.workunit.client.0.smithi049.stdout:2/843: mkdir d3/db/d5b/d125 0 2024-05-15T15:54:32.564 INFO:tasks.workunit.client.0.smithi049.stdout:2/844: chown d3/db/d1d 1186860345 1 2024-05-15T15:54:32.565 INFO:tasks.workunit.client.0.smithi049.stdout:8/885: dwrite d2/de/d71/d86/d5d/d7d/d1b/d69/fde [4194304,4194304] 0 2024-05-15T15:54:32.571 INFO:tasks.workunit.client.0.smithi049.stdout:0/995: fsync dc/d13/d14/d25/d31/dae/dc5/f6c 0 2024-05-15T15:54:32.572 INFO:tasks.workunit.client.0.smithi049.stdout:0/996: chown dc/d13/d56/d6f/dce/def/cf6 16 1 2024-05-15T15:54:32.576 INFO:tasks.workunit.client.0.smithi049.stdout:9/821: write d0/d8/d1b/d40/f58 [775668,51718] 0 2024-05-15T15:54:32.580 INFO:tasks.workunit.client.0.smithi049.stdout:4/862: write d18/fd9 [1173566,120288] 0 2024-05-15T15:54:32.581 INFO:tasks.workunit.client.0.smithi049.stdout:7/878: dwrite d0/d6/d23/fe5 [0,4194304] 0 2024-05-15T15:54:32.583 INFO:tasks.workunit.client.0.smithi049.stdout:8/886: rename d2/de/fa3 to d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/f114 0 2024-05-15T15:54:32.589 INFO:tasks.workunit.client.0.smithi049.stdout:2/845: creat d3/d8/dad/f126 x:0 0 0 2024-05-15T15:54:32.591 INFO:tasks.workunit.client.0.smithi049.stdout:7/879: dwrite d0/d6/da/f61 [0,4194304] 0 2024-05-15T15:54:32.596 INFO:tasks.workunit.client.0.smithi049.stdout:7/880: chown d0/d6/da/f61 795239919 1 2024-05-15T15:54:32.596 INFO:tasks.workunit.client.0.smithi049.stdout:7/881: stat d0/d6/d23/ffa 0 2024-05-15T15:54:32.597 INFO:tasks.workunit.client.0.smithi049.stdout:9/822: read d0/d8/d12/f59 [528461,6857] 0 2024-05-15T15:54:32.603 INFO:tasks.workunit.client.0.smithi049.stdout:8/887: fdatasync d2/de/d71/f84 0 2024-05-15T15:54:32.604 INFO:tasks.workunit.client.0.smithi049.stdout:8/888: chown d2/de/d71/d86/f102 0 1 2024-05-15T15:54:32.605 INFO:tasks.workunit.client.0.smithi049.stdout:3/921: truncate f7 741143 0 2024-05-15T15:54:32.607 INFO:tasks.workunit.client.0.smithi049.stdout:2/846: creat d3/da/d15/d24/d107/f127 x:0 0 0 2024-05-15T15:54:32.608 INFO:tasks.workunit.client.0.smithi049.stdout:7/882: mknod d0/d2f/d5c/c117 0 2024-05-15T15:54:32.610 INFO:tasks.workunit.client.0.smithi049.stdout:7/883: chown d0/ddb/dde/dcc/fa8 102 1 2024-05-15T15:54:32.613 INFO:tasks.workunit.client.0.smithi049.stdout:4/863: rename f10 to d18/d1e/d63/d99/f117 0 2024-05-15T15:54:32.614 INFO:tasks.workunit.client.0.smithi049.stdout:3/922: dwrite d12/d14/d52/db9/dc4/d68/d70/d75/dac/fe8 [0,4194304] 0 2024-05-15T15:54:32.617 INFO:tasks.workunit.client.0.smithi049.stdout:3/923: chown d12/d14/d52/db9/dc4/d68/f9b 1752092 1 2024-05-15T15:54:32.623 INFO:tasks.workunit.client.0.smithi049.stdout:4/864: dwrite d18/d1e/d2a/d36/ff3 [8388608,4194304] 0 2024-05-15T15:54:32.628 INFO:tasks.workunit.client.0.smithi049.stdout:3/924: dwrite d12/d14/d52/db9/dc4/d68/f91 [0,4194304] 0 2024-05-15T15:54:32.630 INFO:tasks.workunit.client.0.smithi049.stdout:7/884: mknod d0/ddb/dde/dca/d5e/c118 0 2024-05-15T15:54:32.634 INFO:tasks.workunit.client.0.smithi049.stdout:7/885: readlink d0/d6/d23/d45/d6a/da1/l78 0 2024-05-15T15:54:32.636 INFO:tasks.workunit.client.0.smithi049.stdout:8/889: mkdir d2/de/d71/d86/d5d/de3/d10e/d115 0 2024-05-15T15:54:32.639 INFO:tasks.workunit.client.0.smithi049.stdout:0/997: dwrite dc/d13/d14/d25/d31/d71/fc1 [0,4194304] 0 2024-05-15T15:54:32.647 INFO:tasks.workunit.client.0.smithi049.stdout:9/823: write d0/d8/d9/d22/f96 [917009,99325] 0 2024-05-15T15:54:32.651 INFO:tasks.workunit.client.0.smithi049.stdout:9/824: truncate d0/d8/d9/d22/dbb/f103 63018 0 2024-05-15T15:54:32.660 INFO:tasks.workunit.client.0.smithi049.stdout:2/847: rename d3/db/d1d/l53 to d3/db/l128 0 2024-05-15T15:54:32.666 INFO:tasks.workunit.client.0.smithi049.stdout:2/848: dread d3/db/d1d/d1e/d2b/d3b/f76 [0,4194304] 0 2024-05-15T15:54:32.666 INFO:tasks.workunit.client.0.smithi049.stdout:7/886: creat d0/d6/d23/f119 x:0 0 0 2024-05-15T15:54:32.668 INFO:tasks.workunit.client.0.smithi049.stdout:4/865: write d18/d21/d2b/fb2 [373467,13255] 0 2024-05-15T15:54:32.672 INFO:tasks.workunit.client.0.smithi049.stdout:3/925: link d12/d67/c109 d12/d14/c11b 0 2024-05-15T15:54:32.672 INFO:tasks.workunit.client.0.smithi049.stdout:0/998: dwrite dc/d13/d3d/f43 [0,4194304] 0 2024-05-15T15:54:32.674 INFO:tasks.workunit.client.0.smithi049.stdout:0/999: chown dc/d13/d56/d6f/cc2 1422093 1 2024-05-15T15:54:32.675 INFO:tasks.workunit.client.0.smithi049.stdout:8/890: dread d2/de/d71/d86/fcf [0,4194304] 0 2024-05-15T15:54:32.677 INFO:tasks.workunit.client.0.smithi049.stdout:2/849: creat d3/d78/d8c/f129 x:0 0 0 2024-05-15T15:54:32.679 INFO:tasks.workunit.client.0.smithi049.stdout:9/825: link d0/d8/d9/d22/f96 d0/d8/d9/d52/d5e/d5f/f123 0 2024-05-15T15:54:32.681 INFO:tasks.workunit.client.0.smithi049.stdout:4/866: mknod d18/d1e/d2a/d50/dc4/c118 0 2024-05-15T15:54:32.682 INFO:tasks.workunit.client.0.smithi049.stdout:3/926: fdatasync d12/d14/d52/db9/da6/f97 0 2024-05-15T15:54:32.687 INFO:tasks.workunit.client.0.smithi049.stdout:9/826: fdatasync d0/d8/d9/d22/f96 0 2024-05-15T15:54:32.687 INFO:tasks.workunit.client.0.smithi049.stdout:2/850: dwrite d3/db/d1d/d1e/f21 [0,4194304] 0 2024-05-15T15:54:32.694 INFO:tasks.workunit.client.0.smithi049.stdout:3/927: dwrite d12/d14/d32/d104/f117 [0,4194304] 0 2024-05-15T15:54:32.696 INFO:tasks.workunit.client.0.smithi049.stdout:3/928: write d12/d14/d52/db9/f8e [1811423,52694] 0 2024-05-15T15:54:32.697 INFO:tasks.workunit.client.0.smithi049.stdout:2/851: truncate d3/da/f45 411961 0 2024-05-15T15:54:32.697 INFO:tasks.workunit.client.0.smithi049.stdout:3/929: chown d12/d14/f27 5295 1 2024-05-15T15:54:32.698 INFO:tasks.workunit.client.0.smithi049.stdout:2/852: stat d3/db/d1d/d1e/d2b/d2e/cbe 0 2024-05-15T15:54:32.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:32 smithi144 ceph-mon[36389]: pgmap v36: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 73 MiB/s rd, 177 MiB/s wr, 437 op/s 2024-05-15T15:54:32.700 INFO:tasks.workunit.client.0.smithi049.stdout:9/827: truncate d0/d8/d9/d45/f4a 3894647 0 2024-05-15T15:54:32.700 INFO:tasks.workunit.client.0.smithi049.stdout:9/828: dread - d0/d8/d1b/f104 zero size 2024-05-15T15:54:32.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:32 smithi049 ceph-mon[28399]: pgmap v36: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 73 MiB/s rd, 177 MiB/s wr, 437 op/s 2024-05-15T15:54:32.707 INFO:tasks.workunit.client.0.smithi049.stdout:2/853: mkdir d3/db/d1d/d1e/d2b/d3b/df8/d12a 0 2024-05-15T15:54:32.707 INFO:tasks.workunit.client.0.smithi049.stdout:7/887: write d0/d6/d23/f40 [1407254,99324] 0 2024-05-15T15:54:32.708 INFO:tasks.workunit.client.0.smithi049.stdout:9/829: symlink d0/d8/d9/d52/d5e/d5f/d56/l124 0 2024-05-15T15:54:32.711 INFO:tasks.workunit.client.0.smithi049.stdout:4/867: write d18/d1e/d63/d99/de1/f9c [1472005,82805] 0 2024-05-15T15:54:32.717 INFO:tasks.workunit.client.0.smithi049.stdout:9/830: rename d0/d8/d9/d52/d5e/d8a/dd3/d7d/dba/f10d to d0/d8/d21/ded/f125 0 2024-05-15T15:54:32.717 INFO:tasks.workunit.client.0.smithi049.stdout:3/930: getdents d12/d26/d73 0 2024-05-15T15:54:32.721 INFO:tasks.workunit.client.0.smithi049.stdout:3/931: fsync d12/d14/d52/db9/dc4/d68/d70/d75/dac/fe8 0 2024-05-15T15:54:32.723 INFO:tasks.workunit.client.0.smithi049.stdout:3/932: chown d12/d14/d52/db9/d7a/la4 0 1 2024-05-15T15:54:32.725 INFO:tasks.workunit.client.0.smithi049.stdout:3/933: mkdir d12/d14/d32/d64/d11c 0 2024-05-15T15:54:32.733 INFO:tasks.workunit.client.0.smithi049.stdout:9/831: dread d0/d8/d1b/d40/f58 [0,4194304] 0 2024-05-15T15:54:32.736 INFO:tasks.workunit.client.0.smithi049.stdout:2/854: dwrite d3/db/d1d/d1e/d2b/d3b/d42/f4f [0,4194304] 0 2024-05-15T15:54:32.738 INFO:tasks.workunit.client.0.smithi049.stdout:7/888: dwrite d0/d6/f14 [0,4194304] 0 2024-05-15T15:54:32.743 INFO:tasks.workunit.client.0.smithi049.stdout:2/855: read d3/db/d1d/fe0 [204292,62511] 0 2024-05-15T15:54:32.749 INFO:tasks.workunit.client.0.smithi049.stdout:3/934: dwrite d12/d14/d52/db9/f6e [0,4194304] 0 2024-05-15T15:54:32.754 INFO:tasks.workunit.client.0.smithi049.stdout:2/856: mknod d3/d78/c12b 0 2024-05-15T15:54:32.756 INFO:tasks.workunit.client.0.smithi049.stdout:2/857: readlink d3/db/d1d/d1e/d2b/d6b/l98 0 2024-05-15T15:54:32.759 INFO:tasks.workunit.client.0.smithi049.stdout:2/858: truncate d3/db/d1d/d1e/d2b/d3b/fa7 5076161 0 2024-05-15T15:54:32.760 INFO:tasks.workunit.client.0.smithi049.stdout:3/935: rename d12/l2b to d12/d26/d73/l11d 0 2024-05-15T15:54:32.760 INFO:tasks.workunit.client.0.smithi049.stdout:7/889: link d0/fc8 d0/d6/d1d/df2/f11a 0 2024-05-15T15:54:32.765 INFO:tasks.workunit.client.0.smithi049.stdout:3/936: mknod d12/d30/dc1/dc7/df8/dcf/c11e 0 2024-05-15T15:54:32.766 INFO:tasks.workunit.client.0.smithi049.stdout:3/937: write d12/d14/f54 [1102930,79093] 0 2024-05-15T15:54:32.770 INFO:tasks.workunit.client.0.smithi049.stdout:2/859: creat d3/db/d1d/d1e/d2b/d3b/df8/d12a/f12c x:0 0 0 2024-05-15T15:54:32.778 INFO:tasks.workunit.client.0.smithi049.stdout:9/832: write d0/d8/d53/faa [1843565,66973] 0 2024-05-15T15:54:32.779 INFO:tasks.workunit.client.0.smithi049.stdout:7/890: link d0/d6/d23/d45/d86/fc5 d0/d2f/d5c/f11b 0 2024-05-15T15:54:32.783 INFO:tasks.workunit.client.0.smithi049.stdout:9/833: creat d0/d8/d1b/d40/d5c/d69/f126 x:0 0 0 2024-05-15T15:54:32.784 INFO:tasks.workunit.client.0.smithi049.stdout:9/834: chown d0/d8/d1b 886343 1 2024-05-15T15:54:32.785 INFO:tasks.workunit.client.0.smithi049.stdout:9/835: write d0/d8/d21/fff [727709,92596] 0 2024-05-15T15:54:32.788 INFO:tasks.workunit.client.0.smithi049.stdout:2/860: dread d3/db/d1d/d1e/d2b/f54 [0,4194304] 0 2024-05-15T15:54:32.791 INFO:tasks.workunit.client.0.smithi049.stdout:9/836: rmdir d0/d8/d12/d9a 39 2024-05-15T15:54:32.792 INFO:tasks.workunit.client.0.smithi049.stdout:9/837: stat d0/d8/d9/d22/lc6 0 2024-05-15T15:54:32.795 INFO:tasks.workunit.client.0.smithi049.stdout:2/861: creat d3/da/da9/f12d x:0 0 0 2024-05-15T15:54:32.798 INFO:tasks.workunit.client.0.smithi049.stdout:9/838: rmdir d0/d8/dd6/de2/dec 39 2024-05-15T15:54:32.801 INFO:tasks.workunit.client.0.smithi049.stdout:3/938: write d12/f25 [1494782,112343] 0 2024-05-15T15:54:32.807 INFO:tasks.workunit.client.0.smithi049.stdout:7/891: write d0/d6/f6f [1032183,27626] 0 2024-05-15T15:54:32.808 INFO:tasks.workunit.client.0.smithi049.stdout:7/892: chown d0/d6/d1d/f27 1 1 2024-05-15T15:54:32.808 INFO:tasks.workunit.client.0.smithi049.stdout:3/939: mknod d12/d14/d52/db9/da6/dc2/c11f 0 2024-05-15T15:54:32.809 INFO:tasks.workunit.client.0.smithi049.stdout:2/862: write d3/d78/d8c/d91/f9c [1566071,91429] 0 2024-05-15T15:54:32.811 INFO:tasks.workunit.client.0.smithi049.stdout:9/839: rename d0/d8/d9/d52/d5e/d8a/fa8 to d0/d8/dd6/de2/dec/d9d/f127 0 2024-05-15T15:54:32.812 INFO:tasks.workunit.client.0.smithi049.stdout:2/863: creat d3/da/d15/d24/f12e x:0 0 0 2024-05-15T15:54:32.813 INFO:tasks.workunit.client.0.smithi049.stdout:2/864: readlink d3/db/d82/lef 0 2024-05-15T15:54:32.815 INFO:tasks.workunit.client.0.smithi049.stdout:3/940: rename d12/d14/d32/d64/d11c to d12/d67/da8/df1/d120 0 2024-05-15T15:54:32.816 INFO:tasks.workunit.client.0.smithi049.stdout:3/941: fsync d12/d14/d52/db9/dc4/f102 0 2024-05-15T15:54:32.822 INFO:tasks.workunit.client.0.smithi049.stdout:2/865: mknod d3/da/c12f 0 2024-05-15T15:54:32.825 INFO:tasks.workunit.client.0.smithi049.stdout:3/942: mkdir d12/d14/d32/d64/d121 0 2024-05-15T15:54:32.828 INFO:tasks.workunit.client.0.smithi049.stdout:3/943: write d12/d14/d52/db9/f6e [5186439,104048] 0 2024-05-15T15:54:32.830 INFO:tasks.workunit.client.0.smithi049.stdout:9/840: dwrite d0/d8/d12/f59 [0,4194304] 0 2024-05-15T15:54:32.833 INFO:tasks.workunit.client.0.smithi049.stdout:2/866: link d3/db/d1d/d1e/d2b/d3b/l9d d3/da/d15/d73/l130 0 2024-05-15T15:54:32.835 INFO:tasks.workunit.client.0.smithi049.stdout:2/867: chown d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d 6212745 1 2024-05-15T15:54:32.837 INFO:tasks.workunit.client.0.smithi049.stdout:9/841: symlink d0/d8/d9/d45/l128 0 2024-05-15T15:54:32.838 INFO:tasks.workunit.client.0.smithi049.stdout:9/842: stat d0/d8/d9/d52/d5e/d5f/f3e 0 2024-05-15T15:54:32.843 INFO:tasks.workunit.client.0.smithi049.stdout:9/843: dread d0/d8/d9/d52/d5e/d5f/f55 [0,4194304] 0 2024-05-15T15:54:32.849 INFO:tasks.workunit.client.0.smithi049.stdout:9/844: rename d0/d8/d9/d22/fb1 to d0/d8/dd6/de2/dec/f129 0 2024-05-15T15:54:32.850 INFO:tasks.workunit.client.0.smithi049.stdout:2/868: link d3/db/d1d/d1e/c2d d3/db/d5b/dbf/dc1/c131 0 2024-05-15T15:54:32.852 INFO:tasks.workunit.client.0.smithi049.stdout:9/845: fdatasync d0/d8/d42/d75/fcf 0 2024-05-15T15:54:32.857 INFO:tasks.workunit.client.0.smithi049.stdout:2/869: dwrite d3/da/d15/d73/fe3 [0,4194304] 0 2024-05-15T15:54:32.867 INFO:tasks.workunit.client.0.smithi049.stdout:2/870: dwrite d3/da/d15/d73/fe3 [0,4194304] 0 2024-05-15T15:54:32.870 INFO:tasks.workunit.client.0.smithi049.stdout:2/871: creat d3/d8/dad/f132 x:0 0 0 2024-05-15T15:54:32.877 INFO:tasks.workunit.client.0.smithi049.stdout:9/846: dread d0/f41 [0,4194304] 0 2024-05-15T15:54:32.877 INFO:tasks.workunit.client.0.smithi049.stdout:2/872: rename d3/db/d1d/d1e/d2b/d3b/l114 to d3/d78/d8c/d91/l133 0 2024-05-15T15:54:32.881 INFO:tasks.workunit.client.0.smithi049.stdout:2/873: mknod d3/d8/c134 0 2024-05-15T15:54:32.884 INFO:tasks.workunit.client.0.smithi049.stdout:2/874: unlink d3/db/d5b/ccf 0 2024-05-15T15:54:32.885 INFO:tasks.workunit.client.0.smithi049.stdout:9/847: getdents d0/d8/d1b/d40/d5c 0 2024-05-15T15:54:32.889 INFO:tasks.workunit.client.0.smithi049.stdout:2/875: getdents d3/db/d1d/d1e 0 2024-05-15T15:54:32.896 INFO:tasks.workunit.client.0.smithi049.stdout:9/848: write d0/d8/d53/dd2/f9f [1059164,11320] 0 2024-05-15T15:54:32.900 INFO:tasks.workunit.client.0.smithi049.stdout:9/849: mknod d0/d8/d9/d52/d5e/d8a/dd3/c12a 0 2024-05-15T15:54:32.901 INFO:tasks.workunit.client.0.smithi049.stdout:2/876: dwrite d3/db/d82/fbc [0,4194304] 0 2024-05-15T15:54:32.902 INFO:tasks.workunit.client.0.smithi049.stdout:2/877: fdatasync d3/da/d15/d9e/ff6 0 2024-05-15T15:54:32.907 INFO:tasks.workunit.client.0.smithi049.stdout:9/850: mknod d0/d8/d1b/db3/df0/c12b 0 2024-05-15T15:54:32.908 INFO:tasks.workunit.client.0.smithi049.stdout:2/878: symlink d3/d52/d116/da3/dc9/dea/l135 0 2024-05-15T15:54:32.913 INFO:tasks.workunit.client.0.smithi049.stdout:9/851: mkdir d0/d8/d53/dd2/d12c 0 2024-05-15T15:54:32.914 INFO:tasks.workunit.client.0.smithi049.stdout:2/879: truncate d3/db/d1d/d1e/d2b/d3b/d47/d59/f8e 614001 0 2024-05-15T15:54:32.916 INFO:tasks.workunit.client.0.smithi049.stdout:9/852: creat d0/d8/d1b/d40/d5c/d69/f12d x:0 0 0 2024-05-15T15:54:32.917 INFO:tasks.workunit.client.0.smithi049.stdout:2/880: symlink d3/d78/d8c/d11a/l136 0 2024-05-15T15:54:32.918 INFO:tasks.workunit.client.0.smithi049.stdout:2/881: chown d3/db/f43 1530 1 2024-05-15T15:54:32.918 INFO:tasks.workunit.client.0.smithi049.stdout:9/853: fsync d0/d8/d42/f62 0 2024-05-15T15:54:32.923 INFO:tasks.workunit.client.0.smithi049.stdout:2/882: dread d3/da/d15/d24/f64 [0,4194304] 0 2024-05-15T15:54:32.931 INFO:tasks.workunit.client.0.smithi049.stdout:2/883: dwrite d3/f4d [0,4194304] 0 2024-05-15T15:54:32.935 INFO:tasks.workunit.client.0.smithi049.stdout:2/884: mknod d3/db/df0/c137 0 2024-05-15T15:54:32.938 INFO:tasks.workunit.client.0.smithi049.stdout:2/885: symlink d3/db/d1d/d1e/l138 0 2024-05-15T15:54:33.029 INFO:tasks.workunit.client.0.smithi049.stdout:2/886: unlink d3/da/d15/d9e/ff6 0 2024-05-15T15:54:33.032 INFO:tasks.workunit.client.0.smithi049.stdout:2/887: dread d3/f4d [0,4194304] 0 2024-05-15T15:54:33.033 INFO:tasks.workunit.client.0.smithi049.stdout:2/888: chown d3/d8/dad/dfe/d106 0 1 2024-05-15T15:54:33.152 INFO:tasks.workunit.client.0.smithi049.stdout:2/889: mknod d3/db/d1d/d1e/de2/c139 0 2024-05-15T15:54:33.157 INFO:tasks.workunit.client.0.smithi049.stdout:2/890: dwrite d3/d8/dad/f126 [0,4194304] 0 2024-05-15T15:54:33.160 INFO:tasks.workunit.client.0.smithi049.stdout:2/891: write d3/db/d1d/d1e/d2b/d3b/d42/f69 [1396106,15095] 0 2024-05-15T15:54:33.181 INFO:tasks.workunit.client.0.smithi049.stdout:2/892: dwrite d3/da/d15/d73/f7f [0,4194304] 0 2024-05-15T15:54:33.184 INFO:tasks.workunit.client.0.smithi049.stdout:2/893: symlink d3/d78/d8c/l13a 0 2024-05-15T15:54:33.193 INFO:tasks.workunit.client.0.smithi049.stdout:2/894: rename d3/d78/d8c/d91/l133 to d3/db/l13b 0 2024-05-15T15:54:33.196 INFO:tasks.workunit.client.0.smithi049.stdout:2/895: mknod d3/db/d1d/d1e/d2b/d3b/d42/c13c 0 2024-05-15T15:54:33.198 INFO:tasks.workunit.client.0.smithi049.stdout:2/896: dread - d3/d52/d116/da3/dc9/fda zero size 2024-05-15T15:54:33.203 INFO:tasks.workunit.client.0.smithi049.stdout:2/897: dread d3/db/d1d/d1e/d2b/d3b/d47/d59/fb2 [0,4194304] 0 2024-05-15T15:54:33.206 INFO:tasks.workunit.client.0.smithi049.stdout:2/898: creat d3/db/d1d/d1e/d2b/d3b/d47/d59/f13d x:0 0 0 2024-05-15T15:54:33.207 INFO:tasks.workunit.client.0.smithi049.stdout:2/899: stat d3/da/d15/d73 0 2024-05-15T15:54:33.208 INFO:tasks.workunit.client.0.smithi049.stdout:2/900: readlink d3/d52/d116/da3/dc9/dea/l135 0 2024-05-15T15:54:33.221 INFO:tasks.workunit.client.0.smithi049.stdout:2/901: dwrite d3/db/f12 [4194304,4194304] 0 2024-05-15T15:54:33.256 INFO:tasks.workunit.client.0.smithi049.stdout:8/891: sync 2024-05-15T15:54:33.256 INFO:tasks.workunit.client.0.smithi049.stdout:4/868: sync 2024-05-15T15:54:33.263 INFO:tasks.workunit.client.0.smithi049.stdout:8/892: mknod d2/de/d71/d86/d5d/d7d/db8/d5b/d60/da5/c116 0 2024-05-15T15:54:33.264 INFO:tasks.workunit.client.0.smithi049.stdout:8/893: chown d2/de/d71/d86/d5d/d7d/db8/d5b/d93/fd2 14766 1 2024-05-15T15:54:33.265 INFO:tasks.workunit.client.0.smithi049.stdout:8/894: readlink d2/de/d71/d86/d5d/d63/lb0 0 2024-05-15T15:54:33.269 INFO:tasks.workunit.client.0.smithi049.stdout:8/895: symlink d2/de/d71/d86/d5d/d7d/db8/l117 0 2024-05-15T15:54:33.270 INFO:tasks.workunit.client.0.smithi049.stdout:8/896: write d2/fc5 [3303360,130054] 0 2024-05-15T15:54:33.271 INFO:tasks.workunit.client.0.smithi049.stdout:4/869: link d18/f1a d18/d1e/d2a/d50/d54/d109/dff/f119 0 2024-05-15T15:54:33.273 INFO:tasks.workunit.client.0.smithi049.stdout:8/897: read - d2/de/d25/fbd zero size 2024-05-15T15:54:33.274 INFO:tasks.workunit.client.0.smithi049.stdout:8/898: read d2/de/dc3/fc4 [2827597,128016] 0 2024-05-15T15:54:33.279 INFO:tasks.workunit.client.0.smithi049.stdout:8/899: symlink d2/de/d95/l118 0 2024-05-15T15:54:33.279 INFO:tasks.workunit.client.0.smithi049.stdout:8/900: chown d2/d18/l29 917191 1 2024-05-15T15:54:33.319 INFO:tasks.workunit.client.0.smithi049.stdout:8/901: write d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/f114 [411476,15395] 0 2024-05-15T15:54:33.321 INFO:tasks.workunit.client.0.smithi049.stdout:4/870: dwrite d18/f7c [0,4194304] 0 2024-05-15T15:54:33.335 INFO:tasks.workunit.client.0.smithi049.stdout:4/871: rename d18/d1e/d2a/d50/d54/d109/fb5 to d18/d1e/f11a 0 2024-05-15T15:54:33.337 INFO:tasks.workunit.client.0.smithi049.stdout:4/872: symlink d18/d69/l11b 0 2024-05-15T15:54:33.342 INFO:tasks.workunit.client.0.smithi049.stdout:8/902: dwrite d2/de/d71/d86/d5d/d7d/d1b/d68/f8e [0,4194304] 0 2024-05-15T15:54:33.348 INFO:tasks.workunit.client.0.smithi049.stdout:8/903: mkdir d2/d119 0 2024-05-15T15:54:33.349 INFO:tasks.workunit.client.0.smithi049.stdout:8/904: read d2/de/d71/d86/fcf [1014239,64695] 0 2024-05-15T15:54:33.352 INFO:tasks.workunit.client.0.smithi049.stdout:8/905: creat d2/de/d71/d86/d5d/d7d/d1b/d68/f11a x:0 0 0 2024-05-15T15:54:33.361 INFO:tasks.workunit.client.0.smithi049.stdout:8/906: dwrite d2/de/d71/d86/d5d/d7d/db8/d5b/d60/f113 [0,4194304] 0 2024-05-15T15:54:33.366 INFO:tasks.workunit.client.0.smithi049.stdout:8/907: readlink d2/de/d71/d86/lc0 0 2024-05-15T15:54:33.375 INFO:tasks.workunit.client.0.smithi049.stdout:8/908: mknod d2/de/d71/d86/d5d/d7d/db8/d5b/dd0/c11b 0 2024-05-15T15:54:33.376 INFO:tasks.workunit.client.0.smithi049.stdout:8/909: chown d2/de/d71/d86/d5d/f80 6856 1 2024-05-15T15:54:33.378 INFO:tasks.workunit.client.0.smithi049.stdout:8/910: read d2/de/d71/d86/d5d/d7d/d1b/f36 [1271141,57900] 0 2024-05-15T15:54:33.379 INFO:tasks.workunit.client.0.smithi049.stdout:8/911: readlink d2/la4 0 2024-05-15T15:54:33.384 INFO:tasks.workunit.client.0.smithi049.stdout:8/912: symlink d2/de/d71/d86/d5d/d7d/d1b/l11c 0 2024-05-15T15:54:33.386 INFO:tasks.workunit.client.0.smithi049.stdout:8/913: creat d2/de/d71/d86/d5d/d7d/d1b/d68/dee/f11d x:0 0 0 2024-05-15T15:54:33.389 INFO:tasks.workunit.client.0.smithi049.stdout:8/914: mkdir d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/d11e 0 2024-05-15T15:54:33.393 INFO:tasks.workunit.client.0.smithi049.stdout:8/915: creat d2/de/f11f x:0 0 0 2024-05-15T15:54:33.395 INFO:tasks.workunit.client.0.smithi049.stdout:8/916: symlink d2/de/d71/d86/d5d/d7d/d1b/d68/dee/d103/l120 0 2024-05-15T15:54:33.547 INFO:tasks.workunit.client.0.smithi049.stdout:9/854: sync 2024-05-15T15:54:33.548 INFO:tasks.workunit.client.0.smithi049.stdout:7/893: sync 2024-05-15T15:54:33.548 INFO:tasks.workunit.client.0.smithi049.stdout:3/944: sync 2024-05-15T15:54:33.551 INFO:tasks.workunit.client.0.smithi049.stdout:3/945: rmdir d12/d14/d32/d104 39 2024-05-15T15:54:33.560 INFO:tasks.workunit.client.0.smithi049.stdout:3/946: rename d12/d14/d52/db9/da6/dc2/c11f to d12/d14/d52/db9/dc4/d68/c122 0 2024-05-15T15:54:33.563 INFO:tasks.workunit.client.0.smithi049.stdout:9/855: dread d0/d25/f4c [0,4194304] 0 2024-05-15T15:54:33.567 INFO:tasks.workunit.client.0.smithi049.stdout:3/947: dwrite d12/d14/d52/db9/dc4/f103 [0,4194304] 0 2024-05-15T15:54:33.569 INFO:tasks.workunit.client.0.smithi049.stdout:9/856: creat d0/d8/d9/d22/dbb/f12e x:0 0 0 2024-05-15T15:54:33.569 INFO:tasks.workunit.client.0.smithi049.stdout:7/894: dread d0/d6/d23/d45/fba [0,4194304] 0 2024-05-15T15:54:33.577 INFO:tasks.workunit.client.0.smithi049.stdout:7/895: mknod d0/d6/d23/d4a/d7b/de2/c11c 0 2024-05-15T15:54:33.578 INFO:tasks.workunit.client.0.smithi049.stdout:3/948: mknod d12/d14/c123 0 2024-05-15T15:54:33.579 INFO:tasks.workunit.client.0.smithi049.stdout:3/949: fdatasync d12/d14/d52/db9/dc4/d68/f91 0 2024-05-15T15:54:33.585 INFO:tasks.workunit.client.0.smithi049.stdout:3/950: creat d12/d30/d66/f124 x:0 0 0 2024-05-15T15:54:33.595 INFO:tasks.workunit.client.0.smithi049.stdout:3/951: truncate d12/d14/d52/db9/dc4/fd3 2147393 0 2024-05-15T15:54:33.595 INFO:tasks.workunit.client.0.smithi049.stdout:9/857: dwrite d0/d8/f39 [0,4194304] 0 2024-05-15T15:54:33.598 INFO:tasks.workunit.client.0.smithi049.stdout:9/858: chown d0/d8/d9/d22/dbb/f103 62031222 1 2024-05-15T15:54:33.598 INFO:tasks.workunit.client.0.smithi049.stdout:9/859: chown d0/d25/f4c 18 1 2024-05-15T15:54:33.600 INFO:tasks.workunit.client.0.smithi049.stdout:9/860: readlink d0/d8/d9/d52/d5e/l119 0 2024-05-15T15:54:33.601 INFO:tasks.workunit.client.0.smithi049.stdout:3/952: creat d12/d14/d52/db9/da6/de3/d113/f125 x:0 0 0 2024-05-15T15:54:33.603 INFO:tasks.workunit.client.0.smithi049.stdout:7/896: dwrite d0/d2f/d5c/f11b [0,4194304] 0 2024-05-15T15:54:33.605 INFO:tasks.workunit.client.0.smithi049.stdout:2/902: sync 2024-05-15T15:54:33.609 INFO:tasks.workunit.client.0.smithi049.stdout:3/953: chown d12/d26/d73/l11d 361827 1 2024-05-15T15:54:33.610 INFO:tasks.workunit.client.0.smithi049.stdout:3/954: chown d12/d26/fe0 0 1 2024-05-15T15:54:33.610 INFO:tasks.workunit.client.0.smithi049.stdout:2/903: mknod d3/db/d1d/d1e/d2b/d3b/d42/da8/c13e 0 2024-05-15T15:54:33.616 INFO:tasks.workunit.client.0.smithi049.stdout:3/955: rename d12/d30/fd6 to d12/d14/d52/db9/da6/dc2/f126 0 2024-05-15T15:54:33.617 INFO:tasks.workunit.client.0.smithi049.stdout:3/956: chown d12/d26/cb7 15294 1 2024-05-15T15:54:33.619 INFO:tasks.workunit.client.0.smithi049.stdout:7/897: creat d0/d6/d23/d45/f11d x:0 0 0 2024-05-15T15:54:33.620 INFO:tasks.workunit.client.0.smithi049.stdout:2/904: creat d3/d78/d124/f13f x:0 0 0 2024-05-15T15:54:33.621 INFO:tasks.workunit.client.0.smithi049.stdout:9/861: write d0/d8/d9/d22/f2e [699909,48843] 0 2024-05-15T15:54:33.622 INFO:tasks.workunit.client.0.smithi049.stdout:3/957: dread - d12/d14/d52/db9/fe1 zero size 2024-05-15T15:54:33.624 INFO:tasks.workunit.client.0.smithi049.stdout:7/898: symlink d0/d34/l11e 0 2024-05-15T15:54:33.625 INFO:tasks.workunit.client.0.smithi049.stdout:2/905: mknod d3/db/d1d/d1e/d2b/d3b/d42/c140 0 2024-05-15T15:54:33.627 INFO:tasks.workunit.client.0.smithi049.stdout:9/862: mknod d0/d8/d9/d52/d5e/d5f/d56/c12f 0 2024-05-15T15:54:33.628 INFO:tasks.workunit.client.0.smithi049.stdout:3/958: mkdir d12/d14/d52/db9/da6/de3/d113/d127 0 2024-05-15T15:54:33.629 INFO:tasks.workunit.client.0.smithi049.stdout:7/899: write d0/d6/d23/d45/d6a/da1/d82/dd3/f10e [2843455,16034] 0 2024-05-15T15:54:33.630 INFO:tasks.workunit.client.0.smithi049.stdout:2/906: rmdir d3/da/da9 39 2024-05-15T15:54:33.634 INFO:tasks.workunit.client.0.smithi049.stdout:3/959: fdatasync d12/d14/d52/db9/dc4/d68/d70/d75/f85 0 2024-05-15T15:54:33.638 INFO:tasks.workunit.client.0.smithi049.stdout:7/900: creat d0/d2f/f11f x:0 0 0 2024-05-15T15:54:33.638 INFO:tasks.workunit.client.0.smithi049.stdout:3/960: dread d12/d14/d52/db9/f6e [0,4194304] 0 2024-05-15T15:54:33.640 INFO:tasks.workunit.client.0.smithi049.stdout:2/907: mknod d3/d8/dad/dfe/c141 0 2024-05-15T15:54:33.641 INFO:tasks.workunit.client.0.smithi049.stdout:2/908: read d3/db/d1d/f28 [169108,6629] 0 2024-05-15T15:54:33.643 INFO:tasks.workunit.client.0.smithi049.stdout:2/909: write d3/db/d1d/d1e/d2b/d6b/f80 [1345793,11866] 0 2024-05-15T15:54:33.652 INFO:tasks.workunit.client.0.smithi049.stdout:3/961: symlink d12/d14/d52/db9/dc4/l128 0 2024-05-15T15:54:33.653 INFO:tasks.workunit.client.0.smithi049.stdout:3/962: write d12/d14/d32/d64/d88/d9c/fb4 [3894330,18850] 0 2024-05-15T15:54:33.655 INFO:tasks.workunit.client.0.smithi049.stdout:9/863: rename d0/d8/d9/d52/d5e/d8a/dd3/cde to d0/d8/d42/c130 0 2024-05-15T15:54:33.656 INFO:tasks.workunit.client.0.smithi049.stdout:2/910: symlink d3/da/l142 0 2024-05-15T15:54:33.661 INFO:tasks.workunit.client.0.smithi049.stdout:2/911: creat d3/d78/d124/f143 x:0 0 0 2024-05-15T15:54:33.669 INFO:tasks.workunit.client.0.smithi049.stdout:9/864: write d0/d8/d21/fa4 [134509,121111] 0 2024-05-15T15:54:33.669 INFO:tasks.workunit.client.0.smithi049.stdout:9/865: chown d0/d8/d12 8 1 2024-05-15T15:54:33.671 INFO:tasks.workunit.client.0.smithi049.stdout:2/912: write d3/db/f1a [912732,113943] 0 2024-05-15T15:54:33.674 INFO:tasks.workunit.client.0.smithi049.stdout:3/963: dread d12/d14/d32/d64/d88/f9f [0,4194304] 0 2024-05-15T15:54:33.677 INFO:tasks.workunit.client.0.smithi049.stdout:9/866: dwrite d0/d8/dd6/de2/dec/f101 [0,4194304] 0 2024-05-15T15:54:33.685 INFO:tasks.workunit.client.0.smithi049.stdout:9/867: write d0/d8/d12/f6e [1619370,63964] 0 2024-05-15T15:54:33.687 INFO:tasks.workunit.client.0.smithi049.stdout:3/964: stat d12/d30/d66/c9e 0 2024-05-15T15:54:33.690 INFO:tasks.workunit.client.0.smithi049.stdout:9/868: creat d0/d25/d5d/f131 x:0 0 0 2024-05-15T15:54:33.690 INFO:tasks.workunit.client.0.smithi049.stdout:2/913: dwrite d3/f11 [0,4194304] 0 2024-05-15T15:54:33.693 INFO:tasks.workunit.client.0.smithi049.stdout:2/914: fdatasync d3/db/d1d/f20 0 2024-05-15T15:54:33.697 INFO:tasks.workunit.client.0.smithi049.stdout:2/915: mknod d3/c144 0 2024-05-15T15:54:33.701 INFO:tasks.workunit.client.0.smithi049.stdout:9/869: getdents d0/d25 0 2024-05-15T15:54:33.704 INFO:tasks.workunit.client.0.smithi049.stdout:3/965: dwrite d12/f4d [0,4194304] 0 2024-05-15T15:54:33.714 INFO:tasks.workunit.client.0.smithi049.stdout:2/916: truncate d3/d52/d116/da3/dc9/f11b 631028 0 2024-05-15T15:54:33.715 INFO:tasks.workunit.client.0.smithi049.stdout:9/870: getdents d0/d8/dd6/de2/dec 0 2024-05-15T15:54:33.721 INFO:tasks.workunit.client.0.smithi049.stdout:9/871: truncate d0/d8/d9/d45/f4a 3287139 0 2024-05-15T15:54:33.723 INFO:tasks.workunit.client.0.smithi049.stdout:9/872: fdatasync d0/d8/d9/d22/dbb/f12e 0 2024-05-15T15:54:33.725 INFO:tasks.workunit.client.0.smithi049.stdout:9/873: write d0/d8/dd6/de2/dec/f101 [3998693,32150] 0 2024-05-15T15:54:33.730 INFO:tasks.workunit.client.0.smithi049.stdout:2/917: dwrite d3/db/d1d/d1e/d2b/d3b/d42/f102 [0,4194304] 0 2024-05-15T15:54:33.737 INFO:tasks.workunit.client.0.smithi049.stdout:2/918: creat d3/db/d5b/dbf/dc1/f145 x:0 0 0 2024-05-15T15:54:33.744 INFO:tasks.workunit.client.0.smithi049.stdout:9/874: write d0/d8/d9/d52/d5e/d8a/dd3/f79 [1043569,4621] 0 2024-05-15T15:54:33.744 INFO:tasks.workunit.client.0.smithi049.stdout:2/919: chown d3/d8/f50 96 1 2024-05-15T15:54:33.745 INFO:tasks.workunit.client.0.smithi049.stdout:9/875: dread - d0/d8/d9/d52/d5e/d5f/fa0 zero size 2024-05-15T15:54:33.746 INFO:tasks.workunit.client.0.smithi049.stdout:2/920: stat d3/lf2 0 2024-05-15T15:54:33.748 INFO:tasks.workunit.client.0.smithi049.stdout:2/921: write d3/d52/d116/da3/dc9/f11b [792560,1920] 0 2024-05-15T15:54:33.749 INFO:tasks.workunit.client.0.smithi049.stdout:2/922: chown d3/c144 16991906 1 2024-05-15T15:54:33.755 INFO:tasks.workunit.client.0.smithi049.stdout:9/876: dwrite d0/d8/d12/d9a/f106 [0,4194304] 0 2024-05-15T15:54:33.758 INFO:tasks.workunit.client.0.smithi049.stdout:2/923: symlink d3/db/d1d/d1e/d2b/d3b/df8/d123/l146 0 2024-05-15T15:54:33.759 INFO:tasks.workunit.client.0.smithi049.stdout:2/924: chown d3/d52/d116/faf 7866 1 2024-05-15T15:54:33.759 INFO:tasks.workunit.client.0.smithi049.stdout:9/877: fsync d0/d8/d42/d75/fcf 0 2024-05-15T15:54:33.762 INFO:tasks.workunit.client.0.smithi049.stdout:2/925: chown d3/d78/d8c/l13a 2516321 1 2024-05-15T15:54:33.766 INFO:tasks.workunit.client.0.smithi049.stdout:9/878: truncate d0/d8/d1b/f28 1824016 0 2024-05-15T15:54:33.768 INFO:tasks.workunit.client.0.smithi049.stdout:9/879: read - d0/d8/d9/d52/d5e/d5f/f89 zero size 2024-05-15T15:54:33.769 INFO:tasks.workunit.client.0.smithi049.stdout:9/880: fdatasync d0/d8/d53/dd2/f93 0 2024-05-15T15:54:33.776 INFO:tasks.workunit.client.0.smithi049.stdout:9/881: dread d0/d8/d1b/f94 [0,4194304] 0 2024-05-15T15:54:33.778 INFO:tasks.workunit.client.0.smithi049.stdout:9/882: write d0/d8/d12/f6e [4877083,87724] 0 2024-05-15T15:54:33.783 INFO:tasks.workunit.client.0.smithi049.stdout:9/883: getdents d0/d25 0 2024-05-15T15:54:33.787 INFO:tasks.workunit.client.0.smithi049.stdout:4/873: sync 2024-05-15T15:54:33.788 INFO:tasks.workunit.client.0.smithi049.stdout:8/917: sync 2024-05-15T15:54:33.790 INFO:tasks.workunit.client.0.smithi049.stdout:8/918: mkdir d2/de/d71/d86/d5d/de3/d121 0 2024-05-15T15:54:33.800 INFO:tasks.workunit.client.0.smithi049.stdout:9/884: write d0/d8/d9/d45/f4a [2454180,47296] 0 2024-05-15T15:54:33.807 INFO:tasks.workunit.client.0.smithi049.stdout:8/919: dread d2/de/d25/d8c/d9c/fd3 [0,4194304] 0 2024-05-15T15:54:33.811 INFO:tasks.workunit.client.0.smithi049.stdout:4/874: dwrite d18/d1e/d63/d10d/ffb [0,4194304] 0 2024-05-15T15:54:33.813 INFO:tasks.workunit.client.0.smithi049.stdout:8/920: chown d2/de/d71/d86/d5d/l81 99 1 2024-05-15T15:54:33.816 INFO:tasks.workunit.client.0.smithi049.stdout:8/921: chown d2/de/d71/d86/f6d 5730045 1 2024-05-15T15:54:33.817 INFO:tasks.workunit.client.0.smithi049.stdout:8/922: chown d2/f3d 0 1 2024-05-15T15:54:33.819 INFO:tasks.workunit.client.0.smithi049.stdout:9/885: dwrite d0/d8/d9/d52/d5e/d8a/dd3/d7d/fea [0,4194304] 0 2024-05-15T15:54:33.822 INFO:tasks.workunit.client.0.smithi049.stdout:4/875: dwrite d18/f3e [0,4194304] 0 2024-05-15T15:54:33.825 INFO:tasks.workunit.client.0.smithi049.stdout:9/886: readlink d0/d8/d9/le4 0 2024-05-15T15:54:33.827 INFO:tasks.workunit.client.0.smithi049.stdout:4/876: unlink d18/d21/d2b/cf6 0 2024-05-15T15:54:33.828 INFO:tasks.workunit.client.0.smithi049.stdout:4/877: fsync d18/d1e/d2a/d50/d54/d109/db6/ff1 0 2024-05-15T15:54:33.829 INFO:tasks.workunit.client.0.smithi049.stdout:8/923: creat d2/de/d71/d86/f122 x:0 0 0 2024-05-15T15:54:33.831 INFO:tasks.workunit.client.0.smithi049.stdout:8/924: fsync d2/de/d71/d86/f102 0 2024-05-15T15:54:33.837 INFO:tasks.workunit.client.0.smithi049.stdout:8/925: truncate d2/de/d71/d86/d5d/d7d/db8/f35 4339978 0 2024-05-15T15:54:33.838 INFO:tasks.workunit.client.0.smithi049.stdout:8/926: chown d2/de/d71/d86/d5d/d63/f6e 306 1 2024-05-15T15:54:33.844 INFO:tasks.workunit.client.0.smithi049.stdout:8/927: dwrite d2/fb [0,4194304] 0 2024-05-15T15:54:33.852 INFO:tasks.workunit.client.0.smithi049.stdout:4/878: write d18/d1e/d2a/d50/d54/f8a [372846,13800] 0 2024-05-15T15:54:33.858 INFO:tasks.workunit.client.0.smithi049.stdout:9/887: dwrite d0/d8/d12/fb0 [0,4194304] 0 2024-05-15T15:54:33.860 INFO:tasks.workunit.client.0.smithi049.stdout:8/928: dread d2/de/d71/fba [0,4194304] 0 2024-05-15T15:54:33.863 INFO:tasks.workunit.client.0.smithi049.stdout:9/888: mknod d0/d8/d21/c132 0 2024-05-15T15:54:33.864 INFO:tasks.workunit.client.0.smithi049.stdout:9/889: dread - d0/d8/d12/dfd/ffe zero size 2024-05-15T15:54:33.865 INFO:tasks.workunit.client.0.smithi049.stdout:8/929: mkdir d2/de/d25/d74/dbe/d123 0 2024-05-15T15:54:33.865 INFO:tasks.workunit.client.0.smithi049.stdout:9/890: chown d0/d8/d9/d52/d5e/d8a/dd3 6 1 2024-05-15T15:54:33.870 INFO:tasks.workunit.client.0.smithi049.stdout:9/891: truncate d0/d25/faf 432099 0 2024-05-15T15:54:33.872 INFO:tasks.workunit.client.0.smithi049.stdout:9/892: creat d0/d8/d12/d9a/f133 x:0 0 0 2024-05-15T15:54:33.873 INFO:tasks.workunit.client.0.smithi049.stdout:9/893: fdatasync d0/d8/d9/d52/d5e/d8a/dd3/f100 0 2024-05-15T15:54:33.875 INFO:tasks.workunit.client.0.smithi049.stdout:9/894: write d0/d8/f39 [4791465,108188] 0 2024-05-15T15:54:33.878 INFO:tasks.workunit.client.0.smithi049.stdout:9/895: creat d0/d8/d1b/db3/df0/f134 x:0 0 0 2024-05-15T15:54:33.880 INFO:tasks.workunit.client.0.smithi049.stdout:8/930: write d2/de/d25/d74/fae [420534,72898] 0 2024-05-15T15:54:33.882 INFO:tasks.workunit.client.0.smithi049.stdout:9/896: truncate d0/d8/d9/d45/f4a 3347467 0 2024-05-15T15:54:33.882 INFO:tasks.workunit.client.0.smithi049.stdout:8/931: write d2/de/f6c [471427,65587] 0 2024-05-15T15:54:33.885 INFO:tasks.workunit.client.0.smithi049.stdout:8/932: chown d2/c75 54991704 1 2024-05-15T15:54:33.887 INFO:tasks.workunit.client.0.smithi049.stdout:9/897: dread d0/d8/d1b/f3c [0,4194304] 0 2024-05-15T15:54:33.888 INFO:tasks.workunit.client.0.smithi049.stdout:8/933: truncate d2/de/d25/fbd 983442 0 2024-05-15T15:54:33.891 INFO:tasks.workunit.client.0.smithi049.stdout:8/934: mknod d2/d6a/c124 0 2024-05-15T15:54:33.894 INFO:tasks.workunit.client.0.smithi049.stdout:8/935: mkdir d2/de/d25/d74/dbe/dd8/d125 0 2024-05-15T15:54:33.896 INFO:tasks.workunit.client.0.smithi049.stdout:8/936: write d2/de/d71/d86/d5d/d7d/db8/d5b/d60/f113 [942160,52296] 0 2024-05-15T15:54:33.896 INFO:tasks.workunit.client.0.smithi049.stdout:9/898: dread d0/d8/d1b/f3a [0,4194304] 0 2024-05-15T15:54:33.910 INFO:tasks.workunit.client.0.smithi049.stdout:9/899: link d0/d8/d53/f113 d0/d8/d9/d52/dc7/f135 0 2024-05-15T15:54:33.914 INFO:tasks.workunit.client.0.smithi049.stdout:8/937: dwrite d2/de/d71/d86/d5d/d7d/f4c [0,4194304] 0 2024-05-15T15:54:33.915 INFO:tasks.workunit.client.0.smithi049.stdout:9/900: unlink d0/d8/d21/lee 0 2024-05-15T15:54:33.920 INFO:tasks.workunit.client.0.smithi049.stdout:8/938: creat d2/de/d95/f126 x:0 0 0 2024-05-15T15:54:33.921 INFO:tasks.workunit.client.0.smithi049.stdout:8/939: fsync d2/fb 0 2024-05-15T15:54:33.922 INFO:tasks.workunit.client.0.smithi049.stdout:9/901: creat d0/d8/d1b/d40/f136 x:0 0 0 2024-05-15T15:54:33.923 INFO:tasks.workunit.client.0.smithi049.stdout:9/902: readlink d0/d8/ld8 0 2024-05-15T15:54:33.929 INFO:tasks.workunit.client.0.smithi049.stdout:9/903: unlink d0/d8/d1b/db3/ffa 0 2024-05-15T15:54:33.934 INFO:tasks.workunit.client.0.smithi049.stdout:9/904: symlink d0/d8/d12/l137 0 2024-05-15T15:54:33.938 INFO:tasks.workunit.client.0.smithi049.stdout:9/905: rename d0/d8/d9/d52/d5e/l119 to d0/d8/d1b/db3/df0/l138 0 2024-05-15T15:54:33.946 INFO:tasks.workunit.client.0.smithi049.stdout:8/940: dread d2/de/d71/d86/d5d/f80 [0,4194304] 0 2024-05-15T15:54:33.949 INFO:tasks.workunit.client.0.smithi049.stdout:9/906: dread d0/d8/d42/f91 [0,4194304] 0 2024-05-15T15:54:33.980 INFO:tasks.workunit.client.0.smithi049.stdout:7/901: sync 2024-05-15T15:54:33.983 INFO:tasks.workunit.client.0.smithi049.stdout:9/907: fdatasync d0/d8/d9/d52/d5e/d8a/dd3/d87/fe6 0 2024-05-15T15:54:33.986 INFO:tasks.workunit.client.0.smithi049.stdout:7/902: rmdir d0/d6/d23/d45/d6a/da1 39 2024-05-15T15:54:33.994 INFO:tasks.workunit.client.0.smithi049.stdout:9/908: creat d0/d8/d9/d52/d5e/d8a/dd3/d87/dd1/f139 x:0 0 0 2024-05-15T15:54:34.038 INFO:tasks.workunit.client.0.smithi049.stdout:9/909: mknod d0/d8/d12/dfd/c13a 0 2024-05-15T15:54:34.039 INFO:tasks.workunit.client.0.smithi049.stdout:7/903: getdents d0/ddb 0 2024-05-15T15:54:34.043 INFO:tasks.workunit.client.0.smithi049.stdout:9/910: rename d0/d8/d9/d52/d5e/d8a/l11f to d0/d8/d9/d52/d5e/d8a/dd3/d7d/l13b 0 2024-05-15T15:54:34.046 INFO:tasks.workunit.client.0.smithi049.stdout:7/904: rename d0/d34/db3/ldf to d0/d6/d23/d45/d86/l120 0 2024-05-15T15:54:34.047 INFO:tasks.workunit.client.0.smithi049.stdout:9/911: truncate d0/d8/d1b/f3c 393953 0 2024-05-15T15:54:34.051 INFO:tasks.workunit.client.0.smithi049.stdout:9/912: rename d0/d8/d9/d52/d5e/d8a/dd3/d7d/dba/lcd to d0/d8/d9/d52/dc7/l13c 0 2024-05-15T15:54:34.052 INFO:tasks.workunit.client.0.smithi049.stdout:7/905: mknod d0/d6/d23/d45/d6a/da1/c121 0 2024-05-15T15:54:34.066 INFO:tasks.workunit.client.0.smithi049.stdout:9/913: dread d0/d8/d42/d75/fcf [0,4194304] 0 2024-05-15T15:54:34.068 INFO:tasks.workunit.client.0.smithi049.stdout:9/914: read d0/d8/d9/fb [1958477,125511] 0 2024-05-15T15:54:34.077 INFO:tasks.workunit.client.0.smithi049.stdout:7/906: truncate d0/d6/d23/d45/d6a/da1/d80/fb2 3667514 0 2024-05-15T15:54:34.078 INFO:tasks.workunit.client.0.smithi049.stdout:7/907: write d0/d6/d23/d76/ff5 [318015,49108] 0 2024-05-15T15:54:34.088 INFO:tasks.workunit.client.0.smithi049.stdout:9/915: dwrite d0/d8/d1b/f32 [0,4194304] 0 2024-05-15T15:54:34.093 INFO:tasks.workunit.client.0.smithi049.stdout:9/916: mknod d0/d8/d9/d52/d5e/d8a/dd3/d87/dd1/c13d 0 2024-05-15T15:54:34.094 INFO:tasks.workunit.client.0.smithi049.stdout:7/908: dread d0/d6/d23/fe5 [0,4194304] 0 2024-05-15T15:54:34.100 INFO:tasks.workunit.client.0.smithi049.stdout:7/909: rmdir d0/d6/d23/d4a/d7b/dc4/de0 39 2024-05-15T15:54:34.102 INFO:tasks.workunit.client.0.smithi049.stdout:7/910: mknod d0/ddb/dde/c122 0 2024-05-15T15:54:34.107 INFO:tasks.workunit.client.0.smithi049.stdout:7/911: mknod d0/d6/d23/d4a/d7b/dc4/de0/c123 0 2024-05-15T15:54:34.108 INFO:tasks.workunit.client.0.smithi049.stdout:9/917: dread d0/f54 [0,4194304] 0 2024-05-15T15:54:34.109 INFO:tasks.workunit.client.0.smithi049.stdout:9/918: dread - d0/d8/d1b/db3/df0/f134 zero size 2024-05-15T15:54:34.110 INFO:tasks.workunit.client.0.smithi049.stdout:9/919: stat d0/d8/d12/c64 0 2024-05-15T15:54:34.111 INFO:tasks.workunit.client.0.smithi049.stdout:7/912: mkdir d0/d6/d23/d45/d6a/d124 0 2024-05-15T15:54:34.112 INFO:tasks.workunit.client.0.smithi049.stdout:9/920: fdatasync d0/d8/d42/d75/fa5 0 2024-05-15T15:54:34.113 INFO:tasks.workunit.client.0.smithi049.stdout:9/921: chown d0/d8/d1b/f3a 87252 1 2024-05-15T15:54:34.114 INFO:tasks.workunit.client.0.smithi049.stdout:7/913: creat d0/d6/d23/d4a/f125 x:0 0 0 2024-05-15T15:54:34.119 INFO:tasks.workunit.client.0.smithi049.stdout:7/914: mknod d0/d6/d23/d45/d6a/c126 0 2024-05-15T15:54:34.122 INFO:tasks.workunit.client.0.smithi049.stdout:9/922: rename d0/d8/f120 to d0/f13e 0 2024-05-15T15:54:34.123 INFO:tasks.workunit.client.0.smithi049.stdout:9/923: chown d0/d8/d1b/c37 1389 1 2024-05-15T15:54:34.123 INFO:tasks.workunit.client.0.smithi049.stdout:9/924: readlink d0/d25/lc8 0 2024-05-15T15:54:34.124 INFO:tasks.workunit.client.0.smithi049.stdout:7/915: fsync d0/d6/da/fea 0 2024-05-15T15:54:34.124 INFO:tasks.workunit.client.0.smithi049.stdout:9/925: chown d0/d8/d12/cc 0 1 2024-05-15T15:54:34.125 INFO:tasks.workunit.client.0.smithi049.stdout:9/926: dread - d0/d25/f76 zero size 2024-05-15T15:54:34.131 INFO:tasks.workunit.client.0.smithi049.stdout:7/916: unlink d0/d6/d23/d45/c7f 0 2024-05-15T15:54:34.136 INFO:tasks.workunit.client.0.smithi049.stdout:7/917: dread d0/d6/d23/d45/d86/fc5 [0,4194304] 0 2024-05-15T15:54:34.137 INFO:tasks.workunit.client.0.smithi049.stdout:9/927: creat d0/f13f x:0 0 0 2024-05-15T15:54:34.138 INFO:tasks.workunit.client.0.smithi049.stdout:9/928: fsync d0/d8/d12/d9a/f106 0 2024-05-15T15:54:34.143 INFO:tasks.workunit.client.0.smithi049.stdout:9/929: creat d0/d8/d9/d22/f140 x:0 0 0 2024-05-15T15:54:34.144 INFO:tasks.workunit.client.0.smithi049.stdout:9/930: chown d0/d8/d9/d52/cd5 18147 1 2024-05-15T15:54:34.147 INFO:tasks.workunit.client.0.smithi049.stdout:7/918: link d0/d2f/lb5 d0/d6/d23/d45/d6a/d102/l127 0 2024-05-15T15:54:34.151 INFO:tasks.workunit.client.0.smithi049.stdout:7/919: rmdir d0/d6/d23/d45/db1 39 2024-05-15T15:54:34.154 INFO:tasks.workunit.client.0.smithi049.stdout:3/966: sync 2024-05-15T15:54:34.155 INFO:tasks.workunit.client.0.smithi049.stdout:9/931: creat d0/d8/d42/d75/f141 x:0 0 0 2024-05-15T15:54:34.157 INFO:tasks.workunit.client.0.smithi049.stdout:7/920: mknod d0/ddb/dde/dca/d115/c128 0 2024-05-15T15:54:34.160 INFO:tasks.workunit.client.0.smithi049.stdout:9/932: creat d0/d8/d9/d52/d5e/d8a/dd3/d87/dd1/f142 x:0 0 0 2024-05-15T15:54:34.164 INFO:tasks.workunit.client.0.smithi049.stdout:7/921: creat d0/ddb/dde/dca/d115/f129 x:0 0 0 2024-05-15T15:54:34.170 INFO:tasks.workunit.client.0.smithi049.stdout:9/933: creat d0/d8/d53/dd2/df3/d10c/f143 x:0 0 0 2024-05-15T15:54:34.172 INFO:tasks.workunit.client.0.smithi049.stdout:9/934: dread - d0/d8/d1b/db3/df0/f134 zero size 2024-05-15T15:54:34.173 INFO:tasks.workunit.client.0.smithi049.stdout:3/967: dwrite d12/d30/f38 [0,4194304] 0 2024-05-15T15:54:34.175 INFO:tasks.workunit.client.0.smithi049.stdout:3/968: read - d12/d14/d52/f105 zero size 2024-05-15T15:54:34.176 INFO:tasks.workunit.client.0.smithi049.stdout:9/935: fdatasync d0/d8/d42/d75/f85 0 2024-05-15T15:54:34.180 INFO:tasks.workunit.client.0.smithi049.stdout:7/922: dwrite d0/d6/f29 [0,4194304] 0 2024-05-15T15:54:34.180 INFO:tasks.workunit.client.0.smithi049.stdout:3/969: creat d12/d67/da8/f129 x:0 0 0 2024-05-15T15:54:34.186 INFO:tasks.workunit.client.0.smithi049.stdout:7/923: mknod d0/d2f/c12a 0 2024-05-15T15:54:34.191 INFO:tasks.workunit.client.0.smithi049.stdout:9/936: creat d0/d8e/f144 x:0 0 0 2024-05-15T15:54:34.191 INFO:tasks.workunit.client.0.smithi049.stdout:9/937: stat d0/l20 0 2024-05-15T15:54:34.194 INFO:tasks.workunit.client.0.smithi049.stdout:2/926: sync 2024-05-15T15:54:34.196 INFO:tasks.workunit.client.0.smithi049.stdout:3/970: creat d12/d14/d52/db9/f12a x:0 0 0 2024-05-15T15:54:34.202 INFO:tasks.workunit.client.0.smithi049.stdout:2/927: dwrite d3/d78/d8c/d11a/fca [0,4194304] 0 2024-05-15T15:54:34.202 INFO:tasks.workunit.client.0.smithi049.stdout:9/938: write d0/d8/d21/f82 [258332,80292] 0 2024-05-15T15:54:34.206 INFO:tasks.workunit.client.0.smithi049.stdout:3/971: rename d12/d14/d32/d64/d88/c10a to d12/d14/d52/db9/da6/de3/c12b 0 2024-05-15T15:54:34.213 INFO:tasks.workunit.client.0.smithi049.stdout:7/924: link d0/d6/d23/d45/d6a/d102/l127 d0/d6/d23/d45/d6a/d102/l12b 0 2024-05-15T15:54:34.214 INFO:tasks.workunit.client.0.smithi049.stdout:7/925: dread - d0/d6/d23/d45/f11d zero size 2024-05-15T15:54:34.215 INFO:tasks.workunit.client.0.smithi049.stdout:3/972: creat d12/d14/d52/db9/dc4/d68/d70/d75/dac/f12c x:0 0 0 2024-05-15T15:54:34.218 INFO:tasks.workunit.client.0.smithi049.stdout:2/928: mknod d3/da/d15/d73/c147 0 2024-05-15T15:54:34.219 INFO:tasks.workunit.client.0.smithi049.stdout:9/939: write d0/d25/f76 [445977,108375] 0 2024-05-15T15:54:34.220 INFO:tasks.workunit.client.0.smithi049.stdout:9/940: chown d0/d8/d9/d52/d5e/d8a/dd3/d7d 46285 1 2024-05-15T15:54:34.222 INFO:tasks.workunit.client.0.smithi049.stdout:9/941: chown d0/d8/d1b/db3/df0/f134 95993792 1 2024-05-15T15:54:34.222 INFO:tasks.workunit.client.0.smithi049.stdout:7/926: rename d0/d6/d23/d4a/d7b/dc4/fe7 to d0/d6/d23/d76/f12c 0 2024-05-15T15:54:34.229 INFO:tasks.workunit.client.0.smithi049.stdout:9/942: creat d0/d8/d53/dd2/df3/f145 x:0 0 0 2024-05-15T15:54:34.229 INFO:tasks.workunit.client.0.smithi049.stdout:9/943: stat d0/d8/f39 0 2024-05-15T15:54:34.235 INFO:tasks.workunit.client.0.smithi049.stdout:9/944: rename d0/d8/d42/d75/cf6 to d0/d25/c146 0 2024-05-15T15:54:34.235 INFO:tasks.workunit.client.0.smithi049.stdout:2/929: write d3/db/d1d/d1e/d2b/d3b/f5f [557686,44883] 0 2024-05-15T15:54:34.236 INFO:tasks.workunit.client.0.smithi049.stdout:8/941: sync 2024-05-15T15:54:34.236 INFO:tasks.workunit.client.0.smithi049.stdout:4/879: sync 2024-05-15T15:54:34.236 INFO:tasks.workunit.client.0.smithi049.stdout:9/945: chown d0/d8/d42/d75 5403 1 2024-05-15T15:54:34.237 INFO:tasks.workunit.client.0.smithi049.stdout:9/946: readlink d0/l20 0 2024-05-15T15:54:34.238 INFO:tasks.workunit.client.0.smithi049.stdout:4/880: read d18/d21/f56 [564529,53738] 0 2024-05-15T15:54:34.240 INFO:tasks.workunit.client.0.smithi049.stdout:7/927: dread d0/d34/f3f [0,4194304] 0 2024-05-15T15:54:34.240 INFO:tasks.workunit.client.0.smithi049.stdout:3/973: getdents d12/d30/dc1 0 2024-05-15T15:54:34.243 INFO:tasks.workunit.client.0.smithi049.stdout:2/930: dread - d3/db/d1d/d1e/d2b/d6b/fd0 zero size 2024-05-15T15:54:34.243 INFO:tasks.workunit.client.0.smithi049.stdout:3/974: dread d12/d14/d52/db9/dc4/f100 [0,4194304] 0 2024-05-15T15:54:34.248 INFO:tasks.workunit.client.0.smithi049.stdout:9/947: truncate d0/d8/d9/d52/d5e/d5f/f63 4326930 0 2024-05-15T15:54:34.249 INFO:tasks.workunit.client.0.smithi049.stdout:9/948: dread - d0/d25/d5d/f131 zero size 2024-05-15T15:54:34.249 INFO:tasks.workunit.client.0.smithi049.stdout:9/949: chown d0/f1 4097 1 2024-05-15T15:54:34.250 INFO:tasks.workunit.client.0.smithi049.stdout:9/950: dread - d0/d8/d1b/db3/df0/f134 zero size 2024-05-15T15:54:34.252 INFO:tasks.workunit.client.0.smithi049.stdout:3/975: mkdir d12/d30/dc1/dc7/d12d 0 2024-05-15T15:54:34.254 INFO:tasks.workunit.client.0.smithi049.stdout:8/942: dread d2/de/d71/d86/d5d/d7d/d1b/f107 [0,4194304] 0 2024-05-15T15:54:34.261 INFO:tasks.workunit.client.0.smithi049.stdout:4/881: dwrite d18/d21/ff2 [0,4194304] 0 2024-05-15T15:54:34.263 INFO:tasks.workunit.client.0.smithi049.stdout:2/931: rename d3/db/l128 to d3/db/d1d/d1e/d2b/d3b/df8/l148 0 2024-05-15T15:54:34.273 INFO:tasks.workunit.client.0.smithi049.stdout:9/951: link d0/fa2 d0/d8/d21/ded/d107/f147 0 2024-05-15T15:54:34.274 INFO:tasks.workunit.client.0.smithi049.stdout:9/952: chown d0/d8/d53/dd2/df3/d10c 31889 1 2024-05-15T15:54:34.274 INFO:tasks.workunit.client.0.smithi049.stdout:3/976: link d12/d14/d52/db9/dc4/d68/lc5 d12/d30/d66/l12e 0 2024-05-15T15:54:34.277 INFO:tasks.workunit.client.0.smithi049.stdout:2/932: truncate d3/d8/f36 1038762 0 2024-05-15T15:54:34.284 INFO:tasks.workunit.client.0.smithi049.stdout:2/933: dwrite d3/d78/d8c/d11a/fe1 [0,4194304] 0 2024-05-15T15:54:34.285 INFO:tasks.workunit.client.0.smithi049.stdout:4/882: link d18/d69/l11b d18/d1e/d2a/d50/d54/d109/dff/l11c 0 2024-05-15T15:54:34.299 INFO:tasks.workunit.client.0.smithi049.stdout:4/883: symlink d18/d69/l11d 0 2024-05-15T15:54:34.299 INFO:tasks.workunit.client.0.smithi049.stdout:8/943: write d2/de/d71/d86/d5d/d7d/db8/f35 [2790664,62876] 0 2024-05-15T15:54:34.308 INFO:tasks.workunit.client.0.smithi049.stdout:4/884: readlink d18/d1e/d2a/d50/d54/d109/dff/l11c 0 2024-05-15T15:54:34.310 INFO:tasks.workunit.client.0.smithi049.stdout:9/953: getdents d0/d8/d1b/db3/df0 0 2024-05-15T15:54:34.312 INFO:tasks.workunit.client.0.smithi049.stdout:9/954: truncate d0/d8/d1b/d40/d5c/d69/f126 808249 0 2024-05-15T15:54:34.315 INFO:tasks.workunit.client.0.smithi049.stdout:2/934: rename d3/db/d5b/f70 to d3/f149 0 2024-05-15T15:54:34.319 INFO:tasks.workunit.client.0.smithi049.stdout:4/885: creat d18/d1e/d2a/d36/d86/f11e x:0 0 0 2024-05-15T15:54:34.321 INFO:tasks.workunit.client.0.smithi049.stdout:3/977: dwrite d12/d14/d52/db9/dc4/d68/d70/dda/f10b [0,4194304] 0 2024-05-15T15:54:34.321 INFO:tasks.workunit.client.0.smithi049.stdout:8/944: mkdir d2/de/d71/d86/d5d/d7d/db8/d5b/da6/d127 0 2024-05-15T15:54:34.332 INFO:tasks.workunit.client.0.smithi049.stdout:9/955: truncate d0/d8/d9/d52/d5e/d5f/f63 4688952 0 2024-05-15T15:54:34.335 INFO:tasks.workunit.client.0.smithi049.stdout:4/886: mknod d18/d21/d107/c11f 0 2024-05-15T15:54:34.339 INFO:tasks.workunit.client.0.smithi049.stdout:8/945: link d2/de/d71/d86/d5d/d7d/d1b/d68/f8e d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/d11e/f128 0 2024-05-15T15:54:34.344 INFO:tasks.workunit.client.0.smithi049.stdout:4/887: mknod d18/d1e/d2a/d50/d54/dad/c120 0 2024-05-15T15:54:34.348 INFO:tasks.workunit.client.0.smithi049.stdout:2/935: link d3/d8/cae d3/d8/dad/dfe/c14a 0 2024-05-15T15:54:34.350 INFO:tasks.workunit.client.0.smithi049.stdout:4/888: read d18/d1e/d63/d99/de1/f74 [2628495,80722] 0 2024-05-15T15:54:34.354 INFO:tasks.workunit.client.0.smithi049.stdout:2/936: chown d3/db/d1d/d1e/d2b/f75 0 1 2024-05-15T15:54:34.356 INFO:tasks.workunit.client.0.smithi049.stdout:4/889: truncate d18/d1e/d2a/d36/ff9 1854993 0 2024-05-15T15:54:34.357 INFO:tasks.workunit.client.0.smithi049.stdout:2/937: creat d3/db/d5b/f14b x:0 0 0 2024-05-15T15:54:34.361 INFO:tasks.workunit.client.0.smithi049.stdout:2/938: creat d3/db/d5b/dbf/f14c x:0 0 0 2024-05-15T15:54:34.364 INFO:tasks.workunit.client.0.smithi049.stdout:3/978: dwrite d12/d14/d52/f56 [0,4194304] 0 2024-05-15T15:54:34.366 INFO:tasks.workunit.client.0.smithi049.stdout:4/890: dwrite d18/d1e/d2a/d36/d86/f115 [0,4194304] 0 2024-05-15T15:54:34.368 INFO:tasks.workunit.client.0.smithi049.stdout:2/939: creat d3/db/d1d/d9f/f14d x:0 0 0 2024-05-15T15:54:34.376 INFO:tasks.workunit.client.0.smithi049.stdout:9/956: dwrite d0/d8/d1b/d40/d5c/f70 [0,4194304] 0 2024-05-15T15:54:34.381 INFO:tasks.workunit.client.0.smithi049.stdout:2/940: chown d3/d52 442 1 2024-05-15T15:54:34.389 INFO:tasks.workunit.client.0.smithi049.stdout:4/891: dread d18/d1e/d2a/d50/d54/d109/d98/fd8 [0,4194304] 0 2024-05-15T15:54:34.390 INFO:tasks.workunit.client.0.smithi049.stdout:4/892: chown d18/d1e/d2a/d50/f5a 0 1 2024-05-15T15:54:34.391 INFO:tasks.workunit.client.0.smithi049.stdout:2/941: unlink d3/d52/d116/da3/dc9/dea/led 0 2024-05-15T15:54:34.393 INFO:tasks.workunit.client.0.smithi049.stdout:4/893: truncate d18/d1e/d2a/d50/d54/d109/f8b 356973 0 2024-05-15T15:54:34.395 INFO:tasks.workunit.client.0.smithi049.stdout:9/957: link d0/d8/ld8 d0/d8/d9/d22/dbb/l148 0 2024-05-15T15:54:34.399 INFO:tasks.workunit.client.0.smithi049.stdout:9/958: dread d0/d8/d1b/d40/d5c/d69/f126 [0,4194304] 0 2024-05-15T15:54:34.402 INFO:tasks.workunit.client.0.smithi049.stdout:2/942: dwrite d3/d78/d124/f13f [0,4194304] 0 2024-05-15T15:54:34.404 INFO:tasks.workunit.client.0.smithi049.stdout:4/894: getdents d18/d1e/d2a/d50/d54 0 2024-05-15T15:54:34.405 INFO:tasks.workunit.client.0.smithi049.stdout:9/959: dread d0/d25/d5d/f8d [0,4194304] 0 2024-05-15T15:54:34.407 INFO:tasks.workunit.client.0.smithi049.stdout:3/979: dwrite d12/f3f [0,4194304] 0 2024-05-15T15:54:34.416 INFO:tasks.workunit.client.0.smithi049.stdout:9/960: dread d0/d8/d21/fff [0,4194304] 0 2024-05-15T15:54:34.418 INFO:tasks.workunit.client.0.smithi049.stdout:9/961: chown d0/d25/d5d/f66 30 1 2024-05-15T15:54:34.418 INFO:tasks.workunit.client.0.smithi049.stdout:3/980: dwrite d12/d30/d66/fca [0,4194304] 0 2024-05-15T15:54:34.420 INFO:tasks.workunit.client.0.smithi049.stdout:4/895: mknod d18/d1e/d2a/d50/d54/d109/d98/c121 0 2024-05-15T15:54:34.428 INFO:tasks.workunit.client.0.smithi049.stdout:9/962: rename d0/d8/d1b/d40/d5c/d69 to d0/d8/d1b/db3/df0/d149 0 2024-05-15T15:54:34.430 INFO:tasks.workunit.client.0.smithi049.stdout:2/943: dread d3/da/d15/d73/fe3 [0,4194304] 0 2024-05-15T15:54:34.432 INFO:tasks.workunit.client.0.smithi049.stdout:3/981: symlink d12/d14/d52/db9/da6/de3/d113/d127/l12f 0 2024-05-15T15:54:34.435 INFO:tasks.workunit.client.0.smithi049.stdout:9/963: creat d0/d8/d9/d52/d5e/d8a/dd3/d87/f14a x:0 0 0 2024-05-15T15:54:34.435 INFO:tasks.workunit.client.0.smithi049.stdout:4/896: write d18/d1e/fa7 [514328,44581] 0 2024-05-15T15:54:34.437 INFO:tasks.workunit.client.0.smithi049.stdout:9/964: write d0/d8/d9/d52/d5e/d5f/f38 [4444988,85791] 0 2024-05-15T15:54:34.437 INFO:tasks.workunit.client.0.smithi049.stdout:4/897: chown d18/d1e/d2a/d50/d54/dad/fce 51321 1 2024-05-15T15:54:34.438 INFO:tasks.workunit.client.0.smithi049.stdout:2/944: rename d3/db/d1d/d1e/d2b/d3b/df8/d123/l146 to d3/db/d1d/d1e/d2b/d3b/df8/l14e 0 2024-05-15T15:54:34.439 INFO:tasks.workunit.client.0.smithi049.stdout:2/945: readlink d3/d78/d8c/d11a/l136 0 2024-05-15T15:54:34.441 INFO:tasks.workunit.client.0.smithi049.stdout:3/982: creat d12/d14/d52/db9/dc4/d68/f130 x:0 0 0 2024-05-15T15:54:34.444 INFO:tasks.workunit.client.0.smithi049.stdout:4/898: creat d18/f122 x:0 0 0 2024-05-15T15:54:34.445 INFO:tasks.workunit.client.0.smithi049.stdout:2/946: mkdir d3/d78/d14f 0 2024-05-15T15:54:34.446 INFO:tasks.workunit.client.0.smithi049.stdout:2/947: read - d3/d78/d8c/d91/f115 zero size 2024-05-15T15:54:34.447 INFO:tasks.workunit.client.0.smithi049.stdout:2/948: fsync d3/db/d1d/d1e/d2b/d3b/f5f 0 2024-05-15T15:54:34.454 INFO:tasks.workunit.client.0.smithi049.stdout:9/965: write d0/d8/d1b/f3c [559693,28863] 0 2024-05-15T15:54:34.457 INFO:tasks.workunit.client.0.smithi049.stdout:2/949: unlink d3/d52/d116/faf 0 2024-05-15T15:54:34.459 INFO:tasks.workunit.client.0.smithi049.stdout:9/966: dread d0/d8/d9/d22/f96 [0,4194304] 0 2024-05-15T15:54:34.462 INFO:tasks.workunit.client.0.smithi049.stdout:2/950: symlink d3/d78/d8c/d91/d7a/l150 0 2024-05-15T15:54:34.463 INFO:tasks.workunit.client.0.smithi049.stdout:3/983: getdents d12/d14/d52/db9/dc4/d68/d70/dda 0 2024-05-15T15:54:34.465 INFO:tasks.workunit.client.0.smithi049.stdout:4/899: dwrite d18/d21/d2b/d4e/f87 [0,4194304] 0 2024-05-15T15:54:34.465 INFO:tasks.workunit.client.0.smithi049.stdout:9/967: read - d0/d8/dd6/de2/dec/d9d/f127 zero size 2024-05-15T15:54:34.469 INFO:tasks.workunit.client.0.smithi049.stdout:2/951: mkdir d3/d8/dad/dfe/d106/d151 0 2024-05-15T15:54:34.470 INFO:tasks.workunit.client.0.smithi049.stdout:2/952: chown d3/d8/dad 175 1 2024-05-15T15:54:34.471 INFO:tasks.workunit.client.0.smithi049.stdout:2/953: chown d3/da/d15/d24/f64 0 1 2024-05-15T15:54:34.473 INFO:tasks.workunit.client.0.smithi049.stdout:3/984: rename d12/d14/d52/db9/da6/de3/d113/d127 to d12/d14/d32/d64/d131 0 2024-05-15T15:54:34.474 INFO:tasks.workunit.client.0.smithi049.stdout:4/900: mknod d18/c123 0 2024-05-15T15:54:34.475 INFO:tasks.workunit.client.0.smithi049.stdout:9/968: symlink d0/l14b 0 2024-05-15T15:54:34.476 INFO:tasks.workunit.client.0.smithi049.stdout:2/954: truncate d3/da/d15/f39 947925 0 2024-05-15T15:54:34.480 INFO:tasks.workunit.client.0.smithi049.stdout:4/901: symlink d18/d69/l124 0 2024-05-15T15:54:34.481 INFO:tasks.workunit.client.0.smithi049.stdout:9/969: dread d0/d8/d21/f82 [0,4194304] 0 2024-05-15T15:54:34.482 INFO:tasks.workunit.client.0.smithi049.stdout:2/955: mknod d3/da/d15/d9e/c152 0 2024-05-15T15:54:34.486 INFO:tasks.workunit.client.0.smithi049.stdout:3/985: creat d12/d30/f132 x:0 0 0 2024-05-15T15:54:34.491 INFO:tasks.workunit.client.0.smithi049.stdout:2/956: dwrite d3/da/d15/d73/f11e [0,4194304] 0 2024-05-15T15:54:34.493 INFO:tasks.workunit.client.0.smithi049.stdout:3/986: unlink d12/d14/l2d 0 2024-05-15T15:54:34.496 INFO:tasks.workunit.client.0.smithi049.stdout:2/957: mknod d3/d78/d8c/d91/d7a/d11f/c153 0 2024-05-15T15:54:34.500 INFO:tasks.workunit.client.0.smithi049.stdout:2/958: dread d3/da/d15/d73/f8a [0,4194304] 0 2024-05-15T15:54:34.502 INFO:tasks.workunit.client.0.smithi049.stdout:3/987: mkdir d12/d14/d52/db9/dc4/d68/d70/d75/d9d/df6/d133 0 2024-05-15T15:54:34.506 INFO:tasks.workunit.client.0.smithi049.stdout:4/902: write d18/d1e/d63/d99/f117 [1813485,66890] 0 2024-05-15T15:54:34.510 INFO:tasks.workunit.client.0.smithi049.stdout:2/959: rename d3/db/d1d/d1e/d2b/d6b/fd2 to d3/db/d1d/d1e/d2b/d6b/f154 0 2024-05-15T15:54:34.513 INFO:tasks.workunit.client.0.smithi049.stdout:4/903: mknod d18/d1e/d2a/d50/dc4/c125 0 2024-05-15T15:54:34.514 INFO:tasks.workunit.client.0.smithi049.stdout:3/988: write d12/d14/d32/f6d [3082929,70800] 0 2024-05-15T15:54:34.515 INFO:tasks.workunit.client.0.smithi049.stdout:2/960: dread d3/d78/d8c/d11a/fca [0,4194304] 0 2024-05-15T15:54:34.518 INFO:tasks.workunit.client.0.smithi049.stdout:2/961: write d3/db/d1d/d1e/d2b/d3b/d42/f6f [2696060,57474] 0 2024-05-15T15:54:34.519 INFO:tasks.workunit.client.0.smithi049.stdout:2/962: readlink d3/d78/d8c/d11a/l118 0 2024-05-15T15:54:34.519 INFO:tasks.workunit.client.0.smithi049.stdout:4/904: dread d18/d21/d2b/d4e/f87 [0,4194304] 0 2024-05-15T15:54:34.521 INFO:tasks.workunit.client.0.smithi049.stdout:4/905: dread - d18/d1e/d2a/d50/dc4/ff5 zero size 2024-05-15T15:54:34.521 INFO:tasks.workunit.client.0.smithi049.stdout:3/989: symlink d12/d14/d52/db9/dc4/d68/d70/d75/d9d/l134 0 2024-05-15T15:54:34.527 INFO:tasks.workunit.client.0.smithi049.stdout:4/906: fsync d18/d1e/ff8 0 2024-05-15T15:54:34.528 INFO:tasks.workunit.client.0.smithi049.stdout:3/990: truncate d12/d14/d52/db9/dc4/fb2 1580651 0 2024-05-15T15:54:34.531 INFO:tasks.workunit.client.0.smithi049.stdout:4/907: unlink d18/l10e 0 2024-05-15T15:54:34.533 INFO:tasks.workunit.client.0.smithi049.stdout:3/991: mkdir d12/d26/d73/d135 0 2024-05-15T15:54:34.535 INFO:tasks.workunit.client.0.smithi049.stdout:3/992: rename d12/d67/da8/df1 to d12/d67/da8/df1/d120/d136 22 2024-05-15T15:54:34.535 INFO:tasks.workunit.client.0.smithi049.stdout:3/993: fsync d12/d14/d52/db9/f12a 0 2024-05-15T15:54:34.539 INFO:tasks.workunit.client.0.smithi049.stdout:2/963: getdents d3/db/d82 0 2024-05-15T15:54:34.541 INFO:tasks.workunit.client.0.smithi049.stdout:3/994: unlink d12/d14/d32/d64/d88/d9c/fb4 0 2024-05-15T15:54:34.543 INFO:tasks.workunit.client.0.smithi049.stdout:2/964: mknod d3/d78/d124/c155 0 2024-05-15T15:54:34.546 INFO:tasks.workunit.client.0.smithi049.stdout:2/965: read d3/db/d1d/d1e/d2b/d3b/d42/f4f [2281705,37499] 0 2024-05-15T15:54:34.547 INFO:tasks.workunit.client.0.smithi049.stdout:4/908: getdents d18/d1e/d2a/d50/d54/d109 0 2024-05-15T15:54:34.548 INFO:tasks.workunit.client.0.smithi049.stdout:3/995: mknod d12/d14/c137 0 2024-05-15T15:54:34.555 INFO:tasks.workunit.client.0.smithi049.stdout:3/996: link d12/d14/d32/d64/f93 d12/d67/da8/df1/d120/f138 0 2024-05-15T15:54:34.560 INFO:tasks.workunit.client.0.smithi049.stdout:3/997: dwrite d12/d30/dc1/dc7/df8/fd7 [0,4194304] 0 2024-05-15T15:54:34.565 INFO:tasks.workunit.client.0.smithi049.stdout:3/998: rename d12/d14/f27 to d12/d30/dc1/dc7/ddc/f139 0 2024-05-15T15:54:34.567 INFO:tasks.workunit.client.0.smithi049.stdout:3/999: dread - d12/d67/da8/ddd/ff4 zero size 2024-05-15T15:54:34.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:34 smithi144 ceph-mon[36389]: pgmap v37: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 98 MiB/s rd, 230 MiB/s wr, 541 op/s 2024-05-15T15:54:34.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:34 smithi049 ceph-mon[28399]: pgmap v37: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 98 MiB/s rd, 230 MiB/s wr, 541 op/s 2024-05-15T15:54:34.763 INFO:tasks.workunit.client.0.smithi049.stdout:7/928: sync 2024-05-15T15:54:34.781 INFO:tasks.workunit.client.0.smithi049.stdout:7/929: dwrite d0/d6/d1d/f4e [0,4194304] 0 2024-05-15T15:54:34.786 INFO:tasks.workunit.client.0.smithi049.stdout:7/930: rename d0/d6/d1d/fb7 to d0/d6/d23/d4a/d7b/f12d 0 2024-05-15T15:54:34.787 INFO:tasks.workunit.client.0.smithi049.stdout:7/931: write d0/d6/fcd [241207,69091] 0 2024-05-15T15:54:34.788 INFO:tasks.workunit.client.0.smithi049.stdout:7/932: write d0/d2f/f11f [978774,108426] 0 2024-05-15T15:54:34.790 INFO:tasks.workunit.client.0.smithi049.stdout:7/933: write d0/d6/d23/d76/ff4 [4654980,56267] 0 2024-05-15T15:54:34.794 INFO:tasks.workunit.client.0.smithi049.stdout:7/934: creat d0/d34/db3/f12e x:0 0 0 2024-05-15T15:54:34.795 INFO:tasks.workunit.client.0.smithi049.stdout:7/935: truncate d0/d2f/f11f 1465763 0 2024-05-15T15:54:34.798 INFO:tasks.workunit.client.0.smithi049.stdout:7/936: mkdir d0/d6/da/d12f 0 2024-05-15T15:54:34.801 INFO:tasks.workunit.client.0.smithi049.stdout:7/937: symlink d0/d6/d1d/df2/l130 0 2024-05-15T15:54:34.805 INFO:tasks.workunit.client.0.smithi049.stdout:7/938: mkdir d0/d6/d23/d4a/d131 0 2024-05-15T15:54:34.807 INFO:tasks.workunit.client.0.smithi049.stdout:7/939: write d0/d6/d23/d45/d6a/d102/f110 [687656,101691] 0 2024-05-15T15:54:34.814 INFO:tasks.workunit.client.0.smithi049.stdout:7/940: write d0/d6/f3c [4052038,75531] 0 2024-05-15T15:54:34.819 INFO:tasks.workunit.client.0.smithi049.stdout:7/941: getdents d0/d6/d23/d45/d6a/da1/d80 0 2024-05-15T15:54:34.819 INFO:tasks.workunit.client.0.smithi049.stdout:7/942: readlink d0/d6/d23/ld7 0 2024-05-15T15:54:34.822 INFO:tasks.workunit.client.0.smithi049.stdout:7/943: creat d0/ddb/dde/dcc/d103/f132 x:0 0 0 2024-05-15T15:54:34.829 INFO:tasks.workunit.client.0.smithi049.stdout:7/944: dwrite d0/d6/d23/d76/ff5 [0,4194304] 0 2024-05-15T15:54:34.834 INFO:tasks.workunit.client.0.smithi049.stdout:7/945: chown d0/d6/d23/d45/db1 0 1 2024-05-15T15:54:34.850 INFO:tasks.workunit.client.0.smithi049.stdout:7/946: dwrite d0/d6/d23/d45/d86/f104 [4194304,4194304] 0 2024-05-15T15:54:34.853 INFO:tasks.workunit.client.0.smithi049.stdout:7/947: fsync d0/d6/fc6 0 2024-05-15T15:54:34.872 INFO:tasks.workunit.client.0.smithi049.stdout:7/948: dread d0/d34/f35 [8388608,4194304] 0 2024-05-15T15:54:34.879 INFO:tasks.workunit.client.0.smithi049.stdout:7/949: write d0/d2f/d5c/f64 [3141178,96684] 0 2024-05-15T15:54:35.073 INFO:tasks.workunit.client.0.smithi049.stdout:4/909: sync 2024-05-15T15:54:35.073 INFO:tasks.workunit.client.0.smithi049.stdout:8/946: sync 2024-05-15T15:54:35.073 INFO:tasks.workunit.client.0.smithi049.stdout:9/970: sync 2024-05-15T15:54:35.073 INFO:tasks.workunit.client.0.smithi049.stdout:2/966: sync 2024-05-15T15:54:35.073 INFO:tasks.workunit.client.0.smithi049.stdout:7/950: sync 2024-05-15T15:54:35.076 INFO:tasks.workunit.client.0.smithi049.stdout:2/967: creat d3/d8/dad/dfe/d106/f156 x:0 0 0 2024-05-15T15:54:35.078 INFO:tasks.workunit.client.0.smithi049.stdout:4/910: mkdir d18/d1e/d2a/d50/d54/dad/d126 0 2024-05-15T15:54:35.080 INFO:tasks.workunit.client.0.smithi049.stdout:8/947: mknod d2/de/d71/d86/d5d/d7d/db8/d5b/da6/dc7/c129 0 2024-05-15T15:54:35.082 INFO:tasks.workunit.client.0.smithi049.stdout:8/948: chown d2/de/d71/d86/d5d/d7d/d1b/d69/fde 25053369 1 2024-05-15T15:54:35.082 INFO:tasks.workunit.client.0.smithi049.stdout:2/968: mknod d3/db/d5b/dbf/dc1/c157 0 2024-05-15T15:54:35.083 INFO:tasks.workunit.client.0.smithi049.stdout:4/911: creat d18/d76/f127 x:0 0 0 2024-05-15T15:54:35.083 INFO:tasks.workunit.client.0.smithi049.stdout:8/949: chown d2/de/d95/cef 29 1 2024-05-15T15:54:35.085 INFO:tasks.workunit.client.0.smithi049.stdout:9/971: link d0/d8/d21/f118 d0/d8/dd6/de2/dec/f14c 0 2024-05-15T15:54:35.086 INFO:tasks.workunit.client.0.smithi049.stdout:7/951: creat d0/d6/d23/d4a/d7b/dc4/f133 x:0 0 0 2024-05-15T15:54:35.089 INFO:tasks.workunit.client.0.smithi049.stdout:4/912: rmdir d18/d1e/d2a/d36/d5d 39 2024-05-15T15:54:35.091 INFO:tasks.workunit.client.0.smithi049.stdout:7/952: mkdir d0/ddb/d134 0 2024-05-15T15:54:35.095 INFO:tasks.workunit.client.0.smithi049.stdout:2/969: dwrite d3/db/d82/ffd [0,4194304] 0 2024-05-15T15:54:35.099 INFO:tasks.workunit.client.0.smithi049.stdout:4/913: dwrite d18/d21/d2b/fb2 [0,4194304] 0 2024-05-15T15:54:35.101 INFO:tasks.workunit.client.0.smithi049.stdout:7/953: dread d0/d2f/d5c/f70 [0,4194304] 0 2024-05-15T15:54:35.110 INFO:tasks.workunit.client.0.smithi049.stdout:4/914: mknod d18/d1e/d2a/d50/d54/d109/db6/c128 0 2024-05-15T15:54:35.116 INFO:tasks.workunit.client.0.smithi049.stdout:7/954: dread d0/d6/d23/d76/ff4 [0,4194304] 0 2024-05-15T15:54:35.117 INFO:tasks.workunit.client.0.smithi049.stdout:7/955: write d0/d6/d23/d76/ff5 [841552,78734] 0 2024-05-15T15:54:35.119 INFO:tasks.workunit.client.0.smithi049.stdout:7/956: chown d0/d6/d23/d45/d86/fc5 639225 1 2024-05-15T15:54:35.120 INFO:tasks.workunit.client.0.smithi049.stdout:7/957: write d0/d6/d23/d45/f11d [780199,30308] 0 2024-05-15T15:54:35.121 INFO:tasks.workunit.client.0.smithi049.stdout:4/915: mknod d18/d21/c129 0 2024-05-15T15:54:35.123 INFO:tasks.workunit.client.0.smithi049.stdout:2/970: getdents d3/d8/dad/dfe/d106 0 2024-05-15T15:54:35.128 INFO:tasks.workunit.client.0.smithi049.stdout:4/916: fsync d18/d1e/d2a/f6a 0 2024-05-15T15:54:35.129 INFO:tasks.workunit.client.0.smithi049.stdout:4/917: readlink d18/d1e/d63/d99/de1/l6b 0 2024-05-15T15:54:35.131 INFO:tasks.workunit.client.0.smithi049.stdout:2/971: truncate d3/db/d1d/d1e/d2b/d3b/d42/f65 218096 0 2024-05-15T15:54:35.133 INFO:tasks.workunit.client.0.smithi049.stdout:8/950: write d2/de/d95/fd1 [693587,105441] 0 2024-05-15T15:54:35.137 INFO:tasks.workunit.client.0.smithi049.stdout:2/972: creat d3/d8/dad/dfe/d106/f158 x:0 0 0 2024-05-15T15:54:35.138 INFO:tasks.workunit.client.0.smithi049.stdout:8/951: rmdir d2/de/d71/d86/d5d/d7d/d45 39 2024-05-15T15:54:35.138 INFO:tasks.workunit.client.0.smithi049.stdout:2/973: chown d3/d8/dad/dfe/c141 127837 1 2024-05-15T15:54:35.143 INFO:tasks.workunit.client.0.smithi049.stdout:2/974: creat d3/db/d1d/d1e/d2b/d3b/d47/d59/f159 x:0 0 0 2024-05-15T15:54:35.149 INFO:tasks.workunit.client.0.smithi049.stdout:2/975: symlink d3/da/d15/d24/d107/l15a 0 2024-05-15T15:54:35.150 INFO:tasks.workunit.client.0.smithi049.stdout:7/958: write d0/d6/d1d/df2/f11a [1589884,58962] 0 2024-05-15T15:54:35.151 INFO:tasks.workunit.client.0.smithi049.stdout:7/959: chown d0/d6/fc6 0 1 2024-05-15T15:54:35.154 INFO:tasks.workunit.client.0.smithi049.stdout:8/952: write d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/d11e/f128 [773443,20366] 0 2024-05-15T15:54:35.155 INFO:tasks.workunit.client.0.smithi049.stdout:7/960: symlink d0/d6/d23/d45/db1/l135 0 2024-05-15T15:54:35.158 INFO:tasks.workunit.client.0.smithi049.stdout:8/953: symlink d2/de/dc3/l12a 0 2024-05-15T15:54:35.163 INFO:tasks.workunit.client.0.smithi049.stdout:7/961: rename d0/d6/d23/d45/d6a/ca7 to d0/d6/d23/d4a/d7b/de2/c136 0 2024-05-15T15:54:35.164 INFO:tasks.workunit.client.0.smithi049.stdout:7/962: write d0/d6/fcd [703842,40246] 0 2024-05-15T15:54:35.167 INFO:tasks.workunit.client.0.smithi049.stdout:8/954: mknod d2/c12b 0 2024-05-15T15:54:35.173 INFO:tasks.workunit.client.0.smithi049.stdout:8/955: dwrite d2/d6a/f110 [0,4194304] 0 2024-05-15T15:54:35.175 INFO:tasks.workunit.client.0.smithi049.stdout:2/976: write d3/da/d15/dec/ddf/fe5 [866853,93272] 0 2024-05-15T15:54:35.187 INFO:tasks.workunit.client.0.smithi049.stdout:7/963: write d0/d6/d23/ffa [845944,111864] 0 2024-05-15T15:54:35.187 INFO:tasks.workunit.client.0.smithi049.stdout:2/977: link d3/da/c12f d3/da/d15/d73/c15b 0 2024-05-15T15:54:35.191 INFO:tasks.workunit.client.0.smithi049.stdout:7/964: creat d0/d6/d23/d4a/d7b/de2/f137 x:0 0 0 2024-05-15T15:54:35.193 INFO:tasks.workunit.client.0.smithi049.stdout:2/978: creat d3/d8/dad/dfe/d106/d151/f15c x:0 0 0 2024-05-15T15:54:35.196 INFO:tasks.workunit.client.0.smithi049.stdout:2/979: creat d3/db/d82/f15d x:0 0 0 2024-05-15T15:54:35.196 INFO:tasks.workunit.client.0.smithi049.stdout:7/965: creat d0/d6/da/f138 x:0 0 0 2024-05-15T15:54:35.197 INFO:tasks.workunit.client.0.smithi049.stdout:2/980: dread - d3/da/d15/d24/f12e zero size 2024-05-15T15:54:35.198 INFO:tasks.workunit.client.0.smithi049.stdout:2/981: chown d3/d52/d116/f105 6 1 2024-05-15T15:54:35.200 INFO:tasks.workunit.client.0.smithi049.stdout:8/956: dread d2/d18/f47 [0,4194304] 0 2024-05-15T15:54:35.201 INFO:tasks.workunit.client.0.smithi049.stdout:2/982: fdatasync d3/db/d5b/f74 0 2024-05-15T15:54:35.202 INFO:tasks.workunit.client.0.smithi049.stdout:8/957: fdatasync d2/de/d71/d86/f122 0 2024-05-15T15:54:35.204 INFO:tasks.workunit.client.0.smithi049.stdout:8/958: dread - d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/fea zero size 2024-05-15T15:54:35.205 INFO:tasks.workunit.client.0.smithi049.stdout:2/983: dread d3/db/d1d/d1e/d2b/d3b/d47/d59/f5d [0,4194304] 0 2024-05-15T15:54:35.209 INFO:tasks.workunit.client.0.smithi049.stdout:2/984: creat d3/db/d1d/d1e/d2b/d2e/f15e x:0 0 0 2024-05-15T15:54:35.210 INFO:tasks.workunit.client.0.smithi049.stdout:2/985: chown d3/da/d15/dec/ddf/c10c 464421008 1 2024-05-15T15:54:35.211 INFO:tasks.workunit.client.0.smithi049.stdout:8/959: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d60/da5/f12c x:0 0 0 2024-05-15T15:54:35.214 INFO:tasks.workunit.client.0.smithi049.stdout:2/986: mknod d3/db/d1d/d1e/d2b/c15f 0 2024-05-15T15:54:35.217 INFO:tasks.workunit.client.0.smithi049.stdout:7/966: dwrite d0/d6/d23/d45/d6a/da1/f2c [0,4194304] 0 2024-05-15T15:54:35.218 INFO:tasks.workunit.client.0.smithi049.stdout:8/960: mkdir d2/de/d71/d86/d5d/d7d/db8/d5b/da6/d127/d12d 0 2024-05-15T15:54:35.221 INFO:tasks.workunit.client.0.smithi049.stdout:2/987: fsync d3/db/d1d/d1e/d2b/d6b/fd5 0 2024-05-15T15:54:35.229 INFO:tasks.workunit.client.0.smithi049.stdout:2/988: mkdir d3/db/d1d/d1e/d2b/d3b/df8/d12a/d160 0 2024-05-15T15:54:35.230 INFO:tasks.workunit.client.0.smithi049.stdout:2/989: chown d3/db/d1d/d1e/d2b/d3b/d42/f6f 1010 1 2024-05-15T15:54:35.238 INFO:tasks.workunit.client.0.smithi049.stdout:2/990: dread d3/db/d1d/d1e/d2b/d6b/f80 [0,4194304] 0 2024-05-15T15:54:35.244 INFO:tasks.workunit.client.0.smithi049.stdout:8/961: dread d2/de/d71/d86/d5d/d7d/fb3 [0,4194304] 0 2024-05-15T15:54:35.245 INFO:tasks.workunit.client.0.smithi049.stdout:8/962: chown d2/d18/l29 249432346 1 2024-05-15T15:54:35.248 INFO:tasks.workunit.client.0.smithi049.stdout:8/963: chown d2/de/d71/d86/d5d/d7d/db8/d5b/d60/da5/c116 7368 1 2024-05-15T15:54:35.252 INFO:tasks.workunit.client.0.smithi049.stdout:2/991: dread d3/da/d15/fc0 [0,4194304] 0 2024-05-15T15:54:35.252 INFO:tasks.workunit.client.0.smithi049.stdout:8/964: fsync d2/de/d71/d86/fdd 0 2024-05-15T15:54:35.253 INFO:tasks.workunit.client.0.smithi049.stdout:7/967: dwrite d0/d34/db3/fc3 [0,4194304] 0 2024-05-15T15:54:35.254 INFO:tasks.workunit.client.0.smithi049.stdout:2/992: fsync d3/d78/d8c/d91/d7a/f101 0 2024-05-15T15:54:35.255 INFO:tasks.workunit.client.0.smithi049.stdout:7/968: chown d0/d6/da/fe1 1 1 2024-05-15T15:54:35.265 INFO:tasks.workunit.client.0.smithi049.stdout:2/993: rename d3/da/d15/dec/ddf to d3/db/d5b/d161 0 2024-05-15T15:54:35.271 INFO:tasks.workunit.client.0.smithi049.stdout:7/969: mkdir d0/d6/d23/d45/d6a/d124/d139 0 2024-05-15T15:54:35.279 INFO:tasks.workunit.client.0.smithi049.stdout:2/994: getdents d3/db/d1d/d1e/d2b 0 2024-05-15T15:54:35.279 INFO:tasks.workunit.client.0.smithi049.stdout:8/965: dwrite d2/de/d71/d86/d5d/d7d/db8/f85 [0,4194304] 0 2024-05-15T15:54:35.289 INFO:tasks.workunit.client.0.smithi049.stdout:7/970: getdents d0/d34 0 2024-05-15T15:54:35.292 INFO:tasks.workunit.client.0.smithi049.stdout:2/995: unlink d3/d8/cae 0 2024-05-15T15:54:35.297 INFO:tasks.workunit.client.0.smithi049.stdout:2/996: creat d3/da/d15/dec/f162 x:0 0 0 2024-05-15T15:54:35.299 INFO:tasks.workunit.client.0.smithi049.stdout:8/966: creat d2/de/d71/d86/d5d/d7d/db8/d5b/f12e x:0 0 0 2024-05-15T15:54:35.301 INFO:tasks.workunit.client.0.smithi049.stdout:7/971: mknod d0/d6/d23/d45/d6a/d114/c13a 0 2024-05-15T15:54:35.306 INFO:tasks.workunit.client.0.smithi049.stdout:2/997: truncate d3/da/f45 978090 0 2024-05-15T15:54:35.307 INFO:tasks.workunit.client.0.smithi049.stdout:2/998: write d3/db/d82/f15d [610791,42669] 0 2024-05-15T15:54:35.308 INFO:tasks.workunit.client.0.smithi049.stdout:2/999: chown d3/d78/lcd 561813365 1 2024-05-15T15:54:35.309 INFO:tasks.workunit.client.0.smithi049.stdout:8/967: rename d2/de/d71/d86/d5d/d7d/d45/cf6 to d2/de/d25/d74/dbe/c12f 0 2024-05-15T15:54:35.311 INFO:tasks.workunit.client.0.smithi049.stdout:8/968: read d2/fc5 [3272619,108252] 0 2024-05-15T15:54:35.326 INFO:tasks.workunit.client.0.smithi049.stdout:7/972: write d0/d2f/f8e [5249595,127075] 0 2024-05-15T15:54:35.330 INFO:tasks.workunit.client.0.smithi049.stdout:7/973: unlink d0/d2f/d5c/f7d 0 2024-05-15T15:54:35.330 INFO:tasks.workunit.client.0.smithi049.stdout:8/969: rmdir d2/de/d25/d74/dbe 39 2024-05-15T15:54:35.331 INFO:tasks.workunit.client.0.smithi049.stdout:8/970: write d2/de/d95/fd1 [3066886,27631] 0 2024-05-15T15:54:35.351 INFO:tasks.workunit.client.0.smithi049.stdout:7/974: getdents d0/d6/d23/d45/db1 0 2024-05-15T15:54:35.354 INFO:tasks.workunit.client.0.smithi049.stdout:7/975: unlink d0/d2f/d5c/f64 0 2024-05-15T15:54:35.359 INFO:tasks.workunit.client.0.smithi049.stdout:7/976: dwrite d0/d6/d23/ffa [0,4194304] 0 2024-05-15T15:54:35.362 INFO:tasks.workunit.client.0.smithi049.stdout:7/977: rmdir d0/d6/d23/d45/d6a 39 2024-05-15T15:54:35.366 INFO:tasks.workunit.client.0.smithi049.stdout:8/971: dwrite d2/de/d71/d86/d5d/d7d/db8/d5b/d60/da5/fd6 [0,4194304] 0 2024-05-15T15:54:35.368 INFO:tasks.workunit.client.0.smithi049.stdout:7/978: creat d0/d6/da/f13b x:0 0 0 2024-05-15T15:54:35.372 INFO:tasks.workunit.client.0.smithi049.stdout:7/979: chown d0/d6/d23/d45/d6a/da1/f33 630822 1 2024-05-15T15:54:35.374 INFO:tasks.workunit.client.0.smithi049.stdout:8/972: creat d2/de/d71/d86/d5d/d7d/db8/d5b/da6/d127/f130 x:0 0 0 2024-05-15T15:54:35.377 INFO:tasks.workunit.client.0.smithi049.stdout:8/973: stat d2/de/d71/d86/d5d/d7d/d1b/d69/fde 0 2024-05-15T15:54:35.379 INFO:tasks.workunit.client.0.smithi049.stdout:7/980: dwrite d0/d6/d23/faf [0,4194304] 0 2024-05-15T15:54:35.384 INFO:tasks.workunit.client.0.smithi049.stdout:8/974: mkdir d2/de/d71/d86/d5d/d7d/db8/d5b/d93/dc2/d131 0 2024-05-15T15:54:35.385 INFO:tasks.workunit.client.0.smithi049.stdout:7/981: fsync d0/d6/da/fea 0 2024-05-15T15:54:35.387 INFO:tasks.workunit.client.0.smithi049.stdout:7/982: write d0/ddb/dde/dcc/d103/f132 [94173,19760] 0 2024-05-15T15:54:35.389 INFO:tasks.workunit.client.0.smithi049.stdout:8/975: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d93/dc2/f132 x:0 0 0 2024-05-15T15:54:35.394 INFO:tasks.workunit.client.0.smithi049.stdout:7/983: mknod d0/d6/d23/d4a/d7b/dc4/c13c 0 2024-05-15T15:54:35.402 INFO:tasks.workunit.client.0.smithi049.stdout:7/984: write d0/d34/f46 [4827271,30168] 0 2024-05-15T15:54:35.408 INFO:tasks.workunit.client.0.smithi049.stdout:7/985: getdents d0/d6/d1d 0 2024-05-15T15:54:35.412 INFO:tasks.workunit.client.0.smithi049.stdout:7/986: getdents d0 0 2024-05-15T15:54:35.414 INFO:tasks.workunit.client.0.smithi049.stdout:7/987: symlink d0/d6/d23/d4a/l13d 0 2024-05-15T15:54:35.422 INFO:tasks.workunit.client.0.smithi049.stdout:7/988: dwrite d0/d6/d23/d45/d86/fc5 [0,4194304] 0 2024-05-15T15:54:35.428 INFO:tasks.workunit.client.0.smithi049.stdout:7/989: write d0/d6/d1d/f59 [180817,55244] 0 2024-05-15T15:54:35.430 INFO:tasks.workunit.client.0.smithi049.stdout:7/990: stat d0/d6/d23/d4a/d7b/dc4/c109 0 2024-05-15T15:54:35.435 INFO:tasks.workunit.client.0.smithi049.stdout:7/991: creat d0/ddb/dde/f13e x:0 0 0 2024-05-15T15:54:35.447 INFO:tasks.workunit.client.0.smithi049.stdout:7/992: dwrite d0/d2f/f8f [0,4194304] 0 2024-05-15T15:54:35.452 INFO:tasks.workunit.client.0.smithi049.stdout:7/993: getdents d0/ddb/dde/dca/d115 0 2024-05-15T15:54:35.461 INFO:tasks.workunit.client.0.smithi049.stdout:7/994: rename d0/d6/d23/d4a/f125 to d0/d6/d23/f13f 0 2024-05-15T15:54:35.468 INFO:tasks.workunit.client.0.smithi049.stdout:7/995: truncate d0/d34/db3/f100 161408 0 2024-05-15T15:54:35.470 INFO:tasks.workunit.client.0.smithi049.stdout:7/996: symlink d0/d2f/d5c/l140 0 2024-05-15T15:54:35.470 INFO:tasks.workunit.client.0.smithi049.stdout:7/997: chown d0/d34/f46 25525745 1 2024-05-15T15:54:35.472 INFO:tasks.workunit.client.0.smithi049.stdout:7/998: write d0/d6/d23/d4a/d7b/dc4/de0/d113/f10c [60800,117168] 0 2024-05-15T15:54:35.474 INFO:tasks.workunit.client.0.smithi049.stdout:7/999: creat d0/d6/d23/d76/f141 x:0 0 0 2024-05-15T15:54:35.504 INFO:tasks.workunit.client.0.smithi049.stdout:9/972: sync 2024-05-15T15:54:35.508 INFO:tasks.workunit.client.0.smithi049.stdout:9/973: unlink d0/d8/d9/d22/dbb/cf5 0 2024-05-15T15:54:35.511 INFO:tasks.workunit.client.0.smithi049.stdout:9/974: unlink d0/d8/d9/d22/dbb/l148 0 2024-05-15T15:54:35.512 INFO:tasks.workunit.client.0.smithi049.stdout:9/975: dread - d0/d8/d12/d9a/f133 zero size 2024-05-15T15:54:35.513 INFO:tasks.workunit.client.0.smithi049.stdout:9/976: chown d0/d8/d12/d9a/lf2 484371 1 2024-05-15T15:54:35.515 INFO:tasks.workunit.client.0.smithi049.stdout:9/977: truncate d0/d8/fd9 25069 0 2024-05-15T15:54:35.517 INFO:tasks.workunit.client.0.smithi049.stdout:9/978: mkdir d0/d8/d9/d52/dc7/de9/d14d 0 2024-05-15T15:54:35.749 INFO:tasks.workunit.client.0.smithi049.stdout:8/976: sync 2024-05-15T15:54:35.749 INFO:tasks.workunit.client.0.smithi049.stdout:4/918: sync 2024-05-15T15:54:35.749 INFO:tasks.workunit.client.0.smithi049.stdout:9/979: sync 2024-05-15T15:54:35.753 INFO:tasks.workunit.client.0.smithi049.stdout:4/919: fsync d18/d1e/d2a/f6a 0 2024-05-15T15:54:35.766 INFO:tasks.workunit.client.0.smithi049.stdout:4/920: write d18/d1e/ff8 [690177,13016] 0 2024-05-15T15:54:35.767 INFO:tasks.workunit.client.0.smithi049.stdout:8/977: getdents d2/de/d71/d86/d5d/d7d/db8/d5b/da6/dc7 0 2024-05-15T15:54:35.772 INFO:tasks.workunit.client.0.smithi049.stdout:4/921: mknod d18/d76/c12a 0 2024-05-15T15:54:35.772 INFO:tasks.workunit.client.0.smithi049.stdout:9/980: dwrite d0/d8/d53/f113 [0,4194304] 0 2024-05-15T15:54:35.773 INFO:tasks.workunit.client.0.smithi049.stdout:9/981: chown d0/d8/d12 20264 1 2024-05-15T15:54:35.777 INFO:tasks.workunit.client.0.smithi049.stdout:8/978: mkdir d2/de/dc3/d133 0 2024-05-15T15:54:35.781 INFO:tasks.workunit.client.0.smithi049.stdout:4/922: dwrite d18/d1e/d63/d10d/f113 [0,4194304] 0 2024-05-15T15:54:35.783 INFO:tasks.workunit.client.0.smithi049.stdout:9/982: link d0/d8/d12/d9a/f133 d0/d8/dd6/de2/dec/d9d/f14e 0 2024-05-15T15:54:35.788 INFO:tasks.workunit.client.0.smithi049.stdout:8/979: symlink d2/de/d71/d86/d5d/d7d/db8/d5b/d60/da5/l134 0 2024-05-15T15:54:35.790 INFO:tasks.workunit.client.0.smithi049.stdout:4/923: dwrite d18/d1e/d2a/d36/d86/f115 [0,4194304] 0 2024-05-15T15:54:35.797 INFO:tasks.workunit.client.0.smithi049.stdout:8/980: truncate d2/de/f16 571524 0 2024-05-15T15:54:35.799 INFO:tasks.workunit.client.0.smithi049.stdout:4/924: creat d18/d1e/d2a/d36/d86/f12b x:0 0 0 2024-05-15T15:54:35.800 INFO:tasks.workunit.client.0.smithi049.stdout:4/925: readlink d18/d21/d2b/d4e/l112 0 2024-05-15T15:54:35.800 INFO:tasks.workunit.client.0.smithi049.stdout:4/926: chown d18 24 1 2024-05-15T15:54:35.804 INFO:tasks.workunit.client.0.smithi049.stdout:4/927: symlink d18/d1e/d2a/d50/l12c 0 2024-05-15T15:54:35.811 INFO:tasks.workunit.client.0.smithi049.stdout:4/928: dwrite d18/d1e/d2a/d50/dc4/ffe [0,4194304] 0 2024-05-15T15:54:35.828 INFO:tasks.workunit.client.0.smithi049.stdout:4/929: dwrite d18/d1e/d2a/f38 [0,4194304] 0 2024-05-15T15:54:35.833 INFO:tasks.workunit.client.0.smithi049.stdout:4/930: rename d18/d1e/d2a/f38 to d18/d1e/d63/d99/f12d 0 2024-05-15T15:54:35.834 INFO:tasks.workunit.client.0.smithi049.stdout:4/931: truncate d18/fd9 1686315 0 2024-05-15T15:54:35.837 INFO:tasks.workunit.client.0.smithi049.stdout:4/932: stat d18/d1e/d2a/d50/d54/d109/d98/dc3/cdf 0 2024-05-15T15:54:35.840 INFO:tasks.workunit.client.0.smithi049.stdout:4/933: fsync d18/d1e/d63/d99/ff4 0 2024-05-15T15:54:35.853 INFO:tasks.workunit.client.0.smithi049.stdout:4/934: dwrite d18/d1e/d2a/d36/d5d/f67 [4194304,4194304] 0 2024-05-15T15:54:35.858 INFO:tasks.workunit.client.0.smithi049.stdout:4/935: truncate d18/d1e/d63/d99/db9/fe0 332654 0 2024-05-15T15:54:36.107 INFO:tasks.workunit.client.0.smithi049.stdout:9/983: sync 2024-05-15T15:54:36.113 INFO:tasks.workunit.client.0.smithi049.stdout:9/984: dwrite d0/d8/d1b/f32 [0,4194304] 0 2024-05-15T15:54:36.114 INFO:tasks.workunit.client.0.smithi049.stdout:9/985: stat d0/d8/d9/d52/d5e 0 2024-05-15T15:54:36.120 INFO:tasks.workunit.client.0.smithi049.stdout:9/986: creat d0/d8/d12/f14f x:0 0 0 2024-05-15T15:54:36.124 INFO:tasks.workunit.client.0.smithi049.stdout:9/987: creat d0/d8/d53/dd2/df3/f150 x:0 0 0 2024-05-15T15:54:36.127 INFO:tasks.workunit.client.0.smithi049.stdout:9/988: symlink d0/d8/d9/d52/d5e/d5f/d56/l151 0 2024-05-15T15:54:36.130 INFO:tasks.workunit.client.0.smithi049.stdout:9/989: unlink d0/d8/d9/le4 0 2024-05-15T15:54:36.133 INFO:tasks.workunit.client.0.smithi049.stdout:9/990: mkdir d0/d8/d9/d52/d5e/d152 0 2024-05-15T15:54:36.134 INFO:tasks.workunit.client.0.smithi049.stdout:9/991: fdatasync d0/d8/d12/dfd/f115 0 2024-05-15T15:54:36.135 INFO:tasks.workunit.client.0.smithi049.stdout:9/992: creat d0/d8/d9/d45/f153 x:0 0 0 2024-05-15T15:54:36.158 INFO:tasks.workunit.client.0.smithi049.stdout:8/981: sync 2024-05-15T15:54:36.158 INFO:tasks.workunit.client.0.smithi049.stdout:4/936: sync 2024-05-15T15:54:36.159 INFO:tasks.workunit.client.0.smithi049.stdout:9/993: dwrite d0/d8/d9/d52/d5e/d8a/f1f [0,4194304] 0 2024-05-15T15:54:36.161 INFO:tasks.workunit.client.0.smithi049.stdout:4/937: chown l15 90493262 1 2024-05-15T15:54:36.163 INFO:tasks.workunit.client.0.smithi049.stdout:8/982: truncate d2/fc5 1046654 0 2024-05-15T15:54:36.164 INFO:tasks.workunit.client.0.smithi049.stdout:9/994: dread d0/d8/d9/d52/d5e/d5f/f123 [0,4194304] 0 2024-05-15T15:54:36.166 INFO:tasks.workunit.client.0.smithi049.stdout:4/938: mknod d18/d21/d2b/d4e/c12e 0 2024-05-15T15:54:36.170 INFO:tasks.workunit.client.0.smithi049.stdout:9/995: mknod d0/d8/d9/d52/d5e/d8a/dd3/d87/c154 0 2024-05-15T15:54:36.172 INFO:tasks.workunit.client.0.smithi049.stdout:9/996: read d0/d8/d9/d22/f2e [522360,90204] 0 2024-05-15T15:54:36.174 INFO:tasks.workunit.client.0.smithi049.stdout:9/997: write d0/d8/d12/dfd/f115 [172267,67284] 0 2024-05-15T15:54:36.176 INFO:tasks.workunit.client.0.smithi049.stdout:8/983: creat d2/de/d71/d86/d5d/d7d/db8/d5b/d60/dbf/d11e/f135 x:0 0 0 2024-05-15T15:54:36.182 INFO:tasks.workunit.client.0.smithi049.stdout:9/998: unlink d0/d8/d9/d52/d5e/d8a/dd3/d87/c154 0 2024-05-15T15:54:36.189 INFO:tasks.workunit.client.0.smithi049.stdout:4/939: truncate d18/d1e/d2a/d50/dc4/ffe 926694 0 2024-05-15T15:54:36.192 INFO:tasks.workunit.client.0.smithi049.stdout:8/984: write d2/de/d71/d86/d5d/d7d/d1b/f36 [2307170,62556] 0 2024-05-15T15:54:36.194 INFO:tasks.workunit.client.0.smithi049.stdout:9/999: unlink d0/d8/d9/fa 0 2024-05-15T15:54:36.195 INFO:tasks.workunit.client.0.smithi049.stdout:4/940: creat d18/d1e/d2a/d50/d54/f12f x:0 0 0 2024-05-15T15:54:36.200 INFO:tasks.workunit.client.0.smithi049.stdout:8/985: truncate d2/de/d25/fbd 1831061 0 2024-05-15T15:54:36.202 INFO:tasks.workunit.client.0.smithi049.stdout:4/941: creat d18/d1e/d2a/f130 x:0 0 0 2024-05-15T15:54:36.205 INFO:tasks.workunit.client.0.smithi049.stdout:8/986: symlink d2/de/dc3/d133/l136 0 2024-05-15T15:54:36.206 INFO:tasks.workunit.client.0.smithi049.stdout:4/942: mknod d18/d1e/d2a/d50/d54/dad/c131 0 2024-05-15T15:54:36.208 INFO:tasks.workunit.client.0.smithi049.stdout:4/943: write d18/d1e/d63/d99/fa5 [835045,34085] 0 2024-05-15T15:54:36.209 INFO:tasks.workunit.client.0.smithi049.stdout:4/944: chown d18/d1e/d2a/d50/lc9 11 1 2024-05-15T15:54:36.210 INFO:tasks.workunit.client.0.smithi049.stdout:4/945: chown d18/d1e/d2a/d36 29 1 2024-05-15T15:54:36.212 INFO:tasks.workunit.client.0.smithi049.stdout:4/946: read d18/d1e/d2a/d50/d54/d109/fab [3585972,127333] 0 2024-05-15T15:54:36.219 INFO:tasks.workunit.client.0.smithi049.stdout:8/987: dread d2/de/d71/d86/d5d/d7d/d1d/f61 [0,4194304] 0 2024-05-15T15:54:36.221 INFO:tasks.workunit.client.0.smithi049.stdout:4/947: truncate d18/d1e/d2a/d50/d54/d109/fd7 470672 0 2024-05-15T15:54:36.224 INFO:tasks.workunit.client.0.smithi049.stdout:8/988: creat d2/de/d25/d8c/f137 x:0 0 0 2024-05-15T15:54:36.229 INFO:tasks.workunit.client.0.smithi049.stdout:4/948: dread d18/d1e/d2a/f93 [0,4194304] 0 2024-05-15T15:54:36.230 INFO:tasks.workunit.client.0.smithi049.stdout:8/989: fsync d2/de/d25/d74/f87 0 2024-05-15T15:54:36.230 INFO:tasks.workunit.client.0.smithi049.stdout:4/949: stat d18/d1e/d63 0 2024-05-15T15:54:36.232 INFO:tasks.workunit.client.0.smithi049.stdout:8/990: stat d2/de/d71/d86/d5d/d7d/db8/d5b/d60/da5/f12c 0 2024-05-15T15:54:36.235 INFO:tasks.workunit.client.0.smithi049.stdout:4/950: truncate d18/d1e/d2a/f6a 1581666 0 2024-05-15T15:54:36.237 INFO:tasks.workunit.client.0.smithi049.stdout:8/991: dread d2/de/d71/d86/d5d/d7d/db8/f85 [0,4194304] 0 2024-05-15T15:54:36.246 INFO:tasks.workunit.client.0.smithi049.stdout:8/992: symlink d2/l138 0 2024-05-15T15:54:36.251 INFO:tasks.workunit.client.0.smithi049.stdout:8/993: rename d2/de/d71/d86/d5d/d7d/d45/cfe to d2/de/d25/d74/dbe/d123/c139 0 2024-05-15T15:54:36.253 INFO:tasks.workunit.client.0.smithi049.stdout:4/951: link d18/cba d18/d21/d2b/c132 0 2024-05-15T15:54:36.255 INFO:tasks.workunit.client.0.smithi049.stdout:8/994: symlink d2/de/d71/d86/d5d/d7d/db8/d5b/da6/d127/l13a 0 2024-05-15T15:54:36.255 INFO:tasks.workunit.client.0.smithi049.stdout:8/995: chown d2/f2c 51 1 2024-05-15T15:54:36.258 INFO:tasks.workunit.client.0.smithi049.stdout:4/952: mkdir d18/d21/d2b/d133 0 2024-05-15T15:54:36.258 INFO:tasks.workunit.client.0.smithi049.stdout:4/953: stat d18/d1e/d2a/d36/d86/c8d 0 2024-05-15T15:54:36.259 INFO:tasks.workunit.client.0.smithi049.stdout:8/996: fsync d2/de/d25/d74/d94/f104 0 2024-05-15T15:54:36.266 INFO:tasks.workunit.client.0.smithi049.stdout:4/954: dwrite d18/d1e/d2a/f111 [0,4194304] 0 2024-05-15T15:54:36.275 INFO:tasks.workunit.client.0.smithi049.stdout:8/997: rmdir d2/de/d71/d86/d5d/de3/d121 0 2024-05-15T15:54:36.285 INFO:tasks.workunit.client.0.smithi049.stdout:8/998: getdents d2/de/d71 0 2024-05-15T15:54:36.289 INFO:tasks.workunit.client.0.smithi049.stdout:4/955: write f16 [8796022,18524] 0 2024-05-15T15:54:36.291 INFO:tasks.workunit.client.0.smithi049.stdout:4/956: fdatasync d18/d1e/d63/d10d/ffb 0 2024-05-15T15:54:36.291 INFO:tasks.workunit.client.0.smithi049.stdout:8/999: symlink d2/de/d71/d86/d5d/d7d/l13b 0 2024-05-15T15:54:36.294 INFO:tasks.workunit.client.0.smithi049.stdout:4/957: creat d18/d1e/d2a/d36/d86/f134 x:0 0 0 2024-05-15T15:54:36.299 INFO:tasks.workunit.client.0.smithi049.stdout:4/958: getdents d18/d21/d114 0 2024-05-15T15:54:36.303 INFO:tasks.workunit.client.0.smithi049.stdout:4/959: symlink d18/d1e/d2a/d50/d54/d109/dff/l135 0 2024-05-15T15:54:36.303 INFO:tasks.workunit.client.0.smithi049.stdout:4/960: chown d18/d21 224 1 2024-05-15T15:54:36.318 INFO:tasks.workunit.client.0.smithi049.stdout:4/961: dwrite d18/f31 [0,4194304] 0 2024-05-15T15:54:36.616 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:36 smithi049 ceph-mon[28399]: pgmap v38: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 154 MiB/s wr, 385 op/s 2024-05-15T15:54:36.677 INFO:tasks.workunit.client.0.smithi049.stdout:4/962: sync 2024-05-15T15:54:36.679 INFO:tasks.workunit.client.0.smithi049.stdout:4/963: write d18/d1e/d2a/d36/d86/f115 [3705003,36247] 0 2024-05-15T15:54:36.680 INFO:tasks.workunit.client.0.smithi049.stdout:4/964: chown d18/d1e/d2a/d50/d54/f8a 8178125 1 2024-05-15T15:54:36.685 INFO:tasks.workunit.client.0.smithi049.stdout:4/965: mkdir d18/d21/d136 0 2024-05-15T15:54:36.686 INFO:tasks.workunit.client.0.smithi049.stdout:4/966: chown d18/d1e/d2a/d50/d54/d109/d98 2323 1 2024-05-15T15:54:36.689 INFO:tasks.workunit.client.0.smithi049.stdout:4/967: unlink d18/d1e/d2a/d36/f7e 0 2024-05-15T15:54:36.691 INFO:tasks.workunit.client.0.smithi049.stdout:4/968: unlink d18/d69/l11d 0 2024-05-15T15:54:36.695 INFO:tasks.workunit.client.0.smithi049.stdout:4/969: rename d18/d69/l124 to d18/d1e/d2a/d50/dc4/l137 0 2024-05-15T15:54:36.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:36 smithi144 ceph-mon[36389]: pgmap v38: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 154 MiB/s wr, 385 op/s 2024-05-15T15:54:36.700 INFO:tasks.workunit.client.0.smithi049.stdout:4/970: getdents d18/d1e/d2a 0 2024-05-15T15:54:36.705 INFO:tasks.workunit.client.0.smithi049.stdout:4/971: rename d18/d1e/d63/d99/db9 to d18/d1e/d63/d10d/d138 0 2024-05-15T15:54:36.706 INFO:tasks.workunit.client.0.smithi049.stdout:4/972: chown d18/d21/l5b 42445 1 2024-05-15T15:54:36.709 INFO:tasks.workunit.client.0.smithi049.stdout:4/973: creat d18/d1e/d2a/f139 x:0 0 0 2024-05-15T15:54:36.721 INFO:tasks.workunit.client.0.smithi049.stdout:4/974: link d18/cba d18/d1e/d2a/d50/d54/dad/d126/c13a 0 2024-05-15T15:54:36.736 INFO:tasks.workunit.client.0.smithi049.stdout:4/975: dread d18/d1e/d63/f96 [0,4194304] 0 2024-05-15T15:54:36.741 INFO:tasks.workunit.client.0.smithi049.stdout:4/976: truncate d18/d1e/d2a/d50/d54/d109/dff/f119 26536 0 2024-05-15T15:54:36.756 INFO:tasks.workunit.client.0.smithi049.stdout:4/977: dwrite d18/d1e/d2a/d50/d54/f5c [0,4194304] 0 2024-05-15T15:54:36.762 INFO:tasks.workunit.client.0.smithi049.stdout:4/978: rename d18/d76 to d18/d21/d2b/d4e/dd2/d13b 0 2024-05-15T15:54:36.765 INFO:tasks.workunit.client.0.smithi049.stdout:4/979: creat d18/d1e/d63/d99/f13c x:0 0 0 2024-05-15T15:54:36.768 INFO:tasks.workunit.client.0.smithi049.stdout:4/980: mknod d18/d21/c13d 0 2024-05-15T15:54:36.772 INFO:tasks.workunit.client.0.smithi049.stdout:4/981: rename d18/d69 to d18/d13e 0 2024-05-15T15:54:36.779 INFO:tasks.workunit.client.0.smithi049.stdout:4/982: write d18/d1e/d2a/d50/d54/d109/fab [1022797,30745] 0 2024-05-15T15:54:36.780 INFO:tasks.workunit.client.0.smithi049.stdout:4/983: chown d18/d1e/d2a/d50/cfc 2297146 1 2024-05-15T15:54:36.783 INFO:tasks.workunit.client.0.smithi049.stdout:4/984: symlink d18/d1e/d2a/d50/l13f 0 2024-05-15T15:54:36.784 INFO:tasks.workunit.client.0.smithi049.stdout:4/985: read - d18/d1e/d63/f106 zero size 2024-05-15T15:54:36.786 INFO:tasks.workunit.client.0.smithi049.stdout:4/986: symlink d18/d1e/d63/d99/de1/da8/l140 0 2024-05-15T15:54:36.790 INFO:tasks.workunit.client.0.smithi049.stdout:4/987: getdents d18/d21/d2b/d4e/dd2/d13b 0 2024-05-15T15:54:36.794 INFO:tasks.workunit.client.0.smithi049.stdout:4/988: rename d18/d21/l25 to d18/d1e/d2a/l141 0 2024-05-15T15:54:36.799 INFO:tasks.workunit.client.0.smithi049.stdout:4/989: dread - d18/d1e/d2a/d50/dbc/f100 zero size 2024-05-15T15:54:36.802 INFO:tasks.workunit.client.0.smithi049.stdout:4/990: rename d18/d1e/d2a/d50/dbc/f100 to d18/d13e/f142 0 2024-05-15T15:54:36.806 INFO:tasks.workunit.client.0.smithi049.stdout:4/991: creat d18/d13e/f143 x:0 0 0 2024-05-15T15:54:36.811 INFO:tasks.workunit.client.0.smithi049.stdout:4/992: rename d18/d1e/d2a/d50/d54/d109/db6/cb7 to d18/d21/d2b/d133/c144 0 2024-05-15T15:54:37.058 INFO:tasks.workunit.client.0.smithi049.stdout:4/993: sync 2024-05-15T15:54:37.063 INFO:tasks.workunit.client.0.smithi049.stdout:4/994: dwrite d18/d21/f30 [4194304,4194304] 0 2024-05-15T15:54:37.226 INFO:tasks.workunit.client.0.smithi049.stdout:4/995: readlink d18/l4c 0 2024-05-15T15:54:37.232 INFO:tasks.workunit.client.0.smithi049.stdout:4/996: fsync d18/d1e/d2a/f93 0 2024-05-15T15:54:37.411 INFO:tasks.workunit.client.0.smithi049.stdout:4/997: write d18/d1e/d63/d99/de1/f47 [29643,29104] 0 2024-05-15T15:54:37.417 INFO:tasks.workunit.client.0.smithi049.stdout:4/998: getdents d18/d1e/d63/d10d 0 2024-05-15T15:54:37.473 INFO:tasks.workunit.client.0.smithi049.stdout:4/999: write d18/d1e/d2a/d50/d54/d109/d98/ddd/fe4 [490298,76364] 0 2024-05-15T15:54:37.477 INFO:tasks.workunit.client.0.smithi049.stderr:+ rm -rf -- ./tmp.HkpAGO5zRK 2024-05-15T15:54:38.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:38 smithi049 ceph-mon[28399]: pgmap v39: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 92 MiB/s rd, 199 MiB/s wr, 518 op/s 2024-05-15T15:54:38.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:38 smithi144 ceph-mon[36389]: pgmap v39: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 92 MiB/s rd, 199 MiB/s wr, 518 op/s 2024-05-15T15:54:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:40 smithi049 ceph-mon[28399]: pgmap v40: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 54 MiB/s rd, 122 MiB/s wr, 322 op/s 2024-05-15T15:54:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:40 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:54:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:40 smithi049 ceph-mon[28399]: Standby manager daemon smithi049.vuhzgf restarted 2024-05-15T15:54:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:40 smithi049 ceph-mon[28399]: Standby manager daemon smithi049.vuhzgf started 2024-05-15T15:54:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:40 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi049.vuhzgf/crt"}]: dispatch 2024-05-15T15:54:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:40 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:54:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:40 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi049.vuhzgf/key"}]: dispatch 2024-05-15T15:54:40.541 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:40 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:54:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:40 smithi144 ceph-mon[36389]: pgmap v40: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 54 MiB/s rd, 122 MiB/s wr, 322 op/s 2024-05-15T15:54:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:40 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:54:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:40 smithi144 ceph-mon[36389]: Standby manager daemon smithi049.vuhzgf restarted 2024-05-15T15:54:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:40 smithi144 ceph-mon[36389]: Standby manager daemon smithi049.vuhzgf started 2024-05-15T15:54:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:40 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi049.vuhzgf/crt"}]: dispatch 2024-05-15T15:54:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:40 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:54:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:40 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi049.vuhzgf/key"}]: dispatch 2024-05-15T15:54:40.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:40 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:54:41.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:41 smithi144 ceph-mon[36389]: mgrmap e26: smithi144.whcjkr(active, since 75s), standbys: smithi049.vuhzgf 2024-05-15T15:54:41.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:41 smithi049 ceph-mon[28399]: mgrmap e26: smithi144.whcjkr(active, since 75s), standbys: smithi049.vuhzgf 2024-05-15T15:54:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:42 smithi144 ceph-mon[36389]: pgmap v41: 65 pgs: 65 active+clean; 3.9 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 54 MiB/s rd, 122 MiB/s wr, 326 op/s 2024-05-15T15:54:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:42 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:42 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:42.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:42 smithi049 ceph-mon[28399]: pgmap v41: 65 pgs: 65 active+clean; 3.9 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 54 MiB/s rd, 122 MiB/s wr, 326 op/s 2024-05-15T15:54:42.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:42 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:42.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:42 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:43.930 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:43 smithi049 ceph-mon[28399]: pgmap v42: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 51 MiB/s rd, 113 MiB/s wr, 332 op/s 2024-05-15T15:54:43.930 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:43 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:43.930 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:43 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:44.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:43 smithi144 ceph-mon[36389]: pgmap v42: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 51 MiB/s rd, 113 MiB/s wr, 332 op/s 2024-05-15T15:54:44.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:43 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:44.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:43 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:45 smithi144 ceph-mon[36389]: pgmap v43: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 21 MiB/s rd, 49 MiB/s wr, 211 op/s 2024-05-15T15:54:46.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:45 smithi049 ceph-mon[28399]: pgmap v43: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 21 MiB/s rd, 49 MiB/s wr, 211 op/s 2024-05-15T15:54:47.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.153+0000 7f824a3dc700 1 -- 172.21.15.49:0/312256895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8244100e50 msgr2=0x7f82441012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:47.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.153+0000 7f824a3dc700 1 --2- 172.21.15.49:0/312256895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8244100e50 0x7f82441012b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8238009910 tx=0x7f8238009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:47.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.153+0000 7f824a3dc700 1 -- 172.21.15.49:0/312256895 shutdown_connections 2024-05-15T15:54:47.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.153+0000 7f824a3dc700 1 --2- 172.21.15.49:0/312256895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8244100e50 0x7f82441012b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:47.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.153+0000 7f824a3dc700 1 --2- 172.21.15.49:0/312256895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82440ffc50 0x7f8244100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:47.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.153+0000 7f824a3dc700 1 -- 172.21.15.49:0/312256895 >> 172.21.15.49:0/312256895 conn(0x7f82440fb1d0 msgr2=0x7f82440fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:47.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.153+0000 7f824a3dc700 1 -- 172.21.15.49:0/312256895 shutdown_connections 2024-05-15T15:54:47.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.153+0000 7f824a3dc700 1 -- 172.21.15.49:0/312256895 wait complete. 2024-05-15T15:54:47.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.154+0000 7f824a3dc700 1 Processor -- start 2024-05-15T15:54:47.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.154+0000 7f824a3dc700 1 -- start start 2024-05-15T15:54:47.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f824a3dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82440ffc50 0x7f8244196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:47.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f824a3dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8244100e50 0x7f8244196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:47.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f824a3dc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82441972c0 con 0x7f82440ffc50 2024-05-15T15:54:47.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f8243fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82440ffc50 0x7f8244196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:47.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f8243fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82440ffc50 0x7f8244196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58802/0 (socket says 172.21.15.49:58802) 2024-05-15T15:54:47.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f8243fff700 1 -- 172.21.15.49:0/1110564003 learned_addr learned my addr 172.21.15.49:0/1110564003 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8244197400 con 0x7f8244100e50 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f82437fe700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8244100e50 0x7f8244196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f8243fff700 1 -- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8244100e50 msgr2=0x7f8244196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f8243fff700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8244100e50 0x7f8244196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.155+0000 7f8243fff700 1 -- 172.21.15.49:0/1110564003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380095f0 con 0x7f82440ffc50 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.156+0000 7f8243fff700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82440ffc50 0x7f8244196760 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f823400d900 tx=0x7f823400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.156+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82340041f0 con 0x7f82440ffc50 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.156+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8234004350 con 0x7f82440ffc50 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.156+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824419be60 con 0x7f82440ffc50 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.156+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824419c2c0 con 0x7f82440ffc50 2024-05-15T15:54:47.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.156+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8234003cf0 con 0x7f82440ffc50 2024-05-15T15:54:47.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.158+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82300052f0 con 0x7f82440ffc50 2024-05-15T15:54:47.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.158+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) v1 ==== 92168+0+0 (secure 0 0 0) 0x7f8234010460 con 0x7f82440ffc50 2024-05-15T15:54:47.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.159+0000 7f82417fa700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f822c06e000 0x7f822c0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:47.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.159+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f823408da40 con 0x7f82440ffc50 2024-05-15T15:54:47.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.159+0000 7f82437fe700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f822c06e000 0x7f822c0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:47.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.160+0000 7f82437fe700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f822c06e000 0x7f822c0704c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8238006010 tx=0x7f823800b540 comp rx=0 tx=0).ready entity=mgr.24413 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:47.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.164+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f823405ad50 con 0x7f82440ffc50 2024-05-15T15:54:47.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.417+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 27) v1 ==== 45957+0+0 (secure 0 0 0) 0x7f82340575f0 con 0x7f82440ffc50 2024-05-15T15:54:47.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.418+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f822c06e000 msgr2=0x7f822c0704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:47.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:47.418+0000 7f82417fa700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f822c06e000 0x7f822c0704c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8238006010 tx=0x7f823800b540 comp rx=0 tx=0).stop 2024-05-15T15:54:48.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.420+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 28) v1 ==== 46378+0+0 (secure 0 0 0) 0x7f8234077db0 con 0x7f82440ffc50 2024-05-15T15:54:48.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.420+0000 7f82417fa700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f822c005ec0 0x7f822c035e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:48.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.420+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f822c0328f0 con 0x7f822c005ec0 2024-05-15T15:54:48.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.420+0000 7f82437fe700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f822c005ec0 0x7f822c035e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:48.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.432+0000 7f82437fe700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f822c005ec0 0x7f822c035e30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8238006010 tx=0x7f8238023000 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:48.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.436+0000 7f82417fa700 1 -- 172.21.15.49:0/1110564003 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f822c0328f0 con 0x7f822c005ec0 2024-05-15T15:54:48.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f822c005ec0 msgr2=0x7f822c035e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:48.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f822c005ec0 0x7f822c035e30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8238006010 tx=0x7f8238023000 comp rx=0 tx=0).stop 2024-05-15T15:54:48.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82440ffc50 msgr2=0x7f8244196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:48.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82440ffc50 0x7f8244196760 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f823400d900 tx=0x7f823400dc10 comp rx=0 tx=0).stop 2024-05-15T15:54:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 shutdown_connections 2024-05-15T15:54:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f822c005ec0 0x7f822c035e30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:6828/177983287,v1:172.21.15.144:6829/177983287] conn(0x7f822c06e000 0x7f822c0704c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82440ffc50 0x7f8244196760 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 --2- 172.21.15.49:0/1110564003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8244100e50 0x7f8244196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 >> 172.21.15.49:0/1110564003 conn(0x7f82440fb1d0 msgr2=0x7f82440686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 shutdown_connections 2024-05-15T15:54:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.440+0000 7f824a3dc700 1 -- 172.21.15.49:0/1110564003 wait complete. 2024-05-15T15:54:48.455 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:54:48.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.591+0000 7efcceb6b700 1 -- 172.21.15.49:0/3145568142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc8100e50 msgr2=0x7efcc81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:48.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.591+0000 7efcceb6b700 1 --2- 172.21.15.49:0/3145568142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc8100e50 0x7efcc81012b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009910 tx=0x7efcb8009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:48.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.591+0000 7efcceb6b700 1 -- 172.21.15.49:0/3145568142 shutdown_connections 2024-05-15T15:54:48.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.591+0000 7efcceb6b700 1 --2- 172.21.15.49:0/3145568142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc8100e50 0x7efcc81012b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.591+0000 7efcceb6b700 1 --2- 172.21.15.49:0/3145568142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc80ffc50 0x7efcc8100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.591+0000 7efcceb6b700 1 -- 172.21.15.49:0/3145568142 >> 172.21.15.49:0/3145568142 conn(0x7efcc80fb1d0 msgr2=0x7efcc80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:48.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.592+0000 7efcceb6b700 1 -- 172.21.15.49:0/3145568142 shutdown_connections 2024-05-15T15:54:48.595 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:48.595 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:48.595 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr fail", "who": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr fail", "who": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: Activating manager daemon smithi049.vuhzgf 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: osdmap e49: 6 total, 6 up, 6 in 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd='[{"prefix": "mgr fail", "who": "smithi144.whcjkr"}]': finished 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: mgrmap e27: smithi049.vuhzgf(active, starting, since 0.00698764s) 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:54:48.596 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:54:48.597 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:54:48.598 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: Manager daemon smithi049.vuhzgf is now available 2024-05-15T15:54:48.598 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:54:48.598 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:54:48.598 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:54:48.598 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/trash_purge_schedule"}]: dispatch 2024-05-15T15:54:48.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.592+0000 7efcceb6b700 1 -- 172.21.15.49:0/3145568142 wait complete. 2024-05-15T15:54:48.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.594+0000 7efcceb6b700 1 Processor -- start 2024-05-15T15:54:48.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.594+0000 7efcceb6b700 1 -- start start 2024-05-15T15:54:48.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.594+0000 7efcceb6b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc80ffc50 0x7efcc8196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.594+0000 7efcceb6b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8100e50 0x7efcc8196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efcceb6b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc8197270 con 0x7efcc8100e50 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efccc907700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc80ffc50 0x7efcc8196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efccc907700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc80ffc50 0x7efcc8196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:43594/0 (socket says 172.21.15.49:43594) 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efccc907700 1 -- 172.21.15.49:0/2385174633 learned_addr learned my addr 172.21.15.49:0/2385174633 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81973b0 con 0x7efcc80ffc50 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efcbffff700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8100e50 0x7efcc8196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efccc907700 1 -- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8100e50 msgr2=0x7efcc8196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efccc907700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8100e50 0x7efcc8196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efccc907700 1 -- 172.21.15.49:0/2385174633 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc4009520 con 0x7efcc80ffc50 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.595+0000 7efcbffff700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8100e50 0x7efcc8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T15:54:48.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.596+0000 7efccc907700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc80ffc50 0x7efcc8196710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efcc400e9d0 tx=0x7efcc400ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:48.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.596+0000 7efcbdffb700 1 -- 172.21.15.49:0/2385174633 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc400cba0 con 0x7efcc80ffc50 2024-05-15T15:54:48.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.596+0000 7efcbdffb700 1 -- 172.21.15.49:0/2385174633 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efcc400cd00 con 0x7efcc80ffc50 2024-05-15T15:54:48.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.596+0000 7efcbdffb700 1 -- 172.21.15.49:0/2385174633 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc4018750 con 0x7efcc80ffc50 2024-05-15T15:54:48.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.597+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb80095f0 con 0x7efcc80ffc50 2024-05-15T15:54:48.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.597+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc81064b0 con 0x7efcc80ffc50 2024-05-15T15:54:48.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.598+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc81908d0 con 0x7efcc80ffc50 2024-05-15T15:54:48.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.599+0000 7efcbdffb700 1 -- 172.21.15.49:0/2385174633 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 28) v1 ==== 46378+0+0 (secure 0 0 0) 0x7efcc40189c0 con 0x7efcc80ffc50 2024-05-15T15:54:48.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.599+0000 7efcbdffb700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7efcb003d6f0 0x7efcb003fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:48.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.599+0000 7efcbdffb700 1 -- 172.21.15.49:0/2385174633 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efcc4014070 con 0x7efcc80ffc50 2024-05-15T15:54:48.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.600+0000 7efcbffff700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7efcb003d6f0 0x7efcb003fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:48.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.600+0000 7efcbffff700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7efcb003d6f0 0x7efcb003fbb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b6f0 tx=0x7efcb8009f90 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:48.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.606+0000 7efcbdffb700 1 -- 172.21.15.49:0/2385174633 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7efcc401f8e0 con 0x7efcc80ffc50 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24413 172.21.15.144:0/1793945209' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr fail", "who": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "mgr fail", "who": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: Activating manager daemon smithi049.vuhzgf 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: osdmap e49: 6 total, 6 up, 6 in 2024-05-15T15:54:48.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24413 ' entity='mgr.smithi144.whcjkr' cmd='[{"prefix": "mgr fail", "who": "smithi144.whcjkr"}]': finished 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: mgrmap e27: smithi049.vuhzgf(active, starting, since 0.00698764s) 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:54:48.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:54:48.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:54:48.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:54:48.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:54:48.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: Manager daemon smithi049.vuhzgf is now available 2024-05-15T15:54:48.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:54:48.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:54:48.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:54:48.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/trash_purge_schedule"}]: dispatch 2024-05-15T15:54:48.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.921+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efcc802ceb0 con 0x7efcb003d6f0 2024-05-15T15:54:48.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.923+0000 7efcbdffb700 1 -- 172.21.15.49:0/2385174633 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7efcc802ceb0 con 0x7efcb003d6f0 2024-05-15T15:54:48.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7efcb003d6f0 msgr2=0x7efcb003fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:48.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7efcb003d6f0 0x7efcb003fbb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b6f0 tx=0x7efcb8009f90 comp rx=0 tx=0).stop 2024-05-15T15:54:48.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc80ffc50 msgr2=0x7efcc8196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:48.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc80ffc50 0x7efcc8196710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efcc400e9d0 tx=0x7efcc400ed90 comp rx=0 tx=0).stop 2024-05-15T15:54:48.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 shutdown_connections 2024-05-15T15:54:48.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7efcb003d6f0 0x7efcb003fbb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc80ffc50 0x7efcc8196710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 --2- 172.21.15.49:0/2385174633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8100e50 0x7efcc8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 >> 172.21.15.49:0/2385174633 conn(0x7efcc80fb1d0 msgr2=0x7efcc8104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 shutdown_connections 2024-05-15T15:54:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:48.926+0000 7efcceb6b700 1 -- 172.21.15.49:0/2385174633 wait complete. 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.077+0000 7fe31a85d700 1 -- 172.21.15.49:0/4212350669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 msgr2=0x7fe3141010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.077+0000 7fe31a85d700 1 --2- 172.21.15.49:0/4212350669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 0x7fe3141010f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe304009910 tx=0x7fe304009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.077+0000 7fe31a85d700 1 -- 172.21.15.49:0/4212350669 shutdown_connections 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.077+0000 7fe31a85d700 1 --2- 172.21.15.49:0/4212350669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe314101ed0 0x7fe314102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.077+0000 7fe31a85d700 1 --2- 172.21.15.49:0/4212350669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 0x7fe3141010f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.077+0000 7fe31a85d700 1 -- 172.21.15.49:0/4212350669 >> 172.21.15.49:0/4212350669 conn(0x7fe3140fc2b0 msgr2=0x7fe3140fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.078+0000 7fe31a85d700 1 -- 172.21.15.49:0/4212350669 shutdown_connections 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.078+0000 7fe31a85d700 1 -- 172.21.15.49:0/4212350669 wait complete. 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.078+0000 7fe31a85d700 1 Processor -- start 2024-05-15T15:54:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.078+0000 7fe31a85d700 1 -- start start 2024-05-15T15:54:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe31a85d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 0x7fe3141966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe31a85d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe314101ed0 0x7fe314196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe31a85d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe314197250 con 0x7fe314100cd0 2024-05-15T15:54:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe31a85d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe314197390 con 0x7fe314101ed0 2024-05-15T15:54:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe313fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 0x7fe3141966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe313fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 0x7fe3141966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58962/0 (socket says 172.21.15.49:58962) 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe313fff700 1 -- 172.21.15.49:0/769923586 learned_addr learned my addr 172.21.15.49:0/769923586 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe3137fe700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe314101ed0 0x7fe314196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe313fff700 1 -- 172.21.15.49:0/769923586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe314101ed0 msgr2=0x7fe314196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe313fff700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe314101ed0 0x7fe314196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.079+0000 7fe313fff700 1 -- 172.21.15.49:0/769923586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3040095f0 con 0x7fe314100cd0 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.080+0000 7fe313fff700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 0x7fe3141966f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe30400b620 tx=0x7fe304005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.080+0000 7fe3117fa700 1 -- 172.21.15.49:0/769923586 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe30401d070 con 0x7fe314100cd0 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.080+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31419bd90 con 0x7fe314100cd0 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.080+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31419c2a0 con 0x7fe314100cd0 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.080+0000 7fe3117fa700 1 -- 172.21.15.49:0/769923586 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3040052a0 con 0x7fe314100cd0 2024-05-15T15:54:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.081+0000 7fe3117fa700 1 -- 172.21.15.49:0/769923586 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe304003e00 con 0x7fe314100cd0 2024-05-15T15:54:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.081+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe300005320 con 0x7fe314100cd0 2024-05-15T15:54:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.082+0000 7fe3117fa700 1 -- 172.21.15.49:0/769923586 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 28) v1 ==== 46378+0+0 (secure 0 0 0) 0x7fe304003680 con 0x7fe314100cd0 2024-05-15T15:54:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.082+0000 7fe3117fa700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe2fc0392e0 0x7fe2fc03b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.082+0000 7fe3117fa700 1 -- 172.21.15.49:0/769923586 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe30404e9c0 con 0x7fe314100cd0 2024-05-15T15:54:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.083+0000 7fe3137fe700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe2fc0392e0 0x7fe2fc03b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:49.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.084+0000 7fe3137fe700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe2fc0392e0 0x7fe2fc03b7a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe308009c60 tx=0x7fe308009260 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:49.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.093+0000 7fe3117fa700 1 -- 172.21.15.49:0/769923586 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fe304021c50 con 0x7fe314100cd0 2024-05-15T15:54:49.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.499+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe300000bf0 con 0x7fe2fc0392e0 2024-05-15T15:54:49.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.515+0000 7fe3117fa700 1 -- 172.21.15.49:0/769923586 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fe300000bf0 con 0x7fe2fc0392e0 2024-05-15T15:54:49.518 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (8m) 8s ago 9m 24.3M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (9m) 8s ago 9m 8032k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (8m) 75s ago 8m 7671k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (9m) 8s ago 9m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (8m) 75s ago 8m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (7m) 8s ago 9m 86.7M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (4m) 8s ago 4m 263M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (3m) 8s ago 3m 250M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (4m) 75s ago 4m 21.4M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (3m) 75s ago 3m 24.4M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (22s) 8s ago 11m 110M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (102s) 75s ago 8m 490M - 19.0.0-2576-gde5210d5 48bc18eb22d8 f3b66ea0d25c 2024-05-15T15:54:49.519 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (11m) 8s ago 11m 51.1M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (8m) 75s ago 8m 46.2M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (9m) 8s ago 9m 15.3M - 1.5.0 0da6a335fe13 171e0a34f565 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (8m) 75s ago 8m 15.5M - 1.5.0 0da6a335fe13 be3a2837bc85 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (7m) 8s ago 7m 202M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (7m) 8s ago 7m 180M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (6m) 8s ago 6m 172M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (6m) 75s ago 6m 88.8M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (5m) 75s ago 5m 79.8M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (5m) 75s ago 5m 87.0M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:54:49.520 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (55s) 8s ago 9m 46.8M - 2.43.0 a07b618ecd1d 6332affb7e87 2024-05-15T15:54:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.519+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe2fc0392e0 msgr2=0x7fe2fc03b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.519+0000 7fe31a85d700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe2fc0392e0 0x7fe2fc03b7a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe308009c60 tx=0x7fe308009260 comp rx=0 tx=0).stop 2024-05-15T15:54:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.519+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 msgr2=0x7fe3141966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.519+0000 7fe31a85d700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 0x7fe3141966f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe30400b620 tx=0x7fe304005f00 comp rx=0 tx=0).stop 2024-05-15T15:54:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.519+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 shutdown_connections 2024-05-15T15:54:49.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.519+0000 7fe31a85d700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe2fc0392e0 0x7fe2fc03b7a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:49.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.519+0000 7fe31a85d700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe314100cd0 0x7fe3141966f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:49.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.519+0000 7fe31a85d700 1 --2- 172.21.15.49:0/769923586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe314101ed0 0x7fe314196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:49.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.519+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 >> 172.21.15.49:0/769923586 conn(0x7fe3140fc2b0 msgr2=0x7fe314105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:49.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.520+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 shutdown_connections 2024-05-15T15:54:49.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.520+0000 7fe31a85d700 1 -- 172.21.15.49:0/769923586 wait complete. 2024-05-15T15:54:49.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.663+0000 7f1c6cabe700 1 -- 172.21.15.49:0/4263170115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 msgr2=0x7f1c681010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:49.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.663+0000 7f1c6cabe700 1 --2- 172.21.15.49:0/4263170115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 0x7f1c681010f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009910 tx=0x7f1c58009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:49.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.664+0000 7f1c6cabe700 1 -- 172.21.15.49:0/4263170115 shutdown_connections 2024-05-15T15:54:49.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.664+0000 7f1c6cabe700 1 --2- 172.21.15.49:0/4263170115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c68101ed0 0x7f1c68102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:49.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.664+0000 7f1c6cabe700 1 --2- 172.21.15.49:0/4263170115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 0x7f1c681010f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:49.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.664+0000 7f1c6cabe700 1 -- 172.21.15.49:0/4263170115 >> 172.21.15.49:0/4263170115 conn(0x7f1c680fc2b0 msgr2=0x7f1c680fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:49.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.664+0000 7f1c6cabe700 1 -- 172.21.15.49:0/4263170115 shutdown_connections 2024-05-15T15:54:49.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.664+0000 7f1c6cabe700 1 -- 172.21.15.49:0/4263170115 wait complete. 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.665+0000 7f1c6cabe700 1 Processor -- start 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.665+0000 7f1c6cabe700 1 -- start start 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.665+0000 7f1c6cabe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 0x7f1c681966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.665+0000 7f1c6cabe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c68101ed0 0x7f1c68196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c6cabe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68197250 con 0x7f1c68100cd0 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c6cabe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68197390 con 0x7f1c68101ed0 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c6659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 0x7f1c681966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c6659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 0x7f1c681966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58982/0 (socket says 172.21.15.49:58982) 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c6659c700 1 -- 172.21.15.49:0/2683383884 learned_addr learned my addr 172.21.15.49:0/2683383884 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:49.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c65d9b700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c68101ed0 0x7f1c68196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:49.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c6659c700 1 -- 172.21.15.49:0/2683383884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c68101ed0 msgr2=0x7f1c68196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:49.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c6659c700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c68101ed0 0x7f1c68196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:49.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c6659c700 1 -- 172.21.15.49:0/2683383884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580095f0 con 0x7f1c68100cd0 2024-05-15T15:54:49.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.666+0000 7f1c6659c700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 0x7f1c681966f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005f00 tx=0x7f1c5800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:49.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.667+0000 7f1c577fe700 1 -- 172.21.15.49:0/2683383884 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c5801d070 con 0x7f1c68100cd0 2024-05-15T15:54:49.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.667+0000 7f1c577fe700 1 -- 172.21.15.49:0/2683383884 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c5800fe30 con 0x7f1c68100cd0 2024-05-15T15:54:49.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.667+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6819bd90 con 0x7f1c68100cd0 2024-05-15T15:54:49.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.667+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6819c2a0 con 0x7f1c68100cd0 2024-05-15T15:54:49.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.667+0000 7f1c577fe700 1 -- 172.21.15.49:0/2683383884 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c58017400 con 0x7f1c68100cd0 2024-05-15T15:54:49.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.668+0000 7f1c577fe700 1 -- 172.21.15.49:0/2683383884 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 28) v1 ==== 46378+0+0 (secure 0 0 0) 0x7f1c58017630 con 0x7f1c68100cd0 2024-05-15T15:54:49.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.668+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6804ecd0 con 0x7f1c68100cd0 2024-05-15T15:54:49.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.669+0000 7f1c577fe700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c50039330 0x7f1c5003b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:49.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.669+0000 7f1c577fe700 1 -- 172.21.15.49:0/2683383884 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1c5804ea70 con 0x7f1c68100cd0 2024-05-15T15:54:49.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.669+0000 7f1c65d9b700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c50039330 0x7f1c5003b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:49.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.670+0000 7f1c65d9b700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c50039330 0x7f1c5003b7f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009bf0 tx=0x7f1c5c009260 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:49.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:49.679+0000 7f1c577fe700 1 -- 172.21.15.49:0/2683383884 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f1c6804ecd0 con 0x7f1c68100cd0 2024-05-15T15:54:49.681 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:49 smithi049 ceph-mon[28399]: mgrmap e28: smithi049.vuhzgf(active, since 1.01173s) 2024-05-15T15:54:49.682 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:49 smithi049 ceph-mon[28399]: from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:49.682 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:49 smithi049 ceph-mon[28399]: pgmap v3: 65 pgs: 65 active+clean; 1.8 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail 2024-05-15T15:54:49.682 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:49 smithi049 ceph-mon[28399]: from='client.24513 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:49.682 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:49 smithi144 ceph-mon[36389]: mgrmap e28: smithi049.vuhzgf(active, since 1.01173s) 2024-05-15T15:54:49.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:49 smithi144 ceph-mon[36389]: from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:49.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:49 smithi144 ceph-mon[36389]: pgmap v3: 65 pgs: 65 active+clean; 1.8 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail 2024-05-15T15:54:49.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:49 smithi144 ceph-mon[36389]: from='client.24513 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:50.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.049+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1c681098d0 con 0x7f1c68100cd0 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.050+0000 7f1c577fe700 1 -- 172.21.15.49:0/2683383884 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f1c58016e20 con 0x7f1c68100cd0 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 1 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:54:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:54:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:54:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:54:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:54:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:54:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:54:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 12, 2024-05-15T15:54:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 1 2024-05-15T15:54:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:54:50.054 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:54:50.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c50039330 msgr2=0x7f1c5003b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:50.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c50039330 0x7f1c5003b7f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009bf0 tx=0x7f1c5c009260 comp rx=0 tx=0).stop 2024-05-15T15:54:50.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 msgr2=0x7f1c681966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:50.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 0x7f1c681966f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005f00 tx=0x7f1c5800f510 comp rx=0 tx=0).stop 2024-05-15T15:54:50.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 shutdown_connections 2024-05-15T15:54:50.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c50039330 0x7f1c5003b7f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c68100cd0 0x7f1c681966f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 --2- 172.21.15.49:0/2683383884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c68101ed0 0x7f1c68196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 >> 172.21.15.49:0/2683383884 conn(0x7f1c680fc2b0 msgr2=0x7f1c68105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:50.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 shutdown_connections 2024-05-15T15:54:50.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.053+0000 7f1c6cabe700 1 -- 172.21.15.49:0/2683383884 wait complete. 2024-05-15T15:54:50.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.204+0000 7fbca0b90700 1 -- 172.21.15.49:0/3070764590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9c104110 msgr2=0x7fbc9c106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:50.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.204+0000 7fbca0b90700 1 --2- 172.21.15.49:0/3070764590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9c104110 0x7fbc9c106500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009910 tx=0x7fbc90009c20 comp rx=0 tx=0).stop 2024-05-15T15:54:50.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.204+0000 7fbca0b90700 1 -- 172.21.15.49:0/3070764590 shutdown_connections 2024-05-15T15:54:50.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.204+0000 7fbca0b90700 1 --2- 172.21.15.49:0/3070764590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9c104110 0x7fbc9c106500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.204+0000 7fbca0b90700 1 --2- 172.21.15.49:0/3070764590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc9c1017e0 0x7fbc9c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.204+0000 7fbca0b90700 1 -- 172.21.15.49:0/3070764590 >> 172.21.15.49:0/3070764590 conn(0x7fbc9c0fb1d0 msgr2=0x7fbc9c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:50.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.205+0000 7fbca0b90700 1 -- 172.21.15.49:0/3070764590 shutdown_connections 2024-05-15T15:54:50.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.205+0000 7fbca0b90700 1 -- 172.21.15.49:0/3070764590 wait complete. 2024-05-15T15:54:50.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.206+0000 7fbca0b90700 1 Processor -- start 2024-05-15T15:54:50.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.206+0000 7fbca0b90700 1 -- start start 2024-05-15T15:54:50.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.207+0000 7fbca0b90700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9c1017e0 0x7fbc9c196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:50.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.207+0000 7fbca0b90700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc9c104110 0x7fbc9c196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:50.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.207+0000 7fbca0b90700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c1972c0 con 0x7fbc9c104110 2024-05-15T15:54:50.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.207+0000 7fbca0b90700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c197400 con 0x7fbc9c1017e0 2024-05-15T15:54:50.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.207+0000 7fbc99d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc9c104110 0x7fbc9c196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:50.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.207+0000 7fbc99d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc9c104110 0x7fbc9c196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58992/0 (socket says 172.21.15.49:58992) 2024-05-15T15:54:50.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.207+0000 7fbc99d9b700 1 -- 172.21.15.49:0/2839754966 learned_addr learned my addr 172.21.15.49:0/2839754966 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:54:50.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.207+0000 7fbc9a59c700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9c1017e0 0x7fbc9c196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:50.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.208+0000 7fbc99d9b700 1 -- 172.21.15.49:0/2839754966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9c1017e0 msgr2=0x7fbc9c196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:50.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.208+0000 7fbc99d9b700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9c1017e0 0x7fbc9c196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.208+0000 7fbc99d9b700 1 -- 172.21.15.49:0/2839754966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900095f0 con 0x7fbc9c104110 2024-05-15T15:54:50.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.208+0000 7fbc99d9b700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc9c104110 0x7fbc9c196ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbc90000c00 tx=0x7fbc90005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.210+0000 7fbc8b7fe700 1 -- 172.21.15.49:0/2839754966 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9001d070 con 0x7fbc9c104110 2024-05-15T15:54:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.210+0000 7fbc8b7fe700 1 -- 172.21.15.49:0/2839754966 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc9000bba0 con 0x7fbc9c104110 2024-05-15T15:54:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.210+0000 7fbc8b7fe700 1 -- 172.21.15.49:0/2839754966 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc90004a00 con 0x7fbc9c104110 2024-05-15T15:54:50.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.211+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9c19be00 con 0x7fbc9c104110 2024-05-15T15:54:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.211+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c19c340 con 0x7fbc9c104110 2024-05-15T15:54:50.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.212+0000 7fbc8b7fe700 1 -- 172.21.15.49:0/2839754966 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 28) v1 ==== 46378+0+0 (secure 0 0 0) 0x7fbc900229c0 con 0x7fbc9c104110 2024-05-15T15:54:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.212+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9c04ec60 con 0x7fbc9c104110 2024-05-15T15:54:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.212+0000 7fbc8b7fe700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fbc84039360 0x7fbc8403b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:54:50.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.213+0000 7fbc8b7fe700 1 -- 172.21.15.49:0/2839754966 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbc9004efa0 con 0x7fbc9c104110 2024-05-15T15:54:50.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.213+0000 7fbc9a59c700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fbc84039360 0x7fbc8403b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:54:50.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.216+0000 7fbc9a59c700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fbc84039360 0x7fbc8403b820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009a10 tx=0x7fbc8c009190 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:54:50.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.223+0000 7fbc8b7fe700 1 -- 172.21.15.49:0/2839754966 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fbc9002be20 con 0x7fbc9c104110 2024-05-15T15:54:50.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.434+0000 7fbc8b7fe700 1 -- 172.21.15.49:0/2839754966 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 29) v1 ==== 46434+0+0 (secure 0 0 0) 0x7fbc9000ece0 con 0x7fbc9c104110 2024-05-15T15:54:50.600 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:50 smithi144 ceph-mon[36389]: [15/May/2024:15:54:49] ENGINE Bus STARTING 2024-05-15T15:54:50.600 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:50 smithi144 ceph-mon[36389]: [15/May/2024:15:54:49] ENGINE Serving on http://172.21.15.49:8765 2024-05-15T15:54:50.600 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:50 smithi144 ceph-mon[36389]: [15/May/2024:15:54:49] ENGINE Serving on https://172.21.15.49:7150 2024-05-15T15:54:50.600 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:50 smithi144 ceph-mon[36389]: [15/May/2024:15:54:49] ENGINE Bus STARTED 2024-05-15T15:54:50.600 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:50 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/2683383884' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:54:50.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.615+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc9c105fb0 con 0x7fbc84039360 2024-05-15T15:54:50.617 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:50 smithi049 ceph-mon[28399]: [15/May/2024:15:54:49] ENGINE Bus STARTING 2024-05-15T15:54:50.617 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:50 smithi049 ceph-mon[28399]: [15/May/2024:15:54:49] ENGINE Serving on http://172.21.15.49:8765 2024-05-15T15:54:50.618 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:50 smithi049 ceph-mon[28399]: [15/May/2024:15:54:49] ENGINE Serving on https://172.21.15.49:7150 2024-05-15T15:54:50.618 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:50 smithi049 ceph-mon[28399]: [15/May/2024:15:54:49] ENGINE Bus STARTED 2024-05-15T15:54:50.618 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:50 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/2683383884' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:54:50.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.616+0000 7fbc8b7fe700 1 -- 172.21.15.49:0/2839754966 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fbc9c105fb0 con 0x7fbc84039360 2024-05-15T15:54:50.618 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr" 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "2/2 daemons upgraded", 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "", 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:54:50.619 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:54:50.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.619+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fbc84039360 msgr2=0x7fbc8403b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:50.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.619+0000 7fbca0b90700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fbc84039360 0x7fbc8403b820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009a10 tx=0x7fbc8c009190 comp rx=0 tx=0).stop 2024-05-15T15:54:50.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.619+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc9c104110 msgr2=0x7fbc9c196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:54:50.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.619+0000 7fbca0b90700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc9c104110 0x7fbc9c196ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbc90000c00 tx=0x7fbc90005dc0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.619+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 shutdown_connections 2024-05-15T15:54:50.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.619+0000 7fbca0b90700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fbc84039360 0x7fbc8403b820 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.619+0000 7fbca0b90700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9c1017e0 0x7fbc9c196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.619+0000 7fbca0b90700 1 --2- 172.21.15.49:0/2839754966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc9c104110 0x7fbc9c196ca0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:54:50.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.619+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 >> 172.21.15.49:0/2839754966 conn(0x7fbc9c0fb1d0 msgr2=0x7fbc9c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:54:50.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.620+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 shutdown_connections 2024-05-15T15:54:50.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:54:50.620+0000 7fbca0b90700 1 -- 172.21.15.49:0/2839754966 wait complete. 2024-05-15T15:54:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:51 smithi144 ceph-mon[36389]: mgrmap e29: smithi049.vuhzgf(active, since 3s) 2024-05-15T15:54:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:51 smithi144 ceph-mon[36389]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:51 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:51 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:51.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:51 smithi049 ceph-mon[28399]: mgrmap e29: smithi049.vuhzgf(active, since 3s) 2024-05-15T15:54:51.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:51 smithi049 ceph-mon[28399]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:54:51.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:51 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:51.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:51 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:52.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:52 smithi144 ceph-mon[36389]: pgmap v5: 65 pgs: 65 active+clean; 1.8 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail 2024-05-15T15:54:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:52 smithi049 ceph-mon[28399]: pgmap v5: 65 pgs: 65 active+clean; 1.8 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail 2024-05-15T15:54:52.912 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-15T15:54:52.912 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-05-15T15:54:53.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:53 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:53.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:53 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:53.875 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:53 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:53.876 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:53 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:54.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:54 smithi144 ceph-mon[36389]: pgmap v6: 65 pgs: 65 active+clean; 1.6 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 13 op/s 2024-05-15T15:54:54.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:54 smithi049 ceph-mon[28399]: pgmap v6: 65 pgs: 65 active+clean; 1.6 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 13 op/s 2024-05-15T15:54:55.438 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-15T15:54:55.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-15T15:54:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:55 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:55 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:55 smithi144 ceph-mon[36389]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:54:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:55 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/2949134471' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:54:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:55 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/2949134471' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:54:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:55 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/2949134471' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:54:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:55 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/2949134471' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:54:55.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:55 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:55.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:55 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:55.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:55 smithi049 ceph-mon[28399]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:54:55.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:55 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/2949134471' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:54:55.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:55 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/2949134471' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:54:55.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:55 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/2949134471' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:54:55.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:55 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/2949134471' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:54:56.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:56 smithi144 ceph-mon[36389]: pgmap v7: 65 pgs: 65 active+clean; 804 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 788 KiB/s rd, 859 KiB/s wr, 126 op/s 2024-05-15T15:54:56.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:56 smithi144 ceph-mon[36389]: mgrmap e30: smithi049.vuhzgf(active, since 8s), standbys: smithi144.whcjkr 2024-05-15T15:54:56.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi144.whcjkr", "id": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:54:56.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:56.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:56 smithi049 ceph-mon[28399]: pgmap v7: 65 pgs: 65 active+clean; 804 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 788 KiB/s rd, 859 KiB/s wr, 126 op/s 2024-05-15T15:54:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:56 smithi049 ceph-mon[28399]: mgrmap e30: smithi049.vuhzgf(active, since 8s), standbys: smithi144.whcjkr 2024-05-15T15:54:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi144.whcjkr", "id": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:54:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:54:58.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:54:58 smithi144 ceph-mon[36389]: pgmap v8: 65 pgs: 65 active+clean; 804 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 612 KiB/s rd, 668 KiB/s wr, 98 op/s 2024-05-15T15:54:58.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:54:58 smithi049 ceph-mon[28399]: pgmap v8: 65 pgs: 65 active+clean; 804 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 612 KiB/s rd, 668 KiB/s wr, 98 op/s 2024-05-15T15:55:00.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:00 smithi144 ceph-mon[36389]: pgmap v9: 65 pgs: 65 active+clean; 284 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-15T15:55:00.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:00 smithi049 ceph-mon[28399]: pgmap v9: 65 pgs: 65 active+clean; 284 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-15T15:55:02.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:02 smithi049 ceph-mon[28399]: pgmap v10: 65 pgs: 65 active+clean; 284 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-05-15T15:55:03.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:02 smithi144 ceph-mon[36389]: pgmap v10: 65 pgs: 65 active+clean; 284 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-05-15T15:55:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:03 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:55:04.047 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:03 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:55:04.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:04 smithi049 ceph-mon[28399]: pgmap v11: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-05-15T15:55:05.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:04 smithi144 ceph-mon[36389]: pgmap v11: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-05-15T15:55:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:06 smithi144 ceph-mon[36389]: pgmap v12: 65 pgs: 65 active+clean; 288 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-05-15T15:55:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:06 smithi049 ceph-mon[28399]: pgmap v12: 65 pgs: 65 active+clean; 288 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-05-15T15:55:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:08 smithi144 ceph-mon[36389]: pgmap v13: 65 pgs: 65 active+clean; 288 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-05-15T15:55:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:08 smithi049 ceph-mon[28399]: pgmap v13: 65 pgs: 65 active+clean; 288 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-05-15T15:55:11.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:10 smithi144 ceph-mon[36389]: pgmap v14: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-05-15T15:55:11.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:10 smithi049 ceph-mon[28399]: pgmap v14: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-05-15T15:55:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:12 smithi144 ceph-mon[36389]: pgmap v15: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 80 op/s 2024-05-15T15:55:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:12 smithi049 ceph-mon[28399]: pgmap v15: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 80 op/s 2024-05-15T15:55:15.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:14 smithi144 ceph-mon[36389]: pgmap v16: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-05-15T15:55:15.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:14 smithi049 ceph-mon[28399]: pgmap v16: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-05-15T15:55:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:16 smithi144 ceph-mon[36389]: pgmap v17: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-05-15T15:55:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:16 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:16 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:16 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:55:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:16 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:55:17.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:16 smithi049 ceph-mon[28399]: pgmap v17: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-05-15T15:55:17.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:16 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:17.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:16 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:17.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:16 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:55:17.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:16 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:55:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:55:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:55:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:55:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:55:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:55:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:17 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:55:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:55:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:55:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:55:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:55:18.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:18.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:18.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:18.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:18.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:17 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:19.006 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:18 smithi049 ceph-mon[28399]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:55:19.007 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:18 smithi049 ceph-mon[28399]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:55:19.007 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:18 smithi049 ceph-mon[28399]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:55:19.007 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:18 smithi049 ceph-mon[28399]: pgmap v18: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-05-15T15:55:19.007 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:18 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:55:19.007 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:18 smithi049 ceph-mon[28399]: Reconfiguring prometheus.smithi049 (dependencies changed)... 2024-05-15T15:55:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:18 smithi144 ceph-mon[36389]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:55:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:18 smithi144 ceph-mon[36389]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:55:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:18 smithi144 ceph-mon[36389]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:55:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:18 smithi144 ceph-mon[36389]: pgmap v18: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-05-15T15:55:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:18 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:55:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:18 smithi144 ceph-mon[36389]: Reconfiguring prometheus.smithi049 (dependencies changed)... 2024-05-15T15:55:20.044 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:19 smithi049 ceph-mon[28399]: Reconfiguring daemon prometheus.smithi049 on smithi049 2024-05-15T15:55:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:19 smithi144 ceph-mon[36389]: Reconfiguring daemon prometheus.smithi049 on smithi049 2024-05-15T15:55:20.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.759+0000 7f36a5678700 1 -- 172.21.15.49:0/2404752242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0101ef0 msgr2=0x7f36a0102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:20.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.759+0000 7f36a5678700 1 --2- 172.21.15.49:0/2404752242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0101ef0 0x7f36a0102350 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3694009910 tx=0x7f3694009c20 comp rx=0 tx=0).stop 2024-05-15T15:55:20.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.759+0000 7f36a5678700 1 -- 172.21.15.49:0/2404752242 shutdown_connections 2024-05-15T15:55:20.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.759+0000 7f36a5678700 1 --2- 172.21.15.49:0/2404752242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0101ef0 0x7f36a0102350 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:20.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.759+0000 7f36a5678700 1 --2- 172.21.15.49:0/2404752242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0100cf0 0x7f36a0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:20.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.759+0000 7f36a5678700 1 -- 172.21.15.49:0/2404752242 >> 172.21.15.49:0/2404752242 conn(0x7f36a00fc290 msgr2=0x7f36a00fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:20.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.759+0000 7f36a5678700 1 -- 172.21.15.49:0/2404752242 shutdown_connections 2024-05-15T15:55:20.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.759+0000 7f36a5678700 1 -- 172.21.15.49:0/2404752242 wait complete. 2024-05-15T15:55:20.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f36a5678700 1 Processor -- start 2024-05-15T15:55:20.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f36a5678700 1 -- start start 2024-05-15T15:55:20.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f36a5678700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0100cf0 0x7f36a0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:20.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f36a5678700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0101ef0 0x7f36a0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f36a5678700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0199460 con 0x7f36a0100cf0 2024-05-15T15:55:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f36a5678700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01995a0 con 0x7f36a0101ef0 2024-05-15T15:55:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f369effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0100cf0 0x7f36a0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f369effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0100cf0 0x7f36a0198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58120/0 (socket says 172.21.15.49:58120) 2024-05-15T15:55:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f369effd700 1 -- 172.21.15.49:0/2848425020 learned_addr learned my addr 172.21.15.49:0/2848425020 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.760+0000 7f369e7fc700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0101ef0 0x7f36a0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.761+0000 7f369effd700 1 -- 172.21.15.49:0/2848425020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0101ef0 msgr2=0x7f36a0198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.761+0000 7f369effd700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0101ef0 0x7f36a0198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.761+0000 7f369effd700 1 -- 172.21.15.49:0/2848425020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36940095f0 con 0x7f36a0100cf0 2024-05-15T15:55:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.761+0000 7f369effd700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0100cf0 0x7f36a0198900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f369000c8f0 tx=0x7f369000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.761+0000 7f3687fff700 1 -- 172.21.15.49:0/2848425020 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36900078e0 con 0x7f36a0100cf0 2024-05-15T15:55:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.761+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a019dfa0 con 0x7f36a0100cf0 2024-05-15T15:55:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.761+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a019e540 con 0x7f36a0100cf0 2024-05-15T15:55:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.761+0000 7f3687fff700 1 -- 172.21.15.49:0/2848425020 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f369000f450 con 0x7f36a0100cf0 2024-05-15T15:55:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.761+0000 7f3687fff700 1 -- 172.21.15.49:0/2848425020 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36900078e0 con 0x7f36a0100cf0 2024-05-15T15:55:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.762+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c0052f0 con 0x7f36a0100cf0 2024-05-15T15:55:20.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.764+0000 7f3687fff700 1 -- 172.21.15.49:0/2848425020 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 30) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f369000fa80 con 0x7f36a0100cf0 2024-05-15T15:55:20.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.764+0000 7f3687fff700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f368806e060 0x7f3688070520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:20.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.765+0000 7f369e7fc700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f368806e060 0x7f3688070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:20.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.765+0000 7f3687fff700 1 -- 172.21.15.49:0/2848425020 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f369008da10 con 0x7f36a0100cf0 2024-05-15T15:55:20.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.765+0000 7f369e7fc700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f368806e060 0x7f3688070520 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3694009fd0 tx=0x7f3694005960 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:20.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:20.771+0000 7f3687fff700 1 -- 172.21.15.49:0/2848425020 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f369005ab30 con 0x7f36a0100cf0 2024-05-15T15:55:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:20 smithi049 ceph-mon[28399]: pgmap v19: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 123 op/s 2024-05-15T15:55:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.120+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f368c000bc0 con 0x7f368806e060 2024-05-15T15:55:21.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.121+0000 7f3687fff700 1 -- 172.21.15.49:0/2848425020 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f368c000bc0 con 0x7f368806e060 2024-05-15T15:55:21.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.123+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f368806e060 msgr2=0x7f3688070520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.123+0000 7f36a5678700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f368806e060 0x7f3688070520 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3694009fd0 tx=0x7f3694005960 comp rx=0 tx=0).stop 2024-05-15T15:55:21.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.123+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0100cf0 msgr2=0x7f36a0198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.123+0000 7f36a5678700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0100cf0 0x7f36a0198900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f369000c8f0 tx=0x7f369000cc00 comp rx=0 tx=0).stop 2024-05-15T15:55:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.123+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 shutdown_connections 2024-05-15T15:55:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.123+0000 7f36a5678700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f368806e060 0x7f3688070520 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.123+0000 7f36a5678700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a0100cf0 0x7f36a0198900 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.123+0000 7f36a5678700 1 --2- 172.21.15.49:0/2848425020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0101ef0 0x7f36a0198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.123+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 >> 172.21.15.49:0/2848425020 conn(0x7f36a00fc290 msgr2=0x7f36a0073500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.124+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 shutdown_connections 2024-05-15T15:55:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.124+0000 7f36a5678700 1 -- 172.21.15.49:0/2848425020 wait complete. 2024-05-15T15:55:21.136 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:55:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:20 smithi144 ceph-mon[36389]: pgmap v19: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 123 op/s 2024-05-15T15:55:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.238+0000 7fcffad6a700 1 -- 172.21.15.49:0/1853619188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 msgr2=0x7fcff41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.238+0000 7fcffad6a700 1 --2- 172.21.15.49:0/1853619188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 0x7fcff41012b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009910 tx=0x7fcfe4009c20 comp rx=0 tx=0).stop 2024-05-15T15:55:21.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.238+0000 7fcffad6a700 1 -- 172.21.15.49:0/1853619188 shutdown_connections 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.238+0000 7fcffad6a700 1 --2- 172.21.15.49:0/1853619188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 0x7fcff41012b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.238+0000 7fcffad6a700 1 --2- 172.21.15.49:0/1853619188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff40ffc50 0x7fcff4100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.238+0000 7fcffad6a700 1 -- 172.21.15.49:0/1853619188 >> 172.21.15.49:0/1853619188 conn(0x7fcff40fb1d0 msgr2=0x7fcff40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.238+0000 7fcffad6a700 1 -- 172.21.15.49:0/1853619188 shutdown_connections 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.238+0000 7fcffad6a700 1 -- 172.21.15.49:0/1853619188 wait complete. 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcffad6a700 1 Processor -- start 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcffad6a700 1 -- start start 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcffad6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff40ffc50 0x7fcff41967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcffad6a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 0x7fcff4196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcffad6a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff4197340 con 0x7fcff4100e50 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcffad6a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff4197480 con 0x7fcff40ffc50 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcfebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 0x7fcff4196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcfebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 0x7fcff4196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58132/0 (socket says 172.21.15.49:58132) 2024-05-15T15:55:21.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcff8b06700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff40ffc50 0x7fcff41967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.239+0000 7fcfebfff700 1 -- 172.21.15.49:0/471550147 learned_addr learned my addr 172.21.15.49:0/471550147 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcff8b06700 1 -- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 msgr2=0x7fcff4196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcff8b06700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 0x7fcff4196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcff8b06700 1 -- 172.21.15.49:0/471550147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe40095f0 con 0x7fcff40ffc50 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcfebfff700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 0x7fcff4196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcff8b06700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff40ffc50 0x7fcff41967e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcff000e940 tx=0x7fcff000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcfe9ffb700 1 -- 172.21.15.49:0/471550147 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff00041f0 con 0x7fcff40ffc50 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcfe9ffb700 1 -- 172.21.15.49:0/471550147 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcff0004350 con 0x7fcff40ffc50 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcfe9ffb700 1 -- 172.21.15.49:0/471550147 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff0003c90 con 0x7fcff40ffc50 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff419be90 con 0x7fcff40ffc50 2024-05-15T15:55:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.240+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff419c350 con 0x7fcff40ffc50 2024-05-15T15:55:21.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.242+0000 7fcfe9ffb700 1 -- 172.21.15.49:0/471550147 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 30) v1 ==== 92166+0+0 (secure 0 0 0) 0x7fcff000c7c0 con 0x7fcff40ffc50 2024-05-15T15:55:21.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.242+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff404ec60 con 0x7fcff40ffc50 2024-05-15T15:55:21.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.242+0000 7fcfe9ffb700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fcfdc06dfc0 0x7fcfdc070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:21.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.242+0000 7fcfe9ffb700 1 -- 172.21.15.49:0/471550147 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcff0014070 con 0x7fcff40ffc50 2024-05-15T15:55:21.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.243+0000 7fcfebfff700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fcfdc06dfc0 0x7fcfdc070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:21.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.243+0000 7fcfebfff700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fcfdc06dfc0 0x7fcfdc070480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009fd0 tx=0x7fcfe4005fb0 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.247+0000 7fcfe9ffb700 1 -- 172.21.15.49:0/471550147 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fcff005b320 con 0x7fcff40ffc50 2024-05-15T15:55:21.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.510+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcff4068de0 con 0x7fcfdc06dfc0 2024-05-15T15:55:21.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.511+0000 7fcfe9ffb700 1 -- 172.21.15.49:0/471550147 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fcff4068de0 con 0x7fcfdc06dfc0 2024-05-15T15:55:21.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.514+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fcfdc06dfc0 msgr2=0x7fcfdc070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.514+0000 7fcffad6a700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fcfdc06dfc0 0x7fcfdc070480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009fd0 tx=0x7fcfe4005fb0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.514+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff40ffc50 msgr2=0x7fcff41967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.514+0000 7fcffad6a700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff40ffc50 0x7fcff41967e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcff000e940 tx=0x7fcff000ec50 comp rx=0 tx=0).stop 2024-05-15T15:55:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.514+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 shutdown_connections 2024-05-15T15:55:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.514+0000 7fcffad6a700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fcfdc06dfc0 0x7fcfdc070480 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.514+0000 7fcffad6a700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff40ffc50 0x7fcff41967e0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcff000e940 tx=0x7fcff000ec50 comp rx=0 tx=0).stop 2024-05-15T15:55:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.514+0000 7fcffad6a700 1 --2- 172.21.15.49:0/471550147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff4100e50 0x7fcff4196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.514+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 >> 172.21.15.49:0/471550147 conn(0x7fcff40fb1d0 msgr2=0x7fcff40686d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.515+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 shutdown_connections 2024-05-15T15:55:21.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.515+0000 7fcffad6a700 1 -- 172.21.15.49:0/471550147 wait complete. 2024-05-15T15:55:21.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.652+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/239753272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ec00ffc50 msgr2=0x7f7ec0100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.652+0000 7f7ec4d6c700 1 --2- 172.21.15.49:0/239753272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ec00ffc50 0x7f7ec0100070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009910 tx=0x7f7eb0009c20 comp rx=0 tx=0).stop 2024-05-15T15:55:21.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.652+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/239753272 shutdown_connections 2024-05-15T15:55:21.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.652+0000 7f7ec4d6c700 1 --2- 172.21.15.49:0/239753272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ec0100e50 0x7f7ec01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.652+0000 7f7ec4d6c700 1 --2- 172.21.15.49:0/239753272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ec00ffc50 0x7f7ec0100070 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.652+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/239753272 >> 172.21.15.49:0/239753272 conn(0x7f7ec00fb1d0 msgr2=0x7f7ec00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:21.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.652+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/239753272 shutdown_connections 2024-05-15T15:55:21.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.652+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/239753272 wait complete. 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.653+0000 7f7ec4d6c700 1 Processor -- start 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.653+0000 7f7ec4d6c700 1 -- start start 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.654+0000 7f7ec4d6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ec00ffc50 0x7f7ec0072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.654+0000 7f7ec4d6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ec0100e50 0x7f7ec0072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.654+0000 7f7ec4d6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec0072ba0 con 0x7f7ec0100e50 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.654+0000 7f7ec4d6c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec01a41e0 con 0x7f7ec00ffc50 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.654+0000 7f7ebdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ec0100e50 0x7f7ec0072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.654+0000 7f7ebdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ec0100e50 0x7f7ec0072580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58152/0 (socket says 172.21.15.49:58152) 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.654+0000 7f7ebdd9b700 1 -- 172.21.15.49:0/3299448722 learned_addr learned my addr 172.21.15.49:0/3299448722 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:21.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.654+0000 7f7ebe59c700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ec00ffc50 0x7f7ec0072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:21.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.655+0000 7f7ebdd9b700 1 -- 172.21.15.49:0/3299448722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ec00ffc50 msgr2=0x7f7ec0072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.655+0000 7f7ebdd9b700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ec00ffc50 0x7f7ec0072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.655+0000 7f7ebdd9b700 1 -- 172.21.15.49:0/3299448722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb00095f0 con 0x7f7ec0100e50 2024-05-15T15:55:21.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.655+0000 7f7ebdd9b700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ec0100e50 0x7f7ec0072580 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400b4b0 tx=0x7f7eb400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:21.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.656+0000 7f7eaf7fe700 1 -- 172.21.15.49:0/3299448722 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb4011800 con 0x7f7ec0100e50 2024-05-15T15:55:21.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.656+0000 7f7eaf7fe700 1 -- 172.21.15.49:0/3299448722 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7eb4011e20 con 0x7f7ec0100e50 2024-05-15T15:55:21.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.656+0000 7f7eaf7fe700 1 -- 172.21.15.49:0/3299448722 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb400f2e0 con 0x7f7ec0100e50 2024-05-15T15:55:21.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.656+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec01063c0 con 0x7f7ec0100e50 2024-05-15T15:55:21.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.656+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec0106990 con 0x7f7ec0100e50 2024-05-15T15:55:21.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.659+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec010a290 con 0x7f7ec0100e50 2024-05-15T15:55:21.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.663+0000 7f7eaf7fe700 1 -- 172.21.15.49:0/3299448722 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 30) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f7eb4010380 con 0x7f7ec0100e50 2024-05-15T15:55:21.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.663+0000 7f7eaf7fe700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ea806e0d0 0x7f7ea8070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:21.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.663+0000 7f7eaf7fe700 1 -- 172.21.15.49:0/3299448722 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7eb408d7f0 con 0x7f7ec0100e50 2024-05-15T15:55:21.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.664+0000 7f7ebe59c700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ea806e0d0 0x7f7ea8070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:21.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.667+0000 7f7ebe59c700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ea806e0d0 0x7f7ea8070590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0005090 tx=0x7f7eb000b540 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:21.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.673+0000 7f7eaf7fe700 1 -- 172.21.15.49:0/3299448722 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f7eb4056f50 con 0x7f7ec0100e50 2024-05-15T15:55:21.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.975+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ec0107e60 con 0x7f7ea806e0d0 2024-05-15T15:55:21.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.990+0000 7f7eaf7fe700 1 -- 172.21.15.49:0/3299448722 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7ec0107e60 con 0x7f7ea806e0d0 2024-05-15T15:55:21.992 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:55:21.992 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (8m) 27s ago 10m 24.5M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (10m) 27s ago 10m 8052k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (9m) 30s ago 9m 7784k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (10m) 27s ago 10m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (9m) 30s ago 9m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (8m) 27s ago 9m 87.3M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (4m) 27s ago 4m 272M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (4m) 27s ago 4m 255M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (4m) 30s ago 4m 24.8M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (4m) 30s ago 4m 28.1M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (54s) 27s ago 11m 478M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (2m) 30s ago 9m 80.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 f3b66ea0d25c 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (11m) 27s ago 11m 53.9M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (9m) 30s ago 8m 45.2M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:55:21.993 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (10m) 27s ago 10m 15.5M - 1.5.0 0da6a335fe13 171e0a34f565 2024-05-15T15:55:21.994 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (9m) 30s ago 9m 15.7M - 1.5.0 0da6a335fe13 be3a2837bc85 2024-05-15T15:55:21.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (8m) 27s ago 8m 182M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:55:21.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (7m) 27s ago 7m 178M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:55:21.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (7m) 27s ago 7m 158M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:55:21.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (6m) 30s ago 6m 212M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:55:21.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (6m) 30s ago 6m 184M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:55:21.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (6m) 30s ago 6m 177M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:55:21.994 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (88s) 27s ago 9m 47.0M - 2.43.0 a07b618ecd1d 6332affb7e87 2024-05-15T15:55:21.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.992+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ea806e0d0 msgr2=0x7f7ea8070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.992+0000 7f7ec4d6c700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ea806e0d0 0x7f7ea8070590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0005090 tx=0x7f7eb000b540 comp rx=0 tx=0).stop 2024-05-15T15:55:21.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.992+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ec0100e50 msgr2=0x7f7ec0072580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:21.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.992+0000 7f7ec4d6c700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ec0100e50 0x7f7ec0072580 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400b4b0 tx=0x7f7eb400b870 comp rx=0 tx=0).stop 2024-05-15T15:55:21.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.992+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 shutdown_connections 2024-05-15T15:55:21.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.992+0000 7f7ec4d6c700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ea806e0d0 0x7f7ea8070590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.993+0000 7f7ec4d6c700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ec00ffc50 0x7f7ec0072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.993+0000 7f7ec4d6c700 1 --2- 172.21.15.49:0/3299448722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ec0100e50 0x7f7ec0072580 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:21.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.993+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 >> 172.21.15.49:0/3299448722 conn(0x7f7ec00fb1d0 msgr2=0x7f7ec0104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:21.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.993+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 shutdown_connections 2024-05-15T15:55:21.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:21.993+0000 7f7ec4d6c700 1 -- 172.21.15.49:0/3299448722 wait complete. 2024-05-15T15:55:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.126+0000 7f2a33da2700 1 -- 172.21.15.49:0/591582028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 msgr2=0x7f2a2c1044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.126+0000 7f2a33da2700 1 --2- 172.21.15.49:0/591582028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 0x7f2a2c1044e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009910 tx=0x7f2a1c009c20 comp rx=0 tx=0).stop 2024-05-15T15:55:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.126+0000 7f2a33da2700 1 -- 172.21.15.49:0/591582028 shutdown_connections 2024-05-15T15:55:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.126+0000 7f2a33da2700 1 --2- 172.21.15.49:0/591582028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 0x7f2a2c1044e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.126+0000 7f2a33da2700 1 --2- 172.21.15.49:0/591582028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2c102f00 0x7f2a2c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.126+0000 7f2a33da2700 1 -- 172.21.15.49:0/591582028 >> 172.21.15.49:0/591582028 conn(0x7f2a2c0fe4e0 msgr2=0x7f2a2c100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.126+0000 7f2a33da2700 1 -- 172.21.15.49:0/591582028 shutdown_connections 2024-05-15T15:55:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.126+0000 7f2a33da2700 1 -- 172.21.15.49:0/591582028 wait complete. 2024-05-15T15:55:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a33da2700 1 Processor -- start 2024-05-15T15:55:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a33da2700 1 -- start start 2024-05-15T15:55:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a33da2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2c102f00 0x7f2a2c078d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a33da2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 0x7f2a2c079240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a33da2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c075840 con 0x7f2a2c104060 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a33da2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c075980 con 0x7f2a2c102f00 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a31b3e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2c102f00 0x7f2a2c078d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a3133d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 0x7f2a2c079240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a31b3e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2c102f00 0x7f2a2c078d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:33734/0 (socket says 172.21.15.49:33734) 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a31b3e700 1 -- 172.21.15.49:0/726062583 learned_addr learned my addr 172.21.15.49:0/726062583 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.127+0000 7f2a3133d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 0x7f2a2c079240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58174/0 (socket says 172.21.15.49:58174) 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a3133d700 1 -- 172.21.15.49:0/726062583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2c102f00 msgr2=0x7f2a2c078d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a3133d700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2c102f00 0x7f2a2c078d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a3133d700 1 -- 172.21.15.49:0/726062583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c0095f0 con 0x7f2a2c104060 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a31b3e700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2c102f00 0x7f2a2c078d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a3133d700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 0x7f2a2c079240 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009910 tx=0x7f2a1c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a22ffd700 1 -- 172.21.15.49:0/726062583 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1c01d070 con 0x7f2a2c104060 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a22ffd700 1 -- 172.21.15.49:0/726062583 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a1c00f460 con 0x7f2a2c104060 2024-05-15T15:55:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a22ffd700 1 -- 172.21.15.49:0/726062583 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1c017630 con 0x7f2a2c104060 2024-05-15T15:55:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c075c00 con 0x7f2a2c104060 2024-05-15T15:55:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.128+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c076170 con 0x7f2a2c104060 2024-05-15T15:55:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.130+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c192a00 con 0x7f2a2c104060 2024-05-15T15:55:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.130+0000 7f2a22ffd700 1 -- 172.21.15.49:0/726062583 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 30) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f2a1c00f5d0 con 0x7f2a2c104060 2024-05-15T15:55:22.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.131+0000 7f2a22ffd700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f2a1806dfb0 0x7f2a18070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:22.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.131+0000 7f2a22ffd700 1 -- 172.21.15.49:0/726062583 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2a1c08f460 con 0x7f2a2c104060 2024-05-15T15:55:22.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.131+0000 7f2a31b3e700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f2a1806dfb0 0x7f2a18070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:22.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.132+0000 7f2a31b3e700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f2a1806dfb0 0x7f2a18070470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a28005950 tx=0x7f2a2800b500 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:22.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.136+0000 7f2a22ffd700 1 -- 172.21.15.49:0/726062583 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f2a1c05c560 con 0x7f2a2c104060 2024-05-15T15:55:22.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.509+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2a2c10b9e0 con 0x7f2a2c104060 2024-05-15T15:55:22.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.509+0000 7f2a22ffd700 1 -- 172.21.15.49:0/726062583 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f2a1c05c0f0 con 0x7f2a2c104060 2024-05-15T15:55:22.514 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:55:22.515 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:55:22.516 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 12, 2024-05-15T15:55:22.516 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:55:22.516 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:55:22.516 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:55:22.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f2a1806dfb0 msgr2=0x7f2a18070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:22.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f2a1806dfb0 0x7f2a18070470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a28005950 tx=0x7f2a2800b500 comp rx=0 tx=0).stop 2024-05-15T15:55:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 msgr2=0x7f2a2c079240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 0x7f2a2c079240 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009910 tx=0x7f2a1c00bde0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 shutdown_connections 2024-05-15T15:55:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f2a1806dfb0 0x7f2a18070470 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2c102f00 0x7f2a2c078d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 --2- 172.21.15.49:0/726062583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a2c104060 0x7f2a2c079240 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 >> 172.21.15.49:0/726062583 conn(0x7f2a2c0fe4e0 msgr2=0x7f2a2c107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 shutdown_connections 2024-05-15T15:55:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.511+0000 7f2a33da2700 1 -- 172.21.15.49:0/726062583 wait complete. 2024-05-15T15:55:22.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.640+0000 7fe169433700 1 -- 172.21.15.49:0/572346698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe164100d00 msgr2=0x7fe164101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:22.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.640+0000 7fe169433700 1 --2- 172.21.15.49:0/572346698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe164100d00 0x7fe164101120 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe154009910 tx=0x7fe154009c20 comp rx=0 tx=0).stop 2024-05-15T15:55:22.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.640+0000 7fe169433700 1 -- 172.21.15.49:0/572346698 shutdown_connections 2024-05-15T15:55:22.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.640+0000 7fe169433700 1 --2- 172.21.15.49:0/572346698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe164101f00 0x7fe164102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.640+0000 7fe169433700 1 --2- 172.21.15.49:0/572346698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe164100d00 0x7fe164101120 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.640+0000 7fe169433700 1 -- 172.21.15.49:0/572346698 >> 172.21.15.49:0/572346698 conn(0x7fe1640fc2c0 msgr2=0x7fe1640fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:22.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.640+0000 7fe169433700 1 -- 172.21.15.49:0/572346698 shutdown_connections 2024-05-15T15:55:22.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.640+0000 7fe169433700 1 -- 172.21.15.49:0/572346698 wait complete. 2024-05-15T15:55:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.641+0000 7fe169433700 1 Processor -- start 2024-05-15T15:55:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.641+0000 7fe169433700 1 -- start start 2024-05-15T15:55:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.641+0000 7fe169433700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe164100d00 0x7fe1641966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.641+0000 7fe169433700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe164101f00 0x7fe164196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.641+0000 7fe169433700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164197220 con 0x7fe164101f00 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.641+0000 7fe169433700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164197360 con 0x7fe164100d00 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe1627fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe164101f00 0x7fe164196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe1627fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe164101f00 0x7fe164196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58196/0 (socket says 172.21.15.49:58196) 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe1627fc700 1 -- 172.21.15.49:0/3373866182 learned_addr learned my addr 172.21.15.49:0/3373866182 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe162ffd700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe164100d00 0x7fe1641966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe1627fc700 1 -- 172.21.15.49:0/3373866182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe164100d00 msgr2=0x7fe1641966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe1627fc700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe164100d00 0x7fe1641966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe1627fc700 1 -- 172.21.15.49:0/3373866182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe158009520 con 0x7fe164101f00 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe1627fc700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe164101f00 0x7fe164196c00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe15800eab0 tx=0x7fe15800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe14bfff700 1 -- 172.21.15.49:0/3373866182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe15800cba0 con 0x7fe164101f00 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe14bfff700 1 -- 172.21.15.49:0/3373866182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1580044e0 con 0x7fe164101f00 2024-05-15T15:55:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1540095f0 con 0x7fe164101f00 2024-05-15T15:55:22.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.642+0000 7fe14bfff700 1 -- 172.21.15.49:0/3373866182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe158005040 con 0x7fe164101f00 2024-05-15T15:55:22.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.643+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16419c200 con 0x7fe164101f00 2024-05-15T15:55:22.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.644+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe164190880 con 0x7fe164101f00 2024-05-15T15:55:22.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.645+0000 7fe14bfff700 1 -- 172.21.15.49:0/3373866182 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 30) v1 ==== 92166+0+0 (secure 0 0 0) 0x7fe158004750 con 0x7fe164101f00 2024-05-15T15:55:22.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.645+0000 7fe14bfff700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe14c06dfb0 0x7fe14c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:22.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.645+0000 7fe14bfff700 1 -- 172.21.15.49:0/3373866182 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe158014070 con 0x7fe164101f00 2024-05-15T15:55:22.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.646+0000 7fe162ffd700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe14c06dfb0 0x7fe14c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:22.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.646+0000 7fe162ffd700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe14c06dfb0 0x7fe14c070470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b6f0 tx=0x7fe154005fb0 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:22.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.653+0000 7fe14bfff700 1 -- 172.21.15.49:0/3373866182 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fe15805b040 con 0x7fe164101f00 2024-05-15T15:55:22.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:22 smithi049 ceph-mon[28399]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:22.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:22 smithi049 ceph-mon[28399]: pgmap v20: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-05-15T15:55:22.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:22 smithi049 ceph-mon[28399]: from='client.24535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:22 smithi049 ceph-mon[28399]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:22 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/726062583' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:22.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.967+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe16402cf00 con 0x7fe14c06dfb0 2024-05-15T15:55:22.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.968+0000 7fe14bfff700 1 -- 172.21.15.49:0/3373866182 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fe16402cf00 con 0x7fe14c06dfb0 2024-05-15T15:55:22.970 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr" 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "2/2 daemons upgraded", 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "", 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:55:22.971 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe14c06dfb0 msgr2=0x7fe14c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe14c06dfb0 0x7fe14c070470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b6f0 tx=0x7fe154005fb0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe164101f00 msgr2=0x7fe164196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe164101f00 0x7fe164196c00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe15800eab0 tx=0x7fe15800edc0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 shutdown_connections 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fe14c06dfb0 0x7fe14c070470 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe164100d00 0x7fe1641966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 --2- 172.21.15.49:0/3373866182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe164101f00 0x7fe164196c00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 >> 172.21.15.49:0/3373866182 conn(0x7fe1640fc2c0 msgr2=0x7fe164105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 shutdown_connections 2024-05-15T15:55:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:22.970+0000 7fe169433700 1 -- 172.21.15.49:0/3373866182 wait complete. 2024-05-15T15:55:23.001 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:22 smithi144 ceph-mon[36389]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:23.001 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:22 smithi144 ceph-mon[36389]: pgmap v20: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-05-15T15:55:23.001 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:22 smithi144 ceph-mon[36389]: from='client.24535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:23.001 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:22 smithi144 ceph-mon[36389]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:23.001 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:22 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/726062583' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:24.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:23 smithi144 ceph-mon[36389]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:24.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:23 smithi049 ceph-mon[28399]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:25.111 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:24 smithi049 ceph-mon[28399]: pgmap v21: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-05-15T15:55:25.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:24 smithi144 ceph-mon[36389]: pgmap v21: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-05-15T15:55:27.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:26 smithi049 ceph-mon[28399]: pgmap v22: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-05-15T15:55:27.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:26 smithi144 ceph-mon[36389]: pgmap v22: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-05-15T15:55:28.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:28 smithi144 ceph-mon[36389]: pgmap v23: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-05-15T15:55:28.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:28 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:28.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:28 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:28.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:28 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:55:28.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:28 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:55:28.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:28 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:55:28.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:28 smithi049 ceph-mon[28399]: pgmap v23: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-05-15T15:55:28.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:28 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:28.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:28 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:28.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:28 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:55:28.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:28 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:55:28.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:28 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:55:30.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:30 smithi144 ceph-mon[36389]: pgmap v24: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 140 op/s 2024-05-15T15:55:30.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:30 smithi049 ceph-mon[28399]: pgmap v24: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 140 op/s 2024-05-15T15:55:31.916 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:31 smithi144 ceph-mon[36389]: Upgrade: Updating mgr.smithi144.whcjkr 2024-05-15T15:55:31.916 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:31 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:55:31.916 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:31 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:55:31.916 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:31 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:55:31.916 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:31 smithi144 ceph-mon[36389]: Deploying daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:55:31.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:31 smithi049 ceph-mon[28399]: Upgrade: Updating mgr.smithi144.whcjkr 2024-05-15T15:55:31.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:31 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:55:31.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:31 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:55:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:31 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:55:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:31 smithi049 ceph-mon[28399]: Deploying daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:55:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:32 smithi144 ceph-mon[36389]: pgmap v25: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-05-15T15:55:32.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:32 smithi049 ceph-mon[28399]: pgmap v25: 65 pgs: 65 active+clean; 304 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 94 op/s 2024-05-15T15:55:33.779 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:33 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:55:33.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:33 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:55:34.773 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:34 smithi144 ceph-mon[36389]: pgmap v26: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-15T15:55:34.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:34 smithi049 ceph-mon[28399]: pgmap v26: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-15T15:55:36.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:36 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:36 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:36 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:55:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:36 smithi049 ceph-mon[28399]: pgmap v27: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-05-15T15:55:36.641 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:36 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:36.641 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:36 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:36.641 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:36 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:55:36.642 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:36 smithi144 ceph-mon[36389]: pgmap v27: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-05-15T15:55:38.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:38 smithi144 ceph-mon[36389]: pgmap v28: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-05-15T15:55:38.817 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:38 smithi049 ceph-mon[28399]: pgmap v28: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-05-15T15:55:39.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:39 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:39.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:39 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:39.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:39 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:39.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:39 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:41.123 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:40 smithi144 ceph-mon[36389]: pgmap v29: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 445 op/s 2024-05-15T15:55:41.123 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:40 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:41.123 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:40 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:41.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:40 smithi049 ceph-mon[28399]: pgmap v29: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 445 op/s 2024-05-15T15:55:41.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:40 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:41.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:40 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:43.228 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:42 smithi144 ceph-mon[36389]: pgmap v30: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 398 op/s 2024-05-15T15:55:43.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:42 smithi049 ceph-mon[28399]: pgmap v30: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 398 op/s 2024-05-15T15:55:44.002 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:43 smithi144 ceph-mon[36389]: Standby manager daemon smithi144.whcjkr restarted 2024-05-15T15:55:44.002 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:43 smithi144 ceph-mon[36389]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:55:44.002 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:43 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/3525110473' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:55:44.002 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:43 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/3525110473' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:55:44.002 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:43 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/3525110473' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:55:44.003 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:43 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/3525110473' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:55:44.003 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:43 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:44.003 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:43 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:44.145 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:43 smithi049 ceph-mon[28399]: Standby manager daemon smithi144.whcjkr restarted 2024-05-15T15:55:44.145 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:43 smithi049 ceph-mon[28399]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:55:44.145 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:43 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/3525110473' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:55:44.145 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:43 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/3525110473' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:55:44.145 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:43 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/3525110473' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:55:44.145 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:43 smithi049 ceph-mon[28399]: from='mgr.? 172.21.15.144:0/3525110473' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:55:44.145 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:43 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:44.146 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:43 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:45.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:44 smithi144 ceph-mon[36389]: pgmap v31: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 408 op/s 2024-05-15T15:55:45.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:44 smithi144 ceph-mon[36389]: mgrmap e31: smithi049.vuhzgf(active, since 56s), standbys: smithi144.whcjkr 2024-05-15T15:55:45.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:44 smithi049 ceph-mon[28399]: pgmap v31: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 408 op/s 2024-05-15T15:55:45.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:44 smithi049 ceph-mon[28399]: mgrmap e31: smithi049.vuhzgf(active, since 56s), standbys: smithi144.whcjkr 2024-05-15T15:55:47.289 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: pgmap v32: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 467 op/s 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi049.vuhzgf"}]: dispatch 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi049.vuhzgf"}]': finished 2024-05-15T15:55:47.290 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.whcjkr"}]: dispatch 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.whcjkr"}]': finished 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.291 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.292 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: pgmap v32: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 467 op/s 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi049.vuhzgf"}]: dispatch 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi049.vuhzgf"}]': finished 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.whcjkr"}]: dispatch 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.whcjkr"}]': finished 2024-05-15T15:55:47.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:47.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:47 smithi144 ceph-mon[36389]: Upgrade: Setting container_image for all mgr 2024-05-15T15:55:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:47 smithi144 ceph-mon[36389]: Upgrade: Setting container_image for all rgw 2024-05-15T15:55:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:47 smithi144 ceph-mon[36389]: Upgrade: Setting container_image for all rbd-mirror 2024-05-15T15:55:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:47 smithi144 ceph-mon[36389]: Upgrade: Setting container_image for all cephfs-mirror 2024-05-15T15:55:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:47 smithi144 ceph-mon[36389]: Upgrade: Setting container_image for all iscsi 2024-05-15T15:55:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:47 smithi144 ceph-mon[36389]: Upgrade: Setting container_image for all nfs 2024-05-15T15:55:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:47 smithi144 ceph-mon[36389]: Upgrade: Setting container_image for all nvmeof 2024-05-15T15:55:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:47 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:55:48.320 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:47 smithi049 ceph-mon[28399]: Upgrade: Setting container_image for all mgr 2024-05-15T15:55:48.320 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:47 smithi049 ceph-mon[28399]: Upgrade: Setting container_image for all rgw 2024-05-15T15:55:48.320 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:47 smithi049 ceph-mon[28399]: Upgrade: Setting container_image for all rbd-mirror 2024-05-15T15:55:48.320 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:47 smithi049 ceph-mon[28399]: Upgrade: Setting container_image for all cephfs-mirror 2024-05-15T15:55:48.320 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:47 smithi049 ceph-mon[28399]: Upgrade: Setting container_image for all iscsi 2024-05-15T15:55:48.320 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:47 smithi049 ceph-mon[28399]: Upgrade: Setting container_image for all nfs 2024-05-15T15:55:48.320 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:47 smithi049 ceph-mon[28399]: Upgrade: Setting container_image for all nvmeof 2024-05-15T15:55:48.321 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:47 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:55:49.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:48 smithi144 ceph-mon[36389]: pgmap v33: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 328 op/s 2024-05-15T15:55:49.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:48 smithi049 ceph-mon[28399]: pgmap v33: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 328 op/s 2024-05-15T15:55:50.434 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:49 smithi049 ceph-mon[28399]: Upgrade: Updating node-exporter.smithi049 (1/2) 2024-05-15T15:55:50.434 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:49 smithi049 ceph-mon[28399]: Deploying daemon node-exporter.smithi049 on smithi049 2024-05-15T15:55:50.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:49 smithi144 ceph-mon[36389]: Upgrade: Updating node-exporter.smithi049 (1/2) 2024-05-15T15:55:50.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:49 smithi144 ceph-mon[36389]: Deploying daemon node-exporter.smithi049 on smithi049 2024-05-15T15:55:51.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:50 smithi144 ceph-mon[36389]: pgmap v34: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 442 op/s 2024-05-15T15:55:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:50 smithi049 ceph-mon[28399]: pgmap v34: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 442 op/s 2024-05-15T15:55:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.093+0000 7fb909fc7700 1 -- 172.21.15.49:0/2690776654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9040ff350 msgr2=0x7fb9040ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.093+0000 7fb909fc7700 1 --2- 172.21.15.49:0/2690776654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9040ff350 0x7fb9040ff770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400a980 tx=0x7fb8f400ac90 comp rx=0 tx=0).stop 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.093+0000 7fb909fc7700 1 -- 172.21.15.49:0/2690776654 shutdown_connections 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.094+0000 7fb909fc7700 1 --2- 172.21.15.49:0/2690776654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb904100550 0x7fb9041009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.094+0000 7fb909fc7700 1 --2- 172.21.15.49:0/2690776654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9040ff350 0x7fb9040ff770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.094+0000 7fb909fc7700 1 -- 172.21.15.49:0/2690776654 >> 172.21.15.49:0/2690776654 conn(0x7fb9040fa8f0 msgr2=0x7fb9040fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.094+0000 7fb909fc7700 1 -- 172.21.15.49:0/2690776654 shutdown_connections 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.094+0000 7fb909fc7700 1 -- 172.21.15.49:0/2690776654 wait complete. 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.094+0000 7fb909fc7700 1 Processor -- start 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.095+0000 7fb909fc7700 1 -- start start 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.095+0000 7fb909fc7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9040ff350 0x7fb904196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.095+0000 7fb909fc7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb904100550 0x7fb904196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.095+0000 7fb909fc7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9041972b0 con 0x7fb9040ff350 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.095+0000 7fb909fc7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9041973f0 con 0x7fb904100550 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.095+0000 7fb9037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9040ff350 0x7fb904196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.095+0000 7fb902ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb904100550 0x7fb904196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.095+0000 7fb9037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9040ff350 0x7fb904196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52134/0 (socket says 172.21.15.49:52134) 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.095+0000 7fb9037fe700 1 -- 172.21.15.49:0/620375609 learned_addr learned my addr 172.21.15.49:0/620375609 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb902ffd700 1 -- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9040ff350 msgr2=0x7fb904196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb902ffd700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9040ff350 0x7fb904196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb902ffd700 1 -- 172.21.15.49:0/620375609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80095f0 con 0x7fb904100550 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb9037fe700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9040ff350 0x7fb904196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T15:55:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb902ffd700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb904100550 0x7fb904196c90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8010ea0 tx=0x7fb8f800b5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb900ff9700 1 -- 172.21.15.49:0/620375609 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8f8015070 con 0x7fb904100550 2024-05-15T15:55:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb900ff9700 1 -- 172.21.15.49:0/620375609 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8f80044e0 con 0x7fb904100550 2024-05-15T15:55:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb900ff9700 1 -- 172.21.15.49:0/620375609 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8f8019640 con 0x7fb904100550 2024-05-15T15:55:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f400a630 con 0x7fb904100550 2024-05-15T15:55:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.096+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb904101c10 con 0x7fb904100550 2024-05-15T15:55:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.098+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb904190650 con 0x7fb904100550 2024-05-15T15:55:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.099+0000 7fb900ff9700 1 -- 172.21.15.49:0/620375609 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7fb8f8003aa0 con 0x7fb904100550 2024-05-15T15:55:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.099+0000 7fb900ff9700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb8ec06dfb0 0x7fb8ec070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.100+0000 7fb900ff9700 1 -- 172.21.15.49:0/620375609 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb8f8091ad0 con 0x7fb904100550 2024-05-15T15:55:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.100+0000 7fb9037fe700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb8ec06dfb0 0x7fb8ec070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.100+0000 7fb9037fe700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb8ec06dfb0 0x7fb8ec070470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40046a0 tx=0x7fb8f4003ba0 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:53.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.104+0000 7fb900ff9700 1 -- 172.21.15.49:0/620375609 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fb8f805e940 con 0x7fb904100550 2024-05-15T15:55:53.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:52 smithi144 ceph-mon[36389]: pgmap v35: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 206 op/s 2024-05-15T15:55:53.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.386+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb90402ce10 con 0x7fb8ec06dfb0 2024-05-15T15:55:53.388 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:52 smithi049 ceph-mon[28399]: pgmap v35: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 206 op/s 2024-05-15T15:55:53.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.388+0000 7fb900ff9700 1 -- 172.21.15.49:0/620375609 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fb90402ce10 con 0x7fb8ec06dfb0 2024-05-15T15:55:53.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb8ec06dfb0 msgr2=0x7fb8ec070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:53.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb8ec06dfb0 0x7fb8ec070470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40046a0 tx=0x7fb8f4003ba0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb904100550 msgr2=0x7fb904196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:53.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb904100550 0x7fb904196c90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8010ea0 tx=0x7fb8f800b5a0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 shutdown_connections 2024-05-15T15:55:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb8ec06dfb0 0x7fb8ec070470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9040ff350 0x7fb904196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 --2- 172.21.15.49:0/620375609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb904100550 0x7fb904196c90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 >> 172.21.15.49:0/620375609 conn(0x7fb9040fa8f0 msgr2=0x7fb904103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 shutdown_connections 2024-05-15T15:55:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.391+0000 7fb909fc7700 1 -- 172.21.15.49:0/620375609 wait complete. 2024-05-15T15:55:53.404 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:55:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.534+0000 7f9f29514700 1 -- 172.21.15.49:0/2831800321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24101f00 msgr2=0x7f9f24102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.534+0000 7f9f29514700 1 --2- 172.21.15.49:0/2831800321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24101f00 0x7f9f24102360 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009910 tx=0x7f9f0c009c20 comp rx=0 tx=0).stop 2024-05-15T15:55:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.534+0000 7f9f29514700 1 -- 172.21.15.49:0/2831800321 shutdown_connections 2024-05-15T15:55:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.534+0000 7f9f29514700 1 --2- 172.21.15.49:0/2831800321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24101f00 0x7f9f24102360 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.534+0000 7f9f29514700 1 --2- 172.21.15.49:0/2831800321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24100d00 0x7f9f24101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.534+0000 7f9f29514700 1 -- 172.21.15.49:0/2831800321 >> 172.21.15.49:0/2831800321 conn(0x7f9f240fc2c0 msgr2=0x7f9f240fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.534+0000 7f9f29514700 1 -- 172.21.15.49:0/2831800321 shutdown_connections 2024-05-15T15:55:53.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.534+0000 7f9f29514700 1 -- 172.21.15.49:0/2831800321 wait complete. 2024-05-15T15:55:53.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.534+0000 7f9f29514700 1 Processor -- start 2024-05-15T15:55:53.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f29514700 1 -- start start 2024-05-15T15:55:53.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f29514700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24100d00 0x7f9f241966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:53.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f29514700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24101f00 0x7f9f24196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:53.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f29514700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24197220 con 0x7f9f24100d00 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f29514700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24197360 con 0x7f9f24101f00 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f22ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24100d00 0x7f9f241966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f22ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24100d00 0x7f9f241966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52160/0 (socket says 172.21.15.49:52160) 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f22ffd700 1 -- 172.21.15.49:0/1828621761 learned_addr learned my addr 172.21.15.49:0/1828621761 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f1a7fc700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24101f00 0x7f9f24196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f22ffd700 1 -- 172.21.15.49:0/1828621761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24101f00 msgr2=0x7f9f24196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f22ffd700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24101f00 0x7f9f24196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f22ffd700 1 -- 172.21.15.49:0/1828621761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0095f0 con 0x7f9f24100d00 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f22ffd700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24100d00 0x7f9f241966c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400d900 tx=0x7f9f1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.535+0000 7f9f20ff9700 1 -- 172.21.15.49:0/1828621761 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f140041f0 con 0x7f9f24100d00 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.536+0000 7f9f20ff9700 1 -- 172.21.15.49:0/1828621761 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f14004350 con 0x7f9f24100d00 2024-05-15T15:55:53.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.536+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2419bdc0 con 0x7f9f24100d00 2024-05-15T15:55:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.536+0000 7f9f20ff9700 1 -- 172.21.15.49:0/1828621761 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f14003dc0 con 0x7f9f24100d00 2024-05-15T15:55:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.536+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2419c310 con 0x7f9f24100d00 2024-05-15T15:55:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.537+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f24190880 con 0x7f9f24100d00 2024-05-15T15:55:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.537+0000 7f9f20ff9700 1 -- 172.21.15.49:0/1828621761 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f9f14009730 con 0x7f9f24100d00 2024-05-15T15:55:53.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.538+0000 7f9f20ff9700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f9f1006dfc0 0x7f9f10070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:53.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.538+0000 7f9f20ff9700 1 -- 172.21.15.49:0/1828621761 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9f1408e030 con 0x7f9f24100d00 2024-05-15T15:55:53.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.538+0000 7f9f1a7fc700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f9f1006dfc0 0x7f9f10070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:53.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.539+0000 7f9f1a7fc700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f9f1006dfc0 0x7f9f10070480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00b5c0 tx=0x7f9f0c005160 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.543+0000 7f9f20ff9700 1 -- 172.21.15.49:0/1828621761 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f9f1405b150 con 0x7f9f24100d00 2024-05-15T15:55:53.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.841+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f2402cf50 con 0x7f9f1006dfc0 2024-05-15T15:55:53.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.843+0000 7f9f20ff9700 1 -- 172.21.15.49:0/1828621761 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f9f2402cf50 con 0x7f9f1006dfc0 2024-05-15T15:55:53.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f9f1006dfc0 msgr2=0x7f9f10070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:53.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f9f1006dfc0 0x7f9f10070480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00b5c0 tx=0x7f9f0c005160 comp rx=0 tx=0).stop 2024-05-15T15:55:53.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24100d00 msgr2=0x7f9f241966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:53.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24100d00 0x7f9f241966c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400d900 tx=0x7f9f1400dcc0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 shutdown_connections 2024-05-15T15:55:53.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f9f1006dfc0 0x7f9f10070480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f24100d00 0x7f9f241966c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 --2- 172.21.15.49:0/1828621761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24101f00 0x7f9f24196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:53.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 >> 172.21.15.49:0/1828621761 conn(0x7f9f240fc2c0 msgr2=0x7f9f241054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:53.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 shutdown_connections 2024-05-15T15:55:53.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.845+0000 7f9f29514700 1 -- 172.21.15.49:0/1828621761 wait complete. 2024-05-15T15:55:53.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.996+0000 7f7edf44f700 1 -- 172.21.15.49:0/2814242291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffbd0 msgr2=0x7f7ed80ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:53.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.996+0000 7f7edf44f700 1 --2- 172.21.15.49:0/2814242291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffbd0 0x7f7ed80ffff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009f90 tx=0x7f7ed400b330 comp rx=0 tx=0).stop 2024-05-15T15:55:53.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.997+0000 7f7edf44f700 1 -- 172.21.15.49:0/2814242291 shutdown_connections 2024-05-15T15:55:54.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.997+0000 7f7edf44f700 1 --2- 172.21.15.49:0/2814242291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100dd0 0x7f7ed8101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:54.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.997+0000 7f7edf44f700 1 --2- 172.21.15.49:0/2814242291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffbd0 0x7f7ed80ffff0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:54.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.997+0000 7f7edf44f700 1 -- 172.21.15.49:0/2814242291 >> 172.21.15.49:0/2814242291 conn(0x7f7ed80fb170 msgr2=0x7f7ed80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:54.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.997+0000 7f7edf44f700 1 -- 172.21.15.49:0/2814242291 shutdown_connections 2024-05-15T15:55:54.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.997+0000 7f7edf44f700 1 -- 172.21.15.49:0/2814242291 wait complete. 2024-05-15T15:55:54.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.998+0000 7f7edf44f700 1 Processor -- start 2024-05-15T15:55:54.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.998+0000 7f7edf44f700 1 -- start start 2024-05-15T15:55:54.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.998+0000 7f7edf44f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffbd0 0x7f7ed8196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:54.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.998+0000 7f7edf44f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100dd0 0x7f7ed8196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:54.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.998+0000 7f7edf44f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed81970e0 con 0x7f7ed80ffbd0 2024-05-15T15:55:54.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.998+0000 7f7edf44f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8197220 con 0x7f7ed8100dd0 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7edd1eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffbd0 0x7f7ed8196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7edd1eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffbd0 0x7f7ed8196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42270/0 (socket says 172.21.15.49:42270) 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7edd1eb700 1 -- 172.21.15.49:0/2155047462 learned_addr learned my addr 172.21.15.49:0/2155047462 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7edc9ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100dd0 0x7f7ed8196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7edc9ea700 1 -- 172.21.15.49:0/2155047462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffbd0 msgr2=0x7f7ed8196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7edc9ea700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffbd0 0x7f7ed8196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7edc9ea700 1 -- 172.21.15.49:0/2155047462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed4009c40 con 0x7f7ed8100dd0 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7edc9ea700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100dd0 0x7f7ed8196ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800e940 tx=0x7f7ec800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7ece7fc700 1 -- 172.21.15.49:0/2155047462 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec800c990 con 0x7f7ed8100dd0 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7ece7fc700 1 -- 172.21.15.49:0/2155047462 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ec8010430 con 0x7f7ed8100dd0 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed819bc30 con 0x7f7ed8100dd0 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:53.999+0000 7f7ece7fc700 1 -- 172.21.15.49:0/2155047462 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec8018630 con 0x7f7ed8100dd0 2024-05-15T15:55:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.000+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed819c200 con 0x7f7ed8100dd0 2024-05-15T15:55:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.001+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed8190850 con 0x7f7ed8100dd0 2024-05-15T15:55:54.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.002+0000 7f7ece7fc700 1 -- 172.21.15.49:0/2155047462 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f7ec80105a0 con 0x7f7ed8100dd0 2024-05-15T15:55:54.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.003+0000 7f7ece7fc700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ec406e000 0x7f7ec40704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:54.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.003+0000 7f7ece7fc700 1 -- 172.21.15.49:0/2155047462 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7ec8014070 con 0x7f7ed8100dd0 2024-05-15T15:55:54.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.003+0000 7f7edd1eb700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ec406e000 0x7f7ec40704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:54.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.004+0000 7f7edd1eb700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ec406e000 0x7f7ec40704c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4015040 tx=0x7f7ed4000f40 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:54.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.006+0000 7f7ece7fc700 1 -- 172.21.15.49:0/2155047462 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f7ec805b700 con 0x7f7ed8100dd0 2024-05-15T15:55:54.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.340+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ed802ceb0 con 0x7f7ec406e000 2024-05-15T15:55:54.368 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:55:54.368 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (9m) 7s ago 10m 24.7M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:55:54.368 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (10m) 7s ago 10m 8116k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:55:54.368 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (9m) 15s ago 9m 7847k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:55:54.368 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (10m) 7s ago 10m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:55:54.368 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (9m) 15s ago 9m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:55:54.368 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (9m) 7s ago 10m 86.7M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:55:54.368 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (5m) 7s ago 5m 233M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:55:54.368 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (5m) 7s ago 5m 230M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (5m) 15s ago 5m 26.2M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (4m) 15s ago 4m 27.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (87s) 7s ago 12m 499M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (19s) 15s ago 9m 65.6M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (12m) 7s ago 12m 54.5M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (9m) 15s ago 9m 46.7M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (10m) 7s ago 10m 15.5M - 1.5.0 0da6a335fe13 171e0a34f565 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (9m) 15s ago 9m 15.2M - 1.5.0 0da6a335fe13 be3a2837bc85 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (8m) 7s ago 8m 191M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (8m) 7s ago 8m 192M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (7m) 7s ago 7m 186M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (7m) 15s ago 7m 238M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (7m) 15s ago 7m 194M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (6m) 15s ago 6m 185M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:55:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (27s) 7s ago 10m 44.2M - 2.43.0 a07b618ecd1d c65cd3f949ef 2024-05-15T15:55:54.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.366+0000 7f7ece7fc700 1 -- 172.21.15.49:0/2155047462 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7ed802ceb0 con 0x7f7ec406e000 2024-05-15T15:55:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.370+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ec406e000 msgr2=0x7f7ec40704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.370+0000 7f7edf44f700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ec406e000 0x7f7ec40704c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4015040 tx=0x7f7ed4000f40 comp rx=0 tx=0).stop 2024-05-15T15:55:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.370+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100dd0 msgr2=0x7f7ed8196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:54.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.370+0000 7f7edf44f700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100dd0 0x7f7ed8196ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800e940 tx=0x7f7ec800ec50 comp rx=0 tx=0).stop 2024-05-15T15:55:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.376+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 shutdown_connections 2024-05-15T15:55:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.376+0000 7f7edf44f700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7ec406e000 0x7f7ec40704c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.376+0000 7f7edf44f700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffbd0 0x7f7ed8196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.376+0000 7f7edf44f700 1 --2- 172.21.15.49:0/2155047462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100dd0 0x7f7ed8196ac0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.376+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 >> 172.21.15.49:0/2155047462 conn(0x7f7ed80fb170 msgr2=0x7f7ed8104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.376+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 shutdown_connections 2024-05-15T15:55:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.376+0000 7f7edf44f700 1 -- 172.21.15.49:0/2155047462 wait complete. 2024-05-15T15:55:54.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.554+0000 7febd0829700 1 -- 172.21.15.49:0/1645449334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febcc0ffc50 msgr2=0x7febcc100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:54.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.554+0000 7febd0829700 1 --2- 172.21.15.49:0/1645449334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febcc0ffc50 0x7febcc100070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7febbc009910 tx=0x7febbc009c20 comp rx=0 tx=0).stop 2024-05-15T15:55:54.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.554+0000 7febd0829700 1 -- 172.21.15.49:0/1645449334 shutdown_connections 2024-05-15T15:55:54.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.554+0000 7febd0829700 1 --2- 172.21.15.49:0/1645449334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febcc100e50 0x7febcc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:54.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.554+0000 7febd0829700 1 --2- 172.21.15.49:0/1645449334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febcc0ffc50 0x7febcc100070 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:54.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.554+0000 7febd0829700 1 -- 172.21.15.49:0/1645449334 >> 172.21.15.49:0/1645449334 conn(0x7febcc0fb1d0 msgr2=0x7febcc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:54.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.554+0000 7febd0829700 1 -- 172.21.15.49:0/1645449334 shutdown_connections 2024-05-15T15:55:54.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.555+0000 7febd0829700 1 -- 172.21.15.49:0/1645449334 wait complete. 2024-05-15T15:55:54.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.555+0000 7febd0829700 1 Processor -- start 2024-05-15T15:55:54.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.555+0000 7febd0829700 1 -- start start 2024-05-15T15:55:54.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febd0829700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febcc0ffc50 0x7febcc196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:54.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febd0829700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febcc100e50 0x7febcc196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:54.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febd0829700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1970d0 con 0x7febcc100e50 2024-05-15T15:55:54.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febd0829700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc197210 con 0x7febcc0ffc50 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febc9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febcc100e50 0x7febcc196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febc9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febcc100e50 0x7febcc196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42274/0 (socket says 172.21.15.49:42274) 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febca59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febcc0ffc50 0x7febcc196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febc9d9b700 1 -- 172.21.15.49:0/3260164523 learned_addr learned my addr 172.21.15.49:0/3260164523 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febca59c700 1 -- 172.21.15.49:0/3260164523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febcc100e50 msgr2=0x7febcc196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febca59c700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febcc100e50 0x7febcc196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.556+0000 7febca59c700 1 -- 172.21.15.49:0/3260164523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc0095f0 con 0x7febcc0ffc50 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.557+0000 7febca59c700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febcc0ffc50 0x7febcc196570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098e0 tx=0x7febbc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.557+0000 7febbb7fe700 1 -- 172.21.15.49:0/3260164523 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febbc01d070 con 0x7febcc0ffc50 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.557+0000 7febbb7fe700 1 -- 172.21.15.49:0/3260164523 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febbc0052a0 con 0x7febcc0ffc50 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.557+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc19bc10 con 0x7febcc0ffc50 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.557+0000 7febbb7fe700 1 -- 172.21.15.49:0/3260164523 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febbc017630 con 0x7febcc0ffc50 2024-05-15T15:55:54.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.557+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc19c100 con 0x7febcc0ffc50 2024-05-15T15:55:54.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.559+0000 7febbb7fe700 1 -- 172.21.15.49:0/3260164523 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7febbc0038a0 con 0x7febcc0ffc50 2024-05-15T15:55:54.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.559+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febcc04ec60 con 0x7febcc0ffc50 2024-05-15T15:55:54.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.559+0000 7febbb7fe700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7febb406e000 0x7febb40704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:54.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.560+0000 7febbb7fe700 1 -- 172.21.15.49:0/3260164523 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7febbc08f4d0 con 0x7febcc0ffc50 2024-05-15T15:55:54.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.560+0000 7febc9d9b700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7febb406e000 0x7febb40704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:54.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.561+0000 7febc9d9b700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7febb406e000 0x7febb40704c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febc0009d20 tx=0x7febc0009260 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:54.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:54.565+0000 7febbb7fe700 1 -- 172.21.15.49:0/3260164523 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7febbc05c5d0 con 0x7febcc0ffc50 2024-05-15T15:55:55.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.005+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7febcc1089b0 con 0x7febcc0ffc50 2024-05-15T15:55:55.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.006+0000 7febbb7fe700 1 -- 172.21.15.49:0/3260164523 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7febbc05c160 con 0x7febcc0ffc50 2024-05-15T15:55:55.008 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:55:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:55:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:55:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:55:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 12, 2024-05-15T15:55:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:55:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:55:55.010 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7febb406e000 msgr2=0x7febb40704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7febb406e000 0x7febb40704c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febc0009d20 tx=0x7febc0009260 comp rx=0 tx=0).stop 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febcc0ffc50 msgr2=0x7febcc196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febcc0ffc50 0x7febcc196570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098e0 tx=0x7febbc00f690 comp rx=0 tx=0).stop 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 shutdown_connections 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7febb406e000 0x7febb40704c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febcc0ffc50 0x7febcc196570 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 --2- 172.21.15.49:0/3260164523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febcc100e50 0x7febcc196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 >> 172.21.15.49:0/3260164523 conn(0x7febcc0fb1d0 msgr2=0x7febcc0686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 shutdown_connections 2024-05-15T15:55:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.008+0000 7febd0829700 1 -- 172.21.15.49:0/3260164523 wait complete. 2024-05-15T15:55:55.046 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:54 smithi049 ceph-mon[28399]: from='client.24555 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:55.047 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:54 smithi049 ceph-mon[28399]: pgmap v36: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 212 op/s 2024-05-15T15:55:55.047 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:54 smithi049 ceph-mon[28399]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.143+0000 7f6e60536700 1 -- 172.21.15.49:0/646269239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58104090 msgr2=0x7f6e58106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.143+0000 7f6e60536700 1 --2- 172.21.15.49:0/646269239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58104090 0x7f6e58106480 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009910 tx=0x7f6e48009c20 comp rx=0 tx=0).stop 2024-05-15T15:55:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.143+0000 7f6e60536700 1 -- 172.21.15.49:0/646269239 shutdown_connections 2024-05-15T15:55:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.143+0000 7f6e60536700 1 --2- 172.21.15.49:0/646269239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58104090 0x7f6e58106480 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.143+0000 7f6e60536700 1 --2- 172.21.15.49:0/646269239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e58101760 0x7f6e58103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.143+0000 7f6e60536700 1 -- 172.21.15.49:0/646269239 >> 172.21.15.49:0/646269239 conn(0x7f6e580fb130 msgr2=0x7f6e580fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.143+0000 7f6e60536700 1 -- 172.21.15.49:0/646269239 shutdown_connections 2024-05-15T15:55:55.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.143+0000 7f6e60536700 1 -- 172.21.15.49:0/646269239 wait complete. 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e60536700 1 Processor -- start 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e60536700 1 -- start start 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e60536700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58101760 0x7f6e58196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e60536700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e58104090 0x7f6e58196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e60536700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581971f0 con 0x7f6e58101760 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e60536700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58197330 con 0x7f6e58104090 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e5e2d2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58101760 0x7f6e58196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e5e2d2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58101760 0x7f6e58196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42304/0 (socket says 172.21.15.49:42304) 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e5e2d2700 1 -- 172.21.15.49:0/2492199539 learned_addr learned my addr 172.21.15.49:0/2492199539 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:55:55.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.144+0000 7f6e5dad1700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e58104090 0x7f6e58196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.145+0000 7f6e5e2d2700 1 -- 172.21.15.49:0/2492199539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e58104090 msgr2=0x7f6e58196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.145+0000 7f6e5e2d2700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e58104090 0x7f6e58196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.145+0000 7f6e5e2d2700 1 -- 172.21.15.49:0/2492199539 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480095f0 con 0x7f6e58101760 2024-05-15T15:55:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.145+0000 7f6e5e2d2700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58101760 0x7f6e58196690 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400e940 tx=0x7f6e5400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.145+0000 7f6e4f7fe700 1 -- 172.21.15.49:0/2492199539 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5400cba0 con 0x7f6e58101760 2024-05-15T15:55:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.145+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5819bd40 con 0x7f6e58101760 2024-05-15T15:55:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.145+0000 7f6e4f7fe700 1 -- 172.21.15.49:0/2492199539 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e5400ee90 con 0x7f6e58101760 2024-05-15T15:55:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.145+0000 7f6e4f7fe700 1 -- 172.21.15.49:0/2492199539 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e540055d0 con 0x7f6e58101760 2024-05-15T15:55:55.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.145+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5819c2b0 con 0x7f6e58101760 2024-05-15T15:55:55.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.147+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5804ec60 con 0x7f6e58101760 2024-05-15T15:55:55.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.147+0000 7f6e4f7fe700 1 -- 172.21.15.49:0/2492199539 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f6e54004750 con 0x7f6e58101760 2024-05-15T15:55:55.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.148+0000 7f6e4f7fe700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6e4406dfc0 0x7f6e44070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:55:55.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.148+0000 7f6e5dad1700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6e4406dfc0 0x7f6e44070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:55:55.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.148+0000 7f6e4f7fe700 1 -- 172.21.15.49:0/2492199539 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6e54014070 con 0x7f6e58101760 2024-05-15T15:55:55.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.149+0000 7f6e5dad1700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6e4406dfc0 0x7f6e44070480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800b800 tx=0x7f6e48009f70 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:55:55.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.153+0000 7f6e4f7fe700 1 -- 172.21.15.49:0/2492199539 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f6e5405b2f0 con 0x7f6e58101760 2024-05-15T15:55:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:54 smithi144 ceph-mon[36389]: from='client.24555 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:54 smithi144 ceph-mon[36389]: pgmap v36: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 212 op/s 2024-05-15T15:55:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:54 smithi144 ceph-mon[36389]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:55.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.528+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e58104a90 con 0x7f6e4406dfc0 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.530+0000 7f6e4f7fe700 1 -- 172.21.15.49:0/2492199539 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f6e58104a90 con 0x7f6e4406dfc0 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr" 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "2/2 daemons upgraded", 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading node-exporter daemons", 2024-05-15T15:55:55.532 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:55:55.533 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:55:55.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.533+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6e4406dfc0 msgr2=0x7f6e44070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:55.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.533+0000 7f6e60536700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6e4406dfc0 0x7f6e44070480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800b800 tx=0x7f6e48009f70 comp rx=0 tx=0).stop 2024-05-15T15:55:55.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.533+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58101760 msgr2=0x7f6e58196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:55:55.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.533+0000 7f6e60536700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58101760 0x7f6e58196690 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400e940 tx=0x7f6e5400ec50 comp rx=0 tx=0).stop 2024-05-15T15:55:55.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.534+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 shutdown_connections 2024-05-15T15:55:55.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.534+0000 7f6e60536700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6e4406dfc0 0x7f6e44070480 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:55.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.534+0000 7f6e60536700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e58101760 0x7f6e58196690 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:55.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.534+0000 7f6e60536700 1 --2- 172.21.15.49:0/2492199539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e58104090 0x7f6e58196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:55:55.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.534+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 >> 172.21.15.49:0/2492199539 conn(0x7f6e580fb130 msgr2=0x7f6e580fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:55:55.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.534+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 shutdown_connections 2024-05-15T15:55:55.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:55:55.534+0000 7f6e60536700 1 -- 172.21.15.49:0/2492199539 wait complete. 2024-05-15T15:55:56.337 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:55 smithi144 ceph-mon[36389]: from='client.24561 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:56.338 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:55 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3260164523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:56.338 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:55 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:56.338 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:55 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:56.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:55 smithi049 ceph-mon[28399]: from='client.24561 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:56.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:55 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3260164523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:55:56.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:55 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:56.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:55 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:55:57.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:56 smithi049 ceph-mon[28399]: pgmap v37: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-05-15T15:55:57.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:56 smithi049 ceph-mon[28399]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:57.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:56 smithi144 ceph-mon[36389]: pgmap v37: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-05-15T15:55:57.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:56 smithi144 ceph-mon[36389]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:55:59.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:55:58 smithi144 ceph-mon[36389]: pgmap v38: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-15T15:55:59.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:55:58 smithi049 ceph-mon[28399]: pgmap v38: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-15T15:56:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:00 smithi144 ceph-mon[36389]: Upgrade: Updating node-exporter.smithi144 (2/2) 2024-05-15T15:56:00.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:00 smithi144 ceph-mon[36389]: Deploying daemon node-exporter.smithi144 on smithi144 2024-05-15T15:56:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:00 smithi049 ceph-mon[28399]: Upgrade: Updating node-exporter.smithi144 (2/2) 2024-05-15T15:56:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:00 smithi049 ceph-mon[28399]: Deploying daemon node-exporter.smithi144 on smithi144 2024-05-15T15:56:01.079 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:01 smithi144 ceph-mon[36389]: pgmap v39: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-05-15T15:56:01.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:01 smithi049 ceph-mon[28399]: pgmap v39: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-05-15T15:56:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:02 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:02 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:02 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:02 smithi144 ceph-mon[36389]: pgmap v40: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-05-15T15:56:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:02 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:02 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:02.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:02 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:02.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:02 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:02.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:02 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:02.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:02 smithi049 ceph-mon[28399]: pgmap v40: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-05-15T15:56:02.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:02 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:02.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:02 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:03 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:56:03.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:03 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:56:04.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:04 smithi144 ceph-mon[36389]: pgmap v41: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-15T15:56:04.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:04 smithi049 ceph-mon[28399]: pgmap v41: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-15T15:56:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:05 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:05 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:05 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:05 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:07.088 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:06 smithi144 ceph-mon[36389]: pgmap v42: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-05-15T15:56:07.088 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:06 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:07.088 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:06 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:06 smithi049 ceph-mon[28399]: pgmap v42: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-05-15T15:56:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:06 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:06 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:09.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:08 smithi144 ceph-mon[36389]: pgmap v43: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-05-15T15:56:09.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:08 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:09.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:08 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:09.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:08 smithi049 ceph-mon[28399]: pgmap v43: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-05-15T15:56:09.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:08 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:09.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:08 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:10.009 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-05-15T15:56:10.010 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-05-15T15:56:10.293 DEBUG:teuthology.parallel:result is None 2024-05-15T15:56:10.688 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-05-15T15:56:10.688 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-15T15:56:10.751 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:10 smithi049 ceph-mon[28399]: pgmap v44: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 268 op/s 2024-05-15T15:56:10.751 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:10 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:10.752 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:10 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:10.752 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:10 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:10.752 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:10 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:11.052 DEBUG:teuthology.parallel:result is None 2024-05-15T15:56:11.052 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-15T15:56:11.127 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-15T15:56:11.127 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-05-15T15:56:11.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:10 smithi144 ceph-mon[36389]: pgmap v44: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 268 op/s 2024-05-15T15:56:11.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:10 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:11.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:10 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:11.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:10 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:11.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:10 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:11.162 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-05-15T15:56:11.162 DEBUG:teuthology.parallel:result is None 2024-05-15T15:56:12.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:12 smithi049 ceph-mon[28399]: pgmap v45: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-05-15T15:56:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:12 smithi144 ceph-mon[36389]: pgmap v45: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-05-15T15:56:15.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: pgmap v46: 65 pgs: 65 active+clean; 268 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-05-15T15:56:15.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:15.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:15.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:56:15.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:56:15.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:15.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:15.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.208 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.208 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.208 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.208 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:15 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: pgmap v46: 65 pgs: 65 active+clean; 268 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-05-15T15:56:15.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:15.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:15 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:17.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:17 smithi144 ceph-mon[36389]: pgmap v47: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 233 op/s 2024-05-15T15:56:17.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:17 smithi049 ceph-mon[28399]: pgmap v47: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 233 op/s 2024-05-15T15:56:18.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:18 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:56:18.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:18 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:56:19.410 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:19 smithi144 ceph-mon[36389]: pgmap v48: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 922 KiB/s rd, 893 KiB/s wr, 170 op/s 2024-05-15T15:56:19.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:19 smithi049 ceph-mon[28399]: pgmap v48: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 922 KiB/s rd, 893 KiB/s wr, 170 op/s 2024-05-15T15:56:20.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:20 smithi144 ceph-mon[36389]: mgrmap e32: smithi049.vuhzgf(active, since 91s), standbys: smithi144.whcjkr 2024-05-15T15:56:20.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:20 smithi144 ceph-mon[36389]: pgmap v49: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 942 KiB/s rd, 912 KiB/s wr, 175 op/s 2024-05-15T15:56:20.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:20 smithi049 ceph-mon[28399]: mgrmap e32: smithi049.vuhzgf(active, since 91s), standbys: smithi144.whcjkr 2024-05-15T15:56:20.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:20 smithi049 ceph-mon[28399]: pgmap v49: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 942 KiB/s rd, 912 KiB/s wr, 175 op/s 2024-05-15T15:56:22.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:22 smithi144 ceph-mon[36389]: Upgrade: Updating prometheus.smithi049 2024-05-15T15:56:22.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:22 smithi144 ceph-mon[36389]: pgmap v50: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 465 KiB/s rd, 339 KiB/s wr, 76 op/s 2024-05-15T15:56:22.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:22 smithi144 ceph-mon[36389]: Deploying daemon prometheus.smithi049 on smithi049 2024-05-15T15:56:22.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:22 smithi049 ceph-mon[28399]: Upgrade: Updating prometheus.smithi049 2024-05-15T15:56:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:22 smithi049 ceph-mon[28399]: pgmap v50: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 465 KiB/s rd, 339 KiB/s wr, 76 op/s 2024-05-15T15:56:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:22 smithi049 ceph-mon[28399]: Deploying daemon prometheus.smithi049 on smithi049 2024-05-15T15:56:24.752 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:24 smithi144 ceph-mon[36389]: pgmap v51: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 465 KiB/s rd, 339 KiB/s wr, 77 op/s 2024-05-15T15:56:24.782 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:24 smithi049 ceph-mon[28399]: pgmap v51: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 465 KiB/s rd, 339 KiB/s wr, 77 op/s 2024-05-15T15:56:25.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.668+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3343270440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 msgr2=0x7f8e68102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:25.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.668+0000 7f8e6dbc5700 1 --2- 172.21.15.49:0/3343270440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 0x7f8e68102360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009910 tx=0x7f8e50009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:25.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.668+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3343270440 shutdown_connections 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.668+0000 7f8e6dbc5700 1 --2- 172.21.15.49:0/3343270440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 0x7f8e68102360 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.668+0000 7f8e6dbc5700 1 --2- 172.21.15.49:0/3343270440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e68100d00 0x7f8e68101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.668+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3343270440 >> 172.21.15.49:0/3343270440 conn(0x7f8e680fc2c0 msgr2=0x7f8e680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.668+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3343270440 shutdown_connections 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.668+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3343270440 wait complete. 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.669+0000 7f8e6dbc5700 1 Processor -- start 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.669+0000 7f8e6dbc5700 1 -- start start 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.669+0000 7f8e6dbc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e68100d00 0x7f8e681966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.669+0000 7f8e6dbc5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 0x7f8e68196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.669+0000 7f8e6dbc5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68197240 con 0x7f8e68101f00 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.669+0000 7f8e6dbc5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68197380 con 0x7f8e68100d00 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 0x7f8e68196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 0x7f8e68196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35230/0 (socket says 172.21.15.49:35230) 2024-05-15T15:56:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e66ffd700 1 -- 172.21.15.49:0/3200008043 learned_addr learned my addr 172.21.15.49:0/3200008043 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e677fe700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e68100d00 0x7f8e681966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e66ffd700 1 -- 172.21.15.49:0/3200008043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e68100d00 msgr2=0x7f8e681966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e66ffd700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e68100d00 0x7f8e681966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e66ffd700 1 -- 172.21.15.49:0/3200008043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e500095f0 con 0x7f8e68101f00 2024-05-15T15:56:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e66ffd700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 0x7f8e68196c20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8e500092e0 tx=0x7f8e5000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e64ff9700 1 -- 172.21.15.49:0/3200008043 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5001d070 con 0x7f8e68101f00 2024-05-15T15:56:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e64ff9700 1 -- 172.21.15.49:0/3200008043 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e5000f460 con 0x7f8e68101f00 2024-05-15T15:56:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6819bd80 con 0x7f8e68101f00 2024-05-15T15:56:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.670+0000 7f8e64ff9700 1 -- 172.21.15.49:0/3200008043 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e50021620 con 0x7f8e68101f00 2024-05-15T15:56:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.671+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6819c2f0 con 0x7f8e68101f00 2024-05-15T15:56:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.672+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e68190880 con 0x7f8e68101f00 2024-05-15T15:56:25.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.672+0000 7f8e64ff9700 1 -- 172.21.15.49:0/3200008043 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f8e5000f5d0 con 0x7f8e68101f00 2024-05-15T15:56:25.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.673+0000 7f8e64ff9700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f8e5406de80 0x7f8e54070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:25.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.673+0000 7f8e64ff9700 1 -- 172.21.15.49:0/3200008043 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8e5008ee30 con 0x7f8e68101f00 2024-05-15T15:56:25.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.673+0000 7f8e677fe700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f8e5406de80 0x7f8e54070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:25.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.673+0000 7f8e677fe700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f8e5406de80 0x7f8e54070340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8e68101d60 tx=0x7f8e5800b410 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.678+0000 7f8e64ff9700 1 -- 172.21.15.49:0/3200008043 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f8e5005c170 con 0x7f8e68101f00 2024-05-15T15:56:25.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.916+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e6802ce10 con 0x7f8e5406de80 2024-05-15T15:56:25.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.917+0000 7f8e64ff9700 1 -- 172.21.15.49:0/3200008043 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f8e6802ce10 con 0x7f8e5406de80 2024-05-15T15:56:25.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.919+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f8e5406de80 msgr2=0x7f8e54070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:25.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.919+0000 7f8e6dbc5700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f8e5406de80 0x7f8e54070340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8e68101d60 tx=0x7f8e5800b410 comp rx=0 tx=0).stop 2024-05-15T15:56:25.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.920+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 msgr2=0x7f8e68196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.920+0000 7f8e6dbc5700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 0x7f8e68196c20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8e500092e0 tx=0x7f8e5000bd70 comp rx=0 tx=0).stop 2024-05-15T15:56:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.920+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 shutdown_connections 2024-05-15T15:56:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.920+0000 7f8e6dbc5700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f8e5406de80 0x7f8e54070340 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.920+0000 7f8e6dbc5700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e68100d00 0x7f8e681966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.920+0000 7f8e6dbc5700 1 --2- 172.21.15.49:0/3200008043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e68101f00 0x7f8e68196c20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.920+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 >> 172.21.15.49:0/3200008043 conn(0x7f8e680fc2c0 msgr2=0x7f8e68105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.920+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 shutdown_connections 2024-05-15T15:56:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:25.920+0000 7f8e6dbc5700 1 -- 172.21.15.49:0/3200008043 wait complete. 2024-05-15T15:56:25.932 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:56:26.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.029+0000 7f435b589700 1 -- 172.21.15.49:0/1387826359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 msgr2=0x7f4354103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.029+0000 7f435b589700 1 --2- 172.21.15.49:0/1387826359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 0x7f4354103c90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4350009910 tx=0x7f4350009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:26.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.029+0000 7f435b589700 1 -- 172.21.15.49:0/1387826359 shutdown_connections 2024-05-15T15:56:26.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.029+0000 7f435b589700 1 --2- 172.21.15.49:0/1387826359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541041d0 0x7f43541065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 --2- 172.21.15.49:0/1387826359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 0x7f4354103c90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 -- 172.21.15.49:0/1387826359 >> 172.21.15.49:0/1387826359 conn(0x7f43540fb1d0 msgr2=0x7f43540fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:26.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 -- 172.21.15.49:0/1387826359 shutdown_connections 2024-05-15T15:56:26.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 -- 172.21.15.49:0/1387826359 wait complete. 2024-05-15T15:56:26.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 Processor -- start 2024-05-15T15:56:26.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 -- start start 2024-05-15T15:56:26.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 0x7f4354196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:26.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541041d0 0x7f4354196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:26.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4354197270 con 0x7f43541018a0 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.030+0000 7f435b589700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43541973b0 con 0x7f43541041d0 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f4359325700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 0x7f4354196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f4359325700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 0x7f4354196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35248/0 (socket says 172.21.15.49:35248) 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f4359325700 1 -- 172.21.15.49:0/3932678881 learned_addr learned my addr 172.21.15.49:0/3932678881 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f4358b24700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541041d0 0x7f4354196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f4359325700 1 -- 172.21.15.49:0/3932678881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541041d0 msgr2=0x7f4354196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f4359325700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541041d0 0x7f4354196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f4359325700 1 -- 172.21.15.49:0/3932678881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500095f0 con 0x7f43541018a0 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f4358b24700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541041d0 0x7f4354196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f4359325700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 0x7f4354196710 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43500092e0 tx=0x7f435000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f434a7fc700 1 -- 172.21.15.49:0/3932678881 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f435001d070 con 0x7f43541018a0 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f434a7fc700 1 -- 172.21.15.49:0/3932678881 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f435000f460 con 0x7f43541018a0 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435419bdb0 con 0x7f43541018a0 2024-05-15T15:56:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f434a7fc700 1 -- 172.21.15.49:0/3932678881 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4350021640 con 0x7f43541018a0 2024-05-15T15:56:26.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.031+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435419c240 con 0x7f43541018a0 2024-05-15T15:56:26.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.032+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435404ecd0 con 0x7f43541018a0 2024-05-15T15:56:26.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.033+0000 7f434a7fc700 1 -- 172.21.15.49:0/3932678881 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f435000fad0 con 0x7f43541018a0 2024-05-15T15:56:26.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.033+0000 7f434a7fc700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f434006de80 0x7f4340070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:26.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.033+0000 7f434a7fc700 1 -- 172.21.15.49:0/3932678881 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f435008ec30 con 0x7f43541018a0 2024-05-15T15:56:26.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.034+0000 7f4358b24700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f434006de80 0x7f4340070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:26.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.034+0000 7f4358b24700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f434006de80 0x7f4340070340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4344005fd0 tx=0x7f4344005ee0 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:26.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.039+0000 7f434a7fc700 1 -- 172.21.15.49:0/3932678881 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f435404ecd0 con 0x7f43541018a0 2024-05-15T15:56:26.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.266+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f435404ec60 con 0x7f434006de80 2024-05-15T15:56:26.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.267+0000 7f434a7fc700 1 -- 172.21.15.49:0/3932678881 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f435404ec60 con 0x7f434006de80 2024-05-15T15:56:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f434006de80 msgr2=0x7f4340070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f434006de80 0x7f4340070340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4344005fd0 tx=0x7f4344005ee0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 msgr2=0x7f4354196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 0x7f4354196710 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43500092e0 tx=0x7f435000be20 comp rx=0 tx=0).stop 2024-05-15T15:56:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 shutdown_connections 2024-05-15T15:56:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f434006de80 0x7f4340070340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43541018a0 0x7f4354196710 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 --2- 172.21.15.49:0/3932678881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43541041d0 0x7f4354196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 >> 172.21.15.49:0/3932678881 conn(0x7f43540fb1d0 msgr2=0x7f43540fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 shutdown_connections 2024-05-15T15:56:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.269+0000 7f435b589700 1 -- 172.21.15.49:0/3932678881 wait complete. 2024-05-15T15:56:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.382+0000 7f77effe1700 1 -- 172.21.15.49:0/712102711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e80ffbe0 msgr2=0x7f77e8100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.382+0000 7f77effe1700 1 --2- 172.21.15.49:0/712102711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e80ffbe0 0x7f77e8100000 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009910 tx=0x7f77e4009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.382+0000 7f77effe1700 1 -- 172.21.15.49:0/712102711 shutdown_connections 2024-05-15T15:56:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.382+0000 7f77effe1700 1 --2- 172.21.15.49:0/712102711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e8100de0 0x7f77e8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.382+0000 7f77effe1700 1 --2- 172.21.15.49:0/712102711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e80ffbe0 0x7f77e8100000 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.382+0000 7f77effe1700 1 -- 172.21.15.49:0/712102711 >> 172.21.15.49:0/712102711 conn(0x7f77e80fb180 msgr2=0x7f77e80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:26.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.382+0000 7f77effe1700 1 -- 172.21.15.49:0/712102711 shutdown_connections 2024-05-15T15:56:26.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.382+0000 7f77effe1700 1 -- 172.21.15.49:0/712102711 wait complete. 2024-05-15T15:56:26.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77effe1700 1 Processor -- start 2024-05-15T15:56:26.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77effe1700 1 -- start start 2024-05-15T15:56:26.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77effe1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e80ffbe0 0x7f77e8103ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:26.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77effe1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e8100de0 0x7f77e81025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:26.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77effe1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8102b20 con 0x7f77e8100de0 2024-05-15T15:56:26.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77effe1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8102c60 con 0x7f77e80ffbe0 2024-05-15T15:56:26.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77ed57c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e8100de0 0x7f77e81025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:26.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77ed57c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e8100de0 0x7f77e81025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35266/0 (socket says 172.21.15.49:35266) 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77ed57c700 1 -- 172.21.15.49:0/1723650106 learned_addr learned my addr 172.21.15.49:0/1723650106 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77edd7d700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e80ffbe0 0x7f77e8103ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77ed57c700 1 -- 172.21.15.49:0/1723650106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e80ffbe0 msgr2=0x7f77e8103ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77ed57c700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e80ffbe0 0x7f77e8103ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.383+0000 7f77ed57c700 1 -- 172.21.15.49:0/1723650106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40095f0 con 0x7f77e8100de0 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.384+0000 7f77ed57c700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e8100de0 0x7f77e81025e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d8d0 tx=0x7f77d800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.384+0000 7f77deffd700 1 -- 172.21.15.49:0/1723650106 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d80098a0 con 0x7f77e8100de0 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.384+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e8102f40 con 0x7f77e8100de0 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.384+0000 7f77deffd700 1 -- 172.21.15.49:0/1723650106 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77d8010460 con 0x7f77e8100de0 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.384+0000 7f77deffd700 1 -- 172.21.15.49:0/1723650106 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d800f5f0 con 0x7f77e8100de0 2024-05-15T15:56:26.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.384+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e8103450 con 0x7f77e8100de0 2024-05-15T15:56:26.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.385+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e804ec60 con 0x7f77e8100de0 2024-05-15T15:56:26.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.386+0000 7f77deffd700 1 -- 172.21.15.49:0/1723650106 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f77d8009a00 con 0x7f77e8100de0 2024-05-15T15:56:26.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.386+0000 7f77deffd700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f77d406de80 0x7f77d4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:26.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.387+0000 7f77deffd700 1 -- 172.21.15.49:0/1723650106 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f77d808d8e0 con 0x7f77e8100de0 2024-05-15T15:56:26.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.387+0000 7f77edd7d700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f77d406de80 0x7f77d4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:26.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.388+0000 7f77edd7d700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f77d406de80 0x7f77d4070340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77e400b5c0 tx=0x7f77e4005960 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:26.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.393+0000 7f77deffd700 1 -- 172.21.15.49:0/1723650106 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f77d805ac20 con 0x7f77e8100de0 2024-05-15T15:56:26.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.610+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f77e8103450 con 0x7f77d406de80 2024-05-15T15:56:26.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.622+0000 7f77deffd700 1 -- 172.21.15.49:0/1723650106 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f77e8103450 con 0x7f77d406de80 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (9m) 18s ago 11m 24.7M - 0.25.0 c8568f914cd2 3c7b950a78d3 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (11m) 18s ago 11m 8150k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (10m) 21s ago 10m 7880k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (11m) 18s ago 11m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (10m) 21s ago 10m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (9m) 18s ago 10m 86.7M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (5m) 18s ago 5m 144M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (5m) 18s ago 5m 182M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (5m) 21s ago 5m 26.3M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:56:26.624 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (5m) 21s ago 5m 28.0M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (119s) 18s ago 12m 501M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (51s) 21s ago 10m 452M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (12m) 18s ago 12m 54.4M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (10m) 21s ago 10m 44.4M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (31s) 18s ago 11m 5200k - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (26s) 21s ago 10m 5200k - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (9m) 18s ago 9m 213M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (8m) 18s ago 8m 213M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (8m) 18s ago 8m 199M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (8m) 21s ago 8m 255M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (7m) 21s ago 7m 210M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (7m) 21s ago 7m 197M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:56:26.625 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (59s) 18s ago 10m 46.7M - 2.43.0 a07b618ecd1d c65cd3f949ef 2024-05-15T15:56:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.624+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f77d406de80 msgr2=0x7f77d4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.624+0000 7f77effe1700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f77d406de80 0x7f77d4070340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77e400b5c0 tx=0x7f77e4005960 comp rx=0 tx=0).stop 2024-05-15T15:56:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.624+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e8100de0 msgr2=0x7f77e81025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.624+0000 7f77effe1700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e8100de0 0x7f77e81025e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d8d0 tx=0x7f77d800dbe0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.624+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 shutdown_connections 2024-05-15T15:56:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.624+0000 7f77effe1700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f77d406de80 0x7f77d4070340 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.624+0000 7f77effe1700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e80ffbe0 0x7f77e8103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.624+0000 7f77effe1700 1 --2- 172.21.15.49:0/1723650106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77e8100de0 0x7f77e81025e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.624+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 >> 172.21.15.49:0/1723650106 conn(0x7f77e80fb180 msgr2=0x7f77e80691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.625+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 shutdown_connections 2024-05-15T15:56:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.625+0000 7f77effe1700 1 -- 172.21.15.49:0/1723650106 wait complete. 2024-05-15T15:56:26.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:26 smithi049 ceph-mon[28399]: pgmap v52: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 235 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-05-15T15:56:26.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:26 smithi049 ceph-mon[28399]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:26.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.739+0000 7f776c349700 1 -- 172.21.15.49:0/623828365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7764100e50 msgr2=0x7f77641012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.739+0000 7f776c349700 1 --2- 172.21.15.49:0/623828365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7764100e50 0x7f77641012b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7754009910 tx=0x7f7754009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:26.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.739+0000 7f776c349700 1 -- 172.21.15.49:0/623828365 shutdown_connections 2024-05-15T15:56:26.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.739+0000 7f776c349700 1 --2- 172.21.15.49:0/623828365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7764100e50 0x7f77641012b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.739+0000 7f776c349700 1 --2- 172.21.15.49:0/623828365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77640ffc50 0x7f7764100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.739+0000 7f776c349700 1 -- 172.21.15.49:0/623828365 >> 172.21.15.49:0/623828365 conn(0x7f77640fb1d0 msgr2=0x7f77640fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:26.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.739+0000 7f776c349700 1 -- 172.21.15.49:0/623828365 shutdown_connections 2024-05-15T15:56:26.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.739+0000 7f776c349700 1 -- 172.21.15.49:0/623828365 wait complete. 2024-05-15T15:56:26.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776c349700 1 Processor -- start 2024-05-15T15:56:26.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776c349700 1 -- start start 2024-05-15T15:56:26.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776c349700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77640ffc50 0x7f7764069140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:26.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776c349700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7764100e50 0x7f776419b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:26.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776c349700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776419bab0 con 0x7f77640ffc50 2024-05-15T15:56:26.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776c349700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776419bbf0 con 0x7f7764100e50 2024-05-15T15:56:26.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776a0e5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77640ffc50 0x7f7764069140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:26.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776a0e5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77640ffc50 0x7f7764069140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35296/0 (socket says 172.21.15.49:35296) 2024-05-15T15:56:26.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776a0e5700 1 -- 172.21.15.49:0/4144166471 learned_addr learned my addr 172.21.15.49:0/4144166471 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:26.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f77698e4700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7764100e50 0x7f776419b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:26.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776a0e5700 1 -- 172.21.15.49:0/4144166471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7764100e50 msgr2=0x7f776419b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:26.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776a0e5700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7764100e50 0x7f776419b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:26.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.740+0000 7f776a0e5700 1 -- 172.21.15.49:0/4144166471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77540095f0 con 0x7f77640ffc50 2024-05-15T15:56:26.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.741+0000 7f776a0e5700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77640ffc50 0x7f7764069140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f776000b480 tx=0x7f776000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:26.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.741+0000 7f775b7fe700 1 -- 172.21.15.49:0/4144166471 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77600118c0 con 0x7f77640ffc50 2024-05-15T15:56:26.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.741+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776419bea0 con 0x7f77640ffc50 2024-05-15T15:56:26.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.741+0000 7f775b7fe700 1 -- 172.21.15.49:0/4144166471 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7760010380 con 0x7f77640ffc50 2024-05-15T15:56:26.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.741+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776419c3e0 con 0x7f77640ffc50 2024-05-15T15:56:26.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.741+0000 7f775b7fe700 1 -- 172.21.15.49:0/4144166471 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f776000f4f0 con 0x7f77640ffc50 2024-05-15T15:56:26.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.743+0000 7f775b7fe700 1 -- 172.21.15.49:0/4144166471 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f77600104f0 con 0x7f77640ffc50 2024-05-15T15:56:26.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.743+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776418f870 con 0x7f77640ffc50 2024-05-15T15:56:26.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.743+0000 7f775b7fe700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f775006de80 0x7f7750070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:26.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.744+0000 7f77698e4700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f775006de80 0x7f7750070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:26.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.744+0000 7f775b7fe700 1 -- 172.21.15.49:0/4144166471 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f776008d800 con 0x7f77640ffc50 2024-05-15T15:56:26.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.744+0000 7f77698e4700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f775006de80 0x7f7750070340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f77540051c0 tx=0x7f7754005920 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:26.749+0000 7f775b7fe700 1 -- 172.21.15.49:0/4144166471 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f776005ab40 con 0x7f77640ffc50 2024-05-15T15:56:26.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:26 smithi144 ceph-mon[36389]: pgmap v52: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 235 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-05-15T15:56:26.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:26 smithi144 ceph-mon[36389]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:27.063 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:56:27.063 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:56:27.063 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:56:27.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:56:27.063 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 12, 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:56:27.064 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:56:27.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.060+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f776404ec60 con 0x7f77640ffc50 2024-05-15T15:56:27.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.061+0000 7f775b7fe700 1 -- 172.21.15.49:0/4144166471 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f776005a6d0 con 0x7f77640ffc50 2024-05-15T15:56:27.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.063+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f775006de80 msgr2=0x7f7750070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:27.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.063+0000 7f776c349700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f775006de80 0x7f7750070340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f77540051c0 tx=0x7f7754005920 comp rx=0 tx=0).stop 2024-05-15T15:56:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.063+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77640ffc50 msgr2=0x7f7764069140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.063+0000 7f776c349700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77640ffc50 0x7f7764069140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f776000b480 tx=0x7f776000b840 comp rx=0 tx=0).stop 2024-05-15T15:56:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.063+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 shutdown_connections 2024-05-15T15:56:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.063+0000 7f776c349700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f775006de80 0x7f7750070340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.063+0000 7f776c349700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77640ffc50 0x7f7764069140 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.063+0000 7f776c349700 1 --2- 172.21.15.49:0/4144166471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7764100e50 0x7f776419b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.063+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 >> 172.21.15.49:0/4144166471 conn(0x7f77640fb1d0 msgr2=0x7f7764104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.064+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 shutdown_connections 2024-05-15T15:56:27.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.064+0000 7f776c349700 1 -- 172.21.15.49:0/4144166471 wait complete. 2024-05-15T15:56:27.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.182+0000 7fa6da9e3700 1 -- 172.21.15.49:0/2107662873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 msgr2=0x7fa6d4101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:27.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.182+0000 7fa6da9e3700 1 --2- 172.21.15.49:0/2107662873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 0x7fa6d4101120 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009910 tx=0x7fa6c4009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:27.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.182+0000 7fa6da9e3700 1 -- 172.21.15.49:0/2107662873 shutdown_connections 2024-05-15T15:56:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.182+0000 7fa6da9e3700 1 --2- 172.21.15.49:0/2107662873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4101f00 0x7fa6d4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.182+0000 7fa6da9e3700 1 --2- 172.21.15.49:0/2107662873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 0x7fa6d4101120 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.182+0000 7fa6da9e3700 1 -- 172.21.15.49:0/2107662873 >> 172.21.15.49:0/2107662873 conn(0x7fa6d40fc2c0 msgr2=0x7fa6d40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.182+0000 7fa6da9e3700 1 -- 172.21.15.49:0/2107662873 shutdown_connections 2024-05-15T15:56:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.182+0000 7fa6da9e3700 1 -- 172.21.15.49:0/2107662873 wait complete. 2024-05-15T15:56:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6da9e3700 1 Processor -- start 2024-05-15T15:56:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6da9e3700 1 -- start start 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6da9e3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 0x7fa6d4074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6da9e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4101f00 0x7fa6d4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6da9e3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40738d0 con 0x7fa6d4100d00 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6da9e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4073a10 con 0x7fa6d4101f00 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6d3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 0x7fa6d4074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6d3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 0x7fa6d4074cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35310/0 (socket says 172.21.15.49:35310) 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6d3fff700 1 -- 172.21.15.49:0/1547184136 learned_addr learned my addr 172.21.15.49:0/1547184136 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.183+0000 7fa6cbfff700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4101f00 0x7fa6d4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.184+0000 7fa6d3fff700 1 -- 172.21.15.49:0/1547184136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4101f00 msgr2=0x7fa6d4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.184+0000 7fa6d3fff700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4101f00 0x7fa6d4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.184+0000 7fa6d3fff700 1 -- 172.21.15.49:0/1547184136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40095f0 con 0x7fa6d4100d00 2024-05-15T15:56:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.184+0000 7fa6d3fff700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 0x7fa6d4074cf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40059d0 tx=0x7fa6c400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.184+0000 7fa6d1ffb700 1 -- 172.21.15.49:0/1547184136 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6c401d070 con 0x7fa6d4100d00 2024-05-15T15:56:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.184+0000 7fa6d1ffb700 1 -- 172.21.15.49:0/1547184136 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6c4005dc0 con 0x7fa6d4100d00 2024-05-15T15:56:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.184+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d4073c90 con 0x7fa6d4100d00 2024-05-15T15:56:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.184+0000 7fa6d1ffb700 1 -- 172.21.15.49:0/1547184136 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6c4021410 con 0x7fa6d4100d00 2024-05-15T15:56:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.184+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d4074200 con 0x7fa6d4100d00 2024-05-15T15:56:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.185+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d4190880 con 0x7fa6d4100d00 2024-05-15T15:56:27.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.186+0000 7fa6d1ffb700 1 -- 172.21.15.49:0/1547184136 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7fa6c400f460 con 0x7fa6d4100d00 2024-05-15T15:56:27.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.187+0000 7fa6d1ffb700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa6c006de80 0x7fa6c0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:27.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.187+0000 7fa6d1ffb700 1 -- 172.21.15.49:0/1547184136 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa6c408ebe0 con 0x7fa6d4100d00 2024-05-15T15:56:27.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.187+0000 7fa6cbfff700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa6c006de80 0x7fa6c0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:27.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.187+0000 7fa6cbfff700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa6c006de80 0x7fa6c0070340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005950 tx=0x7fa6bc00b410 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:27.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.192+0000 7fa6d1ffb700 1 -- 172.21.15.49:0/1547184136 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fa6c405bf20 con 0x7fa6d4100d00 2024-05-15T15:56:27.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.426+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6d4074200 con 0x7fa6c006de80 2024-05-15T15:56:27.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.427+0000 7fa6d1ffb700 1 -- 172.21.15.49:0/1547184136 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fa6d4074200 con 0x7fa6c006de80 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr" 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "2/2 daemons upgraded", 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading prometheus daemons", 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:56:27.429 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa6c006de80 msgr2=0x7fa6c0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa6c006de80 0x7fa6c0070340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005950 tx=0x7fa6bc00b410 comp rx=0 tx=0).stop 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 msgr2=0x7fa6d4074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 0x7fa6d4074cf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40059d0 tx=0x7fa6c400bd70 comp rx=0 tx=0).stop 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 shutdown_connections 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa6c006de80 0x7fa6c0070340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4100d00 0x7fa6d4074cf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 --2- 172.21.15.49:0/1547184136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4101f00 0x7fa6d4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 >> 172.21.15.49:0/1547184136 conn(0x7fa6d40fc2c0 msgr2=0x7fa6d4105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 shutdown_connections 2024-05-15T15:56:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:27.429+0000 7fa6da9e3700 1 -- 172.21.15.49:0/1547184136 wait complete. 2024-05-15T15:56:27.588 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:27 smithi049 ceph-mon[28399]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:27.588 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:27 smithi049 ceph-mon[28399]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:27.588 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:27 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/4144166471' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:27 smithi144 ceph-mon[36389]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:27 smithi144 ceph-mon[36389]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:27 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/4144166471' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:28.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:28 smithi049 ceph-mon[28399]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:28.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:28 smithi049 ceph-mon[28399]: pgmap v53: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 20 KiB/s wr, 6 op/s 2024-05-15T15:56:28.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:28 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:28.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:28 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:28.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:28 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:29.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:28 smithi144 ceph-mon[36389]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:29.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:28 smithi144 ceph-mon[36389]: pgmap v53: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 20 KiB/s wr, 6 op/s 2024-05-15T15:56:29.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:28 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:29.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:28 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:29.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:28 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:31.100 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:30 smithi049 ceph-mon[28399]: pgmap v54: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-05-15T15:56:31.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:30 smithi144 ceph-mon[36389]: pgmap v54: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-05-15T15:56:33.174 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:32 smithi049 ceph-mon[28399]: pgmap v55: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:33.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:32 smithi144 ceph-mon[36389]: pgmap v55: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:34.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:33 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:56:34.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:33 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:34.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:33 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:34.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:33 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:56:34.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:33 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:34.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:33 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:35 smithi144 ceph-mon[36389]: pgmap v56: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:35 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:35 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:35.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:35 smithi049 ceph-mon[28399]: pgmap v56: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:35.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:35 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:35.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:35 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:37.401 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:37 smithi049 ceph-mon[28399]: pgmap v57: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:56:37.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:37 smithi144 ceph-mon[36389]: pgmap v57: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:56:38.921 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: pgmap v58: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:38.922 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:38.923 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:38.923 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:38.923 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:38 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: pgmap v58: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:39.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:38 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:41.160 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:40 smithi049 ceph-mon[28399]: pgmap v59: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:41.160 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:40 smithi049 ceph-mon[28399]: Upgrade: Updating alertmanager.smithi049 2024-05-15T15:56:41.160 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:40 smithi049 ceph-mon[28399]: Deploying daemon alertmanager.smithi049 on smithi049 2024-05-15T15:56:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:40 smithi144 ceph-mon[36389]: pgmap v59: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:40 smithi144 ceph-mon[36389]: Upgrade: Updating alertmanager.smithi049 2024-05-15T15:56:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:40 smithi144 ceph-mon[36389]: Deploying daemon alertmanager.smithi049 on smithi049 2024-05-15T15:56:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:42 smithi144 ceph-mon[36389]: pgmap v60: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:43.200 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:42 smithi049 ceph-mon[28399]: pgmap v60: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:44.927 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:44 smithi049 ceph-mon[28399]: pgmap v61: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:44.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:44 smithi144 ceph-mon[36389]: pgmap v61: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:46.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:46.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:46.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:46 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:46.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:46 smithi144 ceph-mon[36389]: pgmap v62: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:46.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:46.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:46.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:46 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:46.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:46 smithi049 ceph-mon[28399]: pgmap v62: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:48.816 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:48 smithi049 ceph-mon[28399]: pgmap v63: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:48.816 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:48 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:56:48.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:48 smithi144 ceph-mon[36389]: pgmap v63: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:48.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:48 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:56:50.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:50 smithi144 ceph-mon[36389]: pgmap v64: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:50.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:50 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:50.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:50 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:50 smithi049 ceph-mon[28399]: pgmap v64: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:50 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:50 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:52.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:52 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:52.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:52 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:52.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:52 smithi144 ceph-mon[36389]: pgmap v65: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:52 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:52 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:52 smithi049 ceph-mon[28399]: pgmap v65: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:54.782 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:54 smithi144 ceph-mon[36389]: pgmap v66: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:54.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:54 smithi049 ceph-mon[28399]: pgmap v66: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:56:56.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:56 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:56.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:56 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:57.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:57 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-15T15:56:57.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:57 smithi144 ceph-mon[36389]: pgmap v67: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:57.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:57 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-15T15:56:57.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:57 smithi049 ceph-mon[28399]: pgmap v67: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:57.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.542+0000 7fd0bf827700 1 -- 172.21.15.49:0/3773506764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 msgr2=0x7fd0b8100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.542+0000 7fd0bf827700 1 --2- 172.21.15.49:0/3773506764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 0x7fd0b8100000 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009910 tx=0x7fd0b4009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.542+0000 7fd0bf827700 1 -- 172.21.15.49:0/3773506764 shutdown_connections 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.542+0000 7fd0bf827700 1 --2- 172.21.15.49:0/3773506764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b8100de0 0x7fd0b8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.542+0000 7fd0bf827700 1 --2- 172.21.15.49:0/3773506764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 0x7fd0b8100000 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.542+0000 7fd0bf827700 1 -- 172.21.15.49:0/3773506764 >> 172.21.15.49:0/3773506764 conn(0x7fd0b80fb180 msgr2=0x7fd0b80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.542+0000 7fd0bf827700 1 -- 172.21.15.49:0/3773506764 shutdown_connections 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.542+0000 7fd0bf827700 1 -- 172.21.15.49:0/3773506764 wait complete. 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bf827700 1 Processor -- start 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bf827700 1 -- start start 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bf827700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 0x7fd0b8196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bf827700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b8100de0 0x7fd0b8196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:57.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bf827700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b81972a0 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bf827700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b81973e0 con 0x7fd0b8100de0 2024-05-15T15:56:57.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bd5c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 0x7fd0b8196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:57.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bd5c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 0x7fd0b8196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41522/0 (socket says 172.21.15.49:41522) 2024-05-15T15:56:57.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bd5c3700 1 -- 172.21.15.49:0/1371990159 learned_addr learned my addr 172.21.15.49:0/1371990159 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:57.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bcdc2700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b8100de0 0x7fd0b8196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:57.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bd5c3700 1 -- 172.21.15.49:0/1371990159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b8100de0 msgr2=0x7fd0b8196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:57.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bd5c3700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b8100de0 0x7fd0b8196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.543+0000 7fd0bd5c3700 1 -- 172.21.15.49:0/1371990159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b40095f0 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.544+0000 7fd0bd5c3700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 0x7fd0b8196740 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40059d0 tx=0x7fd0b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:57.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.544+0000 7fd0ae7fc700 1 -- 172.21.15.49:0/1371990159 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0b401d070 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.544+0000 7fd0ae7fc700 1 -- 172.21.15.49:0/1371990159 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0b4005dc0 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.544+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b819bcd0 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.544+0000 7fd0ae7fc700 1 -- 172.21.15.49:0/1371990159 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0b4021410 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.544+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b819c1c0 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.545+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b804ec60 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.545+0000 7fd0ae7fc700 1 -- 172.21.15.49:0/1371990159 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7fd0b400f840 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.546+0000 7fd0ae7fc700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd0a406de80 0x7fd0a4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:57.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.546+0000 7fd0ae7fc700 1 -- 172.21.15.49:0/1371990159 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd0b408ec20 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.546+0000 7fd0bcdc2700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd0a406de80 0x7fd0a4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:57.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.546+0000 7fd0bcdc2700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd0a406de80 0x7fd0a4070340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8005950 tx=0x7fd0a80058e0 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:57.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.551+0000 7fd0ae7fc700 1 -- 172.21.15.49:0/1371990159 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fd0b405c090 con 0x7fd0b80ffbe0 2024-05-15T15:56:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.781+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0b8069880 con 0x7fd0a406de80 2024-05-15T15:56:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.782+0000 7fd0ae7fc700 1 -- 172.21.15.49:0/1371990159 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fd0b8069880 con 0x7fd0a406de80 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd0a406de80 msgr2=0x7fd0a4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd0a406de80 0x7fd0a4070340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8005950 tx=0x7fd0a80058e0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 msgr2=0x7fd0b8196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 0x7fd0b8196740 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40059d0 tx=0x7fd0b400be20 comp rx=0 tx=0).stop 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 shutdown_connections 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd0a406de80 0x7fd0a4070340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0b80ffbe0 0x7fd0b8196740 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 --2- 172.21.15.49:0/1371990159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b8100de0 0x7fd0b8196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 >> 172.21.15.49:0/1371990159 conn(0x7fd0b80fb180 msgr2=0x7fd0b8069170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 shutdown_connections 2024-05-15T15:56:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.784+0000 7fd0bf827700 1 -- 172.21.15.49:0/1371990159 wait complete. 2024-05-15T15:56:57.797 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:56:57.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.896+0000 7f03c4334700 1 -- 172.21.15.49:0/691127804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc101ef0 msgr2=0x7f03bc102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:57.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.896+0000 7f03c4334700 1 --2- 172.21.15.49:0/691127804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc101ef0 0x7f03bc102350 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009910 tx=0x7f03ac009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:57.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.896+0000 7f03c4334700 1 -- 172.21.15.49:0/691127804 shutdown_connections 2024-05-15T15:56:57.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.896+0000 7f03c4334700 1 --2- 172.21.15.49:0/691127804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc101ef0 0x7f03bc102350 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.896+0000 7f03c4334700 1 --2- 172.21.15.49:0/691127804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc100cf0 0x7f03bc101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.896+0000 7f03c4334700 1 -- 172.21.15.49:0/691127804 >> 172.21.15.49:0/691127804 conn(0x7f03bc0fc290 msgr2=0x7f03bc0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:57.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.896+0000 7f03c4334700 1 -- 172.21.15.49:0/691127804 shutdown_connections 2024-05-15T15:56:57.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.896+0000 7f03c4334700 1 -- 172.21.15.49:0/691127804 wait complete. 2024-05-15T15:56:57.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.897+0000 7f03c4334700 1 Processor -- start 2024-05-15T15:56:57.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.897+0000 7f03c4334700 1 -- start start 2024-05-15T15:56:57.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.897+0000 7f03c4334700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc100cf0 0x7f03bc1967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:57.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.897+0000 7f03c4334700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc101ef0 0x7f03bc196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:57.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.897+0000 7f03c4334700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc197340 con 0x7f03bc100cf0 2024-05-15T15:56:57.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.897+0000 7f03c4334700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc197480 con 0x7f03bc101ef0 2024-05-15T15:56:57.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.897+0000 7f03c20d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc100cf0 0x7f03bc1967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.897+0000 7f03c20d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc100cf0 0x7f03bc1967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41544/0 (socket says 172.21.15.49:41544) 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03c20d0700 1 -- 172.21.15.49:0/946806792 learned_addr learned my addr 172.21.15.49:0/946806792 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03c18cf700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc101ef0 0x7f03bc196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03c20d0700 1 -- 172.21.15.49:0/946806792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc101ef0 msgr2=0x7f03bc196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03c20d0700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc101ef0 0x7f03bc196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03c20d0700 1 -- 172.21.15.49:0/946806792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac0095f0 con 0x7f03bc100cf0 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03c18cf700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc101ef0 0x7f03bc196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03c20d0700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc100cf0 0x7f03bc1967e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f03b800b480 tx=0x7f03b800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03b37fe700 1 -- 172.21.15.49:0/946806792 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b80118c0 con 0x7f03bc100cf0 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03b37fe700 1 -- 172.21.15.49:0/946806792 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03b8010380 con 0x7f03bc100cf0 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc19bee0 con 0x7f03bc100cf0 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03b37fe700 1 -- 172.21.15.49:0/946806792 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b800f4f0 con 0x7f03bc100cf0 2024-05-15T15:56:57.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.898+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc19c420 con 0x7f03bc100cf0 2024-05-15T15:56:57.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.899+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc190910 con 0x7f03bc100cf0 2024-05-15T15:56:57.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.900+0000 7f03b37fe700 1 -- 172.21.15.49:0/946806792 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f03b80104f0 con 0x7f03bc100cf0 2024-05-15T15:56:57.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.900+0000 7f03b37fe700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f03a806de80 0x7f03a8070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:57.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.900+0000 7f03b37fe700 1 -- 172.21.15.49:0/946806792 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f03b808d7f0 con 0x7f03bc100cf0 2024-05-15T15:56:57.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.900+0000 7f03c18cf700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f03a806de80 0x7f03a8070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:57.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.901+0000 7f03c18cf700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f03a806de80 0x7f03a8070340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0059d0 tx=0x7f03ac005960 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:57.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:57.905+0000 7f03b37fe700 1 -- 172.21.15.49:0/946806792 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f03b805ab30 con 0x7f03bc100cf0 2024-05-15T15:56:58.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.135+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03bc02cf00 con 0x7f03a806de80 2024-05-15T15:56:58.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.136+0000 7f03b37fe700 1 -- 172.21.15.49:0/946806792 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f03bc02cf00 con 0x7f03a806de80 2024-05-15T15:56:58.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f03a806de80 msgr2=0x7f03a8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f03a806de80 0x7f03a8070340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0059d0 tx=0x7f03ac005960 comp rx=0 tx=0).stop 2024-05-15T15:56:58.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc100cf0 msgr2=0x7f03bc1967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc100cf0 0x7f03bc1967e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f03b800b480 tx=0x7f03b800b840 comp rx=0 tx=0).stop 2024-05-15T15:56:58.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 shutdown_connections 2024-05-15T15:56:58.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f03a806de80 0x7f03a8070340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03bc100cf0 0x7f03bc1967e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 --2- 172.21.15.49:0/946806792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc101ef0 0x7f03bc196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 >> 172.21.15.49:0/946806792 conn(0x7f03bc0fc290 msgr2=0x7f03bc1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:58.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 shutdown_connections 2024-05-15T15:56:58.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.139+0000 7f03c4334700 1 -- 172.21.15.49:0/946806792 wait complete. 2024-05-15T15:56:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.252+0000 7f1caf66c700 1 -- 172.21.15.49:0/399376364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca8100e50 msgr2=0x7f1ca81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.252+0000 7f1caf66c700 1 --2- 172.21.15.49:0/399376364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca8100e50 0x7f1ca81012b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009910 tx=0x7f1c98009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.252+0000 7f1caf66c700 1 -- 172.21.15.49:0/399376364 shutdown_connections 2024-05-15T15:56:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.252+0000 7f1caf66c700 1 --2- 172.21.15.49:0/399376364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca8100e50 0x7f1ca81012b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.252+0000 7f1caf66c700 1 --2- 172.21.15.49:0/399376364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca80ffc50 0x7f1ca8100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.252+0000 7f1caf66c700 1 -- 172.21.15.49:0/399376364 >> 172.21.15.49:0/399376364 conn(0x7f1ca80fb1d0 msgr2=0x7f1ca80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.252+0000 7f1caf66c700 1 -- 172.21.15.49:0/399376364 shutdown_connections 2024-05-15T15:56:58.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.252+0000 7f1caf66c700 1 -- 172.21.15.49:0/399376364 wait complete. 2024-05-15T15:56:58.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.253+0000 7f1caf66c700 1 Processor -- start 2024-05-15T15:56:58.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.253+0000 7f1caf66c700 1 -- start start 2024-05-15T15:56:58.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1caf66c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80ffc50 0x7f1ca8069160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:58.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1caf66c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca8100e50 0x7f1ca819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:58.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1caf66c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca819ba90 con 0x7f1ca80ffc50 2024-05-15T15:56:58.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1caf66c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca819bbd0 con 0x7f1ca8100e50 2024-05-15T15:56:58.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1cad408700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80ffc50 0x7f1ca8069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1cad408700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80ffc50 0x7f1ca8069160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41566/0 (socket says 172.21.15.49:41566) 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1cad408700 1 -- 172.21.15.49:0/3047868272 learned_addr learned my addr 172.21.15.49:0/3047868272 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1cad408700 1 -- 172.21.15.49:0/3047868272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca8100e50 msgr2=0x7f1ca819b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1cad408700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca8100e50 0x7f1ca819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1cad408700 1 -- 172.21.15.49:0/3047868272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980095f0 con 0x7f1ca80ffc50 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1cad408700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80ffc50 0x7f1ca8069160 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400b480 tx=0x7f1ca400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1c9e7fc700 1 -- 172.21.15.49:0/3047868272 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca40118c0 con 0x7f1ca80ffc50 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca819beb0 con 0x7f1ca80ffc50 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1c9e7fc700 1 -- 172.21.15.49:0/3047868272 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ca4010380 con 0x7f1ca80ffc50 2024-05-15T15:56:58.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.254+0000 7f1c9e7fc700 1 -- 172.21.15.49:0/3047868272 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca400f4f0 con 0x7f1ca80ffc50 2024-05-15T15:56:58.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.255+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca819c420 con 0x7f1ca80ffc50 2024-05-15T15:56:58.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.256+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca818f870 con 0x7f1ca80ffc50 2024-05-15T15:56:58.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.256+0000 7f1c9e7fc700 1 -- 172.21.15.49:0/3047868272 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f1ca400f7b0 con 0x7f1ca80ffc50 2024-05-15T15:56:58.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.257+0000 7f1c9e7fc700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c9406de80 0x7f1c94070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:58.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.257+0000 7f1c9e7fc700 1 -- 172.21.15.49:0/3047868272 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1ca408d8c0 con 0x7f1ca80ffc50 2024-05-15T15:56:58.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.257+0000 7f1cacc07700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c9406de80 0x7f1c94070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:58.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.257+0000 7f1cacc07700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c9406de80 0x7f1c94070340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800b5c0 tx=0x7f1c98005160 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:58.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.262+0000 7f1c9e7fc700 1 -- 172.21.15.49:0/3047868272 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f1ca405ab70 con 0x7f1ca80ffc50 2024-05-15T15:56:58.370 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:58 smithi049 ceph-mon[28399]: Upgrade: Updating grafana.smithi049 2024-05-15T15:56:58.370 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:58 smithi049 ceph-mon[28399]: Deploying daemon grafana.smithi049 on smithi049 2024-05-15T15:56:58.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:58 smithi144 ceph-mon[36389]: Upgrade: Updating grafana.smithi049 2024-05-15T15:56:58.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:58 smithi144 ceph-mon[36389]: Deploying daemon grafana.smithi049 on smithi049 2024-05-15T15:56:58.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.491+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ca802cf60 con 0x7f1c9406de80 2024-05-15T15:56:58.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.512+0000 7f1c9e7fc700 1 -- 172.21.15.49:0/3047868272 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1ca802cf60 con 0x7f1c9406de80 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (13s) 8s ago 11m 17.1M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (12m) 8s ago 12m 8216k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (10m) 53s ago 10m 7880k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (11m) 8s ago 11m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (10m) 53s ago 10m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (10m) 8s ago 11m 86.1M - 9.4.7 954c08fa6188 dbe37e003c4a 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (6m) 8s ago 6m 90.2M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (6m) 8s ago 6m 153M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (6m) 53s ago 6m 26.3M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (6m) 53s ago 6m 28.0M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (2m) 8s ago 13m 505M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (83s) 53s ago 10m 452M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (13m) 8s ago 13m 55.2M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (10m) 53s ago 10m 44.4M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (63s) 8s ago 11m 7906k - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (57s) 53s ago 10m 5200k - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (9m) 8s ago 9m 217M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (9m) 8s ago 9m 217M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (8m) 8s ago 8m 212M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (8m) 53s ago 8m 255M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (8m) 53s ago 8m 210M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (7m) 53s ago 7m 197M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (31s) 8s ago 11m 46.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c9406de80 msgr2=0x7f1c94070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c9406de80 0x7f1c94070340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800b5c0 tx=0x7f1c98005160 comp rx=0 tx=0).stop 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80ffc50 msgr2=0x7f1ca8069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80ffc50 0x7f1ca8069160 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400b480 tx=0x7f1ca400b840 comp rx=0 tx=0).stop 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 shutdown_connections 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f1c9406de80 0x7f1c94070340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80ffc50 0x7f1ca8069160 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 --2- 172.21.15.49:0/3047868272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca8100e50 0x7f1ca819b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 >> 172.21.15.49:0/3047868272 conn(0x7f1ca80fb1d0 msgr2=0x7f1ca8104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 shutdown_connections 2024-05-15T15:56:58.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.515+0000 7f1caf66c700 1 -- 172.21.15.49:0/3047868272 wait complete. 2024-05-15T15:56:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.627+0000 7f690f2ae700 1 -- 172.21.15.49:0/1417500598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908101840 msgr2=0x7f6908103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.627+0000 7f690f2ae700 1 --2- 172.21.15.49:0/1417500598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908101840 0x7f6908103c30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6904009910 tx=0x7f6904009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.627+0000 7f690f2ae700 1 -- 172.21.15.49:0/1417500598 shutdown_connections 2024-05-15T15:56:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.627+0000 7f690f2ae700 1 --2- 172.21.15.49:0/1417500598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6908104170 0x7f6908106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.627+0000 7f690f2ae700 1 --2- 172.21.15.49:0/1417500598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908101840 0x7f6908103c30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.627+0000 7f690f2ae700 1 -- 172.21.15.49:0/1417500598 >> 172.21.15.49:0/1417500598 conn(0x7f69080fb1d0 msgr2=0x7f69080fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.627+0000 7f690f2ae700 1 -- 172.21.15.49:0/1417500598 shutdown_connections 2024-05-15T15:56:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.627+0000 7f690f2ae700 1 -- 172.21.15.49:0/1417500598 wait complete. 2024-05-15T15:56:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.627+0000 7f690f2ae700 1 Processor -- start 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690f2ae700 1 -- start start 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690f2ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6908101840 0x7f69081966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690f2ae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908104170 0x7f6908196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690f2ae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6908197250 con 0x7f6908104170 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690f2ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6908197390 con 0x7f6908101840 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690c849700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908104170 0x7f6908196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690c849700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908104170 0x7f6908196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41580/0 (socket says 172.21.15.49:41580) 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690c849700 1 -- 172.21.15.49:0/718219213 learned_addr learned my addr 172.21.15.49:0/718219213 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690d04a700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6908101840 0x7f69081966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690c849700 1 -- 172.21.15.49:0/718219213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6908101840 msgr2=0x7f69081966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690c849700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6908101840 0x7f69081966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690c849700 1 -- 172.21.15.49:0/718219213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040095f0 con 0x7f6908104170 2024-05-15T15:56:58.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.628+0000 7f690c849700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908104170 0x7f6908196c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f68f800d8d0 tx=0x7f68f800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.629+0000 7f68fe7fc700 1 -- 172.21.15.49:0/718219213 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68f80098a0 con 0x7f6908104170 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.629+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690819bdf0 con 0x7f6908104170 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.629+0000 7f68fe7fc700 1 -- 172.21.15.49:0/718219213 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68f8010460 con 0x7f6908104170 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.629+0000 7f68fe7fc700 1 -- 172.21.15.49:0/718219213 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68f800f5f0 con 0x7f6908104170 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.629+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690819c310 con 0x7f6908104170 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.630+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690804ec60 con 0x7f6908104170 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.630+0000 7f68fe7fc700 1 -- 172.21.15.49:0/718219213 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f68f8009a00 con 0x7f6908104170 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.631+0000 7f68fe7fc700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f68f406de80 0x7f68f4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.631+0000 7f68fe7fc700 1 -- 172.21.15.49:0/718219213 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f68f808db00 con 0x7f6908104170 2024-05-15T15:56:58.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.631+0000 7f690d04a700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f68f406de80 0x7f68f4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:58.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.632+0000 7f690d04a700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f68f406de80 0x7f68f4070340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6904005090 tx=0x7f6904005960 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:58.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.636+0000 7f68fe7fc700 1 -- 172.21.15.49:0/718219213 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f68f805ada0 con 0x7f6908104170 2024-05-15T15:56:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.944+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f69081098b0 con 0x7f6908104170 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.945+0000 7f68fe7fc700 1 -- 172.21.15.49:0/718219213 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f68f805a930 con 0x7f6908104170 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:56:58.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:56:58.948 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:56:58.948 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:56:58.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:56:58.948 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:56:58.948 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 12, 2024-05-15T15:56:58.948 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:56:58.948 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:56:58.948 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f68f406de80 msgr2=0x7f68f4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f68f406de80 0x7f68f4070340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6904005090 tx=0x7f6904005960 comp rx=0 tx=0).stop 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908104170 msgr2=0x7f6908196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908104170 0x7f6908196c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f68f800d8d0 tx=0x7f68f800dbe0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 shutdown_connections 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f68f406de80 0x7f68f4070340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6908101840 0x7f69081966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 --2- 172.21.15.49:0/718219213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908104170 0x7f6908196c30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 >> 172.21.15.49:0/718219213 conn(0x7f69080fb1d0 msgr2=0x7f69080fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 shutdown_connections 2024-05-15T15:56:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:58.947+0000 7f690f2ae700 1 -- 172.21.15.49:0/718219213 wait complete. 2024-05-15T15:56:59.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.060+0000 7f66c2352700 1 -- 172.21.15.49:0/1755784196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc101ed0 msgr2=0x7f66bc102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:59.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.060+0000 7f66c2352700 1 --2- 172.21.15.49:0/1755784196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc101ed0 0x7f66bc102330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009910 tx=0x7f66b0009c20 comp rx=0 tx=0).stop 2024-05-15T15:56:59.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.060+0000 7f66c2352700 1 -- 172.21.15.49:0/1755784196 shutdown_connections 2024-05-15T15:56:59.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.060+0000 7f66c2352700 1 --2- 172.21.15.49:0/1755784196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc101ed0 0x7f66bc102330 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:59.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.060+0000 7f66c2352700 1 --2- 172.21.15.49:0/1755784196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc100cd0 0x7f66bc1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:59.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.060+0000 7f66c2352700 1 -- 172.21.15.49:0/1755784196 >> 172.21.15.49:0/1755784196 conn(0x7f66bc0fc2b0 msgr2=0x7f66bc0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:59.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.060+0000 7f66c2352700 1 -- 172.21.15.49:0/1755784196 shutdown_connections 2024-05-15T15:56:59.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.060+0000 7f66c2352700 1 -- 172.21.15.49:0/1755784196 wait complete. 2024-05-15T15:56:59.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.061+0000 7f66c2352700 1 Processor -- start 2024-05-15T15:56:59.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.061+0000 7f66c2352700 1 -- start start 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66c2352700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc100cd0 0x7f66bc196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66c2352700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc101ed0 0x7f66bc196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66c2352700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc1973d0 con 0x7f66bc100cd0 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66c2352700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc197510 con 0x7f66bc101ed0 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc100cd0 0x7f66bc196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc100cd0 0x7f66bc196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41598/0 (socket says 172.21.15.49:41598) 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66bbfff700 1 -- 172.21.15.49:0/4287590581 learned_addr learned my addr 172.21.15.49:0/4287590581 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66bb7fe700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc101ed0 0x7f66bc196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66bbfff700 1 -- 172.21.15.49:0/4287590581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc101ed0 msgr2=0x7f66bc196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66bbfff700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc101ed0 0x7f66bc196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66bbfff700 1 -- 172.21.15.49:0/4287590581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b00095f0 con 0x7f66bc100cd0 2024-05-15T15:56:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66bb7fe700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc101ed0 0x7f66bc196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:56:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.062+0000 7f66bbfff700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc100cd0 0x7f66bc196870 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00b480 tx=0x7f66ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.063+0000 7f66b97fa700 1 -- 172.21.15.49:0/4287590581 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66ac0118c0 con 0x7f66bc100cd0 2024-05-15T15:56:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.063+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc19beb0 con 0x7f66bc100cd0 2024-05-15T15:56:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.063+0000 7f66b97fa700 1 -- 172.21.15.49:0/4287590581 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66ac010380 con 0x7f66bc100cd0 2024-05-15T15:56:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.063+0000 7f66b97fa700 1 -- 172.21.15.49:0/4287590581 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66ac00f4f0 con 0x7f66bc100cd0 2024-05-15T15:56:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.063+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc19c3f0 con 0x7f66bc100cd0 2024-05-15T15:56:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.064+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66bc1908f0 con 0x7f66bc100cd0 2024-05-15T15:56:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.064+0000 7f66b97fa700 1 -- 172.21.15.49:0/4287590581 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f66ac0104f0 con 0x7f66bc100cd0 2024-05-15T15:56:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.065+0000 7f66b97fa700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f66a406de80 0x7f66a4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:56:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.065+0000 7f66b97fa700 1 -- 172.21.15.49:0/4287590581 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f66ac08d7f0 con 0x7f66bc100cd0 2024-05-15T15:56:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.065+0000 7f66bb7fe700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f66a406de80 0x7f66a4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:56:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.065+0000 7f66bb7fe700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f66a406de80 0x7f66a4070340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f66b00059d0 tx=0x7f66b0005960 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:56:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.070+0000 7f66b97fa700 1 -- 172.21.15.49:0/4287590581 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f66ac05ab30 con 0x7f66bc100cd0 2024-05-15T15:56:59.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:59 smithi049 ceph-mon[28399]: pgmap v68: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:59.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:59 smithi049 ceph-mon[28399]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:59.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:56:59 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/718219213' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:56:59.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.311+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66bc02cf00 con 0x7f66a406de80 2024-05-15T15:56:59.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.312+0000 7f66b97fa700 1 -- 172.21.15.49:0/4287590581 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f66bc02cf00 con 0x7f66a406de80 2024-05-15T15:56:59.313 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr" 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "2/2 daemons upgraded", 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading grafana daemons", 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:56:59.314 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:56:59.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f66a406de80 msgr2=0x7f66a4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:59.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f66a406de80 0x7f66a4070340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f66b00059d0 tx=0x7f66b0005960 comp rx=0 tx=0).stop 2024-05-15T15:56:59.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc100cd0 msgr2=0x7f66bc196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:56:59.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc100cd0 0x7f66bc196870 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00b480 tx=0x7f66ac00b840 comp rx=0 tx=0).stop 2024-05-15T15:56:59.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 shutdown_connections 2024-05-15T15:56:59.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f66a406de80 0x7f66a4070340 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:59.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66bc100cd0 0x7f66bc196870 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:59.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 --2- 172.21.15.49:0/4287590581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc101ed0 0x7f66bc196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:56:59.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 >> 172.21.15.49:0/4287590581 conn(0x7f66bc0fc2b0 msgr2=0x7f66bc105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:56:59.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 shutdown_connections 2024-05-15T15:56:59.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:56:59.314+0000 7f66c2352700 1 -- 172.21.15.49:0/4287590581 wait complete. 2024-05-15T15:56:59.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:59 smithi144 ceph-mon[36389]: pgmap v68: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:56:59.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:59 smithi144 ceph-mon[36389]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:56:59.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:56:59 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/718219213' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:00.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:00 smithi144 ceph-mon[36389]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:00.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:00 smithi144 ceph-mon[36389]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:00 smithi049 ceph-mon[28399]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:00.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:00 smithi049 ceph-mon[28399]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:01.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:01 smithi144 ceph-mon[36389]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:01.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:01 smithi144 ceph-mon[36389]: pgmap v69: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:01.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:01 smithi049 ceph-mon[28399]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:01.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:01 smithi049 ceph-mon[28399]: pgmap v69: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:02.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:02 smithi144 ceph-mon[36389]: pgmap v70: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:02.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:02 smithi049 ceph-mon[28399]: pgmap v70: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:03 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:57:03.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:03 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:57:04.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:04 smithi144 ceph-mon[36389]: pgmap v71: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:04.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:04 smithi049 ceph-mon[28399]: pgmap v71: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:06.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:06 smithi144 ceph-mon[36389]: pgmap v72: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:06 smithi049 ceph-mon[28399]: pgmap v72: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:08.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:08 smithi049 ceph-mon[28399]: pgmap v73: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:08 smithi144 ceph-mon[36389]: pgmap v73: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:10.655 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:10 smithi049 ceph-mon[28399]: pgmap v74: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:10.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:10 smithi144 ceph-mon[36389]: pgmap v74: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:12.777 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:12 smithi049 ceph-mon[28399]: pgmap v75: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:12.777 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:12 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:12.777 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:12 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:12.777 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:12 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:12.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:12 smithi144 ceph-mon[36389]: pgmap v75: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:12.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:12 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:12.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:12 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:12.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:12 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:14.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:14 smithi144 ceph-mon[36389]: pgmap v76: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:14.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:14 smithi049 ceph-mon[28399]: pgmap v76: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:16.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:16 smithi144 ceph-mon[36389]: pgmap v77: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:16.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:16 smithi049 ceph-mon[28399]: pgmap v77: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:17 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:17 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:17 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:57:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:17 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:17 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:17 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:57:19.361 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:19 smithi049 ceph-mon[28399]: pgmap v78: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:19.361 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:19 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:19.361 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:19 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:19.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:19 smithi144 ceph-mon[36389]: pgmap v78: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:19.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:19 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:19.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:19 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:21.161 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:21 smithi049 ceph-mon[28399]: pgmap v79: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:21 smithi144 ceph-mon[36389]: pgmap v79: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:22.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: pgmap v80: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:22.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:57:22.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:57:22.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: Upgrade: Finalizing container_image settings 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-15T15:57:22.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-15T15:57:22.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: Upgrade: Complete! 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-15T15:57:22.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-15T15:57:22.955 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:22.955 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:57:22.955 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:57:22.955 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.955 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:22.955 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:57:22.955 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:57:22.955 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:22 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: pgmap v80: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:57:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:57:22.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: Upgrade: Finalizing container_image settings 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-15T15:57:22.958 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.959 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: Upgrade: Complete! 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:57:22.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:57:22.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:22 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:24.137 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:23 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:24.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:23 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:25.100 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:24 smithi144 ceph-mon[36389]: pgmap v81: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:25.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:24 smithi049 ceph-mon[28399]: pgmap v81: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:27.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:26 smithi144 ceph-mon[36389]: pgmap v82: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:27.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:26 smithi049 ceph-mon[28399]: pgmap v82: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:29.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:28 smithi144 ceph-mon[36389]: pgmap v83: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:29.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:28 smithi049 ceph-mon[28399]: pgmap v83: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:29.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.438+0000 7fd9352c3700 1 -- 172.21.15.49:0/868631975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 msgr2=0x7fd930101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:29.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.438+0000 7fd9352c3700 1 --2- 172.21.15.49:0/868631975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 0x7fd930101240 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd924009910 tx=0x7fd924009c20 comp rx=0 tx=0).stop 2024-05-15T15:57:29.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.438+0000 7fd9352c3700 1 -- 172.21.15.49:0/868631975 shutdown_connections 2024-05-15T15:57:29.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.438+0000 7fd9352c3700 1 --2- 172.21.15.49:0/868631975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 0x7fd930101240 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:29.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.438+0000 7fd9352c3700 1 --2- 172.21.15.49:0/868631975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9300ffbe0 0x7fd930100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:29.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.438+0000 7fd9352c3700 1 -- 172.21.15.49:0/868631975 >> 172.21.15.49:0/868631975 conn(0x7fd9300fb180 msgr2=0x7fd9300fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:29.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.438+0000 7fd9352c3700 1 -- 172.21.15.49:0/868631975 shutdown_connections 2024-05-15T15:57:29.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.438+0000 7fd9352c3700 1 -- 172.21.15.49:0/868631975 wait complete. 2024-05-15T15:57:29.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.439+0000 7fd9352c3700 1 Processor -- start 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.439+0000 7fd9352c3700 1 -- start start 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.439+0000 7fd9352c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9300ffbe0 0x7fd930103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.439+0000 7fd9352c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 0x7fd9301021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.439+0000 7fd9352c3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd930102700 con 0x7fd930100de0 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.439+0000 7fd9352c3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd930102840 con 0x7fd9300ffbe0 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.439+0000 7fd92e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 0x7fd9301021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.439+0000 7fd92e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 0x7fd9301021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43952/0 (socket says 172.21.15.49:43952) 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.439+0000 7fd92e7fc700 1 -- 172.21.15.49:0/3780233804 learned_addr learned my addr 172.21.15.49:0/3780233804 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd92effd700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9300ffbe0 0x7fd930103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd92e7fc700 1 -- 172.21.15.49:0/3780233804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9300ffbe0 msgr2=0x7fd930103b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd92e7fc700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9300ffbe0 0x7fd930103b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:29.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd92e7fc700 1 -- 172.21.15.49:0/3780233804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9240095f0 con 0x7fd930100de0 2024-05-15T15:57:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd92effd700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9300ffbe0 0x7fd930103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:57:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd92e7fc700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 0x7fd9301021c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd924006010 tx=0x7fd92400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd917fff700 1 -- 172.21.15.49:0/3780233804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd92401d070 con 0x7fd930100de0 2024-05-15T15:57:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd917fff700 1 -- 172.21.15.49:0/3780233804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd92400f460 con 0x7fd930100de0 2024-05-15T15:57:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd930102ac0 con 0x7fd930100de0 2024-05-15T15:57:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd917fff700 1 -- 172.21.15.49:0/3780233804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd924021640 con 0x7fd930100de0 2024-05-15T15:57:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.440+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd930103030 con 0x7fd930100de0 2024-05-15T15:57:29.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.441+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93004ec60 con 0x7fd930100de0 2024-05-15T15:57:29.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.442+0000 7fd917fff700 1 -- 172.21.15.49:0/3780233804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7fd92400fab0 con 0x7fd930100de0 2024-05-15T15:57:29.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.442+0000 7fd917fff700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd91806de80 0x7fd918070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:29.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.442+0000 7fd917fff700 1 -- 172.21.15.49:0/3780233804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd92408ed20 con 0x7fd930100de0 2024-05-15T15:57:29.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.442+0000 7fd92effd700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd91806de80 0x7fd918070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:29.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.443+0000 7fd92effd700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd91806de80 0x7fd918070340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd930100c40 tx=0x7fd920008040 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:29.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.448+0000 7fd917fff700 1 -- 172.21.15.49:0/3780233804 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fd92405c0e0 con 0x7fd930100de0 2024-05-15T15:57:29.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.674+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd930103030 con 0x7fd91806de80 2024-05-15T15:57:29.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.675+0000 7fd917fff700 1 -- 172.21.15.49:0/3780233804 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fd930103030 con 0x7fd91806de80 2024-05-15T15:57:29.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.677+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd91806de80 msgr2=0x7fd918070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:29.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.677+0000 7fd9352c3700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd91806de80 0x7fd918070340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd930100c40 tx=0x7fd920008040 comp rx=0 tx=0).stop 2024-05-15T15:57:29.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.677+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 msgr2=0x7fd9301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:29.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.677+0000 7fd9352c3700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 0x7fd9301021c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd924006010 tx=0x7fd92400be20 comp rx=0 tx=0).stop 2024-05-15T15:57:29.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.677+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 shutdown_connections 2024-05-15T15:57:29.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.678+0000 7fd9352c3700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fd91806de80 0x7fd918070340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:29.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.678+0000 7fd9352c3700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9300ffbe0 0x7fd930103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:29.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.678+0000 7fd9352c3700 1 --2- 172.21.15.49:0/3780233804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd930100de0 0x7fd9301021c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:29.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.678+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 >> 172.21.15.49:0/3780233804 conn(0x7fd9300fb180 msgr2=0x7fd930069220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:29.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.678+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 shutdown_connections 2024-05-15T15:57:29.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:29.678+0000 7fd9352c3700 1 -- 172.21.15.49:0/3780233804 wait complete. 2024-05-15T15:57:30.321 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-05-15T15:57:30.541 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:57:30.927 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:30 smithi049 ceph-mon[28399]: pgmap v84: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:30.928 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:30 smithi049 ceph-mon[28399]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:31.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:30 smithi144 ceph-mon[36389]: pgmap v84: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:31.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:30 smithi144 ceph-mon[36389]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.285+0000 7fb4a5de0700 1 -- 172.21.15.49:0/44930385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 msgr2=0x7fb4a01001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.285+0000 7fb4a5de0700 1 --2- 172.21.15.49:0/44930385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 0x7fb4a01001a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb490009910 tx=0x7fb490009c20 comp rx=0 tx=0).stop 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.285+0000 7fb4a5de0700 1 -- 172.21.15.49:0/44930385 shutdown_connections 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.285+0000 7fb4a5de0700 1 --2- 172.21.15.49:0/44930385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a0100770 0x7fb4a01047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.285+0000 7fb4a5de0700 1 --2- 172.21.15.49:0/44930385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 0x7fb4a01001a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.285+0000 7fb4a5de0700 1 -- 172.21.15.49:0/44930385 >> 172.21.15.49:0/44930385 conn(0x7fb4a0076800 msgr2=0x7fb4a00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.286+0000 7fb4a5de0700 1 -- 172.21.15.49:0/44930385 shutdown_connections 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.286+0000 7fb4a5de0700 1 -- 172.21.15.49:0/44930385 wait complete. 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.286+0000 7fb4a5de0700 1 Processor -- start 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.286+0000 7fb4a5de0700 1 -- start start 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb4a5de0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 0x7fb4a00fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb4a5de0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a0100770 0x7fb4a00fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb4a5de0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00fc760 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb4a5de0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00fc8a0 con 0x7fb4a0100770 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb49f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 0x7fb4a00fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb49f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 0x7fb4a00fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43956/0 (socket says 172.21.15.49:43956) 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb49f7fe700 1 -- 172.21.15.49:0/502133197 learned_addr learned my addr 172.21.15.49:0/502133197 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb49effd700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a0100770 0x7fb4a00fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb49f7fe700 1 -- 172.21.15.49:0/502133197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a0100770 msgr2=0x7fb4a00fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb49f7fe700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a0100770 0x7fb4a00fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb49f7fe700 1 -- 172.21.15.49:0/502133197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900095f0 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb49f7fe700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 0x7fb4a00fbce0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb49000bd70 tx=0x7fb49000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.287+0000 7fb49cff9700 1 -- 172.21.15.49:0/502133197 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49001d070 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.288+0000 7fb49cff9700 1 -- 172.21.15.49:0/502133197 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb49000f460 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.288+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a00fcb20 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.288+0000 7fb49cff9700 1 -- 172.21.15.49:0/502133197 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb490017630 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.288+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a01a3000 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.288+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a01012a0 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.289+0000 7fb49cff9700 1 -- 172.21.15.49:0/502133197 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7fb49000f5d0 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.290+0000 7fb49cff9700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb48806de80 0x7fb488070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.290+0000 7fb49cff9700 1 -- 172.21.15.49:0/502133197 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb49008f000 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.290+0000 7fb49effd700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb48806de80 0x7fb488070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.290+0000 7fb49effd700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb48806de80 0x7fb488070340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00fd410 tx=0x7fb494005eb0 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:32.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.295+0000 7fb49cff9700 1 -- 172.21.15.49:0/502133197 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fb490058890 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.588+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb4a010ce60 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.588+0000 7fb49cff9700 1 -- 172.21.15.49:0/502133197 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7fb49005bf50 con 0x7fb4a00ffdc0 2024-05-15T15:57:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.590+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb48806de80 msgr2=0x7fb488070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.590+0000 7fb4a5de0700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb48806de80 0x7fb488070340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00fd410 tx=0x7fb494005eb0 comp rx=0 tx=0).stop 2024-05-15T15:57:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.590+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 msgr2=0x7fb4a00fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.590+0000 7fb4a5de0700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 0x7fb4a00fbce0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb49000bd70 tx=0x7fb49000be50 comp rx=0 tx=0).stop 2024-05-15T15:57:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.590+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 shutdown_connections 2024-05-15T15:57:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.590+0000 7fb4a5de0700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb48806de80 0x7fb488070340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.590+0000 7fb4a5de0700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a00ffdc0 0x7fb4a00fbce0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.590+0000 7fb4a5de0700 1 --2- 172.21.15.49:0/502133197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a0100770 0x7fb4a00fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.590+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 >> 172.21.15.49:0/502133197 conn(0x7fb4a0076800 msgr2=0x7fb4a00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.591+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 shutdown_connections 2024-05-15T15:57:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:32.591+0000 7fb4a5de0700 1 -- 172.21.15.49:0/502133197 wait complete. 2024-05-15T15:57:32.604 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:57:32.915 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:32 smithi049 ceph-mon[28399]: pgmap v85: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:32.915 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:32 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/502133197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:32.915 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:32 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:57:33.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:32 smithi144 ceph-mon[36389]: pgmap v85: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:33.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:32 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/502133197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:32 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:57:33.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-05-15T15:57:33.438 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:57:34.941 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:34 smithi049 ceph-mon[28399]: pgmap v86: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:35.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:34 smithi144 ceph-mon[36389]: pgmap v86: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.212+0000 7f90594b4700 1 -- 172.21.15.49:0/42470854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 msgr2=0x7f90541047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.212+0000 7f90594b4700 1 --2- 172.21.15.49:0/42470854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 0x7f90541047e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9048009910 tx=0x7f9048009c20 comp rx=0 tx=0).stop 2024-05-15T15:57:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.212+0000 7f90594b4700 1 -- 172.21.15.49:0/42470854 shutdown_connections 2024-05-15T15:57:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.212+0000 7f90594b4700 1 --2- 172.21.15.49:0/42470854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 0x7f90541047e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.212+0000 7f90594b4700 1 --2- 172.21.15.49:0/42470854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90540ffde0 0x7f90541001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.212+0000 7f90594b4700 1 -- 172.21.15.49:0/42470854 >> 172.21.15.49:0/42470854 conn(0x7f90540fb690 msgr2=0x7f90540fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.212+0000 7f90594b4700 1 -- 172.21.15.49:0/42470854 shutdown_connections 2024-05-15T15:57:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.212+0000 7f90594b4700 1 -- 172.21.15.49:0/42470854 wait complete. 2024-05-15T15:57:35.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f90594b4700 1 Processor -- start 2024-05-15T15:57:35.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f90594b4700 1 -- start start 2024-05-15T15:57:35.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f90594b4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90540ffde0 0x7f905419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:35.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f90594b4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 0x7f905419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:35.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f90594b4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905419d460 con 0x7f9054100790 2024-05-15T15:57:35.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f90594b4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9054196a40 con 0x7f90540ffde0 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f90527fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 0x7f905419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f90527fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 0x7f905419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42590/0 (socket says 172.21.15.49:42590) 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f90527fc700 1 -- 172.21.15.49:0/1116949886 learned_addr learned my addr 172.21.15.49:0/1116949886 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.213+0000 7f9052ffd700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90540ffde0 0x7f905419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.214+0000 7f90527fc700 1 -- 172.21.15.49:0/1116949886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90540ffde0 msgr2=0x7f905419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.214+0000 7f90527fc700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90540ffde0 0x7f905419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.214+0000 7f90527fc700 1 -- 172.21.15.49:0/1116949886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480095f0 con 0x7f9054100790 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.214+0000 7f90527fc700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 0x7f905419ce40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9048009fd0 tx=0x7f904800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.214+0000 7f903bfff700 1 -- 172.21.15.49:0/1116949886 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f904801d070 con 0x7f9054100790 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.214+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9054196cc0 con 0x7f9054100790 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.214+0000 7f903bfff700 1 -- 172.21.15.49:0/1116949886 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90480052a0 con 0x7f9054100790 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.214+0000 7f903bfff700 1 -- 172.21.15.49:0/1116949886 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9048022470 con 0x7f9054100790 2024-05-15T15:57:35.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.214+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90541971b0 con 0x7f9054100790 2024-05-15T15:57:35.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.215+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90541091f0 con 0x7f9054100790 2024-05-15T15:57:35.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.216+0000 7f903bfff700 1 -- 172.21.15.49:0/1116949886 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f904800f6f0 con 0x7f9054100790 2024-05-15T15:57:35.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.216+0000 7f903bfff700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f903c06de80 0x7f903c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:35.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.216+0000 7f903bfff700 1 -- 172.21.15.49:0/1116949886 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f904808ed80 con 0x7f9054100790 2024-05-15T15:57:35.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.216+0000 7f9052ffd700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f903c06de80 0x7f903c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:35.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.217+0000 7f9052ffd700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f903c06de80 0x7f903c070340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9044009c60 tx=0x7f9044009260 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:35.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.221+0000 7f903bfff700 1 -- 172.21.15.49:0/1116949886 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f904805c140 con 0x7f9054100790 2024-05-15T15:57:35.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.514+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f90541012c0 con 0x7f9054100790 2024-05-15T15:57:35.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.515+0000 7f903bfff700 1 -- 172.21.15.49:0/1116949886 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f904805bcd0 con 0x7f9054100790 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f903c06de80 msgr2=0x7f903c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f903c06de80 0x7f903c070340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9044009c60 tx=0x7f9044009260 comp rx=0 tx=0).stop 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 msgr2=0x7f905419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 0x7f905419ce40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9048009fd0 tx=0x7f904800b5a0 comp rx=0 tx=0).stop 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 shutdown_connections 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f903c06de80 0x7f903c070340 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90540ffde0 0x7f905419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 --2- 172.21.15.49:0/1116949886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9054100790 0x7f905419ce40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 >> 172.21.15.49:0/1116949886 conn(0x7f90540fb690 msgr2=0x7f9054069820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 shutdown_connections 2024-05-15T15:57:35.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:35.518+0000 7f90594b4700 1 -- 172.21.15.49:0/1116949886 wait complete. 2024-05-15T15:57:35.530 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)" 2024-05-15T15:57:36.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:35 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1116949886' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:36.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:35 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1116949886' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:37.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:36 smithi144 ceph-mon[36389]: pgmap v87: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:37.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:36 smithi049 ceph-mon[28399]: pgmap v87: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:38.588 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-05-15T15:57:38.809 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:57:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:38 smithi144 ceph-mon[36389]: pgmap v88: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:39.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:38 smithi049 ceph-mon[28399]: pgmap v88: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:40.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.428+0000 7f7bc90a2700 1 -- 172.21.15.49:0/1425910667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 msgr2=0x7f7bc41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.428+0000 7f7bc90a2700 1 --2- 172.21.15.49:0/1425910667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 0x7f7bc41012b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009910 tx=0x7f7bac009c20 comp rx=0 tx=0).stop 2024-05-15T15:57:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.428+0000 7f7bc90a2700 1 -- 172.21.15.49:0/1425910667 shutdown_connections 2024-05-15T15:57:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.428+0000 7f7bc90a2700 1 --2- 172.21.15.49:0/1425910667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4101880 0x7f7bc41058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.428+0000 7f7bc90a2700 1 --2- 172.21.15.49:0/1425910667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 0x7f7bc41012b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.428+0000 7f7bc90a2700 1 -- 172.21.15.49:0/1425910667 >> 172.21.15.49:0/1425910667 conn(0x7f7bc40fc7a0 msgr2=0x7f7bc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.428+0000 7f7bc90a2700 1 -- 172.21.15.49:0/1425910667 shutdown_connections 2024-05-15T15:57:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.428+0000 7f7bc90a2700 1 -- 172.21.15.49:0/1425910667 wait complete. 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc90a2700 1 Processor -- start 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc90a2700 1 -- start start 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc90a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 0x7f7bc40732c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc90a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4101880 0x7f7bc4073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc90a2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40752a0 con 0x7f7bc4100ed0 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc90a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc4073d40 con 0x7f7bc4101880 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 0x7f7bc40732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 0x7f7bc40732c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42608/0 (socket says 172.21.15.49:42608) 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc2d9d700 1 -- 172.21.15.49:0/963488062 learned_addr learned my addr 172.21.15.49:0/963488062 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc259c700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4101880 0x7f7bc4073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc2d9d700 1 -- 172.21.15.49:0/963488062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4101880 msgr2=0x7f7bc4073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc2d9d700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4101880 0x7f7bc4073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.429+0000 7f7bc2d9d700 1 -- 172.21.15.49:0/963488062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac0095f0 con 0x7f7bc4100ed0 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.430+0000 7f7bc259c700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4101880 0x7f7bc4073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.430+0000 7f7bc2d9d700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 0x7f7bc40732c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7bac006010 tx=0x7f7bac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.430+0000 7f7bbbfff700 1 -- 172.21.15.49:0/963488062 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bac01d070 con 0x7f7bc4100ed0 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.430+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc4073fc0 con 0x7f7bc4100ed0 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.430+0000 7f7bbbfff700 1 -- 172.21.15.49:0/963488062 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bac0056e0 con 0x7f7bc4100ed0 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.430+0000 7f7bbbfff700 1 -- 172.21.15.49:0/963488062 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bac017400 con 0x7f7bc4100ed0 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.430+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc41a3000 con 0x7f7bc4100ed0 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.431+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc4109270 con 0x7f7bc4100ed0 2024-05-15T15:57:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.431+0000 7f7bbbfff700 1 -- 172.21.15.49:0/963488062 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f7bac00f6f0 con 0x7f7bc4100ed0 2024-05-15T15:57:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.432+0000 7f7bbbfff700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7bb006de80 0x7f7bb0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.432+0000 7f7bbbfff700 1 -- 172.21.15.49:0/963488062 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7bac08ef40 con 0x7f7bc4100ed0 2024-05-15T15:57:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.432+0000 7f7bc259c700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7bb006de80 0x7f7bb0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.432+0000 7f7bc259c700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7bb006de80 0x7f7bb0070340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4005d90 tx=0x7f7bb4005d00 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.437+0000 7f7bbbfff700 1 -- 172.21.15.49:0/963488062 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f7bac0c3830 con 0x7f7bc4100ed0 2024-05-15T15:57:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.731+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7bc40746e0 con 0x7f7bc4100ed0 2024-05-15T15:57:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.732+0000 7f7bbbfff700 1 -- 172.21.15.49:0/963488062 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f7bac058750 con 0x7f7bc4100ed0 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7bb006de80 msgr2=0x7f7bb0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7bb006de80 0x7f7bb0070340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4005d90 tx=0x7f7bb4005d00 comp rx=0 tx=0).stop 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 msgr2=0x7f7bc40732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 0x7f7bc40732c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7bac006010 tx=0x7f7bac00be20 comp rx=0 tx=0).stop 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 shutdown_connections 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f7bb006de80 0x7f7bb0070340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bc4100ed0 0x7f7bc40732c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 --2- 172.21.15.49:0/963488062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4101880 0x7f7bc4073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 >> 172.21.15.49:0/963488062 conn(0x7f7bc40fc7a0 msgr2=0x7f7bc40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 shutdown_connections 2024-05-15T15:57:40.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:40.734+0000 7f7bc90a2700 1 -- 172.21.15.49:0/963488062 wait complete. 2024-05-15T15:57:40.747 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:57:41.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:40 smithi049 ceph-mon[28399]: pgmap v89: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:41.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:40 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/963488062' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:40 smithi144 ceph-mon[36389]: pgmap v89: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:41.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:40 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/963488062' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:57:41.395 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-05-15T15:57:41.613 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:57:42.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:42 smithi049 ceph-mon[28399]: pgmap v90: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.189+0000 7f975b7d4700 1 -- 172.21.15.49:0/1776101553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 msgr2=0x7f97540ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.189+0000 7f975b7d4700 1 --2- 172.21.15.49:0/1776101553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 0x7f97540ff8f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9750009910 tx=0x7f9750009c20 comp rx=0 tx=0).stop 2024-05-15T15:57:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.189+0000 7f975b7d4700 1 -- 172.21.15.49:0/1776101553 shutdown_connections 2024-05-15T15:57:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.189+0000 7f975b7d4700 1 --2- 172.21.15.49:0/1776101553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 0x7f97540ff8f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.189+0000 7f975b7d4700 1 --2- 172.21.15.49:0/1776101553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754105c90 0x7f9754106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.189+0000 7f975b7d4700 1 -- 172.21.15.49:0/1776101553 >> 172.21.15.49:0/1776101553 conn(0x7f9754074e70 msgr2=0x7f9754075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.189+0000 7f975b7d4700 1 -- 172.21.15.49:0/1776101553 shutdown_connections 2024-05-15T15:57:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.189+0000 7f975b7d4700 1 -- 172.21.15.49:0/1776101553 wait complete. 2024-05-15T15:57:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f975b7d4700 1 Processor -- start 2024-05-15T15:57:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f975b7d4700 1 -- start start 2024-05-15T15:57:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f975b7d4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 0x7f9754100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f975b7d4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754105c90 0x7f9754100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f975b7d4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97541010f0 con 0x7f97540ff490 2024-05-15T15:57:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f975b7d4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9754101230 con 0x7f9754105c90 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f9759570700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 0x7f9754100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f9759570700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 0x7f9754100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42624/0 (socket says 172.21.15.49:42624) 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f9759570700 1 -- 172.21.15.49:0/3638810633 learned_addr learned my addr 172.21.15.49:0/3638810633 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f9759570700 1 -- 172.21.15.49:0/3638810633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754105c90 msgr2=0x7f9754100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f9759570700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754105c90 0x7f9754100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.190+0000 7f9759570700 1 -- 172.21.15.49:0/3638810633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97500095f0 con 0x7f97540ff490 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.191+0000 7f9759570700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 0x7f9754100670 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f974800b480 tx=0x7f974800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.191+0000 7f97467fc700 1 -- 172.21.15.49:0/3638810633 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97480118c0 con 0x7f97540ff490 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.191+0000 7f97467fc700 1 -- 172.21.15.49:0/3638810633 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9748010380 con 0x7f97540ff490 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.191+0000 7f97467fc700 1 -- 172.21.15.49:0/3638810633 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974800f4f0 con 0x7f97540ff490 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.191+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97541a2ea0 con 0x7f97540ff490 2024-05-15T15:57:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.191+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97541a3200 con 0x7f97540ff490 2024-05-15T15:57:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.192+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97541092d0 con 0x7f97540ff490 2024-05-15T15:57:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.192+0000 7f97467fc700 1 -- 172.21.15.49:0/3638810633 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f97480104f0 con 0x7f97540ff490 2024-05-15T15:57:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.193+0000 7f97467fc700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f974006de80 0x7f9740070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.193+0000 7f97467fc700 1 -- 172.21.15.49:0/3638810633 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f974808d710 con 0x7f97540ff490 2024-05-15T15:57:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.193+0000 7f9758d6f700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f974006de80 0x7f9740070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.193+0000 7f9758d6f700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f974006de80 0x7f9740070340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f975000b5c0 tx=0x7f9750005960 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:42 smithi144 ceph-mon[36389]: pgmap v90: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:43.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.198+0000 7f97467fc700 1 -- 172.21.15.49:0/3638810633 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f9748056fa0 con 0x7f97540ff490 2024-05-15T15:57:43.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:43.425+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8", "target": ["mon-mgr", ""]}) v1 -- 0x7f975410df20 con 0x7f974006de80 2024-05-15T15:57:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:44 smithi144 ceph-mon[36389]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:44 smithi144 ceph-mon[36389]: pgmap v91: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:45.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:44 smithi049 ceph-mon[28399]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:45.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:44 smithi049 ceph-mon[28399]: pgmap v91: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.696+0000 7f97467fc700 1 -- 172.21.15.49:0/3638810633 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5756 (secure 0 0 0) 0x7f975410df20 con 0x7f974006de80 2024-05-15T15:57:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f974006de80 msgr2=0x7f9740070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f974006de80 0x7f9740070340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f975000b5c0 tx=0x7f9750005960 comp rx=0 tx=0).stop 2024-05-15T15:57:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 msgr2=0x7f9754100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 0x7f9754100670 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f974800b480 tx=0x7f974800b840 comp rx=0 tx=0).stop 2024-05-15T15:57:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 shutdown_connections 2024-05-15T15:57:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f974006de80 0x7f9740070340 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97540ff490 0x7f9754100670 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 --2- 172.21.15.49:0/3638810633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754105c90 0x7f9754100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 >> 172.21.15.49:0/3638810633 conn(0x7f9754074e70 msgr2=0x7f97540fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 shutdown_connections 2024-05-15T15:57:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:45.698+0000 7f975b7d4700 1 -- 172.21.15.49:0/3638810633 wait complete. 2024-05-15T15:57:45.712 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:57:46.362 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph orch ps' 2024-05-15T15:57:46.583 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:57:47.041 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:46 smithi049 ceph-mon[28399]: pgmap v92: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:47.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:46 smithi144 ceph-mon[36389]: pgmap v92: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:47.940 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:47 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:57:48.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.144+0000 7f76d5f75700 1 -- 172.21.15.49:0/2855756649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d01007c0 msgr2=0x7f76d0104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:48.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.144+0000 7f76d5f75700 1 --2- 172.21.15.49:0/2855756649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d01007c0 0x7f76d0104810 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009910 tx=0x7f76c4009c20 comp rx=0 tx=0).stop 2024-05-15T15:57:48.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.144+0000 7f76d5f75700 1 -- 172.21.15.49:0/2855756649 shutdown_connections 2024-05-15T15:57:48.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.144+0000 7f76d5f75700 1 --2- 172.21.15.49:0/2855756649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d01007c0 0x7f76d0104810 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:48.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.144+0000 7f76d5f75700 1 --2- 172.21.15.49:0/2855756649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d00ffe10 0x7f76d01001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:48.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.144+0000 7f76d5f75700 1 -- 172.21.15.49:0/2855756649 >> 172.21.15.49:0/2855756649 conn(0x7f76d00fb700 msgr2=0x7f76d00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:48.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.144+0000 7f76d5f75700 1 -- 172.21.15.49:0/2855756649 shutdown_connections 2024-05-15T15:57:48.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.144+0000 7f76d5f75700 1 -- 172.21.15.49:0/2855756649 wait complete. 2024-05-15T15:57:48.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76d5f75700 1 Processor -- start 2024-05-15T15:57:48.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76d5f75700 1 -- start start 2024-05-15T15:57:48.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76d5f75700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d00ffe10 0x7f76d01018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:48.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76d5f75700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d01007c0 0x7f76d0101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:48.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76d5f75700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0102330 con 0x7f76d00ffe10 2024-05-15T15:57:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76d5f75700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0102470 con 0x7f76d01007c0 2024-05-15T15:57:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76cf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d00ffe10 0x7f76d01018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76cf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d00ffe10 0x7f76d01018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46958/0 (socket says 172.21.15.49:46958) 2024-05-15T15:57:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76cf7fe700 1 -- 172.21.15.49:0/2935701993 learned_addr learned my addr 172.21.15.49:0/2935701993 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.145+0000 7f76ceffd700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d01007c0 0x7f76d0101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.146+0000 7f76cf7fe700 1 -- 172.21.15.49:0/2935701993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d01007c0 msgr2=0x7f76d0101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.146+0000 7f76cf7fe700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d01007c0 0x7f76d0101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.146+0000 7f76cf7fe700 1 -- 172.21.15.49:0/2935701993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40095f0 con 0x7f76d00ffe10 2024-05-15T15:57:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.146+0000 7f76cf7fe700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d00ffe10 0x7f76d01018b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f76c000d8d0 tx=0x7f76c000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.146+0000 7f76ccff9700 1 -- 172.21.15.49:0/2935701993 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c00098a0 con 0x7f76d00ffe10 2024-05-15T15:57:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.146+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d01a2e30 con 0x7f76d00ffe10 2024-05-15T15:57:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.146+0000 7f76ccff9700 1 -- 172.21.15.49:0/2935701993 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76c0010460 con 0x7f76d00ffe10 2024-05-15T15:57:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.146+0000 7f76ccff9700 1 -- 172.21.15.49:0/2935701993 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c000f5f0 con 0x7f76d00ffe10 2024-05-15T15:57:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.146+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d01a3190 con 0x7f76d00ffe10 2024-05-15T15:57:48.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.147+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d01012f0 con 0x7f76d00ffe10 2024-05-15T15:57:48.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.148+0000 7f76ccff9700 1 -- 172.21.15.49:0/2935701993 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f76c0009a00 con 0x7f76d00ffe10 2024-05-15T15:57:48.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.148+0000 7f76ccff9700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f76b806de80 0x7f76b8070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:48.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.148+0000 7f76ccff9700 1 -- 172.21.15.49:0/2935701993 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f76c008da70 con 0x7f76d00ffe10 2024-05-15T15:57:48.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.148+0000 7f76ceffd700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f76b806de80 0x7f76b8070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:48.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.149+0000 7f76ceffd700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f76b806de80 0x7f76b8070340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76d0102fe0 tx=0x7f76c401a040 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:48.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.153+0000 7f76ccff9700 1 -- 172.21.15.49:0/2935701993 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f76c0057280 con 0x7f76d00ffe10 2024-05-15T15:57:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:47 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:57:48.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.371+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f76d0102630 con 0x7f76b806de80 2024-05-15T15:57:48.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.382+0000 7f76ccff9700 1 -- 172.21.15.49:0/2935701993 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f76d0102630 con 0x7f76b806de80 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (63s) 31s ago 12m 17.2M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (12m) 31s ago 12m 8250k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (11m) 103s ago 11m 7880k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (12m) 31s ago 12m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (11m) 103s ago 11m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (37s) 31s ago 12m 56.7M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (7m) 31s ago 7m 89.8M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (6m) 31s ago 6m 153M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (7m) 103s ago 7m 26.3M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:57:48.385 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (6m) 103s ago 6m 28.0M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (3m) 31s ago 14m 513M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (2m) 103s ago 11m 452M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (14m) 31s ago 14m 56.0M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (11m) 103s ago 11m 44.4M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (113s) 31s ago 12m 13.6M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (107s) 103s ago 11m 5200k - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (10m) 31s ago 10m 218M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (10m) 31s ago 10m 218M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (9m) 31s ago 9m 212M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (9m) 103s ago 9m 255M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:57:48.386 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (8m) 103s ago 8m 210M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:57:48.387 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (8m) 103s ago 8m 197M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:57:48.387 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (81s) 31s ago 12m 51.5M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T15:57:48.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.384+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f76b806de80 msgr2=0x7f76b8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.384+0000 7f76d5f75700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f76b806de80 0x7f76b8070340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76d0102fe0 tx=0x7f76c401a040 comp rx=0 tx=0).stop 2024-05-15T15:57:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.384+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d00ffe10 msgr2=0x7f76d01018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.384+0000 7f76d5f75700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d00ffe10 0x7f76d01018b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f76c000d8d0 tx=0x7f76c000dbe0 comp rx=0 tx=0).stop 2024-05-15T15:57:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.385+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 shutdown_connections 2024-05-15T15:57:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.385+0000 7f76d5f75700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f76b806de80 0x7f76b8070340 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.385+0000 7f76d5f75700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76d00ffe10 0x7f76d01018b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.385+0000 7f76d5f75700 1 --2- 172.21.15.49:0/2935701993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d01007c0 0x7f76d0101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.385+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 >> 172.21.15.49:0/2935701993 conn(0x7f76d00fb700 msgr2=0x7f76d0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.385+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 shutdown_connections 2024-05-15T15:57:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:48.385+0000 7f76d5f75700 1 -- 172.21.15.49:0/2935701993 wait complete. 2024-05-15T15:57:48.987 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:48 smithi049 ceph-mon[28399]: pgmap v93: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:49.009 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-15T15:57:49.010 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:57:49.010 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-05-15T15:57:49.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:48 smithi144 ceph-mon[36389]: pgmap v93: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:49.227 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:57:50.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:49 smithi144 ceph-mon[36389]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:49 smithi049 ceph-mon[28399]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:57:50.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.835+0000 7f98b98b6700 1 -- 172.21.15.49:0/1297158975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b41027e0 msgr2=0x7f98b410acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:50.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.835+0000 7f98b98b6700 1 --2- 172.21.15.49:0/1297158975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b41027e0 0x7f98b410acd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009940 tx=0x7f98a8009c50 comp rx=0 tx=0).stop 2024-05-15T15:57:50.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.836+0000 7f98b98b6700 1 -- 172.21.15.49:0/1297158975 shutdown_connections 2024-05-15T15:57:50.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.836+0000 7f98b98b6700 1 --2- 172.21.15.49:0/1297158975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b41027e0 0x7f98b410acd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:50.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.836+0000 7f98b98b6700 1 --2- 172.21.15.49:0/1297158975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b4101ec0 0x7f98b41022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:50.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.836+0000 7f98b98b6700 1 -- 172.21.15.49:0/1297158975 >> 172.21.15.49:0/1297158975 conn(0x7f98b40fb6d0 msgr2=0x7f98b40fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:50.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.836+0000 7f98b98b6700 1 -- 172.21.15.49:0/1297158975 shutdown_connections 2024-05-15T15:57:50.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.836+0000 7f98b98b6700 1 -- 172.21.15.49:0/1297158975 wait complete. 2024-05-15T15:57:50.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.836+0000 7f98b98b6700 1 Processor -- start 2024-05-15T15:57:50.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.836+0000 7f98b98b6700 1 -- start start 2024-05-15T15:57:50.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b98b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b4101ec0 0x7f98b40ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:50.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b98b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b41027e0 0x7f98b4100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:50.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b98b6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b41005a0 con 0x7f98b4101ec0 2024-05-15T15:57:50.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b98b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b41006e0 con 0x7f98b41027e0 2024-05-15T15:57:50.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b4101ec0 0x7f98b40ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:50.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b4101ec0 0x7f98b40ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46976/0 (socket says 172.21.15.49:46976) 2024-05-15T15:57:50.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b2ffd700 1 -- 172.21.15.49:0/1523472412 learned_addr learned my addr 172.21.15.49:0/1523472412 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:50.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b2ffd700 1 -- 172.21.15.49:0/1523472412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b41027e0 msgr2=0x7f98b4100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:50.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b2ffd700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b41027e0 0x7f98b4100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:50.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b2ffd700 1 -- 172.21.15.49:0/1523472412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a80095f0 con 0x7f98b4101ec0 2024-05-15T15:57:50.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b2ffd700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b4101ec0 0x7f98b40ffb20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f98a400b4b0 tx=0x7f98a400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:50.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b88b4700 1 -- 172.21.15.49:0/1523472412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a4011800 con 0x7f98b4101ec0 2024-05-15T15:57:50.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b88b4700 1 -- 172.21.15.49:0/1523472412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98a4011e20 con 0x7f98b4101ec0 2024-05-15T15:57:50.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b41a2ec0 con 0x7f98b4101ec0 2024-05-15T15:57:50.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.837+0000 7f98b88b4700 1 -- 172.21.15.49:0/1523472412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a400f2e0 con 0x7f98b4101ec0 2024-05-15T15:57:50.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.838+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b41a3220 con 0x7f98b4101ec0 2024-05-15T15:57:50.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.838+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b4069a30 con 0x7f98b4101ec0 2024-05-15T15:57:50.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.839+0000 7f98b88b4700 1 -- 172.21.15.49:0/1523472412 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f98a4010380 con 0x7f98b4101ec0 2024-05-15T15:57:50.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.840+0000 7f98b88b4700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f989c06de90 0x7f989c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:50.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.840+0000 7f98b88b4700 1 -- 172.21.15.49:0/1523472412 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f98a408d7c0 con 0x7f98b4101ec0 2024-05-15T15:57:50.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.840+0000 7f98b27fc700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f989c06de90 0x7f989c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:50.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.840+0000 7f98b27fc700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f989c06de90 0x7f989c070350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f98a8006010 tx=0x7f98a8000bc0 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:50.845+0000 7f98b88b4700 1 -- 172.21.15.49:0/1523472412 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f98a4056fd0 con 0x7f98b4101ec0 2024-05-15T15:57:51.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.052+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f98b404ec60 con 0x7f98b4101ec0 2024-05-15T15:57:51.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.056+0000 7f98b88b4700 1 -- 172.21.15.49:0/1523472412 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v48)=0 v48) v1 ==== 125+0+0 (secure 0 0 0) 0x7f98a405a690 con 0x7f98b4101ec0 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f989c06de90 msgr2=0x7f989c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f989c06de90 0x7f989c070350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f98a8006010 tx=0x7f98a8000bc0 comp rx=0 tx=0).stop 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b4101ec0 msgr2=0x7f98b40ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b4101ec0 0x7f98b40ffb20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f98a400b4b0 tx=0x7f98a400b870 comp rx=0 tx=0).stop 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 shutdown_connections 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f989c06de90 0x7f989c070350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b4101ec0 0x7f98b40ffb20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 --2- 172.21.15.49:0/1523472412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b41027e0 0x7f98b4100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 >> 172.21.15.49:0/1523472412 conn(0x7f98b40fb6d0 msgr2=0x7f98b41031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 shutdown_connections 2024-05-15T15:57:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:51.059+0000 7f98b98b6700 1 -- 172.21.15.49:0/1523472412 wait complete. 2024-05-15T15:57:51.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:50 smithi049 ceph-mon[28399]: pgmap v94: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:50 smithi144 ceph-mon[36389]: pgmap v94: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:51.657 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-15T15:57:51.658 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:57:51.658 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-15T15:57:51.872 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:57:52.331 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:52 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/1523472412' entity='client.admin' 2024-05-15T15:57:52.331 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:52 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:52.331 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:52 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:57:52.331 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:52 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:57:52.332 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:52 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:52.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:52 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/1523472412' entity='client.admin' 2024-05-15T15:57:52.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:52 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:57:52.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:52 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:57:52.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:52 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:57:52.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:52 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:57:53.261 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:53 smithi049 ceph-mon[28399]: pgmap v95: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:53.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:53 smithi144 ceph-mon[36389]: pgmap v95: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:53.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 -- 172.21.15.49:0/2920747865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 msgr2=0x7f6b001012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 --2- 172.21.15.49:0/2920747865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 0x7f6b001012c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009910 tx=0x7f6af0009c20 comp rx=0 tx=0).stop 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 -- 172.21.15.49:0/2920747865 shutdown_connections 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 --2- 172.21.15.49:0/2920747865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b00101890 0x7f6b001058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 --2- 172.21.15.49:0/2920747865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 0x7f6b001012c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 -- 172.21.15.49:0/2920747865 >> 172.21.15.49:0/2920747865 conn(0x7f6b000fc790 msgr2=0x7f6b000febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 -- 172.21.15.49:0/2920747865 shutdown_connections 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 -- 172.21.15.49:0/2920747865 wait complete. 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 Processor -- start 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.493+0000 7f6b06660700 1 -- start start 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6b06660700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 0x7f6b000732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6b06660700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b00101890 0x7f6b000737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6b06660700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00073d20 con 0x7f6b00100ee0 2024-05-15T15:57:53.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6b06660700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00073e60 con 0x7f6b00101890 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6afffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 0x7f6b000732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6afffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 0x7f6b000732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46992/0 (socket says 172.21.15.49:46992) 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6afffff700 1 -- 172.21.15.49:0/536227916 learned_addr learned my addr 172.21.15.49:0/536227916 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6af7fff700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b00101890 0x7f6b000737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6afffff700 1 -- 172.21.15.49:0/536227916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b00101890 msgr2=0x7f6b000737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6afffff700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b00101890 0x7f6b000737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6afffff700 1 -- 172.21.15.49:0/536227916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00095f0 con 0x7f6b00100ee0 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6af7fff700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b00101890 0x7f6b000737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.494+0000 7f6afffff700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 0x7f6b000732a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6af000bd70 tx=0x7f6af000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.495+0000 7f6afdffb700 1 -- 172.21.15.49:0/536227916 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af001d070 con 0x7f6b00100ee0 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.495+0000 7f6afdffb700 1 -- 172.21.15.49:0/536227916 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6af000f460 con 0x7f6b00100ee0 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.495+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b000740e0 con 0x7f6b00100ee0 2024-05-15T15:57:53.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.495+0000 7f6afdffb700 1 -- 172.21.15.49:0/536227916 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af0017630 con 0x7f6b00100ee0 2024-05-15T15:57:53.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.495+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b001a3140 con 0x7f6b00100ee0 2024-05-15T15:57:53.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.496+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b00109280 con 0x7f6b00100ee0 2024-05-15T15:57:53.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.496+0000 7f6afdffb700 1 -- 172.21.15.49:0/536227916 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f6af000f5d0 con 0x7f6b00100ee0 2024-05-15T15:57:53.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.497+0000 7f6afdffb700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6aec06de80 0x7f6aec070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:53.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.497+0000 7f6afdffb700 1 -- 172.21.15.49:0/536227916 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6af008f000 con 0x7f6b00100ee0 2024-05-15T15:57:53.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.497+0000 7f6af7fff700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6aec06de80 0x7f6aec070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:53.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.497+0000 7f6af7fff700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6aec06de80 0x7f6aec070340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b000749d0 tx=0x7f6ae8008040 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:53.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.503+0000 7f6afdffb700 1 -- 172.21.15.49:0/536227916 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f6af0058890 con 0x7f6b00100ee0 2024-05-15T15:57:53.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.710+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f6b001023c0 con 0x7f6b00100ee0 2024-05-15T15:57:53.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.710+0000 7f6afdffb700 1 -- 172.21.15.49:0/536227916 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v48)=0 v48) v1 ==== 155+0+0 (secure 0 0 0) 0x7f6af005bf50 con 0x7f6b00100ee0 2024-05-15T15:57:53.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.712+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6aec06de80 msgr2=0x7f6aec070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:53.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.712+0000 7f6b06660700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6aec06de80 0x7f6aec070340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b000749d0 tx=0x7f6ae8008040 comp rx=0 tx=0).stop 2024-05-15T15:57:53.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.712+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 msgr2=0x7f6b000732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:53.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.712+0000 7f6b06660700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 0x7f6b000732a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6af000bd70 tx=0x7f6af000be50 comp rx=0 tx=0).stop 2024-05-15T15:57:53.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.712+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 shutdown_connections 2024-05-15T15:57:53.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.712+0000 7f6b06660700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6aec06de80 0x7f6aec070340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:53.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.712+0000 7f6b06660700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b00100ee0 0x7f6b000732a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:53.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.712+0000 7f6b06660700 1 --2- 172.21.15.49:0/536227916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b00101890 0x7f6b000737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:53.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.712+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 >> 172.21.15.49:0/536227916 conn(0x7f6b000fc790 msgr2=0x7f6b000fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:53.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.713+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 shutdown_connections 2024-05-15T15:57:53.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:53.713+0000 7f6b06660700 1 -- 172.21.15.49:0/536227916 wait complete. 2024-05-15T15:57:54.366 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-15T15:57:54.582 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:57:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:55 smithi049 ceph-mon[28399]: pgmap v96: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:55.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:55 smithi144 ceph-mon[36389]: pgmap v96: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:57:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.208+0000 7faf8156d700 1 -- 172.21.15.49:0/3551209043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 msgr2=0x7faf7c102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.208+0000 7faf8156d700 1 --2- 172.21.15.49:0/3551209043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 0x7faf7c102180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faf7400b0a0 tx=0x7faf7400b3b0 comp rx=0 tx=0).stop 2024-05-15T15:57:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.208+0000 7faf8156d700 1 -- 172.21.15.49:0/3551209043 shutdown_connections 2024-05-15T15:57:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.208+0000 7faf8156d700 1 --2- 172.21.15.49:0/3551209043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c102750 0x7faf7c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.208+0000 7faf8156d700 1 --2- 172.21.15.49:0/3551209043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 0x7faf7c102180 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.208+0000 7faf8156d700 1 -- 172.21.15.49:0/3551209043 >> 172.21.15.49:0/3551209043 conn(0x7faf7c0fb6a0 msgr2=0x7faf7c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.208+0000 7faf8156d700 1 -- 172.21.15.49:0/3551209043 shutdown_connections 2024-05-15T15:57:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.208+0000 7faf8156d700 1 -- 172.21.15.49:0/3551209043 wait complete. 2024-05-15T15:57:56.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf8156d700 1 Processor -- start 2024-05-15T15:57:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf8156d700 1 -- start start 2024-05-15T15:57:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf8156d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 0x7faf7c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf8156d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c102750 0x7faf7c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf8156d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c19d570 con 0x7faf7c101da0 2024-05-15T15:57:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf8156d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c196a90 con 0x7faf7c102750 2024-05-15T15:57:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf7affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 0x7faf7c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf7affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 0x7faf7c19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56360/0 (socket says 172.21.15.49:56360) 2024-05-15T15:57:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf7affd700 1 -- 172.21.15.49:0/1779605421 learned_addr learned my addr 172.21.15.49:0/1779605421 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf7a7fc700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c102750 0x7faf7c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf7affd700 1 -- 172.21.15.49:0/1779605421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c102750 msgr2=0x7faf7c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf7affd700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c102750 0x7faf7c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.209+0000 7faf7affd700 1 -- 172.21.15.49:0/1779605421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf74009d00 con 0x7faf7c101da0 2024-05-15T15:57:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.210+0000 7faf7affd700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 0x7faf7c19c9e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faf74000f80 tx=0x7faf74007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.210+0000 7faf73fff700 1 -- 172.21.15.49:0/1779605421 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf7400e040 con 0x7faf7c101da0 2024-05-15T15:57:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.210+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c196d10 con 0x7faf7c101da0 2024-05-15T15:57:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.210+0000 7faf73fff700 1 -- 172.21.15.49:0/1779605421 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf74003de0 con 0x7faf7c101da0 2024-05-15T15:57:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.210+0000 7faf73fff700 1 -- 172.21.15.49:0/1779605421 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf7401d720 con 0x7faf7c101da0 2024-05-15T15:57:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.210+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c197200 con 0x7faf7c101da0 2024-05-15T15:57:56.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.211+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7c0697f0 con 0x7faf7c101da0 2024-05-15T15:57:56.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.211+0000 7faf73fff700 1 -- 172.21.15.49:0/1779605421 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7faf74004690 con 0x7faf7c101da0 2024-05-15T15:57:56.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.212+0000 7faf73fff700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7faf6806de80 0x7faf68070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:56.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.212+0000 7faf73fff700 1 -- 172.21.15.49:0/1779605421 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faf7408f400 con 0x7faf7c101da0 2024-05-15T15:57:56.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.212+0000 7faf7a7fc700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7faf6806de80 0x7faf68070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:56.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.212+0000 7faf7a7fc700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7faf6806de80 0x7faf68070340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf64005ea0 tx=0x7faf64005e30 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:56.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.217+0000 7faf73fff700 1 -- 172.21.15.49:0/1779605421 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7faf74058c10 con 0x7faf7c101da0 2024-05-15T15:57:56.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.422+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7faf7c10de90 con 0x7faf7c101da0 2024-05-15T15:57:56.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.422+0000 7faf73fff700 1 -- 172.21.15.49:0/1779605421 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v48)=0 v48) v1 ==== 163+0+0 (secure 0 0 0) 0x7faf7405c2d0 con 0x7faf7c101da0 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7faf6806de80 msgr2=0x7faf68070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7faf6806de80 0x7faf68070340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf64005ea0 tx=0x7faf64005e30 comp rx=0 tx=0).stop 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 msgr2=0x7faf7c19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 0x7faf7c19c9e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faf74000f80 tx=0x7faf74007bd0 comp rx=0 tx=0).stop 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 shutdown_connections 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7faf6806de80 0x7faf68070340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf7c101da0 0x7faf7c19c9e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 --2- 172.21.15.49:0/1779605421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf7c102750 0x7faf7c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 >> 172.21.15.49:0/1779605421 conn(0x7faf7c0fb6a0 msgr2=0x7faf7c103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 shutdown_connections 2024-05-15T15:57:56.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:56.425+0000 7faf8156d700 1 -- 172.21.15.49:0/1779605421 wait complete. 2024-05-15T15:57:57.069 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-15T15:57:57.290 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:57:57.411 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:57 smithi049 ceph-mon[28399]: pgmap v97: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:57.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:57 smithi144 ceph-mon[36389]: pgmap v97: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.930+0000 7fa637d2e700 1 -- 172.21.15.49:0/3740151058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa630100770 msgr2=0x7fa6301047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.930+0000 7fa637d2e700 1 --2- 172.21.15.49:0/3740151058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa630100770 0x7fa6301047c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa620009910 tx=0x7fa620009c20 comp rx=0 tx=0).stop 2024-05-15T15:57:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.930+0000 7fa637d2e700 1 -- 172.21.15.49:0/3740151058 shutdown_connections 2024-05-15T15:57:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.930+0000 7fa637d2e700 1 --2- 172.21.15.49:0/3740151058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa630100770 0x7fa6301047c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.930+0000 7fa637d2e700 1 --2- 172.21.15.49:0/3740151058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6300ffdc0 0x7fa6301001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.930+0000 7fa637d2e700 1 -- 172.21.15.49:0/3740151058 >> 172.21.15.49:0/3740151058 conn(0x7fa630076800 msgr2=0x7fa6300fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.930+0000 7fa637d2e700 1 -- 172.21.15.49:0/3740151058 shutdown_connections 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.930+0000 7fa637d2e700 1 -- 172.21.15.49:0/3740151058 wait complete. 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa637d2e700 1 Processor -- start 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa637d2e700 1 -- start start 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa637d2e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6300ffdc0 0x7fa630198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa637d2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa630100770 0x7fa6301993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa637d2e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630199a20 con 0x7fa6300ffdc0 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa637d2e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630199b60 con 0x7fa630100770 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa635aca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6300ffdc0 0x7fa630198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa635aca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6300ffdc0 0x7fa630198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56380/0 (socket says 172.21.15.49:56380) 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa635aca700 1 -- 172.21.15.49:0/762572259 learned_addr learned my addr 172.21.15.49:0/762572259 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:57:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.931+0000 7fa6352c9700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa630100770 0x7fa6301993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:58.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.932+0000 7fa635aca700 1 -- 172.21.15.49:0/762572259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa630100770 msgr2=0x7fa6301993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:58.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.932+0000 7fa635aca700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa630100770 0x7fa6301993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:58.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.932+0000 7fa635aca700 1 -- 172.21.15.49:0/762572259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6200095f0 con 0x7fa6300ffdc0 2024-05-15T15:57:58.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.932+0000 7fa635aca700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6300ffdc0 0x7fa630198e90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c930 tx=0x7fa62c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:58.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.932+0000 7fa626ffd700 1 -- 172.21.15.49:0/762572259 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa62c007ad0 con 0x7fa6300ffdc0 2024-05-15T15:57:58.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.932+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63019d8a0 con 0x7fa6300ffdc0 2024-05-15T15:57:58.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.932+0000 7fa626ffd700 1 -- 172.21.15.49:0/762572259 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa62c00ce80 con 0x7fa6300ffdc0 2024-05-15T15:57:58.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.932+0000 7fa626ffd700 1 -- 172.21.15.49:0/762572259 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa62c018630 con 0x7fa6300ffdc0 2024-05-15T15:57:58.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.932+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63019ddf0 con 0x7fa6300ffdc0 2024-05-15T15:57:58.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.934+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6301012a0 con 0x7fa6300ffdc0 2024-05-15T15:57:58.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.934+0000 7fa626ffd700 1 -- 172.21.15.49:0/762572259 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7fa62c018790 con 0x7fa6300ffdc0 2024-05-15T15:57:58.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.935+0000 7fa626ffd700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa61c06de80 0x7fa61c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:57:58.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.935+0000 7fa626ffd700 1 -- 172.21.15.49:0/762572259 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa62c08db70 con 0x7fa6300ffdc0 2024-05-15T15:57:58.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.935+0000 7fa6352c9700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa61c06de80 0x7fa61c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:57:58.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.935+0000 7fa6352c9700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa61c06de80 0x7fa61c070340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa63019a3a0 tx=0x7fa62001a040 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:57:58.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:58.940+0000 7fa626ffd700 1 -- 172.21.15.49:0/762572259 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fa62c058400 con 0x7fa6300ffdc0 2024-05-15T15:57:59.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.159+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fa63004ec60 con 0x7fa6300ffdc0 2024-05-15T15:57:59.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.160+0000 7fa626ffd700 1 -- 172.21.15.49:0/762572259 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v48)=0 v48) v1 ==== 135+0+0 (secure 0 0 0) 0x7fa62c05bac0 con 0x7fa6300ffdc0 2024-05-15T15:57:59.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.162+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa61c06de80 msgr2=0x7fa61c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:59.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.162+0000 7fa637d2e700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa61c06de80 0x7fa61c070340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa63019a3a0 tx=0x7fa62001a040 comp rx=0 tx=0).stop 2024-05-15T15:57:59.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.162+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6300ffdc0 msgr2=0x7fa630198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:57:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.163+0000 7fa637d2e700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6300ffdc0 0x7fa630198e90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c930 tx=0x7fa62c00cc40 comp rx=0 tx=0).stop 2024-05-15T15:57:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.163+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 shutdown_connections 2024-05-15T15:57:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.163+0000 7fa637d2e700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa61c06de80 0x7fa61c070340 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.163+0000 7fa637d2e700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6300ffdc0 0x7fa630198e90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.163+0000 7fa637d2e700 1 --2- 172.21.15.49:0/762572259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa630100770 0x7fa6301993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:57:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.163+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 >> 172.21.15.49:0/762572259 conn(0x7fa630076800 msgr2=0x7fa630103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:57:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.163+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 shutdown_connections 2024-05-15T15:57:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:57:59.163+0000 7fa637d2e700 1 -- 172.21.15.49:0/762572259 wait complete. 2024-05-15T15:57:59.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:57:59 smithi049 ceph-mon[28399]: pgmap v98: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:59.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:57:59 smithi144 ceph-mon[36389]: pgmap v98: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:57:59.832 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-05-15T15:58:00.062 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:58:01.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:01 smithi049 ceph-mon[28399]: pgmap v99: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:01.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:01 smithi144 ceph-mon[36389]: pgmap v99: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.776+0000 7fa7c05d3700 1 -- 172.21.15.49:0/3070539070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8101da0 msgr2=0x7fa7b8102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.776+0000 7fa7c05d3700 1 --2- 172.21.15.49:0/3070539070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8101da0 0x7fa7b8102180 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400b0a0 tx=0x7fa7b400b3b0 comp rx=0 tx=0).stop 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.776+0000 7fa7c05d3700 1 -- 172.21.15.49:0/3070539070 shutdown_connections 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.776+0000 7fa7c05d3700 1 --2- 172.21.15.49:0/3070539070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b8102750 0x7fa7b810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.776+0000 7fa7c05d3700 1 --2- 172.21.15.49:0/3070539070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8101da0 0x7fa7b8102180 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.776+0000 7fa7c05d3700 1 -- 172.21.15.49:0/3070539070 >> 172.21.15.49:0/3070539070 conn(0x7fa7b80fb6a0 msgr2=0x7fa7b80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.776+0000 7fa7c05d3700 1 -- 172.21.15.49:0/3070539070 shutdown_connections 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.776+0000 7fa7c05d3700 1 -- 172.21.15.49:0/3070539070 wait complete. 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.777+0000 7fa7c05d3700 1 Processor -- start 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.777+0000 7fa7c05d3700 1 -- start start 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.777+0000 7fa7c05d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b8101da0 0x7fa7b8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.777+0000 7fa7c05d3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8102750 0x7fa7b8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:01.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.777+0000 7fa7c05d3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b81999f0 con 0x7fa7b8102750 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.777+0000 7fa7c05d3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b819d6c0 con 0x7fa7b8101da0 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7bdb6e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8102750 0x7fa7b8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7bdb6e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8102750 0x7fa7b8199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56396/0 (socket says 172.21.15.49:56396) 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7bdb6e700 1 -- 172.21.15.49:0/70806226 learned_addr learned my addr 172.21.15.49:0/70806226 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7be36f700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b8101da0 0x7fa7b8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7bdb6e700 1 -- 172.21.15.49:0/70806226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b8101da0 msgr2=0x7fa7b8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7bdb6e700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b8101da0 0x7fa7b8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7bdb6e700 1 -- 172.21.15.49:0/70806226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4009d00 con 0x7fa7b8102750 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7bdb6e700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8102750 0x7fa7b8199310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800d900 tx=0x7fa7a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7af7fe700 1 -- 172.21.15.49:0/70806226 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a8009900 con 0x7fa7b8102750 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b819d9a0 con 0x7fa7b8102750 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7af7fe700 1 -- 172.21.15.49:0/70806226 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7a8010460 con 0x7fa7b8102750 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7af7fe700 1 -- 172.21.15.49:0/70806226 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a800f5f0 con 0x7fa7b8102750 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.778+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b819def0 con 0x7fa7b8102750 2024-05-15T15:58:01.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.779+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b81097a0 con 0x7fa7b8102750 2024-05-15T15:58:01.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.780+0000 7fa7af7fe700 1 -- 172.21.15.49:0/70806226 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7fa7a80105d0 con 0x7fa7b8102750 2024-05-15T15:58:01.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.780+0000 7fa7af7fe700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa7a406de80 0x7fa7a4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:01.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.780+0000 7fa7af7fe700 1 -- 172.21.15.49:0/70806226 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa7a808d7a0 con 0x7fa7b8102750 2024-05-15T15:58:01.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.780+0000 7fa7be36f700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa7a406de80 0x7fa7a4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:01.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.781+0000 7fa7be36f700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa7a406de80 0x7fa7a4070340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4000f80 tx=0x7fa7b40060c0 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:01.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:01.786+0000 7fa7af7fe700 1 -- 172.21.15.49:0/70806226 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fa7a8056fb0 con 0x7fa7b8102750 2024-05-15T15:58:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.016+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7b810de90 con 0x7fa7a406de80 2024-05-15T15:58:02.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.022+0000 7fa7af7fe700 1 -- 172.21.15.49:0/70806226 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fa7b810de90 con 0x7fa7a406de80 2024-05-15T15:58:02.025 INFO:teuthology.orchestra.run.smithi049.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa7a406de80 msgr2=0x7fa7a4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa7a406de80 0x7fa7a4070340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4000f80 tx=0x7fa7b40060c0 comp rx=0 tx=0).stop 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8102750 msgr2=0x7fa7b8199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8102750 0x7fa7b8199310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800d900 tx=0x7fa7a800dc10 comp rx=0 tx=0).stop 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 shutdown_connections 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fa7a406de80 0x7fa7a4070340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b8101da0 0x7fa7b8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 --2- 172.21.15.49:0/70806226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b8102750 0x7fa7b8199310 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 >> 172.21.15.49:0/70806226 conn(0x7fa7b80fb6a0 msgr2=0x7fa7b810a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:02.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 shutdown_connections 2024-05-15T15:58:02.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:02.024+0000 7fa7c05d3700 1 -- 172.21.15.49:0/70806226 wait complete. 2024-05-15T15:58:02.624 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-15T15:58:02.625 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T15:58:02.625 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-05-15T15:58:02.850 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T15:58:03.296 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:03 smithi049 ceph-mon[28399]: pgmap v100: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:03.297 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:03 smithi049 ceph-mon[28399]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:03.297 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:03 smithi049 ceph-mon[28399]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:58:03.297 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:03 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:03.297 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:03 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:03.297 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:03 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:03.297 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:03 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:58:03.297 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:03 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:03.297 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:03 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:58:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:03 smithi144 ceph-mon[36389]: pgmap v100: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:03 smithi144 ceph-mon[36389]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:03 smithi144 ceph-mon[36389]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:58:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:03 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:03 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:03 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:03 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:58:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:03 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:03 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:58:04.295 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:04 smithi049 ceph-mon[28399]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:58:04.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:04 smithi144 ceph-mon[36389]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:de5210d58560a4684fdb5b238ef3e6e0285a1de8 2024-05-15T15:58:05.316 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:05 smithi049 ceph-mon[28399]: pgmap v101: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:05.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 -- 172.21.15.49:0/3553945144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 msgr2=0x7f6d741001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 --2- 172.21.15.49:0/3553945144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 0x7f6d741001a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009910 tx=0x7f6d70009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 -- 172.21.15.49:0/3553945144 shutdown_connections 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 --2- 172.21.15.49:0/3553945144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d74100770 0x7f6d741047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 --2- 172.21.15.49:0/3553945144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 0x7f6d741001a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 -- 172.21.15.49:0/3553945144 >> 172.21.15.49:0/3553945144 conn(0x7f6d74076800 msgr2=0x7f6d740fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 -- 172.21.15.49:0/3553945144 shutdown_connections 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 -- 172.21.15.49:0/3553945144 wait complete. 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 Processor -- start 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.074+0000 7f6d7bef2700 1 -- start start 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d7bef2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 0x7f6d740fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d7bef2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d74100770 0x7f6d740fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d7bef2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d740fc760 con 0x7f6d740ffdc0 2024-05-15T15:58:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d7bef2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d740fc8a0 con 0x7f6d74100770 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d79c8e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 0x7f6d740fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d79c8e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 0x7f6d740fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39224/0 (socket says 172.21.15.49:39224) 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d79c8e700 1 -- 172.21.15.49:0/2757229884 learned_addr learned my addr 172.21.15.49:0/2757229884 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d79c8e700 1 -- 172.21.15.49:0/2757229884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d74100770 msgr2=0x7f6d740fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d7948d700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d74100770 0x7f6d740fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d79c8e700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d74100770 0x7f6d740fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d79c8e700 1 -- 172.21.15.49:0/2757229884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d700095f0 con 0x7f6d740ffdc0 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d7948d700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d74100770 0x7f6d740fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d79c8e700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 0x7f6d740fbce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000bd70 tx=0x7f6d7000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d6affd700 1 -- 172.21.15.49:0/2757229884 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7001d070 con 0x7f6d740ffdc0 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d6affd700 1 -- 172.21.15.49:0/2757229884 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d7000f460 con 0x7f6d740ffdc0 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d740fcb20 con 0x7f6d740ffdc0 2024-05-15T15:58:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.075+0000 7f6d6affd700 1 -- 172.21.15.49:0/2757229884 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d70017630 con 0x7f6d740ffdc0 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.076+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d741a30a0 con 0x7f6d740ffdc0 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.077+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d741012a0 con 0x7f6d740ffdc0 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.077+0000 7f6d6affd700 1 -- 172.21.15.49:0/2757229884 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f6d7000f5d0 con 0x7f6d740ffdc0 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.078+0000 7f6d6affd700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6d6006de80 0x7f6d60070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.078+0000 7f6d6affd700 1 -- 172.21.15.49:0/2757229884 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6d7008efd0 con 0x7f6d740ffdc0 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.078+0000 7f6d7948d700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6d6006de80 0x7f6d60070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.078+0000 7f6d7948d700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6d6006de80 0x7f6d60070340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d740fd410 tx=0x7f6d64008040 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.083+0000 7f6d6affd700 1 -- 172.21.15.49:0/2757229884 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f6d70058860 con 0x7f6d740ffdc0 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.313+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d7410cf10 con 0x7f6d6006de80 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.314+0000 7f6d6affd700 1 -- 172.21.15.49:0/2757229884 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f6d7410cf10 con 0x7f6d6006de80 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6d6006de80 msgr2=0x7f6d60070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6d6006de80 0x7f6d60070340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d740fd410 tx=0x7f6d64008040 comp rx=0 tx=0).stop 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 msgr2=0x7f6d740fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 0x7f6d740fbce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000bd70 tx=0x7f6d7000be50 comp rx=0 tx=0).stop 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 shutdown_connections 2024-05-15T15:58:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6d6006de80 0x7f6d60070340 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d740ffdc0 0x7f6d740fbce0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 --2- 172.21.15.49:0/2757229884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d74100770 0x7f6d740fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 >> 172.21.15.49:0/2757229884 conn(0x7f6d74076800 msgr2=0x7f6d740fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 shutdown_connections 2024-05-15T15:58:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.316+0000 7f6d7bef2700 1 -- 172.21.15.49:0/2757229884 wait complete. 2024-05-15T15:58:05.330 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:58:05.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.428+0000 7f5e338e7700 1 -- 172.21.15.49:0/2641887185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c1007c0 msgr2=0x7f5e2c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.428+0000 7f5e338e7700 1 --2- 172.21.15.49:0/2641887185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c1007c0 0x7f5e2c104810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009910 tx=0x7f5e1c009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:05.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.428+0000 7f5e338e7700 1 -- 172.21.15.49:0/2641887185 shutdown_connections 2024-05-15T15:58:05.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.428+0000 7f5e338e7700 1 --2- 172.21.15.49:0/2641887185 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c1007c0 0x7f5e2c104810 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.428+0000 7f5e338e7700 1 --2- 172.21.15.49:0/2641887185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2c0ffe10 0x7f5e2c1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.428+0000 7f5e338e7700 1 -- 172.21.15.49:0/2641887185 >> 172.21.15.49:0/2641887185 conn(0x7f5e2c0fb6a0 msgr2=0x7f5e2c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:05.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.429+0000 7f5e338e7700 1 -- 172.21.15.49:0/2641887185 shutdown_connections 2024-05-15T15:58:05.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.429+0000 7f5e338e7700 1 -- 172.21.15.49:0/2641887185 wait complete. 2024-05-15T15:58:05.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.429+0000 7f5e338e7700 1 Processor -- start 2024-05-15T15:58:05.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.429+0000 7f5e338e7700 1 -- start start 2024-05-15T15:58:05.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e338e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c0ffe10 0x7f5e2c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:05.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e338e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2c1007c0 0x7f5e2c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:05.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e338e7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c198620 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e338e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c198760 con 0x7f5e2c1007c0 2024-05-15T15:58:05.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e31683700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c0ffe10 0x7f5e2c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e31683700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c0ffe10 0x7f5e2c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39232/0 (socket says 172.21.15.49:39232) 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e31683700 1 -- 172.21.15.49:0/147749311 learned_addr learned my addr 172.21.15.49:0/147749311 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e30e82700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2c1007c0 0x7f5e2c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e31683700 1 -- 172.21.15.49:0/147749311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2c1007c0 msgr2=0x7f5e2c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e31683700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2c1007c0 0x7f5e2c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e31683700 1 -- 172.21.15.49:0/147749311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c0095f0 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e31683700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c0ffe10 0x7f5e2c197b10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800b4b0 tx=0x7f5e2800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e227fc700 1 -- 172.21.15.49:0/147749311 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e28011800 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e227fc700 1 -- 172.21.15.49:0/147749311 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e28011e20 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.430+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c1a0b80 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.431+0000 7f5e227fc700 1 -- 172.21.15.49:0/147749311 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e2800f2e0 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.431+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c1a0f90 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.432+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c108140 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.432+0000 7f5e227fc700 1 -- 172.21.15.49:0/147749311 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f5e28010380 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.433+0000 7f5e227fc700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5e1806de90 0x7f5e18070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:05.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.433+0000 7f5e227fc700 1 -- 172.21.15.49:0/147749311 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5e2808d6e0 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.433+0000 7f5e30e82700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5e1806de90 0x7f5e18070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:05.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.433+0000 7f5e30e82700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5e1806de90 0x7f5e18070350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c069980 tx=0x7f5e1c01a040 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:05.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.438+0000 7f5e227fc700 1 -- 172.21.15.49:0/147749311 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f5e28056f70 con 0x7f5e2c0ffe10 2024-05-15T15:58:05.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:05 smithi144 ceph-mon[36389]: pgmap v101: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:05.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.669+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e2c0697d0 con 0x7f5e1806de90 2024-05-15T15:58:05.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.670+0000 7f5e227fc700 1 -- 172.21.15.49:0/147749311 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f5e2c0697d0 con 0x7f5e1806de90 2024-05-15T15:58:05.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5e1806de90 msgr2=0x7f5e18070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5e1806de90 0x7f5e18070350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c069980 tx=0x7f5e1c01a040 comp rx=0 tx=0).stop 2024-05-15T15:58:05.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c0ffe10 msgr2=0x7f5e2c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c0ffe10 0x7f5e2c197b10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800b4b0 tx=0x7f5e2800b870 comp rx=0 tx=0).stop 2024-05-15T15:58:05.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 shutdown_connections 2024-05-15T15:58:05.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5e1806de90 0x7f5e18070350 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c0ffe10 0x7f5e2c197b10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 --2- 172.21.15.49:0/147749311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2c1007c0 0x7f5e2c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 >> 172.21.15.49:0/147749311 conn(0x7f5e2c0fb6a0 msgr2=0x7f5e2c0fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:05.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 shutdown_connections 2024-05-15T15:58:05.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.672+0000 7f5e338e7700 1 -- 172.21.15.49:0/147749311 wait complete. 2024-05-15T15:58:05.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.781+0000 7f6f03854700 1 -- 172.21.15.49:0/3058784078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 msgr2=0x7f6efc073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.781+0000 7f6f03854700 1 --2- 172.21.15.49:0/3058784078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 0x7f6efc073770 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009940 tx=0x7f6ef8009c50 comp rx=0 tx=0).stop 2024-05-15T15:58:05.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.781+0000 7f6f03854700 1 -- 172.21.15.49:0/3058784078 shutdown_connections 2024-05-15T15:58:05.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.781+0000 7f6f03854700 1 --2- 172.21.15.49:0/3058784078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6efc073cb0 0x7f6efc10cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.781+0000 7f6f03854700 1 --2- 172.21.15.49:0/3058784078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 0x7f6efc073770 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.781+0000 7f6f03854700 1 -- 172.21.15.49:0/3058784078 >> 172.21.15.49:0/3058784078 conn(0x7f6efc0fc790 msgr2=0x7f6efc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:05.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.782+0000 7f6f03854700 1 -- 172.21.15.49:0/3058784078 shutdown_connections 2024-05-15T15:58:05.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.782+0000 7f6f03854700 1 -- 172.21.15.49:0/3058784078 wait complete. 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.782+0000 7f6f03854700 1 Processor -- start 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.782+0000 7f6f03854700 1 -- start start 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.782+0000 7f6f03854700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 0x7f6efc198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.782+0000 7f6f03854700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6efc073cb0 0x7f6efc1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.782+0000 7f6f03854700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc199ac0 con 0x7f6efc073390 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.782+0000 7f6f03854700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc19d790 con 0x7f6efc073cb0 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f015f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 0x7f6efc198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f015f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 0x7f6efc198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39242/0 (socket says 172.21.15.49:39242) 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f015f0700 1 -- 172.21.15.49:0/3634707336 learned_addr learned my addr 172.21.15.49:0/3634707336 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:05.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f00def700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6efc073cb0 0x7f6efc1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f015f0700 1 -- 172.21.15.49:0/3634707336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6efc073cb0 msgr2=0x7f6efc1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f015f0700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6efc073cb0 0x7f6efc1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f015f0700 1 -- 172.21.15.49:0/3634707336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef80095f0 con 0x7f6efc073390 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f015f0700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 0x7f6efc198ea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8005620 tx=0x7f6ef8005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6eee7fc700 1 -- 172.21.15.49:0/3634707336 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef801d070 con 0x7f6efc073390 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6eee7fc700 1 -- 172.21.15.49:0/3634707336 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ef8003d50 con 0x7f6efc073390 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc19da10 con 0x7f6efc073390 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6eee7fc700 1 -- 172.21.15.49:0/3634707336 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef8021410 con 0x7f6efc073390 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.783+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc19df00 con 0x7f6efc073390 2024-05-15T15:58:05.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.784+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc074a40 con 0x7f6efc073390 2024-05-15T15:58:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.785+0000 7f6eee7fc700 1 -- 172.21.15.49:0/3634707336 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f6ef800f810 con 0x7f6efc073390 2024-05-15T15:58:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.785+0000 7f6eee7fc700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6ee806de80 0x7f6ee8070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.786+0000 7f6eee7fc700 1 -- 172.21.15.49:0/3634707336 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6ef808eca0 con 0x7f6efc073390 2024-05-15T15:58:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.786+0000 7f6f00def700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6ee806de80 0x7f6ee8070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:05.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.786+0000 7f6f00def700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6ee806de80 0x7f6ee8070340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6efc19a400 tx=0x7f6ef0009190 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:05.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:05.791+0000 7f6eee7fc700 1 -- 172.21.15.49:0/3634707336 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f6ef8093070 con 0x7f6efc073390 2024-05-15T15:58:06.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.027+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6efc110110 con 0x7f6ee806de80 2024-05-15T15:58:06.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.038+0000 7f6eee7fc700 1 -- 172.21.15.49:0/3634707336 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6efc110110 con 0x7f6ee806de80 2024-05-15T15:58:06.041 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:58:06.041 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (81s) 49s ago 12m 17.2M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T15:58:06.041 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (13m) 49s ago 13m 8250k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:58:06.041 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (12m) 2m ago 12m 7880k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:58:06.041 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (13m) 49s ago 13m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:58:06.041 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (11m) 2m ago 11m 6656k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (54s) 49s ago 12m 56.7M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (7m) 49s ago 7m 89.8M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (7m) 49s ago 7m 153M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (7m) 2m ago 7m 26.3M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (7m) 2m ago 7m 28.0M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (3m) 49s ago 14m 513M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (2m) 2m ago 11m 452M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (14m) 49s ago 14m 56.0M 2048M 18.2.2-1343-g2965081f a04a73ffe079 73d4d6879c0c 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (11m) 2m ago 11m 44.4M 2048M 18.2.2-1343-g2965081f a04a73ffe079 bf0fd6a44ac8 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (2m) 49s ago 13m 13.6M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (2m) 2m ago 11m 5200k - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (11m) 49s ago 11m 218M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (10m) 49s ago 10m 218M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (10m) 49s ago 10m 212M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:58:06.042 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (9m) 2m ago 9m 255M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:58:06.043 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (9m) 2m ago 9m 210M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:58:06.043 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (8m) 2m ago 8m 197M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:58:06.043 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (98s) 49s ago 12m 51.5M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T15:58:06.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6ee806de80 msgr2=0x7f6ee8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6ee806de80 0x7f6ee8070340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6efc19a400 tx=0x7f6ef0009190 comp rx=0 tx=0).stop 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 msgr2=0x7f6efc198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 0x7f6efc198ea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8005620 tx=0x7f6ef8005650 comp rx=0 tx=0).stop 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 shutdown_connections 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f6ee806de80 0x7f6ee8070340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6efc073390 0x7f6efc198ea0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 --2- 172.21.15.49:0/3634707336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6efc073cb0 0x7f6efc1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 >> 172.21.15.49:0/3634707336 conn(0x7f6efc0fc790 msgr2=0x7f6efc1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 shutdown_connections 2024-05-15T15:58:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.041+0000 7f6f03854700 1 -- 172.21.15.49:0/3634707336 wait complete. 2024-05-15T15:58:06.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.150+0000 7f69f0cdb700 1 -- 172.21.15.49:0/1200771924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0fff90 msgr2=0x7f69ec103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.150+0000 7f69f0cdb700 1 --2- 172.21.15.49:0/1200771924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0fff90 0x7f69ec103fe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009910 tx=0x7f69e0009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:06.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.151+0000 7f69f0cdb700 1 -- 172.21.15.49:0/1200771924 shutdown_connections 2024-05-15T15:58:06.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.151+0000 7f69f0cdb700 1 --2- 172.21.15.49:0/1200771924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0fff90 0x7f69ec103fe0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.151+0000 7f69f0cdb700 1 --2- 172.21.15.49:0/1200771924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69ec0ff5e0 0x7f69ec0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.151+0000 7f69f0cdb700 1 -- 172.21.15.49:0/1200771924 >> 172.21.15.49:0/1200771924 conn(0x7f69ec075fc0 msgr2=0x7f69ec0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:06.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.151+0000 7f69f0cdb700 1 -- 172.21.15.49:0/1200771924 shutdown_connections 2024-05-15T15:58:06.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.151+0000 7f69f0cdb700 1 -- 172.21.15.49:0/1200771924 wait complete. 2024-05-15T15:58:06.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.151+0000 7f69f0cdb700 1 Processor -- start 2024-05-15T15:58:06.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.151+0000 7f69f0cdb700 1 -- start start 2024-05-15T15:58:06.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69f0cdb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0ff5e0 0x7f69ec0fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:06.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69f0cdb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69ec0fff90 0x7f69ec0fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69f0cdb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0fbf30 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69f0cdb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0fc070 con 0x7f69ec0fff90 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69ea59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0ff5e0 0x7f69ec0fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69ea59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0ff5e0 0x7f69ec0fb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39260/0 (socket says 172.21.15.49:39260) 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69ea59c700 1 -- 172.21.15.49:0/3685773542 learned_addr learned my addr 172.21.15.49:0/3685773542 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69e9d9b700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69ec0fff90 0x7f69ec0fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69ea59c700 1 -- 172.21.15.49:0/3685773542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69ec0fff90 msgr2=0x7f69ec0fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69ea59c700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69ec0fff90 0x7f69ec0fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69ea59c700 1 -- 172.21.15.49:0/3685773542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e00095f0 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.152+0000 7f69ea59c700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0ff5e0 0x7f69ec0fb4b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00b4b0 tx=0x7f69dc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.153+0000 7f69db7fe700 1 -- 172.21.15.49:0/3685773542 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69dc011800 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.153+0000 7f69db7fe700 1 -- 172.21.15.49:0/3685773542 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69dc011e20 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.153+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec1a2f00 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.153+0000 7f69db7fe700 1 -- 172.21.15.49:0/3685773542 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69dc00f2e0 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.153+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec1a3260 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.154+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec100ac0 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.155+0000 7f69db7fe700 1 -- 172.21.15.49:0/3685773542 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f69dc010380 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.155+0000 7f69db7fe700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f69d406de80 0x7f69d4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.155+0000 7f69db7fe700 1 -- 172.21.15.49:0/3685773542 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f69dc08d750 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.155+0000 7f69e9d9b700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f69d406de80 0x7f69d4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:06.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.156+0000 7f69e9d9b700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f69d406de80 0x7f69d4070340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0fcbe0 tx=0x7f69e001a040 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:06.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.160+0000 7f69db7fe700 1 -- 172.21.15.49:0/3685773542 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f69dc056f40 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.478+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f69ec0fc230 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.478+0000 7f69db7fe700 1 -- 172.21.15.49:0/3685773542 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f69dc05a600 con 0x7f69ec0ff5e0 2024-05-15T15:58:06.481 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:58:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:58:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 12, 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:58:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:58:06.483 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:58:06.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f69d406de80 msgr2=0x7f69d4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f69d406de80 0x7f69d4070340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0fcbe0 tx=0x7f69e001a040 comp rx=0 tx=0).stop 2024-05-15T15:58:06.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0ff5e0 msgr2=0x7f69ec0fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0ff5e0 0x7f69ec0fb4b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00b4b0 tx=0x7f69dc00b870 comp rx=0 tx=0).stop 2024-05-15T15:58:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 shutdown_connections 2024-05-15T15:58:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f69d406de80 0x7f69d4070340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69ec0ff5e0 0x7f69ec0fb4b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 --2- 172.21.15.49:0/3685773542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69ec0fff90 0x7f69ec0fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 >> 172.21.15.49:0/3685773542 conn(0x7f69ec075fc0 msgr2=0x7f69ec0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 shutdown_connections 2024-05-15T15:58:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.480+0000 7f69f0cdb700 1 -- 172.21.15.49:0/3685773542 wait complete. 2024-05-15T15:58:06.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.591+0000 7fb9ca996700 1 -- 172.21.15.49:0/3974729772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40fff90 msgr2=0x7fb9c4103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.591+0000 7fb9ca996700 1 --2- 172.21.15.49:0/3974729772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40fff90 0x7fb9c4103fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009910 tx=0x7fb9ac009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:06.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.591+0000 7fb9ca996700 1 -- 172.21.15.49:0/3974729772 shutdown_connections 2024-05-15T15:58:06.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.591+0000 7fb9ca996700 1 --2- 172.21.15.49:0/3974729772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40fff90 0x7fb9c4103fe0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.591+0000 7fb9ca996700 1 --2- 172.21.15.49:0/3974729772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c40ff5e0 0x7fb9c40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.591+0000 7fb9ca996700 1 -- 172.21.15.49:0/3974729772 >> 172.21.15.49:0/3974729772 conn(0x7fb9c4075fc0 msgr2=0x7fb9c40fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:06.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.591+0000 7fb9ca996700 1 -- 172.21.15.49:0/3974729772 shutdown_connections 2024-05-15T15:58:06.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.591+0000 7fb9ca996700 1 -- 172.21.15.49:0/3974729772 wait complete. 2024-05-15T15:58:06.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9ca996700 1 Processor -- start 2024-05-15T15:58:06.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9ca996700 1 -- start start 2024-05-15T15:58:06.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9ca996700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40ff5e0 0x7fb9c419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:06.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9ca996700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c40fff90 0x7fb9c419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:06.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9ca996700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c419d4c0 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9ca996700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4196aa0 con 0x7fb9c40fff90 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9c3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40ff5e0 0x7fb9c419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9c3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40ff5e0 0x7fb9c419c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39282/0 (socket says 172.21.15.49:39282) 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9c3fff700 1 -- 172.21.15.49:0/3921741813 learned_addr learned my addr 172.21.15.49:0/3921741813 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9bbfff700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c40fff90 0x7fb9c419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9c3fff700 1 -- 172.21.15.49:0/3921741813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c40fff90 msgr2=0x7fb9c419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9c3fff700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c40fff90 0x7fb9c419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9c3fff700 1 -- 172.21.15.49:0/3921741813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac0095f0 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.592+0000 7fb9bbfff700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c40fff90 0x7fb9c419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.593+0000 7fb9c3fff700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40ff5e0 0x7fb9c419c960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400d900 tx=0x7fb9b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.593+0000 7fb9c1ffb700 1 -- 172.21.15.49:0/3921741813 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b4004a00 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.593+0000 7fb9c1ffb700 1 -- 172.21.15.49:0/3921741813 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9b40054e0 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.593+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c4196d80 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.593+0000 7fb9c1ffb700 1 -- 172.21.15.49:0/3921741813 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b4009d60 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.593+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c41972d0 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.594+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c4100ac0 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.594+0000 7fb9c1ffb700 1 -- 172.21.15.49:0/3921741813 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7fb9b4005020 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.595+0000 7fb9c1ffb700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb9b006de90 0x7fb9b0070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:06.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.595+0000 7fb9c1ffb700 1 -- 172.21.15.49:0/3921741813 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb9b408ddb0 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.595+0000 7fb9bbfff700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb9b006de90 0x7fb9b0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:06.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.596+0000 7fb9bbfff700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb9b006de90 0x7fb9b0070350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4197c60 tx=0x7fb9ac00b560 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:06.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.600+0000 7fb9c1ffb700 1 -- 172.21.15.49:0/3921741813 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7fb9b405b0f0 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.864+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb9c410ce20 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.865+0000 7fb9c1ffb700 1 -- 172.21.15.49:0/3921741813 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1871 (secure 0 0 0) 0x7fb9b4017760 con 0x7fb9c40ff5e0 2024-05-15T15:58:06.868 INFO:teuthology.orchestra.run.smithi049.stdout:e17 2024-05-15T15:58:06.868 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:58:06.868 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:58:06.868 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T15:58:06.868 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:58:06.868 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T15:58:06.868 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 17 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:51:00.559034+0000 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T15:58:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:06.870 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:06.871 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:06.871 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:58:06.871 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb9b006de90 msgr2=0x7fb9b0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb9b006de90 0x7fb9b0070350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4197c60 tx=0x7fb9ac00b560 comp rx=0 tx=0).stop 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40ff5e0 msgr2=0x7fb9c419c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40ff5e0 0x7fb9c419c960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400d900 tx=0x7fb9b400dcc0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 shutdown_connections 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7fb9b006de90 0x7fb9b0070350 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9c40ff5e0 0x7fb9c419c960 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 --2- 172.21.15.49:0/3921741813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c40fff90 0x7fb9c419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 >> 172.21.15.49:0/3921741813 conn(0x7fb9c4075fc0 msgr2=0x7fb9c410a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 shutdown_connections 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.867+0000 7fb9ca996700 1 -- 172.21.15.49:0/3921741813 wait complete. 2024-05-15T15:58:06.872 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.981+0000 7f936ba8d700 1 -- 172.21.15.49:0/3204719276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9364100f00 msgr2=0x7f93641012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.981+0000 7f936ba8d700 1 --2- 172.21.15.49:0/3204719276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9364100f00 0x7f93641012e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f936000b0a0 tx=0x7f936000b3b0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.981+0000 7f936ba8d700 1 -- 172.21.15.49:0/3204719276 shutdown_connections 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.982+0000 7f936ba8d700 1 --2- 172.21.15.49:0/3204719276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93641018b0 0x7f9364105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.982+0000 7f936ba8d700 1 --2- 172.21.15.49:0/3204719276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9364100f00 0x7f93641012e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.982+0000 7f936ba8d700 1 -- 172.21.15.49:0/3204719276 >> 172.21.15.49:0/3204719276 conn(0x7f93640fc7d0 msgr2=0x7f93640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.982+0000 7f936ba8d700 1 -- 172.21.15.49:0/3204719276 shutdown_connections 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.982+0000 7f936ba8d700 1 -- 172.21.15.49:0/3204719276 wait complete. 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.982+0000 7f936ba8d700 1 Processor -- start 2024-05-15T15:58:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.982+0000 7f936ba8d700 1 -- start start 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f936ba8d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9364100f00 0x7f9364196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f936ba8d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93641018b0 0x7f9364197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f936ba8d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93641978c0 con 0x7f93641018b0 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f936ba8d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936419a530 con 0x7f9364100f00 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f9369028700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93641018b0 0x7f9364197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f9369028700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93641018b0 0x7f9364197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39302/0 (socket says 172.21.15.49:39302) 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f9369028700 1 -- 172.21.15.49:0/1616053992 learned_addr learned my addr 172.21.15.49:0/1616053992 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f9369829700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9364100f00 0x7f9364196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f9369028700 1 -- 172.21.15.49:0/1616053992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9364100f00 msgr2=0x7f9364196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:06.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f9369028700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9364100f00 0x7f9364196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:06.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f9369028700 1 -- 172.21.15.49:0/1616053992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360009d00 con 0x7f93641018b0 2024-05-15T15:58:06.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f9369028700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93641018b0 0x7f9364197230 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f935400b480 tx=0x7f935400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:06.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f935affd700 1 -- 172.21.15.49:0/1616053992 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93540118c0 con 0x7f93641018b0 2024-05-15T15:58:06.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f935affd700 1 -- 172.21.15.49:0/1616053992 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9354010380 con 0x7f93641018b0 2024-05-15T15:58:06.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936419a810 con 0x7f93641018b0 2024-05-15T15:58:06.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f935affd700 1 -- 172.21.15.49:0/1616053992 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935400f4f0 con 0x7f93641018b0 2024-05-15T15:58:06.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.983+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936419ad60 con 0x7f93641018b0 2024-05-15T15:58:06.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.984+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93641092a0 con 0x7f93641018b0 2024-05-15T15:58:06.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.985+0000 7f935affd700 1 -- 172.21.15.49:0/1616053992 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f93540104f0 con 0x7f93641018b0 2024-05-15T15:58:06.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.985+0000 7f935affd700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f935006de80 0x7f9350070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:06.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.985+0000 7f935affd700 1 -- 172.21.15.49:0/1616053992 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f935408d7f0 con 0x7f93641018b0 2024-05-15T15:58:06.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.985+0000 7f9369829700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f935006de80 0x7f9350070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:06.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.986+0000 7f9369829700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f935006de80 0x7f9350070340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f936000b830 tx=0x7f9360006000 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:06.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:06.990+0000 7f935affd700 1 -- 172.21.15.49:0/1616053992 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f9354057000 con 0x7f93641018b0 2024-05-15T15:58:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: pgmap v102: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: Upgrade: Target is version 19.0.0-2576-gde5210d5 (squid) 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a'] 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: Upgrade: Setting container_image for all mgr 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3685773542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:58:07.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:06 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/3921741813' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:58:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: pgmap v102: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: Upgrade: Target is version 19.0.0-2576-gde5210d5 (squid) 2024-05-15T15:58:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a'] 2024-05-15T15:58:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:58:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: Upgrade: Setting container_image for all mgr 2024-05-15T15:58:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-15T15:58:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3685773542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:58:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:06 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/3921741813' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:58:07.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.221+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 --> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9364198030 con 0x7f935006de80 2024-05-15T15:58:07.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.222+0000 7f935affd700 1 -- 172.21.15.49:0/1616053992 <== mgr.24483 v2:172.21.15.49:6800/70490983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9364198030 con 0x7f935006de80 2024-05-15T15:58:07.225 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:58:07.225 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:58:07.225 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:58:07.225 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T15:58:07.225 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T15:58:07.225 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr" 2024-05-15T15:58:07.226 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T15:58:07.226 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "2/23 daemons upgraded", 2024-05-15T15:58:07.226 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mon daemons", 2024-05-15T15:58:07.226 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:58:07.226 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:58:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.224+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f935006de80 msgr2=0x7f9350070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:07.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.224+0000 7f936ba8d700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f935006de80 0x7f9350070340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f936000b830 tx=0x7f9360006000 comp rx=0 tx=0).stop 2024-05-15T15:58:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.224+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93641018b0 msgr2=0x7f9364197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.224+0000 7f936ba8d700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93641018b0 0x7f9364197230 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f935400b480 tx=0x7f935400b840 comp rx=0 tx=0).stop 2024-05-15T15:58:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.225+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 shutdown_connections 2024-05-15T15:58:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.225+0000 7f936ba8d700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f935006de80 0x7f9350070340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.225+0000 7f936ba8d700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9364100f00 0x7f9364196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.225+0000 7f936ba8d700 1 --2- 172.21.15.49:0/1616053992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93641018b0 0x7f9364197230 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.225+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 >> 172.21.15.49:0/1616053992 conn(0x7f93640fc7d0 msgr2=0x7f93640fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.225+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 shutdown_connections 2024-05-15T15:58:07.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.225+0000 7f936ba8d700 1 -- 172.21.15.49:0/1616053992 wait complete. 2024-05-15T15:58:07.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.334+0000 7f5f6d806700 1 -- 172.21.15.49:0/1065887952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 msgr2=0x7f5f681012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:07.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.334+0000 7f5f6d806700 1 --2- 172.21.15.49:0/1065887952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 0x7f5f681012c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009910 tx=0x7f5f58009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:07.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.334+0000 7f5f6d806700 1 -- 172.21.15.49:0/1065887952 shutdown_connections 2024-05-15T15:58:07.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.334+0000 7f5f6d806700 1 --2- 172.21.15.49:0/1065887952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f68101890 0x7f5f681058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.334+0000 7f5f6d806700 1 --2- 172.21.15.49:0/1065887952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 0x7f5f681012c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.334+0000 7f5f6d806700 1 -- 172.21.15.49:0/1065887952 >> 172.21.15.49:0/1065887952 conn(0x7f5f680fc790 msgr2=0x7f5f680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:07.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.334+0000 7f5f6d806700 1 -- 172.21.15.49:0/1065887952 shutdown_connections 2024-05-15T15:58:07.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.334+0000 7f5f6d806700 1 -- 172.21.15.49:0/1065887952 wait complete. 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.335+0000 7f5f6d806700 1 Processor -- start 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.335+0000 7f5f6d806700 1 -- start start 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.335+0000 7f5f6d806700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 0x7f5f6819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.335+0000 7f5f6d806700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f68101890 0x7f5f6819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.335+0000 7f5f6d806700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6819d480 con 0x7f5f68100ee0 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.335+0000 7f5f6d806700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f68196ab0 con 0x7f5f68101890 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 0x7f5f6819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 0x7f5f6819c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39328/0 (socket says 172.21.15.49:39328) 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f66ffd700 1 -- 172.21.15.49:0/226617210 learned_addr learned my addr 172.21.15.49:0/226617210 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f667fc700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f68101890 0x7f5f6819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f66ffd700 1 -- 172.21.15.49:0/226617210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f68101890 msgr2=0x7f5f6819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f66ffd700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f68101890 0x7f5f6819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f66ffd700 1 -- 172.21.15.49:0/226617210 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f580095f0 con 0x7f5f68100ee0 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f667fc700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f68101890 0x7f5f6819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f66ffd700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 0x7f5f6819c920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5f58000c00 tx=0x7f5f5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f6c804700 1 -- 172.21.15.49:0/226617210 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5801d070 con 0x7f5f68100ee0 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f6c804700 1 -- 172.21.15.49:0/226617210 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f580056e0 con 0x7f5f68100ee0 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f6c804700 1 -- 172.21.15.49:0/226617210 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5800fe80 con 0x7f5f68100ee0 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f68196d30 con 0x7f5f68100ee0 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.336+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f68197220 con 0x7f5f68100ee0 2024-05-15T15:58:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.337+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f68109280 con 0x7f5f68100ee0 2024-05-15T15:58:07.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.338+0000 7f5f6c804700 1 -- 172.21.15.49:0/226617210 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 32) v1 ==== 92064+0+0 (secure 0 0 0) 0x7f5f5800f6f0 con 0x7f5f68100ee0 2024-05-15T15:58:07.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.338+0000 7f5f6c804700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5f5006de90 0x7f5f50070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:07.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.338+0000 7f5f6c804700 1 -- 172.21.15.49:0/226617210 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5f5808ef60 con 0x7f5f68100ee0 2024-05-15T15:58:07.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.338+0000 7f5f667fc700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5f5006de90 0x7f5f50070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:07.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.339+0000 7f5f667fc700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5f5006de90 0x7f5f50070350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c005fd0 tx=0x7f5f5c005ee0 comp rx=0 tx=0).ready entity=mgr.24483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:07.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.344+0000 7f5f6c804700 1 -- 172.21.15.49:0/226617210 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186398 (secure 0 0 0) 0x7f5f580c3830 con 0x7f5f68100ee0 2024-05-15T15:58:07.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.641+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5f68197f20 con 0x7f5f68100ee0 2024-05-15T15:58:07.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.641+0000 7f5f6c804700 1 -- 172.21.15.49:0/226617210 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5f5805c320 con 0x7f5f68100ee0 2024-05-15T15:58:07.644 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:07.644 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:58:07.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5f5006de90 msgr2=0x7f5f50070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5f5006de90 0x7f5f50070350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c005fd0 tx=0x7f5f5c005ee0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 msgr2=0x7f5f6819c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 0x7f5f6819c920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5f58000c00 tx=0x7f5f5800be20 comp rx=0 tx=0).stop 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 shutdown_connections 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:6800/70490983,v1:172.21.15.49:6801/70490983] conn(0x7f5f5006de90 0x7f5f50070350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f68100ee0 0x7f5f6819c920 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 --2- 172.21.15.49:0/226617210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f68101890 0x7f5f6819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 >> 172.21.15.49:0/226617210 conn(0x7f5f680fc790 msgr2=0x7f5f680fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 shutdown_connections 2024-05-15T15:58:07.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:07.644+0000 7f5f6d806700 1 -- 172.21.15.49:0/226617210 wait complete. 2024-05-15T15:58:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:07 smithi144 ceph-mon[36389]: from='client.? 172.21.15.49:0/226617210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T15:58:08.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:07 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:08.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:07 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:58:08.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:07 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:58:08.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:07 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:08.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:07 smithi049 ceph-mon[28399]: from='client.? 172.21.15.49:0/226617210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T15:58:08.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:07 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:08.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:07 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:58:08.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:07 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:58:08.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:07 smithi049 ceph-mon[28399]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:08 smithi144 ceph-mon[36389]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:09.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:08 smithi144 ceph-mon[36389]: pgmap v103: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:09.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:08 smithi144 ceph-mon[36389]: Upgrade: Updating mon.smithi049 2024-05-15T15:58:09.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:08 smithi144 ceph-mon[36389]: Deploying daemon mon.smithi049 on smithi049 2024-05-15T15:58:09.200 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:08 smithi049 ceph-mon[28399]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:09.201 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:08 smithi049 ceph-mon[28399]: pgmap v103: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:09.201 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:08 smithi049 ceph-mon[28399]: Upgrade: Updating mon.smithi049 2024-05-15T15:58:09.201 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:08 smithi049 ceph-mon[28399]: Deploying daemon mon.smithi049 on smithi049 2024-05-15T15:58:09.773 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:09 smithi049 systemd[1]: Stopping Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T15:58:10.023 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:09 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[28395]: 2024-05-15T15:58:09.770+0000 7f08dbd76700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi049 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T15:58:10.023 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:09 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[28395]: 2024-05-15T15:58:09.770+0000 7f08dbd76700 -1 mon.smithi049@0(leader) e2 *** Got Signal Terminated *** 2024-05-15T15:58:10.024 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:09 smithi049 podman[111971]: 2024-05-15 15:58:09.852431735 +0000 UTC m=+0.190381705 container stop 73d4d6879c0ceb100ce0513a9453332c17a54be572b3a38785a4cd10c6cbaf49 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=reef-2965081, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T15:58:10.024 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:09 smithi049 podman[111971]: 2024-05-15 15:58:09.881463634 +0000 UTC m=+0.219413622 container died 73d4d6879c0ceb100ce0513a9453332c17a54be572b3a38785a4cd10c6cbaf49 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, RELEASE=reef-2965081, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-15T15:58:10.274 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:10 smithi049 podman[111971]: 2024-05-15 15:58:10.27084196 +0000 UTC m=+0.608791921 container cleanup 73d4d6879c0ceb100ce0513a9453332c17a54be572b3a38785a4cd10c6cbaf49 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, RELEASE=reef-2965081, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T15:58:10.275 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:10 smithi049 bash[111971]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049 2024-05-15T15:58:10.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:10 smithi049 podman[111994]: 2024-05-15 15:58:10.463990848 +0000 UTC m=+0.605493861 container remove 73d4d6879c0ceb100ce0513a9453332c17a54be572b3a38785a4cd10c6cbaf49 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-2965081, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-15T15:58:10.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:10 smithi049 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049.service: Deactivated successfully. 2024-05-15T15:58:10.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:10 smithi049 systemd[1]: Stopped Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:58:10.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:10 smithi049 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049.service: Consumed 15.318s CPU time. 2024-05-15T15:58:11.156 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:10 smithi049 systemd[1]: Starting Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T15:58:11.414 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 podman[112124]: 2024-05-15 15:58:11.058116112 +0000 UTC m=+0.018036520 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T15:58:11.414 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 podman[112124]: 2024-05-15 15:58:11.165137219 +0000 UTC m=+0.125057660 container create 05295c0417ca4abb31f8485a422428c93418f1a88cd789f5d49e2dc457bfcca8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS) 2024-05-15T15:58:11.415 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 podman[112124]: 2024-05-15 15:58:11.384692539 +0000 UTC m=+0.344612949 container init 05295c0417ca4abb31f8485a422428c93418f1a88cd789f5d49e2dc457bfcca8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-15T15:58:11.415 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 podman[112124]: 2024-05-15 15:58:11.388399226 +0000 UTC m=+0.348319623 container start 05295c0417ca4abb31f8485a422428c93418f1a88cd789f5d49e2dc457bfcca8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-15T15:58:11.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: set uid:gid to 167:167 (ceph:ceph) 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev), process ceph-mon, pid 2 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: pidfile_write: ignore empty --pid-file 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: load: jerasure load: lrc 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: RocksDB version: 7.9.2 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Git sha 0 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Compile date 2024-05-15 11:18:19 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: DB SUMMARY 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: DB Session ID: WR0HC8WPEVSFLXE5LJIA 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: CURRENT file: CURRENT 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: IDENTITY file: IDENTITY 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1498 Bytes 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi049/store.db dir, Total Num: 1, files: 000032.sst 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi049/store.db: 000030.log size: 2344392 ; 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.error_if_exists: 0 2024-05-15T15:58:11.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.create_if_missing: 0 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.paranoid_checks: 1 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.env: 0x55ad9c3abc60 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.fs: PosixFileSystem 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.info_log: 0x55ad9cc89980 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_file_opening_threads: 16 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.statistics: (nil) 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.use_fsync: 0 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_log_file_size: 0 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.keep_log_file_num: 1000 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.recycle_log_file_num: 0 2024-05-15T15:58:11.708 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.allow_fallocate: 1 2024-05-15T15:58:11.709 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.allow_mmap_reads: 0 2024-05-15T15:58:11.709 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.allow_mmap_writes: 0 2024-05-15T15:58:11.709 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.use_direct_reads: 0 2024-05-15T15:58:11.709 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-15T15:58:11.709 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.create_missing_column_families: 0 2024-05-15T15:58:11.709 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.db_log_dir: 2024-05-15T15:58:11.709 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.wal_dir: 2024-05-15T15:58:11.710 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-15T15:58:11.710 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-15T15:58:11.710 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-15T15:58:11.710 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-15T15:58:11.710 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-15T15:58:11.710 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-15T15:58:11.710 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.advise_random_on_open: 1 2024-05-15T15:58:11.710 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.db_write_buffer_size: 0 2024-05-15T15:58:11.710 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.write_buffer_manager: 0x55ad9cc5c1e0 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.rate_limiter: (nil) 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.wal_recovery_mode: 2 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.enable_thread_tracking: 0 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.enable_pipelined_write: 0 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.unordered_write: 0 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.row_cache: None 2024-05-15T15:58:11.711 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.wal_filter: None 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.allow_ingest_behind: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.two_write_queues: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.manual_wal_flush: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.wal_compression: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.atomic_flush: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.log_readahead_size: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.best_efforts_recovery: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.allow_data_in_errors: 0 2024-05-15T15:58:11.712 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.db_host_id: __hostname__ 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_background_jobs: 2 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_background_compactions: -1 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_subcompactions: 1 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_total_wal_size: 0 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_open_files: -1 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bytes_per_sync: 0 2024-05-15T15:58:11.713 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_readahead_size: 0 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_background_flushes: -1 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Compression algorithms supported: 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: kZSTD supported: 0 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: kXpressCompression supported: 0 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: kLZ4HCCompression supported: 1 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: kZlibCompression supported: 1 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: kSnappyCompression supported: 1 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: kLZ4Compression supported: 1 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: kBZip2Compression supported: 0 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi049/store.db/MANIFEST-000015 2024-05-15T15:58:11.714 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.merge_operator: 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_filter: None 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_filter_factory: None 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.sst_partitioner_factory: None 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ad9cc89a80) 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: cache_index_and_filter_blocks: 1 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: pin_top_level_index_and_filter: 1 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: index_type: 0 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: data_block_index_type: 0 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: index_shortening: 1 2024-05-15T15:58:11.715 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: checksum: 4 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: no_block_cache: 0 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_cache: 0x55ad9cc8d090 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_cache_name: BinnedLRUCache 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_cache_options: 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: capacity : 536870912 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: num_shard_bits : 4 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: strict_capacity_limit : 0 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: high_pri_pool_ratio: 0.000 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_cache_compressed: (nil) 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: persistent_cache: (nil) 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_size: 4096 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_size_deviation: 10 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_restart_interval: 16 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: index_block_restart_interval: 1 2024-05-15T15:58:11.716 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: metadata_block_size: 4096 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: partition_filters: 0 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: use_delta_encoding: 1 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: filter_policy: bloomfilter 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: whole_key_filtering: 1 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: verify_compression: 0 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: read_amp_bytes_per_bit: 0 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: format_version: 5 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: enable_index_compression: 1 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: block_align: 0 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: max_auto_readahead_size: 262144 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: prepopulate_block_cache: 0 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: initial_auto_readahead_size: 8192 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.write_buffer_size: 33554432 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_write_buffer_number: 2 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression: NoCompression 2024-05-15T15:58:11.717 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression: Disabled 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.prefix_extractor: nullptr 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.num_levels: 7 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-15T15:58:11.718 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression_opts.level: 32767 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression_opts.strategy: 0 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression_opts.enabled: false 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.target_file_size_base: 67108864 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-15T15:58:11.719 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.arena_block_size: 1048576 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.disable_auto_compactions: 0 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-15T15:58:11.720 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.table_properties_collectors: 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.inplace_update_support: 0 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.bloom_locality: 0 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.max_successive_merges: 0 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-15T15:58:11.721 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.paranoid_file_checks: 0 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.force_consistency_checks: 1 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.report_bg_io_stats: 0 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.ttl: 2592000 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.enable_blob_files: false 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.min_blob_size: 0 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.blob_file_size: 268435456 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.blob_file_starting_level: 0 2024-05-15T15:58:11.722 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-15T15:58:11.723 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-15T15:58:11.723 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi049/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 11996, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-05-15T15:58:11.723 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-05-15T15:58:11.723 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 454e95b1-b7c0-4b85-a0b8-7f5a235bc408 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715788691419371, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715788691427480, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 2053815, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11997, "largest_seqno": 13727, "table_properties": {"data_size": 2045695, "index_size": 5184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1925, "raw_key_size": 17505, "raw_average_key_size": 23, "raw_value_size": 2029405, "raw_average_value_size": 2680, "num_data_blocks": 238, "num_entries": 757, "num_filter_entries": 757, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715788691, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "454e95b1-b7c0-4b85-a0b8-7f5a235bc408", "db_session_id": "WR0HC8WPEVSFLXE5LJIA", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715788691427588, "job": 1, "event": "recovery_finished"} 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi049/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ad9cd7e000 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: DB pointer 0x55ad9cd68000 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ** DB Stats ** 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T15:58:11.724 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ** Compaction Stats [default] ** 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: L0 1/0 1.96 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 489.1 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: L6 1/0 6.79 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Sum 2/0 8.75 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 489.1 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 489.1 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ** Compaction Stats [default] ** 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-15T15:58:11.725 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 489.1 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Cumulative compaction: 0.00 GB write, 185.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Interval compaction: 0.00 GB write, 185.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Block cache BinnedLRUCache@0x55ad9cc8d090#2 capacity: 512.00 MB usage: 8.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-15T15:58:11.726 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: starting mon.smithi049 rank 0 at public addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] at bind addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi049 fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: mon.smithi049@-1(???) e2 preinit fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: mon.smithi049@-1(???).mds e17 new map 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: mon.smithi049@-1(???).mds e17 print_map 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: e17 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: legacy client fscid: 1 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: Filesystem 'cephfs' (1) 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: fs_name cephfs 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: epoch 17 2024-05-15T15:58:11.727 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: created 2024-05-15T15:50:40.850889+0000 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: modified 2024-05-15T15:51:00.559034+0000 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: tableserver 0 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: root 0 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: session_timeout 60 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: session_autoclose 300 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: max_file_size 1099511627776 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: max_xattr_size 65536 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: required_client_features {} 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: last_failure 0 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: last_failure_osd_epoch 47 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: max_mds 2 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: in 0,1 2024-05-15T15:58:11.728 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: up {0=14484,1=24303} 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: failed 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: damaged 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: stopped 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: data_pools [3] 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: metadata_pool 2 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: inline_data enabled 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: balancer 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: bal_rank_mask -1 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: standby_count_wanted 1 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: qdb_cluster leader: 0 members: 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: [mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: [mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: [mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: [mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:11.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:58:11.730 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout: 2024-05-15T15:58:11.730 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: mon.smithi049@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-15T15:58:11.730 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: mon.smithi049@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-15T15:58:11.730 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: mon.smithi049@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-15T15:58:11.730 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: mon.smithi049@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-15T15:58:11.730 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 ceph-mon[112142]: mon.smithi049@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-05-15T15:58:11.730 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 bash[112124]: 05295c0417ca4abb31f8485a422428c93418f1a88cd789f5d49e2dc457bfcca8 2024-05-15T15:58:11.730 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:11 smithi049 systemd[1]: Started Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:58:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: pgmap v105: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: mon.smithi049 calling monitor election 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: mon.smithi049 is new leader, mons smithi049,smithi144 in quorum (ranks 0,1) 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: monmap epoch 2 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: last_changed 2024-05-15T15:46:22.117585+0000 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: created 2024-05-15T15:43:32.672725+0000 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: min_mon_release 18 (reef) 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: election_strategy: 1 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.smithi049 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-05-15T15:58:13.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:58:13.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: osdmap e49: 6 total, 6 up, 6 in 2024-05-15T15:58:13.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: mgrmap e32: smithi049.vuhzgf(active, since 3m), standbys: smithi144.whcjkr 2024-05-15T15:58:13.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:13.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:13.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:58:13.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: from='mgr.24483 ' entity='' 2024-05-15T15:58:13.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:12 smithi144 ceph-mon[36389]: mgrmap e33: smithi049.vuhzgf(active, since 3m), standbys: smithi144.whcjkr 2024-05-15T15:58:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:58:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: from='mgr.24483 172.21.15.49:0/2811657049' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: pgmap v105: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: mon.smithi049 calling monitor election 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: mon.smithi049 is new leader, mons smithi049,smithi144 in quorum (ranks 0,1) 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: monmap epoch 2 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: last_changed 2024-05-15T15:46:22.117585+0000 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: created 2024-05-15T15:43:32.672725+0000 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: min_mon_release 18 (reef) 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: election_strategy: 1 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.smithi049 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: osdmap e49: 6 total, 6 up, 6 in 2024-05-15T15:58:13.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: mgrmap e32: smithi049.vuhzgf(active, since 3m), standbys: smithi144.whcjkr 2024-05-15T15:58:13.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:13.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:13.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:58:13.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: from='mgr.24483 ' entity='' 2024-05-15T15:58:13.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:12 smithi049 ceph-mon[112142]: mgrmap e33: smithi049.vuhzgf(active, since 3m), standbys: smithi144.whcjkr 2024-05-15T15:58:20.089 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:19 smithi049 ceph-mon[112142]: Standby manager daemon smithi144.whcjkr restarted 2024-05-15T15:58:20.089 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:19 smithi049 ceph-mon[112142]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:58:20.089 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:19 smithi049 ceph-mon[112142]: from='mgr.? 172.21.15.144:0/1911369446' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:58:20.089 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:19 smithi049 ceph-mon[112142]: from='mgr.? 172.21.15.144:0/1911369446' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:58:20.090 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:19 smithi049 ceph-mon[112142]: from='mgr.? 172.21.15.144:0/1911369446' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:58:20.090 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:19 smithi049 ceph-mon[112142]: from='mgr.? 172.21.15.144:0/1911369446' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:58:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:19 smithi144 ceph-mon[36389]: Standby manager daemon smithi144.whcjkr restarted 2024-05-15T15:58:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:19 smithi144 ceph-mon[36389]: Standby manager daemon smithi144.whcjkr started 2024-05-15T15:58:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:19 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1911369446' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/crt"}]: dispatch 2024-05-15T15:58:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:19 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1911369446' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T15:58:20.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:19 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1911369446' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.whcjkr/key"}]: dispatch 2024-05-15T15:58:20.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:19 smithi144 ceph-mon[36389]: from='mgr.? 172.21.15.144:0/1911369446' entity='mgr.smithi144.whcjkr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: mgrmap e34: smithi049.vuhzgf(active, since 3m), standbys: smithi144.whcjkr 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: Active manager daemon smithi049.vuhzgf restarted 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: Activating manager daemon smithi049.vuhzgf 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: osdmap e50: 6 total, 6 up, 6 in 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: mgrmap e35: smithi049.vuhzgf(active, starting, since 0.00574083s), standbys: smithi144.whcjkr 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T15:58:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi144.whcjkr", "id": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: Manager daemon smithi049.vuhzgf is now available 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:58:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:20 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/trash_purge_schedule"}]: dispatch 2024-05-15T15:58:21.101 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: mgrmap e34: smithi049.vuhzgf(active, since 3m), standbys: smithi144.whcjkr 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: Active manager daemon smithi049.vuhzgf restarted 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: Activating manager daemon smithi049.vuhzgf 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: osdmap e50: 6 total, 6 up, 6 in 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: mgrmap e35: smithi049.vuhzgf(active, starting, since 0.00574083s), standbys: smithi144.whcjkr 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi049.vuhzgf", "id": "smithi049.vuhzgf"}]: dispatch 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr metadata", "who": "smithi144.whcjkr", "id": "smithi144.whcjkr"}]: dispatch 2024-05-15T15:58:21.102 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: Manager daemon smithi049.vuhzgf is now available 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/mirror_snapshot_schedule"}]: dispatch 2024-05-15T15:58:21.103 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi049.vuhzgf/trash_purge_schedule"}]: dispatch 2024-05-15T15:58:22.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:21 smithi049 ceph-mon[112142]: [15/May/2024:15:58:20] ENGINE Bus STARTING 2024-05-15T15:58:22.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:21 smithi049 ceph-mon[112142]: [15/May/2024:15:58:20] ENGINE Serving on https://172.21.15.49:7150 2024-05-15T15:58:22.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:21 smithi049 ceph-mon[112142]: [15/May/2024:15:58:20] ENGINE Serving on http://172.21.15.49:8765 2024-05-15T15:58:22.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:21 smithi049 ceph-mon[112142]: [15/May/2024:15:58:20] ENGINE Bus STARTED 2024-05-15T15:58:22.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:21 smithi049 ceph-mon[112142]: mgrmap e36: smithi049.vuhzgf(active, since 1.00963s), standbys: smithi144.whcjkr 2024-05-15T15:58:22.129 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:21 smithi144 ceph-mon[36389]: [15/May/2024:15:58:20] ENGINE Bus STARTING 2024-05-15T15:58:22.129 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:21 smithi144 ceph-mon[36389]: [15/May/2024:15:58:20] ENGINE Serving on https://172.21.15.49:7150 2024-05-15T15:58:22.129 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:21 smithi144 ceph-mon[36389]: [15/May/2024:15:58:20] ENGINE Serving on http://172.21.15.49:8765 2024-05-15T15:58:22.129 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:21 smithi144 ceph-mon[36389]: [15/May/2024:15:58:20] ENGINE Bus STARTED 2024-05-15T15:58:22.129 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:21 smithi144 ceph-mon[36389]: mgrmap e36: smithi049.vuhzgf(active, since 1.00963s), standbys: smithi144.whcjkr 2024-05-15T15:58:22.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:22 smithi144 ceph-mon[36389]: pgmap v3: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-15T15:58:23.166 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:22 smithi049 ceph-mon[112142]: pgmap v3: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-15T15:58:24.076 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:23 smithi049 ceph-mon[112142]: pgmap v4: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-15T15:58:24.077 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:23 smithi049 ceph-mon[112142]: mgrmap e37: smithi049.vuhzgf(active, since 2s), standbys: smithi144.whcjkr 2024-05-15T15:58:24.077 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:24.077 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:24.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:23 smithi144 ceph-mon[36389]: pgmap v4: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-15T15:58:24.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:23 smithi144 ceph-mon[36389]: mgrmap e37: smithi049.vuhzgf(active, since 2s), standbys: smithi144.whcjkr 2024-05-15T15:58:24.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:23 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:24.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:23 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:25.601 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:25 smithi049 ceph-mon[112142]: pgmap v5: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-15T15:58:25.601 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:25.602 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:25.602 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:25 smithi049 ceph-mon[112142]: mgrmap e38: smithi049.vuhzgf(active, since 4s), standbys: smithi144.whcjkr 2024-05-15T15:58:25.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:25 smithi144 ceph-mon[36389]: pgmap v5: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-15T15:58:25.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:25 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:25.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:25 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:25.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:25 smithi144 ceph-mon[36389]: mgrmap e38: smithi049.vuhzgf(active, since 4s), standbys: smithi144.whcjkr 2024-05-15T15:58:27.312 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:27 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:27.313 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:27 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:27.313 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:27 smithi144 ceph-mon[36389]: pgmap v6: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-15T15:58:27.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:27 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:27.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:27 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:27.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:27 smithi049 ceph-mon[112142]: pgmap v6: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-15T15:58:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:28 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:28 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:28 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:28 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:28.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:28 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:28.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:28 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:28.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:28 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:28.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:28 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:29.296 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:29 smithi049 ceph-mon[112142]: pgmap v7: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-05-15T15:58:29.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:29 smithi144 ceph-mon[36389]: pgmap v7: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-05-15T15:58:31.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:31 smithi144 ceph-mon[36389]: pgmap v8: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-15T15:58:31.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:31 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:31.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:31 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:31.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:31 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:31.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:31 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:58:31.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:31 smithi144 ceph-mon[36389]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:58:31.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:31 smithi144 ceph-mon[36389]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:58:31.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:31 smithi049 ceph-mon[112142]: pgmap v8: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-15T15:58:31.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:31 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:31.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:31 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:31 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:31 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:58:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:31 smithi049 ceph-mon[112142]: Updating smithi049:/etc/ceph/ceph.conf 2024-05-15T15:58:31.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:31 smithi049 ceph-mon[112142]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-15T15:58:33.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:58:33.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:32 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-15T15:58:33.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:58:33.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: Updating smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: Updating smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:58:33.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-15T15:58:34.078 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:33 smithi144 ceph-mon[36389]: pgmap v9: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-15T15:58:34.078 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:33 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:34.079 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:33 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:58:34.079 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:33 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:58:34.079 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:33 smithi144 ceph-mon[36389]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:34.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:33 smithi049 ceph-mon[112142]: pgmap v9: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-15T15:58:34.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:33 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:34.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:33 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:58:34.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:33 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:58:34.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:33 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:34.904 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:34 smithi144 ceph-mon[36389]: Upgrade: Updating mon.smithi144 2024-05-15T15:58:34.904 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:34 smithi144 ceph-mon[36389]: Deploying daemon mon.smithi144 on smithi144 2024-05-15T15:58:35.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:34 smithi049 ceph-mon[112142]: Upgrade: Updating mon.smithi144 2024-05-15T15:58:35.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:34 smithi049 ceph-mon[112142]: Deploying daemon mon.smithi144 on smithi144 2024-05-15T15:58:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:35 smithi144 systemd[1]: Stopping Ceph mon.smithi144 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T15:58:35.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:35 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144[36360]: 2024-05-15T15:58:35.430+0000 7f126b3fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi144 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T15:58:35.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:35 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144[36360]: 2024-05-15T15:58:35.430+0000 7f126b3fc700 -1 mon.smithi144@1(peon) e2 *** Got Signal Terminated *** 2024-05-15T15:58:35.893 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:35 smithi144 podman[86476]: 2024-05-15 15:58:35.605089879 +0000 UTC m=+0.274422064 container died bf0fd6a44ac8f2bc27fa7a7960e690c0104915e5b2bd6edf77ed1353a9eb20c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-2965081, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T15:58:36.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:35 smithi144 podman[86476]: 2024-05-15 15:58:35.989945662 +0000 UTC m=+0.659277836 container cleanup bf0fd6a44ac8f2bc27fa7a7960e690c0104915e5b2bd6edf77ed1353a9eb20c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=reef-2965081, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-15T15:58:36.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:35 smithi144 bash[86476]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144 2024-05-15T15:58:36.871 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:36 smithi144 podman[86488]: 2024-05-15 15:58:36.450545038 +0000 UTC m=+0.843966871 container remove bf0fd6a44ac8f2bc27fa7a7960e690c0104915e5b2bd6edf77ed1353a9eb20c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, RELEASE=reef-2965081, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T15:58:36.871 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:36 smithi144 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi144.service: Deactivated successfully. 2024-05-15T15:58:36.871 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:36 smithi144 systemd[1]: Stopped Ceph mon.smithi144 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:58:36.871 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:36 smithi144 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi144.service: Consumed 7.551s CPU time. 2024-05-15T15:58:37.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:36 smithi144 systemd[1]: Starting Ceph mon.smithi144 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T15:58:37.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 podman[86616]: 2024-05-15 15:58:36.987133398 +0000 UTC m=+0.017476787 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T15:58:37.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 podman[86616]: 2024-05-15 15:58:37.093495989 +0000 UTC m=+0.123839407 container create 2e8f6c238d313d2e56bceedea00eb0131c9d5d579ece4b290b81cd9d31322de2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-05-15T15:58:37.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 podman[86616]: 2024-05-15 15:58:37.287589404 +0000 UTC m=+0.317932784 container init 2e8f6c238d313d2e56bceedea00eb0131c9d5d579ece4b290b81cd9d31322de2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-15T15:58:37.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 podman[86616]: 2024-05-15 15:58:37.291016703 +0000 UTC m=+0.321360079 container start 2e8f6c238d313d2e56bceedea00eb0131c9d5d579ece4b290b81cd9d31322de2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-15T15:58:37.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: set uid:gid to 167:167 (ceph:ceph) 2024-05-15T15:58:37.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev), process ceph-mon, pid 2 2024-05-15T15:58:37.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: pidfile_write: ignore empty --pid-file 2024-05-15T15:58:37.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: load: jerasure load: lrc 2024-05-15T15:58:37.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: RocksDB version: 7.9.2 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Git sha 0 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Compile date 2024-05-15 11:18:19 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: DB SUMMARY 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: DB Session ID: GO6QZ737LTSF49BFG042 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: CURRENT file: CURRENT 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: IDENTITY file: IDENTITY 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1391 Bytes 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi144/store.db dir, Total Num: 1, files: 000027.sst 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi144/store.db: 000025.log size: 6501700 ; 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.error_if_exists: 0 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.create_if_missing: 0 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.paranoid_checks: 1 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-15T15:58:37.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.env: 0x559a69f25c60 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.fs: PosixFileSystem 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.info_log: 0x559a6b3e3980 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_file_opening_threads: 16 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.statistics: (nil) 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.use_fsync: 0 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_log_file_size: 0 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.keep_log_file_num: 1000 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.recycle_log_file_num: 0 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.allow_fallocate: 1 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.allow_mmap_reads: 0 2024-05-15T15:58:37.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.allow_mmap_writes: 0 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.use_direct_reads: 0 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.create_missing_column_families: 0 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.db_log_dir: 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.wal_dir: 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.advise_random_on_open: 1 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.db_write_buffer_size: 0 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.write_buffer_manager: 0x559a6b3b61e0 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-15T15:58:37.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.rate_limiter: (nil) 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.wal_recovery_mode: 2 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.enable_thread_tracking: 0 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.enable_pipelined_write: 0 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.unordered_write: 0 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.row_cache: None 2024-05-15T15:58:37.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.wal_filter: None 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.allow_ingest_behind: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.two_write_queues: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.manual_wal_flush: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.wal_compression: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.atomic_flush: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.log_readahead_size: 0 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-15T15:58:37.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.best_efforts_recovery: 0 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.allow_data_in_errors: 0 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.db_host_id: __hostname__ 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_background_jobs: 2 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_background_compactions: -1 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_subcompactions: 1 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_total_wal_size: 0 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-15T15:58:37.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_open_files: -1 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bytes_per_sync: 0 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_readahead_size: 0 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_background_flushes: -1 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Compression algorithms supported: 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: kZSTD supported: 0 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: kXpressCompression supported: 0 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: kLZ4HCCompression supported: 1 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: kZlibCompression supported: 1 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: kSnappyCompression supported: 1 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: kLZ4Compression supported: 1 2024-05-15T15:58:37.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: kBZip2Compression supported: 0 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi144/store.db/MANIFEST-000010 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.merge_operator: 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_filter: None 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_filter_factory: None 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.sst_partitioner_factory: None 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559a6b3e3a80) 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: cache_index_and_filter_blocks: 1 2024-05-15T15:58:37.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: pin_top_level_index_and_filter: 1 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_type: 0 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_block_index_type: 0 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_shortening: 1 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: checksum: 4 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: no_block_cache: 0 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache: 0x559a6b3e7090 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_name: BinnedLRUCache 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_options: 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: capacity : 536870912 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: num_shard_bits : 4 2024-05-15T15:58:37.710 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: strict_capacity_limit : 0 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: high_pri_pool_ratio: 0.000 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_compressed: (nil) 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: persistent_cache: (nil) 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_size: 4096 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_size_deviation: 10 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_restart_interval: 16 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_block_restart_interval: 1 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: metadata_block_size: 4096 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: partition_filters: 0 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: use_delta_encoding: 1 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: filter_policy: bloomfilter 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: whole_key_filtering: 1 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: verify_compression: 0 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: read_amp_bytes_per_bit: 0 2024-05-15T15:58:37.711 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: format_version: 5 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: enable_index_compression: 1 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_align: 0 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_auto_readahead_size: 262144 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: prepopulate_block_cache: 0 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: initial_auto_readahead_size: 8192 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: num_file_reads_for_auto_readahead: 2 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.write_buffer_size: 33554432 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_write_buffer_number: 2 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression: NoCompression 2024-05-15T15:58:37.712 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression: Disabled 2024-05-15T15:58:37.713 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.prefix_extractor: nullptr 2024-05-15T15:58:37.713 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-15T15:58:37.713 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.num_levels: 7 2024-05-15T15:58:37.713 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-15T15:58:37.713 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-15T15:58:37.713 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-15T15:58:37.713 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression_opts.level: 32767 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression_opts.strategy: 0 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-15T15:58:37.714 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression_opts.enabled: false 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.target_file_size_base: 67108864 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-15T15:58:37.715 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.arena_block_size: 1048576 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.disable_auto_compactions: 0 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-15T15:58:37.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.table_properties_collectors: 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.inplace_update_support: 0 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.bloom_locality: 0 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.max_successive_merges: 0 2024-05-15T15:58:37.717 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.paranoid_file_checks: 0 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.force_consistency_checks: 1 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.report_bg_io_stats: 0 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.ttl: 2592000 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.enable_blob_files: false 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.min_blob_size: 0 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.blob_file_size: 268435456 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-15T15:58:37.718 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.blob_file_starting_level: 0 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi144/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12150, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 90e1ecc4-6a35-411e-8195-7ed58cc7c8dc 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715788717320857, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-05-15T15:58:37.719 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715788717339908, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3925011, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12155, "largest_seqno": 14854, "table_properties": {"data_size": 3914016, "index_size": 6838, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3141, "raw_key_size": 26937, "raw_average_key_size": 21, "raw_value_size": 3889738, "raw_average_value_size": 3164, "num_data_blocks": 310, "num_entries": 1229, "num_filter_entries": 1229, "num_deletions": 257, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715788717, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "90e1ecc4-6a35-411e-8195-7ed58cc7c8dc", "db_session_id": "GO6QZ737LTSF49BFG042", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715788717340022, "job": 1, "event": "recovery_finished"} 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi144/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559a6b4d8000 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: DB pointer 0x559a6b4c2000 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** DB Stats ** 2024-05-15T15:58:37.720 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** Compaction Stats [default] ** 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: L0 1/0 3.74 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 492.6 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: L6 1/0 6.79 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Sum 2/0 10.53 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 492.6 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 492.6 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-05-15T15:58:37.721 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** Compaction Stats [default] ** 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 492.6 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-15T15:58:37.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative compaction: 0.00 GB write, 169.34 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval compaction: 0.00 GB write, 169.34 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Block cache BinnedLRUCache@0x559a6b3e7090#2 capacity: 512.00 MB usage: 11.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,3.28 KB,0.000625849%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: starting mon.smithi144 rank 1 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi144 fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: mon.smithi144@-1(???) e2 preinit fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: mon.smithi144@-1(???).mds e17 new map 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: mon.smithi144@-1(???).mds e17 print_map 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: e17 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: legacy client fscid: 1 2024-05-15T15:58:37.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Filesystem 'cephfs' (1) 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: fs_name cephfs 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: epoch 17 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: created 2024-05-15T15:50:40.850889+0000 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: modified 2024-05-15T15:51:00.559034+0000 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: tableserver 0 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: root 0 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: session_timeout 60 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: session_autoclose 300 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_file_size 1099511627776 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_xattr_size 65536 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: required_client_features {} 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: last_failure 0 2024-05-15T15:58:37.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: last_failure_osd_epoch 47 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_mds 2 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: in 0,1 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: up {0=14484,1=24303} 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: failed 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: damaged 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: stopped 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_pools [3] 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: metadata_pool 2 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: inline_data enabled 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: balancer 2024-05-15T15:58:37.725 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: bal_rank_mask -1 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: standby_count_wanted 1 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: qdb_cluster leader: 0 members: 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: [mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: [mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: [mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: [mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: mon.smithi144@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: mon.smithi144@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: mon.smithi144@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: mon.smithi144@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 ceph-mon[86635]: mon.smithi144@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 bash[86616]: 2e8f6c238d313d2e56bceedea00eb0131c9d5d579ece4b290b81cd9d31322de2 2024-05-15T15:58:37.726 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:37 smithi144 systemd[1]: Started Ceph mon.smithi144 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:58:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.767+0000 7fefa5e6c700 1 -- 172.21.15.49:0/2646736099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 msgr2=0x7fefa010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.767+0000 7fefa5e6c700 1 --2- 172.21.15.49:0/2646736099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 0x7fefa010ac60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fef94009910 tx=0x7fef94009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.767+0000 7fefa5e6c700 1 -- 172.21.15.49:0/2646736099 shutdown_connections 2024-05-15T15:58:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.767+0000 7fefa5e6c700 1 --2- 172.21.15.49:0/2646736099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 0x7fefa010ac60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.767+0000 7fefa5e6c700 1 --2- 172.21.15.49:0/2646736099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 0x7fefa0102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.767+0000 7fefa5e6c700 1 -- 172.21.15.49:0/2646736099 >> 172.21.15.49:0/2646736099 conn(0x7fefa00fb6a0 msgr2=0x7fefa00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.768+0000 7fefa5e6c700 1 -- 172.21.15.49:0/2646736099 shutdown_connections 2024-05-15T15:58:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.768+0000 7fefa5e6c700 1 -- 172.21.15.49:0/2646736099 wait complete. 2024-05-15T15:58:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.768+0000 7fefa5e6c700 1 Processor -- start 2024-05-15T15:58:37.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fefa5e6c700 1 -- start start 2024-05-15T15:58:37.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fefa5e6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 0x7fefa019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:37.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fefa5e6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 0x7fefa019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:37.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fefa5e6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa019d490 con 0x7fefa0102770 2024-05-15T15:58:37.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fefa5e6c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0196a70 con 0x7fefa0101e50 2024-05-15T15:58:37.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 0x7fefa019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:37.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 0x7fefa019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58336/0 (socket says 172.21.15.49:58336) 2024-05-15T15:58:37.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9effd700 1 -- 172.21.15.49:0/3281430173 learned_addr learned my addr 172.21.15.49:0/3281430173 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9f7fe700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 0x7fefa019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9f7fe700 1 -- 172.21.15.49:0/3281430173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 msgr2=0x7fefa019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9f7fe700 1 -- 172.21.15.49:0/3281430173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 msgr2=0x7fefa019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9f7fe700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 0x7fefa019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9f7fe700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 0x7fefa019c930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9effd700 1 -- 172.21.15.49:0/3281430173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 msgr2=0x7fefa019c930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9effd700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 0x7fefa019c930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.769+0000 7fef9effd700 1 -- 172.21.15.49:0/3281430173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940095f0 con 0x7fefa0102770 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.770+0000 7fef9effd700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 0x7fefa019ce70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fef94000c00 tx=0x7fef9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.770+0000 7fef9cff9700 1 -- 172.21.15.49:0/3281430173 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef9401d070 con 0x7fefa0102770 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.770+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa0196cf0 con 0x7fefa0102770 2024-05-15T15:58:37.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.770+0000 7fef9cff9700 1 -- 172.21.15.49:0/3281430173 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef94005700 con 0x7fefa0102770 2024-05-15T15:58:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.770+0000 7fef9cff9700 1 -- 172.21.15.49:0/3281430173 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef9400fe50 con 0x7fefa0102770 2024-05-15T15:58:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.770+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa01971e0 con 0x7fefa0102770 2024-05-15T15:58:37.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.771+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa0105570 con 0x7fefa0102770 2024-05-15T15:58:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.771+0000 7fef9cff9700 1 -- 172.21.15.49:0/3281430173 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fef9400f640 con 0x7fefa0102770 2024-05-15T15:58:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.772+0000 7fef9cff9700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef8806e330 0x7fef880707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.772+0000 7fef9cff9700 1 -- 172.21.15.49:0/3281430173 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fef9408f760 con 0x7fefa0102770 2024-05-15T15:58:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.772+0000 7fef9f7fe700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef8806e330 0x7fef880707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:37.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.773+0000 7fef9f7fe700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef8806e330 0x7fef880707f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fef90005920 tx=0x7fef9000b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:37.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:37.777+0000 7fef9cff9700 1 -- 172.21.15.49:0/3281430173 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fef9405c400 con 0x7fefa0102770 2024-05-15T15:58:38.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.008+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fefa0197fa0 con 0x7fef8806e330 2024-05-15T15:58:38.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.009+0000 7fef9cff9700 1 -- 172.21.15.49:0/3281430173 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fefa0197fa0 con 0x7fef8806e330 2024-05-15T15:58:38.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef8806e330 msgr2=0x7fef880707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef8806e330 0x7fef880707f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fef90005920 tx=0x7fef9000b2c0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 msgr2=0x7fefa019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 0x7fefa019ce70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fef94000c00 tx=0x7fef9400be20 comp rx=0 tx=0).stop 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 shutdown_connections 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef8806e330 0x7fef880707f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa0101e50 0x7fefa019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 --2- 172.21.15.49:0/3281430173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0102770 0x7fefa019ce70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 >> 172.21.15.49:0/3281430173 conn(0x7fefa00fb6a0 msgr2=0x7fefa0103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 shutdown_connections 2024-05-15T15:58:38.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.012+0000 7fefa5e6c700 1 -- 172.21.15.49:0/3281430173 wait complete. 2024-05-15T15:58:38.025 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:58:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.122+0000 7ff6597e9700 1 -- 172.21.15.49:0/2352012208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 msgr2=0x7ff654105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.122+0000 7ff6597e9700 1 --2- 172.21.15.49:0/2352012208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 0x7ff654105900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff648009910 tx=0x7ff648009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.122+0000 7ff6597e9700 1 -- 172.21.15.49:0/2352012208 shutdown_connections 2024-05-15T15:58:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.122+0000 7ff6597e9700 1 --2- 172.21.15.49:0/2352012208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 0x7ff654105900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.122+0000 7ff6597e9700 1 --2- 172.21.15.49:0/2352012208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 0x7ff6541012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.122+0000 7ff6597e9700 1 -- 172.21.15.49:0/2352012208 >> 172.21.15.49:0/2352012208 conn(0x7ff6540fc7d0 msgr2=0x7ff6540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:38.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.122+0000 7ff6597e9700 1 -- 172.21.15.49:0/2352012208 shutdown_connections 2024-05-15T15:58:38.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.122+0000 7ff6597e9700 1 -- 172.21.15.49:0/2352012208 wait complete. 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff6597e9700 1 Processor -- start 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff6597e9700 1 -- start start 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff6597e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 0x7ff65419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff6597e9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 0x7ff65419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff6597e9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65419d440 con 0x7ff6541018b0 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff6597e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff654196a20 con 0x7ff654100f00 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff6527fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 0x7ff65419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff6527fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 0x7ff65419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58348/0 (socket says 172.21.15.49:58348) 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff6527fc700 1 -- 172.21.15.49:0/1755685184 learned_addr learned my addr 172.21.15.49:0/1755685184 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.123+0000 7ff652ffd700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 0x7ff65419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:38.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff652ffd700 1 -- 172.21.15.49:0/1755685184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 msgr2=0x7ff65419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff652ffd700 1 -- 172.21.15.49:0/1755685184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 msgr2=0x7ff65419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff652ffd700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 0x7ff65419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff652ffd700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 0x7ff65419c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff6527fc700 1 -- 172.21.15.49:0/1755685184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 msgr2=0x7ff65419c8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff6527fc700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 0x7ff65419c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff6527fc700 1 -- 172.21.15.49:0/1755685184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480095f0 con 0x7ff6541018b0 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff6527fc700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 0x7ff65419ce20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff648000c00 tx=0x7ff64800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff63bfff700 1 -- 172.21.15.49:0/1755685184 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64801d070 con 0x7ff6541018b0 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff63bfff700 1 -- 172.21.15.49:0/1755685184 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff648005700 con 0x7ff6541018b0 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff654196ca0 con 0x7ff6541018b0 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff63bfff700 1 -- 172.21.15.49:0/1755685184 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64800fe50 con 0x7ff6541018b0 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.124+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff654197190 con 0x7ff6541018b0 2024-05-15T15:58:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.125+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff654109300 con 0x7ff6541018b0 2024-05-15T15:58:38.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.126+0000 7ff63bfff700 1 -- 172.21.15.49:0/1755685184 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff64800f640 con 0x7ff6541018b0 2024-05-15T15:58:38.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.126+0000 7ff63bfff700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff63c06e330 0x7ff63c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:38.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.126+0000 7ff63bfff700 1 -- 172.21.15.49:0/1755685184 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff64808f7a0 con 0x7ff6541018b0 2024-05-15T15:58:38.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.126+0000 7ff652ffd700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff63c06e330 0x7ff63c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:38.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.127+0000 7ff652ffd700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff63c06e330 0x7ff63c0707f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff644005920 tx=0x7ff64400b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:38.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.131+0000 7ff63bfff700 1 -- 172.21.15.49:0/1755685184 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff64805c420 con 0x7ff6541018b0 2024-05-15T15:58:38.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.357+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff654197f50 con 0x7ff63c06e330 2024-05-15T15:58:38.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.358+0000 7ff63bfff700 1 -- 172.21.15.49:0/1755685184 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7ff654197f50 con 0x7ff63c06e330 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff63c06e330 msgr2=0x7ff63c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff63c06e330 0x7ff63c0707f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff644005920 tx=0x7ff64400b2c0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 msgr2=0x7ff65419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 0x7ff65419ce20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff648000c00 tx=0x7ff64800be20 comp rx=0 tx=0).stop 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 shutdown_connections 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff63c06e330 0x7ff63c0707f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff654100f00 0x7ff65419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 --2- 172.21.15.49:0/1755685184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6541018b0 0x7ff65419ce20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 >> 172.21.15.49:0/1755685184 conn(0x7ff6540fc7d0 msgr2=0x7ff6540fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 shutdown_connections 2024-05-15T15:58:38.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.360+0000 7ff6597e9700 1 -- 172.21.15.49:0/1755685184 wait complete. 2024-05-15T15:58:38.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.469+0000 7f5b51ddd700 1 -- 172.21.15.49:0/3430616742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ff430 msgr2=0x7f5b4c0ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.469+0000 7f5b51ddd700 1 --2- 172.21.15.49:0/3430616742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ff430 0x7f5b4c0ff810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009910 tx=0x7f5b3c009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:38.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.469+0000 7f5b51ddd700 1 -- 172.21.15.49:0/3430616742 shutdown_connections 2024-05-15T15:58:38.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.469+0000 7f5b51ddd700 1 --2- 172.21.15.49:0/3430616742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b4c0ffde0 0x7f5b4c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.469+0000 7f5b51ddd700 1 --2- 172.21.15.49:0/3430616742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ff430 0x7f5b4c0ff810 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.469+0000 7f5b51ddd700 1 -- 172.21.15.49:0/3430616742 >> 172.21.15.49:0/3430616742 conn(0x7f5b4c076000 msgr2=0x7f5b4c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:38.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.469+0000 7f5b51ddd700 1 -- 172.21.15.49:0/3430616742 shutdown_connections 2024-05-15T15:58:38.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.469+0000 7f5b51ddd700 1 -- 172.21.15.49:0/3430616742 wait complete. 2024-05-15T15:58:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b51ddd700 1 Processor -- start 2024-05-15T15:58:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b51ddd700 1 -- start start 2024-05-15T15:58:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b51ddd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b4c0ff430 0x7f5b4c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b51ddd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ffde0 0x7f5b4c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b51ddd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c1978c0 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b51ddd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c19a530 con 0x7f5b4c0ff430 2024-05-15T15:58:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b42ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ffde0 0x7f5b4c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b42ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ffde0 0x7f5b4c197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58376/0 (socket says 172.21.15.49:58376) 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b42ffd700 1 -- 172.21.15.49:0/1710546242 learned_addr learned my addr 172.21.15.49:0/1710546242 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b42ffd700 1 -- 172.21.15.49:0/1710546242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b4c0ff430 msgr2=0x7f5b4c196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b42ffd700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b4c0ff430 0x7f5b4c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.470+0000 7f5b42ffd700 1 -- 172.21.15.49:0/1710546242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c0095f0 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.471+0000 7f5b42ffd700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ffde0 0x7f5b4c197230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400b4d0 tx=0x7f5b3400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.471+0000 7f5b497fa700 1 -- 172.21.15.49:0/1710546242 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b34011800 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.471+0000 7f5b497fa700 1 -- 172.21.15.49:0/1710546242 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b34011e20 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.471+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c19a810 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.471+0000 7f5b497fa700 1 -- 172.21.15.49:0/1710546242 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b3400f2e0 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.471+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c19ad60 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.472+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c1092c0 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.473+0000 7f5b497fa700 1 -- 172.21.15.49:0/1710546242 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f5b34010380 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.473+0000 7f5b497fa700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b3806e340 0x7f5b38070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.473+0000 7f5b497fa700 1 -- 172.21.15.49:0/1710546242 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5b3408de40 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.473+0000 7f5b4b7fe700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b3806e340 0x7f5b38070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:38.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.474+0000 7f5b4b7fe700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b3806e340 0x7f5b38070800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00b5c0 tx=0x7f5b3c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:38.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.478+0000 7f5b497fa700 1 -- 172.21.15.49:0/1710546242 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5b340570c0 con 0x7f5b4c0ffde0 2024-05-15T15:58:38.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.693+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b4c10df60 con 0x7f5b3806e340 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.705+0000 7f5b497fa700 1 -- 172.21.15.49:0/1710546242 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f5b4c10df60 con 0x7f5b3806e340 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (114s) 12s ago 13m 17.3M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (13m) 12s ago 13m 8342k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (12m) 15s ago 12m 8066k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (13m) 12s ago 13m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (12m) 15s ago 12m 6660k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (87s) 12s ago 13m 74.6M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (7m) 12s ago 7m 90.2M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (7m) 12s ago 7m 152M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (7m) 15s ago 7m 26.6M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (7m) 15s ago 7m 28.4M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:58:38.708 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (4m) 12s ago 14m 476M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (3m) 15s ago 12m 449M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (27s) 12s ago 15m 28.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 starting - - - 2048M 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (2m) 12s ago 13m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (2m) 15s ago 12m 16.2M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (11m) 12s ago 11m 219M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (11m) 12s ago 11m 219M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (10m) 12s ago 10m 213M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (10m) 15s ago 10m 260M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (9m) 15s ago 9m 218M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (9m) 15s ago 9m 202M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:58:38.709 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (2m) 12s ago 12m 60.1M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T15:58:38.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b3806e340 msgr2=0x7f5b38070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b3806e340 0x7f5b38070800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00b5c0 tx=0x7f5b3c000bc0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ffde0 msgr2=0x7f5b4c197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ffde0 0x7f5b4c197230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400b4d0 tx=0x7f5b3400b890 comp rx=0 tx=0).stop 2024-05-15T15:58:38.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 shutdown_connections 2024-05-15T15:58:38.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b3806e340 0x7f5b38070800 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b4c0ff430 0x7f5b4c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 --2- 172.21.15.49:0/1710546242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b4c0ffde0 0x7f5b4c197230 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 >> 172.21.15.49:0/1710546242 conn(0x7f5b4c076000 msgr2=0x7f5b4c104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:38.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 shutdown_connections 2024-05-15T15:58:38.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.707+0000 7f5b51ddd700 1 -- 172.21.15.49:0/1710546242 wait complete. 2024-05-15T15:58:38.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.820+0000 7f6b74a28700 1 -- 172.21.15.49:0/1382966340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b70100790 msgr2=0x7f6b701047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:38.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.820+0000 7f6b74a28700 1 --2- 172.21.15.49:0/1382966340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b70100790 0x7f6b701047e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009910 tx=0x7f6b58009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:38.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.820+0000 7f6b74a28700 1 -- 172.21.15.49:0/1382966340 shutdown_connections 2024-05-15T15:58:38.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.820+0000 7f6b74a28700 1 --2- 172.21.15.49:0/1382966340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b70100790 0x7f6b701047e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.820+0000 7f6b74a28700 1 --2- 172.21.15.49:0/1382966340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b700ffde0 0x7f6b701001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.820+0000 7f6b74a28700 1 -- 172.21.15.49:0/1382966340 >> 172.21.15.49:0/1382966340 conn(0x7f6b700fb690 msgr2=0x7f6b700fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:38.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.820+0000 7f6b74a28700 1 -- 172.21.15.49:0/1382966340 shutdown_connections 2024-05-15T15:58:38.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.820+0000 7f6b74a28700 1 -- 172.21.15.49:0/1382966340 wait complete. 2024-05-15T15:58:38.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.821+0000 7f6b74a28700 1 Processor -- start 2024-05-15T15:58:38.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.821+0000 7f6b74a28700 1 -- start start 2024-05-15T15:58:38.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.821+0000 7f6b74a28700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b700ffde0 0x7f6b70197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:38.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.821+0000 7f6b74a28700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b70100790 0x7f6b70197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.821+0000 7f6b74a28700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70198530 con 0x7f6b700ffde0 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.821+0000 7f6b74a28700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70198670 con 0x7f6b70100790 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b6e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b700ffde0 0x7f6b70197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b6e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b700ffde0 0x7f6b70197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58398/0 (socket says 172.21.15.49:58398) 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b6e59c700 1 -- 172.21.15.49:0/3406995778 learned_addr learned my addr 172.21.15.49:0/3406995778 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b6e59c700 1 -- 172.21.15.49:0/3406995778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b70100790 msgr2=0x7f6b70197f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b6e59c700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b70100790 0x7f6b70197f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b6e59c700 1 -- 172.21.15.49:0/3406995778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b580095f0 con 0x7f6b700ffde0 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b6e59c700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b700ffde0 0x7f6b70197a20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000d8d0 tx=0x7f6b6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b67fff700 1 -- 172.21.15.49:0/3406995778 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b60009940 con 0x7f6b700ffde0 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b67fff700 1 -- 172.21.15.49:0/3406995778 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b60010460 con 0x7f6b700ffde0 2024-05-15T15:58:38.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b701a0a90 con 0x7f6b700ffde0 2024-05-15T15:58:38.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b67fff700 1 -- 172.21.15.49:0/3406995778 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b6000f5f0 con 0x7f6b700ffde0 2024-05-15T15:58:38.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.822+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b701a0ea0 con 0x7f6b700ffde0 2024-05-15T15:58:38.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.823+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b701081e0 con 0x7f6b700ffde0 2024-05-15T15:58:38.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.824+0000 7f6b67fff700 1 -- 172.21.15.49:0/3406995778 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f6b60009aa0 con 0x7f6b700ffde0 2024-05-15T15:58:38.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.824+0000 7f6b67fff700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b5c06e340 0x7f6b5c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:38.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.824+0000 7f6b67fff700 1 -- 172.21.15.49:0/3406995778 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6b6008e380 con 0x7f6b700ffde0 2024-05-15T15:58:38.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.825+0000 7f6b65d9b700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b5c06e340 0x7f6b5c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:38.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.825+0000 7f6b65d9b700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b5c06e340 0x7f6b5c070800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b58006010 tx=0x7f6b58005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:38.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:38.830+0000 7f6b67fff700 1 -- 172.21.15.49:0/3406995778 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6b600574f0 con 0x7f6b700ffde0 2024-05-15T15:58:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:38 smithi049 ceph-mon[112142]: pgmap v10: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-15T15:58:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:38 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:58:39.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.127+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b7010cd80 con 0x7f6b700ffde0 2024-05-15T15:58:39.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.128+0000 7f6b67fff700 1 -- 172.21.15.49:0/3406995778 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f6b6005abb0 con 0x7f6b700ffde0 2024-05-15T15:58:39.130 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:58:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:58:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 10, 2024-05-15T15:58:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T15:58:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:58:39.132 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b5c06e340 msgr2=0x7f6b5c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b5c06e340 0x7f6b5c070800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b58006010 tx=0x7f6b58005960 comp rx=0 tx=0).stop 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b700ffde0 msgr2=0x7f6b70197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b700ffde0 0x7f6b70197a20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000d8d0 tx=0x7f6b6000dc90 comp rx=0 tx=0).stop 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 shutdown_connections 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b5c06e340 0x7f6b5c070800 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b700ffde0 0x7f6b70197a20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 --2- 172.21.15.49:0/3406995778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b70100790 0x7f6b70197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 >> 172.21.15.49:0/3406995778 conn(0x7f6b700fb690 msgr2=0x7f6b700fba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 shutdown_connections 2024-05-15T15:58:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.130+0000 7f6b74a28700 1 -- 172.21.15.49:0/3406995778 wait complete. 2024-05-15T15:58:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:38 smithi144 ceph-mon[86635]: pgmap v10: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-15T15:58:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:38 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:58:39.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 -- 172.21.15.49:0/893855921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 msgr2=0x7f63d41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 --2- 172.21.15.49:0/893855921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 0x7f63d41012c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009910 tx=0x7f63d0009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 -- 172.21.15.49:0/893855921 shutdown_connections 2024-05-15T15:58:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 --2- 172.21.15.49:0/893855921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63d4101890 0x7f63d41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 --2- 172.21.15.49:0/893855921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 0x7f63d41012c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 -- 172.21.15.49:0/893855921 >> 172.21.15.49:0/893855921 conn(0x7f63d40fc790 msgr2=0x7f63d40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 -- 172.21.15.49:0/893855921 shutdown_connections 2024-05-15T15:58:39.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 -- 172.21.15.49:0/893855921 wait complete. 2024-05-15T15:58:39.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 Processor -- start 2024-05-15T15:58:39.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.239+0000 7f63dbc07700 1 -- start start 2024-05-15T15:58:39.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63dbc07700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 0x7f63d40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:39.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63dbc07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63d4101890 0x7f63d40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:39.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63dbc07700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4073d20 con 0x7f63d4100ee0 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63dbc07700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4073e60 con 0x7f63d4101890 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63d99a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 0x7f63d40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63d99a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 0x7f63d40732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58424/0 (socket says 172.21.15.49:58424) 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63d99a3700 1 -- 172.21.15.49:0/1442964111 learned_addr learned my addr 172.21.15.49:0/1442964111 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63d91a2700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63d4101890 0x7f63d40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63d99a3700 1 -- 172.21.15.49:0/1442964111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63d4101890 msgr2=0x7f63d40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63d99a3700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63d4101890 0x7f63d40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63d99a3700 1 -- 172.21.15.49:0/1442964111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d00095f0 con 0x7f63d4100ee0 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63d99a3700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 0x7f63d40732a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63d0000c00 tx=0x7f63d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63caffd700 1 -- 172.21.15.49:0/1442964111 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d001d070 con 0x7f63d4100ee0 2024-05-15T15:58:39.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63caffd700 1 -- 172.21.15.49:0/1442964111 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63d00056e0 con 0x7f63d4100ee0 2024-05-15T15:58:39.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d40740e0 con 0x7f63d4100ee0 2024-05-15T15:58:39.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.240+0000 7f63caffd700 1 -- 172.21.15.49:0/1442964111 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d000fe80 con 0x7f63d4100ee0 2024-05-15T15:58:39.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.241+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d41a3040 con 0x7f63d4100ee0 2024-05-15T15:58:39.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.241+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d4109280 con 0x7f63d4100ee0 2024-05-15T15:58:39.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.242+0000 7f63caffd700 1 -- 172.21.15.49:0/1442964111 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f63d000f6f0 con 0x7f63d4100ee0 2024-05-15T15:58:39.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.242+0000 7f63caffd700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f63c006e330 0x7f63c00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:39.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.242+0000 7f63caffd700 1 -- 172.21.15.49:0/1442964111 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f63d008f6e0 con 0x7f63d4100ee0 2024-05-15T15:58:39.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.243+0000 7f63d91a2700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f63c006e330 0x7f63c00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:39.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.243+0000 7f63d91a2700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f63c006e330 0x7f63c00707f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005950 tx=0x7f63c40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:39.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.247+0000 7f63caffd700 1 -- 172.21.15.49:0/1442964111 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f63d00588d0 con 0x7f63d4100ee0 2024-05-15T15:58:39.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.497+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f63d40747f0 con 0x7f63d4100ee0 2024-05-15T15:58:39.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.497+0000 7f63caffd700 1 -- 172.21.15.49:0/1442964111 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f63d005bf90 con 0x7f63d4100ee0 2024-05-15T15:58:39.500 INFO:teuthology.orchestra.run.smithi049.stdout:e17 2024-05-15T15:58:39.500 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:58:39.500 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:58:39.500 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 17 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:51:00.559034+0000 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T15:58:39.501 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T15:58:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T15:58:39.503 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T15:58:39.503 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T15:58:39.503 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:39.503 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:39.503 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:39.503 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:58:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:58:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:58:39.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.499+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f63c006e330 msgr2=0x7f63c00707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.499+0000 7f63dbc07700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f63c006e330 0x7f63c00707f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005950 tx=0x7f63c40058e0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.500+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 msgr2=0x7f63d40732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.500+0000 7f63dbc07700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 0x7f63d40732a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63d0000c00 tx=0x7f63d000be20 comp rx=0 tx=0).stop 2024-05-15T15:58:39.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.500+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 shutdown_connections 2024-05-15T15:58:39.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.500+0000 7f63dbc07700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f63c006e330 0x7f63c00707f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.500+0000 7f63dbc07700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63d4100ee0 0x7f63d40732a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.500+0000 7f63dbc07700 1 --2- 172.21.15.49:0/1442964111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63d4101890 0x7f63d40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.500+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 >> 172.21.15.49:0/1442964111 conn(0x7f63d40fc790 msgr2=0x7f63d40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:39.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.500+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 shutdown_connections 2024-05-15T15:58:39.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.500+0000 7f63dbc07700 1 -- 172.21.15.49:0/1442964111 wait complete. 2024-05-15T15:58:39.505 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T15:58:39.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.619+0000 7f659ab0e700 1 -- 172.21.15.49:0/1424034883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 msgr2=0x7f65940ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.619+0000 7f659ab0e700 1 --2- 172.21.15.49:0/1424034883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 0x7f65940ffad0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6588009910 tx=0x7f6588009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:39.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.619+0000 7f659ab0e700 1 -- 172.21.15.49:0/1424034883 shutdown_connections 2024-05-15T15:58:39.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.619+0000 7f659ab0e700 1 --2- 172.21.15.49:0/1424034883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 0x7f6594103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.619+0000 7f659ab0e700 1 --2- 172.21.15.49:0/1424034883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 0x7f65940ffad0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.619+0000 7f659ab0e700 1 -- 172.21.15.49:0/1424034883 >> 172.21.15.49:0/1424034883 conn(0x7f6594074e70 msgr2=0x7f6594075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:39.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.619+0000 7f659ab0e700 1 -- 172.21.15.49:0/1424034883 shutdown_connections 2024-05-15T15:58:39.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.619+0000 7f659ab0e700 1 -- 172.21.15.49:0/1424034883 wait complete. 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.620+0000 7f659ab0e700 1 Processor -- start 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.620+0000 7f659ab0e700 1 -- start start 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.620+0000 7f659ab0e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 0x7f659419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.620+0000 7f659ab0e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 0x7f659419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.620+0000 7f659ab0e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594196940 con 0x7f65940ff6f0 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.620+0000 7f659ab0e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594196ab0 con 0x7f65941000a0 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.620+0000 7f65988aa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 0x7f659419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.620+0000 7f65988aa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 0x7f659419c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58442/0 (socket says 172.21.15.49:58442) 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.620+0000 7f65988aa700 1 -- 172.21.15.49:0/3583071169 learned_addr learned my addr 172.21.15.49:0/3583071169 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f6593fff700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 0x7f659419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:39.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f6593fff700 1 -- 172.21.15.49:0/3583071169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 msgr2=0x7f659419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f6593fff700 1 -- 172.21.15.49:0/3583071169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 msgr2=0x7f659419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f6593fff700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 0x7f659419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f6593fff700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 0x7f659419cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f65988aa700 1 -- 172.21.15.49:0/3583071169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 msgr2=0x7f659419cdd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f65988aa700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 0x7f659419cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f65988aa700 1 -- 172.21.15.49:0/3583071169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65880095f0 con 0x7f65940ff6f0 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f65988aa700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 0x7f659419c890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6588000c00 tx=0x7f658800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f6591ffb700 1 -- 172.21.15.49:0/3583071169 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f658801d070 con 0x7f65940ff6f0 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f6591ffb700 1 -- 172.21.15.49:0/3583071169 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65880056e0 con 0x7f65940ff6f0 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f6591ffb700 1 -- 172.21.15.49:0/3583071169 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f658800fe80 con 0x7f65940ff6f0 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.621+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6594196d30 con 0x7f65940ff6f0 2024-05-15T15:58:39.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.622+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65941972a0 con 0x7f65940ff6f0 2024-05-15T15:58:39.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.623+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6594109920 con 0x7f65940ff6f0 2024-05-15T15:58:39.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.623+0000 7f6591ffb700 1 -- 172.21.15.49:0/3583071169 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f658800f6f0 con 0x7f65940ff6f0 2024-05-15T15:58:39.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.624+0000 7f6591ffb700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f658406e330 0x7f65840707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:39.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.624+0000 7f6591ffb700 1 -- 172.21.15.49:0/3583071169 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f658808f6c0 con 0x7f65940ff6f0 2024-05-15T15:58:39.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.624+0000 7f6593fff700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f658406e330 0x7f65840707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:39.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.624+0000 7f6593fff700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f658406e330 0x7f65840707f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6580005950 tx=0x7f65800058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:39.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.629+0000 7f6591ffb700 1 -- 172.21.15.49:0/3583071169 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f65880588b0 con 0x7f65940ff6f0 2024-05-15T15:58:39.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.858+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6594197e70 con 0x7f658406e330 2024-05-15T15:58:39.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.859+0000 7f6591ffb700 1 -- 172.21.15.49:0/3583071169 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f6594197e70 con 0x7f658406e330 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr" 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "3/23 daemons upgraded", 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mon daemons", 2024-05-15T15:58:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:58:39.863 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:58:39.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f658406e330 msgr2=0x7f65840707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f658406e330 0x7f65840707f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6580005950 tx=0x7f65800058e0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 msgr2=0x7f659419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 0x7f659419c890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6588000c00 tx=0x7f658800be20 comp rx=0 tx=0).stop 2024-05-15T15:58:39.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 shutdown_connections 2024-05-15T15:58:39.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f658406e330 0x7f65840707f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65940ff6f0 0x7f659419c890 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 --2- 172.21.15.49:0/3583071169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65941000a0 0x7f659419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 >> 172.21.15.49:0/3583071169 conn(0x7f6594074e70 msgr2=0x7f65940fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:39.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 shutdown_connections 2024-05-15T15:58:39.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.861+0000 7f659ab0e700 1 -- 172.21.15.49:0/3583071169 wait complete. 2024-05-15T15:58:39.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:39.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: pgmap v12: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-15T15:58:39.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:39.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:39.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-05-15T15:58:39.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:58:39.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: osdmap e50: 6 total, 6 up, 6 in 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: mgrmap e38: smithi049.vuhzgf(active, since 18s), standbys: smithi144.whcjkr 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: mon.smithi144 calling monitor election 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: mon.smithi049 calling monitor election 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: mon.smithi049 is new leader, mons smithi049,smithi144 in quorum (ranks 0,1) 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: monmap epoch 3 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: last_changed 2024-05-15T15:58:38.784516+0000 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: created 2024-05-15T15:43:32.672725+0000 2024-05-15T15:58:39.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: min_mon_release 19 (squid) 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: election_strategy: 1 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.smithi049 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: osdmap e50: 6 total, 6 up, 6 in 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: mgrmap e38: smithi049.vuhzgf(active, since 18s), standbys: smithi144.whcjkr 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:39.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:39.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:39.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3406995778' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:58:39.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:39 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1442964111' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:58:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.969+0000 7f0b01af1700 1 -- 172.21.15.49:0/3773806922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc0ffdf0 msgr2=0x7f0afc1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.969+0000 7f0b01af1700 1 --2- 172.21.15.49:0/3773806922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc0ffdf0 0x7f0afc1001d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009910 tx=0x7f0aec009c20 comp rx=0 tx=0).stop 2024-05-15T15:58:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.969+0000 7f0b01af1700 1 -- 172.21.15.49:0/3773806922 shutdown_connections 2024-05-15T15:58:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.969+0000 7f0b01af1700 1 --2- 172.21.15.49:0/3773806922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0afc1007a0 0x7f0afc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.969+0000 7f0b01af1700 1 --2- 172.21.15.49:0/3773806922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc0ffdf0 0x7f0afc1001d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.970+0000 7f0b01af1700 1 -- 172.21.15.49:0/3773806922 >> 172.21.15.49:0/3773806922 conn(0x7f0afc0fb6a0 msgr2=0x7f0afc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:39.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.970+0000 7f0b01af1700 1 -- 172.21.15.49:0/3773806922 shutdown_connections 2024-05-15T15:58:39.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.970+0000 7f0b01af1700 1 -- 172.21.15.49:0/3773806922 wait complete. 2024-05-15T15:58:39.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.970+0000 7f0b01af1700 1 Processor -- start 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0b01af1700 1 -- start start 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0b01af1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0afc0ffdf0 0x7f0afc195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0b01af1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc1007a0 0x7f0afc195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0b01af1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc06a160 con 0x7f0afc1007a0 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0b01af1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc1963c0 con 0x7f0afc0ffdf0 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0afaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc1007a0 0x7f0afc195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0afaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc1007a0 0x7f0afc195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58452/0 (socket says 172.21.15.49:58452) 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0afaffd700 1 -- 172.21.15.49:0/1074647979 learned_addr learned my addr 172.21.15.49:0/1074647979 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0afb7fe700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0afc0ffdf0 0x7f0afc195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0afaffd700 1 -- 172.21.15.49:0/1074647979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0afc0ffdf0 msgr2=0x7f0afc195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0afaffd700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0afc0ffdf0 0x7f0afc195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0afaffd700 1 -- 172.21.15.49:0/1074647979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec0095f0 con 0x7f0afc1007a0 2024-05-15T15:58:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.971+0000 7f0afaffd700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc1007a0 0x7f0afc195e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0af000b480 tx=0x7f0af000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.972+0000 7f0af8ff9700 1 -- 172.21.15.49:0/1074647979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0af00118c0 con 0x7f0afc1007a0 2024-05-15T15:58:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.972+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0afc1966a0 con 0x7f0afc1007a0 2024-05-15T15:58:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.972+0000 7f0af8ff9700 1 -- 172.21.15.49:0/1074647979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0af0010380 con 0x7f0afc1007a0 2024-05-15T15:58:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.972+0000 7f0af8ff9700 1 -- 172.21.15.49:0/1074647979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0af000f4f0 con 0x7f0afc1007a0 2024-05-15T15:58:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.972+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc1a3230 con 0x7f0afc1007a0 2024-05-15T15:58:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.973+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0afc108190 con 0x7f0afc1007a0 2024-05-15T15:58:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.973+0000 7f0af8ff9700 1 -- 172.21.15.49:0/1074647979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0af00104f0 con 0x7f0afc1007a0 2024-05-15T15:58:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.974+0000 7f0af8ff9700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0ae406e340 0x7f0ae4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:58:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.974+0000 7f0af8ff9700 1 -- 172.21.15.49:0/1074647979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0af008deb0 con 0x7f0afc1007a0 2024-05-15T15:58:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.974+0000 7f0afb7fe700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0ae406e340 0x7f0ae4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:58:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.974+0000 7f0afb7fe700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0ae406e340 0x7f0ae4070800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0aec005f50 tx=0x7f0aec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:58:39.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:39.978+0000 7f0af8ff9700 1 -- 172.21.15.49:0/1074647979 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0af005ac60 con 0x7f0afc1007a0 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: pgmap v12: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: osdmap e50: 6 total, 6 up, 6 in 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: mgrmap e38: smithi049.vuhzgf(active, since 18s), standbys: smithi144.whcjkr 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:58:40.086 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi049"}]: dispatch 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: mon.smithi144 calling monitor election 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: mon.smithi049 calling monitor election 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: mon.smithi049 is new leader, mons smithi049,smithi144 in quorum (ranks 0,1) 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: monmap epoch 3 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: fsid c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: last_changed 2024-05-15T15:58:38.784516+0000 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: created 2024-05-15T15:43:32.672725+0000 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: min_mon_release 19 (squid) 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: election_strategy: 1 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.smithi049 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2 up:standby-replay 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: osdmap e50: 6 total, 6 up, 6 in 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: mgrmap e38: smithi049.vuhzgf(active, since 18s), standbys: smithi144.whcjkr 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:40.087 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:40.088 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:58:40.088 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:40.088 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:40.088 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:58:40.088 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3406995778' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:58:40.088 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:39 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1442964111' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:58:40.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.274+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0afc069770 con 0x7f0afc1007a0 2024-05-15T15:58:40.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.274+0000 7f0af8ff9700 1 -- 172.21.15.49:0/1074647979 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f0af005a7f0 con 0x7f0afc1007a0 2024-05-15T15:58:40.277 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:40.277 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:58:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:58:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0ae406e340 msgr2=0x7f0ae4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0ae406e340 0x7f0ae4070800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0aec005f50 tx=0x7f0aec01a040 comp rx=0 tx=0).stop 2024-05-15T15:58:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc1007a0 msgr2=0x7f0afc195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:58:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc1007a0 0x7f0afc195e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0af000b480 tx=0x7f0af000b840 comp rx=0 tx=0).stop 2024-05-15T15:58:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 shutdown_connections 2024-05-15T15:58:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0ae406e340 0x7f0ae4070800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0afc0ffdf0 0x7f0afc195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 --2- 172.21.15.49:0/1074647979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0afc1007a0 0x7f0afc195e80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:58:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 >> 172.21.15.49:0/1074647979 conn(0x7f0afc0fb6a0 msgr2=0x7f0afc0fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:58:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 shutdown_connections 2024-05-15T15:58:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:58:40.276+0000 7f0b01af1700 1 -- 172.21.15.49:0/1074647979 wait complete. 2024-05-15T15:58:41.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:40 smithi144 ceph-mon[86635]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:41.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:40 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1074647979' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T15:58:41.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:40 smithi049 ceph-mon[112142]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:58:41.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:40 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1074647979' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T15:58:42.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:41 smithi144 ceph-mon[86635]: pgmap v13: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:42.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:41 smithi049 ceph-mon[112142]: pgmap v13: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:42 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:42 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:43.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:42 smithi144 ceph-mon[86635]: pgmap v14: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:43.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:42 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:43.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:42 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:43.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:42 smithi049 ceph-mon[112142]: pgmap v14: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:44.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:44 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:44.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:44 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:44.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:44 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:44.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:44 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:45 smithi144 ceph-mon[86635]: pgmap v15: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:45.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:45 smithi049 ceph-mon[112142]: pgmap v15: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: pgmap v16: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:48.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:58:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: Reconfiguring mon.smithi049 (monmap changed)... 2024-05-15T15:58:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:58:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:58:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:48.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:47 smithi144 ceph-mon[86635]: Reconfiguring daemon mon.smithi049 on smithi049 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: pgmap v16: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: Reconfiguring mon.smithi049 (monmap changed)... 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:48.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:47 smithi049 ceph-mon[112142]: Reconfiguring daemon mon.smithi049 on smithi049 2024-05-15T15:58:49.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:49 smithi144 ceph-mon[86635]: pgmap v17: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:49.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:49 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:49.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:49 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:49.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:49 smithi144 ceph-mon[86635]: Reconfiguring mgr.smithi049.vuhzgf (monmap changed)... 2024-05-15T15:58:49.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:49 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi049.vuhzgf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:58:49.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:49 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:58:49.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:49 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:49.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:49 smithi144 ceph-mon[86635]: Reconfiguring daemon mgr.smithi049.vuhzgf on smithi049 2024-05-15T15:58:49.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:49 smithi049 ceph-mon[112142]: pgmap v17: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:49.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:49 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:49.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:49 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:49.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:49 smithi049 ceph-mon[112142]: Reconfiguring mgr.smithi049.vuhzgf (monmap changed)... 2024-05-15T15:58:49.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:49 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi049.vuhzgf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:58:49.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:49 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:58:49.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:49 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:49.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:49 smithi049 ceph-mon[112142]: Reconfiguring daemon mgr.smithi049.vuhzgf on smithi049 2024-05-15T15:58:50.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:58:50.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:50.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:50.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:58:50.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:58:50.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-15T15:58:50.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi049"}]: dispatch 2024-05-15T15:58:50.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:58:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:58:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:58:50.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-15T15:58:50.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi049"}]: dispatch 2024-05-15T15:58:50.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:51.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:51 smithi049 ceph-mon[112142]: pgmap v18: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:51 smithi049 ceph-mon[112142]: Reconfiguring ceph-exporter.smithi049 (monmap changed)... 2024-05-15T15:58:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:51 smithi049 ceph-mon[112142]: Unable to update caps for client.ceph-exporter.smithi049 2024-05-15T15:58:51.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:51 smithi049 ceph-mon[112142]: Reconfiguring daemon ceph-exporter.smithi049 on smithi049 2024-05-15T15:58:51.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:51 smithi144 ceph-mon[86635]: pgmap v18: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:51.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:51 smithi144 ceph-mon[86635]: Reconfiguring ceph-exporter.smithi049 (monmap changed)... 2024-05-15T15:58:51.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:51 smithi144 ceph-mon[86635]: Unable to update caps for client.ceph-exporter.smithi049 2024-05-15T15:58:51.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:51 smithi144 ceph-mon[86635]: Reconfiguring daemon ceph-exporter.smithi049 on smithi049 2024-05-15T15:58:53.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:53 smithi144 ceph-mon[86635]: pgmap v19: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:53.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:53 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:53.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:53 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:53.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:53 smithi144 ceph-mon[86635]: Reconfiguring crash.smithi049 (monmap changed)... 2024-05-15T15:58:53.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:53 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi049", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:58:53.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:53 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:53.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:53 smithi144 ceph-mon[86635]: Reconfiguring daemon crash.smithi049 on smithi049 2024-05-15T15:58:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:53 smithi049 ceph-mon[112142]: pgmap v19: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:58:53.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:53 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:53.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:53 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:53.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:53 smithi049 ceph-mon[112142]: Reconfiguring crash.smithi049 (monmap changed)... 2024-05-15T15:58:53.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:53 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi049", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:58:53.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:53 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:53.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:53 smithi049 ceph-mon[112142]: Reconfiguring daemon crash.smithi049 on smithi049 2024-05-15T15:58:55.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:55.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:55.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:54 smithi049 ceph-mon[112142]: Reconfiguring osd.0 (monmap changed)... 2024-05-15T15:58:55.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-15T15:58:55.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:55.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:54 smithi049 ceph-mon[112142]: Reconfiguring daemon osd.0 on smithi049 2024-05-15T15:58:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:54 smithi144 ceph-mon[86635]: Reconfiguring osd.0 (monmap changed)... 2024-05-15T15:58:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-15T15:58:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:54 smithi144 ceph-mon[86635]: Reconfiguring daemon osd.0 on smithi049 2024-05-15T15:58:56.056 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:55 smithi144 ceph-mon[86635]: pgmap v20: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:56.056 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:56.056 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:56.056 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-15T15:58:56.056 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:56.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:55 smithi049 ceph-mon[112142]: pgmap v20: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:58:56.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:56.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:56.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-15T15:58:56.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:57.067 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:56 smithi049 ceph-mon[112142]: Reconfiguring osd.1 (monmap changed)... 2024-05-15T15:58:57.067 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:56 smithi049 ceph-mon[112142]: Reconfiguring daemon osd.1 on smithi049 2024-05-15T15:58:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:56 smithi144 ceph-mon[86635]: Reconfiguring osd.1 (monmap changed)... 2024-05-15T15:58:57.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:56 smithi144 ceph-mon[86635]: Reconfiguring daemon osd.1 on smithi049 2024-05-15T15:58:58.029 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:57 smithi049 ceph-mon[112142]: pgmap v21: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T15:58:58.030 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:57 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:58.030 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:57 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:58.030 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:57 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-15T15:58:58.030 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:57 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:57 smithi144 ceph-mon[86635]: pgmap v21: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T15:58:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:57 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:57 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:58:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:57 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-15T15:58:58.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:57 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:58:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:58 smithi144 ceph-mon[86635]: Reconfiguring osd.2 (monmap changed)... 2024-05-15T15:58:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:58 smithi144 ceph-mon[86635]: Reconfiguring daemon osd.2 on smithi049 2024-05-15T15:58:59.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:58 smithi049 ceph-mon[112142]: Reconfiguring osd.2 (monmap changed)... 2024-05-15T15:58:59.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:58 smithi049 ceph-mon[112142]: Reconfiguring daemon osd.2 on smithi049 2024-05-15T15:59:00.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:59 smithi144 ceph-mon[86635]: pgmap v22: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:00.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:59 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:00.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:59 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:00.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:59 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.abbagx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:59:00.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:58:59 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:59 smithi049 ceph-mon[112142]: pgmap v22: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:59 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:59 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:59 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.abbagx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:59:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:58:59 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:01.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:00 smithi049 ceph-mon[112142]: Reconfiguring mds.cephfs.smithi049.abbagx (monmap changed)... 2024-05-15T15:59:01.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:00 smithi049 ceph-mon[112142]: Reconfiguring daemon mds.cephfs.smithi049.abbagx on smithi049 2024-05-15T15:59:01.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:01.165 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:01.165 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.gocadf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:59:01.165 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:00 smithi144 ceph-mon[86635]: Reconfiguring mds.cephfs.smithi049.abbagx (monmap changed)... 2024-05-15T15:59:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:00 smithi144 ceph-mon[86635]: Reconfiguring daemon mds.cephfs.smithi049.abbagx on smithi049 2024-05-15T15:59:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.gocadf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:59:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:02.182 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:01 smithi049 ceph-mon[112142]: pgmap v23: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:02.182 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:01 smithi049 ceph-mon[112142]: Reconfiguring mds.cephfs.smithi049.gocadf (monmap changed)... 2024-05-15T15:59:02.182 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:01 smithi049 ceph-mon[112142]: Reconfiguring daemon mds.cephfs.smithi049.gocadf on smithi049 2024-05-15T15:59:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:01 smithi144 ceph-mon[86635]: pgmap v23: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:01 smithi144 ceph-mon[86635]: Reconfiguring mds.cephfs.smithi049.gocadf (monmap changed)... 2024-05-15T15:59:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:01 smithi144 ceph-mon[86635]: Reconfiguring daemon mds.cephfs.smithi049.gocadf on smithi049 2024-05-15T15:59:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: pgmap v24: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: Reconfiguring ceph-exporter.smithi144 (monmap changed)... 2024-05-15T15:59:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:59:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:59:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-15T15:59:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: Unable to update caps for client.ceph-exporter.smithi144 2024-05-15T15:59:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi144"}]: dispatch 2024-05-15T15:59:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:03 smithi144 ceph-mon[86635]: Reconfiguring daemon ceph-exporter.smithi144 on smithi144 2024-05-15T15:59:03.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: pgmap v24: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:03.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:03.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:03.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: Reconfiguring ceph-exporter.smithi144 (monmap changed)... 2024-05-15T15:59:03.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:59:03.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T15:59:03.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-15T15:59:03.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: Unable to update caps for client.ceph-exporter.smithi144 2024-05-15T15:59:03.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi144"}]: dispatch 2024-05-15T15:59:03.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:03.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:03 smithi049 ceph-mon[112142]: Reconfiguring daemon ceph-exporter.smithi144 on smithi144 2024-05-15T15:59:04.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:04.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:04.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:04 smithi144 ceph-mon[86635]: Reconfiguring crash.smithi144 (monmap changed)... 2024-05-15T15:59:04.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:59:04.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:04.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:04 smithi144 ceph-mon[86635]: Reconfiguring daemon crash.smithi144 on smithi144 2024-05-15T15:59:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:04 smithi049 ceph-mon[112142]: Reconfiguring crash.smithi144 (monmap changed)... 2024-05-15T15:59:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:59:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:05.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:04 smithi049 ceph-mon[112142]: Reconfiguring daemon crash.smithi144 on smithi144 2024-05-15T15:59:06.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:05 smithi144 ceph-mon[86635]: pgmap v25: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:06.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:59:06.146 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:06.146 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:06.146 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:59:06.146 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:59:06.146 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:05 smithi049 ceph-mon[112142]: pgmap v25: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:59:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.whcjkr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T15:59:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T15:59:06.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:06.986 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:06 smithi144 ceph-mon[86635]: Reconfiguring mgr.smithi144.whcjkr (monmap changed)... 2024-05-15T15:59:06.986 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:06 smithi144 ceph-mon[86635]: Reconfiguring daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:59:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:06 smithi049 ceph-mon[112142]: Reconfiguring mgr.smithi144.whcjkr (monmap changed)... 2024-05-15T15:59:07.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:06 smithi049 ceph-mon[112142]: Reconfiguring daemon mgr.smithi144.whcjkr on smithi144 2024-05-15T15:59:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:07 smithi144 ceph-mon[86635]: pgmap v26: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:07 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:07 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:07 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:59:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:07 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:59:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:07 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:08.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:07 smithi049 ceph-mon[112142]: pgmap v26: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:08.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:07 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:08.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:07 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:08.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:07 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T15:59:08.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:07 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T15:59:08.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:07 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:08 smithi144 ceph-mon[86635]: Reconfiguring mon.smithi144 (monmap changed)... 2024-05-15T15:59:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:08 smithi144 ceph-mon[86635]: Reconfiguring daemon mon.smithi144 on smithi144 2024-05-15T15:59:08.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:08 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:08.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:08 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:08.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:08 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-15T15:59:08.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:08 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:09.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:08 smithi049 ceph-mon[112142]: Reconfiguring mon.smithi144 (monmap changed)... 2024-05-15T15:59:09.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:08 smithi049 ceph-mon[112142]: Reconfiguring daemon mon.smithi144 on smithi144 2024-05-15T15:59:09.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:08 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:09.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:08 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:09.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:08 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-15T15:59:09.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:08 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:09.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:09 smithi144 ceph-mon[86635]: pgmap v27: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:59:09.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:09 smithi144 ceph-mon[86635]: Reconfiguring osd.3 (monmap changed)... 2024-05-15T15:59:09.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:09 smithi144 ceph-mon[86635]: Reconfiguring daemon osd.3 on smithi144 2024-05-15T15:59:10.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:09 smithi049 ceph-mon[112142]: pgmap v27: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T15:59:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:09 smithi049 ceph-mon[112142]: Reconfiguring osd.3 (monmap changed)... 2024-05-15T15:59:10.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:09 smithi049 ceph-mon[112142]: Reconfiguring daemon osd.3 on smithi144 2024-05-15T15:59:10.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.400+0000 7ff4ff473700 1 -- 172.21.15.49:0/2672576395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8102790 msgr2=0x7ff4f810ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:10.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.400+0000 7ff4ff473700 1 --2- 172.21.15.49:0/2672576395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8102790 0x7ff4f810ac80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009910 tx=0x7ff4e8009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:10.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.400+0000 7ff4ff473700 1 -- 172.21.15.49:0/2672576395 shutdown_connections 2024-05-15T15:59:10.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.400+0000 7ff4ff473700 1 --2- 172.21.15.49:0/2672576395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8102790 0x7ff4f810ac80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.400+0000 7ff4ff473700 1 --2- 172.21.15.49:0/2672576395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f8101e70 0x7ff4f8102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.400+0000 7ff4ff473700 1 -- 172.21.15.49:0/2672576395 >> 172.21.15.49:0/2672576395 conn(0x7ff4f80fb700 msgr2=0x7ff4f80fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:10.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.400+0000 7ff4ff473700 1 -- 172.21.15.49:0/2672576395 shutdown_connections 2024-05-15T15:59:10.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.400+0000 7ff4ff473700 1 -- 172.21.15.49:0/2672576395 wait complete. 2024-05-15T15:59:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.401+0000 7ff4ff473700 1 Processor -- start 2024-05-15T15:59:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.401+0000 7ff4ff473700 1 -- start start 2024-05-15T15:59:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.401+0000 7ff4ff473700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8101e70 0x7ff4f819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.401+0000 7ff4ff473700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f8102790 0x7ff4f819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.401+0000 7ff4ff473700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f819d490 con 0x7ff4f8101e70 2024-05-15T15:59:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.401+0000 7ff4ff473700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f8196a70 con 0x7ff4f8102790 2024-05-15T15:59:10.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.401+0000 7ff4fd20f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8101e70 0x7ff4f819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.401+0000 7ff4fd20f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8101e70 0x7ff4f819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43790/0 (socket says 172.21.15.49:43790) 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.401+0000 7ff4fd20f700 1 -- 172.21.15.49:0/3748600501 learned_addr learned my addr 172.21.15.49:0/3748600501 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.402+0000 7ff4fd20f700 1 -- 172.21.15.49:0/3748600501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f8102790 msgr2=0x7ff4f819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.402+0000 7ff4fd20f700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f8102790 0x7ff4f819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.402+0000 7ff4fd20f700 1 -- 172.21.15.49:0/3748600501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80095f0 con 0x7ff4f8101e70 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.402+0000 7ff4fd20f700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8101e70 0x7ff4f819c930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400e920 tx=0x7ff4f400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.402+0000 7ff4ee7fc700 1 -- 172.21.15.49:0/3748600501 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f4009790 con 0x7ff4f8101e70 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.402+0000 7ff4ee7fc700 1 -- 172.21.15.49:0/3748600501 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4f400cba0 con 0x7ff4f8101e70 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.402+0000 7ff4ee7fc700 1 -- 172.21.15.49:0/3748600501 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f4018650 con 0x7ff4f8101e70 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.402+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f8196d50 con 0x7ff4f8101e70 2024-05-15T15:59:10.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.402+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f81972a0 con 0x7ff4f8101e70 2024-05-15T15:59:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.403+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f8105590 con 0x7ff4f8101e70 2024-05-15T15:59:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.404+0000 7ff4ee7fc700 1 -- 172.21.15.49:0/3748600501 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff4f400cd10 con 0x7ff4f8101e70 2024-05-15T15:59:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.404+0000 7ff4ee7fc700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4e406e340 0x7ff4e4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.404+0000 7ff4ee7fc700 1 -- 172.21.15.49:0/3748600501 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff4f4014070 con 0x7ff4f8101e70 2024-05-15T15:59:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.404+0000 7ff4fca0e700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4e406e340 0x7ff4e4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:10.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.405+0000 7ff4fca0e700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4e406e340 0x7ff4e4070800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009fd0 tx=0x7ff4e8000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.409+0000 7ff4ee7fc700 1 -- 172.21.15.49:0/3748600501 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff4f4053850 con 0x7ff4f8101e70 2024-05-15T15:59:10.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.636+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4f8197fd0 con 0x7ff4e406e340 2024-05-15T15:59:10.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.637+0000 7ff4ee7fc700 1 -- 172.21.15.49:0/3748600501 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ff4f8197fd0 con 0x7ff4e406e340 2024-05-15T15:59:10.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.639+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4e406e340 msgr2=0x7ff4e4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:10.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.639+0000 7ff4ff473700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4e406e340 0x7ff4e4070800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009fd0 tx=0x7ff4e8000bc0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.639+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8101e70 msgr2=0x7ff4f819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:10.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.639+0000 7ff4ff473700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8101e70 0x7ff4f819c930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400e920 tx=0x7ff4f400ece0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.640+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 shutdown_connections 2024-05-15T15:59:10.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.640+0000 7ff4ff473700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4e406e340 0x7ff4e4070800 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.640+0000 7ff4ff473700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4f8101e70 0x7ff4f819c930 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.640+0000 7ff4ff473700 1 --2- 172.21.15.49:0/3748600501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f8102790 0x7ff4f819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.640+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 >> 172.21.15.49:0/3748600501 conn(0x7ff4f80fb700 msgr2=0x7ff4f8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:10.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.640+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 shutdown_connections 2024-05-15T15:59:10.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.640+0000 7ff4ff473700 1 -- 172.21.15.49:0/3748600501 wait complete. 2024-05-15T15:59:10.653 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:59:10.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.749+0000 7f2ed279e700 1 -- 172.21.15.49:0/4204951002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 msgr2=0x7f2ecc105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:10.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.749+0000 7f2ed279e700 1 --2- 172.21.15.49:0/4204951002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 0x7f2ecc105d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009910 tx=0x7f2ebc009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:10.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.749+0000 7f2ed279e700 1 -- 172.21.15.49:0/4204951002 shutdown_connections 2024-05-15T15:59:10.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.749+0000 7f2ed279e700 1 --2- 172.21.15.49:0/4204951002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ecc0ff9b0 0x7f2ecc0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.749+0000 7f2ed279e700 1 --2- 172.21.15.49:0/4204951002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 0x7f2ecc105d90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.749+0000 7f2ed279e700 1 -- 172.21.15.49:0/4204951002 >> 172.21.15.49:0/4204951002 conn(0x7f2ecc0fb500 msgr2=0x7f2ecc0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:10.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.750+0000 7f2ed279e700 1 -- 172.21.15.49:0/4204951002 shutdown_connections 2024-05-15T15:59:10.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.750+0000 7f2ed279e700 1 -- 172.21.15.49:0/4204951002 wait complete. 2024-05-15T15:59:10.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.750+0000 7f2ed279e700 1 Processor -- start 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.750+0000 7f2ed279e700 1 -- start start 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed279e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ecc0ff9b0 0x7f2ecc101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed279e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 0x7f2ecc101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed279e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc102000 con 0x7f2ecc1059b0 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed279e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc102140 con 0x7f2ecc0ff9b0 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed0f9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 0x7f2ecc101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed0f9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 0x7f2ecc101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43804/0 (socket says 172.21.15.49:43804) 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed0f9b700 1 -- 172.21.15.49:0/1413052627 learned_addr learned my addr 172.21.15.49:0/1413052627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed179c700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ecc0ff9b0 0x7f2ecc101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed0f9b700 1 -- 172.21.15.49:0/1413052627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ecc0ff9b0 msgr2=0x7f2ecc101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed0f9b700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ecc0ff9b0 0x7f2ecc101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed0f9b700 1 -- 172.21.15.49:0/1413052627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0095f0 con 0x7f2ecc1059b0 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed0f9b700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 0x7f2ecc101ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000b4d0 tx=0x7f2ec000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:10.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2eca7fc700 1 -- 172.21.15.49:0/1413052627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ec0011800 con 0x7f2ecc1059b0 2024-05-15T15:59:10.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2eca7fc700 1 -- 172.21.15.49:0/1413052627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ec0011e20 con 0x7f2ecc1059b0 2024-05-15T15:59:10.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.751+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ecc1a2cf0 con 0x7f2ecc1059b0 2024-05-15T15:59:10.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.752+0000 7f2eca7fc700 1 -- 172.21.15.49:0/1413052627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ec000f2e0 con 0x7f2ecc1059b0 2024-05-15T15:59:10.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.752+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc1a3050 con 0x7f2ecc1059b0 2024-05-15T15:59:10.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.753+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc1010f0 con 0x7f2ecc1059b0 2024-05-15T15:59:10.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.753+0000 7f2eca7fc700 1 -- 172.21.15.49:0/1413052627 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f2ec0011960 con 0x7f2ecc1059b0 2024-05-15T15:59:10.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.753+0000 7f2eca7fc700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2eb406e330 0x7f2eb40707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:10.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.754+0000 7f2eca7fc700 1 -- 172.21.15.49:0/1413052627 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2ec008de10 con 0x7f2ecc1059b0 2024-05-15T15:59:10.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.754+0000 7f2ed179c700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2eb406e330 0x7f2eb40707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:10.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.754+0000 7f2ed179c700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2eb406e330 0x7f2eb40707f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009fd0 tx=0x7f2ebc00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:10.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.758+0000 7f2eca7fc700 1 -- 172.21.15.49:0/1413052627 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2ec0057090 con 0x7f2ecc1059b0 2024-05-15T15:59:10.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.988+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ecc102300 con 0x7f2eb406e330 2024-05-15T15:59:10.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.989+0000 7f2eca7fc700 1 -- 172.21.15.49:0/1413052627 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f2ecc102300 con 0x7f2eb406e330 2024-05-15T15:59:10.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.991+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2eb406e330 msgr2=0x7f2eb40707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:10.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2eb406e330 0x7f2eb40707f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009fd0 tx=0x7f2ebc00b540 comp rx=0 tx=0).stop 2024-05-15T15:59:10.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 msgr2=0x7f2ecc101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:10.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 0x7f2ecc101ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000b4d0 tx=0x7f2ec000b890 comp rx=0 tx=0).stop 2024-05-15T15:59:10.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 shutdown_connections 2024-05-15T15:59:10.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2eb406e330 0x7f2eb40707f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ecc0ff9b0 0x7f2ecc101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 --2- 172.21.15.49:0/1413052627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ecc1059b0 0x7f2ecc101ac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:10.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 >> 172.21.15.49:0/1413052627 conn(0x7f2ecc0fb500 msgr2=0x7f2ecc104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:10.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 shutdown_connections 2024-05-15T15:59:10.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:10.992+0000 7f2ed279e700 1 -- 172.21.15.49:0/1413052627 wait complete. 2024-05-15T15:59:11.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.099+0000 7fd63eeb1700 1 -- 172.21.15.49:0/1557092544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638101e20 msgr2=0x7fd638102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.099+0000 7fd63eeb1700 1 --2- 172.21.15.49:0/1557092544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638101e20 0x7fd638102200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009910 tx=0x7fd62c009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:11.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.100+0000 7fd63eeb1700 1 -- 172.21.15.49:0/1557092544 shutdown_connections 2024-05-15T15:59:11.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.100+0000 7fd63eeb1700 1 --2- 172.21.15.49:0/1557092544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102740 0x7fd63810ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.100+0000 7fd63eeb1700 1 --2- 172.21.15.49:0/1557092544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638101e20 0x7fd638102200 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.100+0000 7fd63eeb1700 1 -- 172.21.15.49:0/1557092544 >> 172.21.15.49:0/1557092544 conn(0x7fd638076800 msgr2=0x7fd6380fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:11.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.100+0000 7fd63eeb1700 1 -- 172.21.15.49:0/1557092544 shutdown_connections 2024-05-15T15:59:11.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.100+0000 7fd63eeb1700 1 -- 172.21.15.49:0/1557092544 wait complete. 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.100+0000 7fd63eeb1700 1 Processor -- start 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.100+0000 7fd63eeb1700 1 -- start start 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd63eeb1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638101e20 0x7fd63819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd63eeb1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638102740 0x7fd63819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd63eeb1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63819d500 con 0x7fd638102740 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd63eeb1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd638196ae0 con 0x7fd638101e20 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd637fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638102740 0x7fd63819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd637fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638102740 0x7fd63819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43812/0 (socket says 172.21.15.49:43812) 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd637fff700 1 -- 172.21.15.49:0/3952552602 learned_addr learned my addr 172.21.15.49:0/3952552602 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd63cc4d700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638101e20 0x7fd63819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd637fff700 1 -- 172.21.15.49:0/3952552602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638101e20 msgr2=0x7fd63819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd637fff700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638101e20 0x7fd63819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd637fff700 1 -- 172.21.15.49:0/3952552602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c0095f0 con 0x7fd638102740 2024-05-15T15:59:11.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd637fff700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638102740 0x7fd63819cee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd62400b3d0 tx=0x7fd62400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:11.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd635ffb700 1 -- 172.21.15.49:0/3952552602 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd624011800 con 0x7fd638102740 2024-05-15T15:59:11.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.101+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd638196dc0 con 0x7fd638102740 2024-05-15T15:59:11.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.102+0000 7fd635ffb700 1 -- 172.21.15.49:0/3952552602 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd624011e20 con 0x7fd638102740 2024-05-15T15:59:11.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.102+0000 7fd635ffb700 1 -- 172.21.15.49:0/3952552602 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd624010ea0 con 0x7fd638102740 2024-05-15T15:59:11.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.102+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd638197310 con 0x7fd638102740 2024-05-15T15:59:11.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.103+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6381055b0 con 0x7fd638102740 2024-05-15T15:59:11.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.103+0000 7fd635ffb700 1 -- 172.21.15.49:0/3952552602 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fd624010380 con 0x7fd638102740 2024-05-15T15:59:11.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.104+0000 7fd635ffb700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd62806e340 0x7fd628070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:11.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.104+0000 7fd635ffb700 1 -- 172.21.15.49:0/3952552602 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd62408dd60 con 0x7fd638102740 2024-05-15T15:59:11.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.104+0000 7fd63cc4d700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd62806e340 0x7fd628070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:11.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.104+0000 7fd63cc4d700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd62806e340 0x7fd628070800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00b5c0 tx=0x7fd62c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:11.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.109+0000 7fd635ffb700 1 -- 172.21.15.49:0/3952552602 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd62405ab10 con 0x7fd638102740 2024-05-15T15:59:11.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.327+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd638198010 con 0x7fd62806e340 2024-05-15T15:59:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.339+0000 7fd635ffb700 1 -- 172.21.15.49:0/3952552602 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd638198010 con 0x7fd62806e340 2024-05-15T15:59:11.341 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (2m) 45s ago 13m 17.3M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (14m) 45s ago 14m 8342k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (13m) 29s ago 13m 8090k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (14m) 45s ago 14m 6665k - 18.2.2-1343-g2965081f a04a73ffe079 be0e0c86c397 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (13m) 29s ago 13m 6660k - 18.2.2-1343-g2965081f a04a73ffe079 306753ea9df0 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (2m) 45s ago 13m 74.6M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (8m) 45s ago 8m 90.2M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (8m) 45s ago 8m 152M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (8m) 29s ago 8m 26.4M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (8m) 29s ago 8m 28.7M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (4m) 45s ago 15m 476M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (3m) 29s ago 12m 450M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (60s) 45s ago 15m 28.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T15:59:11.342 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (34s) 29s ago 12m 24.6M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T15:59:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (3m) 45s ago 14m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T15:59:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (3m) 29s ago 12m 16.1M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T15:59:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (12m) 45s ago 12m 219M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:59:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (11m) 45s ago 11m 219M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:59:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (11m) 45s ago 11m 213M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:59:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (10m) 29s ago 10m 260M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:59:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (10m) 29s ago 10m 217M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:59:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (9m) 29s ago 9m 202M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:59:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (2m) 45s ago 13m 60.1M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T15:59:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd62806e340 msgr2=0x7fd628070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd62806e340 0x7fd628070800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00b5c0 tx=0x7fd62c01a040 comp rx=0 tx=0).stop 2024-05-15T15:59:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638102740 msgr2=0x7fd63819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638102740 0x7fd63819cee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd62400b3d0 tx=0x7fd62400b790 comp rx=0 tx=0).stop 2024-05-15T15:59:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 shutdown_connections 2024-05-15T15:59:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd62806e340 0x7fd628070800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638101e20 0x7fd63819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 --2- 172.21.15.49:0/3952552602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd638102740 0x7fd63819cee0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 >> 172.21.15.49:0/3952552602 conn(0x7fd638076800 msgr2=0x7fd6381030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 shutdown_connections 2024-05-15T15:59:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.341+0000 7fd63eeb1700 1 -- 172.21.15.49:0/3952552602 wait complete. 2024-05-15T15:59:11.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.449+0000 7f92a1e4a700 1 -- 172.21.15.49:0/309107559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c0ffdc0 msgr2=0x7f929c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.449+0000 7f92a1e4a700 1 --2- 172.21.15.49:0/309107559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c0ffdc0 0x7f929c1001a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f928c00b0a0 tx=0x7f928c00b3b0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.449+0000 7f92a1e4a700 1 -- 172.21.15.49:0/309107559 shutdown_connections 2024-05-15T15:59:11.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.449+0000 7f92a1e4a700 1 --2- 172.21.15.49:0/309107559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c100770 0x7f929c1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.449+0000 7f92a1e4a700 1 --2- 172.21.15.49:0/309107559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c0ffdc0 0x7f929c1001a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.449+0000 7f92a1e4a700 1 -- 172.21.15.49:0/309107559 >> 172.21.15.49:0/309107559 conn(0x7f929c076800 msgr2=0x7f929c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:11.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.449+0000 7f92a1e4a700 1 -- 172.21.15.49:0/309107559 shutdown_connections 2024-05-15T15:59:11.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.449+0000 7f92a1e4a700 1 -- 172.21.15.49:0/309107559 wait complete. 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f92a1e4a700 1 Processor -- start 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f92a1e4a700 1 -- start start 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f92a1e4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c0ffdc0 0x7f929c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f92a1e4a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c100770 0x7f929c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f92a1e4a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c19d470 con 0x7f929c100770 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f92a1e4a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c196a50 con 0x7f929c0ffdc0 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f929affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c100770 0x7f929c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f929affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c100770 0x7f929c19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43832/0 (socket says 172.21.15.49:43832) 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f929affd700 1 -- 172.21.15.49:0/332993163 learned_addr learned my addr 172.21.15.49:0/332993163 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:11.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.450+0000 7f929b7fe700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c0ffdc0 0x7f929c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:11.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.451+0000 7f929affd700 1 -- 172.21.15.49:0/332993163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c0ffdc0 msgr2=0x7f929c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.451+0000 7f929affd700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c0ffdc0 0x7f929c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.451+0000 7f929affd700 1 -- 172.21.15.49:0/332993163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c009d00 con 0x7f929c100770 2024-05-15T15:59:11.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.451+0000 7f929affd700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c100770 0x7f929c19ce50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f928400d900 tx=0x7f928400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:11.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.451+0000 7f9298ff9700 1 -- 172.21.15.49:0/332993163 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9284009900 con 0x7f929c100770 2024-05-15T15:59:11.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.451+0000 7f9298ff9700 1 -- 172.21.15.49:0/332993163 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9284010460 con 0x7f929c100770 2024-05-15T15:59:11.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.451+0000 7f9298ff9700 1 -- 172.21.15.49:0/332993163 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f928400f5f0 con 0x7f929c100770 2024-05-15T15:59:11.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.451+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c196d30 con 0x7f929c100770 2024-05-15T15:59:11.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.451+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c197280 con 0x7f929c100770 2024-05-15T15:59:11.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.452+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929c1012a0 con 0x7f929c100770 2024-05-15T15:59:11.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.452+0000 7f9298ff9700 1 -- 172.21.15.49:0/332993163 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f92840105d0 con 0x7f929c100770 2024-05-15T15:59:11.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.453+0000 7f9298ff9700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f928806e330 0x7f92880707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:11.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.453+0000 7f9298ff9700 1 -- 172.21.15.49:0/332993163 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f928408de20 con 0x7f929c100770 2024-05-15T15:59:11.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.453+0000 7f929b7fe700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f928806e330 0x7f92880707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:11.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.453+0000 7f929b7fe700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f928806e330 0x7f92880707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f928c000f80 tx=0x7f928c0061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:11.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.458+0000 7f9298ff9700 1 -- 172.21.15.49:0/332993163 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f928405ab40 con 0x7f929c100770 2024-05-15T15:59:11.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:11 smithi144 ceph-mon[86635]: pgmap v28: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:11.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:11.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:11.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:11 smithi144 ceph-mon[86635]: Reconfiguring osd.4 (monmap changed)... 2024-05-15T15:59:11.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-15T15:59:11.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:11.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:11 smithi144 ceph-mon[86635]: Reconfiguring daemon osd.4 on smithi144 2024-05-15T15:59:11.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:11 smithi144 ceph-mon[86635]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:11.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:11 smithi049 ceph-mon[112142]: pgmap v28: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:11.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:11.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:11.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:11 smithi049 ceph-mon[112142]: Reconfiguring osd.4 (monmap changed)... 2024-05-15T15:59:11.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-15T15:59:11.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:11.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:11 smithi049 ceph-mon[112142]: Reconfiguring daemon osd.4 on smithi144 2024-05-15T15:59:11.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:11 smithi049 ceph-mon[112142]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:11.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.752+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f929c04ec60 con 0x7f929c100770 2024-05-15T15:59:11.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.752+0000 7f9298ff9700 1 -- 172.21.15.49:0/332993163 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f92840172f0 con 0x7f929c100770 2024-05-15T15:59:11.755 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:59:11.755 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:59:11.755 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:59:11.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:59:11.755 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 10, 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:59:11.756 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:59:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f928806e330 msgr2=0x7f92880707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f928806e330 0x7f92880707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f928c000f80 tx=0x7f928c0061f0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c100770 msgr2=0x7f929c19ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c100770 0x7f929c19ce50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f928400d900 tx=0x7f928400dc10 comp rx=0 tx=0).stop 2024-05-15T15:59:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 shutdown_connections 2024-05-15T15:59:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f928806e330 0x7f92880707f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f929c0ffdc0 0x7f929c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 --2- 172.21.15.49:0/332993163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f929c100770 0x7f929c19ce50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 >> 172.21.15.49:0/332993163 conn(0x7f929c076800 msgr2=0x7f929c0686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:11.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 shutdown_connections 2024-05-15T15:59:11.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.755+0000 7f92a1e4a700 1 -- 172.21.15.49:0/332993163 wait complete. 2024-05-15T15:59:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.864+0000 7fed26135700 1 -- 172.21.15.49:0/2739004915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 msgr2=0x7fed20104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.864+0000 7fed26135700 1 --2- 172.21.15.49:0/2739004915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 0x7fed20104880 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fed08009910 tx=0x7fed08009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.864+0000 7fed26135700 1 -- 172.21.15.49:0/2739004915 shutdown_connections 2024-05-15T15:59:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.864+0000 7fed26135700 1 --2- 172.21.15.49:0/2739004915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 0x7fed20104880 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.864+0000 7fed26135700 1 --2- 172.21.15.49:0/2739004915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed200ff690 0x7fed200ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.864+0000 7fed26135700 1 -- 172.21.15.49:0/2739004915 >> 172.21.15.49:0/2739004915 conn(0x7fed20074eb0 msgr2=0x7fed200752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.864+0000 7fed26135700 1 -- 172.21.15.49:0/2739004915 shutdown_connections 2024-05-15T15:59:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.864+0000 7fed26135700 1 -- 172.21.15.49:0/2739004915 wait complete. 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.864+0000 7fed26135700 1 Processor -- start 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed26135700 1 -- start start 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed26135700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed200ff690 0x7fed20198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed26135700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 0x7fed201992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed26135700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20199920 con 0x7fed20100040 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed26135700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20199a60 con 0x7fed200ff690 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed1effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 0x7fed201992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed1effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 0x7fed201992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43854/0 (socket says 172.21.15.49:43854) 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed1effd700 1 -- 172.21.15.49:0/2582712975 learned_addr learned my addr 172.21.15.49:0/2582712975 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed1f7fe700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed200ff690 0x7fed20198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed1effd700 1 -- 172.21.15.49:0/2582712975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed200ff690 msgr2=0x7fed20198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed1effd700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed200ff690 0x7fed20198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.865+0000 7fed1effd700 1 -- 172.21.15.49:0/2582712975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed080095f0 con 0x7fed20100040 2024-05-15T15:59:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.866+0000 7fed1effd700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 0x7fed201992d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed080092e0 tx=0x7fed0800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.866+0000 7fed1cff9700 1 -- 172.21.15.49:0/2582712975 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed0801d070 con 0x7fed20100040 2024-05-15T15:59:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.866+0000 7fed1cff9700 1 -- 172.21.15.49:0/2582712975 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed0800f460 con 0x7fed20100040 2024-05-15T15:59:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.866+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2019d790 con 0x7fed20100040 2024-05-15T15:59:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.866+0000 7fed1cff9700 1 -- 172.21.15.49:0/2582712975 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed08017630 con 0x7fed20100040 2024-05-15T15:59:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.866+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2019dc80 con 0x7fed20100040 2024-05-15T15:59:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.867+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed200febc0 con 0x7fed20100040 2024-05-15T15:59:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.868+0000 7fed1cff9700 1 -- 172.21.15.49:0/2582712975 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fed0800f5d0 con 0x7fed20100040 2024-05-15T15:59:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.868+0000 7fed1cff9700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fed0c06e330 0x7fed0c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.868+0000 7fed1cff9700 1 -- 172.21.15.49:0/2582712975 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fed0808f700 con 0x7fed20100040 2024-05-15T15:59:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.868+0000 7fed1f7fe700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fed0c06e330 0x7fed0c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:11.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.868+0000 7fed1f7fe700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fed0c06e330 0x7fed0c0707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fed1000f440 tx=0x7fed100079e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:11.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:11.873+0000 7fed1cff9700 1 -- 172.21.15.49:0/2582712975 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fed080588f0 con 0x7fed20100040 2024-05-15T15:59:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.125+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fed20068ee0 con 0x7fed20100040 2024-05-15T15:59:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.126+0000 7fed1cff9700 1 -- 172.21.15.49:0/2582712975 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1903 (secure 0 0 0) 0x7fed0805bfb0 con 0x7fed20100040 2024-05-15T15:59:12.129 INFO:teuthology.orchestra.run.smithi049.stdout:e17 2024-05-15T15:59:12.129 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:59:12.129 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:59:12.129 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T15:59:12.129 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:59:12.129 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T15:59:12.129 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T15:59:12.129 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 17 2024-05-15T15:59:12.129 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:51:00.559034+0000 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T15:59:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:59:12.131 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:59:12.132 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:59:12.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.128+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fed0c06e330 msgr2=0x7fed0c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.128+0000 7fed26135700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fed0c06e330 0x7fed0c0707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fed1000f440 tx=0x7fed100079e0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.128+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 msgr2=0x7fed201992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.128+0000 7fed26135700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 0x7fed201992d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed080092e0 tx=0x7fed0800bd70 comp rx=0 tx=0).stop 2024-05-15T15:59:12.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.129+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 shutdown_connections 2024-05-15T15:59:12.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.129+0000 7fed26135700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fed0c06e330 0x7fed0c0707f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.129+0000 7fed26135700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed200ff690 0x7fed20198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.129+0000 7fed26135700 1 --2- 172.21.15.49:0/2582712975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed20100040 0x7fed201992d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.129+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 >> 172.21.15.49:0/2582712975 conn(0x7fed20074eb0 msgr2=0x7fed201007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:12.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.129+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 shutdown_connections 2024-05-15T15:59:12.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.129+0000 7fed26135700 1 -- 172.21.15.49:0/2582712975 wait complete. 2024-05-15T15:59:12.133 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T15:59:12.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.238+0000 7f3a219d3700 1 -- 172.21.15.49:0/3468464278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c0ff690 msgr2=0x7f3a1c0ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.239+0000 7f3a219d3700 1 --2- 172.21.15.49:0/3468464278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c0ff690 0x7f3a1c0ffa70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009910 tx=0x7f3a0c009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:12.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.239+0000 7f3a219d3700 1 -- 172.21.15.49:0/3468464278 shutdown_connections 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.239+0000 7f3a219d3700 1 --2- 172.21.15.49:0/3468464278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1c100040 0x7f3a1c104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.239+0000 7f3a219d3700 1 --2- 172.21.15.49:0/3468464278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c0ff690 0x7f3a1c0ffa70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.239+0000 7f3a219d3700 1 -- 172.21.15.49:0/3468464278 >> 172.21.15.49:0/3468464278 conn(0x7f3a1c074eb0 msgr2=0x7f3a1c0752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.239+0000 7f3a219d3700 1 -- 172.21.15.49:0/3468464278 shutdown_connections 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.239+0000 7f3a219d3700 1 -- 172.21.15.49:0/3468464278 wait complete. 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.239+0000 7f3a219d3700 1 Processor -- start 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.239+0000 7f3a219d3700 1 -- start start 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a219d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1c0ff690 0x7f3a1c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a219d3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c100040 0x7f3a1c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a219d3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c19d3f0 con 0x7f3a1c100040 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a219d3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c196a20 con 0x7f3a1c0ff690 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a1a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c100040 0x7f3a1c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a1a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c100040 0x7f3a1c19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43872/0 (socket says 172.21.15.49:43872) 2024-05-15T15:59:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a1a7fc700 1 -- 172.21.15.49:0/596424952 learned_addr learned my addr 172.21.15.49:0/596424952 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a1affd700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1c0ff690 0x7f3a1c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a1a7fc700 1 -- 172.21.15.49:0/596424952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1c0ff690 msgr2=0x7f3a1c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a1a7fc700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1c0ff690 0x7f3a1c19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a1a7fc700 1 -- 172.21.15.49:0/596424952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c0095f0 con 0x7f3a1c100040 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a1a7fc700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c100040 0x7f3a1c19cdd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000d8d0 tx=0x7f3a1000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.240+0000 7f3a209d1700 1 -- 172.21.15.49:0/596424952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a100098a0 con 0x7f3a1c100040 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.241+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c196d00 con 0x7f3a1c100040 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.241+0000 7f3a209d1700 1 -- 172.21.15.49:0/596424952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a10010460 con 0x7f3a1c100040 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.241+0000 7f3a209d1700 1 -- 172.21.15.49:0/596424952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1000f5f0 con 0x7f3a1c100040 2024-05-15T15:59:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.241+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c197250 con 0x7f3a1c100040 2024-05-15T15:59:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.242+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c109300 con 0x7f3a1c100040 2024-05-15T15:59:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.242+0000 7f3a209d1700 1 -- 172.21.15.49:0/596424952 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f3a10009a00 con 0x7f3a1c100040 2024-05-15T15:59:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.243+0000 7f3a209d1700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a0406e340 0x7f3a04070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.243+0000 7f3a209d1700 1 -- 172.21.15.49:0/596424952 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3a1008e230 con 0x7f3a1c100040 2024-05-15T15:59:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.243+0000 7f3a1affd700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a0406e340 0x7f3a04070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.243+0000 7f3a1affd700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a0406e340 0x7f3a04070800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006010 tx=0x7f3a0c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:12.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.248+0000 7f3a209d1700 1 -- 172.21.15.49:0/596424952 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3a1005aed0 con 0x7f3a1c100040 2024-05-15T15:59:12.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.477+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a1c197f00 con 0x7f3a0406e340 2024-05-15T15:59:12.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.478+0000 7f3a209d1700 1 -- 172.21.15.49:0/596424952 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f3a1c197f00 con 0x7f3a0406e340 2024-05-15T15:59:12.480 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "4/23 daemons upgraded", 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mon daemons", 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:59:12.481 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a0406e340 msgr2=0x7f3a04070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a0406e340 0x7f3a04070800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006010 tx=0x7f3a0c01a040 comp rx=0 tx=0).stop 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c100040 msgr2=0x7f3a1c19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c100040 0x7f3a1c19cdd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000d8d0 tx=0x7f3a1000dbe0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 shutdown_connections 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a0406e340 0x7f3a04070800 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1c0ff690 0x7f3a1c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 --2- 172.21.15.49:0/596424952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1c100040 0x7f3a1c19cdd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 >> 172.21.15.49:0/596424952 conn(0x7f3a1c074eb0 msgr2=0x7f3a1c102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:12.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 shutdown_connections 2024-05-15T15:59:12.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.480+0000 7f3a219d3700 1 -- 172.21.15.49:0/596424952 wait complete. 2024-05-15T15:59:12.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.589+0000 7fead1d86700 1 -- 172.21.15.49:0/924162821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 msgr2=0x7feacc10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.589+0000 7fead1d86700 1 --2- 172.21.15.49:0/924162821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 0x7feacc10ac60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feab4009910 tx=0x7feab4009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:12.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.589+0000 7fead1d86700 1 -- 172.21.15.49:0/924162821 shutdown_connections 2024-05-15T15:59:12.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.589+0000 7fead1d86700 1 --2- 172.21.15.49:0/924162821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 0x7feacc10ac60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.589+0000 7fead1d86700 1 --2- 172.21.15.49:0/924162821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feacc101e50 0x7feacc102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.589+0000 7fead1d86700 1 -- 172.21.15.49:0/924162821 >> 172.21.15.49:0/924162821 conn(0x7feacc0fb6a0 msgr2=0x7feacc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:12.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.589+0000 7fead1d86700 1 -- 172.21.15.49:0/924162821 shutdown_connections 2024-05-15T15:59:12.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.589+0000 7fead1d86700 1 -- 172.21.15.49:0/924162821 wait complete. 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.589+0000 7fead1d86700 1 Processor -- start 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7fead1d86700 1 -- start start 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7fead1d86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feacc101e50 0x7feacc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7fead1d86700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 0x7feacc19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7fead1d86700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc196b70 con 0x7feacc102770 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7fead1d86700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc196ce0 con 0x7feacc101e50 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7feacaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 0x7feacc19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7feacaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 0x7feacc19d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43882/0 (socket says 172.21.15.49:43882) 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7feacaffd700 1 -- 172.21.15.49:0/1837300804 learned_addr learned my addr 172.21.15.49:0/1837300804 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7feacb7fe700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feacc101e50 0x7feacc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:12.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7feacaffd700 1 -- 172.21.15.49:0/1837300804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feacc101e50 msgr2=0x7feacc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7feacaffd700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feacc101e50 0x7feacc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.590+0000 7feacaffd700 1 -- 172.21.15.49:0/1837300804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40095f0 con 0x7feacc102770 2024-05-15T15:59:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.591+0000 7feacaffd700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 0x7feacc19d000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feab400bd70 tx=0x7feab400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.591+0000 7feac8ff9700 1 -- 172.21.15.49:0/1837300804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feab401d070 con 0x7feacc102770 2024-05-15T15:59:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.591+0000 7feac8ff9700 1 -- 172.21.15.49:0/1837300804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feab400f460 con 0x7feacc102770 2024-05-15T15:59:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.591+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc196f60 con 0x7feacc102770 2024-05-15T15:59:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.591+0000 7feac8ff9700 1 -- 172.21.15.49:0/1837300804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feab4017630 con 0x7feacc102770 2024-05-15T15:59:12.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.591+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc197450 con 0x7feacc102770 2024-05-15T15:59:12.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.592+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc069ab0 con 0x7feacc102770 2024-05-15T15:59:12.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.592+0000 7feac8ff9700 1 -- 172.21.15.49:0/1837300804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7feab400f5d0 con 0x7feacc102770 2024-05-15T15:59:12.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.593+0000 7feac8ff9700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feab806e340 0x7feab8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:12.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.593+0000 7feac8ff9700 1 -- 172.21.15.49:0/1837300804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7feab408f8e0 con 0x7feacc102770 2024-05-15T15:59:12.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.593+0000 7feacb7fe700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feab806e340 0x7feab8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:12.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.593+0000 7feacb7fe700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feab806e340 0x7feab8070800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feabc009f20 tx=0x7feabc0094f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:12.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.597+0000 7feac8ff9700 1 -- 172.21.15.49:0/1837300804 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7feab4058a50 con 0x7feacc102770 2024-05-15T15:59:12.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:12 smithi144 ceph-mon[86635]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:12.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:12 smithi144 ceph-mon[86635]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:12.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:12 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/332993163' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:12.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:12 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:12.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:12 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:12.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:12 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-15T15:59:12.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:12 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:12.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:12 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2582712975' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:59:12.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:12 smithi049 ceph-mon[112142]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:12.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:12 smithi049 ceph-mon[112142]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:12.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:12 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/332993163' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:12.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:12 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:12.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:12 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:12.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:12 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-15T15:59:12.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:12 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:12.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:12 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2582712975' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:59:12.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.894+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feacc197fe0 con 0x7feacc102770 2024-05-15T15:59:12.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.895+0000 7feac8ff9700 1 -- 172.21.15.49:0/1837300804 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7feab405c110 con 0x7feacc102770 2024-05-15T15:59:12.897 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T15:59:12.898 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:59:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:59:12.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feab806e340 msgr2=0x7feab8070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feab806e340 0x7feab8070800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feabc009f20 tx=0x7feabc0094f0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 msgr2=0x7feacc19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:12.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 0x7feacc19d000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feab400bd70 tx=0x7feab400be50 comp rx=0 tx=0).stop 2024-05-15T15:59:12.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 shutdown_connections 2024-05-15T15:59:12.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feab806e340 0x7feab8070800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feacc101e50 0x7feacc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 --2- 172.21.15.49:0/1837300804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feacc102770 0x7feacc19d000 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:12.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 >> 172.21.15.49:0/1837300804 conn(0x7feacc0fb6a0 msgr2=0x7feacc1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:12.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 shutdown_connections 2024-05-15T15:59:12.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:12.897+0000 7fead1d86700 1 -- 172.21.15.49:0/1837300804 wait complete. 2024-05-15T15:59:13.573 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:13 smithi144 ceph-mon[86635]: Reconfiguring osd.5 (monmap changed)... 2024-05-15T15:59:13.573 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:13 smithi144 ceph-mon[86635]: Reconfiguring daemon osd.5 on smithi144 2024-05-15T15:59:13.573 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:13 smithi144 ceph-mon[86635]: pgmap v29: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:13.573 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:13 smithi144 ceph-mon[86635]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:13.573 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:13 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1837300804' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T15:59:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:13 smithi049 ceph-mon[112142]: Reconfiguring osd.5 (monmap changed)... 2024-05-15T15:59:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:13 smithi049 ceph-mon[112142]: Reconfiguring daemon osd.5 on smithi144 2024-05-15T15:59:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:13 smithi049 ceph-mon[112142]: pgmap v29: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:13 smithi049 ceph-mon[112142]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:13 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1837300804' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T15:59:14.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:14 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:14.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:14 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:14.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:14 smithi144 ceph-mon[86635]: Reconfiguring mds.cephfs.smithi144.jgodzs (monmap changed)... 2024-05-15T15:59:14.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:14 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jgodzs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:59:14.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:14 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:14.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:14 smithi144 ceph-mon[86635]: Reconfiguring daemon mds.cephfs.smithi144.jgodzs on smithi144 2024-05-15T15:59:14.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:14 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:14.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:14 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:14.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:14 smithi049 ceph-mon[112142]: Reconfiguring mds.cephfs.smithi144.jgodzs (monmap changed)... 2024-05-15T15:59:14.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:14 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jgodzs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:59:14.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:14 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:14.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:14 smithi049 ceph-mon[112142]: Reconfiguring daemon mds.cephfs.smithi144.jgodzs on smithi144 2024-05-15T15:59:15.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:15 smithi144 ceph-mon[86635]: pgmap v30: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:15.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:15.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:15.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.kziuke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:59:15.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:15.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:15 smithi049 ceph-mon[112142]: pgmap v30: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:15.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:15.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:15.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.kziuke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T15:59:15.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:16.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:16 smithi144 ceph-mon[86635]: Reconfiguring mds.cephfs.smithi144.kziuke (monmap changed)... 2024-05-15T15:59:16.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:16 smithi144 ceph-mon[86635]: Reconfiguring daemon mds.cephfs.smithi144.kziuke on smithi144 2024-05-15T15:59:16.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:16 smithi049 ceph-mon[112142]: Reconfiguring mds.cephfs.smithi144.kziuke (monmap changed)... 2024-05-15T15:59:16.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:16 smithi049 ceph-mon[112142]: Reconfiguring daemon mds.cephfs.smithi144.kziuke on smithi144 2024-05-15T15:59:17.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: pgmap v31: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:17.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:17.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:17.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:59:17.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:17.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:17.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:17.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi049"}]: dispatch 2024-05-15T15:59:17.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi049"}]': finished 2024-05-15T15:59:17.854 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi144"}]: dispatch 2024-05-15T15:59:17.854 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi144"}]': finished 2024-05-15T15:59:17.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: pgmap v31: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi049"}]: dispatch 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi049"}]': finished 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi144"}]: dispatch 2024-05-15T15:59:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi144"}]': finished 2024-05-15T15:59:18.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:18 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all mon 2024-05-15T15:59:18.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:18 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all mon 2024-05-15T15:59:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:19 smithi049 ceph-mon[112142]: pgmap v32: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-15T15:59:19.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:19 smithi049 ceph-mon[112142]: Upgrade: Updating crash.smithi049 (1/2) 2024-05-15T15:59:19.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:19.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi049", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:59:19.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:19.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:19 smithi049 ceph-mon[112142]: Deploying daemon crash.smithi049 on smithi049 2024-05-15T15:59:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:19 smithi144 ceph-mon[86635]: pgmap v32: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-15T15:59:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:19 smithi144 ceph-mon[86635]: Upgrade: Updating crash.smithi049 (1/2) 2024-05-15T15:59:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi049", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:59:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:20.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:19 smithi144 ceph-mon[86635]: Deploying daemon crash.smithi049 on smithi049 2024-05-15T15:59:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:59:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:59:22.147 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:21 smithi049 ceph-mon[112142]: pgmap v33: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:22.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:21 smithi144 ceph-mon[86635]: pgmap v33: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:24.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:23 smithi144 ceph-mon[86635]: pgmap v34: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:24.235 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:23 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:24.235 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:23 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:24.236 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:23 smithi049 ceph-mon[112142]: pgmap v34: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:24.236 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:24.236 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:25.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:25 smithi144 ceph-mon[86635]: pgmap v35: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:25.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:25 smithi144 ceph-mon[86635]: Upgrade: Updating crash.smithi144 (2/2) 2024-05-15T15:59:25.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:25 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:25.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:25 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:59:25.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:25 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:25.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:25 smithi144 ceph-mon[86635]: Deploying daemon crash.smithi144 on smithi144 2024-05-15T15:59:25.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:25 smithi049 ceph-mon[112142]: pgmap v35: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:25.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:25 smithi049 ceph-mon[112142]: Upgrade: Updating crash.smithi144 (2/2) 2024-05-15T15:59:25.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:25.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-15T15:59:25.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:25.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:25 smithi049 ceph-mon[112142]: Deploying daemon crash.smithi144 on smithi144 2024-05-15T15:59:27.658 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:27 smithi144 ceph-mon[86635]: pgmap v36: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:27.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:27 smithi049 ceph-mon[112142]: pgmap v36: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:29.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:29 smithi144 ceph-mon[86635]: pgmap v37: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:29.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:29 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:29.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:29 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:29.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:29 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:59:29.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:29 smithi049 ceph-mon[112142]: pgmap v37: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:29.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:29 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:29.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:29 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:29.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:29 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:59:31.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:31 smithi049 ceph-mon[112142]: pgmap v38: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:31.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:31 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:31.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:31 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:31 smithi144 ceph-mon[86635]: pgmap v38: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:31 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:31 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:33.595 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:33 smithi144 ceph-mon[86635]: pgmap v39: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:33.595 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:33 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:33.595 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:33 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:33.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:33 smithi049 ceph-mon[112142]: pgmap v39: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:33.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:33 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:33.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:33 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:34.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:34 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:34.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:34 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:34.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:34 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:34.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:34 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:35 smithi144 ceph-mon[86635]: pgmap v40: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:59:36.047 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:35 smithi049 ceph-mon[112142]: pgmap v40: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:36.047 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:36.047 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:36.047 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:59:36.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:36.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:37.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:37.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:37.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:37 smithi049 ceph-mon[112142]: pgmap v41: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:38.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:37 smithi144 ceph-mon[86635]: pgmap v41: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:39.371 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: pgmap v42: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:39.371 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all crash 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi049"}]: dispatch 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi049"}]': finished 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi144"}]: dispatch 2024-05-15T15:59:39.372 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi144"}]': finished 2024-05-15T15:59:39.373 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-15T15:59:39.373 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-15T15:59:39.373 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:39 smithi049 ceph-mon[112142]: Upgrade: osd.0 is safe to restart 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: pgmap v42: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:39.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:39.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all crash 2024-05-15T15:59:39.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:39.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi049"}]: dispatch 2024-05-15T15:59:39.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi049"}]': finished 2024-05-15T15:59:39.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi144"}]: dispatch 2024-05-15T15:59:39.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi144"}]': finished 2024-05-15T15:59:39.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-15T15:59:39.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-15T15:59:39.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:39 smithi144 ceph-mon[86635]: Upgrade: osd.0 is safe to restart 2024-05-15T15:59:41.413 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:41 smithi049 ceph-mon[112142]: pgmap v43: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:41.413 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:41 smithi049 ceph-mon[112142]: Upgrade: Updating osd.0 2024-05-15T15:59:41.413 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:41 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:41.413 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:41 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-15T15:59:41.413 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:41 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:41.413 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:41 smithi049 ceph-mon[112142]: Deploying daemon osd.0 on smithi049 2024-05-15T15:59:41.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:41 smithi144 ceph-mon[86635]: pgmap v43: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:41.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:41 smithi144 ceph-mon[86635]: Upgrade: Updating osd.0 2024-05-15T15:59:41.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:41 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:41.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:41 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-15T15:59:41.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:41 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T15:59:41.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:41 smithi144 ceph-mon[86635]: Deploying daemon osd.0 on smithi049 2024-05-15T15:59:43.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.009+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/236084944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc100ee0 msgr2=0x7f2adc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.009+0000 7f2ae3fc9700 1 --2- 172.21.15.49:0/236084944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc100ee0 0x7f2adc1012c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009910 tx=0x7f2ad8009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:43.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.010+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/236084944 shutdown_connections 2024-05-15T15:59:43.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.010+0000 7f2ae3fc9700 1 --2- 172.21.15.49:0/236084944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2adc101890 0x7f2adc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.010+0000 7f2ae3fc9700 1 --2- 172.21.15.49:0/236084944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc100ee0 0x7f2adc1012c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.010+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/236084944 >> 172.21.15.49:0/236084944 conn(0x7f2adc0fc790 msgr2=0x7f2adc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:43.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.010+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/236084944 shutdown_connections 2024-05-15T15:59:43.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.010+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/236084944 wait complete. 2024-05-15T15:59:43.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.010+0000 7f2ae3fc9700 1 Processor -- start 2024-05-15T15:59:43.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae3fc9700 1 -- start start 2024-05-15T15:59:43.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae3fc9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2adc100ee0 0x7f2adc19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:43.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae3fc9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc101890 0x7f2adc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:43.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae3fc9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc19d3d0 con 0x7f2adc101890 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae3fc9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc196a90 con 0x7f2adc100ee0 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae1564700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc101890 0x7f2adc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae1564700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc101890 0x7f2adc19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60836/0 (socket says 172.21.15.49:60836) 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae1564700 1 -- 172.21.15.49:0/3479918137 learned_addr learned my addr 172.21.15.49:0/3479918137 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae1d65700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2adc100ee0 0x7f2adc19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae1564700 1 -- 172.21.15.49:0/3479918137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2adc100ee0 msgr2=0x7f2adc19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae1564700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2adc100ee0 0x7f2adc19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae1564700 1 -- 172.21.15.49:0/3479918137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad80095f0 con 0x7f2adc101890 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.011+0000 7f2ae1564700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc101890 0x7f2adc19ce40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000c960 tx=0x7f2ad000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.012+0000 7f2aceffd700 1 -- 172.21.15.49:0/3479918137 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ad0007a70 con 0x7f2adc101890 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.012+0000 7f2aceffd700 1 -- 172.21.15.49:0/3479918137 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ad0007bd0 con 0x7f2adc101890 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.012+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2adc196d70 con 0x7f2adc101890 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.012+0000 7f2aceffd700 1 -- 172.21.15.49:0/3479918137 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ad0018630 con 0x7f2adc101890 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.012+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc1972c0 con 0x7f2adc101890 2024-05-15T15:59:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.013+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2adc109280 con 0x7f2adc101890 2024-05-15T15:59:43.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.013+0000 7f2aceffd700 1 -- 172.21.15.49:0/3479918137 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f2ad000f450 con 0x7f2adc101890 2024-05-15T15:59:43.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.014+0000 7f2aceffd700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2ac806e340 0x7f2ac8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:43.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.014+0000 7f2aceffd700 1 -- 172.21.15.49:0/3479918137 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2ad008e2e0 con 0x7f2adc101890 2024-05-15T15:59:43.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.014+0000 7f2ae1d65700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2ac806e340 0x7f2ac8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:43.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.014+0000 7f2ae1d65700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2ac806e340 0x7f2ac8070800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8006010 tx=0x7f2ad801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:43.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.019+0000 7f2aceffd700 1 -- 172.21.15.49:0/3479918137 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2ad005af80 con 0x7f2adc101890 2024-05-15T15:59:43.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.247+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2adc197f70 con 0x7f2ac806e340 2024-05-15T15:59:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.248+0000 7f2aceffd700 1 -- 172.21.15.49:0/3479918137 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2adc197f70 con 0x7f2ac806e340 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2ac806e340 msgr2=0x7f2ac8070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2ac806e340 0x7f2ac8070800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8006010 tx=0x7f2ad801a040 comp rx=0 tx=0).stop 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc101890 msgr2=0x7f2adc19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc101890 0x7f2adc19ce40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000c960 tx=0x7f2ad000cc70 comp rx=0 tx=0).stop 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 shutdown_connections 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2ac806e340 0x7f2ac8070800 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2adc100ee0 0x7f2adc19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 --2- 172.21.15.49:0/3479918137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2adc101890 0x7f2adc19ce40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 >> 172.21.15.49:0/3479918137 conn(0x7f2adc0fc790 msgr2=0x7f2adc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 shutdown_connections 2024-05-15T15:59:43.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.250+0000 7f2ae3fc9700 1 -- 172.21.15.49:0/3479918137 wait complete. 2024-05-15T15:59:43.263 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T15:59:43.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.359+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2246813575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 msgr2=0x7fd99c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.359+0000 7fd9a1b11700 1 --2- 172.21.15.49:0/2246813575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 0x7fd99c1001d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009910 tx=0x7fd98c009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:43.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.359+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2246813575 shutdown_connections 2024-05-15T15:59:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.359+0000 7fd9a1b11700 1 --2- 172.21.15.49:0/2246813575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd99c1007a0 0x7fd99c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.359+0000 7fd9a1b11700 1 --2- 172.21.15.49:0/2246813575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 0x7fd99c1001d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.359+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2246813575 >> 172.21.15.49:0/2246813575 conn(0x7fd99c0fb6a0 msgr2=0x7fd99c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.359+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2246813575 shutdown_connections 2024-05-15T15:59:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.359+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2246813575 wait complete. 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.360+0000 7fd9a1b11700 1 Processor -- start 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.360+0000 7fd9a1b11700 1 -- start start 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd9a1b11700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 0x7fd99c19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd9a1b11700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd99c1007a0 0x7fd99c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd9a1b11700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c19d430 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd9a1b11700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c196a10 con 0x7fd99c1007a0 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd99b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 0x7fd99c19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd99b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 0x7fd99c19c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60850/0 (socket says 172.21.15.49:60850) 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd99b7fe700 1 -- 172.21.15.49:0/2063328488 learned_addr learned my addr 172.21.15.49:0/2063328488 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd99affd700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd99c1007a0 0x7fd99c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd99b7fe700 1 -- 172.21.15.49:0/2063328488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd99c1007a0 msgr2=0x7fd99c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd99b7fe700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd99c1007a0 0x7fd99c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd99b7fe700 1 -- 172.21.15.49:0/2063328488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c0095f0 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.361+0000 7fd99b7fe700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 0x7fd99c19c8d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00bd70 tx=0x7fd98c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.362+0000 7fd998ff9700 1 -- 172.21.15.49:0/2063328488 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd98c01d070 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.362+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99c196c90 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.362+0000 7fd998ff9700 1 -- 172.21.15.49:0/2063328488 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd98c0056e0 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.362+0000 7fd998ff9700 1 -- 172.21.15.49:0/2063328488 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd98c00fe40 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.362+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c197180 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.363+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99c1091a0 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.363+0000 7fd998ff9700 1 -- 172.21.15.49:0/2063328488 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fd98c00f680 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.364+0000 7fd998ff9700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd98406e2c0 0x7fd984070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.364+0000 7fd998ff9700 1 -- 172.21.15.49:0/2063328488 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd98c08f780 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.364+0000 7fd99affd700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd98406e2c0 0x7fd984070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.364+0000 7fd99affd700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd98406e2c0 0x7fd984070780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd99c197bd0 tx=0x7fd9900073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:43.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.368+0000 7fd998ff9700 1 -- 172.21.15.49:0/2063328488 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd98c05c420 con 0x7fd99c0ffdf0 2024-05-15T15:59:43.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.612+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd99c197f70 con 0x7fd98406e2c0 2024-05-15T15:59:43.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.613+0000 7fd998ff9700 1 -- 172.21.15.49:0/2063328488 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd99c197f70 con 0x7fd98406e2c0 2024-05-15T15:59:43.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd98406e2c0 msgr2=0x7fd984070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd98406e2c0 0x7fd984070780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd99c197bd0 tx=0x7fd9900073d0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 msgr2=0x7fd99c19c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 0x7fd99c19c8d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00bd70 tx=0x7fd98c00bda0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 shutdown_connections 2024-05-15T15:59:43.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd98406e2c0 0x7fd984070780 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c0ffdf0 0x7fd99c19c8d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 --2- 172.21.15.49:0/2063328488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd99c1007a0 0x7fd99c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 >> 172.21.15.49:0/2063328488 conn(0x7fd99c0fb6a0 msgr2=0x7fd99c069700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:43.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 shutdown_connections 2024-05-15T15:59:43.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.616+0000 7fd9a1b11700 1 -- 172.21.15.49:0/2063328488 wait complete. 2024-05-15T15:59:43.642 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:43 smithi049 ceph-mon[112142]: pgmap v44: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:43.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.737+0000 7efd709ee700 1 -- 172.21.15.49:0/956774922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 msgr2=0x7efd6c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.737+0000 7efd709ee700 1 --2- 172.21.15.49:0/956774922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 0x7efd6c1047c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd60009910 tx=0x7efd60009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.737+0000 7efd709ee700 1 -- 172.21.15.49:0/956774922 shutdown_connections 2024-05-15T15:59:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.737+0000 7efd709ee700 1 --2- 172.21.15.49:0/956774922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 0x7efd6c1047c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.737+0000 7efd709ee700 1 --2- 172.21.15.49:0/956774922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd6c0ffdc0 0x7efd6c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.737+0000 7efd709ee700 1 -- 172.21.15.49:0/956774922 >> 172.21.15.49:0/956774922 conn(0x7efd6c076800 msgr2=0x7efd6c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.737+0000 7efd709ee700 1 -- 172.21.15.49:0/956774922 shutdown_connections 2024-05-15T15:59:43.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.737+0000 7efd709ee700 1 -- 172.21.15.49:0/956774922 wait complete. 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.738+0000 7efd709ee700 1 Processor -- start 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.738+0000 7efd709ee700 1 -- start start 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd709ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd6c0ffdc0 0x7efd6c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd709ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 0x7efd6c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd709ee700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c1984f0 con 0x7efd6c100770 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd709ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c198630 con 0x7efd6c0ffdc0 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd69d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 0x7efd6c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd69d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 0x7efd6c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60234/0 (socket says 172.21.15.49:60234) 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd69d9b700 1 -- 172.21.15.49:0/1347467127 learned_addr learned my addr 172.21.15.49:0/1347467127 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd6a59c700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd6c0ffdc0 0x7efd6c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd69d9b700 1 -- 172.21.15.49:0/1347467127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd6c0ffdc0 msgr2=0x7efd6c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd69d9b700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd6c0ffdc0 0x7efd6c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd69d9b700 1 -- 172.21.15.49:0/1347467127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600095f0 con 0x7efd6c100770 2024-05-15T15:59:43.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.739+0000 7efd69d9b700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 0x7efd6c197f20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd60009fd0 tx=0x7efd6000b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:43.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.740+0000 7efd5b7fe700 1 -- 172.21.15.49:0/1347467127 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6001d070 con 0x7efd6c100770 2024-05-15T15:59:43.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.740+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c198850 con 0x7efd6c100770 2024-05-15T15:59:43.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.740+0000 7efd5b7fe700 1 -- 172.21.15.49:0/1347467127 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd600052a0 con 0x7efd6c100770 2024-05-15T15:59:43.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.740+0000 7efd5b7fe700 1 -- 172.21.15.49:0/1347467127 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd60022470 con 0x7efd6c100770 2024-05-15T15:59:43.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.740+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c1a0db0 con 0x7efd6c100770 2024-05-15T15:59:43.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.741+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c1081c0 con 0x7efd6c100770 2024-05-15T15:59:43.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.741+0000 7efd5b7fe700 1 -- 172.21.15.49:0/1347467127 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7efd6000f6f0 con 0x7efd6c100770 2024-05-15T15:59:43.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.742+0000 7efd5b7fe700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd5406e340 0x7efd54070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:43.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.742+0000 7efd5b7fe700 1 -- 172.21.15.49:0/1347467127 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efd6008f490 con 0x7efd6c100770 2024-05-15T15:59:43.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.742+0000 7efd6a59c700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd5406e340 0x7efd54070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:43.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.742+0000 7efd6a59c700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd5406e340 0x7efd54070800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009c60 tx=0x7efd5c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:43.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.747+0000 7efd5b7fe700 1 -- 172.21.15.49:0/1347467127 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efd60058600 con 0x7efd6c100770 2024-05-15T15:59:43.909 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:43 smithi049 systemd[1]: Stopping Ceph osd.0 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T15:59:43.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:43 smithi144 ceph-mon[86635]: pgmap v44: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T15:59:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.976+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efd6c198850 con 0x7efd5406e340 2024-05-15T15:59:43.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.988+0000 7efd5b7fe700 1 -- 172.21.15.49:0/1347467127 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7efd6c198850 con 0x7efd5406e340 2024-05-15T15:59:43.990 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (2m) 10s ago 14m 26.2M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (14m) 10s ago 14m 8434k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (13m) 12s ago 13m 8162k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (21s) 10s ago 14m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (16s) 12s ago 13m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (2m) 10s ago 14m 73.0M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (8m) 10s ago 8m 90.0M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (8m) 10s ago 8m 152M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (8m) 12s ago 8m 26.7M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (8m) 12s ago 8m 28.4M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (5m) 10s ago 16m 498M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (4m) 12s ago 13m 451M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (92s) 10s ago 16m 43.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (66s) 12s ago 13m 35.0M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T15:59:43.991 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (3m) 10s ago 14m 15.4M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T15:59:43.992 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (3m) 12s ago 13m 16.2M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T15:59:43.992 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (12m) 10s ago 12m 219M 1216M 18.2.2-1343-g2965081f a04a73ffe079 9d74a0af6b57 2024-05-15T15:59:43.992 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (12m) 10s ago 12m 219M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T15:59:43.992 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (11m) 10s ago 11m 215M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T15:59:43.992 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (11m) 12s ago 11m 261M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T15:59:43.992 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (10m) 12s ago 10m 217M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T15:59:43.992 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (10m) 12s ago 10m 203M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T15:59:43.992 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (3m) 10s ago 14m 64.5M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd5406e340 msgr2=0x7efd54070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd5406e340 0x7efd54070800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009c60 tx=0x7efd5c009260 comp rx=0 tx=0).stop 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 msgr2=0x7efd6c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 0x7efd6c197f20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd60009fd0 tx=0x7efd6000b5a0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 shutdown_connections 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd5406e340 0x7efd54070800 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd6c0ffdc0 0x7efd6c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 --2- 172.21.15.49:0/1347467127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd6c100770 0x7efd6c197f20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 >> 172.21.15.49:0/1347467127 conn(0x7efd6c076800 msgr2=0x7efd6c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 shutdown_connections 2024-05-15T15:59:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:43.990+0000 7efd709ee700 1 -- 172.21.15.49:0/1347467127 wait complete. 2024-05-15T15:59:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.097+0000 7f27f7014700 1 -- 172.21.15.49:0/842578852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f01007c0 msgr2=0x7f27f0104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.097+0000 7f27f7014700 1 --2- 172.21.15.49:0/842578852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f01007c0 0x7f27f0104810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009910 tx=0x7f27dc009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.097+0000 7f27f7014700 1 -- 172.21.15.49:0/842578852 shutdown_connections 2024-05-15T15:59:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.097+0000 7f27f7014700 1 --2- 172.21.15.49:0/842578852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f01007c0 0x7f27f0104810 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.097+0000 7f27f7014700 1 --2- 172.21.15.49:0/842578852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27f00ffe10 0x7f27f01001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.097+0000 7f27f7014700 1 -- 172.21.15.49:0/842578852 >> 172.21.15.49:0/842578852 conn(0x7f27f00fb700 msgr2=0x7f27f00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.097+0000 7f27f7014700 1 -- 172.21.15.49:0/842578852 shutdown_connections 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.097+0000 7f27f7014700 1 -- 172.21.15.49:0/842578852 wait complete. 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.098+0000 7f27f7014700 1 Processor -- start 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.098+0000 7f27f7014700 1 -- start start 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.098+0000 7f27f7014700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f00ffe10 0x7f27f019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.098+0000 7f27f7014700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27f01007c0 0x7f27f019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.098+0000 7f27f7014700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0196a20 con 0x7f27f00ffe10 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.098+0000 7f27f7014700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0196b90 con 0x7f27f01007c0 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.098+0000 7f27f4db0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f00ffe10 0x7f27f019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.098+0000 7f27f4db0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f00ffe10 0x7f27f019c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60244/0 (socket says 172.21.15.49:60244) 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.098+0000 7f27f4db0700 1 -- 172.21.15.49:0/2972392241 learned_addr learned my addr 172.21.15.49:0/2972392241 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27effff700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27f01007c0 0x7f27f019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27f4db0700 1 -- 172.21.15.49:0/2972392241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27f01007c0 msgr2=0x7f27f019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27f4db0700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27f01007c0 0x7f27f019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27f4db0700 1 -- 172.21.15.49:0/2972392241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0095f0 con 0x7f27f00ffe10 2024-05-15T15:59:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27f4db0700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f00ffe10 0x7f27f019c970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c8a0 tx=0x7f27e400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27edffb700 1 -- 172.21.15.49:0/2972392241 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27e40078e0 con 0x7f27f00ffe10 2024-05-15T15:59:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f0196e70 con 0x7f27f00ffe10 2024-05-15T15:59:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27edffb700 1 -- 172.21.15.49:0/2972392241 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27e400f450 con 0x7f27f00ffe10 2024-05-15T15:59:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27edffb700 1 -- 172.21.15.49:0/2972392241 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27e400e5e0 con 0x7f27f00ffe10 2024-05-15T15:59:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.099+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f0197390 con 0x7f27f00ffe10 2024-05-15T15:59:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.100+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f01012f0 con 0x7f27f00ffe10 2024-05-15T15:59:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.101+0000 7f27edffb700 1 -- 172.21.15.49:0/2972392241 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f27e400f5c0 con 0x7f27f00ffe10 2024-05-15T15:59:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.101+0000 7f27edffb700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f27e006e340 0x7f27e0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.102+0000 7f27edffb700 1 -- 172.21.15.49:0/2972392241 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f27e408e390 con 0x7f27f00ffe10 2024-05-15T15:59:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.102+0000 7f27effff700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f27e006e340 0x7f27e0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.102+0000 7f27effff700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f27e006e340 0x7f27e0070800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f27f0198220 tx=0x7f27dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:44.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.106+0000 7f27edffb700 1 -- 172.21.15.49:0/2972392241 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f27e4057500 con 0x7f27f00ffe10 2024-05-15T15:59:44.205 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:43 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[47019]: 2024-05-15T15:59:43.906+0000 7f6b8208d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T15:59:44.205 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:43 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[47019]: 2024-05-15T15:59:43.906+0000 7f6b8208d700 -1 osd.0 50 *** Got signal Terminated *** 2024-05-15T15:59:44.206 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:43 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[47019]: 2024-05-15T15:59:43.906+0000 7f6b8208d700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T15:59:44.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.401+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f27f010cdb0 con 0x7f27f00ffe10 2024-05-15T15:59:44.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.401+0000 7f27edffb700 1 -- 172.21.15.49:0/2972392241 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f27e405abc0 con 0x7f27f00ffe10 2024-05-15T15:59:44.405 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:59:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T15:59:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:59:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:59:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T15:59:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T15:59:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:59:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T15:59:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6 2024-05-15T15:59:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:59:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T15:59:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T15:59:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T15:59:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T15:59:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 10, 2024-05-15T15:59:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T15:59:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T15:59:44.407 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:59:44.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.403+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f27e006e340 msgr2=0x7f27e0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.403+0000 7f27f7014700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f27e006e340 0x7f27e0070800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f27f0198220 tx=0x7f27dc01a040 comp rx=0 tx=0).stop 2024-05-15T15:59:44.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.403+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f00ffe10 msgr2=0x7f27f019c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.403+0000 7f27f7014700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f00ffe10 0x7f27f019c970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c8a0 tx=0x7f27e400cbb0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.403+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 shutdown_connections 2024-05-15T15:59:44.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.403+0000 7f27f7014700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f27e006e340 0x7f27e0070800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.403+0000 7f27f7014700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27f00ffe10 0x7f27f019c970 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.403+0000 7f27f7014700 1 --2- 172.21.15.49:0/2972392241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27f01007c0 0x7f27f019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.403+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 >> 172.21.15.49:0/2972392241 conn(0x7f27f00fb700 msgr2=0x7f27f0069570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:44.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.404+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 shutdown_connections 2024-05-15T15:59:44.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.404+0000 7f27f7014700 1 -- 172.21.15.49:0/2972392241 wait complete. 2024-05-15T15:59:44.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.521+0000 7f5c80b22700 1 -- 172.21.15.49:0/398096506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c101890 msgr2=0x7f5c7c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.521+0000 7f5c80b22700 1 --2- 172.21.15.49:0/398096506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c101890 0x7f5c7c1058e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009910 tx=0x7f5c70009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:44.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.521+0000 7f5c80b22700 1 -- 172.21.15.49:0/398096506 shutdown_connections 2024-05-15T15:59:44.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.521+0000 7f5c80b22700 1 --2- 172.21.15.49:0/398096506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c101890 0x7f5c7c1058e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.521+0000 7f5c80b22700 1 --2- 172.21.15.49:0/398096506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c7c100ee0 0x7f5c7c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.521+0000 7f5c80b22700 1 -- 172.21.15.49:0/398096506 >> 172.21.15.49:0/398096506 conn(0x7f5c7c0fc790 msgr2=0x7f5c7c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.521+0000 7f5c80b22700 1 -- 172.21.15.49:0/398096506 shutdown_connections 2024-05-15T15:59:44.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.521+0000 7f5c80b22700 1 -- 172.21.15.49:0/398096506 wait complete. 2024-05-15T15:59:44.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.522+0000 7f5c80b22700 1 Processor -- start 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.522+0000 7f5c80b22700 1 -- start start 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c80b22700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c100ee0 0x7f5c7c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c80b22700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c7c101890 0x7f5c7c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c80b22700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c073d20 con 0x7f5c7c100ee0 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c80b22700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c073e60 con 0x7f5c7c101890 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c7a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c100ee0 0x7f5c7c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c7a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c100ee0 0x7f5c7c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60258/0 (socket says 172.21.15.49:60258) 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c7a59c700 1 -- 172.21.15.49:0/2511774906 learned_addr learned my addr 172.21.15.49:0/2511774906 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c79d9b700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c7c101890 0x7f5c7c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c7a59c700 1 -- 172.21.15.49:0/2511774906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c7c101890 msgr2=0x7f5c7c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c7a59c700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c7c101890 0x7f5c7c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c7a59c700 1 -- 172.21.15.49:0/2511774906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700095f0 con 0x7f5c7c100ee0 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c7a59c700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c100ee0 0x7f5c7c0732a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c8a0 tx=0x7f5c6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c6b7fe700 1 -- 172.21.15.49:0/2511774906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6c00cea0 con 0x7f5c7c100ee0 2024-05-15T15:59:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c6b7fe700 1 -- 172.21.15.49:0/2511774906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c6c004830 con 0x7f5c7c100ee0 2024-05-15T15:59:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c1a2da0 con 0x7f5c7c100ee0 2024-05-15T15:59:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c6b7fe700 1 -- 172.21.15.49:0/2511774906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6c0055b0 con 0x7f5c7c100ee0 2024-05-15T15:59:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.523+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1a3100 con 0x7f5c7c100ee0 2024-05-15T15:59:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.525+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c1092e0 con 0x7f5c7c100ee0 2024-05-15T15:59:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.525+0000 7f5c6b7fe700 1 -- 172.21.15.49:0/2511774906 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f5c6c0049a0 con 0x7f5c7c100ee0 2024-05-15T15:59:44.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.526+0000 7f5c6b7fe700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c6406e340 0x7f5c64070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:44.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.526+0000 7f5c6b7fe700 1 -- 172.21.15.49:0/2511774906 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5c6c08df40 con 0x7f5c7c100ee0 2024-05-15T15:59:44.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.526+0000 7f5c79d9b700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c6406e340 0x7f5c64070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:44.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.526+0000 7f5c79d9b700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c6406e340 0x7f5c64070800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000b5c0 tx=0x7f5c70005380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:44.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.530+0000 7f5c6b7fe700 1 -- 172.21.15.49:0/2511774906 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5c6c0c2850 con 0x7f5c7c100ee0 2024-05-15T15:59:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:44 smithi049 ceph-mon[112142]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:44 smithi049 ceph-mon[112142]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:44 smithi049 ceph-mon[112142]: osd.0 marked itself down and dead 2024-05-15T15:59:44.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:44 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2972392241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:44.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.784+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5c7c074820 con 0x7f5c7c100ee0 2024-05-15T15:59:44.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.784+0000 7f5c6b7fe700 1 -- 172.21.15.49:0/2511774906 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f5c6c01d020 con 0x7f5c7c100ee0 2024-05-15T15:59:44.787 INFO:teuthology.orchestra.run.smithi049.stdout:e17 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 17 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:51:00.559034+0000 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T15:59:44.788 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T15:59:44.789 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:59:44.790 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T15:59:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:59:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T15:59:44.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.787+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c6406e340 msgr2=0x7f5c64070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.787+0000 7f5c80b22700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c6406e340 0x7f5c64070800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000b5c0 tx=0x7f5c70005380 comp rx=0 tx=0).stop 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.788+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c100ee0 msgr2=0x7f5c7c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.788+0000 7f5c80b22700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c100ee0 0x7f5c7c0732a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c8a0 tx=0x7f5c6c00cc60 comp rx=0 tx=0).stop 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.788+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 shutdown_connections 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.788+0000 7f5c80b22700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c6406e340 0x7f5c64070800 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.788+0000 7f5c80b22700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c7c100ee0 0x7f5c7c0732a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.788+0000 7f5c80b22700 1 --2- 172.21.15.49:0/2511774906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c7c101890 0x7f5c7c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.788+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 >> 172.21.15.49:0/2511774906 conn(0x7f5c7c0fc790 msgr2=0x7f5c7c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.788+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 shutdown_connections 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.788+0000 7f5c80b22700 1 -- 172.21.15.49:0/2511774906 wait complete. 2024-05-15T15:59:44.792 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T15:59:44.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.903+0000 7fb229cbd700 1 -- 172.21.15.49:0/194647080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224101890 msgr2=0x7fb2241058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.903+0000 7fb229cbd700 1 --2- 172.21.15.49:0/194647080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224101890 0x7fb2241058e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb218009910 tx=0x7fb218009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:44.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.903+0000 7fb229cbd700 1 -- 172.21.15.49:0/194647080 shutdown_connections 2024-05-15T15:59:44.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.903+0000 7fb229cbd700 1 --2- 172.21.15.49:0/194647080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224101890 0x7fb2241058e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.903+0000 7fb229cbd700 1 --2- 172.21.15.49:0/194647080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb224100ee0 0x7fb2241012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.903+0000 7fb229cbd700 1 -- 172.21.15.49:0/194647080 >> 172.21.15.49:0/194647080 conn(0x7fb2240fc790 msgr2=0x7fb2240febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:44.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.903+0000 7fb229cbd700 1 -- 172.21.15.49:0/194647080 shutdown_connections 2024-05-15T15:59:44.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.903+0000 7fb229cbd700 1 -- 172.21.15.49:0/194647080 wait complete. 2024-05-15T15:59:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.904+0000 7fb229cbd700 1 Processor -- start 2024-05-15T15:59:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.904+0000 7fb229cbd700 1 -- start start 2024-05-15T15:59:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.904+0000 7fb229cbd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224100ee0 0x7fb2240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.904+0000 7fb229cbd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb224101890 0x7fb2240737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.904+0000 7fb229cbd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224075270 con 0x7fb224100ee0 2024-05-15T15:59:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.904+0000 7fb229cbd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224073d20 con 0x7fb224101890 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb2237fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224100ee0 0x7fb2240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb2237fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224100ee0 0x7fb2240732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60270/0 (socket says 172.21.15.49:60270) 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb2237fe700 1 -- 172.21.15.49:0/2482429704 learned_addr learned my addr 172.21.15.49:0/2482429704 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb222ffd700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb224101890 0x7fb2240737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb2237fe700 1 -- 172.21.15.49:0/2482429704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb224101890 msgr2=0x7fb2240737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb2237fe700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb224101890 0x7fb2240737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb2237fe700 1 -- 172.21.15.49:0/2482429704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180095f0 con 0x7fb224100ee0 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb2237fe700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224100ee0 0x7fb2240732a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb21400c930 tx=0x7fb21400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb220ff9700 1 -- 172.21.15.49:0/2482429704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb214007ad0 con 0x7fb224100ee0 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb220ff9700 1 -- 172.21.15.49:0/2482429704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb214007c30 con 0x7fb224100ee0 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb220ff9700 1 -- 172.21.15.49:0/2482429704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb214018610 con 0x7fb224100ee0 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb224073fd0 con 0x7fb224100ee0 2024-05-15T15:59:44.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.905+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2241a2fc0 con 0x7fb224100ee0 2024-05-15T15:59:44.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.906+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2241092e0 con 0x7fb224100ee0 2024-05-15T15:59:44.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.907+0000 7fb220ff9700 1 -- 172.21.15.49:0/2482429704 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fb21400f450 con 0x7fb224100ee0 2024-05-15T15:59:44.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.907+0000 7fb220ff9700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb20c06e330 0x7fb20c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:44.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.907+0000 7fb220ff9700 1 -- 172.21.15.49:0/2482429704 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb21408e480 con 0x7fb224100ee0 2024-05-15T15:59:44.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.907+0000 7fb222ffd700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb20c06e330 0x7fb20c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:44.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.908+0000 7fb222ffd700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb20c06e330 0x7fb20c0707f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2240749d0 tx=0x7fb21801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:44.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:44.912+0000 7fb220ff9700 1 -- 172.21.15.49:0/2482429704 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb214057670 con 0x7fb224100ee0 2024-05-15T15:59:44.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:44 smithi144 ceph-mon[86635]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:44.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:44 smithi144 ceph-mon[86635]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:44.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:44 smithi144 ceph-mon[86635]: osd.0 marked itself down and dead 2024-05-15T15:59:44.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:44 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2972392241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T15:59:45.119 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:44 smithi049 podman[123654]: 2024-05-15 15:59:44.811262789 +0000 UTC m=+1.004702223 container died 9d74a0af6b578bbfe6ee14860043b9bc817cddb8b1186b476782f5fb234d8348 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=reef-2965081, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-15T15:59:45.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.141+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb224074860 con 0x7fb20c06e330 2024-05-15T15:59:45.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.142+0000 7fb220ff9700 1 -- 172.21.15.49:0/2482429704 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb224074860 con 0x7fb20c06e330 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "6/23 daemons upgraded", 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading osd daemons", 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T15:59:45.145 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T15:59:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb20c06e330 msgr2=0x7fb20c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb20c06e330 0x7fb20c0707f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2240749d0 tx=0x7fb21801a040 comp rx=0 tx=0).stop 2024-05-15T15:59:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224100ee0 msgr2=0x7fb2240732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224100ee0 0x7fb2240732a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb21400c930 tx=0x7fb21400ccf0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 shutdown_connections 2024-05-15T15:59:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb20c06e330 0x7fb20c0707f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb224100ee0 0x7fb2240732a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 --2- 172.21.15.49:0/2482429704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb224101890 0x7fb2240737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 >> 172.21.15.49:0/2482429704 conn(0x7fb2240fc790 msgr2=0x7fb2240fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 shutdown_connections 2024-05-15T15:59:45.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.144+0000 7fb229cbd700 1 -- 172.21.15.49:0/2482429704 wait complete. 2024-05-15T15:59:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.254+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3938895342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 msgr2=0x7f80f0104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.254+0000 7f80f7a3a700 1 --2- 172.21.15.49:0/3938895342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 0x7f80f0104810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009910 tx=0x7f80e4009c20 comp rx=0 tx=0).stop 2024-05-15T15:59:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.254+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3938895342 shutdown_connections 2024-05-15T15:59:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.254+0000 7f80f7a3a700 1 --2- 172.21.15.49:0/3938895342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 0x7f80f0104810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.254+0000 7f80f7a3a700 1 --2- 172.21.15.49:0/3938895342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f00ffe10 0x7f80f01001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.254+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3938895342 >> 172.21.15.49:0/3938895342 conn(0x7f80f00fb700 msgr2=0x7f80f00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.254+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3938895342 shutdown_connections 2024-05-15T15:59:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.254+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3938895342 wait complete. 2024-05-15T15:59:45.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.254+0000 7f80f7a3a700 1 Processor -- start 2024-05-15T15:59:45.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f7a3a700 1 -- start start 2024-05-15T15:59:45.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f7a3a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f00ffe10 0x7f80f0198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:45.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f7a3a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 0x7f80f01993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:45.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f7a3a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f0199aa0 con 0x7f80f01007c0 2024-05-15T15:59:45.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f7a3a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f019d770 con 0x7f80f00ffe10 2024-05-15T15:59:45.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f4fd5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 0x7f80f01993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:45.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f4fd5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 0x7f80f01993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60292/0 (socket says 172.21.15.49:60292) 2024-05-15T15:59:45.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f4fd5700 1 -- 172.21.15.49:0/3544090536 learned_addr learned my addr 172.21.15.49:0/3544090536 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f4fd5700 1 -- 172.21.15.49:0/3544090536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f00ffe10 msgr2=0x7f80f0198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f57d6700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f00ffe10 0x7f80f0198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f4fd5700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f00ffe10 0x7f80f0198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f4fd5700 1 -- 172.21.15.49:0/3544090536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40095f0 con 0x7f80f01007c0 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.255+0000 7f80f4fd5700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 0x7f80f01993c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f80e40092e0 tx=0x7f80e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.256+0000 7f80ee7fc700 1 -- 172.21.15.49:0/3544090536 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e401d070 con 0x7f80f01007c0 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.256+0000 7f80ee7fc700 1 -- 172.21.15.49:0/3544090536 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80e400f460 con 0x7f80f01007c0 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.256+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f019d9f0 con 0x7f80f01007c0 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.256+0000 7f80ee7fc700 1 -- 172.21.15.49:0/3544090536 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e4017630 con 0x7f80f01007c0 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.256+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f019dee0 con 0x7f80f01007c0 2024-05-15T15:59:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.257+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f01012f0 con 0x7f80f01007c0 2024-05-15T15:59:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.257+0000 7f80ee7fc700 1 -- 172.21.15.49:0/3544090536 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f80e400f5d0 con 0x7f80f01007c0 2024-05-15T15:59:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.258+0000 7f80ee7fc700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d806e340 0x7f80d8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T15:59:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.258+0000 7f80ee7fc700 1 -- 172.21.15.49:0/3544090536 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f80e408f7d0 con 0x7f80f01007c0 2024-05-15T15:59:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.258+0000 7f80f57d6700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d806e340 0x7f80d8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T15:59:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.258+0000 7f80f57d6700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d806e340 0x7f80d8070800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f80e00060b0 tx=0x7f80e0008640 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T15:59:45.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.263+0000 7f80ee7fc700 1 -- 172.21.15.49:0/3544090536 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f80e4058940 con 0x7f80f01007c0 2024-05-15T15:59:45.379 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:45 smithi049 podman[123654]: 2024-05-15 15:59:45.358724711 +0000 UTC m=+1.552164142 container cleanup 9d74a0af6b578bbfe6ee14860043b9bc817cddb8b1186b476782f5fb234d8348 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=reef-2965081, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T15:59:45.380 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:45 smithi049 bash[123654]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0 2024-05-15T15:59:45.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.593+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f80f010cdb0 con 0x7f80f01007c0 2024-05-15T15:59:45.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.593+0000 7f80ee7fc700 1 -- 172.21.15.49:0/3544090536 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+290 (secure 0 0 0) 0x7f80e408de60 con 0x7f80f01007c0 2024-05-15T15:59:45.596 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down 2024-05-15T15:59:45.596 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T15:59:45.596 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T15:59:45.596 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-15T15:59:45.596 INFO:teuthology.orchestra.run.smithi049.stdout: osd.0 (root=default,host=smithi049) is down 2024-05-15T15:59:45.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d806e340 msgr2=0x7f80d8070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:45.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d806e340 0x7f80d8070800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f80e00060b0 tx=0x7f80e0008640 comp rx=0 tx=0).stop 2024-05-15T15:59:45.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 msgr2=0x7f80f01993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T15:59:45.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 0x7f80f01993c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f80e40092e0 tx=0x7f80e400be20 comp rx=0 tx=0).stop 2024-05-15T15:59:45.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 shutdown_connections 2024-05-15T15:59:45.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d806e340 0x7f80d8070800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f00ffe10 0x7f80f0198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 --2- 172.21.15.49:0/3544090536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f01007c0 0x7f80f01993c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T15:59:45.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 >> 172.21.15.49:0/3544090536 conn(0x7f80f00fb700 msgr2=0x7f80f0069aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T15:59:45.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.595+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 shutdown_connections 2024-05-15T15:59:45.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T15:59:45.596+0000 7f80f7a3a700 1 -- 172.21.15.49:0/3544090536 wait complete. 2024-05-15T15:59:45.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:45 smithi144 ceph-mon[86635]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:45.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:45 smithi144 ceph-mon[86635]: pgmap v45: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:45.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:45 smithi144 ceph-mon[86635]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T15:59:45.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:45 smithi144 ceph-mon[86635]: osdmap e51: 6 total, 5 up, 6 in 2024-05-15T15:59:45.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:45 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2511774906' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:59:45.955 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:45 smithi049 podman[123718]: 2024-05-15 15:59:45.68561281 +0000 UTC m=+0.878316972 container remove 9d74a0af6b578bbfe6ee14860043b9bc817cddb8b1186b476782f5fb234d8348 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=reef-2965081, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-15T15:59:45.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:45 smithi049 ceph-mon[112142]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:45.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:45 smithi049 ceph-mon[112142]: pgmap v45: 65 pgs: 65 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T15:59:45.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:45 smithi049 ceph-mon[112142]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T15:59:45.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:45 smithi049 ceph-mon[112142]: osdmap e51: 6 total, 5 up, 6 in 2024-05-15T15:59:45.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:45 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2511774906' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T15:59:46.332 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:46 smithi049 podman[123853]: 2024-05-15 15:59:45.922692543 +0000 UTC m=+0.017784970 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T15:59:46.332 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:46 smithi049 podman[123853]: 2024-05-15 15:59:46.061899746 +0000 UTC m=+0.156992187 container create 5a196df72303864d9a76c36b4dd3fc0d259bc74a7c4a4cc978be1aa78c476137 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-15T15:59:46.333 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:46 smithi049 podman[123853]: 2024-05-15 15:59:46.256196085 +0000 UTC m=+0.351288505 container init 5a196df72303864d9a76c36b4dd3fc0d259bc74a7c4a4cc978be1aa78c476137 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-65480-squid-de5210d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-15T15:59:46.333 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:46 smithi049 podman[123853]: 2024-05-15 15:59:46.259698933 +0000 UTC m=+0.354791361 container start 5a196df72303864d9a76c36b4dd3fc0d259bc74a7c4a4cc978be1aa78c476137 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-65480-squid-de5210d) 2024-05-15T15:59:46.333 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:46 smithi049 podman[123853]: 2024-05-15 15:59:46.331049824 +0000 UTC m=+0.426142246 container attach 5a196df72303864d9a76c36b4dd3fc0d259bc74a7c4a4cc978be1aa78c476137 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-15T15:59:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:46 smithi049 ceph-mon[112142]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:46 smithi049 ceph-mon[112142]: osdmap e52: 6 total, 5 up, 6 in 2024-05-15T15:59:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:46 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3544090536' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T15:59:46.706 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:46 smithi049 podman[123853]: 2024-05-15 15:59:46.573215783 +0000 UTC m=+0.668308205 container died 5a196df72303864d9a76c36b4dd3fc0d259bc74a7c4a4cc978be1aa78c476137 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-15T15:59:46.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:46 smithi144 ceph-mon[86635]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T15:59:46.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:46 smithi144 ceph-mon[86635]: osdmap e52: 6 total, 5 up, 6 in 2024-05-15T15:59:46.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:46 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3544090536' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T15:59:47.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:47 smithi049 ceph-mon[112142]: pgmap v48: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/321 objects degraded (19.003%) 2024-05-15T15:59:47.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:47 smithi049 ceph-mon[112142]: Health check failed: Degraded data redundancy: 61/321 objects degraded (19.003%), 20 pgs degraded (PG_DEGRADED) 2024-05-15T15:59:47.706 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:47 smithi049 podman[123879]: 2024-05-15 15:59:47.26073066 +0000 UTC m=+0.678534774 container remove 5a196df72303864d9a76c36b4dd3fc0d259bc74a7c4a4cc978be1aa78c476137 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-65480-squid-de5210d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-15T15:59:47.706 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:47 smithi049 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.0.service: Deactivated successfully. 2024-05-15T15:59:47.706 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:47 smithi049 systemd[1]: Stopped Ceph osd.0 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:59:47.706 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:47 smithi049 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.0.service: Consumed 57.995s CPU time. 2024-05-15T15:59:47.706 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:47 smithi049 systemd[1]: Starting Ceph osd.0 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T15:59:47.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:47 smithi144 ceph-mon[86635]: pgmap v48: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/321 objects degraded (19.003%) 2024-05-15T15:59:47.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:47 smithi144 ceph-mon[86635]: Health check failed: Degraded data redundancy: 61/321 objects degraded (19.003%), 20 pgs degraded (PG_DEGRADED) 2024-05-15T15:59:48.016 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:47 smithi049 podman[123979]: 2024-05-15 15:59:47.629670357 +0000 UTC m=+0.018370046 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T15:59:48.016 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:47 smithi049 podman[123979]: 2024-05-15 15:59:47.761972328 +0000 UTC m=+0.150672011 container create 568e6e8051bdb279cd318de5df7048bb3cba4dcccd19c7fc695555b4e0c1b763 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T15:59:48.455 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 podman[123979]: 2024-05-15 15:59:48.014844822 +0000 UTC m=+0.403544497 container init 568e6e8051bdb279cd318de5df7048bb3cba4dcccd19c7fc695555b4e0c1b763 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T15:59:48.455 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 podman[123979]: 2024-05-15 15:59:48.018885548 +0000 UTC m=+0.407585223 container start 568e6e8051bdb279cd318de5df7048bb3cba4dcccd19c7fc695555b4e0c1b763 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-15T15:59:48.456 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 podman[123979]: 2024-05-15 15:59:48.114577828 +0000 UTC m=+0.503277519 container attach 568e6e8051bdb279cd318de5df7048bb3cba4dcccd19c7fc695555b4e0c1b763 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-65480-squid-de5210d, ceph=True) 2024-05-15T15:59:48.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate[123993]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T15:59:48.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate[123993]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T15:59:48.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 bash[123979]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T15:59:48.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 bash[123979]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T15:59:48.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 bash[123979]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-15T15:59:48.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 bash[123979]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 bash[123979]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 bash[123979]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 bash[123979]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 bash[123979]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 bash[123979]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate[123993]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate[123993]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate[123993]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate[123993]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate[123993]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate[123993]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate[123993]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-15T15:59:48.957 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:48 smithi049 podman[123979]: 2024-05-15 15:59:48.554088327 +0000 UTC m=+0.942788004 container died 568e6e8051bdb279cd318de5df7048bb3cba4dcccd19c7fc695555b4e0c1b763 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-15T15:59:49.702 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:49 smithi049 ceph-mon[112142]: pgmap v49: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/321 objects degraded (19.003%) 2024-05-15T15:59:49.702 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:49 smithi049 podman[124138]: 2024-05-15 15:59:49.360542623 +0000 UTC m=+0.800305386 container remove 568e6e8051bdb279cd318de5df7048bb3cba4dcccd19c7fc695555b4e0c1b763 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-15T15:59:49.702 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:49 smithi049 podman[124200]: 2024-05-15 15:59:49.586463578 +0000 UTC m=+0.114808426 container create 84a251f158d31b88de5e48bb252056fa43b3ae4664e0b583b81095e1b04aca80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-15T15:59:49.702 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:49 smithi049 podman[124200]: 2024-05-15 15:59:49.490645005 +0000 UTC m=+0.018989873 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T15:59:49.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:49 smithi144 ceph-mon[86635]: pgmap v49: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/321 objects degraded (19.003%) 2024-05-15T15:59:49.955 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:49 smithi049 podman[124200]: 2024-05-15 15:59:49.79711482 +0000 UTC m=+0.325459689 container init 84a251f158d31b88de5e48bb252056fa43b3ae4664e0b583b81095e1b04aca80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-15T15:59:49.955 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:49 smithi049 podman[124200]: 2024-05-15 15:59:49.800545198 +0000 UTC m=+0.328890053 container start 84a251f158d31b88de5e48bb252056fa43b3ae4664e0b583b81095e1b04aca80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T15:59:49.955 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:49 smithi049 ceph-osd[124219]: -- 172.21.15.49:0/2458437452 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5ac6ab680 con 0x55f5ac675000 2024-05-15T15:59:49.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:49 smithi049 bash[124200]: 84a251f158d31b88de5e48bb252056fa43b3ae4664e0b583b81095e1b04aca80 2024-05-15T15:59:49.956 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:49 smithi049 systemd[1]: Started Ceph osd.0 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T15:59:50.705 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:50 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[124215]: 2024-05-15T15:59:50.343+0000 7ff43f7c77c0 -1 Falling back to public interface 2024-05-15T15:59:51.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:51.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:51.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:59:51.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:50 smithi049 ceph-mon[112142]: pgmap v50: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/321 objects degraded (19.003%) 2024-05-15T15:59:51.165 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:51.165 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:59:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T15:59:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:50 smithi144 ceph-mon[86635]: pgmap v50: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/321 objects degraded (19.003%) 2024-05-15T15:59:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T15:59:53.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:53 smithi144 ceph-mon[86635]: pgmap v51: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 639 B/s wr, 0 op/s; 61/321 objects degraded (19.003%) 2024-05-15T15:59:53.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:53 smithi144 ceph-mon[86635]: mgrmap e39: smithi049.vuhzgf(active, since 92s), standbys: smithi144.whcjkr 2024-05-15T15:59:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:53 smithi049 ceph-mon[112142]: pgmap v51: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 639 B/s wr, 0 op/s; 61/321 objects degraded (19.003%) 2024-05-15T15:59:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:53 smithi049 ceph-mon[112142]: mgrmap e39: smithi049.vuhzgf(active, since 92s), standbys: smithi144.whcjkr 2024-05-15T15:59:55.422 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:55 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[124215]: 2024-05-15T15:59:55.146+0000 7ff43f7c77c0 -1 osd.0 50 log_to_monitors true 2024-05-15T15:59:55.423 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:55 smithi049 ceph-mon[112142]: pgmap v52: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 529 B/s wr, 0 op/s; 61/321 objects degraded (19.003%) 2024-05-15T15:59:55.423 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:55.423 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:55.423 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:55 smithi049 ceph-mon[112142]: from='osd.0 [v2:172.21.15.49:6802/1701263964,v1:172.21.15.49:6803/1701263964]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-15T15:59:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:55 smithi144 ceph-mon[86635]: pgmap v52: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 529 B/s wr, 0 op/s; 61/321 objects degraded (19.003%) 2024-05-15T15:59:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:55.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:55 smithi144 ceph-mon[86635]: from='osd.0 [v2:172.21.15.49:6802/1701263964,v1:172.21.15.49:6803/1701263964]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-15T15:59:56.455 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 15:59:56 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[124215]: 2024-05-15T15:59:56.175+0000 7ff42cc16700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T15:59:57.187 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:57 smithi049 ceph-mon[112142]: pgmap v53: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 480 B/s wr, 0 op/s; 61/321 objects degraded (19.003%) 2024-05-15T15:59:57.188 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:57 smithi049 ceph-mon[112142]: from='osd.0 [v2:172.21.15.49:6802/1701263964,v1:172.21.15.49:6803/1701263964]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-15T15:59:57.188 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:57 smithi049 ceph-mon[112142]: osdmap e53: 6 total, 5 up, 6 in 2024-05-15T15:59:57.188 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:57 smithi049 ceph-mon[112142]: from='osd.0 [v2:172.21.15.49:6802/1701263964,v1:172.21.15.49:6803/1701263964]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T15:59:57.188 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:57 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:57.188 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:57 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:57.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:57 smithi144 ceph-mon[86635]: pgmap v53: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 480 B/s wr, 0 op/s; 61/321 objects degraded (19.003%) 2024-05-15T15:59:57.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:57 smithi144 ceph-mon[86635]: from='osd.0 [v2:172.21.15.49:6802/1701263964,v1:172.21.15.49:6803/1701263964]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-15T15:59:57.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:57 smithi144 ceph-mon[86635]: osdmap e53: 6 total, 5 up, 6 in 2024-05-15T15:59:57.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:57 smithi144 ceph-mon[86635]: from='osd.0 [v2:172.21.15.49:6802/1701263964,v1:172.21.15.49:6803/1701263964]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T15:59:57.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:57 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:57.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:57 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T15:59:58.341 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:58 smithi049 ceph-mon[112142]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T15:59:58.341 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:58 smithi049 ceph-mon[112142]: osd.0 [v2:172.21.15.49:6802/1701263964,v1:172.21.15.49:6803/1701263964] boot 2024-05-15T15:59:58.341 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:58 smithi049 ceph-mon[112142]: osdmap e54: 6 total, 6 up, 6 in 2024-05-15T15:59:58.341 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:58 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:59:58.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:58 smithi144 ceph-mon[86635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T15:59:58.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:58 smithi144 ceph-mon[86635]: osd.0 [v2:172.21.15.49:6802/1701263964,v1:172.21.15.49:6803/1701263964] boot 2024-05-15T15:59:58.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:58 smithi144 ceph-mon[86635]: osdmap e54: 6 total, 6 up, 6 in 2024-05-15T15:59:58.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:58 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T15:59:59.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:59 smithi144 ceph-mon[86635]: pgmap v56: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s; 61/321 objects degraded (19.003%) 2024-05-15T15:59:59.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 15:59:59 smithi144 ceph-mon[86635]: osdmap e55: 6 total, 6 up, 6 in 2024-05-15T15:59:59.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:59 smithi049 ceph-mon[112142]: pgmap v56: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s; 61/321 objects degraded (19.003%) 2024-05-15T15:59:59.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 15:59:59 smithi049 ceph-mon[112142]: osdmap e55: 6 total, 6 up, 6 in 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 61/321 objects degraded (19.003%), 20 pgs degraded 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: [WRN] PG_DEGRADED: Degraded data redundancy: 61/321 objects degraded (19.003%), 20 pgs degraded 2024-05-15T16:00:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.c is active+undersized+degraded, acting [2,3] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.e is active+undersized+degraded, acting [2,3] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.f is active+undersized+degraded, acting [4,5] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-05-15T16:00:01.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:01.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 61/321 objects degraded (19.003%), 20 pgs degraded 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: [WRN] PG_DEGRADED: Degraded data redundancy: 61/321 objects degraded (19.003%), 20 pgs degraded 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-05-15T16:00:01.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.c is active+undersized+degraded, acting [2,3] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.e is active+undersized+degraded, acting [2,3] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.f is active+undersized+degraded, acting [4,5] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-05-15T16:00:01.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-05-15T16:00:01.208 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:00:01.208 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:01.208 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:01.208 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:01.208 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-15T16:00:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:01 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-15T16:00:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:01 smithi144 ceph-mon[86635]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-15T16:00:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:01 smithi144 ceph-mon[86635]: pgmap v58: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 61/321 objects degraded (19.003%) 2024-05-15T16:00:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:01 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-15T16:00:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:01 smithi049 ceph-mon[112142]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-15T16:00:02.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:01 smithi049 ceph-mon[112142]: pgmap v58: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 61/321 objects degraded (19.003%) 2024-05-15T16:00:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:02 smithi144 ceph-mon[86635]: pgmap v59: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s; 70 B/s, 0 objects/s recovering 2024-05-15T16:00:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:02 smithi144 ceph-mon[86635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 61/321 objects degraded (19.003%), 20 pgs degraded) 2024-05-15T16:00:03.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:02 smithi049 ceph-mon[112142]: pgmap v59: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s; 70 B/s, 0 objects/s recovering 2024-05-15T16:00:03.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:02 smithi049 ceph-mon[112142]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 61/321 objects degraded (19.003%), 20 pgs degraded) 2024-05-15T16:00:05.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:05 smithi144 ceph-mon[86635]: pgmap v60: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 899 B/s rd, 0 op/s; 53 B/s, 0 objects/s recovering 2024-05-15T16:00:05.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:05.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:05 smithi049 ceph-mon[112142]: pgmap v60: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 899 B/s rd, 0 op/s; 53 B/s, 0 objects/s recovering 2024-05-15T16:00:05.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:06.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:00:06.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:00:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:07 smithi144 ceph-mon[86635]: pgmap v61: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s; 47 B/s, 0 objects/s recovering 2024-05-15T16:00:07.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:07 smithi049 ceph-mon[112142]: pgmap v61: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s; 47 B/s, 0 objects/s recovering 2024-05-15T16:00:09.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:09 smithi144 ceph-mon[86635]: pgmap v62: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 42 B/s, 0 objects/s recovering 2024-05-15T16:00:09.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:09 smithi049 ceph-mon[112142]: pgmap v62: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 42 B/s, 0 objects/s recovering 2024-05-15T16:00:11.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:11 smithi144 ceph-mon[86635]: pgmap v63: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 1 op/s; 35 B/s, 0 objects/s recovering 2024-05-15T16:00:11.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:11.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:11 smithi049 ceph-mon[112142]: pgmap v63: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 1 op/s; 35 B/s, 0 objects/s recovering 2024-05-15T16:00:11.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:13 smithi144 ceph-mon[86635]: pgmap v64: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 35 B/s, 0 objects/s recovering 2024-05-15T16:00:13.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:13 smithi049 ceph-mon[112142]: pgmap v64: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 35 B/s, 0 objects/s recovering 2024-05-15T16:00:15.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:15 smithi144 ceph-mon[86635]: pgmap v65: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:15.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-15T16:00:15.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:15 smithi049 ceph-mon[112142]: pgmap v65: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:15.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-15T16:00:15.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.719+0000 7fd82527c700 1 -- 172.21.15.49:0/2610683978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 msgr2=0x7fd8201058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.719+0000 7fd82527c700 1 --2- 172.21.15.49:0/2610683978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 0x7fd8201058d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd814009910 tx=0x7fd814009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.719+0000 7fd82527c700 1 -- 172.21.15.49:0/2610683978 shutdown_connections 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.719+0000 7fd82527c700 1 --2- 172.21.15.49:0/2610683978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 0x7fd8201058d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.719+0000 7fd82527c700 1 --2- 172.21.15.49:0/2610683978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd820100ed0 0x7fd8201012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.719+0000 7fd82527c700 1 -- 172.21.15.49:0/2610683978 >> 172.21.15.49:0/2610683978 conn(0x7fd8200fc7a0 msgr2=0x7fd8200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.719+0000 7fd82527c700 1 -- 172.21.15.49:0/2610683978 shutdown_connections 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.719+0000 7fd82527c700 1 -- 172.21.15.49:0/2610683978 wait complete. 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.719+0000 7fd82527c700 1 Processor -- start 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd82527c700 1 -- start start 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd82527c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd820100ed0 0x7fd8200732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd82527c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 0x7fd8200737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd82527c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820073d20 con 0x7fd820101880 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd82527c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820073e60 con 0x7fd820100ed0 2024-05-15T16:00:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd81e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 0x7fd8200737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd81e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 0x7fd8200737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36270/0 (socket says 172.21.15.49:36270) 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd81e7fc700 1 -- 172.21.15.49:0/550648924 learned_addr learned my addr 172.21.15.49:0/550648924 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd81effd700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd820100ed0 0x7fd8200732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd81e7fc700 1 -- 172.21.15.49:0/550648924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd820100ed0 msgr2=0x7fd8200732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd81e7fc700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd820100ed0 0x7fd8200732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.720+0000 7fd81e7fc700 1 -- 172.21.15.49:0/550648924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8140095f0 con 0x7fd820101880 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.721+0000 7fd81e7fc700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 0x7fd8200737e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd814009fd0 tx=0x7fd81400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.721+0000 7fd807fff700 1 -- 172.21.15.49:0/550648924 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81401d070 con 0x7fd820101880 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.721+0000 7fd807fff700 1 -- 172.21.15.49:0/550648924 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8140052a0 con 0x7fd820101880 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.721+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8200740e0 con 0x7fd820101880 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.721+0000 7fd807fff700 1 -- 172.21.15.49:0/550648924 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd814022470 con 0x7fd820101880 2024-05-15T16:00:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.721+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8201a3020 con 0x7fd820101880 2024-05-15T16:00:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.722+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8201092d0 con 0x7fd820101880 2024-05-15T16:00:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.722+0000 7fd807fff700 1 -- 172.21.15.49:0/550648924 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd81400f6f0 con 0x7fd820101880 2024-05-15T16:00:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.723+0000 7fd807fff700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd80806e200 0x7fd8080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.723+0000 7fd807fff700 1 -- 172.21.15.49:0/550648924 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd81408f290 con 0x7fd820101880 2024-05-15T16:00:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.723+0000 7fd81effd700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd80806e200 0x7fd8080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.723+0000 7fd81effd700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd80806e200 0x7fd8080706c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd810009c60 tx=0x7fd810009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:15.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.728+0000 7fd807fff700 1 -- 172.21.15.49:0/550648924 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd8140586a0 con 0x7fd820101880 2024-05-15T16:00:15.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.959+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd82010de70 con 0x7fd80806e200 2024-05-15T16:00:15.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.960+0000 7fd807fff700 1 -- 172.21.15.49:0/550648924 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd82010de70 con 0x7fd80806e200 2024-05-15T16:00:15.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd80806e200 msgr2=0x7fd8080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd80806e200 0x7fd8080706c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd810009c60 tx=0x7fd810009260 comp rx=0 tx=0).stop 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 msgr2=0x7fd8200737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 0x7fd8200737e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd814009fd0 tx=0x7fd81400b5a0 comp rx=0 tx=0).stop 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 shutdown_connections 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd80806e200 0x7fd8080706c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd820100ed0 0x7fd8200732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 --2- 172.21.15.49:0/550648924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd820101880 0x7fd8200737e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 >> 172.21.15.49:0/550648924 conn(0x7fd8200fc7a0 msgr2=0x7fd8200fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 shutdown_connections 2024-05-15T16:00:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:15.963+0000 7fd82527c700 1 -- 172.21.15.49:0/550648924 wait complete. 2024-05-15T16:00:15.977 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:00:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.076+0000 7f15f7065700 1 -- 172.21.15.49:0/2213700249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f0105d30 msgr2=0x7f15f0106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.076+0000 7f15f7065700 1 --2- 172.21.15.49:0/2213700249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f0105d30 0x7f15f0106110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009910 tx=0x7f15e4009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:16.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.076+0000 7f15f7065700 1 -- 172.21.15.49:0/2213700249 shutdown_connections 2024-05-15T16:00:16.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.076+0000 7f15f7065700 1 --2- 172.21.15.49:0/2213700249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f00ff150 0x7f15f00ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.076+0000 7f15f7065700 1 --2- 172.21.15.49:0/2213700249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f0105d30 0x7f15f0106110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.076+0000 7f15f7065700 1 -- 172.21.15.49:0/2213700249 >> 172.21.15.49:0/2213700249 conn(0x7f15f0075fc0 msgr2=0x7f15f00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:16.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.076+0000 7f15f7065700 1 -- 172.21.15.49:0/2213700249 shutdown_connections 2024-05-15T16:00:16.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.076+0000 7f15f7065700 1 -- 172.21.15.49:0/2213700249 wait complete. 2024-05-15T16:00:16.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.077+0000 7f15f7065700 1 Processor -- start 2024-05-15T16:00:16.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.077+0000 7f15f7065700 1 -- start start 2024-05-15T16:00:16.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.077+0000 7f15f7065700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f00ff150 0x7f15f019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:16.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.077+0000 7f15f7065700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f0105d30 0x7f15f019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:16.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.077+0000 7f15f7065700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f019d500 con 0x7f15f00ff150 2024-05-15T16:00:16.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.077+0000 7f15f7065700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f0196ae0 con 0x7f15f0105d30 2024-05-15T16:00:16.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.077+0000 7f15f4e01700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f00ff150 0x7f15f019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.077+0000 7f15f4e01700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f00ff150 0x7f15f019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36292/0 (socket says 172.21.15.49:36292) 2024-05-15T16:00:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.077+0000 7f15f4e01700 1 -- 172.21.15.49:0/553882229 learned_addr learned my addr 172.21.15.49:0/553882229 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:16.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15effff700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f0105d30 0x7f15f019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15f4e01700 1 -- 172.21.15.49:0/553882229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f0105d30 msgr2=0x7f15f019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15f4e01700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f0105d30 0x7f15f019cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15f4e01700 1 -- 172.21.15.49:0/553882229 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40095f0 con 0x7f15f00ff150 2024-05-15T16:00:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15f4e01700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f00ff150 0x7f15f019c9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f15e400bd70 tx=0x7f15e400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15edffb700 1 -- 172.21.15.49:0/553882229 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e401d070 con 0x7f15f00ff150 2024-05-15T16:00:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15edffb700 1 -- 172.21.15.49:0/553882229 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15e40056e0 con 0x7f15f00ff150 2024-05-15T16:00:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f0196d60 con 0x7f15f00ff150 2024-05-15T16:00:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15edffb700 1 -- 172.21.15.49:0/553882229 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e400fe40 con 0x7f15f00ff150 2024-05-15T16:00:16.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.078+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f0197250 con 0x7f15f00ff150 2024-05-15T16:00:16.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.079+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f0109300 con 0x7f15f00ff150 2024-05-15T16:00:16.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.080+0000 7f15edffb700 1 -- 172.21.15.49:0/553882229 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f15e400f680 con 0x7f15f00ff150 2024-05-15T16:00:16.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.080+0000 7f15edffb700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15e006e1a0 0x7f15e0070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:16.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.081+0000 7f15edffb700 1 -- 172.21.15.49:0/553882229 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f15e408f4f0 con 0x7f15f00ff150 2024-05-15T16:00:16.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.081+0000 7f15effff700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15e006e1a0 0x7f15e0070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:16.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.081+0000 7f15effff700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15e006e1a0 0x7f15e0070660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15f0197ca0 tx=0x7f15dc00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:16.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.085+0000 7f15edffb700 1 -- 172.21.15.49:0/553882229 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f15e405c3b0 con 0x7f15f00ff150 2024-05-15T16:00:16.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.320+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15f0197fa0 con 0x7f15e006e1a0 2024-05-15T16:00:16.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.321+0000 7f15edffb700 1 -- 172.21.15.49:0/553882229 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f15f0197fa0 con 0x7f15e006e1a0 2024-05-15T16:00:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.323+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15e006e1a0 msgr2=0x7f15e0070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.323+0000 7f15f7065700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15e006e1a0 0x7f15e0070660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15f0197ca0 tx=0x7f15dc00b410 comp rx=0 tx=0).stop 2024-05-15T16:00:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.323+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f00ff150 msgr2=0x7f15f019c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.323+0000 7f15f7065700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f00ff150 0x7f15f019c9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f15e400bd70 tx=0x7f15e400bda0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.323+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 shutdown_connections 2024-05-15T16:00:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.323+0000 7f15f7065700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15e006e1a0 0x7f15e0070660 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.323+0000 7f15f7065700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15f00ff150 0x7f15f019c9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.323+0000 7f15f7065700 1 --2- 172.21.15.49:0/553882229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f0105d30 0x7f15f019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.323+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 >> 172.21.15.49:0/553882229 conn(0x7f15f0075fc0 msgr2=0x7f15f00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.324+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 shutdown_connections 2024-05-15T16:00:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.324+0000 7f15f7065700 1 -- 172.21.15.49:0/553882229 wait complete. 2024-05-15T16:00:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.433+0000 7fa1789c2700 1 -- 172.21.15.49:0/3986203310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa174100770 msgr2=0x7fa1741047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.433+0000 7fa1789c2700 1 --2- 172.21.15.49:0/3986203310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa174100770 0x7fa1741047c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009910 tx=0x7fa15c009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.433+0000 7fa1789c2700 1 -- 172.21.15.49:0/3986203310 shutdown_connections 2024-05-15T16:00:16.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.433+0000 7fa1789c2700 1 --2- 172.21.15.49:0/3986203310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa174100770 0x7fa1741047c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.433+0000 7fa1789c2700 1 --2- 172.21.15.49:0/3986203310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1740ffdc0 0x7fa1741001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.433+0000 7fa1789c2700 1 -- 172.21.15.49:0/3986203310 >> 172.21.15.49:0/3986203310 conn(0x7fa174076800 msgr2=0x7fa1740fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:16.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.433+0000 7fa1789c2700 1 -- 172.21.15.49:0/3986203310 shutdown_connections 2024-05-15T16:00:16.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.433+0000 7fa1789c2700 1 -- 172.21.15.49:0/3986203310 wait complete. 2024-05-15T16:00:16.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.434+0000 7fa1789c2700 1 Processor -- start 2024-05-15T16:00:16.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.434+0000 7fa1789c2700 1 -- start start 2024-05-15T16:00:16.438 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:16 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-15T16:00:16.439 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:16 smithi049 ceph-mon[112142]: Upgrade: osd.1 is safe to restart 2024-05-15T16:00:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.434+0000 7fa1789c2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1740ffdc0 0x7fa17419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.434+0000 7fa1789c2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa174100770 0x7fa17419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.434+0000 7fa1789c2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17419d470 con 0x7fa1740ffdc0 2024-05-15T16:00:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.434+0000 7fa1789c2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1741969e0 con 0x7fa174100770 2024-05-15T16:00:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.434+0000 7fa17259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1740ffdc0 0x7fa17419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.434+0000 7fa17259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1740ffdc0 0x7fa17419c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36310/0 (socket says 172.21.15.49:36310) 2024-05-15T16:00:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.434+0000 7fa17259c700 1 -- 172.21.15.49:0/783910200 learned_addr learned my addr 172.21.15.49:0/783910200 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:16.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa171d9b700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa174100770 0x7fa17419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa17259c700 1 -- 172.21.15.49:0/783910200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa174100770 msgr2=0x7fa17419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa17259c700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa174100770 0x7fa17419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa17259c700 1 -- 172.21.15.49:0/783910200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c0095f0 con 0x7fa1740ffdc0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa171d9b700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa174100770 0x7fa17419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa17259c700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1740ffdc0 0x7fa17419c8a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa1640060b0 tx=0x7fa16400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa16b7fe700 1 -- 172.21.15.49:0/783910200 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa16400dda0 con 0x7fa1740ffdc0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa174196cc0 con 0x7fa1740ffdc0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa16b7fe700 1 -- 172.21.15.49:0/783910200 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa16400f040 con 0x7fa1740ffdc0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa16b7fe700 1 -- 172.21.15.49:0/783910200 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa164014790 con 0x7fa1740ffdc0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.435+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa174197210 con 0x7fa1740ffdc0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.436+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1741081c0 con 0x7fa1740ffdc0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.437+0000 7fa16b7fe700 1 -- 172.21.15.49:0/783910200 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa1640148f0 con 0x7fa1740ffdc0 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.437+0000 7fa16b7fe700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa16006e210 0x7fa1600706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:16.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.438+0000 7fa171d9b700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa16006e210 0x7fa1600706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:16.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.438+0000 7fa16b7fe700 1 -- 172.21.15.49:0/783910200 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa16408daf0 con 0x7fa1740ffdc0 2024-05-15T16:00:16.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.438+0000 7fa171d9b700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa16006e210 0x7fa1600706d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa1740fbf00 tx=0x7fa15c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:16.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.442+0000 7fa16b7fe700 1 -- 172.21.15.49:0/783910200 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa16405ab50 con 0x7fa1740ffdc0 2024-05-15T16:00:16.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:16 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-15T16:00:16.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:16 smithi144 ceph-mon[86635]: Upgrade: osd.1 is safe to restart 2024-05-15T16:00:16.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.660+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa17410de60 con 0x7fa16006e210 2024-05-15T16:00:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.671+0000 7fa16b7fe700 1 -- 172.21.15.49:0/783910200 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa17410de60 con 0x7fa16006e210 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (3m) 21s ago 15m 26.2M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (15m) 21s ago 15m 8464k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (14m) 45s ago 14m 8162k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (54s) 21s ago 15m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (48s) 45s ago 14m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (3m) 21s ago 14m 75.7M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (9m) 21s ago 9m 90.0M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (9m) 21s ago 9m 152M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (9m) 45s ago 9m 26.7M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (9m) 45s ago 9m 28.4M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:00:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (5m) 21s ago 16m 500M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (4m) 45s ago 13m 451M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (2m) 21s ago 16m 45.0M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (99s) 45s ago 13m 35.0M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (4m) 21s ago 15m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (4m) 45s ago 14m 16.2M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (27s) 21s ago 13m 13.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (12m) 21s ago 12m 220M 1216M 18.2.2-1343-g2965081f a04a73ffe079 204e65e37659 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (12m) 21s ago 12m 216M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (11m) 45s ago 11m 261M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (11m) 45s ago 11m 217M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (10m) 45s ago 10m 203M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T16:00:16.675 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (3m) 21s ago 14m 64.5M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa16006e210 msgr2=0x7fa1600706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa16006e210 0x7fa1600706d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa1740fbf00 tx=0x7fa15c01a040 comp rx=0 tx=0).stop 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1740ffdc0 msgr2=0x7fa17419c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1740ffdc0 0x7fa17419c8a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa1640060b0 tx=0x7fa16400d3b0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 shutdown_connections 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa16006e210 0x7fa1600706d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1740ffdc0 0x7fa17419c8a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 --2- 172.21.15.49:0/783910200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa174100770 0x7fa17419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 >> 172.21.15.49:0/783910200 conn(0x7fa174076800 msgr2=0x7fa174103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 shutdown_connections 2024-05-15T16:00:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.673+0000 7fa1789c2700 1 -- 172.21.15.49:0/783910200 wait complete. 2024-05-15T16:00:16.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.781+0000 7f0474462700 1 -- 172.21.15.49:0/111247436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 msgr2=0x7f046c1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.781+0000 7f0474462700 1 --2- 172.21.15.49:0/111247436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 0x7f046c1022a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0460009910 tx=0x7f0460009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:16.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.781+0000 7f0474462700 1 -- 172.21.15.49:0/111247436 shutdown_connections 2024-05-15T16:00:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.781+0000 7f0474462700 1 --2- 172.21.15.49:0/111247436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c1027e0 0x7f046c10acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.781+0000 7f0474462700 1 --2- 172.21.15.49:0/111247436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 0x7f046c1022a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.781+0000 7f0474462700 1 -- 172.21.15.49:0/111247436 >> 172.21.15.49:0/111247436 conn(0x7f046c0fb690 msgr2=0x7f046c0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.781+0000 7f0474462700 1 -- 172.21.15.49:0/111247436 shutdown_connections 2024-05-15T16:00:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.781+0000 7f0474462700 1 -- 172.21.15.49:0/111247436 wait complete. 2024-05-15T16:00:16.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.782+0000 7f0474462700 1 Processor -- start 2024-05-15T16:00:16.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.782+0000 7f0474462700 1 -- start start 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f0474462700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 0x7f046c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f0474462700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c1027e0 0x7f046c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f0474462700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c19d520 con 0x7f046c101ec0 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f0474462700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c196b00 con 0x7f046c1027e0 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f04721fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 0x7f046c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f04721fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 0x7f046c19c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36326/0 (socket says 172.21.15.49:36326) 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f04721fe700 1 -- 172.21.15.49:0/2456027297 learned_addr learned my addr 172.21.15.49:0/2456027297 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f04719fd700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c1027e0 0x7f046c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f04721fe700 1 -- 172.21.15.49:0/2456027297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c1027e0 msgr2=0x7f046c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f04721fe700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c1027e0 0x7f046c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f04721fe700 1 -- 172.21.15.49:0/2456027297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04600095f0 con 0x7f046c101ec0 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f04721fe700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 0x7f046c19c9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0460006010 tx=0x7f046000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f045f7fe700 1 -- 172.21.15.49:0/2456027297 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f046001d070 con 0x7f046c101ec0 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f045f7fe700 1 -- 172.21.15.49:0/2456027297 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f046000f460 con 0x7f046c101ec0 2024-05-15T16:00:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046c196d80 con 0x7f046c101ec0 2024-05-15T16:00:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f045f7fe700 1 -- 172.21.15.49:0/2456027297 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0460017630 con 0x7f046c101ec0 2024-05-15T16:00:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.783+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046c197270 con 0x7f046c101ec0 2024-05-15T16:00:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.784+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046c105650 con 0x7f046c101ec0 2024-05-15T16:00:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.785+0000 7f045f7fe700 1 -- 172.21.15.49:0/2456027297 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f046000f5d0 con 0x7f046c101ec0 2024-05-15T16:00:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.785+0000 7f045f7fe700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.785+0000 7f045f7fe700 1 -- 172.21.15.49:0/2456027297 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f046008f550 con 0x7f046c101ec0 2024-05-15T16:00:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.785+0000 7f04719fd700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.786+0000 7f04719fd700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0468006fd0 tx=0x7f0468008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:16.790+0000 7f045f7fe700 1 -- 172.21.15.49:0/2456027297 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f046005c550 con 0x7f046c101ec0 2024-05-15T16:00:17.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.092+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f046c197fc0 con 0x7f046c101ec0 2024-05-15T16:00:17.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.093+0000 7f045f7fe700 1 -- 172.21.15.49:0/2456027297 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f046005c0e0 con 0x7f046c101ec0 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 5, 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 1 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T16:00:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:00:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:00:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 9, 2024-05-15T16:00:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 5 2024-05-15T16:00:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:00:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:00:17.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.095+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 msgr2=0x7f04580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.095+0000 7f0474462700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0468006fd0 tx=0x7f0468008040 comp rx=0 tx=0).stop 2024-05-15T16:00:17.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.095+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 msgr2=0x7f046c19c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.095+0000 7f0474462700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 0x7f046c19c9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0460006010 tx=0x7f046000be20 comp rx=0 tx=0).stop 2024-05-15T16:00:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.095+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 shutdown_connections 2024-05-15T16:00:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.095+0000 7f0474462700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.095+0000 7f0474462700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101ec0 0x7f046c19c9c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.095+0000 7f0474462700 1 --2- 172.21.15.49:0/2456027297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c1027e0 0x7f046c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.095+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 >> 172.21.15.49:0/2456027297 conn(0x7f046c0fb690 msgr2=0x7f046c1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.096+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 shutdown_connections 2024-05-15T16:00:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.096+0000 7f0474462700 1 -- 172.21.15.49:0/2456027297 wait complete. 2024-05-15T16:00:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 -- 172.21.15.49:0/903353883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed8100770 msgr2=0x7f7ed81047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 --2- 172.21.15.49:0/903353883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed8100770 0x7f7ed81047c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009910 tx=0x7f7ec8009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 -- 172.21.15.49:0/903353883 shutdown_connections 2024-05-15T16:00:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 --2- 172.21.15.49:0/903353883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed8100770 0x7f7ed81047c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 --2- 172.21.15.49:0/903353883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed80ffdc0 0x7f7ed81001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 -- 172.21.15.49:0/903353883 >> 172.21.15.49:0/903353883 conn(0x7f7ed8076800 msgr2=0x7f7ed80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 -- 172.21.15.49:0/903353883 shutdown_connections 2024-05-15T16:00:17.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 -- 172.21.15.49:0/903353883 wait complete. 2024-05-15T16:00:17.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 Processor -- start 2024-05-15T16:00:17.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.207+0000 7f7ee01a1700 1 -- start start 2024-05-15T16:00:17.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7ee01a1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffdc0 0x7f7ed80fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:17.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7ee01a1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100770 0x7f7ed80fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:17.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7ee01a1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed80fc760 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7ee01a1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed80fc8a0 con 0x7f7ed8100770 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7eddf3d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffdc0 0x7f7ed80fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7eddf3d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffdc0 0x7f7ed80fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36336/0 (socket says 172.21.15.49:36336) 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7eddf3d700 1 -- 172.21.15.49:0/1557981891 learned_addr learned my addr 172.21.15.49:0/1557981891 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7edd73c700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100770 0x7f7ed80fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7eddf3d700 1 -- 172.21.15.49:0/1557981891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100770 msgr2=0x7f7ed80fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7eddf3d700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100770 0x7f7ed80fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7eddf3d700 1 -- 172.21.15.49:0/1557981891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec80095f0 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7edd73c700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100770 0x7f7ed80fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.208+0000 7f7eddf3d700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffdc0 0x7f7ed80fbce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400d8d0 tx=0x7f7ed400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.209+0000 7f7eceffd700 1 -- 172.21.15.49:0/1557981891 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ed4009940 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.209+0000 7f7eceffd700 1 -- 172.21.15.49:0/1557981891 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ed4010460 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.209+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed81a2d70 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.209+0000 7f7eceffd700 1 -- 172.21.15.49:0/1557981891 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ed400f5f0 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.209+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed81a30d0 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.210+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed81012a0 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.210+0000 7f7eceffd700 1 -- 172.21.15.49:0/1557981891 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7ed40105d0 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.211+0000 7f7eceffd700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7ec406e210 0x7f7ec40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.211+0000 7f7eceffd700 1 -- 172.21.15.49:0/1557981891 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7ed408e0a0 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.211+0000 7f7edd73c700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7ec406e210 0x7f7ec40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:17.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.212+0000 7f7edd73c700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7ec406e210 0x7f7ec40706d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80fd3c0 tx=0x7f7ec801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:17.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.215+0000 7f7eceffd700 1 -- 172.21.15.49:0/1557981891 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7ed4057540 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.479+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7ed810ce00 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.480+0000 7f7eceffd700 1 -- 172.21.15.49:0/1557981891 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f7ed405ac00 con 0x7f7ed80ffdc0 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:e17 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 17 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:51:00.559034+0000 2024-05-15T16:00:17.483 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T16:00:17.484 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:00:17.485 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7ec406e210 msgr2=0x7f7ec40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7ec406e210 0x7f7ec40706d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80fd3c0 tx=0x7f7ec801a040 comp rx=0 tx=0).stop 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffdc0 msgr2=0x7f7ed80fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffdc0 0x7f7ed80fbce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400d8d0 tx=0x7f7ed400dc90 comp rx=0 tx=0).stop 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 shutdown_connections 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7ec406e210 0x7f7ec40706d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ed80ffdc0 0x7f7ed80fbce0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 --2- 172.21.15.49:0/1557981891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed8100770 0x7f7ed80fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 >> 172.21.15.49:0/1557981891 conn(0x7f7ed8076800 msgr2=0x7f7ed8068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 shutdown_connections 2024-05-15T16:00:17.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.483+0000 7f7ee01a1700 1 -- 172.21.15.49:0/1557981891 wait complete. 2024-05-15T16:00:17.487 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T16:00:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 -- 172.21.15.49:0/3787169758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 msgr2=0x7fb570100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 --2- 172.21.15.49:0/3787169758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 0x7fb570100250 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009910 tx=0x7fb56c009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 -- 172.21.15.49:0/3787169758 shutdown_connections 2024-05-15T16:00:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 --2- 172.21.15.49:0/3787169758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb570100820 0x7fb570104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 --2- 172.21.15.49:0/3787169758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 0x7fb570100250 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 -- 172.21.15.49:0/3787169758 >> 172.21.15.49:0/3787169758 conn(0x7fb5700fb760 msgr2=0x7fb5700fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 -- 172.21.15.49:0/3787169758 shutdown_connections 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 -- 172.21.15.49:0/3787169758 wait complete. 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 Processor -- start 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.595+0000 7fb577f6c700 1 -- start start 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb577f6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 0x7fb57019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb577f6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb570100820 0x7fb57019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb577f6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57019d4e0 con 0x7fb5700ffe70 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb577f6c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb570196ac0 con 0x7fb570100820 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb575d08700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 0x7fb57019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb575d08700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 0x7fb57019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36360/0 (socket says 172.21.15.49:36360) 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb575d08700 1 -- 172.21.15.49:0/3532230060 learned_addr learned my addr 172.21.15.49:0/3532230060 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb575507700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb570100820 0x7fb57019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb575d08700 1 -- 172.21.15.49:0/3532230060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb570100820 msgr2=0x7fb57019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb575d08700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb570100820 0x7fb57019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb575d08700 1 -- 172.21.15.49:0/3532230060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c0095f0 con 0x7fb5700ffe70 2024-05-15T16:00:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb575d08700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 0x7fb57019c980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00bd70 tx=0x7fb56c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb566ffd700 1 -- 172.21.15.49:0/3532230060 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56c01d070 con 0x7fb5700ffe70 2024-05-15T16:00:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb566ffd700 1 -- 172.21.15.49:0/3532230060 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb56c0056e0 con 0x7fb5700ffe70 2024-05-15T16:00:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb570196d40 con 0x7fb5700ffe70 2024-05-15T16:00:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.596+0000 7fb566ffd700 1 -- 172.21.15.49:0/3532230060 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56c00fe40 con 0x7fb5700ffe70 2024-05-15T16:00:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.597+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5701971e0 con 0x7fb5700ffe70 2024-05-15T16:00:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.598+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb570101350 con 0x7fb5700ffe70 2024-05-15T16:00:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.598+0000 7fb566ffd700 1 -- 172.21.15.49:0/3532230060 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb56c00f680 con 0x7fb5700ffe70 2024-05-15T16:00:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.599+0000 7fb566ffd700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb55c06e200 0x7fb55c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.599+0000 7fb566ffd700 1 -- 172.21.15.49:0/3532230060 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb56c08f470 con 0x7fb5700ffe70 2024-05-15T16:00:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.599+0000 7fb575507700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb55c06e200 0x7fb55c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:17.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.599+0000 7fb575507700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb55c06e200 0x7fb55c0706c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb570197c80 tx=0x7fb56000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:17.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.606+0000 7fb566ffd700 1 -- 172.21.15.49:0/3532230060 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb56c05c3b0 con 0x7fb5700ffe70 2024-05-15T16:00:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.835+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb57010ce40 con 0x7fb55c06e200 2024-05-15T16:00:17.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.836+0000 7fb566ffd700 1 -- 172.21.15.49:0/3532230060 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb57010ce40 con 0x7fb55c06e200 2024-05-15T16:00:17.839 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:00:17.839 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:00:17.839 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "7/23 daemons upgraded", 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading osd daemons", 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:00:17.840 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:00:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb55c06e200 msgr2=0x7fb55c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb55c06e200 0x7fb55c0706c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb570197c80 tx=0x7fb56000b410 comp rx=0 tx=0).stop 2024-05-15T16:00:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 msgr2=0x7fb57019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 0x7fb57019c980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00bd70 tx=0x7fb56c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 shutdown_connections 2024-05-15T16:00:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb55c06e200 0x7fb55c0706c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5700ffe70 0x7fb57019c980 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 --2- 172.21.15.49:0/3532230060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb570100820 0x7fb57019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 >> 172.21.15.49:0/3532230060 conn(0x7fb5700fb760 msgr2=0x7fb570069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 shutdown_connections 2024-05-15T16:00:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.839+0000 7fb577f6c700 1 -- 172.21.15.49:0/3532230060 wait complete. 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.948+0000 7fcdc271b700 1 -- 172.21.15.49:0/1947547889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc1007a0 msgr2=0x7fcdbc1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.948+0000 7fcdc271b700 1 --2- 172.21.15.49:0/1947547889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc1007a0 0x7fcdbc1047f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009910 tx=0x7fcdb0009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.948+0000 7fcdc271b700 1 -- 172.21.15.49:0/1947547889 shutdown_connections 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.948+0000 7fcdc271b700 1 --2- 172.21.15.49:0/1947547889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc1007a0 0x7fcdbc1047f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.948+0000 7fcdc271b700 1 --2- 172.21.15.49:0/1947547889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc0ffdf0 0x7fcdbc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.948+0000 7fcdc271b700 1 -- 172.21.15.49:0/1947547889 >> 172.21.15.49:0/1947547889 conn(0x7fcdbc0fb6a0 msgr2=0x7fcdbc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.948+0000 7fcdc271b700 1 -- 172.21.15.49:0/1947547889 shutdown_connections 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.949+0000 7fcdc271b700 1 -- 172.21.15.49:0/1947547889 wait complete. 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.949+0000 7fcdc271b700 1 Processor -- start 2024-05-15T16:00:17.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.949+0000 7fcdc271b700 1 -- start start 2024-05-15T16:00:17.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.949+0000 7fcdc271b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc0ffdf0 0x7fcdbc19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:17.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdc271b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc1007a0 0x7fcdbc19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:17.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdc271b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc1969f0 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdc271b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc196b60 con 0x7fcdbc1007a0 2024-05-15T16:00:17.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdbbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc0ffdf0 0x7fcdbc19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:17.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdbbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc0ffdf0 0x7fcdbc19c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36372/0 (socket says 172.21.15.49:36372) 2024-05-15T16:00:17.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdbbfff700 1 -- 172.21.15.49:0/4013356069 learned_addr learned my addr 172.21.15.49:0/4013356069 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdbb7fe700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc1007a0 0x7fcdbc19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdbbfff700 1 -- 172.21.15.49:0/4013356069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc1007a0 msgr2=0x7fcdbc19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdbbfff700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc1007a0 0x7fcdbc19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdbbfff700 1 -- 172.21.15.49:0/4013356069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb00095f0 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdbbfff700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc0ffdf0 0x7fcdbc19c940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00d8d0 tx=0x7fcdac00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdb97fa700 1 -- 172.21.15.49:0/4013356069 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdac0098a0 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdb97fa700 1 -- 172.21.15.49:0/4013356069 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdac010460 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc196e40 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdb97fa700 1 -- 172.21.15.49:0/4013356069 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdac00f5f0 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.950+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc197360 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.951+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc1012d0 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.952+0000 7fcdb97fa700 1 -- 172.21.15.49:0/4013356069 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcdac009a00 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.952+0000 7fcdb97fa700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcda406e200 0x7fcda40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:17.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.953+0000 7fcdb97fa700 1 -- 172.21.15.49:0/4013356069 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcdac08dfc0 con 0x7fcdbc0ffdf0 2024-05-15T16:00:17.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.953+0000 7fcdbb7fe700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcda406e200 0x7fcda40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:17.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.953+0000 7fcdbb7fe700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcda406e200 0x7fcda40706c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc1981f0 tx=0x7fcdb001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:17.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:17.957+0000 7fcdb97fa700 1 -- 172.21.15.49:0/4013356069 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcdac057350 con 0x7fcdbc0ffdf0 2024-05-15T16:00:18.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:18.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: pgmap v66: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:00:18.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:18.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:18.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: Upgrade: Updating osd.1 2024-05-15T16:00:18.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:18.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-15T16:00:18.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:00:18.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: Deploying daemon osd.1 on smithi049 2024-05-15T16:00:18.117 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2456027297' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:18.117 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:17 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1557981891' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:00:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: pgmap v66: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:00:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: Upgrade: Updating osd.1 2024-05-15T16:00:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-15T16:00:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:00:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: Deploying daemon osd.1 on smithi049 2024-05-15T16:00:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2456027297' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:18.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:17 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1557981891' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:00:18.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.252+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcdbc10cd90 con 0x7fcdbc0ffdf0 2024-05-15T16:00:18.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.253+0000 7fcdb97fa700 1 -- 172.21.15.49:0/4013356069 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fcdac05aa10 con 0x7fcdbc0ffdf0 2024-05-15T16:00:18.256 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T16:00:18.256 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:00:18.256 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:00:18.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcda406e200 msgr2=0x7fcda40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:18.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcda406e200 0x7fcda40706c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc1981f0 tx=0x7fcdb001a040 comp rx=0 tx=0).stop 2024-05-15T16:00:18.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc0ffdf0 msgr2=0x7fcdbc19c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:18.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc0ffdf0 0x7fcdbc19c940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00d8d0 tx=0x7fcdac00dbe0 comp rx=0 tx=0).stop 2024-05-15T16:00:18.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 shutdown_connections 2024-05-15T16:00:18.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcda406e200 0x7fcda40706c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:18.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdbc0ffdf0 0x7fcdbc19c940 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:18.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 --2- 172.21.15.49:0/4013356069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdbc1007a0 0x7fcdbc19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:18.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 >> 172.21.15.49:0/4013356069 conn(0x7fcdbc0fb6a0 msgr2=0x7fcdbc069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:18.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 shutdown_connections 2024-05-15T16:00:18.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:18.255+0000 7fcdc271b700 1 -- 172.21.15.49:0/4013356069 wait complete. 2024-05-15T16:00:19.154 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:18 smithi049 ceph-mon[112142]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:19.155 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:18 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4013356069' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:00:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:18 smithi144 ceph-mon[86635]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:18 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4013356069' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:00:20.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:19 smithi144 ceph-mon[86635]: pgmap v67: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:20.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:19 smithi049 ceph-mon[112142]: pgmap v67: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:20.555 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:20 smithi049 systemd[1]: Stopping Ceph osd.1 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:00:20.864 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:20 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[54740]: 2024-05-15T16:00:20.551+0000 7fb611a7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:00:20.864 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:20 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[54740]: 2024-05-15T16:00:20.551+0000 7fb611a7e700 -1 osd.1 55 *** Got signal Terminated *** 2024-05-15T16:00:20.864 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:20 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[54740]: 2024-05-15T16:00:20.551+0000 7fb611a7e700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:00:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:00:21.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:20 smithi049 ceph-mon[112142]: osd.1 marked itself down and dead 2024-05-15T16:00:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:00:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:20 smithi144 ceph-mon[86635]: osd.1 marked itself down and dead 2024-05-15T16:00:21.455 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:21 smithi049 podman[129285]: 2024-05-15 16:00:21.411154543 +0000 UTC m=+0.974223527 container died 204e65e376599822433f1d3d4742f415dbbdffcac2f3df7c9876abc752f41790 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-2965081, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-05-15T16:00:21.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:21 smithi049 ceph-mon[112142]: pgmap v68: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:21.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:21 smithi049 ceph-mon[112142]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:00:21.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:21 smithi049 ceph-mon[112142]: osdmap e56: 6 total, 5 up, 6 in 2024-05-15T16:00:21.956 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:21 smithi049 podman[129285]: 2024-05-15 16:00:21.727159727 +0000 UTC m=+1.290228711 container cleanup 204e65e376599822433f1d3d4742f415dbbdffcac2f3df7c9876abc752f41790 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1, GIT_CLEAN=True, RELEASE=reef-2965081, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-15T16:00:21.956 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:21 smithi049 bash[129285]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1 2024-05-15T16:00:21.956 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:21 smithi049 podman[129302]: 2024-05-15 16:00:21.911948283 +0000 UTC m=+0.498884971 container remove 204e65e376599822433f1d3d4742f415dbbdffcac2f3df7c9876abc752f41790 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-2965081, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-15T16:00:22.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:21 smithi144 ceph-mon[86635]: pgmap v68: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:22.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:21 smithi144 ceph-mon[86635]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:00:22.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:21 smithi144 ceph-mon[86635]: osdmap e56: 6 total, 5 up, 6 in 2024-05-15T16:00:22.683 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:22 smithi049 podman[129391]: 2024-05-15 16:00:22.354517712 +0000 UTC m=+0.113307801 container create a6bfb2317880970be36afd2a68e7c9ab77311bd562316087d39bde2a65e4eb5f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-05-15T16:00:22.684 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:22 smithi049 podman[129391]: 2024-05-15 16:00:22.259343592 +0000 UTC m=+0.018133646 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:00:22.684 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:22 smithi049 podman[129391]: 2024-05-15 16:00:22.590457366 +0000 UTC m=+0.349247424 container init a6bfb2317880970be36afd2a68e7c9ab77311bd562316087d39bde2a65e4eb5f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-65480-squid-de5210d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-15T16:00:22.684 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:22 smithi049 podman[129391]: 2024-05-15 16:00:22.594064434 +0000 UTC m=+0.352854482 container start a6bfb2317880970be36afd2a68e7c9ab77311bd562316087d39bde2a65e4eb5f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:00:22.684 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:22 smithi049 podman[129391]: 2024-05-15 16:00:22.682111366 +0000 UTC m=+0.440901413 container attach a6bfb2317880970be36afd2a68e7c9ab77311bd562316087d39bde2a65e4eb5f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-15T16:00:22.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:22 smithi049 ceph-mon[112142]: osdmap e57: 6 total, 5 up, 6 in 2024-05-15T16:00:22.956 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:22 smithi049 podman[129391]: 2024-05-15 16:00:22.900679879 +0000 UTC m=+0.659469929 container died a6bfb2317880970be36afd2a68e7c9ab77311bd562316087d39bde2a65e4eb5f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:00:23.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:22 smithi144 ceph-mon[86635]: osdmap e57: 6 total, 5 up, 6 in 2024-05-15T16:00:23.775 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:23 smithi049 podman[129417]: 2024-05-15 16:00:23.52003717 +0000 UTC m=+0.613808125 container remove a6bfb2317880970be36afd2a68e7c9ab77311bd562316087d39bde2a65e4eb5f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-15T16:00:23.775 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:23 smithi049 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.1.service: Deactivated successfully. 2024-05-15T16:00:23.775 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:23 smithi049 systemd[1]: Stopped Ceph osd.1 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:00:23.775 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:23 smithi049 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.1.service: Consumed 1min 1.495s CPU time. 2024-05-15T16:00:24.112 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:23 smithi049 ceph-mon[112142]: pgmap v71: 65 pgs: 34 peering, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-15T16:00:24.112 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:23 smithi049 ceph-mon[112142]: Health check failed: Reduced data availability: 11 pgs peering (PG_AVAILABILITY) 2024-05-15T16:00:24.112 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:23 smithi049 systemd[1]: Starting Ceph osd.1 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:00:24.112 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:23 smithi049 podman[129516]: 2024-05-15 16:00:23.897472357 +0000 UTC m=+0.018144892 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:00:24.112 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 podman[129516]: 2024-05-15 16:00:24.004511406 +0000 UTC m=+0.125183943 container create d9dd5626b033eecd6c69544b479dd125076c2a44bfa30248ef73cc16e47f0119 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T16:00:24.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:23 smithi144 ceph-mon[86635]: pgmap v71: 65 pgs: 34 peering, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-15T16:00:24.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:23 smithi144 ceph-mon[86635]: Health check failed: Reduced data availability: 11 pgs peering (PG_AVAILABILITY) 2024-05-15T16:00:24.456 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 podman[129516]: 2024-05-15 16:00:24.257228359 +0000 UTC m=+0.377900887 container init d9dd5626b033eecd6c69544b479dd125076c2a44bfa30248ef73cc16e47f0119 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-15T16:00:24.456 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 podman[129516]: 2024-05-15 16:00:24.261024855 +0000 UTC m=+0.381697373 container start d9dd5626b033eecd6c69544b479dd125076c2a44bfa30248ef73cc16e47f0119 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-15T16:00:24.456 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 podman[129516]: 2024-05-15 16:00:24.340311433 +0000 UTC m=+0.460983960 container attach d9dd5626b033eecd6c69544b479dd125076c2a44bfa30248ef73cc16e47f0119 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-15T16:00:25.072 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate[129532]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:00:25.072 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate[129532]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:00:25.072 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 bash[129516]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 bash[129516]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 bash[129516]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 bash[129516]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 bash[129516]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 bash[129516]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 bash[129516]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 bash[129516]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 bash[129516]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate[129532]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate[129532]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate[129532]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate[129532]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate[129532]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-15T16:00:25.073 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate[129532]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-15T16:00:25.074 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate[129532]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-15T16:00:25.074 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:24 smithi049 podman[129516]: 2024-05-15 16:00:24.789011642 +0000 UTC m=+0.909684160 container died d9dd5626b033eecd6c69544b479dd125076c2a44bfa30248ef73cc16e47f0119 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate, RELEASE=wip-65480-squid-de5210d, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-15T16:00:25.706 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:25 smithi049 podman[129677]: 2024-05-15 16:00:25.453446776 +0000 UTC m=+0.654007661 container remove d9dd5626b033eecd6c69544b479dd125076c2a44bfa30248ef73cc16e47f0119 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5) 2024-05-15T16:00:25.719 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:25 smithi049 podman[129738]: 2024-05-15 16:00:25.577541176 +0000 UTC m=+0.018093620 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:00:25.719 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:25 smithi049 podman[129738]: 2024-05-15 16:00:25.696052507 +0000 UTC m=+0.136604926 container create defc7d001185ba4cd3f24bc8c049e500a168f329b1d9a0da110c93b18a225f3a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:00:26.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:25 smithi144 ceph-mon[86635]: pgmap v72: 65 pgs: 34 peering, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-15T16:00:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:25 smithi049 ceph-mon[112142]: pgmap v72: 65 pgs: 34 peering, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-15T16:00:26.205 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:25 smithi049 podman[129738]: 2024-05-15 16:00:25.923606863 +0000 UTC m=+0.364159288 container init defc7d001185ba4cd3f24bc8c049e500a168f329b1d9a0da110c93b18a225f3a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:00:26.206 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:25 smithi049 podman[129738]: 2024-05-15 16:00:25.927097099 +0000 UTC m=+0.367649516 container start defc7d001185ba4cd3f24bc8c049e500a168f329b1d9a0da110c93b18a225f3a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:00:26.206 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:25 smithi049 ceph-osd[129756]: -- 172.21.15.49:0/3269864358 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55715cc79680 con 0x55715cc42c00 2024-05-15T16:00:26.206 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:26 smithi049 bash[129738]: defc7d001185ba4cd3f24bc8c049e500a168f329b1d9a0da110c93b18a225f3a 2024-05-15T16:00:26.206 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:26 smithi049 systemd[1]: Started Ceph osd.1 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:00:26.931 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:26 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[129752]: 2024-05-15T16:00:26.470+0000 7fc0d44927c0 -1 Falling back to public interface 2024-05-15T16:00:27.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:27 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:27.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:27 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:27.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:27 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:00:27.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:27 smithi049 ceph-mon[112142]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 5 peering, 16 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/321 objects degraded (13.396%) 2024-05-15T16:00:27.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:27 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:27 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:27 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:00:27.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:27 smithi144 ceph-mon[86635]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 5 peering, 16 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/321 objects degraded (13.396%) 2024-05-15T16:00:28.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:28 smithi049 ceph-mon[112142]: Health check failed: Degraded data redundancy: 43/321 objects degraded (13.396%), 13 pgs degraded (PG_DEGRADED) 2024-05-15T16:00:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:28 smithi144 ceph-mon[86635]: Health check failed: Degraded data redundancy: 43/321 objects degraded (13.396%), 13 pgs degraded (PG_DEGRADED) 2024-05-15T16:00:29.340 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:29 smithi049 ceph-mon[112142]: pgmap v74: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:00:29.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:29 smithi144 ceph-mon[86635]: pgmap v74: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:00:30.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:30 smithi049 ceph-mon[112142]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-15T16:00:30.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:30 smithi144 ceph-mon[86635]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-15T16:00:31.399 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:31 smithi049 ceph-mon[112142]: pgmap v75: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:00:31.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:31 smithi144 ceph-mon[86635]: pgmap v75: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:00:31.705 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:31 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[129752]: 2024-05-15T16:00:31.396+0000 7fc0d44927c0 -1 osd.1 55 log_to_monitors true 2024-05-15T16:00:32.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:32 smithi144 ceph-mon[86635]: from='osd.1 [v2:172.21.15.49:6810/3720875621,v1:172.21.15.49:6811/3720875621]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T16:00:32.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:32 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:32.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:32 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:32.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:32 smithi049 ceph-mon[112142]: from='osd.1 [v2:172.21.15.49:6810/3720875621,v1:172.21.15.49:6811/3720875621]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T16:00:32.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:32.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:32.455 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:00:32 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[129752]: 2024-05-15T16:00:32.099+0000 7fc0c58e9700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T16:00:33.385 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:33 smithi049 ceph-mon[112142]: from='osd.1 [v2:172.21.15.49:6810/3720875621,v1:172.21.15.49:6811/3720875621]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-15T16:00:33.385 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:33 smithi049 ceph-mon[112142]: osdmap e58: 6 total, 5 up, 6 in 2024-05-15T16:00:33.385 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:33 smithi049 ceph-mon[112142]: from='osd.1 [v2:172.21.15.49:6810/3720875621,v1:172.21.15.49:6811/3720875621]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T16:00:33.385 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:33 smithi049 ceph-mon[112142]: pgmap v77: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:00:33.385 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:33 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:33.385 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:33 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:33.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:33 smithi144 ceph-mon[86635]: from='osd.1 [v2:172.21.15.49:6810/3720875621,v1:172.21.15.49:6811/3720875621]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-15T16:00:33.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:33 smithi144 ceph-mon[86635]: osdmap e58: 6 total, 5 up, 6 in 2024-05-15T16:00:33.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:33 smithi144 ceph-mon[86635]: from='osd.1 [v2:172.21.15.49:6810/3720875621,v1:172.21.15.49:6811/3720875621]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T16:00:33.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:33 smithi144 ceph-mon[86635]: pgmap v77: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:00:33.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:33 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:33.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:33 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:34.406 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:34 smithi049 ceph-mon[112142]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:00:34.406 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:34 smithi049 ceph-mon[112142]: osd.1 [v2:172.21.15.49:6810/3720875621,v1:172.21.15.49:6811/3720875621] boot 2024-05-15T16:00:34.406 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:34 smithi049 ceph-mon[112142]: osdmap e59: 6 total, 6 up, 6 in 2024-05-15T16:00:34.406 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:34 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T16:00:34.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:34 smithi144 ceph-mon[86635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:00:34.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:34 smithi144 ceph-mon[86635]: osd.1 [v2:172.21.15.49:6810/3720875621,v1:172.21.15.49:6811/3720875621] boot 2024-05-15T16:00:34.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:34 smithi144 ceph-mon[86635]: osdmap e59: 6 total, 6 up, 6 in 2024-05-15T16:00:34.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:34 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T16:00:35.376 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:35 smithi049 ceph-mon[112142]: osdmap e60: 6 total, 6 up, 6 in 2024-05-15T16:00:35.376 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:35 smithi049 ceph-mon[112142]: pgmap v80: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:00:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:35 smithi144 ceph-mon[86635]: osdmap e60: 6 total, 6 up, 6 in 2024-05-15T16:00:35.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:35 smithi144 ceph-mon[86635]: pgmap v80: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:00:36.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:36.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:00:36.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:36.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:36.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:00:36.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:00:36.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:00:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:00:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:00:36.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:37.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:37 smithi144 ceph-mon[86635]: pgmap v81: 65 pgs: 7 active+undersized+degraded, 7 active+undersized, 51 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 4 op/s; 24/321 objects degraded (7.477%) 2024-05-15T16:00:37.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:00:37.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:37.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:37.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:37.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-15T16:00:37.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:37 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-15T16:00:37.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:37 smithi144 ceph-mon[86635]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-05-15T16:00:37.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:37 smithi144 ceph-mon[86635]: Health check update: Degraded data redundancy: 47/321 objects degraded (14.642%), 15 pgs degraded (PG_DEGRADED) 2024-05-15T16:00:37.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:37 smithi049 ceph-mon[112142]: pgmap v81: 65 pgs: 7 active+undersized+degraded, 7 active+undersized, 51 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 4 op/s; 24/321 objects degraded (7.477%) 2024-05-15T16:00:37.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:00:37.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:37.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:37.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:37.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-15T16:00:37.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:37 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-15T16:00:37.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:37 smithi049 ceph-mon[112142]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-05-15T16:00:37.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:37 smithi049 ceph-mon[112142]: Health check update: Degraded data redundancy: 47/321 objects degraded (14.642%), 15 pgs degraded (PG_DEGRADED) 2024-05-15T16:00:38.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:38 smithi144 ceph-mon[86635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/321 objects degraded (7.477%), 7 pgs degraded) 2024-05-15T16:00:38.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:38 smithi049 ceph-mon[112142]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/321 objects degraded (7.477%), 7 pgs degraded) 2024-05-15T16:00:39.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:39 smithi144 ceph-mon[86635]: pgmap v82: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-15T16:00:39.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:39 smithi049 ceph-mon[112142]: pgmap v82: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-15T16:00:41.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:41 smithi144 ceph-mon[86635]: pgmap v83: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-05-15T16:00:41.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:41 smithi049 ceph-mon[112142]: pgmap v83: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-05-15T16:00:43.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:43 smithi144 ceph-mon[86635]: pgmap v84: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:00:43.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:43 smithi049 ceph-mon[112142]: pgmap v84: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:00:45.496 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:45 smithi049 ceph-mon[112142]: pgmap v85: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-05-15T16:00:45.496 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:45 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:45.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:45 smithi144 ceph-mon[86635]: pgmap v85: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-05-15T16:00:45.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:45 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:47.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:47 smithi144 ceph-mon[86635]: pgmap v86: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-05-15T16:00:47.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:47 smithi049 ceph-mon[112142]: pgmap v86: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-05-15T16:00:48.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.381+0000 7fb90037d700 1 -- 172.21.15.49:0/426483515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f8100f00 msgr2=0x7fb8f81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.381+0000 7fb90037d700 1 --2- 172.21.15.49:0/426483515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f8100f00 0x7fb8f81012e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009910 tx=0x7fb8f4009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.381+0000 7fb90037d700 1 -- 172.21.15.49:0/426483515 shutdown_connections 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.381+0000 7fb90037d700 1 --2- 172.21.15.49:0/426483515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f81018b0 0x7fb8f8105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.381+0000 7fb90037d700 1 --2- 172.21.15.49:0/426483515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f8100f00 0x7fb8f81012e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.381+0000 7fb90037d700 1 -- 172.21.15.49:0/426483515 >> 172.21.15.49:0/426483515 conn(0x7fb8f80fc7d0 msgr2=0x7fb8f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.381+0000 7fb90037d700 1 -- 172.21.15.49:0/426483515 shutdown_connections 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.381+0000 7fb90037d700 1 -- 172.21.15.49:0/426483515 wait complete. 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb90037d700 1 Processor -- start 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb90037d700 1 -- start start 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb90037d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f8100f00 0x7fb8f8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb90037d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f81018b0 0x7fb8f8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb90037d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f81999f0 con 0x7fb8f81018b0 2024-05-15T16:00:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb90037d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f819d6c0 con 0x7fb8f8100f00 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb8fd918700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f81018b0 0x7fb8f8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb8fd918700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f81018b0 0x7fb8f8199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37962/0 (socket says 172.21.15.49:37962) 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb8fd918700 1 -- 172.21.15.49:0/491372949 learned_addr learned my addr 172.21.15.49:0/491372949 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb8fe119700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f8100f00 0x7fb8f8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb8fd918700 1 -- 172.21.15.49:0/491372949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f8100f00 msgr2=0x7fb8f8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb8fd918700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f8100f00 0x7fb8f8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.382+0000 7fb8fd918700 1 -- 172.21.15.49:0/491372949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f40095f0 con 0x7fb8f81018b0 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.383+0000 7fb8fd918700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f81018b0 0x7fb8f8199310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800d900 tx=0x7fb8e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.383+0000 7fb8ef7fe700 1 -- 172.21.15.49:0/491372949 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e8009900 con 0x7fb8f81018b0 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.383+0000 7fb8ef7fe700 1 -- 172.21.15.49:0/491372949 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8e8010460 con 0x7fb8f81018b0 2024-05-15T16:00:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.383+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f819d9a0 con 0x7fb8f81018b0 2024-05-15T16:00:48.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.383+0000 7fb8ef7fe700 1 -- 172.21.15.49:0/491372949 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e800f5f0 con 0x7fb8f81018b0 2024-05-15T16:00:48.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.383+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f819def0 con 0x7fb8f81018b0 2024-05-15T16:00:48.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.384+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f8105080 con 0x7fb8f81018b0 2024-05-15T16:00:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.385+0000 7fb8ef7fe700 1 -- 172.21.15.49:0/491372949 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb8e80105d0 con 0x7fb8f81018b0 2024-05-15T16:00:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.385+0000 7fb8ef7fe700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb8e406e200 0x7fb8e40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.385+0000 7fb8ef7fe700 1 -- 172.21.15.49:0/491372949 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb8e808dcb0 con 0x7fb8f81018b0 2024-05-15T16:00:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.385+0000 7fb8fe119700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb8e406e200 0x7fb8e40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:48.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.386+0000 7fb8fe119700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb8e406e200 0x7fb8e40706c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4000c00 tx=0x7fb8f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:48.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.390+0000 7fb8ef7fe700 1 -- 172.21.15.49:0/491372949 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb8e8057040 con 0x7fb8f81018b0 2024-05-15T16:00:48.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.619+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8f819a150 con 0x7fb8e406e200 2024-05-15T16:00:48.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.621+0000 7fb8ef7fe700 1 -- 172.21.15.49:0/491372949 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb8f819a150 con 0x7fb8e406e200 2024-05-15T16:00:48.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb8e406e200 msgr2=0x7fb8e40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:48.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb8e406e200 0x7fb8e40706c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4000c00 tx=0x7fb8f401a040 comp rx=0 tx=0).stop 2024-05-15T16:00:48.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f81018b0 msgr2=0x7fb8f8199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:48.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f81018b0 0x7fb8f8199310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800d900 tx=0x7fb8e800dc10 comp rx=0 tx=0).stop 2024-05-15T16:00:48.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 shutdown_connections 2024-05-15T16:00:48.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb8e406e200 0x7fb8e40706c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f8100f00 0x7fb8f8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 --2- 172.21.15.49:0/491372949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f81018b0 0x7fb8f8199310 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 >> 172.21.15.49:0/491372949 conn(0x7fb8f80fc7d0 msgr2=0x7fb8f80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:48.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 shutdown_connections 2024-05-15T16:00:48.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.623+0000 7fb90037d700 1 -- 172.21.15.49:0/491372949 wait complete. 2024-05-15T16:00:48.636 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:00:48.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.733+0000 7f4505ffe700 1 -- 172.21.15.49:0/2859894177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 msgr2=0x7f45001022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:48.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.733+0000 7f4505ffe700 1 --2- 172.21.15.49:0/2859894177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 0x7f45001022b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009940 tx=0x7f44f0009c50 comp rx=0 tx=0).stop 2024-05-15T16:00:48.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.733+0000 7f4505ffe700 1 -- 172.21.15.49:0/2859894177 shutdown_connections 2024-05-15T16:00:48.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.733+0000 7f4505ffe700 1 --2- 172.21.15.49:0/2859894177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45001027f0 0x7f450010ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.733+0000 7f4505ffe700 1 --2- 172.21.15.49:0/2859894177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 0x7f45001022b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.733+0000 7f4505ffe700 1 -- 172.21.15.49:0/2859894177 >> 172.21.15.49:0/2859894177 conn(0x7f45000fb700 msgr2=0x7f45000fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:48.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.733+0000 7f4505ffe700 1 -- 172.21.15.49:0/2859894177 shutdown_connections 2024-05-15T16:00:48.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.733+0000 7f4505ffe700 1 -- 172.21.15.49:0/2859894177 wait complete. 2024-05-15T16:00:48.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.734+0000 7f4505ffe700 1 Processor -- start 2024-05-15T16:00:48.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.734+0000 7f4505ffe700 1 -- start start 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.734+0000 7f4505ffe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 0x7f450019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.734+0000 7f4505ffe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45001027f0 0x7f450019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.734+0000 7f4505ffe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450019d4b0 con 0x7f4500101ed0 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.734+0000 7f4505ffe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4500196a90 con 0x7f45001027f0 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44ff7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 0x7f450019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44ff7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 0x7f450019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37968/0 (socket says 172.21.15.49:37968) 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44ff7fe700 1 -- 172.21.15.49:0/1646147431 learned_addr learned my addr 172.21.15.49:0/1646147431 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44feffd700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45001027f0 0x7f450019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44ff7fe700 1 -- 172.21.15.49:0/1646147431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45001027f0 msgr2=0x7f450019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:48.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44ff7fe700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45001027f0 0x7f450019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44ff7fe700 1 -- 172.21.15.49:0/1646147431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f00095f0 con 0x7f4500101ed0 2024-05-15T16:00:48.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44ff7fe700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 0x7f450019c950 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005620 tx=0x7f44f0005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:48.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44fcff9700 1 -- 172.21.15.49:0/1646147431 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44f001d070 con 0x7f4500101ed0 2024-05-15T16:00:48.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44fcff9700 1 -- 172.21.15.49:0/1646147431 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44f0003d50 con 0x7f4500101ed0 2024-05-15T16:00:48.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4500196d10 con 0x7f4500101ed0 2024-05-15T16:00:48.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f44fcff9700 1 -- 172.21.15.49:0/1646147431 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44f0021410 con 0x7f4500101ed0 2024-05-15T16:00:48.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.735+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4500197200 con 0x7f4500101ed0 2024-05-15T16:00:48.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.736+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45001055f0 con 0x7f4500101ed0 2024-05-15T16:00:48.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.737+0000 7f44fcff9700 1 -- 172.21.15.49:0/1646147431 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f44f000f7d0 con 0x7f4500101ed0 2024-05-15T16:00:48.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.737+0000 7f44fcff9700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44e806e200 0x7f44e80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:48.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.738+0000 7f44fcff9700 1 -- 172.21.15.49:0/1646147431 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f44f008f1c0 con 0x7f4500101ed0 2024-05-15T16:00:48.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.738+0000 7f44feffd700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44e806e200 0x7f44e80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:48.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.738+0000 7f44feffd700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44e806e200 0x7f44e80706c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4500197c50 tx=0x7f44f400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.742+0000 7f44fcff9700 1 -- 172.21.15.49:0/1646147431 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f44f005c080 con 0x7f4500101ed0 2024-05-15T16:00:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.971+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4500197f50 con 0x7f44e806e200 2024-05-15T16:00:48.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.972+0000 7f44fcff9700 1 -- 172.21.15.49:0/1646147431 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4500197f50 con 0x7f44e806e200 2024-05-15T16:00:48.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44e806e200 msgr2=0x7f44e80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:48.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44e806e200 0x7f44e80706c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4500197c50 tx=0x7f44f400b410 comp rx=0 tx=0).stop 2024-05-15T16:00:48.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 msgr2=0x7f450019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:48.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 0x7f450019c950 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005620 tx=0x7f44f0005650 comp rx=0 tx=0).stop 2024-05-15T16:00:48.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 shutdown_connections 2024-05-15T16:00:48.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44e806e200 0x7f44e80706c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4500101ed0 0x7f450019c950 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 --2- 172.21.15.49:0/1646147431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45001027f0 0x7f450019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:48.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 >> 172.21.15.49:0/1646147431 conn(0x7f45000fb700 msgr2=0x7f45001031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:48.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 shutdown_connections 2024-05-15T16:00:48.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:48.975+0000 7f4505ffe700 1 -- 172.21.15.49:0/1646147431 wait complete. 2024-05-15T16:00:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.083+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/3504832588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac102790 msgr2=0x7fd6ac10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.083+0000 7fd6b0b3a700 1 --2- 172.21.15.49:0/3504832588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac102790 0x7fd6ac10ac80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009910 tx=0x7fd6a0009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.083+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/3504832588 shutdown_connections 2024-05-15T16:00:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.083+0000 7fd6b0b3a700 1 --2- 172.21.15.49:0/3504832588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac102790 0x7fd6ac10ac80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.083+0000 7fd6b0b3a700 1 --2- 172.21.15.49:0/3504832588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ac101e70 0x7fd6ac102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.083+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/3504832588 >> 172.21.15.49:0/3504832588 conn(0x7fd6ac0fb700 msgr2=0x7fd6ac0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.083+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/3504832588 shutdown_connections 2024-05-15T16:00:49.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.083+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/3504832588 wait complete. 2024-05-15T16:00:49.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.084+0000 7fd6b0b3a700 1 Processor -- start 2024-05-15T16:00:49.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.084+0000 7fd6b0b3a700 1 -- start start 2024-05-15T16:00:49.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.084+0000 7fd6b0b3a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac101e70 0x7fd6ac19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:49.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.084+0000 7fd6b0b3a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ac102790 0x7fd6ac19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:49.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.084+0000 7fd6b0b3a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac19d490 con 0x7fd6ac101e70 2024-05-15T16:00:49.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.084+0000 7fd6b0b3a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac196a70 con 0x7fd6ac102790 2024-05-15T16:00:49.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6aa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac101e70 0x7fd6ac19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:49.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6aa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac101e70 0x7fd6ac19c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37982/0 (socket says 172.21.15.49:37982) 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6aa59c700 1 -- 172.21.15.49:0/1133952997 learned_addr learned my addr 172.21.15.49:0/1133952997 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6a9d9b700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ac102790 0x7fd6ac19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6aa59c700 1 -- 172.21.15.49:0/1133952997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ac102790 msgr2=0x7fd6ac19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6aa59c700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ac102790 0x7fd6ac19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6aa59c700 1 -- 172.21.15.49:0/1133952997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a00095f0 con 0x7fd6ac101e70 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6aa59c700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac101e70 0x7fd6ac19c930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d8d0 tx=0x7fd69c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd69b7fe700 1 -- 172.21.15.49:0/1133952997 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd69c0098a0 con 0x7fd6ac101e70 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd69b7fe700 1 -- 172.21.15.49:0/1133952997 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd69c010460 con 0x7fd6ac101e70 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac196d50 con 0x7fd6ac101e70 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd69b7fe700 1 -- 172.21.15.49:0/1133952997 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd69c00f5f0 con 0x7fd6ac101e70 2024-05-15T16:00:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.085+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac1972a0 con 0x7fd6ac101e70 2024-05-15T16:00:49.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.086+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac105590 con 0x7fd6ac101e70 2024-05-15T16:00:49.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.087+0000 7fd69b7fe700 1 -- 172.21.15.49:0/1133952997 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd69c009a00 con 0x7fd6ac101e70 2024-05-15T16:00:49.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.088+0000 7fd69b7fe700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd69406e200 0x7fd6940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.088+0000 7fd69b7fe700 1 -- 172.21.15.49:0/1133952997 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd69c08e410 con 0x7fd6ac101e70 2024-05-15T16:00:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.088+0000 7fd6a9d9b700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd69406e200 0x7fd6940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.088+0000 7fd6a9d9b700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd69406e200 0x7fd6940706c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac197c30 tx=0x7fd6a001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:49.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.092+0000 7fd69b7fe700 1 -- 172.21.15.49:0/1133952997 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd69c05b350 con 0x7fd6ac101e70 2024-05-15T16:00:49.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.310+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6ac197fa0 con 0x7fd69406e200 2024-05-15T16:00:49.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.321+0000 7fd69b7fe700 1 -- 172.21.15.49:0/1133952997 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd6ac197fa0 con 0x7fd69406e200 2024-05-15T16:00:49.323 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:00:49.323 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (4m) 17s ago 15m 26.2M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:00:49.323 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (15m) 17s ago 15m 8598k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:00:49.323 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (14m) 78s ago 14m 8162k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:00:49.323 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (87s) 17s ago 15m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:00:49.323 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (81s) 78s ago 14m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:00:49.323 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (3m) 17s ago 15m 78.0M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:00:49.323 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (10m) 17s ago 10m 90.1M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (9m) 17s ago 9m 152M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (10m) 78s ago 10m 26.7M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (9m) 78s ago 9m 28.4M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (6m) 17s ago 17m 506M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (5m) 78s ago 14m 451M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (2m) 17s ago 17m 47.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (2m) 78s ago 14m 35.0M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (4m) 17s ago 15m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (4m) 78s ago 14m 16.2M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (59s) 17s ago 13m 81.9M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (23s) 17s ago 13m 12.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (12m) 17s ago 12m 214M 1216M 18.2.2-1343-g2965081f a04a73ffe079 487c81d140af 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (12m) 78s ago 12m 261M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (11m) 78s ago 11m 217M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T16:00:49.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (11m) 78s ago 11m 203M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T16:00:49.325 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (4m) 17s ago 15m 68.7M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:00:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.323+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd69406e200 msgr2=0x7fd6940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.323+0000 7fd6b0b3a700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd69406e200 0x7fd6940706c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac197c30 tx=0x7fd6a001a040 comp rx=0 tx=0).stop 2024-05-15T16:00:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.323+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac101e70 msgr2=0x7fd6ac19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.323+0000 7fd6b0b3a700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac101e70 0x7fd6ac19c930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d8d0 tx=0x7fd69c00dbe0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.324+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 shutdown_connections 2024-05-15T16:00:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.324+0000 7fd6b0b3a700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd69406e200 0x7fd6940706c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.324+0000 7fd6b0b3a700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ac101e70 0x7fd6ac19c930 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.324+0000 7fd6b0b3a700 1 --2- 172.21.15.49:0/1133952997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ac102790 0x7fd6ac19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.324+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 >> 172.21.15.49:0/1133952997 conn(0x7fd6ac0fb700 msgr2=0x7fd6ac103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.324+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 shutdown_connections 2024-05-15T16:00:49.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.324+0000 7fd6b0b3a700 1 -- 172.21.15.49:0/1133952997 wait complete. 2024-05-15T16:00:49.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.431+0000 7f3613a0e700 1 -- 172.21.15.49:0/2729115673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c101880 msgr2=0x7f360c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.431+0000 7f3613a0e700 1 --2- 172.21.15.49:0/2729115673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c101880 0x7f360c1058d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3600009910 tx=0x7f3600009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:49.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.431+0000 7f3613a0e700 1 -- 172.21.15.49:0/2729115673 shutdown_connections 2024-05-15T16:00:49.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.431+0000 7f3613a0e700 1 --2- 172.21.15.49:0/2729115673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c101880 0x7f360c1058d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.431+0000 7f3613a0e700 1 --2- 172.21.15.49:0/2729115673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c100ed0 0x7f360c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.431+0000 7f3613a0e700 1 -- 172.21.15.49:0/2729115673 >> 172.21.15.49:0/2729115673 conn(0x7f360c0fc7a0 msgr2=0x7f360c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:49.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.431+0000 7f3613a0e700 1 -- 172.21.15.49:0/2729115673 shutdown_connections 2024-05-15T16:00:49.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.431+0000 7f3613a0e700 1 -- 172.21.15.49:0/2729115673 wait complete. 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f3613a0e700 1 Processor -- start 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f3613a0e700 1 -- start start 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f3613a0e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c100ed0 0x7f360c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f3613a0e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c101880 0x7f360c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f3613a0e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c19d440 con 0x7f360c100ed0 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f3613a0e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c196a70 con 0x7f360c101880 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f36117aa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c100ed0 0x7f360c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f36117aa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c100ed0 0x7f360c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37994/0 (socket says 172.21.15.49:37994) 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f36117aa700 1 -- 172.21.15.49:0/4066194370 learned_addr learned my addr 172.21.15.49:0/4066194370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.432+0000 7f3610fa9700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c101880 0x7f360c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.433+0000 7f36117aa700 1 -- 172.21.15.49:0/4066194370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c101880 msgr2=0x7f360c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.433+0000 7f36117aa700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c101880 0x7f360c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.433+0000 7f36117aa700 1 -- 172.21.15.49:0/4066194370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36000095f0 con 0x7f360c100ed0 2024-05-15T16:00:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.433+0000 7f36117aa700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c100ed0 0x7f360c19c8e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f360800e920 tx=0x7f360800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.433+0000 7f35fe7fc700 1 -- 172.21.15.49:0/4066194370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f360800c9d0 con 0x7f360c100ed0 2024-05-15T16:00:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.433+0000 7f35fe7fc700 1 -- 172.21.15.49:0/4066194370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3608010430 con 0x7f360c100ed0 2024-05-15T16:00:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.433+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c196d50 con 0x7f360c100ed0 2024-05-15T16:00:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.433+0000 7f35fe7fc700 1 -- 172.21.15.49:0/4066194370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3608018630 con 0x7f360c100ed0 2024-05-15T16:00:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.433+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c197270 con 0x7f360c100ed0 2024-05-15T16:00:49.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.434+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c1092d0 con 0x7f360c100ed0 2024-05-15T16:00:49.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.434+0000 7f35fe7fc700 1 -- 172.21.15.49:0/4066194370 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f36080105a0 con 0x7f360c100ed0 2024-05-15T16:00:49.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.435+0000 7f35fe7fc700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f35f806e200 0x7f35f80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:49.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.435+0000 7f3610fa9700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f35f806e200 0x7f35f80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:49.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.435+0000 7f35fe7fc700 1 -- 172.21.15.49:0/4066194370 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3608014070 con 0x7f360c100ed0 2024-05-15T16:00:49.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.436+0000 7f3610fa9700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f35f806e200 0x7f35f80706c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f36000059d0 tx=0x7f3600005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.440+0000 7f35fe7fc700 1 -- 172.21.15.49:0/4066194370 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3608053850 con 0x7f360c100ed0 2024-05-15T16:00:49.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:49 smithi144 ceph-mon[86635]: pgmap v87: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-15T16:00:49.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:49 smithi144 ceph-mon[86635]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:49.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:49 smithi049 ceph-mon[112142]: pgmap v87: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-15T16:00:49.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:49 smithi049 ceph-mon[112142]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:49.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.733+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f360c197ee0 con 0x7f360c100ed0 2024-05-15T16:00:49.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.734+0000 7f35fe7fc700 1 -- 172.21.15.49:0/4066194370 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f360805b260 con 0x7f360c100ed0 2024-05-15T16:00:49.735 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:00:49.735 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:00:49.735 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:00:49.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:00:49.735 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4, 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 8, 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:00:49.736 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:00:49.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f35f806e200 msgr2=0x7f35f80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f35f806e200 0x7f35f80706c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f36000059d0 tx=0x7f3600005940 comp rx=0 tx=0).stop 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c100ed0 msgr2=0x7f360c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c100ed0 0x7f360c19c8e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f360800e920 tx=0x7f360800ec30 comp rx=0 tx=0).stop 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 shutdown_connections 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f35f806e200 0x7f35f80706c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c100ed0 0x7f360c19c8e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 --2- 172.21.15.49:0/4066194370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c101880 0x7f360c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 >> 172.21.15.49:0/4066194370 conn(0x7f360c0fc7a0 msgr2=0x7f360c0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 shutdown_connections 2024-05-15T16:00:49.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.736+0000 7f3613a0e700 1 -- 172.21.15.49:0/4066194370 wait complete. 2024-05-15T16:00:49.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.845+0000 7f3c29739700 1 -- 172.21.15.49:0/3944889017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 msgr2=0x7f3c241047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.845+0000 7f3c29739700 1 --2- 172.21.15.49:0/3944889017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 0x7f3c241047c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009910 tx=0x7f3c0c009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:49.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.845+0000 7f3c29739700 1 -- 172.21.15.49:0/3944889017 shutdown_connections 2024-05-15T16:00:49.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.845+0000 7f3c29739700 1 --2- 172.21.15.49:0/3944889017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 0x7f3c241047c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.845+0000 7f3c29739700 1 --2- 172.21.15.49:0/3944889017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c240ffdc0 0x7f3c241001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.845+0000 7f3c29739700 1 -- 172.21.15.49:0/3944889017 >> 172.21.15.49:0/3944889017 conn(0x7f3c24076800 msgr2=0x7f3c240fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:49.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.845+0000 7f3c29739700 1 -- 172.21.15.49:0/3944889017 shutdown_connections 2024-05-15T16:00:49.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.845+0000 7f3c29739700 1 -- 172.21.15.49:0/3944889017 wait complete. 2024-05-15T16:00:49.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.845+0000 7f3c29739700 1 Processor -- start 2024-05-15T16:00:49.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c29739700 1 -- start start 2024-05-15T16:00:49.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c29739700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c240ffdc0 0x7f3c240fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:49.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c29739700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 0x7f3c240fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:49.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c29739700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240fc760 con 0x7f3c24100770 2024-05-15T16:00:49.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c29739700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240fc8d0 con 0x7f3c240ffdc0 2024-05-15T16:00:49.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c1a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 0x7f3c240fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:49.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c1a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 0x7f3c240fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38010/0 (socket says 172.21.15.49:38010) 2024-05-15T16:00:49.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c1a7fc700 1 -- 172.21.15.49:0/3488045499 learned_addr learned my addr 172.21.15.49:0/3488045499 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c22ffd700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c240ffdc0 0x7f3c240fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c1a7fc700 1 -- 172.21.15.49:0/3488045499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c240ffdc0 msgr2=0x7f3c240fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c1a7fc700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c240ffdc0 0x7f3c240fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.846+0000 7f3c1a7fc700 1 -- 172.21.15.49:0/3488045499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c0095f0 con 0x7f3c24100770 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.847+0000 7f3c1a7fc700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 0x7f3c240fc220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00bd70 tx=0x7f3c0c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.847+0000 7f3c20ff9700 1 -- 172.21.15.49:0/3488045499 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0c01d070 con 0x7f3c24100770 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.847+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c240fcb50 con 0x7f3c24100770 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.847+0000 7f3c20ff9700 1 -- 172.21.15.49:0/3488045499 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c0c0056e0 con 0x7f3c24100770 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.847+0000 7f3c20ff9700 1 -- 172.21.15.49:0/3488045499 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0c00fe50 con 0x7f3c24100770 2024-05-15T16:00:49.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.847+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c241a3000 con 0x7f3c24100770 2024-05-15T16:00:49.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.848+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c241012a0 con 0x7f3c24100770 2024-05-15T16:00:49.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.848+0000 7f3c20ff9700 1 -- 172.21.15.49:0/3488045499 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3c0c00f680 con 0x7f3c24100770 2024-05-15T16:00:49.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.849+0000 7f3c20ff9700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3c1006e200 0x7f3c100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:49.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.849+0000 7f3c20ff9700 1 -- 172.21.15.49:0/3488045499 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3c0c08f240 con 0x7f3c24100770 2024-05-15T16:00:49.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.849+0000 7f3c22ffd700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3c1006e200 0x7f3c100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:49.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.849+0000 7f3c22ffd700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3c1006e200 0x7f3c100706c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3c14005950 tx=0x7f3c1400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:49.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:49.854+0000 7f3c20ff9700 1 -- 172.21.15.49:0/3488045499 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3c0c0585d0 con 0x7f3c24100770 2024-05-15T16:00:50.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.109+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3c240fcb50 con 0x7f3c24100770 2024-05-15T16:00:50.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.109+0000 7f3c20ff9700 1 -- 172.21.15.49:0/3488045499 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f3c240fcb50 con 0x7f3c24100770 2024-05-15T16:00:50.111 INFO:teuthology.orchestra.run.smithi049.stdout:e17 2024-05-15T16:00:50.111 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:00:50.111 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:00:50.111 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:00:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:00:50.111 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 17 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:51:00.559034+0000 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:00:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:00:50.113 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:00:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:00:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3c1006e200 msgr2=0x7f3c100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3c1006e200 0x7f3c100706c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3c14005950 tx=0x7f3c1400b410 comp rx=0 tx=0).stop 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 msgr2=0x7f3c240fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 0x7f3c240fc220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00bd70 tx=0x7f3c0c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 shutdown_connections 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3c1006e200 0x7f3c100706c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c240ffdc0 0x7f3c240fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 --2- 172.21.15.49:0/3488045499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c24100770 0x7f3c240fc220 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 >> 172.21.15.49:0/3488045499 conn(0x7f3c24076800 msgr2=0x7f3c240688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 shutdown_connections 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.113+0000 7f3c29739700 1 -- 172.21.15.49:0/3488045499 wait complete. 2024-05-15T16:00:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T16:00:50.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.225+0000 7f746f964700 1 -- 172.21.15.49:0/434148224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468101e20 msgr2=0x7f7468102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.225+0000 7f746f964700 1 --2- 172.21.15.49:0/434148224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468101e20 0x7f7468102200 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7464009910 tx=0x7f7464009c20 comp rx=0 tx=0).stop 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.225+0000 7f746f964700 1 -- 172.21.15.49:0/434148224 shutdown_connections 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.225+0000 7f746f964700 1 --2- 172.21.15.49:0/434148224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468102740 0x7f746810ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.225+0000 7f746f964700 1 --2- 172.21.15.49:0/434148224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468101e20 0x7f7468102200 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.225+0000 7f746f964700 1 -- 172.21.15.49:0/434148224 >> 172.21.15.49:0/434148224 conn(0x7f7468076800 msgr2=0x7f74680fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.225+0000 7f746f964700 1 -- 172.21.15.49:0/434148224 shutdown_connections 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.225+0000 7f746f964700 1 -- 172.21.15.49:0/434148224 wait complete. 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746f964700 1 Processor -- start 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746f964700 1 -- start start 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746f964700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468101e20 0x7f746819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746f964700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468102740 0x7f746819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746f964700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819d420 con 0x7f7468102740 2024-05-15T16:00:50.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746f964700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7468196a00 con 0x7f7468101e20 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746ceff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468102740 0x7f746819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746ceff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468102740 0x7f746819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38020/0 (socket says 172.21.15.49:38020) 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746ceff700 1 -- 172.21.15.49:0/1976363856 learned_addr learned my addr 172.21.15.49:0/1976363856 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746d700700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468101e20 0x7f746819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746ceff700 1 -- 172.21.15.49:0/1976363856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468101e20 msgr2=0x7f746819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746ceff700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468101e20 0x7f746819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746ceff700 1 -- 172.21.15.49:0/1976363856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74640095f0 con 0x7f7468102740 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746d700700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468101e20 0x7f746819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.226+0000 7f746ceff700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468102740 0x7f746819ce00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f745800d6d0 tx=0x7f745800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.227+0000 7f745e7fc700 1 -- 172.21.15.49:0/1976363856 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f745800f860 con 0x7f7468102740 2024-05-15T16:00:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.227+0000 7f745e7fc700 1 -- 172.21.15.49:0/1976363856 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f745800fe80 con 0x7f7468102740 2024-05-15T16:00:50.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.227+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7468196ce0 con 0x7f7468102740 2024-05-15T16:00:50.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.227+0000 7f745e7fc700 1 -- 172.21.15.49:0/1976363856 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7458010ea0 con 0x7f7468102740 2024-05-15T16:00:50.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.227+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7468197230 con 0x7f7468102740 2024-05-15T16:00:50.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.228+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7468105540 con 0x7f7468102740 2024-05-15T16:00:50.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.228+0000 7f745e7fc700 1 -- 172.21.15.49:0/1976363856 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7458010460 con 0x7f7468102740 2024-05-15T16:00:50.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.229+0000 7f745e7fc700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f745406e210 0x7f74540706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:50.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.229+0000 7f745e7fc700 1 -- 172.21.15.49:0/1976363856 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f745808dc10 con 0x7f7468102740 2024-05-15T16:00:50.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.229+0000 7f746d700700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f745406e210 0x7f74540706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:50.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.229+0000 7f746d700700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f745406e210 0x7f74540706d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f746400b5c0 tx=0x7f746401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:50.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.233+0000 7f745e7fc700 1 -- 172.21.15.49:0/1976363856 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f745805ab50 con 0x7f7468102740 2024-05-15T16:00:50.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:50 smithi049 ceph-mon[112142]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:50.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:50 smithi049 ceph-mon[112142]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:50.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:50 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4066194370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:50.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:50 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3488045499' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:00:50.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:50.457 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:00:50.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.466+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7468197f30 con 0x7f745406e210 2024-05-15T16:00:50.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.467+0000 7f745e7fc700 1 -- 172.21.15.49:0/1976363856 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7468197f30 con 0x7f745406e210 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "8/23 daemons upgraded", 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading osd daemons", 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:00:50.469 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:00:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.469+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f745406e210 msgr2=0x7f74540706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.469+0000 7f746f964700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f745406e210 0x7f74540706d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f746400b5c0 tx=0x7f746401a040 comp rx=0 tx=0).stop 2024-05-15T16:00:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.469+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468102740 msgr2=0x7f746819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.469+0000 7f746f964700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468102740 0x7f746819ce00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f745800d6d0 tx=0x7f745800da90 comp rx=0 tx=0).stop 2024-05-15T16:00:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.470+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 shutdown_connections 2024-05-15T16:00:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.470+0000 7f746f964700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f745406e210 0x7f74540706d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.470+0000 7f746f964700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468101e20 0x7f746819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.470+0000 7f746f964700 1 --2- 172.21.15.49:0/1976363856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7468102740 0x7f746819ce00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.470+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 >> 172.21.15.49:0/1976363856 conn(0x7f7468076800 msgr2=0x7f7468103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.470+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 shutdown_connections 2024-05-15T16:00:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.470+0000 7f746f964700 1 -- 172.21.15.49:0/1976363856 wait complete. 2024-05-15T16:00:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.578+0000 7f74b4aaf700 1 -- 172.21.15.49:0/3423177261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b00ffdc0 msgr2=0x7f74b01001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.578+0000 7f74b4aaf700 1 --2- 172.21.15.49:0/3423177261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b00ffdc0 0x7f74b01001a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f74a000b0a0 tx=0x7f74a000b3b0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.578+0000 7f74b4aaf700 1 -- 172.21.15.49:0/3423177261 shutdown_connections 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.578+0000 7f74b4aaf700 1 --2- 172.21.15.49:0/3423177261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b0100770 0x7f74b01047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.578+0000 7f74b4aaf700 1 --2- 172.21.15.49:0/3423177261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b00ffdc0 0x7f74b01001a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.578+0000 7f74b4aaf700 1 -- 172.21.15.49:0/3423177261 >> 172.21.15.49:0/3423177261 conn(0x7f74b0076800 msgr2=0x7f74b00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.578+0000 7f74b4aaf700 1 -- 172.21.15.49:0/3423177261 shutdown_connections 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.578+0000 7f74b4aaf700 1 -- 172.21.15.49:0/3423177261 wait complete. 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.579+0000 7f74b4aaf700 1 Processor -- start 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.579+0000 7f74b4aaf700 1 -- start start 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.579+0000 7f74b4aaf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00ffdc0 0x7f74b0198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.579+0000 7f74b4aaf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b0100770 0x7f74b0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.579+0000 7f74b4aaf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0199a10 con 0x7f74b0100770 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.579+0000 7f74b4aaf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b019d690 con 0x7f74b00ffdc0 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.579+0000 7f74add9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b0100770 0x7f74b0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.579+0000 7f74add9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b0100770 0x7f74b0199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38038/0 (socket says 172.21.15.49:38038) 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.579+0000 7f74add9b700 1 -- 172.21.15.49:0/2065450603 learned_addr learned my addr 172.21.15.49:0/2065450603 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74ae59c700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00ffdc0 0x7f74b0198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74add9b700 1 -- 172.21.15.49:0/2065450603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00ffdc0 msgr2=0x7f74b0198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74add9b700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00ffdc0 0x7f74b0198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74add9b700 1 -- 172.21.15.49:0/2065450603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a0009d00 con 0x7f74b0100770 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74ae59c700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00ffdc0 0x7f74b0198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74add9b700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b0100770 0x7f74b0199330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f749800d900 tx=0x7f749800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74a77fe700 1 -- 172.21.15.49:0/2065450603 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7498009900 con 0x7f74b0100770 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74a77fe700 1 -- 172.21.15.49:0/2065450603 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7498010460 con 0x7f74b0100770 2024-05-15T16:00:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b019d970 con 0x7f74b0100770 2024-05-15T16:00:50.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74a77fe700 1 -- 172.21.15.49:0/2065450603 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f749800f5f0 con 0x7f74b0100770 2024-05-15T16:00:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.580+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b019dec0 con 0x7f74b0100770 2024-05-15T16:00:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.581+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b01012a0 con 0x7f74b0100770 2024-05-15T16:00:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.582+0000 7f74a77fe700 1 -- 172.21.15.49:0/2065450603 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f74980105d0 con 0x7f74b0100770 2024-05-15T16:00:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.582+0000 7f74a77fe700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f749c06e210 0x7f749c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:00:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.582+0000 7f74a77fe700 1 -- 172.21.15.49:0/2065450603 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f749808df70 con 0x7f74b0100770 2024-05-15T16:00:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.582+0000 7f74ae59c700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f749c06e210 0x7f749c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:00:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.583+0000 7f74ae59c700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f749c06e210 0x7f749c0706d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f74a000b830 tx=0x7f74a0006210 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:00:50.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.587+0000 7f74a77fe700 1 -- 172.21.15.49:0/2065450603 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f74980c2850 con 0x7f74b0100770 2024-05-15T16:00:50.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:50 smithi144 ceph-mon[86635]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:50.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:50 smithi144 ceph-mon[86635]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:50.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:50 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4066194370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:00:50.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:50 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3488045499' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:00:50.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:50.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:00:50.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.878+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f74b010cd80 con 0x7f74b0100770 2024-05-15T16:00:50.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.878+0000 7f74a77fe700 1 -- 172.21.15.49:0/2065450603 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f7498057300 con 0x7f74b0100770 2024-05-15T16:00:50.880 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T16:00:50.880 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:00:50.880 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:00:50.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.880+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f749c06e210 msgr2=0x7f749c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.880+0000 7f74b4aaf700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f749c06e210 0x7f749c0706d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f74a000b830 tx=0x7f74a0006210 comp rx=0 tx=0).stop 2024-05-15T16:00:50.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.880+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b0100770 msgr2=0x7f74b0199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:00:50.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.880+0000 7f74b4aaf700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b0100770 0x7f74b0199330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f749800d900 tx=0x7f749800dcc0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.880+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 shutdown_connections 2024-05-15T16:00:50.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.880+0000 7f74b4aaf700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f749c06e210 0x7f749c0706d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.880+0000 7f74b4aaf700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00ffdc0 0x7f74b0198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.880+0000 7f74b4aaf700 1 --2- 172.21.15.49:0/2065450603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74b0100770 0x7f74b0199330 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:00:50.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.880+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 >> 172.21.15.49:0/2065450603 conn(0x7f74b0076800 msgr2=0x7f74b0103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:00:50.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.881+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 shutdown_connections 2024-05-15T16:00:50.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:00:50.881+0000 7f74b4aaf700 1 -- 172.21.15.49:0/2065450603 wait complete. 2024-05-15T16:00:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:51 smithi049 ceph-mon[112142]: pgmap v88: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:51 smithi049 ceph-mon[112142]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:51 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2065450603' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:00:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:51 smithi144 ceph-mon[86635]: pgmap v88: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:51.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:51 smithi144 ceph-mon[86635]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:00:51.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:51 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2065450603' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:00:52.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:52 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-15T16:00:52.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:52 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-15T16:00:52.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:52 smithi144 ceph-mon[86635]: Upgrade: osd.2 is safe to restart 2024-05-15T16:00:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:52 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-15T16:00:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:52 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-15T16:00:52.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:52 smithi049 ceph-mon[112142]: Upgrade: osd.2 is safe to restart 2024-05-15T16:00:53.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:53 smithi144 ceph-mon[86635]: pgmap v89: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:00:53.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:53 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:53.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:53 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-15T16:00:53.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:53 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:00:53.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:53 smithi049 ceph-mon[112142]: pgmap v89: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:00:53.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:53 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:00:53.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:53 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-15T16:00:53.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:53 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:00:54.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:54 smithi049 ceph-mon[112142]: Upgrade: Updating osd.2 2024-05-15T16:00:54.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:54 smithi049 ceph-mon[112142]: Deploying daemon osd.2 on smithi049 2024-05-15T16:00:54.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:54 smithi144 ceph-mon[86635]: Upgrade: Updating osd.2 2024-05-15T16:00:54.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:54 smithi144 ceph-mon[86635]: Deploying daemon osd.2 on smithi049 2024-05-15T16:00:55.402 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:55 smithi049 ceph-mon[112142]: pgmap v90: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:55.600 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:55 smithi144 ceph-mon[86635]: pgmap v90: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:00:56.706 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:56 smithi049 systemd[1]: Stopping Ceph osd.2 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:00:57.205 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:56 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[62084]: 2024-05-15T16:00:56.827+0000 7f24f5981700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:00:57.205 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:56 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[62084]: 2024-05-15T16:00:56.827+0000 7f24f5981700 -1 osd.2 60 *** Got signal Terminated *** 2024-05-15T16:00:57.205 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:56 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[62084]: 2024-05-15T16:00:56.827+0000 7f24f5981700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:00:57.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:57 smithi144 ceph-mon[86635]: pgmap v91: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:00:57.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:57 smithi144 ceph-mon[86635]: osd.2 marked itself down and dead 2024-05-15T16:00:57.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:57 smithi049 ceph-mon[112142]: pgmap v91: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:00:57.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:57 smithi049 ceph-mon[112142]: osd.2 marked itself down and dead 2024-05-15T16:00:58.087 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:57 smithi049 podman[134954]: 2024-05-15 16:00:57.809024515 +0000 UTC m=+1.152220131 container died 487c81d140af730deb250f110d589c1c924a9564de29780f5045e017d81dbefb (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-2965081, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-15T16:00:58.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:58 smithi049 ceph-mon[112142]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:00:58.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:58 smithi049 ceph-mon[112142]: osdmap e61: 6 total, 5 up, 6 in 2024-05-15T16:00:58.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:58 smithi049 podman[134954]: 2024-05-15 16:00:58.183488864 +0000 UTC m=+1.526684472 container cleanup 487c81d140af730deb250f110d589c1c924a9564de29780f5045e017d81dbefb (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=reef-2965081, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-05-15T16:00:58.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:58 smithi049 bash[134954]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2 2024-05-15T16:00:58.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:58 smithi144 ceph-mon[86635]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:00:58.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:58 smithi144 ceph-mon[86635]: osdmap e61: 6 total, 5 up, 6 in 2024-05-15T16:00:58.866 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:58 smithi049 podman[134969]: 2024-05-15 16:00:58.476810118 +0000 UTC m=+0.664635006 container remove 487c81d140af730deb250f110d589c1c924a9564de29780f5045e017d81dbefb (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=reef-2965081, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-15T16:00:59.119 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:58 smithi049 podman[135068]: 2024-05-15 16:00:58.767992405 +0000 UTC m=+0.018436683 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:00:59.119 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:58 smithi049 podman[135068]: 2024-05-15 16:00:58.902926217 +0000 UTC m=+0.153370481 container create ec8c53a3524294f179077f4c6cdae18a4029f109513efc46e977542029b7d19a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:00:59.119 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:59 smithi049 podman[135068]: 2024-05-15 16:00:59.114060122 +0000 UTC m=+0.364504386 container init ec8c53a3524294f179077f4c6cdae18a4029f109513efc46e977542029b7d19a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-15T16:00:59.119 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:59 smithi049 podman[135068]: 2024-05-15 16:00:59.117585042 +0000 UTC m=+0.368029308 container start ec8c53a3524294f179077f4c6cdae18a4029f109513efc46e977542029b7d19a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-15T16:00:59.443 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:59 smithi049 ceph-mon[112142]: pgmap v93: 65 pgs: 9 stale+active+clean, 56 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:00:59.443 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:00:59 smithi049 ceph-mon[112142]: osdmap e62: 6 total, 5 up, 6 in 2024-05-15T16:00:59.444 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:59 smithi049 podman[135068]: 2024-05-15 16:00:59.230982069 +0000 UTC m=+0.481426341 container attach ec8c53a3524294f179077f4c6cdae18a4029f109513efc46e977542029b7d19a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-15T16:00:59.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:59 smithi144 ceph-mon[86635]: pgmap v93: 65 pgs: 9 stale+active+clean, 56 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:00:59.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:00:59 smithi144 ceph-mon[86635]: osdmap e62: 6 total, 5 up, 6 in 2024-05-15T16:00:59.705 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:00:59 smithi049 podman[135068]: 2024-05-15 16:00:59.442473335 +0000 UTC m=+0.692917615 container died ec8c53a3524294f179077f4c6cdae18a4029f109513efc46e977542029b7d19a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-15T16:01:00.455 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 podman[135097]: 2024-05-15 16:01:00.084977108 +0000 UTC m=+0.636723518 container remove ec8c53a3524294f179077f4c6cdae18a4029f109513efc46e977542029b7d19a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-15T16:01:00.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.2.service: Deactivated successfully. 2024-05-15T16:01:00.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 systemd[1]: Stopped Ceph osd.2 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:01:00.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.2.service: Consumed 54.004s CPU time. 2024-05-15T16:01:00.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 systemd[1]: Starting Ceph osd.2 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:01:00.901 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 podman[135196]: 2024-05-15 16:01:00.552736536 +0000 UTC m=+0.114329602 container create ab8dfcc6f388cd70b41dc2c0b3fc41352a047552e647b7d011ec626e2bdec5eb (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T16:01:00.901 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 podman[135196]: 2024-05-15 16:01:00.45685484 +0000 UTC m=+0.018447902 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:01:00.901 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 podman[135196]: 2024-05-15 16:01:00.788840536 +0000 UTC m=+0.350433587 container init ab8dfcc6f388cd70b41dc2c0b3fc41352a047552e647b7d011ec626e2bdec5eb (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:01:00.901 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 podman[135196]: 2024-05-15 16:01:00.792484975 +0000 UTC m=+0.354078030 container start ab8dfcc6f388cd70b41dc2c0b3fc41352a047552e647b7d011ec626e2bdec5eb (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:01:00.901 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:00 smithi049 podman[135196]: 2024-05-15 16:01:00.899465337 +0000 UTC m=+0.461058387 container attach ab8dfcc6f388cd70b41dc2c0b3fc41352a047552e647b7d011ec626e2bdec5eb (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T16:01:01.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:01 smithi049 ceph-mon[112142]: pgmap v95: 65 pgs: 9 stale+active+clean, 56 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate[135210]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate[135210]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 bash[135196]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 bash[135196]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 bash[135196]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 bash[135196]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 bash[135196]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 bash[135196]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 bash[135196]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 bash[135196]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 bash[135196]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate[135210]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate[135210]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate[135210]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-15T16:01:01.456 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate[135210]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-15T16:01:01.457 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate[135210]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-15T16:01:01.457 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate[135210]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-15T16:01:01.457 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate[135210]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-15T16:01:01.457 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:01 smithi049 podman[135196]: 2024-05-15 16:01:01.32742709 +0000 UTC m=+0.889020137 container died ab8dfcc6f388cd70b41dc2c0b3fc41352a047552e647b7d011ec626e2bdec5eb (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-15T16:01:01.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:01 smithi144 ceph-mon[86635]: pgmap v95: 65 pgs: 9 stale+active+clean, 56 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-15T16:01:02.407 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:02 smithi049 ceph-mon[112142]: Health check failed: Degraded data redundancy: 43/321 objects degraded (13.396%), 13 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:02.407 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:02 smithi049 podman[135355]: 2024-05-15 16:01:02.093526988 +0000 UTC m=+0.760325662 container remove ab8dfcc6f388cd70b41dc2c0b3fc41352a047552e647b7d011ec626e2bdec5eb (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-activate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:01:02.407 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:02 smithi049 podman[135430]: 2024-05-15 16:01:02.302662369 +0000 UTC m=+0.102229693 container create b2e567da235f543dfb801aaa02fa47b7f7c19d215c8b145314cd94097097e797 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-15T16:01:02.407 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:02 smithi049 podman[135430]: 2024-05-15 16:01:02.218279536 +0000 UTC m=+0.017846864 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:01:02.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:02 smithi144 ceph-mon[86635]: Health check failed: Degraded data redundancy: 43/321 objects degraded (13.396%), 13 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:02.705 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:02 smithi049 podman[135430]: 2024-05-15 16:01:02.514289653 +0000 UTC m=+0.313856974 container init b2e567da235f543dfb801aaa02fa47b7f7c19d215c8b145314cd94097097e797 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-15T16:01:02.705 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:02 smithi049 podman[135430]: 2024-05-15 16:01:02.517583084 +0000 UTC m=+0.317150407 container start b2e567da235f543dfb801aaa02fa47b7f7c19d215c8b145314cd94097097e797 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-15T16:01:02.705 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:02 smithi049 ceph-osd[135448]: -- 172.21.15.49:0/884921804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f32c9c9680 con 0x55f32c992c00 2024-05-15T16:01:02.705 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:02 smithi049 bash[135430]: b2e567da235f543dfb801aaa02fa47b7f7c19d215c8b145314cd94097097e797 2024-05-15T16:01:02.705 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:02 smithi049 systemd[1]: Started Ceph osd.2 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:01:03.455 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:03 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[135444]: 2024-05-15T16:01:03.056+0000 7f6dff8fb7c0 -1 Falling back to public interface 2024-05-15T16:01:03.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:03 smithi144 ceph-mon[86635]: pgmap v96: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:03.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:03.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:03.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:01:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:03 smithi049 ceph-mon[112142]: pgmap v96: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:01:05.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:05 smithi144 ceph-mon[86635]: pgmap v97: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:05.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:05.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:01:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:05 smithi049 ceph-mon[112142]: pgmap v97: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:01:07.673 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:07 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[135444]: 2024-05-15T16:01:07.283+0000 7f6dff8fb7c0 -1 osd.2 60 log_to_monitors true 2024-05-15T16:01:07.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:07 smithi144 ceph-mon[86635]: pgmap v98: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:07.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:07 smithi144 ceph-mon[86635]: from='osd.2 [v2:172.21.15.49:6818/3306250339,v1:172.21.15.49:6819/3306250339]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T16:01:07.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:07 smithi049 ceph-mon[112142]: pgmap v98: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:07.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:07 smithi049 ceph-mon[112142]: from='osd.2 [v2:172.21.15.49:6818/3306250339,v1:172.21.15.49:6819/3306250339]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T16:01:07.955 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:01:07 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[135444]: 2024-05-15T16:01:07.693+0000 7f6decd4a700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T16:01:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:08 smithi144 ceph-mon[86635]: from='osd.2 [v2:172.21.15.49:6818/3306250339,v1:172.21.15.49:6819/3306250339]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-15T16:01:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:08 smithi144 ceph-mon[86635]: osdmap e63: 6 total, 5 up, 6 in 2024-05-15T16:01:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:08 smithi144 ceph-mon[86635]: from='osd.2 [v2:172.21.15.49:6818/3306250339,v1:172.21.15.49:6819/3306250339]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T16:01:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:08 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:08.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:08 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:08 smithi049 ceph-mon[112142]: from='osd.2 [v2:172.21.15.49:6818/3306250339,v1:172.21.15.49:6819/3306250339]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-15T16:01:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:08 smithi049 ceph-mon[112142]: osdmap e63: 6 total, 5 up, 6 in 2024-05-15T16:01:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:08 smithi049 ceph-mon[112142]: from='osd.2 [v2:172.21.15.49:6818/3306250339,v1:172.21.15.49:6819/3306250339]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-05-15T16:01:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:08 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:08 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:09.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:09 smithi144 ceph-mon[86635]: pgmap v100: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:09.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:09 smithi144 ceph-mon[86635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:01:09.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:09 smithi144 ceph-mon[86635]: osd.2 [v2:172.21.15.49:6818/3306250339,v1:172.21.15.49:6819/3306250339] boot 2024-05-15T16:01:09.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:09 smithi144 ceph-mon[86635]: osdmap e64: 6 total, 6 up, 6 in 2024-05-15T16:01:09.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:09 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T16:01:09.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:09 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:09.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:09 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:09.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:09 smithi144 ceph-mon[86635]: osdmap e65: 6 total, 6 up, 6 in 2024-05-15T16:01:09.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:09 smithi049 ceph-mon[112142]: pgmap v100: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:09.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:09 smithi049 ceph-mon[112142]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:01:09.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:09 smithi049 ceph-mon[112142]: osd.2 [v2:172.21.15.49:6818/3306250339,v1:172.21.15.49:6819/3306250339] boot 2024-05-15T16:01:09.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:09 smithi049 ceph-mon[112142]: osdmap e64: 6 total, 6 up, 6 in 2024-05-15T16:01:09.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:09 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T16:01:09.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:09 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:09.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:09 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:09.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:09 smithi049 ceph-mon[112142]: osdmap e65: 6 total, 6 up, 6 in 2024-05-15T16:01:11.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:11 smithi144 ceph-mon[86635]: pgmap v103: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:11 smithi049 ceph-mon[112142]: pgmap v103: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 43/321 objects degraded (13.396%) 2024-05-15T16:01:13.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: pgmap v104: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 4 op/s; 19/321 objects degraded (5.919%) 2024-05-15T16:01:13.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-05-15T16:01:13.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:13 smithi144 ceph-mon[86635]: Health check update: Degraded data redundancy: 19/321 objects degraded (5.919%), 5 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:13.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: pgmap v104: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 4 op/s; 19/321 objects degraded (5.919%) 2024-05-15T16:01:13.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:13.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:13.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:01:13.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:01:13.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:01:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-15T16:01:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-15T16:01:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-05-15T16:01:13.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:13 smithi049 ceph-mon[112142]: Health check update: Degraded data redundancy: 19/321 objects degraded (5.919%), 5 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:14.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:14 smithi144 ceph-mon[86635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/321 objects degraded (5.919%), 5 pgs degraded) 2024-05-15T16:01:14.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:14 smithi049 ceph-mon[112142]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/321 objects degraded (5.919%), 5 pgs degraded) 2024-05-15T16:01:15.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:15 smithi144 ceph-mon[86635]: pgmap v105: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-15T16:01:15.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:15 smithi049 ceph-mon[112142]: pgmap v105: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-15T16:01:17.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:17 smithi144 ceph-mon[86635]: pgmap v106: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-05-15T16:01:17.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:17 smithi049 ceph-mon[112142]: pgmap v106: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-05-15T16:01:19.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:19 smithi144 ceph-mon[86635]: pgmap v107: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:01:19.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:19 smithi049 ceph-mon[112142]: pgmap v107: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:01:21.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 -- 172.21.15.49:0/3593530769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073340 msgr2=0x7fc0e8073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 --2- 172.21.15.49:0/3593530769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073340 0x7fc0e8073720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800b0a0 tx=0x7fc0d800b3b0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 -- 172.21.15.49:0/3593530769 shutdown_connections 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 --2- 172.21.15.49:0/3593530769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e8073c60 0x7fc0e8111770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 --2- 172.21.15.49:0/3593530769 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073340 0x7fc0e8073720 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 -- 172.21.15.49:0/3593530769 >> 172.21.15.49:0/3593530769 conn(0x7fc0e80fc7d0 msgr2=0x7fc0e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 -- 172.21.15.49:0/3593530769 shutdown_connections 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 -- 172.21.15.49:0/3593530769 wait complete. 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 Processor -- start 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.007+0000 7fc0efb9e700 1 -- start start 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0efb9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e8073340 0x7fc0e819d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0efb9e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073c60 0x7fc0e819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0efb9e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e819ddb0 con 0x7fc0e8073c60 2024-05-15T16:01:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0efb9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e81a1a80 con 0x7fc0e8073340 2024-05-15T16:01:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0ed139700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073c60 0x7fc0e819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0ed139700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073c60 0x7fc0e819d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54310/0 (socket says 172.21.15.49:54310) 2024-05-15T16:01:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0ed139700 1 -- 172.21.15.49:0/416462982 learned_addr learned my addr 172.21.15.49:0/416462982 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0ed93a700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e8073340 0x7fc0e819d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0ed139700 1 -- 172.21.15.49:0/416462982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e8073340 msgr2=0x7fc0e819d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0ed139700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e8073340 0x7fc0e819d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0ed139700 1 -- 172.21.15.49:0/416462982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d8009d00 con 0x7fc0e8073c60 2024-05-15T16:01:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.008+0000 7fc0ed139700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073c60 0x7fc0e819d6d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e920 tx=0x7fc0dc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.009+0000 7fc0e6ffd700 1 -- 172.21.15.49:0/416462982 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0dc00c9d0 con 0x7fc0e8073c60 2024-05-15T16:01:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.009+0000 7fc0e6ffd700 1 -- 172.21.15.49:0/416462982 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0dc010430 con 0x7fc0e8073c60 2024-05-15T16:01:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.009+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e81a1d60 con 0x7fc0e8073c60 2024-05-15T16:01:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.009+0000 7fc0e6ffd700 1 -- 172.21.15.49:0/416462982 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0dc018630 con 0x7fc0e8073c60 2024-05-15T16:01:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.009+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e81a22b0 con 0x7fc0e8073c60 2024-05-15T16:01:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.010+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e8074a80 con 0x7fc0e8073c60 2024-05-15T16:01:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.010+0000 7fc0e6ffd700 1 -- 172.21.15.49:0/416462982 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc0dc0105a0 con 0x7fc0e8073c60 2024-05-15T16:01:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.011+0000 7fc0e6ffd700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc0d006e200 0x7fc0d00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.011+0000 7fc0e6ffd700 1 -- 172.21.15.49:0/416462982 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc0dc014070 con 0x7fc0e8073c60 2024-05-15T16:01:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.011+0000 7fc0ed93a700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc0d006e200 0x7fc0d00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.011+0000 7fc0ed93a700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc0d006e200 0x7fc0d00706c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009480 tx=0x7fc0d800b700 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:21.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.016+0000 7fc0e6ffd700 1 -- 172.21.15.49:0/416462982 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc0dc053850 con 0x7fc0e8073c60 2024-05-15T16:01:21.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.243+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0e81144b0 con 0x7fc0d006e200 2024-05-15T16:01:21.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.244+0000 7fc0e6ffd700 1 -- 172.21.15.49:0/416462982 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc0e81144b0 con 0x7fc0d006e200 2024-05-15T16:01:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc0d006e200 msgr2=0x7fc0d00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc0d006e200 0x7fc0d00706c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009480 tx=0x7fc0d800b700 comp rx=0 tx=0).stop 2024-05-15T16:01:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073c60 msgr2=0x7fc0e819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073c60 0x7fc0e819d6d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e920 tx=0x7fc0dc00ec30 comp rx=0 tx=0).stop 2024-05-15T16:01:21.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 shutdown_connections 2024-05-15T16:01:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc0d006e200 0x7fc0d00706c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e8073340 0x7fc0e819d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 --2- 172.21.15.49:0/416462982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0e8073c60 0x7fc0e819d6d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 >> 172.21.15.49:0/416462982 conn(0x7fc0e80fc7d0 msgr2=0x7fc0e8100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 shutdown_connections 2024-05-15T16:01:21.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.247+0000 7fc0efb9e700 1 -- 172.21.15.49:0/416462982 wait complete. 2024-05-15T16:01:21.259 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:01:21.259 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:21 smithi049 ceph-mon[112142]: pgmap v108: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-05-15T16:01:21.259 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:21.259 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:01:21.260 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:21.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.358+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/4093311293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64101e70 msgr2=0x7f4b64102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.358+0000 7f4b6b9f2700 1 --2- 172.21.15.49:0/4093311293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64101e70 0x7f4b64102250 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009910 tx=0x7f4b60009c20 comp rx=0 tx=0).stop 2024-05-15T16:01:21.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.358+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/4093311293 shutdown_connections 2024-05-15T16:01:21.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.358+0000 7f4b6b9f2700 1 --2- 172.21.15.49:0/4093311293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b64102790 0x7f4b6410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.358+0000 7f4b6b9f2700 1 --2- 172.21.15.49:0/4093311293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64101e70 0x7f4b64102250 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.358+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/4093311293 >> 172.21.15.49:0/4093311293 conn(0x7f4b640fb700 msgr2=0x7f4b640fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:21.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.358+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/4093311293 shutdown_connections 2024-05-15T16:01:21.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.358+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/4093311293 wait complete. 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b6b9f2700 1 Processor -- start 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b6b9f2700 1 -- start start 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b6b9f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b64101e70 0x7f4b640ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b6b9f2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64102790 0x7f4b64100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b6b9f2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b64100550 con 0x7f4b64102790 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b6b9f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b64100690 con 0x7f4b64101e70 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b68f8d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64102790 0x7f4b64100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b68f8d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64102790 0x7f4b64100010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54336/0 (socket says 172.21.15.49:54336) 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b68f8d700 1 -- 172.21.15.49:0/220322809 learned_addr learned my addr 172.21.15.49:0/220322809 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b6978e700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b64101e70 0x7f4b640ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.359+0000 7f4b68f8d700 1 -- 172.21.15.49:0/220322809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b64101e70 msgr2=0x7f4b640ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.360+0000 7f4b68f8d700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b64101e70 0x7f4b640ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.360+0000 7f4b68f8d700 1 -- 172.21.15.49:0/220322809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600095f0 con 0x7f4b64102790 2024-05-15T16:01:21.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.360+0000 7f4b68f8d700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64102790 0x7f4b64100010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400d6d0 tx=0x7f4b5400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:21.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.360+0000 7f4b5a7fc700 1 -- 172.21.15.49:0/220322809 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b5400f860 con 0x7f4b64102790 2024-05-15T16:01:21.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.360+0000 7f4b5a7fc700 1 -- 172.21.15.49:0/220322809 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b5400fe80 con 0x7f4b64102790 2024-05-15T16:01:21.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.360+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b641a2e10 con 0x7f4b64102790 2024-05-15T16:01:21.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.360+0000 7f4b5a7fc700 1 -- 172.21.15.49:0/220322809 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b54010ea0 con 0x7f4b64102790 2024-05-15T16:01:21.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.360+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b641a3170 con 0x7f4b64102790 2024-05-15T16:01:21.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.361+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b64069ab0 con 0x7f4b64102790 2024-05-15T16:01:21.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.361+0000 7f4b5a7fc700 1 -- 172.21.15.49:0/220322809 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4b54010460 con 0x7f4b64102790 2024-05-15T16:01:21.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.362+0000 7f4b5a7fc700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4b5006e200 0x7f4b500706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:21.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.362+0000 7f4b5a7fc700 1 -- 172.21.15.49:0/220322809 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4b5408dca0 con 0x7f4b64102790 2024-05-15T16:01:21.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.362+0000 7f4b6978e700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4b5006e200 0x7f4b500706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:21.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.362+0000 7f4b6978e700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4b5006e200 0x7f4b500706c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000b5c0 tx=0x7f4b6001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:21.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.367+0000 7f4b5a7fc700 1 -- 172.21.15.49:0/220322809 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4b54092070 con 0x7f4b64102790 2024-05-15T16:01:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:21 smithi144 ceph-mon[86635]: pgmap v108: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-05-15T16:01:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:01:21.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:21.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.601+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b6410def0 con 0x7f4b5006e200 2024-05-15T16:01:21.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.602+0000 7f4b5a7fc700 1 -- 172.21.15.49:0/220322809 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4b6410def0 con 0x7f4b5006e200 2024-05-15T16:01:21.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.604+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4b5006e200 msgr2=0x7f4b500706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4b5006e200 0x7f4b500706c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000b5c0 tx=0x7f4b6001a040 comp rx=0 tx=0).stop 2024-05-15T16:01:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64102790 msgr2=0x7f4b64100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64102790 0x7f4b64100010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400d6d0 tx=0x7f4b5400da90 comp rx=0 tx=0).stop 2024-05-15T16:01:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 shutdown_connections 2024-05-15T16:01:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4b5006e200 0x7f4b500706c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b64101e70 0x7f4b640ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 --2- 172.21.15.49:0/220322809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b64102790 0x7f4b64100010 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 >> 172.21.15.49:0/220322809 conn(0x7f4b640fb700 msgr2=0x7f4b64103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 shutdown_connections 2024-05-15T16:01:21.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.605+0000 7f4b6b9f2700 1 -- 172.21.15.49:0/220322809 wait complete. 2024-05-15T16:01:21.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.714+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/39522095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e80ffdf0 msgr2=0x7ff4e81001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.714+0000 7ff4ed0c7700 1 --2- 172.21.15.49:0/39522095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e80ffdf0 0x7ff4e81001d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800b0a0 tx=0x7ff4d800b3b0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.714+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/39522095 shutdown_connections 2024-05-15T16:01:21.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.714+0000 7ff4ed0c7700 1 --2- 172.21.15.49:0/39522095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e81007a0 0x7ff4e81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.714+0000 7ff4ed0c7700 1 --2- 172.21.15.49:0/39522095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e80ffdf0 0x7ff4e81001d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.714+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/39522095 >> 172.21.15.49:0/39522095 conn(0x7ff4e80fb6a0 msgr2=0x7ff4e80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:21.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.714+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/39522095 shutdown_connections 2024-05-15T16:01:21.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.714+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/39522095 wait complete. 2024-05-15T16:01:21.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4ed0c7700 1 Processor -- start 2024-05-15T16:01:21.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4ed0c7700 1 -- start start 2024-05-15T16:01:21.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4ed0c7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e80ffdf0 0x7ff4e819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:21.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4ed0c7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e81007a0 0x7ff4e819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:21.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4ed0c7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e819d430 con 0x7ff4e81007a0 2024-05-15T16:01:21.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4ed0c7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e8196a10 con 0x7ff4e80ffdf0 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4e659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e81007a0 0x7ff4e819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4e659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e81007a0 0x7ff4e819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54354/0 (socket says 172.21.15.49:54354) 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4e659c700 1 -- 172.21.15.49:0/3753421757 learned_addr learned my addr 172.21.15.49:0/3753421757 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.715+0000 7ff4e6d9d700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e80ffdf0 0x7ff4e819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4e659c700 1 -- 172.21.15.49:0/3753421757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e80ffdf0 msgr2=0x7ff4e819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4e659c700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e80ffdf0 0x7ff4e819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4e659c700 1 -- 172.21.15.49:0/3753421757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d8009d00 con 0x7ff4e81007a0 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4e6d9d700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e80ffdf0 0x7ff4e819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4e659c700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e81007a0 0x7ff4e819ce10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000e8f0 tx=0x7ff4d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4dffff700 1 -- 172.21.15.49:0/3753421757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4d0009790 con 0x7ff4e81007a0 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e8196cf0 con 0x7ff4e81007a0 2024-05-15T16:01:21.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4dffff700 1 -- 172.21.15.49:0/3753421757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4d0004cf0 con 0x7ff4e81007a0 2024-05-15T16:01:21.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4dffff700 1 -- 172.21.15.49:0/3753421757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4d0005690 con 0x7ff4e81007a0 2024-05-15T16:01:21.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.716+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e8197240 con 0x7ff4e81007a0 2024-05-15T16:01:21.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.717+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e81091a0 con 0x7ff4e81007a0 2024-05-15T16:01:21.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.718+0000 7ff4dffff700 1 -- 172.21.15.49:0/3753421757 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff4d0004750 con 0x7ff4e81007a0 2024-05-15T16:01:21.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.718+0000 7ff4dffff700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4d406e200 0x7ff4d40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:21.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.718+0000 7ff4dffff700 1 -- 172.21.15.49:0/3753421757 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff4d0014070 con 0x7ff4e81007a0 2024-05-15T16:01:21.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.718+0000 7ff4e6d9d700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4d406e200 0x7ff4d40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:21.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.719+0000 7ff4e6d9d700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4d406e200 0x7ff4d40706c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009480 tx=0x7ff4d8006210 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:21.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.723+0000 7ff4dffff700 1 -- 172.21.15.49:0/3753421757 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff4d005b3c0 con 0x7ff4e81007a0 2024-05-15T16:01:21.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.941+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4e8197f40 con 0x7ff4d406e200 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.953+0000 7ff4dffff700 1 -- 172.21.15.49:0/3753421757 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff4e8197f40 con 0x7ff4d406e200 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (4m) 14s ago 16m 24.9M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (16m) 14s ago 16m 8883k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (15m) 110s ago 15m 8162k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (119s) 14s ago 16m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (114s) 110s ago 15m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (4m) 14s ago 15m 79.1M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (10m) 14s ago 10m 90.1M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T16:01:21.955 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (10m) 14s ago 10m 152M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (10m) 110s ago 10m 26.7M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (10m) 110s ago 10m 28.4M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (6m) 14s ago 17m 507M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (5m) 110s ago 15m 451M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (3m) 14s ago 17m 47.3M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (2m) 110s ago 14m 35.0M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (5m) 14s ago 16m 15.8M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (5m) 110s ago 15m 16.2M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (92s) 14s ago 14m 82.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (56s) 14s ago 13m 84.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (19s) 14s ago 13m 14.8M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:01:21.956 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (12m) 110s ago 12m 261M 2240M 18.2.2-1343-g2965081f a04a73ffe079 58f6c9765c0b 2024-05-15T16:01:21.957 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (12m) 110s ago 12m 217M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T16:01:21.957 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (12m) 110s ago 12m 203M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T16:01:21.957 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (4m) 14s ago 15m 68.7M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:01:21.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.955+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4d406e200 msgr2=0x7ff4d40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.955+0000 7ff4ed0c7700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4d406e200 0x7ff4d40706c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009480 tx=0x7ff4d8006210 comp rx=0 tx=0).stop 2024-05-15T16:01:21.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.955+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e81007a0 msgr2=0x7ff4e819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.955+0000 7ff4ed0c7700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e81007a0 0x7ff4e819ce10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000e8f0 tx=0x7ff4d000ecb0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.955+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 shutdown_connections 2024-05-15T16:01:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.955+0000 7ff4ed0c7700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff4d406e200 0x7ff4d40706c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.955+0000 7ff4ed0c7700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4e80ffdf0 0x7ff4e819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.955+0000 7ff4ed0c7700 1 --2- 172.21.15.49:0/3753421757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4e81007a0 0x7ff4e819ce10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.955+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 >> 172.21.15.49:0/3753421757 conn(0x7ff4e80fb6a0 msgr2=0x7ff4e8069740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.956+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 shutdown_connections 2024-05-15T16:01:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:21.956+0000 7ff4ed0c7700 1 -- 172.21.15.49:0/3753421757 wait complete. 2024-05-15T16:01:22.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.063+0000 7f52e1e29700 1 -- 172.21.15.49:0/1916916076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 msgr2=0x7f52dc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.063+0000 7f52e1e29700 1 --2- 172.21.15.49:0/1916916076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 0x7f52dc1012c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009910 tx=0x7f52cc009c20 comp rx=0 tx=0).stop 2024-05-15T16:01:22.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.063+0000 7f52e1e29700 1 -- 172.21.15.49:0/1916916076 shutdown_connections 2024-05-15T16:01:22.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.063+0000 7f52e1e29700 1 --2- 172.21.15.49:0/1916916076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52dc101890 0x7f52dc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.063+0000 7f52e1e29700 1 --2- 172.21.15.49:0/1916916076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 0x7f52dc1012c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.063+0000 7f52e1e29700 1 -- 172.21.15.49:0/1916916076 >> 172.21.15.49:0/1916916076 conn(0x7f52dc0fc790 msgr2=0x7f52dc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:22.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.063+0000 7f52e1e29700 1 -- 172.21.15.49:0/1916916076 shutdown_connections 2024-05-15T16:01:22.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.063+0000 7f52e1e29700 1 -- 172.21.15.49:0/1916916076 wait complete. 2024-05-15T16:01:22.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52e1e29700 1 Processor -- start 2024-05-15T16:01:22.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52e1e29700 1 -- start start 2024-05-15T16:01:22.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52e1e29700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 0x7f52dc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:22.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52e1e29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52dc101890 0x7f52dc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:22.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52e1e29700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc073d20 con 0x7f52dc100ee0 2024-05-15T16:01:22.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52e1e29700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc073e60 con 0x7f52dc101890 2024-05-15T16:01:22.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52db7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 0x7f52dc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:22.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52db7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 0x7f52dc0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54376/0 (socket says 172.21.15.49:54376) 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52db7fe700 1 -- 172.21.15.49:0/1536756818 learned_addr learned my addr 172.21.15.49:0/1536756818 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.064+0000 7f52d2ffd700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52dc101890 0x7f52dc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52db7fe700 1 -- 172.21.15.49:0/1536756818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52dc101890 msgr2=0x7f52dc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52db7fe700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52dc101890 0x7f52dc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52db7fe700 1 -- 172.21.15.49:0/1536756818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0095f0 con 0x7f52dc100ee0 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52d2ffd700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52dc101890 0x7f52dc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52db7fe700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 0x7f52dc0732a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52cc000c00 tx=0x7f52cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52d97fa700 1 -- 172.21.15.49:0/1536756818 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52cc01d070 con 0x7f52dc100ee0 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52d97fa700 1 -- 172.21.15.49:0/1536756818 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52cc0056e0 con 0x7f52dc100ee0 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc0740e0 con 0x7f52dc100ee0 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52d97fa700 1 -- 172.21.15.49:0/1536756818 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52cc00fe80 con 0x7f52dc100ee0 2024-05-15T16:01:22.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.065+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc1a3040 con 0x7f52dc100ee0 2024-05-15T16:01:22.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.066+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc109280 con 0x7f52dc100ee0 2024-05-15T16:01:22.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.067+0000 7f52d97fa700 1 -- 172.21.15.49:0/1536756818 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f52cc00f6f0 con 0x7f52dc100ee0 2024-05-15T16:01:22.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.067+0000 7f52d97fa700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52c806e210 0x7f52c80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:22.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.067+0000 7f52d97fa700 1 -- 172.21.15.49:0/1536756818 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f52cc08f540 con 0x7f52dc100ee0 2024-05-15T16:01:22.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.067+0000 7f52d2ffd700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52c806e210 0x7f52c80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:22.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.068+0000 7f52d2ffd700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52c806e210 0x7f52c80706d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f52c4005950 tx=0x7f52c40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:22.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.072+0000 7f52d97fa700 1 -- 172.21.15.49:0/1536756818 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f52cc0588d0 con 0x7f52dc100ee0 2024-05-15T16:01:22.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:22 smithi049 ceph-mon[112142]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:22.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:22 smithi049 ceph-mon[112142]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:22.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:22 smithi049 ceph-mon[112142]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:22.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.365+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f52dc10deb0 con 0x7f52dc100ee0 2024-05-15T16:01:22.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.366+0000 7f52d97fa700 1 -- 172.21.15.49:0/1536756818 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f52cc05bf90 con 0x7f52dc100ee0 2024-05-15T16:01:22.367 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:01:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:01:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 3, 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 7, 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 7 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:01:22.368 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:01:22.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52c806e210 msgr2=0x7f52c80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52c806e210 0x7f52c80706d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f52c4005950 tx=0x7f52c40058e0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 msgr2=0x7f52dc0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 0x7f52dc0732a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52cc000c00 tx=0x7f52cc00be20 comp rx=0 tx=0).stop 2024-05-15T16:01:22.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 shutdown_connections 2024-05-15T16:01:22.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52c806e210 0x7f52c80706d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52dc100ee0 0x7f52dc0732a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 --2- 172.21.15.49:0/1536756818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52dc101890 0x7f52dc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 >> 172.21.15.49:0/1536756818 conn(0x7f52dc0fc790 msgr2=0x7f52dc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:22.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 shutdown_connections 2024-05-15T16:01:22.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.368+0000 7f52e1e29700 1 -- 172.21.15.49:0/1536756818 wait complete. 2024-05-15T16:01:22.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:22 smithi144 ceph-mon[86635]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:22.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:22 smithi144 ceph-mon[86635]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:22.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:22 smithi144 ceph-mon[86635]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:22.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.492+0000 7f1471178700 1 -- 172.21.15.49:0/2500514156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c100f00 msgr2=0x7f146c1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.492+0000 7f1471178700 1 --2- 172.21.15.49:0/2500514156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c100f00 0x7f146c1012e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f145c009910 tx=0x7f145c009c20 comp rx=0 tx=0).stop 2024-05-15T16:01:22.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.492+0000 7f1471178700 1 -- 172.21.15.49:0/2500514156 shutdown_connections 2024-05-15T16:01:22.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.492+0000 7f1471178700 1 --2- 172.21.15.49:0/2500514156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146c1018b0 0x7f146c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.492+0000 7f1471178700 1 --2- 172.21.15.49:0/2500514156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c100f00 0x7f146c1012e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.492+0000 7f1471178700 1 -- 172.21.15.49:0/2500514156 >> 172.21.15.49:0/2500514156 conn(0x7f146c0fc7d0 msgr2=0x7f146c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:22.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.492+0000 7f1471178700 1 -- 172.21.15.49:0/2500514156 shutdown_connections 2024-05-15T16:01:22.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.492+0000 7f1471178700 1 -- 172.21.15.49:0/2500514156 wait complete. 2024-05-15T16:01:22.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.493+0000 7f1471178700 1 Processor -- start 2024-05-15T16:01:22.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.493+0000 7f1471178700 1 -- start start 2024-05-15T16:01:22.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.493+0000 7f1471178700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146c100f00 0x7f146c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:22.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.493+0000 7f1471178700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c1018b0 0x7f146c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:22.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.493+0000 7f1471178700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c073d20 con 0x7f146c1018b0 2024-05-15T16:01:22.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.493+0000 7f1471178700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c073e60 con 0x7f146c100f00 2024-05-15T16:01:22.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.493+0000 7f146a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c1018b0 0x7f146c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f146a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c1018b0 0x7f146c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54400/0 (socket says 172.21.15.49:54400) 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f146a59c700 1 -- 172.21.15.49:0/2546836234 learned_addr learned my addr 172.21.15.49:0/2546836234 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f146ad9d700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146c100f00 0x7f146c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f146a59c700 1 -- 172.21.15.49:0/2546836234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146c100f00 msgr2=0x7f146c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f146a59c700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146c100f00 0x7f146c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f146a59c700 1 -- 172.21.15.49:0/2546836234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c0095f0 con 0x7f146c1018b0 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f146a59c700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c1018b0 0x7f146c0737e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f145400d900 tx=0x7f145400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f1463fff700 1 -- 172.21.15.49:0/2546836234 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1454009900 con 0x7f146c1018b0 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c1a2e10 con 0x7f146c1018b0 2024-05-15T16:01:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f1463fff700 1 -- 172.21.15.49:0/2546836234 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1454010460 con 0x7f146c1018b0 2024-05-15T16:01:22.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f1463fff700 1 -- 172.21.15.49:0/2546836234 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145400f5f0 con 0x7f146c1018b0 2024-05-15T16:01:22.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.494+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c1a3170 con 0x7f146c1018b0 2024-05-15T16:01:22.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.495+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c1092a0 con 0x7f146c1018b0 2024-05-15T16:01:22.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.496+0000 7f1463fff700 1 -- 172.21.15.49:0/2546836234 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f14540105d0 con 0x7f146c1018b0 2024-05-15T16:01:22.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.496+0000 7f1463fff700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f145806e200 0x7f14580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:22.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.497+0000 7f146ad9d700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f145806e200 0x7f14580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:22.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.497+0000 7f1463fff700 1 -- 172.21.15.49:0/2546836234 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f145408dca0 con 0x7f146c1018b0 2024-05-15T16:01:22.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.497+0000 7f146ad9d700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f145806e200 0x7f14580706c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f145c00b5c0 tx=0x7f145c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:22.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.501+0000 7f1463fff700 1 -- 172.21.15.49:0/2546836234 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1454092070 con 0x7f146c1018b0 2024-05-15T16:01:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.753+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f146c074730 con 0x7f146c1018b0 2024-05-15T16:01:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.754+0000 7f1463fff700 1 -- 172.21.15.49:0/2546836234 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f1454017780 con 0x7f146c1018b0 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:e17 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 17 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:51:00.559034+0000 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:01:22.756 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:01:22.757 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:01:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f145806e200 msgr2=0x7f14580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f145806e200 0x7f14580706c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f145c00b5c0 tx=0x7f145c01a040 comp rx=0 tx=0).stop 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c1018b0 msgr2=0x7f146c0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c1018b0 0x7f146c0737e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f145400d900 tx=0x7f145400dc10 comp rx=0 tx=0).stop 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 shutdown_connections 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f145806e200 0x7f14580706c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146c100f00 0x7f146c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 --2- 172.21.15.49:0/2546836234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f146c1018b0 0x7f146c0737e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 >> 172.21.15.49:0/2546836234 conn(0x7f146c0fc7d0 msgr2=0x7f146c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:22.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 shutdown_connections 2024-05-15T16:01:22.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.756+0000 7f1471178700 1 -- 172.21.15.49:0/2546836234 wait complete. 2024-05-15T16:01:22.760 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3644720189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f8101eb0 msgr2=0x7f56f8102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 --2- 172.21.15.49:0/3644720189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f8101eb0 0x7f56f8102290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009940 tx=0x7f56e8009c50 comp rx=0 tx=0).stop 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3644720189 shutdown_connections 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 --2- 172.21.15.49:0/3644720189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f81027d0 0x7f56f810acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 --2- 172.21.15.49:0/3644720189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f8101eb0 0x7f56f8102290 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3644720189 >> 172.21.15.49:0/3644720189 conn(0x7f56f80fb6a0 msgr2=0x7f56f80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3644720189 shutdown_connections 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3644720189 wait complete. 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 Processor -- start 2024-05-15T16:01:22.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.866+0000 7f56fdf3d700 1 -- start start 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56fdf3d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f8101eb0 0x7f56f8196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56fdf3d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f81027d0 0x7f56f8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56fdf3d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f81978d0 con 0x7f56f81027d0 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56fdf3d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f819a540 con 0x7f56f8101eb0 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56eeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f81027d0 0x7f56f8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56eeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f81027d0 0x7f56f8197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54424/0 (socket says 172.21.15.49:54424) 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56eeffd700 1 -- 172.21.15.49:0/3024577947 learned_addr learned my addr 172.21.15.49:0/3024577947 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56f77fe700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f8101eb0 0x7f56f8196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56eeffd700 1 -- 172.21.15.49:0/3024577947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f8101eb0 msgr2=0x7f56f8196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56eeffd700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f8101eb0 0x7f56f8196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56eeffd700 1 -- 172.21.15.49:0/3024577947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e80095f0 con 0x7f56f81027d0 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.867+0000 7f56f77fe700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f8101eb0 0x7f56f8196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:01:22.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.868+0000 7f56eeffd700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f81027d0 0x7f56f8197240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f56e000b480 tx=0x7f56e000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:22.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.868+0000 7f56f57fa700 1 -- 172.21.15.49:0/3024577947 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e00118c0 con 0x7f56f81027d0 2024-05-15T16:01:22.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.868+0000 7f56f57fa700 1 -- 172.21.15.49:0/3024577947 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56e0010380 con 0x7f56f81027d0 2024-05-15T16:01:22.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.868+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f819a820 con 0x7f56f81027d0 2024-05-15T16:01:22.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.868+0000 7f56f57fa700 1 -- 172.21.15.49:0/3024577947 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e000f4f0 con 0x7f56f81027d0 2024-05-15T16:01:22.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.868+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f819ad70 con 0x7f56f81027d0 2024-05-15T16:01:22.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.869+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f8069a50 con 0x7f56f81027d0 2024-05-15T16:01:22.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.869+0000 7f56f57fa700 1 -- 172.21.15.49:0/3024577947 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f56e00104f0 con 0x7f56f81027d0 2024-05-15T16:01:22.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.870+0000 7f56f57fa700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56e406e200 0x7f56e40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:22.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.870+0000 7f56f57fa700 1 -- 172.21.15.49:0/3024577947 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f56e008dc70 con 0x7f56f81027d0 2024-05-15T16:01:22.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.870+0000 7f56f77fe700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56e406e200 0x7f56e40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:22.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.870+0000 7f56f77fe700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56e406e200 0x7f56e40706c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f56e800b870 tx=0x7f56e80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:22.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:22.875+0000 7f56f57fa700 1 -- 172.21.15.49:0/3024577947 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f56e0057080 con 0x7f56f81027d0 2024-05-15T16:01:23.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.101+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56f81055d0 con 0x7f56e406e200 2024-05-15T16:01:23.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.103+0000 7f56f57fa700 1 -- 172.21.15.49:0/3024577947 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f56f81055d0 con 0x7f56e406e200 2024-05-15T16:01:23.104 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:01:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:01:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:01:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:01:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:01:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:01:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:01:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:01:23.105 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:01:23.105 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "9/23 daemons upgraded", 2024-05-15T16:01:23.105 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading osd daemons", 2024-05-15T16:01:23.105 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:01:23.105 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56e406e200 msgr2=0x7f56e40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56e406e200 0x7f56e40706c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f56e800b870 tx=0x7f56e80058e0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f81027d0 msgr2=0x7f56f8197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f81027d0 0x7f56f8197240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f56e000b480 tx=0x7f56e000b840 comp rx=0 tx=0).stop 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 shutdown_connections 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56e406e200 0x7f56e40706c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f8101eb0 0x7f56f8196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 --2- 172.21.15.49:0/3024577947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f81027d0 0x7f56f8197240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 >> 172.21.15.49:0/3024577947 conn(0x7f56f80fb6a0 msgr2=0x7f56f81031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 shutdown_connections 2024-05-15T16:01:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.105+0000 7f56fdf3d700 1 -- 172.21.15.49:0/3024577947 wait complete. 2024-05-15T16:01:23.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.215+0000 7f8209f02700 1 -- 172.21.15.49:0/1282894142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 msgr2=0x7f82041047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:23.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.215+0000 7f8209f02700 1 --2- 172.21.15.49:0/1282894142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 0x7f82041047f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009910 tx=0x7f81ec009c20 comp rx=0 tx=0).stop 2024-05-15T16:01:23.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.215+0000 7f8209f02700 1 -- 172.21.15.49:0/1282894142 shutdown_connections 2024-05-15T16:01:23.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.215+0000 7f8209f02700 1 --2- 172.21.15.49:0/1282894142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 0x7f82041047f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.215+0000 7f8209f02700 1 --2- 172.21.15.49:0/1282894142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ffdf0 0x7f82041001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.215+0000 7f8209f02700 1 -- 172.21.15.49:0/1282894142 >> 172.21.15.49:0/1282894142 conn(0x7f82040fb6a0 msgr2=0x7f82040fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:23.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.215+0000 7f8209f02700 1 -- 172.21.15.49:0/1282894142 shutdown_connections 2024-05-15T16:01:23.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.215+0000 7f8209f02700 1 -- 172.21.15.49:0/1282894142 wait complete. 2024-05-15T16:01:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f8209f02700 1 Processor -- start 2024-05-15T16:01:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f8209f02700 1 -- start start 2024-05-15T16:01:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f8209f02700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ffdf0 0x7f82041959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f8209f02700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 0x7f8204195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f8209f02700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820406a140 con 0x7f82041007a0 2024-05-15T16:01:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f8209f02700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8204196420 con 0x7f82040ffdf0 2024-05-15T16:01:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f81faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 0x7f8204195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f81faffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 0x7f8204195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54434/0 (socket says 172.21.15.49:54434) 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f81faffd700 1 -- 172.21.15.49:0/179865308 learned_addr learned my addr 172.21.15.49:0/179865308 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.216+0000 7f82037fe700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ffdf0 0x7f82041959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.217+0000 7f81faffd700 1 -- 172.21.15.49:0/179865308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ffdf0 msgr2=0x7f82041959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.217+0000 7f81faffd700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ffdf0 0x7f82041959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.217+0000 7f81faffd700 1 -- 172.21.15.49:0/179865308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0095f0 con 0x7f82041007a0 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.217+0000 7f81faffd700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 0x7f8204195ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00bd70 tx=0x7f81ec00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.217+0000 7f82017fa700 1 -- 172.21.15.49:0/179865308 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81ec01d070 con 0x7f82041007a0 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.217+0000 7f82017fa700 1 -- 172.21.15.49:0/179865308 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81ec0056e0 con 0x7f82041007a0 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.217+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8204196670 con 0x7f82041007a0 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.217+0000 7f82017fa700 1 -- 172.21.15.49:0/179865308 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81ec00fe50 con 0x7f82041007a0 2024-05-15T16:01:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.217+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82041a31e0 con 0x7f82041007a0 2024-05-15T16:01:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.218+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82041081f0 con 0x7f82041007a0 2024-05-15T16:01:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.219+0000 7f82017fa700 1 -- 172.21.15.49:0/179865308 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f81ec00f680 con 0x7f82041007a0 2024-05-15T16:01:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.219+0000 7f82017fa700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f81f006e200 0x7f81f00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.219+0000 7f82017fa700 1 -- 172.21.15.49:0/179865308 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f81ec08f420 con 0x7f82041007a0 2024-05-15T16:01:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.219+0000 7f82037fe700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f81f006e200 0x7f81f00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.220+0000 7f82037fe700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f81f006e200 0x7f81f00706c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81f4005950 tx=0x7f81f400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:23.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.224+0000 7f82017fa700 1 -- 172.21.15.49:0/179865308 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f81ec0587b0 con 0x7f82041007a0 2024-05-15T16:01:23.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:23 smithi144 ceph-mon[86635]: pgmap v109: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-15T16:01:23.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:23 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1536756818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:23.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:23 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2546836234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:01:23.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:23 smithi049 ceph-mon[112142]: pgmap v109: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-15T16:01:23.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:23 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1536756818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:23.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:23 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2546836234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:01:23.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.520+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8204069770 con 0x7f82041007a0 2024-05-15T16:01:23.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.521+0000 7f82017fa700 1 -- 172.21.15.49:0/179865308 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f81ec05be70 con 0x7f82041007a0 2024-05-15T16:01:23.523 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T16:01:23.523 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:01:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f81f006e200 msgr2=0x7f81f00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f81f006e200 0x7f81f00706c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81f4005950 tx=0x7f81f400b410 comp rx=0 tx=0).stop 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 msgr2=0x7f8204195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 0x7f8204195ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00bd70 tx=0x7f81ec00bda0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 shutdown_connections 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f81f006e200 0x7f81f00706c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ffdf0 0x7f82041959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 --2- 172.21.15.49:0/179865308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82041007a0 0x7f8204195ee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 >> 172.21.15.49:0/179865308 conn(0x7f82040fb6a0 msgr2=0x7f82040fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 shutdown_connections 2024-05-15T16:01:23.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:23.523+0000 7f8209f02700 1 -- 172.21.15.49:0/179865308 wait complete. 2024-05-15T16:01:24.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:24 smithi144 ceph-mon[86635]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:24.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:24 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/179865308' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:01:24.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:24 smithi049 ceph-mon[112142]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:24.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:24 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/179865308' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:01:25.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:25 smithi144 ceph-mon[86635]: pgmap v110: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:01:25.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:25 smithi049 ceph-mon[112142]: pgmap v110: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:01:27.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:27 smithi144 ceph-mon[86635]: pgmap v111: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:01:27.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:27 smithi049 ceph-mon[112142]: pgmap v111: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:01:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:28 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-15T16:01:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:28 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-15T16:01:28.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:28 smithi144 ceph-mon[86635]: Upgrade: osd.3 is safe to restart 2024-05-15T16:01:28.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:28 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-15T16:01:28.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:28 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-15T16:01:28.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:28 smithi049 ceph-mon[112142]: Upgrade: osd.3 is safe to restart 2024-05-15T16:01:29.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:29 smithi144 ceph-mon[86635]: pgmap v112: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-15T16:01:29.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:29 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:29.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:29 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-15T16:01:29.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:29 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:01:29.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:29 smithi049 ceph-mon[112142]: pgmap v112: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-15T16:01:29.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:29 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:29.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:29 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-15T16:01:29.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:29 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:01:30.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:30 smithi144 ceph-mon[86635]: Upgrade: Updating osd.3 2024-05-15T16:01:30.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:30 smithi144 ceph-mon[86635]: Deploying daemon osd.3 on smithi144 2024-05-15T16:01:30.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:30 smithi049 ceph-mon[112142]: Upgrade: Updating osd.3 2024-05-15T16:01:30.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:30 smithi049 ceph-mon[112142]: Deploying daemon osd.3 on smithi144 2024-05-15T16:01:31.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:31 smithi144 ceph-mon[86635]: pgmap v113: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:01:31.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:31 smithi049 ceph-mon[112142]: pgmap v113: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:01:32.629 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:32 smithi144 systemd[1]: Stopping Ceph osd.3 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:01:32.949 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:32 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[42263]: 2024-05-15T16:01:32.657+0000 7f3f422ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:01:32.949 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:32 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[42263]: 2024-05-15T16:01:32.657+0000 7f3f422ff700 -1 osd.3 65 *** Got signal Terminated *** 2024-05-15T16:01:32.949 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:32 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[42263]: 2024-05-15T16:01:32.657+0000 7f3f422ff700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:01:33.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:33 smithi049 ceph-mon[112142]: pgmap v114: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:01:33.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:33 smithi049 ceph-mon[112142]: osd.3 marked itself down and dead 2024-05-15T16:01:33.571 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:33 smithi144 ceph-mon[86635]: pgmap v114: 65 pgs: 65 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:01:33.571 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:33 smithi144 ceph-mon[86635]: osd.3 marked itself down and dead 2024-05-15T16:01:33.879 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:33 smithi144 podman[95555]: 2024-05-15 16:01:33.604236394 +0000 UTC m=+1.079196432 container died 58f6c9765c0b0b3451f2fb8cad73fa055980ef04ef54e4e243fa914caa3eff19 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=reef-2965081, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:01:34.199 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:33 smithi144 podman[95555]: 2024-05-15 16:01:33.986763876 +0000 UTC m=+1.461723943 container cleanup 58f6c9765c0b0b3451f2fb8cad73fa055980ef04ef54e4e243fa914caa3eff19 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=reef-2965081, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-15T16:01:34.199 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:33 smithi144 bash[95555]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3 2024-05-15T16:01:34.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:34 smithi049 ceph-mon[112142]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:01:34.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:34 smithi049 ceph-mon[112142]: osdmap e66: 6 total, 5 up, 6 in 2024-05-15T16:01:34.637 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:34 smithi144 ceph-mon[86635]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:01:34.638 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:34 smithi144 ceph-mon[86635]: osdmap e66: 6 total, 5 up, 6 in 2024-05-15T16:01:34.638 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:34 smithi144 podman[95580]: 2024-05-15 16:01:34.270780077 +0000 UTC m=+0.663752194 container remove 58f6c9765c0b0b3451f2fb8cad73fa055980ef04ef54e4e243fa914caa3eff19 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=reef-2965081, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-15T16:01:34.926 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:34 smithi144 podman[95672]: 2024-05-15 16:01:34.538726303 +0000 UTC m=+0.017219360 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:01:34.926 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:34 smithi144 podman[95672]: 2024-05-15 16:01:34.646761316 +0000 UTC m=+0.125254360 container create f1c07acfac49a4b94e0e882cb47d9daef447e00ab54ce8438daf2474cc836273 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-15T16:01:34.926 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:34 smithi144 podman[95672]: 2024-05-15 16:01:34.858283635 +0000 UTC m=+0.336776690 container init f1c07acfac49a4b94e0e882cb47d9daef447e00ab54ce8438daf2474cc836273 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-15T16:01:34.926 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:34 smithi144 podman[95672]: 2024-05-15 16:01:34.861817247 +0000 UTC m=+0.340310305 container start f1c07acfac49a4b94e0e882cb47d9daef447e00ab54ce8438daf2474cc836273 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:01:34.926 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:34 smithi144 podman[95672]: 2024-05-15 16:01:34.924286082 +0000 UTC m=+0.402779148 container attach f1c07acfac49a4b94e0e882cb47d9daef447e00ab54ce8438daf2474cc836273 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-15T16:01:35.199 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:35 smithi144 podman[95672]: 2024-05-15 16:01:35.149672854 +0000 UTC m=+0.628165909 container died f1c07acfac49a4b94e0e882cb47d9daef447e00ab54ce8438daf2474cc836273 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-15T16:01:35.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:35 smithi144 ceph-mon[86635]: pgmap v116: 65 pgs: 16 stale+active+clean, 49 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:01:35.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:35 smithi144 ceph-mon[86635]: osdmap e67: 6 total, 5 up, 6 in 2024-05-15T16:01:35.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:35.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:01:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:35 smithi049 ceph-mon[112142]: pgmap v116: 65 pgs: 16 stale+active+clean, 49 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:01:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:35 smithi049 ceph-mon[112142]: osdmap e67: 6 total, 5 up, 6 in 2024-05-15T16:01:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:01:36.161 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:35 smithi144 podman[95698]: 2024-05-15 16:01:35.719638362 +0000 UTC m=+0.559572531 container remove f1c07acfac49a4b94e0e882cb47d9daef447e00ab54ce8438daf2474cc836273 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-65480-squid-de5210d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T16:01:36.161 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:35 smithi144 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.3.service: Deactivated successfully. 2024-05-15T16:01:36.161 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:35 smithi144 systemd[1]: Stopped Ceph osd.3 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:01:36.161 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:35 smithi144 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.3.service: Consumed 1min 6.071s CPU time. 2024-05-15T16:01:36.161 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:35 smithi144 systemd[1]: Starting Ceph osd.3 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:01:36.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:36 smithi144 ceph-mon[86635]: Health check failed: Degraded data redundancy: 35/321 objects degraded (10.903%), 8 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:36.449 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 podman[95799]: 2024-05-15 16:01:36.061803572 +0000 UTC m=+0.017131676 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:01:36.449 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 podman[95799]: 2024-05-15 16:01:36.198123641 +0000 UTC m=+0.153451731 container create f4bb95a7b5fd12e99964b17f9a6118c0054f80aa06936c9837230e8a86f815e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-05-15T16:01:36.588 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:36 smithi049 ceph-mon[112142]: Health check failed: Degraded data redundancy: 35/321 objects degraded (10.903%), 8 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:36.935 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 podman[95799]: 2024-05-15 16:01:36.448993164 +0000 UTC m=+0.404321250 container init f4bb95a7b5fd12e99964b17f9a6118c0054f80aa06936c9837230e8a86f815e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-15T16:01:36.935 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 podman[95799]: 2024-05-15 16:01:36.45283043 +0000 UTC m=+0.408158526 container start f4bb95a7b5fd12e99964b17f9a6118c0054f80aa06936c9837230e8a86f815e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS) 2024-05-15T16:01:36.936 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 podman[95799]: 2024-05-15 16:01:36.532108525 +0000 UTC m=+0.487436622 container attach f4bb95a7b5fd12e99964b17f9a6118c0054f80aa06936c9837230e8a86f815e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-15T16:01:37.199 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate[95814]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:01:37.199 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate[95814]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:01:37.199 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 bash[95799]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 bash[95799]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 bash[95799]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 bash[95799]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 bash[95799]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 bash[95799]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 bash[95799]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 bash[95799]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 bash[95799]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate[95814]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate[95814]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate[95814]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate[95814]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate[95814]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate[95814]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-15T16:01:37.200 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate[95814]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-15T16:01:37.201 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:36 smithi144 podman[95799]: 2024-05-15 16:01:36.95879018 +0000 UTC m=+0.914118272 container died f4bb95a7b5fd12e99964b17f9a6118c0054f80aa06936c9837230e8a86f815e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-15T16:01:37.529 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:37 smithi144 ceph-mon[86635]: pgmap v118: 65 pgs: 8 active+undersized+degraded, 6 stale+active+clean, 10 active+undersized, 41 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/321 objects degraded (10.903%) 2024-05-15T16:01:37.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:37 smithi049 ceph-mon[112142]: pgmap v118: 65 pgs: 8 active+undersized+degraded, 6 stale+active+clean, 10 active+undersized, 41 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/321 objects degraded (10.903%) 2024-05-15T16:01:37.780 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:37 smithi144 podman[95944]: 2024-05-15 16:01:37.527698673 +0000 UTC m=+0.563341915 container remove f4bb95a7b5fd12e99964b17f9a6118c0054f80aa06936c9837230e8a86f815e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-activate, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-05-15T16:01:37.780 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:37 smithi144 podman[96005]: 2024-05-15 16:01:37.649302827 +0000 UTC m=+0.017721706 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:01:37.780 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:37 smithi144 podman[96005]: 2024-05-15 16:01:37.779075158 +0000 UTC m=+0.147494020 container create 0a1e2d34dad4623c971846a84231a36cae7184032bc14457ccd267f554673873 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-05-15T16:01:38.048 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:38 smithi144 podman[96005]: 2024-05-15 16:01:38.023349454 +0000 UTC m=+0.391768325 container init 0a1e2d34dad4623c971846a84231a36cae7184032bc14457ccd267f554673873 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3, RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-15T16:01:38.048 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:38 smithi144 podman[96005]: 2024-05-15 16:01:38.026513467 +0000 UTC m=+0.394932338 container start 0a1e2d34dad4623c971846a84231a36cae7184032bc14457ccd267f554673873 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-65480-squid-de5210d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-15T16:01:38.449 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:38 smithi144 ceph-osd[96024]: -- 172.21.15.144:0/1514826581 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b88300f680 con 0x55b882fd8c00 2024-05-15T16:01:38.450 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:38 smithi144 bash[96005]: 0a1e2d34dad4623c971846a84231a36cae7184032bc14457ccd267f554673873 2024-05-15T16:01:38.450 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:38 smithi144 systemd[1]: Started Ceph osd.3 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:01:38.949 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:38 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[96020]: 2024-05-15T16:01:38.569+0000 7f670d3b57c0 -1 Falling back to public interface 2024-05-15T16:01:39.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:39 smithi144 ceph-mon[86635]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-05-15T16:01:39.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:39.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:39.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:39 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:01:39.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:39 smithi049 ceph-mon[112142]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-05-15T16:01:39.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:39.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:39.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:39 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:01:41.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:41 smithi144 ceph-mon[86635]: pgmap v120: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-05-15T16:01:41.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:41 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:41.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:41 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:41.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:41 smithi049 ceph-mon[112142]: pgmap v120: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-05-15T16:01:41.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:41 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:41.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:41 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:42.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:42 smithi049 ceph-mon[112142]: Health check update: Degraded data redundancy: 67/321 objects degraded (20.872%), 19 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:42 smithi144 ceph-mon[86635]: Health check update: Degraded data redundancy: 67/321 objects degraded (20.872%), 19 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:43.331 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:43 smithi144 ceph-mon[86635]: pgmap v121: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-05-15T16:01:43.332 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:43.332 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:43.631 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:43 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[96020]: 2024-05-15T16:01:43.499+0000 7f670d3b57c0 -1 osd.3 65 log_to_monitors true 2024-05-15T16:01:43.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:43 smithi049 ceph-mon[112142]: pgmap v121: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-05-15T16:01:43.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:43.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:44.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:44 smithi144 ceph-mon[86635]: from='osd.3 [v2:172.21.15.144:6800/4258967767,v1:172.21.15.144:6801/4258967767]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T16:01:44.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:44 smithi144 ceph-mon[86635]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T16:01:44.700 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:01:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[96020]: 2024-05-15T16:01:44.318+0000 7f66fe80c700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T16:01:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:44 smithi049 ceph-mon[112142]: from='osd.3 [v2:172.21.15.144:6800/4258967767,v1:172.21.15.144:6801/4258967767]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T16:01:44.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:44 smithi049 ceph-mon[112142]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T16:01:45.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:45 smithi144 ceph-mon[86635]: pgmap v122: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-05-15T16:01:45.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:45 smithi144 ceph-mon[86635]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-15T16:01:45.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:45 smithi144 ceph-mon[86635]: from='osd.3 [v2:172.21.15.144:6800/4258967767,v1:172.21.15.144:6801/4258967767]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:01:45.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:45 smithi144 ceph-mon[86635]: osdmap e68: 6 total, 5 up, 6 in 2024-05-15T16:01:45.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:45 smithi144 ceph-mon[86635]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:01:45.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:45 smithi049 ceph-mon[112142]: pgmap v122: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/321 objects degraded (20.872%) 2024-05-15T16:01:45.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:45 smithi049 ceph-mon[112142]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-15T16:01:45.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:45 smithi049 ceph-mon[112142]: from='osd.3 [v2:172.21.15.144:6800/4258967767,v1:172.21.15.144:6801/4258967767]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:01:45.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:45 smithi049 ceph-mon[112142]: osdmap e68: 6 total, 5 up, 6 in 2024-05-15T16:01:45.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:45 smithi049 ceph-mon[112142]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:01:46.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:01:46.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: osd.3 [v2:172.21.15.144:6800/4258967767,v1:172.21.15.144:6801/4258967767] boot 2024-05-15T16:01:46.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: osdmap e69: 6 total, 6 up, 6 in 2024-05-15T16:01:46.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-15T16:01:46.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:46 smithi144 ceph-mon[86635]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-15T16:01:46.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:01:46.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: osd.3 [v2:172.21.15.144:6800/4258967767,v1:172.21.15.144:6801/4258967767] boot 2024-05-15T16:01:46.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: osdmap e69: 6 total, 6 up, 6 in 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:46.706 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-15T16:01:46.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-15T16:01:46.707 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:46 smithi049 ceph-mon[112142]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-15T16:01:47.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:47 smithi144 ceph-mon[86635]: pgmap v125: 65 pgs: 8 peering, 17 active+undersized+degraded, 13 active+undersized, 27 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 58/321 objects degraded (18.069%) 2024-05-15T16:01:47.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:47 smithi144 ceph-mon[86635]: osdmap e70: 6 total, 6 up, 6 in 2024-05-15T16:01:47.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:47 smithi144 ceph-mon[86635]: Health check update: Degraded data redundancy: 58/321 objects degraded (18.069%), 17 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:47.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:47 smithi049 ceph-mon[112142]: pgmap v125: 65 pgs: 8 peering, 17 active+undersized+degraded, 13 active+undersized, 27 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 58/321 objects degraded (18.069%) 2024-05-15T16:01:47.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:47 smithi049 ceph-mon[112142]: osdmap e70: 6 total, 6 up, 6 in 2024-05-15T16:01:47.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:47 smithi049 ceph-mon[112142]: Health check update: Degraded data redundancy: 58/321 objects degraded (18.069%), 17 pgs degraded (PG_DEGRADED) 2024-05-15T16:01:49.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:49 smithi144 ceph-mon[86635]: pgmap v127: 65 pgs: 12 peering, 10 active+undersized+degraded, 6 active+undersized, 37 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 35/321 objects degraded (10.903%) 2024-05-15T16:01:49.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:49 smithi049 ceph-mon[112142]: pgmap v127: 65 pgs: 12 peering, 10 active+undersized+degraded, 6 active+undersized, 37 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 35/321 objects degraded (10.903%) 2024-05-15T16:01:51.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:51 smithi144 ceph-mon[86635]: pgmap v128: 65 pgs: 12 peering, 53 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:01:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:01:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:51 smithi144 ceph-mon[86635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/321 objects degraded (10.903%), 10 pgs degraded) 2024-05-15T16:01:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:51 smithi049 ceph-mon[112142]: pgmap v128: 65 pgs: 12 peering, 53 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:01:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:01:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:01:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:51 smithi049 ceph-mon[112142]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/321 objects degraded (10.903%), 10 pgs degraded) 2024-05-15T16:01:53.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:53 smithi144 ceph-mon[86635]: pgmap v129: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-15T16:01:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:53 smithi049 ceph-mon[112142]: pgmap v129: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-15T16:01:53.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.647+0000 7f7f39705700 1 -- 172.21.15.49:0/1530287436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f341007a0 msgr2=0x7f7f341047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:53.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.647+0000 7f7f39705700 1 --2- 172.21.15.49:0/1530287436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f341007a0 0x7f7f341047f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009910 tx=0x7f7f28009c20 comp rx=0 tx=0).stop 2024-05-15T16:01:53.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.647+0000 7f7f39705700 1 -- 172.21.15.49:0/1530287436 shutdown_connections 2024-05-15T16:01:53.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.647+0000 7f7f39705700 1 --2- 172.21.15.49:0/1530287436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f341007a0 0x7f7f341047f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:53.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.647+0000 7f7f39705700 1 --2- 172.21.15.49:0/1530287436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f340ffdf0 0x7f7f341001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.647+0000 7f7f39705700 1 -- 172.21.15.49:0/1530287436 >> 172.21.15.49:0/1530287436 conn(0x7f7f340fb6a0 msgr2=0x7f7f340fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.647+0000 7f7f39705700 1 -- 172.21.15.49:0/1530287436 shutdown_connections 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.647+0000 7f7f39705700 1 -- 172.21.15.49:0/1530287436 wait complete. 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.647+0000 7f7f39705700 1 Processor -- start 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f39705700 1 -- start start 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f39705700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f340ffdf0 0x7f7f3419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f39705700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f341007a0 0x7f7f3419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f39705700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3419c470 con 0x7f7f340ffdf0 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f39705700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f34195a50 con 0x7f7f341007a0 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f340ffdf0 0x7f7f3419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:53.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f340ffdf0 0x7f7f3419b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46330/0 (socket says 172.21.15.49:46330) 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f32ffd700 1 -- 172.21.15.49:0/270518515 learned_addr learned my addr 172.21.15.49:0/270518515 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f327fc700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f341007a0 0x7f7f3419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f32ffd700 1 -- 172.21.15.49:0/270518515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f341007a0 msgr2=0x7f7f3419be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f32ffd700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f341007a0 0x7f7f3419be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f32ffd700 1 -- 172.21.15.49:0/270518515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f280095f0 con 0x7f7f340ffdf0 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.648+0000 7f7f327fc700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f341007a0 0x7f7f3419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.649+0000 7f7f32ffd700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f340ffdf0 0x7f7f3419b910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400b4b0 tx=0x7f7f2400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.649+0000 7f7f1bfff700 1 -- 172.21.15.49:0/270518515 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f24011800 con 0x7f7f340ffdf0 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.649+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f34195d30 con 0x7f7f340ffdf0 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.649+0000 7f7f1bfff700 1 -- 172.21.15.49:0/270518515 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f24011e20 con 0x7f7f340ffdf0 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.649+0000 7f7f1bfff700 1 -- 172.21.15.49:0/270518515 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f2400f2e0 con 0x7f7f340ffdf0 2024-05-15T16:01:53.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.649+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f34196280 con 0x7f7f340ffdf0 2024-05-15T16:01:53.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.650+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f34108120 con 0x7f7f340ffdf0 2024-05-15T16:01:53.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.651+0000 7f7f1bfff700 1 -- 172.21.15.49:0/270518515 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7f24010380 con 0x7f7f340ffdf0 2024-05-15T16:01:53.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.651+0000 7f7f1bfff700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f1c06e200 0x7f7f1c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:53.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.651+0000 7f7f1bfff700 1 -- 172.21.15.49:0/270518515 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7f2408dca0 con 0x7f7f340ffdf0 2024-05-15T16:01:53.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.651+0000 7f7f327fc700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f1c06e200 0x7f7f1c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:53.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.652+0000 7f7f327fc700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f1c06e200 0x7f7f1c0706c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f28005f50 tx=0x7f7f28005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:53.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.656+0000 7f7f1bfff700 1 -- 172.21.15.49:0/270518515 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7f24092070 con 0x7f7f340ffdf0 2024-05-15T16:01:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.883+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f3410ce10 con 0x7f7f1c06e200 2024-05-15T16:01:53.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.885+0000 7f7f1bfff700 1 -- 172.21.15.49:0/270518515 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7f3410ce10 con 0x7f7f1c06e200 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f1c06e200 msgr2=0x7f7f1c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f1c06e200 0x7f7f1c0706c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f28005f50 tx=0x7f7f28005dc0 comp rx=0 tx=0).stop 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f340ffdf0 msgr2=0x7f7f3419b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f340ffdf0 0x7f7f3419b910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400b4b0 tx=0x7f7f2400b870 comp rx=0 tx=0).stop 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 shutdown_connections 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f1c06e200 0x7f7f1c0706c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f340ffdf0 0x7f7f3419b910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 --2- 172.21.15.49:0/270518515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f341007a0 0x7f7f3419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 >> 172.21.15.49:0/270518515 conn(0x7f7f340fb6a0 msgr2=0x7f7f340fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 shutdown_connections 2024-05-15T16:01:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.887+0000 7f7f39705700 1 -- 172.21.15.49:0/270518515 wait complete. 2024-05-15T16:01:53.900 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:01:54.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.998+0000 7f3821e54700 1 -- 172.21.15.49:0/2263341253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c100f00 msgr2=0x7f381c1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.998+0000 7f3821e54700 1 --2- 172.21.15.49:0/2263341253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c100f00 0x7f381c1012e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f380c009910 tx=0x7f380c009c20 comp rx=0 tx=0).stop 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.998+0000 7f3821e54700 1 -- 172.21.15.49:0/2263341253 shutdown_connections 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.998+0000 7f3821e54700 1 --2- 172.21.15.49:0/2263341253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c1018b0 0x7f381c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.998+0000 7f3821e54700 1 --2- 172.21.15.49:0/2263341253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c100f00 0x7f381c1012e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.998+0000 7f3821e54700 1 -- 172.21.15.49:0/2263341253 >> 172.21.15.49:0/2263341253 conn(0x7f381c0fc7d0 msgr2=0x7f381c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.998+0000 7f3821e54700 1 -- 172.21.15.49:0/2263341253 shutdown_connections 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.998+0000 7f3821e54700 1 -- 172.21.15.49:0/2263341253 wait complete. 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f3821e54700 1 Processor -- start 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f3821e54700 1 -- start start 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f3821e54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c100f00 0x7f381c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f3821e54700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c1018b0 0x7f381c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:54.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f3821e54700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c19d460 con 0x7f381c1018b0 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f3821e54700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c196a90 con 0x7f381c100f00 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f381affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c1018b0 0x7f381c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f381affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c1018b0 0x7f381c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46346/0 (socket says 172.21.15.49:46346) 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f381affd700 1 -- 172.21.15.49:0/2430738259 learned_addr learned my addr 172.21.15.49:0/2430738259 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f381affd700 1 -- 172.21.15.49:0/2430738259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c100f00 msgr2=0x7f381c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f381affd700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c100f00 0x7f381c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:53.999+0000 7f381affd700 1 -- 172.21.15.49:0/2430738259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c0095f0 con 0x7f381c1018b0 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.000+0000 7f381affd700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c1018b0 0x7f381c19ce40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f381000d900 tx=0x7f381000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.000+0000 7f3818ff9700 1 -- 172.21.15.49:0/2430738259 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3810009900 con 0x7f381c1018b0 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.000+0000 7f3818ff9700 1 -- 172.21.15.49:0/2430738259 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3810010460 con 0x7f381c1018b0 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.000+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c196d70 con 0x7f381c1018b0 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.000+0000 7f3818ff9700 1 -- 172.21.15.49:0/2430738259 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381000f5f0 con 0x7f381c1018b0 2024-05-15T16:01:54.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.000+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c1972c0 con 0x7f381c1018b0 2024-05-15T16:01:54.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.001+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c1092a0 con 0x7f381c1018b0 2024-05-15T16:01:54.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.001+0000 7f3818ff9700 1 -- 172.21.15.49:0/2430738259 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f38100105d0 con 0x7f381c1018b0 2024-05-15T16:01:54.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.002+0000 7f3818ff9700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f380406e200 0x7f38040706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:54.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.002+0000 7f3818ff9700 1 -- 172.21.15.49:0/2430738259 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f381008dc40 con 0x7f381c1018b0 2024-05-15T16:01:54.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.002+0000 7f381b7fe700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f380406e200 0x7f38040706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:54.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.002+0000 7f381b7fe700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f380406e200 0x7f38040706c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b5c0 tx=0x7f380c005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:54.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.006+0000 7f3818ff9700 1 -- 172.21.15.49:0/2430738259 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f381005ab80 con 0x7f381c1018b0 2024-05-15T16:01:54.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.236+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f381c197f70 con 0x7f380406e200 2024-05-15T16:01:54.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.237+0000 7f3818ff9700 1 -- 172.21.15.49:0/2430738259 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f381c197f70 con 0x7f380406e200 2024-05-15T16:01:54.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f380406e200 msgr2=0x7f38040706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f380406e200 0x7f38040706c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b5c0 tx=0x7f380c005e20 comp rx=0 tx=0).stop 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c1018b0 msgr2=0x7f381c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c1018b0 0x7f381c19ce40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f381000d900 tx=0x7f381000dcc0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 shutdown_connections 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f380406e200 0x7f38040706c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c100f00 0x7f381c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 --2- 172.21.15.49:0/2430738259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381c1018b0 0x7f381c19ce40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 >> 172.21.15.49:0/2430738259 conn(0x7f381c0fc7d0 msgr2=0x7f381c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 shutdown_connections 2024-05-15T16:01:54.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.239+0000 7f3821e54700 1 -- 172.21.15.49:0/2430738259 wait complete. 2024-05-15T16:01:54.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.350+0000 7f8929698700 1 -- 172.21.15.49:0/3989547065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 msgr2=0x7f89241058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.350+0000 7f8929698700 1 --2- 172.21.15.49:0/3989547065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 0x7f89241058f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8918009910 tx=0x7f8918009c20 comp rx=0 tx=0).stop 2024-05-15T16:01:54.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.350+0000 7f8929698700 1 -- 172.21.15.49:0/3989547065 shutdown_connections 2024-05-15T16:01:54.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.350+0000 7f8929698700 1 --2- 172.21.15.49:0/3989547065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 0x7f89241058f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.350+0000 7f8929698700 1 --2- 172.21.15.49:0/3989547065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924100ef0 0x7f89241012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.350+0000 7f8929698700 1 -- 172.21.15.49:0/3989547065 >> 172.21.15.49:0/3989547065 conn(0x7f89240fc7c0 msgr2=0x7f89240febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:54.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.350+0000 7f8929698700 1 -- 172.21.15.49:0/3989547065 shutdown_connections 2024-05-15T16:01:54.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.350+0000 7f8929698700 1 -- 172.21.15.49:0/3989547065 wait complete. 2024-05-15T16:01:54.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f8929698700 1 Processor -- start 2024-05-15T16:01:54.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f8929698700 1 -- start start 2024-05-15T16:01:54.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f8929698700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924100ef0 0x7f89240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:54.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f8929698700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 0x7f89240737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:54.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f8929698700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8924073d20 con 0x7f89241018a0 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f8929698700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8924073e60 con 0x7f8924100ef0 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f89227fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 0x7f89240737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f89227fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 0x7f89240737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46364/0 (socket says 172.21.15.49:46364) 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f89227fc700 1 -- 172.21.15.49:0/2565723717 learned_addr learned my addr 172.21.15.49:0/2565723717 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.351+0000 7f8922ffd700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924100ef0 0x7f89240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.352+0000 7f89227fc700 1 -- 172.21.15.49:0/2565723717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924100ef0 msgr2=0x7f89240732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.352+0000 7f89227fc700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924100ef0 0x7f89240732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.352+0000 7f89227fc700 1 -- 172.21.15.49:0/2565723717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89180095f0 con 0x7f89241018a0 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.352+0000 7f89227fc700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 0x7f89240737e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8918009fd0 tx=0x7f891800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.352+0000 7f890bfff700 1 -- 172.21.15.49:0/2565723717 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f891801d070 con 0x7f89241018a0 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.352+0000 7f890bfff700 1 -- 172.21.15.49:0/2565723717 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89180052a0 con 0x7f89241018a0 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.352+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89240740e0 con 0x7f89241018a0 2024-05-15T16:01:54.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.352+0000 7f890bfff700 1 -- 172.21.15.49:0/2565723717 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8918022470 con 0x7f89241018a0 2024-05-15T16:01:54.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.352+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89241a3030 con 0x7f89241018a0 2024-05-15T16:01:54.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.353+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89241092f0 con 0x7f89241018a0 2024-05-15T16:01:54.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.354+0000 7f890bfff700 1 -- 172.21.15.49:0/2565723717 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f891800f6f0 con 0x7f89241018a0 2024-05-15T16:01:54.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.354+0000 7f890bfff700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f890c06e200 0x7f890c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:54.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.354+0000 7f890bfff700 1 -- 172.21.15.49:0/2565723717 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f891808f290 con 0x7f89241018a0 2024-05-15T16:01:54.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.354+0000 7f8922ffd700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f890c06e200 0x7f890c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:54.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.355+0000 7f8922ffd700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f890c06e200 0x7f890c0706c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8914009c60 tx=0x7f8914009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:54.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.359+0000 7f890bfff700 1 -- 172.21.15.49:0/2565723717 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f89180586a0 con 0x7f89241018a0 2024-05-15T16:01:54.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:54 smithi144 ceph-mon[86635]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:54.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:54 smithi049 ceph-mon[112142]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:54.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.574+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f892410de90 con 0x7f890c06e200 2024-05-15T16:01:54.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.585+0000 7f890bfff700 1 -- 172.21.15.49:0/2565723717 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f892410de90 con 0x7f890c06e200 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (5m) 46s ago 16m 24.9M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (16m) 46s ago 16m 8883k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (15m) 13s ago 15m 8334k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (2m) 46s ago 16m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (2m) 13s ago 15m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (4m) 46s ago 16m 79.1M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (11m) 46s ago 11m 90.1M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (11m) 46s ago 11m 152M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (11m) 13s ago 11m 26.7M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (11m) 13s ago 10m 28.5M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (7m) 46s ago 18m 507M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (6m) 13s ago 15m 453M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (3m) 46s ago 18m 47.3M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:01:54.587 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (3m) 13s ago 15m 39.1M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:01:54.588 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (5m) 46s ago 16m 15.8M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:01:54.588 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (5m) 13s ago 15m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:01:54.588 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (2m) 46s ago 14m 82.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:01:54.588 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (88s) 46s ago 14m 84.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:01:54.588 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (52s) 46s ago 13m 14.8M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:01:54.588 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (16s) 13s ago 13m 15.0M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:01:54.588 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (13m) 13s ago 13m 220M 2240M 18.2.2-1343-g2965081f a04a73ffe079 bca268a6580d 2024-05-15T16:01:54.588 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (12m) 13s ago 12m 202M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T16:01:54.588 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (5m) 46s ago 16m 68.7M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.587+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f890c06e200 msgr2=0x7f890c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.587+0000 7f8929698700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f890c06e200 0x7f890c0706c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8914009c60 tx=0x7f8914009260 comp rx=0 tx=0).stop 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.587+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 msgr2=0x7f89240737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.587+0000 7f8929698700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 0x7f89240737e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8918009fd0 tx=0x7f891800b5a0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.587+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 shutdown_connections 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.587+0000 7f8929698700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f890c06e200 0x7f890c0706c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.587+0000 7f8929698700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924100ef0 0x7f89240732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.587+0000 7f8929698700 1 --2- 172.21.15.49:0/2565723717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89241018a0 0x7f89240737e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.587+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 >> 172.21.15.49:0/2565723717 conn(0x7f89240fc7c0 msgr2=0x7f89240fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.588+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 shutdown_connections 2024-05-15T16:01:54.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.588+0000 7f8929698700 1 -- 172.21.15.49:0/2565723717 wait complete. 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 -- 172.21.15.49:0/1159408050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 msgr2=0x7fc2a41020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 --2- 172.21.15.49:0/1159408050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 0x7fc2a41020d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc298009940 tx=0x7fc298009c50 comp rx=0 tx=0).stop 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 -- 172.21.15.49:0/1159408050 shutdown_connections 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 --2- 172.21.15.49:0/1159408050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a4102610 0x7fc2a410ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 --2- 172.21.15.49:0/1159408050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 0x7fc2a41020d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 -- 172.21.15.49:0/1159408050 >> 172.21.15.49:0/1159408050 conn(0x7fc2a40fb500 msgr2=0x7fc2a40fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 -- 172.21.15.49:0/1159408050 shutdown_connections 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 -- 172.21.15.49:0/1159408050 wait complete. 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 Processor -- start 2024-05-15T16:01:54.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.696+0000 7fc2a999d700 1 -- start start 2024-05-15T16:01:54.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a999d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 0x7fc2a40ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:54.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a999d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a4102610 0x7fc2a40ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:54.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a999d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4101920 con 0x7fc2a4101cf0 2024-05-15T16:01:54.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a999d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a41003d0 con 0x7fc2a4102610 2024-05-15T16:01:54.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a899b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 0x7fc2a40ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:54.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a899b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 0x7fc2a40ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46378/0 (socket says 172.21.15.49:46378) 2024-05-15T16:01:54.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a899b700 1 -- 172.21.15.49:0/3478304982 learned_addr learned my addr 172.21.15.49:0/3478304982 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:54.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a3fff700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a4102610 0x7fc2a40ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:54.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a899b700 1 -- 172.21.15.49:0/3478304982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a4102610 msgr2=0x7fc2a40ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:54.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a899b700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a4102610 0x7fc2a40ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:54.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a899b700 1 -- 172.21.15.49:0/3478304982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2980095f0 con 0x7fc2a4101cf0 2024-05-15T16:01:54.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a3fff700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a4102610 0x7fc2a40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:01:54.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.697+0000 7fc2a899b700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 0x7fc2a40ff950 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc29800b580 tx=0x7fc2980056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:54.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.698+0000 7fc2a1ffb700 1 -- 172.21.15.49:0/3478304982 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29801d070 con 0x7fc2a4101cf0 2024-05-15T16:01:54.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.698+0000 7fc2a1ffb700 1 -- 172.21.15.49:0/3478304982 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc29800bed0 con 0x7fc2a4101cf0 2024-05-15T16:01:54.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.698+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a4100620 con 0x7fc2a4101cf0 2024-05-15T16:01:54.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.698+0000 7fc2a1ffb700 1 -- 172.21.15.49:0/3478304982 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc298021410 con 0x7fc2a4101cf0 2024-05-15T16:01:54.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.698+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a41a2d60 con 0x7fc2a4101cf0 2024-05-15T16:01:54.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.699+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a4069770 con 0x7fc2a4101cf0 2024-05-15T16:01:54.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.700+0000 7fc2a1ffb700 1 -- 172.21.15.49:0/3478304982 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc29800f460 con 0x7fc2a4101cf0 2024-05-15T16:01:54.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.700+0000 7fc2a1ffb700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc29406e200 0x7fc2940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:54.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.700+0000 7fc2a1ffb700 1 -- 172.21.15.49:0/3478304982 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc29808f400 con 0x7fc2a4101cf0 2024-05-15T16:01:54.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.700+0000 7fc2a3fff700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc29406e200 0x7fc2940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:54.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.701+0000 7fc2a3fff700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc29406e200 0x7fc2940706c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc290005fd0 tx=0x7fc290005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:54.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.705+0000 7fc2a1ffb700 1 -- 172.21.15.49:0/3478304982 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc298058790 con 0x7fc2a4101cf0 2024-05-15T16:01:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.997+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc2a4105380 con 0x7fc2a4101cf0 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.997+0000 7fc2a1ffb700 1 -- 172.21.15.49:0/3478304982 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7fc29805be50 con 0x7fc2a4101cf0 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2, 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:01:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:01:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:01:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T16:01:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:01:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:01:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 6, 2024-05-15T16:01:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 8 2024-05-15T16:01:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:01:55.000 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.999+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc29406e200 msgr2=0x7fc2940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.999+0000 7fc2a999d700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc29406e200 0x7fc2940706c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc290005fd0 tx=0x7fc290005e30 comp rx=0 tx=0).stop 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.999+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 msgr2=0x7fc2a40ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.999+0000 7fc2a999d700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 0x7fc2a40ff950 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc29800b580 tx=0x7fc2980056a0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.999+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 shutdown_connections 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.999+0000 7fc2a999d700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc29406e200 0x7fc2940706c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.999+0000 7fc2a999d700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a4101cf0 0x7fc2a40ff950 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.999+0000 7fc2a999d700 1 --2- 172.21.15.49:0/3478304982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a4102610 0x7fc2a40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:54.999+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 >> 172.21.15.49:0/3478304982 conn(0x7fc2a40fb500 msgr2=0x7fc2a4102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.000+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 shutdown_connections 2024-05-15T16:01:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.000+0000 7fc2a999d700 1 -- 172.21.15.49:0/3478304982 wait complete. 2024-05-15T16:01:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.108+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3357749581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 msgr2=0x7f69a41001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.108+0000 7f69ab1e9700 1 --2- 172.21.15.49:0/3357749581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 0x7f69a41001a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f698c009910 tx=0x7f698c009c20 comp rx=0 tx=0).stop 2024-05-15T16:01:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.108+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3357749581 shutdown_connections 2024-05-15T16:01:55.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.108+0000 7f69ab1e9700 1 --2- 172.21.15.49:0/3357749581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4100770 0x7f69a41047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.108+0000 7f69ab1e9700 1 --2- 172.21.15.49:0/3357749581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 0x7f69a41001a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.108+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3357749581 >> 172.21.15.49:0/3357749581 conn(0x7f69a4076800 msgr2=0x7f69a40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.108+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3357749581 shutdown_connections 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.108+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3357749581 wait complete. 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69ab1e9700 1 Processor -- start 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69ab1e9700 1 -- start start 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69ab1e9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 0x7f69a419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69ab1e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4100770 0x7f69a419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69ab1e9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a419d3b0 con 0x7f69a40ffdc0 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69ab1e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a4196a70 con 0x7f69a4100770 2024-05-15T16:01:55.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69a8f85700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 0x7f69a419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69a8f85700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 0x7f69a419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46392/0 (socket says 172.21.15.49:46392) 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69a8f85700 1 -- 172.21.15.49:0/3581278998 learned_addr learned my addr 172.21.15.49:0/3581278998 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.109+0000 7f69a3fff700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4100770 0x7f69a419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69a8f85700 1 -- 172.21.15.49:0/3581278998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4100770 msgr2=0x7f69a419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69a8f85700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4100770 0x7f69a419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69a8f85700 1 -- 172.21.15.49:0/3581278998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c0095f0 con 0x7f69a40ffdc0 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69a3fff700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4100770 0x7f69a419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69a8f85700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 0x7f69a419c8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f698c006010 tx=0x7f698c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69a1ffb700 1 -- 172.21.15.49:0/3581278998 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f698c01d070 con 0x7f69a40ffdc0 2024-05-15T16:01:55.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69a1ffb700 1 -- 172.21.15.49:0/3581278998 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f698c00f460 con 0x7f69a40ffdc0 2024-05-15T16:01:55.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a4196cf0 con 0x7f69a40ffdc0 2024-05-15T16:01:55.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69a1ffb700 1 -- 172.21.15.49:0/3581278998 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f698c017630 con 0x7f69a40ffdc0 2024-05-15T16:01:55.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.110+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a41971e0 con 0x7f69a40ffdc0 2024-05-15T16:01:55.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.111+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a41012a0 con 0x7f69a40ffdc0 2024-05-15T16:01:55.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.112+0000 7f69a1ffb700 1 -- 172.21.15.49:0/3581278998 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f698c00f5d0 con 0x7f69a40ffdc0 2024-05-15T16:01:55.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.112+0000 7f69a1ffb700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f699006e200 0x7f69900706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:55.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.112+0000 7f69a1ffb700 1 -- 172.21.15.49:0/3581278998 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f698c08f570 con 0x7f69a40ffdc0 2024-05-15T16:01:55.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.112+0000 7f69a3fff700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f699006e200 0x7f69900706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:55.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.113+0000 7f69a3fff700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f699006e200 0x7f69900706c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69a4197be0 tx=0x7f6994009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:55.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.117+0000 7f69a1ffb700 1 -- 172.21.15.49:0/3581278998 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f698c05c570 con 0x7f69a40ffdc0 2024-05-15T16:01:55.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.374+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f69a410cd80 con 0x7f69a40ffdc0 2024-05-15T16:01:55.376 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:55 smithi049 ceph-mon[112142]: pgmap v130: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-05-15T16:01:55.376 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:55 smithi049 ceph-mon[112142]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:55.377 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:55 smithi049 ceph-mon[112142]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:55.377 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:55 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3478304982' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:55.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.375+0000 7f69a1ffb700 1 -- 172.21.15.49:0/3581278998 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f698c05c100 con 0x7f69a40ffdc0 2024-05-15T16:01:55.377 INFO:teuthology.orchestra.run.smithi049.stdout:e17 2024-05-15T16:01:55.377 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:01:55.377 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:01:55.377 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 17 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:51:00.559034+0000 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:01:55.378 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:01:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:01:55.380 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:01:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f699006e200 msgr2=0x7f69900706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f699006e200 0x7f69900706c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69a4197be0 tx=0x7f6994009040 comp rx=0 tx=0).stop 2024-05-15T16:01:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 msgr2=0x7f69a419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 0x7f69a419c8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f698c006010 tx=0x7f698c00be20 comp rx=0 tx=0).stop 2024-05-15T16:01:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 shutdown_connections 2024-05-15T16:01:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f699006e200 0x7f69900706c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a40ffdc0 0x7f69a419c8e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 --2- 172.21.15.49:0/3581278998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4100770 0x7f69a419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 >> 172.21.15.49:0/3581278998 conn(0x7f69a4076800 msgr2=0x7f69a40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 shutdown_connections 2024-05-15T16:01:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.377+0000 7f69ab1e9700 1 -- 172.21.15.49:0/3581278998 wait complete. 2024-05-15T16:01:55.382 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T16:01:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:55 smithi144 ceph-mon[86635]: pgmap v130: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-05-15T16:01:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:55 smithi144 ceph-mon[86635]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:55 smithi144 ceph-mon[86635]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:55.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:55 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3478304982' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:01:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.487+0000 7f9c6614d700 1 -- 172.21.15.49:0/3319432858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60100ee0 msgr2=0x7f9c601012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.487+0000 7f9c6614d700 1 --2- 172.21.15.49:0/3319432858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60100ee0 0x7f9c601012c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000b0a0 tx=0x7f9c5000b3b0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.487+0000 7f9c6614d700 1 -- 172.21.15.49:0/3319432858 shutdown_connections 2024-05-15T16:01:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.487+0000 7f9c6614d700 1 --2- 172.21.15.49:0/3319432858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c60101890 0x7f9c601058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.487+0000 7f9c6614d700 1 --2- 172.21.15.49:0/3319432858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60100ee0 0x7f9c601012c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.487+0000 7f9c6614d700 1 -- 172.21.15.49:0/3319432858 >> 172.21.15.49:0/3319432858 conn(0x7f9c600fc790 msgr2=0x7f9c600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.487+0000 7f9c6614d700 1 -- 172.21.15.49:0/3319432858 shutdown_connections 2024-05-15T16:01:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.487+0000 7f9c6614d700 1 -- 172.21.15.49:0/3319432858 wait complete. 2024-05-15T16:01:55.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c6614d700 1 Processor -- start 2024-05-15T16:01:55.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c6614d700 1 -- start start 2024-05-15T16:01:55.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c6614d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c60100ee0 0x7f9c6019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:55.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c6614d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60101890 0x7f9c6019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c6614d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6019d3d0 con 0x7f9c60101890 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c6614d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c60196a90 con 0x7f9c60100ee0 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c5effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60101890 0x7f9c6019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c5effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60101890 0x7f9c6019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46402/0 (socket says 172.21.15.49:46402) 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c5effd700 1 -- 172.21.15.49:0/2971942697 learned_addr learned my addr 172.21.15.49:0/2971942697 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c5f7fe700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c60100ee0 0x7f9c6019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c5effd700 1 -- 172.21.15.49:0/2971942697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c60100ee0 msgr2=0x7f9c6019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c5effd700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c60100ee0 0x7f9c6019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.488+0000 7f9c5effd700 1 -- 172.21.15.49:0/2971942697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50009d00 con 0x7f9c60101890 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.489+0000 7f9c5effd700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60101890 0x7f9c6019ce40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c960 tx=0x7f9c4800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.489+0000 7f9c5cff9700 1 -- 172.21.15.49:0/2971942697 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c48007a70 con 0x7f9c60101890 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.489+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c60196d70 con 0x7f9c60101890 2024-05-15T16:01:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.489+0000 7f9c5cff9700 1 -- 172.21.15.49:0/2971942697 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c48007bd0 con 0x7f9c60101890 2024-05-15T16:01:55.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.489+0000 7f9c5cff9700 1 -- 172.21.15.49:0/2971942697 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c48018630 con 0x7f9c60101890 2024-05-15T16:01:55.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.489+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c601972c0 con 0x7f9c60101890 2024-05-15T16:01:55.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.490+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c60109280 con 0x7f9c60101890 2024-05-15T16:01:55.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.491+0000 7f9c5cff9700 1 -- 172.21.15.49:0/2971942697 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9c4800f450 con 0x7f9c60101890 2024-05-15T16:01:55.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.491+0000 7f9c5cff9700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c4c06e200 0x7f9c4c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:55.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.491+0000 7f9c5cff9700 1 -- 172.21.15.49:0/2971942697 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9c4808e050 con 0x7f9c60101890 2024-05-15T16:01:55.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.491+0000 7f9c5f7fe700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c4c06e200 0x7f9c4c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:55.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.492+0000 7f9c5f7fe700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c4c06e200 0x7f9c4c0706c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000f80 tx=0x7f9c500060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:55.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.496+0000 7f9c5cff9700 1 -- 172.21.15.49:0/2971942697 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9c4805af10 con 0x7f9c60101890 2024-05-15T16:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.723+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c60197f00 con 0x7f9c4c06e200 2024-05-15T16:01:55.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.724+0000 7f9c5cff9700 1 -- 172.21.15.49:0/2971942697 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9c60197f00 con 0x7f9c4c06e200 2024-05-15T16:01:55.725 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:01:55.725 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:01:55.725 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:01:55.725 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:01:55.726 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:01:55.726 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:01:55.726 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:01:55.726 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:01:55.726 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:01:55.726 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "10/23 daemons upgraded", 2024-05-15T16:01:55.726 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading osd daemons", 2024-05-15T16:01:55.726 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:01:55.726 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c4c06e200 msgr2=0x7f9c4c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c4c06e200 0x7f9c4c0706c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000f80 tx=0x7f9c500060c0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60101890 msgr2=0x7f9c6019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60101890 0x7f9c6019ce40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c960 tx=0x7f9c4800cc70 comp rx=0 tx=0).stop 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 shutdown_connections 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c4c06e200 0x7f9c4c0706c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c60100ee0 0x7f9c6019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 --2- 172.21.15.49:0/2971942697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c60101890 0x7f9c6019ce40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 >> 172.21.15.49:0/2971942697 conn(0x7f9c600fc790 msgr2=0x7f9c600fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 shutdown_connections 2024-05-15T16:01:55.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.726+0000 7f9c6614d700 1 -- 172.21.15.49:0/2971942697 wait complete. 2024-05-15T16:01:55.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.835+0000 7f36bebad700 1 -- 172.21.15.49:0/1832462490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8102800 msgr2=0x7f36b810acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.835+0000 7f36bebad700 1 --2- 172.21.15.49:0/1832462490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8102800 0x7f36b810acf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009910 tx=0x7f36ac009c20 comp rx=0 tx=0).stop 2024-05-15T16:01:55.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.835+0000 7f36bebad700 1 -- 172.21.15.49:0/1832462490 shutdown_connections 2024-05-15T16:01:55.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.835+0000 7f36bebad700 1 --2- 172.21.15.49:0/1832462490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8102800 0x7f36b810acf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.835+0000 7f36bebad700 1 --2- 172.21.15.49:0/1832462490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b8101ee0 0x7f36b81022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.835+0000 7f36bebad700 1 -- 172.21.15.49:0/1832462490 >> 172.21.15.49:0/1832462490 conn(0x7f36b8076800 msgr2=0x7f36b80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:55.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.835+0000 7f36bebad700 1 -- 172.21.15.49:0/1832462490 shutdown_connections 2024-05-15T16:01:55.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.835+0000 7f36bebad700 1 -- 172.21.15.49:0/1832462490 wait complete. 2024-05-15T16:01:55.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bebad700 1 Processor -- start 2024-05-15T16:01:55.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bebad700 1 -- start start 2024-05-15T16:01:55.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bebad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8101ee0 0x7f36b80ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bebad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b8102800 0x7f36b8100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bebad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81005c0 con 0x7f36b8101ee0 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bebad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8100730 con 0x7f36b8102800 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bc949700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8101ee0 0x7f36b80ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bc949700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8101ee0 0x7f36b80ffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46418/0 (socket says 172.21.15.49:46418) 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bc949700 1 -- 172.21.15.49:0/1263339596 learned_addr learned my addr 172.21.15.49:0/1263339596 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36b7fff700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b8102800 0x7f36b8100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bc949700 1 -- 172.21.15.49:0/1263339596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b8102800 msgr2=0x7f36b8100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bc949700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b8102800 0x7f36b8100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.836+0000 7f36bc949700 1 -- 172.21.15.49:0/1263339596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0095f0 con 0x7f36b8101ee0 2024-05-15T16:01:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.837+0000 7f36bc949700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8101ee0 0x7f36b80ffb40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36a800d900 tx=0x7f36a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:55.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.837+0000 7f36b5ffb700 1 -- 172.21.15.49:0/1263339596 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a80041f0 con 0x7f36b8101ee0 2024-05-15T16:01:55.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.837+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b81a2e30 con 0x7f36b8101ee0 2024-05-15T16:01:55.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.837+0000 7f36b5ffb700 1 -- 172.21.15.49:0/1263339596 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36a8004cf0 con 0x7f36b8101ee0 2024-05-15T16:01:55.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.837+0000 7f36b5ffb700 1 -- 172.21.15.49:0/1263339596 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a800b750 con 0x7f36b8101ee0 2024-05-15T16:01:55.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.837+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b81a31e0 con 0x7f36b8101ee0 2024-05-15T16:01:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.838+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b8068ea0 con 0x7f36b8101ee0 2024-05-15T16:01:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.839+0000 7f36b5ffb700 1 -- 172.21.15.49:0/1263339596 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f36a8004750 con 0x7f36b8101ee0 2024-05-15T16:01:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.839+0000 7f36b5ffb700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f36a006e210 0x7f36a00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:01:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.839+0000 7f36b5ffb700 1 -- 172.21.15.49:0/1263339596 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f36a808dfc0 con 0x7f36b8101ee0 2024-05-15T16:01:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.839+0000 7f36b7fff700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f36a006e210 0x7f36a00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:01:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.840+0000 7f36b7fff700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f36a006e210 0x7f36a00706d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36b81015f0 tx=0x7f36ac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:01:55.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:55.844+0000 7f36b5ffb700 1 -- 172.21.15.49:0/1263339596 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f36a8057350 con 0x7f36b8101ee0 2024-05-15T16:01:56.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.139+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f36b81009b0 con 0x7f36b8101ee0 2024-05-15T16:01:56.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.139+0000 7f36b5ffb700 1 -- 172.21.15.49:0/1263339596 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f36b81009b0 con 0x7f36b8101ee0 2024-05-15T16:01:56.141 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T16:01:56.141 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:01:56.141 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:01:56.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f36a006e210 msgr2=0x7f36a00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:56.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f36a006e210 0x7f36a00706d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36b81015f0 tx=0x7f36ac01a040 comp rx=0 tx=0).stop 2024-05-15T16:01:56.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8101ee0 msgr2=0x7f36b80ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:01:56.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8101ee0 0x7f36b80ffb40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36a800d900 tx=0x7f36a800dc10 comp rx=0 tx=0).stop 2024-05-15T16:01:56.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 shutdown_connections 2024-05-15T16:01:56.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f36a006e210 0x7f36a00706d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:56.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36b8101ee0 0x7f36b80ffb40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:56.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 --2- 172.21.15.49:0/1263339596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b8102800 0x7f36b8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:01:56.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 >> 172.21.15.49:0/1263339596 conn(0x7f36b8076800 msgr2=0x7f36b81031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:01:56.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 shutdown_connections 2024-05-15T16:01:56.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:01:56.142+0000 7f36bebad700 1 -- 172.21.15.49:0/1263339596 wait complete. 2024-05-15T16:01:56.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:56 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3581278998' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:01:56.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:56 smithi144 ceph-mon[86635]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:56.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:56 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1263339596' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:01:56.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:56 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3581278998' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:01:56.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:56 smithi049 ceph-mon[112142]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:01:56.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:56 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1263339596' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:01:57.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:57 smithi144 ceph-mon[86635]: pgmap v131: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-15T16:01:57.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:57 smithi049 ceph-mon[112142]: pgmap v131: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-15T16:01:59.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:01:59 smithi144 ceph-mon[86635]: pgmap v132: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-15T16:01:59.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:01:59 smithi049 ceph-mon[112142]: pgmap v132: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-15T16:02:01.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:01 smithi049 ceph-mon[112142]: pgmap v133: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-05-15T16:02:01.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:01 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:01.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:01 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-15T16:02:01.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:01 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-15T16:02:01.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:01 smithi049 ceph-mon[112142]: Upgrade: osd.4 is safe to restart 2024-05-15T16:02:01.536 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:01 smithi144 ceph-mon[86635]: pgmap v133: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-05-15T16:02:01.536 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:01 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:01.536 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:01 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-15T16:02:01.536 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:01 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-15T16:02:01.536 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:01 smithi144 ceph-mon[86635]: Upgrade: osd.4 is safe to restart 2024-05-15T16:02:03.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:03 smithi144 ceph-mon[86635]: pgmap v134: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:02:03.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:03 smithi144 ceph-mon[86635]: Upgrade: Updating osd.4 2024-05-15T16:02:03.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:03.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-15T16:02:03.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:03 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:03.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:03 smithi144 ceph-mon[86635]: Deploying daemon osd.4 on smithi144 2024-05-15T16:02:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:03 smithi049 ceph-mon[112142]: pgmap v134: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:02:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:03 smithi049 ceph-mon[112142]: Upgrade: Updating osd.4 2024-05-15T16:02:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-15T16:02:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:03 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:03 smithi049 ceph-mon[112142]: Deploying daemon osd.4 on smithi144 2024-05-15T16:02:05.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:05 smithi144 ceph-mon[86635]: pgmap v135: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:02:05.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:05.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:02:05.950 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:05 smithi144 systemd[1]: Stopping Ceph osd.4 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:02:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:05 smithi049 ceph-mon[112142]: pgmap v135: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:02:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:02:06.449 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:05 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[48698]: 2024-05-15T16:02:05.975+0000 7f725a983700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:02:06.449 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:05 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[48698]: 2024-05-15T16:02:05.975+0000 7f725a983700 -1 osd.4 70 *** Got signal Terminated *** 2024-05-15T16:02:06.449 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:05 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[48698]: 2024-05-15T16:02:05.975+0000 7f725a983700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:02:06.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:06 smithi144 ceph-mon[86635]: osd.4 marked itself down and dead 2024-05-15T16:02:06.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:06 smithi049 ceph-mon[112142]: osd.4 marked itself down and dead 2024-05-15T16:02:07.387 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:07 smithi144 podman[100606]: 2024-05-15 16:02:07.090861253 +0000 UTC m=+1.231307615 container died bca268a6580d8eec713859d6a027c5301ea774336d0af94739033a33a93aafa2 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=reef-2965081, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-15T16:02:07.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:07 smithi144 ceph-mon[86635]: pgmap v136: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:02:07.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:07 smithi144 ceph-mon[86635]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:02:07.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:07 smithi144 ceph-mon[86635]: osdmap e71: 6 total, 5 up, 6 in 2024-05-15T16:02:07.699 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:07 smithi144 podman[100606]: 2024-05-15 16:02:07.484955348 +0000 UTC m=+1.625401711 container cleanup bca268a6580d8eec713859d6a027c5301ea774336d0af94739033a33a93aafa2 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4, RELEASE=reef-2965081, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-15T16:02:07.700 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:07 smithi144 bash[100606]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4 2024-05-15T16:02:07.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:07 smithi049 ceph-mon[112142]: pgmap v136: 65 pgs: 65 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:02:07.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:07 smithi049 ceph-mon[112142]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:02:07.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:07 smithi049 ceph-mon[112142]: osdmap e71: 6 total, 5 up, 6 in 2024-05-15T16:02:08.067 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:07 smithi144 podman[100623]: 2024-05-15 16:02:07.744780209 +0000 UTC m=+0.647073693 container remove bca268a6580d8eec713859d6a027c5301ea774336d0af94739033a33a93aafa2 (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=reef-2965081, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-15T16:02:08.366 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:08 smithi144 podman[100709]: 2024-05-15 16:02:07.968278972 +0000 UTC m=+0.017328415 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:02:08.366 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:08 smithi144 podman[100709]: 2024-05-15 16:02:08.070744748 +0000 UTC m=+0.119794177 container create 3fb6050e78ff9d48fca932a9c2bc45ea849692bc14612a3f55b1e7944d2f2353 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True) 2024-05-15T16:02:08.366 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:08 smithi144 podman[100709]: 2024-05-15 16:02:08.277397944 +0000 UTC m=+0.326447380 container init 3fb6050e78ff9d48fca932a9c2bc45ea849692bc14612a3f55b1e7944d2f2353 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-15T16:02:08.366 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:08 smithi144 podman[100709]: 2024-05-15 16:02:08.280636673 +0000 UTC m=+0.329686115 container start 3fb6050e78ff9d48fca932a9c2bc45ea849692bc14612a3f55b1e7944d2f2353 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:02:08.366 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:08 smithi144 podman[100709]: 2024-05-15 16:02:08.364918523 +0000 UTC m=+0.413967960 container attach 3fb6050e78ff9d48fca932a9c2bc45ea849692bc14612a3f55b1e7944d2f2353 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:02:08.636 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:08 smithi144 podman[100709]: 2024-05-15 16:02:08.57578992 +0000 UTC m=+0.624839358 container died 3fb6050e78ff9d48fca932a9c2bc45ea849692bc14612a3f55b1e7944d2f2353 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:02:08.895 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:08 smithi144 ceph-mon[86635]: osdmap e72: 6 total, 5 up, 6 in 2024-05-15T16:02:08.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:08 smithi049 ceph-mon[112142]: osdmap e72: 6 total, 5 up, 6 in 2024-05-15T16:02:09.449 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 podman[100734]: 2024-05-15 16:02:09.160857241 +0000 UTC m=+0.579178143 container remove 3fb6050e78ff9d48fca932a9c2bc45ea849692bc14612a3f55b1e7944d2f2353 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, ceph=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-15T16:02:09.449 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-15T16:02:09.450 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 systemd[1]: Stopped Ceph osd.4 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:02:09.450 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.4.service: Consumed 52.951s CPU time. 2024-05-15T16:02:09.450 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 systemd[1]: Starting Ceph osd.4 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:02:09.923 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:09 smithi144 ceph-mon[86635]: pgmap v139: 65 pgs: 8 peering, 4 stale+active+clean, 2 active+undersized, 51 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-05-15T16:02:09.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:09 smithi144 ceph-mon[86635]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-15T16:02:09.924 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 podman[100833]: 2024-05-15 16:02:09.504638235 +0000 UTC m=+0.017429592 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:02:09.924 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 podman[100833]: 2024-05-15 16:02:09.620195918 +0000 UTC m=+0.132987293 container create 07985e69d8969e2984024abd9150d0b73d26f45f89774bdd18ff8f954c556e68 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-15T16:02:09.924 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 podman[100833]: 2024-05-15 16:02:09.839522336 +0000 UTC m=+0.352313689 container init 07985e69d8969e2984024abd9150d0b73d26f45f89774bdd18ff8f954c556e68 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, ceph=True) 2024-05-15T16:02:09.924 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 podman[100833]: 2024-05-15 16:02:09.843238772 +0000 UTC m=+0.356030129 container start 07985e69d8969e2984024abd9150d0b73d26f45f89774bdd18ff8f954c556e68 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2) 2024-05-15T16:02:09.924 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:09 smithi144 podman[100833]: 2024-05-15 16:02:09.922574098 +0000 UTC m=+0.435365460 container attach 07985e69d8969e2984024abd9150d0b73d26f45f89774bdd18ff8f954c556e68 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-15T16:02:09.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:09 smithi049 ceph-mon[112142]: pgmap v139: 65 pgs: 8 peering, 4 stale+active+clean, 2 active+undersized, 51 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-05-15T16:02:09.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:09 smithi049 ceph-mon[112142]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-15T16:02:10.686 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate[100848]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:02:10.686 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate[100848]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 bash[100833]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 bash[100833]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 bash[100833]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 bash[100833]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 bash[100833]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 bash[100833]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 bash[100833]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 bash[100833]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 bash[100833]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate[100848]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate[100848]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate[100848]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate[100848]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-15T16:02:10.687 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate[100848]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-15T16:02:10.688 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate[100848]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-15T16:02:10.688 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate[100848]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-15T16:02:10.688 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:10 smithi144 podman[100833]: 2024-05-15 16:02:10.347838644 +0000 UTC m=+0.860630004 container died 07985e69d8969e2984024abd9150d0b73d26f45f89774bdd18ff8f954c556e68 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2) 2024-05-15T16:02:10.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:10 smithi144 ceph-mon[86635]: Health check failed: Degraded data redundancy: 26/321 objects degraded (8.100%), 5 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:10.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:10 smithi049 ceph-mon[112142]: Health check failed: Degraded data redundancy: 26/321 objects degraded (8.100%), 5 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:11.382 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:11 smithi144 podman[100973]: 2024-05-15 16:02:11.043747863 +0000 UTC m=+0.690394108 container remove 07985e69d8969e2984024abd9150d0b73d26f45f89774bdd18ff8f954c556e68 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-15T16:02:11.382 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:11 smithi144 podman[101032]: 2024-05-15 16:02:11.162367587 +0000 UTC m=+0.017305065 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:02:11.382 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:11 smithi144 podman[101032]: 2024-05-15 16:02:11.2781489 +0000 UTC m=+0.133086399 container create d7e6dd64b07e0add4553557c489b4b3bad2c0aa4927131b5d70cfa7d86c0c26a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-15T16:02:11.693 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:11 smithi144 podman[101032]: 2024-05-15 16:02:11.455956046 +0000 UTC m=+0.310893533 container init d7e6dd64b07e0add4553557c489b4b3bad2c0aa4927131b5d70cfa7d86c0c26a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-65480-squid-de5210d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:02:11.693 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:11 smithi144 podman[101032]: 2024-05-15 16:02:11.459290691 +0000 UTC m=+0.314228171 container start d7e6dd64b07e0add4553557c489b4b3bad2c0aa4927131b5d70cfa7d86c0c26a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-15T16:02:11.693 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:11 smithi144 ceph-osd[101051]: -- 172.21.15.144:0/4096018540 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a7e6f9680 con 0x557a7e6c2c00 2024-05-15T16:02:11.693 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:11 smithi144 bash[101032]: d7e6dd64b07e0add4553557c489b4b3bad2c0aa4927131b5d70cfa7d86c0c26a 2024-05-15T16:02:11.693 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:11 smithi144 systemd[1]: Started Ceph osd.4 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:02:11.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:11 smithi144 ceph-mon[86635]: pgmap v140: 65 pgs: 5 active+undersized+degraded, 8 peering, 3 stale+active+clean, 8 active+undersized, 41 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 26/321 objects degraded (8.100%) 2024-05-15T16:02:11.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:11.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:11.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:11 smithi049 ceph-mon[112142]: pgmap v140: 65 pgs: 5 active+undersized+degraded, 8 peering, 3 stale+active+clean, 8 active+undersized, 41 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 26/321 objects degraded (8.100%) 2024-05-15T16:02:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:12.449 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:12 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:02:12.007+0000 7f5f3a52a7c0 -1 Falling back to public interface 2024-05-15T16:02:12.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:12 smithi144 ceph-mon[86635]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-15T16:02:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:12 smithi049 ceph-mon[112142]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-15T16:02:13.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:13 smithi144 ceph-mon[86635]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s; 53/321 objects degraded (16.511%) 2024-05-15T16:02:14.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:13 smithi049 ceph-mon[112142]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s; 53/321 objects degraded (16.511%) 2024-05-15T16:02:16.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:15 smithi144 ceph-mon[86635]: pgmap v142: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 2 op/s; 53/321 objects degraded (16.511%) 2024-05-15T16:02:16.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:16.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:16.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:15 smithi049 ceph-mon[112142]: pgmap v142: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 2 op/s; 53/321 objects degraded (16.511%) 2024-05-15T16:02:16.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:16.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:16.930 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:16 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:02:16.891+0000 7f5f3a52a7c0 -1 osd.4 70 log_to_monitors true 2024-05-15T16:02:17.199 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:02:17 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:02:17.068+0000 7f5f2b981700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T16:02:17.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:17.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:17.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:17 smithi144 ceph-mon[86635]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 53/321 objects degraded (16.511%) 2024-05-15T16:02:17.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:17 smithi144 ceph-mon[86635]: Health check update: Degraded data redundancy: 53/321 objects degraded (16.511%), 15 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:17.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:17 smithi144 ceph-mon[86635]: from='osd.4 [v2:172.21.15.144:6808/691212832,v1:172.21.15.144:6809/691212832]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T16:02:17.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:17 smithi144 ceph-mon[86635]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T16:02:17.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:17.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:17.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:17 smithi049 ceph-mon[112142]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 53/321 objects degraded (16.511%) 2024-05-15T16:02:17.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:17 smithi049 ceph-mon[112142]: Health check update: Degraded data redundancy: 53/321 objects degraded (16.511%), 15 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:17.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:17 smithi049 ceph-mon[112142]: from='osd.4 [v2:172.21.15.144:6808/691212832,v1:172.21.15.144:6809/691212832]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T16:02:17.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:17 smithi049 ceph-mon[112142]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T16:02:18.314 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:18 smithi144 ceph-mon[86635]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-15T16:02:18.314 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:18 smithi144 ceph-mon[86635]: osdmap e73: 6 total, 5 up, 6 in 2024-05-15T16:02:18.314 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:18 smithi144 ceph-mon[86635]: from='osd.4 [v2:172.21.15.144:6808/691212832,v1:172.21.15.144:6809/691212832]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:02:18.314 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:18 smithi144 ceph-mon[86635]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:02:18.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:18 smithi049 ceph-mon[112142]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-15T16:02:18.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:18 smithi049 ceph-mon[112142]: osdmap e73: 6 total, 5 up, 6 in 2024-05-15T16:02:18.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:18 smithi049 ceph-mon[112142]: from='osd.4 [v2:172.21.15.144:6808/691212832,v1:172.21.15.144:6809/691212832]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:02:18.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:18 smithi049 ceph-mon[112142]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:02:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:19 smithi144 ceph-mon[86635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:02:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:19 smithi144 ceph-mon[86635]: osd.4 [v2:172.21.15.144:6808/691212832,v1:172.21.15.144:6809/691212832] boot 2024-05-15T16:02:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:19 smithi144 ceph-mon[86635]: osdmap e74: 6 total, 6 up, 6 in 2024-05-15T16:02:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T16:02:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:19 smithi144 ceph-mon[86635]: pgmap v146: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s; 53/321 objects degraded (16.511%) 2024-05-15T16:02:19.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:19 smithi144 ceph-mon[86635]: osdmap e75: 6 total, 6 up, 6 in 2024-05-15T16:02:19.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:19 smithi049 ceph-mon[112142]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:02:19.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:19 smithi049 ceph-mon[112142]: osd.4 [v2:172.21.15.144:6808/691212832,v1:172.21.15.144:6809/691212832] boot 2024-05-15T16:02:19.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:19 smithi049 ceph-mon[112142]: osdmap e74: 6 total, 6 up, 6 in 2024-05-15T16:02:19.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T16:02:19.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:19 smithi049 ceph-mon[112142]: pgmap v146: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s; 53/321 objects degraded (16.511%) 2024-05-15T16:02:19.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:19 smithi049 ceph-mon[112142]: osdmap e75: 6 total, 6 up, 6 in 2024-05-15T16:02:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:02:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:21.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:21.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-15T16:02:21.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-15T16:02:21.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-15T16:02:21.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:21.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:02:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:02:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:21.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:21.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:21.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:21.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-15T16:02:21.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-15T16:02:21.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-15T16:02:21.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:21.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:02:22.150 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:21 smithi049 ceph-mon[112142]: pgmap v148: 65 pgs: 10 active+undersized+degraded, 14 active+undersized, 41 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 27/321 objects degraded (8.411%) 2024-05-15T16:02:22.150 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:21 smithi049 ceph-mon[112142]: Health check update: Degraded data redundancy: 27/321 objects degraded (8.411%), 10 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:22.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:21 smithi144 ceph-mon[86635]: pgmap v148: 65 pgs: 10 active+undersized+degraded, 14 active+undersized, 41 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 27/321 objects degraded (8.411%) 2024-05-15T16:02:22.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:21 smithi144 ceph-mon[86635]: Health check update: Degraded data redundancy: 27/321 objects degraded (8.411%), 10 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:23.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:22 smithi144 ceph-mon[86635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/321 objects degraded (8.411%), 10 pgs degraded) 2024-05-15T16:02:23.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:22 smithi049 ceph-mon[112142]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/321 objects degraded (8.411%), 10 pgs degraded) 2024-05-15T16:02:24.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:23 smithi144 ceph-mon[86635]: pgmap v149: 65 pgs: 1 active+undersized, 64 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 4 op/s 2024-05-15T16:02:24.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:23 smithi049 ceph-mon[112142]: pgmap v149: 65 pgs: 1 active+undersized, 64 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 4 op/s 2024-05-15T16:02:26.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:25 smithi144 ceph-mon[86635]: pgmap v150: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s 2024-05-15T16:02:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:25 smithi049 ceph-mon[112142]: pgmap v150: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s 2024-05-15T16:02:26.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.266+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3364854442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0101e20 msgr2=0x7f78b0102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.266+0000 7f78b5a4a700 1 --2- 172.21.15.49:0/3364854442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0101e20 0x7f78b0102200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009910 tx=0x7f78a0009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:26.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.266+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3364854442 shutdown_connections 2024-05-15T16:02:26.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.266+0000 7f78b5a4a700 1 --2- 172.21.15.49:0/3364854442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78b0102740 0x7f78b010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.266+0000 7f78b5a4a700 1 --2- 172.21.15.49:0/3364854442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0101e20 0x7f78b0102200 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.266+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3364854442 >> 172.21.15.49:0/3364854442 conn(0x7f78b0076800 msgr2=0x7f78b00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:26.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.266+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3364854442 shutdown_connections 2024-05-15T16:02:26.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.266+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3364854442 wait complete. 2024-05-15T16:02:26.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.267+0000 7f78b5a4a700 1 Processor -- start 2024-05-15T16:02:26.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.267+0000 7f78b5a4a700 1 -- start start 2024-05-15T16:02:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.267+0000 7f78b5a4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78b0101e20 0x7f78b019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.267+0000 7f78b5a4a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0102740 0x7f78b019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.267+0000 7f78b5a4a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b019d420 con 0x7f78b0102740 2024-05-15T16:02:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.267+0000 7f78b5a4a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0196a00 con 0x7f78b0101e20 2024-05-15T16:02:26.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78ae7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0102740 0x7f78b019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78ae7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0102740 0x7f78b019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33976/0 (socket says 172.21.15.49:33976) 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78ae7fc700 1 -- 172.21.15.49:0/3498327157 learned_addr learned my addr 172.21.15.49:0/3498327157 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78aeffd700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78b0101e20 0x7f78b019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78ae7fc700 1 -- 172.21.15.49:0/3498327157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78b0101e20 msgr2=0x7f78b019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78ae7fc700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78b0101e20 0x7f78b019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78ae7fc700 1 -- 172.21.15.49:0/3498327157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a00095f0 con 0x7f78b0102740 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78ae7fc700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0102740 0x7f78b019ce00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f78a400e940 tx=0x7f78a400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78b4a48700 1 -- 172.21.15.49:0/3498327157 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a40096e0 con 0x7f78b0102740 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b0196ce0 con 0x7f78b0102740 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78b4a48700 1 -- 172.21.15.49:0/3498327157 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78a400c7c0 con 0x7f78b0102740 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78b4a48700 1 -- 172.21.15.49:0/3498327157 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a4005580 con 0x7f78b0102740 2024-05-15T16:02:26.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.268+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b0197230 con 0x7f78b0102740 2024-05-15T16:02:26.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.269+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b0105540 con 0x7f78b0102740 2024-05-15T16:02:26.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.270+0000 7f78b4a48700 1 -- 172.21.15.49:0/3498327157 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f78a400c930 con 0x7f78b0102740 2024-05-15T16:02:26.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.271+0000 7f78b4a48700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f789806e200 0x7f78980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:26.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.271+0000 7f78aeffd700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f789806e200 0x7f78980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:26.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.271+0000 7f78b4a48700 1 -- 172.21.15.49:0/3498327157 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f78a4014070 con 0x7f78b0102740 2024-05-15T16:02:26.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.271+0000 7f78aeffd700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f789806e200 0x7f78980706c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78a00059d0 tx=0x7f78a0005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:26.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.276+0000 7f78b4a48700 1 -- 172.21.15.49:0/3498327157 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f78a405b170 con 0x7f78b0102740 2024-05-15T16:02:26.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.506+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78b0197f30 con 0x7f789806e200 2024-05-15T16:02:26.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.507+0000 7f78b4a48700 1 -- 172.21.15.49:0/3498327157 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f78b0197f30 con 0x7f789806e200 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.509+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f789806e200 msgr2=0x7f78980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.509+0000 7f78b5a4a700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f789806e200 0x7f78980706c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78a00059d0 tx=0x7f78a0005940 comp rx=0 tx=0).stop 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.509+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0102740 msgr2=0x7f78b019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.509+0000 7f78b5a4a700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0102740 0x7f78b019ce00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f78a400e940 tx=0x7f78a400ec50 comp rx=0 tx=0).stop 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.509+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 shutdown_connections 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.509+0000 7f78b5a4a700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f789806e200 0x7f78980706c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.509+0000 7f78b5a4a700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78b0101e20 0x7f78b019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.509+0000 7f78b5a4a700 1 --2- 172.21.15.49:0/3498327157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b0102740 0x7f78b019ce00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.509+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 >> 172.21.15.49:0/3498327157 conn(0x7f78b0076800 msgr2=0x7f78b0103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.510+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 shutdown_connections 2024-05-15T16:02:26.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.510+0000 7f78b5a4a700 1 -- 172.21.15.49:0/3498327157 wait complete. 2024-05-15T16:02:26.523 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:02:26.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.620+0000 7f46514fe700 1 -- 172.21.15.49:0/3218076460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101890 msgr2=0x7f464c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.620+0000 7f46514fe700 1 --2- 172.21.15.49:0/3218076460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101890 0x7f464c1058e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4640009910 tx=0x7f4640009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:26.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.620+0000 7f46514fe700 1 -- 172.21.15.49:0/3218076460 shutdown_connections 2024-05-15T16:02:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.620+0000 7f46514fe700 1 --2- 172.21.15.49:0/3218076460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101890 0x7f464c1058e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.620+0000 7f46514fe700 1 --2- 172.21.15.49:0/3218076460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464c100ee0 0x7f464c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.620+0000 7f46514fe700 1 -- 172.21.15.49:0/3218076460 >> 172.21.15.49:0/3218076460 conn(0x7f464c0fc790 msgr2=0x7f464c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.621+0000 7f46514fe700 1 -- 172.21.15.49:0/3218076460 shutdown_connections 2024-05-15T16:02:26.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.621+0000 7f46514fe700 1 -- 172.21.15.49:0/3218076460 wait complete. 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.621+0000 7f46514fe700 1 Processor -- start 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f46514fe700 1 -- start start 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f46514fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c100ee0 0x7f464c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f46514fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464c101890 0x7f464c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f46514fe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c073d20 con 0x7f464c100ee0 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f46514fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c073e60 con 0x7f464c101890 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f464affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c100ee0 0x7f464c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f464affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c100ee0 0x7f464c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33996/0 (socket says 172.21.15.49:33996) 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f464affd700 1 -- 172.21.15.49:0/4022065131 learned_addr learned my addr 172.21.15.49:0/4022065131 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f464a7fc700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464c101890 0x7f464c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f464affd700 1 -- 172.21.15.49:0/4022065131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464c101890 msgr2=0x7f464c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f464affd700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464c101890 0x7f464c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f464affd700 1 -- 172.21.15.49:0/4022065131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46400095f0 con 0x7f464c100ee0 2024-05-15T16:02:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.622+0000 7f464affd700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c100ee0 0x7f464c0732a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d8d0 tx=0x7f463c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.623+0000 7f4633fff700 1 -- 172.21.15.49:0/4022065131 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463c0098a0 con 0x7f464c100ee0 2024-05-15T16:02:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.623+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c1a2da0 con 0x7f464c100ee0 2024-05-15T16:02:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.623+0000 7f4633fff700 1 -- 172.21.15.49:0/4022065131 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f463c010460 con 0x7f464c100ee0 2024-05-15T16:02:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.623+0000 7f4633fff700 1 -- 172.21.15.49:0/4022065131 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463c00f5f0 con 0x7f464c100ee0 2024-05-15T16:02:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.623+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c1a3100 con 0x7f464c100ee0 2024-05-15T16:02:26.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.624+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c1092e0 con 0x7f464c100ee0 2024-05-15T16:02:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.624+0000 7f4633fff700 1 -- 172.21.15.49:0/4022065131 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f463c009a00 con 0x7f464c100ee0 2024-05-15T16:02:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.625+0000 7f4633fff700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f463406e200 0x7f46340706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.625+0000 7f4633fff700 1 -- 172.21.15.49:0/4022065131 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f463c08df80 con 0x7f464c100ee0 2024-05-15T16:02:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.625+0000 7f464a7fc700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f463406e200 0x7f46340706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.625+0000 7f464a7fc700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f463406e200 0x7f46340706c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f464c0749d0 tx=0x7f464001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:26.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.630+0000 7f4633fff700 1 -- 172.21.15.49:0/4022065131 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f463c057310 con 0x7f464c100ee0 2024-05-15T16:02:26.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.858+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f464c10de80 con 0x7f463406e200 2024-05-15T16:02:26.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.860+0000 7f4633fff700 1 -- 172.21.15.49:0/4022065131 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f464c10de80 con 0x7f463406e200 2024-05-15T16:02:26.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.862+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f463406e200 msgr2=0x7f46340706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.862+0000 7f46514fe700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f463406e200 0x7f46340706c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f464c0749d0 tx=0x7f464001a040 comp rx=0 tx=0).stop 2024-05-15T16:02:26.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.862+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c100ee0 msgr2=0x7f464c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.862+0000 7f46514fe700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c100ee0 0x7f464c0732a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d8d0 tx=0x7f463c00dbe0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.863+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 shutdown_connections 2024-05-15T16:02:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.863+0000 7f46514fe700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f463406e200 0x7f46340706c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.863+0000 7f46514fe700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c100ee0 0x7f464c0732a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.863+0000 7f46514fe700 1 --2- 172.21.15.49:0/4022065131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464c101890 0x7f464c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.863+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 >> 172.21.15.49:0/4022065131 conn(0x7f464c0fc790 msgr2=0x7f464c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.863+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 shutdown_connections 2024-05-15T16:02:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.863+0000 7f46514fe700 1 -- 172.21.15.49:0/4022065131 wait complete. 2024-05-15T16:02:26.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.972+0000 7fc00c783700 1 -- 172.21.15.49:0/1610946410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 msgr2=0x7fc0041012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.972+0000 7fc00c783700 1 --2- 172.21.15.49:0/1610946410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 0x7fc0041012c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc000009910 tx=0x7fc000009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:26.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.972+0000 7fc00c783700 1 -- 172.21.15.49:0/1610946410 shutdown_connections 2024-05-15T16:02:26.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.972+0000 7fc00c783700 1 --2- 172.21.15.49:0/1610946410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc004101890 0x7fc0041058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.972+0000 7fc00c783700 1 --2- 172.21.15.49:0/1610946410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 0x7fc0041012c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.972+0000 7fc00c783700 1 -- 172.21.15.49:0/1610946410 >> 172.21.15.49:0/1610946410 conn(0x7fc0040fc790 msgr2=0x7fc0040febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:26.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.973+0000 7fc00c783700 1 -- 172.21.15.49:0/1610946410 shutdown_connections 2024-05-15T16:02:26.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.973+0000 7fc00c783700 1 -- 172.21.15.49:0/1610946410 wait complete. 2024-05-15T16:02:26.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.973+0000 7fc00c783700 1 Processor -- start 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.973+0000 7fc00c783700 1 -- start start 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.973+0000 7fc00c783700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 0x7fc00419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.973+0000 7fc00c783700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc004101890 0x7fc00419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.973+0000 7fc00c783700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00419d460 con 0x7fc004100ee0 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.973+0000 7fc00c783700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc004196a90 con 0x7fc004101890 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc00a51f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 0x7fc00419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc00a51f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 0x7fc00419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34020/0 (socket says 172.21.15.49:34020) 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc00a51f700 1 -- 172.21.15.49:0/1263810421 learned_addr learned my addr 172.21.15.49:0/1263810421 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc009d1e700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc004101890 0x7fc00419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc00a51f700 1 -- 172.21.15.49:0/1263810421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc004101890 msgr2=0x7fc00419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc00a51f700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc004101890 0x7fc00419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc00a51f700 1 -- 172.21.15.49:0/1263810421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000095f0 con 0x7fc004100ee0 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc00a51f700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 0x7fc00419c900 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0000092e0 tx=0x7fc00000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:26.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fbff77fe700 1 -- 172.21.15.49:0/1263810421 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00001d070 con 0x7fc004100ee0 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc004196d10 con 0x7fc004100ee0 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fbff77fe700 1 -- 172.21.15.49:0/1263810421 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0000056e0 con 0x7fc004100ee0 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fbff77fe700 1 -- 172.21.15.49:0/1263810421 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00000fe10 con 0x7fc004100ee0 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.974+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc004197200 con 0x7fc004100ee0 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.975+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc004109280 con 0x7fc004100ee0 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.976+0000 7fbff77fe700 1 -- 172.21.15.49:0/1263810421 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc00000f680 con 0x7fc004100ee0 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.976+0000 7fbff77fe700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbff006e210 0x7fbff00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.976+0000 7fbff77fe700 1 -- 172.21.15.49:0/1263810421 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc00008f4c0 con 0x7fc004100ee0 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.976+0000 7fc009d1e700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbff006e210 0x7fbff00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:26.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.977+0000 7fc009d1e700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbff006e210 0x7fbff00706d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc004197c00 tx=0x7fbff80073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:26.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:26.981+0000 7fbff77fe700 1 -- 172.21.15.49:0/1263810421 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc00005c380 con 0x7fc004100ee0 2024-05-15T16:02:27.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.203+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc004197f70 con 0x7fbff006e210 2024-05-15T16:02:27.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.214+0000 7fbff77fe700 1 -- 172.21.15.49:0/1263810421 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc004197f70 con 0x7fbff006e210 2024-05-15T16:02:27.216 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:02:27.216 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (5m) 79s ago 17m 24.9M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:02:27.216 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (17m) 79s ago 17m 8883k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:02:27.216 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (16m) 12s ago 16m 8702k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (3m) 79s ago 17m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (2m) 12s ago 16m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (5m) 79s ago 16m 79.1M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (11m) 79s ago 11m 90.1M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (11m) 79s ago 11m 152M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (11m) 12s ago 11m 26.7M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (11m) 12s ago 11m 28.3M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (8m) 79s ago 18m 507M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (6m) 12s ago 16m 453M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (4m) 79s ago 18m 47.3M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (3m) 12s ago 16m 38.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (6m) 79s ago 17m 15.8M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (6m) 12s ago 16m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:02:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (2m) 79s ago 15m 82.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:02:27.218 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (2m) 79s ago 14m 84.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:02:27.218 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (84s) 79s ago 14m 14.8M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:02:27.218 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (49s) 12s ago 14m 97.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:02:27.218 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (16s) 12s ago 13m 11.8M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:02:27.218 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (13m) 12s ago 13m 203M 2240M 18.2.2-1343-g2965081f a04a73ffe079 0ba3921f264e 2024-05-15T16:02:27.218 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (6m) 79s ago 16m 68.7M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:02:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbff006e210 msgr2=0x7fbff00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbff006e210 0x7fbff00706d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc004197c00 tx=0x7fbff80073d0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 msgr2=0x7fc00419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 0x7fc00419c900 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0000092e0 tx=0x7fc00000bd70 comp rx=0 tx=0).stop 2024-05-15T16:02:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 shutdown_connections 2024-05-15T16:02:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbff006e210 0x7fbff00706d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc004100ee0 0x7fc00419c900 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 --2- 172.21.15.49:0/1263810421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc004101890 0x7fc00419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 >> 172.21.15.49:0/1263810421 conn(0x7fc0040fc790 msgr2=0x7fc0040fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 shutdown_connections 2024-05-15T16:02:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.216+0000 7fc00c783700 1 -- 172.21.15.49:0/1263810421 wait complete. 2024-05-15T16:02:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.329+0000 7f2bb481b700 1 -- 172.21.15.49:0/3797608710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 msgr2=0x7f2bb01022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.329+0000 7f2bb481b700 1 --2- 172.21.15.49:0/3797608710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 0x7f2bb01022c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009910 tx=0x7f2ba0009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.329+0000 7f2bb481b700 1 -- 172.21.15.49:0/3797608710 shutdown_connections 2024-05-15T16:02:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.329+0000 7f2bb481b700 1 --2- 172.21.15.49:0/3797608710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bb0102800 0x7f2bb010acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.329+0000 7f2bb481b700 1 --2- 172.21.15.49:0/3797608710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 0x7f2bb01022c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.329+0000 7f2bb481b700 1 -- 172.21.15.49:0/3797608710 >> 172.21.15.49:0/3797608710 conn(0x7f2bb0076800 msgr2=0x7f2bb00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.329+0000 7f2bb481b700 1 -- 172.21.15.49:0/3797608710 shutdown_connections 2024-05-15T16:02:27.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.329+0000 7f2bb481b700 1 -- 172.21.15.49:0/3797608710 wait complete. 2024-05-15T16:02:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2bb481b700 1 Processor -- start 2024-05-15T16:02:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2bb481b700 1 -- start start 2024-05-15T16:02:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2bb481b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 0x7f2bb019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2bb481b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bb0102800 0x7f2bb019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2bb481b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb019d3a0 con 0x7f2bb0101ee0 2024-05-15T16:02:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2bb481b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb0196a10 con 0x7f2bb0102800 2024-05-15T16:02:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2bae59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 0x7f2bb019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2bae59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 0x7f2bb019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34036/0 (socket says 172.21.15.49:34036) 2024-05-15T16:02:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2bae59c700 1 -- 172.21.15.49:0/2593265030 learned_addr learned my addr 172.21.15.49:0/2593265030 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.330+0000 7f2badd9b700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bb0102800 0x7f2bb019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.331+0000 7f2bae59c700 1 -- 172.21.15.49:0/2593265030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bb0102800 msgr2=0x7f2bb019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.331+0000 7f2bae59c700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bb0102800 0x7f2bb019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.331+0000 7f2bae59c700 1 -- 172.21.15.49:0/2593265030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba00095f0 con 0x7f2bb0101ee0 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.331+0000 7f2bae59c700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 0x7f2bb019c8d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00092e0 tx=0x7f2ba000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.331+0000 7f2b9f7fe700 1 -- 172.21.15.49:0/2593265030 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba001d070 con 0x7f2bb0101ee0 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.331+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb0196c90 con 0x7f2bb0101ee0 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.331+0000 7f2b9f7fe700 1 -- 172.21.15.49:0/2593265030 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ba000f460 con 0x7f2bb0101ee0 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.331+0000 7f2b9f7fe700 1 -- 172.21.15.49:0/2593265030 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba0017630 con 0x7f2bb0101ee0 2024-05-15T16:02:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.331+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb0197180 con 0x7f2bb0101ee0 2024-05-15T16:02:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.332+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb0105600 con 0x7f2bb0101ee0 2024-05-15T16:02:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.333+0000 7f2b9f7fe700 1 -- 172.21.15.49:0/2593265030 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2ba000f5d0 con 0x7f2bb0101ee0 2024-05-15T16:02:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.333+0000 7f2b9f7fe700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b9806e210 0x7f2b980706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.333+0000 7f2b9f7fe700 1 -- 172.21.15.49:0/2593265030 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2ba008f4c0 con 0x7f2bb0101ee0 2024-05-15T16:02:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.333+0000 7f2badd9b700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b9806e210 0x7f2b980706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.334+0000 7f2badd9b700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b9806e210 0x7f2b980706d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0197bd0 tx=0x7f2ba400a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.338+0000 7f2b9f7fe700 1 -- 172.21.15.49:0/2593265030 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2ba005c400 con 0x7f2bb0101ee0 2024-05-15T16:02:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.640+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2bb0197ed0 con 0x7f2bb0101ee0 2024-05-15T16:02:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.641+0000 7f2b9f7fe700 1 -- 172.21.15.49:0/2593265030 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f2ba005bf90 con 0x7f2bb0101ee0 2024-05-15T16:02:27.643 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:02:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:02:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:02:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:02:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:02:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:02:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:02:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 5 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 5, 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 9 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:02:27.644 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b9806e210 msgr2=0x7f2b980706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b9806e210 0x7f2b980706d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0197bd0 tx=0x7f2ba400a380 comp rx=0 tx=0).stop 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 msgr2=0x7f2bb019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 0x7f2bb019c8d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00092e0 tx=0x7f2ba000bd70 comp rx=0 tx=0).stop 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 shutdown_connections 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b9806e210 0x7f2b980706d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb0101ee0 0x7f2bb019c8d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 --2- 172.21.15.49:0/2593265030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bb0102800 0x7f2bb019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 >> 172.21.15.49:0/2593265030 conn(0x7f2bb0076800 msgr2=0x7f2bb01031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 shutdown_connections 2024-05-15T16:02:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.643+0000 7f2bb481b700 1 -- 172.21.15.49:0/2593265030 wait complete. 2024-05-15T16:02:27.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.753+0000 7f6c1af6c700 1 -- 172.21.15.49:0/3567485645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 msgr2=0x7f6c1410cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.753+0000 7f6c1af6c700 1 --2- 172.21.15.49:0/3567485645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 0x7f6c1410cfe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009910 tx=0x7f6c08009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.753+0000 7f6c1af6c700 1 -- 172.21.15.49:0/3567485645 shutdown_connections 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.753+0000 7f6c1af6c700 1 --2- 172.21.15.49:0/3567485645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 0x7f6c1410cfe0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.753+0000 7f6c1af6c700 1 --2- 172.21.15.49:0/3567485645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c14073370 0x7f6c14073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.753+0000 7f6c1af6c700 1 -- 172.21.15.49:0/3567485645 >> 172.21.15.49:0/3567485645 conn(0x7f6c140fc790 msgr2=0x7f6c140febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.753+0000 7f6c1af6c700 1 -- 172.21.15.49:0/3567485645 shutdown_connections 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.753+0000 7f6c1af6c700 1 -- 172.21.15.49:0/3567485645 wait complete. 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.753+0000 7f6c1af6c700 1 Processor -- start 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c1af6c700 1 -- start start 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c1af6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c14073370 0x7f6c14198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c1af6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 0x7f6c141992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c1af6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c141999d0 con 0x7f6c14073c90 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c1af6c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1419d6a0 con 0x7f6c14073370 2024-05-15T16:02:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c13fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 0x7f6c141992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c13fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 0x7f6c141992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34052/0 (socket says 172.21.15.49:34052) 2024-05-15T16:02:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c13fff700 1 -- 172.21.15.49:0/2039175238 learned_addr learned my addr 172.21.15.49:0/2039175238 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c18d08700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c14073370 0x7f6c14198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c13fff700 1 -- 172.21.15.49:0/2039175238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c14073370 msgr2=0x7f6c14198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c13fff700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c14073370 0x7f6c14198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c13fff700 1 -- 172.21.15.49:0/2039175238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c080095f0 con 0x7f6c14073c90 2024-05-15T16:02:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c18d08700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c14073370 0x7f6c14198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:02:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.754+0000 7f6c13fff700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 0x7f6c141992f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c08000c00 tx=0x7f6c0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.755+0000 7f6c11ffb700 1 -- 172.21.15.49:0/2039175238 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c0801d070 con 0x7f6c14073c90 2024-05-15T16:02:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.755+0000 7f6c11ffb700 1 -- 172.21.15.49:0/2039175238 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c080056e0 con 0x7f6c14073c90 2024-05-15T16:02:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.755+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1419d920 con 0x7f6c14073c90 2024-05-15T16:02:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.755+0000 7f6c11ffb700 1 -- 172.21.15.49:0/2039175238 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c0800fe80 con 0x7f6c14073c90 2024-05-15T16:02:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.755+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1419de10 con 0x7f6c14073c90 2024-05-15T16:02:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.756+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c14074a20 con 0x7f6c14073c90 2024-05-15T16:02:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.756+0000 7f6c11ffb700 1 -- 172.21.15.49:0/2039175238 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6c0800f6f0 con 0x7f6c14073c90 2024-05-15T16:02:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.757+0000 7f6c11ffb700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6bfc06e210 0x7f6bfc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.757+0000 7f6c11ffb700 1 -- 172.21.15.49:0/2039175238 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6c0808f470 con 0x7f6c14073c90 2024-05-15T16:02:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.757+0000 7f6c18d08700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6bfc06e210 0x7f6bfc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.758+0000 7f6c18d08700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6bfc06e210 0x7f6bfc0706d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400bc10 tx=0x7f6c0400b5f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:27.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:27.762+0000 7f6c11ffb700 1 -- 172.21.15.49:0/2039175238 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6c08058880 con 0x7f6c14073c90 2024-05-15T16:02:27.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:27 smithi049 ceph-mon[112142]: pgmap v151: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-05-15T16:02:27.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:27 smithi049 ceph-mon[112142]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:02:27.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:27 smithi049 ceph-mon[112142]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:02:27.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:27 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2593265030' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.017+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6c1404ec60 con 0x7f6c14073c90 2024-05-15T16:02:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.018+0000 7f6c11ffb700 1 -- 172.21.15.49:0/2039175238 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f6c0805bf40 con 0x7f6c14073c90 2024-05-15T16:02:28.020 INFO:teuthology.orchestra.run.smithi049.stdout:e17 2024-05-15T16:02:28.020 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:02:28.020 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:02:28.020 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:02:28.020 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 17 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T15:51:00.559034+0000 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:02:28.021 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 47 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=14484,1=24303} 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:02:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:02:28.023 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:02:28.023 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:02:28.023 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.49:6828/2512292397,v1:172.21.15.49:6829/2512292397] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:02:28.023 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3374515140,v1:172.21.15.49:6827/3374515140] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:02:28.023 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{1:24303} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.144:6824/2552333341,v1:172.21.15.144:6825/2552333341] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:02:28.023 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:24313} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.144:6826/2463281667,v1:172.21.15.144:6827/2463281667] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:02:28.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:02:28.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6bfc06e210 msgr2=0x7f6bfc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6bfc06e210 0x7f6bfc0706d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400bc10 tx=0x7f6c0400b5f0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 msgr2=0x7f6c141992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 0x7f6c141992f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c08000c00 tx=0x7f6c0800be20 comp rx=0 tx=0).stop 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 shutdown_connections 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6bfc06e210 0x7f6bfc0706d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c14073370 0x7f6c14198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 --2- 172.21.15.49:0/2039175238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c14073c90 0x7f6c141992f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 >> 172.21.15.49:0/2039175238 conn(0x7f6c140fc790 msgr2=0x7f6c141054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 shutdown_connections 2024-05-15T16:02:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.020+0000 7f6c1af6c700 1 -- 172.21.15.49:0/2039175238 wait complete. 2024-05-15T16:02:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.130+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/862170356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d581007a0 msgr2=0x7f7d581047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.130+0000 7f7d5ddc5700 1 --2- 172.21.15.49:0/862170356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d581007a0 0x7f7d581047f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009910 tx=0x7f7d40009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.130+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/862170356 shutdown_connections 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.130+0000 7f7d5ddc5700 1 --2- 172.21.15.49:0/862170356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d581007a0 0x7f7d581047f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.130+0000 7f7d5ddc5700 1 --2- 172.21.15.49:0/862170356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d580ffdf0 0x7f7d581001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.130+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/862170356 >> 172.21.15.49:0/862170356 conn(0x7f7d580fb6a0 msgr2=0x7f7d580fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.130+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/862170356 shutdown_connections 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.130+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/862170356 wait complete. 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d5ddc5700 1 Processor -- start 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d5ddc5700 1 -- start start 2024-05-15T16:02:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d5ddc5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d580ffdf0 0x7f7d58101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d5ddc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d581007a0 0x7f7d58101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d5ddc5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58102310 con 0x7f7d580ffdf0 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d5ddc5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58102450 con 0x7f7d581007a0 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d580ffdf0 0x7f7d58101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d580ffdf0 0x7f7d58101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34066/0 (socket says 172.21.15.49:34066) 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d577fe700 1 -- 172.21.15.49:0/1738425197 learned_addr learned my addr 172.21.15.49:0/1738425197 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d56ffd700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d581007a0 0x7f7d58101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d577fe700 1 -- 172.21.15.49:0/1738425197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d581007a0 msgr2=0x7f7d58101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d577fe700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d581007a0 0x7f7d58101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.131+0000 7f7d577fe700 1 -- 172.21.15.49:0/1738425197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d400095f0 con 0x7f7d580ffdf0 2024-05-15T16:02:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.132+0000 7f7d577fe700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d580ffdf0 0x7f7d58101890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800d8d0 tx=0x7f7d4800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.132+0000 7f7d54ff9700 1 -- 172.21.15.49:0/1738425197 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d480098a0 con 0x7f7d580ffdf0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.132+0000 7f7d54ff9700 1 -- 172.21.15.49:0/1738425197 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d48010460 con 0x7f7d580ffdf0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.132+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d581a2e10 con 0x7f7d580ffdf0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.132+0000 7f7d54ff9700 1 -- 172.21.15.49:0/1738425197 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d4800f5f0 con 0x7f7d580ffdf0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.132+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d581a3170 con 0x7f7d580ffdf0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.133+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d581012d0 con 0x7f7d580ffdf0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.133+0000 7f7d54ff9700 1 -- 172.21.15.49:0/1738425197 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7d48009a00 con 0x7f7d580ffdf0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.134+0000 7f7d54ff9700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d4406e200 0x7f7d440706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.134+0000 7f7d56ffd700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d4406e200 0x7f7d440706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.134+0000 7f7d54ff9700 1 -- 172.21.15.49:0/1738425197 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7d4808dea0 con 0x7f7d580ffdf0 2024-05-15T16:02:28.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.135+0000 7f7d56ffd700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d4406e200 0x7f7d440706c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d58102fc0 tx=0x7f7d4001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:28.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.139+0000 7f7d54ff9700 1 -- 172.21.15.49:0/1738425197 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7d480572b0 con 0x7f7d580ffdf0 2024-05-15T16:02:28.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:27 smithi144 ceph-mon[86635]: pgmap v151: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-05-15T16:02:28.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:27 smithi144 ceph-mon[86635]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:02:28.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:27 smithi144 ceph-mon[86635]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:02:28.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:27 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2593265030' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:28.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.365+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d5810ce30 con 0x7f7d4406e200 2024-05-15T16:02:28.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.366+0000 7f7d54ff9700 1 -- 172.21.15.49:0/1738425197 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7d5810ce30 con 0x7f7d4406e200 2024-05-15T16:02:28.368 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:02:28.368 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:02:28.368 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:02:28.368 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:02:28.369 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:02:28.369 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:02:28.369 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:02:28.369 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:02:28.369 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:02:28.369 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "11/23 daemons upgraded", 2024-05-15T16:02:28.369 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading osd daemons", 2024-05-15T16:02:28.369 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:02:28.369 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:02:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d4406e200 msgr2=0x7f7d440706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d4406e200 0x7f7d440706c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d58102fc0 tx=0x7f7d4001a040 comp rx=0 tx=0).stop 2024-05-15T16:02:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d580ffdf0 msgr2=0x7f7d58101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d580ffdf0 0x7f7d58101890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800d8d0 tx=0x7f7d4800dbe0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 shutdown_connections 2024-05-15T16:02:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d4406e200 0x7f7d440706c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d580ffdf0 0x7f7d58101890 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 --2- 172.21.15.49:0/1738425197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d581007a0 0x7f7d58101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 >> 172.21.15.49:0/1738425197 conn(0x7f7d580fb6a0 msgr2=0x7f7d580695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 shutdown_connections 2024-05-15T16:02:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.369+0000 7f7d5ddc5700 1 -- 172.21.15.49:0/1738425197 wait complete. 2024-05-15T16:02:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.477+0000 7f39e6753700 1 -- 172.21.15.49:0/3655763735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e00ffdf0 msgr2=0x7f39e01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.478+0000 7f39e6753700 1 --2- 172.21.15.49:0/3655763735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e00ffdf0 0x7f39e01001d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009910 tx=0x7f39d0009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.478+0000 7f39e6753700 1 -- 172.21.15.49:0/3655763735 shutdown_connections 2024-05-15T16:02:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.478+0000 7f39e6753700 1 --2- 172.21.15.49:0/3655763735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e01007a0 0x7f39e01047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.478+0000 7f39e6753700 1 --2- 172.21.15.49:0/3655763735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e00ffdf0 0x7f39e01001d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.478+0000 7f39e6753700 1 -- 172.21.15.49:0/3655763735 >> 172.21.15.49:0/3655763735 conn(0x7f39e00fb6a0 msgr2=0x7f39e00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.478+0000 7f39e6753700 1 -- 172.21.15.49:0/3655763735 shutdown_connections 2024-05-15T16:02:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.478+0000 7f39e6753700 1 -- 172.21.15.49:0/3655763735 wait complete. 2024-05-15T16:02:28.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.478+0000 7f39e6753700 1 Processor -- start 2024-05-15T16:02:28.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.478+0000 7f39e6753700 1 -- start start 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39e6753700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e00ffdf0 0x7f39e0101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39e6753700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01007a0 0x7f39e0101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39e6753700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0102310 con 0x7f39e01007a0 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39e6753700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0102450 con 0x7f39e00ffdf0 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39df7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01007a0 0x7f39e0101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39df7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01007a0 0x7f39e0101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34088/0 (socket says 172.21.15.49:34088) 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39df7fe700 1 -- 172.21.15.49:0/1163795532 learned_addr learned my addr 172.21.15.49:0/1163795532 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39dffff700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e00ffdf0 0x7f39e0101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39df7fe700 1 -- 172.21.15.49:0/1163795532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e00ffdf0 msgr2=0x7f39e0101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39df7fe700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e00ffdf0 0x7f39e0101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39df7fe700 1 -- 172.21.15.49:0/1163795532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d00095f0 con 0x7f39e01007a0 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39df7fe700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01007a0 0x7f39e0101dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39d400c8f0 tx=0x7f39d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.479+0000 7f39dd7fa700 1 -- 172.21.15.49:0/1163795532 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39d4007a50 con 0x7f39e01007a0 2024-05-15T16:02:28.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.480+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e01a2d90 con 0x7f39e01007a0 2024-05-15T16:02:28.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.480+0000 7f39dd7fa700 1 -- 172.21.15.49:0/1163795532 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39d4007bb0 con 0x7f39e01007a0 2024-05-15T16:02:28.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.480+0000 7f39dd7fa700 1 -- 172.21.15.49:0/1163795532 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39d4018630 con 0x7f39e01007a0 2024-05-15T16:02:28.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.480+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e01a30f0 con 0x7f39e01007a0 2024-05-15T16:02:28.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.481+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e01091a0 con 0x7f39e01007a0 2024-05-15T16:02:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.481+0000 7f39dd7fa700 1 -- 172.21.15.49:0/1163795532 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f39d4007d20 con 0x7f39e01007a0 2024-05-15T16:02:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.482+0000 7f39dd7fa700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39c806e200 0x7f39c80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.482+0000 7f39dd7fa700 1 -- 172.21.15.49:0/1163795532 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f39d408e0b0 con 0x7f39e01007a0 2024-05-15T16:02:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.482+0000 7f39dffff700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39c806e200 0x7f39c80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:28.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.482+0000 7f39dffff700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39c806e200 0x7f39c80706c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f39d000b5c0 tx=0x7f39d001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:28.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.486+0000 7f39dd7fa700 1 -- 172.21.15.49:0/1163795532 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f39d4057440 con 0x7f39e01007a0 2024-05-15T16:02:28.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.780+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f39e01012d0 con 0x7f39e01007a0 2024-05-15T16:02:28.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.781+0000 7f39dd7fa700 1 -- 172.21.15.49:0/1163795532 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f39d405ab00 con 0x7f39e01007a0 2024-05-15T16:02:28.783 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T16:02:28.783 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:02:28.783 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39c806e200 msgr2=0x7f39c80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39c806e200 0x7f39c80706c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f39d000b5c0 tx=0x7f39d001a040 comp rx=0 tx=0).stop 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01007a0 msgr2=0x7f39e0101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01007a0 0x7f39e0101dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39d400c8f0 tx=0x7f39d400ccb0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 shutdown_connections 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39c806e200 0x7f39c80706c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e00ffdf0 0x7f39e0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 --2- 172.21.15.49:0/1163795532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e01007a0 0x7f39e0101dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 >> 172.21.15.49:0/1163795532 conn(0x7f39e00fb6a0 msgr2=0x7f39e0069700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 shutdown_connections 2024-05-15T16:02:28.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:28.783+0000 7f39e6753700 1 -- 172.21.15.49:0/1163795532 wait complete. 2024-05-15T16:02:29.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:28 smithi144 ceph-mon[86635]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:02:29.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:28 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2039175238' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:02:29.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:28 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1163795532' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:02:29.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:28 smithi049 ceph-mon[112142]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:02:29.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:28 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2039175238' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:02:29.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:28 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1163795532' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:02:30.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:29 smithi144 ceph-mon[86635]: pgmap v152: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-15T16:02:30.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:29 smithi144 ceph-mon[86635]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:02:30.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:29 smithi049 ceph-mon[112142]: pgmap v152: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-15T16:02:30.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:29 smithi049 ceph-mon[112142]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:02:31.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:31 smithi049 ceph-mon[112142]: pgmap v153: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-05-15T16:02:31.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:31 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:31 smithi144 ceph-mon[86635]: pgmap v153: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-05-15T16:02:31.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:31 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:33.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:33 smithi144 ceph-mon[86635]: pgmap v154: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:02:33.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:33 smithi049 ceph-mon[112142]: pgmap v154: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:02:35.492 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:35 smithi144 ceph-mon[86635]: pgmap v155: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:02:35.492 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-15T16:02:35.492 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:35 smithi144 ceph-mon[86635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-15T16:02:35.492 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:35 smithi144 ceph-mon[86635]: Upgrade: osd.5 is safe to restart 2024-05-15T16:02:35.492 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:35.492 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:02:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:35 smithi049 ceph-mon[112142]: pgmap v155: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:02:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-15T16:02:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:35 smithi049 ceph-mon[112142]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-15T16:02:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:35 smithi049 ceph-mon[112142]: Upgrade: osd.5 is safe to restart 2024-05-15T16:02:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:02:37.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:37 smithi144 ceph-mon[86635]: pgmap v156: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:02:37.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:37 smithi144 ceph-mon[86635]: Upgrade: Updating osd.5 2024-05-15T16:02:37.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:37.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-15T16:02:37.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:37.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:37 smithi144 ceph-mon[86635]: Deploying daemon osd.5 on smithi144 2024-05-15T16:02:37.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:37 smithi049 ceph-mon[112142]: pgmap v156: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T16:02:37.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:37 smithi049 ceph-mon[112142]: Upgrade: Updating osd.5 2024-05-15T16:02:37.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:37.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-15T16:02:37.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:37.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:37 smithi049 ceph-mon[112142]: Deploying daemon osd.5 on smithi144 2024-05-15T16:02:39.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:39 smithi144 ceph-mon[86635]: pgmap v157: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-15T16:02:39.944 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:39 smithi144 systemd[1]: Stopping Ceph osd.5 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:02:39.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:39 smithi049 ceph-mon[112142]: pgmap v157: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-05-15T16:02:40.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:39 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[55371]: 2024-05-15T16:02:39.945+0000 7f31ed631700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:02:40.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:39 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[55371]: 2024-05-15T16:02:39.945+0000 7f31ed631700 -1 osd.5 75 *** Got signal Terminated *** 2024-05-15T16:02:40.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:39 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[55371]: 2024-05-15T16:02:39.945+0000 7f31ed631700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:02:40.888 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:40 smithi144 ceph-mon[86635]: osd.5 marked itself down and dead 2024-05-15T16:02:40.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:40 smithi049 ceph-mon[112142]: osd.5 marked itself down and dead 2024-05-15T16:02:41.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:40 smithi144 podman[105744]: 2024-05-15 16:02:40.965239547 +0000 UTC m=+1.127248502 container died 0ba3921f264e43f5cdbd65728c5617498b8664990f51f0086feae3fdfbe320af (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-2965081, io.buildah.version=1.33.5) 2024-05-15T16:02:41.598 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:41 smithi144 podman[105744]: 2024-05-15 16:02:41.345041622 +0000 UTC m=+1.507050577 container cleanup 0ba3921f264e43f5cdbd65728c5617498b8664990f51f0086feae3fdfbe320af (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-2965081, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:02:41.598 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:41 smithi144 bash[105744]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5 2024-05-15T16:02:41.598 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:41 smithi144 ceph-mon[86635]: pgmap v158: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:02:41.598 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:41 smithi144 ceph-mon[86635]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:02:41.598 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:41 smithi144 ceph-mon[86635]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-15T16:02:41.599 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:41 smithi144 ceph-mon[86635]: osdmap e76: 6 total, 5 up, 6 in 2024-05-15T16:02:41.930 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:41 smithi144 podman[105774]: 2024-05-15 16:02:41.596478702 +0000 UTC m=+0.624137212 container remove 0ba3921f264e43f5cdbd65728c5617498b8664990f51f0086feae3fdfbe320af (image=quay.ceph.io/ceph-ci/ceph@sha256:0764a18419358e118953c7a2137c56896dd1b4a53ba86e3e915181d824ff62f1, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=reef-2965081, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-15T16:02:41.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:41 smithi049 ceph-mon[112142]: pgmap v158: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:02:41.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:41 smithi049 ceph-mon[112142]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-15T16:02:41.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:41 smithi049 ceph-mon[112142]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-15T16:02:41.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:41 smithi049 ceph-mon[112142]: osdmap e76: 6 total, 5 up, 6 in 2024-05-15T16:02:42.185 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:41 smithi144 podman[105861]: 2024-05-15 16:02:41.831632737 +0000 UTC m=+0.017402246 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:02:42.185 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:41 smithi144 podman[105861]: 2024-05-15 16:02:41.964457369 +0000 UTC m=+0.150226870 container create 8aeffbe0791615ea9453b2627d420e1e610ec881a2129504f9d4c412c773d327 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-05-15T16:02:42.449 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:42 smithi144 podman[105861]: 2024-05-15 16:02:42.183454494 +0000 UTC m=+0.369223992 container init 8aeffbe0791615ea9453b2627d420e1e610ec881a2129504f9d4c412c773d327 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:02:42.449 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:42 smithi144 podman[105861]: 2024-05-15 16:02:42.187299349 +0000 UTC m=+0.373068856 container start 8aeffbe0791615ea9453b2627d420e1e610ec881a2129504f9d4c412c773d327 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, org.label-schema.schema-version=1.0) 2024-05-15T16:02:42.449 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:42 smithi144 podman[105861]: 2024-05-15 16:02:42.266827178 +0000 UTC m=+0.452596691 container attach 8aeffbe0791615ea9453b2627d420e1e610ec881a2129504f9d4c412c773d327 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:02:42.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:42 smithi049 ceph-mon[112142]: osdmap e77: 6 total, 5 up, 6 in 2024-05-15T16:02:42.814 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:42 smithi144 ceph-mon[86635]: osdmap e77: 6 total, 5 up, 6 in 2024-05-15T16:02:42.814 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:42 smithi144 podman[105861]: 2024-05-15 16:02:42.496595267 +0000 UTC m=+0.682364769 container died 8aeffbe0791615ea9453b2627d420e1e610ec881a2129504f9d4c412c773d327 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:02:43.449 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 podman[105887]: 2024-05-15 16:02:43.145585286 +0000 UTC m=+0.638602595 container remove 8aeffbe0791615ea9453b2627d420e1e610ec881a2129504f9d4c412c773d327 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-15T16:02:43.449 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.5.service: Deactivated successfully. 2024-05-15T16:02:43.450 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 systemd[1]: Stopped Ceph osd.5 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:02:43.450 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 systemd[1]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.5.service: Consumed 55.880s CPU time. 2024-05-15T16:02:43.450 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 systemd[1]: Starting Ceph osd.5 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:02:43.751 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:43 smithi144 ceph-mon[86635]: pgmap v161: 65 pgs: 5 active+undersized+degraded, 3 peering, 6 stale+active+clean, 5 active+undersized, 46 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 14/321 objects degraded (4.361%) 2024-05-15T16:02:43.751 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:43 smithi144 ceph-mon[86635]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-15T16:02:43.751 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:43 smithi144 ceph-mon[86635]: Health check failed: Degraded data redundancy: 14/321 objects degraded (4.361%), 5 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:43.752 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 podman[105985]: 2024-05-15 16:02:43.504799734 +0000 UTC m=+0.021723644 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:02:43.752 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 podman[105985]: 2024-05-15 16:02:43.613761295 +0000 UTC m=+0.130685231 container create c2595499477ee33addcf9c6dc972497e6bbca78c42fe4bd34393b80f75a307f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-15T16:02:43.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:43 smithi049 ceph-mon[112142]: pgmap v161: 65 pgs: 5 active+undersized+degraded, 3 peering, 6 stale+active+clean, 5 active+undersized, 46 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 14/321 objects degraded (4.361%) 2024-05-15T16:02:43.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:43 smithi049 ceph-mon[112142]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-15T16:02:43.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:43 smithi049 ceph-mon[112142]: Health check failed: Degraded data redundancy: 14/321 objects degraded (4.361%), 5 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:44.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 podman[105985]: 2024-05-15 16:02:43.866386819 +0000 UTC m=+0.383310706 container init c2595499477ee33addcf9c6dc972497e6bbca78c42fe4bd34393b80f75a307f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-15T16:02:44.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 podman[105985]: 2024-05-15 16:02:43.869691611 +0000 UTC m=+0.386615501 container start c2595499477ee33addcf9c6dc972497e6bbca78c42fe4bd34393b80f75a307f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate, ceph=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-15T16:02:44.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:43 smithi144 podman[105985]: 2024-05-15 16:02:43.941195895 +0000 UTC m=+0.458119782 container attach c2595499477ee33addcf9c6dc972497e6bbca78c42fe4bd34393b80f75a307f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True) 2024-05-15T16:02:44.699 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate[106000]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:02:44.699 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate[106000]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:02:44.700 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 bash[105985]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:02:44.700 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 bash[105985]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-15T16:02:44.700 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 bash[105985]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-15T16:02:44.700 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 bash[105985]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-15T16:02:44.700 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 bash[105985]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-15T16:02:44.700 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 bash[105985]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-15T16:02:44.700 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 bash[105985]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-15T16:02:44.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 bash[105985]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-15T16:02:44.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 bash[105985]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-15T16:02:44.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate[106000]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-15T16:02:44.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate[106000]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-15T16:02:44.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate[106000]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-15T16:02:44.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate[106000]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-15T16:02:44.702 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate[106000]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-15T16:02:44.702 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate[106000]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-15T16:02:44.702 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate[106000]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-15T16:02:44.702 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:44 smithi144 podman[105985]: 2024-05-15 16:02:44.381284689 +0000 UTC m=+0.898208577 container died c2595499477ee33addcf9c6dc972497e6bbca78c42fe4bd34393b80f75a307f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-15T16:02:45.430 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:45 smithi144 podman[106131]: 2024-05-15 16:02:45.019972024 +0000 UTC m=+0.630178811 container remove c2595499477ee33addcf9c6dc972497e6bbca78c42fe4bd34393b80f75a307f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-15T16:02:45.430 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:45 smithi144 podman[106191]: 2024-05-15 16:02:45.136991082 +0000 UTC m=+0.017388511 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:02:45.430 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:45 smithi144 podman[106191]: 2024-05-15 16:02:45.429245044 +0000 UTC m=+0.309642476 container create eec3c5d0c5eed40fa380b745da6feb5f46345be0550bae94610a8fb7ab175388 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-15T16:02:45.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:45 smithi144 ceph-mon[86635]: pgmap v162: 65 pgs: 8 active+undersized+degraded, 3 peering, 3 stale+active+clean, 7 active+undersized, 44 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22/321 objects degraded (6.854%) 2024-05-15T16:02:45.699 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:45 smithi144 podman[106191]: 2024-05-15 16:02:45.673660362 +0000 UTC m=+0.554057772 container init eec3c5d0c5eed40fa380b745da6feb5f46345be0550bae94610a8fb7ab175388 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-15T16:02:45.699 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:45 smithi144 podman[106191]: 2024-05-15 16:02:45.676951464 +0000 UTC m=+0.557348880 container start eec3c5d0c5eed40fa380b745da6feb5f46345be0550bae94610a8fb7ab175388 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-15T16:02:45.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:45 smithi049 ceph-mon[112142]: pgmap v162: 65 pgs: 8 active+undersized+degraded, 3 peering, 3 stale+active+clean, 7 active+undersized, 44 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22/321 objects degraded (6.854%) 2024-05-15T16:02:46.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:45 smithi144 ceph-osd[106209]: -- 172.21.15.144:0/1805254555 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce799e9680 con 0x55ce799b3000 2024-05-15T16:02:46.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:45 smithi144 bash[106191]: eec3c5d0c5eed40fa380b745da6feb5f46345be0550bae94610a8fb7ab175388 2024-05-15T16:02:46.199 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:45 smithi144 systemd[1]: Started Ceph osd.5 for c619ac44-12d1-11ef-bc98-c7b262605968. 2024-05-15T16:02:46.647 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:46 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:02:46.224+0000 7f36053487c0 -1 Falling back to public interface 2024-05-15T16:02:46.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:46.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:46.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:47.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:47.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:47.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:48.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:47 smithi144 ceph-mon[86635]: pgmap v163: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 50/321 objects degraded (15.576%) 2024-05-15T16:02:48.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:47 smithi049 ceph-mon[112142]: pgmap v163: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 50/321 objects degraded (15.576%) 2024-05-15T16:02:49.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:48 smithi144 ceph-mon[86635]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-15T16:02:49.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:48 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:49.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:48 smithi049 ceph-mon[112142]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-15T16:02:49.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:48 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:50.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:49 smithi144 ceph-mon[86635]: pgmap v164: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 50/321 objects degraded (15.576%) 2024-05-15T16:02:50.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:49 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:49 smithi049 ceph-mon[112142]: pgmap v164: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 50/321 objects degraded (15.576%) 2024-05-15T16:02:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:49 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:51.449 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:50 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:02:50.949+0000 7f36053487c0 -1 osd.5 75 log_to_monitors true 2024-05-15T16:02:51.449 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:02:51 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:02:51.195+0000 7f35f679f700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T16:02:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:51 smithi144 ceph-mon[86635]: pgmap v165: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 50/321 objects degraded (15.576%) 2024-05-15T16:02:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:02:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:51 smithi144 ceph-mon[86635]: from='osd.5 [v2:172.21.15.144:6816/255063313,v1:172.21.15.144:6817/255063313]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T16:02:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:51 smithi144 ceph-mon[86635]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T16:02:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:51.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:51 smithi049 ceph-mon[112142]: pgmap v165: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 50/321 objects degraded (15.576%) 2024-05-15T16:02:51.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:51.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:02:51.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:51 smithi049 ceph-mon[112142]: from='osd.5 [v2:172.21.15.144:6816/255063313,v1:172.21.15.144:6817/255063313]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T16:02:51.456 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:51 smithi049 ceph-mon[112142]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T16:02:52.281 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:52 smithi144 ceph-mon[86635]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-15T16:02:52.282 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:52 smithi144 ceph-mon[86635]: osdmap e78: 6 total, 5 up, 6 in 2024-05-15T16:02:52.282 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:52 smithi144 ceph-mon[86635]: from='osd.5 [v2:172.21.15.144:6816/255063313,v1:172.21.15.144:6817/255063313]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:02:52.282 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:52 smithi144 ceph-mon[86635]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:02:52.282 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:52 smithi144 ceph-mon[86635]: Health check update: Degraded data redundancy: 50/321 objects degraded (15.576%), 14 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:52.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:52 smithi049 ceph-mon[112142]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-15T16:02:52.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:52 smithi049 ceph-mon[112142]: osdmap e78: 6 total, 5 up, 6 in 2024-05-15T16:02:52.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:52 smithi049 ceph-mon[112142]: from='osd.5 [v2:172.21.15.144:6816/255063313,v1:172.21.15.144:6817/255063313]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:02:52.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:52 smithi049 ceph-mon[112142]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-15T16:02:52.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:52 smithi049 ceph-mon[112142]: Health check update: Degraded data redundancy: 50/321 objects degraded (15.576%), 14 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:53.301 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:53 smithi144 ceph-mon[86635]: pgmap v167: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 50/321 objects degraded (15.576%) 2024-05-15T16:02:53.301 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:53 smithi144 ceph-mon[86635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:02:53.302 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:53 smithi144 ceph-mon[86635]: osd.5 [v2:172.21.15.144:6816/255063313,v1:172.21.15.144:6817/255063313] boot 2024-05-15T16:02:53.302 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:53 smithi144 ceph-mon[86635]: osdmap e79: 6 total, 6 up, 6 in 2024-05-15T16:02:53.302 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:53 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T16:02:53.302 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:53 smithi144 ceph-mon[86635]: osdmap e80: 6 total, 6 up, 6 in 2024-05-15T16:02:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:53 smithi049 ceph-mon[112142]: pgmap v167: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 50/321 objects degraded (15.576%) 2024-05-15T16:02:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:53 smithi049 ceph-mon[112142]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-15T16:02:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:53 smithi049 ceph-mon[112142]: osd.5 [v2:172.21.15.144:6816/255063313,v1:172.21.15.144:6817/255063313] boot 2024-05-15T16:02:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:53 smithi049 ceph-mon[112142]: osdmap e79: 6 total, 6 up, 6 in 2024-05-15T16:02:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:53 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T16:02:53.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:53 smithi049 ceph-mon[112142]: osdmap e80: 6 total, 6 up, 6 in 2024-05-15T16:02:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:55.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all osd 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-15T16:02:55.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-15T16:02:55.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-15T16:02:55.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-15T16:02:55.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-15T16:02:55.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-15T16:02:55.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-15T16:02:55.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: Upgrade: Setting require_osd_release to 19 squid 2024-05-15T16:02:55.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-15T16:02:55.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all osd 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-15T16:02:55.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: Upgrade: Setting require_osd_release to 19 squid 2024-05-15T16:02:55.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-15T16:02:56.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:55 smithi049 ceph-mon[112142]: pgmap v170: 65 pgs: 2 peering, 13 active+undersized+degraded, 13 active+undersized, 37 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:02:56.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:55 smithi049 ceph-mon[112142]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-15T16:02:56.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-15T16:02:56.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:55 smithi049 ceph-mon[112142]: osdmap e81: 6 total, 6 up, 6 in 2024-05-15T16:02:56.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:55 smithi049 ceph-mon[112142]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-15T16:02:56.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:56.079 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-15T16:02:56.100 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:55 smithi144 ceph-mon[86635]: pgmap v170: 65 pgs: 2 peering, 13 active+undersized+degraded, 13 active+undersized, 37 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 47/321 objects degraded (14.642%) 2024-05-15T16:02:56.100 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:55 smithi144 ceph-mon[86635]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-15T16:02:56.100 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-15T16:02:56.100 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:55 smithi144 ceph-mon[86635]: osdmap e81: 6 total, 6 up, 6 in 2024-05-15T16:02:56.100 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:55 smithi144 ceph-mon[86635]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-15T16:02:56.100 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:56.100 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-15T16:02:56.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:02:56.076+0000 7f6ffa9ef700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T16:02:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: osdmap e82: 6 total, 6 up, 6 in 2024-05-15T16:02:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-15T16:02:56.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-15T16:02:56.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-15T16:02:56.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: osdmap e83: 6 total, 6 up, 6 in 2024-05-15T16:02:56.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] up:boot 2024-05-15T16:02:56.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6826/3089875224,v1:172.21.15.49:6827/3089875224] up:boot 2024-05-15T16:02:56.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-05-15T16:02:56.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T16:02:56.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T16:02:56.957 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:56 smithi049 ceph-mon[112142]: Health check update: Degraded data redundancy: 47/321 objects degraded (14.642%), 13 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:57.112 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: osdmap e82: 6 total, 6 up, 6 in 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: fsmap cephfs:2 {0=cephfs.smithi049.gocadf=up:active,1=cephfs.smithi144.jgodzs=up:active} 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:57.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: osdmap e83: 6 total, 6 up, 6 in 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] up:boot 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6826/3089875224,v1:172.21.15.49:6827/3089875224] up:boot 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T16:02:57.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:56 smithi144 ceph-mon[86635]: Health check update: Degraded data redundancy: 47/321 objects degraded (14.642%), 13 pgs degraded (PG_DEGRADED) 2024-05-15T16:02:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:57 smithi144 ceph-mon[86635]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-05-15T16:02:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:57 smithi144 ceph-mon[86635]: pgmap v174: 65 pgs: 2 peering, 8 active+undersized+degraded, 11 active+undersized, 44 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 24/321 objects degraded (7.477%) 2024-05-15T16:02:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:57 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.144:6824/1712325654,v1:172.21.15.144:6825/1712325654] up:boot 2024-05-15T16:02:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:57 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6828/210029258,v1:172.21.15.49:6829/210029258] up:boot 2024-05-15T16:02:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:57 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:02:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:57 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T16:02:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:57 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T16:02:58.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:57 smithi049 ceph-mon[112142]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-05-15T16:02:58.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:57 smithi049 ceph-mon[112142]: pgmap v174: 65 pgs: 2 peering, 8 active+undersized+degraded, 11 active+undersized, 44 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 24/321 objects degraded (7.477%) 2024-05-15T16:02:58.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:57 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.144:6824/1712325654,v1:172.21.15.144:6825/1712325654] up:boot 2024-05-15T16:02:58.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:57 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6828/210029258,v1:172.21.15.49:6829/210029258] up:boot 2024-05-15T16:02:58.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:57 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:02:58.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:57 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T16:02:58.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:57 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T16:02:58.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.896+0000 7f2111c63700 1 -- 172.21.15.49:0/1881879964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 msgr2=0x7f210c105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:58.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.896+0000 7f2111c63700 1 --2- 172.21.15.49:0/1881879964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 0x7f210c105900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2100009910 tx=0x7f2100009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:58.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.896+0000 7f2111c63700 1 -- 172.21.15.49:0/1881879964 shutdown_connections 2024-05-15T16:02:58.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.896+0000 7f2111c63700 1 --2- 172.21.15.49:0/1881879964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 0x7f210c105900 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:58.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.896+0000 7f2111c63700 1 --2- 172.21.15.49:0/1881879964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c100f00 0x7f210c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:58.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.896+0000 7f2111c63700 1 -- 172.21.15.49:0/1881879964 >> 172.21.15.49:0/1881879964 conn(0x7f210c0fc7d0 msgr2=0x7f210c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:58.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.896+0000 7f2111c63700 1 -- 172.21.15.49:0/1881879964 shutdown_connections 2024-05-15T16:02:58.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.896+0000 7f2111c63700 1 -- 172.21.15.49:0/1881879964 wait complete. 2024-05-15T16:02:58.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.897+0000 7f2111c63700 1 Processor -- start 2024-05-15T16:02:58.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.897+0000 7f2111c63700 1 -- start start 2024-05-15T16:02:58.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.897+0000 7f2111c63700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c100f00 0x7f210c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:58.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.897+0000 7f2111c63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 0x7f210c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.897+0000 7f2111c63700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c19d440 con 0x7f210c1018b0 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.897+0000 7f2111c63700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c196a20 con 0x7f210c100f00 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f210affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 0x7f210c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f210affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 0x7f210c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60486/0 (socket says 172.21.15.49:60486) 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f210affd700 1 -- 172.21.15.49:0/2875048230 learned_addr learned my addr 172.21.15.49:0/2875048230 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f210b7fe700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c100f00 0x7f210c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f210affd700 1 -- 172.21.15.49:0/2875048230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c100f00 msgr2=0x7f210c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f210affd700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c100f00 0x7f210c19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f210affd700 1 -- 172.21.15.49:0/2875048230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21000095f0 con 0x7f210c1018b0 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f210affd700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 0x7f210c19ce20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f210000bd70 tx=0x7f210000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f2108ff9700 1 -- 172.21.15.49:0/2875048230 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f210001d070 con 0x7f210c1018b0 2024-05-15T16:02:58.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c196ca0 con 0x7f210c1018b0 2024-05-15T16:02:58.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f2108ff9700 1 -- 172.21.15.49:0/2875048230 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21000056c0 con 0x7f210c1018b0 2024-05-15T16:02:58.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f2108ff9700 1 -- 172.21.15.49:0/2875048230 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2100017400 con 0x7f210c1018b0 2024-05-15T16:02:58.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.898+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c197190 con 0x7f210c1018b0 2024-05-15T16:02:58.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.900+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c109300 con 0x7f210c1018b0 2024-05-15T16:02:58.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.900+0000 7f2108ff9700 1 -- 172.21.15.49:0/2875048230 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2100003a60 con 0x7f210c1018b0 2024-05-15T16:02:58.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.901+0000 7f2108ff9700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20f406e200 0x7f20f40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:58.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.901+0000 7f2108ff9700 1 -- 172.21.15.49:0/2875048230 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f210008f4a0 con 0x7f210c1018b0 2024-05-15T16:02:58.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.901+0000 7f210b7fe700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20f406e200 0x7f20f40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:58.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.901+0000 7f210b7fe700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20f406e200 0x7f20f40706c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f20fc005950 tx=0x7f20fc00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:58.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:58.906+0000 7f2108ff9700 1 -- 172.21.15.49:0/2875048230 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f210005c330 con 0x7f210c1018b0 2024-05-15T16:02:59.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.136+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f210c197ee0 con 0x7f20f406e200 2024-05-15T16:02:59.139 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:58 smithi049 ceph-mon[112142]: Upgrade: Updating mds.cephfs.smithi049.abbagx 2024-05-15T16:02:59.139 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:58 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:59.139 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:58 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.abbagx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T16:02:59.139 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:58 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:59.139 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:58 smithi049 ceph-mon[112142]: Deploying daemon mds.cephfs.smithi049.abbagx on smithi049 2024-05-15T16:02:59.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.138+0000 7f2108ff9700 1 -- 172.21.15.49:0/2875048230 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f210c197ee0 con 0x7f20f406e200 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20f406e200 msgr2=0x7f20f40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20f406e200 0x7f20f40706c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f20fc005950 tx=0x7f20fc00a300 comp rx=0 tx=0).stop 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 msgr2=0x7f210c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 0x7f210c19ce20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f210000bd70 tx=0x7f210000bda0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 shutdown_connections 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20f406e200 0x7f20f40706c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c100f00 0x7f210c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 --2- 172.21.15.49:0/2875048230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f210c1018b0 0x7f210c19ce20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 >> 172.21.15.49:0/2875048230 conn(0x7f210c0fc7d0 msgr2=0x7f210c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.140+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 shutdown_connections 2024-05-15T16:02:59.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.141+0000 7f2111c63700 1 -- 172.21.15.49:0/2875048230 wait complete. 2024-05-15T16:02:59.154 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:02:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:58 smithi144 ceph-mon[86635]: Upgrade: Updating mds.cephfs.smithi049.abbagx 2024-05-15T16:02:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:58 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:02:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:58 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.abbagx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T16:02:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:58 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:02:59.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:58 smithi144 ceph-mon[86635]: Deploying daemon mds.cephfs.smithi049.abbagx on smithi049 2024-05-15T16:02:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.252+0000 7f4f72727700 1 -- 172.21.15.49:0/1892605426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 msgr2=0x7f4f6c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.252+0000 7f4f72727700 1 --2- 172.21.15.49:0/1892605426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 0x7f4f6c1058e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009910 tx=0x7f4f54009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:59.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.252+0000 7f4f72727700 1 -- 172.21.15.49:0/1892605426 shutdown_connections 2024-05-15T16:02:59.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.252+0000 7f4f72727700 1 --2- 172.21.15.49:0/1892605426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 0x7f4f6c1058e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.252+0000 7f4f72727700 1 --2- 172.21.15.49:0/1892605426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c100ee0 0x7f4f6c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.252+0000 7f4f72727700 1 -- 172.21.15.49:0/1892605426 >> 172.21.15.49:0/1892605426 conn(0x7f4f6c0fc790 msgr2=0x7f4f6c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:59.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.252+0000 7f4f72727700 1 -- 172.21.15.49:0/1892605426 shutdown_connections 2024-05-15T16:02:59.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.252+0000 7f4f72727700 1 -- 172.21.15.49:0/1892605426 wait complete. 2024-05-15T16:02:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f72727700 1 Processor -- start 2024-05-15T16:02:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f72727700 1 -- start start 2024-05-15T16:02:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f72727700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c100ee0 0x7f4f6c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f72727700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 0x7f4f6c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f72727700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19d440 con 0x7f4f6c101890 2024-05-15T16:02:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f72727700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c196a20 con 0x7f4f6c100ee0 2024-05-15T16:02:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f63fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 0x7f4f6c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f63fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 0x7f4f6c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60506/0 (socket says 172.21.15.49:60506) 2024-05-15T16:02:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f63fff700 1 -- 172.21.15.49:0/310837299 learned_addr learned my addr 172.21.15.49:0/310837299 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.253+0000 7f4f6bfff700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c100ee0 0x7f4f6c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.254+0000 7f4f63fff700 1 -- 172.21.15.49:0/310837299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c100ee0 msgr2=0x7f4f6c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.254+0000 7f4f63fff700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c100ee0 0x7f4f6c19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.254+0000 7f4f63fff700 1 -- 172.21.15.49:0/310837299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540095f0 con 0x7f4f6c101890 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.254+0000 7f4f63fff700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 0x7f4f6c19ce20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400bd70 tx=0x7f4f5400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.254+0000 7f4f69ffb700 1 -- 172.21.15.49:0/310837299 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5401d070 con 0x7f4f6c101890 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.254+0000 7f4f69ffb700 1 -- 172.21.15.49:0/310837299 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f540056e0 con 0x7f4f6c101890 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.254+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c196ca0 con 0x7f4f6c101890 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.254+0000 7f4f69ffb700 1 -- 172.21.15.49:0/310837299 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5400fe50 con 0x7f4f6c101890 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.254+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c197190 con 0x7f4f6c101890 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.255+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c1092e0 con 0x7f4f6c101890 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.256+0000 7f4f69ffb700 1 -- 172.21.15.49:0/310837299 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4f5400f680 con 0x7f4f6c101890 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.256+0000 7f4f69ffb700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4f5806e200 0x7f4f580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.256+0000 7f4f69ffb700 1 -- 172.21.15.49:0/310837299 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4f5408f400 con 0x7f4f6c101890 2024-05-15T16:02:59.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.256+0000 7f4f6bfff700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4f5806e200 0x7f4f580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:59.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.257+0000 7f4f6bfff700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4f5806e200 0x7f4f580706c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c005950 tx=0x7f4f5c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:59.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.261+0000 7f4f69ffb700 1 -- 172.21.15.49:0/310837299 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4f5405c290 con 0x7f4f6c101890 2024-05-15T16:02:59.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.488+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f6c197f50 con 0x7f4f5806e200 2024-05-15T16:02:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.489+0000 7f4f69ffb700 1 -- 172.21.15.49:0/310837299 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4f6c197f50 con 0x7f4f5806e200 2024-05-15T16:02:59.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.491+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4f5806e200 msgr2=0x7f4f580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.491+0000 7f4f72727700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4f5806e200 0x7f4f580706c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c005950 tx=0x7f4f5c00b410 comp rx=0 tx=0).stop 2024-05-15T16:02:59.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.491+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 msgr2=0x7f4f6c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.491+0000 7f4f72727700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 0x7f4f6c19ce20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400bd70 tx=0x7f4f5400bda0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.492+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 shutdown_connections 2024-05-15T16:02:59.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.492+0000 7f4f72727700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4f5806e200 0x7f4f580706c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.492+0000 7f4f72727700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c100ee0 0x7f4f6c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.492+0000 7f4f72727700 1 --2- 172.21.15.49:0/310837299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f6c101890 0x7f4f6c19ce20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.492+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 >> 172.21.15.49:0/310837299 conn(0x7f4f6c0fc790 msgr2=0x7f4f6c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:59.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.492+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 shutdown_connections 2024-05-15T16:02:59.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.492+0000 7f4f72727700 1 -- 172.21.15.49:0/310837299 wait complete. 2024-05-15T16:02:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.601+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2341487190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 msgr2=0x7f6c181048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.601+0000 7f6c1ea43700 1 --2- 172.21.15.49:0/2341487190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 0x7f6c181048b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009910 tx=0x7f6c0c009c20 comp rx=0 tx=0).stop 2024-05-15T16:02:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.601+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2341487190 shutdown_connections 2024-05-15T16:02:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.601+0000 7f6c1ea43700 1 --2- 172.21.15.49:0/2341487190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 0x7f6c181048b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.601+0000 7f6c1ea43700 1 --2- 172.21.15.49:0/2341487190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c181004c0 0x7f6c181008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.601+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2341487190 >> 172.21.15.49:0/2341487190 conn(0x7f6c18075fc0 msgr2=0x7f6c180fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.601+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2341487190 shutdown_connections 2024-05-15T16:02:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.601+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2341487190 wait complete. 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.602+0000 7f6c1ea43700 1 Processor -- start 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.602+0000 7f6c1ea43700 1 -- start start 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.602+0000 7f6c1ea43700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c181004c0 0x7f6c1819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.602+0000 7f6c1ea43700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 0x7f6c1819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c1ea43700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1819d4c0 con 0x7f6c18100e70 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c1ea43700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c18196aa0 con 0x7f6c181004c0 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 0x7f6c1819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 0x7f6c1819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60520/0 (socket says 172.21.15.49:60520) 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c177fe700 1 -- 172.21.15.49:0/2371428267 learned_addr learned my addr 172.21.15.49:0/2371428267 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c17fff700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c181004c0 0x7f6c1819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c177fe700 1 -- 172.21.15.49:0/2371428267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c181004c0 msgr2=0x7f6c1819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c177fe700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c181004c0 0x7f6c1819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c177fe700 1 -- 172.21.15.49:0/2371428267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0095f0 con 0x7f6c18100e70 2024-05-15T16:02:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c177fe700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 0x7f6c1819cea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009fd0 tx=0x7f6c0c00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c157fa700 1 -- 172.21.15.49:0/2371428267 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c0c01d070 con 0x7f6c18100e70 2024-05-15T16:02:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c18196d20 con 0x7f6c18100e70 2024-05-15T16:02:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c157fa700 1 -- 172.21.15.49:0/2371428267 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c0c0052a0 con 0x7f6c18100e70 2024-05-15T16:02:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.603+0000 7f6c157fa700 1 -- 172.21.15.49:0/2371428267 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c0c022470 con 0x7f6c18100e70 2024-05-15T16:02:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.604+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c18197210 con 0x7f6c18100e70 2024-05-15T16:02:59.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.605+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c18101480 con 0x7f6c18100e70 2024-05-15T16:02:59.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.605+0000 7f6c157fa700 1 -- 172.21.15.49:0/2371428267 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6c0c0225d0 con 0x7f6c18100e70 2024-05-15T16:02:59.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.606+0000 7f6c157fa700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c0006e200 0x7f6c000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:59.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.606+0000 7f6c157fa700 1 -- 172.21.15.49:0/2371428267 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6c0c08f480 con 0x7f6c18100e70 2024-05-15T16:02:59.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.606+0000 7f6c17fff700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c0006e200 0x7f6c000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:59.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.606+0000 7f6c17fff700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c0006e200 0x7f6c000706c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009c60 tx=0x7f6c08009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:59.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.610+0000 7f6c157fa700 1 -- 172.21.15.49:0/2371428267 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6c0c05c290 con 0x7f6c18100e70 2024-05-15T16:02:59.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.833+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c1810ce90 con 0x7f6c0006e200 2024-05-15T16:02:59.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.845+0000 7f6c157fa700 1 -- 172.21.15.49:0/2371428267 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6c1810ce90 con 0x7f6c0006e200 2024-05-15T16:02:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:02:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (6m) 112s ago 17m 24.9M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:02:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (18m) 112s ago 18m 8883k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:02:59.847 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (16m) 10s ago 16m 9013k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (3m) 112s ago 18m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (3m) 10s ago 16m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (5m) 112s ago 17m 79.1M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (12m) 112s ago 12m 90.1M - 18.2.2-1343-g2965081f a04a73ffe079 92583d54bc88 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (12m) 112s ago 12m 152M - 18.2.2-1343-g2965081f a04a73ffe079 4365ee4f0ccb 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (12m) 10s ago 12m 26.7M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (12m) 10s ago 12m 28.7M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (8m) 112s ago 19m 507M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (7m) 10s ago 16m 453M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (4m) 112s ago 19m 47.3M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (4m) 10s ago 16m 40.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (7m) 112s ago 17m 15.8M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (6m) 10s ago 16m 15.7M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (3m) 112s ago 15m 82.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:02:59.848 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (2m) 112s ago 15m 84.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:02:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (117s) 112s ago 14m 14.8M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:02:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (82s) 10s ago 14m 98.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:02:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (48s) 10s ago 14m 85.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:02:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (14s) 10s ago 13m 14.9M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:02:59.849 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (6m) 112s ago 17m 68.7M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:02:59.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c0006e200 msgr2=0x7f6c000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c0006e200 0x7f6c000706c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009c60 tx=0x7f6c08009260 comp rx=0 tx=0).stop 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 msgr2=0x7f6c1819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 0x7f6c1819cea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009fd0 tx=0x7f6c0c00b5a0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 shutdown_connections 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c0006e200 0x7f6c000706c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c181004c0 0x7f6c1819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 --2- 172.21.15.49:0/2371428267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c18100e70 0x7f6c1819cea0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 >> 172.21.15.49:0/2371428267 conn(0x7f6c18075fc0 msgr2=0x7f6c180fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 shutdown_connections 2024-05-15T16:02:59.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.847+0000 7f6c1ea43700 1 -- 172.21.15.49:0/2371428267 wait complete. 2024-05-15T16:02:59.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.967+0000 7fca40d03700 1 -- 172.21.15.49:0/2053535970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 msgr2=0x7fca3c10ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.967+0000 7fca40d03700 1 --2- 172.21.15.49:0/2053535970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 0x7fca3c10ad00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca30009940 tx=0x7fca30009c50 comp rx=0 tx=0).stop 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.967+0000 7fca40d03700 1 -- 172.21.15.49:0/2053535970 shutdown_connections 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.967+0000 7fca40d03700 1 --2- 172.21.15.49:0/2053535970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 0x7fca3c10ad00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.967+0000 7fca40d03700 1 --2- 172.21.15.49:0/2053535970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c101ef0 0x7fca3c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.967+0000 7fca40d03700 1 -- 172.21.15.49:0/2053535970 >> 172.21.15.49:0/2053535970 conn(0x7fca3c0fb6a0 msgr2=0x7fca3c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.967+0000 7fca40d03700 1 -- 172.21.15.49:0/2053535970 shutdown_connections 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.967+0000 7fca40d03700 1 -- 172.21.15.49:0/2053535970 wait complete. 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.968+0000 7fca40d03700 1 Processor -- start 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.968+0000 7fca40d03700 1 -- start start 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.968+0000 7fca40d03700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c101ef0 0x7fca3c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.968+0000 7fca40d03700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 0x7fca3c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.968+0000 7fca40d03700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c1999d0 con 0x7fca3c102810 2024-05-15T16:02:59.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.968+0000 7fca40d03700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c19d6a0 con 0x7fca3c101ef0 2024-05-15T16:02:59.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca39d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 0x7fca3c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:59.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca39d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 0x7fca3c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60542/0 (socket says 172.21.15.49:60542) 2024-05-15T16:02:59.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca39d9b700 1 -- 172.21.15.49:0/3431431350 learned_addr learned my addr 172.21.15.49:0/3431431350 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca3a59c700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c101ef0 0x7fca3c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca39d9b700 1 -- 172.21.15.49:0/3431431350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c101ef0 msgr2=0x7fca3c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca39d9b700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c101ef0 0x7fca3c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca39d9b700 1 -- 172.21.15.49:0/3431431350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300095f0 con 0x7fca3c102810 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca39d9b700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 0x7fca3c1992f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca3000b580 tx=0x7fca300055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca2b7fe700 1 -- 172.21.15.49:0/3431431350 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3001d070 con 0x7fca3c102810 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c19d920 con 0x7fca3c102810 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.969+0000 7fca2b7fe700 1 -- 172.21.15.49:0/3431431350 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca30003d50 con 0x7fca3c102810 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.970+0000 7fca2b7fe700 1 -- 172.21.15.49:0/3431431350 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca30021410 con 0x7fca3c102810 2024-05-15T16:02:59.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.970+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c19de10 con 0x7fca3c102810 2024-05-15T16:02:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.971+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c069730 con 0x7fca3c102810 2024-05-15T16:02:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.971+0000 7fca2b7fe700 1 -- 172.21.15.49:0/3431431350 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fca3000f6f0 con 0x7fca3c102810 2024-05-15T16:02:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.972+0000 7fca2b7fe700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca2406e210 0x7fca240706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:02:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.972+0000 7fca2b7fe700 1 -- 172.21.15.49:0/3431431350 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fca3008f260 con 0x7fca3c102810 2024-05-15T16:02:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.972+0000 7fca3a59c700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca2406e210 0x7fca240706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:02:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.973+0000 7fca3a59c700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca2406e210 0x7fca240706d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005950 tx=0x7fca2c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:02:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:02:59.977+0000 7fca2b7fe700 1 -- 172.21.15.49:0/3431431350 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fca30058540 con 0x7fca3c102810 2024-05-15T16:03:00.165 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:59 smithi049 ceph-mon[112142]: pgmap v175: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-15T16:03:00.165 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:02:59 smithi049 ceph-mon[112142]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/321 objects degraded (7.477%), 8 pgs degraded) 2024-05-15T16:03:00.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:59 smithi144 ceph-mon[86635]: pgmap v175: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-05-15T16:03:00.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:02:59 smithi144 ceph-mon[86635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/321 objects degraded (7.477%), 8 pgs degraded) 2024-05-15T16:03:00.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.281+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fca3c04ec60 con 0x7fca3c102810 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.282+0000 7fca2b7fe700 1 -- 172.21.15.49:0/3431431350 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7fca3005bc00 con 0x7fca3c102810 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:03:00.285 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:03:00.286 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4 2024-05-15T16:03:00.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:03:00.286 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:03:00.286 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 4, 2024-05-15T16:03:00.286 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 10 2024-05-15T16:03:00.286 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:03:00.286 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.284+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca2406e210 msgr2=0x7fca240706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.284+0000 7fca40d03700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca2406e210 0x7fca240706d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005950 tx=0x7fca2c00b410 comp rx=0 tx=0).stop 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.284+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 msgr2=0x7fca3c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.284+0000 7fca40d03700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 0x7fca3c1992f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca3000b580 tx=0x7fca300055f0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.284+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 shutdown_connections 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.284+0000 7fca40d03700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca2406e210 0x7fca240706d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.284+0000 7fca40d03700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c101ef0 0x7fca3c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.284+0000 7fca40d03700 1 --2- 172.21.15.49:0/3431431350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca3c102810 0x7fca3c1992f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.284+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 >> 172.21.15.49:0/3431431350 conn(0x7fca3c0fb6a0 msgr2=0x7fca3c10a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.285+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 shutdown_connections 2024-05-15T16:03:00.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.285+0000 7fca40d03700 1 -- 172.21.15.49:0/3431431350 wait complete. 2024-05-15T16:03:00.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.397+0000 7fd8af0e5700 1 -- 172.21.15.49:0/507675824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8104c30 msgr2=0x7fd8a8105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.397+0000 7fd8af0e5700 1 --2- 172.21.15.49:0/507675824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8104c30 0x7fd8a8105010 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd898009910 tx=0x7fd898009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.397+0000 7fd8af0e5700 1 -- 172.21.15.49:0/507675824 shutdown_connections 2024-05-15T16:03:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.397+0000 7fd8af0e5700 1 --2- 172.21.15.49:0/507675824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8a8100320 0x7fd8a81007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.397+0000 7fd8af0e5700 1 --2- 172.21.15.49:0/507675824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8104c30 0x7fd8a8105010 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.397+0000 7fd8af0e5700 1 -- 172.21.15.49:0/507675824 >> 172.21.15.49:0/507675824 conn(0x7fd8a8075fc0 msgr2=0x7fd8a80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.397+0000 7fd8af0e5700 1 -- 172.21.15.49:0/507675824 shutdown_connections 2024-05-15T16:03:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.397+0000 7fd8af0e5700 1 -- 172.21.15.49:0/507675824 wait complete. 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.398+0000 7fd8af0e5700 1 Processor -- start 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.398+0000 7fd8af0e5700 1 -- start start 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.398+0000 7fd8af0e5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8100320 0x7fd8a80fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.398+0000 7fd8af0e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8a8104c30 0x7fd8a80fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.398+0000 7fd8af0e5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80fbf30 con 0x7fd8a8100320 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.398+0000 7fd8af0e5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80fc0a0 con 0x7fd8a8104c30 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.398+0000 7fd8ace81700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8100320 0x7fd8a80fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.398+0000 7fd8ace81700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8100320 0x7fd8a80fb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60570/0 (socket says 172.21.15.49:60570) 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.398+0000 7fd8ace81700 1 -- 172.21.15.49:0/3741962821 learned_addr learned my addr 172.21.15.49:0/3741962821 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8a7fff700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8a8104c30 0x7fd8a80fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8ace81700 1 -- 172.21.15.49:0/3741962821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8a8104c30 msgr2=0x7fd8a80fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8ace81700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8a8104c30 0x7fd8a80fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8ace81700 1 -- 172.21.15.49:0/3741962821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8980095f0 con 0x7fd8a8100320 2024-05-15T16:03:00.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8ace81700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8100320 0x7fd8a80fb4b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd89800bd70 tx=0x7fd89800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:00.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8a5ffb700 1 -- 172.21.15.49:0/3741962821 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd89801d070 con 0x7fd8a8100320 2024-05-15T16:03:00.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8a5ffb700 1 -- 172.21.15.49:0/3741962821 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8980056e0 con 0x7fd8a8100320 2024-05-15T16:03:00.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a80fc320 con 0x7fd8a8100320 2024-05-15T16:03:00.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8a5ffb700 1 -- 172.21.15.49:0/3741962821 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd89800fe40 con 0x7fd8a8100320 2024-05-15T16:03:00.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.399+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a81a30e0 con 0x7fd8a8100320 2024-05-15T16:03:00.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.401+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a8101540 con 0x7fd8a8100320 2024-05-15T16:03:00.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.401+0000 7fd8a5ffb700 1 -- 172.21.15.49:0/3741962821 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd898003a60 con 0x7fd8a8100320 2024-05-15T16:03:00.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.401+0000 7fd8a5ffb700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd89006e1a0 0x7fd890070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:00.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.402+0000 7fd8a5ffb700 1 -- 172.21.15.49:0/3741962821 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd89808f410 con 0x7fd8a8100320 2024-05-15T16:03:00.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.402+0000 7fd8a7fff700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd89006e1a0 0x7fd890070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:00.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.402+0000 7fd8a7fff700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd89006e1a0 0x7fd890070660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80fcb90 tx=0x7fd89c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:00.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.410+0000 7fd8a5ffb700 1 -- 172.21.15.49:0/3741962821 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd8980586f0 con 0x7fd8a8100320 2024-05-15T16:03:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.668+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd8a80fc320 con 0x7fd8a8100320 2024-05-15T16:03:00.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.668+0000 7fd8a5ffb700 1 -- 172.21.15.49:0/3741962821 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1868 (secure 0 0 0) 0x7fd8a80fc320 con 0x7fd8a8100320 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:e20 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:03:00.671 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:03:00.672 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34380} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/3089875224,v1:172.21.15.49:6827/3089875224] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/210029258,v1:172.21.15.49:6829/210029258] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:03:00.673 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/1712325654,v1:172.21.15.144:6825/1712325654] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:03:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.670+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd89006e1a0 msgr2=0x7fd890070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.670+0000 7fd8af0e5700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd89006e1a0 0x7fd890070660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80fcb90 tx=0x7fd89c00b410 comp rx=0 tx=0).stop 2024-05-15T16:03:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.670+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8100320 msgr2=0x7fd8a80fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.670+0000 7fd8af0e5700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8100320 0x7fd8a80fb4b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd89800bd70 tx=0x7fd89800bda0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.671+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 shutdown_connections 2024-05-15T16:03:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.671+0000 7fd8af0e5700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd89006e1a0 0x7fd890070660 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.671+0000 7fd8af0e5700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8a8100320 0x7fd8a80fb4b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.671+0000 7fd8af0e5700 1 --2- 172.21.15.49:0/3741962821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8a8104c30 0x7fd8a80fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.671+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 >> 172.21.15.49:0/3741962821 conn(0x7fd8a8075fc0 msgr2=0x7fd8a80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.671+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 shutdown_connections 2024-05-15T16:03:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.671+0000 7fd8af0e5700 1 -- 172.21.15.49:0/3741962821 wait complete. 2024-05-15T16:03:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 20 2024-05-15T16:03:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.781+0000 7fc87931d700 1 -- 172.21.15.49:0/180699945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8740ffdd0 msgr2=0x7fc8741001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.781+0000 7fc87931d700 1 --2- 172.21.15.49:0/180699945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8740ffdd0 0x7fc8741001b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc864009910 tx=0x7fc864009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.781+0000 7fc87931d700 1 -- 172.21.15.49:0/180699945 shutdown_connections 2024-05-15T16:03:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.781+0000 7fc87931d700 1 --2- 172.21.15.49:0/180699945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc874100780 0x7fc8741047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.781+0000 7fc87931d700 1 --2- 172.21.15.49:0/180699945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8740ffdd0 0x7fc8741001b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.781+0000 7fc87931d700 1 -- 172.21.15.49:0/180699945 >> 172.21.15.49:0/180699945 conn(0x7fc8740fb6a0 msgr2=0x7fc8740fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.782+0000 7fc87931d700 1 -- 172.21.15.49:0/180699945 shutdown_connections 2024-05-15T16:03:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.782+0000 7fc87931d700 1 -- 172.21.15.49:0/180699945 wait complete. 2024-05-15T16:03:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.782+0000 7fc87931d700 1 Processor -- start 2024-05-15T16:03:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.782+0000 7fc87931d700 1 -- start start 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc87931d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740ffdd0 0x7fc87419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc87931d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc874100780 0x7fc87419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc87931d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87419d3d0 con 0x7fc874100780 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc87931d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741969b0 con 0x7fc8740ffdd0 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc8727fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc874100780 0x7fc87419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc8727fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc874100780 0x7fc87419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60594/0 (socket says 172.21.15.49:60594) 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc8727fc700 1 -- 172.21.15.49:0/929101182 learned_addr learned my addr 172.21.15.49:0/929101182 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc872ffd700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740ffdd0 0x7fc87419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc8727fc700 1 -- 172.21.15.49:0/929101182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740ffdd0 msgr2=0x7fc87419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc8727fc700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740ffdd0 0x7fc87419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc8727fc700 1 -- 172.21.15.49:0/929101182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640095f0 con 0x7fc874100780 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc872ffd700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740ffdd0 0x7fc87419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:03:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.783+0000 7fc8727fc700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc874100780 0x7fc87419cdb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00d900 tx=0x7fc85c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.784+0000 7fc86bfff700 1 -- 172.21.15.49:0/929101182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc85c009900 con 0x7fc874100780 2024-05-15T16:03:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.784+0000 7fc86bfff700 1 -- 172.21.15.49:0/929101182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc85c010460 con 0x7fc874100780 2024-05-15T16:03:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.784+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc874196c90 con 0x7fc874100780 2024-05-15T16:03:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.784+0000 7fc86bfff700 1 -- 172.21.15.49:0/929101182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc85c00f5f0 con 0x7fc874100780 2024-05-15T16:03:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.784+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741971e0 con 0x7fc874100780 2024-05-15T16:03:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.785+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874108170 con 0x7fc874100780 2024-05-15T16:03:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.785+0000 7fc86bfff700 1 -- 172.21.15.49:0/929101182 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc85c0105d0 con 0x7fc874100780 2024-05-15T16:03:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.786+0000 7fc86bfff700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc86006e200 0x7fc8600706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.786+0000 7fc86bfff700 1 -- 172.21.15.49:0/929101182 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc85c08de10 con 0x7fc874100780 2024-05-15T16:03:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.786+0000 7fc872ffd700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc86006e200 0x7fc8600706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.787+0000 7fc872ffd700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc86006e200 0x7fc8600706c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8640059d0 tx=0x7fc86401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:00.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:00.791+0000 7fc86bfff700 1 -- 172.21.15.49:0/929101182 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc85c05ac20 con 0x7fc874100780 2024-05-15T16:03:00.920 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:00 smithi049 ceph-mon[112142]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:00.920 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:00 smithi049 ceph-mon[112142]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:00.920 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:00 smithi049 ceph-mon[112142]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:00.920 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:00 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3431431350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:00.921 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:00 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3741962821' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:03:01.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.025+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc874197e70 con 0x7fc86006e200 2024-05-15T16:03:01.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.027+0000 7fc86bfff700 1 -- 172.21.15.49:0/929101182 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc874197e70 con 0x7fc86006e200 2024-05-15T16:03:01.029 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:03:01.029 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:03:01.029 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "12/23 daemons upgraded", 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:03:01.030 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:03:01.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc86006e200 msgr2=0x7fc8600706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:01.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc86006e200 0x7fc8600706c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8640059d0 tx=0x7fc86401a040 comp rx=0 tx=0).stop 2024-05-15T16:03:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc874100780 msgr2=0x7fc87419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc874100780 0x7fc87419cdb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00d900 tx=0x7fc85c00dcc0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 shutdown_connections 2024-05-15T16:03:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc86006e200 0x7fc8600706c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740ffdd0 0x7fc87419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 --2- 172.21.15.49:0/929101182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc874100780 0x7fc87419cdb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 >> 172.21.15.49:0/929101182 conn(0x7fc8740fb6a0 msgr2=0x7fc874109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 shutdown_connections 2024-05-15T16:03:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.029+0000 7fc87931d700 1 -- 172.21.15.49:0/929101182 wait complete. 2024-05-15T16:03:01.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.139+0000 7fe330bce700 1 -- 172.21.15.49:0/891582243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 msgr2=0x7fe32c1000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:01.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.139+0000 7fe330bce700 1 --2- 172.21.15.49:0/891582243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 0x7fe32c1000e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe320009910 tx=0x7fe320009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:01.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.139+0000 7fe330bce700 1 -- 172.21.15.49:0/891582243 shutdown_connections 2024-05-15T16:03:01.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.139+0000 7fe330bce700 1 --2- 172.21.15.49:0/891582243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 0x7fe32c1000e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.139+0000 7fe330bce700 1 --2- 172.21.15.49:0/891582243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c105b80 0x7fe32c105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.139+0000 7fe330bce700 1 -- 172.21.15.49:0/891582243 >> 172.21.15.49:0/891582243 conn(0x7fe32c076850 msgr2=0x7fe32c076c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:01.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.139+0000 7fe330bce700 1 -- 172.21.15.49:0/891582243 shutdown_connections 2024-05-15T16:03:01.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.139+0000 7fe330bce700 1 -- 172.21.15.49:0/891582243 wait complete. 2024-05-15T16:03:01.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.140+0000 7fe330bce700 1 Processor -- start 2024-05-15T16:03:01.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.140+0000 7fe330bce700 1 -- start start 2024-05-15T16:03:01.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.140+0000 7fe330bce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 0x7fe32c101750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:01.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.140+0000 7fe330bce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c105b80 0x7fe32c101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:01.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.140+0000 7fe330bce700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1021d0 con 0x7fe32c0ffc80 2024-05-15T16:03:01.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.140+0000 7fe330bce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c102310 con 0x7fe32c105b80 2024-05-15T16:03:01.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.140+0000 7fe32a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 0x7fe32c101750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:01.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.140+0000 7fe32a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 0x7fe32c101750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60612/0 (socket says 172.21.15.49:60612) 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.140+0000 7fe32a59c700 1 -- 172.21.15.49:0/2544633936 learned_addr learned my addr 172.21.15.49:0/2544633936 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.141+0000 7fe329d9b700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c105b80 0x7fe32c101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.141+0000 7fe32a59c700 1 -- 172.21.15.49:0/2544633936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c105b80 msgr2=0x7fe32c101c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.141+0000 7fe32a59c700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c105b80 0x7fe32c101c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.141+0000 7fe32a59c700 1 -- 172.21.15.49:0/2544633936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3200095f0 con 0x7fe32c0ffc80 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.141+0000 7fe32a59c700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 0x7fe32c101750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d8d0 tx=0x7fe31c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.141+0000 7fe31b7fe700 1 -- 172.21.15.49:0/2544633936 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe31c0098a0 con 0x7fe32c0ffc80 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.141+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c1a2d90 con 0x7fe32c0ffc80 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.141+0000 7fe31b7fe700 1 -- 172.21.15.49:0/2544633936 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe31c010460 con 0x7fe32c0ffc80 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.141+0000 7fe31b7fe700 1 -- 172.21.15.49:0/2544633936 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe31c00f5f0 con 0x7fe32c0ffc80 2024-05-15T16:03:01.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.142+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c1a30f0 con 0x7fe32c0ffc80 2024-05-15T16:03:01.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.143+0000 7fe31b7fe700 1 -- 172.21.15.49:0/2544633936 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe31c009a00 con 0x7fe32c0ffc80 2024-05-15T16:03:01.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.143+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c109090 con 0x7fe32c0ffc80 2024-05-15T16:03:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.143+0000 7fe31b7fe700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe31406e200 0x7fe3140706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.144+0000 7fe31b7fe700 1 -- 172.21.15.49:0/2544633936 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe31c08e030 con 0x7fe32c0ffc80 2024-05-15T16:03:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.144+0000 7fe329d9b700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe31406e200 0x7fe3140706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.144+0000 7fe329d9b700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe31406e200 0x7fe3140706c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe32c102e80 tx=0x7fe32001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:01.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.148+0000 7fe31b7fe700 1 -- 172.21.15.49:0/2544633936 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe31c057310 con 0x7fe32c0ffc80 2024-05-15T16:03:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:00 smithi144 ceph-mon[86635]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:00 smithi144 ceph-mon[86635]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:00 smithi144 ceph-mon[86635]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:00 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3431431350' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:00 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3741962821' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:03:01.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.444+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe32c102cd0 con 0x7fe32c0ffc80 2024-05-15T16:03:01.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.445+0000 7fe31b7fe700 1 -- 172.21.15.49:0/2544633936 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fe31c05a9d0 con 0x7fe32c0ffc80 2024-05-15T16:03:01.447 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:03:01.447 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:03:01.447 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:03:01.447 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:03:01.448 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:03:01.448 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:03:01.448 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:03:01.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe31406e200 msgr2=0x7fe3140706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:01.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe31406e200 0x7fe3140706c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe32c102e80 tx=0x7fe32001a040 comp rx=0 tx=0).stop 2024-05-15T16:03:01.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 msgr2=0x7fe32c101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:01.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 0x7fe32c101750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d8d0 tx=0x7fe31c00dbe0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 shutdown_connections 2024-05-15T16:03:01.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe31406e200 0x7fe3140706c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe32c0ffc80 0x7fe32c101750 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 --2- 172.21.15.49:0/2544633936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c105b80 0x7fe32c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 >> 172.21.15.49:0/2544633936 conn(0x7fe32c076850 msgr2=0x7fe32c068ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 shutdown_connections 2024-05-15T16:03:01.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:01.447+0000 7fe330bce700 1 -- 172.21.15.49:0/2544633936 wait complete. 2024-05-15T16:03:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:01 smithi144 ceph-mon[86635]: pgmap v176: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 1 op/s 2024-05-15T16:03:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:01 smithi144 ceph-mon[86635]: osdmap e84: 6 total, 6 up, 6 in 2024-05-15T16:03:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:01 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-15T16:03:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:01 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T16:03:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:01 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2544633936' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:03:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:01 smithi049 ceph-mon[112142]: pgmap v176: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 1 op/s 2024-05-15T16:03:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:01 smithi049 ceph-mon[112142]: osdmap e84: 6 total, 6 up, 6 in 2024-05-15T16:03:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:01 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-15T16:03:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:01 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T16:03:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:01 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2544633936' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:03:03.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:02 smithi144 ceph-mon[86635]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:03.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:02 smithi049 ceph-mon[112142]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:04.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:03 smithi144 ceph-mon[86635]: pgmap v178: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-15T16:03:04.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:03 smithi049 ceph-mon[112142]: pgmap v178: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-15T16:03:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:05 smithi049 ceph-mon[112142]: pgmap v179: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-15T16:03:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:03:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:05 smithi144 ceph-mon[86635]: pgmap v179: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-15T16:03:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:03:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:07.577 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:07 smithi049 ceph-mon[112142]: pgmap v180: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-15T16:03:07.577 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:07 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:boot 2024-05-15T16:03:07.578 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:07 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:03:07.578 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:07 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T16:03:07.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:07 smithi144 ceph-mon[86635]: pgmap v180: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-15T16:03:07.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:07 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:boot 2024-05-15T16:03:07.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:07 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:03:07.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:07 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T16:03:09.494 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:09 smithi049 ceph-mon[112142]: pgmap v181: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-05-15T16:03:09.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:09 smithi144 ceph-mon[86635]: pgmap v181: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-05-15T16:03:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:11 smithi144 ceph-mon[86635]: pgmap v182: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:12.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:11 smithi049 ceph-mon[112142]: pgmap v182: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:12.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:12.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:13.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:13 smithi144 ceph-mon[86635]: pgmap v183: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:13.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:13.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:13.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:13 smithi049 ceph-mon[112142]: pgmap v183: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:15.515 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:15 smithi049 ceph-mon[112142]: pgmap v184: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:15.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:15 smithi144 ceph-mon[86635]: pgmap v184: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:16.643 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:16.643 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:16.643 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:03:16.643 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:03:16.643 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:16.643 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:16.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:16.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:16.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:16.644 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:16 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:16.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:16.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:16.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:03:16.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:03:16.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:16.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:16.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:16.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:16.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:16.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:16 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:17.918 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:17 smithi049 ceph-mon[112142]: pgmap v185: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:17.918 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:17.919 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.gocadf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T16:03:17.919 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:17 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:03:17.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:17 smithi144 ceph-mon[86635]: pgmap v185: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:17.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:17.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi049.gocadf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T16:03:17.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:17 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:03:18.869 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:18 smithi144 ceph-mon[86635]: Upgrade: Updating mds.cephfs.smithi049.gocadf 2024-05-15T16:03:18.869 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:18 smithi144 ceph-mon[86635]: Deploying daemon mds.cephfs.smithi049.gocadf on smithi049 2024-05-15T16:03:18.912 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:18 smithi049 ceph-mon[112142]: Upgrade: Updating mds.cephfs.smithi049.gocadf 2024-05-15T16:03:18.912 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:18 smithi049 ceph-mon[112142]: Deploying daemon mds.cephfs.smithi049.gocadf on smithi049 2024-05-15T16:03:19.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:19 smithi144 ceph-mon[86635]: pgmap v186: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:19 smithi049 ceph-mon[112142]: pgmap v186: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:20 smithi144 ceph-mon[86635]: osdmap e85: 6 total, 6 up, 6 in 2024-05-15T16:03:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:20 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-15T16:03:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:03:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:20 smithi049 ceph-mon[112142]: osdmap e85: 6 total, 6 up, 6 in 2024-05-15T16:03:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:20 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-15T16:03:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:03:21.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:21 smithi144 ceph-mon[86635]: pgmap v188: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:21.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:21 smithi049 ceph-mon[112142]: pgmap v188: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:22.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:22 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T16:03:22.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:22 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T16:03:23.768 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:23 smithi049 ceph-mon[112142]: pgmap v189: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:23.922 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:23 smithi144 ceph-mon[86635]: pgmap v189: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:25.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:25 smithi144 ceph-mon[86635]: pgmap v190: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:25.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:25 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:25.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:25 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:25.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:25 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:25.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:25 smithi049 ceph-mon[112142]: pgmap v190: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:25.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:25.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:25.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:26.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:26 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:boot 2024-05-15T16:03:26.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:26 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:03:26.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:26 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T16:03:26.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:26 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:boot 2024-05-15T16:03:26.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:26 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:03:26.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:26 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T16:03:27.808 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:27 smithi049 ceph-mon[112142]: pgmap v191: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:27.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:27 smithi144 ceph-mon[86635]: pgmap v191: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:29.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:29 smithi144 ceph-mon[86635]: pgmap v192: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:29.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:29 smithi049 ceph-mon[112142]: pgmap v192: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.562+0000 7f301e51c700 1 -- 172.21.15.49:0/3806035831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180ffdd0 msgr2=0x7f30181001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.562+0000 7f301e51c700 1 --2- 172.21.15.49:0/3806035831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180ffdd0 0x7f30181001b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3008009910 tx=0x7f3008009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.562+0000 7f301e51c700 1 -- 172.21.15.49:0/3806035831 shutdown_connections 2024-05-15T16:03:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.562+0000 7f301e51c700 1 --2- 172.21.15.49:0/3806035831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3018100780 0x7f30181047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.562+0000 7f301e51c700 1 --2- 172.21.15.49:0/3806035831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180ffdd0 0x7f30181001b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.562+0000 7f301e51c700 1 -- 172.21.15.49:0/3806035831 >> 172.21.15.49:0/3806035831 conn(0x7f30180fb6a0 msgr2=0x7f30180fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.562+0000 7f301e51c700 1 -- 172.21.15.49:0/3806035831 shutdown_connections 2024-05-15T16:03:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.562+0000 7f301e51c700 1 -- 172.21.15.49:0/3806035831 wait complete. 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.563+0000 7f301e51c700 1 Processor -- start 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.563+0000 7f301e51c700 1 -- start start 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.563+0000 7f301e51c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30180ffdd0 0x7f301819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.563+0000 7f301e51c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3018100780 0x7f301819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.563+0000 7f301e51c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301819d3d0 con 0x7f3018100780 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.563+0000 7f301e51c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181969b0 con 0x7f30180ffdd0 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.563+0000 7f30177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3018100780 0x7f301819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.563+0000 7f30177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3018100780 0x7f301819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38910/0 (socket says 172.21.15.49:38910) 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.563+0000 7f30177fe700 1 -- 172.21.15.49:0/1793071751 learned_addr learned my addr 172.21.15.49:0/1793071751 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f3017fff700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30180ffdd0 0x7f301819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f30177fe700 1 -- 172.21.15.49:0/1793071751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30180ffdd0 msgr2=0x7f301819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f30177fe700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30180ffdd0 0x7f301819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f30177fe700 1 -- 172.21.15.49:0/1793071751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080095f0 con 0x7f3018100780 2024-05-15T16:03:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f30177fe700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3018100780 0x7f301819cdb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f300000d900 tx=0x7f300000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f30157fa700 1 -- 172.21.15.49:0/1793071751 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3000009900 con 0x7f3018100780 2024-05-15T16:03:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f30157fa700 1 -- 172.21.15.49:0/1793071751 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3000010460 con 0x7f3018100780 2024-05-15T16:03:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3018196c90 con 0x7f3018100780 2024-05-15T16:03:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30181971e0 con 0x7f3018100780 2024-05-15T16:03:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.564+0000 7f30157fa700 1 -- 172.21.15.49:0/1793071751 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f300000f5f0 con 0x7f3018100780 2024-05-15T16:03:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.565+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3018108170 con 0x7f3018100780 2024-05-15T16:03:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.566+0000 7f30157fa700 1 -- 172.21.15.49:0/1793071751 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f300000fa80 con 0x7f3018100780 2024-05-15T16:03:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.566+0000 7f30157fa700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f300406e210 0x7f30040706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.566+0000 7f3017fff700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f300406e210 0x7f30040706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.566+0000 7f30157fa700 1 -- 172.21.15.49:0/1793071751 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f300008e050 con 0x7f3018100780 2024-05-15T16:03:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.567+0000 7f3017fff700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f300406e210 0x7f30040706d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f300800b5c0 tx=0x7f300801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:31.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.571+0000 7f30157fa700 1 -- 172.21.15.49:0/1793071751 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f300005ae60 con 0x7f3018100780 2024-05-15T16:03:31.820 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:31 smithi049 ceph-mon[112142]: pgmap v193: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:31.820 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:31 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:31.821 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:31 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:31.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.817+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3018197ee0 con 0x7f300406e210 2024-05-15T16:03:31.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.819+0000 7f30157fa700 1 -- 172.21.15.49:0/1793071751 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3018197ee0 con 0x7f300406e210 2024-05-15T16:03:31.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f300406e210 msgr2=0x7f30040706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:31.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f300406e210 0x7f30040706d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f300800b5c0 tx=0x7f300801a040 comp rx=0 tx=0).stop 2024-05-15T16:03:31.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3018100780 msgr2=0x7f301819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:31.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3018100780 0x7f301819cdb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f300000d900 tx=0x7f300000dcc0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 shutdown_connections 2024-05-15T16:03:31.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f300406e210 0x7f30040706d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30180ffdd0 0x7f301819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 --2- 172.21.15.49:0/1793071751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3018100780 0x7f301819cdb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 >> 172.21.15.49:0/1793071751 conn(0x7f30180fb6a0 msgr2=0x7f3018109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:31.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 shutdown_connections 2024-05-15T16:03:31.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.821+0000 7f301e51c700 1 -- 172.21.15.49:0/1793071751 wait complete. 2024-05-15T16:03:31.835 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:03:31.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.934+0000 7fc3d54a3700 1 -- 172.21.15.49:0/1704187567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ff470 msgr2=0x7fc3d00ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:31.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.934+0000 7fc3d54a3700 1 --2- 172.21.15.49:0/1704187567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ff470 0x7fc3d00ff850 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009910 tx=0x7fc3c0009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:31.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.934+0000 7fc3d54a3700 1 -- 172.21.15.49:0/1704187567 shutdown_connections 2024-05-15T16:03:31.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.934+0000 7fc3d54a3700 1 --2- 172.21.15.49:0/1704187567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffe20 0x7fc3d0105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.934+0000 7fc3d54a3700 1 --2- 172.21.15.49:0/1704187567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ff470 0x7fc3d00ff850 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.934+0000 7fc3d54a3700 1 -- 172.21.15.49:0/1704187567 >> 172.21.15.49:0/1704187567 conn(0x7fc3d0075fc0 msgr2=0x7fc3d00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.934+0000 7fc3d54a3700 1 -- 172.21.15.49:0/1704187567 shutdown_connections 2024-05-15T16:03:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.934+0000 7fc3d54a3700 1 -- 172.21.15.49:0/1704187567 wait complete. 2024-05-15T16:03:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.935+0000 7fc3d54a3700 1 Processor -- start 2024-05-15T16:03:31.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.935+0000 7fc3d54a3700 1 -- start start 2024-05-15T16:03:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.935+0000 7fc3d54a3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ff470 0x7fc3d01003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.935+0000 7fc3d54a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ffe20 0x7fc3d0100930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:31.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.935+0000 7fc3d54a3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d0100e70 con 0x7fc3d00ffe20 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.935+0000 7fc3d54a3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d0100fb0 con 0x7fc3d00ff470 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.935+0000 7fc3ce7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ffe20 0x7fc3d0100930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.935+0000 7fc3ce7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ffe20 0x7fc3d0100930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38934/0 (socket says 172.21.15.49:38934) 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.935+0000 7fc3ce7fc700 1 -- 172.21.15.49:0/2287779323 learned_addr learned my addr 172.21.15.49:0/2287779323 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3ceffd700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ff470 0x7fc3d01003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3ce7fc700 1 -- 172.21.15.49:0/2287779323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ff470 msgr2=0x7fc3d01003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3ce7fc700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ff470 0x7fc3d01003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3ce7fc700 1 -- 172.21.15.49:0/2287779323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c00095f0 con 0x7fc3d00ffe20 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3ce7fc700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ffe20 0x7fc3d0100930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400d8d0 tx=0x7fc3c400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3b7fff700 1 -- 172.21.15.49:0/2287779323 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c40041f0 con 0x7fc3d00ffe20 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d01a2df0 con 0x7fc3d00ffe20 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3b7fff700 1 -- 172.21.15.49:0/2287779323 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3c4007620 con 0x7fc3d00ffe20 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3b7fff700 1 -- 172.21.15.49:0/2287779323 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c40056b0 con 0x7fc3d00ffe20 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.936+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d01a3150 con 0x7fc3d00ffe20 2024-05-15T16:03:31.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.937+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d0109300 con 0x7fc3d00ffe20 2024-05-15T16:03:31.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.938+0000 7fc3b7fff700 1 -- 172.21.15.49:0/2287779323 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc3c4004750 con 0x7fc3d00ffe20 2024-05-15T16:03:31.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.938+0000 7fc3b7fff700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3b806e200 0x7fc3b80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:31.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.938+0000 7fc3b7fff700 1 -- 172.21.15.49:0/2287779323 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc3c408df20 con 0x7fc3d00ffe20 2024-05-15T16:03:31.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.938+0000 7fc3ceffd700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3b806e200 0x7fc3b80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:31.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.939+0000 7fc3ceffd700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3b806e200 0x7fc3b80706c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0000c00 tx=0x7fc3c001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:31.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:31.943+0000 7fc3b7fff700 1 -- 172.21.15.49:0/2287779323 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc3c40c2850 con 0x7fc3d00ffe20 2024-05-15T16:03:31.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:31 smithi144 ceph-mon[86635]: pgmap v193: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:31.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:31 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:31.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:31 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:32.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.178+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3d0102500 con 0x7fc3b806e200 2024-05-15T16:03:32.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.179+0000 7fc3b7fff700 1 -- 172.21.15.49:0/2287779323 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc3d0102500 con 0x7fc3b806e200 2024-05-15T16:03:32.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3b806e200 msgr2=0x7fc3b80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3b806e200 0x7fc3b80706c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0000c00 tx=0x7fc3c001a040 comp rx=0 tx=0).stop 2024-05-15T16:03:32.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ffe20 msgr2=0x7fc3d0100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ffe20 0x7fc3d0100930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400d8d0 tx=0x7fc3c400dbe0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 shutdown_connections 2024-05-15T16:03:32.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3b806e200 0x7fc3b80706c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ff470 0x7fc3d01003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 --2- 172.21.15.49:0/2287779323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d00ffe20 0x7fc3d0100930 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 >> 172.21.15.49:0/2287779323 conn(0x7fc3d0075fc0 msgr2=0x7fc3d0104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:32.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 shutdown_connections 2024-05-15T16:03:32.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.182+0000 7fc3d54a3700 1 -- 172.21.15.49:0/2287779323 wait complete. 2024-05-15T16:03:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.304+0000 7f304d137700 1 -- 172.21.15.49:0/1360279761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 msgr2=0x7f30481012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.304+0000 7f304d137700 1 --2- 172.21.15.49:0/1360279761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 0x7f30481012c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3038009910 tx=0x7f3038009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.305+0000 7f304d137700 1 -- 172.21.15.49:0/1360279761 shutdown_connections 2024-05-15T16:03:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.305+0000 7f304d137700 1 --2- 172.21.15.49:0/1360279761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3048101890 0x7f30481058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.305+0000 7f304d137700 1 --2- 172.21.15.49:0/1360279761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 0x7f30481012c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.305+0000 7f304d137700 1 -- 172.21.15.49:0/1360279761 >> 172.21.15.49:0/1360279761 conn(0x7f30480fc790 msgr2=0x7f30480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.305+0000 7f304d137700 1 -- 172.21.15.49:0/1360279761 shutdown_connections 2024-05-15T16:03:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.305+0000 7f304d137700 1 -- 172.21.15.49:0/1360279761 wait complete. 2024-05-15T16:03:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.305+0000 7f304d137700 1 Processor -- start 2024-05-15T16:03:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f304d137700 1 -- start start 2024-05-15T16:03:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f304d137700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 0x7f30480732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f304d137700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3048101890 0x7f30480737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f304d137700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3048073d20 con 0x7f3048100ee0 2024-05-15T16:03:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f304d137700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3048073e60 con 0x7f3048101890 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f3046d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 0x7f30480732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f3046d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 0x7f30480732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38942/0 (socket says 172.21.15.49:38942) 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f3046d9d700 1 -- 172.21.15.49:0/2250527285 learned_addr learned my addr 172.21.15.49:0/2250527285 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f304659c700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3048101890 0x7f30480737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f3046d9d700 1 -- 172.21.15.49:0/2250527285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3048101890 msgr2=0x7f30480737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f3046d9d700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3048101890 0x7f30480737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f3046d9d700 1 -- 172.21.15.49:0/2250527285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30380095f0 con 0x7f3048100ee0 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.306+0000 7f3046d9d700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 0x7f30480732a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f303800bd70 tx=0x7f303800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.307+0000 7f302ffff700 1 -- 172.21.15.49:0/2250527285 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303801d070 con 0x7f3048100ee0 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.307+0000 7f302ffff700 1 -- 172.21.15.49:0/2250527285 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3038005700 con 0x7f3048100ee0 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.307+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30480740e0 con 0x7f3048100ee0 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.307+0000 7f302ffff700 1 -- 172.21.15.49:0/2250527285 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3038017400 con 0x7f3048100ee0 2024-05-15T16:03:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.307+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30481a3040 con 0x7f3048100ee0 2024-05-15T16:03:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.308+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3048109280 con 0x7f3048100ee0 2024-05-15T16:03:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.308+0000 7f302ffff700 1 -- 172.21.15.49:0/2250527285 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3038003a60 con 0x7f3048100ee0 2024-05-15T16:03:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.309+0000 7f302ffff700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f303006e200 0x7f30300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.309+0000 7f302ffff700 1 -- 172.21.15.49:0/2250527285 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f303808f600 con 0x7f3048100ee0 2024-05-15T16:03:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.309+0000 7f304659c700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f303006e200 0x7f30300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.309+0000 7f304659c700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f303006e200 0x7f30300706c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3048074980 tx=0x7f303c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:32.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.314+0000 7f302ffff700 1 -- 172.21.15.49:0/2250527285 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f30380588e0 con 0x7f3048100ee0 2024-05-15T16:03:32.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.537+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f30480747f0 con 0x7f303006e200 2024-05-15T16:03:32.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.548+0000 7f302ffff700 1 -- 172.21.15.49:0/2250527285 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f30480747f0 con 0x7f303006e200 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (6m) 2s ago 18m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (18m) 2s ago 18m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (17m) 43s ago 17m 9013k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (4m) 2s ago 18m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (4m) 43s ago 17m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (6m) 2s ago 18m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (27s) 2s ago 12m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (7s) 2s ago 12m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (12m) 43s ago 12m 26.7M - 18.2.2-1343-g2965081f a04a73ffe079 9d66ece8f8dc 2024-05-15T16:03:32.551 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (12m) 43s ago 12m 28.7M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (9m) 2s ago 19m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (7m) 43s ago 17m 453M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (5m) 2s ago 19m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (4m) 43s ago 17m 40.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (7m) 2s ago 18m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (7m) 43s ago 17m 15.7M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (3m) 2s ago 16m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (3m) 2s ago 16m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (2m) 2s ago 15m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (114s) 43s ago 15m 98.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (81s) 43s ago 14m 85.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:03:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (47s) 43s ago 14m 14.9M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:03:32.553 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (7m) 2s ago 17m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:03:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f303006e200 msgr2=0x7f30300706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f303006e200 0x7f30300706c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3048074980 tx=0x7f303c00a300 comp rx=0 tx=0).stop 2024-05-15T16:03:32.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 msgr2=0x7f30480732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 0x7f30480732a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f303800bd70 tx=0x7f303800bda0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 shutdown_connections 2024-05-15T16:03:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f303006e200 0x7f30300706c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3048100ee0 0x7f30480732a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 --2- 172.21.15.49:0/2250527285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3048101890 0x7f30480737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 >> 172.21.15.49:0/2250527285 conn(0x7f30480fc790 msgr2=0x7f30480fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 shutdown_connections 2024-05-15T16:03:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.550+0000 7f304d137700 1 -- 172.21.15.49:0/2250527285 wait complete. 2024-05-15T16:03:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.661+0000 7f91839b8700 1 -- 172.21.15.49:0/3124314681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c101890 msgr2=0x7f917c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.661+0000 7f91839b8700 1 --2- 172.21.15.49:0/3124314681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c101890 0x7f917c1058e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9170009910 tx=0x7f9170009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.661+0000 7f91839b8700 1 -- 172.21.15.49:0/3124314681 shutdown_connections 2024-05-15T16:03:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.661+0000 7f91839b8700 1 --2- 172.21.15.49:0/3124314681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c101890 0x7f917c1058e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.661+0000 7f91839b8700 1 --2- 172.21.15.49:0/3124314681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917c100ee0 0x7f917c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.661+0000 7f91839b8700 1 -- 172.21.15.49:0/3124314681 >> 172.21.15.49:0/3124314681 conn(0x7f917c0fc790 msgr2=0x7f917c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.661+0000 7f91839b8700 1 -- 172.21.15.49:0/3124314681 shutdown_connections 2024-05-15T16:03:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.661+0000 7f91839b8700 1 -- 172.21.15.49:0/3124314681 wait complete. 2024-05-15T16:03:32.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f91839b8700 1 Processor -- start 2024-05-15T16:03:32.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f91839b8700 1 -- start start 2024-05-15T16:03:32.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f91839b8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c100ee0 0x7f917c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:32.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f91839b8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917c101890 0x7f917c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:32.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f91839b8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c073d20 con 0x7f917c100ee0 2024-05-15T16:03:32.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f91839b8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c073e60 con 0x7f917c101890 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f9181754700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c100ee0 0x7f917c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f9181754700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c100ee0 0x7f917c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38958/0 (socket says 172.21.15.49:38958) 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f9181754700 1 -- 172.21.15.49:0/4099248839 learned_addr learned my addr 172.21.15.49:0/4099248839 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f9180f53700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917c101890 0x7f917c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f9181754700 1 -- 172.21.15.49:0/4099248839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917c101890 msgr2=0x7f917c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f9181754700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917c101890 0x7f917c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.662+0000 7f9181754700 1 -- 172.21.15.49:0/4099248839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91700095f0 con 0x7f917c100ee0 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.663+0000 7f9181754700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c100ee0 0x7f917c0732a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f917800d8d0 tx=0x7f917800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.663+0000 7f916e7fc700 1 -- 172.21.15.49:0/4099248839 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91780098a0 con 0x7f917c100ee0 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.663+0000 7f916e7fc700 1 -- 172.21.15.49:0/4099248839 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9178010460 con 0x7f917c100ee0 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.663+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c1a2da0 con 0x7f917c100ee0 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.663+0000 7f916e7fc700 1 -- 172.21.15.49:0/4099248839 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f917800f5f0 con 0x7f917c100ee0 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.663+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c1a3100 con 0x7f917c100ee0 2024-05-15T16:03:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.664+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917c1092e0 con 0x7f917c100ee0 2024-05-15T16:03:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.664+0000 7f916e7fc700 1 -- 172.21.15.49:0/4099248839 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9178009a00 con 0x7f917c100ee0 2024-05-15T16:03:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.665+0000 7f916e7fc700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f916806e200 0x7f91680706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.665+0000 7f916e7fc700 1 -- 172.21.15.49:0/4099248839 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f917808df80 con 0x7f917c100ee0 2024-05-15T16:03:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.665+0000 7f9180f53700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f916806e200 0x7f91680706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.665+0000 7f9180f53700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f916806e200 0x7f91680706c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9170000c00 tx=0x7f917001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.669+0000 7f916e7fc700 1 -- 172.21.15.49:0/4099248839 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f91780572e0 con 0x7f917c100ee0 2024-05-15T16:03:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:32 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:32 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:32.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:32 smithi144 ceph-mon[86635]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:32.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:32.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:32 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:32.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:32 smithi049 ceph-mon[112142]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.967+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f917c074820 con 0x7f917c100ee0 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.968+0000 7f916e7fc700 1 -- 172.21.15.49:0/4099248839 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f917805a9a0 con 0x7f917c100ee0 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:03:32.970 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2, 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 2, 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 12 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:03:32.971 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f916806e200 msgr2=0x7f91680706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f916806e200 0x7f91680706c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9170000c00 tx=0x7f917001a040 comp rx=0 tx=0).stop 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c100ee0 msgr2=0x7f917c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c100ee0 0x7f917c0732a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f917800d8d0 tx=0x7f917800dbe0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 shutdown_connections 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f916806e200 0x7f91680706c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f917c100ee0 0x7f917c0732a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 --2- 172.21.15.49:0/4099248839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917c101890 0x7f917c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 >> 172.21.15.49:0/4099248839 conn(0x7f917c0fc790 msgr2=0x7f917c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 shutdown_connections 2024-05-15T16:03:32.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:32.971+0000 7f91839b8700 1 -- 172.21.15.49:0/4099248839 wait complete. 2024-05-15T16:03:33.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.084+0000 7f3f896a8700 1 -- 172.21.15.49:0/4002115223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f840ff690 msgr2=0x7f3f840ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.084+0000 7f3f896a8700 1 --2- 172.21.15.49:0/4002115223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f840ff690 0x7f3f840ffa70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009910 tx=0x7f3f74009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:33.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.085+0000 7f3f896a8700 1 -- 172.21.15.49:0/4002115223 shutdown_connections 2024-05-15T16:03:33.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.085+0000 7f3f896a8700 1 --2- 172.21.15.49:0/4002115223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f84100040 0x7f3f84104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.085+0000 7f3f896a8700 1 --2- 172.21.15.49:0/4002115223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f840ff690 0x7f3f840ffa70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.085+0000 7f3f896a8700 1 -- 172.21.15.49:0/4002115223 >> 172.21.15.49:0/4002115223 conn(0x7f3f84074eb0 msgr2=0x7f3f840752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:33.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.085+0000 7f3f896a8700 1 -- 172.21.15.49:0/4002115223 shutdown_connections 2024-05-15T16:03:33.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.085+0000 7f3f896a8700 1 -- 172.21.15.49:0/4002115223 wait complete. 2024-05-15T16:03:33.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.085+0000 7f3f896a8700 1 Processor -- start 2024-05-15T16:03:33.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.085+0000 7f3f896a8700 1 -- start start 2024-05-15T16:03:33.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f896a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f840ff690 0x7f3f8419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:33.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f896a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f84100040 0x7f3f8419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:33.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f896a8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8419d360 con 0x7f3f84100040 2024-05-15T16:03:33.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f896a8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84196a20 con 0x7f3f840ff690 2024-05-15T16:03:33.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f827fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f84100040 0x7f3f8419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:33.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f827fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f84100040 0x7f3f8419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38986/0 (socket says 172.21.15.49:38986) 2024-05-15T16:03:33.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f827fc700 1 -- 172.21.15.49:0/2726759402 learned_addr learned my addr 172.21.15.49:0/2726759402 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:33.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f82ffd700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f840ff690 0x7f3f8419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:33.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f827fc700 1 -- 172.21.15.49:0/2726759402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f840ff690 msgr2=0x7f3f8419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f827fc700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f840ff690 0x7f3f8419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f827fc700 1 -- 172.21.15.49:0/2726759402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740095f0 con 0x7f3f84100040 2024-05-15T16:03:33.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f827fc700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f84100040 0x7f3f8419cdd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c930 tx=0x7f3f7800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:33.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.086+0000 7f3f6bfff700 1 -- 172.21.15.49:0/2726759402 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f78007ad0 con 0x7f3f84100040 2024-05-15T16:03:33.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.087+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f84196d00 con 0x7f3f84100040 2024-05-15T16:03:33.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.087+0000 7f3f6bfff700 1 -- 172.21.15.49:0/2726759402 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f7800ce80 con 0x7f3f84100040 2024-05-15T16:03:33.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.087+0000 7f3f6bfff700 1 -- 172.21.15.49:0/2726759402 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f78018630 con 0x7f3f84100040 2024-05-15T16:03:33.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.087+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f84197250 con 0x7f3f84100040 2024-05-15T16:03:33.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.088+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f84109300 con 0x7f3f84100040 2024-05-15T16:03:33.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.088+0000 7f3f6bfff700 1 -- 172.21.15.49:0/2726759402 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3f78007c30 con 0x7f3f84100040 2024-05-15T16:03:33.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.089+0000 7f3f6bfff700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3f6c06e200 0x7f3f6c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:33.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.089+0000 7f3f6bfff700 1 -- 172.21.15.49:0/2726759402 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3f7808e1e0 con 0x7f3f84100040 2024-05-15T16:03:33.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.089+0000 7f3f82ffd700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3f6c06e200 0x7f3f6c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:33.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.089+0000 7f3f82ffd700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3f6c06e200 0x7f3f6c0706c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3f74006010 tx=0x7f3f7401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:33.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.093+0000 7f3f6bfff700 1 -- 172.21.15.49:0/2726759402 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3f7805aff0 con 0x7f3f84100040 2024-05-15T16:03:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.345+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3f84068ee0 con 0x7f3f84100040 2024-05-15T16:03:33.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.346+0000 7f3f6bfff700 1 -- 172.21.15.49:0/2726759402 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1870 (secure 0 0 0) 0x7f3f7805ab80 con 0x7f3f84100040 2024-05-15T16:03:33.348 INFO:teuthology.orchestra.run.smithi049.stdout:e24 2024-05-15T16:03:33.348 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:03:33.348 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:03:33.349 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:03:33.350 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:03:33.351 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/1712325654,v1:172.21.15.144:6825/1712325654] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:03:33.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.348+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3f6c06e200 msgr2=0x7f3f6c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.348+0000 7f3f896a8700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3f6c06e200 0x7f3f6c0706c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3f74006010 tx=0x7f3f7401a040 comp rx=0 tx=0).stop 2024-05-15T16:03:33.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.348+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f84100040 msgr2=0x7f3f8419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.348+0000 7f3f896a8700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f84100040 0x7f3f8419cdd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c930 tx=0x7f3f7800cc40 comp rx=0 tx=0).stop 2024-05-15T16:03:33.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.348+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 shutdown_connections 2024-05-15T16:03:33.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.348+0000 7f3f896a8700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3f6c06e200 0x7f3f6c0706c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.348+0000 7f3f896a8700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f840ff690 0x7f3f8419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.348+0000 7f3f896a8700 1 --2- 172.21.15.49:0/2726759402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f84100040 0x7f3f8419cdd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.349+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 >> 172.21.15.49:0/2726759402 conn(0x7f3f84074eb0 msgr2=0x7f3f84102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:33.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.349+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 shutdown_connections 2024-05-15T16:03:33.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.349+0000 7f3f896a8700 1 -- 172.21.15.49:0/2726759402 wait complete. 2024-05-15T16:03:33.353 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-05-15T16:03:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.460+0000 7f7e84832700 1 -- 172.21.15.49:0/17501163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80102740 msgr2=0x7f7e8010ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.460+0000 7f7e84832700 1 --2- 172.21.15.49:0/17501163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80102740 0x7f7e8010ac30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009910 tx=0x7f7e68009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.460+0000 7f7e84832700 1 -- 172.21.15.49:0/17501163 shutdown_connections 2024-05-15T16:03:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.460+0000 7f7e84832700 1 --2- 172.21.15.49:0/17501163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80102740 0x7f7e8010ac30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.460+0000 7f7e84832700 1 --2- 172.21.15.49:0/17501163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e80101e20 0x7f7e80102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.460+0000 7f7e84832700 1 -- 172.21.15.49:0/17501163 >> 172.21.15.49:0/17501163 conn(0x7f7e80076800 msgr2=0x7f7e800fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.460+0000 7f7e84832700 1 -- 172.21.15.49:0/17501163 shutdown_connections 2024-05-15T16:03:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.460+0000 7f7e84832700 1 -- 172.21.15.49:0/17501163 wait complete. 2024-05-15T16:03:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.461+0000 7f7e84832700 1 Processor -- start 2024-05-15T16:03:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.461+0000 7f7e84832700 1 -- start start 2024-05-15T16:03:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.461+0000 7f7e84832700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80101e20 0x7f7e8019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.461+0000 7f7e84832700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e80102740 0x7f7e8019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.461+0000 7f7e84832700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8019d460 con 0x7f7e80101e20 2024-05-15T16:03:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.461+0000 7f7e84832700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e80196a40 con 0x7f7e80102740 2024-05-15T16:03:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.461+0000 7f7e7e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80101e20 0x7f7e8019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.461+0000 7f7e7e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80101e20 0x7f7e8019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39004/0 (socket says 172.21.15.49:39004) 2024-05-15T16:03:33.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e7e59c700 1 -- 172.21.15.49:0/2902001110 learned_addr learned my addr 172.21.15.49:0/2902001110 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e7dd9b700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e80102740 0x7f7e8019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e7e59c700 1 -- 172.21.15.49:0/2902001110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e80102740 msgr2=0x7f7e8019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e7e59c700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e80102740 0x7f7e8019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e7e59c700 1 -- 172.21.15.49:0/2902001110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680095f0 con 0x7f7e80101e20 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e7e59c700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80101e20 0x7f7e8019c900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e700060b0 tx=0x7f7e7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e777fe700 1 -- 172.21.15.49:0/2902001110 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e7000dda0 con 0x7f7e80101e20 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e777fe700 1 -- 172.21.15.49:0/2902001110 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e7000f040 con 0x7f7e80101e20 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e80196d20 con 0x7f7e80101e20 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e777fe700 1 -- 172.21.15.49:0/2902001110 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e70014790 con 0x7f7e80101e20 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.462+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e80197270 con 0x7f7e80101e20 2024-05-15T16:03:33.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.463+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e80105540 con 0x7f7e80101e20 2024-05-15T16:03:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.464+0000 7f7e777fe700 1 -- 172.21.15.49:0/2902001110 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7e70014980 con 0x7f7e80101e20 2024-05-15T16:03:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.464+0000 7f7e777fe700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7e6c06e200 0x7f7e6c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.464+0000 7f7e777fe700 1 -- 172.21.15.49:0/2902001110 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7e7008dc30 con 0x7f7e80101e20 2024-05-15T16:03:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.464+0000 7f7e7dd9b700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7e6c06e200 0x7f7e6c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.465+0000 7f7e7dd9b700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7e6c06e200 0x7f7e6c0706c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e80197c00 tx=0x7f7e6801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:33.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.469+0000 7f7e777fe700 1 -- 172.21.15.49:0/2902001110 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7e7005aaf0 con 0x7f7e80101e20 2024-05-15T16:03:33.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.715+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e80197f70 con 0x7f7e6c06e200 2024-05-15T16:03:33.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.716+0000 7f7e777fe700 1 -- 172.21.15.49:0/2902001110 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7e80197f70 con 0x7f7e6c06e200 2024-05-15T16:03:33.718 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:03:33.718 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:03:33.718 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:03:33.718 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:03:33.718 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:03:33.719 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:03:33.719 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:03:33.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:03:33.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:03:33.719 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:03:33.719 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "14/23 daemons upgraded", 2024-05-15T16:03:33.719 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:03:33.719 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:03:33.719 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.718+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7e6c06e200 msgr2=0x7f7e6c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.718+0000 7f7e84832700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7e6c06e200 0x7f7e6c0706c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e80197c00 tx=0x7f7e6801a040 comp rx=0 tx=0).stop 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.718+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80101e20 msgr2=0x7f7e8019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.718+0000 7f7e84832700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80101e20 0x7f7e8019c900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e700060b0 tx=0x7f7e7000d3b0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.719+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 shutdown_connections 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.719+0000 7f7e84832700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7e6c06e200 0x7f7e6c0706c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.719+0000 7f7e84832700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e80101e20 0x7f7e8019c900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.719+0000 7f7e84832700 1 --2- 172.21.15.49:0/2902001110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e80102740 0x7f7e8019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.719+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 >> 172.21.15.49:0/2902001110 conn(0x7f7e80076800 msgr2=0x7f7e80103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.719+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 shutdown_connections 2024-05-15T16:03:33.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.719+0000 7f7e84832700 1 -- 172.21.15.49:0/2902001110 wait complete. 2024-05-15T16:03:33.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 -- 172.21.15.49:0/2604199130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 msgr2=0x7f63a40ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 --2- 172.21.15.49:0/2604199130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 0x7f63a40ffe40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6398009910 tx=0x7f6398009c20 comp rx=0 tx=0).stop 2024-05-15T16:03:33.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 -- 172.21.15.49:0/2604199130 shutdown_connections 2024-05-15T16:03:33.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 --2- 172.21.15.49:0/2604199130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 0x7f63a40ffe40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 --2- 172.21.15.49:0/2604199130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63a41059c0 0x7f63a4105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 -- 172.21.15.49:0/2604199130 >> 172.21.15.49:0/2604199130 conn(0x7f63a40fb4b0 msgr2=0x7f63a40fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:33.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 -- 172.21.15.49:0/2604199130 shutdown_connections 2024-05-15T16:03:33.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 -- 172.21.15.49:0/2604199130 wait complete. 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 Processor -- start 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 -- start start 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 0x7f63a419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63a41059c0 0x7f63a419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a419d380 con 0x7f63a40ff9c0 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.831+0000 7f63a8ba1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4196960 con 0x7f63a41059c0 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 0x7f63a419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 0x7f63a419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53938/0 (socket says 172.21.15.49:53938) 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a37fe700 1 -- 172.21.15.49:0/1323733312 learned_addr learned my addr 172.21.15.49:0/1323733312 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a2ffd700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63a41059c0 0x7f63a419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a37fe700 1 -- 172.21.15.49:0/1323733312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63a41059c0 msgr2=0x7f63a419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a37fe700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63a41059c0 0x7f63a419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a37fe700 1 -- 172.21.15.49:0/1323733312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63980095f0 con 0x7f63a40ff9c0 2024-05-15T16:03:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a2ffd700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63a41059c0 0x7f63a419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:03:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a37fe700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 0x7f63a419c820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f639400b4b0 tx=0x7f639400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a0ff9700 1 -- 172.21.15.49:0/1323733312 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6394011800 con 0x7f63a40ff9c0 2024-05-15T16:03:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a0ff9700 1 -- 172.21.15.49:0/1323733312 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6394011e20 con 0x7f63a40ff9c0 2024-05-15T16:03:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a4196c40 con 0x7f63a40ff9c0 2024-05-15T16:03:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a0ff9700 1 -- 172.21.15.49:0/1323733312 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f639400f2e0 con 0x7f63a40ff9c0 2024-05-15T16:03:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a4197140 con 0x7f63a40ff9c0 2024-05-15T16:03:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a4107fe0 con 0x7f63a40ff9c0 2024-05-15T16:03:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a0ff9700 1 -- 172.21.15.49:0/1323733312 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6394010380 con 0x7f63a40ff9c0 2024-05-15T16:03:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a0ff9700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f638c06e200 0x7f638c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:03:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a0ff9700 1 -- 172.21.15.49:0/1323733312 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f639408dd50 con 0x7f63a40ff9c0 2024-05-15T16:03:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a2ffd700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f638c06e200 0x7f638c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:03:33.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.834+0000 7f63a2ffd700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f638c06e200 0x7f638c0706c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f63a4197b20 tx=0x7f639801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:03:33.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:33.840+0000 7f63a0ff9700 1 -- 172.21.15.49:0/1323733312 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f639405ab60 con 0x7f63a40ff9c0 2024-05-15T16:03:33.931 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:33 smithi049 ceph-mon[112142]: pgmap v194: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:33.931 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:33 smithi049 ceph-mon[112142]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:33.931 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:33 smithi049 ceph-mon[112142]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:33.932 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:33 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4099248839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:33.932 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:33 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2726759402' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:03:33.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:33 smithi144 ceph-mon[86635]: pgmap v194: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:33.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:33 smithi144 ceph-mon[86635]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:33.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:33 smithi144 ceph-mon[86635]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:33.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:33 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4099248839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:33.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:33 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2726759402' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:03:34.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.142+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f63a4101100 con 0x7f63a40ff9c0 2024-05-15T16:03:34.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.142+0000 7f63a0ff9700 1 -- 172.21.15.49:0/1323733312 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f639405a6f0 con 0x7f63a40ff9c0 2024-05-15T16:03:34.145 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:03:34.145 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:03:34.145 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:03:34.145 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:03:34.145 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:03:34.145 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:03:34.145 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:03:34.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f638c06e200 msgr2=0x7f638c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:34.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f638c06e200 0x7f638c0706c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f63a4197b20 tx=0x7f639801a040 comp rx=0 tx=0).stop 2024-05-15T16:03:34.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 msgr2=0x7f63a419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:03:34.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 0x7f63a419c820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f639400b4b0 tx=0x7f639400b870 comp rx=0 tx=0).stop 2024-05-15T16:03:34.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 shutdown_connections 2024-05-15T16:03:34.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f638c06e200 0x7f638c0706c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:34.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63a40ff9c0 0x7f63a419c820 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:34.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 --2- 172.21.15.49:0/1323733312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63a41059c0 0x7f63a419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:03:34.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 >> 172.21.15.49:0/1323733312 conn(0x7f63a40fb4b0 msgr2=0x7f63a4109e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:03:34.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 shutdown_connections 2024-05-15T16:03:34.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:03:34.145+0000 7f63a8ba1700 1 -- 172.21.15.49:0/1323733312 wait complete. 2024-05-15T16:03:34.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:34 smithi144 ceph-mon[86635]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:34.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:34 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1323733312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:03:34.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:34 smithi049 ceph-mon[112142]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:03:34.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:34 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1323733312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: pgmap v195: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:35.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:35.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: pgmap v195: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:35.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:35.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:03:37.942 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:37 smithi144 ceph-mon[86635]: pgmap v196: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:37.943 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:37.943 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jgodzs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T16:03:37.943 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:37 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:03:37.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:37 smithi049 ceph-mon[112142]: pgmap v196: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:37.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:37.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jgodzs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T16:03:37.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:37 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:03:38.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:38 smithi144 ceph-mon[86635]: Upgrade: Updating mds.cephfs.smithi144.jgodzs 2024-05-15T16:03:38.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:38 smithi144 ceph-mon[86635]: Deploying daemon mds.cephfs.smithi144.jgodzs on smithi144 2024-05-15T16:03:38.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:38 smithi049 ceph-mon[112142]: Upgrade: Updating mds.cephfs.smithi144.jgodzs 2024-05-15T16:03:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:38 smithi049 ceph-mon[112142]: Deploying daemon mds.cephfs.smithi144.jgodzs on smithi144 2024-05-15T16:03:39.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:39 smithi049 ceph-mon[112142]: pgmap v197: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:40.142 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:39 smithi144 ceph-mon[86635]: pgmap v197: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:40.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:40 smithi144 ceph-mon[86635]: osdmap e86: 6 total, 6 up, 6 in 2024-05-15T16:03:40.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:40 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-15T16:03:40.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:40 smithi049 ceph-mon[112142]: osdmap e86: 6 total, 6 up, 6 in 2024-05-15T16:03:40.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:40 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-15T16:03:41.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:41 smithi049 ceph-mon[112142]: pgmap v199: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:41.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:41 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:41.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:41 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:41 smithi144 ceph-mon[86635]: pgmap v199: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:42.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:41 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:42.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:41 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:42.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:42 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:42.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:42 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] up:boot 2024-05-15T16:03:42.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:42 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:03:42.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:42 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T16:03:43.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:42 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:03:43.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:42 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] up:boot 2024-05-15T16:03:43.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:42 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:03:43.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:42 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T16:03:43.976 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:43 smithi144 ceph-mon[86635]: pgmap v200: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:44.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:43 smithi049 ceph-mon[112142]: pgmap v200: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:45.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:45 smithi144 ceph-mon[86635]: pgmap v201: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:45.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:45 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:45.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:45 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:45.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:45 smithi049 ceph-mon[112142]: pgmap v201: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:45.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:45 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:45.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:45 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:46.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:46.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:46 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:46.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:46.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:46 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:03:47.896 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:47 smithi144 ceph-mon[86635]: pgmap v202: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:47.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:47 smithi049 ceph-mon[112142]: pgmap v202: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:49.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:49 smithi144 ceph-mon[86635]: pgmap v203: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:49.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:49 smithi049 ceph-mon[112142]: pgmap v203: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:03:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:03:51.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:51 smithi049 ceph-mon[112142]: pgmap v204: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:51 smithi144 ceph-mon[86635]: pgmap v204: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:53 smithi144 ceph-mon[86635]: pgmap v205: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:54.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:53 smithi049 ceph-mon[112142]: pgmap v205: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:56.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:55 smithi144 ceph-mon[86635]: pgmap v206: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:56.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:55 smithi049 ceph-mon[112142]: pgmap v206: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:58.007 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:57 smithi049 ceph-mon[112142]: pgmap v207: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:03:58.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:57 smithi144 ceph-mon[86635]: pgmap v207: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:00.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:03:59 smithi144 ceph-mon[86635]: pgmap v208: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:03:59 smithi049 ceph-mon[112142]: pgmap v208: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:01 smithi144 ceph-mon[86635]: pgmap v209: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:01 smithi049 ceph-mon[112142]: pgmap v209: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:04.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:03 smithi144 ceph-mon[86635]: pgmap v210: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:03 smithi049 ceph-mon[112142]: pgmap v210: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:04.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.270+0000 7f19af98f700 1 -- 172.21.15.49:0/4042702751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 msgr2=0x7f19a80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.270+0000 7f19af98f700 1 --2- 172.21.15.49:0/4042702751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 0x7f19a80ffec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f199800b0a0 tx=0x7f199800b3b0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.271+0000 7f19af98f700 1 -- 172.21.15.49:0/4042702751 shutdown_connections 2024-05-15T16:04:04.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.271+0000 7f19af98f700 1 --2- 172.21.15.49:0/4042702751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8100490 0x7f19a810d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.271+0000 7f19af98f700 1 --2- 172.21.15.49:0/4042702751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 0x7f19a80ffec0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.271+0000 7f19af98f700 1 -- 172.21.15.49:0/4042702751 >> 172.21.15.49:0/4042702751 conn(0x7f19a80fb760 msgr2=0x7f19a80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.271+0000 7f19af98f700 1 -- 172.21.15.49:0/4042702751 shutdown_connections 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.271+0000 7f19af98f700 1 -- 172.21.15.49:0/4042702751 wait complete. 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.271+0000 7f19af98f700 1 Processor -- start 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.271+0000 7f19af98f700 1 -- start start 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19af98f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 0x7f19a8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19af98f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8100490 0x7f19a8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19af98f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a8199a70 con 0x7f19a80ffae0 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19af98f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a819d740 con 0x7f19a8100490 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19ad72b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 0x7f19a8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19ad72b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 0x7f19a8198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54028/0 (socket says 172.21.15.49:54028) 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19ad72b700 1 -- 172.21.15.49:0/1472488519 learned_addr learned my addr 172.21.15.49:0/1472488519 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19acf2a700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8100490 0x7f19a8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:04.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19ad72b700 1 -- 172.21.15.49:0/1472488519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8100490 msgr2=0x7f19a8199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19ad72b700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8100490 0x7f19a8199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19ad72b700 1 -- 172.21.15.49:0/1472488519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998009d00 con 0x7f19a80ffae0 2024-05-15T16:04:04.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19acf2a700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8100490 0x7f19a8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:04:04.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19ad72b700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 0x7f19a8198e50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f19980062a0 tx=0x7f199800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:04.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19a67fc700 1 -- 172.21.15.49:0/1472488519 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199800e040 con 0x7f19a80ffae0 2024-05-15T16:04:04.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19a67fc700 1 -- 172.21.15.49:0/1472488519 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1998003c10 con 0x7f19a80ffae0 2024-05-15T16:04:04.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.272+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a819d9c0 con 0x7f19a80ffae0 2024-05-15T16:04:04.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.273+0000 7f19a67fc700 1 -- 172.21.15.49:0/1472488519 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199801d7f0 con 0x7f19a80ffae0 2024-05-15T16:04:04.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.273+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a819df10 con 0x7f19a80ffae0 2024-05-15T16:04:04.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.273+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a80699d0 con 0x7f19a80ffae0 2024-05-15T16:04:04.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.274+0000 7f19a67fc700 1 -- 172.21.15.49:0/1472488519 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1998004720 con 0x7f19a80ffae0 2024-05-15T16:04:04.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.275+0000 7f19a67fc700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199006e200 0x7f19900706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:04.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.275+0000 7f19a67fc700 1 -- 172.21.15.49:0/1472488519 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f199808f9e0 con 0x7f19a80ffae0 2024-05-15T16:04:04.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.275+0000 7f19acf2a700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199006e200 0x7f19900706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:04.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.275+0000 7f19acf2a700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199006e200 0x7f19900706c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f199c005fd0 tx=0x7f199c005f00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:04.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.280+0000 7f19a67fc700 1 -- 172.21.15.49:0/1472488519 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1998058d70 con 0x7f19a80ffae0 2024-05-15T16:04:04.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.506+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19a81100e0 con 0x7f199006e200 2024-05-15T16:04:04.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.508+0000 7f19a67fc700 1 -- 172.21.15.49:0/1472488519 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f19a81100e0 con 0x7f199006e200 2024-05-15T16:04:04.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199006e200 msgr2=0x7f19900706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199006e200 0x7f19900706c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f199c005fd0 tx=0x7f199c005f00 comp rx=0 tx=0).stop 2024-05-15T16:04:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 msgr2=0x7f19a8198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 0x7f19a8198e50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f19980062a0 tx=0x7f199800cd50 comp rx=0 tx=0).stop 2024-05-15T16:04:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 shutdown_connections 2024-05-15T16:04:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199006e200 0x7f19900706c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19a80ffae0 0x7f19a8198e50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 --2- 172.21.15.49:0/1472488519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8100490 0x7f19a8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 >> 172.21.15.49:0/1472488519 conn(0x7f19a80fb760 msgr2=0x7f19a80fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 shutdown_connections 2024-05-15T16:04:04.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.510+0000 7f19af98f700 1 -- 172.21.15.49:0/1472488519 wait complete. 2024-05-15T16:04:04.523 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:04:04.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.619+0000 7fe928a73700 1 -- 172.21.15.49:0/1406451447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 msgr2=0x7fe924105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.619+0000 7fe928a73700 1 --2- 172.21.15.49:0/1406451447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 0x7fe924105900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe918009910 tx=0x7fe918009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:04.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.620+0000 7fe928a73700 1 -- 172.21.15.49:0/1406451447 shutdown_connections 2024-05-15T16:04:04.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.620+0000 7fe928a73700 1 --2- 172.21.15.49:0/1406451447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 0x7fe924105900 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.620+0000 7fe928a73700 1 --2- 172.21.15.49:0/1406451447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe924100f00 0x7fe9241012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.620+0000 7fe928a73700 1 -- 172.21.15.49:0/1406451447 >> 172.21.15.49:0/1406451447 conn(0x7fe9240fc7d0 msgr2=0x7fe9240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.620+0000 7fe928a73700 1 -- 172.21.15.49:0/1406451447 shutdown_connections 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.620+0000 7fe928a73700 1 -- 172.21.15.49:0/1406451447 wait complete. 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.620+0000 7fe928a73700 1 Processor -- start 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.620+0000 7fe928a73700 1 -- start start 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe928a73700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe924100f00 0x7fe92419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe928a73700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 0x7fe92419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe928a73700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92419d440 con 0x7fe9241018b0 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe928a73700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe924196a20 con 0x7fe924100f00 2024-05-15T16:04:04.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe921d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 0x7fe92419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe921d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 0x7fe92419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54054/0 (socket says 172.21.15.49:54054) 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe921d9b700 1 -- 172.21.15.49:0/2810415607 learned_addr learned my addr 172.21.15.49:0/2810415607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe92259c700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe924100f00 0x7fe92419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe921d9b700 1 -- 172.21.15.49:0/2810415607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe924100f00 msgr2=0x7fe92419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe921d9b700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe924100f00 0x7fe92419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe921d9b700 1 -- 172.21.15.49:0/2810415607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180095f0 con 0x7fe9241018b0 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe921d9b700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 0x7fe92419ce20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe9180092e0 tx=0x7fe91800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.621+0000 7fe9137fe700 1 -- 172.21.15.49:0/2810415607 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe91801d070 con 0x7fe9241018b0 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.622+0000 7fe9137fe700 1 -- 172.21.15.49:0/2810415607 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe918005700 con 0x7fe9241018b0 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.622+0000 7fe9137fe700 1 -- 172.21.15.49:0/2810415607 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe91800fdb0 con 0x7fe9241018b0 2024-05-15T16:04:04.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.622+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe924196ca0 con 0x7fe9241018b0 2024-05-15T16:04:04.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.622+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe924197140 con 0x7fe9241018b0 2024-05-15T16:04:04.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.623+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe924109300 con 0x7fe9241018b0 2024-05-15T16:04:04.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.623+0000 7fe9137fe700 1 -- 172.21.15.49:0/2810415607 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe91800f5f0 con 0x7fe9241018b0 2024-05-15T16:04:04.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.623+0000 7fe9137fe700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe90c06e200 0x7fe90c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:04.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.624+0000 7fe9137fe700 1 -- 172.21.15.49:0/2810415607 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe91808f480 con 0x7fe9241018b0 2024-05-15T16:04:04.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.624+0000 7fe92259c700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe90c06e200 0x7fe90c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:04.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.624+0000 7fe92259c700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe90c06e200 0x7fe90c0706c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe914007be0 tx=0x7fe9140073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:04.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.628+0000 7fe9137fe700 1 -- 172.21.15.49:0/2810415607 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe91805c290 con 0x7fe9241018b0 2024-05-15T16:04:04.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.855+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe924197f50 con 0x7fe90c06e200 2024-05-15T16:04:04.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.857+0000 7fe9137fe700 1 -- 172.21.15.49:0/2810415607 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe924197f50 con 0x7fe90c06e200 2024-05-15T16:04:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe90c06e200 msgr2=0x7fe90c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe90c06e200 0x7fe90c0706c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe914007be0 tx=0x7fe9140073d0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 msgr2=0x7fe92419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 0x7fe92419ce20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe9180092e0 tx=0x7fe91800bd70 comp rx=0 tx=0).stop 2024-05-15T16:04:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 shutdown_connections 2024-05-15T16:04:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe90c06e200 0x7fe90c0706c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe924100f00 0x7fe92419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 --2- 172.21.15.49:0/2810415607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9241018b0 0x7fe92419ce20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 >> 172.21.15.49:0/2810415607 conn(0x7fe9240fc7d0 msgr2=0x7fe9240fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 shutdown_connections 2024-05-15T16:04:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.859+0000 7fe928a73700 1 -- 172.21.15.49:0/2810415607 wait complete. 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.967+0000 7f95d3c10700 1 -- 172.21.15.49:0/3733480780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 msgr2=0x7f95cc1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.967+0000 7f95d3c10700 1 --2- 172.21.15.49:0/3733480780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 0x7f95cc1047c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009910 tx=0x7f95c0009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.967+0000 7f95d3c10700 1 -- 172.21.15.49:0/3733480780 shutdown_connections 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.967+0000 7f95d3c10700 1 --2- 172.21.15.49:0/3733480780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 0x7f95cc1047c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.967+0000 7f95d3c10700 1 --2- 172.21.15.49:0/3733480780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0ffdc0 0x7f95cc1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.967+0000 7f95d3c10700 1 -- 172.21.15.49:0/3733480780 >> 172.21.15.49:0/3733480780 conn(0x7f95cc076800 msgr2=0x7f95cc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.967+0000 7f95d3c10700 1 -- 172.21.15.49:0/3733480780 shutdown_connections 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.967+0000 7f95d3c10700 1 -- 172.21.15.49:0/3733480780 wait complete. 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d3c10700 1 Processor -- start 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d3c10700 1 -- start start 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d3c10700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0ffdc0 0x7f95cc19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:04.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d3c10700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 0x7f95cc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d3c10700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc19d3c0 con 0x7f95cc100770 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d3c10700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc1969f0 con 0x7f95cc0ffdc0 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d11ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 0x7f95cc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d11ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 0x7f95cc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54070/0 (socket says 172.21.15.49:54070) 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d11ab700 1 -- 172.21.15.49:0/3417030486 learned_addr learned my addr 172.21.15.49:0/3417030486 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.968+0000 7f95d19ac700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0ffdc0 0x7f95cc19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.969+0000 7f95d11ab700 1 -- 172.21.15.49:0/3417030486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0ffdc0 msgr2=0x7f95cc19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.969+0000 7f95d11ab700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0ffdc0 0x7f95cc19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.969+0000 7f95d11ab700 1 -- 172.21.15.49:0/3417030486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00095f0 con 0x7f95cc100770 2024-05-15T16:04:04.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.969+0000 7f95d11ab700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 0x7f95cc19cda0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95c00092e0 tx=0x7f95c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:04.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.969+0000 7f95beffd700 1 -- 172.21.15.49:0/3417030486 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c001d070 con 0x7f95cc100770 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.969+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc196c70 con 0x7f95cc100770 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.969+0000 7f95beffd700 1 -- 172.21.15.49:0/3417030486 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95c000f460 con 0x7f95cc100770 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.969+0000 7f95beffd700 1 -- 172.21.15.49:0/3417030486 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c0017610 con 0x7f95cc100770 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.969+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc197160 con 0x7f95cc100770 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.970+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc1012a0 con 0x7f95cc100770 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.971+0000 7f95beffd700 1 -- 172.21.15.49:0/3417030486 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f95c000f5d0 con 0x7f95cc100770 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.971+0000 7f95beffd700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f95b806e210 0x7f95b80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.971+0000 7f95beffd700 1 -- 172.21.15.49:0/3417030486 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f95c008f6b0 con 0x7f95cc100770 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.971+0000 7f95d19ac700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f95b806e210 0x7f95b80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:04.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.972+0000 7f95d19ac700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f95b806e210 0x7f95b80706d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95c8005950 tx=0x7f95c800a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:04.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:04.976+0000 7f95beffd700 1 -- 172.21.15.49:0/3417030486 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f95c005c540 con 0x7f95cc100770 2024-05-15T16:04:05.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.194+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f95cc10cd60 con 0x7f95b806e210 2024-05-15T16:04:05.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.207+0000 7f95beffd700 1 -- 172.21.15.49:0/3417030486 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f95cc10cd60 con 0x7f95b806e210 2024-05-15T16:04:05.209 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:04:05.209 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (7m) 34s ago 18m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:04:05.209 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (19m) 34s ago 19m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:04:05.209 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (18m) 20s ago 18m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:04:05.209 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (4m) 34s ago 19m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (4m) 20s ago 17m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (6m) 34s ago 18m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (59s) 34s ago 13m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (40s) 34s ago 13m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (23s) 20s ago 13m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (13m) 20s ago 13m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (9m) 34s ago 20m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (8m) 20s ago 17m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (5m) 34s ago 20m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (5m) 20s ago 17m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:04:05.210 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (8m) 34s ago 19m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:04:05.211 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (8m) 20s ago 17m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:04:05.211 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (4m) 34s ago 17m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:04:05.211 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (3m) 34s ago 16m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:04:05.211 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (3m) 34s ago 16m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:04:05.211 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (2m) 20s ago 15m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:04:05.211 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (114s) 20s ago 15m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:04:05.211 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (80s) 20s ago 14m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:04:05.211 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (7m) 34s ago 18m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f95b806e210 msgr2=0x7f95b80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f95b806e210 0x7f95b80706d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95c8005950 tx=0x7f95c800a300 comp rx=0 tx=0).stop 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 msgr2=0x7f95cc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 0x7f95cc19cda0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95c00092e0 tx=0x7f95c000bd70 comp rx=0 tx=0).stop 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 shutdown_connections 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f95b806e210 0x7f95b80706d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0ffdc0 0x7f95cc19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 --2- 172.21.15.49:0/3417030486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95cc100770 0x7f95cc19cda0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 >> 172.21.15.49:0/3417030486 conn(0x7f95cc076800 msgr2=0x7f95cc068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 shutdown_connections 2024-05-15T16:04:05.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.209+0000 7f95d3c10700 1 -- 172.21.15.49:0/3417030486 wait complete. 2024-05-15T16:04:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.317+0000 7ffaf89e9700 1 -- 172.21.15.49:0/2491734686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4101860 msgr2=0x7ffaf41058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.317+0000 7ffaf89e9700 1 --2- 172.21.15.49:0/2491734686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4101860 0x7ffaf41058b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009910 tx=0x7ffae8009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.317+0000 7ffaf89e9700 1 -- 172.21.15.49:0/2491734686 shutdown_connections 2024-05-15T16:04:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.317+0000 7ffaf89e9700 1 --2- 172.21.15.49:0/2491734686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4101860 0x7ffaf41058b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.317+0000 7ffaf89e9700 1 --2- 172.21.15.49:0/2491734686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaf4100eb0 0x7ffaf4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.317+0000 7ffaf89e9700 1 -- 172.21.15.49:0/2491734686 >> 172.21.15.49:0/2491734686 conn(0x7ffaf40fc760 msgr2=0x7ffaf40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.317+0000 7ffaf89e9700 1 -- 172.21.15.49:0/2491734686 shutdown_connections 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.317+0000 7ffaf89e9700 1 -- 172.21.15.49:0/2491734686 wait complete. 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf89e9700 1 Processor -- start 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf89e9700 1 -- start start 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf89e9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4100eb0 0x7ffaf419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf89e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaf4101860 0x7ffaf419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf89e9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf419d430 con 0x7ffaf4100eb0 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf89e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf4196a10 con 0x7ffaf4101860 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4100eb0 0x7ffaf419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4100eb0 0x7ffaf419c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54092/0 (socket says 172.21.15.49:54092) 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf259c700 1 -- 172.21.15.49:0/4236456929 learned_addr learned my addr 172.21.15.49:0/4236456929 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf1d9b700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaf4101860 0x7ffaf419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:05.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf259c700 1 -- 172.21.15.49:0/4236456929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaf4101860 msgr2=0x7ffaf419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf259c700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaf4101860 0x7ffaf419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf259c700 1 -- 172.21.15.49:0/4236456929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80095f0 con 0x7ffaf4100eb0 2024-05-15T16:04:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.318+0000 7ffaf1d9b700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaf4101860 0x7ffaf419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:04:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.319+0000 7ffaf259c700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4100eb0 0x7ffaf419c8d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffae400d8d0 tx=0x7ffae400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.319+0000 7ffae37fe700 1 -- 172.21.15.49:0/4236456929 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae4009940 con 0x7ffaf4100eb0 2024-05-15T16:04:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.319+0000 7ffae37fe700 1 -- 172.21.15.49:0/4236456929 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffae4010460 con 0x7ffaf4100eb0 2024-05-15T16:04:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.319+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf4196cf0 con 0x7ffaf4100eb0 2024-05-15T16:04:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.319+0000 7ffae37fe700 1 -- 172.21.15.49:0/4236456929 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae400f5f0 con 0x7ffaf4100eb0 2024-05-15T16:04:05.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.319+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf4197240 con 0x7ffaf4100eb0 2024-05-15T16:04:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.320+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf41092b0 con 0x7ffaf4100eb0 2024-05-15T16:04:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.320+0000 7ffae37fe700 1 -- 172.21.15.49:0/4236456929 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffae40105d0 con 0x7ffaf4100eb0 2024-05-15T16:04:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.321+0000 7ffae37fe700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffadc06e200 0x7ffadc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.321+0000 7ffae37fe700 1 -- 172.21.15.49:0/4236456929 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ffae408e1a0 con 0x7ffaf4100eb0 2024-05-15T16:04:05.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.321+0000 7ffaf1d9b700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffadc06e200 0x7ffadc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:05.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.322+0000 7ffaf1d9b700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffadc06e200 0x7ffadc0706c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4197bd0 tx=0x7ffae801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:05.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.326+0000 7ffae37fe700 1 -- 172.21.15.49:0/4236456929 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ffae405afb0 con 0x7ffaf4100eb0 2024-05-15T16:04:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.617+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffaf4197ed0 con 0x7ffaf4100eb0 2024-05-15T16:04:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.617+0000 7ffae37fe700 1 -- 172.21.15.49:0/4236456929 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7ffae405ab40 con 0x7ffaf4100eb0 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:04:05.620 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:04:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:04:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:04:05.621 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:04:05.621 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:04:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffadc06e200 msgr2=0x7ffadc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffadc06e200 0x7ffadc0706c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4197bd0 tx=0x7ffae801a040 comp rx=0 tx=0).stop 2024-05-15T16:04:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4100eb0 msgr2=0x7ffaf419c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4100eb0 0x7ffaf419c8d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffae400d8d0 tx=0x7ffae400dc90 comp rx=0 tx=0).stop 2024-05-15T16:04:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 shutdown_connections 2024-05-15T16:04:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffadc06e200 0x7ffadc0706c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffaf4100eb0 0x7ffaf419c8d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 --2- 172.21.15.49:0/4236456929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaf4101860 0x7ffaf419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 >> 172.21.15.49:0/4236456929 conn(0x7ffaf40fc760 msgr2=0x7ffaf40fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 shutdown_connections 2024-05-15T16:04:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.620+0000 7ffaf89e9700 1 -- 172.21.15.49:0/4236456929 wait complete. 2024-05-15T16:04:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.729+0000 7f10a14a5700 1 -- 172.21.15.49:0/3701666952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 msgr2=0x7f109c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.729+0000 7f10a14a5700 1 --2- 172.21.15.49:0/3701666952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 0x7f109c10ac40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1084009910 tx=0x7f1084009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.729+0000 7f10a14a5700 1 -- 172.21.15.49:0/3701666952 shutdown_connections 2024-05-15T16:04:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.729+0000 7f10a14a5700 1 --2- 172.21.15.49:0/3701666952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 0x7f109c10ac40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.729+0000 7f10a14a5700 1 --2- 172.21.15.49:0/3701666952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c101da0 0x7f109c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.729+0000 7f10a14a5700 1 -- 172.21.15.49:0/3701666952 >> 172.21.15.49:0/3701666952 conn(0x7f109c0fb6a0 msgr2=0x7f109c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.729+0000 7f10a14a5700 1 -- 172.21.15.49:0/3701666952 shutdown_connections 2024-05-15T16:04:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.729+0000 7f10a14a5700 1 -- 172.21.15.49:0/3701666952 wait complete. 2024-05-15T16:04:05.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f10a14a5700 1 Processor -- start 2024-05-15T16:04:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f10a14a5700 1 -- start start 2024-05-15T16:04:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f10a14a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c101da0 0x7f109c0ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f10a14a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 0x7f109c0fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f10a14a5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c100510 con 0x7f109c102750 2024-05-15T16:04:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f10a14a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c100650 con 0x7f109c101da0 2024-05-15T16:04:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f10927fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 0x7f109c0fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f10927fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 0x7f109c0fffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54118/0 (socket says 172.21.15.49:54118) 2024-05-15T16:04:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f10927fc700 1 -- 172.21.15.49:0/3988742002 learned_addr learned my addr 172.21.15.49:0/3988742002 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:05.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.730+0000 7f109affd700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c101da0 0x7f109c0ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f10927fc700 1 -- 172.21.15.49:0/3988742002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c101da0 msgr2=0x7f109c0ffa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f10927fc700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c101da0 0x7f109c0ffa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f10927fc700 1 -- 172.21.15.49:0/3988742002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10840095f0 con 0x7f109c102750 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f109affd700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c101da0 0x7f109c0ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f10927fc700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 0x7f109c0fffd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1084000c00 tx=0x7f108400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f1098ff9700 1 -- 172.21.15.49:0/3988742002 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108401d070 con 0x7f109c102750 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c1008d0 con 0x7f109c102750 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f1098ff9700 1 -- 172.21.15.49:0/3988742002 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1084005700 con 0x7f109c102750 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f1098ff9700 1 -- 172.21.15.49:0/3988742002 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108400fe50 con 0x7f109c102750 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.731+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c1a2ff0 con 0x7f109c102750 2024-05-15T16:04:05.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.732+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c0697f0 con 0x7f109c102750 2024-05-15T16:04:05.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.733+0000 7f1098ff9700 1 -- 172.21.15.49:0/3988742002 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f108400f640 con 0x7f109c102750 2024-05-15T16:04:05.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.733+0000 7f1098ff9700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f108806e210 0x7f10880706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:05.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.733+0000 7f1098ff9700 1 -- 172.21.15.49:0/3988742002 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f108408f630 con 0x7f109c102750 2024-05-15T16:04:05.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.733+0000 7f109affd700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f108806e210 0x7f10880706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:05.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.733+0000 7f109affd700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f108806e210 0x7f10880706d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f108c009dd0 tx=0x7f108c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:05.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.738+0000 7f1098ff9700 1 -- 172.21.15.49:0/3988742002 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f10840588b0 con 0x7f109c102750 2024-05-15T16:04:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:05 smithi049 ceph-mon[112142]: pgmap v211: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:05 smithi049 ceph-mon[112142]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:05 smithi049 ceph-mon[112142]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:04:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:05 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4236456929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:04:05.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.989+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f109c10de70 con 0x7f109c102750 2024-05-15T16:04:05.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.989+0000 7f1098ff9700 1 -- 172.21.15.49:0/3988742002 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f108405bf70 con 0x7f109c102750 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:04:05.992 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:04:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:04:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:04:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:04:05.995 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:04:05.995 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:04:05.995 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:04:05.995 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:04:05.995 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:04:05.995 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:04:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f108806e210 msgr2=0x7f10880706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f108806e210 0x7f10880706d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f108c009dd0 tx=0x7f108c009040 comp rx=0 tx=0).stop 2024-05-15T16:04:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 msgr2=0x7f109c0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 0x7f109c0fffd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1084000c00 tx=0x7f108400be20 comp rx=0 tx=0).stop 2024-05-15T16:04:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 shutdown_connections 2024-05-15T16:04:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f108806e210 0x7f10880706d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c101da0 0x7f109c0ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 --2- 172.21.15.49:0/3988742002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f109c102750 0x7f109c0fffd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:05.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 >> 172.21.15.49:0/3988742002 conn(0x7f109c0fb6a0 msgr2=0x7f109c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:05.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 shutdown_connections 2024-05-15T16:04:05.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:05.992+0000 7f10a14a5700 1 -- 172.21.15.49:0/3988742002 wait complete. 2024-05-15T16:04:05.997 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:04:06.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.102+0000 7fa3ebe93700 1 -- 172.21.15.49:0/1563614224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e40ff9b0 msgr2=0x7fa3e40ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:06.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.102+0000 7fa3ebe93700 1 --2- 172.21.15.49:0/1563614224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e40ff9b0 0x7fa3e40ffd90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009910 tx=0x7fa3e0009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:06.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.102+0000 7fa3ebe93700 1 -- 172.21.15.49:0/1563614224 shutdown_connections 2024-05-15T16:04:06.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.102+0000 7fa3ebe93700 1 --2- 172.21.15.49:0/1563614224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e4100360 0x7fa3e410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.102+0000 7fa3ebe93700 1 --2- 172.21.15.49:0/1563614224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e40ff9b0 0x7fa3e40ffd90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.102+0000 7fa3ebe93700 1 -- 172.21.15.49:0/1563614224 >> 172.21.15.49:0/1563614224 conn(0x7fa3e4076800 msgr2=0x7fa3e40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.102+0000 7fa3ebe93700 1 -- 172.21.15.49:0/1563614224 shutdown_connections 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.102+0000 7fa3ebe93700 1 -- 172.21.15.49:0/1563614224 wait complete. 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3ebe93700 1 Processor -- start 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3ebe93700 1 -- start start 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3ebe93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e40ff9b0 0x7fa3e4198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3ebe93700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4100360 0x7fa3e41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3ebe93700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e4199990 con 0x7fa3e4100360 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3ebe93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e419d660 con 0x7fa3e40ff9b0 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3e942e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4100360 0x7fa3e41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3e942e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4100360 0x7fa3e41992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54148/0 (socket says 172.21.15.49:54148) 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3e942e700 1 -- 172.21.15.49:0/3799933838 learned_addr learned my addr 172.21.15.49:0/3799933838 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3e9c2f700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e40ff9b0 0x7fa3e4198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3e942e700 1 -- 172.21.15.49:0/3799933838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e40ff9b0 msgr2=0x7fa3e4198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3e942e700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e40ff9b0 0x7fa3e4198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.103+0000 7fa3e942e700 1 -- 172.21.15.49:0/3799933838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00095f0 con 0x7fa3e4100360 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.104+0000 7fa3e942e700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4100360 0x7fa3e41992b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400d8d0 tx=0x7fa3d400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.104+0000 7fa3daffd700 1 -- 172.21.15.49:0/3799933838 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3d40098a0 con 0x7fa3e4100360 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.104+0000 7fa3daffd700 1 -- 172.21.15.49:0/3799933838 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3d4010460 con 0x7fa3e4100360 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.104+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e419d940 con 0x7fa3e4100360 2024-05-15T16:04:06.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.104+0000 7fa3daffd700 1 -- 172.21.15.49:0/3799933838 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3d400f5f0 con 0x7fa3e4100360 2024-05-15T16:04:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.104+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e419de90 con 0x7fa3e4100360 2024-05-15T16:04:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.105+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e4068f60 con 0x7fa3e4100360 2024-05-15T16:04:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.105+0000 7fa3daffd700 1 -- 172.21.15.49:0/3799933838 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa3d4009a00 con 0x7fa3e4100360 2024-05-15T16:04:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.106+0000 7fa3daffd700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3d006e210 0x7fa3d00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.106+0000 7fa3daffd700 1 -- 172.21.15.49:0/3799933838 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa3d408e110 con 0x7fa3e4100360 2024-05-15T16:04:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.106+0000 7fa3e9c2f700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3d006e210 0x7fa3d00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.106+0000 7fa3e9c2f700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3d006e210 0x7fa3d00706d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009fd0 tx=0x7fa3e0005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.110+0000 7fa3daffd700 1 -- 172.21.15.49:0/3799933838 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa3d4057350 con 0x7fa3e4100360 2024-05-15T16:04:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:05 smithi144 ceph-mon[86635]: pgmap v211: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:05 smithi144 ceph-mon[86635]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:05 smithi144 ceph-mon[86635]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:04:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:05 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4236456929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:04:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.340+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3e41078c0 con 0x7fa3d006e210 2024-05-15T16:04:06.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.341+0000 7fa3daffd700 1 -- 172.21.15.49:0/3799933838 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa3e41078c0 con 0x7fa3d006e210 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:04:06.344 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:04:06.345 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:04:06.345 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:04:06.345 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:04:06.345 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:04:06.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3d006e210 msgr2=0x7fa3d00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3d006e210 0x7fa3d00706d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009fd0 tx=0x7fa3e0005ab0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4100360 msgr2=0x7fa3e41992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4100360 0x7fa3e41992b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400d8d0 tx=0x7fa3d400dbe0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 shutdown_connections 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3d006e210 0x7fa3d00706d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e40ff9b0 0x7fa3e4198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 --2- 172.21.15.49:0/3799933838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4100360 0x7fa3e41992b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 >> 172.21.15.49:0/3799933838 conn(0x7fa3e4076800 msgr2=0x7fa3e40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 shutdown_connections 2024-05-15T16:04:06.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.344+0000 7fa3ebe93700 1 -- 172.21.15.49:0/3799933838 wait complete. 2024-05-15T16:04:06.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.453+0000 7f50ab16f700 1 -- 172.21.15.49:0/870931802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4100ee0 msgr2=0x7f50a41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:06.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.453+0000 7f50ab16f700 1 --2- 172.21.15.49:0/870931802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4100ee0 0x7f50a41012c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5098009910 tx=0x7f5098009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.453+0000 7f50ab16f700 1 -- 172.21.15.49:0/870931802 shutdown_connections 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.453+0000 7f50ab16f700 1 --2- 172.21.15.49:0/870931802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4101890 0x7f50a41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.453+0000 7f50ab16f700 1 --2- 172.21.15.49:0/870931802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4100ee0 0x7f50a41012c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.453+0000 7f50ab16f700 1 -- 172.21.15.49:0/870931802 >> 172.21.15.49:0/870931802 conn(0x7f50a40fc790 msgr2=0x7f50a40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.453+0000 7f50ab16f700 1 -- 172.21.15.49:0/870931802 shutdown_connections 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.453+0000 7f50ab16f700 1 -- 172.21.15.49:0/870931802 wait complete. 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.454+0000 7f50ab16f700 1 Processor -- start 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.454+0000 7f50ab16f700 1 -- start start 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.454+0000 7f50ab16f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4100ee0 0x7f50a419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.454+0000 7f50ab16f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4101890 0x7f50a419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.454+0000 7f50ab16f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a419d460 con 0x7f50a4101890 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.454+0000 7f50ab16f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4196a90 con 0x7f50a4100ee0 2024-05-15T16:04:06.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.454+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4101890 0x7f50a419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.454+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4101890 0x7f50a419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54168/0 (socket says 172.21.15.49:54168) 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.454+0000 7f50a3fff700 1 -- 172.21.15.49:0/128125288 learned_addr learned my addr 172.21.15.49:0/128125288 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50a8f0b700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4100ee0 0x7f50a419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50a3fff700 1 -- 172.21.15.49:0/128125288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4100ee0 msgr2=0x7f50a419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50a3fff700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4100ee0 0x7f50a419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50a3fff700 1 -- 172.21.15.49:0/128125288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50980095f0 con 0x7f50a4101890 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50a3fff700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4101890 0x7f50a419ce40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f509000c8f0 tx=0x7f509000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50a1ffb700 1 -- 172.21.15.49:0/128125288 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5090007a50 con 0x7f50a4101890 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50a1ffb700 1 -- 172.21.15.49:0/128125288 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5090007bb0 con 0x7f50a4101890 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a4196d70 con 0x7f50a4101890 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50a1ffb700 1 -- 172.21.15.49:0/128125288 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5090018630 con 0x7f50a4101890 2024-05-15T16:04:06.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.455+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a41972c0 con 0x7f50a4101890 2024-05-15T16:04:06.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.456+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a4109280 con 0x7f50a4101890 2024-05-15T16:04:06.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.457+0000 7f50a1ffb700 1 -- 172.21.15.49:0/128125288 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5090007d20 con 0x7f50a4101890 2024-05-15T16:04:06.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.457+0000 7f50a1ffb700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f509406e200 0x7f50940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:06.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.457+0000 7f50a1ffb700 1 -- 172.21.15.49:0/128125288 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f509008e160 con 0x7f50a4101890 2024-05-15T16:04:06.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.457+0000 7f50a8f0b700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f509406e200 0x7f50940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:06.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.458+0000 7f50a8f0b700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f509406e200 0x7f50940706c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f509800b5c0 tx=0x7f509801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:06.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.462+0000 7f50a1ffb700 1 -- 172.21.15.49:0/128125288 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f509005af70 con 0x7f50a4101890 2024-05-15T16:04:06.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.758+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f50a41023c0 con 0x7f50a4101890 2024-05-15T16:04:06.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.758+0000 7f50a1ffb700 1 -- 172.21.15.49:0/128125288 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f509005ab00 con 0x7f50a4101890 2024-05-15T16:04:06.760 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:04:06.761 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:04:06.761 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:04:06.761 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:04:06.761 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:04:06.761 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:04:06.761 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:04:06.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f509406e200 msgr2=0x7f50940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f509406e200 0x7f50940706c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f509800b5c0 tx=0x7f509801a040 comp rx=0 tx=0).stop 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4101890 msgr2=0x7f50a419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4101890 0x7f50a419ce40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f509000c8f0 tx=0x7f509000ccb0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 shutdown_connections 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f509406e200 0x7f50940706c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4100ee0 0x7f50a419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 --2- 172.21.15.49:0/128125288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a4101890 0x7f50a419ce40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 >> 172.21.15.49:0/128125288 conn(0x7f50a40fc790 msgr2=0x7f50a40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.760+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 shutdown_connections 2024-05-15T16:04:06.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:06.761+0000 7f50ab16f700 1 -- 172.21.15.49:0/128125288 wait complete. 2024-05-15T16:04:07.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:06 smithi144 ceph-mon[86635]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:07.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:06 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3988742002' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:04:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:06 smithi049 ceph-mon[112142]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:07.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:06 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3988742002' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:04:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:07 smithi144 ceph-mon[86635]: pgmap v212: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:07 smithi144 ceph-mon[86635]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:07 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/128125288' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:04:08.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:07 smithi049 ceph-mon[112142]: pgmap v212: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:08.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:07 smithi049 ceph-mon[112142]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:08.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:07 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/128125288' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:04:10.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:09 smithi144 ceph-mon[86635]: pgmap v213: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:10.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:09 smithi049 ceph-mon[112142]: pgmap v213: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:11 smithi144 ceph-mon[86635]: pgmap v214: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:12.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:11 smithi049 ceph-mon[112142]: pgmap v214: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:14.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:13 smithi144 ceph-mon[86635]: pgmap v215: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:14.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:13 smithi049 ceph-mon[112142]: pgmap v215: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:16.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:15 smithi144 ceph-mon[86635]: pgmap v216: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:15 smithi049 ceph-mon[112142]: pgmap v216: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:18.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:17 smithi144 ceph-mon[86635]: pgmap v217: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:17 smithi049 ceph-mon[112142]: pgmap v217: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:19.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:19 smithi144 ceph-mon[86635]: pgmap v218: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:20.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:19 smithi049 ceph-mon[112142]: pgmap v218: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:04:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:04:22.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:21 smithi144 ceph-mon[86635]: pgmap v219: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:21 smithi049 ceph-mon[112142]: pgmap v219: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:24.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:23 smithi144 ceph-mon[86635]: pgmap v220: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:23 smithi049 ceph-mon[112142]: pgmap v220: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:26.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:25 smithi144 ceph-mon[86635]: pgmap v221: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:26.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:25 smithi049 ceph-mon[112142]: pgmap v221: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:28.025 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:27 smithi049 ceph-mon[112142]: pgmap v222: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:27 smithi144 ceph-mon[86635]: pgmap v222: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:29.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:29 smithi144 ceph-mon[86635]: pgmap v223: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:30.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:29 smithi049 ceph-mon[112142]: pgmap v223: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:31 smithi144 ceph-mon[86635]: pgmap v224: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:31 smithi049 ceph-mon[112142]: pgmap v224: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:33.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:33 smithi049 ceph-mon[112142]: pgmap v225: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:33 smithi144 ceph-mon[86635]: pgmap v225: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:36.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:35 smithi144 ceph-mon[86635]: pgmap v226: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:36.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:04:36.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:35 smithi049 ceph-mon[112142]: pgmap v226: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:36.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:04:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.880+0000 7ff628a1b700 1 -- 172.21.15.49:0/2335979975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 msgr2=0x7ff624104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.880+0000 7ff628a1b700 1 --2- 172.21.15.49:0/2335979975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 0x7ff624104810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009910 tx=0x7ff60c009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.881+0000 7ff628a1b700 1 -- 172.21.15.49:0/2335979975 shutdown_connections 2024-05-15T16:04:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.881+0000 7ff628a1b700 1 --2- 172.21.15.49:0/2335979975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 0x7ff624104810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.881+0000 7ff628a1b700 1 --2- 172.21.15.49:0/2335979975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6240ffe10 0x7ff6241001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.881+0000 7ff628a1b700 1 -- 172.21.15.49:0/2335979975 >> 172.21.15.49:0/2335979975 conn(0x7ff6240fb700 msgr2=0x7ff6240fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.881+0000 7ff628a1b700 1 -- 172.21.15.49:0/2335979975 shutdown_connections 2024-05-15T16:04:36.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.881+0000 7ff628a1b700 1 -- 172.21.15.49:0/2335979975 wait complete. 2024-05-15T16:04:36.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff628a1b700 1 Processor -- start 2024-05-15T16:04:36.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff628a1b700 1 -- start start 2024-05-15T16:04:36.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff628a1b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6240ffe10 0x7ff624197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:36.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff628a1b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 0x7ff624197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff628a1b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624198540 con 0x7ff6241007c0 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff628a1b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624198680 con 0x7ff6240ffe10 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff621d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 0x7ff624197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff621d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 0x7ff624197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55660/0 (socket says 172.21.15.49:55660) 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff621d9b700 1 -- 172.21.15.49:0/728774607 learned_addr learned my addr 172.21.15.49:0/728774607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff62259c700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6240ffe10 0x7ff624197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff621d9b700 1 -- 172.21.15.49:0/728774607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6240ffe10 msgr2=0x7ff624197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff621d9b700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6240ffe10 0x7ff624197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.882+0000 7ff621d9b700 1 -- 172.21.15.49:0/728774607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0095f0 con 0x7ff6241007c0 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.883+0000 7ff621d9b700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 0x7ff624197f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00bd70 tx=0x7ff60c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.883+0000 7ff61b7fe700 1 -- 172.21.15.49:0/728774607 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff60c01d070 con 0x7ff6241007c0 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.883+0000 7ff61b7fe700 1 -- 172.21.15.49:0/728774607 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff60c0056e0 con 0x7ff6241007c0 2024-05-15T16:04:36.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.883+0000 7ff61b7fe700 1 -- 172.21.15.49:0/728774607 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff60c00fe50 con 0x7ff6241007c0 2024-05-15T16:04:36.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.883+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6241a0aa0 con 0x7ff6241007c0 2024-05-15T16:04:36.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.883+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6241a0e50 con 0x7ff6241007c0 2024-05-15T16:04:36.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.884+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff624108210 con 0x7ff6241007c0 2024-05-15T16:04:36.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.885+0000 7ff61b7fe700 1 -- 172.21.15.49:0/728774607 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff60c00f680 con 0x7ff6241007c0 2024-05-15T16:04:36.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.885+0000 7ff61b7fe700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61006e200 0x7ff6100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:36.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.885+0000 7ff61b7fe700 1 -- 172.21.15.49:0/728774607 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff60c08f430 con 0x7ff6241007c0 2024-05-15T16:04:36.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.885+0000 7ff62259c700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61006e200 0x7ff6100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:36.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.886+0000 7ff62259c700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61006e200 0x7ff6100706c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff614009550 tx=0x7ff6140091f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:36.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:36.890+0000 7ff61b7fe700 1 -- 172.21.15.49:0/728774607 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff60c058790 con 0x7ff6241007c0 2024-05-15T16:04:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.118+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff624069840 con 0x7ff61006e200 2024-05-15T16:04:37.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.119+0000 7ff61b7fe700 1 -- 172.21.15.49:0/728774607 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff624069840 con 0x7ff61006e200 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.121+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61006e200 msgr2=0x7ff6100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.121+0000 7ff628a1b700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61006e200 0x7ff6100706c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff614009550 tx=0x7ff6140091f0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.121+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 msgr2=0x7ff624197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.122+0000 7ff628a1b700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 0x7ff624197f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00bd70 tx=0x7ff60c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.122+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 shutdown_connections 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.122+0000 7ff628a1b700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61006e200 0x7ff6100706c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.122+0000 7ff628a1b700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6240ffe10 0x7ff624197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.122+0000 7ff628a1b700 1 --2- 172.21.15.49:0/728774607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6241007c0 0x7ff624197f70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.122+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 >> 172.21.15.49:0/728774607 conn(0x7ff6240fb700 msgr2=0x7ff6240fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:37.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.122+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 shutdown_connections 2024-05-15T16:04:37.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.122+0000 7ff628a1b700 1 -- 172.21.15.49:0/728774607 wait complete. 2024-05-15T16:04:37.134 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:04:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.233+0000 7f4ddeb11700 1 -- 172.21.15.49:0/815874593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8101e20 msgr2=0x7f4dd8102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.233+0000 7f4ddeb11700 1 --2- 172.21.15.49:0/815874593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8101e20 0x7f4dd8102200 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009910 tx=0x7f4dd4009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.233+0000 7f4ddeb11700 1 -- 172.21.15.49:0/815874593 shutdown_connections 2024-05-15T16:04:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.233+0000 7f4ddeb11700 1 --2- 172.21.15.49:0/815874593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd8102740 0x7f4dd810ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.233+0000 7f4ddeb11700 1 --2- 172.21.15.49:0/815874593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8101e20 0x7f4dd8102200 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.233+0000 7f4ddeb11700 1 -- 172.21.15.49:0/815874593 >> 172.21.15.49:0/815874593 conn(0x7f4dd8076800 msgr2=0x7f4dd80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.233+0000 7f4ddeb11700 1 -- 172.21.15.49:0/815874593 shutdown_connections 2024-05-15T16:04:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.233+0000 7f4ddeb11700 1 -- 172.21.15.49:0/815874593 wait complete. 2024-05-15T16:04:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4ddeb11700 1 Processor -- start 2024-05-15T16:04:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4ddeb11700 1 -- start start 2024-05-15T16:04:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4ddeb11700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd8101e20 0x7f4dd819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4ddeb11700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8102740 0x7f4dd819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4ddeb11700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd819d420 con 0x7f4dd8102740 2024-05-15T16:04:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4ddeb11700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8196a00 con 0x7f4dd8101e20 2024-05-15T16:04:37.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8102740 0x7f4dd819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8102740 0x7f4dd819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55672/0 (socket says 172.21.15.49:55672) 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4dcffff700 1 -- 172.21.15.49:0/2487759697 learned_addr learned my addr 172.21.15.49:0/2487759697 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4ddc8ad700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd8101e20 0x7f4dd819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4dcffff700 1 -- 172.21.15.49:0/2487759697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd8101e20 msgr2=0x7f4dd819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4dcffff700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd8101e20 0x7f4dd819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4dcffff700 1 -- 172.21.15.49:0/2487759697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40095f0 con 0x7f4dd8102740 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.234+0000 7f4dcffff700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8102740 0x7f4dd819ce00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800c8f0 tx=0x7f4dc800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.235+0000 7f4dcdffb700 1 -- 172.21.15.49:0/2487759697 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dc800e850 con 0x7f4dd8102740 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.235+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd8196ce0 con 0x7f4dd8102740 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.235+0000 7f4dcdffb700 1 -- 172.21.15.49:0/2487759697 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dc800ee70 con 0x7f4dd8102740 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.235+0000 7f4dcdffb700 1 -- 172.21.15.49:0/2487759697 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dc8018630 con 0x7f4dd8102740 2024-05-15T16:04:37.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.235+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd8197230 con 0x7f4dd8102740 2024-05-15T16:04:37.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.236+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd8105540 con 0x7f4dd8102740 2024-05-15T16:04:37.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.236+0000 7f4dcdffb700 1 -- 172.21.15.49:0/2487759697 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4dc800f450 con 0x7f4dd8102740 2024-05-15T16:04:37.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.237+0000 7f4dcdffb700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4dc006e200 0x7f4dc00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:37.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.237+0000 7f4dcdffb700 1 -- 172.21.15.49:0/2487759697 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4dc808e000 con 0x7f4dd8102740 2024-05-15T16:04:37.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.237+0000 7f4ddc8ad700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4dc006e200 0x7f4dc00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:37.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.237+0000 7f4ddc8ad700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4dc006e200 0x7f4dc00706c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000c00 tx=0x7f4dd401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:37.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.242+0000 7f4dcdffb700 1 -- 172.21.15.49:0/2487759697 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4dc805ae10 con 0x7f4dd8102740 2024-05-15T16:04:37.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.466+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dd8197f30 con 0x7f4dc006e200 2024-05-15T16:04:37.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.468+0000 7f4dcdffb700 1 -- 172.21.15.49:0/2487759697 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4dd8197f30 con 0x7f4dc006e200 2024-05-15T16:04:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4dc006e200 msgr2=0x7f4dc00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4dc006e200 0x7f4dc00706c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000c00 tx=0x7f4dd401a040 comp rx=0 tx=0).stop 2024-05-15T16:04:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8102740 msgr2=0x7f4dd819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8102740 0x7f4dd819ce00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800c8f0 tx=0x7f4dc800cc00 comp rx=0 tx=0).stop 2024-05-15T16:04:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 shutdown_connections 2024-05-15T16:04:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4dc006e200 0x7f4dc00706c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd8101e20 0x7f4dd819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 --2- 172.21.15.49:0/2487759697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dd8102740 0x7f4dd819ce00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 >> 172.21.15.49:0/2487759697 conn(0x7f4dd8076800 msgr2=0x7f4dd8103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 shutdown_connections 2024-05-15T16:04:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.470+0000 7f4ddeb11700 1 -- 172.21.15.49:0/2487759697 wait complete. 2024-05-15T16:04:37.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.579+0000 7f1e3a228700 1 -- 172.21.15.49:0/3910584830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 msgr2=0x7f1e34105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.579+0000 7f1e3a228700 1 --2- 172.21.15.49:0/3910584830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 0x7f1e34105900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009910 tx=0x7f1e28009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:37.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.579+0000 7f1e3a228700 1 -- 172.21.15.49:0/3910584830 shutdown_connections 2024-05-15T16:04:37.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.579+0000 7f1e3a228700 1 --2- 172.21.15.49:0/3910584830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 0x7f1e34105900 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.579+0000 7f1e3a228700 1 --2- 172.21.15.49:0/3910584830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e34100f00 0x7f1e341012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.579+0000 7f1e3a228700 1 -- 172.21.15.49:0/3910584830 >> 172.21.15.49:0/3910584830 conn(0x7f1e340fc7d0 msgr2=0x7f1e340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:37.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.580+0000 7f1e3a228700 1 -- 172.21.15.49:0/3910584830 shutdown_connections 2024-05-15T16:04:37.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.580+0000 7f1e3a228700 1 -- 172.21.15.49:0/3910584830 wait complete. 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.580+0000 7f1e3a228700 1 Processor -- start 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.580+0000 7f1e3a228700 1 -- start start 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e3a228700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e34100f00 0x7f1e34198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e3a228700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 0x7f1e34199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e3a228700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e341999e0 con 0x7f1e341018b0 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e3a228700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3419d6b0 con 0x7f1e34100f00 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 0x7f1e34199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 0x7f1e34199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55700/0 (socket says 172.21.15.49:55700) 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e32ffd700 1 -- 172.21.15.49:0/2906563467 learned_addr learned my addr 172.21.15.49:0/2906563467 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:37.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e337fe700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e34100f00 0x7f1e34198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e32ffd700 1 -- 172.21.15.49:0/2906563467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e34100f00 msgr2=0x7f1e34198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e32ffd700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e34100f00 0x7f1e34198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e32ffd700 1 -- 172.21.15.49:0/2906563467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e280095f0 con 0x7f1e341018b0 2024-05-15T16:04:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e32ffd700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 0x7f1e34199300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800bd70 tx=0x7f1e2800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e30ff9700 1 -- 172.21.15.49:0/2906563467 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e2801d070 con 0x7f1e341018b0 2024-05-15T16:04:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e30ff9700 1 -- 172.21.15.49:0/2906563467 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e280056e0 con 0x7f1e341018b0 2024-05-15T16:04:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3419d930 con 0x7f1e341018b0 2024-05-15T16:04:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.581+0000 7f1e30ff9700 1 -- 172.21.15.49:0/2906563467 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e28017400 con 0x7f1e341018b0 2024-05-15T16:04:37.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.582+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3419de20 con 0x7f1e341018b0 2024-05-15T16:04:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.583+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e34105080 con 0x7f1e341018b0 2024-05-15T16:04:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.583+0000 7f1e30ff9700 1 -- 172.21.15.49:0/2906563467 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1e2800f460 con 0x7f1e341018b0 2024-05-15T16:04:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.584+0000 7f1e30ff9700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e1c06e200 0x7f1e1c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.584+0000 7f1e30ff9700 1 -- 172.21.15.49:0/2906563467 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1e2808f5b0 con 0x7f1e341018b0 2024-05-15T16:04:37.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.584+0000 7f1e337fe700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e1c06e200 0x7f1e1c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:37.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.584+0000 7f1e337fe700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e1c06e200 0x7f1e1c0706c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1e24005950 tx=0x7f1e240058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.588+0000 7f1e30ff9700 1 -- 172.21.15.49:0/2906563467 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1e28058950 con 0x7f1e341018b0 2024-05-15T16:04:37.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.809+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e3410dea0 con 0x7f1e1c06e200 2024-05-15T16:04:37.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.820+0000 7f1e30ff9700 1 -- 172.21.15.49:0/2906563467 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1e3410dea0 con 0x7f1e1c06e200 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (7m) 67s ago 19m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (19m) 67s ago 19m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (18m) 53s ago 18m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (5m) 67s ago 19m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (5m) 53s ago 18m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (7m) 67s ago 19m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (92s) 67s ago 13m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (73s) 67s ago 13m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (56s) 53s ago 13m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:04:37.822 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (13m) 53s ago 13m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (10m) 67s ago 20m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (9m) 53s ago 18m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (6m) 67s ago 21m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (6m) 53s ago 18m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (8m) 67s ago 19m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (8m) 53s ago 18m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (4m) 67s ago 17m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (4m) 67s ago 17m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (3m) 67s ago 16m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (3m) 53s ago 16m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (2m) 53s ago 15m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (112s) 53s ago 15m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:04:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (8m) 67s ago 18m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:04:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e1c06e200 msgr2=0x7f1e1c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e1c06e200 0x7f1e1c0706c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1e24005950 tx=0x7f1e240058e0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 msgr2=0x7f1e34199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 0x7f1e34199300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800bd70 tx=0x7f1e2800be50 comp rx=0 tx=0).stop 2024-05-15T16:04:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 shutdown_connections 2024-05-15T16:04:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e1c06e200 0x7f1e1c0706c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e34100f00 0x7f1e34198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 --2- 172.21.15.49:0/2906563467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e341018b0 0x7f1e34199300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 >> 172.21.15.49:0/2906563467 conn(0x7f1e340fc7d0 msgr2=0x7f1e340fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 shutdown_connections 2024-05-15T16:04:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.822+0000 7f1e3a228700 1 -- 172.21.15.49:0/2906563467 wait complete. 2024-05-15T16:04:37.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.931+0000 7fc309b0e700 1 -- 172.21.15.49:0/3280086970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 msgr2=0x7fc3041001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.931+0000 7fc309b0e700 1 --2- 172.21.15.49:0/3280086970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 0x7fc3041001b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009910 tx=0x7fc2f4009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:37.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.931+0000 7fc309b0e700 1 -- 172.21.15.49:0/3280086970 shutdown_connections 2024-05-15T16:04:37.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.931+0000 7fc309b0e700 1 --2- 172.21.15.49:0/3280086970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc304100780 0x7fc3041047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.931+0000 7fc309b0e700 1 --2- 172.21.15.49:0/3280086970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 0x7fc3041001b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.931+0000 7fc309b0e700 1 -- 172.21.15.49:0/3280086970 >> 172.21.15.49:0/3280086970 conn(0x7fc3040fb6a0 msgr2=0x7fc3040fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:37.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.931+0000 7fc309b0e700 1 -- 172.21.15.49:0/3280086970 shutdown_connections 2024-05-15T16:04:37.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.931+0000 7fc309b0e700 1 -- 172.21.15.49:0/3280086970 wait complete. 2024-05-15T16:04:37.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.932+0000 7fc309b0e700 1 Processor -- start 2024-05-15T16:04:37.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.932+0000 7fc309b0e700 1 -- start start 2024-05-15T16:04:37.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.932+0000 7fc309b0e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 0x7fc30419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.932+0000 7fc309b0e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc304100780 0x7fc30419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.932+0000 7fc309b0e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30419d3d0 con 0x7fc3040ffdd0 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.932+0000 7fc309b0e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041969b0 con 0x7fc304100780 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.932+0000 7fc3037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 0x7fc30419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.932+0000 7fc3037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 0x7fc30419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55718/0 (socket says 172.21.15.49:55718) 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.932+0000 7fc3037fe700 1 -- 172.21.15.49:0/3805682483 learned_addr learned my addr 172.21.15.49:0/3805682483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc302ffd700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc304100780 0x7fc30419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc3037fe700 1 -- 172.21.15.49:0/3805682483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc304100780 msgr2=0x7fc30419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc3037fe700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc304100780 0x7fc30419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc3037fe700 1 -- 172.21.15.49:0/3805682483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f40095f0 con 0x7fc3040ffdd0 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc3037fe700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 0x7fc30419c870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40092e0 tx=0x7fc2f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc300ff9700 1 -- 172.21.15.49:0/3805682483 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f401d070 con 0x7fc3040ffdd0 2024-05-15T16:04:37.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc304196c30 con 0x7fc3040ffdd0 2024-05-15T16:04:37.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc300ff9700 1 -- 172.21.15.49:0/3805682483 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2f40056e0 con 0x7fc3040ffdd0 2024-05-15T16:04:37.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc300ff9700 1 -- 172.21.15.49:0/3805682483 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f400fe80 con 0x7fc3040ffdd0 2024-05-15T16:04:37.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.933+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304197120 con 0x7fc3040ffdd0 2024-05-15T16:04:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.934+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc304108170 con 0x7fc3040ffdd0 2024-05-15T16:04:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.935+0000 7fc300ff9700 1 -- 172.21.15.49:0/3805682483 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc2f4003a60 con 0x7fc3040ffdd0 2024-05-15T16:04:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.935+0000 7fc300ff9700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc2ec06e210 0x7fc2ec0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.935+0000 7fc300ff9700 1 -- 172.21.15.49:0/3805682483 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc2f408f5a0 con 0x7fc3040ffdd0 2024-05-15T16:04:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.935+0000 7fc302ffd700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc2ec06e210 0x7fc2ec0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:37.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.936+0000 7fc302ffd700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc2ec06e210 0x7fc2ec0706d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8005fd0 tx=0x7fc2f8005f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:37.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:37.940+0000 7fc300ff9700 1 -- 172.21.15.49:0/3805682483 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc2f405c3b0 con 0x7fc3040ffdd0 2024-05-15T16:04:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:37 smithi144 ceph-mon[86635]: pgmap v227: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:37 smithi049 ceph-mon[112142]: pgmap v227: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:38.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.240+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc304197e70 con 0x7fc3040ffdd0 2024-05-15T16:04:38.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.240+0000 7fc300ff9700 1 -- 172.21.15.49:0/3805682483 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fc2f405bf40 con 0x7fc3040ffdd0 2024-05-15T16:04:38.242 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:04:38.242 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:04:38.242 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:04:38.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:04:38.243 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:04:38.244 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc2ec06e210 msgr2=0x7fc2ec0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc2ec06e210 0x7fc2ec0706d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8005fd0 tx=0x7fc2f8005f60 comp rx=0 tx=0).stop 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 msgr2=0x7fc30419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 0x7fc30419c870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40092e0 tx=0x7fc2f400be20 comp rx=0 tx=0).stop 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 shutdown_connections 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc2ec06e210 0x7fc2ec0706d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3040ffdd0 0x7fc30419c870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 --2- 172.21.15.49:0/3805682483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc304100780 0x7fc30419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 >> 172.21.15.49:0/3805682483 conn(0x7fc3040fb6a0 msgr2=0x7fc304109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 shutdown_connections 2024-05-15T16:04:38.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.243+0000 7fc309b0e700 1 -- 172.21.15.49:0/3805682483 wait complete. 2024-05-15T16:04:38.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 -- 172.21.15.49:0/2732352409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514100ae0 msgr2=0x7f2514100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 --2- 172.21.15.49:0/2732352409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514100ae0 0x7f2514100f60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2504009910 tx=0x7f2504009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:38.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 -- 172.21.15.49:0/2732352409 shutdown_connections 2024-05-15T16:04:38.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 --2- 172.21.15.49:0/2732352409 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514100ae0 0x7f2514100f60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 --2- 172.21.15.49:0/2732352409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2514106b70 0x7f2514106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 -- 172.21.15.49:0/2732352409 >> 172.21.15.49:0/2732352409 conn(0x7f25140fc5f0 msgr2=0x7f25140fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:38.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 -- 172.21.15.49:0/2732352409 shutdown_connections 2024-05-15T16:04:38.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 -- 172.21.15.49:0/2732352409 wait complete. 2024-05-15T16:04:38.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 Processor -- start 2024-05-15T16:04:38.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.351+0000 7f251a19d700 1 -- start start 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251a19d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2514100ae0 0x7f251419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251a19d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514106b70 0x7f251419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251a19d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251419d2e0 con 0x7f2514106b70 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251a19d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141968c0 con 0x7f2514100ae0 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251899a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514106b70 0x7f251419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251899a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514106b70 0x7f251419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55730/0 (socket says 172.21.15.49:55730) 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251899a700 1 -- 172.21.15.49:0/2284027001 learned_addr learned my addr 172.21.15.49:0/2284027001 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251919b700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2514100ae0 0x7f251419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251899a700 1 -- 172.21.15.49:0/2284027001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2514100ae0 msgr2=0x7f251419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251899a700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2514100ae0 0x7f251419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251899a700 1 -- 172.21.15.49:0/2284027001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25040095f0 con 0x7f2514106b70 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251919b700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2514100ae0 0x7f251419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:04:38.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f251899a700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514106b70 0x7f251419ccc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f250400bd70 tx=0x7f250400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f250a7fc700 1 -- 172.21.15.49:0/2284027001 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f250401d070 con 0x7f2514106b70 2024-05-15T16:04:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.352+0000 7f250a7fc700 1 -- 172.21.15.49:0/2284027001 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25040056e0 con 0x7f2514106b70 2024-05-15T16:04:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.353+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2514196b40 con 0x7f2514106b70 2024-05-15T16:04:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.353+0000 7f250a7fc700 1 -- 172.21.15.49:0/2284027001 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2504017400 con 0x7f2514106b70 2024-05-15T16:04:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.353+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2514197030 con 0x7f2514106b70 2024-05-15T16:04:38.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.354+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2514109070 con 0x7f2514106b70 2024-05-15T16:04:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.354+0000 7f250a7fc700 1 -- 172.21.15.49:0/2284027001 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2504003a60 con 0x7f2514106b70 2024-05-15T16:04:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.355+0000 7f250a7fc700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f250006e200 0x7f25000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.355+0000 7f250a7fc700 1 -- 172.21.15.49:0/2284027001 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f250408f6c0 con 0x7f2514106b70 2024-05-15T16:04:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.355+0000 7f251919b700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f250006e200 0x7f25000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:38.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.355+0000 7f251919b700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f250006e200 0x7f25000706c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f251000bcf0 tx=0x7f251000b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:38.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.359+0000 7f250a7fc700 1 -- 172.21.15.49:0/2284027001 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f250405c430 con 0x7f2514106b70 2024-05-15T16:04:38.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.610+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2514197d80 con 0x7f2514106b70 2024-05-15T16:04:38.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.610+0000 7f250a7fc700 1 -- 172.21.15.49:0/2284027001 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f250405bfc0 con 0x7f2514106b70 2024-05-15T16:04:38.612 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:04:38.613 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:04:38.614 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:04:38.615 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:04:38.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.613+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f250006e200 msgr2=0x7f25000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.613+0000 7f251a19d700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f250006e200 0x7f25000706c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f251000bcf0 tx=0x7f251000b2c0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.613+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514106b70 msgr2=0x7f251419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.613+0000 7f251a19d700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514106b70 0x7f251419ccc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f250400bd70 tx=0x7f250400be50 comp rx=0 tx=0).stop 2024-05-15T16:04:38.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.613+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 shutdown_connections 2024-05-15T16:04:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.613+0000 7f251a19d700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f250006e200 0x7f25000706c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.614+0000 7f251a19d700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2514100ae0 0x7f251419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.614+0000 7f251a19d700 1 --2- 172.21.15.49:0/2284027001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2514106b70 0x7f251419ccc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.614+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 >> 172.21.15.49:0/2284027001 conn(0x7f25140fc5f0 msgr2=0x7f25140fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.614+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 shutdown_connections 2024-05-15T16:04:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.614+0000 7f251a19d700 1 -- 172.21.15.49:0/2284027001 wait complete. 2024-05-15T16:04:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:04:38.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.730+0000 7f2f90bcb700 1 -- 172.21.15.49:0/3940650980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c100770 msgr2=0x7f2f8c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.730+0000 7f2f90bcb700 1 --2- 172.21.15.49:0/3940650980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c100770 0x7f2f8c1047c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009910 tx=0x7f2f7c009c20 comp rx=0 tx=0).stop 2024-05-15T16:04:38.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.730+0000 7f2f90bcb700 1 -- 172.21.15.49:0/3940650980 shutdown_connections 2024-05-15T16:04:38.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.730+0000 7f2f90bcb700 1 --2- 172.21.15.49:0/3940650980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c100770 0x7f2f8c1047c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.730+0000 7f2f90bcb700 1 --2- 172.21.15.49:0/3940650980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c0ffdc0 0x7f2f8c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.730+0000 7f2f90bcb700 1 -- 172.21.15.49:0/3940650980 >> 172.21.15.49:0/3940650980 conn(0x7f2f8c076800 msgr2=0x7f2f8c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:38.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.730+0000 7f2f90bcb700 1 -- 172.21.15.49:0/3940650980 shutdown_connections 2024-05-15T16:04:38.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.730+0000 7f2f90bcb700 1 -- 172.21.15.49:0/3940650980 wait complete. 2024-05-15T16:04:38.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f90bcb700 1 Processor -- start 2024-05-15T16:04:38.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f90bcb700 1 -- start start 2024-05-15T16:04:38.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f90bcb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c0ffdc0 0x7f2f8c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:38.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f90bcb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c100770 0x7f2f8c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:38.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f90bcb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c1984f0 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f90bcb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c198630 con 0x7f2f8c100770 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f8a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c0ffdc0 0x7f2f8c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f8a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c0ffdc0 0x7f2f8c1979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55752/0 (socket says 172.21.15.49:55752) 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f8a59c700 1 -- 172.21.15.49:0/668701251 learned_addr learned my addr 172.21.15.49:0/668701251 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.731+0000 7f2f89d9b700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c100770 0x7f2f8c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.732+0000 7f2f8a59c700 1 -- 172.21.15.49:0/668701251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c100770 msgr2=0x7f2f8c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.732+0000 7f2f8a59c700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c100770 0x7f2f8c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.732+0000 7f2f8a59c700 1 -- 172.21.15.49:0/668701251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c0095f0 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.732+0000 7f2f8a59c700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c0ffdc0 0x7f2f8c1979e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400e920 tx=0x7f2f7400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.732+0000 7f2f837fe700 1 -- 172.21.15.49:0/668701251 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f7400c9d0 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.732+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8c1a0a50 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.732+0000 7f2f837fe700 1 -- 172.21.15.49:0/668701251 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f74010430 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.732+0000 7f2f837fe700 1 -- 172.21.15.49:0/668701251 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f74018630 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.732+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8c1a0e60 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.733+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8c1081c0 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.734+0000 7f2f837fe700 1 -- 172.21.15.49:0/668701251 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2f740105a0 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.734+0000 7f2f837fe700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2f7806e200 0x7f2f780706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:38.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.734+0000 7f2f837fe700 1 -- 172.21.15.49:0/668701251 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2f74014070 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.734+0000 7f2f89d9b700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2f7806e200 0x7f2f780706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:38.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.735+0000 7f2f89d9b700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2f7806e200 0x7f2f780706c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0059d0 tx=0x7f2f7c005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:38.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.739+0000 7f2f837fe700 1 -- 172.21.15.49:0/668701251 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2f74053850 con 0x7f2f8c0ffdc0 2024-05-15T16:04:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:38 smithi049 ceph-mon[112142]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:38 smithi049 ceph-mon[112142]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:38 smithi049 ceph-mon[112142]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:38 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3805682483' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:04:38.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:38 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2284027001' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:04:38.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.967+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f8c10cd60 con 0x7f2f7806e200 2024-05-15T16:04:38.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.969+0000 7f2f837fe700 1 -- 172.21.15.49:0/668701251 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2f8c10cd60 con 0x7f2f7806e200 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:04:38.971 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:04:38.972 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:04:38.972 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:04:38.972 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:04:38.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2f7806e200 msgr2=0x7f2f780706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2f7806e200 0x7f2f780706c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0059d0 tx=0x7f2f7c005940 comp rx=0 tx=0).stop 2024-05-15T16:04:38.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c0ffdc0 msgr2=0x7f2f8c1979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:38.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c0ffdc0 0x7f2f8c1979e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400e920 tx=0x7f2f7400ec30 comp rx=0 tx=0).stop 2024-05-15T16:04:38.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 shutdown_connections 2024-05-15T16:04:38.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2f7806e200 0x7f2f780706c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f8c0ffdc0 0x7f2f8c1979e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 --2- 172.21.15.49:0/668701251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c100770 0x7f2f8c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:38.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 >> 172.21.15.49:0/668701251 conn(0x7f2f8c076800 msgr2=0x7f2f8c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:38.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 shutdown_connections 2024-05-15T16:04:38.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:38.971+0000 7f2f90bcb700 1 -- 172.21.15.49:0/668701251 wait complete. 2024-05-15T16:04:39.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.079+0000 7f4dfb880700 1 -- 172.21.15.49:0/2770681254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 msgr2=0x7f4df4103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:39.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.079+0000 7f4dfb880700 1 --2- 172.21.15.49:0/2770681254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 0x7f4df4103fa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4de400aa20 tx=0x7f4de400ad30 comp rx=0 tx=0).stop 2024-05-15T16:04:39.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.079+0000 7f4dfb880700 1 -- 172.21.15.49:0/2770681254 shutdown_connections 2024-05-15T16:04:39.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.079+0000 7f4dfb880700 1 --2- 172.21.15.49:0/2770681254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 0x7f4df4103fa0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:39.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.079+0000 7f4dfb880700 1 --2- 172.21.15.49:0/2770681254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4df40ff5a0 0x7f4df40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:39.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.079+0000 7f4dfb880700 1 -- 172.21.15.49:0/2770681254 >> 172.21.15.49:0/2770681254 conn(0x7f4df4076000 msgr2=0x7f4df40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:39.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.080+0000 7f4dfb880700 1 -- 172.21.15.49:0/2770681254 shutdown_connections 2024-05-15T16:04:39.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.080+0000 7f4dfb880700 1 -- 172.21.15.49:0/2770681254 wait complete. 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.080+0000 7f4dfb880700 1 Processor -- start 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.080+0000 7f4dfb880700 1 -- start start 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.080+0000 7f4dfb880700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4df40ff5a0 0x7f4df419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.080+0000 7f4dfb880700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 0x7f4df419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.080+0000 7f4dfb880700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4196ad0 con 0x7f4df40fff50 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.080+0000 7f4dfb880700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4196c40 con 0x7f4df40ff5a0 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4df8e1b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 0x7f4df419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4df8e1b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 0x7f4df419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55768/0 (socket says 172.21.15.49:55768) 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4df8e1b700 1 -- 172.21.15.49:0/155382200 learned_addr learned my addr 172.21.15.49:0/155382200 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:04:39.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4df961c700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4df40ff5a0 0x7f4df419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:39.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4df8e1b700 1 -- 172.21.15.49:0/155382200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4df40ff5a0 msgr2=0x7f4df419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:39.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4df8e1b700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4df40ff5a0 0x7f4df419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:39.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4df8e1b700 1 -- 172.21.15.49:0/155382200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de400a700 con 0x7f4df40fff50 2024-05-15T16:04:39.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4df8e1b700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 0x7f4df419cf10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4de400d010 tx=0x7f4de400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:39.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4dea7fc700 1 -- 172.21.15.49:0/155382200 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de4003a60 con 0x7f4df40fff50 2024-05-15T16:04:39.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4dea7fc700 1 -- 172.21.15.49:0/155382200 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4de4003bc0 con 0x7f4df40fff50 2024-05-15T16:04:39.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df4196ec0 con 0x7f4df40fff50 2024-05-15T16:04:39.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4dea7fc700 1 -- 172.21.15.49:0/155382200 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de4016680 con 0x7f4df40fff50 2024-05-15T16:04:39.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.081+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df41973b0 con 0x7f4df40fff50 2024-05-15T16:04:39.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.082+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df41099b0 con 0x7f4df40fff50 2024-05-15T16:04:39.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.083+0000 7f4dea7fc700 1 -- 172.21.15.49:0/155382200 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4de400eaa0 con 0x7f4df40fff50 2024-05-15T16:04:39.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.083+0000 7f4dea7fc700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4de006e200 0x7f4de00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:04:39.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.084+0000 7f4dea7fc700 1 -- 172.21.15.49:0/155382200 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4de4012070 con 0x7f4df40fff50 2024-05-15T16:04:39.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.084+0000 7f4df961c700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4de006e200 0x7f4de00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:04:39.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.084+0000 7f4df961c700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4de006e200 0x7f4de00706c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4df0005950 tx=0x7f4df000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:04:39.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.088+0000 7f4dea7fc700 1 -- 172.21.15.49:0/155382200 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4de4058fd0 con 0x7f4df40fff50 2024-05-15T16:04:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:38 smithi144 ceph-mon[86635]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:38 smithi144 ceph-mon[86635]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:38 smithi144 ceph-mon[86635]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:38 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3805682483' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:04:39.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:38 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2284027001' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:04:39.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.382+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4df4197f60 con 0x7f4df40fff50 2024-05-15T16:04:39.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.383+0000 7f4dea7fc700 1 -- 172.21.15.49:0/155382200 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f4de405c690 con 0x7f4df40fff50 2024-05-15T16:04:39.385 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:04:39.385 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:04:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:04:39.385 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:04:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:04:39.385 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:04:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4de006e200 msgr2=0x7f4de00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4de006e200 0x7f4de00706c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4df0005950 tx=0x7f4df000b410 comp rx=0 tx=0).stop 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 msgr2=0x7f4df419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 0x7f4df419cf10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4de400d010 tx=0x7f4de400caf0 comp rx=0 tx=0).stop 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 shutdown_connections 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4de006e200 0x7f4de00706c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4df40ff5a0 0x7f4df419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 --2- 172.21.15.49:0/155382200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4df40fff50 0x7f4df419cf10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 >> 172.21.15.49:0/155382200 conn(0x7f4df4076000 msgr2=0x7f4df4103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 shutdown_connections 2024-05-15T16:04:39.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:04:39.385+0000 7f4dfb880700 1 -- 172.21.15.49:0/155382200 wait complete. 2024-05-15T16:04:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:39 smithi144 ceph-mon[86635]: pgmap v228: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:39 smithi144 ceph-mon[86635]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:39 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/155382200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:04:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:39 smithi049 ceph-mon[112142]: pgmap v228: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:39 smithi049 ceph-mon[112142]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:04:40.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:39 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/155382200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:04:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:41 smithi144 ceph-mon[86635]: pgmap v229: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:41 smithi049 ceph-mon[112142]: pgmap v229: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:43.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:43 smithi049 ceph-mon[112142]: pgmap v230: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:43 smithi144 ceph-mon[86635]: pgmap v230: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:45 smithi144 ceph-mon[86635]: pgmap v231: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:45 smithi049 ceph-mon[112142]: pgmap v231: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:47 smithi144 ceph-mon[86635]: pgmap v232: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:48.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:47 smithi049 ceph-mon[112142]: pgmap v232: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:49 smithi144 ceph-mon[86635]: pgmap v233: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:49 smithi049 ceph-mon[112142]: pgmap v233: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:51.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:04:51.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:04:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:51 smithi144 ceph-mon[86635]: pgmap v234: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:51 smithi049 ceph-mon[112142]: pgmap v234: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:53 smithi144 ceph-mon[86635]: pgmap v235: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:54.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:53 smithi049 ceph-mon[112142]: pgmap v235: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:55 smithi144 ceph-mon[86635]: pgmap v236: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:55 smithi049 ceph-mon[112142]: pgmap v236: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:57 smithi144 ceph-mon[86635]: pgmap v237: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:04:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:57 smithi049 ceph-mon[112142]: pgmap v237: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:00.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:04:59 smithi144 ceph-mon[86635]: pgmap v238: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:04:59 smithi049 ceph-mon[112142]: pgmap v238: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:01 smithi144 ceph-mon[86635]: pgmap v239: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:02.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:01 smithi049 ceph-mon[112142]: pgmap v239: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:03 smithi144 ceph-mon[86635]: pgmap v240: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:03 smithi049 ceph-mon[112142]: pgmap v240: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:05 smithi144 ceph-mon[86635]: pgmap v241: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:05:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:05 smithi049 ceph-mon[112142]: pgmap v241: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:05:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:07 smithi144 ceph-mon[86635]: pgmap v242: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:07 smithi049 ceph-mon[112142]: pgmap v242: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.510+0000 7f31d7de7700 1 -- 172.21.15.49:0/2906526043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073c60 msgr2=0x7f31d0111750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.510+0000 7f31d7de7700 1 --2- 172.21.15.49:0/2906526043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073c60 0x7f31d0111750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009910 tx=0x7f31c0009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.510+0000 7f31d7de7700 1 -- 172.21.15.49:0/2906526043 shutdown_connections 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.510+0000 7f31d7de7700 1 --2- 172.21.15.49:0/2906526043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073c60 0x7f31d0111750 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.510+0000 7f31d7de7700 1 --2- 172.21.15.49:0/2906526043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31d0073340 0x7f31d0073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.510+0000 7f31d7de7700 1 -- 172.21.15.49:0/2906526043 >> 172.21.15.49:0/2906526043 conn(0x7f31d00fc790 msgr2=0x7f31d00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.510+0000 7f31d7de7700 1 -- 172.21.15.49:0/2906526043 shutdown_connections 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.510+0000 7f31d7de7700 1 -- 172.21.15.49:0/2906526043 wait complete. 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d7de7700 1 Processor -- start 2024-05-15T16:05:09.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d7de7700 1 -- start start 2024-05-15T16:05:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d7de7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073340 0x7f31d019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d7de7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31d0073c60 0x7f31d019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d7de7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d019de10 con 0x7f31d0073340 2024-05-15T16:05:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d7de7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d01a1ae0 con 0x7f31d0073c60 2024-05-15T16:05:09.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d5b83700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073340 0x7f31d019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d5b83700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073340 0x7f31d019d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32866/0 (socket says 172.21.15.49:32866) 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d5b83700 1 -- 172.21.15.49:0/1884656260 learned_addr learned my addr 172.21.15.49:0/1884656260 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.511+0000 7f31d5382700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31d0073c60 0x7f31d019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.512+0000 7f31d5b83700 1 -- 172.21.15.49:0/1884656260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31d0073c60 msgr2=0x7f31d019d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.512+0000 7f31d5b83700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31d0073c60 0x7f31d019d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.512+0000 7f31d5b83700 1 -- 172.21.15.49:0/1884656260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c00095f0 con 0x7f31d0073340 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.512+0000 7f31d5b83700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073340 0x7f31d019d1f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00d8d0 tx=0x7f31cc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.512+0000 7f31c6ffd700 1 -- 172.21.15.49:0/1884656260 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31cc0098a0 con 0x7f31d0073340 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.512+0000 7f31c6ffd700 1 -- 172.21.15.49:0/1884656260 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31cc010460 con 0x7f31d0073340 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.512+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d01a1dc0 con 0x7f31d0073340 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.512+0000 7f31c6ffd700 1 -- 172.21.15.49:0/1884656260 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31cc00f5f0 con 0x7f31d0073340 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.512+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d01a22e0 con 0x7f31d0073340 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.513+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d0074a80 con 0x7f31d0073340 2024-05-15T16:05:09.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.514+0000 7f31c6ffd700 1 -- 172.21.15.49:0/1884656260 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f31cc009a00 con 0x7f31d0073340 2024-05-15T16:05:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.514+0000 7f31c6ffd700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f31bc06e210 0x7f31bc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.514+0000 7f31c6ffd700 1 -- 172.21.15.49:0/1884656260 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f31cc08df70 con 0x7f31d0073340 2024-05-15T16:05:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.514+0000 7f31d5382700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f31bc06e210 0x7f31bc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:09.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.515+0000 7f31d5382700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f31bc06e210 0x7f31bc0706d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f31c0006010 tx=0x7f31c001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:09.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.519+0000 7f31c6ffd700 1 -- 172.21.15.49:0/1884656260 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f31cc0c2850 con 0x7f31d0073340 2024-05-15T16:05:09.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.757+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31d01144b0 con 0x7f31bc06e210 2024-05-15T16:05:09.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.758+0000 7f31c6ffd700 1 -- 172.21.15.49:0/1884656260 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f31d01144b0 con 0x7f31bc06e210 2024-05-15T16:05:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f31bc06e210 msgr2=0x7f31bc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f31bc06e210 0x7f31bc0706d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f31c0006010 tx=0x7f31c001a040 comp rx=0 tx=0).stop 2024-05-15T16:05:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073340 msgr2=0x7f31d019d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073340 0x7f31d019d1f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00d8d0 tx=0x7f31cc00dbe0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 shutdown_connections 2024-05-15T16:05:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f31bc06e210 0x7f31bc0706d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31d0073340 0x7f31d019d1f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 --2- 172.21.15.49:0/1884656260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31d0073c60 0x7f31d019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 >> 172.21.15.49:0/1884656260 conn(0x7f31d00fc790 msgr2=0x7f31d01009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 shutdown_connections 2024-05-15T16:05:09.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.760+0000 7f31d7de7700 1 -- 172.21.15.49:0/1884656260 wait complete. 2024-05-15T16:05:09.772 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:05:09.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.870+0000 7fbdaa338700 1 -- 172.21.15.49:0/3806234379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 msgr2=0x7fbda41012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.870+0000 7fbdaa338700 1 --2- 172.21.15.49:0/3806234379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 0x7fbda41012e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009910 tx=0x7fbd94009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.870+0000 7fbdaa338700 1 -- 172.21.15.49:0/3806234379 shutdown_connections 2024-05-15T16:05:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.870+0000 7fbdaa338700 1 --2- 172.21.15.49:0/3806234379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda41018b0 0x7fbda4105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.870+0000 7fbdaa338700 1 --2- 172.21.15.49:0/3806234379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 0x7fbda41012e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.870+0000 7fbdaa338700 1 -- 172.21.15.49:0/3806234379 >> 172.21.15.49:0/3806234379 conn(0x7fbda40fc7d0 msgr2=0x7fbda40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.870+0000 7fbdaa338700 1 -- 172.21.15.49:0/3806234379 shutdown_connections 2024-05-15T16:05:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.870+0000 7fbdaa338700 1 -- 172.21.15.49:0/3806234379 wait complete. 2024-05-15T16:05:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.871+0000 7fbdaa338700 1 Processor -- start 2024-05-15T16:05:09.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.871+0000 7fbdaa338700 1 -- start start 2024-05-15T16:05:09.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.871+0000 7fbdaa338700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 0x7fbda419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:09.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.871+0000 7fbdaa338700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda41018b0 0x7fbda419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:09.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.871+0000 7fbdaa338700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda419d460 con 0x7fbda4100f00 2024-05-15T16:05:09.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.871+0000 7fbdaa338700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4196a90 con 0x7fbda41018b0 2024-05-15T16:05:09.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.871+0000 7fbda3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 0x7fbda419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:09.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.871+0000 7fbda3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 0x7fbda419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32880/0 (socket says 172.21.15.49:32880) 2024-05-15T16:05:09.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.871+0000 7fbda3fff700 1 -- 172.21.15.49:0/4257276108 learned_addr learned my addr 172.21.15.49:0/4257276108 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:09.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbda37fe700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda41018b0 0x7fbda419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:09.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbda3fff700 1 -- 172.21.15.49:0/4257276108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda41018b0 msgr2=0x7fbda419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbda3fff700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda41018b0 0x7fbda419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbda3fff700 1 -- 172.21.15.49:0/4257276108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd940095f0 con 0x7fbda4100f00 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbda37fe700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda41018b0 0x7fbda419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbda3fff700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 0x7fbda419c900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd94000c00 tx=0x7fbd9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbda17fa700 1 -- 172.21.15.49:0/4257276108 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd9401d070 con 0x7fbda4100f00 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbda17fa700 1 -- 172.21.15.49:0/4257276108 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd940056e0 con 0x7fbda4100f00 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda4196d10 con 0x7fbda4100f00 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbda17fa700 1 -- 172.21.15.49:0/4257276108 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd9400fe80 con 0x7fbda4100f00 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.872+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda4197200 con 0x7fbda4100f00 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.873+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda41092a0 con 0x7fbda4100f00 2024-05-15T16:05:09.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.874+0000 7fbda17fa700 1 -- 172.21.15.49:0/4257276108 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbd9400f6f0 con 0x7fbda4100f00 2024-05-15T16:05:09.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.874+0000 7fbda17fa700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbd9006e210 0x7fbd900706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:09.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.874+0000 7fbda17fa700 1 -- 172.21.15.49:0/4257276108 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbd9408f4e0 con 0x7fbda4100f00 2024-05-15T16:05:09.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.874+0000 7fbda37fe700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbd9006e210 0x7fbd900706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:09.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.875+0000 7fbda37fe700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbd9006e210 0x7fbd900706d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c005fd0 tx=0x7fbd8c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:09.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:09.879+0000 7fbda17fa700 1 -- 172.21.15.49:0/4257276108 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbd9405c370 con 0x7fbda4100f00 2024-05-15T16:05:10.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.106+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbda410ded0 con 0x7fbd9006e210 2024-05-15T16:05:10.107 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:09 smithi049 ceph-mon[112142]: pgmap v243: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:10.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.107+0000 7fbda17fa700 1 -- 172.21.15.49:0/4257276108 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbda410ded0 con 0x7fbd9006e210 2024-05-15T16:05:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbd9006e210 msgr2=0x7fbd900706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbd9006e210 0x7fbd900706d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c005fd0 tx=0x7fbd8c005ee0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 msgr2=0x7fbda419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 0x7fbda419c900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd94000c00 tx=0x7fbd9400be20 comp rx=0 tx=0).stop 2024-05-15T16:05:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 shutdown_connections 2024-05-15T16:05:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbd9006e210 0x7fbd900706d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbda4100f00 0x7fbda419c900 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 --2- 172.21.15.49:0/4257276108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda41018b0 0x7fbda419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 >> 172.21.15.49:0/4257276108 conn(0x7fbda40fc7d0 msgr2=0x7fbda40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:10.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 shutdown_connections 2024-05-15T16:05:10.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.110+0000 7fbdaa338700 1 -- 172.21.15.49:0/4257276108 wait complete. 2024-05-15T16:05:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:09 smithi144 ceph-mon[86635]: pgmap v243: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.219+0000 7f92ffe11700 1 -- 172.21.15.49:0/2743674130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8100ed0 msgr2=0x7f92f81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.219+0000 7f92ffe11700 1 --2- 172.21.15.49:0/2743674130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8100ed0 0x7f92f81012b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009910 tx=0x7f92f4009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.219+0000 7f92ffe11700 1 -- 172.21.15.49:0/2743674130 shutdown_connections 2024-05-15T16:05:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.219+0000 7f92ffe11700 1 --2- 172.21.15.49:0/2743674130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f8101880 0x7f92f81058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.219+0000 7f92ffe11700 1 --2- 172.21.15.49:0/2743674130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8100ed0 0x7f92f81012b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.219+0000 7f92ffe11700 1 -- 172.21.15.49:0/2743674130 >> 172.21.15.49:0/2743674130 conn(0x7f92f80fc7a0 msgr2=0x7f92f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.219+0000 7f92ffe11700 1 -- 172.21.15.49:0/2743674130 shutdown_connections 2024-05-15T16:05:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.219+0000 7f92ffe11700 1 -- 172.21.15.49:0/2743674130 wait complete. 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92ffe11700 1 Processor -- start 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92ffe11700 1 -- start start 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92ffe11700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f8100ed0 0x7f92f819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92ffe11700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8101880 0x7f92f819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92ffe11700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f819d500 con 0x7f92f8101880 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92ffe11700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8196b30 con 0x7f92f8100ed0 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92fd3ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8101880 0x7f92f819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92fd3ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8101880 0x7f92f819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32894/0 (socket says 172.21.15.49:32894) 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92fd3ac700 1 -- 172.21.15.49:0/2044372013 learned_addr learned my addr 172.21.15.49:0/2044372013 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.220+0000 7f92fdbad700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f8100ed0 0x7f92f819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.221+0000 7f92fd3ac700 1 -- 172.21.15.49:0/2044372013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f8100ed0 msgr2=0x7f92f819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.221+0000 7f92fd3ac700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f8100ed0 0x7f92f819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.221+0000 7f92fd3ac700 1 -- 172.21.15.49:0/2044372013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40095f0 con 0x7f92f8101880 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.221+0000 7f92fd3ac700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8101880 0x7f92f819cee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b480 tx=0x7f92e800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.221+0000 7f92eeffd700 1 -- 172.21.15.49:0/2044372013 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e8011800 con 0x7f92f8101880 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.221+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f8196e10 con 0x7f92f8101880 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.221+0000 7f92eeffd700 1 -- 172.21.15.49:0/2044372013 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92e8011e20 con 0x7f92f8101880 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.221+0000 7f92eeffd700 1 -- 172.21.15.49:0/2044372013 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e8010ea0 con 0x7f92f8101880 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.221+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f8197360 con 0x7f92f8101880 2024-05-15T16:05:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.222+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f8109270 con 0x7f92f8101880 2024-05-15T16:05:10.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.223+0000 7f92eeffd700 1 -- 172.21.15.49:0/2044372013 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f92e8010380 con 0x7f92f8101880 2024-05-15T16:05:10.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.223+0000 7f92eeffd700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92e406e210 0x7f92e40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:10.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.223+0000 7f92eeffd700 1 -- 172.21.15.49:0/2044372013 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f92e808dc90 con 0x7f92f8101880 2024-05-15T16:05:10.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.223+0000 7f92fdbad700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92e406e210 0x7f92e40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:10.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.224+0000 7f92fdbad700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92e406e210 0x7f92e40706d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92f4006010 tx=0x7f92f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:10.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.228+0000 7f92eeffd700 1 -- 172.21.15.49:0/2044372013 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f92e805aaa0 con 0x7f92f8101880 2024-05-15T16:05:10.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.446+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f92f8198010 con 0x7f92e406e210 2024-05-15T16:05:10.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.457+0000 7f92eeffd700 1 -- 172.21.15.49:0/2044372013 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f92f8198010 con 0x7f92e406e210 2024-05-15T16:05:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:05:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (8m) 99s ago 19m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:05:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (20m) 99s ago 20m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:05:10.458 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (19m) 85s ago 19m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (5m) 99s ago 20m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (5m) 85s ago 19m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (7m) 99s ago 19m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (2m) 99s ago 14m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (105s) 99s ago 14m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (89s) 85s ago 14m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (14m) 85s ago 14m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (10m) 99s ago 21m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (9m) 85s ago 18m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (6m) 99s ago 21m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (6m) 85s ago 18m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (9m) 99s ago 20m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (9m) 85s ago 18m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:05:10.459 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (5m) 99s ago 18m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:05:10.460 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (4m) 99s ago 17m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:05:10.460 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (4m) 99s ago 17m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:05:10.460 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (3m) 85s ago 16m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:05:10.460 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (2m) 85s ago 16m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:05:10.460 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (2m) 85s ago 15m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:05:10.460 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (8m) 99s ago 19m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.459+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92e406e210 msgr2=0x7f92e40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.459+0000 7f92ffe11700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92e406e210 0x7f92e40706d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92f4006010 tx=0x7f92f401a040 comp rx=0 tx=0).stop 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.459+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8101880 msgr2=0x7f92f819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.459+0000 7f92ffe11700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8101880 0x7f92f819cee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b480 tx=0x7f92e800b790 comp rx=0 tx=0).stop 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.460+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 shutdown_connections 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.460+0000 7f92ffe11700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92e406e210 0x7f92e40706d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.460+0000 7f92ffe11700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f8100ed0 0x7f92f819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.460+0000 7f92ffe11700 1 --2- 172.21.15.49:0/2044372013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92f8101880 0x7f92f819cee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.460+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 >> 172.21.15.49:0/2044372013 conn(0x7f92f80fc7a0 msgr2=0x7f92f80fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.460+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 shutdown_connections 2024-05-15T16:05:10.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.460+0000 7f92ffe11700 1 -- 172.21.15.49:0/2044372013 wait complete. 2024-05-15T16:05:10.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.567+0000 7f00432c6700 1 -- 172.21.15.49:0/1146230538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c1018b0 msgr2=0x7f003c105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.567+0000 7f00432c6700 1 --2- 172.21.15.49:0/1146230538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c1018b0 0x7f003c105900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0030009910 tx=0x7f0030009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.567+0000 7f00432c6700 1 -- 172.21.15.49:0/1146230538 shutdown_connections 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.567+0000 7f00432c6700 1 --2- 172.21.15.49:0/1146230538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c1018b0 0x7f003c105900 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.567+0000 7f00432c6700 1 --2- 172.21.15.49:0/1146230538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003c100f00 0x7f003c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.567+0000 7f00432c6700 1 -- 172.21.15.49:0/1146230538 >> 172.21.15.49:0/1146230538 conn(0x7f003c0fc7d0 msgr2=0x7f003c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.567+0000 7f00432c6700 1 -- 172.21.15.49:0/1146230538 shutdown_connections 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.567+0000 7f00432c6700 1 -- 172.21.15.49:0/1146230538 wait complete. 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.567+0000 7f00432c6700 1 Processor -- start 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f00432c6700 1 -- start start 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f00432c6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c100f00 0x7f003c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f00432c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003c1018b0 0x7f003c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f00432c6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c073d20 con 0x7f003c100f00 2024-05-15T16:05:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f00432c6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c073e60 con 0x7f003c1018b0 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f0041062700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c100f00 0x7f003c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f0041062700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c100f00 0x7f003c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32904/0 (socket says 172.21.15.49:32904) 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f0041062700 1 -- 172.21.15.49:0/900274657 learned_addr learned my addr 172.21.15.49:0/900274657 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f0040861700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003c1018b0 0x7f003c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f0041062700 1 -- 172.21.15.49:0/900274657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003c1018b0 msgr2=0x7f003c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f0041062700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003c1018b0 0x7f003c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f0041062700 1 -- 172.21.15.49:0/900274657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00300095f0 con 0x7f003c100f00 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f0041062700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c100f00 0x7f003c0732a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f003800d900 tx=0x7f003800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f002e7fc700 1 -- 172.21.15.49:0/900274657 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0038004a00 con 0x7f003c100f00 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f002e7fc700 1 -- 172.21.15.49:0/900274657 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00380054e0 con 0x7f003c100f00 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.568+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c1a2e60 con 0x7f003c100f00 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.569+0000 7f002e7fc700 1 -- 172.21.15.49:0/900274657 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0038009d60 con 0x7f003c100f00 2024-05-15T16:05:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.569+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c1a31c0 con 0x7f003c100f00 2024-05-15T16:05:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.570+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c109300 con 0x7f003c100f00 2024-05-15T16:05:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.570+0000 7f002e7fc700 1 -- 172.21.15.49:0/900274657 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0038005020 con 0x7f003c100f00 2024-05-15T16:05:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.571+0000 7f002e7fc700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002806e200 0x7f00280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:10.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.571+0000 7f002e7fc700 1 -- 172.21.15.49:0/900274657 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f003808e2f0 con 0x7f003c100f00 2024-05-15T16:05:10.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.571+0000 7f0040861700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002806e200 0x7f00280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:10.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.571+0000 7f0040861700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002806e200 0x7f00280706c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f003c074980 tx=0x7f003001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.575+0000 7f002e7fc700 1 -- 172.21.15.49:0/900274657 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0038057650 con 0x7f003c100f00 2024-05-15T16:05:10.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.871+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f003c0740e0 con 0x7f003c100f00 2024-05-15T16:05:10.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.871+0000 7f002e7fc700 1 -- 172.21.15.49:0/900274657 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f003805ad10 con 0x7f003c100f00 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:05:10.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:05:10.875 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:05:10.875 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:05:10.875 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:05:10.875 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:05:10.875 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002806e200 msgr2=0x7f00280706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002806e200 0x7f00280706c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f003c074980 tx=0x7f003001a040 comp rx=0 tx=0).stop 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c100f00 msgr2=0x7f003c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c100f00 0x7f003c0732a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f003800d900 tx=0x7f003800dcc0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 shutdown_connections 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002806e200 0x7f00280706c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c100f00 0x7f003c0732a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 --2- 172.21.15.49:0/900274657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003c1018b0 0x7f003c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 >> 172.21.15.49:0/900274657 conn(0x7f003c0fc7d0 msgr2=0x7f003c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 shutdown_connections 2024-05-15T16:05:10.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.874+0000 7f00432c6700 1 -- 172.21.15.49:0/900274657 wait complete. 2024-05-15T16:05:10.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.983+0000 7fa58ad46700 1 -- 172.21.15.49:0/2413066176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 msgr2=0x7fa5841047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.983+0000 7fa58ad46700 1 --2- 172.21.15.49:0/2413066176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5841047c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa578009910 tx=0x7fa578009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:10.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.983+0000 7fa58ad46700 1 -- 172.21.15.49:0/2413066176 shutdown_connections 2024-05-15T16:05:10.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.983+0000 7fa58ad46700 1 --2- 172.21.15.49:0/2413066176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5841047c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.983+0000 7fa58ad46700 1 --2- 172.21.15.49:0/2413066176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5840ffdc0 0x7fa5841001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.983+0000 7fa58ad46700 1 -- 172.21.15.49:0/2413066176 >> 172.21.15.49:0/2413066176 conn(0x7fa584076800 msgr2=0x7fa5840fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.983+0000 7fa58ad46700 1 -- 172.21.15.49:0/2413066176 shutdown_connections 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.983+0000 7fa58ad46700 1 -- 172.21.15.49:0/2413066176 wait complete. 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa58ad46700 1 Processor -- start 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa58ad46700 1 -- start start 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa58ad46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa58ad46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa584100770 0x7fa5840fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa58ad46700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840fc760 con 0x7fa5840ffdc0 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa58ad46700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840fc8a0 con 0x7fa584100770 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa588ae2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa588ae2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32926/0 (socket says 172.21.15.49:32926) 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa588ae2700 1 -- 172.21.15.49:0/1896670688 learned_addr learned my addr 172.21.15.49:0/1896670688 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:10.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa583fff700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa584100770 0x7fa5840fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:10.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa588ae2700 1 -- 172.21.15.49:0/1896670688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa584100770 msgr2=0x7fa5840fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:10.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa588ae2700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa584100770 0x7fa5840fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:10.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.984+0000 7fa588ae2700 1 -- 172.21.15.49:0/1896670688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780095f0 con 0x7fa5840ffdc0 2024-05-15T16:05:10.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.985+0000 7fa588ae2700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa57400d900 tx=0x7fa57400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:10.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.985+0000 7fa581ffb700 1 -- 172.21.15.49:0/1896670688 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa574004a00 con 0x7fa5840ffdc0 2024-05-15T16:05:10.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.985+0000 7fa581ffb700 1 -- 172.21.15.49:0/1896670688 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5740054e0 con 0x7fa5840ffdc0 2024-05-15T16:05:10.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.985+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5841a2d80 con 0x7fa5840ffdc0 2024-05-15T16:05:10.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.985+0000 7fa581ffb700 1 -- 172.21.15.49:0/1896670688 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa574009d00 con 0x7fa5840ffdc0 2024-05-15T16:05:10.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.985+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5841a30e0 con 0x7fa5840ffdc0 2024-05-15T16:05:10.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.986+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5841012a0 con 0x7fa5840ffdc0 2024-05-15T16:05:10.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.986+0000 7fa581ffb700 1 -- 172.21.15.49:0/1896670688 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa574005020 con 0x7fa5840ffdc0 2024-05-15T16:05:10.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.987+0000 7fa581ffb700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e200 0x7fa56c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:10.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.987+0000 7fa581ffb700 1 -- 172.21.15.49:0/1896670688 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa57408e020 con 0x7fa5840ffdc0 2024-05-15T16:05:10.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.987+0000 7fa583fff700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e200 0x7fa56c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:10.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.988+0000 7fa583fff700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e200 0x7fa56c0706c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa5840fd410 tx=0x7fa57801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:10.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:10.992+0000 7fa581ffb700 1 -- 172.21.15.49:0/1896670688 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa574057380 con 0x7fa5840ffdc0 2024-05-15T16:05:11.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:10 smithi049 ceph-mon[112142]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:11.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:10 smithi144 ceph-mon[86635]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:11.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.249+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa58404ec60 con 0x7fa5840ffdc0 2024-05-15T16:05:11.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.249+0000 7fa581ffb700 1 -- 172.21.15.49:0/1896670688 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7fa574017070 con 0x7fa5840ffdc0 2024-05-15T16:05:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:05:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:05:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:05:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:05:11.252 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:05:11.253 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:05:11.253 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:05:11.253 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:05:11.253 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:05:11.253 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:05:11.253 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:05:11.253 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:05:11.254 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:05:11.254 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:05:11.254 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:05:11.254 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:05:11.254 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:05:11.254 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:05:11.254 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:05:11.254 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:05:11.255 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:05:11.256 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:05:11.256 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:05:11.256 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:05:11.256 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:05:11.256 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:05:11.256 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:05:11.256 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e200 msgr2=0x7fa56c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e200 0x7fa56c0706c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa5840fd410 tx=0x7fa57801a040 comp rx=0 tx=0).stop 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5840ffdc0 msgr2=0x7fa5840fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa57400d900 tx=0x7fa57400dc10 comp rx=0 tx=0).stop 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 shutdown_connections 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e200 0x7fa56c0706c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 --2- 172.21.15.49:0/1896670688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa584100770 0x7fa5840fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 >> 172.21.15.49:0/1896670688 conn(0x7fa584076800 msgr2=0x7fa5840fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:11.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 shutdown_connections 2024-05-15T16:05:11.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.251+0000 7fa58ad46700 1 -- 172.21.15.49:0/1896670688 wait complete. 2024-05-15T16:05:11.258 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:05:11.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.361+0000 7fcd283fe700 1 -- 172.21.15.49:0/2036648745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 msgr2=0x7fcd201047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.361+0000 7fcd283fe700 1 --2- 172.21.15.49:0/2036648745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 0x7fcd201047c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009910 tx=0x7fcd10009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.361+0000 7fcd283fe700 1 -- 172.21.15.49:0/2036648745 shutdown_connections 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.361+0000 7fcd283fe700 1 --2- 172.21.15.49:0/2036648745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 0x7fcd201047c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.361+0000 7fcd283fe700 1 --2- 172.21.15.49:0/2036648745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd200ffdc0 0x7fcd201001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.361+0000 7fcd283fe700 1 -- 172.21.15.49:0/2036648745 >> 172.21.15.49:0/2036648745 conn(0x7fcd20076800 msgr2=0x7fcd200fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.361+0000 7fcd283fe700 1 -- 172.21.15.49:0/2036648745 shutdown_connections 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.361+0000 7fcd283fe700 1 -- 172.21.15.49:0/2036648745 wait complete. 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd283fe700 1 Processor -- start 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd283fe700 1 -- start start 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd283fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd200ffdc0 0x7fcd2019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd283fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 0x7fcd2019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd283fe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20196990 con 0x7fcd20100770 2024-05-15T16:05:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd283fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20196b00 con 0x7fcd200ffdc0 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd25999700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 0x7fcd2019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd25999700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 0x7fcd2019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32940/0 (socket says 172.21.15.49:32940) 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd25999700 1 -- 172.21.15.49:0/1257052238 learned_addr learned my addr 172.21.15.49:0/1257052238 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.362+0000 7fcd2619a700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd200ffdc0 0x7fcd2019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.363+0000 7fcd25999700 1 -- 172.21.15.49:0/1257052238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd200ffdc0 msgr2=0x7fcd2019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.363+0000 7fcd25999700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd200ffdc0 0x7fcd2019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.363+0000 7fcd25999700 1 -- 172.21.15.49:0/1257052238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100095f0 con 0x7fcd20100770 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.363+0000 7fcd25999700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 0x7fcd2019ce20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000c00 tx=0x7fcd1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.363+0000 7fcd177fe700 1 -- 172.21.15.49:0/1257052238 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd1001d070 con 0x7fcd20100770 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.363+0000 7fcd177fe700 1 -- 172.21.15.49:0/1257052238 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd10005700 con 0x7fcd20100770 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.363+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd20196d80 con 0x7fcd20100770 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.363+0000 7fcd177fe700 1 -- 172.21.15.49:0/1257052238 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd1000fe50 con 0x7fcd20100770 2024-05-15T16:05:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.363+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd20197270 con 0x7fcd20100770 2024-05-15T16:05:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.364+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd201012a0 con 0x7fcd20100770 2024-05-15T16:05:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.365+0000 7fcd177fe700 1 -- 172.21.15.49:0/1257052238 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcd1000f640 con 0x7fcd20100770 2024-05-15T16:05:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.365+0000 7fcd177fe700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd0c06e210 0x7fcd0c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.365+0000 7fcd177fe700 1 -- 172.21.15.49:0/1257052238 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcd1008f520 con 0x7fcd20100770 2024-05-15T16:05:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.365+0000 7fcd2619a700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd0c06e210 0x7fcd0c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.366+0000 7fcd2619a700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd0c06e210 0x7fcd0c0706d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c005950 tx=0x7fcd1c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.370+0000 7fcd177fe700 1 -- 172.21.15.49:0/1257052238 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcd10058880 con 0x7fcd20100770 2024-05-15T16:05:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.597+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd2010cd60 con 0x7fcd0c06e210 2024-05-15T16:05:11.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.598+0000 7fcd177fe700 1 -- 172.21.15.49:0/1257052238 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcd2010cd60 con 0x7fcd0c06e210 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:05:11.601 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:05:11.602 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:05:11.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd0c06e210 msgr2=0x7fcd0c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:11.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd0c06e210 0x7fcd0c0706d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c005950 tx=0x7fcd1c0058e0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 msgr2=0x7fcd2019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:11.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 0x7fcd2019ce20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000c00 tx=0x7fcd1000be20 comp rx=0 tx=0).stop 2024-05-15T16:05:11.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 shutdown_connections 2024-05-15T16:05:11.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd0c06e210 0x7fcd0c0706d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd200ffdc0 0x7fcd2019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 --2- 172.21.15.49:0/1257052238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20100770 0x7fcd2019ce20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 >> 172.21.15.49:0/1257052238 conn(0x7fcd20076800 msgr2=0x7fcd200fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:11.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 shutdown_connections 2024-05-15T16:05:11.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.601+0000 7fcd283fe700 1 -- 172.21.15.49:0/1257052238 wait complete. 2024-05-15T16:05:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.710+0000 7f55a8c57700 1 -- 172.21.15.49:0/913335955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 msgr2=0x7f55a4102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.710+0000 7f55a8c57700 1 --2- 172.21.15.49:0/913335955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 0x7f55a4102250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f559400b0a0 tx=0x7f559400b3b0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.710+0000 7f55a8c57700 1 -- 172.21.15.49:0/913335955 shutdown_connections 2024-05-15T16:05:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.710+0000 7f55a8c57700 1 --2- 172.21.15.49:0/913335955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a4102790 0x7f55a410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.710+0000 7f55a8c57700 1 --2- 172.21.15.49:0/913335955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 0x7f55a4102250 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.710+0000 7f55a8c57700 1 -- 172.21.15.49:0/913335955 >> 172.21.15.49:0/913335955 conn(0x7f55a40fb700 msgr2=0x7f55a40fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.710+0000 7f55a8c57700 1 -- 172.21.15.49:0/913335955 shutdown_connections 2024-05-15T16:05:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.710+0000 7f55a8c57700 1 -- 172.21.15.49:0/913335955 wait complete. 2024-05-15T16:05:11.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.711+0000 7f55a8c57700 1 Processor -- start 2024-05-15T16:05:11.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.711+0000 7f55a8c57700 1 -- start start 2024-05-15T16:05:11.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.711+0000 7f55a8c57700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 0x7f55a419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.711+0000 7f55a8c57700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a4102790 0x7f55a419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.711+0000 7f55a8c57700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a419d4b0 con 0x7f55a4101e70 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.711+0000 7f55a8c57700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4196a90 con 0x7f55a4102790 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.711+0000 7f55a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 0x7f55a419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.711+0000 7f55a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 0x7f55a419c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32958/0 (socket says 172.21.15.49:32958) 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.711+0000 7f55a259c700 1 -- 172.21.15.49:0/2230962202 learned_addr learned my addr 172.21.15.49:0/2230962202 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f55a1d9b700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a4102790 0x7f55a419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f55a259c700 1 -- 172.21.15.49:0/2230962202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a4102790 msgr2=0x7f55a419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f55a259c700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a4102790 0x7f55a419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f55a259c700 1 -- 172.21.15.49:0/2230962202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5594009d00 con 0x7f55a4101e70 2024-05-15T16:05:11.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f55a259c700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 0x7f55a419c950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5594015040 tx=0x7f5594003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f559b7fe700 1 -- 172.21.15.49:0/2230962202 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559400e040 con 0x7f55a4101e70 2024-05-15T16:05:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a4196d10 con 0x7f55a4101e70 2024-05-15T16:05:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f559b7fe700 1 -- 172.21.15.49:0/2230962202 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5594004040 con 0x7f55a4101e70 2024-05-15T16:05:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f559b7fe700 1 -- 172.21.15.49:0/2230962202 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559401d6b0 con 0x7f55a4101e70 2024-05-15T16:05:11.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.712+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a4197200 con 0x7f55a4101e70 2024-05-15T16:05:11.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.713+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a4105600 con 0x7f55a4101e70 2024-05-15T16:05:11.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.714+0000 7f559b7fe700 1 -- 172.21.15.49:0/2230962202 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5594007b00 con 0x7f55a4101e70 2024-05-15T16:05:11.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.714+0000 7f559b7fe700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f559006e200 0x7f55900706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:11.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.714+0000 7f559b7fe700 1 -- 172.21.15.49:0/2230962202 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f559408f960 con 0x7f55a4101e70 2024-05-15T16:05:11.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.714+0000 7f55a1d9b700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f559006e200 0x7f55900706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:11.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.715+0000 7f55a1d9b700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f559006e200 0x7f55900706c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f558c005950 tx=0x7f558c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:11.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:11.719+0000 7f559b7fe700 1 -- 172.21.15.49:0/2230962202 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f559405c770 con 0x7f55a4101e70 2024-05-15T16:05:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:11 smithi049 ceph-mon[112142]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:11 smithi049 ceph-mon[112142]: pgmap v244: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:11 smithi049 ceph-mon[112142]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:11 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/900274657' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:05:11.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:11 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1896670688' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:05:12.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.011+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f55a4197f50 con 0x7f55a4101e70 2024-05-15T16:05:12.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.011+0000 7f559b7fe700 1 -- 172.21.15.49:0/2230962202 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f559405c300 con 0x7f55a4101e70 2024-05-15T16:05:12.014 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:05:12.014 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:05:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:05:12.014 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:05:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:05:12.014 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:05:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:05:12.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.013+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f559006e200 msgr2=0x7f55900706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:12.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.013+0000 7f55a8c57700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f559006e200 0x7f55900706c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f558c005950 tx=0x7f558c0058e0 comp rx=0 tx=0).stop 2024-05-15T16:05:12.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.013+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 msgr2=0x7f55a419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:12.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.013+0000 7f55a8c57700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 0x7f55a419c950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5594015040 tx=0x7f5594003ce0 comp rx=0 tx=0).stop 2024-05-15T16:05:12.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.013+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 shutdown_connections 2024-05-15T16:05:12.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.013+0000 7f55a8c57700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f559006e200 0x7f55900706c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:12.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.013+0000 7f55a8c57700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55a4101e70 0x7f55a419c950 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:12.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.013+0000 7f55a8c57700 1 --2- 172.21.15.49:0/2230962202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a4102790 0x7f55a419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:12.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.014+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 >> 172.21.15.49:0/2230962202 conn(0x7f55a40fb700 msgr2=0x7f55a4103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:12.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.014+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 shutdown_connections 2024-05-15T16:05:12.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:12.014+0000 7f55a8c57700 1 -- 172.21.15.49:0/2230962202 wait complete. 2024-05-15T16:05:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:11 smithi144 ceph-mon[86635]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:11 smithi144 ceph-mon[86635]: pgmap v244: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:11 smithi144 ceph-mon[86635]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:11 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/900274657' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:05:12.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:11 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1896670688' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:05:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:12 smithi144 ceph-mon[86635]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:13.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:12 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2230962202' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:05:13.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:12 smithi049 ceph-mon[112142]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:13.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:12 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2230962202' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:05:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:13 smithi144 ceph-mon[86635]: pgmap v245: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:13 smithi049 ceph-mon[112142]: pgmap v245: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:15 smithi144 ceph-mon[86635]: pgmap v246: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:15 smithi049 ceph-mon[112142]: pgmap v246: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:17 smithi144 ceph-mon[86635]: pgmap v247: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:18.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:17 smithi049 ceph-mon[112142]: pgmap v247: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:20.148 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:19 smithi144 ceph-mon[86635]: pgmap v248: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:19 smithi049 ceph-mon[112142]: pgmap v248: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:05:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:05:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:21 smithi144 ceph-mon[86635]: pgmap v249: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:21 smithi049 ceph-mon[112142]: pgmap v249: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:24.133 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:23 smithi049 ceph-mon[112142]: pgmap v250: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:23 smithi144 ceph-mon[86635]: pgmap v250: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:25 smithi144 ceph-mon[86635]: pgmap v251: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:26.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:25 smithi049 ceph-mon[112142]: pgmap v251: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:28.202 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:27 smithi144 ceph-mon[86635]: pgmap v252: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:27 smithi049 ceph-mon[112142]: pgmap v252: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:29 smithi144 ceph-mon[86635]: pgmap v253: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:29 smithi049 ceph-mon[112142]: pgmap v253: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:31 smithi144 ceph-mon[86635]: pgmap v254: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:32.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:31 smithi049 ceph-mon[112142]: pgmap v254: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:33 smithi144 ceph-mon[86635]: pgmap v255: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:33 smithi049 ceph-mon[112142]: pgmap v255: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:36.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:35 smithi144 ceph-mon[86635]: pgmap v256: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:36.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:05:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:35 smithi049 ceph-mon[112142]: pgmap v256: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:05:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:37 smithi144 ceph-mon[86635]: pgmap v257: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:37 smithi049 ceph-mon[112142]: pgmap v257: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:39 smithi144 ceph-mon[86635]: pgmap v258: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:39 smithi049 ceph-mon[112142]: pgmap v258: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.138+0000 7f3aa1323700 1 -- 172.21.15.49:0/2696479370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 msgr2=0x7f3a9c1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.138+0000 7f3aa1323700 1 --2- 172.21.15.49:0/2696479370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 0x7f3a9c1012e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009910 tx=0x7f3a8c009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.138+0000 7f3aa1323700 1 -- 172.21.15.49:0/2696479370 shutdown_connections 2024-05-15T16:05:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.138+0000 7f3aa1323700 1 --2- 172.21.15.49:0/2696479370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a9c1018b0 0x7f3a9c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.138+0000 7f3aa1323700 1 --2- 172.21.15.49:0/2696479370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 0x7f3a9c1012e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.138+0000 7f3aa1323700 1 -- 172.21.15.49:0/2696479370 >> 172.21.15.49:0/2696479370 conn(0x7f3a9c0fc7d0 msgr2=0x7f3a9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:42.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.138+0000 7f3aa1323700 1 -- 172.21.15.49:0/2696479370 shutdown_connections 2024-05-15T16:05:42.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.138+0000 7f3aa1323700 1 -- 172.21.15.49:0/2696479370 wait complete. 2024-05-15T16:05:42.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.139+0000 7f3aa1323700 1 Processor -- start 2024-05-15T16:05:42.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.139+0000 7f3aa1323700 1 -- start start 2024-05-15T16:05:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.139+0000 7f3aa1323700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 0x7f3a9c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.139+0000 7f3aa1323700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a9c1018b0 0x7f3a9c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.139+0000 7f3aa1323700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c073d20 con 0x7f3a9c100f00 2024-05-15T16:05:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.139+0000 7f3aa1323700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c073e60 con 0x7f3a9c1018b0 2024-05-15T16:05:42.143 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:41 smithi049 ceph-mon[112142]: pgmap v259: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:42.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a9affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 0x7f3a9c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:42.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a9affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 0x7f3a9c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44894/0 (socket says 172.21.15.49:44894) 2024-05-15T16:05:42.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a9affd700 1 -- 172.21.15.49:0/1251200646 learned_addr learned my addr 172.21.15.49:0/1251200646 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:42.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a927fc700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a9c1018b0 0x7f3a9c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:42.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a9affd700 1 -- 172.21.15.49:0/1251200646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a9c1018b0 msgr2=0x7f3a9c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a9affd700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a9c1018b0 0x7f3a9c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a9affd700 1 -- 172.21.15.49:0/1251200646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0095f0 con 0x7f3a9c100f00 2024-05-15T16:05:42.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a927fc700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a9c1018b0 0x7f3a9c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a9affd700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 0x7f3a9c0732a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006010 tx=0x7f3a8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.140+0000 7f3a98ff9700 1 -- 172.21.15.49:0/1251200646 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a8c01d070 con 0x7f3a9c100f00 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.141+0000 7f3a98ff9700 1 -- 172.21.15.49:0/1251200646 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a8c00f460 con 0x7f3a9c100f00 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.141+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c0740e0 con 0x7f3a9c100f00 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.141+0000 7f3a98ff9700 1 -- 172.21.15.49:0/1251200646 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a8c017630 con 0x7f3a9c100f00 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.141+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c1a3040 con 0x7f3a9c100f00 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.142+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c1092a0 con 0x7f3a9c100f00 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.142+0000 7f3a98ff9700 1 -- 172.21.15.49:0/1251200646 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3a8c00f5d0 con 0x7f3a9c100f00 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.143+0000 7f3a98ff9700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a8806e200 0x7f3a880706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.143+0000 7f3a98ff9700 1 -- 172.21.15.49:0/1251200646 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3a8c08f650 con 0x7f3a9c100f00 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.143+0000 7f3a927fc700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a8806e200 0x7f3a880706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:42.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.143+0000 7f3a927fc700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a8806e200 0x7f3a880706c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c074980 tx=0x7f3a84005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:42.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.148+0000 7f3a98ff9700 1 -- 172.21.15.49:0/1251200646 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3a8c0589f0 con 0x7f3a9c100f00 2024-05-15T16:05:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:41 smithi144 ceph-mon[86635]: pgmap v259: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.373+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a9c10ded0 con 0x7f3a8806e200 2024-05-15T16:05:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.375+0000 7f3a98ff9700 1 -- 172.21.15.49:0/1251200646 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3a9c10ded0 con 0x7f3a8806e200 2024-05-15T16:05:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a8806e200 msgr2=0x7f3a880706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a8806e200 0x7f3a880706c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c074980 tx=0x7f3a84005f50 comp rx=0 tx=0).stop 2024-05-15T16:05:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 msgr2=0x7f3a9c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 0x7f3a9c0732a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006010 tx=0x7f3a8c00be20 comp rx=0 tx=0).stop 2024-05-15T16:05:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 shutdown_connections 2024-05-15T16:05:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a8806e200 0x7f3a880706c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c100f00 0x7f3a9c0732a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 --2- 172.21.15.49:0/1251200646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a9c1018b0 0x7f3a9c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 >> 172.21.15.49:0/1251200646 conn(0x7f3a9c0fc7d0 msgr2=0x7f3a9c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 shutdown_connections 2024-05-15T16:05:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.377+0000 7f3aa1323700 1 -- 172.21.15.49:0/1251200646 wait complete. 2024-05-15T16:05:42.389 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:05:42.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.487+0000 7f6618ab1700 1 -- 172.21.15.49:0/3550210298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66140ffdf0 msgr2=0x7f66141001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.487+0000 7f6618ab1700 1 --2- 172.21.15.49:0/3550210298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66140ffdf0 0x7f66141001d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6604009910 tx=0x7f6604009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.487+0000 7f6618ab1700 1 -- 172.21.15.49:0/3550210298 shutdown_connections 2024-05-15T16:05:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.487+0000 7f6618ab1700 1 --2- 172.21.15.49:0/3550210298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66141007a0 0x7f66141047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.487+0000 7f6618ab1700 1 --2- 172.21.15.49:0/3550210298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66140ffdf0 0x7f66141001d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.487+0000 7f6618ab1700 1 -- 172.21.15.49:0/3550210298 >> 172.21.15.49:0/3550210298 conn(0x7f66140fb6a0 msgr2=0x7f66140fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.487+0000 7f6618ab1700 1 -- 172.21.15.49:0/3550210298 shutdown_connections 2024-05-15T16:05:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.487+0000 7f6618ab1700 1 -- 172.21.15.49:0/3550210298 wait complete. 2024-05-15T16:05:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f6618ab1700 1 Processor -- start 2024-05-15T16:05:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f6618ab1700 1 -- start start 2024-05-15T16:05:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f6618ab1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ffdf0 0x7f6614101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f6618ab1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66141007a0 0x7f6614101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f6618ab1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6614102310 con 0x7f66141007a0 2024-05-15T16:05:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f6618ab1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6614102480 con 0x7f66140ffdf0 2024-05-15T16:05:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f660bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66141007a0 0x7f6614101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f660bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66141007a0 0x7f6614101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44914/0 (socket says 172.21.15.49:44914) 2024-05-15T16:05:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f660bfff700 1 -- 172.21.15.49:0/76391041 learned_addr learned my addr 172.21.15.49:0/76391041 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.488+0000 7f661259c700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ffdf0 0x7f6614101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.489+0000 7f660bfff700 1 -- 172.21.15.49:0/76391041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ffdf0 msgr2=0x7f6614101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.489+0000 7f660bfff700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ffdf0 0x7f6614101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.489+0000 7f660bfff700 1 -- 172.21.15.49:0/76391041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040095f0 con 0x7f66141007a0 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.489+0000 7f660bfff700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66141007a0 0x7f6614101dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00e8f0 tx=0x7f65fc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.489+0000 7f660b7fe700 1 -- 172.21.15.49:0/76391041 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65fc009790 con 0x7f66141007a0 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.489+0000 7f660b7fe700 1 -- 172.21.15.49:0/76391041 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65fc00c810 con 0x7f66141007a0 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.489+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66141a2d90 con 0x7f66141007a0 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.489+0000 7f660b7fe700 1 -- 172.21.15.49:0/76391041 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65fc005580 con 0x7f66141007a0 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.489+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66141a30f0 con 0x7f66141007a0 2024-05-15T16:05:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.490+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66141091a0 con 0x7f66141007a0 2024-05-15T16:05:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.491+0000 7f660b7fe700 1 -- 172.21.15.49:0/76391041 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f65fc00c980 con 0x7f66141007a0 2024-05-15T16:05:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.491+0000 7f660b7fe700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f660006e200 0x7f66000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.491+0000 7f660b7fe700 1 -- 172.21.15.49:0/76391041 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f65fc014070 con 0x7f66141007a0 2024-05-15T16:05:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.491+0000 7f661259c700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f660006e200 0x7f66000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.492+0000 7f661259c700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f660006e200 0x7f66000706c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66040059d0 tx=0x7f6604005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:42.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.496+0000 7f660b7fe700 1 -- 172.21.15.49:0/76391041 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f65fc057710 con 0x7f66141007a0 2024-05-15T16:05:42.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.721+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f661410cdb0 con 0x7f660006e200 2024-05-15T16:05:42.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.723+0000 7f660b7fe700 1 -- 172.21.15.49:0/76391041 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f661410cdb0 con 0x7f660006e200 2024-05-15T16:05:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f660006e200 msgr2=0x7f66000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f660006e200 0x7f66000706c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66040059d0 tx=0x7f6604005940 comp rx=0 tx=0).stop 2024-05-15T16:05:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66141007a0 msgr2=0x7f6614101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66141007a0 0x7f6614101dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00e8f0 tx=0x7f65fc00ec00 comp rx=0 tx=0).stop 2024-05-15T16:05:42.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 shutdown_connections 2024-05-15T16:05:42.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f660006e200 0x7f66000706c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ffdf0 0x7f6614101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 --2- 172.21.15.49:0/76391041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66141007a0 0x7f6614101dd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 >> 172.21.15.49:0/76391041 conn(0x7f66140fb6a0 msgr2=0x7f6614069700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:42.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 shutdown_connections 2024-05-15T16:05:42.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.725+0000 7f6618ab1700 1 -- 172.21.15.49:0/76391041 wait complete. 2024-05-15T16:05:42.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.834+0000 7f5fe3224700 1 -- 172.21.15.49:0/3593922070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 msgr2=0x7f5fdc10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.834+0000 7f5fe3224700 1 --2- 172.21.15.49:0/3593922070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 0x7f5fdc10cfe0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009910 tx=0x7f5fcc009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:42.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.834+0000 7f5fe3224700 1 -- 172.21.15.49:0/3593922070 shutdown_connections 2024-05-15T16:05:42.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.834+0000 7f5fe3224700 1 --2- 172.21.15.49:0/3593922070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 0x7f5fdc10cfe0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.834+0000 7f5fe3224700 1 --2- 172.21.15.49:0/3593922070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc073370 0x7f5fdc073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.834+0000 7f5fe3224700 1 -- 172.21.15.49:0/3593922070 >> 172.21.15.49:0/3593922070 conn(0x7f5fdc0fc790 msgr2=0x7f5fdc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:42.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.834+0000 7f5fe3224700 1 -- 172.21.15.49:0/3593922070 shutdown_connections 2024-05-15T16:05:42.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.834+0000 7f5fe3224700 1 -- 172.21.15.49:0/3593922070 wait complete. 2024-05-15T16:05:42.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fe3224700 1 Processor -- start 2024-05-15T16:05:42.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fe3224700 1 -- start start 2024-05-15T16:05:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fe3224700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc073370 0x7f5fdc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fe3224700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 0x7f5fdc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:42.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fe3224700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc1999d0 con 0x7f5fdc073c90 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fe3224700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc19d6a0 con 0x7f5fdc073370 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 0x7f5fdc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 0x7f5fdc1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44930/0 (socket says 172.21.15.49:44930) 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fdbfff700 1 -- 172.21.15.49:0/2543155041 learned_addr learned my addr 172.21.15.49:0/2543155041 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fe0fc0700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc073370 0x7f5fdc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fdbfff700 1 -- 172.21.15.49:0/2543155041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc073370 msgr2=0x7f5fdc198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fdbfff700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc073370 0x7f5fdc198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.835+0000 7f5fdbfff700 1 -- 172.21.15.49:0/2543155041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc0095f0 con 0x7f5fdc073c90 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.836+0000 7f5fdbfff700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 0x7f5fdc1992f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.836+0000 7f5fd9ffb700 1 -- 172.21.15.49:0/2543155041 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fcc01d070 con 0x7f5fdc073c90 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.836+0000 7f5fd9ffb700 1 -- 172.21.15.49:0/2543155041 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fcc0056e0 con 0x7f5fdc073c90 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.836+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fdc19d920 con 0x7f5fdc073c90 2024-05-15T16:05:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.836+0000 7f5fd9ffb700 1 -- 172.21.15.49:0/2543155041 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fcc00fe40 con 0x7f5fdc073c90 2024-05-15T16:05:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.836+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc19de10 con 0x7f5fdc073c90 2024-05-15T16:05:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.837+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc074a20 con 0x7f5fdc073c90 2024-05-15T16:05:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.838+0000 7f5fd9ffb700 1 -- 172.21.15.49:0/2543155041 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5fcc00f680 con 0x7f5fdc073c90 2024-05-15T16:05:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.839+0000 7f5fd9ffb700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5fc806e210 0x7f5fc80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.839+0000 7f5fd9ffb700 1 -- 172.21.15.49:0/2543155041 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5fcc08f480 con 0x7f5fdc073c90 2024-05-15T16:05:42.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.839+0000 7f5fe0fc0700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5fc806e210 0x7f5fc80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:42.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.839+0000 7f5fe0fc0700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5fc806e210 0x7f5fc80706d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005950 tx=0x7f5fc400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:42.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:42.843+0000 7f5fd9ffb700 1 -- 172.21.15.49:0/2543155041 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5fcc0587e0 con 0x7f5fdc073c90 2024-05-15T16:05:43.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.065+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fdc1100a0 con 0x7f5fc806e210 2024-05-15T16:05:43.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.076+0000 7f5fd9ffb700 1 -- 172.21.15.49:0/2543155041 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5fdc1100a0 con 0x7f5fc806e210 2024-05-15T16:05:43.095 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:05:43.095 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (8m) 2m ago 20m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (20m) 2m ago 20m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (19m) 118s ago 19m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (6m) 2m ago 20m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (6m) 118s ago 19m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (8m) 2m ago 20m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (2m) 2m ago 14m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (2m) 2m ago 14m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (2m) 118s ago 14m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (14m) 118s ago 14m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (11m) 2m ago 22m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (10m) 118s ago 19m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (7m) 2m ago 22m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (7m) 118s ago 19m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:05:43.096 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (9m) 2m ago 20m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:05:43.097 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (9m) 118s ago 19m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:05:43.097 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (5m) 2m ago 18m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:05:43.097 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (5m) 2m ago 18m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:05:43.097 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (4m) 2m ago 17m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:05:43.097 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (4m) 118s ago 17m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:05:43.097 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (3m) 118s ago 16m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:05:43.097 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (2m) 118s ago 16m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:05:43.097 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (9m) 2m ago 20m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:05:43.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5fc806e210 msgr2=0x7f5fc80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5fc806e210 0x7f5fc80706d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005950 tx=0x7f5fc400b410 comp rx=0 tx=0).stop 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 msgr2=0x7f5fdc1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 0x7f5fdc1992f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc00bd70 comp rx=0 tx=0).stop 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 shutdown_connections 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5fc806e210 0x7f5fc80706d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc073370 0x7f5fdc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 --2- 172.21.15.49:0/2543155041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fdc073c90 0x7f5fdc1992f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 >> 172.21.15.49:0/2543155041 conn(0x7f5fdc0fc790 msgr2=0x7f5fdc1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 shutdown_connections 2024-05-15T16:05:43.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.079+0000 7f5fe3224700 1 -- 172.21.15.49:0/2543155041 wait complete. 2024-05-15T16:05:43.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.187+0000 7f3c11406700 1 -- 172.21.15.49:0/3553517160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c0ffdd0 msgr2=0x7f3c0c1001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.187+0000 7f3c11406700 1 --2- 172.21.15.49:0/3553517160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c0ffdd0 0x7f3c0c1001b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009910 tx=0x7f3bfc009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:43.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.187+0000 7f3c11406700 1 -- 172.21.15.49:0/3553517160 shutdown_connections 2024-05-15T16:05:43.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.187+0000 7f3c11406700 1 --2- 172.21.15.49:0/3553517160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c0c100780 0x7f3c0c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.187+0000 7f3c11406700 1 --2- 172.21.15.49:0/3553517160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c0ffdd0 0x7f3c0c1001b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.187+0000 7f3c11406700 1 -- 172.21.15.49:0/3553517160 >> 172.21.15.49:0/3553517160 conn(0x7f3c0c0fb6a0 msgr2=0x7f3c0c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:43.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.188+0000 7f3c11406700 1 -- 172.21.15.49:0/3553517160 shutdown_connections 2024-05-15T16:05:43.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.188+0000 7f3c11406700 1 -- 172.21.15.49:0/3553517160 wait complete. 2024-05-15T16:05:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.188+0000 7f3c11406700 1 Processor -- start 2024-05-15T16:05:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.188+0000 7f3c11406700 1 -- start start 2024-05-15T16:05:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c11406700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c0c0ffdd0 0x7f3c0c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c11406700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c100780 0x7f3c0c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c11406700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c19d400 con 0x7f3c0c100780 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c11406700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c1969e0 con 0x7f3c0c0ffdd0 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c0a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c100780 0x7f3c0c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c0a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c100780 0x7f3c0c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44944/0 (socket says 172.21.15.49:44944) 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c0a7fc700 1 -- 172.21.15.49:0/3259223660 learned_addr learned my addr 172.21.15.49:0/3259223660 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c0affd700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c0c0ffdd0 0x7f3c0c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c0a7fc700 1 -- 172.21.15.49:0/3259223660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c0c0ffdd0 msgr2=0x7f3c0c19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c0a7fc700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c0c0ffdd0 0x7f3c0c19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.189+0000 7f3c0a7fc700 1 -- 172.21.15.49:0/3259223660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0095f0 con 0x7f3c0c100780 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.190+0000 7f3c0a7fc700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c100780 0x7f3c0c19cde0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000d8d0 tx=0x7f3c0000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.190+0000 7f3bf3fff700 1 -- 172.21.15.49:0/3259223660 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c00004a00 con 0x7f3c0c100780 2024-05-15T16:05:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.190+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0c196cc0 con 0x7f3c0c100780 2024-05-15T16:05:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.190+0000 7f3bf3fff700 1 -- 172.21.15.49:0/3259223660 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c000054e0 con 0x7f3c0c100780 2024-05-15T16:05:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.190+0000 7f3bf3fff700 1 -- 172.21.15.49:0/3259223660 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0000fc20 con 0x7f3c0c100780 2024-05-15T16:05:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.190+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0c197210 con 0x7f3c0c100780 2024-05-15T16:05:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.191+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0c1012b0 con 0x7f3c0c100780 2024-05-15T16:05:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.191+0000 7f3bf3fff700 1 -- 172.21.15.49:0/3259223660 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3c00005020 con 0x7f3c0c100780 2024-05-15T16:05:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.192+0000 7f3bf3fff700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.192+0000 7f3bf3fff700 1 -- 172.21.15.49:0/3259223660 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3c0008da30 con 0x7f3c0c100780 2024-05-15T16:05:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.192+0000 7f3c0affd700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.193+0000 7f3c0affd700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc000c00 tx=0x7f3bfc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:43.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.197+0000 7f3bf3fff700 1 -- 172.21.15.49:0/3259223660 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3c0005a840 con 0x7f3c0c100780 2024-05-15T16:05:43.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.490+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3c0c10ce00 con 0x7f3c0c100780 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.491+0000 7f3bf3fff700 1 -- 172.21.15.49:0/3259223660 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f3c0005a3d0 con 0x7f3c0c100780 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:05:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:05:43.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 msgr2=0x7f3bf40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc000c00 tx=0x7f3bfc01a040 comp rx=0 tx=0).stop 2024-05-15T16:05:43.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c100780 msgr2=0x7f3c0c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c100780 0x7f3c0c19cde0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000d8d0 tx=0x7f3c0000dbe0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 shutdown_connections 2024-05-15T16:05:43.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c0c0ffdd0 0x7f3c0c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 --2- 172.21.15.49:0/3259223660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c0c100780 0x7f3c0c19cde0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 >> 172.21.15.49:0/3259223660 conn(0x7f3c0c0fb6a0 msgr2=0x7f3c0c069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:43.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 shutdown_connections 2024-05-15T16:05:43.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.493+0000 7f3c11406700 1 -- 172.21.15.49:0/3259223660 wait complete. 2024-05-15T16:05:43.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.601+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/3825411591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 msgr2=0x7fe1e810ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.601+0000 7fe1ed4c9700 1 --2- 172.21.15.49:0/3825411591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 0x7fe1e810ac80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009910 tx=0x7fe1dc009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:43.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.601+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/3825411591 shutdown_connections 2024-05-15T16:05:43.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.601+0000 7fe1ed4c9700 1 --2- 172.21.15.49:0/3825411591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 0x7fe1e810ac80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.601+0000 7fe1ed4c9700 1 --2- 172.21.15.49:0/3825411591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e8101e70 0x7fe1e8102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.602+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/3825411591 >> 172.21.15.49:0/3825411591 conn(0x7fe1e80fb700 msgr2=0x7fe1e80fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:43.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.602+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/3825411591 shutdown_connections 2024-05-15T16:05:43.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.602+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/3825411591 wait complete. 2024-05-15T16:05:43.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.602+0000 7fe1ed4c9700 1 Processor -- start 2024-05-15T16:05:43.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.602+0000 7fe1ed4c9700 1 -- start start 2024-05-15T16:05:43.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1ed4c9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e8101e70 0x7fe1e819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:43.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1ed4c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 0x7fe1e819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:43.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1ed4c9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e819d490 con 0x7fe1e8102790 2024-05-15T16:05:43.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1ed4c9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8196a70 con 0x7fe1e8101e70 2024-05-15T16:05:43.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1e67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 0x7fe1e819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:43.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1e67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 0x7fe1e819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41948/0 (socket says 172.21.15.49:41948) 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1e67fc700 1 -- 172.21.15.49:0/981254903 learned_addr learned my addr 172.21.15.49:0/981254903 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1e6ffd700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e8101e70 0x7fe1e819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1e67fc700 1 -- 172.21.15.49:0/981254903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e8101e70 msgr2=0x7fe1e819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1e67fc700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e8101e70 0x7fe1e819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1e67fc700 1 -- 172.21.15.49:0/981254903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0095f0 con 0x7fe1e8102790 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1e67fc700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 0x7fe1e819ce70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009fd0 tx=0x7fe1dc00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1cffff700 1 -- 172.21.15.49:0/981254903 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1dc01d070 con 0x7fe1e8102790 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e8196cf0 con 0x7fe1e8102790 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1cffff700 1 -- 172.21.15.49:0/981254903 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1dc0052a0 con 0x7fe1e8102790 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1cffff700 1 -- 172.21.15.49:0/981254903 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1dc022470 con 0x7fe1e8102790 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.603+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e81971e0 con 0x7fe1e8102790 2024-05-15T16:05:43.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.605+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e8105590 con 0x7fe1e8102790 2024-05-15T16:05:43.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.605+0000 7fe1cffff700 1 -- 172.21.15.49:0/981254903 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe1dc00f6f0 con 0x7fe1e8102790 2024-05-15T16:05:43.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.606+0000 7fe1cffff700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe1d006e210 0x7fe1d00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:43.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.606+0000 7fe1cffff700 1 -- 172.21.15.49:0/981254903 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe1dc08f4d0 con 0x7fe1e8102790 2024-05-15T16:05:43.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.606+0000 7fe1e6ffd700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe1d006e210 0x7fe1d00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:43.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.606+0000 7fe1e6ffd700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe1d006e210 0x7fe1d00706d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009bf0 tx=0x7fe1d8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:43.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.610+0000 7fe1cffff700 1 -- 172.21.15.49:0/981254903 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe1dc05c2e0 con 0x7fe1e8102790 2024-05-15T16:05:43.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.865+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe1e8197f30 con 0x7fe1e8102790 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.865+0000 7fe1cffff700 1 -- 172.21.15.49:0/981254903 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7fe1dc05be70 con 0x7fe1e8102790 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:05:43.867 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:05:43.868 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:05:43.869 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:05:43.870 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:05:43.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.867+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe1d006e210 msgr2=0x7fe1d00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.867+0000 7fe1ed4c9700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe1d006e210 0x7fe1d00706d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009bf0 tx=0x7fe1d8009260 comp rx=0 tx=0).stop 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.868+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 msgr2=0x7fe1e819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.868+0000 7fe1ed4c9700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 0x7fe1e819ce70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009fd0 tx=0x7fe1dc00b650 comp rx=0 tx=0).stop 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.868+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 shutdown_connections 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.868+0000 7fe1ed4c9700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe1d006e210 0x7fe1d00706d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.868+0000 7fe1ed4c9700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1e8101e70 0x7fe1e819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.868+0000 7fe1ed4c9700 1 --2- 172.21.15.49:0/981254903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e8102790 0x7fe1e819ce70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.868+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 >> 172.21.15.49:0/981254903 conn(0x7fe1e80fb700 msgr2=0x7fe1e8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.868+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 shutdown_connections 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.868+0000 7fe1ed4c9700 1 -- 172.21.15.49:0/981254903 wait complete. 2024-05-15T16:05:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:05:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 -- 172.21.15.49:0/1980753789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf280ff9e0 msgr2=0x7faf280ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 --2- 172.21.15.49:0/1980753789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf280ff9e0 0x7faf280ffdc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf24009910 tx=0x7faf24009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 -- 172.21.15.49:0/1980753789 shutdown_connections 2024-05-15T16:05:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 --2- 172.21.15.49:0/1980753789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf28100390 0x7faf2810d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 --2- 172.21.15.49:0/1980753789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf280ff9e0 0x7faf280ffdc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 -- 172.21.15.49:0/1980753789 >> 172.21.15.49:0/1980753789 conn(0x7faf280fb6a0 msgr2=0x7faf280fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 -- 172.21.15.49:0/1980753789 shutdown_connections 2024-05-15T16:05:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 -- 172.21.15.49:0/1980753789 wait complete. 2024-05-15T16:05:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 Processor -- start 2024-05-15T16:05:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.980+0000 7faf2fa92700 1 -- start start 2024-05-15T16:05:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2fa92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf280ff9e0 0x7faf28198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2fa92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28100390 0x7faf281992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2fa92700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf281999c0 con 0x7faf28100390 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2fa92700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2819d690 con 0x7faf280ff9e0 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2d02d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28100390 0x7faf281992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2d02d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28100390 0x7faf281992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41958/0 (socket says 172.21.15.49:41958) 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2d02d700 1 -- 172.21.15.49:0/641619004 learned_addr learned my addr 172.21.15.49:0/641619004 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2d82e700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf280ff9e0 0x7faf28198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2d02d700 1 -- 172.21.15.49:0/641619004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf280ff9e0 msgr2=0x7faf28198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2d02d700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf280ff9e0 0x7faf28198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2d02d700 1 -- 172.21.15.49:0/641619004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240095f0 con 0x7faf28100390 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf2d02d700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28100390 0x7faf281992e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf1800d8d0 tx=0x7faf1800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.981+0000 7faf1effd700 1 -- 172.21.15.49:0/641619004 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf180098a0 con 0x7faf28100390 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.982+0000 7faf1effd700 1 -- 172.21.15.49:0/641619004 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf18010460 con 0x7faf28100390 2024-05-15T16:05:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.982+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2819d970 con 0x7faf28100390 2024-05-15T16:05:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.982+0000 7faf1effd700 1 -- 172.21.15.49:0/641619004 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf1800f5f0 con 0x7faf28100390 2024-05-15T16:05:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.982+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2819dec0 con 0x7faf28100390 2024-05-15T16:05:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.983+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2810aed0 con 0x7faf28100390 2024-05-15T16:05:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.983+0000 7faf1effd700 1 -- 172.21.15.49:0/641619004 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faf18009a00 con 0x7faf28100390 2024-05-15T16:05:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.984+0000 7faf1effd700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf1406e200 0x7faf140706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.984+0000 7faf2d82e700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf1406e200 0x7faf140706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.984+0000 7faf1effd700 1 -- 172.21.15.49:0/641619004 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faf1808df70 con 0x7faf28100390 2024-05-15T16:05:43.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.984+0000 7faf2d82e700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf1406e200 0x7faf140706c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf24000c00 tx=0x7faf2400b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:43.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:43.989+0000 7faf1effd700 1 -- 172.21.15.49:0/641619004 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7faf180c2850 con 0x7faf28100390 2024-05-15T16:05:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:43 smithi144 ceph-mon[86635]: pgmap v260: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:44.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:43 smithi144 ceph-mon[86635]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:44.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:43 smithi144 ceph-mon[86635]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:44.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:43 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3259223660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:05:44.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:43 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/981254903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:05:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:43 smithi049 ceph-mon[112142]: pgmap v260: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:43 smithi049 ceph-mon[112142]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:44.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:43 smithi049 ceph-mon[112142]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:44.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:43 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3259223660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:05:44.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:43 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/981254903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:05:44.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.214+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf28110050 con 0x7faf1406e200 2024-05-15T16:05:44.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.216+0000 7faf1effd700 1 -- 172.21.15.49:0/641619004 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faf28110050 con 0x7faf1406e200 2024-05-15T16:05:44.216 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:05:44.217 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:05:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf1406e200 msgr2=0x7faf140706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf1406e200 0x7faf140706c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf24000c00 tx=0x7faf2400b560 comp rx=0 tx=0).stop 2024-05-15T16:05:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28100390 msgr2=0x7faf281992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28100390 0x7faf281992e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf1800d8d0 tx=0x7faf1800dbe0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 shutdown_connections 2024-05-15T16:05:44.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf1406e200 0x7faf140706c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf280ff9e0 0x7faf28198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 --2- 172.21.15.49:0/641619004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28100390 0x7faf281992e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 >> 172.21.15.49:0/641619004 conn(0x7faf280fb6a0 msgr2=0x7faf28108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:44.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 shutdown_connections 2024-05-15T16:05:44.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.218+0000 7faf2fa92700 1 -- 172.21.15.49:0/641619004 wait complete. 2024-05-15T16:05:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.327+0000 7f26546cc700 1 -- 172.21.15.49:0/1503684050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 msgr2=0x7f264c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.327+0000 7f26546cc700 1 --2- 172.21.15.49:0/1503684050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 0x7f264c1047c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2648009910 tx=0x7f2648009c20 comp rx=0 tx=0).stop 2024-05-15T16:05:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.327+0000 7f26546cc700 1 -- 172.21.15.49:0/1503684050 shutdown_connections 2024-05-15T16:05:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.327+0000 7f26546cc700 1 --2- 172.21.15.49:0/1503684050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 0x7f264c1047c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.327+0000 7f26546cc700 1 --2- 172.21.15.49:0/1503684050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f264c0ffdc0 0x7f264c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.327+0000 7f26546cc700 1 -- 172.21.15.49:0/1503684050 >> 172.21.15.49:0/1503684050 conn(0x7f264c076800 msgr2=0x7f264c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.327+0000 7f26546cc700 1 -- 172.21.15.49:0/1503684050 shutdown_connections 2024-05-15T16:05:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.327+0000 7f26546cc700 1 -- 172.21.15.49:0/1503684050 wait complete. 2024-05-15T16:05:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.328+0000 7f26546cc700 1 Processor -- start 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.328+0000 7f26546cc700 1 -- start start 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.328+0000 7f26546cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f264c0ffdc0 0x7f264c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.328+0000 7f26546cc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 0x7f264c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.328+0000 7f26546cc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1984f0 con 0x7f264c100770 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.328+0000 7f26546cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c198630 con 0x7f264c0ffdc0 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.328+0000 7f2651c67700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 0x7f264c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.328+0000 7f2651c67700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 0x7f264c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41986/0 (socket says 172.21.15.49:41986) 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.328+0000 7f2651c67700 1 -- 172.21.15.49:0/3845166546 learned_addr learned my addr 172.21.15.49:0/3845166546 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f2652468700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f264c0ffdc0 0x7f264c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f2651c67700 1 -- 172.21.15.49:0/3845166546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f264c0ffdc0 msgr2=0x7f264c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f2651c67700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f264c0ffdc0 0x7f264c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f2651c67700 1 -- 172.21.15.49:0/3845166546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26480095f0 con 0x7f264c100770 2024-05-15T16:05:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f2652468700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f264c0ffdc0 0x7f264c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:05:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f2651c67700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 0x7f264c197f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f26480092e0 tx=0x7f264800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f263f7fe700 1 -- 172.21.15.49:0/3845166546 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f264801d070 con 0x7f264c100770 2024-05-15T16:05:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f263f7fe700 1 -- 172.21.15.49:0/3845166546 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f264800f460 con 0x7f264c100770 2024-05-15T16:05:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c198850 con 0x7f264c100770 2024-05-15T16:05:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f263f7fe700 1 -- 172.21.15.49:0/3845166546 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2648017610 con 0x7f264c100770 2024-05-15T16:05:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.329+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c1a0db0 con 0x7f264c100770 2024-05-15T16:05:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.330+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c1081c0 con 0x7f264c100770 2024-05-15T16:05:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.331+0000 7f263f7fe700 1 -- 172.21.15.49:0/3845166546 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f264800f5d0 con 0x7f264c100770 2024-05-15T16:05:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.331+0000 7f263f7fe700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f263806e210 0x7f26380706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:05:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.331+0000 7f263f7fe700 1 -- 172.21.15.49:0/3845166546 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f264808f4e0 con 0x7f264c100770 2024-05-15T16:05:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.331+0000 7f2652468700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f263806e210 0x7f26380706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:05:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.332+0000 7f2652468700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f263806e210 0x7f26380706d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2640005950 tx=0x7f2640005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:05:44.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.336+0000 7f263f7fe700 1 -- 172.21.15.49:0/3845166546 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f26480587c0 con 0x7f264c100770 2024-05-15T16:05:44.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.661+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f264c1012a0 con 0x7f264c100770 2024-05-15T16:05:44.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.662+0000 7f263f7fe700 1 -- 172.21.15.49:0/3845166546 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f264805be80 con 0x7f264c100770 2024-05-15T16:05:44.663 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:05:44.663 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:05:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:05:44.664 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:05:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:05:44.664 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:05:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:05:44.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f263806e210 msgr2=0x7f26380706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:44.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f263806e210 0x7f26380706d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2640005950 tx=0x7f2640005f50 comp rx=0 tx=0).stop 2024-05-15T16:05:44.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 msgr2=0x7f264c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:05:44.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 0x7f264c197f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f26480092e0 tx=0x7f264800be20 comp rx=0 tx=0).stop 2024-05-15T16:05:44.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 shutdown_connections 2024-05-15T16:05:44.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f263806e210 0x7f26380706d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f264c0ffdc0 0x7f264c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 --2- 172.21.15.49:0/3845166546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264c100770 0x7f264c197f20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:05:44.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 >> 172.21.15.49:0/3845166546 conn(0x7f264c076800 msgr2=0x7f264c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:05:44.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 shutdown_connections 2024-05-15T16:05:44.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:05:44.664+0000 7f26546cc700 1 -- 172.21.15.49:0/3845166546 wait complete. 2024-05-15T16:05:44.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:44 smithi049 ceph-mon[112142]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:44.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:44 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3845166546' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:05:45.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:44 smithi144 ceph-mon[86635]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:45.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:44 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3845166546' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:05:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:45 smithi144 ceph-mon[86635]: pgmap v261: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:46.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:45 smithi144 ceph-mon[86635]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:45 smithi049 ceph-mon[112142]: pgmap v261: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:46.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:45 smithi049 ceph-mon[112142]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:05:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:47 smithi144 ceph-mon[86635]: pgmap v262: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:48.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:47 smithi049 ceph-mon[112142]: pgmap v262: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:49 smithi144 ceph-mon[86635]: pgmap v263: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:49 smithi049 ceph-mon[112142]: pgmap v263: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:05:51.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:05:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:51 smithi144 ceph-mon[86635]: pgmap v264: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:51 smithi049 ceph-mon[112142]: pgmap v264: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:53 smithi144 ceph-mon[86635]: pgmap v265: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:53 smithi049 ceph-mon[112142]: pgmap v265: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:55 smithi144 ceph-mon[86635]: pgmap v266: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:55 smithi049 ceph-mon[112142]: pgmap v266: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:57 smithi144 ceph-mon[86635]: pgmap v267: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:05:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:57 smithi049 ceph-mon[112142]: pgmap v267: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:05:59 smithi144 ceph-mon[86635]: pgmap v268: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:05:59 smithi049 ceph-mon[112142]: pgmap v268: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:02.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:01 smithi144 ceph-mon[86635]: pgmap v269: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:01 smithi049 ceph-mon[112142]: pgmap v269: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:04.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:03 smithi144 ceph-mon[86635]: pgmap v270: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:03 smithi049 ceph-mon[112142]: pgmap v270: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:05 smithi144 ceph-mon[86635]: pgmap v271: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:06:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:05 smithi049 ceph-mon[112142]: pgmap v271: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:06:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:07 smithi144 ceph-mon[86635]: pgmap v272: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:07 smithi049 ceph-mon[112142]: pgmap v272: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:09 smithi144 ceph-mon[86635]: pgmap v273: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:09 smithi049 ceph-mon[112142]: pgmap v273: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:11 smithi144 ceph-mon[86635]: pgmap v274: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:12.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:11 smithi049 ceph-mon[112142]: pgmap v274: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:13 smithi144 ceph-mon[86635]: pgmap v275: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:13 smithi049 ceph-mon[112142]: pgmap v275: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:14.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.790+0000 7f219c08f700 1 -- 172.21.15.49:0/1431537328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 msgr2=0x7f21941058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:14.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.790+0000 7f219c08f700 1 --2- 172.21.15.49:0/1431537328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 0x7f21941058e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2188009910 tx=0x7f2188009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:14.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.790+0000 7f219c08f700 1 -- 172.21.15.49:0/1431537328 shutdown_connections 2024-05-15T16:06:14.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.790+0000 7f219c08f700 1 --2- 172.21.15.49:0/1431537328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 0x7f21941058e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:14.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.790+0000 7f219c08f700 1 --2- 172.21.15.49:0/1431537328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194100ee0 0x7f21941012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:14.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.790+0000 7f219c08f700 1 -- 172.21.15.49:0/1431537328 >> 172.21.15.49:0/1431537328 conn(0x7f21940fc790 msgr2=0x7f21940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:14.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.790+0000 7f219c08f700 1 -- 172.21.15.49:0/1431537328 shutdown_connections 2024-05-15T16:06:14.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.790+0000 7f219c08f700 1 -- 172.21.15.49:0/1431537328 wait complete. 2024-05-15T16:06:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.791+0000 7f219c08f700 1 Processor -- start 2024-05-15T16:06:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.791+0000 7f219c08f700 1 -- start start 2024-05-15T16:06:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.791+0000 7f219c08f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194100ee0 0x7f21940732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.791+0000 7f219c08f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 0x7f21940737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.791+0000 7f219c08f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2194073d20 con 0x7f2194101890 2024-05-15T16:06:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.791+0000 7f219c08f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2194073e60 con 0x7f2194100ee0 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.791+0000 7f219962a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 0x7f21940737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.791+0000 7f219962a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 0x7f21940737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36796/0 (socket says 172.21.15.49:36796) 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.791+0000 7f219962a700 1 -- 172.21.15.49:0/3009921120 learned_addr learned my addr 172.21.15.49:0/3009921120 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f2199e2b700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194100ee0 0x7f21940732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f219962a700 1 -- 172.21.15.49:0/3009921120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194100ee0 msgr2=0x7f21940732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f219962a700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194100ee0 0x7f21940732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f219962a700 1 -- 172.21.15.49:0/3009921120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21880095f0 con 0x7f2194101890 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f219962a700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 0x7f21940737e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f218800bd70 tx=0x7f218800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f2192ffd700 1 -- 172.21.15.49:0/3009921120 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218801d070 con 0x7f2194101890 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f2192ffd700 1 -- 172.21.15.49:0/3009921120 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21880056e0 con 0x7f2194101890 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21940740e0 con 0x7f2194101890 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f2192ffd700 1 -- 172.21.15.49:0/3009921120 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218800fe50 con 0x7f2194101890 2024-05-15T16:06:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.792+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21941a3020 con 0x7f2194101890 2024-05-15T16:06:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.793+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21941092e0 con 0x7f2194101890 2024-05-15T16:06:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.794+0000 7f2192ffd700 1 -- 172.21.15.49:0/3009921120 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f218800f680 con 0x7f2194101890 2024-05-15T16:06:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.794+0000 7f2192ffd700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f217c06e200 0x7f217c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.794+0000 7f2192ffd700 1 -- 172.21.15.49:0/3009921120 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f218808f400 con 0x7f2194101890 2024-05-15T16:06:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.794+0000 7f2199e2b700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f217c06e200 0x7f217c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:14.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.795+0000 7f2199e2b700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f217c06e200 0x7f217c0706c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2184009550 tx=0x7f21840091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:14.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:14.799+0000 7f2192ffd700 1 -- 172.21.15.49:0/3009921120 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2188058760 con 0x7f2194101890 2024-05-15T16:06:15.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.026+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f219410de80 con 0x7f217c06e200 2024-05-15T16:06:15.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.027+0000 7f2192ffd700 1 -- 172.21.15.49:0/3009921120 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f219410de80 con 0x7f217c06e200 2024-05-15T16:06:15.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f217c06e200 msgr2=0x7f217c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f217c06e200 0x7f217c0706c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2184009550 tx=0x7f21840091b0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 msgr2=0x7f21940737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 0x7f21940737e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f218800bd70 tx=0x7f218800bda0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 shutdown_connections 2024-05-15T16:06:15.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f217c06e200 0x7f217c0706c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194100ee0 0x7f21940732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 --2- 172.21.15.49:0/3009921120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2194101890 0x7f21940737e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 >> 172.21.15.49:0/3009921120 conn(0x7f21940fc790 msgr2=0x7f21940fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:15.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 shutdown_connections 2024-05-15T16:06:15.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.029+0000 7f219c08f700 1 -- 172.21.15.49:0/3009921120 wait complete. 2024-05-15T16:06:15.041 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:06:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.139+0000 7ff01babf700 1 -- 172.21.15.49:0/2868353304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 msgr2=0x7ff0141001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.139+0000 7ff01babf700 1 --2- 172.21.15.49:0/2868353304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 0x7ff0141001d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff010009910 tx=0x7ff010009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.139+0000 7ff01babf700 1 -- 172.21.15.49:0/2868353304 shutdown_connections 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.139+0000 7ff01babf700 1 --2- 172.21.15.49:0/2868353304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141007a0 0x7ff0141047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.139+0000 7ff01babf700 1 --2- 172.21.15.49:0/2868353304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 0x7ff0141001d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.139+0000 7ff01babf700 1 -- 172.21.15.49:0/2868353304 >> 172.21.15.49:0/2868353304 conn(0x7ff0140fb6a0 msgr2=0x7ff0140fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.139+0000 7ff01babf700 1 -- 172.21.15.49:0/2868353304 shutdown_connections 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.140+0000 7ff01babf700 1 -- 172.21.15.49:0/2868353304 wait complete. 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.140+0000 7ff01babf700 1 Processor -- start 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.140+0000 7ff01babf700 1 -- start start 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.140+0000 7ff01babf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 0x7ff014101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.140+0000 7ff01babf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141007a0 0x7ff014101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.140+0000 7ff01babf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff014102310 con 0x7ff0140ffdf0 2024-05-15T16:06:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.140+0000 7ff01babf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff014102450 con 0x7ff0141007a0 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.140+0000 7ff01985b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 0x7ff014101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff01985b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 0x7ff014101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36818/0 (socket says 172.21.15.49:36818) 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff01985b700 1 -- 172.21.15.49:0/844531258 learned_addr learned my addr 172.21.15.49:0/844531258 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff01905a700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141007a0 0x7ff014101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff01985b700 1 -- 172.21.15.49:0/844531258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141007a0 msgr2=0x7ff014101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff01985b700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141007a0 0x7ff014101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff01985b700 1 -- 172.21.15.49:0/844531258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0100095f0 con 0x7ff0140ffdf0 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff01985b700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 0x7ff014101890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff01000bd70 tx=0x7ff01000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff00affd700 1 -- 172.21.15.49:0/844531258 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff01001d070 con 0x7ff0140ffdf0 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff00affd700 1 -- 172.21.15.49:0/844531258 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff010005700 con 0x7ff0140ffdf0 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0141026d0 con 0x7ff0140ffdf0 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff00affd700 1 -- 172.21.15.49:0/844531258 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff010017400 con 0x7ff0140ffdf0 2024-05-15T16:06:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.141+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0141a2ff0 con 0x7ff0140ffdf0 2024-05-15T16:06:15.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.142+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0141012d0 con 0x7ff0140ffdf0 2024-05-15T16:06:15.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.143+0000 7ff00affd700 1 -- 172.21.15.49:0/844531258 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff010003a60 con 0x7ff0140ffdf0 2024-05-15T16:06:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.143+0000 7ff00affd700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff00006e200 0x7ff0000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.143+0000 7ff00affd700 1 -- 172.21.15.49:0/844531258 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff01008f640 con 0x7ff0140ffdf0 2024-05-15T16:06:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.143+0000 7ff01905a700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff00006e200 0x7ff0000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.144+0000 7ff01905a700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff00006e200 0x7ff0000706c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff014102fc0 tx=0x7ff00400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:15.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.148+0000 7ff00affd700 1 -- 172.21.15.49:0/844531258 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff010058920 con 0x7ff0140ffdf0 2024-05-15T16:06:15.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.377+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff01410cec0 con 0x7ff00006e200 2024-05-15T16:06:15.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.378+0000 7ff00affd700 1 -- 172.21.15.49:0/844531258 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff01410cec0 con 0x7ff00006e200 2024-05-15T16:06:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff00006e200 msgr2=0x7ff0000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff00006e200 0x7ff0000706c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff014102fc0 tx=0x7ff00400a300 comp rx=0 tx=0).stop 2024-05-15T16:06:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 msgr2=0x7ff014101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 0x7ff014101890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff01000bd70 tx=0x7ff01000bda0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 shutdown_connections 2024-05-15T16:06:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff00006e200 0x7ff0000706c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0140ffdf0 0x7ff014101890 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 --2- 172.21.15.49:0/844531258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141007a0 0x7ff014101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 >> 172.21.15.49:0/844531258 conn(0x7ff0140fb6a0 msgr2=0x7ff014069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:15.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 shutdown_connections 2024-05-15T16:06:15.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.381+0000 7ff01babf700 1 -- 172.21.15.49:0/844531258 wait complete. 2024-05-15T16:06:15.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.489+0000 7fa294475700 1 -- 172.21.15.49:0/4123736686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 msgr2=0x7fa28c102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.489+0000 7fa294475700 1 --2- 172.21.15.49:0/4123736686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 0x7fa28c102290 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa288009940 tx=0x7fa288009c50 comp rx=0 tx=0).stop 2024-05-15T16:06:15.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.489+0000 7fa294475700 1 -- 172.21.15.49:0/4123736686 shutdown_connections 2024-05-15T16:06:15.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.489+0000 7fa294475700 1 --2- 172.21.15.49:0/4123736686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa28c1027d0 0x7fa28c10acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.489+0000 7fa294475700 1 --2- 172.21.15.49:0/4123736686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 0x7fa28c102290 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.489+0000 7fa294475700 1 -- 172.21.15.49:0/4123736686 >> 172.21.15.49:0/4123736686 conn(0x7fa28c0fb6a0 msgr2=0x7fa28c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:15.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.489+0000 7fa294475700 1 -- 172.21.15.49:0/4123736686 shutdown_connections 2024-05-15T16:06:15.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.489+0000 7fa294475700 1 -- 172.21.15.49:0/4123736686 wait complete. 2024-05-15T16:06:15.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.490+0000 7fa294475700 1 Processor -- start 2024-05-15T16:06:15.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.490+0000 7fa294475700 1 -- start start 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.490+0000 7fa294475700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 0x7fa28c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.490+0000 7fa294475700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa28c1027d0 0x7fa28c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.490+0000 7fa294475700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c19d450 con 0x7fa28c101eb0 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.490+0000 7fa294475700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c196a30 con 0x7fa28c1027d0 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa292211700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 0x7fa28c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa292211700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 0x7fa28c19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36824/0 (socket says 172.21.15.49:36824) 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa292211700 1 -- 172.21.15.49:0/2063409724 learned_addr learned my addr 172.21.15.49:0/2063409724 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa291a10700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa28c1027d0 0x7fa28c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa292211700 1 -- 172.21.15.49:0/2063409724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa28c1027d0 msgr2=0x7fa28c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa292211700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa28c1027d0 0x7fa28c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa292211700 1 -- 172.21.15.49:0/2063409724 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2880095f0 con 0x7fa28c101eb0 2024-05-15T16:06:15.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa292211700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 0x7fa28c19c8f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa288005620 tx=0x7fa288005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:15.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa2837fe700 1 -- 172.21.15.49:0/2063409724 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa28801d070 con 0x7fa28c101eb0 2024-05-15T16:06:15.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa2837fe700 1 -- 172.21.15.49:0/2063409724 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa28800f460 con 0x7fa28c101eb0 2024-05-15T16:06:15.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c196cb0 con 0x7fa28c101eb0 2024-05-15T16:06:15.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa2837fe700 1 -- 172.21.15.49:0/2063409724 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa288021620 con 0x7fa28c101eb0 2024-05-15T16:06:15.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.491+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c1971a0 con 0x7fa28c101eb0 2024-05-15T16:06:15.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.492+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c1055d0 con 0x7fa28c101eb0 2024-05-15T16:06:15.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.493+0000 7fa2837fe700 1 -- 172.21.15.49:0/2063409724 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa28800f5d0 con 0x7fa28c101eb0 2024-05-15T16:06:15.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.493+0000 7fa2837fe700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa27806e200 0x7fa2780706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:15.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.493+0000 7fa2837fe700 1 -- 172.21.15.49:0/2063409724 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa28808f370 con 0x7fa28c101eb0 2024-05-15T16:06:15.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.493+0000 7fa291a10700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa27806e200 0x7fa2780706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:15.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.494+0000 7fa291a10700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa27806e200 0x7fa2780706c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa27c005fd0 tx=0x7fa27c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:15.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.498+0000 7fa2837fe700 1 -- 172.21.15.49:0/2063409724 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa28805c200 con 0x7fa28c101eb0 2024-05-15T16:06:15.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.714+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa28c197ef0 con 0x7fa27806e200 2024-05-15T16:06:15.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.726+0000 7fa2837fe700 1 -- 172.21.15.49:0/2063409724 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa28c197ef0 con 0x7fa27806e200 2024-05-15T16:06:15.726 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (9m) 2m ago 21m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (21m) 2m ago 21m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (20m) 2m ago 20m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (6m) 2m ago 21m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (6m) 2m ago 20m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (9m) 2m ago 20m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (3m) 2m ago 15m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (2m) 2m ago 15m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (2m) 2m ago 15m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (15m) 2m ago 15m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (11m) 2m ago 22m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (10m) 2m ago 19m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (8m) 2m ago 22m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:06:15.727 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (7m) 2m ago 19m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:06:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (10m) 2m ago 21m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:06:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (10m) 2m ago 20m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:06:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (6m) 2m ago 19m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:06:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (5m) 2m ago 18m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:06:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (5m) 2m ago 18m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:06:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (4m) 2m ago 17m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:06:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (4m) 2m ago 17m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:06:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (3m) 2m ago 16m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:06:15.728 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (9m) 2m ago 20m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa27806e200 msgr2=0x7fa2780706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa27806e200 0x7fa2780706c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa27c005fd0 tx=0x7fa27c00a560 comp rx=0 tx=0).stop 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 msgr2=0x7fa28c19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 0x7fa28c19c8f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa288005620 tx=0x7fa288005700 comp rx=0 tx=0).stop 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 shutdown_connections 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa27806e200 0x7fa2780706c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa28c101eb0 0x7fa28c19c8f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 --2- 172.21.15.49:0/2063409724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa28c1027d0 0x7fa28c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 >> 172.21.15.49:0/2063409724 conn(0x7fa28c0fb6a0 msgr2=0x7fa28c1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 shutdown_connections 2024-05-15T16:06:15.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.728+0000 7fa294475700 1 -- 172.21.15.49:0/2063409724 wait complete. 2024-05-15T16:06:15.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.836+0000 7f9570bda700 1 -- 172.21.15.49:0/63398158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c101ed0 msgr2=0x7f956c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.836+0000 7f9570bda700 1 --2- 172.21.15.49:0/63398158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c101ed0 0x7f956c1022b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9554009910 tx=0x7f9554009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:15.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.836+0000 7f9570bda700 1 -- 172.21.15.49:0/63398158 shutdown_connections 2024-05-15T16:06:15.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.836+0000 7f9570bda700 1 --2- 172.21.15.49:0/63398158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c1027f0 0x7f956c10ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.836+0000 7f9570bda700 1 --2- 172.21.15.49:0/63398158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c101ed0 0x7f956c1022b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.836+0000 7f9570bda700 1 -- 172.21.15.49:0/63398158 >> 172.21.15.49:0/63398158 conn(0x7f956c0fb6a0 msgr2=0x7f956c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:15.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.836+0000 7f9570bda700 1 -- 172.21.15.49:0/63398158 shutdown_connections 2024-05-15T16:06:15.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.836+0000 7f9570bda700 1 -- 172.21.15.49:0/63398158 wait complete. 2024-05-15T16:06:15.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9570bda700 1 Processor -- start 2024-05-15T16:06:15.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9570bda700 1 -- start start 2024-05-15T16:06:15.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9570bda700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c101ed0 0x7f956c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:15.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9570bda700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c1027f0 0x7f956c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:15.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9570bda700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c19d3f0 con 0x7f956c1027f0 2024-05-15T16:06:15.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9570bda700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c196a60 con 0x7f956c101ed0 2024-05-15T16:06:15.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9569d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c1027f0 0x7f956c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9569d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c1027f0 0x7f956c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36856/0 (socket says 172.21.15.49:36856) 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9569d9b700 1 -- 172.21.15.49:0/1784223967 learned_addr learned my addr 172.21.15.49:0/1784223967 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f956a59c700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c101ed0 0x7f956c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9569d9b700 1 -- 172.21.15.49:0/1784223967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c101ed0 msgr2=0x7f956c19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9569d9b700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c101ed0 0x7f956c19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.837+0000 7f9569d9b700 1 -- 172.21.15.49:0/1784223967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95540095f0 con 0x7f956c1027f0 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.838+0000 7f9569d9b700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c1027f0 0x7f956c19ce60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f955c00c960 tx=0x7f955c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.838+0000 7f95637fe700 1 -- 172.21.15.49:0/1784223967 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955c007a70 con 0x7f956c1027f0 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.838+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956c196d40 con 0x7f956c1027f0 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.838+0000 7f95637fe700 1 -- 172.21.15.49:0/1784223967 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f955c007bd0 con 0x7f956c1027f0 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.838+0000 7f95637fe700 1 -- 172.21.15.49:0/1784223967 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955c0186c0 con 0x7f956c1027f0 2024-05-15T16:06:15.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.838+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956c197290 con 0x7f956c1027f0 2024-05-15T16:06:15.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.839+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956c1055f0 con 0x7f956c1027f0 2024-05-15T16:06:15.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.840+0000 7f95637fe700 1 -- 172.21.15.49:0/1784223967 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f955c00faa0 con 0x7f956c1027f0 2024-05-15T16:06:15.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.840+0000 7f95637fe700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f955806e210 0x7f95580706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:15.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.840+0000 7f95637fe700 1 -- 172.21.15.49:0/1784223967 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f955c08e1d0 con 0x7f956c1027f0 2024-05-15T16:06:15.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.840+0000 7f956a59c700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f955806e210 0x7f95580706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:15.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.841+0000 7f956a59c700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f955806e210 0x7f95580706d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f955400b5c0 tx=0x7f955401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:15.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:15.845+0000 7f95637fe700 1 -- 172.21.15.49:0/1784223967 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f955c05ad50 con 0x7f956c1027f0 2024-05-15T16:06:16.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.143+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f956c10dec0 con 0x7f956c1027f0 2024-05-15T16:06:16.145 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:15 smithi049 ceph-mon[112142]: pgmap v276: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:16.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.144+0000 7f95637fe700 1 -- 172.21.15.49:0/1784223967 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f955c05ab70 con 0x7f956c1027f0 2024-05-15T16:06:16.145 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:06:16.145 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:06:16.145 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:06:16.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:06:16.146 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:06:16.147 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:06:16.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.146+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f955806e210 msgr2=0x7f95580706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.146+0000 7f9570bda700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f955806e210 0x7f95580706d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f955400b5c0 tx=0x7f955401a040 comp rx=0 tx=0).stop 2024-05-15T16:06:16.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.146+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c1027f0 msgr2=0x7f956c19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.146+0000 7f9570bda700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c1027f0 0x7f956c19ce60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f955c00c960 tx=0x7f955c00cd20 comp rx=0 tx=0).stop 2024-05-15T16:06:16.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.146+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 shutdown_connections 2024-05-15T16:06:16.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.146+0000 7f9570bda700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f955806e210 0x7f95580706d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.146+0000 7f9570bda700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c101ed0 0x7f956c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.146+0000 7f9570bda700 1 --2- 172.21.15.49:0/1784223967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f956c1027f0 0x7f956c19ce60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.146+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 >> 172.21.15.49:0/1784223967 conn(0x7f956c0fb6a0 msgr2=0x7f956c1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:16.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.147+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 shutdown_connections 2024-05-15T16:06:16.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.147+0000 7f9570bda700 1 -- 172.21.15.49:0/1784223967 wait complete. 2024-05-15T16:06:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:15 smithi144 ceph-mon[86635]: pgmap v276: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.256+0000 7f4a2c27d700 1 -- 172.21.15.49:0/4000118921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a240ffde0 msgr2=0x7f4a241001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.256+0000 7f4a2c27d700 1 --2- 172.21.15.49:0/4000118921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a240ffde0 0x7f4a241001c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009910 tx=0x7f4a20009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.256+0000 7f4a2c27d700 1 -- 172.21.15.49:0/4000118921 shutdown_connections 2024-05-15T16:06:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.256+0000 7f4a2c27d700 1 --2- 172.21.15.49:0/4000118921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a24100790 0x7f4a241047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.256+0000 7f4a2c27d700 1 --2- 172.21.15.49:0/4000118921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a240ffde0 0x7f4a241001c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.256+0000 7f4a2c27d700 1 -- 172.21.15.49:0/4000118921 >> 172.21.15.49:0/4000118921 conn(0x7f4a240fb690 msgr2=0x7f4a240fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.256+0000 7f4a2c27d700 1 -- 172.21.15.49:0/4000118921 shutdown_connections 2024-05-15T16:06:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.256+0000 7f4a2c27d700 1 -- 172.21.15.49:0/4000118921 wait complete. 2024-05-15T16:06:16.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a2c27d700 1 Processor -- start 2024-05-15T16:06:16.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a2c27d700 1 -- start start 2024-05-15T16:06:16.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a2c27d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a240ffde0 0x7f4a24196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:16.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a2c27d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a24100790 0x7f4a24197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a2c27d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a241978c0 con 0x7f4a24100790 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a2c27d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2419a530 con 0x7f4a240ffde0 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a29818700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a24100790 0x7f4a24197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a29818700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a24100790 0x7f4a24197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36878/0 (socket says 172.21.15.49:36878) 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a29818700 1 -- 172.21.15.49:0/3359953871 learned_addr learned my addr 172.21.15.49:0/3359953871 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a2a019700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a240ffde0 0x7f4a24196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a29818700 1 -- 172.21.15.49:0/3359953871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a240ffde0 msgr2=0x7f4a24196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a29818700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a240ffde0 0x7f4a24196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a29818700 1 -- 172.21.15.49:0/3359953871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a200095f0 con 0x7f4a24100790 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.257+0000 7f4a2a019700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a240ffde0 0x7f4a24196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.258+0000 7f4a29818700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a24100790 0x7f4a24197230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400b480 tx=0x7f4a1400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.258+0000 7f4a1b7fe700 1 -- 172.21.15.49:0/3359953871 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a140118c0 con 0x7f4a24100790 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.258+0000 7f4a1b7fe700 1 -- 172.21.15.49:0/3359953871 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a14010380 con 0x7f4a24100790 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.258+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2419a810 con 0x7f4a24100790 2024-05-15T16:06:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.258+0000 7f4a1b7fe700 1 -- 172.21.15.49:0/3359953871 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a1400f4f0 con 0x7f4a24100790 2024-05-15T16:06:16.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.258+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2419ad60 con 0x7f4a24100790 2024-05-15T16:06:16.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.259+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a241012c0 con 0x7f4a24100790 2024-05-15T16:06:16.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.260+0000 7f4a1b7fe700 1 -- 172.21.15.49:0/3359953871 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4a140104f0 con 0x7f4a24100790 2024-05-15T16:06:16.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.260+0000 7f4a1b7fe700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4a1006e210 0x7f4a100706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:16.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.260+0000 7f4a1b7fe700 1 -- 172.21.15.49:0/3359953871 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4a1408dde0 con 0x7f4a24100790 2024-05-15T16:06:16.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.260+0000 7f4a2a019700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4a1006e210 0x7f4a100706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:16.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.261+0000 7f4a2a019700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4a1006e210 0x7f4a100706d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009fd0 tx=0x7f4a2000b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.265+0000 7f4a1b7fe700 1 -- 172.21.15.49:0/3359953871 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4a140570c0 con 0x7f4a24100790 2024-05-15T16:06:16.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.514+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4a2410ce20 con 0x7f4a24100790 2024-05-15T16:06:16.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.515+0000 7f4a1b7fe700 1 -- 172.21.15.49:0/3359953871 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f4a1405a780 con 0x7f4a24100790 2024-05-15T16:06:16.516 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:06:16.516 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:06:16.516 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:06:16.516 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:06:16.516 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:06:16.516 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:06:16.516 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:06:16.517 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:06:16.518 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:06:16.519 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:06:16.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4a1006e210 msgr2=0x7f4a100706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4a1006e210 0x7f4a100706d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009fd0 tx=0x7f4a2000b560 comp rx=0 tx=0).stop 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a24100790 msgr2=0x7f4a24197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a24100790 0x7f4a24197230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400b480 tx=0x7f4a1400b840 comp rx=0 tx=0).stop 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 shutdown_connections 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4a1006e210 0x7f4a100706d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a240ffde0 0x7f4a24196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 --2- 172.21.15.49:0/3359953871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a24100790 0x7f4a24197230 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 >> 172.21.15.49:0/3359953871 conn(0x7f4a240fb690 msgr2=0x7f4a240694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 shutdown_connections 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.517+0000 7f4a2c27d700 1 -- 172.21.15.49:0/3359953871 wait complete. 2024-05-15T16:06:16.521 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:06:16.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.628+0000 7f1e896ab700 1 -- 172.21.15.49:0/2496464680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 msgr2=0x7f1e841058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.628+0000 7f1e896ab700 1 --2- 172.21.15.49:0/2496464680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 0x7f1e841058b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009910 tx=0x7f1e6c009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:16.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.628+0000 7f1e896ab700 1 -- 172.21.15.49:0/2496464680 shutdown_connections 2024-05-15T16:06:16.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.628+0000 7f1e896ab700 1 --2- 172.21.15.49:0/2496464680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 0x7f1e841058b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.628+0000 7f1e896ab700 1 --2- 172.21.15.49:0/2496464680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84100eb0 0x7f1e84101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.628+0000 7f1e896ab700 1 -- 172.21.15.49:0/2496464680 >> 172.21.15.49:0/2496464680 conn(0x7f1e840fc760 msgr2=0x7f1e840feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:16.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.628+0000 7f1e896ab700 1 -- 172.21.15.49:0/2496464680 shutdown_connections 2024-05-15T16:06:16.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.628+0000 7f1e896ab700 1 -- 172.21.15.49:0/2496464680 wait complete. 2024-05-15T16:06:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.628+0000 7f1e896ab700 1 Processor -- start 2024-05-15T16:06:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e896ab700 1 -- start start 2024-05-15T16:06:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e896ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84100eb0 0x7f1e840732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e896ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 0x7f1e840737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e896ab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e84075270 con 0x7f1e84101860 2024-05-15T16:06:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e896ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e84073d20 con 0x7f1e84100eb0 2024-05-15T16:06:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e827fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 0x7f1e840737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e827fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 0x7f1e840737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36904/0 (socket says 172.21.15.49:36904) 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e827fc700 1 -- 172.21.15.49:0/2842983387 learned_addr learned my addr 172.21.15.49:0/2842983387 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e82ffd700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84100eb0 0x7f1e840732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e827fc700 1 -- 172.21.15.49:0/2842983387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84100eb0 msgr2=0x7f1e840732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e827fc700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84100eb0 0x7f1e840732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e827fc700 1 -- 172.21.15.49:0/2842983387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c0095f0 con 0x7f1e84101860 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e82ffd700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84100eb0 0x7f1e840732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.629+0000 7f1e827fc700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 0x7f1e840737e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0092e0 tx=0x7f1e6c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.630+0000 7f1e7bfff700 1 -- 172.21.15.49:0/2842983387 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e6c01d070 con 0x7f1e84101860 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.630+0000 7f1e7bfff700 1 -- 172.21.15.49:0/2842983387 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e6c00f460 con 0x7f1e84101860 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.630+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e84073f70 con 0x7f1e84101860 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.630+0000 7f1e7bfff700 1 -- 172.21.15.49:0/2842983387 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e6c017630 con 0x7f1e84101860 2024-05-15T16:06:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.630+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e841a2f60 con 0x7f1e84101860 2024-05-15T16:06:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.631+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e841092b0 con 0x7f1e84101860 2024-05-15T16:06:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.631+0000 7f1e7bfff700 1 -- 172.21.15.49:0/2842983387 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1e6c00f5d0 con 0x7f1e84101860 2024-05-15T16:06:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.632+0000 7f1e7bfff700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e7006e200 0x7f1e700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.632+0000 7f1e82ffd700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e7006e200 0x7f1e700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.632+0000 7f1e7bfff700 1 -- 172.21.15.49:0/2842983387 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1e6c08f5f0 con 0x7f1e84101860 2024-05-15T16:06:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.632+0000 7f1e82ffd700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e7006e200 0x7f1e700706c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e74005fd0 tx=0x7f1e74005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:16.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.637+0000 7f1e7bfff700 1 -- 172.21.15.49:0/2842983387 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1e6c0588d0 con 0x7f1e84101860 2024-05-15T16:06:16.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.862+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e8410de50 con 0x7f1e7006e200 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.863+0000 7f1e7bfff700 1 -- 172.21.15.49:0/2842983387 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1e8410de50 con 0x7f1e7006e200 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:06:16.864 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:06:16.865 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:06:16.865 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:06:16.865 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:06:16.865 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:06:16.865 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:06:16.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.866+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e7006e200 msgr2=0x7f1e700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.866+0000 7f1e896ab700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e7006e200 0x7f1e700706c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e74005fd0 tx=0x7f1e74005e20 comp rx=0 tx=0).stop 2024-05-15T16:06:16.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.866+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 msgr2=0x7f1e840737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.866+0000 7f1e896ab700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 0x7f1e840737e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0092e0 tx=0x7f1e6c00be20 comp rx=0 tx=0).stop 2024-05-15T16:06:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.866+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 shutdown_connections 2024-05-15T16:06:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.866+0000 7f1e896ab700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1e7006e200 0x7f1e700706c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.866+0000 7f1e896ab700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e84100eb0 0x7f1e840732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.866+0000 7f1e896ab700 1 --2- 172.21.15.49:0/2842983387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84101860 0x7f1e840737e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.866+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 >> 172.21.15.49:0/2842983387 conn(0x7f1e840fc760 msgr2=0x7f1e840fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.867+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 shutdown_connections 2024-05-15T16:06:16.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.867+0000 7f1e896ab700 1 -- 172.21.15.49:0/2842983387 wait complete. 2024-05-15T16:06:16.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.978+0000 7ff533154700 1 -- 172.21.15.49:0/1378536379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 msgr2=0x7ff52c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.978+0000 7ff533154700 1 --2- 172.21.15.49:0/1378536379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 0x7ff52c1012d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00b0a0 tx=0x7ff51c00b3b0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.978+0000 7ff533154700 1 -- 172.21.15.49:0/1378536379 shutdown_connections 2024-05-15T16:06:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.978+0000 7ff533154700 1 --2- 172.21.15.49:0/1378536379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff52c1018a0 0x7ff52c1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.978+0000 7ff533154700 1 --2- 172.21.15.49:0/1378536379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 0x7ff52c1012d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.978+0000 7ff533154700 1 -- 172.21.15.49:0/1378536379 >> 172.21.15.49:0/1378536379 conn(0x7ff52c0fc7c0 msgr2=0x7ff52c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.978+0000 7ff533154700 1 -- 172.21.15.49:0/1378536379 shutdown_connections 2024-05-15T16:06:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.978+0000 7ff533154700 1 -- 172.21.15.49:0/1378536379 wait complete. 2024-05-15T16:06:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff533154700 1 Processor -- start 2024-05-15T16:06:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff533154700 1 -- start start 2024-05-15T16:06:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff533154700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 0x7ff52c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff533154700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff52c1018a0 0x7ff52c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff533154700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c19d550 con 0x7ff52c100ef0 2024-05-15T16:06:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff533154700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c196ac0 con 0x7ff52c1018a0 2024-05-15T16:06:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff530ef0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 0x7ff52c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff530ef0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 0x7ff52c19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36930/0 (socket says 172.21.15.49:36930) 2024-05-15T16:06:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff530ef0700 1 -- 172.21.15.49:0/3317977063 learned_addr learned my addr 172.21.15.49:0/3317977063 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff530ef0700 1 -- 172.21.15.49:0/3317977063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff52c1018a0 msgr2=0x7ff52c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff52bfff700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff52c1018a0 0x7ff52c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff530ef0700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff52c1018a0 0x7ff52c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff530ef0700 1 -- 172.21.15.49:0/3317977063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c009d00 con 0x7ff52c100ef0 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.979+0000 7ff52bfff700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff52c1018a0 0x7ff52c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.980+0000 7ff530ef0700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 0x7ff52c19c980 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff51c000f80 tx=0x7ff51c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.980+0000 7ff529ffb700 1 -- 172.21.15.49:0/3317977063 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51c00e040 con 0x7ff52c100ef0 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.980+0000 7ff529ffb700 1 -- 172.21.15.49:0/3317977063 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff51c003de0 con 0x7ff52c100ef0 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.980+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c196d40 con 0x7ff52c100ef0 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.980+0000 7ff529ffb700 1 -- 172.21.15.49:0/3317977063 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51c01d720 con 0x7ff52c100ef0 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.980+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c197230 con 0x7ff52c100ef0 2024-05-15T16:06:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.981+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c109290 con 0x7ff52c100ef0 2024-05-15T16:06:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.982+0000 7ff529ffb700 1 -- 172.21.15.49:0/3317977063 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff51c004690 con 0x7ff52c100ef0 2024-05-15T16:06:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.982+0000 7ff529ffb700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff51806e200 0x7ff5180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.982+0000 7ff529ffb700 1 -- 172.21.15.49:0/3317977063 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff51c08f9e0 con 0x7ff52c100ef0 2024-05-15T16:06:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.982+0000 7ff52bfff700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff51806e200 0x7ff5180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.983+0000 7ff52bfff700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff51806e200 0x7ff5180706c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff514005fd0 tx=0x7ff514005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:16.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:16.987+0000 7ff529ffb700 1 -- 172.21.15.49:0/3317977063 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff51c05c7f0 con 0x7ff52c100ef0 2024-05-15T16:06:17.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:16 smithi049 ceph-mon[112142]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:17.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:16 smithi049 ceph-mon[112142]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:17.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:16 smithi049 ceph-mon[112142]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:17.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:16 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1784223967' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:06:17.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:16 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3359953871' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:06:17.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.278+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff52c1023d0 con 0x7ff52c100ef0 2024-05-15T16:06:17.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.279+0000 7ff529ffb700 1 -- 172.21.15.49:0/3317977063 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7ff51c05c380 con 0x7ff52c100ef0 2024-05-15T16:06:17.280 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:06:17.280 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:06:17.280 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:06:17.280 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:06:17.280 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:06:17.280 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:06:17.280 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:06:17.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff51806e200 msgr2=0x7ff5180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:17.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff51806e200 0x7ff5180706c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff514005fd0 tx=0x7ff514005dc0 comp rx=0 tx=0).stop 2024-05-15T16:06:17.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 msgr2=0x7ff52c19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:17.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 0x7ff52c19c980 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff51c000f80 tx=0x7ff51c007bd0 comp rx=0 tx=0).stop 2024-05-15T16:06:17.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 shutdown_connections 2024-05-15T16:06:17.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff51806e200 0x7ff5180706c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:17.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff52c100ef0 0x7ff52c19c980 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:17.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 --2- 172.21.15.49:0/3317977063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff52c1018a0 0x7ff52c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:17.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 >> 172.21.15.49:0/3317977063 conn(0x7ff52c0fc7c0 msgr2=0x7ff52c0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:17.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 shutdown_connections 2024-05-15T16:06:17.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:17.281+0000 7ff533154700 1 -- 172.21.15.49:0/3317977063 wait complete. 2024-05-15T16:06:17.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:16 smithi144 ceph-mon[86635]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:17.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:16 smithi144 ceph-mon[86635]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:17.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:16 smithi144 ceph-mon[86635]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:17.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:16 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1784223967' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:06:17.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:16 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3359953871' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:06:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:17 smithi144 ceph-mon[86635]: pgmap v277: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:17 smithi144 ceph-mon[86635]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:17 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3317977063' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:06:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:17 smithi049 ceph-mon[112142]: pgmap v277: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:18.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:17 smithi049 ceph-mon[112142]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:18.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:17 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3317977063' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:06:20.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:19 smithi049 ceph-mon[112142]: pgmap v278: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:19 smithi144 ceph-mon[86635]: pgmap v278: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:06:21.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:06:22.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:21 smithi144 ceph-mon[86635]: pgmap v279: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:21 smithi049 ceph-mon[112142]: pgmap v279: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:23 smithi144 ceph-mon[86635]: pgmap v280: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:23 smithi049 ceph-mon[112142]: pgmap v280: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:25 smithi144 ceph-mon[86635]: pgmap v281: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:25 smithi049 ceph-mon[112142]: pgmap v281: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:28.257 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:27 smithi049 ceph-mon[112142]: pgmap v282: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:27 smithi144 ceph-mon[86635]: pgmap v282: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:30.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:29 smithi049 ceph-mon[112142]: pgmap v283: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:29 smithi144 ceph-mon[86635]: pgmap v283: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:31 smithi144 ceph-mon[86635]: pgmap v284: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:31 smithi049 ceph-mon[112142]: pgmap v284: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:33 smithi144 ceph-mon[86635]: pgmap v285: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:33 smithi049 ceph-mon[112142]: pgmap v285: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:35 smithi144 ceph-mon[86635]: pgmap v286: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:06:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:35 smithi049 ceph-mon[112142]: pgmap v286: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:06:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:37 smithi144 ceph-mon[86635]: pgmap v287: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:37 smithi049 ceph-mon[112142]: pgmap v287: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:39 smithi144 ceph-mon[86635]: pgmap v288: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:39 smithi049 ceph-mon[112142]: pgmap v288: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:41 smithi144 ceph-mon[86635]: pgmap v289: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:41 smithi049 ceph-mon[112142]: pgmap v289: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:43 smithi144 ceph-mon[86635]: pgmap v290: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:43 smithi049 ceph-mon[112142]: pgmap v290: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:45 smithi144 ceph-mon[86635]: pgmap v291: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:45 smithi049 ceph-mon[112142]: pgmap v291: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:47.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.404+0000 7ff40e650700 1 -- 172.21.15.49:0/22279626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4081007a0 msgr2=0x7ff4081047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:47.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.404+0000 7ff40e650700 1 --2- 172.21.15.49:0/22279626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4081007a0 0x7ff4081047f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009910 tx=0x7ff3f0009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.404+0000 7ff40e650700 1 -- 172.21.15.49:0/22279626 shutdown_connections 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.404+0000 7ff40e650700 1 --2- 172.21.15.49:0/22279626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4081007a0 0x7ff4081047f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.404+0000 7ff40e650700 1 --2- 172.21.15.49:0/22279626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4080ffdf0 0x7ff4081001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.405+0000 7ff40e650700 1 -- 172.21.15.49:0/22279626 >> 172.21.15.49:0/22279626 conn(0x7ff4080fb6a0 msgr2=0x7ff4080fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.405+0000 7ff40e650700 1 -- 172.21.15.49:0/22279626 shutdown_connections 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.405+0000 7ff40e650700 1 -- 172.21.15.49:0/22279626 wait complete. 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.405+0000 7ff40e650700 1 Processor -- start 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.405+0000 7ff40e650700 1 -- start start 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.405+0000 7ff40e650700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4080ffdf0 0x7ff40810ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.405+0000 7ff40e650700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4081007a0 0x7ff40810b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff40e650700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40810b5e0 con 0x7ff4080ffdf0 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff40e650700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40810b750 con 0x7ff4081007a0 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff407fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4080ffdf0 0x7ff40810ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff407fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4080ffdf0 0x7ff40810ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49662/0 (socket says 172.21.15.49:49662) 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff407fff700 1 -- 172.21.15.49:0/2537610440 learned_addr learned my addr 172.21.15.49:0/2537610440 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff3fffff700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4081007a0 0x7ff40810b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff407fff700 1 -- 172.21.15.49:0/2537610440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4081007a0 msgr2=0x7ff40810b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff407fff700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4081007a0 0x7ff40810b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff407fff700 1 -- 172.21.15.49:0/2537610440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f00095f0 con 0x7ff4080ffdf0 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff407fff700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4080ffdf0 0x7ff40810ab60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c8a0 tx=0x7ff3f800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff405ffb700 1 -- 172.21.15.49:0/2537610440 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f800cea0 con 0x7ff4080ffdf0 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4081a2f10 con 0x7ff4080ffdf0 2024-05-15T16:06:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff405ffb700 1 -- 172.21.15.49:0/2537610440 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3f8004830 con 0x7ff4080ffdf0 2024-05-15T16:06:47.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff405ffb700 1 -- 172.21.15.49:0/2537610440 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f80055b0 con 0x7ff4080ffdf0 2024-05-15T16:06:47.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.406+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4081a3270 con 0x7ff4080ffdf0 2024-05-15T16:06:47.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.407+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4081081f0 con 0x7ff4080ffdf0 2024-05-15T16:06:47.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.408+0000 7ff405ffb700 1 -- 172.21.15.49:0/2537610440 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff3f80049a0 con 0x7ff4080ffdf0 2024-05-15T16:06:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.409+0000 7ff405ffb700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff3f406e200 0x7ff3f40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.409+0000 7ff405ffb700 1 -- 172.21.15.49:0/2537610440 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff3f808ddc0 con 0x7ff4080ffdf0 2024-05-15T16:06:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.409+0000 7ff3fffff700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff3f406e200 0x7ff3f40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.409+0000 7ff3fffff700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff3f406e200 0x7ff3f40706c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff40810c610 tx=0x7ff3f001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:47.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.413+0000 7ff405ffb700 1 -- 172.21.15.49:0/2537610440 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff3f80570a0 con 0x7ff4080ffdf0 2024-05-15T16:06:47.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.642+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff40810c040 con 0x7ff3f406e200 2024-05-15T16:06:47.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.644+0000 7ff405ffb700 1 -- 172.21.15.49:0/2537610440 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff40810c040 con 0x7ff3f406e200 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff3f406e200 msgr2=0x7ff3f40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff3f406e200 0x7ff3f40706c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff40810c610 tx=0x7ff3f001a040 comp rx=0 tx=0).stop 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4080ffdf0 msgr2=0x7ff40810ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4080ffdf0 0x7ff40810ab60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c8a0 tx=0x7ff3f800cc60 comp rx=0 tx=0).stop 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 shutdown_connections 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff3f406e200 0x7ff3f40706c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4080ffdf0 0x7ff40810ab60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 --2- 172.21.15.49:0/2537610440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4081007a0 0x7ff40810b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 >> 172.21.15.49:0/2537610440 conn(0x7ff4080fb6a0 msgr2=0x7ff4080fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 shutdown_connections 2024-05-15T16:06:47.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.646+0000 7ff40e650700 1 -- 172.21.15.49:0/2537610440 wait complete. 2024-05-15T16:06:47.658 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:06:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.756+0000 7f5917f5e700 1 -- 172.21.15.49:0/2110638056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 msgr2=0x7f591010acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.756+0000 7f5917f5e700 1 --2- 172.21.15.49:0/2110638056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 0x7f591010acd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5900009910 tx=0x7f5900009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.756+0000 7f5917f5e700 1 -- 172.21.15.49:0/2110638056 shutdown_connections 2024-05-15T16:06:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.756+0000 7f5917f5e700 1 --2- 172.21.15.49:0/2110638056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 0x7f591010acd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.756+0000 7f5917f5e700 1 --2- 172.21.15.49:0/2110638056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910101ec0 0x7f59101022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.756+0000 7f5917f5e700 1 -- 172.21.15.49:0/2110638056 >> 172.21.15.49:0/2110638056 conn(0x7f5910076800 msgr2=0x7f59100fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.756+0000 7f5917f5e700 1 -- 172.21.15.49:0/2110638056 shutdown_connections 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.756+0000 7f5917f5e700 1 -- 172.21.15.49:0/2110638056 wait complete. 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f5917f5e700 1 Processor -- start 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f5917f5e700 1 -- start start 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f5917f5e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910101ec0 0x7f591019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f5917f5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 0x7f591019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f5917f5e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5910196aa0 con 0x7f59101027e0 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f5917f5e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5910196c10 con 0x7f5910101ec0 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f59154f9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 0x7f591019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f59154f9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 0x7f591019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49680/0 (socket says 172.21.15.49:49680) 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f59154f9700 1 -- 172.21.15.49:0/2028167195 learned_addr learned my addr 172.21.15.49:0/2028167195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.757+0000 7f5915cfa700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910101ec0 0x7f591019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.758+0000 7f59154f9700 1 -- 172.21.15.49:0/2028167195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910101ec0 msgr2=0x7f591019c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.758+0000 7f59154f9700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910101ec0 0x7f591019c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.758+0000 7f59154f9700 1 -- 172.21.15.49:0/2028167195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000095f0 con 0x7f59101027e0 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.758+0000 7f59154f9700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 0x7f591019cf30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f590000bd70 tx=0x7f590000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.758+0000 7f5906ffd700 1 -- 172.21.15.49:0/2028167195 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590001d070 con 0x7f59101027e0 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.758+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5910196e90 con 0x7f59101027e0 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.758+0000 7f5906ffd700 1 -- 172.21.15.49:0/2028167195 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59000056e0 con 0x7f59101027e0 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.758+0000 7f5906ffd700 1 -- 172.21.15.49:0/2028167195 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590000fe50 con 0x7f59101027e0 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.758+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5910197380 con 0x7f59101027e0 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.759+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5910068ec0 con 0x7f59101027e0 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.760+0000 7f5906ffd700 1 -- 172.21.15.49:0/2028167195 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f590000f680 con 0x7f59101027e0 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.760+0000 7f5906ffd700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f58fc06e200 0x7f58fc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.760+0000 7f5906ffd700 1 -- 172.21.15.49:0/2028167195 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f590008f3f0 con 0x7f59101027e0 2024-05-15T16:06:47.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.760+0000 7f5915cfa700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f58fc06e200 0x7f58fc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:47.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.761+0000 7f5915cfa700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f58fc06e200 0x7f58fc0706c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f590c009550 tx=0x7f590c0091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:47.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.765+0000 7f5906ffd700 1 -- 172.21.15.49:0/2028167195 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5900058750 con 0x7f59101027e0 2024-05-15T16:06:47.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.991+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5910197f80 con 0x7f58fc06e200 2024-05-15T16:06:47.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.992+0000 7f5906ffd700 1 -- 172.21.15.49:0/2028167195 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5910197f80 con 0x7f58fc06e200 2024-05-15T16:06:47.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f58fc06e200 msgr2=0x7f58fc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:47.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f58fc06e200 0x7f58fc0706c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f590c009550 tx=0x7f590c0091b0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 msgr2=0x7f591019cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:47.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 0x7f591019cf30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f590000bd70 tx=0x7f590000bda0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 shutdown_connections 2024-05-15T16:06:47.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f58fc06e200 0x7f58fc0706c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910101ec0 0x7f591019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 --2- 172.21.15.49:0/2028167195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59101027e0 0x7f591019cf30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 >> 172.21.15.49:0/2028167195 conn(0x7f5910076800 msgr2=0x7f59101031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 shutdown_connections 2024-05-15T16:06:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:47.994+0000 7f5917f5e700 1 -- 172.21.15.49:0/2028167195 wait complete. 2024-05-15T16:06:48.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.103+0000 7f489928c700 1 -- 172.21.15.49:0/18349238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894102790 msgr2=0x7f489410ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:48.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.103+0000 7f489928c700 1 --2- 172.21.15.49:0/18349238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894102790 0x7f489410ac80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4888009910 tx=0x7f4888009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:48.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.103+0000 7f489928c700 1 -- 172.21.15.49:0/18349238 shutdown_connections 2024-05-15T16:06:48.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.103+0000 7f489928c700 1 --2- 172.21.15.49:0/18349238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894102790 0x7f489410ac80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.103+0000 7f489928c700 1 --2- 172.21.15.49:0/18349238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4894101e70 0x7f4894102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.103+0000 7f489928c700 1 -- 172.21.15.49:0/18349238 >> 172.21.15.49:0/18349238 conn(0x7f48940fb700 msgr2=0x7f48940fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:48.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.104+0000 7f489928c700 1 -- 172.21.15.49:0/18349238 shutdown_connections 2024-05-15T16:06:48.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.104+0000 7f489928c700 1 -- 172.21.15.49:0/18349238 wait complete. 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.104+0000 7f489928c700 1 Processor -- start 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.104+0000 7f489928c700 1 -- start start 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f489928c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894101e70 0x7f48940ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f489928c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4894102790 0x7f4894100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f489928c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894100550 con 0x7f4894101e70 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f489928c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894100690 con 0x7f4894102790 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f4892ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894101e70 0x7f48940ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f4892ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894101e70 0x7f48940ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49710/0 (socket says 172.21.15.49:49710) 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f4892ffd700 1 -- 172.21.15.49:0/3439809453 learned_addr learned my addr 172.21.15.49:0/3439809453 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f48927fc700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4894102790 0x7f4894100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f4892ffd700 1 -- 172.21.15.49:0/3439809453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4894102790 msgr2=0x7f4894100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f4892ffd700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4894102790 0x7f4894100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f4892ffd700 1 -- 172.21.15.49:0/3439809453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48880095f0 con 0x7f4894101e70 2024-05-15T16:06:48.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f48927fc700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4894102790 0x7f4894100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:06:48.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f4892ffd700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894101e70 0x7f48940ffad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f488400d8d0 tx=0x7f488400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:48.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f487bfff700 1 -- 172.21.15.49:0/3439809453 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4884009940 con 0x7f4894101e70 2024-05-15T16:06:48.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f487bfff700 1 -- 172.21.15.49:0/3439809453 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4884010460 con 0x7f4894101e70 2024-05-15T16:06:48.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.105+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48941a2df0 con 0x7f4894101e70 2024-05-15T16:06:48.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.106+0000 7f487bfff700 1 -- 172.21.15.49:0/3439809453 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f488400f5f0 con 0x7f4894101e70 2024-05-15T16:06:48.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.106+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48941a3150 con 0x7f4894101e70 2024-05-15T16:06:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.106+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4894069ab0 con 0x7f4894101e70 2024-05-15T16:06:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.107+0000 7f487bfff700 1 -- 172.21.15.49:0/3439809453 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f48840105d0 con 0x7f4894101e70 2024-05-15T16:06:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.107+0000 7f487bfff700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f487c06e200 0x7f487c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.108+0000 7f487bfff700 1 -- 172.21.15.49:0/3439809453 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f488408e0b0 con 0x7f4894101e70 2024-05-15T16:06:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.108+0000 7f48927fc700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f487c06e200 0x7f487c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:48.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.108+0000 7f48927fc700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f487c06e200 0x7f487c0706c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4894101200 tx=0x7f488801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:48.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.112+0000 7f487bfff700 1 -- 172.21.15.49:0/3439809453 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4884057410 con 0x7f4894101e70 2024-05-15T16:06:48.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.328+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f48941055c0 con 0x7f487c06e200 2024-05-15T16:06:48.329 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:48 smithi049 ceph-mon[112142]: pgmap v292: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:48.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.339+0000 7f487bfff700 1 -- 172.21.15.49:0/3439809453 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f48880055c0 con 0x7f487c06e200 2024-05-15T16:06:48.340 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:06:48.340 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (10m) 3m ago 21m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:06:48.340 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (21m) 3m ago 21m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:06:48.340 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (20m) 3m ago 20m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:06:48.340 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (7m) 3m ago 21m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:06:48.340 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (7m) 3m ago 20m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:06:48.340 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (9m) 3m ago 21m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:06:48.340 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (3m) 3m ago 16m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (3m) 3m ago 15m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (3m) 3m ago 16m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (15m) 3m ago 15m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (12m) 3m ago 23m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (11m) 3m ago 20m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (8m) 3m ago 23m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (8m) 3m ago 20m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (10m) 3m ago 21m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (10m) 3m ago 20m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (6m) 3m ago 19m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (6m) 3m ago 19m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (5m) 3m ago 18m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (5m) 3m ago 18m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (4m) 3m ago 17m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:06:48.341 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (4m) 3m ago 17m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:06:48.342 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (10m) 3m ago 21m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:06:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.341+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f487c06e200 msgr2=0x7f487c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.341+0000 7f489928c700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f487c06e200 0x7f487c0706c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4894101200 tx=0x7f488801a040 comp rx=0 tx=0).stop 2024-05-15T16:06:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.341+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894101e70 msgr2=0x7f48940ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.341+0000 7f489928c700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894101e70 0x7f48940ffad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f488400d8d0 tx=0x7f488400dc90 comp rx=0 tx=0).stop 2024-05-15T16:06:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.341+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 shutdown_connections 2024-05-15T16:06:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.341+0000 7f489928c700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f487c06e200 0x7f487c0706c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.341+0000 7f489928c700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4894101e70 0x7f48940ffad0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.341+0000 7f489928c700 1 --2- 172.21.15.49:0/3439809453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4894102790 0x7f4894100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.341+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 >> 172.21.15.49:0/3439809453 conn(0x7f48940fb700 msgr2=0x7f4894103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.342+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 shutdown_connections 2024-05-15T16:06:48.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.342+0000 7f489928c700 1 -- 172.21.15.49:0/3439809453 wait complete. 2024-05-15T16:06:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:48 smithi144 ceph-mon[86635]: pgmap v292: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:48.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.450+0000 7fc6007bc700 1 -- 172.21.15.49:0/321442831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 msgr2=0x7fc5f81058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:48.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.450+0000 7fc6007bc700 1 --2- 172.21.15.49:0/321442831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 0x7fc5f81058e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009910 tx=0x7fc5e8009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.450+0000 7fc6007bc700 1 -- 172.21.15.49:0/321442831 shutdown_connections 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.450+0000 7fc6007bc700 1 --2- 172.21.15.49:0/321442831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 0x7fc5f81058e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.450+0000 7fc6007bc700 1 --2- 172.21.15.49:0/321442831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f8100ee0 0x7fc5f81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.450+0000 7fc6007bc700 1 -- 172.21.15.49:0/321442831 >> 172.21.15.49:0/321442831 conn(0x7fc5f80fc790 msgr2=0x7fc5f80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.450+0000 7fc6007bc700 1 -- 172.21.15.49:0/321442831 shutdown_connections 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.450+0000 7fc6007bc700 1 -- 172.21.15.49:0/321442831 wait complete. 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.450+0000 7fc6007bc700 1 Processor -- start 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc6007bc700 1 -- start start 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc6007bc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f8100ee0 0x7fc5f819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc6007bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 0x7fc5f819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc6007bc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f819d440 con 0x7fc5f8101890 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc6007bc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8196a20 con 0x7fc5f8100ee0 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc5fdd57700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 0x7fc5f819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:48.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc5fdd57700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 0x7fc5f819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49718/0 (socket says 172.21.15.49:49718) 2024-05-15T16:06:48.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc5fdd57700 1 -- 172.21.15.49:0/277955884 learned_addr learned my addr 172.21.15.49:0/277955884 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc5fdd57700 1 -- 172.21.15.49:0/277955884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f8100ee0 msgr2=0x7fc5f819c8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc5fdd57700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f8100ee0 0x7fc5f819c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc5fdd57700 1 -- 172.21.15.49:0/277955884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80095f0 con 0x7fc5f8101890 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc5fdd57700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 0x7fc5f819ce20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009fd0 tx=0x7fc5e800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.451+0000 7fc5ef7fe700 1 -- 172.21.15.49:0/277955884 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e801d070 con 0x7fc5f8101890 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.452+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f8196ca0 con 0x7fc5f8101890 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.452+0000 7fc5ef7fe700 1 -- 172.21.15.49:0/277955884 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5e80052a0 con 0x7fc5f8101890 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.452+0000 7fc5ef7fe700 1 -- 172.21.15.49:0/277955884 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e8022470 con 0x7fc5f8101890 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.452+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f8197190 con 0x7fc5f8101890 2024-05-15T16:06:48.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.453+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f81092e0 con 0x7fc5f8101890 2024-05-15T16:06:48.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.453+0000 7fc5ef7fe700 1 -- 172.21.15.49:0/277955884 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc5e800f770 con 0x7fc5f8101890 2024-05-15T16:06:48.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.454+0000 7fc5ef7fe700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc5e406e200 0x7fc5e40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:48.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.454+0000 7fc5ef7fe700 1 -- 172.21.15.49:0/277955884 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc5e808f450 con 0x7fc5f8101890 2024-05-15T16:06:48.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.454+0000 7fc5fe558700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc5e406e200 0x7fc5e40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:48.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.454+0000 7fc5fe558700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc5e406e200 0x7fc5e40706c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009af0 tx=0x7fc5f4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:48.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.458+0000 7fc5ef7fe700 1 -- 172.21.15.49:0/277955884 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc5e805c2e0 con 0x7fc5f8101890 2024-05-15T16:06:48.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.759+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc5f81023c0 con 0x7fc5f8101890 2024-05-15T16:06:48.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.760+0000 7fc5ef7fe700 1 -- 172.21.15.49:0/277955884 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fc5e805be70 con 0x7fc5f8101890 2024-05-15T16:06:48.760 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:06:48.760 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:06:48.761 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:06:48.762 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:06:48.762 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:06:48.762 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:06:48.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc5e406e200 msgr2=0x7fc5e40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc5e406e200 0x7fc5e40706c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009af0 tx=0x7fc5f4009260 comp rx=0 tx=0).stop 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 msgr2=0x7fc5f819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 0x7fc5f819ce20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009fd0 tx=0x7fc5e800b650 comp rx=0 tx=0).stop 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 shutdown_connections 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc5e406e200 0x7fc5e40706c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f8100ee0 0x7fc5f819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 --2- 172.21.15.49:0/277955884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f8101890 0x7fc5f819ce20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 >> 172.21.15.49:0/277955884 conn(0x7fc5f80fc790 msgr2=0x7fc5f80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 shutdown_connections 2024-05-15T16:06:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.762+0000 7fc6007bc700 1 -- 172.21.15.49:0/277955884 wait complete. 2024-05-15T16:06:48.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.871+0000 7f9fdb570700 1 -- 172.21.15.49:0/3636132182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 msgr2=0x7f9fd410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:48.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.871+0000 7f9fdb570700 1 --2- 172.21.15.49:0/3636132182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 0x7f9fd410d070 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009910 tx=0x7f9fc4009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:48.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.871+0000 7f9fdb570700 1 -- 172.21.15.49:0/3636132182 shutdown_connections 2024-05-15T16:06:48.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.871+0000 7f9fdb570700 1 --2- 172.21.15.49:0/3636132182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 0x7f9fd410d070 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.871+0000 7f9fdb570700 1 --2- 172.21.15.49:0/3636132182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd40ffa00 0x7f9fd40ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.871+0000 7f9fdb570700 1 -- 172.21.15.49:0/3636132182 >> 172.21.15.49:0/3636132182 conn(0x7f9fd40fb700 msgr2=0x7f9fd40fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:48.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.871+0000 7f9fdb570700 1 -- 172.21.15.49:0/3636132182 shutdown_connections 2024-05-15T16:06:48.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.871+0000 7f9fdb570700 1 -- 172.21.15.49:0/3636132182 wait complete. 2024-05-15T16:06:48.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fdb570700 1 Processor -- start 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fdb570700 1 -- start start 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fdb570700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd40ffa00 0x7f9fd4198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fdb570700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 0x7f9fd41993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fdb570700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4199a90 con 0x7f9fd41003b0 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fdb570700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd419d760 con 0x7f9fd40ffa00 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fd8b0b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 0x7f9fd41993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fd8b0b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 0x7f9fd41993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49732/0 (socket says 172.21.15.49:49732) 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fd8b0b700 1 -- 172.21.15.49:0/1935780221 learned_addr learned my addr 172.21.15.49:0/1935780221 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fd930c700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd40ffa00 0x7f9fd4198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fd8b0b700 1 -- 172.21.15.49:0/1935780221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd40ffa00 msgr2=0x7f9fd4198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fd8b0b700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd40ffa00 0x7f9fd4198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.872+0000 7f9fd8b0b700 1 -- 172.21.15.49:0/1935780221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40095f0 con 0x7f9fd41003b0 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.873+0000 7f9fd8b0b700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 0x7f9fd41993b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4006010 tx=0x7f9fc40055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:48.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.873+0000 7f9fca7fc700 1 -- 172.21.15.49:0/1935780221 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fc401d070 con 0x7f9fd41003b0 2024-05-15T16:06:48.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.873+0000 7f9fca7fc700 1 -- 172.21.15.49:0/1935780221 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9fc4003d50 con 0x7f9fd41003b0 2024-05-15T16:06:48.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.873+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd419d9e0 con 0x7f9fd41003b0 2024-05-15T16:06:48.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.873+0000 7f9fca7fc700 1 -- 172.21.15.49:0/1935780221 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fc4021410 con 0x7f9fd41003b0 2024-05-15T16:06:48.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.873+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd419df30 con 0x7f9fd41003b0 2024-05-15T16:06:48.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.874+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd410aef0 con 0x7f9fd41003b0 2024-05-15T16:06:48.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.874+0000 7f9fca7fc700 1 -- 172.21.15.49:0/1935780221 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9fc4003890 con 0x7f9fd41003b0 2024-05-15T16:06:48.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.875+0000 7f9fca7fc700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fc006e200 0x7f9fc00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:48.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.875+0000 7f9fca7fc700 1 -- 172.21.15.49:0/1935780221 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9fc408f2a0 con 0x7f9fd41003b0 2024-05-15T16:06:48.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.875+0000 7f9fd930c700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fc006e200 0x7f9fc00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:48.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.875+0000 7f9fd930c700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fc006e200 0x7f9fc00706c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0005950 tx=0x7f9fd000a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:48.880+0000 7f9fca7fc700 1 -- 172.21.15.49:0/1935780221 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9fc4058600 con 0x7f9fd41003b0 2024-05-15T16:06:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.131+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9fd419a180 con 0x7f9fd41003b0 2024-05-15T16:06:49.132 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:49 smithi049 ceph-mon[112142]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:49.132 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:49 smithi049 ceph-mon[112142]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:49.132 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:49 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/277955884' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:06:49.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.131+0000 7f9fca7fc700 1 -- 172.21.15.49:0/1935780221 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f9fc405bcc0 con 0x7f9fd41003b0 2024-05-15T16:06:49.132 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:06:49.132 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:06:49.133 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:06:49.134 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:06:49.135 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:06:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.133+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fc006e200 msgr2=0x7f9fc00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.133+0000 7f9fdb570700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fc006e200 0x7f9fc00706c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0005950 tx=0x7f9fd000a300 comp rx=0 tx=0).stop 2024-05-15T16:06:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.133+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 msgr2=0x7f9fd41993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.133+0000 7f9fdb570700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 0x7f9fd41993b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4006010 tx=0x7f9fc40055f0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.134+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 shutdown_connections 2024-05-15T16:06:49.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.134+0000 7f9fdb570700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fc006e200 0x7f9fc00706c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.134+0000 7f9fdb570700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd40ffa00 0x7f9fd4198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.134+0000 7f9fdb570700 1 --2- 172.21.15.49:0/1935780221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fd41003b0 0x7f9fd41993b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.134+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 >> 172.21.15.49:0/1935780221 conn(0x7f9fd40fb700 msgr2=0x7f9fd4108030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:49.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.134+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 shutdown_connections 2024-05-15T16:06:49.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.134+0000 7f9fdb570700 1 -- 172.21.15.49:0/1935780221 wait complete. 2024-05-15T16:06:49.137 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:06:49.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.243+0000 7fdd8a47f700 1 -- 172.21.15.49:0/1866478577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84100780 msgr2=0x7fdd841047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.243+0000 7fdd8a47f700 1 --2- 172.21.15.49:0/1866478577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84100780 0x7fdd841047d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009910 tx=0x7fdd6c009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.243+0000 7fdd8a47f700 1 -- 172.21.15.49:0/1866478577 shutdown_connections 2024-05-15T16:06:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.243+0000 7fdd8a47f700 1 --2- 172.21.15.49:0/1866478577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd84100780 0x7fdd841047d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.243+0000 7fdd8a47f700 1 --2- 172.21.15.49:0/1866478577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd840ffdd0 0x7fdd841001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.243+0000 7fdd8a47f700 1 -- 172.21.15.49:0/1866478577 >> 172.21.15.49:0/1866478577 conn(0x7fdd840fb6a0 msgr2=0x7fdd840fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.243+0000 7fdd8a47f700 1 -- 172.21.15.49:0/1866478577 shutdown_connections 2024-05-15T16:06:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.243+0000 7fdd8a47f700 1 -- 172.21.15.49:0/1866478577 wait complete. 2024-05-15T16:06:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.244+0000 7fdd8a47f700 1 Processor -- start 2024-05-15T16:06:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.244+0000 7fdd8a47f700 1 -- start start 2024-05-15T16:06:49.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.244+0000 7fdd8a47f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd840ffdd0 0x7fdd8419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:49.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.244+0000 7fdd8a47f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd84100780 0x7fdd8419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:49.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.244+0000 7fdd8a47f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8419d460 con 0x7fdd840ffdd0 2024-05-15T16:06:49.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.244+0000 7fdd8a47f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84196a40 con 0x7fdd84100780 2024-05-15T16:06:49.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.244+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd840ffdd0 0x7fdd8419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:49.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd83fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd840ffdd0 0x7fdd8419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49750/0 (socket says 172.21.15.49:49750) 2024-05-15T16:06:49.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd83fff700 1 -- 172.21.15.49:0/4221389405 learned_addr learned my addr 172.21.15.49:0/4221389405 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd837fe700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd84100780 0x7fdd8419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd83fff700 1 -- 172.21.15.49:0/4221389405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd84100780 msgr2=0x7fdd8419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd83fff700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd84100780 0x7fdd8419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd83fff700 1 -- 172.21.15.49:0/4221389405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0095f0 con 0x7fdd840ffdd0 2024-05-15T16:06:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd83fff700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd840ffdd0 0x7fdd8419c900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400c8a0 tx=0x7fdd7400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd817fa700 1 -- 172.21.15.49:0/4221389405 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd740078e0 con 0x7fdd840ffdd0 2024-05-15T16:06:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd817fa700 1 -- 172.21.15.49:0/4221389405 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd7400f450 con 0x7fdd840ffdd0 2024-05-15T16:06:49.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd84196d20 con 0x7fdd840ffdd0 2024-05-15T16:06:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd817fa700 1 -- 172.21.15.49:0/4221389405 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd7400e5e0 con 0x7fdd840ffdd0 2024-05-15T16:06:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.245+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd84197240 con 0x7fdd840ffdd0 2024-05-15T16:06:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.246+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd841012b0 con 0x7fdd840ffdd0 2024-05-15T16:06:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.247+0000 7fdd817fa700 1 -- 172.21.15.49:0/4221389405 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdd7400f5c0 con 0x7fdd840ffdd0 2024-05-15T16:06:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.247+0000 7fdd817fa700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdd7006e210 0x7fdd700706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.247+0000 7fdd817fa700 1 -- 172.21.15.49:0/4221389405 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdd7408e1b0 con 0x7fdd840ffdd0 2024-05-15T16:06:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.248+0000 7fdd837fe700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdd7006e210 0x7fdd700706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.248+0000 7fdd837fe700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdd7006e210 0x7fdd700706d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009fd0 tx=0x7fdd6c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:49.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.252+0000 7fdd817fa700 1 -- 172.21.15.49:0/4221389405 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdd7405afc0 con 0x7fdd840ffdd0 2024-05-15T16:06:49.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:49 smithi144 ceph-mon[86635]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:49.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:49 smithi144 ceph-mon[86635]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:49.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:49 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/277955884' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:06:49.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.478+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd8410cdc0 con 0x7fdd7006e210 2024-05-15T16:06:49.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.480+0000 7fdd817fa700 1 -- 172.21.15.49:0/4221389405 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdd8410cdc0 con 0x7fdd7006e210 2024-05-15T16:06:49.480 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:06:49.481 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:06:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.482+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdd7006e210 msgr2=0x7fdd700706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.482+0000 7fdd8a47f700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdd7006e210 0x7fdd700706d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009fd0 tx=0x7fdd6c01a040 comp rx=0 tx=0).stop 2024-05-15T16:06:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.482+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd840ffdd0 msgr2=0x7fdd8419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.482+0000 7fdd8a47f700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd840ffdd0 0x7fdd8419c900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400c8a0 tx=0x7fdd7400cbb0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.482+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 shutdown_connections 2024-05-15T16:06:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.482+0000 7fdd8a47f700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdd7006e210 0x7fdd700706d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.482+0000 7fdd8a47f700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd840ffdd0 0x7fdd8419c900 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.482+0000 7fdd8a47f700 1 --2- 172.21.15.49:0/4221389405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd84100780 0x7fdd8419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.482+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 >> 172.21.15.49:0/4221389405 conn(0x7fdd840fb6a0 msgr2=0x7fdd840692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.483+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 shutdown_connections 2024-05-15T16:06:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.483+0000 7fdd8a47f700 1 -- 172.21.15.49:0/4221389405 wait complete. 2024-05-15T16:06:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.591+0000 7fd9d037a700 1 -- 172.21.15.49:0/108807368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c8100770 msgr2=0x7fd9c81047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.591+0000 7fd9d037a700 1 --2- 172.21.15.49:0/108807368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c8100770 0x7fd9c81047c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009910 tx=0x7fd9b8009c20 comp rx=0 tx=0).stop 2024-05-15T16:06:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.591+0000 7fd9d037a700 1 -- 172.21.15.49:0/108807368 shutdown_connections 2024-05-15T16:06:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.591+0000 7fd9d037a700 1 --2- 172.21.15.49:0/108807368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c8100770 0x7fd9c81047c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.591+0000 7fd9d037a700 1 --2- 172.21.15.49:0/108807368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9c80ffdc0 0x7fd9c81001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.591+0000 7fd9d037a700 1 -- 172.21.15.49:0/108807368 >> 172.21.15.49:0/108807368 conn(0x7fd9c8076800 msgr2=0x7fd9c80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.591+0000 7fd9d037a700 1 -- 172.21.15.49:0/108807368 shutdown_connections 2024-05-15T16:06:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.591+0000 7fd9d037a700 1 -- 172.21.15.49:0/108807368 wait complete. 2024-05-15T16:06:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9d037a700 1 Processor -- start 2024-05-15T16:06:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9d037a700 1 -- start start 2024-05-15T16:06:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9d037a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80ffdc0 0x7fd9c8197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9d037a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9c8100770 0x7fd9c8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9d037a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8198510 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9d037a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8198650 con 0x7fd9c8100770 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9ce116700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80ffdc0 0x7fd9c8197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9ce116700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80ffdc0 0x7fd9c8197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49780/0 (socket says 172.21.15.49:49780) 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9ce116700 1 -- 172.21.15.49:0/1715367695 learned_addr learned my addr 172.21.15.49:0/1715367695 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.592+0000 7fd9cd915700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9c8100770 0x7fd9c8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.593+0000 7fd9ce116700 1 -- 172.21.15.49:0/1715367695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9c8100770 msgr2=0x7fd9c8197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.593+0000 7fd9ce116700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9c8100770 0x7fd9c8197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.593+0000 7fd9ce116700 1 -- 172.21.15.49:0/1715367695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b80095f0 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.593+0000 7fd9ce116700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80ffdc0 0x7fd9c8197a00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400d8d0 tx=0x7fd9c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.593+0000 7fd9bf7fe700 1 -- 172.21.15.49:0/1715367695 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9c4009940 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.593+0000 7fd9bf7fe700 1 -- 172.21.15.49:0/1715367695 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9c4010460 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.593+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c81a0a70 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.593+0000 7fd9bf7fe700 1 -- 172.21.15.49:0/1715367695 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9c400f5f0 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.593+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c81a0e80 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.594+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c81081c0 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.595+0000 7fd9bf7fe700 1 -- 172.21.15.49:0/1715367695 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd9c40105d0 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.595+0000 7fd9bf7fe700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd9b406e210 0x7fd9b40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:06:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.595+0000 7fd9bf7fe700 1 -- 172.21.15.49:0/1715367695 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd9c408e090 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.595+0000 7fd9cd915700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd9b406e210 0x7fd9b40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:06:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.596+0000 7fd9cd915700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd9b406e210 0x7fd9b40706d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8068e70 tx=0x7fd9b801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:06:49.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.600+0000 7fd9bf7fe700 1 -- 172.21.15.49:0/1715367695 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd9c40573f0 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.893+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd9c8068c50 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.893+0000 7fd9bf7fe700 1 -- 172.21.15.49:0/1715367695 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd9c40161c0 con 0x7fd9c80ffdc0 2024-05-15T16:06:49.894 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:06:49.894 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:06:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:06:49.894 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:06:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:06:49.895 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:06:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:06:49.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.895+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd9b406e210 msgr2=0x7fd9b40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.895+0000 7fd9d037a700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd9b406e210 0x7fd9b40706d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8068e70 tx=0x7fd9b801a040 comp rx=0 tx=0).stop 2024-05-15T16:06:49.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.895+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80ffdc0 msgr2=0x7fd9c8197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:06:49.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.895+0000 7fd9d037a700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80ffdc0 0x7fd9c8197a00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400d8d0 tx=0x7fd9c400dc90 comp rx=0 tx=0).stop 2024-05-15T16:06:49.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.896+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 shutdown_connections 2024-05-15T16:06:49.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.896+0000 7fd9d037a700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd9b406e210 0x7fd9b40706d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.896+0000 7fd9d037a700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80ffdc0 0x7fd9c8197a00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.896+0000 7fd9d037a700 1 --2- 172.21.15.49:0/1715367695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9c8100770 0x7fd9c8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:06:49.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.896+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 >> 172.21.15.49:0/1715367695 conn(0x7fd9c8076800 msgr2=0x7fd9c8103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:06:49.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.896+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 shutdown_connections 2024-05-15T16:06:49.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:06:49.896+0000 7fd9d037a700 1 -- 172.21.15.49:0/1715367695 wait complete. 2024-05-15T16:06:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:50 smithi049 ceph-mon[112142]: pgmap v293: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:50 smithi049 ceph-mon[112142]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:50 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1935780221' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:06:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:50 smithi049 ceph-mon[112142]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:50 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1715367695' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:06:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:50 smithi144 ceph-mon[86635]: pgmap v293: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:50 smithi144 ceph-mon[86635]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:50 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1935780221' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:06:50.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:50 smithi144 ceph-mon[86635]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:06:50.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:50 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1715367695' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:06:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:06:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:51 smithi144 ceph-mon[86635]: pgmap v294: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:06:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:51 smithi049 ceph-mon[112142]: pgmap v294: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:53.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:53 smithi144 ceph-mon[86635]: pgmap v295: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:53.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:53 smithi049 ceph-mon[112142]: pgmap v295: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:55.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:55 smithi049 ceph-mon[112142]: pgmap v296: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:55.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:55 smithi144 ceph-mon[86635]: pgmap v296: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:57.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:57 smithi144 ceph-mon[86635]: pgmap v297: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:57.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:57 smithi049 ceph-mon[112142]: pgmap v297: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:59.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:06:59 smithi144 ceph-mon[86635]: pgmap v298: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:06:59.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:06:59 smithi049 ceph-mon[112142]: pgmap v298: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:01.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:01 smithi049 ceph-mon[112142]: pgmap v299: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:01.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:01 smithi144 ceph-mon[86635]: pgmap v299: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:03.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:03 smithi144 ceph-mon[86635]: pgmap v300: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:03.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:03 smithi049 ceph-mon[112142]: pgmap v300: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:05.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:05 smithi144 ceph-mon[86635]: pgmap v301: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:05.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:07:05.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:05 smithi049 ceph-mon[112142]: pgmap v301: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:05.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:07:07.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:07 smithi144 ceph-mon[86635]: pgmap v302: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:07.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:07 smithi049 ceph-mon[112142]: pgmap v302: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:09.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:09 smithi144 ceph-mon[86635]: pgmap v303: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:09.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:09 smithi049 ceph-mon[112142]: pgmap v303: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:11.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:11 smithi049 ceph-mon[112142]: pgmap v304: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:11.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:11 smithi144 ceph-mon[86635]: pgmap v304: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:13.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:13 smithi144 ceph-mon[86635]: pgmap v305: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:13.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:13 smithi049 ceph-mon[112142]: pgmap v305: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:15.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:15 smithi144 ceph-mon[86635]: pgmap v306: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:15.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:15 smithi049 ceph-mon[112142]: pgmap v306: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:17.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:17 smithi144 ceph-mon[86635]: pgmap v307: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:17.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:17 smithi049 ceph-mon[112142]: pgmap v307: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:19.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:19 smithi144 ceph-mon[86635]: pgmap v308: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:19.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:19 smithi049 ceph-mon[112142]: pgmap v308: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:20.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.018+0000 7ff459c76700 1 -- 172.21.15.49:0/2445945877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 msgr2=0x7ff4541012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.018+0000 7ff459c76700 1 --2- 172.21.15.49:0/2445945877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 0x7ff4541012e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff444009910 tx=0x7ff444009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:20.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.018+0000 7ff459c76700 1 -- 172.21.15.49:0/2445945877 shutdown_connections 2024-05-15T16:07:20.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.018+0000 7ff459c76700 1 --2- 172.21.15.49:0/2445945877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4541018b0 0x7ff454105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.018+0000 7ff459c76700 1 --2- 172.21.15.49:0/2445945877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 0x7ff4541012e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.018+0000 7ff459c76700 1 -- 172.21.15.49:0/2445945877 >> 172.21.15.49:0/2445945877 conn(0x7ff4540fc7d0 msgr2=0x7ff4540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:20.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.018+0000 7ff459c76700 1 -- 172.21.15.49:0/2445945877 shutdown_connections 2024-05-15T16:07:20.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.018+0000 7ff459c76700 1 -- 172.21.15.49:0/2445945877 wait complete. 2024-05-15T16:07:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.019+0000 7ff459c76700 1 Processor -- start 2024-05-15T16:07:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.019+0000 7ff459c76700 1 -- start start 2024-05-15T16:07:20.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.019+0000 7ff459c76700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 0x7ff45419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.019+0000 7ff459c76700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4541018b0 0x7ff45419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.019+0000 7ff459c76700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45419d3d0 con 0x7ff454100f00 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.019+0000 7ff459c76700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454196a90 con 0x7ff4541018b0 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4537fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 0x7ff45419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4537fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 0x7ff45419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47922/0 (socket says 172.21.15.49:47922) 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4537fe700 1 -- 172.21.15.49:0/2422615571 learned_addr learned my addr 172.21.15.49:0/2422615571 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff44affd700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4541018b0 0x7ff45419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4537fe700 1 -- 172.21.15.49:0/2422615571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4541018b0 msgr2=0x7ff45419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4537fe700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4541018b0 0x7ff45419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4537fe700 1 -- 172.21.15.49:0/2422615571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440095f0 con 0x7ff454100f00 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4537fe700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 0x7ff45419c900 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff444006010 tx=0x7ff44400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4517fa700 1 -- 172.21.15.49:0/2422615571 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff44401d070 con 0x7ff454100f00 2024-05-15T16:07:20.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4517fa700 1 -- 172.21.15.49:0/2422615571 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff44400f460 con 0x7ff454100f00 2024-05-15T16:07:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff454196d10 con 0x7ff454100f00 2024-05-15T16:07:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff4517fa700 1 -- 172.21.15.49:0/2422615571 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff444017630 con 0x7ff454100f00 2024-05-15T16:07:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.020+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff454197200 con 0x7ff454100f00 2024-05-15T16:07:20.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.021+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4541092a0 con 0x7ff454100f00 2024-05-15T16:07:20.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.022+0000 7ff4517fa700 1 -- 172.21.15.49:0/2422615571 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff44400f5d0 con 0x7ff454100f00 2024-05-15T16:07:20.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.022+0000 7ff4517fa700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44006e200 0x7ff4400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:20.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.023+0000 7ff4517fa700 1 -- 172.21.15.49:0/2422615571 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff44408f620 con 0x7ff454100f00 2024-05-15T16:07:20.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.023+0000 7ff44affd700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44006e200 0x7ff4400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:20.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.023+0000 7ff44affd700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44006e200 0x7ff4400706c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff43c005ea0 tx=0x7ff43c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:20.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.027+0000 7ff4517fa700 1 -- 172.21.15.49:0/2422615571 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff44405c570 con 0x7ff454100f00 2024-05-15T16:07:20.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.261+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff454197f00 con 0x7ff44006e200 2024-05-15T16:07:20.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.263+0000 7ff4517fa700 1 -- 172.21.15.49:0/2422615571 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff454197f00 con 0x7ff44006e200 2024-05-15T16:07:20.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44006e200 msgr2=0x7ff4400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44006e200 0x7ff4400706c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff43c005ea0 tx=0x7ff43c005e30 comp rx=0 tx=0).stop 2024-05-15T16:07:20.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 msgr2=0x7ff45419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 0x7ff45419c900 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff444006010 tx=0x7ff44400be20 comp rx=0 tx=0).stop 2024-05-15T16:07:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 shutdown_connections 2024-05-15T16:07:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44006e200 0x7ff4400706c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff454100f00 0x7ff45419c900 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 --2- 172.21.15.49:0/2422615571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4541018b0 0x7ff45419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 >> 172.21.15.49:0/2422615571 conn(0x7ff4540fc7d0 msgr2=0x7ff4540fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:20.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 shutdown_connections 2024-05-15T16:07:20.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.265+0000 7ff459c76700 1 -- 172.21.15.49:0/2422615571 wait complete. 2024-05-15T16:07:20.278 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:07:20.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.376+0000 7f3a89af4700 1 -- 172.21.15.49:0/2524518848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 msgr2=0x7f3a8410ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.376+0000 7f3a89af4700 1 --2- 172.21.15.49:0/2524518848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 0x7f3a8410ac60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009910 tx=0x7f3a78009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:20.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.376+0000 7f3a89af4700 1 -- 172.21.15.49:0/2524518848 shutdown_connections 2024-05-15T16:07:20.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.376+0000 7f3a89af4700 1 --2- 172.21.15.49:0/2524518848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 0x7f3a8410ac60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.376+0000 7f3a89af4700 1 --2- 172.21.15.49:0/2524518848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a84101e50 0x7f3a84102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.376+0000 7f3a89af4700 1 -- 172.21.15.49:0/2524518848 >> 172.21.15.49:0/2524518848 conn(0x7f3a840fb6c0 msgr2=0x7f3a840fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:20.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.376+0000 7f3a89af4700 1 -- 172.21.15.49:0/2524518848 shutdown_connections 2024-05-15T16:07:20.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.377+0000 7f3a89af4700 1 -- 172.21.15.49:0/2524518848 wait complete. 2024-05-15T16:07:20.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.377+0000 7f3a89af4700 1 Processor -- start 2024-05-15T16:07:20.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a89af4700 1 -- start start 2024-05-15T16:07:20.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a89af4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a84101e50 0x7f3a840ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:20.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a89af4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 0x7f3a840ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:20.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a89af4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84100530 con 0x7f3a84102770 2024-05-15T16:07:20.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a89af4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84100670 con 0x7f3a84101e50 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a82ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 0x7f3a840ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a82ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 0x7f3a840ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47942/0 (socket says 172.21.15.49:47942) 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a82ffd700 1 -- 172.21.15.49:0/2617386519 learned_addr learned my addr 172.21.15.49:0/2617386519 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a837fe700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a84101e50 0x7f3a840ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a82ffd700 1 -- 172.21.15.49:0/2617386519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a84101e50 msgr2=0x7f3a840ffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a82ffd700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a84101e50 0x7f3a840ffab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a82ffd700 1 -- 172.21.15.49:0/2617386519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780095f0 con 0x7f3a84102770 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.378+0000 7f3a82ffd700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 0x7f3a840ffff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a78005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.379+0000 7f3a80ff9700 1 -- 172.21.15.49:0/2617386519 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a7801d070 con 0x7f3a84102770 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.379+0000 7f3a80ff9700 1 -- 172.21.15.49:0/2617386519 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a780052a0 con 0x7f3a84102770 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.379+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a841008f0 con 0x7f3a84102770 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.379+0000 7f3a80ff9700 1 -- 172.21.15.49:0/2617386519 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a7800fe30 con 0x7f3a84102770 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.379+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a841a3070 con 0x7f3a84102770 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.380+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a84069af0 con 0x7f3a84102770 2024-05-15T16:07:20.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.380+0000 7f3a80ff9700 1 -- 172.21.15.49:0/2617386519 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3a7800f670 con 0x7f3a84102770 2024-05-15T16:07:20.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.381+0000 7f3a80ff9700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a6c06e210 0x7f3a6c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:20.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.381+0000 7f3a80ff9700 1 -- 172.21.15.49:0/2617386519 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3a7808f2a0 con 0x7f3a84102770 2024-05-15T16:07:20.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.381+0000 7f3a837fe700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a6c06e210 0x7f3a6c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:20.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.381+0000 7f3a837fe700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a6c06e210 0x7f3a6c0706d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009c60 tx=0x7f3a74009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:20.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.386+0000 7f3a80ff9700 1 -- 172.21.15.49:0/2617386519 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3a78058600 con 0x7f3a84102770 2024-05-15T16:07:20.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.613+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a8410deb0 con 0x7f3a6c06e210 2024-05-15T16:07:20.626 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:07:20.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.614+0000 7f3a80ff9700 1 -- 172.21.15.49:0/2617386519 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3a8410deb0 con 0x7f3a6c06e210 2024-05-15T16:07:20.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.616+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a6c06e210 msgr2=0x7f3a6c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.616+0000 7f3a89af4700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a6c06e210 0x7f3a6c0706d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009c60 tx=0x7f3a74009260 comp rx=0 tx=0).stop 2024-05-15T16:07:20.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.616+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 msgr2=0x7f3a840ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.616+0000 7f3a89af4700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 0x7f3a840ffff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a78005ee0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.616+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 shutdown_connections 2024-05-15T16:07:20.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.616+0000 7f3a89af4700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3a6c06e210 0x7f3a6c0706d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.616+0000 7f3a89af4700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a84101e50 0x7f3a840ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.616+0000 7f3a89af4700 1 --2- 172.21.15.49:0/2617386519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a84102770 0x7f3a840ffff0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.616+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 >> 172.21.15.49:0/2617386519 conn(0x7f3a840fb6c0 msgr2=0x7f3a84103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:20.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.617+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 shutdown_connections 2024-05-15T16:07:20.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.617+0000 7f3a89af4700 1 -- 172.21.15.49:0/2617386519 wait complete. 2024-05-15T16:07:20.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:07:20.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 -- 172.21.15.49:0/6399506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 msgr2=0x7fcb341012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 --2- 172.21.15.49:0/6399506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 0x7fcb341012e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009910 tx=0x7fcb24009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:20.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 -- 172.21.15.49:0/6399506 shutdown_connections 2024-05-15T16:07:20.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 --2- 172.21.15.49:0/6399506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb341018b0 0x7fcb34105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 --2- 172.21.15.49:0/6399506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 0x7fcb341012e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 -- 172.21.15.49:0/6399506 >> 172.21.15.49:0/6399506 conn(0x7fcb340fc7d0 msgr2=0x7fcb340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:20.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 -- 172.21.15.49:0/6399506 shutdown_connections 2024-05-15T16:07:20.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 -- 172.21.15.49:0/6399506 wait complete. 2024-05-15T16:07:20.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 Processor -- start 2024-05-15T16:07:20.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.726+0000 7fcb3a881700 1 -- start start 2024-05-15T16:07:20.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb3a881700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 0x7fcb340732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:20.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb3a881700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb341018b0 0x7fcb340737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:20.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb3a881700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34073d20 con 0x7fcb34100f00 2024-05-15T16:07:20.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb3a881700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34073e60 con 0x7fcb341018b0 2024-05-15T16:07:20.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 0x7fcb340732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:20.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 0x7fcb340732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47952/0 (socket says 172.21.15.49:47952) 2024-05-15T16:07:20.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb33fff700 1 -- 172.21.15.49:0/2608214543 learned_addr learned my addr 172.21.15.49:0/2608214543 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:20.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb2bfff700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb341018b0 0x7fcb340737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb33fff700 1 -- 172.21.15.49:0/2608214543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb341018b0 msgr2=0x7fcb340737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb33fff700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb341018b0 0x7fcb340737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb33fff700 1 -- 172.21.15.49:0/2608214543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb240095f0 con 0x7fcb34100f00 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb2bfff700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb341018b0 0x7fcb340737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb33fff700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 0x7fcb340732a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400bd70 tx=0x7fcb2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb31ffb700 1 -- 172.21.15.49:0/2608214543 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb2401d070 con 0x7fcb34100f00 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb31ffb700 1 -- 172.21.15.49:0/2608214543 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb240056e0 con 0x7fcb34100f00 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.727+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb340740e0 con 0x7fcb34100f00 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.728+0000 7fcb31ffb700 1 -- 172.21.15.49:0/2608214543 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb24017400 con 0x7fcb34100f00 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.728+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb341a3040 con 0x7fcb34100f00 2024-05-15T16:07:20.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.729+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb341092a0 con 0x7fcb34100f00 2024-05-15T16:07:20.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.729+0000 7fcb31ffb700 1 -- 172.21.15.49:0/2608214543 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcb24003a60 con 0x7fcb34100f00 2024-05-15T16:07:20.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.729+0000 7fcb31ffb700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcb2006e200 0x7fcb200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:20.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.730+0000 7fcb31ffb700 1 -- 172.21.15.49:0/2608214543 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcb2408f550 con 0x7fcb34100f00 2024-05-15T16:07:20.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.730+0000 7fcb2bfff700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcb2006e200 0x7fcb200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:20.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.730+0000 7fcb2bfff700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcb2006e200 0x7fcb200706c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c005950 tx=0x7fcb1c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:20.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.734+0000 7fcb31ffb700 1 -- 172.21.15.49:0/2608214543 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcb240588b0 con 0x7fcb34100f00 2024-05-15T16:07:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.950+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb3410ded0 con 0x7fcb2006e200 2024-05-15T16:07:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.961+0000 7fcb31ffb700 1 -- 172.21.15.49:0/2608214543 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcb3410ded0 con 0x7fcb2006e200 2024-05-15T16:07:20.963 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (10m) 3m ago 22m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (22m) 3m ago 22m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (21m) 3m ago 21m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (7m) 3m ago 22m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (7m) 3m ago 21m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (10m) 3m ago 21m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (4m) 3m ago 16m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (3m) 3m ago 16m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (3m) 3m ago 16m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (16m) 3m ago 16m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (12m) 3m ago 23m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (11m) 3m ago 21m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (9m) 3m ago 23m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:07:20.964 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (8m) 3m ago 20m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:07:20.965 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (11m) 3m ago 22m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:07:20.965 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (11m) 3m ago 21m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:07:20.965 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (7m) 3m ago 20m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:07:20.965 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (6m) 3m ago 19m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:07:20.965 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (6m) 3m ago 19m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:07:20.965 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (5m) 3m ago 18m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:07:20.965 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (5m) 3m ago 18m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:07:20.965 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (4m) 3m ago 18m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:07:20.965 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (10m) 3m ago 21m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcb2006e200 msgr2=0x7fcb200706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcb2006e200 0x7fcb200706c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c005950 tx=0x7fcb1c0058e0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 msgr2=0x7fcb340732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 0x7fcb340732a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400bd70 tx=0x7fcb2400be50 comp rx=0 tx=0).stop 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 shutdown_connections 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcb2006e200 0x7fcb200706c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcb34100f00 0x7fcb340732a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 --2- 172.21.15.49:0/2608214543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb341018b0 0x7fcb340737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 >> 172.21.15.49:0/2608214543 conn(0x7fcb340fc7d0 msgr2=0x7fcb340fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 shutdown_connections 2024-05-15T16:07:20.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:20.963+0000 7fcb3a881700 1 -- 172.21.15.49:0/2608214543 wait complete. 2024-05-15T16:07:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.071+0000 7fb49e608700 1 -- 172.21.15.49:0/3226705216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498100ee0 msgr2=0x7fb4981012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.071+0000 7fb49e608700 1 --2- 172.21.15.49:0/3226705216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498100ee0 0x7fb4981012c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb488009910 tx=0x7fb488009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.071+0000 7fb49e608700 1 -- 172.21.15.49:0/3226705216 shutdown_connections 2024-05-15T16:07:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.071+0000 7fb49e608700 1 --2- 172.21.15.49:0/3226705216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb498101890 0x7fb4981058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.071+0000 7fb49e608700 1 --2- 172.21.15.49:0/3226705216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498100ee0 0x7fb4981012c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.071+0000 7fb49e608700 1 -- 172.21.15.49:0/3226705216 >> 172.21.15.49:0/3226705216 conn(0x7fb4980fc790 msgr2=0x7fb4980febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.071+0000 7fb49e608700 1 -- 172.21.15.49:0/3226705216 shutdown_connections 2024-05-15T16:07:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.071+0000 7fb49e608700 1 -- 172.21.15.49:0/3226705216 wait complete. 2024-05-15T16:07:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb49e608700 1 Processor -- start 2024-05-15T16:07:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb49e608700 1 -- start start 2024-05-15T16:07:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb49e608700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb498100ee0 0x7fb4980732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb49e608700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498101890 0x7fb4980737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb49e608700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb498073d20 con 0x7fb498101890 2024-05-15T16:07:21.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb49e608700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb498073e60 con 0x7fb498100ee0 2024-05-15T16:07:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb4977fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498101890 0x7fb4980737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb4977fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498101890 0x7fb4980737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47976/0 (socket says 172.21.15.49:47976) 2024-05-15T16:07:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb4977fe700 1 -- 172.21.15.49:0/208860655 learned_addr learned my addr 172.21.15.49:0/208860655 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.072+0000 7fb497fff700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb498100ee0 0x7fb4980732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb4977fe700 1 -- 172.21.15.49:0/208860655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb498100ee0 msgr2=0x7fb4980732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb4977fe700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb498100ee0 0x7fb4980732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb4977fe700 1 -- 172.21.15.49:0/208860655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4880095f0 con 0x7fb498101890 2024-05-15T16:07:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb497fff700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb498100ee0 0x7fb4980732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:07:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb4977fe700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498101890 0x7fb4980737e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb48000e8f0 tx=0x7fb48000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb4957fa700 1 -- 172.21.15.49:0/208860655 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb480009790 con 0x7fb498101890 2024-05-15T16:07:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4981a2e10 con 0x7fb498101890 2024-05-15T16:07:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb4957fa700 1 -- 172.21.15.49:0/208860655 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb480004cf0 con 0x7fb498101890 2024-05-15T16:07:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4981a3170 con 0x7fb498101890 2024-05-15T16:07:21.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.073+0000 7fb4957fa700 1 -- 172.21.15.49:0/208860655 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4800056b0 con 0x7fb498101890 2024-05-15T16:07:21.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.074+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb498109280 con 0x7fb498101890 2024-05-15T16:07:21.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.075+0000 7fb4957fa700 1 -- 172.21.15.49:0/208860655 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb480004750 con 0x7fb498101890 2024-05-15T16:07:21.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.075+0000 7fb4957fa700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb48406e200 0x7fb4840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.075+0000 7fb4957fa700 1 -- 172.21.15.49:0/208860655 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb480014070 con 0x7fb498101890 2024-05-15T16:07:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.075+0000 7fb497fff700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb48406e200 0x7fb4840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:21.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.076+0000 7fb497fff700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb48406e200 0x7fb4840706c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb488000c00 tx=0x7fb488009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:21.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.080+0000 7fb4957fa700 1 -- 172.21.15.49:0/208860655 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb480057960 con 0x7fb498101890 2024-05-15T16:07:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.374+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb498074730 con 0x7fb498101890 2024-05-15T16:07:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.375+0000 7fb4957fa700 1 -- 172.21.15.49:0/208860655 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fb48005b020 con 0x7fb498101890 2024-05-15T16:07:21.377 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:07:21.377 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:07:21.377 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:07:21.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:07:21.377 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:07:21.377 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:07:21.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:07:21.377 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:07:21.378 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:07:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb48406e200 msgr2=0x7fb4840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb48406e200 0x7fb4840706c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb488000c00 tx=0x7fb488009f70 comp rx=0 tx=0).stop 2024-05-15T16:07:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498101890 msgr2=0x7fb4980737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498101890 0x7fb4980737e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb48000e8f0 tx=0x7fb48000ecb0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 shutdown_connections 2024-05-15T16:07:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb48406e200 0x7fb4840706c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb498100ee0 0x7fb4980732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 --2- 172.21.15.49:0/208860655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb498101890 0x7fb4980737e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 >> 172.21.15.49:0/208860655 conn(0x7fb4980fc790 msgr2=0x7fb4980fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:21.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 shutdown_connections 2024-05-15T16:07:21.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.377+0000 7fb49e608700 1 -- 172.21.15.49:0/208860655 wait complete. 2024-05-15T16:07:21.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.485+0000 7f51eb594700 1 -- 172.21.15.49:0/3602953779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e4101e80 msgr2=0x7f51e4102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.485+0000 7f51eb594700 1 --2- 172.21.15.49:0/3602953779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e4101e80 0x7f51e4102260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009940 tx=0x7f51e0009c50 comp rx=0 tx=0).stop 2024-05-15T16:07:21.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.485+0000 7f51eb594700 1 -- 172.21.15.49:0/3602953779 shutdown_connections 2024-05-15T16:07:21.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.485+0000 7f51eb594700 1 --2- 172.21.15.49:0/3602953779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51e41027a0 0x7f51e410ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.485+0000 7f51eb594700 1 --2- 172.21.15.49:0/3602953779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e4101e80 0x7f51e4102260 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.485+0000 7f51eb594700 1 -- 172.21.15.49:0/3602953779 >> 172.21.15.49:0/3602953779 conn(0x7f51e4076800 msgr2=0x7f51e40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:21.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.485+0000 7f51eb594700 1 -- 172.21.15.49:0/3602953779 shutdown_connections 2024-05-15T16:07:21.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.485+0000 7f51eb594700 1 -- 172.21.15.49:0/3602953779 wait complete. 2024-05-15T16:07:21.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.486+0000 7f51eb594700 1 Processor -- start 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.486+0000 7f51eb594700 1 -- start start 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.486+0000 7f51eb594700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51e4101e80 0x7f51e419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.486+0000 7f51eb594700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e41027a0 0x7f51e419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.486+0000 7f51eb594700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e419d420 con 0x7f51e41027a0 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.486+0000 7f51eb594700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e4196a00 con 0x7f51e4101e80 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.486+0000 7f51e8b2f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e41027a0 0x7f51e419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51e8b2f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e41027a0 0x7f51e419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47992/0 (socket says 172.21.15.49:47992) 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51e8b2f700 1 -- 172.21.15.49:0/4274061466 learned_addr learned my addr 172.21.15.49:0/4274061466 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51e9330700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51e4101e80 0x7f51e419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51e8b2f700 1 -- 172.21.15.49:0/4274061466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51e4101e80 msgr2=0x7f51e419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51e8b2f700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51e4101e80 0x7f51e419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51e8b2f700 1 -- 172.21.15.49:0/4274061466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00095f0 con 0x7f51e41027a0 2024-05-15T16:07:21.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51e8b2f700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e41027a0 0x7f51e419ce00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f51d400d900 tx=0x7f51d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51da7fc700 1 -- 172.21.15.49:0/4274061466 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51d4009900 con 0x7f51e41027a0 2024-05-15T16:07:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51da7fc700 1 -- 172.21.15.49:0/4274061466 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51d4010460 con 0x7f51e41027a0 2024-05-15T16:07:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51da7fc700 1 -- 172.21.15.49:0/4274061466 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51d400f5f0 con 0x7f51e41027a0 2024-05-15T16:07:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e4196ce0 con 0x7f51e41027a0 2024-05-15T16:07:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.487+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e41971e0 con 0x7f51e41027a0 2024-05-15T16:07:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.488+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e41055a0 con 0x7f51e41027a0 2024-05-15T16:07:21.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.489+0000 7f51da7fc700 1 -- 172.21.15.49:0/4274061466 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f51d40105d0 con 0x7f51e41027a0 2024-05-15T16:07:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.489+0000 7f51da7fc700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f51d006e200 0x7f51d00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.489+0000 7f51da7fc700 1 -- 172.21.15.49:0/4274061466 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f51d408dd60 con 0x7f51e41027a0 2024-05-15T16:07:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.489+0000 7f51e9330700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f51d006e200 0x7f51d00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:21.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.490+0000 7f51e9330700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f51d006e200 0x7f51d00706c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51e0005b40 tx=0x7f51e0005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.494+0000 7f51da7fc700 1 -- 172.21.15.49:0/4274061466 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f51d405ab70 con 0x7f51e41027a0 2024-05-15T16:07:21.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:21 smithi144 ceph-mon[86635]: pgmap v309: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:21.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:21 smithi144 ceph-mon[86635]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:21.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:21 smithi144 ceph-mon[86635]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:21.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:21 smithi144 ceph-mon[86635]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:21.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:21 smithi049 ceph-mon[112142]: pgmap v309: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:21.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:21 smithi049 ceph-mon[112142]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:21.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:21 smithi049 ceph-mon[112142]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:21.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:21 smithi049 ceph-mon[112142]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:21.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.748+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f51e4197ec0 con 0x7f51e41027a0 2024-05-15T16:07:21.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.748+0000 7f51da7fc700 1 -- 172.21.15.49:0/4274061466 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f51d4017760 con 0x7f51e41027a0 2024-05-15T16:07:21.751 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:07:21.751 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:07:21.751 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:07:21.752 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:07:21.752 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:07:21.752 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:07:21.752 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:07:21.753 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:07:21.753 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:07:21.753 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:07:21.753 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:07:21.753 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:07:21.753 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:07:21.753 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:07:21.753 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:07:21.754 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:07:21.754 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:07:21.754 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:07:21.755 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:07:21.755 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:07:21.756 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:07:21.756 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:07:21.756 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:07:21.756 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:07:21.756 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:07:21.756 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:07:21.756 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:07:21.757 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:07:21.757 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:07:21.757 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:07:21.757 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:07:21.757 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:07:21.757 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:07:21.757 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:07:21.757 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:07:21.758 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:07:21.758 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:07:21.758 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:07:21.758 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:07:21.759 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:07:21.759 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:07:21.760 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:07:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f51d006e200 msgr2=0x7f51d00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f51d006e200 0x7f51d00706c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51e0005b40 tx=0x7f51e0005a90 comp rx=0 tx=0).stop 2024-05-15T16:07:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e41027a0 msgr2=0x7f51e419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e41027a0 0x7f51e419ce00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f51d400d900 tx=0x7f51d400dc10 comp rx=0 tx=0).stop 2024-05-15T16:07:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 shutdown_connections 2024-05-15T16:07:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f51d006e200 0x7f51d00706c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51e4101e80 0x7f51e419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 --2- 172.21.15.49:0/4274061466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51e41027a0 0x7f51e419ce00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 >> 172.21.15.49:0/4274061466 conn(0x7f51e4076800 msgr2=0x7f51e4103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:21.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 shutdown_connections 2024-05-15T16:07:21.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.751+0000 7f51eb594700 1 -- 172.21.15.49:0/4274061466 wait complete. 2024-05-15T16:07:21.762 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:07:21.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.860+0000 7fa2b537c700 1 -- 172.21.15.49:0/2731352837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 msgr2=0x7fa2b0102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.860+0000 7fa2b537c700 1 --2- 172.21.15.49:0/2731352837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 0x7fa2b0102250 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009910 tx=0x7fa2a0009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:21.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.861+0000 7fa2b537c700 1 -- 172.21.15.49:0/2731352837 shutdown_connections 2024-05-15T16:07:21.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.861+0000 7fa2b537c700 1 --2- 172.21.15.49:0/2731352837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0102790 0x7fa2b010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.861+0000 7fa2b537c700 1 --2- 172.21.15.49:0/2731352837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 0x7fa2b0102250 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.861+0000 7fa2b537c700 1 -- 172.21.15.49:0/2731352837 >> 172.21.15.49:0/2731352837 conn(0x7fa2b00fb700 msgr2=0x7fa2b00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:21.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.861+0000 7fa2b537c700 1 -- 172.21.15.49:0/2731352837 shutdown_connections 2024-05-15T16:07:21.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.861+0000 7fa2b537c700 1 -- 172.21.15.49:0/2731352837 wait complete. 2024-05-15T16:07:21.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.861+0000 7fa2b537c700 1 Processor -- start 2024-05-15T16:07:21.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2b537c700 1 -- start start 2024-05-15T16:07:21.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2b537c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 0x7fa2b0196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:21.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2b537c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0102790 0x7fa2b0197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:21.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2b537c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b01978f0 con 0x7fa2b0101e70 2024-05-15T16:07:21.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2b537c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b019a560 con 0x7fa2b0102790 2024-05-15T16:07:21.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2aeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 0x7fa2b0196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:21.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2aeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 0x7fa2b0196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48000/0 (socket says 172.21.15.49:48000) 2024-05-15T16:07:21.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2aeffd700 1 -- 172.21.15.49:0/2925815930 learned_addr learned my addr 172.21.15.49:0/2925815930 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:21.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2ae7fc700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0102790 0x7fa2b0197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:21.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2aeffd700 1 -- 172.21.15.49:0/2925815930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0102790 msgr2=0x7fa2b0197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:21.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2aeffd700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0102790 0x7fa2b0197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:21.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2aeffd700 1 -- 172.21.15.49:0/2925815930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00095f0 con 0x7fa2b0101e70 2024-05-15T16:07:21.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2ae7fc700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0102790 0x7fa2b0197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:07:21.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.862+0000 7fa2aeffd700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 0x7fa2b0196d20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00092e0 tx=0x7fa2a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:21.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.863+0000 7fa297fff700 1 -- 172.21.15.49:0/2925815930 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a001d070 con 0x7fa2b0101e70 2024-05-15T16:07:21.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.863+0000 7fa297fff700 1 -- 172.21.15.49:0/2925815930 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2a000f460 con 0x7fa2b0101e70 2024-05-15T16:07:21.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.863+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b019a7e0 con 0x7fa2b0101e70 2024-05-15T16:07:21.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.863+0000 7fa297fff700 1 -- 172.21.15.49:0/2925815930 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a0017630 con 0x7fa2b0101e70 2024-05-15T16:07:21.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.863+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b019acd0 con 0x7fa2b0101e70 2024-05-15T16:07:21.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.864+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b0069ab0 con 0x7fa2b0101e70 2024-05-15T16:07:21.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.864+0000 7fa297fff700 1 -- 172.21.15.49:0/2925815930 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa2a000f5d0 con 0x7fa2b0101e70 2024-05-15T16:07:21.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.865+0000 7fa297fff700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa29806e210 0x7fa2980706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:21.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.865+0000 7fa297fff700 1 -- 172.21.15.49:0/2925815930 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa2a008f5d0 con 0x7fa2b0101e70 2024-05-15T16:07:21.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.865+0000 7fa2ae7fc700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa29806e210 0x7fa2980706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:21.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.865+0000 7fa2ae7fc700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa29806e210 0x7fa2980706d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0198240 tx=0x7fa2a4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:21.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:21.870+0000 7fa297fff700 1 -- 172.21.15.49:0/2925815930 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa2a0058930 con 0x7fa2b0101e70 2024-05-15T16:07:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.098+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2b010df70 con 0x7fa29806e210 2024-05-15T16:07:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.099+0000 7fa297fff700 1 -- 172.21.15.49:0/2925815930 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa2b010df70 con 0x7fa29806e210 2024-05-15T16:07:22.100 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:07:22.101 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:07:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa29806e210 msgr2=0x7fa2980706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa29806e210 0x7fa2980706d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0198240 tx=0x7fa2a4008040 comp rx=0 tx=0).stop 2024-05-15T16:07:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 msgr2=0x7fa2b0196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 0x7fa2b0196d20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00092e0 tx=0x7fa2a000be20 comp rx=0 tx=0).stop 2024-05-15T16:07:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 shutdown_connections 2024-05-15T16:07:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa29806e210 0x7fa2980706d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2b0101e70 0x7fa2b0196d20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 --2- 172.21.15.49:0/2925815930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0102790 0x7fa2b0197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 >> 172.21.15.49:0/2925815930 conn(0x7fa2b00fb700 msgr2=0x7fa2b0103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 shutdown_connections 2024-05-15T16:07:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.102+0000 7fa2b537c700 1 -- 172.21.15.49:0/2925815930 wait complete. 2024-05-15T16:07:22.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.212+0000 7efd90687700 1 -- 172.21.15.49:0/1662393611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 msgr2=0x7efd88105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:22.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.212+0000 7efd90687700 1 --2- 172.21.15.49:0/1662393611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 0x7efd88105900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efd78009910 tx=0x7efd78009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:22.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.212+0000 7efd90687700 1 -- 172.21.15.49:0/1662393611 shutdown_connections 2024-05-15T16:07:22.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.212+0000 7efd90687700 1 --2- 172.21.15.49:0/1662393611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 0x7efd88105900 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:22.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.212+0000 7efd90687700 1 --2- 172.21.15.49:0/1662393611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd88100f00 0x7efd881012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:22.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.212+0000 7efd90687700 1 -- 172.21.15.49:0/1662393611 >> 172.21.15.49:0/1662393611 conn(0x7efd880fc7d0 msgr2=0x7efd880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:22.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.212+0000 7efd90687700 1 -- 172.21.15.49:0/1662393611 shutdown_connections 2024-05-15T16:07:22.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.212+0000 7efd90687700 1 -- 172.21.15.49:0/1662393611 wait complete. 2024-05-15T16:07:22.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd90687700 1 Processor -- start 2024-05-15T16:07:22.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd90687700 1 -- start start 2024-05-15T16:07:22.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd90687700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd88100f00 0x7efd8819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:22.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd90687700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 0x7efd8819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:22.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd90687700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8819d3b0 con 0x7efd881018b0 2024-05-15T16:07:22.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd90687700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd88196a20 con 0x7efd88100f00 2024-05-15T16:07:22.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd8dc22700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 0x7efd8819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:22.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd8dc22700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 0x7efd8819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48016/0 (socket says 172.21.15.49:48016) 2024-05-15T16:07:22.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd8dc22700 1 -- 172.21.15.49:0/3266296431 learned_addr learned my addr 172.21.15.49:0/3266296431 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:22.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.213+0000 7efd8e423700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd88100f00 0x7efd8819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:22.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.214+0000 7efd8dc22700 1 -- 172.21.15.49:0/3266296431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd88100f00 msgr2=0x7efd8819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:22.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.214+0000 7efd8dc22700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd88100f00 0x7efd8819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:22.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.214+0000 7efd8dc22700 1 -- 172.21.15.49:0/3266296431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd780095f0 con 0x7efd881018b0 2024-05-15T16:07:22.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.214+0000 7efd8dc22700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 0x7efd8819ce20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efd78005b40 tx=0x7efd7800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:22.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.214+0000 7efd7f7fe700 1 -- 172.21.15.49:0/3266296431 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd7801d070 con 0x7efd881018b0 2024-05-15T16:07:22.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.214+0000 7efd7f7fe700 1 -- 172.21.15.49:0/3266296431 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd7800f460 con 0x7efd881018b0 2024-05-15T16:07:22.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.214+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd88196ca0 con 0x7efd881018b0 2024-05-15T16:07:22.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.214+0000 7efd7f7fe700 1 -- 172.21.15.49:0/3266296431 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd78017630 con 0x7efd881018b0 2024-05-15T16:07:22.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.214+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd88197190 con 0x7efd881018b0 2024-05-15T16:07:22.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.215+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd88109300 con 0x7efd881018b0 2024-05-15T16:07:22.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.215+0000 7efd7f7fe700 1 -- 172.21.15.49:0/3266296431 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efd7800f5d0 con 0x7efd881018b0 2024-05-15T16:07:22.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.216+0000 7efd7f7fe700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd7406e210 0x7efd740706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:22.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.216+0000 7efd8e423700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd7406e210 0x7efd740706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:22.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.216+0000 7efd7f7fe700 1 -- 172.21.15.49:0/3266296431 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efd7808f6e0 con 0x7efd881018b0 2024-05-15T16:07:22.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.217+0000 7efd8e423700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd7406e210 0x7efd740706d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd8400ba60 tx=0x7efd84005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:22.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.221+0000 7efd7f7fe700 1 -- 172.21.15.49:0/3266296431 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efd7805c4f0 con 0x7efd881018b0 2024-05-15T16:07:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:22 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/208860655' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:07:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:22 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4274061466' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:07:22.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.514+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efd881023e0 con 0x7efd881018b0 2024-05-15T16:07:22.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.515+0000 7efd7f7fe700 1 -- 172.21.15.49:0/3266296431 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7efd7805c080 con 0x7efd881018b0 2024-05-15T16:07:22.517 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:07:22.517 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:07:22.517 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:07:22.517 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:07:22.517 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:07:22.517 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:07:22.517 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:07:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd7406e210 msgr2=0x7efd740706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd7406e210 0x7efd740706d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd8400ba60 tx=0x7efd84005d50 comp rx=0 tx=0).stop 2024-05-15T16:07:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 msgr2=0x7efd8819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 0x7efd8819ce20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efd78005b40 tx=0x7efd7800be20 comp rx=0 tx=0).stop 2024-05-15T16:07:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 shutdown_connections 2024-05-15T16:07:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd7406e210 0x7efd740706d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:22.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd88100f00 0x7efd8819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:22.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 --2- 172.21.15.49:0/3266296431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd881018b0 0x7efd8819ce20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:22.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 >> 172.21.15.49:0/3266296431 conn(0x7efd880fc7d0 msgr2=0x7efd880fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:22.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 shutdown_connections 2024-05-15T16:07:22.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:22.517+0000 7efd90687700 1 -- 172.21.15.49:0/3266296431 wait complete. 2024-05-15T16:07:22.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:22 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/208860655' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:07:22.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:22 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4274061466' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:07:23.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:23 smithi144 ceph-mon[86635]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:23.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:23 smithi144 ceph-mon[86635]: pgmap v310: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:23.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:23 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3266296431' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:07:23.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:23 smithi049 ceph-mon[112142]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:23.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:23 smithi049 ceph-mon[112142]: pgmap v310: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:23.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:23 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3266296431' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:07:25.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:25 smithi144 ceph-mon[86635]: pgmap v311: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:25.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:25 smithi049 ceph-mon[112142]: pgmap v311: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:27.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:27 smithi144 ceph-mon[86635]: pgmap v312: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:27.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:27 smithi049 ceph-mon[112142]: pgmap v312: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:29.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:29 smithi144 ceph-mon[86635]: pgmap v313: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:29.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:29 smithi049 ceph-mon[112142]: pgmap v313: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:31.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:31 smithi049 ceph-mon[112142]: pgmap v314: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:31.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:31 smithi144 ceph-mon[86635]: pgmap v314: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:33.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:33 smithi144 ceph-mon[86635]: pgmap v315: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:33.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:33 smithi049 ceph-mon[112142]: pgmap v315: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:35.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:35 smithi144 ceph-mon[86635]: pgmap v316: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:35.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:07:35.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:35 smithi049 ceph-mon[112142]: pgmap v316: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:07:37.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:37 smithi144 ceph-mon[86635]: pgmap v317: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:37.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:37 smithi049 ceph-mon[112142]: pgmap v317: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:39.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:39 smithi144 ceph-mon[86635]: pgmap v318: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:39.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:39 smithi049 ceph-mon[112142]: pgmap v318: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:41.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:41 smithi049 ceph-mon[112142]: pgmap v319: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:41.645 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:41 smithi144 ceph-mon[86635]: pgmap v319: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:43.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:43 smithi144 ceph-mon[86635]: pgmap v320: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:43.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:43 smithi049 ceph-mon[112142]: pgmap v320: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:45.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:45 smithi144 ceph-mon[86635]: pgmap v321: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:45 smithi049 ceph-mon[112142]: pgmap v321: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:47.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:47 smithi144 ceph-mon[86635]: pgmap v322: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:47.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:47 smithi049 ceph-mon[112142]: pgmap v322: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:49.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:49 smithi144 ceph-mon[86635]: pgmap v323: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:49.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:49 smithi049 ceph-mon[112142]: pgmap v323: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:07:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:07:51.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:51 smithi144 ceph-mon[86635]: pgmap v324: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:51.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:51 smithi049 ceph-mon[112142]: pgmap v324: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.642+0000 7f7c23de6700 1 -- 172.21.15.49:0/3133063874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101890 msgr2=0x7f7c1c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.642+0000 7f7c23de6700 1 --2- 172.21.15.49:0/3133063874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101890 0x7f7c1c1058e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009910 tx=0x7f7c10009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.642+0000 7f7c23de6700 1 -- 172.21.15.49:0/3133063874 shutdown_connections 2024-05-15T16:07:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.642+0000 7f7c23de6700 1 --2- 172.21.15.49:0/3133063874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c101890 0x7f7c1c1058e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.642+0000 7f7c23de6700 1 --2- 172.21.15.49:0/3133063874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c1c100ee0 0x7f7c1c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.642+0000 7f7c23de6700 1 -- 172.21.15.49:0/3133063874 >> 172.21.15.49:0/3133063874 conn(0x7f7c1c0fc790 msgr2=0x7f7c1c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.642+0000 7f7c23de6700 1 -- 172.21.15.49:0/3133063874 shutdown_connections 2024-05-15T16:07:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.642+0000 7f7c23de6700 1 -- 172.21.15.49:0/3133063874 wait complete. 2024-05-15T16:07:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.642+0000 7f7c23de6700 1 Processor -- start 2024-05-15T16:07:52.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c23de6700 1 -- start start 2024-05-15T16:07:52.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c23de6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c100ee0 0x7f7c1c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:52.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c23de6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c1c101890 0x7f7c1c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:52.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c23de6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c19d440 con 0x7f7c1c100ee0 2024-05-15T16:07:52.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c23de6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c196a20 con 0x7f7c1c101890 2024-05-15T16:07:52.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c21b82700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c100ee0 0x7f7c1c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c21b82700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c100ee0 0x7f7c1c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47266/0 (socket says 172.21.15.49:47266) 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c21b82700 1 -- 172.21.15.49:0/3135709113 learned_addr learned my addr 172.21.15.49:0/3135709113 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c21381700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c1c101890 0x7f7c1c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c21b82700 1 -- 172.21.15.49:0/3135709113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c1c101890 msgr2=0x7f7c1c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c21b82700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c1c101890 0x7f7c1c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c21b82700 1 -- 172.21.15.49:0/3135709113 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c100095f0 con 0x7f7c1c100ee0 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.643+0000 7f7c21b82700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c100ee0 0x7f7c1c19c8e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800d900 tx=0x7f7c1800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.644+0000 7f7c0effd700 1 -- 172.21.15.49:0/3135709113 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c18004a00 con 0x7f7c1c100ee0 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.644+0000 7f7c0effd700 1 -- 172.21.15.49:0/3135709113 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c180054e0 con 0x7f7c1c100ee0 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.644+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1c196d00 con 0x7f7c1c100ee0 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.644+0000 7f7c0effd700 1 -- 172.21.15.49:0/3135709113 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c18009d00 con 0x7f7c1c100ee0 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.644+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c197250 con 0x7f7c1c100ee0 2024-05-15T16:07:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.645+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1c1092e0 con 0x7f7c1c100ee0 2024-05-15T16:07:52.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.645+0000 7f7c0effd700 1 -- 172.21.15.49:0/3135709113 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7c18005020 con 0x7f7c1c100ee0 2024-05-15T16:07:52.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.646+0000 7f7c0effd700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7c0806e200 0x7f7c080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:52.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.646+0000 7f7c0effd700 1 -- 172.21.15.49:0/3135709113 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7c1808dff0 con 0x7f7c1c100ee0 2024-05-15T16:07:52.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.646+0000 7f7c21381700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7c0806e200 0x7f7c080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:52.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.646+0000 7f7c21381700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7c0806e200 0x7f7c080706c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c197be0 tx=0x7f7c1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:52.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.651+0000 7f7c0effd700 1 -- 172.21.15.49:0/3135709113 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7c1805ae80 con 0x7f7c1c100ee0 2024-05-15T16:07:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.881+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c1c197ee0 con 0x7f7c0806e200 2024-05-15T16:07:52.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.882+0000 7f7c0effd700 1 -- 172.21.15.49:0/3135709113 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7c1c197ee0 con 0x7f7c0806e200 2024-05-15T16:07:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.884+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7c0806e200 msgr2=0x7f7c080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.884+0000 7f7c23de6700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7c0806e200 0x7f7c080706c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c197be0 tx=0x7f7c1001a040 comp rx=0 tx=0).stop 2024-05-15T16:07:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.884+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c100ee0 msgr2=0x7f7c1c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.885+0000 7f7c23de6700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c100ee0 0x7f7c1c19c8e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800d900 tx=0x7f7c1800dc10 comp rx=0 tx=0).stop 2024-05-15T16:07:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.885+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 shutdown_connections 2024-05-15T16:07:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.885+0000 7f7c23de6700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7c0806e200 0x7f7c080706c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.885+0000 7f7c23de6700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c1c100ee0 0x7f7c1c19c8e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.885+0000 7f7c23de6700 1 --2- 172.21.15.49:0/3135709113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c1c101890 0x7f7c1c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.885+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 >> 172.21.15.49:0/3135709113 conn(0x7f7c1c0fc790 msgr2=0x7f7c1c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.885+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 shutdown_connections 2024-05-15T16:07:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.885+0000 7f7c23de6700 1 -- 172.21.15.49:0/3135709113 wait complete. 2024-05-15T16:07:52.897 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:07:52.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.995+0000 7f15d3476700 1 -- 172.21.15.49:0/3174166016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 msgr2=0x7f15cc1001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:52.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.995+0000 7f15d3476700 1 --2- 172.21.15.49:0/3174166016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 0x7f15cc1001c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00b0a0 tx=0x7f15bc00b3b0 comp rx=0 tx=0).stop 2024-05-15T16:07:52.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.995+0000 7f15d3476700 1 -- 172.21.15.49:0/3174166016 shutdown_connections 2024-05-15T16:07:52.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.995+0000 7f15d3476700 1 --2- 172.21.15.49:0/3174166016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc100790 0x7f15cc1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:52.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.995+0000 7f15d3476700 1 --2- 172.21.15.49:0/3174166016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 0x7f15cc1001c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:52.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.995+0000 7f15d3476700 1 -- 172.21.15.49:0/3174166016 >> 172.21.15.49:0/3174166016 conn(0x7f15cc0fb690 msgr2=0x7f15cc0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:52.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.995+0000 7f15d3476700 1 -- 172.21.15.49:0/3174166016 shutdown_connections 2024-05-15T16:07:52.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.995+0000 7f15d3476700 1 -- 172.21.15.49:0/3174166016 wait complete. 2024-05-15T16:07:52.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d3476700 1 Processor -- start 2024-05-15T16:07:52.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d3476700 1 -- start start 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d3476700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 0x7f15cc101880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d3476700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc100790 0x7f15cc101dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d3476700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc102300 con 0x7f15cc0ffde0 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d3476700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc102440 con 0x7f15cc100790 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d1212700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 0x7f15cc101880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d1212700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 0x7f15cc101880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47290/0 (socket says 172.21.15.49:47290) 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d1212700 1 -- 172.21.15.49:0/1999238001 learned_addr learned my addr 172.21.15.49:0/1999238001 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d0a11700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc100790 0x7f15cc101dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d1212700 1 -- 172.21.15.49:0/1999238001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc100790 msgr2=0x7f15cc101dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d1212700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc100790 0x7f15cc101dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.996+0000 7f15d1212700 1 -- 172.21.15.49:0/1999238001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc009d00 con 0x7f15cc0ffde0 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.997+0000 7f15d1212700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 0x7f15cc101880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00b070 tx=0x7f15bc007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.997+0000 7f15ca7fc700 1 -- 172.21.15.49:0/1999238001 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15bc00e040 con 0x7f15cc0ffde0 2024-05-15T16:07:53.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.997+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc1026c0 con 0x7f15cc0ffde0 2024-05-15T16:07:53.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.997+0000 7f15ca7fc700 1 -- 172.21.15.49:0/1999238001 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15bc003c10 con 0x7f15cc0ffde0 2024-05-15T16:07:53.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.997+0000 7f15ca7fc700 1 -- 172.21.15.49:0/1999238001 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15bc01d6b0 con 0x7f15cc0ffde0 2024-05-15T16:07:53.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.997+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc1a3070 con 0x7f15cc0ffde0 2024-05-15T16:07:53.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.998+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc1012c0 con 0x7f15cc0ffde0 2024-05-15T16:07:53.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.999+0000 7f15ca7fc700 1 -- 172.21.15.49:0/1999238001 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f15bc004600 con 0x7f15cc0ffde0 2024-05-15T16:07:53.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.999+0000 7f15ca7fc700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15b406e210 0x7f15b40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:53.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.999+0000 7f15ca7fc700 1 -- 172.21.15.49:0/1999238001 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f15bc08f950 con 0x7f15cc0ffde0 2024-05-15T16:07:53.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:52.999+0000 7f15d0a11700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15b406e210 0x7f15b40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:53.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.000+0000 7f15d0a11700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15b406e210 0x7f15b40706d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f15c0005950 tx=0x7f15c00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:53.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.004+0000 7f15ca7fc700 1 -- 172.21.15.49:0/1999238001 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f15bc058c30 con 0x7f15cc0ffde0 2024-05-15T16:07:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.231+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15cc10cdc0 con 0x7f15b406e210 2024-05-15T16:07:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.232+0000 7f15ca7fc700 1 -- 172.21.15.49:0/1999238001 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f15cc10cdc0 con 0x7f15b406e210 2024-05-15T16:07:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15b406e210 msgr2=0x7f15b40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15b406e210 0x7f15b40706d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f15c0005950 tx=0x7f15c00058e0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 msgr2=0x7f15cc101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 0x7f15cc101880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00b070 tx=0x7f15bc007bd0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 shutdown_connections 2024-05-15T16:07:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f15b406e210 0x7f15b40706d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15cc0ffde0 0x7f15cc101880 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 --2- 172.21.15.49:0/1999238001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc100790 0x7f15cc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 >> 172.21.15.49:0/1999238001 conn(0x7f15cc0fb690 msgr2=0x7f15cc069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 shutdown_connections 2024-05-15T16:07:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.234+0000 7f15d3476700 1 -- 172.21.15.49:0/1999238001 wait complete. 2024-05-15T16:07:53.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.342+0000 7fddf1346700 1 -- 172.21.15.49:0/337687699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 msgr2=0x7fddec10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.342+0000 7fddf1346700 1 --2- 172.21.15.49:0/337687699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 0x7fddec10ac80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009910 tx=0x7fdde0009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:53.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.342+0000 7fddf1346700 1 -- 172.21.15.49:0/337687699 shutdown_connections 2024-05-15T16:07:53.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.342+0000 7fddf1346700 1 --2- 172.21.15.49:0/337687699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 0x7fddec10ac80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.342+0000 7fddf1346700 1 --2- 172.21.15.49:0/337687699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddec101e70 0x7fddec102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.342+0000 7fddf1346700 1 -- 172.21.15.49:0/337687699 >> 172.21.15.49:0/337687699 conn(0x7fddec0fb700 msgr2=0x7fddec0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:53.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.343+0000 7fddf1346700 1 -- 172.21.15.49:0/337687699 shutdown_connections 2024-05-15T16:07:53.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.343+0000 7fddf1346700 1 -- 172.21.15.49:0/337687699 wait complete. 2024-05-15T16:07:53.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.343+0000 7fddf1346700 1 Processor -- start 2024-05-15T16:07:53.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.343+0000 7fddf1346700 1 -- start start 2024-05-15T16:07:53.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddf1346700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddec101e70 0x7fddec196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:53.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddf1346700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 0x7fddec197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:53.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddf1346700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec1978d0 con 0x7fddec102790 2024-05-15T16:07:53.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddf1346700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec19a540 con 0x7fddec101e70 2024-05-15T16:07:53.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddea7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 0x7fddec197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:53.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddea7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 0x7fddec197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47318/0 (socket says 172.21.15.49:47318) 2024-05-15T16:07:53.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddea7fc700 1 -- 172.21.15.49:0/151531757 learned_addr learned my addr 172.21.15.49:0/151531757 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:53.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddeaffd700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddec101e70 0x7fddec196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:53.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddea7fc700 1 -- 172.21.15.49:0/151531757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddec101e70 msgr2=0x7fddec196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddea7fc700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddec101e70 0x7fddec196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddea7fc700 1 -- 172.21.15.49:0/151531757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00095f0 con 0x7fddec102790 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddea7fc700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 0x7fddec197240 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdde0005b40 tx=0x7fdde000bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddd3fff700 1 -- 172.21.15.49:0/151531757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde001d070 con 0x7fddec102790 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddd3fff700 1 -- 172.21.15.49:0/151531757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdde0003d50 con 0x7fddec102790 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.344+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddec19a7c0 con 0x7fddec102790 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.345+0000 7fddd3fff700 1 -- 172.21.15.49:0/151531757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde000fe90 con 0x7fddec102790 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.345+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddec19acb0 con 0x7fddec102790 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.346+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddec069ab0 con 0x7fddec102790 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.346+0000 7fddd3fff700 1 -- 172.21.15.49:0/151531757 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdde000f6d0 con 0x7fddec102790 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.347+0000 7fddd3fff700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddd406e210 0x7fddd40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:53.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.347+0000 7fddd3fff700 1 -- 172.21.15.49:0/151531757 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdde008f6e0 con 0x7fddec102790 2024-05-15T16:07:53.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.347+0000 7fddeaffd700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddd406e210 0x7fddd40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:53.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.347+0000 7fddeaffd700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddd406e210 0x7fddd40706d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdddc006fd0 tx=0x7fdddc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:53.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.351+0000 7fddd3fff700 1 -- 172.21.15.49:0/151531757 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdde0058920 con 0x7fddec102790 2024-05-15T16:07:53.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.568+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fddec105590 con 0x7fddd406e210 2024-05-15T16:07:53.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.579+0000 7fddd3fff700 1 -- 172.21.15.49:0/151531757 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fddec105590 con 0x7fddd406e210 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (11m) 4m ago 22m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (22m) 4m ago 22m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (21m) 4m ago 21m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (8m) 4m ago 22m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (8m) 4m ago 21m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (10m) 4m ago 22m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (4m) 4m ago 17m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (4m) 4m ago 17m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (4m) 4m ago 17m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (16m) 4m ago 16m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:07:53.581 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (13m) 4m ago 24m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (12m) 4m ago 21m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (9m) 4m ago 24m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (9m) 4m ago 21m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (11m) 4m ago 22m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (11m) 4m ago 21m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (8m) 4m ago 20m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (7m) 4m ago 20m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (6m) 4m ago 19m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (6m) 4m ago 19m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (5m) 4m ago 19m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (5m) 4m ago 18m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:07:53.582 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (11m) 4m ago 22m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:07:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddd406e210 msgr2=0x7fddd40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddd406e210 0x7fddd40706d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdddc006fd0 tx=0x7fdddc008040 comp rx=0 tx=0).stop 2024-05-15T16:07:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 msgr2=0x7fddec197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 0x7fddec197240 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdde0005b40 tx=0x7fdde000bc10 comp rx=0 tx=0).stop 2024-05-15T16:07:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 shutdown_connections 2024-05-15T16:07:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddd406e210 0x7fddd40706d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddec101e70 0x7fddec196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 --2- 172.21.15.49:0/151531757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddec102790 0x7fddec197240 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 >> 172.21.15.49:0/151531757 conn(0x7fddec0fb700 msgr2=0x7fddec103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 shutdown_connections 2024-05-15T16:07:53.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.581+0000 7fddf1346700 1 -- 172.21.15.49:0/151531757 wait complete. 2024-05-15T16:07:53.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.689+0000 7f1629eb8700 1 -- 172.21.15.49:0/1629264249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 msgr2=0x7f1624104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.689+0000 7f1629eb8700 1 --2- 172.21.15.49:0/1629264249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 0x7f1624104830 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1618009910 tx=0x7f1618009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:53.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.689+0000 7f1629eb8700 1 -- 172.21.15.49:0/1629264249 shutdown_connections 2024-05-15T16:07:53.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.689+0000 7f1629eb8700 1 --2- 172.21.15.49:0/1629264249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 0x7f1624104830 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.689+0000 7f1629eb8700 1 --2- 172.21.15.49:0/1629264249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1624100800 0x7f1624100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.689+0000 7f1629eb8700 1 -- 172.21.15.49:0/1629264249 >> 172.21.15.49:0/1629264249 conn(0x7f1624074e70 msgr2=0x7f1624075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.689+0000 7f1629eb8700 1 -- 172.21.15.49:0/1629264249 shutdown_connections 2024-05-15T16:07:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.689+0000 7f1629eb8700 1 -- 172.21.15.49:0/1629264249 wait complete. 2024-05-15T16:07:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1629eb8700 1 Processor -- start 2024-05-15T16:07:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1629eb8700 1 -- start start 2024-05-15T16:07:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1629eb8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1624100800 0x7f162419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1629eb8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 0x7f162419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1629eb8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162419d4d0 con 0x7f16241011b0 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1629eb8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1624196ab0 con 0x7f1624100800 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1622ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 0x7f162419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1622ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 0x7f162419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56090/0 (socket says 172.21.15.49:56090) 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1622ffd700 1 -- 172.21.15.49:0/489667791 learned_addr learned my addr 172.21.15.49:0/489667791 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f16237fe700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1624100800 0x7f162419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1622ffd700 1 -- 172.21.15.49:0/489667791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1624100800 msgr2=0x7f162419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1622ffd700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1624100800 0x7f162419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.690+0000 7f1622ffd700 1 -- 172.21.15.49:0/489667791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16180095f0 con 0x7f16241011b0 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.691+0000 7f1622ffd700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 0x7f162419ceb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f161800bd70 tx=0x7f161800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.691+0000 7f1620ff9700 1 -- 172.21.15.49:0/489667791 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f161801d070 con 0x7f16241011b0 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.691+0000 7f1620ff9700 1 -- 172.21.15.49:0/489667791 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16180056c0 con 0x7f16241011b0 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.691+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1624196d30 con 0x7f16241011b0 2024-05-15T16:07:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.691+0000 7f1620ff9700 1 -- 172.21.15.49:0/489667791 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1618017400 con 0x7f16241011b0 2024-05-15T16:07:53.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.691+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1624197220 con 0x7f16241011b0 2024-05-15T16:07:53.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.692+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16241080f0 con 0x7f16241011b0 2024-05-15T16:07:53.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.693+0000 7f1620ff9700 1 -- 172.21.15.49:0/489667791 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1618003a60 con 0x7f16241011b0 2024-05-15T16:07:53.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.693+0000 7f1620ff9700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f160c06e200 0x7f160c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:53.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.693+0000 7f1620ff9700 1 -- 172.21.15.49:0/489667791 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f161808f4a0 con 0x7f16241011b0 2024-05-15T16:07:53.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.693+0000 7f16237fe700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f160c06e200 0x7f160c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:53.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.694+0000 7f16237fe700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f160c06e200 0x7f160c0706c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1614005950 tx=0x7f161400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:53.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.698+0000 7f1620ff9700 1 -- 172.21.15.49:0/489667791 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f161805c330 con 0x7f16241011b0 2024-05-15T16:07:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:53 smithi144 ceph-mon[86635]: pgmap v325: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:53 smithi144 ceph-mon[86635]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:53 smithi049 ceph-mon[112142]: pgmap v325: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:53 smithi049 ceph-mon[112142]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:53.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.990+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1624068f10 con 0x7f16241011b0 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.991+0000 7f1620ff9700 1 -- 172.21.15.49:0/489667791 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f161805bec0 con 0x7f16241011b0 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f160c06e200 msgr2=0x7f160c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f160c06e200 0x7f160c0706c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1614005950 tx=0x7f161400a300 comp rx=0 tx=0).stop 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 msgr2=0x7f162419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 0x7f162419ceb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f161800bd70 tx=0x7f161800bda0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 shutdown_connections 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f160c06e200 0x7f160c0706c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1624100800 0x7f162419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 --2- 172.21.15.49:0/489667791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16241011b0 0x7f162419ceb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 >> 172.21.15.49:0/489667791 conn(0x7f1624074e70 msgr2=0x7f1624103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 shutdown_connections 2024-05-15T16:07:53.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:53.993+0000 7f1629eb8700 1 -- 172.21.15.49:0/489667791 wait complete. 2024-05-15T16:07:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.102+0000 7fec337aa700 1 -- 172.21.15.49:0/4159232686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 msgr2=0x7fec2c0736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.102+0000 7fec337aa700 1 --2- 172.21.15.49:0/4159232686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 0x7fec2c0736a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fec20009910 tx=0x7fec20009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.102+0000 7fec337aa700 1 -- 172.21.15.49:0/4159232686 shutdown_connections 2024-05-15T16:07:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.102+0000 7fec337aa700 1 --2- 172.21.15.49:0/4159232686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c073be0 0x7fec2c10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.102+0000 7fec337aa700 1 --2- 172.21.15.49:0/4159232686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 0x7fec2c0736a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.102+0000 7fec337aa700 1 -- 172.21.15.49:0/4159232686 >> 172.21.15.49:0/4159232686 conn(0x7fec2c0fc790 msgr2=0x7fec2c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:54.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.102+0000 7fec337aa700 1 -- 172.21.15.49:0/4159232686 shutdown_connections 2024-05-15T16:07:54.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.102+0000 7fec337aa700 1 -- 172.21.15.49:0/4159232686 wait complete. 2024-05-15T16:07:54.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec337aa700 1 Processor -- start 2024-05-15T16:07:54.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec337aa700 1 -- start start 2024-05-15T16:07:54.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec337aa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 0x7fec2c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:54.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec337aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c073be0 0x7fec2c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:54.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec337aa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c1999d0 con 0x7fec2c0732c0 2024-05-15T16:07:54.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec337aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c19d6a0 con 0x7fec2c073be0 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec31546700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 0x7fec2c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec31546700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 0x7fec2c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56100/0 (socket says 172.21.15.49:56100) 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec31546700 1 -- 172.21.15.49:0/1896459384 learned_addr learned my addr 172.21.15.49:0/1896459384 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec30d45700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c073be0 0x7fec2c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec31546700 1 -- 172.21.15.49:0/1896459384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c073be0 msgr2=0x7fec2c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec31546700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c073be0 0x7fec2c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.103+0000 7fec31546700 1 -- 172.21.15.49:0/1896459384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200095f0 con 0x7fec2c0732c0 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.104+0000 7fec31546700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 0x7fec2c198db0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec2000bd70 tx=0x7fec2000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.104+0000 7fec1e7fc700 1 -- 172.21.15.49:0/1896459384 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2001d070 con 0x7fec2c0732c0 2024-05-15T16:07:54.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.104+0000 7fec1e7fc700 1 -- 172.21.15.49:0/1896459384 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec200056e0 con 0x7fec2c0732c0 2024-05-15T16:07:54.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.104+0000 7fec1e7fc700 1 -- 172.21.15.49:0/1896459384 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2000fe40 con 0x7fec2c0732c0 2024-05-15T16:07:54.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.104+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c19d920 con 0x7fec2c0732c0 2024-05-15T16:07:54.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.104+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c19de10 con 0x7fec2c0732c0 2024-05-15T16:07:54.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.105+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c074a00 con 0x7fec2c0732c0 2024-05-15T16:07:54.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.105+0000 7fec1e7fc700 1 -- 172.21.15.49:0/1896459384 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fec2000f680 con 0x7fec2c0732c0 2024-05-15T16:07:54.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.106+0000 7fec1e7fc700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fec1806e1a0 0x7fec18070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:54.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.106+0000 7fec1e7fc700 1 -- 172.21.15.49:0/1896459384 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fec2008f600 con 0x7fec2c0732c0 2024-05-15T16:07:54.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.106+0000 7fec30d45700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fec1806e1a0 0x7fec18070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:54.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.107+0000 7fec30d45700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fec1806e1a0 0x7fec18070660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fec2c19a310 tx=0x7fec2800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:54.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.111+0000 7fec1e7fc700 1 -- 172.21.15.49:0/1896459384 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fec20058880 con 0x7fec2c0732c0 2024-05-15T16:07:54.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.367+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fec2c1078f0 con 0x7fec2c0732c0 2024-05-15T16:07:54.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.367+0000 7fec1e7fc700 1 -- 172.21.15.49:0/1896459384 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7fec2005bf40 con 0x7fec2c0732c0 2024-05-15T16:07:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:07:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:07:54.370 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:07:54.371 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:07:54.371 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:07:54.371 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:07:54.371 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:07:54.371 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:07:54.371 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:07:54.371 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:07:54.371 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:07:54.372 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:07:54.373 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:07:54.373 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:07:54.373 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:07:54.373 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:07:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fec1806e1a0 msgr2=0x7fec18070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fec1806e1a0 0x7fec18070660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fec2c19a310 tx=0x7fec2800b410 comp rx=0 tx=0).stop 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 msgr2=0x7fec2c198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 0x7fec2c198db0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec2000bd70 tx=0x7fec2000bda0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 shutdown_connections 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fec1806e1a0 0x7fec18070660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec2c0732c0 0x7fec2c198db0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 --2- 172.21.15.49:0/1896459384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c073be0 0x7fec2c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 >> 172.21.15.49:0/1896459384 conn(0x7fec2c0fc790 msgr2=0x7fec2c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.370+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 shutdown_connections 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.371+0000 7fec337aa700 1 -- 172.21.15.49:0/1896459384 wait complete. 2024-05-15T16:07:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:07:54.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 -- 172.21.15.49:0/387054885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 msgr2=0x7fa5841047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 --2- 172.21.15.49:0/387054885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5841047c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa578009910 tx=0x7fa578009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 -- 172.21.15.49:0/387054885 shutdown_connections 2024-05-15T16:07:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 --2- 172.21.15.49:0/387054885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5841047c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 --2- 172.21.15.49:0/387054885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5840ffdc0 0x7fa5841001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 -- 172.21.15.49:0/387054885 >> 172.21.15.49:0/387054885 conn(0x7fa584076800 msgr2=0x7fa5840fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 -- 172.21.15.49:0/387054885 shutdown_connections 2024-05-15T16:07:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 -- 172.21.15.49:0/387054885 wait complete. 2024-05-15T16:07:54.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 Processor -- start 2024-05-15T16:07:54.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.482+0000 7fa5894fa700 1 -- start start 2024-05-15T16:07:54.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5894fa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:54.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5894fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5840fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:54.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5894fa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840fc760 con 0x7fa584100770 2024-05-15T16:07:54.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5894fa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840fc8d0 con 0x7fa5840ffdc0 2024-05-15T16:07:54.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5827fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5840fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:54.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5827fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5840fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56124/0 (socket says 172.21.15.49:56124) 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5827fc700 1 -- 172.21.15.49:0/664774882 learned_addr learned my addr 172.21.15.49:0/664774882 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa582ffd700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5827fc700 1 -- 172.21.15.49:0/664774882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5840ffdc0 msgr2=0x7fa5840fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5827fc700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.483+0000 7fa5827fc700 1 -- 172.21.15.49:0/664774882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780095f0 con 0x7fa584100770 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.484+0000 7fa5827fc700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5840fc220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa578005b40 tx=0x7fa57800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.484+0000 7fa56bfff700 1 -- 172.21.15.49:0/664774882 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57801d070 con 0x7fa584100770 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.484+0000 7fa56bfff700 1 -- 172.21.15.49:0/664774882 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa57800f460 con 0x7fa584100770 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.484+0000 7fa56bfff700 1 -- 172.21.15.49:0/664774882 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa578005720 con 0x7fa584100770 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.484+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5840fcb50 con 0x7fa584100770 2024-05-15T16:07:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.484+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5841a2fa0 con 0x7fa584100770 2024-05-15T16:07:54.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.485+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5841012a0 con 0x7fa584100770 2024-05-15T16:07:54.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.485+0000 7fa56bfff700 1 -- 172.21.15.49:0/664774882 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa57800f5d0 con 0x7fa584100770 2024-05-15T16:07:54.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.486+0000 7fa56bfff700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e210 0x7fa56c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:54.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.486+0000 7fa56bfff700 1 -- 172.21.15.49:0/664774882 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa57808f390 con 0x7fa584100770 2024-05-15T16:07:54.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.486+0000 7fa582ffd700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e210 0x7fa56c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:54.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.487+0000 7fa582ffd700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e210 0x7fa56c0706d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa57400ac50 tx=0x7fa57400a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:54.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.491+0000 7fa56bfff700 1 -- 172.21.15.49:0/664774882 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa5780586f0 con 0x7fa584100770 2024-05-15T16:07:54.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:54 smithi049 ceph-mon[112142]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:54.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:54 smithi049 ceph-mon[112142]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:54.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:54 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/489667791' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:07:54.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:54 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1896459384' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:07:54.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.720+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5840fcb50 con 0x7fa56c06e210 2024-05-15T16:07:54.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.721+0000 7fa56bfff700 1 -- 172.21.15.49:0/664774882 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa5840fcb50 con 0x7fa56c06e210 2024-05-15T16:07:54.722 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:07:54.722 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:07:54.723 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e210 msgr2=0x7fa56c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e210 0x7fa56c0706d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa57400ac50 tx=0x7fa57400a380 comp rx=0 tx=0).stop 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 msgr2=0x7fa5840fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5840fc220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa578005b40 tx=0x7fa57800bd70 comp rx=0 tx=0).stop 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 shutdown_connections 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa56c06e210 0x7fa56c0706d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5840ffdc0 0x7fa5840fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 --2- 172.21.15.49:0/664774882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa584100770 0x7fa5840fc220 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 >> 172.21.15.49:0/664774882 conn(0x7fa584076800 msgr2=0x7fa5840688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 shutdown_connections 2024-05-15T16:07:54.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.723+0000 7fa5894fa700 1 -- 172.21.15.49:0/664774882 wait complete. 2024-05-15T16:07:54.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.831+0000 7f86624cf700 1 -- 172.21.15.49:0/3196462036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c1002c0 msgr2=0x7f865c100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.831+0000 7f86624cf700 1 --2- 172.21.15.49:0/3196462036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c1002c0 0x7f865c100740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8650009910 tx=0x7f8650009c20 comp rx=0 tx=0).stop 2024-05-15T16:07:54.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.831+0000 7f86624cf700 1 -- 172.21.15.49:0/3196462036 shutdown_connections 2024-05-15T16:07:54.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.831+0000 7f86624cf700 1 --2- 172.21.15.49:0/3196462036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c1002c0 0x7f865c100740 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.831+0000 7f86624cf700 1 --2- 172.21.15.49:0/3196462036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c105d50 0x7f865c106130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.831+0000 7f86624cf700 1 -- 172.21.15.49:0/3196462036 >> 172.21.15.49:0/3196462036 conn(0x7f865c076000 msgr2=0x7f865c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:54.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.832+0000 7f86624cf700 1 -- 172.21.15.49:0/3196462036 shutdown_connections 2024-05-15T16:07:54.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.832+0000 7f86624cf700 1 -- 172.21.15.49:0/3196462036 wait complete. 2024-05-15T16:07:54.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.832+0000 7f86624cf700 1 Processor -- start 2024-05-15T16:07:54.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.832+0000 7f86624cf700 1 -- start start 2024-05-15T16:07:54.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f86624cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c1002c0 0x7f865c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f86624cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c105d50 0x7f865c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f86624cf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c198630 con 0x7f865c105d50 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f86624cf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c198770 con 0x7f865c1002c0 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f865b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c105d50 0x7f865c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f865b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c105d50 0x7f865c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56150/0 (socket says 172.21.15.49:56150) 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f865b7fe700 1 -- 172.21.15.49:0/4055140630 learned_addr learned my addr 172.21.15.49:0/4055140630 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f865bfff700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c1002c0 0x7f865c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f865b7fe700 1 -- 172.21.15.49:0/4055140630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c1002c0 msgr2=0x7f865c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f865b7fe700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c1002c0 0x7f865c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f865b7fe700 1 -- 172.21.15.49:0/4055140630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500095f0 con 0x7f865c105d50 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f865b7fe700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c105d50 0x7f865c198060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f865000bd70 tx=0x7f865000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f86597fa700 1 -- 172.21.15.49:0/4055140630 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f865001d070 con 0x7f865c105d50 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c1a0b90 con 0x7f865c105d50 2024-05-15T16:07:54.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f86597fa700 1 -- 172.21.15.49:0/4055140630 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8650005700 con 0x7f865c105d50 2024-05-15T16:07:54.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.833+0000 7f86597fa700 1 -- 172.21.15.49:0/4055140630 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8650017400 con 0x7f865c105d50 2024-05-15T16:07:54.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.834+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c1a0ef0 con 0x7f865c105d50 2024-05-15T16:07:54.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.834+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c108200 con 0x7f865c105d50 2024-05-15T16:07:54.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.835+0000 7f86597fa700 1 -- 172.21.15.49:0/4055140630 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8650003a60 con 0x7f865c105d50 2024-05-15T16:07:54.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.836+0000 7f86597fa700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f864406e200 0x7f86440706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:07:54.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.836+0000 7f86597fa700 1 -- 172.21.15.49:0/4055140630 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f865008f650 con 0x7f865c105d50 2024-05-15T16:07:54.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.836+0000 7f865bfff700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f864406e200 0x7f86440706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:07:54.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.836+0000 7f865bfff700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f864406e200 0x7f86440706c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f864c005950 tx=0x7f864c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:07:54.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:54.840+0000 7f86597fa700 1 -- 172.21.15.49:0/4055140630 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8650058930 con 0x7f865c105d50 2024-05-15T16:07:54.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:54 smithi144 ceph-mon[86635]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:54.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:54 smithi144 ceph-mon[86635]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:54.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:54 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/489667791' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:07:54.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:54 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1896459384' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:07:55.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.133+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f865c0ff750 con 0x7f865c105d50 2024-05-15T16:07:55.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.133+0000 7f86597fa700 1 -- 172.21.15.49:0/4055140630 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f865005bff0 con 0x7f865c105d50 2024-05-15T16:07:55.135 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:07:55.135 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:07:55.135 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:07:55.135 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:07:55.135 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:07:55.135 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:07:55.135 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:07:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.135+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f864406e200 msgr2=0x7f86440706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.135+0000 7f86624cf700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f864406e200 0x7f86440706c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f864c005950 tx=0x7f864c00a300 comp rx=0 tx=0).stop 2024-05-15T16:07:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.135+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c105d50 msgr2=0x7f865c198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:07:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.135+0000 7f86624cf700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c105d50 0x7f865c198060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f865000bd70 tx=0x7f865000bda0 comp rx=0 tx=0).stop 2024-05-15T16:07:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.135+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 shutdown_connections 2024-05-15T16:07:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.135+0000 7f86624cf700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f864406e200 0x7f86440706c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.135+0000 7f86624cf700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c1002c0 0x7f865c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:55.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.135+0000 7f86624cf700 1 --2- 172.21.15.49:0/4055140630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f865c105d50 0x7f865c198060 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:07:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.135+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 >> 172.21.15.49:0/4055140630 conn(0x7f865c076000 msgr2=0x7f865c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:07:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.136+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 shutdown_connections 2024-05-15T16:07:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:07:55.136+0000 7f86624cf700 1 -- 172.21.15.49:0/4055140630 wait complete. 2024-05-15T16:07:55.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:55 smithi144 ceph-mon[86635]: pgmap v326: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:55 smithi144 ceph-mon[86635]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:55.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:55 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4055140630' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:07:55.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:55 smithi049 ceph-mon[112142]: pgmap v326: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:55.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:55 smithi049 ceph-mon[112142]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:07:55.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:55 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4055140630' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:07:57.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:57 smithi144 ceph-mon[86635]: pgmap v327: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:57 smithi049 ceph-mon[112142]: pgmap v327: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:07:59 smithi144 ceph-mon[86635]: pgmap v328: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:07:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:07:59 smithi049 ceph-mon[112142]: pgmap v328: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:01.856 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:01 smithi144 ceph-mon[86635]: pgmap v329: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:01.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:01 smithi049 ceph-mon[112142]: pgmap v329: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:03.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:03 smithi144 ceph-mon[86635]: pgmap v330: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:03.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:03 smithi049 ceph-mon[112142]: pgmap v330: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:05.869 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:05 smithi049 ceph-mon[112142]: pgmap v331: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:05.869 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:08:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:05 smithi144 ceph-mon[86635]: pgmap v331: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:08:07.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:07 smithi144 ceph-mon[86635]: pgmap v332: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:07.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:07 smithi049 ceph-mon[112142]: pgmap v332: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:09.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:09 smithi144 ceph-mon[86635]: pgmap v333: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:09.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:09 smithi049 ceph-mon[112142]: pgmap v333: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:11.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:11 smithi144 ceph-mon[86635]: pgmap v334: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:11 smithi049 ceph-mon[112142]: pgmap v334: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:13.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:13 smithi144 ceph-mon[86635]: pgmap v335: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:13.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:13 smithi049 ceph-mon[112142]: pgmap v335: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:15.890 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:15 smithi144 ceph-mon[86635]: pgmap v336: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:15.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:15 smithi049 ceph-mon[112142]: pgmap v336: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:17.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:17 smithi144 ceph-mon[86635]: pgmap v337: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:17.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:17 smithi049 ceph-mon[112142]: pgmap v337: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:19 smithi144 ceph-mon[86635]: pgmap v338: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:19.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:19 smithi049 ceph-mon[112142]: pgmap v338: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:08:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:08:21.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:21 smithi144 ceph-mon[86635]: pgmap v339: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:21.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:21 smithi049 ceph-mon[112142]: pgmap v339: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:23.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:23 smithi144 ceph-mon[86635]: pgmap v340: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:23.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:23 smithi049 ceph-mon[112142]: pgmap v340: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:25.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.259+0000 7f739e5f4700 1 -- 172.21.15.49:0/1408608131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398101f10 msgr2=0x7f73981022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.259+0000 7f739e5f4700 1 --2- 172.21.15.49:0/1408608131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398101f10 0x7f73981022f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7388009910 tx=0x7f7388009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:25.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.259+0000 7f739e5f4700 1 -- 172.21.15.49:0/1408608131 shutdown_connections 2024-05-15T16:08:25.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.259+0000 7f739e5f4700 1 --2- 172.21.15.49:0/1408608131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7398102830 0x7f739810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.259+0000 7f739e5f4700 1 --2- 172.21.15.49:0/1408608131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398101f10 0x7f73981022f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.259+0000 7f739e5f4700 1 -- 172.21.15.49:0/1408608131 >> 172.21.15.49:0/1408608131 conn(0x7f73980fb6a0 msgr2=0x7f73980fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:25.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.259+0000 7f739e5f4700 1 -- 172.21.15.49:0/1408608131 shutdown_connections 2024-05-15T16:08:25.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.259+0000 7f739e5f4700 1 -- 172.21.15.49:0/1408608131 wait complete. 2024-05-15T16:08:25.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.260+0000 7f739e5f4700 1 Processor -- start 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.260+0000 7f739e5f4700 1 -- start start 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.260+0000 7f739e5f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7398101f10 0x7f7398198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.260+0000 7f739e5f4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398102830 0x7f7398199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.260+0000 7f739e5f4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73981999f0 con 0x7f7398102830 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.260+0000 7f739e5f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739819d6c0 con 0x7f7398101f10 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.260+0000 7f73977fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398102830 0x7f7398199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.260+0000 7f73977fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398102830 0x7f7398199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37810/0 (socket says 172.21.15.49:37810) 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.260+0000 7f73977fe700 1 -- 172.21.15.49:0/475703266 learned_addr learned my addr 172.21.15.49:0/475703266 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f7397fff700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7398101f10 0x7f7398198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f73977fe700 1 -- 172.21.15.49:0/475703266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7398101f10 msgr2=0x7f7398198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f73977fe700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7398101f10 0x7f7398198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f73977fe700 1 -- 172.21.15.49:0/475703266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73880095f0 con 0x7f7398102830 2024-05-15T16:08:25.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f73977fe700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398102830 0x7f7398199310 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f738c00d900 tx=0x7f738c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:25.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f73957fa700 1 -- 172.21.15.49:0/475703266 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738c009900 con 0x7f7398102830 2024-05-15T16:08:25.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f73957fa700 1 -- 172.21.15.49:0/475703266 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f738c010460 con 0x7f7398102830 2024-05-15T16:08:25.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739819d9a0 con 0x7f7398102830 2024-05-15T16:08:25.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f73957fa700 1 -- 172.21.15.49:0/475703266 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738c00f5f0 con 0x7f7398102830 2024-05-15T16:08:25.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.261+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739819def0 con 0x7f7398102830 2024-05-15T16:08:25.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.262+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7398069a30 con 0x7f7398102830 2024-05-15T16:08:25.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.263+0000 7f73957fa700 1 -- 172.21.15.49:0/475703266 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f738c0105d0 con 0x7f7398102830 2024-05-15T16:08:25.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.263+0000 7f73957fa700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f738006e210 0x7f73800706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:25.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.263+0000 7f73957fa700 1 -- 172.21.15.49:0/475703266 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f738c08e020 con 0x7f7398102830 2024-05-15T16:08:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.263+0000 7f7397fff700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f738006e210 0x7f73800706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.264+0000 7f7397fff700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f738006e210 0x7f73800706d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f738800b5c0 tx=0x7f738801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:25.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.268+0000 7f73957fa700 1 -- 172.21.15.49:0/475703266 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f738c057300 con 0x7f7398102830 2024-05-15T16:08:25.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.494+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f739810df30 con 0x7f738006e210 2024-05-15T16:08:25.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.495+0000 7f73957fa700 1 -- 172.21.15.49:0/475703266 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f739810df30 con 0x7f738006e210 2024-05-15T16:08:25.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f738006e210 msgr2=0x7f73800706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f738006e210 0x7f73800706d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f738800b5c0 tx=0x7f738801a040 comp rx=0 tx=0).stop 2024-05-15T16:08:25.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398102830 msgr2=0x7f7398199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398102830 0x7f7398199310 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f738c00d900 tx=0x7f738c00dcc0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 shutdown_connections 2024-05-15T16:08:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f738006e210 0x7f73800706d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7398101f10 0x7f7398198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 --2- 172.21.15.49:0/475703266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7398102830 0x7f7398199310 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 >> 172.21.15.49:0/475703266 conn(0x7f73980fb6a0 msgr2=0x7f739810a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 shutdown_connections 2024-05-15T16:08:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.497+0000 7f739e5f4700 1 -- 172.21.15.49:0/475703266 wait complete. 2024-05-15T16:08:25.509 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:08:25.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.607+0000 7f09916a1700 1 -- 172.21.15.49:0/2415249857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c101e20 msgr2=0x7f098c102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.607+0000 7f09916a1700 1 --2- 172.21.15.49:0/2415249857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c101e20 0x7f098c102200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f097c009910 tx=0x7f097c009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:25.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.607+0000 7f09916a1700 1 -- 172.21.15.49:0/2415249857 shutdown_connections 2024-05-15T16:08:25.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.607+0000 7f09916a1700 1 --2- 172.21.15.49:0/2415249857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c102740 0x7f098c10ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.607+0000 7f09916a1700 1 --2- 172.21.15.49:0/2415249857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c101e20 0x7f098c102200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.607+0000 7f09916a1700 1 -- 172.21.15.49:0/2415249857 >> 172.21.15.49:0/2415249857 conn(0x7f098c076800 msgr2=0x7f098c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:25.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.607+0000 7f09916a1700 1 -- 172.21.15.49:0/2415249857 shutdown_connections 2024-05-15T16:08:25.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.607+0000 7f09916a1700 1 -- 172.21.15.49:0/2415249857 wait complete. 2024-05-15T16:08:25.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.608+0000 7f09916a1700 1 Processor -- start 2024-05-15T16:08:25.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.608+0000 7f09916a1700 1 -- start start 2024-05-15T16:08:25.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.608+0000 7f09916a1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c101e20 0x7f098c0ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:25.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.608+0000 7f09916a1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c102740 0x7f098c0fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:25.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.608+0000 7f09916a1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c100500 con 0x7f098c102740 2024-05-15T16:08:25.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.608+0000 7f09916a1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c100640 con 0x7f098c101e20 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f098a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c102740 0x7f098c0fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f098a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c102740 0x7f098c0fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37828/0 (socket says 172.21.15.49:37828) 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f098a7fc700 1 -- 172.21.15.49:0/2109941776 learned_addr learned my addr 172.21.15.49:0/2109941776 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f098affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c101e20 0x7f098c0ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f098a7fc700 1 -- 172.21.15.49:0/2109941776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c101e20 msgr2=0x7f098c0ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f098a7fc700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c101e20 0x7f098c0ffa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f098a7fc700 1 -- 172.21.15.49:0/2109941776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c0095f0 con 0x7f098c102740 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f098a7fc700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c102740 0x7f098c0fffc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f098000b3d0 tx=0x7f098000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f0973fff700 1 -- 172.21.15.49:0/2109941776 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0980011800 con 0x7f098c102740 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c1a2e60 con 0x7f098c102740 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f0973fff700 1 -- 172.21.15.49:0/2109941776 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f098000be90 con 0x7f098c102740 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f0973fff700 1 -- 172.21.15.49:0/2109941776 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0980010e50 con 0x7f098c102740 2024-05-15T16:08:25.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.609+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c1a31c0 con 0x7f098c102740 2024-05-15T16:08:25.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.611+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c068f60 con 0x7f098c102740 2024-05-15T16:08:25.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.612+0000 7f0973fff700 1 -- 172.21.15.49:0/2109941776 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f098001a400 con 0x7f098c102740 2024-05-15T16:08:25.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.612+0000 7f0973fff700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f097406e1a0 0x7f0974070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:25.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.612+0000 7f0973fff700 1 -- 172.21.15.49:0/2109941776 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f098008dc40 con 0x7f098c102740 2024-05-15T16:08:25.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.612+0000 7f098affd700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f097406e1a0 0x7f0974070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:25.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.613+0000 7f098affd700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f097406e1a0 0x7f0974070660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f097c000c00 tx=0x7f097c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:25.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.617+0000 7f0973fff700 1 -- 172.21.15.49:0/2109941776 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0980056fd0 con 0x7f098c102740 2024-05-15T16:08:25.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.845+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f098c10df20 con 0x7f097406e1a0 2024-05-15T16:08:25.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.846+0000 7f0973fff700 1 -- 172.21.15.49:0/2109941776 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f098c10df20 con 0x7f097406e1a0 2024-05-15T16:08:25.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f097406e1a0 msgr2=0x7f0974070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f097406e1a0 0x7f0974070660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f097c000c00 tx=0x7f097c01a040 comp rx=0 tx=0).stop 2024-05-15T16:08:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c102740 msgr2=0x7f098c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c102740 0x7f098c0fffc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f098000b3d0 tx=0x7f098000b6e0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 shutdown_connections 2024-05-15T16:08:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f097406e1a0 0x7f0974070660 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f098c101e20 0x7f098c0ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 --2- 172.21.15.49:0/2109941776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f098c102740 0x7f098c0fffc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 >> 172.21.15.49:0/2109941776 conn(0x7f098c076800 msgr2=0x7f098c1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:25.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 shutdown_connections 2024-05-15T16:08:25.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.848+0000 7f09916a1700 1 -- 172.21.15.49:0/2109941776 wait complete. 2024-05-15T16:08:25.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:25 smithi144 ceph-mon[86635]: pgmap v341: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:25.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:25 smithi049 ceph-mon[112142]: pgmap v341: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 -- 172.21.15.49:0/3212240970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac101890 msgr2=0x7fcfac1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 --2- 172.21.15.49:0/3212240970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac101890 0x7fcfac1058e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009910 tx=0x7fcfa0009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 -- 172.21.15.49:0/3212240970 shutdown_connections 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 --2- 172.21.15.49:0/3212240970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac101890 0x7fcfac1058e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 --2- 172.21.15.49:0/3212240970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac100ee0 0x7fcfac1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 -- 172.21.15.49:0/3212240970 >> 172.21.15.49:0/3212240970 conn(0x7fcfac0fc790 msgr2=0x7fcfac0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 -- 172.21.15.49:0/3212240970 shutdown_connections 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 -- 172.21.15.49:0/3212240970 wait complete. 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 Processor -- start 2024-05-15T16:08:25.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.957+0000 7fcfb26e8700 1 -- start start 2024-05-15T16:08:25.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfb26e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac100ee0 0x7fcfac19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:25.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfb26e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac101890 0x7fcfac19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:25.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfb26e8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac19d440 con 0x7fcfac100ee0 2024-05-15T16:08:25.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfb26e8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac196a20 con 0x7fcfac101890 2024-05-15T16:08:25.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfabfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac100ee0 0x7fcfac19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfabfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac100ee0 0x7fcfac19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37852/0 (socket says 172.21.15.49:37852) 2024-05-15T16:08:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfabfff700 1 -- 172.21.15.49:0/1751699852 learned_addr learned my addr 172.21.15.49:0/1751699852 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfab7fe700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac101890 0x7fcfac19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfabfff700 1 -- 172.21.15.49:0/1751699852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac101890 msgr2=0x7fcfac19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfabfff700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac101890 0x7fcfac19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfabfff700 1 -- 172.21.15.49:0/1751699852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa00095f0 con 0x7fcfac100ee0 2024-05-15T16:08:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.958+0000 7fcfabfff700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac100ee0 0x7fcfac19c8e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c8a0 tx=0x7fcf9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.959+0000 7fcfa97fa700 1 -- 172.21.15.49:0/1751699852 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf9c00cea0 con 0x7fcfac100ee0 2024-05-15T16:08:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.959+0000 7fcfa97fa700 1 -- 172.21.15.49:0/1751699852 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf9c004830 con 0x7fcfac100ee0 2024-05-15T16:08:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.959+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac196d00 con 0x7fcfac100ee0 2024-05-15T16:08:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.959+0000 7fcfa97fa700 1 -- 172.21.15.49:0/1751699852 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf9c0055b0 con 0x7fcfac100ee0 2024-05-15T16:08:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.959+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac197250 con 0x7fcfac100ee0 2024-05-15T16:08:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.960+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac1092e0 con 0x7fcfac100ee0 2024-05-15T16:08:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.960+0000 7fcfa97fa700 1 -- 172.21.15.49:0/1751699852 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcf9c0049a0 con 0x7fcfac100ee0 2024-05-15T16:08:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.961+0000 7fcfa97fa700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf9406e200 0x7fcf940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.961+0000 7fcfa97fa700 1 -- 172.21.15.49:0/1751699852 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcf9c08dd40 con 0x7fcfac100ee0 2024-05-15T16:08:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.961+0000 7fcfab7fe700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf9406e200 0x7fcf940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.961+0000 7fcfab7fe700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf9406e200 0x7fcf940706c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcfac197be0 tx=0x7fcfa001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:25.965+0000 7fcfa97fa700 1 -- 172.21.15.49:0/1751699852 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcf9c05abd0 con 0x7fcfac100ee0 2024-05-15T16:08:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.183+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfac10de80 con 0x7fcf9406e200 2024-05-15T16:08:26.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.193+0000 7fcfa97fa700 1 -- 172.21.15.49:0/1751699852 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcfac10de80 con 0x7fcf9406e200 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (11m) 4m ago 23m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (23m) 4m ago 23m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (22m) 4m ago 22m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (9m) 4m ago 23m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (8m) 4m ago 22m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (11m) 4m ago 22m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (5m) 4m ago 17m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (5m) 4m ago 17m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (4m) 4m ago 17m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (17m) 4m ago 17m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:08:26.196 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (13m) 4m ago 24m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (12m) 4m ago 22m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (10m) 4m ago 24m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (9m) 4m ago 22m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (12m) 4m ago 23m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (12m) 4m ago 22m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (8m) 4m ago 21m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (8m) 4m ago 20m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (7m) 4m ago 20m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (6m) 4m ago 20m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (6m) 4m ago 19m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (5m) 4m ago 19m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:08:26.197 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (11m) 4m ago 22m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf9406e200 msgr2=0x7fcf940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf9406e200 0x7fcf940706c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcfac197be0 tx=0x7fcfa001a040 comp rx=0 tx=0).stop 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac100ee0 msgr2=0x7fcfac19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac100ee0 0x7fcfac19c8e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c8a0 tx=0x7fcf9c00cc60 comp rx=0 tx=0).stop 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 shutdown_connections 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf9406e200 0x7fcf940706c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfac100ee0 0x7fcfac19c8e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 --2- 172.21.15.49:0/1751699852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac101890 0x7fcfac19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 >> 172.21.15.49:0/1751699852 conn(0x7fcfac0fc790 msgr2=0x7fcfac0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 shutdown_connections 2024-05-15T16:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.196+0000 7fcfb26e8700 1 -- 172.21.15.49:0/1751699852 wait complete. 2024-05-15T16:08:26.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.304+0000 7f847715c700 1 -- 172.21.15.49:0/1548334652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84701007c0 msgr2=0x7f8470104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.304+0000 7f847715c700 1 --2- 172.21.15.49:0/1548334652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84701007c0 0x7f8470104810 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f845c009910 tx=0x7f845c009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.304+0000 7f847715c700 1 -- 172.21.15.49:0/1548334652 shutdown_connections 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.304+0000 7f847715c700 1 --2- 172.21.15.49:0/1548334652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84701007c0 0x7f8470104810 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.304+0000 7f847715c700 1 --2- 172.21.15.49:0/1548334652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84700ffe10 0x7f84701001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.304+0000 7f847715c700 1 -- 172.21.15.49:0/1548334652 >> 172.21.15.49:0/1548334652 conn(0x7f84700fb6a0 msgr2=0x7f84700fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.305+0000 7f847715c700 1 -- 172.21.15.49:0/1548334652 shutdown_connections 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.305+0000 7f847715c700 1 -- 172.21.15.49:0/1548334652 wait complete. 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.305+0000 7f847715c700 1 Processor -- start 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.305+0000 7f847715c700 1 -- start start 2024-05-15T16:08:26.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f847715c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84700ffe10 0x7f847019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:26.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f847715c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84701007c0 0x7f847019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:26.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f847715c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847019d460 con 0x7f84700ffe10 2024-05-15T16:08:26.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f847715c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84701969d0 con 0x7f84701007c0 2024-05-15T16:08:26.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f8474ef8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84700ffe10 0x7f847019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:26.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f8474ef8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84700ffe10 0x7f847019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37870/0 (socket says 172.21.15.49:37870) 2024-05-15T16:08:26.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f8474ef8700 1 -- 172.21.15.49:0/3891601237 learned_addr learned my addr 172.21.15.49:0/3891601237 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:26.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f846ffff700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84701007c0 0x7f847019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:26.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f8474ef8700 1 -- 172.21.15.49:0/3891601237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84701007c0 msgr2=0x7f847019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f8474ef8700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84701007c0 0x7f847019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f8474ef8700 1 -- 172.21.15.49:0/3891601237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c0095f0 con 0x7f84700ffe10 2024-05-15T16:08:26.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f846ffff700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84701007c0 0x7f847019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:08:26.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f8474ef8700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84700ffe10 0x7f847019c890 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f846400d8d0 tx=0x7f846400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:26.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f846dffb700 1 -- 172.21.15.49:0/3891601237 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8464009940 con 0x7f84700ffe10 2024-05-15T16:08:26.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f846dffb700 1 -- 172.21.15.49:0/3891601237 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8464010460 con 0x7f84700ffe10 2024-05-15T16:08:26.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8470196cb0 con 0x7f84700ffe10 2024-05-15T16:08:26.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.306+0000 7f846dffb700 1 -- 172.21.15.49:0/3891601237 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f846400f5f0 con 0x7f84700ffe10 2024-05-15T16:08:26.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.307+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8470197200 con 0x7f84700ffe10 2024-05-15T16:08:26.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.308+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84701012f0 con 0x7f84700ffe10 2024-05-15T16:08:26.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.308+0000 7f846dffb700 1 -- 172.21.15.49:0/3891601237 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f84640105d0 con 0x7f84700ffe10 2024-05-15T16:08:26.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.309+0000 7f846dffb700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f846006e210 0x7f84600706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:26.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.309+0000 7f846dffb700 1 -- 172.21.15.49:0/3891601237 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f846408e090 con 0x7f84700ffe10 2024-05-15T16:08:26.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.309+0000 7f846ffff700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f846006e210 0x7f84600706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:26.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.310+0000 7f846ffff700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f846006e210 0x7f84600706d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f845c006010 tx=0x7f845c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:26.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.313+0000 7f846dffb700 1 -- 172.21.15.49:0/3891601237 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f846405af20 con 0x7f84700ffe10 2024-05-15T16:08:26.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.613+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f847010cdb0 con 0x7f84700ffe10 2024-05-15T16:08:26.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.614+0000 7f846dffb700 1 -- 172.21.15.49:0/3891601237 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f846405aab0 con 0x7f84700ffe10 2024-05-15T16:08:26.615 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:08:26.616 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:08:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:08:26.617 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:08:26.617 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f846006e210 msgr2=0x7f84600706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f846006e210 0x7f84600706d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f845c006010 tx=0x7f845c00b540 comp rx=0 tx=0).stop 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84700ffe10 msgr2=0x7f847019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84700ffe10 0x7f847019c890 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f846400d8d0 tx=0x7f846400dc90 comp rx=0 tx=0).stop 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 shutdown_connections 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f846006e210 0x7f84600706d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84700ffe10 0x7f847019c890 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 --2- 172.21.15.49:0/3891601237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84701007c0 0x7f847019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 >> 172.21.15.49:0/3891601237 conn(0x7f84700fb6a0 msgr2=0x7f8470069630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 shutdown_connections 2024-05-15T16:08:26.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.616+0000 7f847715c700 1 -- 172.21.15.49:0/3891601237 wait complete. 2024-05-15T16:08:26.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.725+0000 7f6c0239f700 1 -- 172.21.15.49:0/3858880374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc1025f0 msgr2=0x7f6bfc10aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.725+0000 7f6c0239f700 1 --2- 172.21.15.49:0/3858880374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc1025f0 0x7f6bfc10aae0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009940 tx=0x7f6bf8009c50 comp rx=0 tx=0).stop 2024-05-15T16:08:26.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.725+0000 7f6c0239f700 1 -- 172.21.15.49:0/3858880374 shutdown_connections 2024-05-15T16:08:26.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.725+0000 7f6c0239f700 1 --2- 172.21.15.49:0/3858880374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc1025f0 0x7f6bfc10aae0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.725+0000 7f6c0239f700 1 --2- 172.21.15.49:0/3858880374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc101cd0 0x7f6bfc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.725+0000 7f6c0239f700 1 -- 172.21.15.49:0/3858880374 >> 172.21.15.49:0/3858880374 conn(0x7f6bfc0fb500 msgr2=0x7f6bfc0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:26.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.726+0000 7f6c0239f700 1 -- 172.21.15.49:0/3858880374 shutdown_connections 2024-05-15T16:08:26.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.726+0000 7f6c0239f700 1 -- 172.21.15.49:0/3858880374 wait complete. 2024-05-15T16:08:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.726+0000 7f6c0239f700 1 Processor -- start 2024-05-15T16:08:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.726+0000 7f6c0239f700 1 -- start start 2024-05-15T16:08:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0239f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc101cd0 0x7f6bfc19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0239f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc1025f0 0x7f6bfc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0239f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc198aa0 con 0x7f6bfc101cd0 2024-05-15T16:08:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0239f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc198c10 con 0x7f6bfc1025f0 2024-05-15T16:08:26.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0139d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc101cd0 0x7f6bfc19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0139d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc101cd0 0x7f6bfc19e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37882/0 (socket says 172.21.15.49:37882) 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0139d700 1 -- 172.21.15.49:0/2092251417 learned_addr learned my addr 172.21.15.49:0/2092251417 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c00b9c700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc1025f0 0x7f6bfc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0139d700 1 -- 172.21.15.49:0/2092251417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc1025f0 msgr2=0x7f6bfc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0139d700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc1025f0 0x7f6bfc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0139d700 1 -- 172.21.15.49:0/2092251417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf80095f0 con 0x7f6bfc101cd0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6c0139d700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc101cd0 0x7f6bfc19e9f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000d8d0 tx=0x7f6bf000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6bee7fc700 1 -- 172.21.15.49:0/2092251417 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf00098a0 con 0x7f6bfc101cd0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.727+0000 7f6bee7fc700 1 -- 172.21.15.49:0/2092251417 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bf0010460 con 0x7f6bfc101cd0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.728+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc198ef0 con 0x7f6bfc101cd0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.728+0000 7f6bee7fc700 1 -- 172.21.15.49:0/2092251417 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf000f5f0 con 0x7f6bfc101cd0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.728+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc199410 con 0x7f6bfc101cd0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.728+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc10a640 con 0x7f6bfc101cd0 2024-05-15T16:08:26.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.729+0000 7f6bee7fc700 1 -- 172.21.15.49:0/2092251417 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6bf0009a00 con 0x7f6bfc101cd0 2024-05-15T16:08:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.730+0000 7f6bee7fc700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6be806e210 0x7f6be80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.730+0000 7f6c00b9c700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6be806e210 0x7f6be80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.730+0000 7f6bee7fc700 1 -- 172.21.15.49:0/2092251417 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6bf008df70 con 0x7f6bfc101cd0 2024-05-15T16:08:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.730+0000 7f6c00b9c700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6be806e210 0x7f6be80706d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009910 tx=0x7f6bf8005ad0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:26.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.735+0000 7f6bee7fc700 1 -- 172.21.15.49:0/2092251417 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6bf00c2850 con 0x7f6bfc101cd0 2024-05-15T16:08:26.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:26 smithi144 ceph-mon[86635]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:26.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:26 smithi144 ceph-mon[86635]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:26.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:26 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3891601237' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:08:26.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:26 smithi049 ceph-mon[112142]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:26.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:26 smithi049 ceph-mon[112142]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:26.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:26 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3891601237' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:08:26.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.991+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6bfc199f80 con 0x7f6bfc101cd0 2024-05-15T16:08:26.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.991+0000 7f6bee7fc700 1 -- 172.21.15.49:0/2092251417 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f6bf0020760 con 0x7f6bfc101cd0 2024-05-15T16:08:26.993 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:08:26.993 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:08:26.993 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:08:26.993 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:08:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:08:26.993 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:08:26.993 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:08:26.994 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:08:26.995 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:08:26.996 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:08:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:08:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:08:26.996 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:08:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:08:26.996 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:08:26.996 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:08:26.996 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:08:26.996 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:08:26.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6be806e210 msgr2=0x7f6be80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6be806e210 0x7f6be80706d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009910 tx=0x7f6bf8005ad0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc101cd0 msgr2=0x7f6bfc19e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:26.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc101cd0 0x7f6bfc19e9f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000d8d0 tx=0x7f6bf000dbe0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 shutdown_connections 2024-05-15T16:08:26.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6be806e210 0x7f6be80706d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc101cd0 0x7f6bfc19e9f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 --2- 172.21.15.49:0/2092251417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc1025f0 0x7f6bfc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:26.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 >> 172.21.15.49:0/2092251417 conn(0x7f6bfc0fb500 msgr2=0x7f6bfc0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:26.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.993+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 shutdown_connections 2024-05-15T16:08:26.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:26.994+0000 7f6c0239f700 1 -- 172.21.15.49:0/2092251417 wait complete. 2024-05-15T16:08:26.998 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:08:27.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.104+0000 7f85d5427700 1 -- 172.21.15.49:0/937748891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 msgr2=0x7f85d0104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:27.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.104+0000 7f85d5427700 1 --2- 172.21.15.49:0/937748891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 0x7f85d0104810 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009910 tx=0x7f85c4009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:27.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.104+0000 7f85d5427700 1 -- 172.21.15.49:0/937748891 shutdown_connections 2024-05-15T16:08:27.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.104+0000 7f85d5427700 1 --2- 172.21.15.49:0/937748891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 0x7f85d0104810 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.104+0000 7f85d5427700 1 --2- 172.21.15.49:0/937748891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85d00ffe10 0x7f85d01001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.104+0000 7f85d5427700 1 -- 172.21.15.49:0/937748891 >> 172.21.15.49:0/937748891 conn(0x7f85d00fb700 msgr2=0x7f85d00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.104+0000 7f85d5427700 1 -- 172.21.15.49:0/937748891 shutdown_connections 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.105+0000 7f85d5427700 1 -- 172.21.15.49:0/937748891 wait complete. 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.105+0000 7f85d5427700 1 Processor -- start 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.105+0000 7f85d5427700 1 -- start start 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.105+0000 7f85d5427700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85d00ffe10 0x7f85d019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85d5427700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 0x7f85d019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85d5427700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d019d4d0 con 0x7f85d01007c0 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85d5427700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0196ab0 con 0x7f85d00ffe10 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85ce7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 0x7f85d019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85ce7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 0x7f85d019ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37898/0 (socket says 172.21.15.49:37898) 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85ce7fc700 1 -- 172.21.15.49:0/3246196764 learned_addr learned my addr 172.21.15.49:0/3246196764 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85ceffd700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85d00ffe10 0x7f85d019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85ce7fc700 1 -- 172.21.15.49:0/3246196764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85d00ffe10 msgr2=0x7f85d019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:27.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85ce7fc700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85d00ffe10 0x7f85d019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85ce7fc700 1 -- 172.21.15.49:0/3246196764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40095f0 con 0x7f85d01007c0 2024-05-15T16:08:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85ce7fc700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 0x7f85d019ceb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f85c40092e0 tx=0x7f85c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85b7fff700 1 -- 172.21.15.49:0/3246196764 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85c401d070 con 0x7f85d01007c0 2024-05-15T16:08:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d0196d30 con 0x7f85d01007c0 2024-05-15T16:08:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85b7fff700 1 -- 172.21.15.49:0/3246196764 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85c400f460 con 0x7f85d01007c0 2024-05-15T16:08:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.106+0000 7f85b7fff700 1 -- 172.21.15.49:0/3246196764 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85c4017610 con 0x7f85d01007c0 2024-05-15T16:08:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.107+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d0197220 con 0x7f85d01007c0 2024-05-15T16:08:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.108+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d01012f0 con 0x7f85d01007c0 2024-05-15T16:08:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.108+0000 7f85b7fff700 1 -- 172.21.15.49:0/3246196764 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f85c400f5d0 con 0x7f85d01007c0 2024-05-15T16:08:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.109+0000 7f85b7fff700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85b806e210 0x7f85b80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.109+0000 7f85b7fff700 1 -- 172.21.15.49:0/3246196764 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f85c408f730 con 0x7f85d01007c0 2024-05-15T16:08:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.109+0000 7f85ceffd700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85b806e210 0x7f85b80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.109+0000 7f85ceffd700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85b806e210 0x7f85b80706d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005fd0 tx=0x7f85c000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:27.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.113+0000 7f85b7fff700 1 -- 172.21.15.49:0/3246196764 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f85c405c4a0 con 0x7f85d01007c0 2024-05-15T16:08:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.338+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85d010cdb0 con 0x7f85b806e210 2024-05-15T16:08:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.340+0000 7f85b7fff700 1 -- 172.21.15.49:0/3246196764 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f85d010cdb0 con 0x7f85b806e210 2024-05-15T16:08:27.342 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:08:27.342 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:08:27.342 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:08:27.342 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:08:27.342 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:08:27.343 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:08:27.343 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:08:27.343 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:08:27.343 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:08:27.343 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:08:27.343 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:08:27.343 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:08:27.343 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:08:27.343 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:08:27.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85b806e210 msgr2=0x7f85b80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85b806e210 0x7f85b80706d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005fd0 tx=0x7f85c000a560 comp rx=0 tx=0).stop 2024-05-15T16:08:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 msgr2=0x7f85d019ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 0x7f85d019ceb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f85c40092e0 tx=0x7f85c400be20 comp rx=0 tx=0).stop 2024-05-15T16:08:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 shutdown_connections 2024-05-15T16:08:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85b806e210 0x7f85b80706d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85d00ffe10 0x7f85d019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 --2- 172.21.15.49:0/3246196764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85d01007c0 0x7f85d019ceb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 >> 172.21.15.49:0/3246196764 conn(0x7f85d00fb700 msgr2=0x7f85d0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 shutdown_connections 2024-05-15T16:08:27.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.343+0000 7f85d5427700 1 -- 172.21.15.49:0/3246196764 wait complete. 2024-05-15T16:08:27.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.451+0000 7efdc4736700 1 -- 172.21.15.49:0/42408509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 msgr2=0x7efdbc105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:27.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.451+0000 7efdc4736700 1 --2- 172.21.15.49:0/42408509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 0x7efdbc105900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efdac009910 tx=0x7efdac009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:27.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.451+0000 7efdc4736700 1 -- 172.21.15.49:0/42408509 shutdown_connections 2024-05-15T16:08:27.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.451+0000 7efdc4736700 1 --2- 172.21.15.49:0/42408509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 0x7efdbc105900 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.451+0000 7efdc4736700 1 --2- 172.21.15.49:0/42408509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdbc100f00 0x7efdbc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.451+0000 7efdc4736700 1 -- 172.21.15.49:0/42408509 >> 172.21.15.49:0/42408509 conn(0x7efdbc0fc7d0 msgr2=0x7efdbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.452+0000 7efdc4736700 1 -- 172.21.15.49:0/42408509 shutdown_connections 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.452+0000 7efdc4736700 1 -- 172.21.15.49:0/42408509 wait complete. 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.452+0000 7efdc4736700 1 Processor -- start 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.452+0000 7efdc4736700 1 -- start start 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.452+0000 7efdc4736700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdbc100f00 0x7efdbc198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.452+0000 7efdc4736700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 0x7efdbc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.452+0000 7efdc4736700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc199a60 con 0x7efdbc1018b0 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.452+0000 7efdc4736700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc19d730 con 0x7efdbc100f00 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc1cd1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 0x7efdbc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc1cd1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 0x7efdbc199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37914/0 (socket says 172.21.15.49:37914) 2024-05-15T16:08:27.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc1cd1700 1 -- 172.21.15.49:0/3281959448 learned_addr learned my addr 172.21.15.49:0/3281959448 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:27.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc24d2700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdbc100f00 0x7efdbc198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:27.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc1cd1700 1 -- 172.21.15.49:0/3281959448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdbc100f00 msgr2=0x7efdbc198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:27.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc1cd1700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdbc100f00 0x7efdbc198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc1cd1700 1 -- 172.21.15.49:0/3281959448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac0095f0 con 0x7efdbc1018b0 2024-05-15T16:08:27.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc1cd1700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 0x7efdbc199380 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efdac0092e0 tx=0x7efdac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:27.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdb37fe700 1 -- 172.21.15.49:0/3281959448 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdac01d070 con 0x7efdbc1018b0 2024-05-15T16:08:27.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc19d9b0 con 0x7efdbc1018b0 2024-05-15T16:08:27.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdb37fe700 1 -- 172.21.15.49:0/3281959448 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdac00f460 con 0x7efdbc1018b0 2024-05-15T16:08:27.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdb37fe700 1 -- 172.21.15.49:0/3281959448 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdac017610 con 0x7efdbc1018b0 2024-05-15T16:08:27.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.453+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc19dea0 con 0x7efdbc1018b0 2024-05-15T16:08:27.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.454+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc105080 con 0x7efdbc1018b0 2024-05-15T16:08:27.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.455+0000 7efdb37fe700 1 -- 172.21.15.49:0/3281959448 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efdac00f5d0 con 0x7efdbc1018b0 2024-05-15T16:08:27.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.455+0000 7efdb37fe700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efda806e210 0x7efda80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:27.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.456+0000 7efdb37fe700 1 -- 172.21.15.49:0/3281959448 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efdac08f5b0 con 0x7efdbc1018b0 2024-05-15T16:08:27.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.456+0000 7efdc24d2700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efda806e210 0x7efda80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:27.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.456+0000 7efdc24d2700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efda806e210 0x7efda80706d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005950 tx=0x7efdb800a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:27.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.460+0000 7efdb37fe700 1 -- 172.21.15.49:0/3281959448 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efdac058890 con 0x7efdbc1018b0 2024-05-15T16:08:27.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.754+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efdbc19a150 con 0x7efdbc1018b0 2024-05-15T16:08:27.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.754+0000 7efdb37fe700 1 -- 172.21.15.49:0/3281959448 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7efdac05bf50 con 0x7efdbc1018b0 2024-05-15T16:08:27.756 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:08:27.756 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:08:27.756 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:08:27.756 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:08:27.756 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:08:27.756 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:08:27.756 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:08:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efda806e210 msgr2=0x7efda80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efda806e210 0x7efda80706d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005950 tx=0x7efdb800a300 comp rx=0 tx=0).stop 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 msgr2=0x7efdbc199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 0x7efdbc199380 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efdac0092e0 tx=0x7efdac00bd70 comp rx=0 tx=0).stop 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 shutdown_connections 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efda806e210 0x7efda80706d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdbc100f00 0x7efdbc198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 --2- 172.21.15.49:0/3281959448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1018b0 0x7efdbc199380 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 >> 172.21.15.49:0/3281959448 conn(0x7efdbc0fc7d0 msgr2=0x7efdbc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 shutdown_connections 2024-05-15T16:08:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:27.757+0000 7efdc4736700 1 -- 172.21.15.49:0/3281959448 wait complete. 2024-05-15T16:08:27.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:27 smithi144 ceph-mon[86635]: from='client.34690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:27.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:27 smithi144 ceph-mon[86635]: pgmap v342: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:27.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:27 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2092251417' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:08:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:27 smithi049 ceph-mon[112142]: from='client.34690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:27 smithi049 ceph-mon[112142]: pgmap v342: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:27 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2092251417' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:08:28.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:28 smithi144 ceph-mon[86635]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:28.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:28 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3281959448' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:08:28.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:28 smithi049 ceph-mon[112142]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:28.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:28 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3281959448' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:08:29.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:29 smithi144 ceph-mon[86635]: pgmap v343: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:29.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:29 smithi049 ceph-mon[112142]: pgmap v343: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:31.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:31 smithi144 ceph-mon[86635]: pgmap v344: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:31.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:31 smithi049 ceph-mon[112142]: pgmap v344: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:33.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:33 smithi144 ceph-mon[86635]: pgmap v345: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:33.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:33 smithi049 ceph-mon[112142]: pgmap v345: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:35.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:35 smithi144 ceph-mon[86635]: pgmap v346: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:35.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:08:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:35 smithi049 ceph-mon[112142]: pgmap v346: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:08:37.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:37 smithi144 ceph-mon[86635]: pgmap v347: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:37.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:37 smithi049 ceph-mon[112142]: pgmap v347: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:39.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:39 smithi144 ceph-mon[86635]: pgmap v348: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:39.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:39 smithi049 ceph-mon[112142]: pgmap v348: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:41.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:41 smithi144 ceph-mon[86635]: pgmap v349: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:41.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:41 smithi049 ceph-mon[112142]: pgmap v349: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:43.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:43 smithi144 ceph-mon[86635]: pgmap v350: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:43.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:43 smithi049 ceph-mon[112142]: pgmap v350: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:45.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:45 smithi144 ceph-mon[86635]: pgmap v351: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:45 smithi049 ceph-mon[112142]: pgmap v351: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:47.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:47 smithi144 ceph-mon[86635]: pgmap v352: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:47.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:47 smithi049 ceph-mon[112142]: pgmap v352: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:49.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:49 smithi144 ceph-mon[86635]: pgmap v353: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:49.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:49 smithi049 ceph-mon[112142]: pgmap v353: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:08:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:08:51.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:51 smithi144 ceph-mon[86635]: pgmap v354: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:51.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:51 smithi049 ceph-mon[112142]: pgmap v354: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:53.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:53 smithi144 ceph-mon[86635]: pgmap v355: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:53 smithi049 ceph-mon[112142]: pgmap v355: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:55.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:55 smithi144 ceph-mon[86635]: pgmap v356: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:55.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:55 smithi049 ceph-mon[112142]: pgmap v356: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:57.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:57 smithi144 ceph-mon[86635]: pgmap v357: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.881+0000 7f90c5e61700 1 -- 172.21.15.49:0/1728945277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 msgr2=0x7f90c01047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.881+0000 7f90c5e61700 1 --2- 172.21.15.49:0/1728945277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 0x7f90c01047c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009910 tx=0x7f90a8009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.882+0000 7f90c5e61700 1 -- 172.21.15.49:0/1728945277 shutdown_connections 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.882+0000 7f90c5e61700 1 --2- 172.21.15.49:0/1728945277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 0x7f90c01047c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.882+0000 7f90c5e61700 1 --2- 172.21.15.49:0/1728945277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c00ffdc0 0x7f90c01001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.882+0000 7f90c5e61700 1 -- 172.21.15.49:0/1728945277 >> 172.21.15.49:0/1728945277 conn(0x7f90c0076800 msgr2=0x7f90c00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.882+0000 7f90c5e61700 1 -- 172.21.15.49:0/1728945277 shutdown_connections 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.882+0000 7f90c5e61700 1 -- 172.21.15.49:0/1728945277 wait complete. 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.882+0000 7f90c5e61700 1 Processor -- start 2024-05-15T16:08:57.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.882+0000 7f90c5e61700 1 -- start start 2024-05-15T16:08:57.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90c5e61700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c00ffdc0 0x7f90c019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:57.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90c5e61700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 0x7f90c019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:57.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90c5e61700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c019d3c0 con 0x7f90c0100770 2024-05-15T16:08:57.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90c5e61700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c01969f0 con 0x7f90c00ffdc0 2024-05-15T16:08:57.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90b6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 0x7f90c019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90b6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 0x7f90c019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46334/0 (socket says 172.21.15.49:46334) 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90b6ffd700 1 -- 172.21.15.49:0/1088111639 learned_addr learned my addr 172.21.15.49:0/1088111639 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90bf7fe700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c00ffdc0 0x7f90c019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90b6ffd700 1 -- 172.21.15.49:0/1088111639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c00ffdc0 msgr2=0x7f90c019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90b6ffd700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c00ffdc0 0x7f90c019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90b6ffd700 1 -- 172.21.15.49:0/1088111639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a80095f0 con 0x7f90c0100770 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.883+0000 7f90b6ffd700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 0x7f90c019cda0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f90a8005b40 tx=0x7f90a800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.884+0000 7f90bd7fa700 1 -- 172.21.15.49:0/1088111639 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a801d070 con 0x7f90c0100770 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.884+0000 7f90bd7fa700 1 -- 172.21.15.49:0/1088111639 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90a8005700 con 0x7f90c0100770 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.884+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c0196c70 con 0x7f90c0100770 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.884+0000 7f90bd7fa700 1 -- 172.21.15.49:0/1088111639 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a800fd60 con 0x7f90c0100770 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.884+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c0197160 con 0x7f90c0100770 2024-05-15T16:08:57.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.885+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c01012a0 con 0x7f90c0100770 2024-05-15T16:08:57.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.886+0000 7f90bd7fa700 1 -- 172.21.15.49:0/1088111639 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f90a800f560 con 0x7f90c0100770 2024-05-15T16:08:57.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.886+0000 7f90bd7fa700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f90ac06e210 0x7f90ac0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:57.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.886+0000 7f90bd7fa700 1 -- 172.21.15.49:0/1088111639 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f90a808f5d0 con 0x7f90c0100770 2024-05-15T16:08:57.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.886+0000 7f90bf7fe700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f90ac06e210 0x7f90ac0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:57.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.887+0000 7f90bf7fe700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f90ac06e210 0x7f90ac0706d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009550 tx=0x7f90b00091f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:57.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:57.891+0000 7f90bd7fa700 1 -- 172.21.15.49:0/1088111639 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f90a805c3e0 con 0x7f90c0100770 2024-05-15T16:08:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:57 smithi049 ceph-mon[112142]: pgmap v357: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:58.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.119+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f90c010cd60 con 0x7f90ac06e210 2024-05-15T16:08:58.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.120+0000 7f90bd7fa700 1 -- 172.21.15.49:0/1088111639 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f90c010cd60 con 0x7f90ac06e210 2024-05-15T16:08:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.122+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f90ac06e210 msgr2=0x7f90ac0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.122+0000 7f90c5e61700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f90ac06e210 0x7f90ac0706d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009550 tx=0x7f90b00091f0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.122+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 msgr2=0x7f90c019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.122+0000 7f90c5e61700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 0x7f90c019cda0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f90a8005b40 tx=0x7f90a800bd70 comp rx=0 tx=0).stop 2024-05-15T16:08:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.122+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 shutdown_connections 2024-05-15T16:08:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.122+0000 7f90c5e61700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f90ac06e210 0x7f90ac0706d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.122+0000 7f90c5e61700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c00ffdc0 0x7f90c019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.122+0000 7f90c5e61700 1 --2- 172.21.15.49:0/1088111639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90c0100770 0x7f90c019cda0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.123+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 >> 172.21.15.49:0/1088111639 conn(0x7f90c0076800 msgr2=0x7f90c0068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.123+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 shutdown_connections 2024-05-15T16:08:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.123+0000 7f90c5e61700 1 -- 172.21.15.49:0/1088111639 wait complete. 2024-05-15T16:08:58.135 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:08:58.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.232+0000 7fb927e8a700 1 -- 172.21.15.49:0/1930040463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920101880 msgr2=0x7fb9201058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.232+0000 7fb927e8a700 1 --2- 172.21.15.49:0/1930040463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920101880 0x7fb9201058d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb914009910 tx=0x7fb914009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:58.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.233+0000 7fb927e8a700 1 -- 172.21.15.49:0/1930040463 shutdown_connections 2024-05-15T16:08:58.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.233+0000 7fb927e8a700 1 --2- 172.21.15.49:0/1930040463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920101880 0x7fb9201058d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.233+0000 7fb927e8a700 1 --2- 172.21.15.49:0/1930040463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb920100ed0 0x7fb9201012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.233+0000 7fb927e8a700 1 -- 172.21.15.49:0/1930040463 >> 172.21.15.49:0/1930040463 conn(0x7fb9200fc7a0 msgr2=0x7fb9200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:58.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.233+0000 7fb927e8a700 1 -- 172.21.15.49:0/1930040463 shutdown_connections 2024-05-15T16:08:58.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.233+0000 7fb927e8a700 1 -- 172.21.15.49:0/1930040463 wait complete. 2024-05-15T16:08:58.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.233+0000 7fb927e8a700 1 Processor -- start 2024-05-15T16:08:58.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.233+0000 7fb927e8a700 1 -- start start 2024-05-15T16:08:58.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.233+0000 7fb927e8a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920100ed0 0x7fb92019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:58.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb927e8a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb920101880 0x7fb92019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb927e8a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92019d440 con 0x7fb920100ed0 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb927e8a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb920196a70 con 0x7fb920101880 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb925c26700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920100ed0 0x7fb92019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb925c26700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920100ed0 0x7fb92019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46354/0 (socket says 172.21.15.49:46354) 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb925c26700 1 -- 172.21.15.49:0/2826285190 learned_addr learned my addr 172.21.15.49:0/2826285190 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb925425700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb920101880 0x7fb92019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb925c26700 1 -- 172.21.15.49:0/2826285190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb920101880 msgr2=0x7fb92019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb925c26700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb920101880 0x7fb92019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb925c26700 1 -- 172.21.15.49:0/2826285190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140095f0 con 0x7fb920100ed0 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb925c26700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920100ed0 0x7fb92019c8e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb91000ef10 tx=0x7fb91000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb91effd700 1 -- 172.21.15.49:0/2826285190 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb91000ca50 con 0x7fb920100ed0 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb91effd700 1 -- 172.21.15.49:0/2826285190 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb91000f040 con 0x7fb920100ed0 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb920196cf0 con 0x7fb920100ed0 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb91effd700 1 -- 172.21.15.49:0/2826285190 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9100146b0 con 0x7fb920100ed0 2024-05-15T16:08:58.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.234+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb920197240 con 0x7fb920100ed0 2024-05-15T16:08:58.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.235+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9201092d0 con 0x7fb920100ed0 2024-05-15T16:08:58.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.236+0000 7fb91effd700 1 -- 172.21.15.49:0/2826285190 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb91000cbb0 con 0x7fb920100ed0 2024-05-15T16:08:58.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.236+0000 7fb91effd700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb90806e200 0x7fb9080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:58.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.236+0000 7fb91effd700 1 -- 172.21.15.49:0/2826285190 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb91008e590 con 0x7fb920100ed0 2024-05-15T16:08:58.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.236+0000 7fb925425700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb90806e200 0x7fb9080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:58.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.237+0000 7fb925425700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb90806e200 0x7fb9080706c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb914009fd0 tx=0x7fb914005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:58.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.241+0000 7fb91effd700 1 -- 172.21.15.49:0/2826285190 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb91005b3a0 con 0x7fb920100ed0 2024-05-15T16:08:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.470+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb920197f50 con 0x7fb90806e200 2024-05-15T16:08:58.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.471+0000 7fb91effd700 1 -- 172.21.15.49:0/2826285190 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb920197f50 con 0x7fb90806e200 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb90806e200 msgr2=0x7fb9080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb90806e200 0x7fb9080706c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb914009fd0 tx=0x7fb914005940 comp rx=0 tx=0).stop 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920100ed0 msgr2=0x7fb92019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920100ed0 0x7fb92019c8e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb91000ef10 tx=0x7fb91000c3a0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 shutdown_connections 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb90806e200 0x7fb9080706c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb920100ed0 0x7fb92019c8e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 --2- 172.21.15.49:0/2826285190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb920101880 0x7fb92019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 >> 172.21.15.49:0/2826285190 conn(0x7fb9200fc7a0 msgr2=0x7fb9200fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 shutdown_connections 2024-05-15T16:08:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.473+0000 7fb927e8a700 1 -- 172.21.15.49:0/2826285190 wait complete. 2024-05-15T16:08:58.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.582+0000 7f7c0255f700 1 -- 172.21.15.49:0/2651655102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc100ef0 msgr2=0x7f7bfc1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.582+0000 7f7c0255f700 1 --2- 172.21.15.49:0/2651655102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc100ef0 0x7f7bfc1012d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009910 tx=0x7f7bec009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.582+0000 7f7c0255f700 1 -- 172.21.15.49:0/2651655102 shutdown_connections 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.582+0000 7f7c0255f700 1 --2- 172.21.15.49:0/2651655102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bfc1018a0 0x7f7bfc1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.582+0000 7f7c0255f700 1 --2- 172.21.15.49:0/2651655102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc100ef0 0x7f7bfc1012d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.582+0000 7f7c0255f700 1 -- 172.21.15.49:0/2651655102 >> 172.21.15.49:0/2651655102 conn(0x7f7bfc0fc7c0 msgr2=0x7f7bfc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.582+0000 7f7c0255f700 1 -- 172.21.15.49:0/2651655102 shutdown_connections 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.582+0000 7f7c0255f700 1 -- 172.21.15.49:0/2651655102 wait complete. 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.583+0000 7f7c0255f700 1 Processor -- start 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.583+0000 7f7c0255f700 1 -- start start 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.583+0000 7f7c0255f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bfc100ef0 0x7f7bfc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:58.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.583+0000 7f7c0255f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc1018a0 0x7f7bfc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.583+0000 7f7c0255f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc073d20 con 0x7f7bfc1018a0 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.583+0000 7f7c0255f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc073e60 con 0x7f7bfc100ef0 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.583+0000 7f7bfb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc1018a0 0x7f7bfc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.583+0000 7f7bfb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc1018a0 0x7f7bfc0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46366/0 (socket says 172.21.15.49:46366) 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.583+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/3022747398 learned_addr learned my addr 172.21.15.49:0/3022747398 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7bfbfff700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bfc100ef0 0x7f7bfc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/3022747398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bfc100ef0 msgr2=0x7f7bfc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7bfb7fe700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bfc100ef0 0x7f7bfc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/3022747398 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0095f0 con 0x7f7bfc1018a0 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7bfb7fe700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc1018a0 0x7f7bfc0737e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000b480 tx=0x7f7bf000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7bf97fa700 1 -- 172.21.15.49:0/3022747398 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf0011800 con 0x7f7bfc1018a0 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7bf97fa700 1 -- 172.21.15.49:0/3022747398 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf0011e20 con 0x7f7bfc1018a0 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc1a2eb0 con 0x7f7bfc1018a0 2024-05-15T16:08:58.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7bf97fa700 1 -- 172.21.15.49:0/3022747398 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf0010ea0 con 0x7f7bfc1018a0 2024-05-15T16:08:58.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.584+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc1a3210 con 0x7f7bfc1018a0 2024-05-15T16:08:58.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.585+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc1023d0 con 0x7f7bfc1018a0 2024-05-15T16:08:58.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.586+0000 7f7bf97fa700 1 -- 172.21.15.49:0/3022747398 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7bf0010380 con 0x7f7bfc1018a0 2024-05-15T16:08:58.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.586+0000 7f7bf97fa700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7be406e210 0x7f7be40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:58.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.586+0000 7f7bf97fa700 1 -- 172.21.15.49:0/3022747398 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7bf008dc60 con 0x7f7bfc1018a0 2024-05-15T16:08:58.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.586+0000 7f7bfbfff700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7be406e210 0x7f7be40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:58.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.587+0000 7f7bfbfff700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7be406e210 0x7f7be40706d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006010 tx=0x7f7bec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:58.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.591+0000 7f7bf97fa700 1 -- 172.21.15.49:0/3022747398 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7bf0056f40 con 0x7f7bfc1018a0 2024-05-15T16:08:58.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.807+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bfc10dfa0 con 0x7f7be406e210 2024-05-15T16:08:58.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.819+0000 7f7bf97fa700 1 -- 172.21.15.49:0/3022747398 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7bfc10dfa0 con 0x7f7be406e210 2024-05-15T16:08:58.821 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:08:58.821 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (12m) 5m ago 23m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:08:58.821 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (24m) 5m ago 24m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:08:58.821 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (22m) 5m ago 22m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:08:58.821 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (9m) 5m ago 23m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (9m) 5m ago 22m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (11m) 5m ago 23m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (5m) 5m ago 18m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (5m) 5m ago 18m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (5m) 5m ago 18m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (18m) 5m ago 18m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (14m) 5m ago 25m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (13m) 5m ago 22m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (10m) 5m ago 25m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (10m) 5m ago 22m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (13m) 5m ago 23m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (12m) 5m ago 22m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (9m) 5m ago 21m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:08:58.822 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (8m) 5m ago 21m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:08:58.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (7m) 5m ago 20m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:08:58.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (7m) 5m ago 20m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:08:58.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (6m) 5m ago 20m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:08:58.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (6m) 5m ago 19m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:08:58.823 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (12m) 5m ago 23m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:08:58.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7be406e210 msgr2=0x7f7be40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7be406e210 0x7f7be40706d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006010 tx=0x7f7bec01a040 comp rx=0 tx=0).stop 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc1018a0 msgr2=0x7f7bfc0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc1018a0 0x7f7bfc0737e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000b480 tx=0x7f7bf000b790 comp rx=0 tx=0).stop 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 shutdown_connections 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7be406e210 0x7f7be40706d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bfc100ef0 0x7f7bfc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 --2- 172.21.15.49:0/3022747398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bfc1018a0 0x7f7bfc0737e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 >> 172.21.15.49:0/3022747398 conn(0x7f7bfc0fc7c0 msgr2=0x7f7bfc1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 shutdown_connections 2024-05-15T16:08:58.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.822+0000 7f7c0255f700 1 -- 172.21.15.49:0/3022747398 wait complete. 2024-05-15T16:08:58.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.929+0000 7f28bbd20700 1 -- 172.21.15.49:0/602630437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 msgr2=0x7f28b410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.929+0000 7f28bbd20700 1 --2- 172.21.15.49:0/602630437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 0x7f28b410d070 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009910 tx=0x7f28a4009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.929+0000 7f28bbd20700 1 -- 172.21.15.49:0/602630437 shutdown_connections 2024-05-15T16:08:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.929+0000 7f28bbd20700 1 --2- 172.21.15.49:0/602630437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 0x7f28b410d070 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.929+0000 7f28bbd20700 1 --2- 172.21.15.49:0/602630437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b40ffa00 0x7f28b40ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.929+0000 7f28bbd20700 1 -- 172.21.15.49:0/602630437 >> 172.21.15.49:0/602630437 conn(0x7f28b40fb700 msgr2=0x7f28b40fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.929+0000 7f28bbd20700 1 -- 172.21.15.49:0/602630437 shutdown_connections 2024-05-15T16:08:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.929+0000 7f28bbd20700 1 -- 172.21.15.49:0/602630437 wait complete. 2024-05-15T16:08:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28bbd20700 1 Processor -- start 2024-05-15T16:08:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28bbd20700 1 -- start start 2024-05-15T16:08:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28bbd20700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b40ffa00 0x7f28b4198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28bbd20700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 0x7f28b41993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28bbd20700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b4199a90 con 0x7f28b41003b0 2024-05-15T16:08:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28bbd20700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b419d760 con 0x7f28b40ffa00 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28b92bb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 0x7f28b41993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28b92bb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 0x7f28b41993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46372/0 (socket says 172.21.15.49:46372) 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28b92bb700 1 -- 172.21.15.49:0/3775949181 learned_addr learned my addr 172.21.15.49:0/3775949181 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28b9abc700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b40ffa00 0x7f28b4198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28b92bb700 1 -- 172.21.15.49:0/3775949181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b40ffa00 msgr2=0x7f28b4198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28b92bb700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b40ffa00 0x7f28b4198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28b92bb700 1 -- 172.21.15.49:0/3775949181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40095f0 con 0x7f28b41003b0 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.930+0000 7f28b9abc700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b40ffa00 0x7f28b4198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.931+0000 7f28b92bb700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 0x7f28b41993b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f28a4006010 tx=0x7f28a40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.931+0000 7f28aaffd700 1 -- 172.21.15.49:0/3775949181 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a401d070 con 0x7f28b41003b0 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.931+0000 7f28aaffd700 1 -- 172.21.15.49:0/3775949181 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28a400bea0 con 0x7f28b41003b0 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.931+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b419d9e0 con 0x7f28b41003b0 2024-05-15T16:08:58.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.931+0000 7f28aaffd700 1 -- 172.21.15.49:0/3775949181 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a400fdc0 con 0x7f28b41003b0 2024-05-15T16:08:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.931+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b419df30 con 0x7f28b41003b0 2024-05-15T16:08:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.932+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b410aef0 con 0x7f28b41003b0 2024-05-15T16:08:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.932+0000 7f28aaffd700 1 -- 172.21.15.49:0/3775949181 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f28a400f5f0 con 0x7f28b41003b0 2024-05-15T16:08:58.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.933+0000 7f28aaffd700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f28a006e210 0x7f28a00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:58.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.933+0000 7f28aaffd700 1 -- 172.21.15.49:0/3775949181 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f28a408f360 con 0x7f28b41003b0 2024-05-15T16:08:58.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.933+0000 7f28b9abc700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f28a006e210 0x7f28a00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:58.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.933+0000 7f28b9abc700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f28a006e210 0x7f28a00706d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28b0005fd0 tx=0x7f28b0005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:58.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:58.938+0000 7f28aaffd700 1 -- 172.21.15.49:0/3775949181 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f28a4058640 con 0x7f28b41003b0 2024-05-15T16:08:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.233+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f28b419a180 con 0x7f28b41003b0 2024-05-15T16:08:59.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.234+0000 7f28aaffd700 1 -- 172.21.15.49:0/3775949181 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f28a405bd00 con 0x7f28b41003b0 2024-05-15T16:08:59.236 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:08:59.236 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:08:59.236 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:08:59.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:08:59.236 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:08:59.237 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:08:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f28a006e210 msgr2=0x7f28a00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f28a006e210 0x7f28a00706d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28b0005fd0 tx=0x7f28b0005ee0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 msgr2=0x7f28b41993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 0x7f28b41993b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f28a4006010 tx=0x7f28a40056a0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 shutdown_connections 2024-05-15T16:08:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f28a006e210 0x7f28a00706d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b40ffa00 0x7f28b4198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 --2- 172.21.15.49:0/3775949181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b41003b0 0x7f28b41993b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 >> 172.21.15.49:0/3775949181 conn(0x7f28b40fb700 msgr2=0x7f28b4108030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 shutdown_connections 2024-05-15T16:08:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.237+0000 7f28bbd20700 1 -- 172.21.15.49:0/3775949181 wait complete. 2024-05-15T16:08:59.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.345+0000 7effb5bc1700 1 -- 172.21.15.49:0/3835265415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00fff10 msgr2=0x7effb0103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.346+0000 7effb5bc1700 1 --2- 172.21.15.49:0/3835265415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00fff10 0x7effb0103f60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7effa4009910 tx=0x7effa4009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:59.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.346+0000 7effb5bc1700 1 -- 172.21.15.49:0/3835265415 shutdown_connections 2024-05-15T16:08:59.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.346+0000 7effb5bc1700 1 --2- 172.21.15.49:0/3835265415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00fff10 0x7effb0103f60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.346+0000 7effb5bc1700 1 --2- 172.21.15.49:0/3835265415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00ff560 0x7effb00ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.346+0000 7effb5bc1700 1 -- 172.21.15.49:0/3835265415 >> 172.21.15.49:0/3835265415 conn(0x7effb0075ff0 msgr2=0x7effb00fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.346+0000 7effb5bc1700 1 -- 172.21.15.49:0/3835265415 shutdown_connections 2024-05-15T16:08:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.346+0000 7effb5bc1700 1 -- 172.21.15.49:0/3835265415 wait complete. 2024-05-15T16:08:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.346+0000 7effb5bc1700 1 Processor -- start 2024-05-15T16:08:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.346+0000 7effb5bc1700 1 -- start start 2024-05-15T16:08:59.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effb5bc1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00ff560 0x7effb019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:59.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effb5bc1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00fff10 0x7effb019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:59.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effb5bc1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb0196ae0 con 0x7effb00ff560 2024-05-15T16:08:59.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effb5bc1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb0196c50 con 0x7effb00fff10 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effaf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00ff560 0x7effb019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effaf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00ff560 0x7effb019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46382/0 (socket says 172.21.15.49:46382) 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effaf7fe700 1 -- 172.21.15.49:0/2215230532 learned_addr learned my addr 172.21.15.49:0/2215230532 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effaeffd700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00fff10 0x7effb019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effaf7fe700 1 -- 172.21.15.49:0/2215230532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00fff10 msgr2=0x7effb019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effaf7fe700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00fff10 0x7effb019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effaf7fe700 1 -- 172.21.15.49:0/2215230532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa40095f0 con 0x7effb00ff560 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.347+0000 7effaf7fe700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00ff560 0x7effb019ca30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7effa000b4b0 tx=0x7effa000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.348+0000 7effacff9700 1 -- 172.21.15.49:0/2215230532 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effa0011800 con 0x7effb00ff560 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.348+0000 7effacff9700 1 -- 172.21.15.49:0/2215230532 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effa0011e20 con 0x7effb00ff560 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.348+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb0196f30 con 0x7effb00ff560 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.348+0000 7effacff9700 1 -- 172.21.15.49:0/2215230532 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effa000f2e0 con 0x7effb00ff560 2024-05-15T16:08:59.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.348+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb0197450 con 0x7effb00ff560 2024-05-15T16:08:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.349+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb0100a40 con 0x7effb00ff560 2024-05-15T16:08:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.349+0000 7effacff9700 1 -- 172.21.15.49:0/2215230532 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7effa0010380 con 0x7effb00ff560 2024-05-15T16:08:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.350+0000 7effacff9700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff9806e210 0x7eff980706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.350+0000 7effacff9700 1 -- 172.21.15.49:0/2215230532 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7effa008dd00 con 0x7effb00ff560 2024-05-15T16:08:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.350+0000 7effaeffd700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff9806e210 0x7eff980706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:59.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.350+0000 7effaeffd700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff9806e210 0x7eff980706d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7effa4009fd0 tx=0x7effa401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.355+0000 7effacff9700 1 -- 172.21.15.49:0/2215230532 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7effa0091030 con 0x7effb00ff560 2024-05-15T16:08:59.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.606+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7effb0197ff0 con 0x7effb00ff560 2024-05-15T16:08:59.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.607+0000 7effacff9700 1 -- 172.21.15.49:0/2215230532 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7effa0056fe0 con 0x7effb00ff560 2024-05-15T16:08:59.609 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:08:59.609 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:08:59.609 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:08:59.609 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:08:59.609 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:08:59.609 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:08:59.609 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:08:59.610 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:08:59.611 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:08:59.612 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:08:59.612 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:08:59.612 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:08:59.612 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:08:59.612 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.610+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff9806e210 msgr2=0x7eff980706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.610+0000 7effb5bc1700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff9806e210 0x7eff980706d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7effa4009fd0 tx=0x7effa401a040 comp rx=0 tx=0).stop 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.610+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00ff560 msgr2=0x7effb019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.610+0000 7effb5bc1700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00ff560 0x7effb019ca30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7effa000b4b0 tx=0x7effa000b7c0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.610+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 shutdown_connections 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.610+0000 7effb5bc1700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff9806e210 0x7eff980706d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.610+0000 7effb5bc1700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effb00ff560 0x7effb019ca30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.610+0000 7effb5bc1700 1 --2- 172.21.15.49:0/2215230532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00fff10 0x7effb019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.610+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 >> 172.21.15.49:0/2215230532 conn(0x7effb0075ff0 msgr2=0x7effb00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.611+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 shutdown_connections 2024-05-15T16:08:59.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.611+0000 7effb5bc1700 1 -- 172.21.15.49:0/2215230532 wait complete. 2024-05-15T16:08:59.614 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:08:59.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.722+0000 7fd59b448700 1 -- 172.21.15.49:0/4193369995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 msgr2=0x7fd5941047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.722+0000 7fd59b448700 1 --2- 172.21.15.49:0/4193369995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 0x7fd5941047d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd584009910 tx=0x7fd584009c20 comp rx=0 tx=0).stop 2024-05-15T16:08:59.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.722+0000 7fd59b448700 1 -- 172.21.15.49:0/4193369995 shutdown_connections 2024-05-15T16:08:59.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.722+0000 7fd59b448700 1 --2- 172.21.15.49:0/4193369995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 0x7fd5941047d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.722+0000 7fd59b448700 1 --2- 172.21.15.49:0/4193369995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5940ffdd0 0x7fd5941001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.722+0000 7fd59b448700 1 -- 172.21.15.49:0/4193369995 >> 172.21.15.49:0/4193369995 conn(0x7fd5940fb6a0 msgr2=0x7fd5940fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:59.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.722+0000 7fd59b448700 1 -- 172.21.15.49:0/4193369995 shutdown_connections 2024-05-15T16:08:59.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.722+0000 7fd59b448700 1 -- 172.21.15.49:0/4193369995 wait complete. 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.723+0000 7fd59b448700 1 Processor -- start 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.723+0000 7fd59b448700 1 -- start start 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.723+0000 7fd59b448700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5940ffdd0 0x7fd59419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.723+0000 7fd59b448700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 0x7fd59419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.723+0000 7fd59b448700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59419d510 con 0x7fd594100780 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.723+0000 7fd59b448700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd594196af0 con 0x7fd5940ffdd0 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.723+0000 7fd5989e3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 0x7fd59419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.723+0000 7fd5989e3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 0x7fd59419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46406/0 (socket says 172.21.15.49:46406) 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.723+0000 7fd5989e3700 1 -- 172.21.15.49:0/284624447 learned_addr learned my addr 172.21.15.49:0/284624447 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd5991e4700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5940ffdd0 0x7fd59419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd5989e3700 1 -- 172.21.15.49:0/284624447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5940ffdd0 msgr2=0x7fd59419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd5989e3700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5940ffdd0 0x7fd59419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd5989e3700 1 -- 172.21.15.49:0/284624447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840095f0 con 0x7fd594100780 2024-05-15T16:08:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd5989e3700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 0x7fd59419cef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd58400bd70 tx=0x7fd58400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd58a7fc700 1 -- 172.21.15.49:0/284624447 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd58401d070 con 0x7fd594100780 2024-05-15T16:08:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd58a7fc700 1 -- 172.21.15.49:0/284624447 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5840056c0 con 0x7fd594100780 2024-05-15T16:08:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd594196d70 con 0x7fd594100780 2024-05-15T16:08:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd58a7fc700 1 -- 172.21.15.49:0/284624447 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd584017400 con 0x7fd594100780 2024-05-15T16:08:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.724+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd594197260 con 0x7fd594100780 2024-05-15T16:08:59.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.725+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd594108120 con 0x7fd594100780 2024-05-15T16:08:59.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.726+0000 7fd58a7fc700 1 -- 172.21.15.49:0/284624447 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd584003a60 con 0x7fd594100780 2024-05-15T16:08:59.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.726+0000 7fd58a7fc700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd58006e200 0x7fd5800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:08:59.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.726+0000 7fd58a7fc700 1 -- 172.21.15.49:0/284624447 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd58408f4a0 con 0x7fd594100780 2024-05-15T16:08:59.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.727+0000 7fd5991e4700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd58006e200 0x7fd5800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:08:59.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.727+0000 7fd5991e4700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd58006e200 0x7fd5800706c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd590009a10 tx=0x7fd590009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:08:59.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.731+0000 7fd58a7fc700 1 -- 172.21.15.49:0/284624447 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd58405c330 con 0x7fd594100780 2024-05-15T16:08:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:59 smithi144 ceph-mon[86635]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:59 smithi144 ceph-mon[86635]: pgmap v358: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:59.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:59 smithi144 ceph-mon[86635]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:59.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:59 smithi144 ceph-mon[86635]: from='client.34718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:59.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:59 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3775949181' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:08:59.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:08:59 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2215230532' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:08:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:59 smithi049 ceph-mon[112142]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:59 smithi049 ceph-mon[112142]: pgmap v358: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:08:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:59 smithi049 ceph-mon[112142]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:59 smithi049 ceph-mon[112142]: from='client.34718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:08:59.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:59 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3775949181' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:08:59.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:08:59 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2215230532' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:08:59.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.958+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd594198020 con 0x7fd58006e200 2024-05-15T16:08:59.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.960+0000 7fd58a7fc700 1 -- 172.21.15.49:0/284624447 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd594198020 con 0x7fd58006e200 2024-05-15T16:08:59.962 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:08:59.962 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:08:59.962 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:08:59.962 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:08:59.963 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:08:59.963 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:08:59.963 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:08:59.963 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:08:59.963 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:08:59.963 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:08:59.964 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:08:59.964 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:08:59.964 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:08:59.964 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:08:59.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd58006e200 msgr2=0x7fd5800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd58006e200 0x7fd5800706c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd590009a10 tx=0x7fd590009190 comp rx=0 tx=0).stop 2024-05-15T16:08:59.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 msgr2=0x7fd59419cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:08:59.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 0x7fd59419cef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd58400bd70 tx=0x7fd58400bda0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 shutdown_connections 2024-05-15T16:08:59.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd58006e200 0x7fd5800706c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5940ffdd0 0x7fd59419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 --2- 172.21.15.49:0/284624447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd594100780 0x7fd59419cef0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:08:59.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 >> 172.21.15.49:0/284624447 conn(0x7fd5940fb6a0 msgr2=0x7fd594109fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:08:59.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 shutdown_connections 2024-05-15T16:08:59.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:08:59.962+0000 7fd59b448700 1 -- 172.21.15.49:0/284624447 wait complete. 2024-05-15T16:09:00.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 -- 172.21.15.49:0/1712437867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e8101ed0 msgr2=0x7f45e81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:00.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 --2- 172.21.15.49:0/1712437867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e8101ed0 0x7f45e81022b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009910 tx=0x7f45e4009c20 comp rx=0 tx=0).stop 2024-05-15T16:09:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 -- 172.21.15.49:0/1712437867 shutdown_connections 2024-05-15T16:09:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 --2- 172.21.15.49:0/1712437867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e81027f0 0x7f45e810ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 --2- 172.21.15.49:0/1712437867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e8101ed0 0x7f45e81022b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 -- 172.21.15.49:0/1712437867 >> 172.21.15.49:0/1712437867 conn(0x7f45e80fb6a0 msgr2=0x7f45e80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 -- 172.21.15.49:0/1712437867 shutdown_connections 2024-05-15T16:09:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 -- 172.21.15.49:0/1712437867 wait complete. 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 Processor -- start 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.071+0000 7f45ef7ab700 1 -- start start 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ef7ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e8101ed0 0x7f45e819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ef7ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e81027f0 0x7f45e819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ef7ab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e819d480 con 0x7f45e81027f0 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ef7ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e8196a60 con 0x7f45e8101ed0 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ecd46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e81027f0 0x7f45e819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ecd46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e81027f0 0x7f45e819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46428/0 (socket says 172.21.15.49:46428) 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ecd46700 1 -- 172.21.15.49:0/3698610699 learned_addr learned my addr 172.21.15.49:0/3698610699 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:09:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ed547700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e8101ed0 0x7f45e819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ecd46700 1 -- 172.21.15.49:0/3698610699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e8101ed0 msgr2=0x7f45e819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ecd46700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e8101ed0 0x7f45e819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ecd46700 1 -- 172.21.15.49:0/3698610699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e40095f0 con 0x7f45e81027f0 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.072+0000 7f45ecd46700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e81027f0 0x7f45e819ce60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c8f0 tx=0x7f45d800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.073+0000 7f45de7fc700 1 -- 172.21.15.49:0/3698610699 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d80078a0 con 0x7f45e81027f0 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.073+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e8196d40 con 0x7f45e81027f0 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.073+0000 7f45de7fc700 1 -- 172.21.15.49:0/3698610699 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45d800f450 con 0x7f45e81027f0 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.073+0000 7f45de7fc700 1 -- 172.21.15.49:0/3698610699 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d8018610 con 0x7f45e81027f0 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.073+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e8197290 con 0x7f45e81027f0 2024-05-15T16:09:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.073+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e81055f0 con 0x7f45e81027f0 2024-05-15T16:09:00.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.074+0000 7f45de7fc700 1 -- 172.21.15.49:0/3698610699 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f45d800f5c0 con 0x7f45e81027f0 2024-05-15T16:09:00.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.075+0000 7f45de7fc700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f45d406e200 0x7f45d40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:00.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.075+0000 7f45de7fc700 1 -- 172.21.15.49:0/3698610699 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f45d808e0e0 con 0x7f45e81027f0 2024-05-15T16:09:00.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.075+0000 7f45ed547700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f45d406e200 0x7f45d40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:00.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.075+0000 7f45ed547700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f45d406e200 0x7f45d40706c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000c00 tx=0x7f45e401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:00.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.079+0000 7f45de7fc700 1 -- 172.21.15.49:0/3698610699 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f45d805aef0 con 0x7f45e81027f0 2024-05-15T16:09:00.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.371+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f45e8069a30 con 0x7f45e81027f0 2024-05-15T16:09:00.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.371+0000 7f45de7fc700 1 -- 172.21.15.49:0/3698610699 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f45d805aa80 con 0x7f45e81027f0 2024-05-15T16:09:00.373 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:09:00.373 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:09:00.373 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:09:00.373 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:09:00.373 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:09:00.373 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:09:00.373 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:09:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.374+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f45d406e200 msgr2=0x7f45d40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.374+0000 7f45ef7ab700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f45d406e200 0x7f45d40706c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000c00 tx=0x7f45e401a040 comp rx=0 tx=0).stop 2024-05-15T16:09:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.374+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e81027f0 msgr2=0x7f45e819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:00.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.374+0000 7f45ef7ab700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e81027f0 0x7f45e819ce60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c8f0 tx=0x7f45d800cc00 comp rx=0 tx=0).stop 2024-05-15T16:09:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.375+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 shutdown_connections 2024-05-15T16:09:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.375+0000 7f45ef7ab700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f45d406e200 0x7f45d40706c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.375+0000 7f45ef7ab700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e8101ed0 0x7f45e819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.375+0000 7f45ef7ab700 1 --2- 172.21.15.49:0/3698610699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45e81027f0 0x7f45e819ce60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.375+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 >> 172.21.15.49:0/3698610699 conn(0x7f45e80fb6a0 msgr2=0x7f45e81031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.375+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 shutdown_connections 2024-05-15T16:09:00.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:00.375+0000 7f45ef7ab700 1 -- 172.21.15.49:0/3698610699 wait complete. 2024-05-15T16:09:00.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:00 smithi144 ceph-mon[86635]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:00.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:00 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3698610699' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:09:00.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:00 smithi049 ceph-mon[112142]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:00.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:00 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3698610699' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:09:01.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:01 smithi144 ceph-mon[86635]: pgmap v359: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:01.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:01 smithi049 ceph-mon[112142]: pgmap v359: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:03.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:03 smithi144 ceph-mon[86635]: pgmap v360: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:03.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:03 smithi049 ceph-mon[112142]: pgmap v360: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:05.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:05 smithi144 ceph-mon[86635]: pgmap v361: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:05.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:09:05.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:05 smithi049 ceph-mon[112142]: pgmap v361: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:05.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:09:07.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:07 smithi144 ceph-mon[86635]: pgmap v362: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:07.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:07 smithi049 ceph-mon[112142]: pgmap v362: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:09.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:09 smithi049 ceph-mon[112142]: pgmap v363: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:09 smithi144 ceph-mon[86635]: pgmap v363: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:11 smithi049 ceph-mon[112142]: pgmap v364: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:11 smithi144 ceph-mon[86635]: pgmap v364: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:13 smithi144 ceph-mon[86635]: pgmap v365: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:13 smithi049 ceph-mon[112142]: pgmap v365: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:15 smithi144 ceph-mon[86635]: pgmap v366: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:15 smithi049 ceph-mon[112142]: pgmap v366: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:17.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:17 smithi144 ceph-mon[86635]: pgmap v367: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:17 smithi049 ceph-mon[112142]: pgmap v367: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:19 smithi144 ceph-mon[86635]: pgmap v368: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:19 smithi049 ceph-mon[112142]: pgmap v368: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:21.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:09:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:09:21.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:21 smithi049 ceph-mon[112142]: pgmap v369: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:21 smithi144 ceph-mon[86635]: pgmap v369: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:23 smithi144 ceph-mon[86635]: pgmap v370: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:23 smithi049 ceph-mon[112142]: pgmap v370: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:25 smithi144 ceph-mon[86635]: pgmap v371: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:25 smithi049 ceph-mon[112142]: pgmap v371: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:27 smithi144 ceph-mon[86635]: pgmap v372: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:27 smithi049 ceph-mon[112142]: pgmap v372: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:30.001 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:29 smithi144 ceph-mon[86635]: pgmap v373: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:30.005 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:29 smithi049 ceph-mon[112142]: pgmap v373: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:30.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.499+0000 7f3525e22700 1 -- 172.21.15.49:0/3729629684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3520101ea0 msgr2=0x7f3520102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:30.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.499+0000 7f3525e22700 1 --2- 172.21.15.49:0/3729629684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3520101ea0 0x7f3520102280 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f351000b0d0 tx=0x7f351000b3e0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.499+0000 7f3525e22700 1 -- 172.21.15.49:0/3729629684 shutdown_connections 2024-05-15T16:09:30.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.499+0000 7f3525e22700 1 --2- 172.21.15.49:0/3729629684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35201027c0 0x7f352010acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.499+0000 7f3525e22700 1 --2- 172.21.15.49:0/3729629684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3520101ea0 0x7f3520102280 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.499+0000 7f3525e22700 1 -- 172.21.15.49:0/3729629684 >> 172.21.15.49:0/3729629684 conn(0x7f3520076800 msgr2=0x7f35200fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:30.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.499+0000 7f3525e22700 1 -- 172.21.15.49:0/3729629684 shutdown_connections 2024-05-15T16:09:30.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.500+0000 7f3525e22700 1 -- 172.21.15.49:0/3729629684 wait complete. 2024-05-15T16:09:30.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.500+0000 7f3525e22700 1 Processor -- start 2024-05-15T16:09:30.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.500+0000 7f3525e22700 1 -- start start 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f3525e22700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3520101ea0 0x7f352019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f3525e22700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35201027c0 0x7f352019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f3525e22700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019d400 con 0x7f35201027c0 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f3525e22700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35201969e0 con 0x7f3520101ea0 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f350effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35201027c0 0x7f352019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f350effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35201027c0 0x7f352019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46302/0 (socket says 172.21.15.49:46302) 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f350effd700 1 -- 172.21.15.49:0/4251407952 learned_addr learned my addr 172.21.15.49:0/4251407952 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f351f7fe700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3520101ea0 0x7f352019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f350effd700 1 -- 172.21.15.49:0/4251407952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3520101ea0 msgr2=0x7f352019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f350effd700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3520101ea0 0x7f352019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f350effd700 1 -- 172.21.15.49:0/4251407952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3510009d00 con 0x7f35201027c0 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.501+0000 7f350effd700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35201027c0 0x7f352019cde0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f350800d8d0 tx=0x7f350800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.502+0000 7f351d7fa700 1 -- 172.21.15.49:0/4251407952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35080041f0 con 0x7f35201027c0 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.502+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3520196cc0 con 0x7f35201027c0 2024-05-15T16:09:30.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.502+0000 7f351d7fa700 1 -- 172.21.15.49:0/4251407952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3508007620 con 0x7f35201027c0 2024-05-15T16:09:30.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.502+0000 7f351d7fa700 1 -- 172.21.15.49:0/4251407952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35080056b0 con 0x7f35201027c0 2024-05-15T16:09:30.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.502+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3520197210 con 0x7f35201027c0 2024-05-15T16:09:30.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.503+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35201055c0 con 0x7f35201027c0 2024-05-15T16:09:30.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.503+0000 7f351d7fa700 1 -- 172.21.15.49:0/4251407952 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f350800f770 con 0x7f35201027c0 2024-05-15T16:09:30.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.504+0000 7f351d7fa700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f350006e200 0x7f35000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:30.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.504+0000 7f351d7fa700 1 -- 172.21.15.49:0/4251407952 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f350808dfa0 con 0x7f35201027c0 2024-05-15T16:09:30.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.504+0000 7f351f7fe700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f350006e200 0x7f35000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:30.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.504+0000 7f351f7fe700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f350006e200 0x7f35000706c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f351000b820 tx=0x7f3510006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:30.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.509+0000 7f351d7fa700 1 -- 172.21.15.49:0/4251407952 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f350805adb0 con 0x7f35201027c0 2024-05-15T16:09:30.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.736+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f352010dea0 con 0x7f350006e200 2024-05-15T16:09:30.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.737+0000 7f351d7fa700 1 -- 172.21.15.49:0/4251407952 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f352010dea0 con 0x7f350006e200 2024-05-15T16:09:30.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f350006e200 msgr2=0x7f35000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:30.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f350006e200 0x7f35000706c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f351000b820 tx=0x7f3510006000 comp rx=0 tx=0).stop 2024-05-15T16:09:30.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35201027c0 msgr2=0x7f352019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:30.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35201027c0 0x7f352019cde0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f350800d8d0 tx=0x7f350800dbe0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 shutdown_connections 2024-05-15T16:09:30.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f350006e200 0x7f35000706c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3520101ea0 0x7f352019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 --2- 172.21.15.49:0/4251407952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35201027c0 0x7f352019cde0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 >> 172.21.15.49:0/4251407952 conn(0x7f3520076800 msgr2=0x7f3520103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:30.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 shutdown_connections 2024-05-15T16:09:30.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.740+0000 7f3525e22700 1 -- 172.21.15.49:0/4251407952 wait complete. 2024-05-15T16:09:30.752 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:09:30.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.850+0000 7efe08f8c700 1 -- 172.21.15.49:0/2956628520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe041027f0 msgr2=0x7efe0410ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:30.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.850+0000 7efe08f8c700 1 --2- 172.21.15.49:0/2956628520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe041027f0 0x7efe0410ace0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009940 tx=0x7efdf8009c50 comp rx=0 tx=0).stop 2024-05-15T16:09:30.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.850+0000 7efe08f8c700 1 -- 172.21.15.49:0/2956628520 shutdown_connections 2024-05-15T16:09:30.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.850+0000 7efe08f8c700 1 --2- 172.21.15.49:0/2956628520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe041027f0 0x7efe0410ace0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.850+0000 7efe08f8c700 1 --2- 172.21.15.49:0/2956628520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe04101ed0 0x7efe041022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.850+0000 7efe08f8c700 1 -- 172.21.15.49:0/2956628520 >> 172.21.15.49:0/2956628520 conn(0x7efe040fb700 msgr2=0x7efe040fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:30.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.850+0000 7efe08f8c700 1 -- 172.21.15.49:0/2956628520 shutdown_connections 2024-05-15T16:09:30.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.850+0000 7efe08f8c700 1 -- 172.21.15.49:0/2956628520 wait complete. 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe08f8c700 1 Processor -- start 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe08f8c700 1 -- start start 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe08f8c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe04101ed0 0x7efe0419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe08f8c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe041027f0 0x7efe0419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe08f8c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0419d580 con 0x7efe04101ed0 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe08f8c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe04196af0 con 0x7efe041027f0 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe0259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe04101ed0 0x7efe0419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe0259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe04101ed0 0x7efe0419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46306/0 (socket says 172.21.15.49:46306) 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe0259c700 1 -- 172.21.15.49:0/855472407 learned_addr learned my addr 172.21.15.49:0/855472407 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe01d9b700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe041027f0 0x7efe0419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe0259c700 1 -- 172.21.15.49:0/855472407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe041027f0 msgr2=0x7efe0419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe0259c700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe041027f0 0x7efe0419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.851+0000 7efe0259c700 1 -- 172.21.15.49:0/855472407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf80095f0 con 0x7efe04101ed0 2024-05-15T16:09:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.852+0000 7efe01d9b700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe041027f0 0x7efe0419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:09:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.852+0000 7efe0259c700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe04101ed0 0x7efe0419c9b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efdf400d8d0 tx=0x7efdf400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.852+0000 7efdf37fe700 1 -- 172.21.15.49:0/855472407 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf4009940 con 0x7efe04101ed0 2024-05-15T16:09:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.852+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe04196dd0 con 0x7efe04101ed0 2024-05-15T16:09:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.852+0000 7efdf37fe700 1 -- 172.21.15.49:0/855472407 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdf4010460 con 0x7efe04101ed0 2024-05-15T16:09:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.852+0000 7efdf37fe700 1 -- 172.21.15.49:0/855472407 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf400f5f0 con 0x7efe04101ed0 2024-05-15T16:09:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.852+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe04197320 con 0x7efe04101ed0 2024-05-15T16:09:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.853+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe04069a50 con 0x7efe04101ed0 2024-05-15T16:09:30.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.854+0000 7efdf37fe700 1 -- 172.21.15.49:0/855472407 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efdf40105d0 con 0x7efe04101ed0 2024-05-15T16:09:30.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.854+0000 7efdf37fe700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdec06e200 0x7efdec0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:30.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.854+0000 7efe01d9b700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdec06e200 0x7efdec0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:30.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.854+0000 7efdf37fe700 1 -- 172.21.15.49:0/855472407 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efdf408e1a0 con 0x7efe04101ed0 2024-05-15T16:09:30.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.855+0000 7efe01d9b700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdec06e200 0x7efdec0706c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efdf80059d0 tx=0x7efdf8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:30.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:30.859+0000 7efdf37fe700 1 -- 172.21.15.49:0/855472407 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efdf405afb0 con 0x7efe04101ed0 2024-05-15T16:09:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.088+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe0410df90 con 0x7efdec06e200 2024-05-15T16:09:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.090+0000 7efdf37fe700 1 -- 172.21.15.49:0/855472407 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efe0410df90 con 0x7efdec06e200 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdec06e200 msgr2=0x7efdec0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdec06e200 0x7efdec0706c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efdf80059d0 tx=0x7efdf8005960 comp rx=0 tx=0).stop 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe04101ed0 msgr2=0x7efe0419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe04101ed0 0x7efe0419c9b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efdf400d8d0 tx=0x7efdf400dc90 comp rx=0 tx=0).stop 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 shutdown_connections 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdec06e200 0x7efdec0706c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe04101ed0 0x7efe0419c9b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 --2- 172.21.15.49:0/855472407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe041027f0 0x7efe0419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 >> 172.21.15.49:0/855472407 conn(0x7efe040fb700 msgr2=0x7efe041031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 shutdown_connections 2024-05-15T16:09:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.092+0000 7efe08f8c700 1 -- 172.21.15.49:0/855472407 wait complete. 2024-05-15T16:09:31.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.201+0000 7fbb6a32b700 1 -- 172.21.15.49:0/2632802631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 msgr2=0x7fbb64102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.201+0000 7fbb6a32b700 1 --2- 172.21.15.49:0/2632802631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 0x7fbb64102200 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009910 tx=0x7fbb54009c20 comp rx=0 tx=0).stop 2024-05-15T16:09:31.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.201+0000 7fbb6a32b700 1 -- 172.21.15.49:0/2632802631 shutdown_connections 2024-05-15T16:09:31.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.201+0000 7fbb6a32b700 1 --2- 172.21.15.49:0/2632802631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb64102740 0x7fbb6410ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.201+0000 7fbb6a32b700 1 --2- 172.21.15.49:0/2632802631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 0x7fbb64102200 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.201+0000 7fbb6a32b700 1 -- 172.21.15.49:0/2632802631 >> 172.21.15.49:0/2632802631 conn(0x7fbb64076800 msgr2=0x7fbb640fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:31.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.201+0000 7fbb6a32b700 1 -- 172.21.15.49:0/2632802631 shutdown_connections 2024-05-15T16:09:31.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.201+0000 7fbb6a32b700 1 -- 172.21.15.49:0/2632802631 wait complete. 2024-05-15T16:09:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb6a32b700 1 Processor -- start 2024-05-15T16:09:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb6a32b700 1 -- start start 2024-05-15T16:09:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb6a32b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 0x7fbb640ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb6a32b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb64102740 0x7fbb640fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb6a32b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64100500 con 0x7fbb64101e20 2024-05-15T16:09:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb6a32b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64100640 con 0x7fbb64102740 2024-05-15T16:09:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb63fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 0x7fbb640ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb63fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 0x7fbb640ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46330/0 (socket says 172.21.15.49:46330) 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb63fff700 1 -- 172.21.15.49:0/3686441200 learned_addr learned my addr 172.21.15.49:0/3686441200 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb5bfff700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb64102740 0x7fbb640fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb63fff700 1 -- 172.21.15.49:0/3686441200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb64102740 msgr2=0x7fbb640fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb63fff700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb64102740 0x7fbb640fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.202+0000 7fbb63fff700 1 -- 172.21.15.49:0/3686441200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540095f0 con 0x7fbb64101e20 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.203+0000 7fbb63fff700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 0x7fbb640ffa80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb54000c00 tx=0x7fbb5400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.203+0000 7fbb61ffb700 1 -- 172.21.15.49:0/3686441200 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5401d070 con 0x7fbb64101e20 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.203+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb641008c0 con 0x7fbb64101e20 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.203+0000 7fbb61ffb700 1 -- 172.21.15.49:0/3686441200 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb540056e0 con 0x7fbb64101e20 2024-05-15T16:09:31.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.203+0000 7fbb61ffb700 1 -- 172.21.15.49:0/3686441200 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5400fe80 con 0x7fbb64101e20 2024-05-15T16:09:31.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.203+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb641a3000 con 0x7fbb64101e20 2024-05-15T16:09:31.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.204+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb64068f60 con 0x7fbb64101e20 2024-05-15T16:09:31.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.204+0000 7fbb61ffb700 1 -- 172.21.15.49:0/3686441200 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbb5400f6f0 con 0x7fbb64101e20 2024-05-15T16:09:31.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.205+0000 7fbb61ffb700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5006e210 0x7fbb500706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:31.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.205+0000 7fbb61ffb700 1 -- 172.21.15.49:0/3686441200 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbb5408f5f0 con 0x7fbb64101e20 2024-05-15T16:09:31.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.205+0000 7fbb5bfff700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5006e210 0x7fbb500706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:31.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.205+0000 7fbb5bfff700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5006e210 0x7fbb500706d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c005950 tx=0x7fbb4c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.210+0000 7fbb61ffb700 1 -- 172.21.15.49:0/3686441200 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbb540588d0 con 0x7fbb64101e20 2024-05-15T16:09:31.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.424+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb6410dea0 con 0x7fbb5006e210 2024-05-15T16:09:31.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.436+0000 7fbb61ffb700 1 -- 172.21.15.49:0/3686441200 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbb6410dea0 con 0x7fbb5006e210 2024-05-15T16:09:31.437 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (12m) 6m ago 24m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (24m) 6m ago 24m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (23m) 5m ago 23m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (10m) 6m ago 24m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (10m) 5m ago 23m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (12m) 6m ago 24m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (6m) 6m ago 18m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (6m) 6m ago 18m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (5m) 5m ago 18m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (18m) 5m ago 18m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (15m) 6m ago 25m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (13m) 5m ago 23m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:09:31.438 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (11m) 6m ago 25m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (10m) 5m ago 23m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (13m) 6m ago 24m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (13m) 5m ago 23m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (9m) 6m ago 22m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (9m) 6m ago 21m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (8m) 6m ago 21m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (7m) 5m ago 21m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (7m) 5m ago 20m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (6m) 5m ago 20m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:09:31.439 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (13m) 6m ago 23m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:09:31.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.438+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5006e210 msgr2=0x7fbb500706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.438+0000 7fbb6a32b700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5006e210 0x7fbb500706d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c005950 tx=0x7fbb4c0058e0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.438+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 msgr2=0x7fbb640ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.438+0000 7fbb6a32b700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 0x7fbb640ffa80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb54000c00 tx=0x7fbb5400be20 comp rx=0 tx=0).stop 2024-05-15T16:09:31.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.438+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 shutdown_connections 2024-05-15T16:09:31.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.438+0000 7fbb6a32b700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5006e210 0x7fbb500706d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.438+0000 7fbb6a32b700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb64101e20 0x7fbb640ffa80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.438+0000 7fbb6a32b700 1 --2- 172.21.15.49:0/3686441200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb64102740 0x7fbb640fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.438+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 >> 172.21.15.49:0/3686441200 conn(0x7fbb64076800 msgr2=0x7fbb64103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:31.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.439+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 shutdown_connections 2024-05-15T16:09:31.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.439+0000 7fbb6a32b700 1 -- 172.21.15.49:0/3686441200 wait complete. 2024-05-15T16:09:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.546+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/2027452931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb08100770 msgr2=0x7ffb081047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.546+0000 7ffb0f3a0700 1 --2- 172.21.15.49:0/2027452931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb08100770 0x7ffb081047c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009910 tx=0x7ffaf8009c20 comp rx=0 tx=0).stop 2024-05-15T16:09:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.546+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/2027452931 shutdown_connections 2024-05-15T16:09:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.546+0000 7ffb0f3a0700 1 --2- 172.21.15.49:0/2027452931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb08100770 0x7ffb081047c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.546+0000 7ffb0f3a0700 1 --2- 172.21.15.49:0/2027452931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb080ffdc0 0x7ffb081001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.546+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/2027452931 >> 172.21.15.49:0/2027452931 conn(0x7ffb08076800 msgr2=0x7ffb080fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.546+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/2027452931 shutdown_connections 2024-05-15T16:09:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.546+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/2027452931 wait complete. 2024-05-15T16:09:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0f3a0700 1 Processor -- start 2024-05-15T16:09:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0f3a0700 1 -- start start 2024-05-15T16:09:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0f3a0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb080ffdc0 0x7ffb080fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0f3a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb08100770 0x7ffb080fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0f3a0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080fc760 con 0x7ffb080ffdc0 2024-05-15T16:09:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0f3a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080fc8a0 con 0x7ffb08100770 2024-05-15T16:09:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0d13c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb080ffdc0 0x7ffb080fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0d13c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb080ffdc0 0x7ffb080fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46358/0 (socket says 172.21.15.49:46358) 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0d13c700 1 -- 172.21.15.49:0/3548306190 learned_addr learned my addr 172.21.15.49:0/3548306190 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0c93b700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb08100770 0x7ffb080fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0d13c700 1 -- 172.21.15.49:0/3548306190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb08100770 msgr2=0x7ffb080fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0d13c700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb08100770 0x7ffb080fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.547+0000 7ffb0d13c700 1 -- 172.21.15.49:0/3548306190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf80095f0 con 0x7ffb080ffdc0 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.548+0000 7ffb0d13c700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb080ffdc0 0x7ffb080fbce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400e920 tx=0x7ffb0400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.548+0000 7ffafe7fc700 1 -- 172.21.15.49:0/3548306190 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb04009790 con 0x7ffb080ffdc0 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.548+0000 7ffafe7fc700 1 -- 172.21.15.49:0/3548306190 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb0400cba0 con 0x7ffb080ffdc0 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.548+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb081a2d70 con 0x7ffb080ffdc0 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.548+0000 7ffafe7fc700 1 -- 172.21.15.49:0/3548306190 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb04018650 con 0x7ffb080ffdc0 2024-05-15T16:09:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.548+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb081a30d0 con 0x7ffb080ffdc0 2024-05-15T16:09:31.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.549+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb081012a0 con 0x7ffb080ffdc0 2024-05-15T16:09:31.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.549+0000 7ffafe7fc700 1 -- 172.21.15.49:0/3548306190 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffb0400cd10 con 0x7ffb080ffdc0 2024-05-15T16:09:31.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.550+0000 7ffafe7fc700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffaf406e210 0x7ffaf40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:31.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.550+0000 7ffafe7fc700 1 -- 172.21.15.49:0/3548306190 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ffb04014070 con 0x7ffb080ffdc0 2024-05-15T16:09:31.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.550+0000 7ffb0c93b700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffaf406e210 0x7ffaf40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:31.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.550+0000 7ffb0c93b700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffaf406e210 0x7ffaf40706d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800b760 tx=0x7ffaf8009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:31.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.555+0000 7ffafe7fc700 1 -- 172.21.15.49:0/3548306190 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ffb04053850 con 0x7ffb080ffdc0 2024-05-15T16:09:31.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.850+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffb0810ce20 con 0x7ffb080ffdc0 2024-05-15T16:09:31.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.851+0000 7ffafe7fc700 1 -- 172.21.15.49:0/3548306190 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7ffb0405b340 con 0x7ffb080ffdc0 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:09:31.853 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:09:31.854 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:09:31.854 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:09:31.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:09:31.854 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:09:31.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffaf406e210 msgr2=0x7ffaf40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffaf406e210 0x7ffaf40706d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800b760 tx=0x7ffaf8009f90 comp rx=0 tx=0).stop 2024-05-15T16:09:31.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb080ffdc0 msgr2=0x7ffb080fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb080ffdc0 0x7ffb080fbce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400e920 tx=0x7ffb0400ece0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 shutdown_connections 2024-05-15T16:09:31.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffaf406e210 0x7ffaf40706d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb080ffdc0 0x7ffb080fbce0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 --2- 172.21.15.49:0/3548306190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb08100770 0x7ffb080fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 >> 172.21.15.49:0/3548306190 conn(0x7ffb08076800 msgr2=0x7ffb08068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:31.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 shutdown_connections 2024-05-15T16:09:31.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.854+0000 7ffb0f3a0700 1 -- 172.21.15.49:0/3548306190 wait complete. 2024-05-15T16:09:31.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.962+0000 7f7db7fde700 1 -- 172.21.15.49:0/3265998030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 msgr2=0x7f7db010ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.962+0000 7f7db7fde700 1 --2- 172.21.15.49:0/3265998030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 0x7f7db010ac30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009910 tx=0x7f7da0009c20 comp rx=0 tx=0).stop 2024-05-15T16:09:31.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.962+0000 7f7db7fde700 1 -- 172.21.15.49:0/3265998030 shutdown_connections 2024-05-15T16:09:31.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.962+0000 7f7db7fde700 1 --2- 172.21.15.49:0/3265998030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 0x7f7db010ac30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.962+0000 7f7db7fde700 1 --2- 172.21.15.49:0/3265998030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db0101e20 0x7f7db0102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.962+0000 7f7db7fde700 1 -- 172.21.15.49:0/3265998030 >> 172.21.15.49:0/3265998030 conn(0x7f7db0076800 msgr2=0x7f7db00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:31.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.962+0000 7f7db7fde700 1 -- 172.21.15.49:0/3265998030 shutdown_connections 2024-05-15T16:09:31.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.962+0000 7f7db7fde700 1 -- 172.21.15.49:0/3265998030 wait complete. 2024-05-15T16:09:31.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db7fde700 1 Processor -- start 2024-05-15T16:09:31.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db7fde700 1 -- start start 2024-05-15T16:09:31.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db7fde700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db0101e20 0x7f7db00ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:31.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db7fde700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 0x7f7db00fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:31.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db7fde700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0100500 con 0x7f7db0102740 2024-05-15T16:09:31.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db7fde700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0100640 con 0x7f7db0101e20 2024-05-15T16:09:31.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db5579700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 0x7f7db00fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:31.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db5579700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 0x7f7db00fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46374/0 (socket says 172.21.15.49:46374) 2024-05-15T16:09:31.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db5579700 1 -- 172.21.15.49:0/1554933870 learned_addr learned my addr 172.21.15.49:0/1554933870 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:09:31.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db5d7a700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db0101e20 0x7f7db00ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:31.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db5579700 1 -- 172.21.15.49:0/1554933870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db0101e20 msgr2=0x7f7db00ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:31.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db5579700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db0101e20 0x7f7db00ffa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:31.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db5579700 1 -- 172.21.15.49:0/1554933870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da00095f0 con 0x7f7db0102740 2024-05-15T16:09:31.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.963+0000 7f7db5d7a700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db0101e20 0x7f7db00ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:09:31.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.964+0000 7f7db5579700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 0x7f7db00fffc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7da00092e0 tx=0x7f7da000bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:31.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.964+0000 7f7da6ffd700 1 -- 172.21.15.49:0/1554933870 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da001d070 con 0x7f7db0102740 2024-05-15T16:09:31.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.964+0000 7f7da6ffd700 1 -- 172.21.15.49:0/1554933870 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7da0005dc0 con 0x7f7db0102740 2024-05-15T16:09:31.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.964+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db01008c0 con 0x7f7db0102740 2024-05-15T16:09:31.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.964+0000 7f7da6ffd700 1 -- 172.21.15.49:0/1554933870 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da000fe20 con 0x7f7db0102740 2024-05-15T16:09:31.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.964+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db01a3040 con 0x7f7db0102740 2024-05-15T16:09:31.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.965+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db0068f60 con 0x7f7db0102740 2024-05-15T16:09:31.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.965+0000 7f7da6ffd700 1 -- 172.21.15.49:0/1554933870 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7da0003850 con 0x7f7db0102740 2024-05-15T16:09:31.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.966+0000 7f7da6ffd700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d9c06e200 0x7f7d9c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:31.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.966+0000 7f7da6ffd700 1 -- 172.21.15.49:0/1554933870 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7da008f3e0 con 0x7f7db0102740 2024-05-15T16:09:31.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.966+0000 7f7db5d7a700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d9c06e200 0x7f7d9c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:31.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.966+0000 7f7db5d7a700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d9c06e200 0x7f7d9c0706c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7dac005fd0 tx=0x7f7dac005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:31.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:31.971+0000 7f7da6ffd700 1 -- 172.21.15.49:0/1554933870 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7da0058740 con 0x7f7db0102740 2024-05-15T16:09:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:31 smithi144 ceph-mon[86635]: pgmap v374: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:31 smithi144 ceph-mon[86635]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:31 smithi049 ceph-mon[112142]: pgmap v374: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:31 smithi049 ceph-mon[112142]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:32.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.230+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7db010de80 con 0x7f7db0102740 2024-05-15T16:09:32.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.230+0000 7f7da6ffd700 1 -- 172.21.15.49:0/1554933870 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f7da005be00 con 0x7f7db0102740 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:09:32.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d9c06e200 msgr2=0x7f7d9c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d9c06e200 0x7f7d9c0706c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7dac005fd0 tx=0x7f7dac005ee0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 msgr2=0x7f7db00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 0x7f7db00fffc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7da00092e0 tx=0x7f7da000bbf0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 shutdown_connections 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d9c06e200 0x7f7d9c0706c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db0101e20 0x7f7db00ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 --2- 172.21.15.49:0/1554933870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db0102740 0x7f7db00fffc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 >> 172.21.15.49:0/1554933870 conn(0x7f7db0076800 msgr2=0x7f7db0103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 shutdown_connections 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.232+0000 7f7db7fde700 1 -- 172.21.15.49:0/1554933870 wait complete. 2024-05-15T16:09:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:09:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.342+0000 7f92bd6c8700 1 -- 172.21.15.49:0/1869788763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 msgr2=0x7f92b810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.342+0000 7f92bd6c8700 1 --2- 172.21.15.49:0/1869788763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 0x7f92b810ac40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009910 tx=0x7f92a0009c20 comp rx=0 tx=0).stop 2024-05-15T16:09:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.342+0000 7f92bd6c8700 1 -- 172.21.15.49:0/1869788763 shutdown_connections 2024-05-15T16:09:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.342+0000 7f92bd6c8700 1 --2- 172.21.15.49:0/1869788763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 0x7f92b810ac40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.342+0000 7f92bd6c8700 1 --2- 172.21.15.49:0/1869788763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b8101da0 0x7f92b8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.342+0000 7f92bd6c8700 1 -- 172.21.15.49:0/1869788763 >> 172.21.15.49:0/1869788763 conn(0x7f92b80fb6a0 msgr2=0x7f92b80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.342+0000 7f92bd6c8700 1 -- 172.21.15.49:0/1869788763 shutdown_connections 2024-05-15T16:09:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.342+0000 7f92bd6c8700 1 -- 172.21.15.49:0/1869788763 wait complete. 2024-05-15T16:09:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92bd6c8700 1 Processor -- start 2024-05-15T16:09:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92bd6c8700 1 -- start start 2024-05-15T16:09:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92bd6c8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b8101da0 0x7f92b819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92bd6c8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 0x7f92b819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92bd6c8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b819d410 con 0x7f92b8102750 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92bd6c8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81969f0 con 0x7f92b8101da0 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92b67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 0x7f92b819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92b67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 0x7f92b819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46380/0 (socket says 172.21.15.49:46380) 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92b67fc700 1 -- 172.21.15.49:0/937272800 learned_addr learned my addr 172.21.15.49:0/937272800 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92b6ffd700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b8101da0 0x7f92b819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92b67fc700 1 -- 172.21.15.49:0/937272800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b8101da0 msgr2=0x7f92b819c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.343+0000 7f92b67fc700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b8101da0 0x7f92b819c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.344+0000 7f92b67fc700 1 -- 172.21.15.49:0/937272800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a00095f0 con 0x7f92b8102750 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.344+0000 7f92b67fc700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 0x7f92b819cdf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009fd0 tx=0x7f92a000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.344+0000 7f92affff700 1 -- 172.21.15.49:0/937272800 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92a001d070 con 0x7f92b8102750 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.344+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b8196c70 con 0x7f92b8102750 2024-05-15T16:09:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.344+0000 7f92affff700 1 -- 172.21.15.49:0/937272800 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92a00052a0 con 0x7f92b8102750 2024-05-15T16:09:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.344+0000 7f92affff700 1 -- 172.21.15.49:0/937272800 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92a0022470 con 0x7f92b8102750 2024-05-15T16:09:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.344+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b8197160 con 0x7f92b8102750 2024-05-15T16:09:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.345+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b81055c0 con 0x7f92b8102750 2024-05-15T16:09:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.346+0000 7f92affff700 1 -- 172.21.15.49:0/937272800 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f92a000f770 con 0x7f92b8102750 2024-05-15T16:09:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.346+0000 7f92affff700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92a406e200 0x7f92a40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.346+0000 7f92affff700 1 -- 172.21.15.49:0/937272800 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f92a008f550 con 0x7f92b8102750 2024-05-15T16:09:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.346+0000 7f92b6ffd700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92a406e200 0x7f92a40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.347+0000 7f92b6ffd700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92a406e200 0x7f92a40706c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009380 tx=0x7f92a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:32.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.351+0000 7f92affff700 1 -- 172.21.15.49:0/937272800 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f92a005c360 con 0x7f92b8102750 2024-05-15T16:09:32.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.577+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92b8197eb0 con 0x7f92a406e200 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.578+0000 7f92affff700 1 -- 172.21.15.49:0/937272800 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f92b8197eb0 con 0x7f92a406e200 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:09:32.580 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:09:32.581 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:09:32.581 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:09:32.581 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:09:32.581 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:09:32.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.580+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92a406e200 msgr2=0x7f92a40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.580+0000 7f92bd6c8700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92a406e200 0x7f92a40706c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009380 tx=0x7f92a800b040 comp rx=0 tx=0).stop 2024-05-15T16:09:32.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.580+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 msgr2=0x7f92b819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.580+0000 7f92bd6c8700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 0x7f92b819cdf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009fd0 tx=0x7f92a000b650 comp rx=0 tx=0).stop 2024-05-15T16:09:32.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.580+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 shutdown_connections 2024-05-15T16:09:32.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.580+0000 7f92bd6c8700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f92a406e200 0x7f92a40706c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.580+0000 7f92bd6c8700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b8101da0 0x7f92b819c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.580+0000 7f92bd6c8700 1 --2- 172.21.15.49:0/937272800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92b8102750 0x7f92b819cdf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.580+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 >> 172.21.15.49:0/937272800 conn(0x7f92b80fb6a0 msgr2=0x7f92b81030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:32.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.581+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 shutdown_connections 2024-05-15T16:09:32.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.581+0000 7f92bd6c8700 1 -- 172.21.15.49:0/937272800 wait complete. 2024-05-15T16:09:32.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.690+0000 7f6ce7ded700 1 -- 172.21.15.49:0/153982634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 msgr2=0x7f6ce0100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.690+0000 7f6ce7ded700 1 --2- 172.21.15.49:0/153982634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 0x7f6ce0100020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009910 tx=0x7f6cdc009c20 comp rx=0 tx=0).stop 2024-05-15T16:09:32.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.690+0000 7f6ce7ded700 1 -- 172.21.15.49:0/153982634 shutdown_connections 2024-05-15T16:09:32.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.690+0000 7f6ce7ded700 1 --2- 172.21.15.49:0/153982634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce0100560 0x7f6ce0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.690+0000 7f6ce7ded700 1 --2- 172.21.15.49:0/153982634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 0x7f6ce0100020 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.690+0000 7f6ce7ded700 1 -- 172.21.15.49:0/153982634 >> 172.21.15.49:0/153982634 conn(0x7f6ce00fb6a0 msgr2=0x7f6ce00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:32.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.690+0000 7f6ce7ded700 1 -- 172.21.15.49:0/153982634 shutdown_connections 2024-05-15T16:09:32.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.690+0000 7f6ce7ded700 1 -- 172.21.15.49:0/153982634 wait complete. 2024-05-15T16:09:32.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.690+0000 7f6ce7ded700 1 Processor -- start 2024-05-15T16:09:32.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce7ded700 1 -- start start 2024-05-15T16:09:32.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce7ded700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 0x7f6ce019d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:32.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce7ded700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce0100560 0x7f6ce019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:32.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce7ded700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce019de00 con 0x7f6ce00ffc40 2024-05-15T16:09:32.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce7ded700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce01a1ad0 con 0x7f6ce0100560 2024-05-15T16:09:32.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce5b89700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 0x7f6ce019d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:32.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce5b89700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 0x7f6ce019d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46396/0 (socket says 172.21.15.49:46396) 2024-05-15T16:09:32.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce5b89700 1 -- 172.21.15.49:0/2619836975 learned_addr learned my addr 172.21.15.49:0/2619836975 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:09:32.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce5388700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce0100560 0x7f6ce019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:32.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce5b89700 1 -- 172.21.15.49:0/2619836975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce0100560 msgr2=0x7f6ce019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce5b89700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce0100560 0x7f6ce019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce5b89700 1 -- 172.21.15.49:0/2619836975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0095f0 con 0x7f6ce00ffc40 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce5388700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce0100560 0x7f6ce019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.691+0000 7f6ce5b89700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 0x7f6ce019d1e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00bd70 tx=0x7f6cdc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.692+0000 7f6cd6ffd700 1 -- 172.21.15.49:0/2619836975 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cdc01d070 con 0x7f6ce00ffc40 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.692+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce01a1d50 con 0x7f6ce00ffc40 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.692+0000 7f6cd6ffd700 1 -- 172.21.15.49:0/2619836975 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cdc0056e0 con 0x7f6ce00ffc40 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.692+0000 7f6cd6ffd700 1 -- 172.21.15.49:0/2619836975 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cdc017400 con 0x7f6ce00ffc40 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.692+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce01a2240 con 0x7f6ce00ffc40 2024-05-15T16:09:32.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.693+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce0069ab0 con 0x7f6ce00ffc40 2024-05-15T16:09:32.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.693+0000 7f6cd6ffd700 1 -- 172.21.15.49:0/2619836975 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6cdc003a60 con 0x7f6ce00ffc40 2024-05-15T16:09:32.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.694+0000 7f6cd6ffd700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ccc06e200 0x7f6ccc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:09:32.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.694+0000 7f6ce5388700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ccc06e200 0x7f6ccc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:09:32.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.694+0000 7f6cd6ffd700 1 -- 172.21.15.49:0/2619836975 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6cdc08f650 con 0x7f6ce00ffc40 2024-05-15T16:09:32.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.695+0000 7f6ce5388700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ccc06e200 0x7f6ccc0706c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0005950 tx=0x7f6cd00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:09:32.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.699+0000 7f6cd6ffd700 1 -- 172.21.15.49:0/2619836975 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6cdc058930 con 0x7f6ce00ffc40 2024-05-15T16:09:32.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:32 smithi049 ceph-mon[112142]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:32.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:32 smithi049 ceph-mon[112142]: from='client.34746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:32.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:32 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3548306190' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:09:32.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:32 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1554933870' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:09:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.993+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6ce01144d0 con 0x7f6ce00ffc40 2024-05-15T16:09:32.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.994+0000 7f6cd6ffd700 1 -- 172.21.15.49:0/2619836975 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f6cdc05bff0 con 0x7f6ce00ffc40 2024-05-15T16:09:32.995 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:09:32.995 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:09:32.996 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:09:32.996 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:09:32.996 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:09:32.996 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:09:32.996 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:09:32.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ccc06e200 msgr2=0x7f6ccc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ccc06e200 0x7f6ccc0706c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0005950 tx=0x7f6cd00058e0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 msgr2=0x7f6ce019d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 0x7f6ce019d1e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00bd70 tx=0x7f6cdc00be50 comp rx=0 tx=0).stop 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 shutdown_connections 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ccc06e200 0x7f6ccc0706c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce00ffc40 0x7f6ce019d1e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 --2- 172.21.15.49:0/2619836975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce0100560 0x7f6ce019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 >> 172.21.15.49:0/2619836975 conn(0x7f6ce00fb6a0 msgr2=0x7f6ce010e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 shutdown_connections 2024-05-15T16:09:32.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:09:32.996+0000 7f6ce7ded700 1 -- 172.21.15.49:0/2619836975 wait complete. 2024-05-15T16:09:33.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:32 smithi144 ceph-mon[86635]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:33.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:32 smithi144 ceph-mon[86635]: from='client.34746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:33.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:32 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3548306190' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:09:33.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:32 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1554933870' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:09:34.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:33 smithi144 ceph-mon[86635]: pgmap v375: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:34.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:33 smithi144 ceph-mon[86635]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:34.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:33 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2619836975' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:09:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:33 smithi049 ceph-mon[112142]: pgmap v375: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:33 smithi049 ceph-mon[112142]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:09:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:33 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2619836975' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:09:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:35 smithi144 ceph-mon[86635]: pgmap v376: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:09:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:35 smithi049 ceph-mon[112142]: pgmap v376: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:09:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:37 smithi144 ceph-mon[86635]: pgmap v377: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:37 smithi049 ceph-mon[112142]: pgmap v377: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:39 smithi144 ceph-mon[86635]: pgmap v378: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:39 smithi049 ceph-mon[112142]: pgmap v378: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:41 smithi144 ceph-mon[86635]: pgmap v379: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:41 smithi049 ceph-mon[112142]: pgmap v379: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:43 smithi144 ceph-mon[86635]: pgmap v380: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:43 smithi049 ceph-mon[112142]: pgmap v380: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:45 smithi144 ceph-mon[86635]: pgmap v381: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:45 smithi049 ceph-mon[112142]: pgmap v381: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:47 smithi144 ceph-mon[86635]: pgmap v382: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:47 smithi049 ceph-mon[112142]: pgmap v382: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:50.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:49 smithi144 ceph-mon[86635]: pgmap v383: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:50.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:49 smithi049 ceph-mon[112142]: pgmap v383: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:51.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:09:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:09:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:51 smithi144 ceph-mon[86635]: pgmap v384: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:51 smithi049 ceph-mon[112142]: pgmap v384: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:53 smithi144 ceph-mon[86635]: pgmap v385: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:53 smithi049 ceph-mon[112142]: pgmap v385: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:55 smithi144 ceph-mon[86635]: pgmap v386: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:55 smithi049 ceph-mon[112142]: pgmap v386: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:58.077 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:57 smithi144 ceph-mon[86635]: pgmap v387: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:09:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:57 smithi049 ceph-mon[112142]: pgmap v387: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:09:59 smithi144 ceph-mon[86635]: pgmap v388: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:09:59 smithi049 ceph-mon[112142]: pgmap v388: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:09:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:10:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:09:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:10:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:09:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-05-15T16:10:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:09:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:10:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:09:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:10:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:09:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:10:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:09:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-05-15T16:10:01.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-mon[112142]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:10:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-mon[112142]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:10:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-mon[112142]: fs cephfs is degraded 2024-05-15T16:10:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-mon[112142]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:10:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-mon[112142]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:10:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-mon[112142]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:10:01.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:00 smithi049 ceph-mon[112142]: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:10:01.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:00 smithi144 ceph-mon[86635]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:10:01.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:00 smithi144 ceph-mon[86635]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:10:01.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:00 smithi144 ceph-mon[86635]: fs cephfs is degraded 2024-05-15T16:10:01.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:00 smithi144 ceph-mon[86635]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:10:01.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:00 smithi144 ceph-mon[86635]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:10:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:00 smithi144 ceph-mon[86635]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:10:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:00 smithi144 ceph-mon[86635]: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:10:02.099 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:01 smithi049 ceph-mon[112142]: pgmap v389: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:01 smithi144 ceph-mon[86635]: pgmap v389: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.120+0000 7fcdd6a93700 1 -- 172.21.15.49:0/3191558523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 msgr2=0x7fcdd01001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.120+0000 7fcdd6a93700 1 --2- 172.21.15.49:0/3191558523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 0x7fcdd01001b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000b0a0 tx=0x7fcdc000b3b0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.120+0000 7fcdd6a93700 1 -- 172.21.15.49:0/3191558523 shutdown_connections 2024-05-15T16:10:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.120+0000 7fcdd6a93700 1 --2- 172.21.15.49:0/3191558523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd0100780 0x7fcdd01047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.120+0000 7fcdd6a93700 1 --2- 172.21.15.49:0/3191558523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 0x7fcdd01001b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.120+0000 7fcdd6a93700 1 -- 172.21.15.49:0/3191558523 >> 172.21.15.49:0/3191558523 conn(0x7fcdd00fb6a0 msgr2=0x7fcdd00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.120+0000 7fcdd6a93700 1 -- 172.21.15.49:0/3191558523 shutdown_connections 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.120+0000 7fcdd6a93700 1 -- 172.21.15.49:0/3191558523 wait complete. 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.121+0000 7fcdd6a93700 1 Processor -- start 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.121+0000 7fcdd6a93700 1 -- start start 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.121+0000 7fcdd6a93700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 0x7fcdd019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.121+0000 7fcdd6a93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd0100780 0x7fcdd019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.121+0000 7fcdd6a93700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd019d3d0 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.121+0000 7fcdd6a93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd01969b0 con 0x7fcdd0100780 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.121+0000 7fcdd482f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 0x7fcdd019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.121+0000 7fcdd482f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 0x7fcdd019c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49882/0 (socket says 172.21.15.49:49882) 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.121+0000 7fcdd482f700 1 -- 172.21.15.49:0/18158050 learned_addr learned my addr 172.21.15.49:0/18158050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdcffff700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd0100780 0x7fcdd019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdd482f700 1 -- 172.21.15.49:0/18158050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd0100780 msgr2=0x7fcdd019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdd482f700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd0100780 0x7fcdd019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdd482f700 1 -- 172.21.15.49:0/18158050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc0009d00 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdd482f700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 0x7fcdd019c870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0000f80 tx=0x7fcdc0007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdcdffb700 1 -- 172.21.15.49:0/18158050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc000e040 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdcdffb700 1 -- 172.21.15.49:0/18158050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdc0003c10 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd0196c30 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdcdffb700 1 -- 172.21.15.49:0/18158050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc001d6b0 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.122+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd0197120 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.123+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd0108170 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.124+0000 7fcdcdffb700 1 -- 172.21.15.49:0/18158050 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcdc0004040 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.125+0000 7fcdcdffb700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdbc06e200 0x7fcdbc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.125+0000 7fcdcdffb700 1 -- 172.21.15.49:0/18158050 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcdc008f910 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.125+0000 7fcdcffff700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdbc06e200 0x7fcdbc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.125+0000 7fcdcffff700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdbc06e200 0x7fcdbc0706c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8005950 tx=0x7fcdb80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:03.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.129+0000 7fcdcdffb700 1 -- 172.21.15.49:0/18158050 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcdc005c720 con 0x7fcdd00ffdd0 2024-05-15T16:10:03.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.358+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdd0197ee0 con 0x7fcdbc06e200 2024-05-15T16:10:03.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.360+0000 7fcdcdffb700 1 -- 172.21.15.49:0/18158050 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcdd0197ee0 con 0x7fcdbc06e200 2024-05-15T16:10:03.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdbc06e200 msgr2=0x7fcdbc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdbc06e200 0x7fcdbc0706c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8005950 tx=0x7fcdb80058e0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 msgr2=0x7fcdd019c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 0x7fcdd019c870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0000f80 tx=0x7fcdc0007bd0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 shutdown_connections 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdbc06e200 0x7fcdbc0706c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdd00ffdd0 0x7fcdd019c870 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 --2- 172.21.15.49:0/18158050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd0100780 0x7fcdd019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 >> 172.21.15.49:0/18158050 conn(0x7fcdd00fb6a0 msgr2=0x7fcdd0109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 shutdown_connections 2024-05-15T16:10:03.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.362+0000 7fcdd6a93700 1 -- 172.21.15.49:0/18158050 wait complete. 2024-05-15T16:10:03.374 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:10:03.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.472+0000 7f4e8ee63700 1 -- 172.21.15.49:0/3468915445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e880ffdc0 msgr2=0x7f4e881001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.472+0000 7f4e8ee63700 1 --2- 172.21.15.49:0/3468915445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e880ffdc0 0x7f4e881001a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009910 tx=0x7f4e7c009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:03.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.472+0000 7f4e8ee63700 1 -- 172.21.15.49:0/3468915445 shutdown_connections 2024-05-15T16:10:03.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.472+0000 7f4e8ee63700 1 --2- 172.21.15.49:0/3468915445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e88100770 0x7f4e881047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.472+0000 7f4e8ee63700 1 --2- 172.21.15.49:0/3468915445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e880ffdc0 0x7f4e881001a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.472+0000 7f4e8ee63700 1 -- 172.21.15.49:0/3468915445 >> 172.21.15.49:0/3468915445 conn(0x7f4e88076800 msgr2=0x7f4e880fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:03.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.472+0000 7f4e8ee63700 1 -- 172.21.15.49:0/3468915445 shutdown_connections 2024-05-15T16:10:03.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.472+0000 7f4e8ee63700 1 -- 172.21.15.49:0/3468915445 wait complete. 2024-05-15T16:10:03.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.473+0000 7f4e8ee63700 1 Processor -- start 2024-05-15T16:10:03.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.473+0000 7f4e8ee63700 1 -- start start 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e8ee63700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e880ffdc0 0x7f4e880fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e8ee63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e88100770 0x7f4e880fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e8ee63700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880fc760 con 0x7f4e88100770 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e8ee63700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880fc8a0 con 0x7f4e880ffdc0 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e87fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e88100770 0x7f4e880fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e87fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e88100770 0x7f4e880fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49896/0 (socket says 172.21.15.49:49896) 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e87fff700 1 -- 172.21.15.49:0/2571213987 learned_addr learned my addr 172.21.15.49:0/2571213987 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e8cbff700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e880ffdc0 0x7f4e880fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e87fff700 1 -- 172.21.15.49:0/2571213987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e880ffdc0 msgr2=0x7f4e880fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e87fff700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e880ffdc0 0x7f4e880fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e87fff700 1 -- 172.21.15.49:0/2571213987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c0095f0 con 0x7f4e88100770 2024-05-15T16:10:03.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e87fff700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e88100770 0x7f4e880fc220 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400b480 tx=0x7f4e7400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:03.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e85ffb700 1 -- 172.21.15.49:0/2571213987 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e740118c0 con 0x7f4e88100770 2024-05-15T16:10:03.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e85ffb700 1 -- 172.21.15.49:0/2571213987 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e74010380 con 0x7f4e88100770 2024-05-15T16:10:03.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e881a2e90 con 0x7f4e88100770 2024-05-15T16:10:03.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.474+0000 7f4e85ffb700 1 -- 172.21.15.49:0/2571213987 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e7400f4f0 con 0x7f4e88100770 2024-05-15T16:10:03.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.475+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e881a31f0 con 0x7f4e88100770 2024-05-15T16:10:03.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.476+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e881012a0 con 0x7f4e88100770 2024-05-15T16:10:03.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.476+0000 7f4e85ffb700 1 -- 172.21.15.49:0/2571213987 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4e740104f0 con 0x7f4e88100770 2024-05-15T16:10:03.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.476+0000 7f4e85ffb700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e7806e200 0x7f4e780706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:03.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.477+0000 7f4e85ffb700 1 -- 172.21.15.49:0/2571213987 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4e7408ddb0 con 0x7f4e88100770 2024-05-15T16:10:03.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.477+0000 7f4e8cbff700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e7806e200 0x7f4e780706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:03.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.477+0000 7f4e8cbff700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e7806e200 0x7f4e780706c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00b5c0 tx=0x7f4e7c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:03.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.481+0000 7f4e85ffb700 1 -- 172.21.15.49:0/2571213987 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4e74057090 con 0x7f4e88100770 2024-05-15T16:10:03.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.706+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e8810cec0 con 0x7f4e7806e200 2024-05-15T16:10:03.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.708+0000 7f4e85ffb700 1 -- 172.21.15.49:0/2571213987 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4e8810cec0 con 0x7f4e7806e200 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e7806e200 msgr2=0x7f4e780706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e7806e200 0x7f4e780706c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00b5c0 tx=0x7f4e7c01a040 comp rx=0 tx=0).stop 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e88100770 msgr2=0x7f4e880fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e88100770 0x7f4e880fc220 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400b480 tx=0x7f4e7400b840 comp rx=0 tx=0).stop 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 shutdown_connections 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e7806e200 0x7f4e780706c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e880ffdc0 0x7f4e880fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 --2- 172.21.15.49:0/2571213987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e88100770 0x7f4e880fc220 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 >> 172.21.15.49:0/2571213987 conn(0x7f4e88076800 msgr2=0x7f4e880fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 shutdown_connections 2024-05-15T16:10:03.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.710+0000 7f4e8ee63700 1 -- 172.21.15.49:0/2571213987 wait complete. 2024-05-15T16:10:03.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.820+0000 7f8828b44700 1 -- 172.21.15.49:0/1983201750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 msgr2=0x7f8824105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.820+0000 7f8828b44700 1 --2- 172.21.15.49:0/1983201750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 0x7f8824105900 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f880c009910 tx=0x7f880c009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:03.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.820+0000 7f8828b44700 1 -- 172.21.15.49:0/1983201750 shutdown_connections 2024-05-15T16:10:03.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.820+0000 7f8828b44700 1 --2- 172.21.15.49:0/1983201750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 0x7f8824105900 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.820+0000 7f8828b44700 1 --2- 172.21.15.49:0/1983201750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8824100f00 0x7f88241012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.820+0000 7f8828b44700 1 -- 172.21.15.49:0/1983201750 >> 172.21.15.49:0/1983201750 conn(0x7f88240fc7d0 msgr2=0x7f88240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:03.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.820+0000 7f8828b44700 1 -- 172.21.15.49:0/1983201750 shutdown_connections 2024-05-15T16:10:03.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.820+0000 7f8828b44700 1 -- 172.21.15.49:0/1983201750 wait complete. 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f8828b44700 1 Processor -- start 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f8828b44700 1 -- start start 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f8828b44700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8824100f00 0x7f882419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f8828b44700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 0x7f882419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f8828b44700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882419d540 con 0x7f88241018b0 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f8828b44700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8824196b20 con 0x7f8824100f00 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f8821d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 0x7f882419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f8821d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 0x7f882419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41638/0 (socket says 172.21.15.49:41638) 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f8821d9b700 1 -- 172.21.15.49:0/3689462503 learned_addr learned my addr 172.21.15.49:0/3689462503 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.821+0000 7f882259c700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8824100f00 0x7f882419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.822+0000 7f8821d9b700 1 -- 172.21.15.49:0/3689462503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8824100f00 msgr2=0x7f882419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:03.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.822+0000 7f8821d9b700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8824100f00 0x7f882419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:03.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.822+0000 7f8821d9b700 1 -- 172.21.15.49:0/3689462503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c0095f0 con 0x7f88241018b0 2024-05-15T16:10:03.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.822+0000 7f8821d9b700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 0x7f882419cf20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f880c005b40 tx=0x7f880c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:03.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.822+0000 7f881b7fe700 1 -- 172.21.15.49:0/3689462503 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f880c01d070 con 0x7f88241018b0 2024-05-15T16:10:03.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.822+0000 7f881b7fe700 1 -- 172.21.15.49:0/3689462503 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f880c00f460 con 0x7f88241018b0 2024-05-15T16:10:03.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.822+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8824196da0 con 0x7f88241018b0 2024-05-15T16:10:03.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.822+0000 7f881b7fe700 1 -- 172.21.15.49:0/3689462503 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f880c005720 con 0x7f88241018b0 2024-05-15T16:10:03.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.822+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8824197290 con 0x7f88241018b0 2024-05-15T16:10:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.823+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8824109250 con 0x7f88241018b0 2024-05-15T16:10:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.824+0000 7f881b7fe700 1 -- 172.21.15.49:0/3689462503 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f880c003a60 con 0x7f88241018b0 2024-05-15T16:10:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.824+0000 7f881b7fe700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f881006e210 0x7f88100706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.824+0000 7f881b7fe700 1 -- 172.21.15.49:0/3689462503 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f880c08f530 con 0x7f88241018b0 2024-05-15T16:10:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.824+0000 7f882259c700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f881006e210 0x7f88100706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.825+0000 7f882259c700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f881006e210 0x7f88100706d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f881400a400 tx=0x7f881400d040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:03.829+0000 7f881b7fe700 1 -- 172.21.15.49:0/3689462503 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f880c05c3c0 con 0x7f88241018b0 2024-05-15T16:10:04.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.046+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8824198050 con 0x7f881006e210 2024-05-15T16:10:04.047 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:03 smithi049 ceph-mon[112142]: pgmap v390: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:04.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.057+0000 7f881b7fe700 1 -- 172.21.15.49:0/3689462503 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8824198050 con 0x7f881006e210 2024-05-15T16:10:04.058 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (13m) 6m ago 24m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (25m) 6m ago 25m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (24m) 6m ago 23m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (10m) 6m ago 25m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (10m) 6m ago 23m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (12m) 6m ago 24m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (6m) 6m ago 19m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (6m) 6m ago 19m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (6m) 6m ago 19m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (19m) 6m ago 19m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (15m) 6m ago 26m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (14m) 6m ago 23m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (11m) 6m ago 26m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:10:04.059 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (11m) 6m ago 23m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:10:04.060 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (14m) 6m ago 25m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:10:04.060 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (14m) 6m ago 23m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:10:04.060 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (10m) 6m ago 22m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:10:04.060 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (9m) 6m ago 22m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:10:04.060 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (9m) 6m ago 22m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:10:04.060 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (8m) 6m ago 21m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:10:04.060 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (7m) 6m ago 21m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:10:04.060 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (7m) 6m ago 20m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:10:04.060 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (13m) 6m ago 24m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f881006e210 msgr2=0x7f88100706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f881006e210 0x7f88100706d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f881400a400 tx=0x7f881400d040 comp rx=0 tx=0).stop 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 msgr2=0x7f882419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 0x7f882419cf20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f880c005b40 tx=0x7f880c00bd70 comp rx=0 tx=0).stop 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 shutdown_connections 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f881006e210 0x7f88100706d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8824100f00 0x7f882419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 --2- 172.21.15.49:0/3689462503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88241018b0 0x7f882419cf20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 >> 172.21.15.49:0/3689462503 conn(0x7f88240fc7d0 msgr2=0x7f88240fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 shutdown_connections 2024-05-15T16:10:04.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.059+0000 7f8828b44700 1 -- 172.21.15.49:0/3689462503 wait complete. 2024-05-15T16:10:04.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.166+0000 7f3634d81700 1 -- 172.21.15.49:0/3390125330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3630100040 msgr2=0x7f3630104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.166+0000 7f3634d81700 1 --2- 172.21.15.49:0/3390125330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3630100040 0x7f3630104880 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3624009910 tx=0x7f3624009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:04.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.166+0000 7f3634d81700 1 -- 172.21.15.49:0/3390125330 shutdown_connections 2024-05-15T16:10:04.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.166+0000 7f3634d81700 1 --2- 172.21.15.49:0/3390125330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3630100040 0x7f3630104880 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.166+0000 7f3634d81700 1 --2- 172.21.15.49:0/3390125330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36300ff690 0x7f36300ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.166+0000 7f3634d81700 1 -- 172.21.15.49:0/3390125330 >> 172.21.15.49:0/3390125330 conn(0x7f3630074eb0 msgr2=0x7f36300752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:04.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.166+0000 7f3634d81700 1 -- 172.21.15.49:0/3390125330 shutdown_connections 2024-05-15T16:10:04.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.166+0000 7f3634d81700 1 -- 172.21.15.49:0/3390125330 wait complete. 2024-05-15T16:10:04.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f3634d81700 1 Processor -- start 2024-05-15T16:10:04.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f3634d81700 1 -- start start 2024-05-15T16:10:04.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f3634d81700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36300ff690 0x7f363019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:04.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f3634d81700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3630100040 0x7f363019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:04.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f3634d81700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363019d430 con 0x7f36300ff690 2024-05-15T16:10:04.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f3634d81700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3630196a10 con 0x7f3630100040 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f362e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36300ff690 0x7f363019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f362e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36300ff690 0x7f363019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41650/0 (socket says 172.21.15.49:41650) 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f362e59c700 1 -- 172.21.15.49:0/4175856686 learned_addr learned my addr 172.21.15.49:0/4175856686 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f362dd9b700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3630100040 0x7f363019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f362e59c700 1 -- 172.21.15.49:0/4175856686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3630100040 msgr2=0x7f363019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f362e59c700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3630100040 0x7f363019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f362e59c700 1 -- 172.21.15.49:0/4175856686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240095f0 con 0x7f36300ff690 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.167+0000 7f362dd9b700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3630100040 0x7f363019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.168+0000 7f362e59c700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36300ff690 0x7f363019c8d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f362000d8d0 tx=0x7f362000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.168+0000 7f361f7fe700 1 -- 172.21.15.49:0/4175856686 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3620009940 con 0x7f36300ff690 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.168+0000 7f361f7fe700 1 -- 172.21.15.49:0/4175856686 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3620010460 con 0x7f36300ff690 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.168+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3630196cf0 con 0x7f36300ff690 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.168+0000 7f361f7fe700 1 -- 172.21.15.49:0/4175856686 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f362000f5f0 con 0x7f36300ff690 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.168+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3630197240 con 0x7f36300ff690 2024-05-15T16:10:04.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.169+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36301092b0 con 0x7f36300ff690 2024-05-15T16:10:04.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.169+0000 7f361f7fe700 1 -- 172.21.15.49:0/4175856686 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f36200105d0 con 0x7f36300ff690 2024-05-15T16:10:04.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.170+0000 7f361f7fe700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f361806e210 0x7f36180706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:04.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.170+0000 7f361f7fe700 1 -- 172.21.15.49:0/4175856686 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f362008e1a0 con 0x7f36300ff690 2024-05-15T16:10:04.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.170+0000 7f362dd9b700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f361806e210 0x7f36180706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:04.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.170+0000 7f362dd9b700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f361806e210 0x7f36180706d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f362400b5c0 tx=0x7f362401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:04.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.175+0000 7f361f7fe700 1 -- 172.21.15.49:0/4175856686 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f362005afb0 con 0x7f36300ff690 2024-05-15T16:10:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:03 smithi144 ceph-mon[86635]: pgmap v390: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:04.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.469+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3630197ed0 con 0x7f36300ff690 2024-05-15T16:10:04.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.470+0000 7f361f7fe700 1 -- 172.21.15.49:0/4175856686 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f362005ab40 con 0x7f36300ff690 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:10:04.472 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:10:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:10:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:10:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:10:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:10:04.473 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:10:04.473 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f361806e210 msgr2=0x7f36180706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f361806e210 0x7f36180706d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f362400b5c0 tx=0x7f362401a040 comp rx=0 tx=0).stop 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36300ff690 msgr2=0x7f363019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36300ff690 0x7f363019c8d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f362000d8d0 tx=0x7f362000dc90 comp rx=0 tx=0).stop 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 shutdown_connections 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f361806e210 0x7f36180706d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36300ff690 0x7f363019c8d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 --2- 172.21.15.49:0/4175856686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3630100040 0x7f363019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 >> 172.21.15.49:0/4175856686 conn(0x7f3630074eb0 msgr2=0x7f3630102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 shutdown_connections 2024-05-15T16:10:04.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.472+0000 7f3634d81700 1 -- 172.21.15.49:0/4175856686 wait complete. 2024-05-15T16:10:04.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.585+0000 7f59848ba700 1 -- 172.21.15.49:0/3164611036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 msgr2=0x7f59801001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.585+0000 7f59848ba700 1 --2- 172.21.15.49:0/3164611036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 0x7f59801001a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5970009910 tx=0x7f5970009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.585+0000 7f59848ba700 1 -- 172.21.15.49:0/3164611036 shutdown_connections 2024-05-15T16:10:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.585+0000 7f59848ba700 1 --2- 172.21.15.49:0/3164611036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5980100770 0x7f59801047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.585+0000 7f59848ba700 1 --2- 172.21.15.49:0/3164611036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 0x7f59801001a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.585+0000 7f59848ba700 1 -- 172.21.15.49:0/3164611036 >> 172.21.15.49:0/3164611036 conn(0x7f5980076800 msgr2=0x7f59800fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.585+0000 7f59848ba700 1 -- 172.21.15.49:0/3164611036 shutdown_connections 2024-05-15T16:10:04.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.585+0000 7f59848ba700 1 -- 172.21.15.49:0/3164611036 wait complete. 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f59848ba700 1 Processor -- start 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f59848ba700 1 -- start start 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f59848ba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 0x7f5980197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f59848ba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5980100770 0x7f5980197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f59848ba700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5980198510 con 0x7f59800ffdc0 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f59848ba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5980198650 con 0x7f5980100770 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f597e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 0x7f5980197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f597e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 0x7f5980197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41666/0 (socket says 172.21.15.49:41666) 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f597e59c700 1 -- 172.21.15.49:0/455098881 learned_addr learned my addr 172.21.15.49:0/455098881 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f597dd9b700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5980100770 0x7f5980197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f597e59c700 1 -- 172.21.15.49:0/455098881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5980100770 msgr2=0x7f5980197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f597e59c700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5980100770 0x7f5980197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.586+0000 7f597e59c700 1 -- 172.21.15.49:0/455098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700095f0 con 0x7f59800ffdc0 2024-05-15T16:10:04.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.587+0000 7f597e59c700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 0x7f5980197a00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5970006010 tx=0x7f597000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:04.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.587+0000 7f596f7fe700 1 -- 172.21.15.49:0/455098881 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f597001d070 con 0x7f59800ffdc0 2024-05-15T16:10:04.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.587+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5980198870 con 0x7f59800ffdc0 2024-05-15T16:10:04.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.587+0000 7f596f7fe700 1 -- 172.21.15.49:0/455098881 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f597000f460 con 0x7f59800ffdc0 2024-05-15T16:10:04.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.587+0000 7f596f7fe700 1 -- 172.21.15.49:0/455098881 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5970017630 con 0x7f59800ffdc0 2024-05-15T16:10:04.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.587+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59801a0dd0 con 0x7f59800ffdc0 2024-05-15T16:10:04.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.588+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5980108160 con 0x7f59800ffdc0 2024-05-15T16:10:04.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.588+0000 7f596f7fe700 1 -- 172.21.15.49:0/455098881 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f597000f5d0 con 0x7f59800ffdc0 2024-05-15T16:10:04.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.589+0000 7f596f7fe700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596806e210 0x7f59680706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:04.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.589+0000 7f596f7fe700 1 -- 172.21.15.49:0/455098881 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f597008f520 con 0x7f59800ffdc0 2024-05-15T16:10:04.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.589+0000 7f597dd9b700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596806e210 0x7f59680706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:04.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.589+0000 7f597dd9b700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596806e210 0x7f59680706d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5980068e90 tx=0x7f5974005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:04.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.593+0000 7f596f7fe700 1 -- 172.21.15.49:0/455098881 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f59700588c0 con 0x7f59800ffdc0 2024-05-15T16:10:04.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.846+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5980068ce0 con 0x7f59800ffdc0 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.846+0000 7f596f7fe700 1 -- 172.21.15.49:0/455098881 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f5970020370 con 0x7f59800ffdc0 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:10:04.848 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:10:04.849 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:10:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:10:04.851 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:10:04.851 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:10:04.851 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:10:04.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.848+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596806e210 msgr2=0x7f59680706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.848+0000 7f59848ba700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596806e210 0x7f59680706d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5980068e90 tx=0x7f5974005cf0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.848+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 msgr2=0x7f5980197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.848+0000 7f59848ba700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 0x7f5980197a00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5970006010 tx=0x7f597000be20 comp rx=0 tx=0).stop 2024-05-15T16:10:04.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.848+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 shutdown_connections 2024-05-15T16:10:04.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.848+0000 7f59848ba700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596806e210 0x7f59680706d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.848+0000 7f59848ba700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59800ffdc0 0x7f5980197a00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.848+0000 7f59848ba700 1 --2- 172.21.15.49:0/455098881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5980100770 0x7f5980197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.848+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 >> 172.21.15.49:0/455098881 conn(0x7f5980076800 msgr2=0x7f5980103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:04.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.849+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 shutdown_connections 2024-05-15T16:10:04.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.849+0000 7f59848ba700 1 -- 172.21.15.49:0/455098881 wait complete. 2024-05-15T16:10:04.853 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:10:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.958+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/2151892307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c101e70 msgr2=0x7fb99c102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.958+0000 7fb9a1fc2700 1 --2- 172.21.15.49:0/2151892307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c101e70 0x7fb99c102250 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009910 tx=0x7fb98c009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.958+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/2151892307 shutdown_connections 2024-05-15T16:10:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.958+0000 7fb9a1fc2700 1 --2- 172.21.15.49:0/2151892307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c102790 0x7fb99c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.958+0000 7fb9a1fc2700 1 --2- 172.21.15.49:0/2151892307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c101e70 0x7fb99c102250 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.958+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/2151892307 >> 172.21.15.49:0/2151892307 conn(0x7fb99c0fb700 msgr2=0x7fb99c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.958+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/2151892307 shutdown_connections 2024-05-15T16:10:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.958+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/2151892307 wait complete. 2024-05-15T16:10:04.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.958+0000 7fb9a1fc2700 1 Processor -- start 2024-05-15T16:10:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb9a1fc2700 1 -- start start 2024-05-15T16:10:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb9a1fc2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c101e70 0x7fb99c0ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb9a1fc2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c102790 0x7fb99c100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb9a1fc2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c100550 con 0x7fb99c102790 2024-05-15T16:10:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb9a1fc2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c100690 con 0x7fb99c101e70 2024-05-15T16:10:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb99affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c102790 0x7fb99c100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb99affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c102790 0x7fb99c100010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41698/0 (socket says 172.21.15.49:41698) 2024-05-15T16:10:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb99affd700 1 -- 172.21.15.49:0/1888752405 learned_addr learned my addr 172.21.15.49:0/1888752405 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:04.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb99b7fe700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c101e70 0x7fb99c0ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb99affd700 1 -- 172.21.15.49:0/1888752405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c101e70 msgr2=0x7fb99c0ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb99affd700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c101e70 0x7fb99c0ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.959+0000 7fb99affd700 1 -- 172.21.15.49:0/1888752405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c0095f0 con 0x7fb99c102790 2024-05-15T16:10:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.960+0000 7fb99affd700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c102790 0x7fb99c100010 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb99000e940 tx=0x7fb99000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.960+0000 7fb998ff9700 1 -- 172.21.15.49:0/1888752405 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9900096e0 con 0x7fb99c102790 2024-05-15T16:10:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.960+0000 7fb998ff9700 1 -- 172.21.15.49:0/1888752405 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb990004cf0 con 0x7fb99c102790 2024-05-15T16:10:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.960+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c1a2e10 con 0x7fb99c102790 2024-05-15T16:10:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.960+0000 7fb998ff9700 1 -- 172.21.15.49:0/1888752405 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb99000eee0 con 0x7fb99c102790 2024-05-15T16:10:04.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.960+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c1a3170 con 0x7fb99c102790 2024-05-15T16:10:04.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.961+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99c069ab0 con 0x7fb99c102790 2024-05-15T16:10:04.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.962+0000 7fb998ff9700 1 -- 172.21.15.49:0/1888752405 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb99000cba0 con 0x7fb99c102790 2024-05-15T16:10:04.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.962+0000 7fb998ff9700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb98406e210 0x7fb9840706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:04.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.962+0000 7fb998ff9700 1 -- 172.21.15.49:0/1888752405 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb990014070 con 0x7fb99c102790 2024-05-15T16:10:04.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.962+0000 7fb99b7fe700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb98406e210 0x7fb9840706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:04.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.963+0000 7fb99b7fe700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb98406e210 0x7fb9840706d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00b760 tx=0x7fb98c009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:04.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:04.967+0000 7fb998ff9700 1 -- 172.21.15.49:0/1888752405 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb990057760 con 0x7fb99c102790 2024-05-15T16:10:05.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:04 smithi144 ceph-mon[86635]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:05.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:04 smithi144 ceph-mon[86635]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:05.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:04 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4175856686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:10:05.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.197+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb99c10def0 con 0x7fb98406e210 2024-05-15T16:10:05.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:04 smithi049 ceph-mon[112142]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:05.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:04 smithi049 ceph-mon[112142]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:05.199 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:04 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4175856686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:10:05.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.198+0000 7fb998ff9700 1 -- 172.21.15.49:0/1888752405 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb99c10def0 con 0x7fb98406e210 2024-05-15T16:10:05.200 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:10:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:10:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:10:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:10:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:10:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:10:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:10:05.200 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:10:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:10:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:10:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:10:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:10:05.201 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:10:05.201 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:10:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb98406e210 msgr2=0x7fb9840706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb98406e210 0x7fb9840706d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00b760 tx=0x7fb98c009f90 comp rx=0 tx=0).stop 2024-05-15T16:10:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c102790 msgr2=0x7fb99c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c102790 0x7fb99c100010 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb99000e940 tx=0x7fb99000ed00 comp rx=0 tx=0).stop 2024-05-15T16:10:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 shutdown_connections 2024-05-15T16:10:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb98406e210 0x7fb9840706d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c101e70 0x7fb99c0ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 --2- 172.21.15.49:0/1888752405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb99c102790 0x7fb99c100010 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 >> 172.21.15.49:0/1888752405 conn(0x7fb99c0fb700 msgr2=0x7fb99c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 shutdown_connections 2024-05-15T16:10:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.201+0000 7fb9a1fc2700 1 -- 172.21.15.49:0/1888752405 wait complete. 2024-05-15T16:10:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.311+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1922130304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 msgr2=0x7f9bc41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.311+0000 7f9bcb3bb700 1 --2- 172.21.15.49:0/1922130304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 0x7f9bc41012c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009910 tx=0x7f9bc0009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.311+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1922130304 shutdown_connections 2024-05-15T16:10:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.311+0000 7f9bcb3bb700 1 --2- 172.21.15.49:0/1922130304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc4101890 0x7f9bc41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.311+0000 7f9bcb3bb700 1 --2- 172.21.15.49:0/1922130304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 0x7f9bc41012c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.311+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1922130304 >> 172.21.15.49:0/1922130304 conn(0x7f9bc40fc790 msgr2=0x7f9bc40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.311+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1922130304 shutdown_connections 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.311+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1922130304 wait complete. 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.311+0000 7f9bcb3bb700 1 Processor -- start 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bcb3bb700 1 -- start start 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bcb3bb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 0x7f9bc419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bcb3bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc4101890 0x7f9bc419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bcb3bb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc419d460 con 0x7f9bc4100ee0 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bcb3bb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4196a90 con 0x7f9bc4101890 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bc9157700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 0x7f9bc419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bc9157700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 0x7f9bc419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41716/0 (socket says 172.21.15.49:41716) 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bc9157700 1 -- 172.21.15.49:0/1743090095 learned_addr learned my addr 172.21.15.49:0/1743090095 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:05.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bc9157700 1 -- 172.21.15.49:0/1743090095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc4101890 msgr2=0x7f9bc419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:05.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bc9157700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc4101890 0x7f9bc419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bc9157700 1 -- 172.21.15.49:0/1743090095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc00095f0 con 0x7f9bc4100ee0 2024-05-15T16:10:05.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bc9157700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 0x7f9bc419c900 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000bd70 tx=0x7f9bc000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:05.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.312+0000 7f9bba7fc700 1 -- 172.21.15.49:0/1743090095 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc001d070 con 0x7f9bc4100ee0 2024-05-15T16:10:05.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.313+0000 7f9bba7fc700 1 -- 172.21.15.49:0/1743090095 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc00056e0 con 0x7f9bc4100ee0 2024-05-15T16:10:05.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.313+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc4196d10 con 0x7f9bc4100ee0 2024-05-15T16:10:05.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.313+0000 7f9bba7fc700 1 -- 172.21.15.49:0/1743090095 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc0017400 con 0x7f9bc4100ee0 2024-05-15T16:10:05.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.313+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc4197200 con 0x7f9bc4100ee0 2024-05-15T16:10:05.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.314+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc4109280 con 0x7f9bc4100ee0 2024-05-15T16:10:05.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.314+0000 7f9bba7fc700 1 -- 172.21.15.49:0/1743090095 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9bc0003a60 con 0x7f9bc4100ee0 2024-05-15T16:10:05.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.315+0000 7f9bba7fc700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bb006e200 0x7f9bb00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:05.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.315+0000 7f9bba7fc700 1 -- 172.21.15.49:0/1743090095 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9bc008f5a0 con 0x7f9bc4100ee0 2024-05-15T16:10:05.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.315+0000 7f9bc8956700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bb006e200 0x7f9bb00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:05.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.315+0000 7f9bc8956700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bb006e200 0x7f9bb00706c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005950 tx=0x7f9bb40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:05.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.319+0000 7f9bba7fc700 1 -- 172.21.15.49:0/1743090095 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9bc005c430 con 0x7f9bc4100ee0 2024-05-15T16:10:05.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.612+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9bc4197f30 con 0x7f9bc4100ee0 2024-05-15T16:10:05.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.613+0000 7f9bba7fc700 1 -- 172.21.15.49:0/1743090095 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f9bc005bfc0 con 0x7f9bc4100ee0 2024-05-15T16:10:05.614 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:10:05.614 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:10:05.614 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:10:05.614 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:10:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:10:05.615 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:10:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:10:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bb006e200 msgr2=0x7f9bb00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bb006e200 0x7f9bb00706c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005950 tx=0x7f9bb40058e0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 msgr2=0x7f9bc419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 0x7f9bc419c900 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000bd70 tx=0x7f9bc000be50 comp rx=0 tx=0).stop 2024-05-15T16:10:05.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 shutdown_connections 2024-05-15T16:10:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bb006e200 0x7f9bb00706c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc4100ee0 0x7f9bc419c900 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 --2- 172.21.15.49:0/1743090095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc4101890 0x7f9bc419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 >> 172.21.15.49:0/1743090095 conn(0x7f9bc40fc790 msgr2=0x7f9bc40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 shutdown_connections 2024-05-15T16:10:05.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:05.615+0000 7f9bcb3bb700 1 -- 172.21.15.49:0/1743090095 wait complete. 2024-05-15T16:10:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:05 smithi049 ceph-mon[112142]: from='client.34774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:05.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:05 smithi049 ceph-mon[112142]: pgmap v391: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:05.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:05 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/455098881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:10:05.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:10:05.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:05 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1743090095' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:10:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:05 smithi144 ceph-mon[86635]: from='client.34774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:05 smithi144 ceph-mon[86635]: pgmap v391: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:05 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/455098881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:10:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:10:06.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:05 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1743090095' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:10:07.153 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:06 smithi049 ceph-mon[112142]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:07.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:06 smithi144 ceph-mon[86635]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:08.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:07 smithi144 ceph-mon[86635]: pgmap v392: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:07 smithi049 ceph-mon[112142]: pgmap v392: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:09 smithi144 ceph-mon[86635]: pgmap v393: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:09 smithi049 ceph-mon[112142]: pgmap v393: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:11 smithi144 ceph-mon[86635]: pgmap v394: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:11 smithi049 ceph-mon[112142]: pgmap v394: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:13 smithi144 ceph-mon[86635]: pgmap v395: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:13 smithi049 ceph-mon[112142]: pgmap v395: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:15 smithi144 ceph-mon[86635]: pgmap v396: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:15 smithi049 ceph-mon[112142]: pgmap v396: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:17 smithi144 ceph-mon[86635]: pgmap v397: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:18.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:17 smithi049 ceph-mon[112142]: pgmap v397: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:19 smithi144 ceph-mon[86635]: pgmap v398: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:19 smithi049 ceph-mon[112142]: pgmap v398: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:21.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:10:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:10:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:21 smithi144 ceph-mon[86635]: pgmap v399: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:21 smithi049 ceph-mon[112142]: pgmap v399: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:23 smithi144 ceph-mon[86635]: pgmap v400: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:23 smithi049 ceph-mon[112142]: pgmap v400: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:25 smithi144 ceph-mon[86635]: pgmap v401: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:25 smithi049 ceph-mon[112142]: pgmap v401: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:27 smithi144 ceph-mon[86635]: pgmap v402: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:27 smithi049 ceph-mon[112142]: pgmap v402: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:29 smithi144 ceph-mon[86635]: pgmap v403: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:29 smithi049 ceph-mon[112142]: pgmap v403: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:31 smithi144 ceph-mon[86635]: pgmap v404: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:31 smithi049 ceph-mon[112142]: pgmap v404: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:33 smithi144 ceph-mon[86635]: pgmap v405: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:33 smithi049 ceph-mon[112142]: pgmap v405: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.739+0000 7f19b407a700 1 -- 172.21.15.49:0/3646298435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac100390 msgr2=0x7f19ac10d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.739+0000 7f19b407a700 1 --2- 172.21.15.49:0/3646298435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac100390 0x7f19ac10d050 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f199c009910 tx=0x7f199c009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.739+0000 7f19b407a700 1 -- 172.21.15.49:0/3646298435 shutdown_connections 2024-05-15T16:10:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.739+0000 7f19b407a700 1 --2- 172.21.15.49:0/3646298435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac100390 0x7f19ac10d050 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.739+0000 7f19b407a700 1 --2- 172.21.15.49:0/3646298435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac0ff9e0 0x7f19ac0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.739+0000 7f19b407a700 1 -- 172.21.15.49:0/3646298435 >> 172.21.15.49:0/3646298435 conn(0x7f19ac0fb6a0 msgr2=0x7f19ac0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.739+0000 7f19b407a700 1 -- 172.21.15.49:0/3646298435 shutdown_connections 2024-05-15T16:10:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.739+0000 7f19b407a700 1 -- 172.21.15.49:0/3646298435 wait complete. 2024-05-15T16:10:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b407a700 1 Processor -- start 2024-05-15T16:10:35.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b407a700 1 -- start start 2024-05-15T16:10:35.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b407a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac0ff9e0 0x7f19ac198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:35.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b407a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac100390 0x7f19ac199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:35.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b407a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac199a30 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b407a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac19d700 con 0x7f19ac100390 2024-05-15T16:10:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b1e16700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac0ff9e0 0x7f19ac198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b1e16700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac0ff9e0 0x7f19ac198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51398/0 (socket says 172.21.15.49:51398) 2024-05-15T16:10:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b1e16700 1 -- 172.21.15.49:0/3286948050 learned_addr learned my addr 172.21.15.49:0/3286948050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b1615700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac100390 0x7f19ac199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b1e16700 1 -- 172.21.15.49:0/3286948050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac100390 msgr2=0x7f19ac199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b1e16700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac100390 0x7f19ac199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b1e16700 1 -- 172.21.15.49:0/3286948050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c0095f0 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.740+0000 7f19b1615700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac100390 0x7f19ac199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.741+0000 7f19b1e16700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac0ff9e0 0x7f19ac198e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f19a800d8d0 tx=0x7f19a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.741+0000 7f19a2ffd700 1 -- 172.21.15.49:0/3286948050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19a8009940 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.741+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac19d9e0 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.741+0000 7f19a2ffd700 1 -- 172.21.15.49:0/3286948050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19a8010460 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.741+0000 7f19a2ffd700 1 -- 172.21.15.49:0/3286948050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19a800f5f0 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.741+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac19df30 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.742+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac10aed0 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.742+0000 7f19a2ffd700 1 -- 172.21.15.49:0/3286948050 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f19a80105d0 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.743+0000 7f19a2ffd700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199806e200 0x7f19980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.743+0000 7f19a2ffd700 1 -- 172.21.15.49:0/3286948050 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f19a808e170 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.743+0000 7f19b1615700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199806e200 0x7f19980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.744+0000 7f19b1615700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199806e200 0x7f19980706c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f199c00b850 tx=0x7f199c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.748+0000 7f19a2ffd700 1 -- 172.21.15.49:0/3286948050 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f19a8057450 con 0x7f19ac0ff9e0 2024-05-15T16:10:35.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.975+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19ac110060 con 0x7f199806e200 2024-05-15T16:10:35.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.976+0000 7f19a2ffd700 1 -- 172.21.15.49:0/3286948050 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f19ac110060 con 0x7f199806e200 2024-05-15T16:10:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.978+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199806e200 msgr2=0x7f19980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.978+0000 7f19b407a700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199806e200 0x7f19980706c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f199c00b850 tx=0x7f199c005dc0 comp rx=0 tx=0).stop 2024-05-15T16:10:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.978+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac0ff9e0 msgr2=0x7f19ac198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.978+0000 7f19b407a700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac0ff9e0 0x7f19ac198e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f19a800d8d0 tx=0x7f19a800dc90 comp rx=0 tx=0).stop 2024-05-15T16:10:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.978+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 shutdown_connections 2024-05-15T16:10:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.978+0000 7f19b407a700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f199806e200 0x7f19980706c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.978+0000 7f19b407a700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ac0ff9e0 0x7f19ac198e10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.978+0000 7f19b407a700 1 --2- 172.21.15.49:0/3286948050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac100390 0x7f19ac199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.979+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 >> 172.21.15.49:0/3286948050 conn(0x7f19ac0fb6a0 msgr2=0x7f19ac108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.979+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 shutdown_connections 2024-05-15T16:10:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:35.979+0000 7f19b407a700 1 -- 172.21.15.49:0/3286948050 wait complete. 2024-05-15T16:10:35.990 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:10:35.990 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:35 smithi049 ceph-mon[112142]: pgmap v406: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:35.990 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:10:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.089+0000 7f29e9676700 1 -- 172.21.15.49:0/711659983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4100ee0 msgr2=0x7f29e41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.089+0000 7f29e9676700 1 --2- 172.21.15.49:0/711659983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4100ee0 0x7f29e41012c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009910 tx=0x7f29d4009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.089+0000 7f29e9676700 1 -- 172.21.15.49:0/711659983 shutdown_connections 2024-05-15T16:10:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.089+0000 7f29e9676700 1 --2- 172.21.15.49:0/711659983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29e4101890 0x7f29e41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.089+0000 7f29e9676700 1 --2- 172.21.15.49:0/711659983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4100ee0 0x7f29e41012c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.089+0000 7f29e9676700 1 -- 172.21.15.49:0/711659983 >> 172.21.15.49:0/711659983 conn(0x7f29e40fc790 msgr2=0x7f29e40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.089+0000 7f29e9676700 1 -- 172.21.15.49:0/711659983 shutdown_connections 2024-05-15T16:10:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.089+0000 7f29e9676700 1 -- 172.21.15.49:0/711659983 wait complete. 2024-05-15T16:10:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.089+0000 7f29e9676700 1 Processor -- start 2024-05-15T16:10:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29e9676700 1 -- start start 2024-05-15T16:10:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29e9676700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29e4100ee0 0x7f29e419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29e9676700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4101890 0x7f29e419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29e9676700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e419d460 con 0x7f29e4101890 2024-05-15T16:10:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29e9676700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e4196a90 con 0x7f29e4100ee0 2024-05-15T16:10:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29da7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4101890 0x7f29e419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29da7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4101890 0x7f29e419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51404/0 (socket says 172.21.15.49:51404) 2024-05-15T16:10:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29da7fc700 1 -- 172.21.15.49:0/4230543351 learned_addr learned my addr 172.21.15.49:0/4230543351 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29e2ffd700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29e4100ee0 0x7f29e419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29da7fc700 1 -- 172.21.15.49:0/4230543351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29e4100ee0 msgr2=0x7f29e419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29da7fc700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29e4100ee0 0x7f29e419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.090+0000 7f29da7fc700 1 -- 172.21.15.49:0/4230543351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40095f0 con 0x7f29e4101890 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.091+0000 7f29da7fc700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4101890 0x7f29e419ce40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c8f0 tx=0x7f29cc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.091+0000 7f29e0ff9700 1 -- 172.21.15.49:0/4230543351 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29cc0078a0 con 0x7f29e4101890 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.091+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e4196d70 con 0x7f29e4101890 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.091+0000 7f29e0ff9700 1 -- 172.21.15.49:0/4230543351 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29cc00f450 con 0x7f29e4101890 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.091+0000 7f29e0ff9700 1 -- 172.21.15.49:0/4230543351 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29cc018610 con 0x7f29e4101890 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.091+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e41972c0 con 0x7f29e4101890 2024-05-15T16:10:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.092+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e4109280 con 0x7f29e4101890 2024-05-15T16:10:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.092+0000 7f29e0ff9700 1 -- 172.21.15.49:0/4230543351 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f29cc00f5c0 con 0x7f29e4101890 2024-05-15T16:10:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.093+0000 7f29e0ff9700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f29d006e200 0x7f29d00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.093+0000 7f29e0ff9700 1 -- 172.21.15.49:0/4230543351 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f29cc08e0e0 con 0x7f29e4101890 2024-05-15T16:10:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.093+0000 7f29e2ffd700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f29d006e200 0x7f29d00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.093+0000 7f29e2ffd700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f29d006e200 0x7f29d00706c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29d4000c00 tx=0x7f29d401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.098+0000 7f29e0ff9700 1 -- 172.21.15.49:0/4230543351 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f29cc05aef0 con 0x7f29e4101890 2024-05-15T16:10:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:35 smithi144 ceph-mon[86635]: pgmap v406: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:10:36.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.326+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29e4197f70 con 0x7f29d006e200 2024-05-15T16:10:36.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.327+0000 7f29e0ff9700 1 -- 172.21.15.49:0/4230543351 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f29e4197f70 con 0x7f29d006e200 2024-05-15T16:10:36.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f29d006e200 msgr2=0x7f29d00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f29d006e200 0x7f29d00706c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29d4000c00 tx=0x7f29d401a040 comp rx=0 tx=0).stop 2024-05-15T16:10:36.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4101890 msgr2=0x7f29e419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4101890 0x7f29e419ce40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c8f0 tx=0x7f29cc00cc00 comp rx=0 tx=0).stop 2024-05-15T16:10:36.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 shutdown_connections 2024-05-15T16:10:36.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f29d006e200 0x7f29d00706c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29e4100ee0 0x7f29e419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 --2- 172.21.15.49:0/4230543351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29e4101890 0x7f29e419ce40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 >> 172.21.15.49:0/4230543351 conn(0x7f29e40fc790 msgr2=0x7f29e40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:36.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.329+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 shutdown_connections 2024-05-15T16:10:36.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.330+0000 7f29e9676700 1 -- 172.21.15.49:0/4230543351 wait complete. 2024-05-15T16:10:36.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.437+0000 7f6d5e63a700 1 -- 172.21.15.49:0/3425524518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 msgr2=0x7f6d58103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.437+0000 7f6d5e63a700 1 --2- 172.21.15.49:0/3425524518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 0x7f6d58103f90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009910 tx=0x7f6d4c009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:36.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.438+0000 7f6d5e63a700 1 -- 172.21.15.49:0/3425524518 shutdown_connections 2024-05-15T16:10:36.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.438+0000 7f6d5e63a700 1 --2- 172.21.15.49:0/3425524518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 0x7f6d58103f90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.438+0000 7f6d5e63a700 1 --2- 172.21.15.49:0/3425524518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d580ff6f0 0x7f6d580ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.438+0000 7f6d5e63a700 1 -- 172.21.15.49:0/3425524518 >> 172.21.15.49:0/3425524518 conn(0x7f6d58074e50 msgr2=0x7f6d58075260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:36.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.438+0000 7f6d5e63a700 1 -- 172.21.15.49:0/3425524518 shutdown_connections 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.438+0000 7f6d5e63a700 1 -- 172.21.15.49:0/3425524518 wait complete. 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.438+0000 7f6d5e63a700 1 Processor -- start 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.438+0000 7f6d5e63a700 1 -- start start 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d5e63a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d580ff6f0 0x7f6d5819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d5e63a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 0x7f6d5819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d5e63a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5819d420 con 0x7f6d581000a0 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d5e63a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d58196a90 con 0x7f6d580ff6f0 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 0x7f6d5819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 0x7f6d5819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51428/0 (socket says 172.21.15.49:51428) 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d577fe700 1 -- 172.21.15.49:0/1190398895 learned_addr learned my addr 172.21.15.49:0/1190398895 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d57fff700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d580ff6f0 0x7f6d5819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d577fe700 1 -- 172.21.15.49:0/1190398895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d580ff6f0 msgr2=0x7f6d5819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d577fe700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d580ff6f0 0x7f6d5819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d577fe700 1 -- 172.21.15.49:0/1190398895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c0095f0 con 0x7f6d581000a0 2024-05-15T16:10:36.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.439+0000 7f6d577fe700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 0x7f6d5819ce90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0092e0 tx=0x7f6d4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:36.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.440+0000 7f6d557fa700 1 -- 172.21.15.49:0/1190398895 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d4c01d070 con 0x7f6d581000a0 2024-05-15T16:10:36.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.440+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d58196d10 con 0x7f6d581000a0 2024-05-15T16:10:36.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.440+0000 7f6d557fa700 1 -- 172.21.15.49:0/1190398895 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d4c00f460 con 0x7f6d581000a0 2024-05-15T16:10:36.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.440+0000 7f6d557fa700 1 -- 172.21.15.49:0/1190398895 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d4c017630 con 0x7f6d581000a0 2024-05-15T16:10:36.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.440+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d58197200 con 0x7f6d581000a0 2024-05-15T16:10:36.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.440+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d58100a70 con 0x7f6d581000a0 2024-05-15T16:10:36.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.441+0000 7f6d557fa700 1 -- 172.21.15.49:0/1190398895 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6d4c00f5d0 con 0x7f6d581000a0 2024-05-15T16:10:36.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.442+0000 7f6d557fa700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d4006e200 0x7f6d400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:36.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.442+0000 7f6d557fa700 1 -- 172.21.15.49:0/1190398895 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6d4c08f5f0 con 0x7f6d581000a0 2024-05-15T16:10:36.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.442+0000 7f6d57fff700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d4006e200 0x7f6d400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:36.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.442+0000 7f6d57fff700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d4006e200 0x7f6d400706c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800ba60 tx=0x7f6d48005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:36.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.446+0000 7f6d557fa700 1 -- 172.21.15.49:0/1190398895 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6d4c05c400 con 0x7f6d581000a0 2024-05-15T16:10:36.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.672+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d58197f50 con 0x7f6d4006e200 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.683+0000 7f6d557fa700 1 -- 172.21.15.49:0/1190398895 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6d58197f50 con 0x7f6d4006e200 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (13m) 7m ago 25m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (25m) 7m ago 25m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (24m) 6m ago 24m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (11m) 7m ago 25m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (11m) 6m ago 24m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (13m) 7m ago 25m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (7m) 7m ago 19m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (7m) 7m ago 19m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:10:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (6m) 6m ago 19m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (19m) 6m ago 19m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (16m) 7m ago 26m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (15m) 6m ago 24m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (12m) 7m ago 27m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (11m) 6m ago 24m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (14m) 7m ago 25m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (14m) 6m ago 24m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (10m) 7m ago 23m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (10m) 7m ago 23m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (9m) 7m ago 22m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (8m) 6m ago 22m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (8m) 6m ago 21m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (7m) 6m ago 21m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:10:36.685 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (14m) 7m ago 24m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:10:36.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d4006e200 msgr2=0x7f6d400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d4006e200 0x7f6d400706c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800ba60 tx=0x7f6d48005d50 comp rx=0 tx=0).stop 2024-05-15T16:10:36.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 msgr2=0x7f6d5819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 0x7f6d5819ce90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0092e0 tx=0x7f6d4c00be20 comp rx=0 tx=0).stop 2024-05-15T16:10:36.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 shutdown_connections 2024-05-15T16:10:36.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d4006e200 0x7f6d400706c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d580ff6f0 0x7f6d5819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 --2- 172.21.15.49:0/1190398895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d581000a0 0x7f6d5819ce90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 >> 172.21.15.49:0/1190398895 conn(0x7f6d58074e50 msgr2=0x7f6d580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 shutdown_connections 2024-05-15T16:10:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.686+0000 7f6d5e63a700 1 -- 172.21.15.49:0/1190398895 wait complete. 2024-05-15T16:10:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.793+0000 7f0cabf9b700 1 -- 172.21.15.49:0/3706215978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4100ee0 msgr2=0x7f0ca41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.793+0000 7f0cabf9b700 1 --2- 172.21.15.49:0/3706215978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4100ee0 0x7f0ca41012c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400b0a0 tx=0x7f0c9400b3b0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.793+0000 7f0cabf9b700 1 -- 172.21.15.49:0/3706215978 shutdown_connections 2024-05-15T16:10:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.793+0000 7f0cabf9b700 1 --2- 172.21.15.49:0/3706215978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca4101890 0x7f0ca41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.793+0000 7f0cabf9b700 1 --2- 172.21.15.49:0/3706215978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4100ee0 0x7f0ca41012c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.793+0000 7f0cabf9b700 1 -- 172.21.15.49:0/3706215978 >> 172.21.15.49:0/3706215978 conn(0x7f0ca40fc790 msgr2=0x7f0ca40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.793+0000 7f0cabf9b700 1 -- 172.21.15.49:0/3706215978 shutdown_connections 2024-05-15T16:10:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.793+0000 7f0cabf9b700 1 -- 172.21.15.49:0/3706215978 wait complete. 2024-05-15T16:10:36.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.794+0000 7f0cabf9b700 1 Processor -- start 2024-05-15T16:10:36.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.794+0000 7f0cabf9b700 1 -- start start 2024-05-15T16:10:36.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.794+0000 7f0cabf9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca4100ee0 0x7f0ca419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.794+0000 7f0cabf9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4101890 0x7f0ca419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.794+0000 7f0cabf9b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca419d460 con 0x7f0ca4101890 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.794+0000 7f0cabf9b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca4196a90 con 0x7f0ca4100ee0 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.794+0000 7f0ca9536700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4101890 0x7f0ca419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.794+0000 7f0ca9536700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4101890 0x7f0ca419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51454/0 (socket says 172.21.15.49:51454) 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.794+0000 7f0ca9536700 1 -- 172.21.15.49:0/263131775 learned_addr learned my addr 172.21.15.49:0/263131775 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0ca9d37700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca4100ee0 0x7f0ca419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0ca9536700 1 -- 172.21.15.49:0/263131775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca4100ee0 msgr2=0x7f0ca419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0ca9536700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca4100ee0 0x7f0ca419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0ca9536700 1 -- 172.21.15.49:0/263131775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c94009d00 con 0x7f0ca4101890 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0ca9536700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4101890 0x7f0ca419ce40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c8f0 tx=0x7f0c9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0ca2ffd700 1 -- 172.21.15.49:0/263131775 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c98007a50 con 0x7f0ca4101890 2024-05-15T16:10:36.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0ca2ffd700 1 -- 172.21.15.49:0/263131775 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c98007bb0 con 0x7f0ca4101890 2024-05-15T16:10:36.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca4196d70 con 0x7f0ca4101890 2024-05-15T16:10:36.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0ca2ffd700 1 -- 172.21.15.49:0/263131775 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c98018630 con 0x7f0ca4101890 2024-05-15T16:10:36.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.795+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca41972c0 con 0x7f0ca4101890 2024-05-15T16:10:36.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.796+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca4109280 con 0x7f0ca4101890 2024-05-15T16:10:36.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.797+0000 7f0ca2ffd700 1 -- 172.21.15.49:0/263131775 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0c9800f450 con 0x7f0ca4101890 2024-05-15T16:10:36.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.797+0000 7f0ca2ffd700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c8c06e200 0x7f0c8c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:36.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.797+0000 7f0ca2ffd700 1 -- 172.21.15.49:0/263131775 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0c9808e1c0 con 0x7f0ca4101890 2024-05-15T16:10:36.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.797+0000 7f0ca9d37700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c8c06e200 0x7f0c8c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:36.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.798+0000 7f0ca9d37700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c8c06e200 0x7f0c8c0706c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400b830 tx=0x7f0c94006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:36.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:36.802+0000 7f0ca2ffd700 1 -- 172.21.15.49:0/263131775 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0c9805b050 con 0x7f0ca4101890 2024-05-15T16:10:36.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:36 smithi049 ceph-mon[112142]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:37.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.098+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0ca41023c0 con 0x7f0ca4101890 2024-05-15T16:10:37.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.098+0000 7f0ca2ffd700 1 -- 172.21.15.49:0/263131775 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f0c9805abe0 con 0x7f0ca4101890 2024-05-15T16:10:37.099 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:10:37.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:10:37.099 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:10:37.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:10:37.100 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:10:37.101 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:10:37.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c8c06e200 msgr2=0x7f0c8c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c8c06e200 0x7f0c8c0706c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400b830 tx=0x7f0c94006000 comp rx=0 tx=0).stop 2024-05-15T16:10:37.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4101890 msgr2=0x7f0ca419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4101890 0x7f0ca419ce40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c8f0 tx=0x7f0c9800ccb0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 shutdown_connections 2024-05-15T16:10:37.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c8c06e200 0x7f0c8c0706c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca4100ee0 0x7f0ca419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 --2- 172.21.15.49:0/263131775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca4101890 0x7f0ca419ce40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 >> 172.21.15.49:0/263131775 conn(0x7f0ca40fc790 msgr2=0x7f0ca40fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:37.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 shutdown_connections 2024-05-15T16:10:37.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.101+0000 7f0cabf9b700 1 -- 172.21.15.49:0/263131775 wait complete. 2024-05-15T16:10:37.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:36 smithi144 ceph-mon[86635]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:37.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.209+0000 7f5c4247b700 1 -- 172.21.15.49:0/1121015165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 msgr2=0x7f5c3c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.209+0000 7f5c4247b700 1 --2- 172.21.15.49:0/1121015165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 0x7f5c3c1058f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009910 tx=0x7f5c30009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:37.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.209+0000 7f5c4247b700 1 -- 172.21.15.49:0/1121015165 shutdown_connections 2024-05-15T16:10:37.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.209+0000 7f5c4247b700 1 --2- 172.21.15.49:0/1121015165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 0x7f5c3c1058f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.209+0000 7f5c4247b700 1 --2- 172.21.15.49:0/1121015165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c100ef0 0x7f5c3c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.209+0000 7f5c4247b700 1 -- 172.21.15.49:0/1121015165 >> 172.21.15.49:0/1121015165 conn(0x7f5c3c0fc7c0 msgr2=0x7f5c3c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:37.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.209+0000 7f5c4247b700 1 -- 172.21.15.49:0/1121015165 shutdown_connections 2024-05-15T16:10:37.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.209+0000 7f5c4247b700 1 -- 172.21.15.49:0/1121015165 wait complete. 2024-05-15T16:10:37.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.210+0000 7f5c4247b700 1 Processor -- start 2024-05-15T16:10:37.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.210+0000 7f5c4247b700 1 -- start start 2024-05-15T16:10:37.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.210+0000 7f5c4247b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c100ef0 0x7f5c3c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:37.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.210+0000 7f5c4247b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 0x7f5c3c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:37.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.210+0000 7f5c4247b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c19d430 con 0x7f5c3c1018a0 2024-05-15T16:10:37.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.210+0000 7f5c4247b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c196a30 con 0x7f5c3c100ef0 2024-05-15T16:10:37.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.210+0000 7f5c3b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 0x7f5c3c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:37.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.210+0000 7f5c3b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 0x7f5c3c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51460/0 (socket says 172.21.15.49:51460) 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.210+0000 7f5c3b7fe700 1 -- 172.21.15.49:0/3058301680 learned_addr learned my addr 172.21.15.49:0/3058301680 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c3bfff700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c100ef0 0x7f5c3c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c3b7fe700 1 -- 172.21.15.49:0/3058301680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c100ef0 msgr2=0x7f5c3c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c3b7fe700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c100ef0 0x7f5c3c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c3b7fe700 1 -- 172.21.15.49:0/3058301680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c300095f0 con 0x7f5c3c1018a0 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c3b7fe700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 0x7f5c3c19ce30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5c300092e0 tx=0x7f5c3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c397fa700 1 -- 172.21.15.49:0/3058301680 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c3001d070 con 0x7f5c3c1018a0 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3c196cb0 con 0x7f5c3c1018a0 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c397fa700 1 -- 172.21.15.49:0/3058301680 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c3000f460 con 0x7f5c3c1018a0 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c397fa700 1 -- 172.21.15.49:0/3058301680 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c30017630 con 0x7f5c3c1018a0 2024-05-15T16:10:37.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.211+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c1971a0 con 0x7f5c3c1018a0 2024-05-15T16:10:37.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.212+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3c1092f0 con 0x7f5c3c1018a0 2024-05-15T16:10:37.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.213+0000 7f5c397fa700 1 -- 172.21.15.49:0/3058301680 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5c3000f5d0 con 0x7f5c3c1018a0 2024-05-15T16:10:37.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.213+0000 7f5c397fa700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c2406e200 0x7f5c240706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:37.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.213+0000 7f5c397fa700 1 -- 172.21.15.49:0/3058301680 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5c3008f5f0 con 0x7f5c3c1018a0 2024-05-15T16:10:37.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.213+0000 7f5c3bfff700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c2406e200 0x7f5c240706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:37.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.214+0000 7f5c3bfff700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c2406e200 0x7f5c240706c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005ea0 tx=0x7f5c2c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.219+0000 7f5c397fa700 1 -- 172.21.15.49:0/3058301680 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5c3005c400 con 0x7f5c3c1018a0 2024-05-15T16:10:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.471+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5c3c197ef0 con 0x7f5c3c1018a0 2024-05-15T16:10:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.472+0000 7f5c397fa700 1 -- 172.21.15.49:0/3058301680 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f5c30005740 con 0x7f5c3c1018a0 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:10:37.473 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:10:37.474 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:10:37.475 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:10:37.476 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:10:37.476 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:10:37.476 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:10:37.476 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:10:37.476 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:10:37.476 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:10:37.476 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:10:37.476 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:10:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c2406e200 msgr2=0x7f5c240706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c2406e200 0x7f5c240706c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005ea0 tx=0x7f5c2c005e30 comp rx=0 tx=0).stop 2024-05-15T16:10:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 msgr2=0x7f5c3c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 0x7f5c3c19ce30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5c300092e0 tx=0x7f5c3000be20 comp rx=0 tx=0).stop 2024-05-15T16:10:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 shutdown_connections 2024-05-15T16:10:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c2406e200 0x7f5c240706c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c100ef0 0x7f5c3c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 --2- 172.21.15.49:0/3058301680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c3c1018a0 0x7f5c3c19ce30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 >> 172.21.15.49:0/3058301680 conn(0x7f5c3c0fc7c0 msgr2=0x7f5c3c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:37.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 shutdown_connections 2024-05-15T16:10:37.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.474+0000 7f5c4247b700 1 -- 172.21.15.49:0/3058301680 wait complete. 2024-05-15T16:10:37.478 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 -- 172.21.15.49:0/666433116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e40ffe50 msgr2=0x7fa0e4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 --2- 172.21.15.49:0/666433116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e40ffe50 0x7fa0e4100230 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009910 tx=0x7fa0d4009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 -- 172.21.15.49:0/666433116 shutdown_connections 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 --2- 172.21.15.49:0/666433116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0e4100800 0x7fa0e4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 --2- 172.21.15.49:0/666433116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e40ffe50 0x7fa0e4100230 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 -- 172.21.15.49:0/666433116 >> 172.21.15.49:0/666433116 conn(0x7fa0e40fb740 msgr2=0x7fa0e40fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 -- 172.21.15.49:0/666433116 shutdown_connections 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 -- 172.21.15.49:0/666433116 wait complete. 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 Processor -- start 2024-05-15T16:10:37.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.596+0000 7fa0e94eb700 1 -- start start 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e94eb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0e40ffe50 0x7fa0e41018f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e94eb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e4100800 0x7fa0e4101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e94eb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e4102370 con 0x7fa0e4100800 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e94eb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e41024b0 con 0x7fa0e40ffe50 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e4100800 0x7fa0e4101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e4100800 0x7fa0e4101e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51476/0 (socket says 172.21.15.49:51476) 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e27fc700 1 -- 172.21.15.49:0/2421930988 learned_addr learned my addr 172.21.15.49:0/2421930988 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e2ffd700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0e40ffe50 0x7fa0e41018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e27fc700 1 -- 172.21.15.49:0/2421930988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0e40ffe50 msgr2=0x7fa0e41018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e27fc700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0e40ffe50 0x7fa0e41018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e27fc700 1 -- 172.21.15.49:0/2421930988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40095f0 con 0x7fa0e4100800 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0e27fc700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e4100800 0x7fa0e4101e30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800d8d0 tx=0x7fa0d800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.597+0000 7fa0cbfff700 1 -- 172.21.15.49:0/2421930988 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0d80041f0 con 0x7fa0e4100800 2024-05-15T16:10:37.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.598+0000 7fa0cbfff700 1 -- 172.21.15.49:0/2421930988 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0d8007620 con 0x7fa0e4100800 2024-05-15T16:10:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.598+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e41a2df0 con 0x7fa0e4100800 2024-05-15T16:10:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.598+0000 7fa0cbfff700 1 -- 172.21.15.49:0/2421930988 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0d80056b0 con 0x7fa0e4100800 2024-05-15T16:10:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.598+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e41a3150 con 0x7fa0e4100800 2024-05-15T16:10:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.599+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e4101330 con 0x7fa0e4100800 2024-05-15T16:10:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.599+0000 7fa0cbfff700 1 -- 172.21.15.49:0/2421930988 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa0d8004750 con 0x7fa0e4100800 2024-05-15T16:10:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.600+0000 7fa0cbfff700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa0cc06e200 0x7fa0cc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.600+0000 7fa0cbfff700 1 -- 172.21.15.49:0/2421930988 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa0d808def0 con 0x7fa0e4100800 2024-05-15T16:10:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.600+0000 7fa0e2ffd700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa0cc06e200 0x7fa0cc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.600+0000 7fa0e2ffd700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa0cc06e200 0x7fa0cc0706c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400b5c0 tx=0x7fa0d401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.604+0000 7fa0cbfff700 1 -- 172.21.15.49:0/2421930988 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa0d8057250 con 0x7fa0e4100800 2024-05-15T16:10:37.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.832+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0e410cf30 con 0x7fa0cc06e200 2024-05-15T16:10:37.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.833+0000 7fa0cbfff700 1 -- 172.21.15.49:0/2421930988 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa0e410cf30 con 0x7fa0cc06e200 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:10:37.834 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:10:37.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.835+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa0cc06e200 msgr2=0x7fa0cc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.835+0000 7fa0e94eb700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa0cc06e200 0x7fa0cc0706c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400b5c0 tx=0x7fa0d401a040 comp rx=0 tx=0).stop 2024-05-15T16:10:37.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.835+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e4100800 msgr2=0x7fa0e4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.835+0000 7fa0e94eb700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e4100800 0x7fa0e4101e30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800d8d0 tx=0x7fa0d800dbe0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.835+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 shutdown_connections 2024-05-15T16:10:37.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.836+0000 7fa0e94eb700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa0cc06e200 0x7fa0cc0706c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.836+0000 7fa0e94eb700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0e40ffe50 0x7fa0e41018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.836+0000 7fa0e94eb700 1 --2- 172.21.15.49:0/2421930988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0e4100800 0x7fa0e4101e30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.836+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 >> 172.21.15.49:0/2421930988 conn(0x7fa0e40fb740 msgr2=0x7fa0e4069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.836+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 shutdown_connections 2024-05-15T16:10:37.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.836+0000 7fa0e94eb700 1 -- 172.21.15.49:0/2421930988 wait complete. 2024-05-15T16:10:37.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.945+0000 7f984d138700 1 -- 172.21.15.49:0/2086688940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848100ee0 msgr2=0x7f98481012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.945+0000 7f984d138700 1 --2- 172.21.15.49:0/2086688940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848100ee0 0x7f98481012c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9838009910 tx=0x7f9838009c20 comp rx=0 tx=0).stop 2024-05-15T16:10:37.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.945+0000 7f984d138700 1 -- 172.21.15.49:0/2086688940 shutdown_connections 2024-05-15T16:10:37.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.945+0000 7f984d138700 1 --2- 172.21.15.49:0/2086688940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9848101890 0x7f98481058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.945+0000 7f984d138700 1 --2- 172.21.15.49:0/2086688940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848100ee0 0x7f98481012c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.945+0000 7f984d138700 1 -- 172.21.15.49:0/2086688940 >> 172.21.15.49:0/2086688940 conn(0x7f98480fc790 msgr2=0x7f98480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:37.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.945+0000 7f984d138700 1 -- 172.21.15.49:0/2086688940 shutdown_connections 2024-05-15T16:10:37.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.945+0000 7f984d138700 1 -- 172.21.15.49:0/2086688940 wait complete. 2024-05-15T16:10:37.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f984d138700 1 Processor -- start 2024-05-15T16:10:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f984d138700 1 -- start start 2024-05-15T16:10:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f984d138700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9848100ee0 0x7f984819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f984d138700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848101890 0x7f984819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f984d138700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984819d460 con 0x7f9848101890 2024-05-15T16:10:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f984d138700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9848196a90 con 0x7f9848100ee0 2024-05-15T16:10:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f983e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848101890 0x7f984819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f983e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848101890 0x7f984819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51496/0 (socket says 172.21.15.49:51496) 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f983e59c700 1 -- 172.21.15.49:0/2209899099 learned_addr learned my addr 172.21.15.49:0/2209899099 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.946+0000 7f9846d9d700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9848100ee0 0x7f984819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.947+0000 7f983e59c700 1 -- 172.21.15.49:0/2209899099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9848100ee0 msgr2=0x7f984819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.947+0000 7f983e59c700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9848100ee0 0x7f984819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.947+0000 7f983e59c700 1 -- 172.21.15.49:0/2209899099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380095f0 con 0x7f9848101890 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.947+0000 7f983e59c700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848101890 0x7f984819ce40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f983000d900 tx=0x7f983000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.947+0000 7f9844d99700 1 -- 172.21.15.49:0/2209899099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9830009900 con 0x7f9848101890 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.947+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9848196d70 con 0x7f9848101890 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.947+0000 7f9844d99700 1 -- 172.21.15.49:0/2209899099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9830010460 con 0x7f9848101890 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.947+0000 7f9844d99700 1 -- 172.21.15.49:0/2209899099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983000f5f0 con 0x7f9848101890 2024-05-15T16:10:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.947+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98481972c0 con 0x7f9848101890 2024-05-15T16:10:37.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.948+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9848109280 con 0x7f9848101890 2024-05-15T16:10:37.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.949+0000 7f9844d99700 1 -- 172.21.15.49:0/2209899099 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f98300105d0 con 0x7f9848101890 2024-05-15T16:10:37.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.949+0000 7f9844d99700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f983406e200 0x7f98340706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:10:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.949+0000 7f9844d99700 1 -- 172.21.15.49:0/2209899099 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f983008dc60 con 0x7f9848101890 2024-05-15T16:10:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.949+0000 7f9846d9d700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f983406e200 0x7f98340706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:10:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.950+0000 7f9846d9d700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f983406e200 0x7f98340706c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f983801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:10:37.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:37.954+0000 7f9844d99700 1 -- 172.21.15.49:0/2209899099 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f983005aaf0 con 0x7f9848101890 2024-05-15T16:10:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:37 smithi144 ceph-mon[86635]: pgmap v407: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:37 smithi144 ceph-mon[86635]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:37 smithi144 ceph-mon[86635]: from='client.34802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:37 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/263131775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:10:38.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:37 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3058301680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:10:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:37 smithi049 ceph-mon[112142]: pgmap v407: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:37 smithi049 ceph-mon[112142]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:37 smithi049 ceph-mon[112142]: from='client.34802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:37 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/263131775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:10:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:37 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3058301680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:10:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.248+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f98481023c0 con 0x7f9848101890 2024-05-15T16:10:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.248+0000 7f9844d99700 1 -- 172.21.15.49:0/2209899099 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f98300201c0 con 0x7f9848101890 2024-05-15T16:10:38.249 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:10:38.249 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:10:38.249 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:10:38.249 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:10:38.249 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:10:38.249 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:10:38.249 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:10:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f983406e200 msgr2=0x7f98340706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f983406e200 0x7f98340706c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f983801a040 comp rx=0 tx=0).stop 2024-05-15T16:10:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848101890 msgr2=0x7f984819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:10:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848101890 0x7f984819ce40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f983000d900 tx=0x7f983000dc10 comp rx=0 tx=0).stop 2024-05-15T16:10:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 shutdown_connections 2024-05-15T16:10:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f983406e200 0x7f98340706c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9848100ee0 0x7f984819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 --2- 172.21.15.49:0/2209899099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9848101890 0x7f984819ce40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:10:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 >> 172.21.15.49:0/2209899099 conn(0x7f98480fc790 msgr2=0x7f98480fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:10:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 shutdown_connections 2024-05-15T16:10:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:10:38.250+0000 7f984d138700 1 -- 172.21.15.49:0/2209899099 wait complete. 2024-05-15T16:10:39.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:38 smithi144 ceph-mon[86635]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:39.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:38 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2209899099' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:10:39.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:38 smithi049 ceph-mon[112142]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:10:39.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:38 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2209899099' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:10:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:39 smithi144 ceph-mon[86635]: pgmap v408: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:39 smithi049 ceph-mon[112142]: pgmap v408: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:41 smithi144 ceph-mon[86635]: pgmap v409: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:41 smithi049 ceph-mon[112142]: pgmap v409: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:43 smithi144 ceph-mon[86635]: pgmap v410: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:43 smithi049 ceph-mon[112142]: pgmap v410: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:45 smithi144 ceph-mon[86635]: pgmap v411: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:45 smithi049 ceph-mon[112142]: pgmap v411: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:47 smithi144 ceph-mon[86635]: pgmap v412: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:47 smithi049 ceph-mon[112142]: pgmap v412: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:49 smithi144 ceph-mon[86635]: pgmap v413: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:49 smithi049 ceph-mon[112142]: pgmap v413: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:51.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:10:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:10:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:51 smithi144 ceph-mon[86635]: pgmap v414: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:51 smithi049 ceph-mon[112142]: pgmap v414: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:53 smithi144 ceph-mon[86635]: pgmap v415: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:53 smithi049 ceph-mon[112142]: pgmap v415: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:55 smithi144 ceph-mon[86635]: pgmap v416: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:55 smithi049 ceph-mon[112142]: pgmap v416: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:57 smithi049 ceph-mon[112142]: pgmap v417: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:10:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:57 smithi144 ceph-mon[86635]: pgmap v417: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:10:59 smithi144 ceph-mon[86635]: pgmap v418: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:10:59 smithi049 ceph-mon[112142]: pgmap v418: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:01 smithi144 ceph-mon[86635]: pgmap v419: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:01 smithi049 ceph-mon[112142]: pgmap v419: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:03 smithi144 ceph-mon[86635]: pgmap v420: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:03 smithi049 ceph-mon[112142]: pgmap v420: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:05 smithi144 ceph-mon[86635]: pgmap v421: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:11:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:05 smithi049 ceph-mon[112142]: pgmap v421: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:11:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:07 smithi144 ceph-mon[86635]: pgmap v422: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:07 smithi049 ceph-mon[112142]: pgmap v422: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:08.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.374+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2940044607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 msgr2=0x7fcd381047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:08.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.374+0000 7fcd3e7bd700 1 --2- 172.21.15.49:0/2940044607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 0x7fcd381047d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009910 tx=0x7fcd2c009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:08.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.374+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2940044607 shutdown_connections 2024-05-15T16:11:08.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.374+0000 7fcd3e7bd700 1 --2- 172.21.15.49:0/2940044607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 0x7fcd381047d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.374+0000 7fcd3e7bd700 1 --2- 172.21.15.49:0/2940044607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380ffdd0 0x7fcd381001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.374+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2940044607 >> 172.21.15.49:0/2940044607 conn(0x7fcd380fb6a0 msgr2=0x7fcd380fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:08.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.374+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2940044607 shutdown_connections 2024-05-15T16:11:08.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.374+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2940044607 wait complete. 2024-05-15T16:11:08.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.375+0000 7fcd3e7bd700 1 Processor -- start 2024-05-15T16:11:08.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.375+0000 7fcd3e7bd700 1 -- start start 2024-05-15T16:11:08.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.375+0000 7fcd3e7bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380ffdd0 0x7fcd3819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:08.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.375+0000 7fcd3e7bd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 0x7fcd3819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:08.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.375+0000 7fcd3e7bd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3819d460 con 0x7fcd38100780 2024-05-15T16:11:08.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.375+0000 7fcd3e7bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd38196a40 con 0x7fcd380ffdd0 2024-05-15T16:11:08.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd377fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 0x7fcd3819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd377fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 0x7fcd3819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58268/0 (socket says 172.21.15.49:58268) 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd377fe700 1 -- 172.21.15.49:0/2025792899 learned_addr learned my addr 172.21.15.49:0/2025792899 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd37fff700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380ffdd0 0x7fcd3819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd377fe700 1 -- 172.21.15.49:0/2025792899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380ffdd0 msgr2=0x7fcd3819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd377fe700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380ffdd0 0x7fcd3819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd377fe700 1 -- 172.21.15.49:0/2025792899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c0095f0 con 0x7fcd38100780 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd377fe700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 0x7fcd3819ce40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00bd70 tx=0x7fcd2c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd357fa700 1 -- 172.21.15.49:0/2025792899 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd2c01d070 con 0x7fcd38100780 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd38196cc0 con 0x7fcd38100780 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd357fa700 1 -- 172.21.15.49:0/2025792899 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd2c0056e0 con 0x7fcd38100780 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd357fa700 1 -- 172.21.15.49:0/2025792899 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd2c00fe50 con 0x7fcd38100780 2024-05-15T16:11:08.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.376+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd381971b0 con 0x7fcd38100780 2024-05-15T16:11:08.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.377+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd381012b0 con 0x7fcd38100780 2024-05-15T16:11:08.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.378+0000 7fcd357fa700 1 -- 172.21.15.49:0/2025792899 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcd2c00f680 con 0x7fcd38100780 2024-05-15T16:11:08.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.378+0000 7fcd357fa700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd2006e200 0x7fcd200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:08.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.379+0000 7fcd357fa700 1 -- 172.21.15.49:0/2025792899 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcd2c08f570 con 0x7fcd38100780 2024-05-15T16:11:08.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.379+0000 7fcd37fff700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd2006e200 0x7fcd200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:08.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.379+0000 7fcd37fff700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd2006e200 0x7fcd200706c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcd28005950 tx=0x7fcd2800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:08.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.383+0000 7fcd357fa700 1 -- 172.21.15.49:0/2025792899 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcd2c05c400 con 0x7fcd38100780 2024-05-15T16:11:08.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.612+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd3810cde0 con 0x7fcd2006e200 2024-05-15T16:11:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.614+0000 7fcd357fa700 1 -- 172.21.15.49:0/2025792899 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcd3810cde0 con 0x7fcd2006e200 2024-05-15T16:11:08.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd2006e200 msgr2=0x7fcd200706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd2006e200 0x7fcd200706c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcd28005950 tx=0x7fcd2800b410 comp rx=0 tx=0).stop 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 msgr2=0x7fcd3819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 0x7fcd3819ce40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00bd70 tx=0x7fcd2c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 shutdown_connections 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcd2006e200 0x7fcd200706c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380ffdd0 0x7fcd3819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 --2- 172.21.15.49:0/2025792899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd38100780 0x7fcd3819ce40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 >> 172.21.15.49:0/2025792899 conn(0x7fcd380fb6a0 msgr2=0x7fcd380692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 shutdown_connections 2024-05-15T16:11:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.616+0000 7fcd3e7bd700 1 -- 172.21.15.49:0/2025792899 wait complete. 2024-05-15T16:11:08.627 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:11:08.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.727+0000 7f8d8dc46700 1 -- 172.21.15.49:0/2880065182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ff430 msgr2=0x7f8d880ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:08.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.727+0000 7f8d8dc46700 1 --2- 172.21.15.49:0/2880065182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ff430 0x7f8d880ff810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009910 tx=0x7f8d78009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:08.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.727+0000 7f8d8dc46700 1 -- 172.21.15.49:0/2880065182 shutdown_connections 2024-05-15T16:11:08.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.727+0000 7f8d8dc46700 1 --2- 172.21.15.49:0/2880065182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d880ffde0 0x7f8d88105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.727+0000 7f8d8dc46700 1 --2- 172.21.15.49:0/2880065182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ff430 0x7f8d880ff810 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.727+0000 7f8d8dc46700 1 -- 172.21.15.49:0/2880065182 >> 172.21.15.49:0/2880065182 conn(0x7f8d88075ff0 msgr2=0x7f8d880fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:08.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.727+0000 7f8d8dc46700 1 -- 172.21.15.49:0/2880065182 shutdown_connections 2024-05-15T16:11:08.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.727+0000 7f8d8dc46700 1 -- 172.21.15.49:0/2880065182 wait complete. 2024-05-15T16:11:08.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d8dc46700 1 Processor -- start 2024-05-15T16:11:08.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d8dc46700 1 -- start start 2024-05-15T16:11:08.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d8dc46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d880ff430 0x7f8d88196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:08.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d8dc46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ffde0 0x7f8d88197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:08.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d8dc46700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d881978e0 con 0x7f8d880ffde0 2024-05-15T16:11:08.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d8dc46700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8819a550 con 0x7f8d880ff430 2024-05-15T16:11:08.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d86ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ffde0 0x7f8d88197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:08.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d86ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ffde0 0x7f8d88197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58286/0 (socket says 172.21.15.49:58286) 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d86ffd700 1 -- 172.21.15.49:0/3218154059 learned_addr learned my addr 172.21.15.49:0/3218154059 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.728+0000 7f8d877fe700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d880ff430 0x7f8d88196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.729+0000 7f8d86ffd700 1 -- 172.21.15.49:0/3218154059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d880ff430 msgr2=0x7f8d88196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.729+0000 7f8d86ffd700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d880ff430 0x7f8d88196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.729+0000 7f8d86ffd700 1 -- 172.21.15.49:0/3218154059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d780095f0 con 0x7f8d880ffde0 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.729+0000 7f8d86ffd700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ffde0 0x7f8d88197250 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00b4d0 tx=0x7f8d7c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.729+0000 7f8d84ff9700 1 -- 172.21.15.49:0/3218154059 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7c011800 con 0x7f8d880ffde0 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.729+0000 7f8d84ff9700 1 -- 172.21.15.49:0/3218154059 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d7c011e20 con 0x7f8d880ffde0 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.729+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8819a830 con 0x7f8d880ffde0 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.729+0000 7f8d84ff9700 1 -- 172.21.15.49:0/3218154059 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7c00f2e0 con 0x7f8d880ffde0 2024-05-15T16:11:08.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.729+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8819ad80 con 0x7f8d880ffde0 2024-05-15T16:11:08.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.730+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d881092b0 con 0x7f8d880ffde0 2024-05-15T16:11:08.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.731+0000 7f8d84ff9700 1 -- 172.21.15.49:0/3218154059 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8d7c010380 con 0x7f8d880ffde0 2024-05-15T16:11:08.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.731+0000 7f8d84ff9700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d7006e200 0x7f8d700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:08.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.731+0000 7f8d84ff9700 1 -- 172.21.15.49:0/3218154059 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8d7c08dc70 con 0x7f8d880ffde0 2024-05-15T16:11:08.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.731+0000 7f8d877fe700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d7006e200 0x7f8d700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:08.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.732+0000 7f8d877fe700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d7006e200 0x7f8d700706c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c00 tx=0x7f8d7801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:08.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.736+0000 7f8d84ff9700 1 -- 172.21.15.49:0/3218154059 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8d7c056f50 con 0x7f8d880ffde0 2024-05-15T16:11:08.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.963+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d88197fe0 con 0x7f8d7006e200 2024-05-15T16:11:08.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.964+0000 7f8d84ff9700 1 -- 172.21.15.49:0/3218154059 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8d88197fe0 con 0x7f8d7006e200 2024-05-15T16:11:08.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d7006e200 msgr2=0x7f8d700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:08.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d7006e200 0x7f8d700706c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c00 tx=0x7f8d7801a040 comp rx=0 tx=0).stop 2024-05-15T16:11:08.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ffde0 msgr2=0x7f8d88197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:08.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ffde0 0x7f8d88197250 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00b4d0 tx=0x7f8d7c00b7e0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 shutdown_connections 2024-05-15T16:11:08.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d7006e200 0x7f8d700706c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d880ff430 0x7f8d88196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 --2- 172.21.15.49:0/3218154059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d880ffde0 0x7f8d88197250 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:08.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 >> 172.21.15.49:0/3218154059 conn(0x7f8d88075ff0 msgr2=0x7f8d88104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:08.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 shutdown_connections 2024-05-15T16:11:08.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:08.967+0000 7f8d8dc46700 1 -- 172.21.15.49:0/3218154059 wait complete. 2024-05-15T16:11:09.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.076+0000 7fd170583700 1 -- 172.21.15.49:0/2438159694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 msgr2=0x7fd168102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.076+0000 7fd170583700 1 --2- 172.21.15.49:0/2438159694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 0x7fd168102180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd15800b0a0 tx=0x7fd15800b3b0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.076+0000 7fd170583700 1 -- 172.21.15.49:0/2438159694 shutdown_connections 2024-05-15T16:11:09.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.076+0000 7fd170583700 1 --2- 172.21.15.49:0/2438159694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168102750 0x7fd16810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.076+0000 7fd170583700 1 --2- 172.21.15.49:0/2438159694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 0x7fd168102180 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.076+0000 7fd170583700 1 -- 172.21.15.49:0/2438159694 >> 172.21.15.49:0/2438159694 conn(0x7fd1680fb6a0 msgr2=0x7fd1680fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:09.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.076+0000 7fd170583700 1 -- 172.21.15.49:0/2438159694 shutdown_connections 2024-05-15T16:11:09.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.076+0000 7fd170583700 1 -- 172.21.15.49:0/2438159694 wait complete. 2024-05-15T16:11:09.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.077+0000 7fd170583700 1 Processor -- start 2024-05-15T16:11:09.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.077+0000 7fd170583700 1 -- start start 2024-05-15T16:11:09.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.077+0000 7fd170583700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 0x7fd16819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:09.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.077+0000 7fd170583700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168102750 0x7fd16819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:09.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.077+0000 7fd170583700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16819d430 con 0x7fd168101da0 2024-05-15T16:11:09.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.077+0000 7fd170583700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd168196a10 con 0x7fd168102750 2024-05-15T16:11:09.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.077+0000 7fd16e31f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 0x7fd16819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:09.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd16e31f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 0x7fd16819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58298/0 (socket says 172.21.15.49:58298) 2024-05-15T16:11:09.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd16e31f700 1 -- 172.21.15.49:0/1843778713 learned_addr learned my addr 172.21.15.49:0/1843778713 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd16db1e700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168102750 0x7fd16819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd16e31f700 1 -- 172.21.15.49:0/1843778713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168102750 msgr2=0x7fd16819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd16e31f700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168102750 0x7fd16819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd16e31f700 1 -- 172.21.15.49:0/1843778713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158009d00 con 0x7fd168101da0 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd16e31f700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 0x7fd16819c8d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd158000f80 tx=0x7fd158007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd1677fe700 1 -- 172.21.15.49:0/1843778713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15800e040 con 0x7fd168101da0 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd168196c90 con 0x7fd168101da0 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd1677fe700 1 -- 172.21.15.49:0/1843778713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1580096a0 con 0x7fd168101da0 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd168197180 con 0x7fd168101da0 2024-05-15T16:11:09.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.078+0000 7fd1677fe700 1 -- 172.21.15.49:0/1843778713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15801d6b0 con 0x7fd168101da0 2024-05-15T16:11:09.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.079+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd168105550 con 0x7fd168101da0 2024-05-15T16:11:09.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.080+0000 7fd1677fe700 1 -- 172.21.15.49:0/1843778713 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd158003c10 con 0x7fd168101da0 2024-05-15T16:11:09.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.080+0000 7fd1677fe700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd15006e200 0x7fd1500706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:09.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.081+0000 7fd1677fe700 1 -- 172.21.15.49:0/1843778713 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd15808f840 con 0x7fd168101da0 2024-05-15T16:11:09.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.081+0000 7fd16db1e700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd15006e200 0x7fd1500706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:09.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.081+0000 7fd16db1e700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd15006e200 0x7fd1500706c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd168197bd0 tx=0x7fd15c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:09.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.085+0000 7fd1677fe700 1 -- 172.21.15.49:0/1843778713 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd15805c650 con 0x7fd168101da0 2024-05-15T16:11:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.301+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd168197ed0 con 0x7fd15006e200 2024-05-15T16:11:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.314+0000 7fd1677fe700 1 -- 172.21.15.49:0/1843778713 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd168197ed0 con 0x7fd15006e200 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (14m) 7m ago 25m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (26m) 7m ago 26m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (25m) 7m ago 25m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (11m) 7m ago 26m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (11m) 7m ago 25m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (13m) 7m ago 25m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (8m) 7m ago 20m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (7m) 7m ago 20m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (7m) 7m ago 20m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (20m) 7m ago 20m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (16m) 7m ago 27m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (15m) 7m ago 24m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (12m) 7m ago 27m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:11:09.315 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (12m) 7m ago 24m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:11:09.316 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (15m) 7m ago 26m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:11:09.316 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (15m) 7m ago 24m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:11:09.316 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (11m) 7m ago 24m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:11:09.316 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (10m) 7m ago 23m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:11:09.316 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (10m) 7m ago 23m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:11:09.316 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (9m) 7m ago 22m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:11:09.316 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (8m) 7m ago 22m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:11:09.316 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (8m) 7m ago 21m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:11:09.316 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (14m) 7m ago 25m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:11:09.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd15006e200 msgr2=0x7fd1500706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd15006e200 0x7fd1500706c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd168197bd0 tx=0x7fd15c00b410 comp rx=0 tx=0).stop 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 msgr2=0x7fd16819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 0x7fd16819c8d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd158000f80 tx=0x7fd158007b20 comp rx=0 tx=0).stop 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 shutdown_connections 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd15006e200 0x7fd1500706c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd168101da0 0x7fd16819c8d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 --2- 172.21.15.49:0/1843778713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168102750 0x7fd16819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 >> 172.21.15.49:0/1843778713 conn(0x7fd1680fb6a0 msgr2=0x7fd168103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 shutdown_connections 2024-05-15T16:11:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.317+0000 7fd170583700 1 -- 172.21.15.49:0/1843778713 wait complete. 2024-05-15T16:11:09.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 -- 172.21.15.49:0/4055327288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 msgr2=0x7fbf84073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 --2- 172.21.15.49:0/4055327288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 0x7fbf84073750 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009910 tx=0x7fbf74009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:09.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 -- 172.21.15.49:0/4055327288 shutdown_connections 2024-05-15T16:11:09.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 --2- 172.21.15.49:0/4055327288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf84073c90 0x7fbf8410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 --2- 172.21.15.49:0/4055327288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 0x7fbf84073750 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 -- 172.21.15.49:0/4055327288 >> 172.21.15.49:0/4055327288 conn(0x7fbf840fc7d0 msgr2=0x7fbf840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:09.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 -- 172.21.15.49:0/4055327288 shutdown_connections 2024-05-15T16:11:09.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 -- 172.21.15.49:0/4055327288 wait complete. 2024-05-15T16:11:09.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 Processor -- start 2024-05-15T16:11:09.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.425+0000 7fbf8a797700 1 -- start start 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf8a797700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 0x7fbf84198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf8a797700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf84073c90 0x7fbf84199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf8a797700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf84199a20 con 0x7fbf84073370 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf8a797700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8419d6a0 con 0x7fbf84073c90 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf83fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 0x7fbf84198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf83fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 0x7fbf84198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58316/0 (socket says 172.21.15.49:58316) 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf83fff700 1 -- 172.21.15.49:0/3696829697 learned_addr learned my addr 172.21.15.49:0/3696829697 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf837fe700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf84073c90 0x7fbf84199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf83fff700 1 -- 172.21.15.49:0/3696829697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf84073c90 msgr2=0x7fbf84199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf83fff700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf84073c90 0x7fbf84199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf83fff700 1 -- 172.21.15.49:0/3696829697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf740095f0 con 0x7fbf84073370 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf837fe700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf84073c90 0x7fbf84199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.426+0000 7fbf83fff700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 0x7fbf84198e00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf74006010 tx=0x7fbf7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:09.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.427+0000 7fbf817fa700 1 -- 172.21.15.49:0/3696829697 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf7401d070 con 0x7fbf84073370 2024-05-15T16:11:09.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.427+0000 7fbf817fa700 1 -- 172.21.15.49:0/3696829697 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf740056e0 con 0x7fbf84073370 2024-05-15T16:11:09.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.427+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8419d920 con 0x7fbf84073370 2024-05-15T16:11:09.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.427+0000 7fbf817fa700 1 -- 172.21.15.49:0/3696829697 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf74017400 con 0x7fbf84073370 2024-05-15T16:11:09.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.427+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8419de10 con 0x7fbf84073370 2024-05-15T16:11:09.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.428+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf84074a20 con 0x7fbf84073370 2024-05-15T16:11:09.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.428+0000 7fbf817fa700 1 -- 172.21.15.49:0/3696829697 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbf74003a60 con 0x7fbf84073370 2024-05-15T16:11:09.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.429+0000 7fbf817fa700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf6c06e200 0x7fbf6c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:09.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.429+0000 7fbf817fa700 1 -- 172.21.15.49:0/3696829697 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbf7408f560 con 0x7fbf84073370 2024-05-15T16:11:09.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.429+0000 7fbf837fe700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf6c06e200 0x7fbf6c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:09.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.429+0000 7fbf837fe700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf6c06e200 0x7fbf6c0706c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005950 tx=0x7fbf780058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:09.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.434+0000 7fbf817fa700 1 -- 172.21.15.49:0/3696829697 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbf74058840 con 0x7fbf84073370 2024-05-15T16:11:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.730+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbf84107960 con 0x7fbf84073370 2024-05-15T16:11:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.730+0000 7fbf817fa700 1 -- 172.21.15.49:0/3696829697 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fbf7405bf00 con 0x7fbf84073370 2024-05-15T16:11:09.731 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:11:09.731 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:11:09.731 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:11:09.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:11:09.731 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:11:09.731 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:11:09.732 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:11:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf6c06e200 msgr2=0x7fbf6c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf6c06e200 0x7fbf6c0706c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005950 tx=0x7fbf780058e0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 msgr2=0x7fbf84198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 0x7fbf84198e00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf74006010 tx=0x7fbf7400be20 comp rx=0 tx=0).stop 2024-05-15T16:11:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 shutdown_connections 2024-05-15T16:11:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf6c06e200 0x7fbf6c0706c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf84073370 0x7fbf84198e00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 --2- 172.21.15.49:0/3696829697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf84073c90 0x7fbf84199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 >> 172.21.15.49:0/3696829697 conn(0x7fbf840fc7d0 msgr2=0x7fbf841054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 shutdown_connections 2024-05-15T16:11:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.733+0000 7fbf8a797700 1 -- 172.21.15.49:0/3696829697 wait complete. 2024-05-15T16:11:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.842+0000 7f89f6617700 1 -- 172.21.15.49:0/1278862906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 msgr2=0x7f89f01012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.842+0000 7f89f6617700 1 --2- 172.21.15.49:0/1278862906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 0x7f89f01012d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009910 tx=0x7f89e0009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.842+0000 7f89f6617700 1 -- 172.21.15.49:0/1278862906 shutdown_connections 2024-05-15T16:11:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.842+0000 7f89f6617700 1 --2- 172.21.15.49:0/1278862906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89f01018a0 0x7f89f01058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.842+0000 7f89f6617700 1 --2- 172.21.15.49:0/1278862906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 0x7f89f01012d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.842+0000 7f89f6617700 1 -- 172.21.15.49:0/1278862906 >> 172.21.15.49:0/1278862906 conn(0x7f89f00fc7a0 msgr2=0x7f89f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.842+0000 7f89f6617700 1 -- 172.21.15.49:0/1278862906 shutdown_connections 2024-05-15T16:11:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.842+0000 7f89f6617700 1 -- 172.21.15.49:0/1278862906 wait complete. 2024-05-15T16:11:09.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.843+0000 7f89f6617700 1 Processor -- start 2024-05-15T16:11:09.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.843+0000 7f89f6617700 1 -- start start 2024-05-15T16:11:09.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.843+0000 7f89f6617700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 0x7f89f019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.843+0000 7f89f6617700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89f01018a0 0x7f89f019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.843+0000 7f89f6617700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f019d4f0 con 0x7f89f0100ef0 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.843+0000 7f89f6617700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0196a60 con 0x7f89f01018a0 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.843+0000 7f89effff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 0x7f89f019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.843+0000 7f89effff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 0x7f89f019c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58346/0 (socket says 172.21.15.49:58346) 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.843+0000 7f89effff700 1 -- 172.21.15.49:0/4004177665 learned_addr learned my addr 172.21.15.49:0/4004177665 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89e7fff700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89f01018a0 0x7f89f019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89effff700 1 -- 172.21.15.49:0/4004177665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89f01018a0 msgr2=0x7f89f019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89effff700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89f01018a0 0x7f89f019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89effff700 1 -- 172.21.15.49:0/4004177665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e00095f0 con 0x7f89f0100ef0 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89e7fff700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89f01018a0 0x7f89f019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89effff700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 0x7f89f019c920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89edffb700 1 -- 172.21.15.49:0/4004177665 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e001d070 con 0x7f89f0100ef0 2024-05-15T16:11:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f0196ce0 con 0x7f89f0100ef0 2024-05-15T16:11:09.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89edffb700 1 -- 172.21.15.49:0/4004177665 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89e00056e0 con 0x7f89f0100ef0 2024-05-15T16:11:09.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f01971d0 con 0x7f89f0100ef0 2024-05-15T16:11:09.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.844+0000 7f89edffb700 1 -- 172.21.15.49:0/4004177665 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e000fe80 con 0x7f89f0100ef0 2024-05-15T16:11:09.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.845+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f0109290 con 0x7f89f0100ef0 2024-05-15T16:11:09.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.846+0000 7f89edffb700 1 -- 172.21.15.49:0/4004177665 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f89e000f6f0 con 0x7f89f0100ef0 2024-05-15T16:11:09.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.846+0000 7f89edffb700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f89dc06e210 0x7f89dc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:09.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.846+0000 7f89edffb700 1 -- 172.21.15.49:0/4004177665 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f89e008f5f0 con 0x7f89f0100ef0 2024-05-15T16:11:09.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.846+0000 7f89e7fff700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f89dc06e210 0x7f89dc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:09.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.847+0000 7f89e7fff700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f89dc06e210 0x7f89dc0706d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89d8005fd0 tx=0x7f89d8005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:09.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:09.851+0000 7f89edffb700 1 -- 172.21.15.49:0/4004177665 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f89e005c400 con 0x7f89f0100ef0 2024-05-15T16:11:10.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.104+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f89f0197f20 con 0x7f89f0100ef0 2024-05-15T16:11:10.105 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:09 smithi049 ceph-mon[112142]: pgmap v423: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:10.105 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:09 smithi049 ceph-mon[112142]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:10.105 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:09 smithi049 ceph-mon[112142]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:10.105 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:09 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3696829697' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:11:10.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.105+0000 7f89edffb700 1 -- 172.21.15.49:0/4004177665 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f89e005bf90 con 0x7f89f0100ef0 2024-05-15T16:11:10.106 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:11:10.106 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:11:10.106 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:11:10.106 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:11:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:11:10.106 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:11:10.106 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:11:10.106 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:11:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:11:10.108 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:11:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:11:10.109 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:11:10.109 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:11:10.109 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:11:10.109 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f89dc06e210 msgr2=0x7f89dc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f89dc06e210 0x7f89dc0706d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89d8005fd0 tx=0x7f89d8005ee0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 msgr2=0x7f89f019c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 0x7f89f019c920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e000be20 comp rx=0 tx=0).stop 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 shutdown_connections 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f89dc06e210 0x7f89dc0706d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89f0100ef0 0x7f89f019c920 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 --2- 172.21.15.49:0/4004177665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89f01018a0 0x7f89f019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 >> 172.21.15.49:0/4004177665 conn(0x7f89f00fc7a0 msgr2=0x7f89f00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 shutdown_connections 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.107+0000 7f89f6617700 1 -- 172.21.15.49:0/4004177665 wait complete. 2024-05-15T16:11:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:11:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:09 smithi144 ceph-mon[86635]: pgmap v423: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:10.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:09 smithi144 ceph-mon[86635]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:10.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:09 smithi144 ceph-mon[86635]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:10.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:09 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3696829697' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:11:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.219+0000 7f362374e700 1 -- 172.21.15.49:0/2300624621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 msgr2=0x7f361c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.219+0000 7f362374e700 1 --2- 172.21.15.49:0/2300624621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 0x7f361c1047c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f360c009910 tx=0x7f360c009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.219+0000 7f362374e700 1 -- 172.21.15.49:0/2300624621 shutdown_connections 2024-05-15T16:11:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.219+0000 7f362374e700 1 --2- 172.21.15.49:0/2300624621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 0x7f361c1047c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.219+0000 7f362374e700 1 --2- 172.21.15.49:0/2300624621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f361c0ffdc0 0x7f361c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.219+0000 7f362374e700 1 -- 172.21.15.49:0/2300624621 >> 172.21.15.49:0/2300624621 conn(0x7f361c076800 msgr2=0x7f361c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.219+0000 7f362374e700 1 -- 172.21.15.49:0/2300624621 shutdown_connections 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.219+0000 7f362374e700 1 -- 172.21.15.49:0/2300624621 wait complete. 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f362374e700 1 Processor -- start 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f362374e700 1 -- start start 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f362374e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f361c0ffdc0 0x7f361c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f362374e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 0x7f361c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f362374e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1969a0 con 0x7f361c100770 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f362374e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c196b10 con 0x7f361c0ffdc0 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f3620ce9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 0x7f361c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f3620ce9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 0x7f361c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58352/0 (socket says 172.21.15.49:58352) 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f3620ce9700 1 -- 172.21.15.49:0/3408173290 learned_addr learned my addr 172.21.15.49:0/3408173290 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:10.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f36214ea700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f361c0ffdc0 0x7f361c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f3620ce9700 1 -- 172.21.15.49:0/3408173290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f361c0ffdc0 msgr2=0x7f361c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f3620ce9700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f361c0ffdc0 0x7f361c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.220+0000 7f3620ce9700 1 -- 172.21.15.49:0/3408173290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c0095f0 con 0x7f361c100770 2024-05-15T16:11:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.221+0000 7f3620ce9700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 0x7f361c19ce30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f360c00bd70 tx=0x7f360c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.221+0000 7f36127fc700 1 -- 172.21.15.49:0/3408173290 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f360c01d070 con 0x7f361c100770 2024-05-15T16:11:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.221+0000 7f36127fc700 1 -- 172.21.15.49:0/3408173290 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f360c0056e0 con 0x7f361c100770 2024-05-15T16:11:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.221+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c196d90 con 0x7f361c100770 2024-05-15T16:11:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.221+0000 7f36127fc700 1 -- 172.21.15.49:0/3408173290 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f360c00fe50 con 0x7f361c100770 2024-05-15T16:11:10.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.221+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c197280 con 0x7f361c100770 2024-05-15T16:11:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.222+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c1012a0 con 0x7f361c100770 2024-05-15T16:11:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.222+0000 7f36127fc700 1 -- 172.21.15.49:0/3408173290 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f360c00f680 con 0x7f361c100770 2024-05-15T16:11:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.223+0000 7f36127fc700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f360806e200 0x7f36080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.223+0000 7f36127fc700 1 -- 172.21.15.49:0/3408173290 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f360c08f4d0 con 0x7f361c100770 2024-05-15T16:11:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.223+0000 7f36214ea700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f360806e200 0x7f36080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.223+0000 7f36214ea700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f360806e200 0x7f36080706c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3618007be0 tx=0x7f36180073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:10.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.228+0000 7f36127fc700 1 -- 172.21.15.49:0/3408173290 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f360c0587b0 con 0x7f361c100770 2024-05-15T16:11:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.454+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f361c197ef0 con 0x7f360806e200 2024-05-15T16:11:10.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.456+0000 7f36127fc700 1 -- 172.21.15.49:0/3408173290 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f361c197ef0 con 0x7f360806e200 2024-05-15T16:11:10.456 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:11:10.456 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:11:10.456 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:11:10.456 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:11:10.457 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:11:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.458+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f360806e200 msgr2=0x7f36080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.458+0000 7f362374e700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f360806e200 0x7f36080706c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3618007be0 tx=0x7f36180073d0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.458+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 msgr2=0x7f361c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.458+0000 7f362374e700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 0x7f361c19ce30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f360c00bd70 tx=0x7f360c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.458+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 shutdown_connections 2024-05-15T16:11:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.458+0000 7f362374e700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f360806e200 0x7f36080706c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.458+0000 7f362374e700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f361c0ffdc0 0x7f361c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.458+0000 7f362374e700 1 --2- 172.21.15.49:0/3408173290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f361c100770 0x7f361c19ce30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.458+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 >> 172.21.15.49:0/3408173290 conn(0x7f361c076800 msgr2=0x7f361c0688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.459+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 shutdown_connections 2024-05-15T16:11:10.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.459+0000 7f362374e700 1 -- 172.21.15.49:0/3408173290 wait complete. 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.567+0000 7f9ac2567700 1 -- 172.21.15.49:0/603082364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ff3f0 msgr2=0x7f9abc0ff7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.567+0000 7f9ac2567700 1 --2- 172.21.15.49:0/603082364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ff3f0 0x7f9abc0ff7d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009910 tx=0x7f9aac009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.567+0000 7f9ac2567700 1 -- 172.21.15.49:0/603082364 shutdown_connections 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.567+0000 7f9ac2567700 1 --2- 172.21.15.49:0/603082364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0ffda0 0x7f9abc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.567+0000 7f9ac2567700 1 --2- 172.21.15.49:0/603082364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ff3f0 0x7f9abc0ff7d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.567+0000 7f9ac2567700 1 -- 172.21.15.49:0/603082364 >> 172.21.15.49:0/603082364 conn(0x7f9abc075ff0 msgr2=0x7f9abc0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.567+0000 7f9ac2567700 1 -- 172.21.15.49:0/603082364 shutdown_connections 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.567+0000 7f9ac2567700 1 -- 172.21.15.49:0/603082364 wait complete. 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.567+0000 7f9ac2567700 1 Processor -- start 2024-05-15T16:11:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9ac2567700 1 -- start start 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9ac2567700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0ff3f0 0x7f9abc19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9ac2567700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ffda0 0x7f9abc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9ac2567700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc19d400 con 0x7f9abc0ffda0 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9ac2567700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc196a30 con 0x7f9abc0ff3f0 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9abb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ffda0 0x7f9abc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9abb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ffda0 0x7f9abc19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58364/0 (socket says 172.21.15.49:58364) 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9abb7fe700 1 -- 172.21.15.49:0/2442747882 learned_addr learned my addr 172.21.15.49:0/2442747882 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9abbfff700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0ff3f0 0x7f9abc19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9abb7fe700 1 -- 172.21.15.49:0/2442747882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0ff3f0 msgr2=0x7f9abc19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9abb7fe700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0ff3f0 0x7f9abc19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.568+0000 7f9abb7fe700 1 -- 172.21.15.49:0/2442747882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0095f0 con 0x7f9abc0ffda0 2024-05-15T16:11:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.569+0000 7f9abb7fe700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ffda0 0x7f9abc19cde0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e8f0 tx=0x7f9ab000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.569+0000 7f9ab97fa700 1 -- 172.21.15.49:0/2442747882 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab0009790 con 0x7f9abc0ffda0 2024-05-15T16:11:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.569+0000 7f9ab97fa700 1 -- 172.21.15.49:0/2442747882 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ab000c810 con 0x7f9abc0ffda0 2024-05-15T16:11:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.569+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9abc196d10 con 0x7f9abc0ffda0 2024-05-15T16:11:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.569+0000 7f9ab97fa700 1 -- 172.21.15.49:0/2442747882 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab0005580 con 0x7f9abc0ffda0 2024-05-15T16:11:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.569+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9abc197260 con 0x7f9abc0ffda0 2024-05-15T16:11:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.570+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9abc109280 con 0x7f9abc0ffda0 2024-05-15T16:11:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.570+0000 7f9ab97fa700 1 -- 172.21.15.49:0/2442747882 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9ab0004a00 con 0x7f9abc0ffda0 2024-05-15T16:11:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.571+0000 7f9ab97fa700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9aa406e210 0x7f9aa40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.571+0000 7f9ab97fa700 1 -- 172.21.15.49:0/2442747882 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9ab0014070 con 0x7f9abc0ffda0 2024-05-15T16:11:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.571+0000 7f9abbfff700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9aa406e210 0x7f9aa40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.571+0000 7f9abbfff700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9aa406e210 0x7f9aa40706d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b760 tx=0x7f9aac005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:10.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.575+0000 7f9ab97fa700 1 -- 172.21.15.49:0/2442747882 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9ab005b260 con 0x7f9abc0ffda0 2024-05-15T16:11:10.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.869+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9abc1024b0 con 0x7f9abc0ffda0 2024-05-15T16:11:10.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.870+0000 7f9ab97fa700 1 -- 172.21.15.49:0/2442747882 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f9ab005adf0 con 0x7f9abc0ffda0 2024-05-15T16:11:10.870 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:11:10.870 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:11:10.871 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:11:10.871 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:11:10.871 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:11:10.871 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:11:10.871 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:11:10.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9aa406e210 msgr2=0x7f9aa40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9aa406e210 0x7f9aa40706d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b760 tx=0x7f9aac005a90 comp rx=0 tx=0).stop 2024-05-15T16:11:10.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ffda0 msgr2=0x7f9abc19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:10.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ffda0 0x7f9abc19cde0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e8f0 tx=0x7f9ab000ec00 comp rx=0 tx=0).stop 2024-05-15T16:11:10.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 shutdown_connections 2024-05-15T16:11:10.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9aa406e210 0x7f9aa40706d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0ff3f0 0x7f9abc19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 --2- 172.21.15.49:0/2442747882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9abc0ffda0 0x7f9abc19cde0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:10.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 >> 172.21.15.49:0/2442747882 conn(0x7f9abc075ff0 msgr2=0x7f9abc104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:10.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 shutdown_connections 2024-05-15T16:11:10.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:10.872+0000 7f9ac2567700 1 -- 172.21.15.49:0/2442747882 wait complete. 2024-05-15T16:11:11.161 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:10 smithi049 ceph-mon[112142]: from='client.34830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:11.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:10 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4004177665' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:11:11.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:10 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2442747882' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:11:11.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:10 smithi144 ceph-mon[86635]: from='client.34830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:11.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:10 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4004177665' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:11:11.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:10 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2442747882' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:11:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:11 smithi144 ceph-mon[86635]: pgmap v424: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:11 smithi144 ceph-mon[86635]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:11 smithi049 ceph-mon[112142]: pgmap v424: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:11 smithi049 ceph-mon[112142]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:13 smithi144 ceph-mon[86635]: pgmap v425: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:13 smithi049 ceph-mon[112142]: pgmap v425: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:15 smithi144 ceph-mon[86635]: pgmap v426: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:15 smithi049 ceph-mon[112142]: pgmap v426: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:17 smithi144 ceph-mon[86635]: pgmap v427: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:17 smithi049 ceph-mon[112142]: pgmap v427: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:19 smithi144 ceph-mon[86635]: pgmap v428: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:19 smithi049 ceph-mon[112142]: pgmap v428: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:11:21.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:11:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:21 smithi144 ceph-mon[86635]: pgmap v429: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:21 smithi049 ceph-mon[112142]: pgmap v429: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:23 smithi144 ceph-mon[86635]: pgmap v430: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:23 smithi049 ceph-mon[112142]: pgmap v430: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:25 smithi049 ceph-mon[112142]: pgmap v431: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:25 smithi144 ceph-mon[86635]: pgmap v431: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:27 smithi049 ceph-mon[112142]: pgmap v432: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:27 smithi144 ceph-mon[86635]: pgmap v432: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:30.349 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:29 smithi144 ceph-mon[86635]: pgmap v433: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:29 smithi049 ceph-mon[112142]: pgmap v433: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:31 smithi144 ceph-mon[86635]: pgmap v434: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:31 smithi049 ceph-mon[112142]: pgmap v434: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:33 smithi144 ceph-mon[86635]: pgmap v435: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:33 smithi049 ceph-mon[112142]: pgmap v435: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:35 smithi144 ceph-mon[86635]: pgmap v436: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:11:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:35 smithi049 ceph-mon[112142]: pgmap v436: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:11:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:37 smithi144 ceph-mon[86635]: pgmap v437: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:37 smithi049 ceph-mon[112142]: pgmap v437: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:39 smithi144 ceph-mon[86635]: pgmap v438: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:39 smithi049 ceph-mon[112142]: pgmap v438: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.996+0000 7fd66748e700 1 -- 172.21.15.49:0/287549641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660073cf0 msgr2=0x7fd66010cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.996+0000 7fd66748e700 1 --2- 172.21.15.49:0/287549641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660073cf0 0x7fd66010cfa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd650009910 tx=0x7fd650009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.996+0000 7fd66748e700 1 -- 172.21.15.49:0/287549641 shutdown_connections 2024-05-15T16:11:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.996+0000 7fd66748e700 1 --2- 172.21.15.49:0/287549641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd660073cf0 0x7fd66010cfa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.996+0000 7fd66748e700 1 --2- 172.21.15.49:0/287549641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6600733d0 0x7fd6600737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.996+0000 7fd66748e700 1 -- 172.21.15.49:0/287549641 >> 172.21.15.49:0/287549641 conn(0x7fd6600fc7d0 msgr2=0x7fd6600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.996+0000 7fd66748e700 1 -- 172.21.15.49:0/287549641 shutdown_connections 2024-05-15T16:11:40.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.996+0000 7fd66748e700 1 -- 172.21.15.49:0/287549641 wait complete. 2024-05-15T16:11:40.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.997+0000 7fd66748e700 1 Processor -- start 2024-05-15T16:11:40.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.997+0000 7fd66748e700 1 -- start start 2024-05-15T16:11:40.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.997+0000 7fd66748e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6600733d0 0x7fd660198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:40.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.997+0000 7fd66748e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd660073cf0 0x7fd660199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:40.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.997+0000 7fd66748e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6601999f0 con 0x7fd6600733d0 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.997+0000 7fd66748e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66019d6c0 con 0x7fd660073cf0 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd66522a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6600733d0 0x7fd660198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd66522a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6600733d0 0x7fd660198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37028/0 (socket says 172.21.15.49:37028) 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd66522a700 1 -- 172.21.15.49:0/2865652090 learned_addr learned my addr 172.21.15.49:0/2865652090 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd664a29700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd660073cf0 0x7fd660199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd66522a700 1 -- 172.21.15.49:0/2865652090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd660073cf0 msgr2=0x7fd660199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd66522a700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd660073cf0 0x7fd660199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd66522a700 1 -- 172.21.15.49:0/2865652090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6500095f0 con 0x7fd6600733d0 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd664a29700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd660073cf0 0x7fd660199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd66522a700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6600733d0 0x7fd660198dd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00d8d0 tx=0x7fd65c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd6567fc700 1 -- 172.21.15.49:0/2865652090 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd65c009940 con 0x7fd6600733d0 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd6567fc700 1 -- 172.21.15.49:0/2865652090 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd65c010460 con 0x7fd6600733d0 2024-05-15T16:11:41.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66019d9a0 con 0x7fd6600733d0 2024-05-15T16:11:41.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd6567fc700 1 -- 172.21.15.49:0/2865652090 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd65c00f5f0 con 0x7fd6600733d0 2024-05-15T16:11:41.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.998+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66019def0 con 0x7fd6600733d0 2024-05-15T16:11:41.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:40.999+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66010a7a0 con 0x7fd6600733d0 2024-05-15T16:11:41.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.000+0000 7fd6567fc700 1 -- 172.21.15.49:0/2865652090 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd65c0105d0 con 0x7fd6600733d0 2024-05-15T16:11:41.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.000+0000 7fd6567fc700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd64c06e200 0x7fd64c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:41.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.000+0000 7fd6567fc700 1 -- 172.21.15.49:0/2865652090 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd65c08e1a0 con 0x7fd6600733d0 2024-05-15T16:11:41.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.001+0000 7fd664a29700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd64c06e200 0x7fd64c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:41.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.001+0000 7fd664a29700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd64c06e200 0x7fd64c0706c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6500059d0 tx=0x7fd65001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:41.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.005+0000 7fd6567fc700 1 -- 172.21.15.49:0/2865652090 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd65c057480 con 0x7fd6600733d0 2024-05-15T16:11:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.234+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6601100c0 con 0x7fd64c06e200 2024-05-15T16:11:41.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.235+0000 7fd6567fc700 1 -- 172.21.15.49:0/2865652090 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd6601100c0 con 0x7fd64c06e200 2024-05-15T16:11:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd64c06e200 msgr2=0x7fd64c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd64c06e200 0x7fd64c0706c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6500059d0 tx=0x7fd65001a040 comp rx=0 tx=0).stop 2024-05-15T16:11:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6600733d0 msgr2=0x7fd660198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6600733d0 0x7fd660198dd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00d8d0 tx=0x7fd65c00dc90 comp rx=0 tx=0).stop 2024-05-15T16:11:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 shutdown_connections 2024-05-15T16:11:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd64c06e200 0x7fd64c0706c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6600733d0 0x7fd660198dd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 --2- 172.21.15.49:0/2865652090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd660073cf0 0x7fd660199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 >> 172.21.15.49:0/2865652090 conn(0x7fd6600fc7d0 msgr2=0x7fd6601078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 shutdown_connections 2024-05-15T16:11:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.237+0000 7fd66748e700 1 -- 172.21.15.49:0/2865652090 wait complete. 2024-05-15T16:11:41.248 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:11:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.347+0000 7f193150b700 1 -- 172.21.15.49:0/107092117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c0733d0 msgr2=0x7f192c0737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.347+0000 7f193150b700 1 --2- 172.21.15.49:0/107092117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c0733d0 0x7f192c0737b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f191c00b0d0 tx=0x7f191c00b3e0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.347+0000 7f193150b700 1 -- 172.21.15.49:0/107092117 shutdown_connections 2024-05-15T16:11:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.347+0000 7f193150b700 1 --2- 172.21.15.49:0/107092117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f192c073d80 0x7f192c10cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.347+0000 7f193150b700 1 --2- 172.21.15.49:0/107092117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c0733d0 0x7f192c0737b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.347+0000 7f193150b700 1 -- 172.21.15.49:0/107092117 >> 172.21.15.49:0/107092117 conn(0x7f192c0fc7c0 msgr2=0x7f192c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.347+0000 7f193150b700 1 -- 172.21.15.49:0/107092117 shutdown_connections 2024-05-15T16:11:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.347+0000 7f193150b700 1 -- 172.21.15.49:0/107092117 wait complete. 2024-05-15T16:11:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f193150b700 1 Processor -- start 2024-05-15T16:11:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f193150b700 1 -- start start 2024-05-15T16:11:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f193150b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f192c0733d0 0x7f192c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f193150b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c073d80 0x7f192c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f193150b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c19f5b0 con 0x7f192c073d80 2024-05-15T16:11:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f193150b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c198be0 con 0x7f192c0733d0 2024-05-15T16:11:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f192a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c073d80 0x7f192c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f192a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c073d80 0x7f192c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37044/0 (socket says 172.21.15.49:37044) 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f192a7fc700 1 -- 172.21.15.49:0/30666018 learned_addr learned my addr 172.21.15.49:0/30666018 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.348+0000 7f192affd700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f192c0733d0 0x7f192c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.349+0000 7f192a7fc700 1 -- 172.21.15.49:0/30666018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f192c0733d0 msgr2=0x7f192c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.349+0000 7f192a7fc700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f192c0733d0 0x7f192c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.349+0000 7f192a7fc700 1 -- 172.21.15.49:0/30666018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c009d00 con 0x7f192c073d80 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.349+0000 7f192a7fc700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c073d80 0x7f192c19ef90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f191400c8a0 tx=0x7f191400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.349+0000 7f1923fff700 1 -- 172.21.15.49:0/30666018 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19140078e0 con 0x7f192c073d80 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.349+0000 7f1923fff700 1 -- 172.21.15.49:0/30666018 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f191400f450 con 0x7f192c073d80 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.349+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192c198ec0 con 0x7f192c073d80 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.349+0000 7f1923fff700 1 -- 172.21.15.49:0/30666018 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f191400e5e0 con 0x7f192c073d80 2024-05-15T16:11:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.349+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192c199410 con 0x7f192c073d80 2024-05-15T16:11:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.350+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192c10a730 con 0x7f192c073d80 2024-05-15T16:11:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.351+0000 7f1923fff700 1 -- 172.21.15.49:0/30666018 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f191400f5c0 con 0x7f192c073d80 2024-05-15T16:11:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.351+0000 7f1923fff700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f191806e200 0x7f19180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.351+0000 7f1923fff700 1 -- 172.21.15.49:0/30666018 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f191408e180 con 0x7f192c073d80 2024-05-15T16:11:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.351+0000 7f192affd700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f191806e200 0x7f19180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:41.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.352+0000 7f192affd700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f191806e200 0x7f19180706c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f191c00b0a0 tx=0x7f191c0060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.356+0000 7f1923fff700 1 -- 172.21.15.49:0/30666018 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f191405af90 con 0x7f192c073d80 2024-05-15T16:11:41.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.580+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f192c199fe0 con 0x7f191806e200 2024-05-15T16:11:41.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.581+0000 7f1923fff700 1 -- 172.21.15.49:0/30666018 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f192c199fe0 con 0x7f191806e200 2024-05-15T16:11:41.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f191806e200 msgr2=0x7f19180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f191806e200 0x7f19180706c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f191c00b0a0 tx=0x7f191c0060c0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c073d80 msgr2=0x7f192c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c073d80 0x7f192c19ef90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f191400c8a0 tx=0x7f191400cbb0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 shutdown_connections 2024-05-15T16:11:41.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f191806e200 0x7f19180706c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f192c0733d0 0x7f192c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 --2- 172.21.15.49:0/30666018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c073d80 0x7f192c19ef90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 >> 172.21.15.49:0/30666018 conn(0x7f192c0fc7c0 msgr2=0x7f192c107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:41.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 shutdown_connections 2024-05-15T16:11:41.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.584+0000 7f193150b700 1 -- 172.21.15.49:0/30666018 wait complete. 2024-05-15T16:11:41.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.692+0000 7fbb52a10700 1 -- 172.21.15.49:0/4270368930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c0ffe10 msgr2=0x7fbb4c1001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.692+0000 7fbb52a10700 1 --2- 172.21.15.49:0/4270368930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c0ffe10 0x7fbb4c1001f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009910 tx=0x7fbb3c009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.692+0000 7fbb52a10700 1 -- 172.21.15.49:0/4270368930 shutdown_connections 2024-05-15T16:11:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.692+0000 7fbb52a10700 1 --2- 172.21.15.49:0/4270368930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4c1007c0 0x7fbb4c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.692+0000 7fbb52a10700 1 --2- 172.21.15.49:0/4270368930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c0ffe10 0x7fbb4c1001f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.692+0000 7fbb52a10700 1 -- 172.21.15.49:0/4270368930 >> 172.21.15.49:0/4270368930 conn(0x7fbb4c0fb700 msgr2=0x7fbb4c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.692+0000 7fbb52a10700 1 -- 172.21.15.49:0/4270368930 shutdown_connections 2024-05-15T16:11:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.692+0000 7fbb52a10700 1 -- 172.21.15.49:0/4270368930 wait complete. 2024-05-15T16:11:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.693+0000 7fbb52a10700 1 Processor -- start 2024-05-15T16:11:41.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.693+0000 7fbb52a10700 1 -- start start 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.693+0000 7fbb52a10700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4c0ffe10 0x7fbb4c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.693+0000 7fbb52a10700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c1007c0 0x7fbb4c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.693+0000 7fbb52a10700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c196a70 con 0x7fbb4c1007c0 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.693+0000 7fbb52a10700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c196be0 con 0x7fbb4c0ffe10 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb4b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c1007c0 0x7fbb4c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb4bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4c0ffe10 0x7fbb4c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb4b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c1007c0 0x7fbb4c19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37070/0 (socket says 172.21.15.49:37070) 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb4b7fe700 1 -- 172.21.15.49:0/2051001590 learned_addr learned my addr 172.21.15.49:0/2051001590 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb4b7fe700 1 -- 172.21.15.49:0/2051001590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4c0ffe10 msgr2=0x7fbb4c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb4b7fe700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4c0ffe10 0x7fbb4c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb4b7fe700 1 -- 172.21.15.49:0/2051001590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0095f0 con 0x7fbb4c1007c0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb4bfff700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4c0ffe10 0x7fbb4c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb4b7fe700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c1007c0 0x7fbb4c19ceb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000d900 tx=0x7fbb4000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb497fa700 1 -- 172.21.15.49:0/2051001590 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb40009900 con 0x7fbb4c1007c0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb497fa700 1 -- 172.21.15.49:0/2051001590 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb40010460 con 0x7fbb4c1007c0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c196ec0 con 0x7fbb4c1007c0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb497fa700 1 -- 172.21.15.49:0/2051001590 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb4000f5f0 con 0x7fbb4c1007c0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.694+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c197410 con 0x7fbb4c1007c0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.695+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c1012f0 con 0x7fbb4c1007c0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.696+0000 7fbb497fa700 1 -- 172.21.15.49:0/2051001590 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbb400105d0 con 0x7fbb4c1007c0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.696+0000 7fbb497fa700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb3406e200 0x7fbb340706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.697+0000 7fbb497fa700 1 -- 172.21.15.49:0/2051001590 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbb4008dd60 con 0x7fbb4c1007c0 2024-05-15T16:11:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.697+0000 7fbb4bfff700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb3406e200 0x7fbb340706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:41.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.697+0000 7fbb4bfff700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb3406e200 0x7fbb340706c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c006010 tx=0x7fbb3c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:41.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.701+0000 7fbb497fa700 1 -- 172.21.15.49:0/2051001590 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbb40057040 con 0x7fbb4c1007c0 2024-05-15T16:11:41.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.917+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb4c1976f0 con 0x7fbb3406e200 2024-05-15T16:11:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.930+0000 7fbb497fa700 1 -- 172.21.15.49:0/2051001590 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbb4c1976f0 con 0x7fbb3406e200 2024-05-15T16:11:41.930 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (14m) 8m ago 26m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (26m) 8m ago 26m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (25m) 7m ago 25m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (12m) 8m ago 26m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (12m) 7m ago 25m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (14m) 8m ago 26m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (8m) 8m ago 20m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (8m) 8m ago 20m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (8m) 7m ago 20m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (20m) 7m ago 20m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (17m) 8m ago 27m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (16m) 7m ago 25m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:11:41.931 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (13m) 8m ago 28m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (13m) 7m ago 25m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (15m) 8m ago 26m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (15m) 7m ago 25m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (11m) 8m ago 24m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (11m) 8m ago 24m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (10m) 8m ago 23m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (10m) 7m ago 23m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (9m) 7m ago 22m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (8m) 7m ago 22m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:11:41.932 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (15m) 8m ago 25m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:11:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.932+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb3406e200 msgr2=0x7fbb340706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.932+0000 7fbb52a10700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb3406e200 0x7fbb340706c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c006010 tx=0x7fbb3c01a040 comp rx=0 tx=0).stop 2024-05-15T16:11:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.932+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c1007c0 msgr2=0x7fbb4c19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.932+0000 7fbb52a10700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c1007c0 0x7fbb4c19ceb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000d900 tx=0x7fbb4000dc10 comp rx=0 tx=0).stop 2024-05-15T16:11:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.932+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 shutdown_connections 2024-05-15T16:11:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.932+0000 7fbb52a10700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb3406e200 0x7fbb340706c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.932+0000 7fbb52a10700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4c0ffe10 0x7fbb4c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.932+0000 7fbb52a10700 1 --2- 172.21.15.49:0/2051001590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb4c1007c0 0x7fbb4c19ceb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:41.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.932+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 >> 172.21.15.49:0/2051001590 conn(0x7fbb4c0fb700 msgr2=0x7fbb4c069670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:41.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.933+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 shutdown_connections 2024-05-15T16:11:41.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:41.933+0000 7fbb52a10700 1 -- 172.21.15.49:0/2051001590 wait complete. 2024-05-15T16:11:42.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.040+0000 7f0ed0890700 1 -- 172.21.15.49:0/3223733967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc1018b0 msgr2=0x7f0ecc105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.040+0000 7f0ed0890700 1 --2- 172.21.15.49:0/3223733967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc1018b0 0x7f0ecc105900 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009910 tx=0x7f0eb4009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:42.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.040+0000 7f0ed0890700 1 -- 172.21.15.49:0/3223733967 shutdown_connections 2024-05-15T16:11:42.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.040+0000 7f0ed0890700 1 --2- 172.21.15.49:0/3223733967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc1018b0 0x7f0ecc105900 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.040+0000 7f0ed0890700 1 --2- 172.21.15.49:0/3223733967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc100f00 0x7f0ecc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.040+0000 7f0ed0890700 1 -- 172.21.15.49:0/3223733967 >> 172.21.15.49:0/3223733967 conn(0x7f0ecc0fc7d0 msgr2=0x7f0ecc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:42.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.040+0000 7f0ed0890700 1 -- 172.21.15.49:0/3223733967 shutdown_connections 2024-05-15T16:11:42.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.040+0000 7f0ed0890700 1 -- 172.21.15.49:0/3223733967 wait complete. 2024-05-15T16:11:42.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.041+0000 7f0ed0890700 1 Processor -- start 2024-05-15T16:11:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.041+0000 7f0ed0890700 1 -- start start 2024-05-15T16:11:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.041+0000 7f0ed0890700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc100f00 0x7f0ecc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.041+0000 7f0ed0890700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc1018b0 0x7f0ecc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.041+0000 7f0ed0890700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc073d20 con 0x7f0ecc100f00 2024-05-15T16:11:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.041+0000 7f0ed0890700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc073e60 con 0x7f0ecc1018b0 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.041+0000 7f0eca59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc100f00 0x7f0ecc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0eca59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc100f00 0x7f0ecc0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37086/0 (socket says 172.21.15.49:37086) 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0eca59c700 1 -- 172.21.15.49:0/2780770076 learned_addr learned my addr 172.21.15.49:0/2780770076 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0ec1d9b700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc1018b0 0x7f0ecc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0eca59c700 1 -- 172.21.15.49:0/2780770076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc1018b0 msgr2=0x7f0ecc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0eca59c700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc1018b0 0x7f0ecc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0eca59c700 1 -- 172.21.15.49:0/2780770076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40095f0 con 0x7f0ecc100f00 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0eca59c700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc100f00 0x7f0ecc0732a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c8a0 tx=0x7f0ebc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0ec3fff700 1 -- 172.21.15.49:0/2780770076 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ebc0078e0 con 0x7f0ecc100f00 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0ec3fff700 1 -- 172.21.15.49:0/2780770076 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ebc00f450 con 0x7f0ecc100f00 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc1a2da0 con 0x7f0ecc100f00 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0ec3fff700 1 -- 172.21.15.49:0/2780770076 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ebc00e5e0 con 0x7f0ecc100f00 2024-05-15T16:11:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.042+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc1a3100 con 0x7f0ecc100f00 2024-05-15T16:11:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.043+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc109300 con 0x7f0ecc100f00 2024-05-15T16:11:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.044+0000 7f0ec3fff700 1 -- 172.21.15.49:0/2780770076 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0ebc00e740 con 0x7f0ecc100f00 2024-05-15T16:11:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.044+0000 7f0ec3fff700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0eb806e200 0x7f0eb80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.044+0000 7f0ec3fff700 1 -- 172.21.15.49:0/2780770076 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0ebc08e180 con 0x7f0ecc100f00 2024-05-15T16:11:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.044+0000 7f0ec1d9b700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0eb806e200 0x7f0eb80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.045+0000 7f0ec1d9b700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0eb806e200 0x7f0eb80706c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0749d0 tx=0x7f0eb401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.049+0000 7f0ec3fff700 1 -- 172.21.15.49:0/2780770076 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0ebc057460 con 0x7f0ecc100f00 2024-05-15T16:11:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:41 smithi049 ceph-mon[112142]: pgmap v439: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:42.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.344+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0ecc10dea0 con 0x7f0ecc100f00 2024-05-15T16:11:42.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.345+0000 7f0ec3fff700 1 -- 172.21.15.49:0/2780770076 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f0ebc05ab20 con 0x7f0ecc100f00 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:11:42.346 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:11:42.347 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:11:42.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:11:42.347 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:11:42.347 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:11:42.347 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:11:42.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:11:42.347 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:11:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0eb806e200 msgr2=0x7f0eb80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0eb806e200 0x7f0eb80706c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0749d0 tx=0x7f0eb401a040 comp rx=0 tx=0).stop 2024-05-15T16:11:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc100f00 msgr2=0x7f0ecc0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc100f00 0x7f0ecc0732a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c8a0 tx=0x7f0ebc00cbb0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 shutdown_connections 2024-05-15T16:11:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0eb806e200 0x7f0eb80706c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ecc100f00 0x7f0ecc0732a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 --2- 172.21.15.49:0/2780770076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc1018b0 0x7f0ecc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 >> 172.21.15.49:0/2780770076 conn(0x7f0ecc0fc7d0 msgr2=0x7f0ecc0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:42.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 shutdown_connections 2024-05-15T16:11:42.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.347+0000 7f0ed0890700 1 -- 172.21.15.49:0/2780770076 wait complete. 2024-05-15T16:11:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:41 smithi144 ceph-mon[86635]: pgmap v439: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.458+0000 7fd54de98700 1 -- 172.21.15.49:0/400806669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 msgr2=0x7fd5481047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.458+0000 7fd54de98700 1 --2- 172.21.15.49:0/400806669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 0x7fd5481047f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd530009910 tx=0x7fd530009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.458+0000 7fd54de98700 1 -- 172.21.15.49:0/400806669 shutdown_connections 2024-05-15T16:11:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.458+0000 7fd54de98700 1 --2- 172.21.15.49:0/400806669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 0x7fd5481047f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.458+0000 7fd54de98700 1 --2- 172.21.15.49:0/400806669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5480ffdf0 0x7fd5481001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.458+0000 7fd54de98700 1 -- 172.21.15.49:0/400806669 >> 172.21.15.49:0/400806669 conn(0x7fd5480fb6a0 msgr2=0x7fd5480fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.458+0000 7fd54de98700 1 -- 172.21.15.49:0/400806669 shutdown_connections 2024-05-15T16:11:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.458+0000 7fd54de98700 1 -- 172.21.15.49:0/400806669 wait complete. 2024-05-15T16:11:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.458+0000 7fd54de98700 1 Processor -- start 2024-05-15T16:11:42.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd54de98700 1 -- start start 2024-05-15T16:11:42.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd54de98700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5480ffdf0 0x7fd548101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:42.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd54de98700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 0x7fd548101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:42.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd54de98700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd548102310 con 0x7fd5481007a0 2024-05-15T16:11:42.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd54de98700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd548102450 con 0x7fd5480ffdf0 2024-05-15T16:11:42.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd546ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 0x7fd548101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:42.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd546ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 0x7fd548101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37104/0 (socket says 172.21.15.49:37104) 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd546ffd700 1 -- 172.21.15.49:0/3399252518 learned_addr learned my addr 172.21.15.49:0/3399252518 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd5477fe700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5480ffdf0 0x7fd548101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd546ffd700 1 -- 172.21.15.49:0/3399252518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5480ffdf0 msgr2=0x7fd548101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd546ffd700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5480ffdf0 0x7fd548101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.459+0000 7fd546ffd700 1 -- 172.21.15.49:0/3399252518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300095f0 con 0x7fd5481007a0 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.460+0000 7fd546ffd700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 0x7fd548101dd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd530000c00 tx=0x7fd53000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.460+0000 7fd544ff9700 1 -- 172.21.15.49:0/3399252518 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd53001d070 con 0x7fd5481007a0 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.460+0000 7fd544ff9700 1 -- 172.21.15.49:0/3399252518 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd530005700 con 0x7fd5481007a0 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.460+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5481026d0 con 0x7fd5481007a0 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.460+0000 7fd544ff9700 1 -- 172.21.15.49:0/3399252518 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd53000fda0 con 0x7fd5481007a0 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.460+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5481a3050 con 0x7fd5481007a0 2024-05-15T16:11:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.461+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd548109200 con 0x7fd5481007a0 2024-05-15T16:11:42.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.461+0000 7fd544ff9700 1 -- 172.21.15.49:0/3399252518 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd53000f5d0 con 0x7fd5481007a0 2024-05-15T16:11:42.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.462+0000 7fd544ff9700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd53406e210 0x7fd5340706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:42.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.462+0000 7fd544ff9700 1 -- 172.21.15.49:0/3399252518 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd53008f510 con 0x7fd5481007a0 2024-05-15T16:11:42.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.462+0000 7fd5477fe700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd53406e210 0x7fd5340706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:42.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.462+0000 7fd5477fe700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd53406e210 0x7fd5340706d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd538009550 tx=0x7fd5380091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:42.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.467+0000 7fd544ff9700 1 -- 172.21.15.49:0/3399252518 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd5300587f0 con 0x7fd5481007a0 2024-05-15T16:11:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.725+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd5481012d0 con 0x7fd5481007a0 2024-05-15T16:11:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.726+0000 7fd544ff9700 1 -- 172.21.15.49:0/3399252518 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7fd53005beb0 con 0x7fd5481007a0 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:11:42.727 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:11:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:11:42.729 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:11:42.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.728+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd53406e210 msgr2=0x7fd5340706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.728+0000 7fd54de98700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd53406e210 0x7fd5340706d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd538009550 tx=0x7fd5380091b0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.728+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 msgr2=0x7fd548101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.728+0000 7fd54de98700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 0x7fd548101dd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd530000c00 tx=0x7fd53000bd70 comp rx=0 tx=0).stop 2024-05-15T16:11:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.728+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 shutdown_connections 2024-05-15T16:11:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.728+0000 7fd54de98700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd53406e210 0x7fd5340706d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.728+0000 7fd54de98700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5480ffdf0 0x7fd548101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.728+0000 7fd54de98700 1 --2- 172.21.15.49:0/3399252518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5481007a0 0x7fd548101dd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.728+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 >> 172.21.15.49:0/3399252518 conn(0x7fd5480fb6a0 msgr2=0x7fd5480697c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.729+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 shutdown_connections 2024-05-15T16:11:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.729+0000 7fd54de98700 1 -- 172.21.15.49:0/3399252518 wait complete. 2024-05-15T16:11:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:11:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.838+0000 7f52d0760700 1 -- 172.21.15.49:0/3864356832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8101e40 msgr2=0x7f52c8102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.838+0000 7f52d0760700 1 --2- 172.21.15.49:0/3864356832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8101e40 0x7f52c8102220 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009910 tx=0x7f52c4009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.838+0000 7f52d0760700 1 -- 172.21.15.49:0/3864356832 shutdown_connections 2024-05-15T16:11:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.838+0000 7f52d0760700 1 --2- 172.21.15.49:0/3864356832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c8102760 0x7f52c810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.838+0000 7f52d0760700 1 --2- 172.21.15.49:0/3864356832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8101e40 0x7f52c8102220 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.838+0000 7f52d0760700 1 -- 172.21.15.49:0/3864356832 >> 172.21.15.49:0/3864356832 conn(0x7f52c80fb6b0 msgr2=0x7f52c80fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.838+0000 7f52d0760700 1 -- 172.21.15.49:0/3864356832 shutdown_connections 2024-05-15T16:11:42.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.838+0000 7f52d0760700 1 -- 172.21.15.49:0/3864356832 wait complete. 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52d0760700 1 Processor -- start 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52d0760700 1 -- start start 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52d0760700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c8101e40 0x7f52c80ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52d0760700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8102760 0x7f52c80fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52d0760700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c8100520 con 0x7f52c8102760 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52d0760700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c8100660 con 0x7f52c8101e40 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52cdcfb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8102760 0x7f52c80fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52cdcfb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8102760 0x7f52c80fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37120/0 (socket says 172.21.15.49:37120) 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52cdcfb700 1 -- 172.21.15.49:0/1735766676 learned_addr learned my addr 172.21.15.49:0/1735766676 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52ce4fc700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c8101e40 0x7f52c80ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:42.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52cdcfb700 1 -- 172.21.15.49:0/1735766676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c8101e40 msgr2=0x7f52c80ffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52cdcfb700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c8101e40 0x7f52c80ffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.839+0000 7f52cdcfb700 1 -- 172.21.15.49:0/1735766676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40095f0 con 0x7f52c8102760 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.840+0000 7f52cdcfb700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8102760 0x7f52c80fffe0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f52b800e940 tx=0x7f52b800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.840+0000 7f52bf7fe700 1 -- 172.21.15.49:0/1735766676 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52b80096e0 con 0x7f52c8102760 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.840+0000 7f52bf7fe700 1 -- 172.21.15.49:0/1735766676 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52b800c7c0 con 0x7f52c8102760 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.840+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c81a2e00 con 0x7f52c8102760 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.840+0000 7f52bf7fe700 1 -- 172.21.15.49:0/1735766676 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52b8005580 con 0x7f52c8102760 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.840+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c81a3160 con 0x7f52c8102760 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.841+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c8069ab0 con 0x7f52c8102760 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.841+0000 7f52bf7fe700 1 -- 172.21.15.49:0/1735766676 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f52b800c930 con 0x7f52c8102760 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.842+0000 7f52bf7fe700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52b406e210 0x7f52b40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.842+0000 7f52bf7fe700 1 -- 172.21.15.49:0/1735766676 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f52b8014070 con 0x7f52c8102760 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.842+0000 7f52ce4fc700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52b406e210 0x7f52b40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:42.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.842+0000 7f52ce4fc700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52b406e210 0x7f52b40706d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52c400b760 tx=0x7f52c4005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:42.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:42.846+0000 7f52bf7fe700 1 -- 172.21.15.49:0/1735766676 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f52b8057600 con 0x7f52c8102760 2024-05-15T16:11:43.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.078+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52c810dec0 con 0x7f52b406e210 2024-05-15T16:11:43.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.080+0000 7f52bf7fe700 1 -- 172.21.15.49:0/1735766676 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f52c810dec0 con 0x7f52b406e210 2024-05-15T16:11:43.080 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:11:43.080 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:11:43.081 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:11:43.081 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:11:43.081 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:11:43.081 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:11:43.081 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:11:43.081 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:11:43.081 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:11:43.082 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:11:43.082 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:11:43.082 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:11:43.082 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:11:43.082 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52b406e210 msgr2=0x7f52b40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52b406e210 0x7f52b40706d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52c400b760 tx=0x7f52c4005a90 comp rx=0 tx=0).stop 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8102760 msgr2=0x7f52c80fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8102760 0x7f52c80fffe0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f52b800e940 tx=0x7f52b800ec50 comp rx=0 tx=0).stop 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 shutdown_connections 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52b406e210 0x7f52b40706d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c8101e40 0x7f52c80ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 --2- 172.21.15.49:0/1735766676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52c8102760 0x7f52c80fffe0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 >> 172.21.15.49:0/1735766676 conn(0x7f52c80fb6b0 msgr2=0x7f52c8103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:43.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 shutdown_connections 2024-05-15T16:11:43.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.082+0000 7f52d0760700 1 -- 172.21.15.49:0/1735766676 wait complete. 2024-05-15T16:11:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.190+0000 7f0046678700 1 -- 172.21.15.49:0/3119952245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040101e20 msgr2=0x7f0040102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.190+0000 7f0046678700 1 --2- 172.21.15.49:0/3119952245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040101e20 0x7f0040102200 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0030009910 tx=0x7f0030009c20 comp rx=0 tx=0).stop 2024-05-15T16:11:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.190+0000 7f0046678700 1 -- 172.21.15.49:0/3119952245 shutdown_connections 2024-05-15T16:11:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.190+0000 7f0046678700 1 --2- 172.21.15.49:0/3119952245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0040102740 0x7f004010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.190+0000 7f0046678700 1 --2- 172.21.15.49:0/3119952245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040101e20 0x7f0040102200 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.190+0000 7f0046678700 1 -- 172.21.15.49:0/3119952245 >> 172.21.15.49:0/3119952245 conn(0x7f0040076800 msgr2=0x7f00400fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.190+0000 7f0046678700 1 -- 172.21.15.49:0/3119952245 shutdown_connections 2024-05-15T16:11:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.190+0000 7f0046678700 1 -- 172.21.15.49:0/3119952245 wait complete. 2024-05-15T16:11:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.191+0000 7f0046678700 1 Processor -- start 2024-05-15T16:11:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.191+0000 7f0046678700 1 -- start start 2024-05-15T16:11:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.191+0000 7f0046678700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0040101e20 0x7f00400ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f0046678700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040102740 0x7f00400fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f0046678700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040100500 con 0x7f0040102740 2024-05-15T16:11:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f0046678700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040100640 con 0x7f0040101e20 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040102740 0x7f00400fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040102740 0x7f00400fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37124/0 (socket says 172.21.15.49:37124) 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003f7fe700 1 -- 172.21.15.49:0/1680857598 learned_addr learned my addr 172.21.15.49:0/1680857598 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003ffff700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0040101e20 0x7f00400ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003f7fe700 1 -- 172.21.15.49:0/1680857598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0040101e20 msgr2=0x7f00400ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003f7fe700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0040101e20 0x7f00400ffa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003f7fe700 1 -- 172.21.15.49:0/1680857598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00300095f0 con 0x7f0040102740 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003ffff700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0040101e20 0x7f00400ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003f7fe700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040102740 0x7f00400fffc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f002800d6a0 tx=0x7f002800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003d7fa700 1 -- 172.21.15.49:0/1680857598 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0028004a00 con 0x7f0040102740 2024-05-15T16:11:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f003d7fa700 1 -- 172.21.15.49:0/1680857598 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00280054e0 con 0x7f0040102740 2024-05-15T16:11:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.192+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00401a2dd0 con 0x7f0040102740 2024-05-15T16:11:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.193+0000 7f003d7fa700 1 -- 172.21.15.49:0/1680857598 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f002800fca0 con 0x7f0040102740 2024-05-15T16:11:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.193+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00401a3130 con 0x7f0040102740 2024-05-15T16:11:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.194+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0040068f60 con 0x7f0040102740 2024-05-15T16:11:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.194+0000 7f003d7fa700 1 -- 172.21.15.49:0/1680857598 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0028005020 con 0x7f0040102740 2024-05-15T16:11:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.195+0000 7f003d7fa700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002c06e200 0x7f002c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:11:43.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.195+0000 7f003d7fa700 1 -- 172.21.15.49:0/1680857598 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f002808d910 con 0x7f0040102740 2024-05-15T16:11:43.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.195+0000 7f003ffff700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002c06e200 0x7f002c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:11:43.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.195+0000 7f003ffff700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002c06e200 0x7f002c0706c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0030006010 tx=0x7f003001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:11:43.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.200+0000 7f003d7fa700 1 -- 172.21.15.49:0/1680857598 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0028056c70 con 0x7f0040102740 2024-05-15T16:11:43.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:42 smithi144 ceph-mon[86635]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:43.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:42 smithi144 ceph-mon[86635]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:43.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:42 smithi144 ceph-mon[86635]: from='client.34858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:43.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:42 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2780770076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:11:43.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:42 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3399252518' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:11:43.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:42 smithi049 ceph-mon[112142]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:43.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:42 smithi049 ceph-mon[112142]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:43.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:42 smithi049 ceph-mon[112142]: from='client.34858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:43.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:42 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2780770076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:11:43.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:42 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3399252518' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:11:43.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.512+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f004004ec60 con 0x7f0040102740 2024-05-15T16:11:43.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.512+0000 7f003d7fa700 1 -- 172.21.15.49:0/1680857598 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f00280171c0 con 0x7f0040102740 2024-05-15T16:11:43.513 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:11:43.513 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:11:43.513 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:11:43.513 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:11:43.513 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:11:43.513 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:11:43.513 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:11:43.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.514+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002c06e200 msgr2=0x7f002c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:43.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.514+0000 7f0046678700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002c06e200 0x7f002c0706c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0030006010 tx=0x7f003001a040 comp rx=0 tx=0).stop 2024-05-15T16:11:43.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.514+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040102740 msgr2=0x7f00400fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:11:43.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.514+0000 7f0046678700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040102740 0x7f00400fffc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f002800d6a0 tx=0x7f002800da60 comp rx=0 tx=0).stop 2024-05-15T16:11:43.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.515+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 shutdown_connections 2024-05-15T16:11:43.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.515+0000 7f0046678700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f002c06e200 0x7f002c0706c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:43.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.515+0000 7f0046678700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0040101e20 0x7f00400ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:43.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.515+0000 7f0046678700 1 --2- 172.21.15.49:0/1680857598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0040102740 0x7f00400fffc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:11:43.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.515+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 >> 172.21.15.49:0/1680857598 conn(0x7f0040076800 msgr2=0x7f0040103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:11:43.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.515+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 shutdown_connections 2024-05-15T16:11:43.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:11:43.515+0000 7f0046678700 1 -- 172.21.15.49:0/1680857598 wait complete. 2024-05-15T16:11:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:43 smithi144 ceph-mon[86635]: pgmap v440: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:43 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1680857598' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:11:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:43 smithi049 ceph-mon[112142]: pgmap v440: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:43 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1680857598' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:11:45.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:44 smithi144 ceph-mon[86635]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:45.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:44 smithi049 ceph-mon[112142]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:11:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:45 smithi144 ceph-mon[86635]: pgmap v441: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:45 smithi049 ceph-mon[112142]: pgmap v441: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:48.257 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:47 smithi049 ceph-mon[112142]: pgmap v442: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:47 smithi144 ceph-mon[86635]: pgmap v442: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:49 smithi144 ceph-mon[86635]: pgmap v443: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:50.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:49 smithi049 ceph-mon[112142]: pgmap v443: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:11:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:11:52.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:51 smithi144 ceph-mon[86635]: pgmap v444: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:51 smithi049 ceph-mon[112142]: pgmap v444: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:54.296 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:54 smithi144 ceph-mon[86635]: pgmap v445: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:54 smithi049 ceph-mon[112142]: pgmap v445: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:56 smithi144 ceph-mon[86635]: pgmap v446: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:56 smithi049 ceph-mon[112142]: pgmap v446: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:58.366 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:11:58 smithi049 ceph-mon[112142]: pgmap v447: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:11:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:11:58 smithi144 ceph-mon[86635]: pgmap v447: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:00 smithi144 ceph-mon[86635]: pgmap v448: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:00 smithi049 ceph-mon[112142]: pgmap v448: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:02 smithi144 ceph-mon[86635]: pgmap v449: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:02.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:02 smithi049 ceph-mon[112142]: pgmap v449: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:04.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:04 smithi144 ceph-mon[86635]: pgmap v450: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:04.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:04 smithi049 ceph-mon[112142]: pgmap v450: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:06 smithi144 ceph-mon[86635]: pgmap v451: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:12:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:06 smithi049 ceph-mon[112142]: pgmap v451: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:06.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:12:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:08 smithi144 ceph-mon[86635]: pgmap v452: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:08 smithi049 ceph-mon[112142]: pgmap v452: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:10 smithi144 ceph-mon[86635]: pgmap v453: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:10 smithi049 ceph-mon[112142]: pgmap v453: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:12 smithi144 ceph-mon[86635]: pgmap v454: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:12.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:12 smithi049 ceph-mon[112142]: pgmap v454: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.639+0000 7f42d3da3700 1 -- 172.21.15.49:0/1623843056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 msgr2=0x7f42cc102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.639+0000 7f42d3da3700 1 --2- 172.21.15.49:0/1623843056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 0x7f42cc102200 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009910 tx=0x7f42c8009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.639+0000 7f42d3da3700 1 -- 172.21.15.49:0/1623843056 shutdown_connections 2024-05-15T16:12:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.639+0000 7f42d3da3700 1 --2- 172.21.15.49:0/1623843056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42cc102740 0x7f42cc10ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:13.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.639+0000 7f42d3da3700 1 --2- 172.21.15.49:0/1623843056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 0x7f42cc102200 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.639+0000 7f42d3da3700 1 -- 172.21.15.49:0/1623843056 >> 172.21.15.49:0/1623843056 conn(0x7f42cc076800 msgr2=0x7f42cc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.640+0000 7f42d3da3700 1 -- 172.21.15.49:0/1623843056 shutdown_connections 2024-05-15T16:12:13.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.640+0000 7f42d3da3700 1 -- 172.21.15.49:0/1623843056 wait complete. 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.640+0000 7f42d3da3700 1 Processor -- start 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.640+0000 7f42d3da3700 1 -- start start 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.640+0000 7f42d3da3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 0x7f42cc19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.640+0000 7f42d3da3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42cc102740 0x7f42cc19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.640+0000 7f42d3da3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc19d420 con 0x7f42cc101e20 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d3da3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc196a00 con 0x7f42cc102740 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d1b3f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 0x7f42cc19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d1b3f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 0x7f42cc19c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42834/0 (socket says 172.21.15.49:42834) 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d1b3f700 1 -- 172.21.15.49:0/2601437726 learned_addr learned my addr 172.21.15.49:0/2601437726 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d133e700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42cc102740 0x7f42cc19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d1b3f700 1 -- 172.21.15.49:0/2601437726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42cc102740 msgr2=0x7f42cc19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d1b3f700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42cc102740 0x7f42cc19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:13.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d1b3f700 1 -- 172.21.15.49:0/2601437726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c80095f0 con 0x7f42cc101e20 2024-05-15T16:12:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d133e700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42cc102740 0x7f42cc19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:12:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d1b3f700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 0x7f42cc19c8c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f42c800bd70 tx=0x7f42c800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42beffd700 1 -- 172.21.15.49:0/2601437726 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c801d070 con 0x7f42cc101e20 2024-05-15T16:12:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42beffd700 1 -- 172.21.15.49:0/2601437726 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42c80056e0 con 0x7f42cc101e20 2024-05-15T16:12:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42cc196c80 con 0x7f42cc101e20 2024-05-15T16:12:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42beffd700 1 -- 172.21.15.49:0/2601437726 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c8017400 con 0x7f42cc101e20 2024-05-15T16:12:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.641+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc197170 con 0x7f42cc101e20 2024-05-15T16:12:13.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.643+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42cc105540 con 0x7f42cc101e20 2024-05-15T16:12:13.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.643+0000 7f42beffd700 1 -- 172.21.15.49:0/2601437726 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f42c8003a60 con 0x7f42cc101e20 2024-05-15T16:12:13.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.643+0000 7f42beffd700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f42b806e200 0x7f42b80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:13.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.644+0000 7f42beffd700 1 -- 172.21.15.49:0/2601437726 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f42c808f570 con 0x7f42cc101e20 2024-05-15T16:12:13.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.644+0000 7f42d133e700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f42b806e200 0x7f42b80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:13.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.644+0000 7f42d133e700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f42b806e200 0x7f42b80706c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f42c0005950 tx=0x7f42c00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:13.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.648+0000 7f42beffd700 1 -- 172.21.15.49:0/2601437726 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f42c805c400 con 0x7f42cc101e20 2024-05-15T16:12:13.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.879+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f42cc197ec0 con 0x7f42b806e200 2024-05-15T16:12:13.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.880+0000 7f42beffd700 1 -- 172.21.15.49:0/2601437726 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f42cc197ec0 con 0x7f42b806e200 2024-05-15T16:12:13.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f42b806e200 msgr2=0x7f42b80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:13.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f42b806e200 0x7f42b80706c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f42c0005950 tx=0x7f42c00058e0 comp rx=0 tx=0).stop 2024-05-15T16:12:13.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 msgr2=0x7f42cc19c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:13.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 0x7f42cc19c8c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f42c800bd70 tx=0x7f42c800be50 comp rx=0 tx=0).stop 2024-05-15T16:12:13.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 shutdown_connections 2024-05-15T16:12:13.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f42b806e200 0x7f42b80706c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:13.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42cc101e20 0x7f42cc19c8c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:13.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 --2- 172.21.15.49:0/2601437726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42cc102740 0x7f42cc19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:13.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 >> 172.21.15.49:0/2601437726 conn(0x7f42cc076800 msgr2=0x7f42cc103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:13.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 shutdown_connections 2024-05-15T16:12:13.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.882+0000 7f42d3da3700 1 -- 172.21.15.49:0/2601437726 wait complete. 2024-05-15T16:12:13.894 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:12:13.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.992+0000 7f8d5d081700 1 -- 172.21.15.49:0/1556379211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 msgr2=0x7f8d581012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:13.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.992+0000 7f8d5d081700 1 --2- 172.21.15.49:0/1556379211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 0x7f8d581012e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009910 tx=0x7f8d48009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:13.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.992+0000 7f8d5d081700 1 -- 172.21.15.49:0/1556379211 shutdown_connections 2024-05-15T16:12:13.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.992+0000 7f8d5d081700 1 --2- 172.21.15.49:0/1556379211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d581018b0 0x7f8d58105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:13.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.992+0000 7f8d5d081700 1 --2- 172.21.15.49:0/1556379211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 0x7f8d581012e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:13.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.992+0000 7f8d5d081700 1 -- 172.21.15.49:0/1556379211 >> 172.21.15.49:0/1556379211 conn(0x7f8d580fc7d0 msgr2=0x7f8d580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:13.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.992+0000 7f8d5d081700 1 -- 172.21.15.49:0/1556379211 shutdown_connections 2024-05-15T16:12:13.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.992+0000 7f8d5d081700 1 -- 172.21.15.49:0/1556379211 wait complete. 2024-05-15T16:12:13.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.993+0000 7f8d5d081700 1 Processor -- start 2024-05-15T16:12:13.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.993+0000 7f8d5d081700 1 -- start start 2024-05-15T16:12:13.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d5d081700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 0x7f8d5819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:13.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d5d081700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d581018b0 0x7f8d5819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:13.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d5d081700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5819d3d0 con 0x7f8d58100f00 2024-05-15T16:12:13.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d5d081700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d58196a90 con 0x7f8d581018b0 2024-05-15T16:12:13.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d56d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 0x7f8d5819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:13.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d56d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 0x7f8d5819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42852/0 (socket says 172.21.15.49:42852) 2024-05-15T16:12:13.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d56d9d700 1 -- 172.21.15.49:0/874627130 learned_addr learned my addr 172.21.15.49:0/874627130 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:13.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d5659c700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d581018b0 0x7f8d5819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:13.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d56d9d700 1 -- 172.21.15.49:0/874627130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d581018b0 msgr2=0x7f8d5819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:13.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d56d9d700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d581018b0 0x7f8d5819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d56d9d700 1 -- 172.21.15.49:0/874627130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d480095f0 con 0x7f8d58100f00 2024-05-15T16:12:14.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d56d9d700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 0x7f8d5819c900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8d480092e0 tx=0x7f8d4800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:14.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d4ffff700 1 -- 172.21.15.49:0/874627130 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4801d070 con 0x7f8d58100f00 2024-05-15T16:12:14.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d4ffff700 1 -- 172.21.15.49:0/874627130 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d4800f460 con 0x7f8d58100f00 2024-05-15T16:12:14.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d4ffff700 1 -- 172.21.15.49:0/874627130 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d48017630 con 0x7f8d58100f00 2024-05-15T16:12:14.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d58196d10 con 0x7f8d58100f00 2024-05-15T16:12:14.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.994+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d58197200 con 0x7f8d58100f00 2024-05-15T16:12:14.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.995+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d581092a0 con 0x7f8d58100f00 2024-05-15T16:12:14.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.996+0000 7f8d4ffff700 1 -- 172.21.15.49:0/874627130 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8d4800f5d0 con 0x7f8d58100f00 2024-05-15T16:12:14.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.996+0000 7f8d4ffff700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d4406e210 0x7f8d440706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:14.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.996+0000 7f8d4ffff700 1 -- 172.21.15.49:0/874627130 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8d4808f5c0 con 0x7f8d58100f00 2024-05-15T16:12:14.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.997+0000 7f8d5659c700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d4406e210 0x7f8d440706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:14.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:13.997+0000 7f8d5659c700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d4406e210 0x7f8d440706d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d58197c00 tx=0x7f8d40005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:14.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.001+0000 7f8d4ffff700 1 -- 172.21.15.49:0/874627130 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8d4805c450 con 0x7f8d58100f00 2024-05-15T16:12:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:14 smithi049 ceph-mon[112142]: pgmap v455: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:14.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.228+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d58197f70 con 0x7f8d4406e210 2024-05-15T16:12:14.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.230+0000 7f8d4ffff700 1 -- 172.21.15.49:0/874627130 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8d58197f70 con 0x7f8d4406e210 2024-05-15T16:12:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d4406e210 msgr2=0x7f8d440706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d4406e210 0x7f8d440706d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d58197c00 tx=0x7f8d40005d50 comp rx=0 tx=0).stop 2024-05-15T16:12:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 msgr2=0x7f8d5819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 0x7f8d5819c900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8d480092e0 tx=0x7f8d4800be20 comp rx=0 tx=0).stop 2024-05-15T16:12:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 shutdown_connections 2024-05-15T16:12:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8d4406e210 0x7f8d440706d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d58100f00 0x7f8d5819c900 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 --2- 172.21.15.49:0/874627130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d581018b0 0x7f8d5819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 >> 172.21.15.49:0/874627130 conn(0x7f8d580fc7d0 msgr2=0x7f8d580fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 shutdown_connections 2024-05-15T16:12:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.232+0000 7f8d5d081700 1 -- 172.21.15.49:0/874627130 wait complete. 2024-05-15T16:12:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.340+0000 7f8bdc27d700 1 -- 172.21.15.49:0/1480268337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 msgr2=0x7f8bd41022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.340+0000 7f8bdc27d700 1 --2- 172.21.15.49:0/1480268337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 0x7f8bd41022f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009910 tx=0x7f8bd0009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.340+0000 7f8bdc27d700 1 -- 172.21.15.49:0/1480268337 shutdown_connections 2024-05-15T16:12:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.340+0000 7f8bdc27d700 1 --2- 172.21.15.49:0/1480268337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd4102830 0x7f8bd410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.340+0000 7f8bdc27d700 1 --2- 172.21.15.49:0/1480268337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 0x7f8bd41022f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.340+0000 7f8bdc27d700 1 -- 172.21.15.49:0/1480268337 >> 172.21.15.49:0/1480268337 conn(0x7f8bd40fb6a0 msgr2=0x7f8bd40fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.340+0000 7f8bdc27d700 1 -- 172.21.15.49:0/1480268337 shutdown_connections 2024-05-15T16:12:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.340+0000 7f8bdc27d700 1 -- 172.21.15.49:0/1480268337 wait complete. 2024-05-15T16:12:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.341+0000 7f8bdc27d700 1 Processor -- start 2024-05-15T16:12:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.341+0000 7f8bdc27d700 1 -- start start 2024-05-15T16:12:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.341+0000 7f8bdc27d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 0x7f8bd419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.341+0000 7f8bdc27d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd4102830 0x7f8bd419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.341+0000 7f8bdc27d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd419d490 con 0x7f8bd4101f10 2024-05-15T16:12:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.341+0000 7f8bdc27d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd4196a70 con 0x7f8bd4102830 2024-05-15T16:12:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.341+0000 7f8bda019700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 0x7f8bd419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.341+0000 7f8bda019700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 0x7f8bd419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42872/0 (socket says 172.21.15.49:42872) 2024-05-15T16:12:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.341+0000 7f8bda019700 1 -- 172.21.15.49:0/912151780 learned_addr learned my addr 172.21.15.49:0/912151780 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bd9818700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd4102830 0x7f8bd419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bda019700 1 -- 172.21.15.49:0/912151780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd4102830 msgr2=0x7f8bd419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bda019700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd4102830 0x7f8bd419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bda019700 1 -- 172.21.15.49:0/912151780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd00095f0 con 0x7f8bd4101f10 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bd9818700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd4102830 0x7f8bd419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bda019700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 0x7f8bd419c930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0006010 tx=0x7f8bd000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bcb7fe700 1 -- 172.21.15.49:0/912151780 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd001d070 con 0x7f8bd4101f10 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd4196cf0 con 0x7f8bd4101f10 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bcb7fe700 1 -- 172.21.15.49:0/912151780 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bd000f460 con 0x7f8bd4101f10 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bcb7fe700 1 -- 172.21.15.49:0/912151780 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd0017630 con 0x7f8bd4101f10 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.342+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd41971e0 con 0x7f8bd4101f10 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.343+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd4105630 con 0x7f8bd4101f10 2024-05-15T16:12:14.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.344+0000 7f8bcb7fe700 1 -- 172.21.15.49:0/912151780 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8bd000f5d0 con 0x7f8bd4101f10 2024-05-15T16:12:14.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.344+0000 7f8bcb7fe700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8bc006e210 0x7f8bc00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:14.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.344+0000 7f8bcb7fe700 1 -- 172.21.15.49:0/912151780 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8bd008f650 con 0x7f8bd4101f10 2024-05-15T16:12:14.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.344+0000 7f8bd9818700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8bc006e210 0x7f8bc00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:14.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.345+0000 7f8bd9818700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8bc006e210 0x7f8bc00706d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4005fd0 tx=0x7f8bc4005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:14.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.349+0000 7f8bcb7fe700 1 -- 172.21.15.49:0/912151780 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8bd005c520 con 0x7f8bd4101f10 2024-05-15T16:12:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:14 smithi144 ceph-mon[86635]: pgmap v455: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:14.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.565+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bd4197f30 con 0x7f8bc006e210 2024-05-15T16:12:14.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.576+0000 7f8bcb7fe700 1 -- 172.21.15.49:0/912151780 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8bd4197f30 con 0x7f8bc006e210 2024-05-15T16:12:14.576 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:12:14.576 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (15m) 8m ago 27m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:12:14.576 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (27m) 8m ago 27m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (26m) 8m ago 26m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (12m) 8m ago 27m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (12m) 8m ago 26m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (15m) 8m ago 26m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (9m) 8m ago 21m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (8m) 8m ago 21m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (8m) 8m ago 21m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (21m) 8m ago 21m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (17m) 8m ago 28m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (16m) 8m ago 25m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (14m) 8m ago 28m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (13m) 8m ago 25m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (16m) 8m ago 27m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (16m) 8m ago 26m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:12:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (12m) 8m ago 25m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:12:14.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (11m) 8m ago 24m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:12:14.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (11m) 8m ago 24m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:12:14.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (10m) 8m ago 23m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:12:14.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (10m) 8m ago 23m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:12:14.578 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (9m) 8m ago 22m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:12:14.578 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (15m) 8m ago 26m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:12:14.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8bc006e210 msgr2=0x7f8bc00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8bc006e210 0x7f8bc00706d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4005fd0 tx=0x7f8bc4005dc0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 msgr2=0x7f8bd419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 0x7f8bd419c930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0006010 tx=0x7f8bd000be20 comp rx=0 tx=0).stop 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 shutdown_connections 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8bc006e210 0x7f8bc00706d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bd4101f10 0x7f8bd419c930 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 --2- 172.21.15.49:0/912151780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd4102830 0x7f8bd419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 >> 172.21.15.49:0/912151780 conn(0x7f8bd40fb6a0 msgr2=0x7f8bd4103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 shutdown_connections 2024-05-15T16:12:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.578+0000 7f8bdc27d700 1 -- 172.21.15.49:0/912151780 wait complete. 2024-05-15T16:12:14.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.685+0000 7ff672b82700 1 -- 172.21.15.49:0/2253696405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 msgr2=0x7ff66c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.685+0000 7ff672b82700 1 --2- 172.21.15.49:0/2253696405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 0x7ff66c1058e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff660009910 tx=0x7ff660009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.685+0000 7ff672b82700 1 -- 172.21.15.49:0/2253696405 shutdown_connections 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.685+0000 7ff672b82700 1 --2- 172.21.15.49:0/2253696405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 0x7ff66c1058e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.685+0000 7ff672b82700 1 --2- 172.21.15.49:0/2253696405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c100ee0 0x7ff66c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.685+0000 7ff672b82700 1 -- 172.21.15.49:0/2253696405 >> 172.21.15.49:0/2253696405 conn(0x7ff66c0fc790 msgr2=0x7ff66c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.685+0000 7ff672b82700 1 -- 172.21.15.49:0/2253696405 shutdown_connections 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.685+0000 7ff672b82700 1 -- 172.21.15.49:0/2253696405 wait complete. 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.685+0000 7ff672b82700 1 Processor -- start 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff672b82700 1 -- start start 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff672b82700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c100ee0 0x7ff66c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff672b82700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 0x7ff66c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff672b82700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c073d20 con 0x7ff66c101890 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff672b82700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c073e60 con 0x7ff66c100ee0 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff66bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 0x7ff66c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff66bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 0x7ff66c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42898/0 (socket says 172.21.15.49:42898) 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff66bfff700 1 -- 172.21.15.49:0/2019871240 learned_addr learned my addr 172.21.15.49:0/2019871240 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff67091e700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c100ee0 0x7ff66c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff66bfff700 1 -- 172.21.15.49:0/2019871240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c100ee0 msgr2=0x7ff66c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff66bfff700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c100ee0 0x7ff66c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.686+0000 7ff66bfff700 1 -- 172.21.15.49:0/2019871240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600095f0 con 0x7ff66c101890 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.687+0000 7ff66bfff700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 0x7ff66c0737e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff6600092e0 tx=0x7ff66000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.687+0000 7ff669ffb700 1 -- 172.21.15.49:0/2019871240 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff66001d070 con 0x7ff66c101890 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.687+0000 7ff669ffb700 1 -- 172.21.15.49:0/2019871240 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff660005700 con 0x7ff66c101890 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.687+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66c0740e0 con 0x7ff66c101890 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.687+0000 7ff669ffb700 1 -- 172.21.15.49:0/2019871240 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff66000fdb0 con 0x7ff66c101890 2024-05-15T16:12:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.687+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c1a3020 con 0x7ff66c101890 2024-05-15T16:12:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.688+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66c1092e0 con 0x7ff66c101890 2024-05-15T16:12:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.689+0000 7ff669ffb700 1 -- 172.21.15.49:0/2019871240 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff66000f5f0 con 0x7ff66c101890 2024-05-15T16:12:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.689+0000 7ff669ffb700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff65406e200 0x7ff6540706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.689+0000 7ff669ffb700 1 -- 172.21.15.49:0/2019871240 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff66008f3b0 con 0x7ff66c101890 2024-05-15T16:12:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.689+0000 7ff67091e700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff65406e200 0x7ff6540706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.690+0000 7ff67091e700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff65406e200 0x7ff6540706c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff65c004630 tx=0x7ff65c00b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:14.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.694+0000 7ff669ffb700 1 -- 172.21.15.49:0/2019871240 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff660058710 con 0x7ff66c101890 2024-05-15T16:12:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.993+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff66c074780 con 0x7ff66c101890 2024-05-15T16:12:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.993+0000 7ff669ffb700 1 -- 172.21.15.49:0/2019871240 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7ff66005bdd0 con 0x7ff66c101890 2024-05-15T16:12:14.994 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:12:14.994 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:12:14.994 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:12:14.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:12:14.994 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:12:14.994 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:12:14.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:12:14.996 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:12:14.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.995+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff65406e200 msgr2=0x7ff6540706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.995+0000 7ff672b82700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff65406e200 0x7ff6540706c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff65c004630 tx=0x7ff65c00b3c0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.995+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 msgr2=0x7ff66c0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.995+0000 7ff672b82700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 0x7ff66c0737e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff6600092e0 tx=0x7ff66000bd70 comp rx=0 tx=0).stop 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.996+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 shutdown_connections 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.996+0000 7ff672b82700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff65406e200 0x7ff6540706c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.996+0000 7ff672b82700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c100ee0 0x7ff66c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.996+0000 7ff672b82700 1 --2- 172.21.15.49:0/2019871240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff66c101890 0x7ff66c0737e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.996+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 >> 172.21.15.49:0/2019871240 conn(0x7ff66c0fc790 msgr2=0x7ff66c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.996+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 shutdown_connections 2024-05-15T16:12:14.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:14.996+0000 7ff672b82700 1 -- 172.21.15.49:0/2019871240 wait complete. 2024-05-15T16:12:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.105+0000 7f124ed48700 1 -- 172.21.15.49:0/1476776103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248100ee0 msgr2=0x7f12481012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.105+0000 7f124ed48700 1 --2- 172.21.15.49:0/1476776103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248100ee0 0x7f12481012c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f123c009910 tx=0x7f123c009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.105+0000 7f124ed48700 1 -- 172.21.15.49:0/1476776103 shutdown_connections 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.105+0000 7f124ed48700 1 --2- 172.21.15.49:0/1476776103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1248101890 0x7f12481058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.105+0000 7f124ed48700 1 --2- 172.21.15.49:0/1476776103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248100ee0 0x7f12481012c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.105+0000 7f124ed48700 1 -- 172.21.15.49:0/1476776103 >> 172.21.15.49:0/1476776103 conn(0x7f12480fc790 msgr2=0x7f12480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.105+0000 7f124ed48700 1 -- 172.21.15.49:0/1476776103 shutdown_connections 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.105+0000 7f124ed48700 1 -- 172.21.15.49:0/1476776103 wait complete. 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.106+0000 7f124ed48700 1 Processor -- start 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.106+0000 7f124ed48700 1 -- start start 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.106+0000 7f124ed48700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1248100ee0 0x7f124819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.106+0000 7f124ed48700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248101890 0x7f124819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.106+0000 7f124ed48700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124819d460 con 0x7f1248101890 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.106+0000 7f124ed48700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1248196a90 con 0x7f1248100ee0 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.106+0000 7f1247fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248101890 0x7f124819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.106+0000 7f1247fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248101890 0x7f124819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42924/0 (socket says 172.21.15.49:42924) 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.106+0000 7f1247fff700 1 -- 172.21.15.49:0/2827790744 learned_addr learned my addr 172.21.15.49:0/2827790744 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f124cae4700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1248100ee0 0x7f124819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f1247fff700 1 -- 172.21.15.49:0/2827790744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1248100ee0 msgr2=0x7f124819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f1247fff700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1248100ee0 0x7f124819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f1247fff700 1 -- 172.21.15.49:0/2827790744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c0095f0 con 0x7f1248101890 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f1247fff700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248101890 0x7f124819ce40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f123400d900 tx=0x7f123400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f1245ffb700 1 -- 172.21.15.49:0/2827790744 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1234009900 con 0x7f1248101890 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f1245ffb700 1 -- 172.21.15.49:0/2827790744 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1234010460 con 0x7f1248101890 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1248196d70 con 0x7f1248101890 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f1245ffb700 1 -- 172.21.15.49:0/2827790744 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123400f5f0 con 0x7f1248101890 2024-05-15T16:12:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.107+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12481972c0 con 0x7f1248101890 2024-05-15T16:12:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.108+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1248109280 con 0x7f1248101890 2024-05-15T16:12:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.109+0000 7f1245ffb700 1 -- 172.21.15.49:0/2827790744 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f12340105d0 con 0x7f1248101890 2024-05-15T16:12:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.109+0000 7f1245ffb700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f123806e210 0x7f12380706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.109+0000 7f1245ffb700 1 -- 172.21.15.49:0/2827790744 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f123408df60 con 0x7f1248101890 2024-05-15T16:12:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.109+0000 7f124cae4700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f123806e210 0x7f12380706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.110+0000 7f124cae4700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f123806e210 0x7f12380706d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f123c006010 tx=0x7f123c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:15.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.114+0000 7f1245ffb700 1 -- 172.21.15.49:0/2827790744 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f12340c2850 con 0x7f1248101890 2024-05-15T16:12:15.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.369+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1248197f00 con 0x7f1248101890 2024-05-15T16:12:15.370 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:15 smithi049 ceph-mon[112142]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:15.370 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:15 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2019871240' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:12:15.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.369+0000 7f1245ffb700 1 -- 172.21.15.49:0/2827790744 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f1234020760 con 0x7f1248101890 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:12:15.371 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:12:15.372 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:12:15.373 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:12:15.374 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:12:15.374 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:12:15.374 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:12:15.374 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:12:15.374 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:12:15.374 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f123806e210 msgr2=0x7f12380706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f123806e210 0x7f12380706d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f123c006010 tx=0x7f123c00b540 comp rx=0 tx=0).stop 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248101890 msgr2=0x7f124819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248101890 0x7f124819ce40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f123400d900 tx=0x7f123400dcc0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 shutdown_connections 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f123806e210 0x7f12380706d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1248100ee0 0x7f124819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 --2- 172.21.15.49:0/2827790744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1248101890 0x7f124819ce40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 >> 172.21.15.49:0/2827790744 conn(0x7f12480fc790 msgr2=0x7f12480fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:15.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 shutdown_connections 2024-05-15T16:12:15.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.372+0000 7f124ed48700 1 -- 172.21.15.49:0/2827790744 wait complete. 2024-05-15T16:12:15.376 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:12:15.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:15 smithi144 ceph-mon[86635]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:15.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:15 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2019871240' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:12:15.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.482+0000 7fb63b25d700 1 -- 172.21.15.49:0/4102641047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 msgr2=0x7fb6341058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.482+0000 7fb63b25d700 1 --2- 172.21.15.49:0/4102641047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 0x7fb6341058e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb628009910 tx=0x7fb628009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:15.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.482+0000 7fb63b25d700 1 -- 172.21.15.49:0/4102641047 shutdown_connections 2024-05-15T16:12:15.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.482+0000 7fb63b25d700 1 --2- 172.21.15.49:0/4102641047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 0x7fb6341058e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.482+0000 7fb63b25d700 1 --2- 172.21.15.49:0/4102641047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb634100ee0 0x7fb6341012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.482+0000 7fb63b25d700 1 -- 172.21.15.49:0/4102641047 >> 172.21.15.49:0/4102641047 conn(0x7fb6340fc790 msgr2=0x7fb6340febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.482+0000 7fb63b25d700 1 -- 172.21.15.49:0/4102641047 shutdown_connections 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.482+0000 7fb63b25d700 1 -- 172.21.15.49:0/4102641047 wait complete. 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.482+0000 7fb63b25d700 1 Processor -- start 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb63b25d700 1 -- start start 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb63b25d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb634100ee0 0x7fb6340732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb63b25d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 0x7fb6340737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb63b25d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb634073d20 con 0x7fb634101890 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb63b25d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb634073e60 con 0x7fb634100ee0 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb633fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 0x7fb6340737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb633fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 0x7fb6340737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42946/0 (socket says 172.21.15.49:42946) 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb633fff700 1 -- 172.21.15.49:0/266633464 learned_addr learned my addr 172.21.15.49:0/266633464 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb638ff9700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb634100ee0 0x7fb6340732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb633fff700 1 -- 172.21.15.49:0/266633464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb634100ee0 msgr2=0x7fb6340732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb633fff700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb634100ee0 0x7fb6340732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb633fff700 1 -- 172.21.15.49:0/266633464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6280095f0 con 0x7fb634101890 2024-05-15T16:12:15.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.483+0000 7fb633fff700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 0x7fb6340737e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb6280092e0 tx=0x7fb62800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:15.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.484+0000 7fb631ffb700 1 -- 172.21.15.49:0/266633464 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb62801d070 con 0x7fb634101890 2024-05-15T16:12:15.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.484+0000 7fb631ffb700 1 -- 172.21.15.49:0/266633464 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb62800f460 con 0x7fb634101890 2024-05-15T16:12:15.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.484+0000 7fb631ffb700 1 -- 172.21.15.49:0/266633464 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb628017610 con 0x7fb634101890 2024-05-15T16:12:15.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.484+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6340740e0 con 0x7fb634101890 2024-05-15T16:12:15.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.484+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6341a2fb0 con 0x7fb634101890 2024-05-15T16:12:15.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.485+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6341092e0 con 0x7fb634101890 2024-05-15T16:12:15.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.486+0000 7fb631ffb700 1 -- 172.21.15.49:0/266633464 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb62800f5d0 con 0x7fb634101890 2024-05-15T16:12:15.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.486+0000 7fb631ffb700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb61c06e210 0x7fb61c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:15.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.486+0000 7fb631ffb700 1 -- 172.21.15.49:0/266633464 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb62808f670 con 0x7fb634101890 2024-05-15T16:12:15.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.486+0000 7fb638ff9700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb61c06e210 0x7fb61c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:15.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.487+0000 7fb638ff9700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb61c06e210 0x7fb61c0706d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb624005fd0 tx=0x7fb62400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:15.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.491+0000 7fb631ffb700 1 -- 172.21.15.49:0/266633464 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb628058950 con 0x7fb634101890 2024-05-15T16:12:15.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.717+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb63410de80 con 0x7fb61c06e210 2024-05-15T16:12:15.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.718+0000 7fb631ffb700 1 -- 172.21.15.49:0/266633464 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb63410de80 con 0x7fb61c06e210 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:12:15.719 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:12:15.720 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:12:15.720 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:12:15.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.720+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb61c06e210 msgr2=0x7fb61c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.720+0000 7fb63b25d700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb61c06e210 0x7fb61c0706d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb624005fd0 tx=0x7fb62400a560 comp rx=0 tx=0).stop 2024-05-15T16:12:15.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.720+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 msgr2=0x7fb6340737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.720+0000 7fb63b25d700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 0x7fb6340737e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb6280092e0 tx=0x7fb62800be20 comp rx=0 tx=0).stop 2024-05-15T16:12:15.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.721+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 shutdown_connections 2024-05-15T16:12:15.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.721+0000 7fb63b25d700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb61c06e210 0x7fb61c0706d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.721+0000 7fb63b25d700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb634100ee0 0x7fb6340732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.721+0000 7fb63b25d700 1 --2- 172.21.15.49:0/266633464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb634101890 0x7fb6340737e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.721+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 >> 172.21.15.49:0/266633464 conn(0x7fb6340fc790 msgr2=0x7fb6340fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:15.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.721+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 shutdown_connections 2024-05-15T16:12:15.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.721+0000 7fb63b25d700 1 -- 172.21.15.49:0/266633464 wait complete. 2024-05-15T16:12:15.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.829+0000 7ff73a65f700 1 -- 172.21.15.49:0/750397240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 msgr2=0x7ff7341012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.829+0000 7ff73a65f700 1 --2- 172.21.15.49:0/750397240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 0x7ff7341012c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff724009910 tx=0x7ff724009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:15.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.829+0000 7ff73a65f700 1 -- 172.21.15.49:0/750397240 shutdown_connections 2024-05-15T16:12:15.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.829+0000 7ff73a65f700 1 --2- 172.21.15.49:0/750397240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff734101890 0x7ff7341058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.829+0000 7ff73a65f700 1 --2- 172.21.15.49:0/750397240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 0x7ff7341012c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.829+0000 7ff73a65f700 1 -- 172.21.15.49:0/750397240 >> 172.21.15.49:0/750397240 conn(0x7ff7340fc790 msgr2=0x7ff7340febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:15.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.829+0000 7ff73a65f700 1 -- 172.21.15.49:0/750397240 shutdown_connections 2024-05-15T16:12:15.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.829+0000 7ff73a65f700 1 -- 172.21.15.49:0/750397240 wait complete. 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.830+0000 7ff73a65f700 1 Processor -- start 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.830+0000 7ff73a65f700 1 -- start start 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.830+0000 7ff73a65f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 0x7ff73419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.830+0000 7ff73a65f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff734101890 0x7ff73419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.830+0000 7ff73a65f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73419d4b0 con 0x7ff734100ee0 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.830+0000 7ff73a65f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff734196a90 con 0x7ff734101890 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.830+0000 7ff733fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 0x7ff73419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.830+0000 7ff733fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 0x7ff73419c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42958/0 (socket says 172.21.15.49:42958) 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.830+0000 7ff733fff700 1 -- 172.21.15.49:0/124071306 learned_addr learned my addr 172.21.15.49:0/124071306 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff72bfff700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff734101890 0x7ff73419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff733fff700 1 -- 172.21.15.49:0/124071306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff734101890 msgr2=0x7ff73419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff733fff700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff734101890 0x7ff73419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff733fff700 1 -- 172.21.15.49:0/124071306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240095f0 con 0x7ff734100ee0 2024-05-15T16:12:15.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff733fff700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 0x7ff73419c950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff724006010 tx=0x7ff72400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:15.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff731ffb700 1 -- 172.21.15.49:0/124071306 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72401d070 con 0x7ff734100ee0 2024-05-15T16:12:15.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff731ffb700 1 -- 172.21.15.49:0/124071306 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7240056e0 con 0x7ff734100ee0 2024-05-15T16:12:15.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff731ffb700 1 -- 172.21.15.49:0/124071306 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff724017400 con 0x7ff734100ee0 2024-05-15T16:12:15.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff734196d10 con 0x7ff734100ee0 2024-05-15T16:12:15.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.831+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff734197200 con 0x7ff734100ee0 2024-05-15T16:12:15.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.832+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff734109280 con 0x7ff734100ee0 2024-05-15T16:12:15.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.833+0000 7ff731ffb700 1 -- 172.21.15.49:0/124071306 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff72400f6f0 con 0x7ff734100ee0 2024-05-15T16:12:15.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.833+0000 7ff731ffb700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff72006e200 0x7ff7200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:15.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.833+0000 7ff72bfff700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff72006e200 0x7ff7200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:15.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.833+0000 7ff731ffb700 1 -- 172.21.15.49:0/124071306 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff72408f520 con 0x7ff734100ee0 2024-05-15T16:12:15.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.834+0000 7ff72bfff700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff72006e200 0x7ff7200706c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff734197c50 tx=0x7ff71c0094f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:15.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:15.838+0000 7ff731ffb700 1 -- 172.21.15.49:0/124071306 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff72405c330 con 0x7ff734100ee0 2024-05-15T16:12:16.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.129+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff734197f50 con 0x7ff734100ee0 2024-05-15T16:12:16.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.130+0000 7ff731ffb700 1 -- 172.21.15.49:0/124071306 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7ff72405bec0 con 0x7ff734100ee0 2024-05-15T16:12:16.131 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:12:16.131 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:12:16.131 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:12:16.131 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:12:16.131 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:12:16.131 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:12:16.131 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:12:16.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff72006e200 msgr2=0x7ff7200706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:16.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff72006e200 0x7ff7200706c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff734197c50 tx=0x7ff71c0094f0 comp rx=0 tx=0).stop 2024-05-15T16:12:16.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 msgr2=0x7ff73419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:16.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 0x7ff73419c950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff724006010 tx=0x7ff72400be20 comp rx=0 tx=0).stop 2024-05-15T16:12:16.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 shutdown_connections 2024-05-15T16:12:16.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff72006e200 0x7ff7200706c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:16.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff734100ee0 0x7ff73419c950 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:16.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 --2- 172.21.15.49:0/124071306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff734101890 0x7ff73419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:16.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 >> 172.21.15.49:0/124071306 conn(0x7ff7340fc790 msgr2=0x7ff7340fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:16.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 shutdown_connections 2024-05-15T16:12:16.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:16.132+0000 7ff73a65f700 1 -- 172.21.15.49:0/124071306 wait complete. 2024-05-15T16:12:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:16 smithi144 ceph-mon[86635]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:16 smithi144 ceph-mon[86635]: pgmap v456: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:16 smithi144 ceph-mon[86635]: from='client.34886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:16.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:16 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2827790744' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:12:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:16 smithi049 ceph-mon[112142]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:16 smithi049 ceph-mon[112142]: pgmap v456: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:16 smithi049 ceph-mon[112142]: from='client.34886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:16 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2827790744' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:12:17.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:17 smithi144 ceph-mon[86635]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:17.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:17 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/124071306' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:12:17.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:17 smithi049 ceph-mon[112142]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:17.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:17 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/124071306' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:12:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:18 smithi144 ceph-mon[86635]: pgmap v457: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:18 smithi049 ceph-mon[112142]: pgmap v457: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:20 smithi144 ceph-mon[86635]: pgmap v458: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:20 smithi049 ceph-mon[112142]: pgmap v458: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:12:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:12:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:22 smithi144 ceph-mon[86635]: pgmap v459: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:22 smithi049 ceph-mon[112142]: pgmap v459: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:24 smithi144 ceph-mon[86635]: pgmap v460: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:24 smithi049 ceph-mon[112142]: pgmap v460: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:26 smithi144 ceph-mon[86635]: pgmap v461: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:26 smithi049 ceph-mon[112142]: pgmap v461: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:28 smithi144 ceph-mon[86635]: pgmap v462: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:28 smithi049 ceph-mon[112142]: pgmap v462: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:29.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:29 smithi144 ceph-mon[86635]: pgmap v463: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:29.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:29 smithi049 ceph-mon[112142]: pgmap v463: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:31.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:31 smithi049 ceph-mon[112142]: pgmap v464: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:31.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:31 smithi144 ceph-mon[86635]: pgmap v464: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:33.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:33 smithi144 ceph-mon[86635]: pgmap v465: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:33.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:33 smithi049 ceph-mon[112142]: pgmap v465: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:35.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:35 smithi144 ceph-mon[86635]: pgmap v466: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:35.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:12:35.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:35 smithi049 ceph-mon[112142]: pgmap v466: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:35.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:12:37.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:37 smithi144 ceph-mon[86635]: pgmap v467: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:37.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:37 smithi049 ceph-mon[112142]: pgmap v467: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:39.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:39 smithi144 ceph-mon[86635]: pgmap v468: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:39.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:39 smithi049 ceph-mon[112142]: pgmap v468: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:41.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:41 smithi049 ceph-mon[112142]: pgmap v469: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:41.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:41 smithi144 ceph-mon[86635]: pgmap v469: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:43.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:43 smithi144 ceph-mon[86635]: pgmap v470: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:43.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:43 smithi049 ceph-mon[112142]: pgmap v470: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:45.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:45 smithi144 ceph-mon[86635]: pgmap v471: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:45.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:45 smithi049 ceph-mon[112142]: pgmap v471: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:46.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.254+0000 7f034c289700 1 -- 172.21.15.49:0/1083104361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0344100f00 msgr2=0x7f03441012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.254+0000 7f034c289700 1 --2- 172.21.15.49:0/1083104361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0344100f00 0x7f03441012e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0340009910 tx=0x7f0340009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.255+0000 7f034c289700 1 -- 172.21.15.49:0/1083104361 shutdown_connections 2024-05-15T16:12:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.255+0000 7f034c289700 1 --2- 172.21.15.49:0/1083104361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03441018b0 0x7f0344105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.255+0000 7f034c289700 1 --2- 172.21.15.49:0/1083104361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0344100f00 0x7f03441012e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.255+0000 7f034c289700 1 -- 172.21.15.49:0/1083104361 >> 172.21.15.49:0/1083104361 conn(0x7f03440fc7d0 msgr2=0x7f03440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.255+0000 7f034c289700 1 -- 172.21.15.49:0/1083104361 shutdown_connections 2024-05-15T16:12:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.255+0000 7f034c289700 1 -- 172.21.15.49:0/1083104361 wait complete. 2024-05-15T16:12:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.255+0000 7f034c289700 1 Processor -- start 2024-05-15T16:12:46.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.255+0000 7f034c289700 1 -- start start 2024-05-15T16:12:46.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f034c289700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0344100f00 0x7f0344198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:46.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f034c289700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03441018b0 0x7f0344199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:46.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f034c289700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0344199b00 con 0x7f03441018b0 2024-05-15T16:12:46.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f034c289700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034419d780 con 0x7f0344100f00 2024-05-15T16:12:46.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f0349824700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03441018b0 0x7f0344199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:46.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f0349824700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03441018b0 0x7f0344199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43970/0 (socket says 172.21.15.49:43970) 2024-05-15T16:12:46.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f0349824700 1 -- 172.21.15.49:0/3523464742 learned_addr learned my addr 172.21.15.49:0/3523464742 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:46.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f034a025700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0344100f00 0x7f0344198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f0349824700 1 -- 172.21.15.49:0/3523464742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0344100f00 msgr2=0x7f0344198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f0349824700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0344100f00 0x7f0344198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f0349824700 1 -- 172.21.15.49:0/3523464742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03400095f0 con 0x7f03441018b0 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f034a025700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0344100f00 0x7f0344198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f0349824700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03441018b0 0x7f0344199420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f033400b480 tx=0x7f033400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f033b7fe700 1 -- 172.21.15.49:0/3523464742 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03340118c0 con 0x7f03441018b0 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f033b7fe700 1 -- 172.21.15.49:0/3523464742 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0334010380 con 0x7f03441018b0 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.256+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034419da60 con 0x7f03441018b0 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.257+0000 7f033b7fe700 1 -- 172.21.15.49:0/3523464742 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f033400f4f0 con 0x7f03441018b0 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.257+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034419dfb0 con 0x7f03441018b0 2024-05-15T16:12:46.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.258+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0344105080 con 0x7f03441018b0 2024-05-15T16:12:46.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.258+0000 7f033b7fe700 1 -- 172.21.15.49:0/3523464742 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f03340104f0 con 0x7f03441018b0 2024-05-15T16:12:46.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.259+0000 7f033b7fe700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f033006e200 0x7f03300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:46.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.259+0000 7f033b7fe700 1 -- 172.21.15.49:0/3523464742 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f033408dd20 con 0x7f03441018b0 2024-05-15T16:12:46.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.259+0000 7f034a025700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f033006e200 0x7f03300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:46.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.259+0000 7f034a025700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f033006e200 0x7f03300706c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0340005f50 tx=0x7f0340005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:46.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.264+0000 7f033b7fe700 1 -- 172.21.15.49:0/3523464742 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0334057080 con 0x7f03441018b0 2024-05-15T16:12:46.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.492+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f034410df40 con 0x7f033006e200 2024-05-15T16:12:46.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.493+0000 7f033b7fe700 1 -- 172.21.15.49:0/3523464742 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f034410df40 con 0x7f033006e200 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f033006e200 msgr2=0x7f03300706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f033006e200 0x7f03300706c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0340005f50 tx=0x7f0340005dc0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03441018b0 msgr2=0x7f0344199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03441018b0 0x7f0344199420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f033400b480 tx=0x7f033400b840 comp rx=0 tx=0).stop 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 shutdown_connections 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f033006e200 0x7f03300706c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0344100f00 0x7f0344198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 --2- 172.21.15.49:0/3523464742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03441018b0 0x7f0344199420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 >> 172.21.15.49:0/3523464742 conn(0x7f03440fc7d0 msgr2=0x7f03440fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 shutdown_connections 2024-05-15T16:12:46.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.495+0000 7f034c289700 1 -- 172.21.15.49:0/3523464742 wait complete. 2024-05-15T16:12:46.507 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:12:46.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.605+0000 7f8f01f78700 1 -- 172.21.15.49:0/1472026201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc100780 msgr2=0x7f8efc1047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.605+0000 7f8f01f78700 1 --2- 172.21.15.49:0/1472026201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc100780 0x7f8efc1047d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009910 tx=0x7f8ef0009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:46.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.605+0000 7f8f01f78700 1 -- 172.21.15.49:0/1472026201 shutdown_connections 2024-05-15T16:12:46.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.605+0000 7f8f01f78700 1 --2- 172.21.15.49:0/1472026201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc100780 0x7f8efc1047d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.605+0000 7f8f01f78700 1 --2- 172.21.15.49:0/1472026201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc0ffdd0 0x7f8efc1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.605+0000 7f8f01f78700 1 -- 172.21.15.49:0/1472026201 >> 172.21.15.49:0/1472026201 conn(0x7f8efc0fb6a0 msgr2=0x7f8efc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:46.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.605+0000 7f8f01f78700 1 -- 172.21.15.49:0/1472026201 shutdown_connections 2024-05-15T16:12:46.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.605+0000 7f8f01f78700 1 -- 172.21.15.49:0/1472026201 wait complete. 2024-05-15T16:12:46.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.606+0000 7f8f01f78700 1 Processor -- start 2024-05-15T16:12:46.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.606+0000 7f8f01f78700 1 -- start start 2024-05-15T16:12:46.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.606+0000 7f8f01f78700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc0ffdd0 0x7f8efc101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:46.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.606+0000 7f8f01f78700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc100780 0x7f8efc101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:46.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.606+0000 7f8f01f78700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc1022f0 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.606+0000 7f8f01f78700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc102430 con 0x7f8efc100780 2024-05-15T16:12:46.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8efb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc0ffdd0 0x7f8efc101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:46.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8efb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc0ffdd0 0x7f8efc101870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43982/0 (socket says 172.21.15.49:43982) 2024-05-15T16:12:46.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8efb7fe700 1 -- 172.21.15.49:0/1233680399 learned_addr learned my addr 172.21.15.49:0/1233680399 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8efaffd700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc100780 0x7f8efc101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8efb7fe700 1 -- 172.21.15.49:0/1233680399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc100780 msgr2=0x7f8efc101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8efb7fe700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc100780 0x7f8efc101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8efb7fe700 1 -- 172.21.15.49:0/1233680399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef00095f0 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8efb7fe700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc0ffdd0 0x7f8efc101870 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00d8d0 tx=0x7f8eec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8ef8ff9700 1 -- 172.21.15.49:0/1233680399 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eec009940 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8ef8ff9700 1 -- 172.21.15.49:0/1233680399 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8eec010460 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc1a2d70 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8ef8ff9700 1 -- 172.21.15.49:0/1233680399 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eec00f5f0 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.607+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc1a30d0 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.608+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc1091e0 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.609+0000 7f8ef8ff9700 1 -- 172.21.15.49:0/1233680399 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8eec0105d0 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.609+0000 7f8ef8ff9700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ee406e200 0x7f8ee40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:46.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.609+0000 7f8ef8ff9700 1 -- 172.21.15.49:0/1233680399 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8eec08e090 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.609+0000 7f8efaffd700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ee406e200 0x7f8ee40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:46.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.610+0000 7f8efaffd700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ee406e200 0x7f8ee40706c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8efc102fa0 tx=0x7f8ef001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:46.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.614+0000 7f8ef8ff9700 1 -- 172.21.15.49:0/1233680399 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8eec0573f0 con 0x7f8efc0ffdd0 2024-05-15T16:12:46.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.839+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8efc10cd70 con 0x7f8ee406e200 2024-05-15T16:12:46.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.841+0000 7f8ef8ff9700 1 -- 172.21.15.49:0/1233680399 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8efc10cd70 con 0x7f8ee406e200 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ee406e200 msgr2=0x7f8ee40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ee406e200 0x7f8ee40706c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8efc102fa0 tx=0x7f8ef001a040 comp rx=0 tx=0).stop 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc0ffdd0 msgr2=0x7f8efc101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc0ffdd0 0x7f8efc101870 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00d8d0 tx=0x7f8eec00dc90 comp rx=0 tx=0).stop 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 shutdown_connections 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ee406e200 0x7f8ee40706c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8efc0ffdd0 0x7f8efc101870 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 --2- 172.21.15.49:0/1233680399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc100780 0x7f8efc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 >> 172.21.15.49:0/1233680399 conn(0x7f8efc0fb6a0 msgr2=0x7f8efc069460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 shutdown_connections 2024-05-15T16:12:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.843+0000 7f8f01f78700 1 -- 172.21.15.49:0/1233680399 wait complete. 2024-05-15T16:12:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.952+0000 7fd2b8317700 1 -- 172.21.15.49:0/4273074101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b00ffdf0 msgr2=0x7fd2b01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.952+0000 7fd2b8317700 1 --2- 172.21.15.49:0/4273074101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b00ffdf0 0x7fd2b01001d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000b0a0 tx=0x7fd2a000b3b0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.952+0000 7fd2b8317700 1 -- 172.21.15.49:0/4273074101 shutdown_connections 2024-05-15T16:12:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.952+0000 7fd2b8317700 1 --2- 172.21.15.49:0/4273074101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b01007a0 0x7fd2b01047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.952+0000 7fd2b8317700 1 --2- 172.21.15.49:0/4273074101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b00ffdf0 0x7fd2b01001d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.952+0000 7fd2b8317700 1 -- 172.21.15.49:0/4273074101 >> 172.21.15.49:0/4273074101 conn(0x7fd2b00fb6a0 msgr2=0x7fd2b00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.952+0000 7fd2b8317700 1 -- 172.21.15.49:0/4273074101 shutdown_connections 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.952+0000 7fd2b8317700 1 -- 172.21.15.49:0/4273074101 wait complete. 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b8317700 1 Processor -- start 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b8317700 1 -- start start 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b8317700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b00ffdf0 0x7fd2b0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b8317700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b01007a0 0x7fd2b0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b8317700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b01985c0 con 0x7fd2b01007a0 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b8317700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0198700 con 0x7fd2b00ffdf0 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b58b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b01007a0 0x7fd2b0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b58b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b01007a0 0x7fd2b0197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43998/0 (socket says 172.21.15.49:43998) 2024-05-15T16:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b58b2700 1 -- 172.21.15.49:0/1250997254 learned_addr learned my addr 172.21.15.49:0/1250997254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b60b3700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b00ffdf0 0x7fd2b0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b58b2700 1 -- 172.21.15.49:0/1250997254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b00ffdf0 msgr2=0x7fd2b0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b58b2700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b00ffdf0 0x7fd2b0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b58b2700 1 -- 172.21.15.49:0/1250997254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0009d00 con 0x7fd2b01007a0 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.953+0000 7fd2b58b2700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b01007a0 0x7fd2b0197ff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400c8f0 tx=0x7fd2a400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.954+0000 7fd2af7fe700 1 -- 172.21.15.49:0/1250997254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a40078a0 con 0x7fd2b01007a0 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.954+0000 7fd2af7fe700 1 -- 172.21.15.49:0/1250997254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2a400f450 con 0x7fd2b01007a0 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.954+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b01a0b20 con 0x7fd2b01007a0 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.954+0000 7fd2af7fe700 1 -- 172.21.15.49:0/1250997254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a4018610 con 0x7fd2b01007a0 2024-05-15T16:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.954+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b01a0f30 con 0x7fd2b01007a0 2024-05-15T16:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.955+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b0108190 con 0x7fd2b01007a0 2024-05-15T16:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.955+0000 7fd2af7fe700 1 -- 172.21.15.49:0/1250997254 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd2a400f5c0 con 0x7fd2b01007a0 2024-05-15T16:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.956+0000 7fd2af7fe700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd29806e210 0x7fd2980706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.956+0000 7fd2af7fe700 1 -- 172.21.15.49:0/1250997254 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd2a408e060 con 0x7fd2b01007a0 2024-05-15T16:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.956+0000 7fd2b60b3700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd29806e210 0x7fd2980706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.956+0000 7fd2b60b3700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd29806e210 0x7fd2980706d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0000f80 tx=0x7fd2a00097e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:46.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:46.960+0000 7fd2af7fe700 1 -- 172.21.15.49:0/1250997254 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd2a40573c0 con 0x7fd2b01007a0 2024-05-15T16:12:47.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.174+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2b010cdb0 con 0x7fd29806e210 2024-05-15T16:12:47.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.185+0000 7fd2af7fe700 1 -- 172.21.15.49:0/1250997254 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd2b010cdb0 con 0x7fd29806e210 2024-05-15T16:12:47.187 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:12:47.187 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (16m) 9m ago 27m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:12:47.187 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (27m) 9m ago 27m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:12:47.187 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (26m) 9m ago 26m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:12:47.187 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (13m) 9m ago 27m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:12:47.187 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (13m) 9m ago 26m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:12:47.187 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (15m) 9m ago 27m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:12:47.187 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (9m) 9m ago 22m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (9m) 9m ago 21m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (9m) 9m ago 21m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (21m) 9m ago 21m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (18m) 9m ago 29m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (17m) 9m ago 26m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (14m) 9m ago 29m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (14m) 9m ago 26m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (16m) 9m ago 27m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (16m) 9m ago 26m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (12m) 9m ago 25m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (12m) 9m ago 25m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (11m) 9m ago 24m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (11m) 9m ago 24m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (10m) 9m ago 23m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:12:47.188 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (10m) 9m ago 23m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:12:47.189 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (16m) 9m ago 27m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:12:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.187+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd29806e210 msgr2=0x7fd2980706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.187+0000 7fd2b8317700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd29806e210 0x7fd2980706d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0000f80 tx=0x7fd2a00097e0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.187+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b01007a0 msgr2=0x7fd2b0197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.187+0000 7fd2b8317700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b01007a0 0x7fd2b0197ff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400c8f0 tx=0x7fd2a400cc00 comp rx=0 tx=0).stop 2024-05-15T16:12:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.188+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 shutdown_connections 2024-05-15T16:12:47.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.188+0000 7fd2b8317700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd29806e210 0x7fd2980706d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.188+0000 7fd2b8317700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b00ffdf0 0x7fd2b0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.188+0000 7fd2b8317700 1 --2- 172.21.15.49:0/1250997254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b01007a0 0x7fd2b0197ff0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.188+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 >> 172.21.15.49:0/1250997254 conn(0x7fd2b00fb6a0 msgr2=0x7fd2b00fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:47.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.188+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 shutdown_connections 2024-05-15T16:12:47.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.188+0000 7fd2b8317700 1 -- 172.21.15.49:0/1250997254 wait complete. 2024-05-15T16:12:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.295+0000 7f30b8fd7700 1 -- 172.21.15.49:0/4230899467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 msgr2=0x7f30b41058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.295+0000 7f30b8fd7700 1 --2- 172.21.15.49:0/4230899467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 0x7f30b41058f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f309c009910 tx=0x7f309c009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.295+0000 7f30b8fd7700 1 -- 172.21.15.49:0/4230899467 shutdown_connections 2024-05-15T16:12:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.295+0000 7f30b8fd7700 1 --2- 172.21.15.49:0/4230899467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 0x7f30b41058f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.295+0000 7f30b8fd7700 1 --2- 172.21.15.49:0/4230899467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30b4100ef0 0x7f30b41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.295+0000 7f30b8fd7700 1 -- 172.21.15.49:0/4230899467 >> 172.21.15.49:0/4230899467 conn(0x7f30b40fc7c0 msgr2=0x7f30b40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.295+0000 7f30b8fd7700 1 -- 172.21.15.49:0/4230899467 shutdown_connections 2024-05-15T16:12:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.295+0000 7f30b8fd7700 1 -- 172.21.15.49:0/4230899467 wait complete. 2024-05-15T16:12:47.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.296+0000 7f30b8fd7700 1 Processor -- start 2024-05-15T16:12:47.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.296+0000 7f30b8fd7700 1 -- start start 2024-05-15T16:12:47.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.296+0000 7f30b8fd7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30b4100ef0 0x7f30b40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:47.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.296+0000 7f30b8fd7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 0x7f30b40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:47.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.296+0000 7f30b8fd7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b4073d20 con 0x7f30b41018a0 2024-05-15T16:12:47.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.296+0000 7f30b8fd7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b4073e60 con 0x7f30b4100ef0 2024-05-15T16:12:47.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 0x7f30b40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 0x7f30b40737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44012/0 (socket says 172.21.15.49:44012) 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b1d9b700 1 -- 172.21.15.49:0/878325906 learned_addr learned my addr 172.21.15.49:0/878325906 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b259c700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30b4100ef0 0x7f30b40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b1d9b700 1 -- 172.21.15.49:0/878325906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30b4100ef0 msgr2=0x7f30b40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b1d9b700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30b4100ef0 0x7f30b40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b1d9b700 1 -- 172.21.15.49:0/878325906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0095f0 con 0x7f30b41018a0 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b1d9b700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 0x7f30b40737e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f309c00bd70 tx=0x7f309c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30ab7fe700 1 -- 172.21.15.49:0/878325906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f309c01d070 con 0x7f30b41018a0 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b40740e0 con 0x7f30b41018a0 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30ab7fe700 1 -- 172.21.15.49:0/878325906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f309c0056e0 con 0x7f30b41018a0 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30ab7fe700 1 -- 172.21.15.49:0/878325906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f309c00fe50 con 0x7f30b41018a0 2024-05-15T16:12:47.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.297+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b41a3030 con 0x7f30b41018a0 2024-05-15T16:12:47.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.298+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b41092f0 con 0x7f30b41018a0 2024-05-15T16:12:47.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.299+0000 7f30ab7fe700 1 -- 172.21.15.49:0/878325906 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f309c00f680 con 0x7f30b41018a0 2024-05-15T16:12:47.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.299+0000 7f30ab7fe700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30a006e200 0x7f30a00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:47.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.300+0000 7f30ab7fe700 1 -- 172.21.15.49:0/878325906 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f309c08f5f0 con 0x7f30b41018a0 2024-05-15T16:12:47.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.300+0000 7f30b259c700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30a006e200 0x7f30a00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:47.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.300+0000 7f30b259c700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30a006e200 0x7f30a00706c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30a4005950 tx=0x7f30a400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.304+0000 7f30ab7fe700 1 -- 172.21.15.49:0/878325906 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f309c0588d0 con 0x7f30b41018a0 2024-05-15T16:12:47.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:47 smithi049 ceph-mon[112142]: pgmap v472: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:47.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:47 smithi049 ceph-mon[112142]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:47.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:47 smithi049 ceph-mon[112142]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.603+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f30b41023d0 con 0x7f30b41018a0 2024-05-15T16:12:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.604+0000 7f30ab7fe700 1 -- 172.21.15.49:0/878325906 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f309c05bf90 con 0x7f30b41018a0 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:12:47.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:12:47.607 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:12:47.607 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:12:47.607 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:12:47.607 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:12:47.607 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30a006e200 msgr2=0x7f30a00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30a006e200 0x7f30a00706c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30a4005950 tx=0x7f30a400b410 comp rx=0 tx=0).stop 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 msgr2=0x7f30b40737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 0x7f30b40737e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f309c00bd70 tx=0x7f309c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 shutdown_connections 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30a006e200 0x7f30a00706c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30b4100ef0 0x7f30b40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 --2- 172.21.15.49:0/878325906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b41018a0 0x7f30b40737e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 >> 172.21.15.49:0/878325906 conn(0x7f30b40fc7c0 msgr2=0x7f30b40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 shutdown_connections 2024-05-15T16:12:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.606+0000 7f30b8fd7700 1 -- 172.21.15.49:0/878325906 wait complete. 2024-05-15T16:12:47.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:47 smithi144 ceph-mon[86635]: pgmap v472: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:47.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:47 smithi144 ceph-mon[86635]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:47.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:47 smithi144 ceph-mon[86635]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:47.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.714+0000 7fe8b6518700 1 -- 172.21.15.49:0/796194720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 msgr2=0x7fe8b01047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.714+0000 7fe8b6518700 1 --2- 172.21.15.49:0/796194720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 0x7fe8b01047f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009910 tx=0x7fe8a4009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.714+0000 7fe8b6518700 1 -- 172.21.15.49:0/796194720 shutdown_connections 2024-05-15T16:12:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.714+0000 7fe8b6518700 1 --2- 172.21.15.49:0/796194720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 0x7fe8b01047f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.714+0000 7fe8b6518700 1 --2- 172.21.15.49:0/796194720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b00ffdf0 0x7fe8b01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.714+0000 7fe8b6518700 1 -- 172.21.15.49:0/796194720 >> 172.21.15.49:0/796194720 conn(0x7fe8b00fb6a0 msgr2=0x7fe8b00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.715+0000 7fe8b6518700 1 -- 172.21.15.49:0/796194720 shutdown_connections 2024-05-15T16:12:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.715+0000 7fe8b6518700 1 -- 172.21.15.49:0/796194720 wait complete. 2024-05-15T16:12:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.715+0000 7fe8b6518700 1 Processor -- start 2024-05-15T16:12:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.715+0000 7fe8b6518700 1 -- start start 2024-05-15T16:12:47.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8b6518700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b00ffdf0 0x7fe8b0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8b6518700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 0x7fe8b0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8b6518700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b0198620 con 0x7fe8b01007a0 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8b6518700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b0198760 con 0x7fe8b00ffdf0 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8af7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 0x7fe8b0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8af7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 0x7fe8b0198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44022/0 (socket says 172.21.15.49:44022) 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8af7fe700 1 -- 172.21.15.49:0/2962520367 learned_addr learned my addr 172.21.15.49:0/2962520367 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8affff700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b00ffdf0 0x7fe8b0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8af7fe700 1 -- 172.21.15.49:0/2962520367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b00ffdf0 msgr2=0x7fe8b0197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8af7fe700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b00ffdf0 0x7fe8b0197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8af7fe700 1 -- 172.21.15.49:0/2962520367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40095f0 con 0x7fe8b01007a0 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.716+0000 7fe8af7fe700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 0x7fe8b0198050 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4005b40 tx=0x7fe8a400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.717+0000 7fe8ad7fa700 1 -- 172.21.15.49:0/2962520367 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8a401d070 con 0x7fe8b01007a0 2024-05-15T16:12:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.717+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b01a0b80 con 0x7fe8b01007a0 2024-05-15T16:12:47.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.717+0000 7fe8ad7fa700 1 -- 172.21.15.49:0/2962520367 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8a400f460 con 0x7fe8b01007a0 2024-05-15T16:12:47.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.717+0000 7fe8ad7fa700 1 -- 172.21.15.49:0/2962520367 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8a4005720 con 0x7fe8b01007a0 2024-05-15T16:12:47.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.717+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b01a0f30 con 0x7fe8b01007a0 2024-05-15T16:12:47.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.718+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b0108140 con 0x7fe8b01007a0 2024-05-15T16:12:47.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.718+0000 7fe8ad7fa700 1 -- 172.21.15.49:0/2962520367 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe8a400f5d0 con 0x7fe8b01007a0 2024-05-15T16:12:47.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.719+0000 7fe8ad7fa700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe89806e210 0x7fe8980706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:47.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.719+0000 7fe8ad7fa700 1 -- 172.21.15.49:0/2962520367 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe8a408f620 con 0x7fe8b01007a0 2024-05-15T16:12:47.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.719+0000 7fe8affff700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe89806e210 0x7fe8980706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:47.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.719+0000 7fe8affff700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe89806e210 0x7fe8980706d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009a10 tx=0x7fe8a0009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:47.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.723+0000 7fe8ad7fa700 1 -- 172.21.15.49:0/2962520367 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe8a4058900 con 0x7fe8b01007a0 2024-05-15T16:12:47.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.976+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe8b01012d0 con 0x7fe8b01007a0 2024-05-15T16:12:47.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.976+0000 7fe8ad7fa700 1 -- 172.21.15.49:0/2962520367 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7fe8a405bfc0 con 0x7fe8b01007a0 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:12:47.978 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:12:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:12:47.980 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:12:47.981 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:12:47.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe89806e210 msgr2=0x7fe8980706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe89806e210 0x7fe8980706d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009a10 tx=0x7fe8a0009190 comp rx=0 tx=0).stop 2024-05-15T16:12:47.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 msgr2=0x7fe8b0198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:47.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 0x7fe8b0198050 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4005b40 tx=0x7fe8a400bd70 comp rx=0 tx=0).stop 2024-05-15T16:12:47.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 shutdown_connections 2024-05-15T16:12:47.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe89806e210 0x7fe8980706d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b00ffdf0 0x7fe8b0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 --2- 172.21.15.49:0/2962520367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8b01007a0 0x7fe8b0198050 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:47.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 >> 172.21.15.49:0/2962520367 conn(0x7fe8b00fb6a0 msgr2=0x7fe8b00fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:47.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 shutdown_connections 2024-05-15T16:12:47.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:47.979+0000 7fe8b6518700 1 -- 172.21.15.49:0/2962520367 wait complete. 2024-05-15T16:12:47.982 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:12:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.089+0000 7fd590e94700 1 -- 172.21.15.49:0/3667932662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 msgr2=0x7fd58c102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.089+0000 7fd590e94700 1 --2- 172.21.15.49:0/3667932662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 0x7fd58c102200 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00b0a0 tx=0x7fd57c00b3b0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.089+0000 7fd590e94700 1 -- 172.21.15.49:0/3667932662 shutdown_connections 2024-05-15T16:12:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.089+0000 7fd590e94700 1 --2- 172.21.15.49:0/3667932662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102740 0x7fd58c10ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.089+0000 7fd590e94700 1 --2- 172.21.15.49:0/3667932662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 0x7fd58c102200 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.089+0000 7fd590e94700 1 -- 172.21.15.49:0/3667932662 >> 172.21.15.49:0/3667932662 conn(0x7fd58c076800 msgr2=0x7fd58c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.089+0000 7fd590e94700 1 -- 172.21.15.49:0/3667932662 shutdown_connections 2024-05-15T16:12:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.089+0000 7fd590e94700 1 -- 172.21.15.49:0/3667932662 wait complete. 2024-05-15T16:12:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.090+0000 7fd590e94700 1 Processor -- start 2024-05-15T16:12:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.090+0000 7fd590e94700 1 -- start start 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.090+0000 7fd590e94700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 0x7fd58c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.090+0000 7fd590e94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102740 0x7fd58c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.090+0000 7fd590e94700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c19d390 con 0x7fd58c101e20 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.090+0000 7fd590e94700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c196a00 con 0x7fd58c102740 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.090+0000 7fd58a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 0x7fd58c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.090+0000 7fd58a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 0x7fd58c19c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44044/0 (socket says 172.21.15.49:44044) 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.090+0000 7fd58a59c700 1 -- 172.21.15.49:0/3908061750 learned_addr learned my addr 172.21.15.49:0/3908061750 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd589d9b700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102740 0x7fd58c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd58a59c700 1 -- 172.21.15.49:0/3908061750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102740 msgr2=0x7fd58c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:48.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd58a59c700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102740 0x7fd58c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd58a59c700 1 -- 172.21.15.49:0/3908061750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c009d00 con 0x7fd58c101e20 2024-05-15T16:12:48.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd58a59c700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 0x7fd58c19c8c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd57c015040 tx=0x7fd57c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:48.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd5837fe700 1 -- 172.21.15.49:0/3908061750 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57c00e040 con 0x7fd58c101e20 2024-05-15T16:12:48.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd5837fe700 1 -- 172.21.15.49:0/3908061750 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd57c007cd0 con 0x7fd58c101e20 2024-05-15T16:12:48.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c196c80 con 0x7fd58c101e20 2024-05-15T16:12:48.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd5837fe700 1 -- 172.21.15.49:0/3908061750 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57c01d6b0 con 0x7fd58c101e20 2024-05-15T16:12:48.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.091+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c197170 con 0x7fd58c101e20 2024-05-15T16:12:48.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.092+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c105540 con 0x7fd58c101e20 2024-05-15T16:12:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.093+0000 7fd5837fe700 1 -- 172.21.15.49:0/3908061750 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd57c004040 con 0x7fd58c101e20 2024-05-15T16:12:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.093+0000 7fd5837fe700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd57806e200 0x7fd5780706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.093+0000 7fd5837fe700 1 -- 172.21.15.49:0/3908061750 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd57c08f920 con 0x7fd58c101e20 2024-05-15T16:12:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.093+0000 7fd589d9b700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd57806e200 0x7fd5780706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.094+0000 7fd589d9b700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd57806e200 0x7fd5780706c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd574005ea0 tx=0x7fd574005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:48.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.098+0000 7fd5837fe700 1 -- 172.21.15.49:0/3908061750 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd57c05c730 con 0x7fd58c101e20 2024-05-15T16:12:48.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.326+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd58c197ec0 con 0x7fd57806e200 2024-05-15T16:12:48.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.327+0000 7fd5837fe700 1 -- 172.21.15.49:0/3908061750 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd58c197ec0 con 0x7fd57806e200 2024-05-15T16:12:48.329 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:12:48.329 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:12:48.329 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:12:48.329 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:12:48.330 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:12:48.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.329+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd57806e200 msgr2=0x7fd5780706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:48.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd57806e200 0x7fd5780706c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd574005ea0 tx=0x7fd574005e30 comp rx=0 tx=0).stop 2024-05-15T16:12:48.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 msgr2=0x7fd58c19c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:48.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 0x7fd58c19c8c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd57c015040 tx=0x7fd57c003ce0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 shutdown_connections 2024-05-15T16:12:48.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd57806e200 0x7fd5780706c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c101e20 0x7fd58c19c8c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 --2- 172.21.15.49:0/3908061750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102740 0x7fd58c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 >> 172.21.15.49:0/3908061750 conn(0x7fd58c076800 msgr2=0x7fd58c103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:48.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 shutdown_connections 2024-05-15T16:12:48.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.330+0000 7fd590e94700 1 -- 172.21.15.49:0/3908061750 wait complete. 2024-05-15T16:12:48.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.438+0000 7efd33512700 1 -- 172.21.15.49:0/3398226716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c0732c0 msgr2=0x7efd2c0736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:48.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.438+0000 7efd33512700 1 --2- 172.21.15.49:0/3398226716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c0732c0 0x7efd2c0736a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efd20009910 tx=0x7efd20009c20 comp rx=0 tx=0).stop 2024-05-15T16:12:48.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.439+0000 7efd33512700 1 -- 172.21.15.49:0/3398226716 shutdown_connections 2024-05-15T16:12:48.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.439+0000 7efd33512700 1 --2- 172.21.15.49:0/3398226716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd2c073be0 0x7efd2c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.439+0000 7efd33512700 1 --2- 172.21.15.49:0/3398226716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c0732c0 0x7efd2c0736a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.439+0000 7efd33512700 1 -- 172.21.15.49:0/3398226716 >> 172.21.15.49:0/3398226716 conn(0x7efd2c0fc7d0 msgr2=0x7efd2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:48.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.439+0000 7efd33512700 1 -- 172.21.15.49:0/3398226716 shutdown_connections 2024-05-15T16:12:48.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.439+0000 7efd33512700 1 -- 172.21.15.49:0/3398226716 wait complete. 2024-05-15T16:12:48.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.439+0000 7efd33512700 1 Processor -- start 2024-05-15T16:12:48.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.439+0000 7efd33512700 1 -- start start 2024-05-15T16:12:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd33512700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd2c0732c0 0x7efd2c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd33512700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c073be0 0x7efd2c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:48.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd33512700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c199940 con 0x7efd2c073be0 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd33512700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c199a80 con 0x7efd2c0732c0 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd30aad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c073be0 0x7efd2c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd30aad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c073be0 0x7efd2c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44062/0 (socket says 172.21.15.49:44062) 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd30aad700 1 -- 172.21.15.49:0/690223268 learned_addr learned my addr 172.21.15.49:0/690223268 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd312ae700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd2c0732c0 0x7efd2c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd30aad700 1 -- 172.21.15.49:0/690223268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd2c0732c0 msgr2=0x7efd2c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd30aad700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd2c0732c0 0x7efd2c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd30aad700 1 -- 172.21.15.49:0/690223268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd200095f0 con 0x7efd2c073be0 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd312ae700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd2c0732c0 0x7efd2c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.440+0000 7efd30aad700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c073be0 0x7efd2c1992f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c930 tx=0x7efd2800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:48.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.441+0000 7efd1e7fc700 1 -- 172.21.15.49:0/690223268 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd28007ad0 con 0x7efd2c073be0 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.441+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2c19d810 con 0x7efd2c073be0 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.441+0000 7efd1e7fc700 1 -- 172.21.15.49:0/690223268 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd28007c30 con 0x7efd2c073be0 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.441+0000 7efd1e7fc700 1 -- 172.21.15.49:0/690223268 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd28018610 con 0x7efd2c073be0 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.441+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c19dd60 con 0x7efd2c073be0 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.442+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2c074a00 con 0x7efd2c073be0 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.442+0000 7efd1e7fc700 1 -- 172.21.15.49:0/690223268 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efd2800f450 con 0x7efd2c073be0 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.443+0000 7efd1e7fc700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd1806e200 0x7efd180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.443+0000 7efd1e7fc700 1 -- 172.21.15.49:0/690223268 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efd2808e360 con 0x7efd2c073be0 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.443+0000 7efd312ae700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd1806e200 0x7efd180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:12:48.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.443+0000 7efd312ae700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd1806e200 0x7efd180706c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efd20005f50 tx=0x7efd20005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:12:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.447+0000 7efd1e7fc700 1 -- 172.21.15.49:0/690223268 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efd28057640 con 0x7efd2c073be0 2024-05-15T16:12:48.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:48 smithi144 ceph-mon[86635]: from='client.34914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:48.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:48 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/878325906' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:12:48.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:48 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2962520367' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:12:48.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:48 smithi049 ceph-mon[112142]: from='client.34914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:48.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:48 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/878325906' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:12:48.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:48 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2962520367' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:12:48.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.740+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efd2c107880 con 0x7efd2c073be0 2024-05-15T16:12:48.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.740+0000 7efd1e7fc700 1 -- 172.21.15.49:0/690223268 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7efd2805ad00 con 0x7efd2c073be0 2024-05-15T16:12:48.742 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:12:48.742 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:12:48.742 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:12:48.742 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:12:48.742 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:12:48.742 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:12:48.742 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:12:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd1806e200 msgr2=0x7efd180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd1806e200 0x7efd180706c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efd20005f50 tx=0x7efd20005dc0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c073be0 msgr2=0x7efd2c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:12:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c073be0 0x7efd2c1992f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c930 tx=0x7efd2800ccf0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 shutdown_connections 2024-05-15T16:12:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd1806e200 0x7efd180706c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd2c0732c0 0x7efd2c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 --2- 172.21.15.49:0/690223268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2c073be0 0x7efd2c1992f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:12:48.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 >> 172.21.15.49:0/690223268 conn(0x7efd2c0fc7d0 msgr2=0x7efd2c105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:12:48.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 shutdown_connections 2024-05-15T16:12:48.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:12:48.742+0000 7efd33512700 1 -- 172.21.15.49:0/690223268 wait complete. 2024-05-15T16:12:49.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:49 smithi144 ceph-mon[86635]: pgmap v473: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:49.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:49 smithi144 ceph-mon[86635]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:49.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:49 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/690223268' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:12:49.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:49 smithi049 ceph-mon[112142]: pgmap v473: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:49.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:49 smithi049 ceph-mon[112142]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:12:49.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:49 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/690223268' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:12:50.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:12:50.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:12:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:51 smithi049 ceph-mon[112142]: pgmap v474: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:51.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:51 smithi144 ceph-mon[86635]: pgmap v474: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:53.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:53 smithi144 ceph-mon[86635]: pgmap v475: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:53.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:53 smithi049 ceph-mon[112142]: pgmap v475: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:55.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:55 smithi144 ceph-mon[86635]: pgmap v476: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:55.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:55 smithi049 ceph-mon[112142]: pgmap v476: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:57.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:57 smithi144 ceph-mon[86635]: pgmap v477: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:57.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:57 smithi049 ceph-mon[112142]: pgmap v477: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:59.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:12:59 smithi144 ceph-mon[86635]: pgmap v478: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:12:59.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:12:59 smithi049 ceph-mon[112142]: pgmap v478: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:01.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:01 smithi144 ceph-mon[86635]: pgmap v479: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:01.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:01 smithi049 ceph-mon[112142]: pgmap v479: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:03.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:03 smithi144 ceph-mon[86635]: pgmap v480: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:03.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:03 smithi049 ceph-mon[112142]: pgmap v480: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:05.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:05 smithi144 ceph-mon[86635]: pgmap v481: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:05.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:13:05.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:05 smithi049 ceph-mon[112142]: pgmap v481: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:05.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:13:07.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:07 smithi144 ceph-mon[86635]: pgmap v482: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:07.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:07 smithi049 ceph-mon[112142]: pgmap v482: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:09.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:09 smithi144 ceph-mon[86635]: pgmap v483: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:09.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:09 smithi049 ceph-mon[112142]: pgmap v483: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:11.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:11 smithi144 ceph-mon[86635]: pgmap v484: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:11.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:11 smithi049 ceph-mon[112142]: pgmap v484: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:13.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:13 smithi144 ceph-mon[86635]: pgmap v485: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:13.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:13 smithi049 ceph-mon[112142]: pgmap v485: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:15.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:15 smithi144 ceph-mon[86635]: pgmap v486: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:15.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:15 smithi049 ceph-mon[112142]: pgmap v486: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:17.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:17 smithi144 ceph-mon[86635]: pgmap v487: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:17.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:17 smithi049 ceph-mon[112142]: pgmap v487: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:18.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.866+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/2557308933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073340 msgr2=0x7fbe54073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.866+0000 7fbe5b6f5700 1 --2- 172.21.15.49:0/2557308933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073340 0x7fbe54073720 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009910 tx=0x7fbe50009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.866+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/2557308933 shutdown_connections 2024-05-15T16:13:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.866+0000 7fbe5b6f5700 1 --2- 172.21.15.49:0/2557308933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54073c60 0x7fbe54111750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.866+0000 7fbe5b6f5700 1 --2- 172.21.15.49:0/2557308933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073340 0x7fbe54073720 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.866+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/2557308933 >> 172.21.15.49:0/2557308933 conn(0x7fbe540fc790 msgr2=0x7fbe540febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.866+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/2557308933 shutdown_connections 2024-05-15T16:13:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.866+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/2557308933 wait complete. 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe5b6f5700 1 Processor -- start 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe5b6f5700 1 -- start start 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe5b6f5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54073340 0x7fbe5419d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe5b6f5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073c60 0x7fbe5419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe5b6f5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5419dd70 con 0x7fbe54073c60 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe5b6f5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe541a1a40 con 0x7fbe54073340 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe58c90700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073c60 0x7fbe5419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe58c90700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073c60 0x7fbe5419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38760/0 (socket says 172.21.15.49:38760) 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe58c90700 1 -- 172.21.15.49:0/3780625630 learned_addr learned my addr 172.21.15.49:0/3780625630 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe59491700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54073340 0x7fbe5419d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe58c90700 1 -- 172.21.15.49:0/3780625630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54073340 msgr2=0x7fbe5419d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe58c90700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54073340 0x7fbe5419d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.867+0000 7fbe58c90700 1 -- 172.21.15.49:0/3780625630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500095f0 con 0x7fbe54073c60 2024-05-15T16:13:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.868+0000 7fbe58c90700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073c60 0x7fbe5419d690 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400d8d0 tx=0x7fbe4400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.868+0000 7fbe4a7fc700 1 -- 172.21.15.49:0/3780625630 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe440098a0 con 0x7fbe54073c60 2024-05-15T16:13:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.868+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe541a1d20 con 0x7fbe54073c60 2024-05-15T16:13:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.868+0000 7fbe4a7fc700 1 -- 172.21.15.49:0/3780625630 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe44010460 con 0x7fbe54073c60 2024-05-15T16:13:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.868+0000 7fbe4a7fc700 1 -- 172.21.15.49:0/3780625630 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4400f5f0 con 0x7fbe54073c60 2024-05-15T16:13:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.868+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe541a2270 con 0x7fbe54073c60 2024-05-15T16:13:18.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.869+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe54074a80 con 0x7fbe54073c60 2024-05-15T16:13:18.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.869+0000 7fbe4a7fc700 1 -- 172.21.15.49:0/3780625630 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbe44009a00 con 0x7fbe54073c60 2024-05-15T16:13:18.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.870+0000 7fbe4a7fc700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe4006e200 0x7fbe400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:18.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.870+0000 7fbe59491700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe4006e200 0x7fbe400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:18.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.870+0000 7fbe4a7fc700 1 -- 172.21.15.49:0/3780625630 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbe4408df70 con 0x7fbe54073c60 2024-05-15T16:13:18.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.871+0000 7fbe59491700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe4006e200 0x7fbe400706c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe50000c00 tx=0x7fbe5001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:18.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:18.875+0000 7fbe4a7fc700 1 -- 172.21.15.49:0/3780625630 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbe440c2850 con 0x7fbe54073c60 2024-05-15T16:13:19.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.104+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe54114490 con 0x7fbe4006e200 2024-05-15T16:13:19.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.105+0000 7fbe4a7fc700 1 -- 172.21.15.49:0/3780625630 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbe54114490 con 0x7fbe4006e200 2024-05-15T16:13:19.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe4006e200 msgr2=0x7fbe400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe4006e200 0x7fbe400706c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe50000c00 tx=0x7fbe5001a040 comp rx=0 tx=0).stop 2024-05-15T16:13:19.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073c60 msgr2=0x7fbe5419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073c60 0x7fbe5419d690 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400d8d0 tx=0x7fbe4400dbe0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 shutdown_connections 2024-05-15T16:13:19.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe4006e200 0x7fbe400706c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54073340 0x7fbe5419d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 --2- 172.21.15.49:0/3780625630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe54073c60 0x7fbe5419d690 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 >> 172.21.15.49:0/3780625630 conn(0x7fbe540fc790 msgr2=0x7fbe541009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:19.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 shutdown_connections 2024-05-15T16:13:19.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.108+0000 7fbe5b6f5700 1 -- 172.21.15.49:0/3780625630 wait complete. 2024-05-15T16:13:19.120 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:13:19.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.217+0000 7f397e4f6700 1 -- 172.21.15.49:0/676512841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3978100040 msgr2=0x7f3978104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.217+0000 7f397e4f6700 1 --2- 172.21.15.49:0/676512841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3978100040 0x7f3978104880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3960009910 tx=0x7f3960009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:19.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.217+0000 7f397e4f6700 1 -- 172.21.15.49:0/676512841 shutdown_connections 2024-05-15T16:13:19.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.217+0000 7f397e4f6700 1 --2- 172.21.15.49:0/676512841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3978100040 0x7f3978104880 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 --2- 172.21.15.49:0/676512841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39780ff690 0x7f39780ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 -- 172.21.15.49:0/676512841 >> 172.21.15.49:0/676512841 conn(0x7f3978074eb0 msgr2=0x7f39780752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 -- 172.21.15.49:0/676512841 shutdown_connections 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 -- 172.21.15.49:0/676512841 wait complete. 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 Processor -- start 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 -- start start 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39780ff690 0x7f397819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3978100040 0x7f397819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397819d3a0 con 0x7f39780ff690 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.218+0000 7f397e4f6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3978196a10 con 0x7f3978100040 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3977fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39780ff690 0x7f397819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3977fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39780ff690 0x7f397819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38772/0 (socket says 172.21.15.49:38772) 2024-05-15T16:13:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3977fff700 1 -- 172.21.15.49:0/2942080727 learned_addr learned my addr 172.21.15.49:0/2942080727 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f396ffff700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3978100040 0x7f397819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3977fff700 1 -- 172.21.15.49:0/2942080727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3978100040 msgr2=0x7f397819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3977fff700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3978100040 0x7f397819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3977fff700 1 -- 172.21.15.49:0/2942080727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39600095f0 con 0x7f39780ff690 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f396ffff700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3978100040 0x7f397819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3977fff700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39780ff690 0x7f397819c8d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f396800c930 tx=0x7f396800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3975ffb700 1 -- 172.21.15.49:0/2942080727 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3968007ad0 con 0x7f39780ff690 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3975ffb700 1 -- 172.21.15.49:0/2942080727 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3968007c30 con 0x7f39780ff690 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3978196cf0 con 0x7f39780ff690 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f3975ffb700 1 -- 172.21.15.49:0/2942080727 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3968018610 con 0x7f39780ff690 2024-05-15T16:13:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.219+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3978197240 con 0x7f39780ff690 2024-05-15T16:13:19.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.220+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39781092b0 con 0x7f39780ff690 2024-05-15T16:13:19.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.221+0000 7f3975ffb700 1 -- 172.21.15.49:0/2942080727 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f396800f450 con 0x7f39780ff690 2024-05-15T16:13:19.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.221+0000 7f3975ffb700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f396406e200 0x7f39640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:19.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.221+0000 7f3975ffb700 1 -- 172.21.15.49:0/2942080727 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f396808e2a0 con 0x7f39780ff690 2024-05-15T16:13:19.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.221+0000 7f396ffff700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f396406e200 0x7f39640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:19.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.222+0000 7f396ffff700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f396406e200 0x7f39640706c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39600059d0 tx=0x7f396001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:19.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.226+0000 7f3975ffb700 1 -- 172.21.15.49:0/2942080727 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f396805b130 con 0x7f39780ff690 2024-05-15T16:13:19.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.460+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3978197f40 con 0x7f396406e200 2024-05-15T16:13:19.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.461+0000 7f3975ffb700 1 -- 172.21.15.49:0/2942080727 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3978197f40 con 0x7f396406e200 2024-05-15T16:13:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.463+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f396406e200 msgr2=0x7f39640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.463+0000 7f397e4f6700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f396406e200 0x7f39640706c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39600059d0 tx=0x7f396001a040 comp rx=0 tx=0).stop 2024-05-15T16:13:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.463+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39780ff690 msgr2=0x7f397819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.463+0000 7f397e4f6700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39780ff690 0x7f397819c8d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f396800c930 tx=0x7f396800ccf0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.463+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 shutdown_connections 2024-05-15T16:13:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.463+0000 7f397e4f6700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f396406e200 0x7f39640706c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.463+0000 7f397e4f6700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39780ff690 0x7f397819c8d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.463+0000 7f397e4f6700 1 --2- 172.21.15.49:0/2942080727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3978100040 0x7f397819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.463+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 >> 172.21.15.49:0/2942080727 conn(0x7f3978074eb0 msgr2=0x7f3978102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:19.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.464+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 shutdown_connections 2024-05-15T16:13:19.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.464+0000 7f397e4f6700 1 -- 172.21.15.49:0/2942080727 wait complete. 2024-05-15T16:13:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.572+0000 7f98f9fdb700 1 -- 172.21.15.49:0/2269775772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f41027d0 msgr2=0x7f98f410acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.572+0000 7f98f9fdb700 1 --2- 172.21.15.49:0/2269775772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f41027d0 0x7f98f410acc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009940 tx=0x7f98e8009c50 comp rx=0 tx=0).stop 2024-05-15T16:13:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.572+0000 7f98f9fdb700 1 -- 172.21.15.49:0/2269775772 shutdown_connections 2024-05-15T16:13:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.572+0000 7f98f9fdb700 1 --2- 172.21.15.49:0/2269775772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f41027d0 0x7f98f410acc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.572+0000 7f98f9fdb700 1 --2- 172.21.15.49:0/2269775772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f4101eb0 0x7f98f4102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.572+0000 7f98f9fdb700 1 -- 172.21.15.49:0/2269775772 >> 172.21.15.49:0/2269775772 conn(0x7f98f40fb6a0 msgr2=0x7f98f40fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.573+0000 7f98f9fdb700 1 -- 172.21.15.49:0/2269775772 shutdown_connections 2024-05-15T16:13:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.573+0000 7f98f9fdb700 1 -- 172.21.15.49:0/2269775772 wait complete. 2024-05-15T16:13:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.573+0000 7f98f9fdb700 1 Processor -- start 2024-05-15T16:13:19.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.573+0000 7f98f9fdb700 1 -- start start 2024-05-15T16:13:19.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f9fdb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f4101eb0 0x7f98f40ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:19.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f9fdb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f41027d0 0x7f98f4100050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f9fdb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4100590 con 0x7f98f4101eb0 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f9fdb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41006d0 con 0x7f98f41027d0 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f4101eb0 0x7f98f40ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f4101eb0 0x7f98f40ffb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38780/0 (socket says 172.21.15.49:38780) 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f37fe700 1 -- 172.21.15.49:0/4249673324 learned_addr learned my addr 172.21.15.49:0/4249673324 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f2ffd700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f41027d0 0x7f98f4100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f37fe700 1 -- 172.21.15.49:0/4249673324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f41027d0 msgr2=0x7f98f4100050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f37fe700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f41027d0 0x7f98f4100050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f37fe700 1 -- 172.21.15.49:0/4249673324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80095f0 con 0x7f98f4101eb0 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.574+0000 7f98f37fe700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f4101eb0 0x7f98f40ffb10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f98e400b4b0 tx=0x7f98e400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.575+0000 7f98f0ff9700 1 -- 172.21.15.49:0/4249673324 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e4011800 con 0x7f98f4101eb0 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.575+0000 7f98f0ff9700 1 -- 172.21.15.49:0/4249673324 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98e4011e20 con 0x7f98f4101eb0 2024-05-15T16:13:19.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.575+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f41a2eb0 con 0x7f98f4101eb0 2024-05-15T16:13:19.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.575+0000 7f98f0ff9700 1 -- 172.21.15.49:0/4249673324 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e400f2e0 con 0x7f98f4101eb0 2024-05-15T16:13:19.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.575+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f41a3210 con 0x7f98f4101eb0 2024-05-15T16:13:19.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.576+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f4069a50 con 0x7f98f4101eb0 2024-05-15T16:13:19.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.576+0000 7f98f0ff9700 1 -- 172.21.15.49:0/4249673324 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f98e4010380 con 0x7f98f4101eb0 2024-05-15T16:13:19.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.577+0000 7f98f0ff9700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f98dc06e200 0x7f98dc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:19.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.577+0000 7f98f0ff9700 1 -- 172.21.15.49:0/4249673324 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f98e408dc60 con 0x7f98f4101eb0 2024-05-15T16:13:19.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.577+0000 7f98f2ffd700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f98dc06e200 0x7f98dc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:19.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.577+0000 7f98f2ffd700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f98dc06e200 0x7f98dc0706c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f98e8006010 tx=0x7f98e800b790 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:19.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.581+0000 7f98f0ff9700 1 -- 172.21.15.49:0/4249673324 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f98e4056f40 con 0x7f98f4101eb0 2024-05-15T16:13:19.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:19 smithi144 ceph-mon[86635]: pgmap v488: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:19.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:19 smithi049 ceph-mon[112142]: pgmap v488: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.801+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f98f410df30 con 0x7f98dc06e200 2024-05-15T16:13:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.812+0000 7f98f0ff9700 1 -- 172.21.15.49:0/4249673324 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f98f410df30 con 0x7f98dc06e200 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (16m) 9m ago 28m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (28m) 9m ago 28m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (27m) 9m ago 27m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (13m) 9m ago 28m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (13m) 9m ago 27m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (16m) 9m ago 27m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (10m) 9m ago 22m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (9m) 9m ago 22m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (9m) 9m ago 22m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:13:19.814 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (22m) 9m ago 22m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (18m) 9m ago 29m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (17m) 9m ago 27m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (15m) 9m ago 29m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (14m) 9m ago 26m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (17m) 9m ago 28m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (17m) 9m ago 27m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (13m) 9m ago 26m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (12m) 9m ago 25m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (12m) 9m ago 25m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (11m) 9m ago 24m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (11m) 9m ago 24m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (10m) 9m ago 24m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:13:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (16m) 9m ago 27m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:13:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f98dc06e200 msgr2=0x7f98dc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f98dc06e200 0x7f98dc0706c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f98e8006010 tx=0x7f98e800b790 comp rx=0 tx=0).stop 2024-05-15T16:13:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f4101eb0 msgr2=0x7f98f40ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f4101eb0 0x7f98f40ffb10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f98e400b4b0 tx=0x7f98e400b7c0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 shutdown_connections 2024-05-15T16:13:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f98dc06e200 0x7f98dc0706c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98f4101eb0 0x7f98f40ffb10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 --2- 172.21.15.49:0/4249673324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f41027d0 0x7f98f4100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 >> 172.21.15.49:0/4249673324 conn(0x7f98f40fb6a0 msgr2=0x7f98f41031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 shutdown_connections 2024-05-15T16:13:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.814+0000 7f98f9fdb700 1 -- 172.21.15.49:0/4249673324 wait complete. 2024-05-15T16:13:19.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.921+0000 7f1c8b654700 1 -- 172.21.15.49:0/67238065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84102790 msgr2=0x7f1c8410ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.921+0000 7f1c8b654700 1 --2- 172.21.15.49:0/67238065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84102790 0x7f1c8410ac80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009910 tx=0x7f1c74009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:19.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.921+0000 7f1c8b654700 1 -- 172.21.15.49:0/67238065 shutdown_connections 2024-05-15T16:13:19.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.921+0000 7f1c8b654700 1 --2- 172.21.15.49:0/67238065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84102790 0x7f1c8410ac80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.921+0000 7f1c8b654700 1 --2- 172.21.15.49:0/67238065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84101e70 0x7f1c84102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.921+0000 7f1c8b654700 1 -- 172.21.15.49:0/67238065 >> 172.21.15.49:0/67238065 conn(0x7f1c840fb6a0 msgr2=0x7f1c840fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:19.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.922+0000 7f1c8b654700 1 -- 172.21.15.49:0/67238065 shutdown_connections 2024-05-15T16:13:19.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.922+0000 7f1c8b654700 1 -- 172.21.15.49:0/67238065 wait complete. 2024-05-15T16:13:19.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.922+0000 7f1c8b654700 1 Processor -- start 2024-05-15T16:13:19.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.922+0000 7f1c8b654700 1 -- start start 2024-05-15T16:13:19.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.922+0000 7f1c8b654700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84101e70 0x7f1c84196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:19.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c8b654700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84102790 0x7f1c84197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:19.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c8b654700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c841978e0 con 0x7f1c84101e70 2024-05-15T16:13:19.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c8b654700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8419a550 con 0x7f1c84102790 2024-05-15T16:13:19.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c893f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84101e70 0x7f1c84196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:19.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c893f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84101e70 0x7f1c84196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38804/0 (socket says 172.21.15.49:38804) 2024-05-15T16:13:19.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c893f0700 1 -- 172.21.15.49:0/2126156941 learned_addr learned my addr 172.21.15.49:0/2126156941 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:19.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c88bef700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84102790 0x7f1c84197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:19.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c893f0700 1 -- 172.21.15.49:0/2126156941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84102790 msgr2=0x7f1c84197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:19.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c893f0700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84102790 0x7f1c84197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c893f0700 1 -- 172.21.15.49:0/2126156941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740095f0 con 0x7f1c84101e70 2024-05-15T16:13:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c88bef700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84102790 0x7f1c84197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:13:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c893f0700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84101e70 0x7f1c84196d10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000b4b0 tx=0x7f1c8000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c7a7fc700 1 -- 172.21.15.49:0/2126156941 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c80011800 con 0x7f1c84101e70 2024-05-15T16:13:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c7a7fc700 1 -- 172.21.15.49:0/2126156941 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c80011e20 con 0x7f1c84101e70 2024-05-15T16:13:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8419a830 con 0x7f1c84101e70 2024-05-15T16:13:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c7a7fc700 1 -- 172.21.15.49:0/2126156941 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c8000f2e0 con 0x7f1c84101e70 2024-05-15T16:13:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.923+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8419ad80 con 0x7f1c84101e70 2024-05-15T16:13:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.924+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c840697b0 con 0x7f1c84101e70 2024-05-15T16:13:19.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.925+0000 7f1c7a7fc700 1 -- 172.21.15.49:0/2126156941 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1c80010380 con 0x7f1c84101e70 2024-05-15T16:13:19.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.925+0000 7f1c7a7fc700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1c7006e200 0x7f1c700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:19.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.925+0000 7f1c7a7fc700 1 -- 172.21.15.49:0/2126156941 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1c8008dd50 con 0x7f1c84101e70 2024-05-15T16:13:19.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.926+0000 7f1c88bef700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1c7006e200 0x7f1c700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:19.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.926+0000 7f1c88bef700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1c7006e200 0x7f1c700706c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1c84198230 tx=0x7f1c74005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:19.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:19.930+0000 7f1c7a7fc700 1 -- 172.21.15.49:0/2126156941 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1c80057030 con 0x7f1c84101e70 2024-05-15T16:13:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.227+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1c8404ec60 con 0x7f1c84101e70 2024-05-15T16:13:20.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.228+0000 7f1c7a7fc700 1 -- 172.21.15.49:0/2126156941 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f1c8005a6f0 con 0x7f1c84101e70 2024-05-15T16:13:20.229 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:13:20.230 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:13:20.231 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:13:20.231 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:13:20.231 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:13:20.231 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1c7006e200 msgr2=0x7f1c700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1c7006e200 0x7f1c700706c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1c84198230 tx=0x7f1c74005a90 comp rx=0 tx=0).stop 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84101e70 msgr2=0x7f1c84196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84101e70 0x7f1c84196d10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000b4b0 tx=0x7f1c8000b870 comp rx=0 tx=0).stop 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 shutdown_connections 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1c7006e200 0x7f1c700706c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c84101e70 0x7f1c84196d10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 --2- 172.21.15.49:0/2126156941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84102790 0x7f1c84197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 >> 172.21.15.49:0/2126156941 conn(0x7f1c840fb6a0 msgr2=0x7f1c84103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 shutdown_connections 2024-05-15T16:13:20.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.230+0000 7f1c8b654700 1 -- 172.21.15.49:0/2126156941 wait complete. 2024-05-15T16:13:20.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.340+0000 7f60e8972700 1 -- 172.21.15.49:0/3324877286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e41027e0 msgr2=0x7f60e410acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.340+0000 7f60e8972700 1 --2- 172.21.15.49:0/3324877286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e41027e0 0x7f60e410acd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009910 tx=0x7f60cc009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:20.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.340+0000 7f60e8972700 1 -- 172.21.15.49:0/3324877286 shutdown_connections 2024-05-15T16:13:20.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.340+0000 7f60e8972700 1 --2- 172.21.15.49:0/3324877286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e41027e0 0x7f60e410acd0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.340+0000 7f60e8972700 1 --2- 172.21.15.49:0/3324877286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60e4101ec0 0x7f60e41022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.340+0000 7f60e8972700 1 -- 172.21.15.49:0/3324877286 >> 172.21.15.49:0/3324877286 conn(0x7f60e4076800 msgr2=0x7f60e40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.340+0000 7f60e8972700 1 -- 172.21.15.49:0/3324877286 shutdown_connections 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.340+0000 7f60e8972700 1 -- 172.21.15.49:0/3324877286 wait complete. 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.341+0000 7f60e8972700 1 Processor -- start 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.341+0000 7f60e8972700 1 -- start start 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.341+0000 7f60e8972700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e4101ec0 0x7f60e40ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.341+0000 7f60e8972700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60e41027e0 0x7f60e4100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.341+0000 7f60e8972700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41005a0 con 0x7f60e4101ec0 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.341+0000 7f60e8972700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41006e0 con 0x7f60e41027e0 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.341+0000 7f60e259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e4101ec0 0x7f60e40ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.341+0000 7f60e259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e4101ec0 0x7f60e40ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38830/0 (socket says 172.21.15.49:38830) 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.341+0000 7f60e259c700 1 -- 172.21.15.49:0/1316188030 learned_addr learned my addr 172.21.15.49:0/1316188030 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60e1d9b700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60e41027e0 0x7f60e4100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60e259c700 1 -- 172.21.15.49:0/1316188030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60e41027e0 msgr2=0x7f60e4100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60e259c700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60e41027e0 0x7f60e4100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60e259c700 1 -- 172.21.15.49:0/1316188030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc0095f0 con 0x7f60e4101ec0 2024-05-15T16:13:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60e259c700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e4101ec0 0x7f60e40ffb20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f60d400ba50 tx=0x7f60d400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60db7fe700 1 -- 172.21.15.49:0/1316188030 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60d400ca00 con 0x7f60e4101ec0 2024-05-15T16:13:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60db7fe700 1 -- 172.21.15.49:0/1316188030 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60d400cb60 con 0x7f60e4101ec0 2024-05-15T16:13:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e41a2ea0 con 0x7f60e4101ec0 2024-05-15T16:13:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60db7fe700 1 -- 172.21.15.49:0/1316188030 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60d40075a0 con 0x7f60e4101ec0 2024-05-15T16:13:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.342+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e41a3200 con 0x7f60e4101ec0 2024-05-15T16:13:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.343+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e4068ec0 con 0x7f60e4101ec0 2024-05-15T16:13:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.344+0000 7f60db7fe700 1 -- 172.21.15.49:0/1316188030 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f60d400ccd0 con 0x7f60e4101ec0 2024-05-15T16:13:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.344+0000 7f60db7fe700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f60d006e200 0x7f60d00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.345+0000 7f60e1d9b700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f60d006e200 0x7f60d00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.345+0000 7f60db7fe700 1 -- 172.21.15.49:0/1316188030 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f60d4006e40 con 0x7f60e4101ec0 2024-05-15T16:13:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.345+0000 7f60e1d9b700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f60d006e200 0x7f60d00706c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f60e4101250 tx=0x7f60cc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.349+0000 7f60db7fe700 1 -- 172.21.15.49:0/1316188030 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f60d4056c90 con 0x7f60e4101ec0 2024-05-15T16:13:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:20 smithi144 ceph-mon[86635]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:20 smithi144 ceph-mon[86635]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:20 smithi144 ceph-mon[86635]: from='client.34942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:13:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:20 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2126156941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:13:20.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.603+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f60e41056a0 con 0x7f60e4101ec0 2024-05-15T16:13:20.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:20 smithi049 ceph-mon[112142]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:20.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:20 smithi049 ceph-mon[112142]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:20.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:20 smithi049 ceph-mon[112142]: from='client.34942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:20.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:13:20.605 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:20 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2126156941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:13:20.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.604+0000 7f60db7fe700 1 -- 172.21.15.49:0/1316188030 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f60d408be60 con 0x7f60e4101ec0 2024-05-15T16:13:20.606 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:13:20.606 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:13:20.606 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:13:20.606 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:13:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:13:20.606 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:13:20.606 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:13:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:13:20.608 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:13:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:13:20.609 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:13:20.609 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:13:20.609 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:13:20.609 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:13:20.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f60d006e200 msgr2=0x7f60d00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f60d006e200 0x7f60d00706c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f60e4101250 tx=0x7f60cc01a040 comp rx=0 tx=0).stop 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e4101ec0 msgr2=0x7f60e40ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e4101ec0 0x7f60e40ffb20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f60d400ba50 tx=0x7f60d400be10 comp rx=0 tx=0).stop 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 shutdown_connections 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f60d006e200 0x7f60d00706c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60e4101ec0 0x7f60e40ffb20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 --2- 172.21.15.49:0/1316188030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60e41027e0 0x7f60e4100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 >> 172.21.15.49:0/1316188030 conn(0x7f60e4076800 msgr2=0x7f60e41031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 shutdown_connections 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.606+0000 7f60e8972700 1 -- 172.21.15.49:0/1316188030 wait complete. 2024-05-15T16:13:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:13:20.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.716+0000 7fca4e763700 1 -- 172.21.15.49:0/193040638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca481007a0 msgr2=0x7fca481047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.716+0000 7fca4e763700 1 --2- 172.21.15.49:0/193040638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca481007a0 0x7fca481047f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fca30009910 tx=0x7fca30009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:20.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.716+0000 7fca4e763700 1 -- 172.21.15.49:0/193040638 shutdown_connections 2024-05-15T16:13:20.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.716+0000 7fca4e763700 1 --2- 172.21.15.49:0/193040638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca481007a0 0x7fca481047f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.716+0000 7fca4e763700 1 --2- 172.21.15.49:0/193040638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca480ffdf0 0x7fca481001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.716+0000 7fca4e763700 1 -- 172.21.15.49:0/193040638 >> 172.21.15.49:0/193040638 conn(0x7fca480fb6a0 msgr2=0x7fca480fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:20.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.716+0000 7fca4e763700 1 -- 172.21.15.49:0/193040638 shutdown_connections 2024-05-15T16:13:20.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.716+0000 7fca4e763700 1 -- 172.21.15.49:0/193040638 wait complete. 2024-05-15T16:13:20.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca4e763700 1 Processor -- start 2024-05-15T16:13:20.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca4e763700 1 -- start start 2024-05-15T16:13:20.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca4e763700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca480ffdf0 0x7fca48197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca4e763700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca481007a0 0x7fca481980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca4e763700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca481986c0 con 0x7fca480ffdf0 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca4e763700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca48198800 con 0x7fca481007a0 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca47fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca480ffdf0 0x7fca48197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca47fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca480ffdf0 0x7fca48197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38848/0 (socket says 172.21.15.49:38848) 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca47fff700 1 -- 172.21.15.49:0/1968537179 learned_addr learned my addr 172.21.15.49:0/1968537179 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.717+0000 7fca477fe700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca481007a0 0x7fca481980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.718+0000 7fca47fff700 1 -- 172.21.15.49:0/1968537179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca481007a0 msgr2=0x7fca481980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.718+0000 7fca47fff700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca481007a0 0x7fca481980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.718+0000 7fca47fff700 1 -- 172.21.15.49:0/1968537179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300095f0 con 0x7fca480ffdf0 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.718+0000 7fca47fff700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca480ffdf0 0x7fca48197bb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d8d0 tx=0x7fca3800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.718+0000 7fca457fa700 1 -- 172.21.15.49:0/1968537179 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca380098a0 con 0x7fca480ffdf0 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.718+0000 7fca457fa700 1 -- 172.21.15.49:0/1968537179 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca38010460 con 0x7fca480ffdf0 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.718+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca481a0c20 con 0x7fca480ffdf0 2024-05-15T16:13:20.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.718+0000 7fca457fa700 1 -- 172.21.15.49:0/1968537179 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3800f5f0 con 0x7fca480ffdf0 2024-05-15T16:13:20.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.718+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca481a1090 con 0x7fca480ffdf0 2024-05-15T16:13:20.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.719+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca481081f0 con 0x7fca480ffdf0 2024-05-15T16:13:20.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.720+0000 7fca457fa700 1 -- 172.21.15.49:0/1968537179 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fca38009a00 con 0x7fca480ffdf0 2024-05-15T16:13:20.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.720+0000 7fca457fa700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca3406e200 0x7fca340706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:20.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.720+0000 7fca457fa700 1 -- 172.21.15.49:0/1968537179 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fca3808e070 con 0x7fca480ffdf0 2024-05-15T16:13:20.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.720+0000 7fca477fe700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca3406e200 0x7fca340706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:20.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.721+0000 7fca477fe700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca3406e200 0x7fca340706c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fca48069d40 tx=0x7fca3001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:20.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.725+0000 7fca457fa700 1 -- 172.21.15.49:0/1968537179 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fca38057350 con 0x7fca480ffdf0 2024-05-15T16:13:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.956+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca48069770 con 0x7fca3406e200 2024-05-15T16:13:20.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.958+0000 7fca457fa700 1 -- 172.21.15.49:0/1968537179 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fca48069770 con 0x7fca3406e200 2024-05-15T16:13:20.959 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:13:20.959 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:13:20.959 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:13:20.959 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:13:20.959 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:13:20.959 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:13:20.959 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:13:20.960 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:13:20.960 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:13:20.960 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:13:20.960 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:13:20.960 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:13:20.960 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:13:20.960 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:13:20.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca3406e200 msgr2=0x7fca340706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca3406e200 0x7fca340706c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fca48069d40 tx=0x7fca3001a040 comp rx=0 tx=0).stop 2024-05-15T16:13:20.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca480ffdf0 msgr2=0x7fca48197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca480ffdf0 0x7fca48197bb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d8d0 tx=0x7fca3800dbe0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 shutdown_connections 2024-05-15T16:13:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca3406e200 0x7fca340706c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca480ffdf0 0x7fca48197bb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 --2- 172.21.15.49:0/1968537179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca481007a0 0x7fca481980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 >> 172.21.15.49:0/1968537179 conn(0x7fca480fb6a0 msgr2=0x7fca480fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 shutdown_connections 2024-05-15T16:13:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:20.960+0000 7fca4e763700 1 -- 172.21.15.49:0/1968537179 wait complete. 2024-05-15T16:13:21.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.068+0000 7ff31d287700 1 -- 172.21.15.49:0/2019669181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 msgr2=0x7ff3181012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:21.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.068+0000 7ff31d287700 1 --2- 172.21.15.49:0/2019669181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 0x7ff3181012e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff308009910 tx=0x7ff308009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:21.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.068+0000 7ff31d287700 1 -- 172.21.15.49:0/2019669181 shutdown_connections 2024-05-15T16:13:21.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.068+0000 7ff31d287700 1 --2- 172.21.15.49:0/2019669181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3181018b0 0x7ff318105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:21.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.068+0000 7ff31d287700 1 --2- 172.21.15.49:0/2019669181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 0x7ff3181012e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:21.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.068+0000 7ff31d287700 1 -- 172.21.15.49:0/2019669181 >> 172.21.15.49:0/2019669181 conn(0x7ff3180fc7d0 msgr2=0x7ff3180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:21.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.068+0000 7ff31d287700 1 -- 172.21.15.49:0/2019669181 shutdown_connections 2024-05-15T16:13:21.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.068+0000 7ff31d287700 1 -- 172.21.15.49:0/2019669181 wait complete. 2024-05-15T16:13:21.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.069+0000 7ff31d287700 1 Processor -- start 2024-05-15T16:13:21.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.069+0000 7ff31d287700 1 -- start start 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.069+0000 7ff31d287700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 0x7ff31819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.069+0000 7ff31d287700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3181018b0 0x7ff31819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.069+0000 7ff31d287700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31819d460 con 0x7ff318100f00 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.069+0000 7ff31d287700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff318196a90 con 0x7ff3181018b0 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff316ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 0x7ff31819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff316ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 0x7ff31819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38870/0 (socket says 172.21.15.49:38870) 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff316ffd700 1 -- 172.21.15.49:0/1462072710 learned_addr learned my addr 172.21.15.49:0/1462072710 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff3167fc700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3181018b0 0x7ff31819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff316ffd700 1 -- 172.21.15.49:0/1462072710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3181018b0 msgr2=0x7ff31819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff316ffd700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3181018b0 0x7ff31819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:21.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff316ffd700 1 -- 172.21.15.49:0/1462072710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080095f0 con 0x7ff318100f00 2024-05-15T16:13:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff3167fc700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3181018b0 0x7ff31819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:13:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff316ffd700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 0x7ff31819c900 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff30800bd70 tx=0x7ff30800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff30ffff700 1 -- 172.21.15.49:0/1462072710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff30801d070 con 0x7ff318100f00 2024-05-15T16:13:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff30ffff700 1 -- 172.21.15.49:0/1462072710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3080056e0 con 0x7ff318100f00 2024-05-15T16:13:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff318196d10 con 0x7ff318100f00 2024-05-15T16:13:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff30ffff700 1 -- 172.21.15.49:0/1462072710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff308017400 con 0x7ff318100f00 2024-05-15T16:13:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.070+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff318197200 con 0x7ff318100f00 2024-05-15T16:13:21.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.071+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3181092a0 con 0x7ff318100f00 2024-05-15T16:13:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.072+0000 7ff30ffff700 1 -- 172.21.15.49:0/1462072710 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff308003a60 con 0x7ff318100f00 2024-05-15T16:13:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.072+0000 7ff30ffff700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff30406e200 0x7ff3040706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.073+0000 7ff30ffff700 1 -- 172.21.15.49:0/1462072710 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff30808f620 con 0x7ff318100f00 2024-05-15T16:13:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.073+0000 7ff3167fc700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff30406e200 0x7ff3040706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:21.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.073+0000 7ff3167fc700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff30406e200 0x7ff3040706c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff300005950 tx=0x7ff3000058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:21.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.078+0000 7ff30ffff700 1 -- 172.21.15.49:0/1462072710 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff30805c430 con 0x7ff318100f00 2024-05-15T16:13:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.374+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff3181023e0 con 0x7ff318100f00 2024-05-15T16:13:21.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.374+0000 7ff30ffff700 1 -- 172.21.15.49:0/1462072710 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7ff30805bfc0 con 0x7ff318100f00 2024-05-15T16:13:21.376 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:13:21.376 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:13:21.376 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:13:21.376 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:13:21.376 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:13:21.376 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:13:21.376 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:13:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff30406e200 msgr2=0x7ff3040706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff30406e200 0x7ff3040706c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff300005950 tx=0x7ff3000058e0 comp rx=0 tx=0).stop 2024-05-15T16:13:21.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 msgr2=0x7ff31819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 0x7ff31819c900 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff30800bd70 tx=0x7ff30800be50 comp rx=0 tx=0).stop 2024-05-15T16:13:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 shutdown_connections 2024-05-15T16:13:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff30406e200 0x7ff3040706c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff318100f00 0x7ff31819c900 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 --2- 172.21.15.49:0/1462072710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3181018b0 0x7ff31819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 >> 172.21.15.49:0/1462072710 conn(0x7ff3180fc7d0 msgr2=0x7ff3180fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 shutdown_connections 2024-05-15T16:13:21.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:21.377+0000 7ff31d287700 1 -- 172.21.15.49:0/1462072710 wait complete. 2024-05-15T16:13:21.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:21 smithi144 ceph-mon[86635]: pgmap v489: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:21.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:21 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1316188030' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:13:21.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:21 smithi144 ceph-mon[86635]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:21.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:21 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1462072710' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:13:21.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:21 smithi049 ceph-mon[112142]: pgmap v489: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:21.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:21 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1316188030' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:13:21.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:21 smithi049 ceph-mon[112142]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:21.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:21 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1462072710' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:13:23.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:23 smithi049 ceph-mon[112142]: pgmap v490: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:23.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:23 smithi144 ceph-mon[86635]: pgmap v490: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:25.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:25 smithi144 ceph-mon[86635]: pgmap v491: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:25.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:25 smithi049 ceph-mon[112142]: pgmap v491: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:27.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:27 smithi144 ceph-mon[86635]: pgmap v492: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:27 smithi049 ceph-mon[112142]: pgmap v492: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:29.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:29 smithi049 ceph-mon[112142]: pgmap v493: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:29.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:29 smithi144 ceph-mon[86635]: pgmap v493: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:31.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:31 smithi144 ceph-mon[86635]: pgmap v494: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:31.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:31 smithi049 ceph-mon[112142]: pgmap v494: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:33.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:33 smithi144 ceph-mon[86635]: pgmap v495: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:33.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:33 smithi049 ceph-mon[112142]: pgmap v495: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:35.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:35 smithi144 ceph-mon[86635]: pgmap v496: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:35.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:13:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:35 smithi049 ceph-mon[112142]: pgmap v496: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:13:37.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:37 smithi144 ceph-mon[86635]: pgmap v497: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:37.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:37 smithi049 ceph-mon[112142]: pgmap v497: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:39.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:39 smithi049 ceph-mon[112142]: pgmap v498: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:39.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:39 smithi144 ceph-mon[86635]: pgmap v498: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:41.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:41 smithi144 ceph-mon[86635]: pgmap v499: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:41.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:41 smithi049 ceph-mon[112142]: pgmap v499: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:43.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:43 smithi144 ceph-mon[86635]: pgmap v500: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:43.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:43 smithi049 ceph-mon[112142]: pgmap v500: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:45.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:45 smithi144 ceph-mon[86635]: pgmap v501: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:45 smithi049 ceph-mon[112142]: pgmap v501: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:47.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:47 smithi144 ceph-mon[86635]: pgmap v502: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:47.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:47 smithi049 ceph-mon[112142]: pgmap v502: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:49.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:49 smithi144 ceph-mon[86635]: pgmap v503: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:49.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:49 smithi049 ceph-mon[112142]: pgmap v503: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:13:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:13:51.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.502+0000 7f1483acc700 1 -- 172.21.15.49:0/3372865895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c0733d0 msgr2=0x7f147c0737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:51.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.502+0000 7f1483acc700 1 --2- 172.21.15.49:0/3372865895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c0733d0 0x7f147c0737b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1478009910 tx=0x7f1478009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:51.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.502+0000 7f1483acc700 1 -- 172.21.15.49:0/3372865895 shutdown_connections 2024-05-15T16:13:51.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.502+0000 7f1483acc700 1 --2- 172.21.15.49:0/3372865895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f147c073cf0 0x7f147c10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:51.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.502+0000 7f1483acc700 1 --2- 172.21.15.49:0/3372865895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c0733d0 0x7f147c0737b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:51.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.502+0000 7f1483acc700 1 -- 172.21.15.49:0/3372865895 >> 172.21.15.49:0/3372865895 conn(0x7f147c0fc7d0 msgr2=0x7f147c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:51.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.502+0000 7f1483acc700 1 -- 172.21.15.49:0/3372865895 shutdown_connections 2024-05-15T16:13:51.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.502+0000 7f1483acc700 1 -- 172.21.15.49:0/3372865895 wait complete. 2024-05-15T16:13:51.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1483acc700 1 Processor -- start 2024-05-15T16:13:51.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1483acc700 1 -- start start 2024-05-15T16:13:51.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1483acc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f147c0733d0 0x7f147c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:51.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1483acc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c073cf0 0x7f147c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:51.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1483acc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c1999d0 con 0x7f147c073cf0 2024-05-15T16:13:51.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1483acc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c19d6a0 con 0x7f147c0733d0 2024-05-15T16:13:51.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1481067700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c073cf0 0x7f147c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:51.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1481067700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c073cf0 0x7f147c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51158/0 (socket says 172.21.15.49:51158) 2024-05-15T16:13:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1481067700 1 -- 172.21.15.49:0/3722962704 learned_addr learned my addr 172.21.15.49:0/3722962704 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.503+0000 7f1481868700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f147c0733d0 0x7f147c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1481067700 1 -- 172.21.15.49:0/3722962704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f147c0733d0 msgr2=0x7f147c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1481067700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f147c0733d0 0x7f147c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1481067700 1 -- 172.21.15.49:0/3722962704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780095f0 con 0x7f147c073cf0 2024-05-15T16:13:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1481868700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f147c0733d0 0x7f147c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:13:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1481067700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c073cf0 0x7f147c1992f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f146c00d8d0 tx=0x7f146c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:51.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1472ffd700 1 -- 172.21.15.49:0/3722962704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f146c009940 con 0x7f147c073cf0 2024-05-15T16:13:51.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1472ffd700 1 -- 172.21.15.49:0/3722962704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f146c010460 con 0x7f147c073cf0 2024-05-15T16:13:51.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147c19d980 con 0x7f147c073cf0 2024-05-15T16:13:51.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1472ffd700 1 -- 172.21.15.49:0/3722962704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f146c00f5f0 con 0x7f147c073cf0 2024-05-15T16:13:51.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.504+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147c19ded0 con 0x7f147c073cf0 2024-05-15T16:13:51.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.505+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147c074a80 con 0x7f147c073cf0 2024-05-15T16:13:51.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.506+0000 7f1472ffd700 1 -- 172.21.15.49:0/3722962704 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f146c0105d0 con 0x7f147c073cf0 2024-05-15T16:13:51.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.506+0000 7f1472ffd700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f146806e200 0x7f14680706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:51.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.506+0000 7f1472ffd700 1 -- 172.21.15.49:0/3722962704 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f146c08e1a0 con 0x7f147c073cf0 2024-05-15T16:13:51.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.506+0000 7f1481868700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f146806e200 0x7f14680706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:51.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.507+0000 7f1481868700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f146806e200 0x7f14680706c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14780059d0 tx=0x7f147801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:51.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.511+0000 7f1472ffd700 1 -- 172.21.15.49:0/3722962704 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f146c057480 con 0x7f147c073cf0 2024-05-15T16:13:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.738+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f147c107840 con 0x7f146806e200 2024-05-15T16:13:51.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.739+0000 7f1472ffd700 1 -- 172.21.15.49:0/3722962704 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f147c107840 con 0x7f146806e200 2024-05-15T16:13:51.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.741+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f146806e200 msgr2=0x7f14680706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:51.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.741+0000 7f1483acc700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f146806e200 0x7f14680706c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14780059d0 tx=0x7f147801a040 comp rx=0 tx=0).stop 2024-05-15T16:13:51.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.741+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c073cf0 msgr2=0x7f147c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:51.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.741+0000 7f1483acc700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c073cf0 0x7f147c1992f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f146c00d8d0 tx=0x7f146c00dc90 comp rx=0 tx=0).stop 2024-05-15T16:13:51.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.741+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 shutdown_connections 2024-05-15T16:13:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.741+0000 7f1483acc700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f146806e200 0x7f14680706c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.741+0000 7f1483acc700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f147c0733d0 0x7f147c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.741+0000 7f1483acc700 1 --2- 172.21.15.49:0/3722962704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f147c073cf0 0x7f147c1992f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.741+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 >> 172.21.15.49:0/3722962704 conn(0x7f147c0fc7d0 msgr2=0x7f147c105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.742+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 shutdown_connections 2024-05-15T16:13:51.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.742+0000 7f1483acc700 1 -- 172.21.15.49:0/3722962704 wait complete. 2024-05-15T16:13:51.754 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:13:51.754 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:51 smithi049 ceph-mon[112142]: pgmap v504: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:51.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.851+0000 7efd4f492700 1 -- 172.21.15.49:0/1074630820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48101880 msgr2=0x7efd481058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.851+0000 7efd4f492700 1 --2- 172.21.15.49:0/1074630820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48101880 0x7efd481058d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efd38009910 tx=0x7efd38009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.851+0000 7efd4f492700 1 -- 172.21.15.49:0/1074630820 shutdown_connections 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.851+0000 7efd4f492700 1 --2- 172.21.15.49:0/1074630820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48101880 0x7efd481058d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.851+0000 7efd4f492700 1 --2- 172.21.15.49:0/1074630820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd48100ed0 0x7efd481012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.851+0000 7efd4f492700 1 -- 172.21.15.49:0/1074630820 >> 172.21.15.49:0/1074630820 conn(0x7efd480fc7a0 msgr2=0x7efd480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.852+0000 7efd4f492700 1 -- 172.21.15.49:0/1074630820 shutdown_connections 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.852+0000 7efd4f492700 1 -- 172.21.15.49:0/1074630820 wait complete. 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.852+0000 7efd4f492700 1 Processor -- start 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.852+0000 7efd4f492700 1 -- start start 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.852+0000 7efd4f492700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48100ed0 0x7efd480732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.852+0000 7efd4f492700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd48101880 0x7efd480737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:51.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.852+0000 7efd4f492700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd48073d20 con 0x7efd48100ed0 2024-05-15T16:13:51.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.852+0000 7efd4f492700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd48073e60 con 0x7efd48101880 2024-05-15T16:13:51.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4d22e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48100ed0 0x7efd480732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:51.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4d22e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48100ed0 0x7efd480732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51178/0 (socket says 172.21.15.49:51178) 2024-05-15T16:13:51.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4d22e700 1 -- 172.21.15.49:0/3072532706 learned_addr learned my addr 172.21.15.49:0/3072532706 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:51.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4ca2d700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd48101880 0x7efd480737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:51.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4d22e700 1 -- 172.21.15.49:0/3072532706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd48101880 msgr2=0x7efd480737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:51.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4d22e700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd48101880 0x7efd480737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:51.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4d22e700 1 -- 172.21.15.49:0/3072532706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380095f0 con 0x7efd48100ed0 2024-05-15T16:13:51.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4d22e700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48100ed0 0x7efd480732a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efd4400d8d0 tx=0x7efd4400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:51.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd3e7fc700 1 -- 172.21.15.49:0/3072532706 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd440098a0 con 0x7efd48100ed0 2024-05-15T16:13:51.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd3e7fc700 1 -- 172.21.15.49:0/3072532706 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd44010460 con 0x7efd48100ed0 2024-05-15T16:13:51.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd481a2df0 con 0x7efd48100ed0 2024-05-15T16:13:51.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd3e7fc700 1 -- 172.21.15.49:0/3072532706 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd4400f5f0 con 0x7efd48100ed0 2024-05-15T16:13:51.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.853+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd481a3150 con 0x7efd48100ed0 2024-05-15T16:13:51.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.854+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd481092d0 con 0x7efd48100ed0 2024-05-15T16:13:51.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.855+0000 7efd3e7fc700 1 -- 172.21.15.49:0/3072532706 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efd44009a00 con 0x7efd48100ed0 2024-05-15T16:13:51.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.855+0000 7efd3e7fc700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd3406e200 0x7efd340706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:51.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.856+0000 7efd3e7fc700 1 -- 172.21.15.49:0/3072532706 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efd4408df70 con 0x7efd48100ed0 2024-05-15T16:13:51.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.856+0000 7efd4ca2d700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd3406e200 0x7efd340706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:51.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.856+0000 7efd4ca2d700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd3406e200 0x7efd340706c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd48074980 tx=0x7efd3801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:51.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:51.860+0000 7efd3e7fc700 1 -- 172.21.15.49:0/3072532706 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efd440c2850 con 0x7efd48100ed0 2024-05-15T16:13:51.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:51 smithi144 ceph-mon[86635]: pgmap v504: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:52.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.087+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd4810de70 con 0x7efd3406e200 2024-05-15T16:13:52.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.088+0000 7efd3e7fc700 1 -- 172.21.15.49:0/3072532706 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efd4810de70 con 0x7efd3406e200 2024-05-15T16:13:52.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd3406e200 msgr2=0x7efd340706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd3406e200 0x7efd340706c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd48074980 tx=0x7efd3801a040 comp rx=0 tx=0).stop 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48100ed0 msgr2=0x7efd480732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48100ed0 0x7efd480732a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efd4400d8d0 tx=0x7efd4400dbe0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 shutdown_connections 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efd3406e200 0x7efd340706c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd48100ed0 0x7efd480732a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 --2- 172.21.15.49:0/3072532706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd48101880 0x7efd480737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 >> 172.21.15.49:0/3072532706 conn(0x7efd480fc7a0 msgr2=0x7efd480fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 shutdown_connections 2024-05-15T16:13:52.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.091+0000 7efd4f492700 1 -- 172.21.15.49:0/3072532706 wait complete. 2024-05-15T16:13:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 -- 172.21.15.49:0/1229098498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 msgr2=0x7fe1201012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 --2- 172.21.15.49:0/1229098498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 0x7fe1201012c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe11000b0a0 tx=0x7fe11000b3b0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 -- 172.21.15.49:0/1229098498 shutdown_connections 2024-05-15T16:13:52.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 --2- 172.21.15.49:0/1229098498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe120101890 0x7fe1201058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 --2- 172.21.15.49:0/1229098498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 0x7fe1201012c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 -- 172.21.15.49:0/1229098498 >> 172.21.15.49:0/1229098498 conn(0x7fe1200fc790 msgr2=0x7fe1200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:52.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 -- 172.21.15.49:0/1229098498 shutdown_connections 2024-05-15T16:13:52.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 -- 172.21.15.49:0/1229098498 wait complete. 2024-05-15T16:13:52.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 Processor -- start 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.200+0000 7fe126ae3700 1 -- start start 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe126ae3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 0x7fe12019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe126ae3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe120101890 0x7fe12019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe126ae3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12019d460 con 0x7fe120100ee0 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe126ae3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe120196a90 con 0x7fe120101890 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe12487f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 0x7fe12019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe12487f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 0x7fe12019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51200/0 (socket says 172.21.15.49:51200) 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe12487f700 1 -- 172.21.15.49:0/2698617228 learned_addr learned my addr 172.21.15.49:0/2698617228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe11ffff700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe120101890 0x7fe12019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe12487f700 1 -- 172.21.15.49:0/2698617228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe120101890 msgr2=0x7fe12019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe12487f700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe120101890 0x7fe12019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe12487f700 1 -- 172.21.15.49:0/2698617228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe110009d00 con 0x7fe120100ee0 2024-05-15T16:13:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe11ffff700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe120101890 0x7fe12019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:13:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe12487f700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 0x7fe12019c900 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe110000f80 tx=0x7fe110007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.201+0000 7fe11dffb700 1 -- 172.21.15.49:0/2698617228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11000e040 con 0x7fe120100ee0 2024-05-15T16:13:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.202+0000 7fe11dffb700 1 -- 172.21.15.49:0/2698617228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe110003de0 con 0x7fe120100ee0 2024-05-15T16:13:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.202+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe120196d10 con 0x7fe120100ee0 2024-05-15T16:13:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.202+0000 7fe11dffb700 1 -- 172.21.15.49:0/2698617228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11001d720 con 0x7fe120100ee0 2024-05-15T16:13:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.202+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe120197200 con 0x7fe120100ee0 2024-05-15T16:13:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.203+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe120109280 con 0x7fe120100ee0 2024-05-15T16:13:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.203+0000 7fe11dffb700 1 -- 172.21.15.49:0/2698617228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe110004690 con 0x7fe120100ee0 2024-05-15T16:13:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.204+0000 7fe11dffb700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe10c06e200 0x7fe10c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.204+0000 7fe11dffb700 1 -- 172.21.15.49:0/2698617228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe11008f9e0 con 0x7fe120100ee0 2024-05-15T16:13:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.204+0000 7fe11ffff700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe10c06e200 0x7fe10c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.204+0000 7fe11ffff700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe10c06e200 0x7fe10c0706c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe120197c00 tx=0x7fe108005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:52.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.208+0000 7fe11dffb700 1 -- 172.21.15.49:0/2698617228 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe11005c7f0 con 0x7fe120100ee0 2024-05-15T16:13:52.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.426+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe120197f00 con 0x7fe10c06e200 2024-05-15T16:13:52.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.438+0000 7fe11dffb700 1 -- 172.21.15.49:0/2698617228 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe120197f00 con 0x7fe10c06e200 2024-05-15T16:13:52.439 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:13:52.439 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (17m) 10m ago 28m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:13:52.439 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (28m) 10m ago 28m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:13:52.439 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (27m) 10m ago 27m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:13:52.439 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (14m) 10m ago 28m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:13:52.439 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (14m) 10m ago 27m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:13:52.439 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (16m) 10m ago 28m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:13:52.439 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (10m) 10m ago 23m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (10m) 10m ago 23m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (10m) 10m ago 23m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (22m) 10m ago 22m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (19m) 10m ago 30m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (18m) 10m ago 27m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (15m) 10m ago 30m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (15m) 10m ago 27m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (17m) 10m ago 28m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (17m) 10m ago 27m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (14m) 10m ago 26m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (13m) 10m ago 26m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (12m) 10m ago 25m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (12m) 10m ago 25m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (11m) 10m ago 25m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:13:52.440 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (11m) 10m ago 24m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:13:52.441 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (17m) 10m ago 28m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:13:52.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe10c06e200 msgr2=0x7fe10c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe10c06e200 0x7fe10c0706c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe120197c00 tx=0x7fe108005f50 comp rx=0 tx=0).stop 2024-05-15T16:13:52.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 msgr2=0x7fe12019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 0x7fe12019c900 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe110000f80 tx=0x7fe110007bd0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 shutdown_connections 2024-05-15T16:13:52.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe10c06e200 0x7fe10c0706c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe120100ee0 0x7fe12019c900 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 --2- 172.21.15.49:0/2698617228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe120101890 0x7fe12019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 >> 172.21.15.49:0/2698617228 conn(0x7fe1200fc790 msgr2=0x7fe1200fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:52.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 shutdown_connections 2024-05-15T16:13:52.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.440+0000 7fe126ae3700 1 -- 172.21.15.49:0/2698617228 wait complete. 2024-05-15T16:13:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 -- 172.21.15.49:0/622675799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8101890 msgr2=0x7f3bd81058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 --2- 172.21.15.49:0/622675799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8101890 0x7f3bd81058e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009910 tx=0x7f3bc4009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 -- 172.21.15.49:0/622675799 shutdown_connections 2024-05-15T16:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 --2- 172.21.15.49:0/622675799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8101890 0x7f3bd81058e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 --2- 172.21.15.49:0/622675799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd8100ee0 0x7f3bd81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 -- 172.21.15.49:0/622675799 >> 172.21.15.49:0/622675799 conn(0x7f3bd80fc790 msgr2=0x7f3bd80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 -- 172.21.15.49:0/622675799 shutdown_connections 2024-05-15T16:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 -- 172.21.15.49:0/622675799 wait complete. 2024-05-15T16:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 Processor -- start 2024-05-15T16:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.548+0000 7f3bdf09d700 1 -- start start 2024-05-15T16:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdf09d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8100ee0 0x7f3bd80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdf09d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd8101890 0x7f3bd80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdf09d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8073d20 con 0x7f3bd8100ee0 2024-05-15T16:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdf09d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8073e60 con 0x7f3bd8101890 2024-05-15T16:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdce39700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8100ee0 0x7f3bd80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdce39700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8100ee0 0x7f3bd80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51214/0 (socket says 172.21.15.49:51214) 2024-05-15T16:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdce39700 1 -- 172.21.15.49:0/2561546018 learned_addr learned my addr 172.21.15.49:0/2561546018 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bd7fff700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd8101890 0x7f3bd80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdce39700 1 -- 172.21.15.49:0/2561546018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd8101890 msgr2=0x7f3bd80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdce39700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd8101890 0x7f3bd80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdce39700 1 -- 172.21.15.49:0/2561546018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc40095f0 con 0x7f3bd8100ee0 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bd7fff700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd8101890 0x7f3bd80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdce39700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8100ee0 0x7f3bd80732a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00d8d0 tx=0x7f3bcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bd5ffb700 1 -- 172.21.15.49:0/2561546018 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bcc009940 con 0x7f3bd8100ee0 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd81a2da0 con 0x7f3bd8100ee0 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bd5ffb700 1 -- 172.21.15.49:0/2561546018 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bcc010460 con 0x7f3bd8100ee0 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.549+0000 7f3bd5ffb700 1 -- 172.21.15.49:0/2561546018 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bcc00f5f0 con 0x7f3bd8100ee0 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.550+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd81a3100 con 0x7f3bd8100ee0 2024-05-15T16:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.551+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd81092e0 con 0x7f3bd8100ee0 2024-05-15T16:13:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.551+0000 7f3bd5ffb700 1 -- 172.21.15.49:0/2561546018 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3bcc0105d0 con 0x7f3bd8100ee0 2024-05-15T16:13:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.552+0000 7f3bd5ffb700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bc806e200 0x7f3bc80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.552+0000 7f3bd5ffb700 1 -- 172.21.15.49:0/2561546018 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3bcc08e170 con 0x7f3bd8100ee0 2024-05-15T16:13:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.552+0000 7f3bd7fff700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bc806e200 0x7f3bc80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.552+0000 7f3bd7fff700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bc806e200 0x7f3bc80706c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80749d0 tx=0x7f3bc401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.557+0000 7f3bd5ffb700 1 -- 172.21.15.49:0/2561546018 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3bcc057450 con 0x7f3bd8100ee0 2024-05-15T16:13:52.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:52 smithi049 ceph-mon[112142]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:52.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.852+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3bd810de80 con 0x7f3bd8100ee0 2024-05-15T16:13:52.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.852+0000 7f3bd5ffb700 1 -- 172.21.15.49:0/2561546018 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f3bcc05ab10 con 0x7f3bd8100ee0 2024-05-15T16:13:52.854 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:13:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:13:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:13:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:13:52.854 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:13:52.855 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:13:52.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:13:52.856 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bc806e200 msgr2=0x7f3bc80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bc806e200 0x7f3bc80706c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80749d0 tx=0x7f3bc401a040 comp rx=0 tx=0).stop 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8100ee0 msgr2=0x7f3bd80732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8100ee0 0x7f3bd80732a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00d8d0 tx=0x7f3bcc00dc90 comp rx=0 tx=0).stop 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 shutdown_connections 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bc806e200 0x7f3bc80706c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bd8100ee0 0x7f3bd80732a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 --2- 172.21.15.49:0/2561546018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd8101890 0x7f3bd80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 >> 172.21.15.49:0/2561546018 conn(0x7f3bd80fc790 msgr2=0x7f3bd80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 shutdown_connections 2024-05-15T16:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.855+0000 7f3bdf09d700 1 -- 172.21.15.49:0/2561546018 wait complete. 2024-05-15T16:13:52.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:52 smithi144 ceph-mon[86635]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:52.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.964+0000 7ff575ecd700 1 -- 172.21.15.49:0/4053619909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570102790 msgr2=0x7ff57010ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.964+0000 7ff575ecd700 1 --2- 172.21.15.49:0/4053619909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570102790 0x7ff57010ac80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff564009910 tx=0x7ff564009c20 comp rx=0 tx=0).stop 2024-05-15T16:13:52.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.964+0000 7ff575ecd700 1 -- 172.21.15.49:0/4053619909 shutdown_connections 2024-05-15T16:13:52.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.964+0000 7ff575ecd700 1 --2- 172.21.15.49:0/4053619909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570102790 0x7ff57010ac80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.964+0000 7ff575ecd700 1 --2- 172.21.15.49:0/4053619909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff570101e70 0x7ff570102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.964+0000 7ff575ecd700 1 -- 172.21.15.49:0/4053619909 >> 172.21.15.49:0/4053619909 conn(0x7ff5700fb700 msgr2=0x7ff5700fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:52.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.964+0000 7ff575ecd700 1 -- 172.21.15.49:0/4053619909 shutdown_connections 2024-05-15T16:13:52.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.964+0000 7ff575ecd700 1 -- 172.21.15.49:0/4053619909 wait complete. 2024-05-15T16:13:52.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff575ecd700 1 Processor -- start 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff575ecd700 1 -- start start 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff575ecd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570101e70 0x7ff5700ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff575ecd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff570102790 0x7ff570100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff575ecd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff570100550 con 0x7ff570101e70 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff575ecd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff570100690 con 0x7ff570102790 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff56f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570101e70 0x7ff5700ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff56f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570101e70 0x7ff5700ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51224/0 (socket says 172.21.15.49:51224) 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff56f7fe700 1 -- 172.21.15.49:0/2280770318 learned_addr learned my addr 172.21.15.49:0/2280770318 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff56effd700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff570102790 0x7ff570100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:52.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.965+0000 7ff56f7fe700 1 -- 172.21.15.49:0/2280770318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff570102790 msgr2=0x7ff570100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:52.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.966+0000 7ff56f7fe700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff570102790 0x7ff570100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:52.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.966+0000 7ff56f7fe700 1 -- 172.21.15.49:0/2280770318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640095f0 con 0x7ff570101e70 2024-05-15T16:13:52.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.966+0000 7ff56effd700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff570102790 0x7ff570100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:13:52.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.966+0000 7ff56f7fe700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570101e70 0x7ff5700ffad0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c8a0 tx=0x7ff56000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:52.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.966+0000 7ff56cff9700 1 -- 172.21.15.49:0/2280770318 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff56000cea0 con 0x7ff570101e70 2024-05-15T16:13:52.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.966+0000 7ff56cff9700 1 -- 172.21.15.49:0/2280770318 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff560004830 con 0x7ff570101e70 2024-05-15T16:13:52.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.966+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5701a2df0 con 0x7ff570101e70 2024-05-15T16:13:52.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.966+0000 7ff56cff9700 1 -- 172.21.15.49:0/2280770318 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5600055b0 con 0x7ff570101e70 2024-05-15T16:13:52.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.966+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5701a3150 con 0x7ff570101e70 2024-05-15T16:13:52.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.967+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff570069ab0 con 0x7ff570101e70 2024-05-15T16:13:52.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.968+0000 7ff56cff9700 1 -- 172.21.15.49:0/2280770318 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff5600049a0 con 0x7ff570101e70 2024-05-15T16:13:52.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.968+0000 7ff56cff9700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff55806e200 0x7ff5580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:52.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.968+0000 7ff56cff9700 1 -- 172.21.15.49:0/2280770318 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff56008de30 con 0x7ff570101e70 2024-05-15T16:13:52.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.968+0000 7ff56effd700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff55806e200 0x7ff5580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:52.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.969+0000 7ff56effd700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff55806e200 0x7ff5580706c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff564006010 tx=0x7ff56400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:52.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:52.973+0000 7ff56cff9700 1 -- 172.21.15.49:0/2280770318 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff560057110 con 0x7ff570101e70 2024-05-15T16:13:53.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.224+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff570100850 con 0x7ff570101e70 2024-05-15T16:13:53.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.224+0000 7ff56cff9700 1 -- 172.21.15.49:0/2280770318 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7ff56001d070 con 0x7ff570101e70 2024-05-15T16:13:53.226 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:13:53.226 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:13:53.227 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:13:53.228 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:13:53.229 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:13:53.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff55806e200 msgr2=0x7ff5580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff55806e200 0x7ff5580706c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff564006010 tx=0x7ff56400b540 comp rx=0 tx=0).stop 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570101e70 msgr2=0x7ff5700ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570101e70 0x7ff5700ffad0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c8a0 tx=0x7ff56000cc60 comp rx=0 tx=0).stop 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 shutdown_connections 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff55806e200 0x7ff5580706c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff570101e70 0x7ff5700ffad0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 --2- 172.21.15.49:0/2280770318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff570102790 0x7ff570100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 >> 172.21.15.49:0/2280770318 conn(0x7ff5700fb700 msgr2=0x7ff570103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 shutdown_connections 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.227+0000 7ff575ecd700 1 -- 172.21.15.49:0/2280770318 wait complete. 2024-05-15T16:13:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:13:53.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.337+0000 7f0e97068700 1 -- 172.21.15.49:0/531293599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90101da0 msgr2=0x7f0e90102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.337+0000 7f0e97068700 1 --2- 172.21.15.49:0/531293599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90101da0 0x7f0e90102180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b0a0 tx=0x7f0e8000b3b0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.337+0000 7f0e97068700 1 -- 172.21.15.49:0/531293599 shutdown_connections 2024-05-15T16:13:53.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.337+0000 7f0e97068700 1 --2- 172.21.15.49:0/531293599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e90102750 0x7f0e9010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.337+0000 7f0e97068700 1 --2- 172.21.15.49:0/531293599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90101da0 0x7f0e90102180 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.337+0000 7f0e97068700 1 -- 172.21.15.49:0/531293599 >> 172.21.15.49:0/531293599 conn(0x7f0e900fb6a0 msgr2=0x7f0e900fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:53.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.337+0000 7f0e97068700 1 -- 172.21.15.49:0/531293599 shutdown_connections 2024-05-15T16:13:53.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.337+0000 7f0e97068700 1 -- 172.21.15.49:0/531293599 wait complete. 2024-05-15T16:13:53.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e97068700 1 Processor -- start 2024-05-15T16:13:53.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e97068700 1 -- start start 2024-05-15T16:13:53.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e97068700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e90101da0 0x7f0e90198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:53.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e97068700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90102750 0x7f0e90199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:53.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e97068700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901999f0 con 0x7f0e90102750 2024-05-15T16:13:53.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e97068700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9019d6c0 con 0x7f0e90101da0 2024-05-15T16:13:53.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e8ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90102750 0x7f0e90199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:53.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e8ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90102750 0x7f0e90199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51240/0 (socket says 172.21.15.49:51240) 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e8ffff700 1 -- 172.21.15.49:0/2785175625 learned_addr learned my addr 172.21.15.49:0/2785175625 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.338+0000 7f0e94e04700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e90101da0 0x7f0e90198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.339+0000 7f0e8ffff700 1 -- 172.21.15.49:0/2785175625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e90101da0 msgr2=0x7f0e90198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.339+0000 7f0e8ffff700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e90101da0 0x7f0e90198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.339+0000 7f0e8ffff700 1 -- 172.21.15.49:0/2785175625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e80009d00 con 0x7f0e90102750 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.339+0000 7f0e8ffff700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90102750 0x7f0e90199310 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800e8f0 tx=0x7f0e7800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.339+0000 7f0e8dffb700 1 -- 172.21.15.49:0/2785175625 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e78009790 con 0x7f0e90102750 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.339+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9019d9a0 con 0x7f0e90102750 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.339+0000 7f0e8dffb700 1 -- 172.21.15.49:0/2785175625 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e7800c810 con 0x7f0e90102750 2024-05-15T16:13:53.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.339+0000 7f0e8dffb700 1 -- 172.21.15.49:0/2785175625 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e78005580 con 0x7f0e90102750 2024-05-15T16:13:53.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.339+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9019def0 con 0x7f0e90102750 2024-05-15T16:13:53.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.340+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e901097a0 con 0x7f0e90102750 2024-05-15T16:13:53.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.341+0000 7f0e8dffb700 1 -- 172.21.15.49:0/2785175625 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0e7800c980 con 0x7f0e90102750 2024-05-15T16:13:53.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.341+0000 7f0e8dffb700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0e7c06e200 0x7f0e7c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:53.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.341+0000 7f0e8dffb700 1 -- 172.21.15.49:0/2785175625 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0e78014070 con 0x7f0e90102750 2024-05-15T16:13:53.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.341+0000 7f0e94e04700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0e7c06e200 0x7f0e7c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:53.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.342+0000 7f0e94e04700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0e7c06e200 0x7f0e7c0706c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0e80000f80 tx=0x7f0e800060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:53.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.346+0000 7f0e8dffb700 1 -- 172.21.15.49:0/2785175625 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0e78057710 con 0x7f0e90102750 2024-05-15T16:13:53.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.576+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e9019a150 con 0x7f0e7c06e200 2024-05-15T16:13:53.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.577+0000 7f0e8dffb700 1 -- 172.21.15.49:0/2785175625 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0e9019a150 con 0x7f0e7c06e200 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:13:53.579 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:13:53.580 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:13:53.580 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:13:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0e7c06e200 msgr2=0x7f0e7c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0e7c06e200 0x7f0e7c0706c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0e80000f80 tx=0x7f0e800060c0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90102750 msgr2=0x7f0e90199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90102750 0x7f0e90199310 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800e8f0 tx=0x7f0e7800ec00 comp rx=0 tx=0).stop 2024-05-15T16:13:53.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 shutdown_connections 2024-05-15T16:13:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0e7c06e200 0x7f0e7c0706c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e90101da0 0x7f0e90198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 --2- 172.21.15.49:0/2785175625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e90102750 0x7f0e90199310 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 >> 172.21.15.49:0/2785175625 conn(0x7f0e900fb6a0 msgr2=0x7f0e9010a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 shutdown_connections 2024-05-15T16:13:53.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.580+0000 7f0e97068700 1 -- 172.21.15.49:0/2785175625 wait complete. 2024-05-15T16:13:53.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.688+0000 7faf67137700 1 -- 172.21.15.49:0/150629207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 msgr2=0x7faf600ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.688+0000 7faf67137700 1 --2- 172.21.15.49:0/150629207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 0x7faf600ff9c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faf5400aa20 tx=0x7faf5400ad30 comp rx=0 tx=0).stop 2024-05-15T16:13:53.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.688+0000 7faf67137700 1 -- 172.21.15.49:0/150629207 shutdown_connections 2024-05-15T16:13:53.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.688+0000 7faf67137700 1 --2- 172.21.15.49:0/150629207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf600fff90 0x7faf60103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.688+0000 7faf67137700 1 --2- 172.21.15.49:0/150629207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 0x7faf600ff9c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.688+0000 7faf67137700 1 -- 172.21.15.49:0/150629207 >> 172.21.15.49:0/150629207 conn(0x7faf60075fc0 msgr2=0x7faf600fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:53.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.688+0000 7faf67137700 1 -- 172.21.15.49:0/150629207 shutdown_connections 2024-05-15T16:13:53.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.688+0000 7faf67137700 1 -- 172.21.15.49:0/150629207 wait complete. 2024-05-15T16:13:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.689+0000 7faf67137700 1 Processor -- start 2024-05-15T16:13:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.689+0000 7faf67137700 1 -- start start 2024-05-15T16:13:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.689+0000 7faf67137700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 0x7faf6019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.689+0000 7faf67137700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf600fff90 0x7faf6019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.689+0000 7faf67137700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6019c450 con 0x7faf600ff5e0 2024-05-15T16:13:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.689+0000 7faf67137700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf60196b20 con 0x7faf600fff90 2024-05-15T16:13:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.689+0000 7faf64ed3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 0x7faf6019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.689+0000 7faf64ed3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 0x7faf6019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41930/0 (socket says 172.21.15.49:41930) 2024-05-15T16:13:53.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.689+0000 7faf64ed3700 1 -- 172.21.15.49:0/2047809112 learned_addr learned my addr 172.21.15.49:0/2047809112 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf5ffff700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf600fff90 0x7faf6019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf64ed3700 1 -- 172.21.15.49:0/2047809112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf600fff90 msgr2=0x7faf6019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf64ed3700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf600fff90 0x7faf6019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf64ed3700 1 -- 172.21.15.49:0/2047809112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5400a700 con 0x7faf600ff5e0 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf64ed3700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 0x7faf6019b830 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faf5400cd20 tx=0x7faf54005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf5dffb700 1 -- 172.21.15.49:0/2047809112 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf54004cb0 con 0x7faf600ff5e0 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf5dffb700 1 -- 172.21.15.49:0/2047809112 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf54004e10 con 0x7faf600ff5e0 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf5dffb700 1 -- 172.21.15.49:0/2047809112 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf54016900 con 0x7faf600ff5e0 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf60196da0 con 0x7faf600ff5e0 2024-05-15T16:13:53.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.690+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf60197290 con 0x7faf600ff5e0 2024-05-15T16:13:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.691+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf60100ac0 con 0x7faf600ff5e0 2024-05-15T16:13:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.692+0000 7faf5dffb700 1 -- 172.21.15.49:0/2047809112 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faf5400ecc0 con 0x7faf600ff5e0 2024-05-15T16:13:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.692+0000 7faf5dffb700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf5006e210 0x7faf500706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:13:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.693+0000 7faf5dffb700 1 -- 172.21.15.49:0/2047809112 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faf54012070 con 0x7faf600ff5e0 2024-05-15T16:13:53.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.693+0000 7faf5ffff700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf5006e210 0x7faf500706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:13:53.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.693+0000 7faf5ffff700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf5006e210 0x7faf500706d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf4c005fd0 tx=0x7faf4c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:13:53.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.697+0000 7faf5dffb700 1 -- 172.21.15.49:0/2047809112 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7faf54059600 con 0x7faf600ff5e0 2024-05-15T16:13:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:53 smithi144 ceph-mon[86635]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:53 smithi144 ceph-mon[86635]: pgmap v505: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:53 smithi144 ceph-mon[86635]: from='client.34970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:53 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2561546018' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:13:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:53 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2280770318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:13:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:53 smithi049 ceph-mon[112142]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:53 smithi049 ceph-mon[112142]: pgmap v505: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:53 smithi049 ceph-mon[112142]: from='client.34970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:53 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2561546018' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:13:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:53 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2280770318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:13:53.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.992+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faf6004ec60 con 0x7faf600ff5e0 2024-05-15T16:13:53.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.992+0000 7faf5dffb700 1 -- 172.21.15.49:0/2047809112 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7faf5405ccc0 con 0x7faf600ff5e0 2024-05-15T16:13:53.994 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:13:53.994 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:13:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:13:53.994 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:13:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:13:53.994 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:13:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:13:53.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.994+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf5006e210 msgr2=0x7faf500706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.994+0000 7faf67137700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf5006e210 0x7faf500706d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf4c005fd0 tx=0x7faf4c00a560 comp rx=0 tx=0).stop 2024-05-15T16:13:53.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.994+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 msgr2=0x7faf6019b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:13:53.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.994+0000 7faf67137700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 0x7faf6019b830 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faf5400cd20 tx=0x7faf54005750 comp rx=0 tx=0).stop 2024-05-15T16:13:53.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.994+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 shutdown_connections 2024-05-15T16:13:53.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.994+0000 7faf67137700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faf5006e210 0x7faf500706d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.994+0000 7faf67137700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf600ff5e0 0x7faf6019b830 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.994+0000 7faf67137700 1 --2- 172.21.15.49:0/2047809112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf600fff90 0x7faf6019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:13:53.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.995+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 >> 172.21.15.49:0/2047809112 conn(0x7faf60075fc0 msgr2=0x7faf601098f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:13:53.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.995+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 shutdown_connections 2024-05-15T16:13:53.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:13:53.995+0000 7faf67137700 1 -- 172.21.15.49:0/2047809112 wait complete. 2024-05-15T16:13:54.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:54 smithi144 ceph-mon[86635]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:54.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:54 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2047809112' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:13:54.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:54 smithi049 ceph-mon[112142]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:13:54.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:54 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2047809112' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:13:55.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:55 smithi144 ceph-mon[86635]: pgmap v506: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:55.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:55 smithi049 ceph-mon[112142]: pgmap v506: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:57.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:57 smithi144 ceph-mon[86635]: pgmap v507: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:57 smithi049 ceph-mon[112142]: pgmap v507: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:13:59 smithi144 ceph-mon[86635]: pgmap v508: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:13:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:13:59 smithi049 ceph-mon[112142]: pgmap v508: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:01.849 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:01 smithi144 ceph-mon[86635]: pgmap v509: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:01.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:01 smithi049 ceph-mon[112142]: pgmap v509: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:03.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:03 smithi144 ceph-mon[86635]: pgmap v510: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:03.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:03 smithi049 ceph-mon[112142]: pgmap v510: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:05 smithi144 ceph-mon[86635]: pgmap v511: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:14:05.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:05 smithi049 ceph-mon[112142]: pgmap v511: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:05.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:14:07.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:07 smithi144 ceph-mon[86635]: pgmap v512: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:07.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:07 smithi049 ceph-mon[112142]: pgmap v512: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:09.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:09 smithi144 ceph-mon[86635]: pgmap v513: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:09.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:09 smithi049 ceph-mon[112142]: pgmap v513: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:11.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:11 smithi144 ceph-mon[86635]: pgmap v514: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:11 smithi049 ceph-mon[112142]: pgmap v514: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:13.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:13 smithi144 ceph-mon[86635]: pgmap v515: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:13.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:13 smithi049 ceph-mon[112142]: pgmap v515: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:15.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:15 smithi144 ceph-mon[86635]: pgmap v516: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:15.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:15 smithi049 ceph-mon[112142]: pgmap v516: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:17.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:17 smithi144 ceph-mon[86635]: pgmap v517: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:17.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:17 smithi049 ceph-mon[112142]: pgmap v517: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:19.873 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:19 smithi049 ceph-mon[112142]: pgmap v518: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:19 smithi144 ceph-mon[86635]: pgmap v518: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:20.832 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:14:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:14:21.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:21 smithi144 ceph-mon[86635]: pgmap v519: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:21.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:21 smithi049 ceph-mon[112142]: pgmap v519: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:23.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:23 smithi144 ceph-mon[86635]: pgmap v520: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:23.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:23 smithi049 ceph-mon[112142]: pgmap v520: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:24.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.118+0000 7f3c08428700 1 -- 172.21.15.49:0/307263975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000fff50 msgr2=0x7f3c00103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.118+0000 7f3c08428700 1 --2- 172.21.15.49:0/307263975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000fff50 0x7f3c00103fa0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000aa20 tx=0x7f3bf000ad30 comp rx=0 tx=0).stop 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.118+0000 7f3c08428700 1 -- 172.21.15.49:0/307263975 shutdown_connections 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.118+0000 7f3c08428700 1 --2- 172.21.15.49:0/307263975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000fff50 0x7f3c00103fa0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.118+0000 7f3c08428700 1 --2- 172.21.15.49:0/307263975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c000ff5a0 0x7f3c000ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.118+0000 7f3c08428700 1 -- 172.21.15.49:0/307263975 >> 172.21.15.49:0/307263975 conn(0x7f3c00076000 msgr2=0x7f3c000fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.118+0000 7f3c08428700 1 -- 172.21.15.49:0/307263975 shutdown_connections 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.118+0000 7f3c08428700 1 -- 172.21.15.49:0/307263975 wait complete. 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.118+0000 7f3c08428700 1 Processor -- start 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c08428700 1 -- start start 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c08428700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000ff5a0 0x7f3c0019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c08428700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c000fff50 0x7f3c0019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c08428700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0019d440 con 0x7f3c000ff5a0 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c08428700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c00196a20 con 0x7f3c000fff50 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c061c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000ff5a0 0x7f3c0019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c061c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000ff5a0 0x7f3c0019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59584/0 (socket says 172.21.15.49:59584) 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c061c4700 1 -- 172.21.15.49:0/2733206406 learned_addr learned my addr 172.21.15.49:0/2733206406 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c059c3700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c000fff50 0x7f3c0019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c061c4700 1 -- 172.21.15.49:0/2733206406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c000fff50 msgr2=0x7f3c0019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c061c4700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c000fff50 0x7f3c0019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c061c4700 1 -- 172.21.15.49:0/2733206406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf000a700 con 0x7f3c000ff5a0 2024-05-15T16:14:24.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.119+0000 7f3c061c4700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000ff5a0 0x7f3c0019c8e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00e920 tx=0x7f3bfc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.120+0000 7f3bf77fe700 1 -- 172.21.15.49:0/2733206406 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bfc00c9d0 con 0x7f3c000ff5a0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.120+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c00196d00 con 0x7f3c000ff5a0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.120+0000 7f3bf77fe700 1 -- 172.21.15.49:0/2733206406 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bfc010430 con 0x7f3c000ff5a0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.120+0000 7f3bf77fe700 1 -- 172.21.15.49:0/2733206406 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bfc018630 con 0x7f3c000ff5a0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.120+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c00197250 con 0x7f3c000ff5a0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.121+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c001099b0 con 0x7f3c000ff5a0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.121+0000 7f3bf77fe700 1 -- 172.21.15.49:0/2733206406 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3bfc0105a0 con 0x7f3c000ff5a0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.122+0000 7f3bf77fe700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bec06e200 0x7f3bec0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.122+0000 7f3bf77fe700 1 -- 172.21.15.49:0/2733206406 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3bfc014070 con 0x7f3c000ff5a0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.122+0000 7f3c059c3700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bec06e200 0x7f3bec0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.122+0000 7f3c059c3700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bec06e200 0x7f3bec0706c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c00197be0 tx=0x7f3bf000c670 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:24.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.126+0000 7f3bf77fe700 1 -- 172.21.15.49:0/2733206406 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3bfc053850 con 0x7f3c000ff5a0 2024-05-15T16:14:24.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.353+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c00197f50 con 0x7f3bec06e200 2024-05-15T16:14:24.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.354+0000 7f3bf77fe700 1 -- 172.21.15.49:0/2733206406 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3c00197f50 con 0x7f3bec06e200 2024-05-15T16:14:24.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bec06e200 msgr2=0x7f3bec0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bec06e200 0x7f3bec0706c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c00197be0 tx=0x7f3bf000c670 comp rx=0 tx=0).stop 2024-05-15T16:14:24.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000ff5a0 msgr2=0x7f3c0019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000ff5a0 0x7f3c0019c8e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00e920 tx=0x7f3bfc00ec30 comp rx=0 tx=0).stop 2024-05-15T16:14:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 shutdown_connections 2024-05-15T16:14:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bec06e200 0x7f3bec0706c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c000ff5a0 0x7f3c0019c8e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 --2- 172.21.15.49:0/2733206406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c000fff50 0x7f3c0019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 >> 172.21.15.49:0/2733206406 conn(0x7f3c00076000 msgr2=0x7f3c000fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 shutdown_connections 2024-05-15T16:14:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.357+0000 7f3c08428700 1 -- 172.21.15.49:0/2733206406 wait complete. 2024-05-15T16:14:24.369 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:14:24.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 -- 172.21.15.49:0/1756119684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 msgr2=0x7f843c102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 --2- 172.21.15.49:0/1756119684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 0x7f843c102050 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f842c009910 tx=0x7f842c009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 -- 172.21.15.49:0/1756119684 shutdown_connections 2024-05-15T16:14:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 --2- 172.21.15.49:0/1756119684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f843c102590 0x7f843c10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 --2- 172.21.15.49:0/1756119684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 0x7f843c102050 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 -- 172.21.15.49:0/1756119684 >> 172.21.15.49:0/1756119684 conn(0x7f843c0fb500 msgr2=0x7f843c0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 -- 172.21.15.49:0/1756119684 shutdown_connections 2024-05-15T16:14:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 -- 172.21.15.49:0/1756119684 wait complete. 2024-05-15T16:14:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 Processor -- start 2024-05-15T16:14:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.468+0000 7f844159d700 1 -- start start 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f844159d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 0x7f843c0ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f844159d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f843c102590 0x7f843c0ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f844159d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c100350 con 0x7f843c101c70 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f844159d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c100490 con 0x7f843c102590 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f843bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 0x7f843c0ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f843bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 0x7f843c0ff8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59602/0 (socket says 172.21.15.49:59602) 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f843bfff700 1 -- 172.21.15.49:0/660952747 learned_addr learned my addr 172.21.15.49:0/660952747 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f843b7fe700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f843c102590 0x7f843c0ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f843bfff700 1 -- 172.21.15.49:0/660952747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f843c102590 msgr2=0x7f843c0ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f843bfff700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f843c102590 0x7f843c0ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f843bfff700 1 -- 172.21.15.49:0/660952747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0095f0 con 0x7f843c101c70 2024-05-15T16:14:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f843b7fe700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f843c102590 0x7f843c0ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:14:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f843bfff700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 0x7f843c0ff8d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f842c00bd70 tx=0x7f842c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.469+0000 7f84397fa700 1 -- 172.21.15.49:0/660952747 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842c01d070 con 0x7f843c101c70 2024-05-15T16:14:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.470+0000 7f84397fa700 1 -- 172.21.15.49:0/660952747 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f842c0056e0 con 0x7f843c101c70 2024-05-15T16:14:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.470+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c100710 con 0x7f843c101c70 2024-05-15T16:14:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.470+0000 7f84397fa700 1 -- 172.21.15.49:0/660952747 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842c017400 con 0x7f843c101c70 2024-05-15T16:14:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.470+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c1a2e30 con 0x7f843c101c70 2024-05-15T16:14:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.471+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c0697f0 con 0x7f843c101c70 2024-05-15T16:14:24.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.471+0000 7f84397fa700 1 -- 172.21.15.49:0/660952747 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f842c003a60 con 0x7f843c101c70 2024-05-15T16:14:24.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.472+0000 7f84397fa700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f842406e200 0x7f84240706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:24.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.472+0000 7f84397fa700 1 -- 172.21.15.49:0/660952747 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f842c08f520 con 0x7f843c101c70 2024-05-15T16:14:24.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.472+0000 7f843b7fe700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f842406e200 0x7f84240706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:24.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.472+0000 7f843b7fe700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f842406e200 0x7f84240706c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8430005950 tx=0x7f84300058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:24.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.476+0000 7f84397fa700 1 -- 172.21.15.49:0/660952747 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f842c058880 con 0x7f843c101c70 2024-05-15T16:14:24.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.715+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f843c10dcf0 con 0x7f842406e200 2024-05-15T16:14:24.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.716+0000 7f84397fa700 1 -- 172.21.15.49:0/660952747 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f843c10dcf0 con 0x7f842406e200 2024-05-15T16:14:24.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f842406e200 msgr2=0x7f84240706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f842406e200 0x7f84240706c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8430005950 tx=0x7f84300058e0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 msgr2=0x7f843c0ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 0x7f843c0ff8d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f842c00bd70 tx=0x7f842c00be50 comp rx=0 tx=0).stop 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 shutdown_connections 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f842406e200 0x7f84240706c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f843c101c70 0x7f843c0ff8d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 --2- 172.21.15.49:0/660952747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f843c102590 0x7f843c0ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 >> 172.21.15.49:0/660952747 conn(0x7f843c0fb500 msgr2=0x7f843c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 shutdown_connections 2024-05-15T16:14:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.718+0000 7f844159d700 1 -- 172.21.15.49:0/660952747 wait complete. 2024-05-15T16:14:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.827+0000 7fb27f570700 1 -- 172.21.15.49:0/747831442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278101f30 msgr2=0x7fb278102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.827+0000 7fb27f570700 1 --2- 172.21.15.49:0/747831442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278101f30 0x7fb278102310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb27400b0a0 tx=0x7fb27400b3b0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.827+0000 7fb27f570700 1 -- 172.21.15.49:0/747831442 shutdown_connections 2024-05-15T16:14:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.827+0000 7fb27f570700 1 --2- 172.21.15.49:0/747831442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb278102850 0x7fb27810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.827+0000 7fb27f570700 1 --2- 172.21.15.49:0/747831442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278101f30 0x7fb278102310 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.827+0000 7fb27f570700 1 -- 172.21.15.49:0/747831442 >> 172.21.15.49:0/747831442 conn(0x7fb2780fb700 msgr2=0x7fb2780fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.827+0000 7fb27f570700 1 -- 172.21.15.49:0/747831442 shutdown_connections 2024-05-15T16:14:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.827+0000 7fb27f570700 1 -- 172.21.15.49:0/747831442 wait complete. 2024-05-15T16:14:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.827+0000 7fb27f570700 1 Processor -- start 2024-05-15T16:14:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27f570700 1 -- start start 2024-05-15T16:14:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27f570700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb278101f30 0x7fb27819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27f570700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278102850 0x7fb27819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27f570700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27819d520 con 0x7fb278102850 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27f570700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278196b00 con 0x7fb278101f30 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27cb0b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278102850 0x7fb27819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27cb0b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278102850 0x7fb27819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59606/0 (socket says 172.21.15.49:59606) 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27cb0b700 1 -- 172.21.15.49:0/2461777274 learned_addr learned my addr 172.21.15.49:0/2461777274 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27d30c700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb278101f30 0x7fb27819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27cb0b700 1 -- 172.21.15.49:0/2461777274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb278101f30 msgr2=0x7fb27819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27cb0b700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb278101f30 0x7fb27819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27cb0b700 1 -- 172.21.15.49:0/2461777274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274009d00 con 0x7fb278102850 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.828+0000 7fb27cb0b700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278102850 0x7fb27819cf00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d8d0 tx=0x7fb26800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.829+0000 7fb26e7fc700 1 -- 172.21.15.49:0/2461777274 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2680041f0 con 0x7fb278102850 2024-05-15T16:14:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.829+0000 7fb26e7fc700 1 -- 172.21.15.49:0/2461777274 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb268007620 con 0x7fb278102850 2024-05-15T16:14:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.829+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb278196de0 con 0x7fb278102850 2024-05-15T16:14:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.829+0000 7fb26e7fc700 1 -- 172.21.15.49:0/2461777274 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb268003c50 con 0x7fb278102850 2024-05-15T16:14:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.829+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb278197330 con 0x7fb278102850 2024-05-15T16:14:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.830+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb278105650 con 0x7fb278102850 2024-05-15T16:14:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.830+0000 7fb26e7fc700 1 -- 172.21.15.49:0/2461777274 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb268004350 con 0x7fb278102850 2024-05-15T16:14:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.831+0000 7fb26e7fc700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb26406e210 0x7fb2640706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.831+0000 7fb26e7fc700 1 -- 172.21.15.49:0/2461777274 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb26808dd30 con 0x7fb278102850 2024-05-15T16:14:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.831+0000 7fb27d30c700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb26406e210 0x7fb2640706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.831+0000 7fb27d30c700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb26406e210 0x7fb2640706d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb2740060b0 tx=0x7fb274006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:24.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:24.836+0000 7fb26e7fc700 1 -- 172.21.15.49:0/2461777274 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb26805ab40 con 0x7fb278102850 2024-05-15T16:14:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.054+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb278197fc0 con 0x7fb26406e210 2024-05-15T16:14:25.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.065+0000 7fb26e7fc700 1 -- 172.21.15.49:0/2461777274 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb278197fc0 con 0x7fb26406e210 2024-05-15T16:14:25.066 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (17m) 10m ago 29m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (29m) 10m ago 29m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (28m) 10m ago 28m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (15m) 10m ago 29m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (14m) 10m ago 28m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (17m) 10m ago 28m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (11m) 10m ago 23m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (11m) 10m ago 23m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (10m) 10m ago 23m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (23m) 10m ago 23m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (19m) 10m ago 30m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (18m) 10m ago 28m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:14:25.067 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (16m) 10m ago 30m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (15m) 10m ago 28m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (18m) 10m ago 29m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (18m) 10m ago 28m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (14m) 10m ago 27m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (13m) 10m ago 26m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (13m) 10m ago 26m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (12m) 10m ago 25m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (12m) 10m ago 25m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (11m) 10m ago 25m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:14:25.068 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (17m) 10m ago 28m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.067+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb26406e210 msgr2=0x7fb2640706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.067+0000 7fb27f570700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb26406e210 0x7fb2640706d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb2740060b0 tx=0x7fb274006000 comp rx=0 tx=0).stop 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.067+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278102850 msgr2=0x7fb27819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.067+0000 7fb27f570700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278102850 0x7fb27819cf00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d8d0 tx=0x7fb26800dc90 comp rx=0 tx=0).stop 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.068+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 shutdown_connections 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.068+0000 7fb27f570700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb26406e210 0x7fb2640706d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.068+0000 7fb27f570700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb278101f30 0x7fb27819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.068+0000 7fb27f570700 1 --2- 172.21.15.49:0/2461777274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb278102850 0x7fb27819cf00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.068+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 >> 172.21.15.49:0/2461777274 conn(0x7fb2780fb700 msgr2=0x7fb278103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.068+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 shutdown_connections 2024-05-15T16:14:25.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.068+0000 7fb27f570700 1 -- 172.21.15.49:0/2461777274 wait complete. 2024-05-15T16:14:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.175+0000 7fda1ed29700 1 -- 172.21.15.49:0/3132580467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18102800 msgr2=0x7fda1810acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.175+0000 7fda1ed29700 1 --2- 172.21.15.49:0/3132580467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18102800 0x7fda1810acf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fda0800b0a0 tx=0x7fda0800b3b0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.175+0000 7fda1ed29700 1 -- 172.21.15.49:0/3132580467 shutdown_connections 2024-05-15T16:14:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.175+0000 7fda1ed29700 1 --2- 172.21.15.49:0/3132580467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18102800 0x7fda1810acf0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.175+0000 7fda1ed29700 1 --2- 172.21.15.49:0/3132580467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda18101ee0 0x7fda181022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.175+0000 7fda1ed29700 1 -- 172.21.15.49:0/3132580467 >> 172.21.15.49:0/3132580467 conn(0x7fda18076800 msgr2=0x7fda180fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.175+0000 7fda1ed29700 1 -- 172.21.15.49:0/3132580467 shutdown_connections 2024-05-15T16:14:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.175+0000 7fda1ed29700 1 -- 172.21.15.49:0/3132580467 wait complete. 2024-05-15T16:14:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda1ed29700 1 Processor -- start 2024-05-15T16:14:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda1ed29700 1 -- start start 2024-05-15T16:14:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda1ed29700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18101ee0 0x7fda1819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda1ed29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda18102800 0x7fda1819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda1ed29700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda18196a90 con 0x7fda18101ee0 2024-05-15T16:14:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda1ed29700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda18196c00 con 0x7fda18102800 2024-05-15T16:14:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda1cac5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18101ee0 0x7fda1819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda1cac5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18101ee0 0x7fda1819c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59628/0 (socket says 172.21.15.49:59628) 2024-05-15T16:14:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda1cac5700 1 -- 172.21.15.49:0/1675392438 learned_addr learned my addr 172.21.15.49:0/1675392438 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.176+0000 7fda17fff700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda18102800 0x7fda1819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.177+0000 7fda1cac5700 1 -- 172.21.15.49:0/1675392438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda18102800 msgr2=0x7fda1819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.177+0000 7fda1cac5700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda18102800 0x7fda1819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.177+0000 7fda1cac5700 1 -- 172.21.15.49:0/1675392438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda08009d00 con 0x7fda18101ee0 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.177+0000 7fda1cac5700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18101ee0 0x7fda1819c9e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d900 tx=0x7fda0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.177+0000 7fda15ffb700 1 -- 172.21.15.49:0/1675392438 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda000041f0 con 0x7fda18101ee0 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.177+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda18196ee0 con 0x7fda18101ee0 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.177+0000 7fda15ffb700 1 -- 172.21.15.49:0/1675392438 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda00004cf0 con 0x7fda18101ee0 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.177+0000 7fda15ffb700 1 -- 172.21.15.49:0/1675392438 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda0000b750 con 0x7fda18101ee0 2024-05-15T16:14:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.177+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda18197400 con 0x7fda18101ee0 2024-05-15T16:14:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.178+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda18068ea0 con 0x7fda18101ee0 2024-05-15T16:14:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.179+0000 7fda15ffb700 1 -- 172.21.15.49:0/1675392438 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fda00004750 con 0x7fda18101ee0 2024-05-15T16:14:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.179+0000 7fda15ffb700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fda0406e200 0x7fda040706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.179+0000 7fda15ffb700 1 -- 172.21.15.49:0/1675392438 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fda0008e070 con 0x7fda18101ee0 2024-05-15T16:14:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.179+0000 7fda17fff700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fda0406e200 0x7fda040706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.180+0000 7fda17fff700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fda0406e200 0x7fda040706c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda08009fd0 tx=0x7fda080060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.184+0000 7fda15ffb700 1 -- 172.21.15.49:0/1675392438 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fda00057350 con 0x7fda18101ee0 2024-05-15T16:14:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.479+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fda1810de70 con 0x7fda18101ee0 2024-05-15T16:14:25.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.479+0000 7fda15ffb700 1 -- 172.21.15.49:0/1675392438 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fda0005aa10 con 0x7fda18101ee0 2024-05-15T16:14:25.481 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:14:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:14:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:14:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:14:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:14:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:14:25.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:14:25.482 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:14:25.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fda0406e200 msgr2=0x7fda040706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fda0406e200 0x7fda040706c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda08009fd0 tx=0x7fda080060c0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18101ee0 msgr2=0x7fda1819c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18101ee0 0x7fda1819c9e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d900 tx=0x7fda0000dc10 comp rx=0 tx=0).stop 2024-05-15T16:14:25.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 shutdown_connections 2024-05-15T16:14:25.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fda0406e200 0x7fda040706c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda18101ee0 0x7fda1819c9e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 --2- 172.21.15.49:0/1675392438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda18102800 0x7fda1819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 >> 172.21.15.49:0/1675392438 conn(0x7fda18076800 msgr2=0x7fda181031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:25.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 shutdown_connections 2024-05-15T16:14:25.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.482+0000 7fda1ed29700 1 -- 172.21.15.49:0/1675392438 wait complete. 2024-05-15T16:14:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 -- 172.21.15.49:0/320188694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c0ffdf0 msgr2=0x7f5c1c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 --2- 172.21.15.49:0/320188694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c0ffdf0 0x7f5c1c1001d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009910 tx=0x7f5c18009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 -- 172.21.15.49:0/320188694 shutdown_connections 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 --2- 172.21.15.49:0/320188694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c1007a0 0x7f5c1c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 --2- 172.21.15.49:0/320188694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c0ffdf0 0x7f5c1c1001d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 -- 172.21.15.49:0/320188694 >> 172.21.15.49:0/320188694 conn(0x7f5c1c0fb6a0 msgr2=0x7f5c1c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 -- 172.21.15.49:0/320188694 shutdown_connections 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 -- 172.21.15.49:0/320188694 wait complete. 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 Processor -- start 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.591+0000 7f5c232d3700 1 -- start start 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c232d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c0ffdf0 0x7f5c1c19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c232d3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c1007a0 0x7f5c1c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c232d3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c19d430 con 0x7f5c1c1007a0 2024-05-15T16:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c232d3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c196a10 con 0x7f5c1c0ffdf0 2024-05-15T16:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c2086e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c1007a0 0x7f5c1c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c2086e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c1007a0 0x7f5c1c19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59642/0 (socket says 172.21.15.49:59642) 2024-05-15T16:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c2086e700 1 -- 172.21.15.49:0/2107579877 learned_addr learned my addr 172.21.15.49:0/2107579877 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c2106f700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c0ffdf0 0x7f5c1c19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c2086e700 1 -- 172.21.15.49:0/2107579877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c0ffdf0 msgr2=0x7f5c1c19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c2086e700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c0ffdf0 0x7f5c1c19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c2086e700 1 -- 172.21.15.49:0/2107579877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c180095f0 con 0x7f5c1c1007a0 2024-05-15T16:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c2086e700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c1007a0 0x7f5c1c19ce10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00d900 tx=0x7f5c0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c127fc700 1 -- 172.21.15.49:0/2107579877 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c0c009900 con 0x7f5c1c1007a0 2024-05-15T16:14:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.592+0000 7f5c127fc700 1 -- 172.21.15.49:0/2107579877 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c0c010460 con 0x7f5c1c1007a0 2024-05-15T16:14:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.593+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c196cf0 con 0x7f5c1c1007a0 2024-05-15T16:14:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.593+0000 7f5c127fc700 1 -- 172.21.15.49:0/2107579877 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c0c00f5f0 con 0x7f5c1c1007a0 2024-05-15T16:14:25.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.593+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c197240 con 0x7f5c1c1007a0 2024-05-15T16:14:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.594+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c1091a0 con 0x7f5c1c1007a0 2024-05-15T16:14:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.594+0000 7f5c127fc700 1 -- 172.21.15.49:0/2107579877 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5c0c0105d0 con 0x7f5c1c1007a0 2024-05-15T16:14:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.595+0000 7f5c127fc700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c0806e200 0x7f5c080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.595+0000 7f5c127fc700 1 -- 172.21.15.49:0/2107579877 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5c0c08e020 con 0x7f5c1c1007a0 2024-05-15T16:14:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.595+0000 7f5c2106f700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c0806e200 0x7f5c080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:25.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.595+0000 7f5c2106f700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c0806e200 0x7f5c080706c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800b5c0 tx=0x7f5c1801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:25.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.599+0000 7f5c127fc700 1 -- 172.21.15.49:0/2107579877 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5c0c05ae30 con 0x7f5c1c1007a0 2024-05-15T16:14:25.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.850+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5c1c197ed0 con 0x7f5c1c1007a0 2024-05-15T16:14:25.852 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:25 smithi049 ceph-mon[112142]: pgmap v521: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:25.852 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:25 smithi049 ceph-mon[112142]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:25.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:25 smithi049 ceph-mon[112142]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:25.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:25 smithi049 ceph-mon[112142]: from='client.34998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:25.853 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:25 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1675392438' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:14:25.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.851+0000 7f5c127fc700 1 -- 172.21.15.49:0/2107579877 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f5c0c05a9c0 con 0x7f5c1c1007a0 2024-05-15T16:14:25.853 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:14:25.853 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:14:25.853 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:14:25.853 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:14:25.854 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:14:25.855 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:14:25.856 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:14:25.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c0806e200 msgr2=0x7f5c080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c0806e200 0x7f5c080706c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800b5c0 tx=0x7f5c1801a040 comp rx=0 tx=0).stop 2024-05-15T16:14:25.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c1007a0 msgr2=0x7f5c1c19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c1007a0 0x7f5c1c19ce10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00d900 tx=0x7f5c0c00dcc0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 shutdown_connections 2024-05-15T16:14:25.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c0806e200 0x7f5c080706c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c0ffdf0 0x7f5c1c19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 --2- 172.21.15.49:0/2107579877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1c1007a0 0x7f5c1c19ce10 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 >> 172.21.15.49:0/2107579877 conn(0x7f5c1c0fb6a0 msgr2=0x7f5c1c069700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:25.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 shutdown_connections 2024-05-15T16:14:25.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.854+0000 7f5c232d3700 1 -- 172.21.15.49:0/2107579877 wait complete. 2024-05-15T16:14:25.858 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:14:25.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:25 smithi144 ceph-mon[86635]: pgmap v521: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:25.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:25 smithi144 ceph-mon[86635]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:25.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:25 smithi144 ceph-mon[86635]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:25.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:25 smithi144 ceph-mon[86635]: from='client.34998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:25.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:25 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1675392438' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:14:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.963+0000 7fb644e1d700 1 -- 172.21.15.49:0/3561424144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 msgr2=0x7fb640105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.963+0000 7fb644e1d700 1 --2- 172.21.15.49:0/3561424144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 0x7fb640105900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb634009910 tx=0x7fb634009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.964+0000 7fb644e1d700 1 -- 172.21.15.49:0/3561424144 shutdown_connections 2024-05-15T16:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.964+0000 7fb644e1d700 1 --2- 172.21.15.49:0/3561424144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 0x7fb640105900 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.964+0000 7fb644e1d700 1 --2- 172.21.15.49:0/3561424144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb640100f00 0x7fb6401012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.964+0000 7fb644e1d700 1 -- 172.21.15.49:0/3561424144 >> 172.21.15.49:0/3561424144 conn(0x7fb6400fc7d0 msgr2=0x7fb6400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.964+0000 7fb644e1d700 1 -- 172.21.15.49:0/3561424144 shutdown_connections 2024-05-15T16:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.964+0000 7fb644e1d700 1 -- 172.21.15.49:0/3561424144 wait complete. 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.964+0000 7fb644e1d700 1 Processor -- start 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.964+0000 7fb644e1d700 1 -- start start 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb644e1d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb640100f00 0x7fb64019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb644e1d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 0x7fb64019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb644e1d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64019d440 con 0x7fb6401018b0 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb644e1d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb640196a20 con 0x7fb640100f00 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb63dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 0x7fb64019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb63dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 0x7fb64019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59664/0 (socket says 172.21.15.49:59664) 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb63dd9b700 1 -- 172.21.15.49:0/1145627662 learned_addr learned my addr 172.21.15.49:0/1145627662 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb63e59c700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb640100f00 0x7fb64019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:25.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb63dd9b700 1 -- 172.21.15.49:0/1145627662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb640100f00 msgr2=0x7fb64019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb63dd9b700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb640100f00 0x7fb64019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb63dd9b700 1 -- 172.21.15.49:0/1145627662 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340095f0 con 0x7fb6401018b0 2024-05-15T16:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb63dd9b700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 0x7fb64019ce20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb6340092e0 tx=0x7fb63400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.965+0000 7fb62f7fe700 1 -- 172.21.15.49:0/1145627662 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb63401d070 con 0x7fb6401018b0 2024-05-15T16:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.966+0000 7fb62f7fe700 1 -- 172.21.15.49:0/1145627662 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb63400f460 con 0x7fb6401018b0 2024-05-15T16:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.966+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb640196ca0 con 0x7fb6401018b0 2024-05-15T16:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.966+0000 7fb62f7fe700 1 -- 172.21.15.49:0/1145627662 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb634017610 con 0x7fb6401018b0 2024-05-15T16:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.966+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb640197190 con 0x7fb6401018b0 2024-05-15T16:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.967+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb640109300 con 0x7fb6401018b0 2024-05-15T16:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.967+0000 7fb62f7fe700 1 -- 172.21.15.49:0/1145627662 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb63400f5d0 con 0x7fb6401018b0 2024-05-15T16:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.968+0000 7fb62f7fe700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb62806e210 0x7fb6280706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.968+0000 7fb63e59c700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb62806e210 0x7fb6280706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.968+0000 7fb62f7fe700 1 -- 172.21.15.49:0/1145627662 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb63408f5c0 con 0x7fb6401018b0 2024-05-15T16:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.968+0000 7fb63e59c700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb62806e210 0x7fb6280706d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb630005fd0 tx=0x7fb63000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:25.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:25.973+0000 7fb62f7fe700 1 -- 172.21.15.49:0/1145627662 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb63405c450 con 0x7fb6401018b0 2024-05-15T16:14:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.204+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb640197ee0 con 0x7fb62806e210 2024-05-15T16:14:26.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.206+0000 7fb62f7fe700 1 -- 172.21.15.49:0/1145627662 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb640197ee0 con 0x7fb62806e210 2024-05-15T16:14:26.207 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:14:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:14:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:14:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:14:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:14:26.207 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:14:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:14:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:14:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:14:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:14:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:14:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:14:26.208 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:14:26.208 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:14:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb62806e210 msgr2=0x7fb6280706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb62806e210 0x7fb6280706d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb630005fd0 tx=0x7fb63000a560 comp rx=0 tx=0).stop 2024-05-15T16:14:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 msgr2=0x7fb64019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 0x7fb64019ce20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb6340092e0 tx=0x7fb63400be20 comp rx=0 tx=0).stop 2024-05-15T16:14:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 shutdown_connections 2024-05-15T16:14:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb62806e210 0x7fb6280706d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb640100f00 0x7fb64019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 --2- 172.21.15.49:0/1145627662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6401018b0 0x7fb64019ce20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 >> 172.21.15.49:0/1145627662 conn(0x7fb6400fc7d0 msgr2=0x7fb6400fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 shutdown_connections 2024-05-15T16:14:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.208+0000 7fb644e1d700 1 -- 172.21.15.49:0/1145627662 wait complete. 2024-05-15T16:14:26.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.317+0000 7f64a6212700 1 -- 172.21.15.49:0/4108831872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0101890 msgr2=0x7f64a01058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:26.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.317+0000 7f64a6212700 1 --2- 172.21.15.49:0/4108831872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0101890 0x7f64a01058e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6488009910 tx=0x7f6488009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:26.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.317+0000 7f64a6212700 1 -- 172.21.15.49:0/4108831872 shutdown_connections 2024-05-15T16:14:26.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.317+0000 7f64a6212700 1 --2- 172.21.15.49:0/4108831872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0101890 0x7f64a01058e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:26.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.317+0000 7f64a6212700 1 --2- 172.21.15.49:0/4108831872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a0100ee0 0x7f64a01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:26.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.317+0000 7f64a6212700 1 -- 172.21.15.49:0/4108831872 >> 172.21.15.49:0/4108831872 conn(0x7f64a00fc790 msgr2=0x7f64a00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:26.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.317+0000 7f64a6212700 1 -- 172.21.15.49:0/4108831872 shutdown_connections 2024-05-15T16:14:26.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.317+0000 7f64a6212700 1 -- 172.21.15.49:0/4108831872 wait complete. 2024-05-15T16:14:26.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f64a6212700 1 Processor -- start 2024-05-15T16:14:26.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f64a6212700 1 -- start start 2024-05-15T16:14:26.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f64a6212700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0100ee0 0x7f64a0198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:26.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f64a6212700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a0101890 0x7f64a0199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:26.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f64a6212700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0199b60 con 0x7f64a0100ee0 2024-05-15T16:14:26.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f64a6212700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a019d7e0 con 0x7f64a0101890 2024-05-15T16:14:26.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f649f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0100ee0 0x7f64a0198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:26.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f649f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0100ee0 0x7f64a0198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59682/0 (socket says 172.21.15.49:59682) 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f649f7fe700 1 -- 172.21.15.49:0/903054804 learned_addr learned my addr 172.21.15.49:0/903054804 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.318+0000 7f649effd700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a0101890 0x7f64a0199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f649f7fe700 1 -- 172.21.15.49:0/903054804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a0101890 msgr2=0x7f64a0199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f649f7fe700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a0101890 0x7f64a0199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f649f7fe700 1 -- 172.21.15.49:0/903054804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880095f0 con 0x7f64a0100ee0 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f649effd700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a0101890 0x7f64a0199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f649f7fe700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0100ee0 0x7f64a0198f40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f649000ba50 tx=0x7f649000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f649cff9700 1 -- 172.21.15.49:0/903054804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f649000ca00 con 0x7f64a0100ee0 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f649cff9700 1 -- 172.21.15.49:0/903054804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f649000cb60 con 0x7f64a0100ee0 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f649cff9700 1 -- 172.21.15.49:0/903054804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64900075a0 con 0x7f64a0100ee0 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a019dac0 con 0x7f64a0100ee0 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.319+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a019e010 con 0x7f64a0100ee0 2024-05-15T16:14:26.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.320+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a01023c0 con 0x7f64a0100ee0 2024-05-15T16:14:26.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.321+0000 7f649cff9700 1 -- 172.21.15.49:0/903054804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f649000ccd0 con 0x7f64a0100ee0 2024-05-15T16:14:26.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.321+0000 7f649cff9700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f648c06e210 0x7f648c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:26.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.321+0000 7f649cff9700 1 -- 172.21.15.49:0/903054804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6490006e40 con 0x7f64a0100ee0 2024-05-15T16:14:26.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.321+0000 7f649effd700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f648c06e210 0x7f648c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:26.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.322+0000 7f649effd700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f648c06e210 0x7f648c0706d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64a019a450 tx=0x7f648801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:26.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.326+0000 7f649cff9700 1 -- 172.21.15.49:0/903054804 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6490056cc0 con 0x7f64a0100ee0 2024-05-15T16:14:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.627+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f64a004ec60 con 0x7f64a0100ee0 2024-05-15T16:14:26.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.627+0000 7f649cff9700 1 -- 172.21.15.49:0/903054804 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f649005a380 con 0x7f64a0100ee0 2024-05-15T16:14:26.629 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:14:26.629 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:14:26.629 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:14:26.629 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:14:26.629 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:14:26.629 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:14:26.629 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:14:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f648c06e210 msgr2=0x7f648c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f648c06e210 0x7f648c0706d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64a019a450 tx=0x7f648801a040 comp rx=0 tx=0).stop 2024-05-15T16:14:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0100ee0 msgr2=0x7f64a0198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0100ee0 0x7f64a0198f40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f649000ba50 tx=0x7f649000be10 comp rx=0 tx=0).stop 2024-05-15T16:14:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 shutdown_connections 2024-05-15T16:14:26.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f648c06e210 0x7f648c0706d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:26.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a0100ee0 0x7f64a0198f40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:26.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 --2- 172.21.15.49:0/903054804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a0101890 0x7f64a0199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:26.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 >> 172.21.15.49:0/903054804 conn(0x7f64a00fc790 msgr2=0x7f64a00745a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:26.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 shutdown_connections 2024-05-15T16:14:26.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:26.630+0000 7f64a6212700 1 -- 172.21.15.49:0/903054804 wait complete. 2024-05-15T16:14:26.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:26 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2107579877' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:14:26.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:26 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2107579877' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:14:27.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:27 smithi144 ceph-mon[86635]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:27.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:27 smithi144 ceph-mon[86635]: pgmap v522: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:27.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:27 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/903054804' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:14:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:27 smithi049 ceph-mon[112142]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:27 smithi049 ceph-mon[112142]: pgmap v522: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:27 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/903054804' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:14:29.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:29 smithi144 ceph-mon[86635]: pgmap v523: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:29.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:29 smithi049 ceph-mon[112142]: pgmap v523: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:31.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:31 smithi144 ceph-mon[86635]: pgmap v524: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:31.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:31 smithi049 ceph-mon[112142]: pgmap v524: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:33.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:33 smithi144 ceph-mon[86635]: pgmap v525: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:33.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:33 smithi049 ceph-mon[112142]: pgmap v525: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:35.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:35 smithi144 ceph-mon[86635]: pgmap v526: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:35.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:14:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:35 smithi049 ceph-mon[112142]: pgmap v526: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:14:37.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:37 smithi144 ceph-mon[86635]: pgmap v527: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:37.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:37 smithi049 ceph-mon[112142]: pgmap v527: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:39.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:39 smithi144 ceph-mon[86635]: pgmap v528: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:39.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:39 smithi049 ceph-mon[112142]: pgmap v528: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:41.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:41 smithi144 ceph-mon[86635]: pgmap v529: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:41.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:41 smithi049 ceph-mon[112142]: pgmap v529: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:43.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:43 smithi144 ceph-mon[86635]: pgmap v530: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:43.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:43 smithi049 ceph-mon[112142]: pgmap v530: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:45.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:45 smithi144 ceph-mon[86635]: pgmap v531: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:45 smithi049 ceph-mon[112142]: pgmap v531: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:47.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:47 smithi144 ceph-mon[86635]: pgmap v532: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:47.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:47 smithi049 ceph-mon[112142]: pgmap v532: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:49.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:49 smithi144 ceph-mon[86635]: pgmap v533: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:49.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:49 smithi049 ceph-mon[112142]: pgmap v533: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:14:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:14:51.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:51 smithi144 ceph-mon[86635]: pgmap v534: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:51.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:51 smithi049 ceph-mon[112142]: pgmap v534: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:53 smithi144 ceph-mon[86635]: pgmap v535: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:53 smithi049 ceph-mon[112142]: pgmap v535: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:55.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:55 smithi144 ceph-mon[86635]: pgmap v536: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:55.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:55 smithi049 ceph-mon[112142]: pgmap v536: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:56.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.751+0000 7fa734bca700 1 -- 172.21.15.49:0/4197774632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 msgr2=0x7fa7301058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:56.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.751+0000 7fa734bca700 1 --2- 172.21.15.49:0/4197774632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 0x7fa7301058e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa724009910 tx=0x7fa724009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:56.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.752+0000 7fa734bca700 1 -- 172.21.15.49:0/4197774632 shutdown_connections 2024-05-15T16:14:56.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.752+0000 7fa734bca700 1 --2- 172.21.15.49:0/4197774632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 0x7fa7301058e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:56.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.752+0000 7fa734bca700 1 --2- 172.21.15.49:0/4197774632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa730100ee0 0x7fa7301012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:56.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.752+0000 7fa734bca700 1 -- 172.21.15.49:0/4197774632 >> 172.21.15.49:0/4197774632 conn(0x7fa7300fc790 msgr2=0x7fa7300febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:56.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.752+0000 7fa734bca700 1 -- 172.21.15.49:0/4197774632 shutdown_connections 2024-05-15T16:14:56.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.752+0000 7fa734bca700 1 -- 172.21.15.49:0/4197774632 wait complete. 2024-05-15T16:14:56.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.752+0000 7fa734bca700 1 Processor -- start 2024-05-15T16:14:56.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa734bca700 1 -- start start 2024-05-15T16:14:56.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa734bca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa730100ee0 0x7fa7300732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:56.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa734bca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 0x7fa7300737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa734bca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa730075270 con 0x7fa730101890 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa734bca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa730073d20 con 0x7fa730100ee0 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa72dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 0x7fa7300737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa72dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 0x7fa7300737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50924/0 (socket says 172.21.15.49:50924) 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa72dd9b700 1 -- 172.21.15.49:0/2153579271 learned_addr learned my addr 172.21.15.49:0/2153579271 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa72e59c700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa730100ee0 0x7fa7300732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa72dd9b700 1 -- 172.21.15.49:0/2153579271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa730100ee0 msgr2=0x7fa7300732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa72dd9b700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa730100ee0 0x7fa7300732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa72dd9b700 1 -- 172.21.15.49:0/2153579271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7240095f0 con 0x7fa730101890 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.753+0000 7fa72dd9b700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 0x7fa7300737e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa7240092e0 tx=0x7fa72400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.754+0000 7fa71f7fe700 1 -- 172.21.15.49:0/2153579271 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa72401d070 con 0x7fa730101890 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.754+0000 7fa71f7fe700 1 -- 172.21.15.49:0/2153579271 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa72400f460 con 0x7fa730101890 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.754+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa730073f70 con 0x7fa730101890 2024-05-15T16:14:56.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.754+0000 7fa71f7fe700 1 -- 172.21.15.49:0/2153579271 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa724017630 con 0x7fa730101890 2024-05-15T16:14:56.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.754+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7301a2f70 con 0x7fa730101890 2024-05-15T16:14:56.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.755+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7301092e0 con 0x7fa730101890 2024-05-15T16:14:56.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.755+0000 7fa71f7fe700 1 -- 172.21.15.49:0/2153579271 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa72400f5d0 con 0x7fa730101890 2024-05-15T16:14:56.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.756+0000 7fa71f7fe700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa71806e200 0x7fa7180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:56.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.756+0000 7fa71f7fe700 1 -- 172.21.15.49:0/2153579271 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa72408f620 con 0x7fa730101890 2024-05-15T16:14:56.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.756+0000 7fa72e59c700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa71806e200 0x7fa7180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:56.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.756+0000 7fa72e59c700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa71806e200 0x7fa7180706c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa720005ea0 tx=0x7fa720005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:56.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.761+0000 7fa71f7fe700 1 -- 172.21.15.49:0/2153579271 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa724058900 con 0x7fa730101890 2024-05-15T16:14:56.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.988+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7300747f0 con 0x7fa71806e200 2024-05-15T16:14:56.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.990+0000 7fa71f7fe700 1 -- 172.21.15.49:0/2153579271 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa7300747f0 con 0x7fa71806e200 2024-05-15T16:14:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa71806e200 msgr2=0x7fa7180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa71806e200 0x7fa7180706c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa720005ea0 tx=0x7fa720005e30 comp rx=0 tx=0).stop 2024-05-15T16:14:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 msgr2=0x7fa7300737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:56.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 0x7fa7300737e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa7240092e0 tx=0x7fa72400be20 comp rx=0 tx=0).stop 2024-05-15T16:14:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 shutdown_connections 2024-05-15T16:14:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa71806e200 0x7fa7180706c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa730100ee0 0x7fa7300732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 --2- 172.21.15.49:0/2153579271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa730101890 0x7fa7300737e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 >> 172.21.15.49:0/2153579271 conn(0x7fa7300fc790 msgr2=0x7fa7300fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 shutdown_connections 2024-05-15T16:14:56.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:56.992+0000 7fa734bca700 1 -- 172.21.15.49:0/2153579271 wait complete. 2024-05-15T16:14:57.004 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:14:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.102+0000 7fb42d353700 1 -- 172.21.15.49:0/2243109231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428100320 msgr2=0x7fb4281007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.102+0000 7fb42d353700 1 --2- 172.21.15.49:0/2243109231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428100320 0x7fb4281007a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009910 tx=0x7fb41c009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.102+0000 7fb42d353700 1 -- 172.21.15.49:0/2243109231 shutdown_connections 2024-05-15T16:14:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.102+0000 7fb42d353700 1 --2- 172.21.15.49:0/2243109231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428100320 0x7fb4281007a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.102+0000 7fb42d353700 1 --2- 172.21.15.49:0/2243109231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb428104bf0 0x7fb428104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.102+0000 7fb42d353700 1 -- 172.21.15.49:0/2243109231 >> 172.21.15.49:0/2243109231 conn(0x7fb428076000 msgr2=0x7fb4280fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.102+0000 7fb42d353700 1 -- 172.21.15.49:0/2243109231 shutdown_connections 2024-05-15T16:14:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.102+0000 7fb42d353700 1 -- 172.21.15.49:0/2243109231 wait complete. 2024-05-15T16:14:57.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.102+0000 7fb42d353700 1 Processor -- start 2024-05-15T16:14:57.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb42d353700 1 -- start start 2024-05-15T16:14:57.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb42d353700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb428100320 0x7fb4280fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb42d353700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428104bf0 0x7fb4280fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb42d353700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280fbec0 con 0x7fb428104bf0 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb42d353700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280fc030 con 0x7fb428100320 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb4267fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428104bf0 0x7fb4280fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb4267fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428104bf0 0x7fb4280fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50944/0 (socket says 172.21.15.49:50944) 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb4267fc700 1 -- 172.21.15.49:0/1400921680 learned_addr learned my addr 172.21.15.49:0/1400921680 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb426ffd700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb428100320 0x7fb4280fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb4267fc700 1 -- 172.21.15.49:0/1400921680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb428100320 msgr2=0x7fb4280fb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb4267fc700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb428100320 0x7fb4280fb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb4267fc700 1 -- 172.21.15.49:0/1400921680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0095f0 con 0x7fb428104bf0 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.103+0000 7fb4267fc700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428104bf0 0x7fb4280fb980 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00bd70 tx=0x7fb41c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:57.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.104+0000 7fb40ffff700 1 -- 172.21.15.49:0/1400921680 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41c01d070 con 0x7fb428104bf0 2024-05-15T16:14:57.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.104+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4280fc2b0 con 0x7fb428104bf0 2024-05-15T16:14:57.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.104+0000 7fb40ffff700 1 -- 172.21.15.49:0/1400921680 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb41c0056e0 con 0x7fb428104bf0 2024-05-15T16:14:57.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.104+0000 7fb40ffff700 1 -- 172.21.15.49:0/1400921680 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41c00fe40 con 0x7fb428104bf0 2024-05-15T16:14:57.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.104+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4281a3090 con 0x7fb428104bf0 2024-05-15T16:14:57.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.105+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42804ec60 con 0x7fb428104bf0 2024-05-15T16:14:57.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.105+0000 7fb40ffff700 1 -- 172.21.15.49:0/1400921680 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb41c00f680 con 0x7fb428104bf0 2024-05-15T16:14:57.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.106+0000 7fb40ffff700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb41006e1a0 0x7fb410070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:57.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.106+0000 7fb40ffff700 1 -- 172.21.15.49:0/1400921680 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb41c08f3e0 con 0x7fb428104bf0 2024-05-15T16:14:57.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.106+0000 7fb426ffd700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb41006e1a0 0x7fb410070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:57.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.106+0000 7fb426ffd700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb41006e1a0 0x7fb410070660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb418005950 tx=0x7fb41800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:57.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.110+0000 7fb40ffff700 1 -- 172.21.15.49:0/1400921680 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb41c0586c0 con 0x7fb428104bf0 2024-05-15T16:14:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.339+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4280fc2b0 con 0x7fb41006e1a0 2024-05-15T16:14:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.340+0000 7fb40ffff700 1 -- 172.21.15.49:0/1400921680 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb4280fc2b0 con 0x7fb41006e1a0 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb41006e1a0 msgr2=0x7fb410070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb41006e1a0 0x7fb410070660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb418005950 tx=0x7fb41800b410 comp rx=0 tx=0).stop 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428104bf0 msgr2=0x7fb4280fb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428104bf0 0x7fb4280fb980 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00bd70 tx=0x7fb41c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 shutdown_connections 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb41006e1a0 0x7fb410070660 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb428100320 0x7fb4280fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 --2- 172.21.15.49:0/1400921680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb428104bf0 0x7fb4280fb980 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 >> 172.21.15.49:0/1400921680 conn(0x7fb428076000 msgr2=0x7fb4280fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 shutdown_connections 2024-05-15T16:14:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.342+0000 7fb42d353700 1 -- 172.21.15.49:0/1400921680 wait complete. 2024-05-15T16:14:57.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.451+0000 7f2d4b826700 1 -- 172.21.15.49:0/1839580996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 msgr2=0x7f2d441047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.451+0000 7f2d4b826700 1 --2- 172.21.15.49:0/1839580996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 0x7f2d441047c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009910 tx=0x7f2d34009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:57.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.451+0000 7f2d4b826700 1 -- 172.21.15.49:0/1839580996 shutdown_connections 2024-05-15T16:14:57.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.451+0000 7f2d4b826700 1 --2- 172.21.15.49:0/1839580996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 0x7f2d441047c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.451+0000 7f2d4b826700 1 --2- 172.21.15.49:0/1839580996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d440ffdc0 0x7f2d441001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.451+0000 7f2d4b826700 1 -- 172.21.15.49:0/1839580996 >> 172.21.15.49:0/1839580996 conn(0x7f2d44076800 msgr2=0x7f2d440fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:57.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.451+0000 7f2d4b826700 1 -- 172.21.15.49:0/1839580996 shutdown_connections 2024-05-15T16:14:57.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.451+0000 7f2d4b826700 1 -- 172.21.15.49:0/1839580996 wait complete. 2024-05-15T16:14:57.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d4b826700 1 Processor -- start 2024-05-15T16:14:57.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d4b826700 1 -- start start 2024-05-15T16:14:57.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d4b826700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d440ffdc0 0x7f2d440fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:57.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d4b826700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 0x7f2d440fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:57.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d4b826700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d440fdcb0 con 0x7f2d44100770 2024-05-15T16:14:57.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d4b826700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d440fc760 con 0x7f2d440ffdc0 2024-05-15T16:14:57.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d48dc1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 0x7f2d440fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d48dc1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 0x7f2d440fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50954/0 (socket says 172.21.15.49:50954) 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d48dc1700 1 -- 172.21.15.49:0/3597527929 learned_addr learned my addr 172.21.15.49:0/3597527929 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d48dc1700 1 -- 172.21.15.49:0/3597527929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d440ffdc0 msgr2=0x7f2d440fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d48dc1700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d440ffdc0 0x7f2d440fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d48dc1700 1 -- 172.21.15.49:0/3597527929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d340095f0 con 0x7f2d44100770 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.452+0000 7f2d48dc1700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 0x7f2d440fc220 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2d340092e0 tx=0x7f2d3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.453+0000 7f2d3a7fc700 1 -- 172.21.15.49:0/3597527929 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d3401d070 con 0x7f2d44100770 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.453+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d440fc9b0 con 0x7f2d44100770 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.453+0000 7f2d3a7fc700 1 -- 172.21.15.49:0/3597527929 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d3400f460 con 0x7f2d44100770 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.453+0000 7f2d3a7fc700 1 -- 172.21.15.49:0/3597527929 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d34017630 con 0x7f2d44100770 2024-05-15T16:14:57.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.453+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d441a2f30 con 0x7f2d44100770 2024-05-15T16:14:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.454+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d441012a0 con 0x7f2d44100770 2024-05-15T16:14:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.454+0000 7f2d3a7fc700 1 -- 172.21.15.49:0/3597527929 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2d3400f5d0 con 0x7f2d44100770 2024-05-15T16:14:57.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.455+0000 7f2d3a7fc700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d3006e200 0x7f2d300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:57.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.455+0000 7f2d495c2700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d3006e200 0x7f2d300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:57.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.455+0000 7f2d3a7fc700 1 -- 172.21.15.49:0/3597527929 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d3408f5f0 con 0x7f2d44100770 2024-05-15T16:14:57.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.455+0000 7f2d495c2700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d3006e200 0x7f2d300706c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d40005fd0 tx=0x7f2d40005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.460+0000 7f2d3a7fc700 1 -- 172.21.15.49:0/3597527929 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2d340588d0 con 0x7f2d44100770 2024-05-15T16:14:57.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.680+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d4410ce70 con 0x7f2d3006e200 2024-05-15T16:14:57.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.691+0000 7f2d3a7fc700 1 -- 172.21.15.49:0/3597527929 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2d4410ce70 con 0x7f2d3006e200 2024-05-15T16:14:57.693 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:14:57.693 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (18m) 11m ago 29m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:14:57.694 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (30m) 11m ago 30m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:14:57.694 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (28m) 11m ago 28m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:14:57.694 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (15m) 11m ago 29m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:14:57.694 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (15m) 11m ago 28m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:14:57.694 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (17m) 11m ago 29m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (11m) 11m ago 24m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (11m) 11m ago 24m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (11m) 11m ago 24m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (24m) 11m ago 24m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (20m) 11m ago 31m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (19m) 11m ago 28m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (16m) 11m ago 31m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (16m) 11m ago 28m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (19m) 11m ago 29m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (18m) 11m ago 28m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (15m) 11m ago 27m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:14:57.695 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (14m) 11m ago 27m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:14:57.696 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (13m) 11m ago 26m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:14:57.696 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (13m) 11m ago 26m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:14:57.696 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (12m) 11m ago 26m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:14:57.696 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (12m) 11m ago 25m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:14:57.696 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (18m) 11m ago 29m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:14:57.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.693+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d3006e200 msgr2=0x7f2d300706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.693+0000 7f2d4b826700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d3006e200 0x7f2d300706c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d40005fd0 tx=0x7f2d40005d00 comp rx=0 tx=0).stop 2024-05-15T16:14:57.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.693+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 msgr2=0x7f2d440fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.693+0000 7f2d4b826700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 0x7f2d440fc220 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2d340092e0 tx=0x7f2d3400be20 comp rx=0 tx=0).stop 2024-05-15T16:14:57.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.693+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 shutdown_connections 2024-05-15T16:14:57.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.693+0000 7f2d4b826700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d3006e200 0x7f2d300706c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.693+0000 7f2d4b826700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d440ffdc0 0x7f2d440fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.693+0000 7f2d4b826700 1 --2- 172.21.15.49:0/3597527929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d44100770 0x7f2d440fc220 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.693+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 >> 172.21.15.49:0/3597527929 conn(0x7f2d44076800 msgr2=0x7f2d44068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:57.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.694+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 shutdown_connections 2024-05-15T16:14:57.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.694+0000 7f2d4b826700 1 -- 172.21.15.49:0/3597527929 wait complete. 2024-05-15T16:14:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.801+0000 7fdb4141a700 1 -- 172.21.15.49:0/3830812648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c100770 msgr2=0x7fdb3c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.801+0000 7fdb4141a700 1 --2- 172.21.15.49:0/3830812648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c100770 0x7fdb3c1047c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009910 tx=0x7fdb24009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.801+0000 7fdb4141a700 1 -- 172.21.15.49:0/3830812648 shutdown_connections 2024-05-15T16:14:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.801+0000 7fdb4141a700 1 --2- 172.21.15.49:0/3830812648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c100770 0x7fdb3c1047c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.801+0000 7fdb4141a700 1 --2- 172.21.15.49:0/3830812648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb3c0ffdc0 0x7fdb3c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.801+0000 7fdb4141a700 1 -- 172.21.15.49:0/3830812648 >> 172.21.15.49:0/3830812648 conn(0x7fdb3c076800 msgr2=0x7fdb3c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.801+0000 7fdb4141a700 1 -- 172.21.15.49:0/3830812648 shutdown_connections 2024-05-15T16:14:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.801+0000 7fdb4141a700 1 -- 172.21.15.49:0/3830812648 wait complete. 2024-05-15T16:14:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb4141a700 1 Processor -- start 2024-05-15T16:14:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb4141a700 1 -- start start 2024-05-15T16:14:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb4141a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c0ffdc0 0x7fdb3c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb4141a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb3c100770 0x7fdb3c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb4141a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c198510 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb4141a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c198650 con 0x7fdb3c100770 2024-05-15T16:14:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb3affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c0ffdc0 0x7fdb3c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb3affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c0ffdc0 0x7fdb3c197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50962/0 (socket says 172.21.15.49:50962) 2024-05-15T16:14:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb3affd700 1 -- 172.21.15.49:0/2416670279 learned_addr learned my addr 172.21.15.49:0/2416670279 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.802+0000 7fdb3a7fc700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb3c100770 0x7fdb3c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.803+0000 7fdb3affd700 1 -- 172.21.15.49:0/2416670279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb3c100770 msgr2=0x7fdb3c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.803+0000 7fdb3affd700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb3c100770 0x7fdb3c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.803+0000 7fdb3affd700 1 -- 172.21.15.49:0/2416670279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb240095f0 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.803+0000 7fdb3affd700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c0ffdc0 0x7fdb3c197a00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00ada0 tx=0x7fdb2c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.803+0000 7fdb33fff700 1 -- 172.21.15.49:0/2416670279 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb2c00f070 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.803+0000 7fdb33fff700 1 -- 172.21.15.49:0/2416670279 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb2c007bf0 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.803+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c1a0a70 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.803+0000 7fdb33fff700 1 -- 172.21.15.49:0/2416670279 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb2c005690 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.803+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c1a0e80 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.804+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c1081c0 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.805+0000 7fdb33fff700 1 -- 172.21.15.49:0/2416670279 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdb2c005900 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.805+0000 7fdb33fff700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdb2806e210 0x7fdb280706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.805+0000 7fdb33fff700 1 -- 172.21.15.49:0/2416670279 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdb2c08dbc0 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.805+0000 7fdb3a7fc700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdb2806e210 0x7fdb280706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.806+0000 7fdb3a7fc700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdb2806e210 0x7fdb280706d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c068e70 tx=0x7fdb2401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:57.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:57.810+0000 7fdb33fff700 1 -- 172.21.15.49:0/2416670279 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdb2c056ea0 con 0x7fdb3c0ffdc0 2024-05-15T16:14:57.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:57 smithi144 ceph-mon[86635]: pgmap v537: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:57.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:57 smithi144 ceph-mon[86635]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:57 smithi049 ceph-mon[112142]: pgmap v537: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:57 smithi049 ceph-mon[112142]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:58.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.112+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdb3c10cd60 con 0x7fdb3c0ffdc0 2024-05-15T16:14:58.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.112+0000 7fdb33fff700 1 -- 172.21.15.49:0/2416670279 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fdb2c05a560 con 0x7fdb3c0ffdc0 2024-05-15T16:14:58.114 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:14:58.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:14:58.114 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:14:58.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:14:58.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:14:58.114 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:14:58.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:14:58.114 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:14:58.115 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:14:58.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdb2806e210 msgr2=0x7fdb280706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdb2806e210 0x7fdb280706d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c068e70 tx=0x7fdb2401a040 comp rx=0 tx=0).stop 2024-05-15T16:14:58.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c0ffdc0 msgr2=0x7fdb3c197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c0ffdc0 0x7fdb3c197a00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00ada0 tx=0x7fdb2c00c330 comp rx=0 tx=0).stop 2024-05-15T16:14:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 shutdown_connections 2024-05-15T16:14:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdb2806e210 0x7fdb280706d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb3c0ffdc0 0x7fdb3c197a00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 --2- 172.21.15.49:0/2416670279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb3c100770 0x7fdb3c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 >> 172.21.15.49:0/2416670279 conn(0x7fdb3c076800 msgr2=0x7fdb3c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 shutdown_connections 2024-05-15T16:14:58.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.115+0000 7fdb4141a700 1 -- 172.21.15.49:0/2416670279 wait complete. 2024-05-15T16:14:58.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.223+0000 7f580dc5f700 1 -- 172.21.15.49:0/3304247983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808100ee0 msgr2=0x7f58081012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.223+0000 7f580dc5f700 1 --2- 172.21.15.49:0/3304247983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808100ee0 0x7f58081012c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009910 tx=0x7f57f8009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:58.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.223+0000 7f580dc5f700 1 -- 172.21.15.49:0/3304247983 shutdown_connections 2024-05-15T16:14:58.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.223+0000 7f580dc5f700 1 --2- 172.21.15.49:0/3304247983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5808101890 0x7f58081058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.223+0000 7f580dc5f700 1 --2- 172.21.15.49:0/3304247983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808100ee0 0x7f58081012c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.223+0000 7f580dc5f700 1 -- 172.21.15.49:0/3304247983 >> 172.21.15.49:0/3304247983 conn(0x7f58080fc790 msgr2=0x7f58080febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:58.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.223+0000 7f580dc5f700 1 -- 172.21.15.49:0/3304247983 shutdown_connections 2024-05-15T16:14:58.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.223+0000 7f580dc5f700 1 -- 172.21.15.49:0/3304247983 wait complete. 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f580dc5f700 1 Processor -- start 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f580dc5f700 1 -- start start 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f580dc5f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5808100ee0 0x7f5808198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f580dc5f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808101890 0x7f5808199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f580dc5f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808199a30 con 0x7f5808101890 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f580dc5f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580819d700 con 0x7f5808100ee0 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f5806ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808101890 0x7f5808199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f5806ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808101890 0x7f5808199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50980/0 (socket says 172.21.15.49:50980) 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f5806ffd700 1 -- 172.21.15.49:0/1111894918 learned_addr learned my addr 172.21.15.49:0/1111894918 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f58077fe700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5808100ee0 0x7f5808198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f5806ffd700 1 -- 172.21.15.49:0/1111894918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5808100ee0 msgr2=0x7f5808198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f5806ffd700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5808100ee0 0x7f5808198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f5806ffd700 1 -- 172.21.15.49:0/1111894918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f80095f0 con 0x7f5808101890 2024-05-15T16:14:58.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.224+0000 7f58077fe700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5808100ee0 0x7f5808198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:14:58.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.225+0000 7f5806ffd700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808101890 0x7f5808199350 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00d900 tx=0x7f57fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:58.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.225+0000 7f5804ff9700 1 -- 172.21.15.49:0/1111894918 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57fc009900 con 0x7f5808101890 2024-05-15T16:14:58.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.225+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580819d9e0 con 0x7f5808101890 2024-05-15T16:14:58.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.225+0000 7f5804ff9700 1 -- 172.21.15.49:0/1111894918 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57fc010460 con 0x7f5808101890 2024-05-15T16:14:58.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.225+0000 7f5804ff9700 1 -- 172.21.15.49:0/1111894918 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57fc00f5f0 con 0x7f5808101890 2024-05-15T16:14:58.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.225+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580819df30 con 0x7f5808101890 2024-05-15T16:14:58.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.226+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5808105060 con 0x7f5808101890 2024-05-15T16:14:58.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.226+0000 7f5804ff9700 1 -- 172.21.15.49:0/1111894918 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f57fc0105d0 con 0x7f5808101890 2024-05-15T16:14:58.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.227+0000 7f5804ff9700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f57f006e210 0x7f57f00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:58.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.227+0000 7f5804ff9700 1 -- 172.21.15.49:0/1111894918 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f57fc08de10 con 0x7f5808101890 2024-05-15T16:14:58.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.227+0000 7f58077fe700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f57f006e210 0x7f57f00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:58.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.227+0000 7f58077fe700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f57f006e210 0x7f57f00706d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57f800b5c0 tx=0x7f57f801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:58.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.232+0000 7f5804ff9700 1 -- 172.21.15.49:0/1111894918 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f57fc0570f0 con 0x7f5808101890 2024-05-15T16:14:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.484+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f58081023c0 con 0x7f5808101890 2024-05-15T16:14:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.485+0000 7f5804ff9700 1 -- 172.21.15.49:0/1111894918 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f57fc05a7b0 con 0x7f5808101890 2024-05-15T16:14:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:14:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:14:58.487 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:14:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:14:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:14:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:14:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:14:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:14:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:14:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:14:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:14:58.488 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:14:58.489 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:14:58.490 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:14:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:14:58.491 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:14:58.491 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:14:58.491 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:14:58.491 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f57f006e210 msgr2=0x7f57f00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f57f006e210 0x7f57f00706d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57f800b5c0 tx=0x7f57f801a040 comp rx=0 tx=0).stop 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808101890 msgr2=0x7f5808199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808101890 0x7f5808199350 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00d900 tx=0x7f57fc00dcc0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 shutdown_connections 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f57f006e210 0x7f57f00706d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5808100ee0 0x7f5808198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 --2- 172.21.15.49:0/1111894918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5808101890 0x7f5808199350 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 >> 172.21.15.49:0/1111894918 conn(0x7f58080fc790 msgr2=0x7f58080fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 shutdown_connections 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.487+0000 7f580dc5f700 1 -- 172.21.15.49:0/1111894918 wait complete. 2024-05-15T16:14:58.492 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:14:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.596+0000 7f6952be7700 1 -- 172.21.15.49:0/951280758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c101890 msgr2=0x7f694c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.596+0000 7f6952be7700 1 --2- 172.21.15.49:0/951280758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c101890 0x7f694c1058e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6938009910 tx=0x7f6938009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.596+0000 7f6952be7700 1 -- 172.21.15.49:0/951280758 shutdown_connections 2024-05-15T16:14:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.596+0000 7f6952be7700 1 --2- 172.21.15.49:0/951280758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c101890 0x7f694c1058e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.596+0000 7f6952be7700 1 --2- 172.21.15.49:0/951280758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f694c100ee0 0x7f694c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.596+0000 7f6952be7700 1 -- 172.21.15.49:0/951280758 >> 172.21.15.49:0/951280758 conn(0x7f694c0fc790 msgr2=0x7f694c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.596+0000 7f6952be7700 1 -- 172.21.15.49:0/951280758 shutdown_connections 2024-05-15T16:14:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.597+0000 7f6952be7700 1 -- 172.21.15.49:0/951280758 wait complete. 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.597+0000 7f6952be7700 1 Processor -- start 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.597+0000 7f6952be7700 1 -- start start 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.597+0000 7f6952be7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c100ee0 0x7f694c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.597+0000 7f6952be7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f694c101890 0x7f694c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.597+0000 7f6952be7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c073d20 con 0x7f694c100ee0 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.597+0000 7f6952be7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c073e60 con 0x7f694c101890 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.597+0000 7f6950983700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c100ee0 0x7f694c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6950983700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c100ee0 0x7f694c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50990/0 (socket says 172.21.15.49:50990) 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6950983700 1 -- 172.21.15.49:0/4012570679 learned_addr learned my addr 172.21.15.49:0/4012570679 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f694bfff700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f694c101890 0x7f694c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6950983700 1 -- 172.21.15.49:0/4012570679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f694c101890 msgr2=0x7f694c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6950983700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f694c101890 0x7f694c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6950983700 1 -- 172.21.15.49:0/4012570679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69380095f0 con 0x7f694c100ee0 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6950983700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c100ee0 0x7f694c0732a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f694000d8d0 tx=0x7f694000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6949ffb700 1 -- 172.21.15.49:0/4012570679 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69400098a0 con 0x7f694c100ee0 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6949ffb700 1 -- 172.21.15.49:0/4012570679 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6940010460 con 0x7f694c100ee0 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694c1a2da0 con 0x7f694c100ee0 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6949ffb700 1 -- 172.21.15.49:0/4012570679 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694000f5f0 con 0x7f694c100ee0 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.598+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c1a3100 con 0x7f694c100ee0 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.599+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694c1092e0 con 0x7f694c100ee0 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.600+0000 7f6949ffb700 1 -- 172.21.15.49:0/4012570679 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6940009a00 con 0x7f694c100ee0 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.600+0000 7f6949ffb700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f693c06e200 0x7f693c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.600+0000 7f6949ffb700 1 -- 172.21.15.49:0/4012570679 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f694008e030 con 0x7f694c100ee0 2024-05-15T16:14:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.600+0000 7f694bfff700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f693c06e200 0x7f693c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.601+0000 7f694bfff700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f693c06e200 0x7f693c0706c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f694c0749d0 tx=0x7f693801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:58.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.605+0000 7f6949ffb700 1 -- 172.21.15.49:0/4012570679 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6940057310 con 0x7f694c100ee0 2024-05-15T16:14:58.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.832+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f694c10de80 con 0x7f693c06e200 2024-05-15T16:14:58.834 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:58 smithi049 ceph-mon[112142]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:58.834 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:58 smithi049 ceph-mon[112142]: from='client.35026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:58.834 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:58 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2416670279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:14:58.834 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:58 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1111894918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.834+0000 7f6949ffb700 1 -- 172.21.15.49:0/4012570679 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f694c10de80 con 0x7f693c06e200 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:14:58.835 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:14:58.836 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:14:58.836 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:14:58.836 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:14:58.836 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:14:58.836 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:14:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f693c06e200 msgr2=0x7f693c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f693c06e200 0x7f693c0706c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f694c0749d0 tx=0x7f693801a040 comp rx=0 tx=0).stop 2024-05-15T16:14:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c100ee0 msgr2=0x7f694c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c100ee0 0x7f694c0732a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f694000d8d0 tx=0x7f694000dbe0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 shutdown_connections 2024-05-15T16:14:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f693c06e200 0x7f693c0706c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f694c100ee0 0x7f694c0732a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 --2- 172.21.15.49:0/4012570679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f694c101890 0x7f694c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 >> 172.21.15.49:0/4012570679 conn(0x7f694c0fc790 msgr2=0x7f694c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 shutdown_connections 2024-05-15T16:14:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.836+0000 7f6952be7700 1 -- 172.21.15.49:0/4012570679 wait complete. 2024-05-15T16:14:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.944+0000 7f0a5d658700 1 -- 172.21.15.49:0/1267257060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580ff490 msgr2=0x7f0a580ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.944+0000 7f0a5d658700 1 --2- 172.21.15.49:0/1267257060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580ff490 0x7f0a580ff8f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009910 tx=0x7f0a4c009c20 comp rx=0 tx=0).stop 2024-05-15T16:14:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.944+0000 7f0a5d658700 1 -- 172.21.15.49:0/1267257060 shutdown_connections 2024-05-15T16:14:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.944+0000 7f0a5d658700 1 --2- 172.21.15.49:0/1267257060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580ff490 0x7f0a580ff8f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.944+0000 7f0a5d658700 1 --2- 172.21.15.49:0/1267257060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a58105c90 0x7f0a58106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.944+0000 7f0a5d658700 1 -- 172.21.15.49:0/1267257060 >> 172.21.15.49:0/1267257060 conn(0x7f0a58074e70 msgr2=0x7f0a58075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.944+0000 7f0a5d658700 1 -- 172.21.15.49:0/1267257060 shutdown_connections 2024-05-15T16:14:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.945+0000 7f0a5d658700 1 -- 172.21.15.49:0/1267257060 wait complete. 2024-05-15T16:14:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.945+0000 7f0a5d658700 1 Processor -- start 2024-05-15T16:14:58.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.945+0000 7f0a5d658700 1 -- start start 2024-05-15T16:14:58.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a5d658700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580ff490 0x7f0a58100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a5d658700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a58105c90 0x7f0a58100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a5d658700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a581010f0 con 0x7f0a58105c90 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a5d658700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a58101230 con 0x7f0a580ff490 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a567fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a58105c90 0x7f0a58100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a567fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a58105c90 0x7f0a58100bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51004/0 (socket says 172.21.15.49:51004) 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a567fc700 1 -- 172.21.15.49:0/3369782666 learned_addr learned my addr 172.21.15.49:0/3369782666 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a56ffd700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580ff490 0x7f0a58100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a567fc700 1 -- 172.21.15.49:0/3369782666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580ff490 msgr2=0x7f0a58100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a567fc700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580ff490 0x7f0a58100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a567fc700 1 -- 172.21.15.49:0/3369782666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c0095f0 con 0x7f0a58105c90 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.946+0000 7f0a567fc700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a58105c90 0x7f0a58100bb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00bd70 tx=0x7f0a4c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.947+0000 7f0a3ffff700 1 -- 172.21.15.49:0/3369782666 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a4c01d070 con 0x7f0a58105c90 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.947+0000 7f0a3ffff700 1 -- 172.21.15.49:0/3369782666 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a4c0056e0 con 0x7f0a58105c90 2024-05-15T16:14:58.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.947+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a581014b0 con 0x7f0a58105c90 2024-05-15T16:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.947+0000 7f0a3ffff700 1 -- 172.21.15.49:0/3369782666 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a4c00fe40 con 0x7f0a58105c90 2024-05-15T16:14:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.947+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a581a3060 con 0x7f0a58105c90 2024-05-15T16:14:58.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:58 smithi144 ceph-mon[86635]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:58.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:58 smithi144 ceph-mon[86635]: from='client.35026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:58.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:58 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2416670279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:14:58.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:58 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1111894918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.948+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a581092b0 con 0x7f0a58105c90 2024-05-15T16:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.948+0000 7f0a3ffff700 1 -- 172.21.15.49:0/3369782666 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0a4c00f680 con 0x7f0a58105c90 2024-05-15T16:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.949+0000 7f0a3ffff700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a4006e200 0x7f0a400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.949+0000 7f0a3ffff700 1 -- 172.21.15.49:0/3369782666 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0a4c08f4d0 con 0x7f0a58105c90 2024-05-15T16:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.949+0000 7f0a56ffd700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a4006e200 0x7f0a400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:14:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.949+0000 7f0a56ffd700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a4006e200 0x7f0a400706c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0a5806a820 tx=0x7f0a4800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:14:58.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:58.953+0000 7f0a3ffff700 1 -- 172.21.15.49:0/3369782666 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0a4c058830 con 0x7f0a58105c90 2024-05-15T16:14:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.245+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0a58068e00 con 0x7f0a58105c90 2024-05-15T16:14:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.245+0000 7f0a3ffff700 1 -- 172.21.15.49:0/3369782666 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f0a4c05bef0 con 0x7f0a58105c90 2024-05-15T16:14:59.247 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:14:59.247 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:14:59.247 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:14:59.247 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:14:59.247 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:14:59.247 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:14:59.247 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a4006e200 msgr2=0x7f0a400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a4006e200 0x7f0a400706c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0a5806a820 tx=0x7f0a4800b410 comp rx=0 tx=0).stop 2024-05-15T16:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a58105c90 msgr2=0x7f0a58100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a58105c90 0x7f0a58100bb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00bd70 tx=0x7f0a4c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 shutdown_connections 2024-05-15T16:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a4006e200 0x7f0a400706c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580ff490 0x7f0a58100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 --2- 172.21.15.49:0/3369782666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a58105c90 0x7f0a58100bb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:14:59.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 >> 172.21.15.49:0/3369782666 conn(0x7f0a58074e70 msgr2=0x7f0a580fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:14:59.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 shutdown_connections 2024-05-15T16:14:59.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:14:59.248+0000 7f0a5d658700 1 -- 172.21.15.49:0/3369782666 wait complete. 2024-05-15T16:14:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:59 smithi144 ceph-mon[86635]: pgmap v538: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:59 smithi144 ceph-mon[86635]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:14:59 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3369782666' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:14:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:59 smithi049 ceph-mon[112142]: pgmap v538: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:14:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:59 smithi049 ceph-mon[112142]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:14:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:14:59 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3369782666' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:15:01.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:01 smithi144 ceph-mon[86635]: pgmap v539: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:01.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:01 smithi049 ceph-mon[112142]: pgmap v539: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:03.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:03 smithi144 ceph-mon[86635]: pgmap v540: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:03.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:03 smithi049 ceph-mon[112142]: pgmap v540: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:05 smithi144 ceph-mon[86635]: pgmap v541: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:15:05.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:05 smithi049 ceph-mon[112142]: pgmap v541: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:05.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:15:07.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:07 smithi144 ceph-mon[86635]: pgmap v542: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:07.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:07 smithi049 ceph-mon[112142]: pgmap v542: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:09.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:09 smithi144 ceph-mon[86635]: pgmap v543: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:09.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:09 smithi049 ceph-mon[112142]: pgmap v543: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:11.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:11 smithi144 ceph-mon[86635]: pgmap v544: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:11 smithi049 ceph-mon[112142]: pgmap v544: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:13.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:13 smithi144 ceph-mon[86635]: pgmap v545: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:13.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:13 smithi049 ceph-mon[112142]: pgmap v545: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:15.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:15 smithi144 ceph-mon[86635]: pgmap v546: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:15.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:15 smithi049 ceph-mon[112142]: pgmap v546: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:17.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:17 smithi144 ceph-mon[86635]: pgmap v547: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:17.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:17 smithi049 ceph-mon[112142]: pgmap v547: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:19 smithi144 ceph-mon[86635]: pgmap v548: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:19.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:19 smithi049 ceph-mon[112142]: pgmap v548: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:15:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:15:21.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:21 smithi144 ceph-mon[86635]: pgmap v549: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:21.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:21 smithi049 ceph-mon[112142]: pgmap v549: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:23.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:23 smithi144 ceph-mon[86635]: pgmap v550: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:23.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:23 smithi049 ceph-mon[112142]: pgmap v550: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:25.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:25 smithi144 ceph-mon[86635]: pgmap v551: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:25.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:25 smithi049 ceph-mon[112142]: pgmap v551: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:27.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:27 smithi144 ceph-mon[86635]: pgmap v552: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:27 smithi049 ceph-mon[112142]: pgmap v552: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:29.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.371+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/644116573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c101e20 msgr2=0x7fb19c102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:29.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.371+0000 7fb1a2aaa700 1 --2- 172.21.15.49:0/644116573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c101e20 0x7fb19c102200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b0a0 tx=0x7fb18c00b3b0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.371+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/644116573 shutdown_connections 2024-05-15T16:15:29.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.371+0000 7fb1a2aaa700 1 --2- 172.21.15.49:0/644116573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c102740 0x7fb19c10ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.371+0000 7fb1a2aaa700 1 --2- 172.21.15.49:0/644116573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c101e20 0x7fb19c102200 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.371+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/644116573 >> 172.21.15.49:0/644116573 conn(0x7fb19c076800 msgr2=0x7fb19c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.371+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/644116573 shutdown_connections 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.371+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/644116573 wait complete. 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.372+0000 7fb1a2aaa700 1 Processor -- start 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.372+0000 7fb1a2aaa700 1 -- start start 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.372+0000 7fb1a2aaa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c101e20 0x7fb19c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.372+0000 7fb1a2aaa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c102740 0x7fb19c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.372+0000 7fb1a2aaa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c199a30 con 0x7fb19c102740 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.372+0000 7fb1a2aaa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c199b70 con 0x7fb19c101e20 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.372+0000 7fb19bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c102740 0x7fb19c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.372+0000 7fb19bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c102740 0x7fb19c1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59848/0 (socket says 172.21.15.49:59848) 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.372+0000 7fb19bfff700 1 -- 172.21.15.49:0/2350101162 learned_addr learned my addr 172.21.15.49:0/2350101162 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb1a0846700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c101e20 0x7fb19c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb19bfff700 1 -- 172.21.15.49:0/2350101162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c101e20 msgr2=0x7fb19c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:29.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb19bfff700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c101e20 0x7fb19c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb19bfff700 1 -- 172.21.15.49:0/2350101162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c009d00 con 0x7fb19c102740 2024-05-15T16:15:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb1a0846700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c101e20 0x7fb19c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:15:29.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb19bfff700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c102740 0x7fb19c1993e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb18400c8f0 tx=0x7fb18400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb199ffb700 1 -- 172.21.15.49:0/2350101162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb184007ad0 con 0x7fb19c102740 2024-05-15T16:15:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb199ffb700 1 -- 172.21.15.49:0/2350101162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb18400ce90 con 0x7fb19c102740 2024-05-15T16:15:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c19d900 con 0x7fb19c102740 2024-05-15T16:15:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb199ffb700 1 -- 172.21.15.49:0/2350101162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb184018630 con 0x7fb19c102740 2024-05-15T16:15:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.373+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c19de50 con 0x7fb19c102740 2024-05-15T16:15:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.374+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c068f60 con 0x7fb19c102740 2024-05-15T16:15:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.375+0000 7fb199ffb700 1 -- 172.21.15.49:0/2350101162 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb184007c30 con 0x7fb19c102740 2024-05-15T16:15:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.375+0000 7fb199ffb700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb18806e210 0x7fb1880706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.375+0000 7fb199ffb700 1 -- 172.21.15.49:0/2350101162 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb18408e140 con 0x7fb19c102740 2024-05-15T16:15:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.376+0000 7fb1a0846700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb18806e210 0x7fb1880706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.376+0000 7fb1a0846700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb18806e210 0x7fb1880706d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b830 tx=0x7fb18c006210 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:29.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.380+0000 7fb199ffb700 1 -- 172.21.15.49:0/2350101162 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb184057420 con 0x7fb19c102740 2024-05-15T16:15:29.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.607+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb19c10dea0 con 0x7fb18806e210 2024-05-15T16:15:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.608+0000 7fb199ffb700 1 -- 172.21.15.49:0/2350101162 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb19c10dea0 con 0x7fb18806e210 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb18806e210 msgr2=0x7fb1880706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb18806e210 0x7fb1880706d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b830 tx=0x7fb18c006210 comp rx=0 tx=0).stop 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c102740 msgr2=0x7fb19c1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c102740 0x7fb19c1993e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb18400c8f0 tx=0x7fb18400ccb0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 shutdown_connections 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb18806e210 0x7fb1880706d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c101e20 0x7fb19c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 --2- 172.21.15.49:0/2350101162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb19c102740 0x7fb19c1993e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 >> 172.21.15.49:0/2350101162 conn(0x7fb19c076800 msgr2=0x7fb19c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.610+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 shutdown_connections 2024-05-15T16:15:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.611+0000 7fb1a2aaa700 1 -- 172.21.15.49:0/2350101162 wait complete. 2024-05-15T16:15:29.622 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:15:29.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.720+0000 7f39eed72700 1 -- 172.21.15.49:0/1376720134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e81007c0 msgr2=0x7f39e8104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:29.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.720+0000 7f39eed72700 1 --2- 172.21.15.49:0/1376720134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e81007c0 0x7f39e8104810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009910 tx=0x7f39d8009c20 comp rx=0 tx=0).stop 2024-05-15T16:15:29.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.720+0000 7f39eed72700 1 -- 172.21.15.49:0/1376720134 shutdown_connections 2024-05-15T16:15:29.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.720+0000 7f39eed72700 1 --2- 172.21.15.49:0/1376720134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e81007c0 0x7f39e8104810 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.720+0000 7f39eed72700 1 --2- 172.21.15.49:0/1376720134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e80ffe10 0x7f39e81001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.720+0000 7f39eed72700 1 -- 172.21.15.49:0/1376720134 >> 172.21.15.49:0/1376720134 conn(0x7f39e80fb700 msgr2=0x7f39e80fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:29.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.721+0000 7f39eed72700 1 -- 172.21.15.49:0/1376720134 shutdown_connections 2024-05-15T16:15:29.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.721+0000 7f39eed72700 1 -- 172.21.15.49:0/1376720134 wait complete. 2024-05-15T16:15:29.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.721+0000 7f39eed72700 1 Processor -- start 2024-05-15T16:15:29.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.721+0000 7f39eed72700 1 -- start start 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39eed72700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e80ffe10 0x7f39e819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39eed72700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e81007c0 0x7f39e819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39eed72700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e819d490 con 0x7f39e80ffe10 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39eed72700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e8196a70 con 0x7f39e81007c0 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39ecb0e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e80ffe10 0x7f39e819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39ecb0e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e80ffe10 0x7f39e819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59864/0 (socket says 172.21.15.49:59864) 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39ecb0e700 1 -- 172.21.15.49:0/336389050 learned_addr learned my addr 172.21.15.49:0/336389050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39e7fff700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e81007c0 0x7f39e819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39ecb0e700 1 -- 172.21.15.49:0/336389050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e81007c0 msgr2=0x7f39e819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39ecb0e700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e81007c0 0x7f39e819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39ecb0e700 1 -- 172.21.15.49:0/336389050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d80095f0 con 0x7f39e80ffe10 2024-05-15T16:15:29.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39e7fff700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e81007c0 0x7f39e819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:15:29.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39ecb0e700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e80ffe10 0x7f39e819c930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c8a0 tx=0x7f39d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:29.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39e5ffb700 1 -- 172.21.15.49:0/336389050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39d000cea0 con 0x7f39e80ffe10 2024-05-15T16:15:29.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39e5ffb700 1 -- 172.21.15.49:0/336389050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39d0004830 con 0x7f39e80ffe10 2024-05-15T16:15:29.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e8196d50 con 0x7f39e80ffe10 2024-05-15T16:15:29.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.722+0000 7f39e5ffb700 1 -- 172.21.15.49:0/336389050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39d00055b0 con 0x7f39e80ffe10 2024-05-15T16:15:29.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.723+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e81972a0 con 0x7f39e80ffe10 2024-05-15T16:15:29.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.723+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e8108210 con 0x7f39e80ffe10 2024-05-15T16:15:29.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.724+0000 7f39e5ffb700 1 -- 172.21.15.49:0/336389050 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f39d00049a0 con 0x7f39e80ffe10 2024-05-15T16:15:29.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.725+0000 7f39e5ffb700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39d406e200 0x7f39d40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:29.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.725+0000 7f39e7fff700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39d406e200 0x7f39d40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:29.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.725+0000 7f39e5ffb700 1 -- 172.21.15.49:0/336389050 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f39d008de40 con 0x7f39e80ffe10 2024-05-15T16:15:29.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.725+0000 7f39e7fff700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39d406e200 0x7f39d40706c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006010 tx=0x7f39d800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.729+0000 7f39e5ffb700 1 -- 172.21.15.49:0/336389050 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f39d005ac50 con 0x7f39e80ffe10 2024-05-15T16:15:29.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:29 smithi144 ceph-mon[86635]: pgmap v553: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:29.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:29 smithi049 ceph-mon[112142]: pgmap v553: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:29.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.954+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f39e8197fd0 con 0x7f39d406e200 2024-05-15T16:15:29.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.955+0000 7f39e5ffb700 1 -- 172.21.15.49:0/336389050 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f39e8197fd0 con 0x7f39d406e200 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39d406e200 msgr2=0x7f39d40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39d406e200 0x7f39d40706c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006010 tx=0x7f39d800b540 comp rx=0 tx=0).stop 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e80ffe10 msgr2=0x7f39e819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e80ffe10 0x7f39e819c930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c8a0 tx=0x7f39d000cc60 comp rx=0 tx=0).stop 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 shutdown_connections 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f39d406e200 0x7f39d40706c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e80ffe10 0x7f39e819c930 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 --2- 172.21.15.49:0/336389050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39e81007c0 0x7f39e819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 >> 172.21.15.49:0/336389050 conn(0x7f39e80fb700 msgr2=0x7f39e80fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 shutdown_connections 2024-05-15T16:15:29.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:29.957+0000 7f39eed72700 1 -- 172.21.15.49:0/336389050 wait complete. 2024-05-15T16:15:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.065+0000 7f5796fca700 1 -- 172.21.15.49:0/2068733160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5790101eb0 msgr2=0x7f5790102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.065+0000 7f5796fca700 1 --2- 172.21.15.49:0/2068733160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5790101eb0 0x7f5790102290 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5784009940 tx=0x7f5784009c50 comp rx=0 tx=0).stop 2024-05-15T16:15:30.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.065+0000 7f5796fca700 1 -- 172.21.15.49:0/2068733160 shutdown_connections 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.066+0000 7f5796fca700 1 --2- 172.21.15.49:0/2068733160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57901027d0 0x7f579010acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.066+0000 7f5796fca700 1 --2- 172.21.15.49:0/2068733160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5790101eb0 0x7f5790102290 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.066+0000 7f5796fca700 1 -- 172.21.15.49:0/2068733160 >> 172.21.15.49:0/2068733160 conn(0x7f57900fb6c0 msgr2=0x7f57900fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.066+0000 7f5796fca700 1 -- 172.21.15.49:0/2068733160 shutdown_connections 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.066+0000 7f5796fca700 1 -- 172.21.15.49:0/2068733160 wait complete. 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.066+0000 7f5796fca700 1 Processor -- start 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.066+0000 7f5796fca700 1 -- start start 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.066+0000 7f5796fca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5790101eb0 0x7f579019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.066+0000 7f5796fca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57901027d0 0x7f579019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f5796fca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579019d430 con 0x7f57901027d0 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f5796fca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790196a10 con 0x7f5790101eb0 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57901027d0 0x7f579019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57901027d0 0x7f579019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59888/0 (socket says 172.21.15.49:59888) 2024-05-15T16:15:30.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578ffff700 1 -- 172.21.15.49:0/3405244817 learned_addr learned my addr 172.21.15.49:0/3405244817 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:15:30.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f5794d66700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5790101eb0 0x7f579019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:30.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578ffff700 1 -- 172.21.15.49:0/3405244817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5790101eb0 msgr2=0x7f579019c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578ffff700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5790101eb0 0x7f579019c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578ffff700 1 -- 172.21.15.49:0/3405244817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57840095f0 con 0x7f57901027d0 2024-05-15T16:15:30.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578ffff700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57901027d0 0x7f579019ce10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f577c00d900 tx=0x7f577c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578dffb700 1 -- 172.21.15.49:0/3405244817 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f577c009900 con 0x7f57901027d0 2024-05-15T16:15:30.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578dffb700 1 -- 172.21.15.49:0/3405244817 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f577c010460 con 0x7f57901027d0 2024-05-15T16:15:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5790196cf0 con 0x7f57901027d0 2024-05-15T16:15:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.067+0000 7f578dffb700 1 -- 172.21.15.49:0/3405244817 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f577c00f5f0 con 0x7f57901027d0 2024-05-15T16:15:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.068+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5790197240 con 0x7f57901027d0 2024-05-15T16:15:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.069+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5790105560 con 0x7f57901027d0 2024-05-15T16:15:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.069+0000 7f578dffb700 1 -- 172.21.15.49:0/3405244817 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f577c0105d0 con 0x7f57901027d0 2024-05-15T16:15:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.069+0000 7f578dffb700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f578006e210 0x7f57800706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.070+0000 7f578dffb700 1 -- 172.21.15.49:0/3405244817 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f577c08dc40 con 0x7f57901027d0 2024-05-15T16:15:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.070+0000 7f5794d66700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f578006e210 0x7f57800706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:30.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.070+0000 7f5794d66700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f578006e210 0x7f57800706d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5784006010 tx=0x7f5784005ad0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:30.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.074+0000 7f578dffb700 1 -- 172.21.15.49:0/3405244817 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f577c05aad0 con 0x7f57901027d0 2024-05-15T16:15:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.291+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5790197f40 con 0x7f578006e210 2024-05-15T16:15:30.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.303+0000 7f578dffb700 1 -- 172.21.15.49:0/3405244817 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5790197f40 con 0x7f578006e210 2024-05-15T16:15:30.304 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:15:30.304 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (18m) 11m ago 30m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:15:30.304 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (30m) 11m ago 30m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:15:30.304 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (29m) 11m ago 29m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:15:30.304 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (16m) 11m ago 30m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (16m) 11m ago 29m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (18m) 11m ago 30m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (12m) 11m ago 24m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (12m) 11m ago 24m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (11m) 11m ago 24m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (24m) 11m ago 24m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (21m) 11m ago 31m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (19m) 11m ago 29m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (17m) 11m ago 31m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (16m) 11m ago 29m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (19m) 11m ago 30m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (19m) 11m ago 29m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (15m) 11m ago 28m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (15m) 11m ago 27m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:15:30.305 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (14m) 11m ago 27m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:15:30.306 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (13m) 11m ago 27m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:15:30.306 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (13m) 11m ago 26m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:15:30.306 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (12m) 11m ago 26m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:15:30.306 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (19m) 11m ago 29m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f578006e210 msgr2=0x7f57800706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f578006e210 0x7f57800706d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5784006010 tx=0x7f5784005ad0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57901027d0 msgr2=0x7f579019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57901027d0 0x7f579019ce10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f577c00d900 tx=0x7f577c00dc10 comp rx=0 tx=0).stop 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 shutdown_connections 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f578006e210 0x7f57800706d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5790101eb0 0x7f579019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 --2- 172.21.15.49:0/3405244817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57901027d0 0x7f579019ce10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 >> 172.21.15.49:0/3405244817 conn(0x7f57900fb6c0 msgr2=0x7f57901031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 shutdown_connections 2024-05-15T16:15:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.305+0000 7f5796fca700 1 -- 172.21.15.49:0/3405244817 wait complete. 2024-05-15T16:15:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.413+0000 7f377515c700 1 -- 172.21.15.49:0/827968121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770100770 msgr2=0x7f37701047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.413+0000 7f377515c700 1 --2- 172.21.15.49:0/827968121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770100770 0x7f37701047c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3764009910 tx=0x7f3764009c20 comp rx=0 tx=0).stop 2024-05-15T16:15:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.413+0000 7f377515c700 1 -- 172.21.15.49:0/827968121 shutdown_connections 2024-05-15T16:15:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.413+0000 7f377515c700 1 --2- 172.21.15.49:0/827968121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3770100770 0x7f37701047c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.413+0000 7f377515c700 1 --2- 172.21.15.49:0/827968121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37700ffdc0 0x7f37701001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.413+0000 7f377515c700 1 -- 172.21.15.49:0/827968121 >> 172.21.15.49:0/827968121 conn(0x7f3770076800 msgr2=0x7f37700fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:30.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.414+0000 7f377515c700 1 -- 172.21.15.49:0/827968121 shutdown_connections 2024-05-15T16:15:30.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.414+0000 7f377515c700 1 -- 172.21.15.49:0/827968121 wait complete. 2024-05-15T16:15:30.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.414+0000 7f377515c700 1 Processor -- start 2024-05-15T16:15:30.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f377515c700 1 -- start start 2024-05-15T16:15:30.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f377515c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37700ffdc0 0x7f3770197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:30.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f377515c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3770100770 0x7f3770197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:30.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f377515c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770198480 con 0x7f37700ffdc0 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f377515c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37701985c0 con 0x7f3770100770 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f376ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37700ffdc0 0x7f3770197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f376ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37700ffdc0 0x7f3770197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59896/0 (socket says 172.21.15.49:59896) 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f376ed9d700 1 -- 172.21.15.49:0/599938297 learned_addr learned my addr 172.21.15.49:0/599938297 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f376e59c700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3770100770 0x7f3770197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f376ed9d700 1 -- 172.21.15.49:0/599938297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3770100770 msgr2=0x7f3770197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f376ed9d700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3770100770 0x7f3770197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f376ed9d700 1 -- 172.21.15.49:0/599938297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37640095f0 con 0x7f37700ffdc0 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.415+0000 7f376e59c700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3770100770 0x7f3770197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.416+0000 7f376ed9d700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37700ffdc0 0x7f3770197a00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f376000c930 tx=0x7f376000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.416+0000 7f3757fff700 1 -- 172.21.15.49:0/599938297 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3760007ad0 con 0x7f37700ffdc0 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.416+0000 7f3757fff700 1 -- 172.21.15.49:0/599938297 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3760007c30 con 0x7f37700ffdc0 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.416+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37701a0a70 con 0x7f37700ffdc0 2024-05-15T16:15:30.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.416+0000 7f3757fff700 1 -- 172.21.15.49:0/599938297 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3760018610 con 0x7f37700ffdc0 2024-05-15T16:15:30.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.416+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37701a0e20 con 0x7f37700ffdc0 2024-05-15T16:15:30.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.417+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37701081c0 con 0x7f37700ffdc0 2024-05-15T16:15:30.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.418+0000 7f3757fff700 1 -- 172.21.15.49:0/599938297 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f376000f450 con 0x7f37700ffdc0 2024-05-15T16:15:30.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.418+0000 7f3757fff700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f375806e200 0x7f37580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:30.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.418+0000 7f3757fff700 1 -- 172.21.15.49:0/599938297 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f376008e3a0 con 0x7f37700ffdc0 2024-05-15T16:15:30.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.418+0000 7f376e59c700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f375806e200 0x7f37580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.419+0000 7f376e59c700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f375806e200 0x7f37580706c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3764005f50 tx=0x7f3764005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:30.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.423+0000 7f3757fff700 1 -- 172.21.15.49:0/599938297 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3760057680 con 0x7f37700ffdc0 2024-05-15T16:15:30.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.724+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3770068c50 con 0x7f37700ffdc0 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.725+0000 7f3757fff700 1 -- 172.21.15.49:0/599938297 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f376005ad40 con 0x7f37700ffdc0 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:15:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:15:30.728 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:15:30.728 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:15:30.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:15:30.728 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:15:30.728 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:15:30.728 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:15:30.728 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:15:30.728 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.727+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f375806e200 msgr2=0x7f37580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.727+0000 7f377515c700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f375806e200 0x7f37580706c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3764005f50 tx=0x7f3764005dc0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.727+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37700ffdc0 msgr2=0x7f3770197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.727+0000 7f377515c700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37700ffdc0 0x7f3770197a00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f376000c930 tx=0x7f376000ccf0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.727+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 shutdown_connections 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.727+0000 7f377515c700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f375806e200 0x7f37580706c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.727+0000 7f377515c700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37700ffdc0 0x7f3770197a00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.727+0000 7f377515c700 1 --2- 172.21.15.49:0/599938297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3770100770 0x7f3770197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.727+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 >> 172.21.15.49:0/599938297 conn(0x7f3770076800 msgr2=0x7f3770103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.728+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 shutdown_connections 2024-05-15T16:15:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.728+0000 7f377515c700 1 -- 172.21.15.49:0/599938297 wait complete. 2024-05-15T16:15:30.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.836+0000 7f3e820c4700 1 -- 172.21.15.49:0/645086335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c101860 msgr2=0x7f3e7c1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.836+0000 7f3e820c4700 1 --2- 172.21.15.49:0/645086335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c101860 0x7f3e7c1058b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009910 tx=0x7f3e70009c20 comp rx=0 tx=0).stop 2024-05-15T16:15:30.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.836+0000 7f3e820c4700 1 -- 172.21.15.49:0/645086335 shutdown_connections 2024-05-15T16:15:30.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.836+0000 7f3e820c4700 1 --2- 172.21.15.49:0/645086335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c101860 0x7f3e7c1058b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.836+0000 7f3e820c4700 1 --2- 172.21.15.49:0/645086335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e7c100eb0 0x7f3e7c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.836+0000 7f3e820c4700 1 -- 172.21.15.49:0/645086335 >> 172.21.15.49:0/645086335 conn(0x7f3e7c0fc760 msgr2=0x7f3e7c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:30.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.836+0000 7f3e820c4700 1 -- 172.21.15.49:0/645086335 shutdown_connections 2024-05-15T16:15:30.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.836+0000 7f3e820c4700 1 -- 172.21.15.49:0/645086335 wait complete. 2024-05-15T16:15:30.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.836+0000 7f3e820c4700 1 Processor -- start 2024-05-15T16:15:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e820c4700 1 -- start start 2024-05-15T16:15:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e820c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c100eb0 0x7f3e7c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e820c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e7c101860 0x7f3e7c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e820c4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c074f90 con 0x7f3e7c100eb0 2024-05-15T16:15:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e820c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c075100 con 0x7f3e7c101860 2024-05-15T16:15:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e7b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c100eb0 0x7f3e7c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e7b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c100eb0 0x7f3e7c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59920/0 (socket says 172.21.15.49:59920) 2024-05-15T16:15:30.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e7b7fe700 1 -- 172.21.15.49:0/431301136 learned_addr learned my addr 172.21.15.49:0/431301136 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e7affd700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e7c101860 0x7f3e7c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e7b7fe700 1 -- 172.21.15.49:0/431301136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e7c101860 msgr2=0x7f3e7c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e7b7fe700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e7c101860 0x7f3e7c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.837+0000 7f3e7b7fe700 1 -- 172.21.15.49:0/431301136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700095f0 con 0x7f3e7c100eb0 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.838+0000 7f3e7b7fe700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c100eb0 0x7f3e7c0732a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00b4b0 tx=0x7f3e6c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.838+0000 7f3e78ff9700 1 -- 172.21.15.49:0/431301136 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e6c011800 con 0x7f3e7c100eb0 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.838+0000 7f3e78ff9700 1 -- 172.21.15.49:0/431301136 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e6c011e20 con 0x7f3e7c100eb0 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.838+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c073d20 con 0x7f3e7c100eb0 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.838+0000 7f3e78ff9700 1 -- 172.21.15.49:0/431301136 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e6c00f2e0 con 0x7f3e7c100eb0 2024-05-15T16:15:30.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.838+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c1a7640 con 0x7f3e7c100eb0 2024-05-15T16:15:30.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.839+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c1091e0 con 0x7f3e7c100eb0 2024-05-15T16:15:30.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.839+0000 7f3e78ff9700 1 -- 172.21.15.49:0/431301136 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3e6c010380 con 0x7f3e7c100eb0 2024-05-15T16:15:30.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.840+0000 7f3e78ff9700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3e6406e200 0x7f3e640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:30.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.840+0000 7f3e78ff9700 1 -- 172.21.15.49:0/431301136 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3e6c08dd90 con 0x7f3e7c100eb0 2024-05-15T16:15:30.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.840+0000 7f3e7affd700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3e6406e200 0x7f3e640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:30.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.840+0000 7f3e7affd700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3e6406e200 0x7f3e640706c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c102b30 tx=0x7f3e7001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:30.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:30.845+0000 7f3e78ff9700 1 -- 172.21.15.49:0/431301136 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3e6c057070 con 0x7f3e7c100eb0 2024-05-15T16:15:30.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:30 smithi144 ceph-mon[86635]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:15:30.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:30 smithi144 ceph-mon[86635]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:15:30.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:30 smithi049 ceph-mon[112142]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:15:30.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:30 smithi049 ceph-mon[112142]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:15:31.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.098+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3e7c102390 con 0x7f3e7c100eb0 2024-05-15T16:15:31.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.098+0000 7f3e78ff9700 1 -- 172.21.15.49:0/431301136 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f3e6c05a730 con 0x7f3e7c100eb0 2024-05-15T16:15:31.100 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:15:31.100 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:15:31.100 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:15:31.100 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:15:31.100 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:15:31.100 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:15:31.100 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:15:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:15:31.102 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:15:31.103 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:15:31.103 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:15:31.103 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:15:31.103 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:15:31.103 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:15:31.103 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3e6406e200 msgr2=0x7f3e640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3e6406e200 0x7f3e640706c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c102b30 tx=0x7f3e7001a040 comp rx=0 tx=0).stop 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c100eb0 msgr2=0x7f3e7c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c100eb0 0x7f3e7c0732a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00b4b0 tx=0x7f3e6c00b870 comp rx=0 tx=0).stop 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 shutdown_connections 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3e6406e200 0x7f3e640706c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7c100eb0 0x7f3e7c0732a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 --2- 172.21.15.49:0/431301136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e7c101860 0x7f3e7c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 >> 172.21.15.49:0/431301136 conn(0x7f3e7c0fc760 msgr2=0x7f3e7c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 shutdown_connections 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.101+0000 7f3e820c4700 1 -- 172.21.15.49:0/431301136 wait complete. 2024-05-15T16:15:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:15:31.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.211+0000 7f697cc4f700 1 -- 172.21.15.49:0/1092304617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 msgr2=0x7f69780ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.211+0000 7f697cc4f700 1 --2- 172.21.15.49:0/1092304617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 0x7f69780ffa70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6968009910 tx=0x7f6968009c20 comp rx=0 tx=0).stop 2024-05-15T16:15:31.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.211+0000 7f697cc4f700 1 -- 172.21.15.49:0/1092304617 shutdown_connections 2024-05-15T16:15:31.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.211+0000 7f697cc4f700 1 --2- 172.21.15.49:0/1092304617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6978100040 0x7f6978104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.211+0000 7f697cc4f700 1 --2- 172.21.15.49:0/1092304617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 0x7f69780ffa70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.211+0000 7f697cc4f700 1 -- 172.21.15.49:0/1092304617 >> 172.21.15.49:0/1092304617 conn(0x7f6978074eb0 msgr2=0x7f69780752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:31.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.211+0000 7f697cc4f700 1 -- 172.21.15.49:0/1092304617 shutdown_connections 2024-05-15T16:15:31.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.211+0000 7f697cc4f700 1 -- 172.21.15.49:0/1092304617 wait complete. 2024-05-15T16:15:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f697cc4f700 1 Processor -- start 2024-05-15T16:15:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f697cc4f700 1 -- start start 2024-05-15T16:15:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f697cc4f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 0x7f697819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f697cc4f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6978100040 0x7f697819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f697cc4f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697819d3f0 con 0x7f69780ff690 2024-05-15T16:15:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f697cc4f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6978196a20 con 0x7f6978100040 2024-05-15T16:15:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f697659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 0x7f697819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f697659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 0x7f697819c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59942/0 (socket says 172.21.15.49:59942) 2024-05-15T16:15:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f697659c700 1 -- 172.21.15.49:0/459150388 learned_addr learned my addr 172.21.15.49:0/459150388 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:15:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.212+0000 7f6975d9b700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6978100040 0x7f697819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.213+0000 7f697659c700 1 -- 172.21.15.49:0/459150388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6978100040 msgr2=0x7f697819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.213+0000 7f697659c700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6978100040 0x7f697819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.213+0000 7f697659c700 1 -- 172.21.15.49:0/459150388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69680095f0 con 0x7f69780ff690 2024-05-15T16:15:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.213+0000 7f697659c700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 0x7f697819c890 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6968006010 tx=0x7f696800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.213+0000 7f69677fe700 1 -- 172.21.15.49:0/459150388 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f696801d070 con 0x7f69780ff690 2024-05-15T16:15:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.213+0000 7f69677fe700 1 -- 172.21.15.49:0/459150388 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69680056e0 con 0x7f69780ff690 2024-05-15T16:15:31.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.213+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6978196ca0 con 0x7f69780ff690 2024-05-15T16:15:31.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.213+0000 7f69677fe700 1 -- 172.21.15.49:0/459150388 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f696800fdf0 con 0x7f69780ff690 2024-05-15T16:15:31.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.213+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6978197190 con 0x7f69780ff690 2024-05-15T16:15:31.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.214+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6978109300 con 0x7f69780ff690 2024-05-15T16:15:31.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.215+0000 7f69677fe700 1 -- 172.21.15.49:0/459150388 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6968003a60 con 0x7f69780ff690 2024-05-15T16:15:31.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.215+0000 7f69677fe700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f696006e200 0x7f69600706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:31.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.215+0000 7f69677fe700 1 -- 172.21.15.49:0/459150388 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f696808f430 con 0x7f69780ff690 2024-05-15T16:15:31.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.215+0000 7f6975d9b700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f696006e200 0x7f69600706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:31.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.216+0000 7f6975d9b700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f696006e200 0x7f69600706c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6978197b90 tx=0x7f696c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:31.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.220+0000 7f69677fe700 1 -- 172.21.15.49:0/459150388 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f696805c2c0 con 0x7f69780ff690 2024-05-15T16:15:31.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.445+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6978197e90 con 0x7f696006e200 2024-05-15T16:15:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.446+0000 7f69677fe700 1 -- 172.21.15.49:0/459150388 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6978197e90 con 0x7f696006e200 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:15:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:15:31.451 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:15:31.451 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:15:31.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.449+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f696006e200 msgr2=0x7f69600706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.449+0000 7f697cc4f700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f696006e200 0x7f69600706c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6978197b90 tx=0x7f696c00b410 comp rx=0 tx=0).stop 2024-05-15T16:15:31.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.449+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 msgr2=0x7f697819c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.449+0000 7f697cc4f700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 0x7f697819c890 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6968006010 tx=0x7f696800bd70 comp rx=0 tx=0).stop 2024-05-15T16:15:31.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.450+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 shutdown_connections 2024-05-15T16:15:31.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.450+0000 7f697cc4f700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f696006e200 0x7f69600706c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.450+0000 7f697cc4f700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69780ff690 0x7f697819c890 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.450+0000 7f697cc4f700 1 --2- 172.21.15.49:0/459150388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6978100040 0x7f697819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.450+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 >> 172.21.15.49:0/459150388 conn(0x7f6978074eb0 msgr2=0x7f6978102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:31.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.450+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 shutdown_connections 2024-05-15T16:15:31.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.450+0000 7f697cc4f700 1 -- 172.21.15.49:0/459150388 wait complete. 2024-05-15T16:15:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.560+0000 7f2cf31a1700 1 -- 172.21.15.49:0/1862331384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 msgr2=0x7f2cec0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.560+0000 7f2cf31a1700 1 --2- 172.21.15.49:0/1862331384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 0x7f2cec0fffc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009910 tx=0x7f2cdc009c20 comp rx=0 tx=0).stop 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.560+0000 7f2cf31a1700 1 -- 172.21.15.49:0/1862331384 shutdown_connections 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.560+0000 7f2cf31a1700 1 --2- 172.21.15.49:0/1862331384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 0x7f2cec0fffc0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.560+0000 7f2cf31a1700 1 --2- 172.21.15.49:0/1862331384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec1059c0 0x7f2cec105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.560+0000 7f2cf31a1700 1 -- 172.21.15.49:0/1862331384 >> 172.21.15.49:0/1862331384 conn(0x7f2cec075520 msgr2=0x7f2cec075930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.560+0000 7f2cf31a1700 1 -- 172.21.15.49:0/1862331384 shutdown_connections 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.560+0000 7f2cf31a1700 1 -- 172.21.15.49:0/1862331384 wait complete. 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.561+0000 7f2cf31a1700 1 Processor -- start 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.561+0000 7f2cf31a1700 1 -- start start 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.561+0000 7f2cf31a1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 0x7f2cec19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.561+0000 7f2cf31a1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec1059c0 0x7f2cec19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.561+0000 7f2cf31a1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec19d2d0 con 0x7f2cec0ffb40 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.561+0000 7f2cf31a1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec196840 con 0x7f2cec1059c0 2024-05-15T16:15:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.561+0000 7f2cf219f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 0x7f2cec19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.561+0000 7f2cf219f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 0x7f2cec19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59966/0 (socket says 172.21.15.49:59966) 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.561+0000 7f2cf219f700 1 -- 172.21.15.49:0/230129328 learned_addr learned my addr 172.21.15.49:0/230129328 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2cf199e700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec1059c0 0x7f2cec19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2cf219f700 1 -- 172.21.15.49:0/230129328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec1059c0 msgr2=0x7f2cec19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2cf219f700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec1059c0 0x7f2cec19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2cf219f700 1 -- 172.21.15.49:0/230129328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0095f0 con 0x7f2cec0ffb40 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2cf219f700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 0x7f2cec19c700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800c8a0 tx=0x7f2ce800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2ce37fe700 1 -- 172.21.15.49:0/230129328 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce800cea0 con 0x7f2cec0ffb40 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2ce37fe700 1 -- 172.21.15.49:0/230129328 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ce8004830 con 0x7f2cec0ffb40 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec196b20 con 0x7f2cec0ffb40 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2ce37fe700 1 -- 172.21.15.49:0/230129328 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce80055b0 con 0x7f2cec0ffb40 2024-05-15T16:15:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.562+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec197070 con 0x7f2cec0ffb40 2024-05-15T16:15:31.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.563+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec108ff0 con 0x7f2cec0ffb40 2024-05-15T16:15:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.564+0000 7f2ce37fe700 1 -- 172.21.15.49:0/230129328 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2ce80049a0 con 0x7f2cec0ffb40 2024-05-15T16:15:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.565+0000 7f2ce37fe700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cd806e200 0x7f2cd80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:15:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.565+0000 7f2cf199e700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cd806e200 0x7f2cd80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:15:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.565+0000 7f2ce37fe700 1 -- 172.21.15.49:0/230129328 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2ce808de40 con 0x7f2cec0ffb40 2024-05-15T16:15:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.565+0000 7f2cf199e700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cd806e200 0x7f2cd80706c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0059d0 tx=0x7f2cdc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:15:31.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.569+0000 7f2ce37fe700 1 -- 172.21.15.49:0/230129328 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2ce805ac50 con 0x7f2cec0ffb40 2024-05-15T16:15:31.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:31 smithi049 ceph-mon[112142]: from='client.35054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:15:31.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:31 smithi049 ceph-mon[112142]: pgmap v554: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:31.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:31 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/599938297' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:15:31.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:31 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/431301136' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:15:31.849 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:31 smithi144 ceph-mon[86635]: from='client.35054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:15:31.849 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:31 smithi144 ceph-mon[86635]: pgmap v554: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:31.849 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:31 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/599938297' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:15:31.849 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:31 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/431301136' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:15:31.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.867+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2cec101120 con 0x7f2cec0ffb40 2024-05-15T16:15:31.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.868+0000 7f2ce37fe700 1 -- 172.21.15.49:0/230129328 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f2ce801d090 con 0x7f2cec0ffb40 2024-05-15T16:15:31.869 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:15:31.869 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:15:31.870 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:15:31.870 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:15:31.870 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:15:31.870 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:15:31.870 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:15:31.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cd806e200 msgr2=0x7f2cd80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cd806e200 0x7f2cd80706c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0059d0 tx=0x7f2cdc01a040 comp rx=0 tx=0).stop 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 msgr2=0x7f2cec19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 0x7f2cec19c700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800c8a0 tx=0x7f2ce800cc60 comp rx=0 tx=0).stop 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 shutdown_connections 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cd806e200 0x7f2cd80706c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ffb40 0x7f2cec19c700 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 --2- 172.21.15.49:0/230129328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec1059c0 0x7f2cec19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 >> 172.21.15.49:0/230129328 conn(0x7f2cec075520 msgr2=0x7f2cec068b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 shutdown_connections 2024-05-15T16:15:31.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:15:31.870+0000 7f2cf31a1700 1 -- 172.21.15.49:0/230129328 wait complete. 2024-05-15T16:15:32.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:32 smithi144 ceph-mon[86635]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:15:32.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:32 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/230129328' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:15:32.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:32 smithi049 ceph-mon[112142]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:15:32.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:32 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/230129328' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:15:33.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:33 smithi144 ceph-mon[86635]: pgmap v555: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:33.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:33 smithi049 ceph-mon[112142]: pgmap v555: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:35.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:35 smithi144 ceph-mon[86635]: pgmap v556: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:35.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:15:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:35 smithi049 ceph-mon[112142]: pgmap v556: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:35.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:15:37.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:37 smithi144 ceph-mon[86635]: pgmap v557: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:37.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:37 smithi049 ceph-mon[112142]: pgmap v557: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:39.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:39 smithi144 ceph-mon[86635]: pgmap v558: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:39.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:39 smithi049 ceph-mon[112142]: pgmap v558: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:41.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:41 smithi144 ceph-mon[86635]: pgmap v559: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:41.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:41 smithi049 ceph-mon[112142]: pgmap v559: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:43.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:43 smithi049 ceph-mon[112142]: pgmap v560: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:43 smithi144 ceph-mon[86635]: pgmap v560: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:45 smithi049 ceph-mon[112142]: pgmap v561: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:45 smithi144 ceph-mon[86635]: pgmap v561: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:47.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:47 smithi049 ceph-mon[112142]: pgmap v562: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:47 smithi144 ceph-mon[86635]: pgmap v562: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:49 smithi144 ceph-mon[86635]: pgmap v563: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:49 smithi049 ceph-mon[112142]: pgmap v563: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:15:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:15:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:51 smithi144 ceph-mon[86635]: pgmap v564: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:51 smithi049 ceph-mon[112142]: pgmap v564: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:53 smithi144 ceph-mon[86635]: pgmap v565: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:53 smithi049 ceph-mon[112142]: pgmap v565: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:55 smithi144 ceph-mon[86635]: pgmap v566: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:55 smithi049 ceph-mon[112142]: pgmap v566: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:57 smithi144 ceph-mon[86635]: pgmap v567: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:15:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:57 smithi049 ceph-mon[112142]: pgmap v567: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:15:59 smithi144 ceph-mon[86635]: pgmap v568: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:15:59 smithi049 ceph-mon[112142]: pgmap v568: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:01.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.994+0000 7f6cb079e700 1 -- 172.21.15.49:0/1148238502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 msgr2=0x7f6ca810ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:01.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.994+0000 7f6cb079e700 1 --2- 172.21.15.49:0/1148238502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 0x7f6ca810ac30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009910 tx=0x7f6c98009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:01.995 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:01 smithi049 ceph-mon[112142]: pgmap v569: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.994+0000 7f6cb079e700 1 -- 172.21.15.49:0/1148238502 shutdown_connections 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.994+0000 7f6cb079e700 1 --2- 172.21.15.49:0/1148238502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 0x7f6ca810ac30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.994+0000 7f6cb079e700 1 --2- 172.21.15.49:0/1148238502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ca8101e20 0x7f6ca8102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.994+0000 7f6cb079e700 1 -- 172.21.15.49:0/1148238502 >> 172.21.15.49:0/1148238502 conn(0x7f6ca8076800 msgr2=0x7f6ca80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.994+0000 7f6cb079e700 1 -- 172.21.15.49:0/1148238502 shutdown_connections 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.994+0000 7f6cb079e700 1 -- 172.21.15.49:0/1148238502 wait complete. 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cb079e700 1 Processor -- start 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cb079e700 1 -- start start 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cb079e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ca8101e20 0x7f6ca819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:01.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cb079e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 0x7f6ca819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cb079e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca819d460 con 0x7f6ca8102740 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cb079e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8196a40 con 0x7f6ca8101e20 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cadd39700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 0x7f6ca819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cadd39700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 0x7f6ca819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46030/0 (socket says 172.21.15.49:46030) 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cadd39700 1 -- 172.21.15.49:0/1608984227 learned_addr learned my addr 172.21.15.49:0/1608984227 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.995+0000 7f6cae53a700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ca8101e20 0x7f6ca819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6cadd39700 1 -- 172.21.15.49:0/1608984227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ca8101e20 msgr2=0x7f6ca819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6cadd39700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ca8101e20 0x7f6ca819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6cadd39700 1 -- 172.21.15.49:0/1608984227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c980095f0 con 0x7f6ca8102740 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6cae53a700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ca8101e20 0x7f6ca819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6cadd39700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 0x7f6ca819ce40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6c980092e0 tx=0x7f6c9800bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6c9f7fe700 1 -- 172.21.15.49:0/1608984227 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9801d070 con 0x7f6ca8102740 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6c9f7fe700 1 -- 172.21.15.49:0/1608984227 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c98005dc0 con 0x7f6ca8102740 2024-05-15T16:16:01.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca8196cc0 con 0x7f6ca8102740 2024-05-15T16:16:01.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6c9f7fe700 1 -- 172.21.15.49:0/1608984227 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9800fe20 con 0x7f6ca8102740 2024-05-15T16:16:01.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.996+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca81971b0 con 0x7f6ca8102740 2024-05-15T16:16:01.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.997+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca8105540 con 0x7f6ca8102740 2024-05-15T16:16:02.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.998+0000 7f6c9f7fe700 1 -- 172.21.15.49:0/1608984227 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6c98003850 con 0x7f6ca8102740 2024-05-15T16:16:02.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.998+0000 7f6c9f7fe700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c9406e200 0x7f6c940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:02.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.998+0000 7f6c9f7fe700 1 -- 172.21.15.49:0/1608984227 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6c9808f460 con 0x7f6ca8102740 2024-05-15T16:16:02.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.998+0000 7f6cae53a700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c9406e200 0x7f6c940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:02.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:01.999+0000 7f6cae53a700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c9406e200 0x7f6c940706c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4005950 tx=0x7f6ca40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:02.000 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:01 smithi144 ceph-mon[86635]: pgmap v569: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.003+0000 7f6c9f7fe700 1 -- 172.21.15.49:0/1608984227 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6c9805c270 con 0x7f6ca8102740 2024-05-15T16:16:02.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.233+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ca8197f70 con 0x7f6c9406e200 2024-05-15T16:16:02.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.234+0000 7f6c9f7fe700 1 -- 172.21.15.49:0/1608984227 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6ca8197f70 con 0x7f6c9406e200 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c9406e200 msgr2=0x7f6c940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c9406e200 0x7f6c940706c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4005950 tx=0x7f6ca40058e0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 msgr2=0x7f6ca819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 0x7f6ca819ce40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6c980092e0 tx=0x7f6c9800bbf0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 shutdown_connections 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c9406e200 0x7f6c940706c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ca8101e20 0x7f6ca819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 --2- 172.21.15.49:0/1608984227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ca8102740 0x7f6ca819ce40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 >> 172.21.15.49:0/1608984227 conn(0x7f6ca8076800 msgr2=0x7f6ca8103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 shutdown_connections 2024-05-15T16:16:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.236+0000 7f6cb079e700 1 -- 172.21.15.49:0/1608984227 wait complete. 2024-05-15T16:16:02.247 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:16:02.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.346+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2793992118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf301018b0 msgr2=0x7fcf30105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.346+0000 7fcf36dc4700 1 --2- 172.21.15.49:0/2793992118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf301018b0 0x7fcf30105900 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009910 tx=0x7fcf24009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.346+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2793992118 shutdown_connections 2024-05-15T16:16:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.346+0000 7fcf36dc4700 1 --2- 172.21.15.49:0/2793992118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf301018b0 0x7fcf30105900 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.346+0000 7fcf36dc4700 1 --2- 172.21.15.49:0/2793992118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf30100f00 0x7fcf301012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.346+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2793992118 >> 172.21.15.49:0/2793992118 conn(0x7fcf300fc7d0 msgr2=0x7fcf300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.346+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2793992118 shutdown_connections 2024-05-15T16:16:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.346+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2793992118 wait complete. 2024-05-15T16:16:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.347+0000 7fcf36dc4700 1 Processor -- start 2024-05-15T16:16:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.347+0000 7fcf36dc4700 1 -- start start 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.347+0000 7fcf36dc4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf30100f00 0x7fcf3019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.347+0000 7fcf36dc4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf301018b0 0x7fcf3019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.347+0000 7fcf36dc4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3019d440 con 0x7fcf30100f00 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.347+0000 7fcf36dc4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf30196a20 con 0x7fcf301018b0 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf34b60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf30100f00 0x7fcf3019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf34b60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf30100f00 0x7fcf3019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46048/0 (socket says 172.21.15.49:46048) 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf34b60700 1 -- 172.21.15.49:0/2191641614 learned_addr learned my addr 172.21.15.49:0/2191641614 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf2ffff700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf301018b0 0x7fcf3019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf34b60700 1 -- 172.21.15.49:0/2191641614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf301018b0 msgr2=0x7fcf3019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf34b60700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf301018b0 0x7fcf3019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf34b60700 1 -- 172.21.15.49:0/2191641614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf240095f0 con 0x7fcf30100f00 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf2ffff700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf301018b0 0x7fcf3019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf34b60700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf30100f00 0x7fcf3019c8e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000e920 tx=0x7fcf2000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf2dffb700 1 -- 172.21.15.49:0/2191641614 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf20009790 con 0x7fcf30100f00 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf30196d00 con 0x7fcf30100f00 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf2dffb700 1 -- 172.21.15.49:0/2191641614 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf2000cba0 con 0x7fcf30100f00 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.348+0000 7fcf2dffb700 1 -- 172.21.15.49:0/2191641614 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf20018650 con 0x7fcf30100f00 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.349+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf30197250 con 0x7fcf30100f00 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.349+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf30109300 con 0x7fcf30100f00 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.350+0000 7fcf2dffb700 1 -- 172.21.15.49:0/2191641614 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcf2000cd10 con 0x7fcf30100f00 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.351+0000 7fcf2dffb700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf1806e200 0x7fcf180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.351+0000 7fcf2dffb700 1 -- 172.21.15.49:0/2191641614 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcf20014070 con 0x7fcf30100f00 2024-05-15T16:16:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.351+0000 7fcf2ffff700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf1806e200 0x7fcf180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:02.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.351+0000 7fcf2ffff700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf1806e200 0x7fcf180706c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf240059d0 tx=0x7fcf24005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:02.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.355+0000 7fcf2dffb700 1 -- 172.21.15.49:0/2191641614 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcf20053850 con 0x7fcf30100f00 2024-05-15T16:16:02.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.584+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf30197f50 con 0x7fcf1806e200 2024-05-15T16:16:02.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.585+0000 7fcf2dffb700 1 -- 172.21.15.49:0/2191641614 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcf30197f50 con 0x7fcf1806e200 2024-05-15T16:16:02.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf1806e200 msgr2=0x7fcf180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf1806e200 0x7fcf180706c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf240059d0 tx=0x7fcf24005940 comp rx=0 tx=0).stop 2024-05-15T16:16:02.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf30100f00 msgr2=0x7fcf3019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf30100f00 0x7fcf3019c8e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000e920 tx=0x7fcf2000ece0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 shutdown_connections 2024-05-15T16:16:02.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcf1806e200 0x7fcf180706c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf30100f00 0x7fcf3019c8e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 --2- 172.21.15.49:0/2191641614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf301018b0 0x7fcf3019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 >> 172.21.15.49:0/2191641614 conn(0x7fcf300fc7d0 msgr2=0x7fcf300fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:02.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 shutdown_connections 2024-05-15T16:16:02.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.587+0000 7fcf36dc4700 1 -- 172.21.15.49:0/2191641614 wait complete. 2024-05-15T16:16:02.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 -- 172.21.15.49:0/3815143172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 msgr2=0x7fea381022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 --2- 172.21.15.49:0/3815143172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 0x7fea381022f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fea28009910 tx=0x7fea28009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 -- 172.21.15.49:0/3815143172 shutdown_connections 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 --2- 172.21.15.49:0/3815143172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea38102830 0x7fea3810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 --2- 172.21.15.49:0/3815143172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 0x7fea381022f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 -- 172.21.15.49:0/3815143172 >> 172.21.15.49:0/3815143172 conn(0x7fea380fb6c0 msgr2=0x7fea380fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 -- 172.21.15.49:0/3815143172 shutdown_connections 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 -- 172.21.15.49:0/3815143172 wait complete. 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 Processor -- start 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.696+0000 7fea3e38b700 1 -- start start 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea3e38b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 0x7fea3819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea3e38b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea38102830 0x7fea3819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea3e38b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3819d440 con 0x7fea38101f10 2024-05-15T16:16:02.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea3e38b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38196a20 con 0x7fea38102830 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 0x7fea3819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 0x7fea3819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46066/0 (socket says 172.21.15.49:46066) 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea37fff700 1 -- 172.21.15.49:0/186469856 learned_addr learned my addr 172.21.15.49:0/186469856 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea377fe700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea38102830 0x7fea3819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea37fff700 1 -- 172.21.15.49:0/186469856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea38102830 msgr2=0x7fea3819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea37fff700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea38102830 0x7fea3819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea37fff700 1 -- 172.21.15.49:0/186469856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea280095f0 con 0x7fea38101f10 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea377fe700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea38102830 0x7fea3819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea37fff700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 0x7fea3819c8e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fea28006010 tx=0x7fea2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea357fa700 1 -- 172.21.15.49:0/186469856 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea2801d070 con 0x7fea38101f10 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.697+0000 7fea357fa700 1 -- 172.21.15.49:0/186469856 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea280056e0 con 0x7fea38101f10 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.698+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea38196ca0 con 0x7fea38101f10 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.698+0000 7fea357fa700 1 -- 172.21.15.49:0/186469856 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea28017400 con 0x7fea38101f10 2024-05-15T16:16:02.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.698+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea38197190 con 0x7fea38101f10 2024-05-15T16:16:02.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.699+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea38105630 con 0x7fea38101f10 2024-05-15T16:16:02.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.699+0000 7fea357fa700 1 -- 172.21.15.49:0/186469856 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fea28003a60 con 0x7fea38101f10 2024-05-15T16:16:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.699+0000 7fea357fa700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fea2006e200 0x7fea200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.700+0000 7fea357fa700 1 -- 172.21.15.49:0/186469856 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fea2808f570 con 0x7fea38101f10 2024-05-15T16:16:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.700+0000 7fea377fe700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fea2006e200 0x7fea200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:02.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.700+0000 7fea377fe700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fea2006e200 0x7fea200706c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fea2c005d90 tx=0x7fea2c005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:02.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.704+0000 7fea357fa700 1 -- 172.21.15.49:0/186469856 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fea2805c380 con 0x7fea38101f10 2024-05-15T16:16:02.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.920+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fea38197f50 con 0x7fea2006e200 2024-05-15T16:16:02.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.932+0000 7fea357fa700 1 -- 172.21.15.49:0/186469856 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fea38197f50 con 0x7fea2006e200 2024-05-15T16:16:02.933 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:16:02.933 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (19m) 12m ago 30m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:16:02.933 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (31m) 12m ago 31m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:16:02.933 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (29m) 12m ago 29m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (16m) 12m ago 31m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (16m) 12m ago 29m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (18m) 12m ago 30m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (12m) 12m ago 25m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (12m) 12m ago 25m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (12m) 12m ago 25m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (25m) 12m ago 25m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (21m) 12m ago 32m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (20m) 12m ago 29m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (17m) 12m ago 32m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (17m) 12m ago 29m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (20m) 12m ago 30m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (20m) 12m ago 29m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:16:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (16m) 12m ago 28m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:16:02.935 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (15m) 12m ago 28m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:16:02.935 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (15m) 12m ago 28m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:16:02.935 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (14m) 12m ago 27m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:16:02.935 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (13m) 12m ago 27m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:16:02.935 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (13m) 12m ago 26m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:16:02.935 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (19m) 12m ago 30m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fea2006e200 msgr2=0x7fea200706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fea2006e200 0x7fea200706c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fea2c005d90 tx=0x7fea2c005d00 comp rx=0 tx=0).stop 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 msgr2=0x7fea3819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 0x7fea3819c8e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fea28006010 tx=0x7fea2800be20 comp rx=0 tx=0).stop 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 shutdown_connections 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fea2006e200 0x7fea200706c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea38101f10 0x7fea3819c8e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 --2- 172.21.15.49:0/186469856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea38102830 0x7fea3819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 >> 172.21.15.49:0/186469856 conn(0x7fea380fb6c0 msgr2=0x7fea38103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 shutdown_connections 2024-05-15T16:16:02.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:02.934+0000 7fea3e38b700 1 -- 172.21.15.49:0/186469856 wait complete. 2024-05-15T16:16:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.042+0000 7f6713ee8700 1 -- 172.21.15.49:0/3022428689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c1007a0 msgr2=0x7f670c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.042+0000 7f6713ee8700 1 --2- 172.21.15.49:0/3022428689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c1007a0 0x7f670c1047f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6700009910 tx=0x7f6700009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.042+0000 7f6713ee8700 1 -- 172.21.15.49:0/3022428689 shutdown_connections 2024-05-15T16:16:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.042+0000 7f6713ee8700 1 --2- 172.21.15.49:0/3022428689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c1007a0 0x7f670c1047f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.042+0000 7f6713ee8700 1 --2- 172.21.15.49:0/3022428689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f670c0ffdf0 0x7f670c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.042+0000 7f6713ee8700 1 -- 172.21.15.49:0/3022428689 >> 172.21.15.49:0/3022428689 conn(0x7f670c0fb6a0 msgr2=0x7f670c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.042+0000 7f6713ee8700 1 -- 172.21.15.49:0/3022428689 shutdown_connections 2024-05-15T16:16:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.042+0000 7f6713ee8700 1 -- 172.21.15.49:0/3022428689 wait complete. 2024-05-15T16:16:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6713ee8700 1 Processor -- start 2024-05-15T16:16:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6713ee8700 1 -- start start 2024-05-15T16:16:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6713ee8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c0ffdf0 0x7f670c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6713ee8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f670c1007a0 0x7f670c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6713ee8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c198660 con 0x7f670c0ffdf0 2024-05-15T16:16:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6713ee8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c1987a0 con 0x7f670c1007a0 2024-05-15T16:16:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6711c84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c0ffdf0 0x7f670c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6711c84700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c0ffdf0 0x7f670c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46078/0 (socket says 172.21.15.49:46078) 2024-05-15T16:16:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6711c84700 1 -- 172.21.15.49:0/159170693 learned_addr learned my addr 172.21.15.49:0/159170693 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6711483700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f670c1007a0 0x7f670c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6711c84700 1 -- 172.21.15.49:0/159170693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f670c1007a0 msgr2=0x7f670c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6711c84700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f670c1007a0 0x7f670c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6711c84700 1 -- 172.21.15.49:0/159170693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67000095f0 con 0x7f670c0ffdf0 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.043+0000 7f6711483700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f670c1007a0 0x7f670c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.044+0000 7f6711c84700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c0ffdf0 0x7f670c197b50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f670800b4b0 tx=0x7f670800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.044+0000 7f66feffd700 1 -- 172.21.15.49:0/159170693 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6708011800 con 0x7f670c0ffdf0 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.044+0000 7f66feffd700 1 -- 172.21.15.49:0/159170693 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6708011e20 con 0x7f670c0ffdf0 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.044+0000 7f66feffd700 1 -- 172.21.15.49:0/159170693 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670800f2e0 con 0x7f670c0ffdf0 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.044+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c1a0bc0 con 0x7f670c0ffdf0 2024-05-15T16:16:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.044+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c1a0fd0 con 0x7f670c0ffdf0 2024-05-15T16:16:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.045+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c108120 con 0x7f670c0ffdf0 2024-05-15T16:16:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.046+0000 7f66feffd700 1 -- 172.21.15.49:0/159170693 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6708010380 con 0x7f670c0ffdf0 2024-05-15T16:16:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.046+0000 7f66feffd700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f66f806e200 0x7f66f80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.046+0000 7f66feffd700 1 -- 172.21.15.49:0/159170693 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f670808dca0 con 0x7f670c0ffdf0 2024-05-15T16:16:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.046+0000 7f6711483700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f66f806e200 0x7f66f80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:03.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.047+0000 7f6711483700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f66f806e200 0x7f66f80706c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6700005f50 tx=0x7f6700005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:03.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.051+0000 7f66feffd700 1 -- 172.21.15.49:0/159170693 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6708092070 con 0x7f670c0ffdf0 2024-05-15T16:16:03.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.349+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f670c069830 con 0x7f670c0ffdf0 2024-05-15T16:16:03.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.350+0000 7f66feffd700 1 -- 172.21.15.49:0/159170693 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f6708057000 con 0x7f670c0ffdf0 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:16:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:16:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:16:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:16:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:16:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:16:03.353 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:16:03.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f66f806e200 msgr2=0x7f66f80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f66f806e200 0x7f66f80706c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6700005f50 tx=0x7f6700005dc0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c0ffdf0 msgr2=0x7f670c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c0ffdf0 0x7f670c197b50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f670800b4b0 tx=0x7f670800b870 comp rx=0 tx=0).stop 2024-05-15T16:16:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 shutdown_connections 2024-05-15T16:16:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f66f806e200 0x7f66f80706c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f670c0ffdf0 0x7f670c197b50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 --2- 172.21.15.49:0/159170693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f670c1007a0 0x7f670c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 >> 172.21.15.49:0/159170693 conn(0x7f670c0fb6a0 msgr2=0x7f670c0fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 shutdown_connections 2024-05-15T16:16:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.352+0000 7f6713ee8700 1 -- 172.21.15.49:0/159170693 wait complete. 2024-05-15T16:16:03.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.460+0000 7feeb955c700 1 -- 172.21.15.49:0/4179726233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 msgr2=0x7feeb410acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.460+0000 7feeb955c700 1 --2- 172.21.15.49:0/4179726233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 0x7feeb410acc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7feea4009910 tx=0x7feea4009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:03.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.460+0000 7feeb955c700 1 -- 172.21.15.49:0/4179726233 shutdown_connections 2024-05-15T16:16:03.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.460+0000 7feeb955c700 1 --2- 172.21.15.49:0/4179726233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 0x7feeb410acc0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.460+0000 7feeb955c700 1 --2- 172.21.15.49:0/4179726233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb4101eb0 0x7feeb4102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.460+0000 7feeb955c700 1 -- 172.21.15.49:0/4179726233 >> 172.21.15.49:0/4179726233 conn(0x7feeb40fb6a0 msgr2=0x7feeb40fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:03.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.460+0000 7feeb955c700 1 -- 172.21.15.49:0/4179726233 shutdown_connections 2024-05-15T16:16:03.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.460+0000 7feeb955c700 1 -- 172.21.15.49:0/4179726233 wait complete. 2024-05-15T16:16:03.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.461+0000 7feeb955c700 1 Processor -- start 2024-05-15T16:16:03.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.461+0000 7feeb955c700 1 -- start start 2024-05-15T16:16:03.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.461+0000 7feeb955c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb4101eb0 0x7feeb419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:03.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.461+0000 7feeb955c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 0x7feeb419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:03.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.461+0000 7feeb955c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb419d410 con 0x7feeb41027d0 2024-05-15T16:16:03.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.461+0000 7feeb955c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb41969f0 con 0x7feeb4101eb0 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.461+0000 7feeb27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 0x7feeb419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb4101eb0 0x7feeb419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb4101eb0 0x7feeb419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.49:39592/0 (socket says 172.21.15.49:39592) 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb2ffd700 1 -- 172.21.15.49:0/1372065302 learned_addr learned my addr 172.21.15.49:0/1372065302 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 0x7feeb419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46094/0 (socket says 172.21.15.49:46094) 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb27fc700 1 -- 172.21.15.49:0/1372065302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb4101eb0 msgr2=0x7feeb419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb27fc700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb4101eb0 0x7feeb419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb27fc700 1 -- 172.21.15.49:0/1372065302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea40095f0 con 0x7feeb41027d0 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb2ffd700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb4101eb0 0x7feeb419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb27fc700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 0x7feeb419cdf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feea4009fd0 tx=0x7feea400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeabfff700 1 -- 172.21.15.49:0/1372065302 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea401d070 con 0x7feeb41027d0 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeabfff700 1 -- 172.21.15.49:0/1372065302 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feea40056e0 con 0x7feeb41027d0 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeabfff700 1 -- 172.21.15.49:0/1372065302 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea400fd80 con 0x7feeb41027d0 2024-05-15T16:16:03.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb4196c70 con 0x7feeb41027d0 2024-05-15T16:16:03.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.462+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb4197160 con 0x7feeb41027d0 2024-05-15T16:16:03.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.463+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb4105640 con 0x7feeb41027d0 2024-05-15T16:16:03.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.464+0000 7feeabfff700 1 -- 172.21.15.49:0/1372065302 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feea400f5f0 con 0x7feeb41027d0 2024-05-15T16:16:03.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.464+0000 7feeabfff700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feea006e200 0x7feea00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:03.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.465+0000 7feeabfff700 1 -- 172.21.15.49:0/1372065302 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7feea408f2f0 con 0x7feeb41027d0 2024-05-15T16:16:03.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.465+0000 7feeb2ffd700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feea006e200 0x7feea00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:03.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.465+0000 7feeb2ffd700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feea006e200 0x7feea00706c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fee9c005950 tx=0x7fee9c00b500 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:03.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.469+0000 7feeabfff700 1 -- 172.21.15.49:0/1372065302 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7feea405c180 con 0x7feeb41027d0 2024-05-15T16:16:03.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.731+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7feeb4069770 con 0x7feeb41027d0 2024-05-15T16:16:03.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.732+0000 7feeabfff700 1 -- 172.21.15.49:0/1372065302 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7feea405bd10 con 0x7feeb41027d0 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:16:03.734 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:16:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:16:03.736 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:16:03.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feea006e200 msgr2=0x7feea00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feea006e200 0x7feea00706c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fee9c005950 tx=0x7fee9c00b500 comp rx=0 tx=0).stop 2024-05-15T16:16:03.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 msgr2=0x7feeb419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 0x7feeb419cdf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feea4009fd0 tx=0x7feea400bd70 comp rx=0 tx=0).stop 2024-05-15T16:16:03.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 shutdown_connections 2024-05-15T16:16:03.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feea006e200 0x7feea00706c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb4101eb0 0x7feeb419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 --2- 172.21.15.49:0/1372065302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feeb41027d0 0x7feeb419cdf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 >> 172.21.15.49:0/1372065302 conn(0x7feeb40fb6a0 msgr2=0x7feeb4103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:03.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 shutdown_connections 2024-05-15T16:16:03.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.735+0000 7feeb955c700 1 -- 172.21.15.49:0/1372065302 wait complete. 2024-05-15T16:16:03.738 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:16:03.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.844+0000 7f9ad1c49700 1 -- 172.21.15.49:0/3704119437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 msgr2=0x7f9acc1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.844+0000 7f9ad1c49700 1 --2- 172.21.15.49:0/3704119437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 0x7f9acc1058e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009910 tx=0x7f9ac0009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:03.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.844+0000 7f9ad1c49700 1 -- 172.21.15.49:0/3704119437 shutdown_connections 2024-05-15T16:16:03.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.844+0000 7f9ad1c49700 1 --2- 172.21.15.49:0/3704119437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 0x7f9acc1058e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.844+0000 7f9ad1c49700 1 --2- 172.21.15.49:0/3704119437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc100ee0 0x7f9acc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.844+0000 7f9ad1c49700 1 -- 172.21.15.49:0/3704119437 >> 172.21.15.49:0/3704119437 conn(0x7f9acc0fc790 msgr2=0x7f9acc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:03.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.844+0000 7f9ad1c49700 1 -- 172.21.15.49:0/3704119437 shutdown_connections 2024-05-15T16:16:03.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.844+0000 7f9ad1c49700 1 -- 172.21.15.49:0/3704119437 wait complete. 2024-05-15T16:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.845+0000 7f9ad1c49700 1 Processor -- start 2024-05-15T16:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.845+0000 7f9ad1c49700 1 -- start start 2024-05-15T16:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.845+0000 7f9ad1c49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc100ee0 0x7f9acc198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.845+0000 7f9ad1c49700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 0x7f9acc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.845+0000 7f9ad1c49700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc199990 con 0x7f9acc101890 2024-05-15T16:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.845+0000 7f9ad1c49700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc199ad0 con 0x7f9acc100ee0 2024-05-15T16:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9acaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 0x7f9acc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:03.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9acb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc100ee0 0x7f9acc198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9acaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 0x7f9acc199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35580/0 (socket says 172.21.15.49:35580) 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9acaffd700 1 -- 172.21.15.49:0/2907763359 learned_addr learned my addr 172.21.15.49:0/2907763359 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9acaffd700 1 -- 172.21.15.49:0/2907763359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc100ee0 msgr2=0x7f9acc198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9acaffd700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc100ee0 0x7f9acc198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9acaffd700 1 -- 172.21.15.49:0/2907763359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac00095f0 con 0x7f9acc101890 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9acaffd700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 0x7f9acc199340 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0005b40 tx=0x7f9ac000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9ac8ff9700 1 -- 172.21.15.49:0/2907763359 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ac001d070 con 0x7f9acc101890 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9ac8ff9700 1 -- 172.21.15.49:0/2907763359 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ac000f460 con 0x7f9acc101890 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc19d800 con 0x7f9acc101890 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9ac8ff9700 1 -- 172.21.15.49:0/2907763359 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ac0005720 con 0x7f9acc101890 2024-05-15T16:16:03.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.846+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc19dcf0 con 0x7f9acc101890 2024-05-15T16:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.848+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc1023c0 con 0x7f9acc101890 2024-05-15T16:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.848+0000 7f9ac8ff9700 1 -- 172.21.15.49:0/2907763359 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9ac000f5d0 con 0x7f9acc101890 2024-05-15T16:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.848+0000 7f9ac8ff9700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9ab406e210 0x7f9ab40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.849+0000 7f9ac8ff9700 1 -- 172.21.15.49:0/2907763359 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9ac008f620 con 0x7f9acc101890 2024-05-15T16:16:03.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.849+0000 7f9acb7fe700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9ab406e210 0x7f9ab40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:03.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.849+0000 7f9acb7fe700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9ab406e210 0x7f9ab40706d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00ac50 tx=0x7f9abc00a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:03.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:03.853+0000 7f9ac8ff9700 1 -- 172.21.15.49:0/2907763359 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9ac0058900 con 0x7f9acc101890 2024-05-15T16:16:04.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.081+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9acc10def0 con 0x7f9ab406e210 2024-05-15T16:16:04.083 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:03 smithi049 ceph-mon[112142]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:04.083 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:03 smithi049 ceph-mon[112142]: pgmap v570: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:04.083 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:03 smithi049 ceph-mon[112142]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:04.083 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:03 smithi049 ceph-mon[112142]: from='client.35082 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:04.083 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:03 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/159170693' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:16:04.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.083+0000 7f9ac8ff9700 1 -- 172.21.15.49:0/2907763359 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9acc10def0 con 0x7f9ab406e210 2024-05-15T16:16:04.084 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:16:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:16:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:16:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:16:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:16:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:16:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:16:04.084 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:16:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:16:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:16:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:16:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:16:04.085 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:16:04.085 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9ab406e210 msgr2=0x7f9ab40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9ab406e210 0x7f9ab40706d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00ac50 tx=0x7f9abc00a380 comp rx=0 tx=0).stop 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 msgr2=0x7f9acc199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 0x7f9acc199340 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0005b40 tx=0x7f9ac000bd70 comp rx=0 tx=0).stop 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 shutdown_connections 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9ab406e210 0x7f9ab40706d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc100ee0 0x7f9acc198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 --2- 172.21.15.49:0/2907763359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9acc101890 0x7f9acc199340 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 >> 172.21.15.49:0/2907763359 conn(0x7f9acc0fc790 msgr2=0x7f9acc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 shutdown_connections 2024-05-15T16:16:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.085+0000 7f9ad1c49700 1 -- 172.21.15.49:0/2907763359 wait complete. 2024-05-15T16:16:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.193+0000 7f5f8e042700 1 -- 172.21.15.49:0/4160441529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 msgr2=0x7f5f8810ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.193+0000 7f5f8e042700 1 --2- 172.21.15.49:0/4160441529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 0x7f5f8810ac80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009910 tx=0x7f5f7c009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:04.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.193+0000 7f5f8e042700 1 -- 172.21.15.49:0/4160441529 shutdown_connections 2024-05-15T16:16:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.193+0000 7f5f8e042700 1 --2- 172.21.15.49:0/4160441529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 0x7f5f8810ac80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.193+0000 7f5f8e042700 1 --2- 172.21.15.49:0/4160441529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88101e70 0x7f5f88102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.193+0000 7f5f8e042700 1 -- 172.21.15.49:0/4160441529 >> 172.21.15.49:0/4160441529 conn(0x7f5f880fb700 msgr2=0x7f5f880fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.193+0000 7f5f8e042700 1 -- 172.21.15.49:0/4160441529 shutdown_connections 2024-05-15T16:16:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.193+0000 7f5f8e042700 1 -- 172.21.15.49:0/4160441529 wait complete. 2024-05-15T16:16:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.194+0000 7f5f8e042700 1 Processor -- start 2024-05-15T16:16:04.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.194+0000 7f5f8e042700 1 -- start start 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.194+0000 7f5f8e042700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88101e70 0x7f5f8819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.194+0000 7f5f8e042700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 0x7f5f8819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.194+0000 7f5f8e042700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8819d490 con 0x7f5f88102790 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.194+0000 7f5f8e042700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f88196a70 con 0x7f5f88101e70 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f86ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 0x7f5f8819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f86ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 0x7f5f8819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35606/0 (socket says 172.21.15.49:35606) 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f86ffd700 1 -- 172.21.15.49:0/2623987955 learned_addr learned my addr 172.21.15.49:0/2623987955 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f877fe700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88101e70 0x7f5f8819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f86ffd700 1 -- 172.21.15.49:0/2623987955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88101e70 msgr2=0x7f5f8819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f86ffd700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88101e70 0x7f5f8819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f86ffd700 1 -- 172.21.15.49:0/2623987955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0095f0 con 0x7f5f88102790 2024-05-15T16:16:04.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f86ffd700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 0x7f5f8819ce70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00ba70 tx=0x7f5f7c00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f84ff9700 1 -- 172.21.15.49:0/2623987955 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f7c01d070 con 0x7f5f88102790 2024-05-15T16:16:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f88196cf0 con 0x7f5f88102790 2024-05-15T16:16:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f84ff9700 1 -- 172.21.15.49:0/2623987955 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f7c003d10 con 0x7f5f88102790 2024-05-15T16:16:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f84ff9700 1 -- 172.21.15.49:0/2623987955 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f7c017400 con 0x7f5f88102790 2024-05-15T16:16:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.195+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f881971e0 con 0x7f5f88102790 2024-05-15T16:16:04.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.196+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f88105590 con 0x7f5f88102790 2024-05-15T16:16:04.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:03 smithi144 ceph-mon[86635]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:04.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:03 smithi144 ceph-mon[86635]: pgmap v570: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:04.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:03 smithi144 ceph-mon[86635]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:04.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:03 smithi144 ceph-mon[86635]: from='client.35082 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:04.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:03 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/159170693' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:16:04.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.197+0000 7f5f84ff9700 1 -- 172.21.15.49:0/2623987955 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5f7c00f6f0 con 0x7f5f88102790 2024-05-15T16:16:04.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.197+0000 7f5f84ff9700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f7006e200 0x7f5f700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:04.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.198+0000 7f5f84ff9700 1 -- 172.21.15.49:0/2623987955 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f7c08f460 con 0x7f5f88102790 2024-05-15T16:16:04.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.198+0000 7f5f877fe700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f7006e200 0x7f5f700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:04.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.198+0000 7f5f877fe700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f7006e200 0x7f5f700706c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5f78005950 tx=0x7f5f780058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:04.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.202+0000 7f5f84ff9700 1 -- 172.21.15.49:0/2623987955 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5f7c05c330 con 0x7f5f88102790 2024-05-15T16:16:04.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.495+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5f88069ab0 con 0x7f5f88102790 2024-05-15T16:16:04.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.496+0000 7f5f84ff9700 1 -- 172.21.15.49:0/2623987955 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5f7c05bec0 con 0x7f5f88102790 2024-05-15T16:16:04.498 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:16:04.498 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:16:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:16:04.498 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:16:04.498 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:16:04.499 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:16:04.499 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:16:04.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f7006e200 msgr2=0x7f5f700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f7006e200 0x7f5f700706c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5f78005950 tx=0x7f5f780058e0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 msgr2=0x7f5f8819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 0x7f5f8819ce70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00ba70 tx=0x7f5f7c00bb50 comp rx=0 tx=0).stop 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 shutdown_connections 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f7006e200 0x7f5f700706c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88101e70 0x7f5f8819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 --2- 172.21.15.49:0/2623987955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f88102790 0x7f5f8819ce70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 >> 172.21.15.49:0/2623987955 conn(0x7f5f880fb700 msgr2=0x7f5f88103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 shutdown_connections 2024-05-15T16:16:04.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:04.498+0000 7f5f8e042700 1 -- 172.21.15.49:0/2623987955 wait complete. 2024-05-15T16:16:04.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:04 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1372065302' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:16:04.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:04 smithi049 ceph-mon[112142]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:04.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:04 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2623987955' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:16:05.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:04 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1372065302' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:16:05.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:04 smithi144 ceph-mon[86635]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:05.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:04 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2623987955' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:16:06.004 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:05 smithi049 ceph-mon[112142]: pgmap v571: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:06.004 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:16:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:05 smithi144 ceph-mon[86635]: pgmap v571: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:16:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:07 smithi144 ceph-mon[86635]: pgmap v572: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:07 smithi049 ceph-mon[112142]: pgmap v572: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:09 smithi144 ceph-mon[86635]: pgmap v573: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:09 smithi049 ceph-mon[112142]: pgmap v573: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:11 smithi144 ceph-mon[86635]: pgmap v574: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:11 smithi049 ceph-mon[112142]: pgmap v574: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:13 smithi144 ceph-mon[86635]: pgmap v575: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:13 smithi049 ceph-mon[112142]: pgmap v575: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:16.113 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:15 smithi049 ceph-mon[112142]: pgmap v576: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:15 smithi144 ceph-mon[86635]: pgmap v576: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:17 smithi144 ceph-mon[86635]: pgmap v577: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:17 smithi049 ceph-mon[112142]: pgmap v577: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:19 smithi144 ceph-mon[86635]: pgmap v578: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:19 smithi049 ceph-mon[112142]: pgmap v578: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:21.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:16:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:16:22.103 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:21 smithi144 ceph-mon[86635]: pgmap v579: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:21 smithi049 ceph-mon[112142]: pgmap v579: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:23 smithi144 ceph-mon[86635]: pgmap v580: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:23 smithi049 ceph-mon[112142]: pgmap v580: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:25 smithi144 ceph-mon[86635]: pgmap v581: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:25 smithi049 ceph-mon[112142]: pgmap v581: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:27 smithi144 ceph-mon[86635]: pgmap v582: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:27 smithi049 ceph-mon[112142]: pgmap v582: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:29 smithi144 ceph-mon[86635]: pgmap v583: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:29 smithi049 ceph-mon[112142]: pgmap v583: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:31 smithi144 ceph-mon[86635]: pgmap v584: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:31 smithi049 ceph-mon[112142]: pgmap v584: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:33 smithi144 ceph-mon[86635]: pgmap v585: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:33 smithi049 ceph-mon[112142]: pgmap v585: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:34.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 -- 172.21.15.49:0/3521596983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e28100f00 msgr2=0x7f5e281012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 --2- 172.21.15.49:0/3521596983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e28100f00 0x7f5e281012e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009910 tx=0x7f5e18009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 -- 172.21.15.49:0/3521596983 shutdown_connections 2024-05-15T16:16:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 --2- 172.21.15.49:0/3521596983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e281018b0 0x7f5e28105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 --2- 172.21.15.49:0/3521596983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e28100f00 0x7f5e281012e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 -- 172.21.15.49:0/3521596983 >> 172.21.15.49:0/3521596983 conn(0x7f5e280fc7d0 msgr2=0x7f5e280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 -- 172.21.15.49:0/3521596983 shutdown_connections 2024-05-15T16:16:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 -- 172.21.15.49:0/3521596983 wait complete. 2024-05-15T16:16:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 Processor -- start 2024-05-15T16:16:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.621+0000 7f5e2e901700 1 -- start start 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e2e901700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e28100f00 0x7f5e2819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e2e901700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e281018b0 0x7f5e2819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e2e901700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2819d460 con 0x7f5e281018b0 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e2e901700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e28196a90 con 0x7f5e28100f00 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e277fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e281018b0 0x7f5e2819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e277fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e281018b0 0x7f5e2819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42006/0 (socket says 172.21.15.49:42006) 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e277fe700 1 -- 172.21.15.49:0/1436910100 learned_addr learned my addr 172.21.15.49:0/1436910100 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e27fff700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e28100f00 0x7f5e2819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e277fe700 1 -- 172.21.15.49:0/1436910100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e28100f00 msgr2=0x7f5e2819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e277fe700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e28100f00 0x7f5e2819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:34.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.622+0000 7f5e277fe700 1 -- 172.21.15.49:0/1436910100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180095f0 con 0x7f5e281018b0 2024-05-15T16:16:34.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.623+0000 7f5e277fe700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e281018b0 0x7f5e2819ce40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c8f0 tx=0x7f5e1c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:34.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.623+0000 7f5e257fa700 1 -- 172.21.15.49:0/1436910100 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1c0078a0 con 0x7f5e281018b0 2024-05-15T16:16:34.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.623+0000 7f5e257fa700 1 -- 172.21.15.49:0/1436910100 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e1c00f450 con 0x7f5e281018b0 2024-05-15T16:16:34.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.623+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e28196d70 con 0x7f5e281018b0 2024-05-15T16:16:34.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.623+0000 7f5e257fa700 1 -- 172.21.15.49:0/1436910100 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1c018610 con 0x7f5e281018b0 2024-05-15T16:16:34.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.623+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e281972c0 con 0x7f5e281018b0 2024-05-15T16:16:34.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.624+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e281092a0 con 0x7f5e281018b0 2024-05-15T16:16:34.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.625+0000 7f5e257fa700 1 -- 172.21.15.49:0/1436910100 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5e1c00f5c0 con 0x7f5e281018b0 2024-05-15T16:16:34.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.625+0000 7f5e257fa700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1006e210 0x7f5e100706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:34.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.625+0000 7f5e257fa700 1 -- 172.21.15.49:0/1436910100 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5e1c08e060 con 0x7f5e281018b0 2024-05-15T16:16:34.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.625+0000 7f5e27fff700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1006e210 0x7f5e100706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:34.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.626+0000 7f5e27fff700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1006e210 0x7f5e100706d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5e18006010 tx=0x7f5e1801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:34.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.630+0000 7f5e257fa700 1 -- 172.21.15.49:0/1436910100 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5e1c05aef0 con 0x7f5e281018b0 2024-05-15T16:16:34.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.859+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e28197f00 con 0x7f5e1006e210 2024-05-15T16:16:34.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.860+0000 7f5e257fa700 1 -- 172.21.15.49:0/1436910100 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5e28197f00 con 0x7f5e1006e210 2024-05-15T16:16:34.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1006e210 msgr2=0x7f5e100706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:34.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1006e210 0x7f5e100706d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5e18006010 tx=0x7f5e1801a040 comp rx=0 tx=0).stop 2024-05-15T16:16:34.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e281018b0 msgr2=0x7f5e2819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:34.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e281018b0 0x7f5e2819ce40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c8f0 tx=0x7f5e1c00cc00 comp rx=0 tx=0).stop 2024-05-15T16:16:34.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 shutdown_connections 2024-05-15T16:16:34.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1006e210 0x7f5e100706d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:34.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e28100f00 0x7f5e2819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:34.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 --2- 172.21.15.49:0/1436910100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e281018b0 0x7f5e2819ce40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:34.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 >> 172.21.15.49:0/1436910100 conn(0x7f5e280fc7d0 msgr2=0x7f5e280fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:34.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 shutdown_connections 2024-05-15T16:16:34.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.863+0000 7f5e2e901700 1 -- 172.21.15.49:0/1436910100 wait complete. 2024-05-15T16:16:34.874 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:16:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.972+0000 7f0608035700 1 -- 172.21.15.49:0/26989013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600100320 msgr2=0x7f06001007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.972+0000 7f0608035700 1 --2- 172.21.15.49:0/26989013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600100320 0x7f06001007a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009910 tx=0x7f05f4009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.972+0000 7f0608035700 1 -- 172.21.15.49:0/26989013 shutdown_connections 2024-05-15T16:16:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.972+0000 7f0608035700 1 --2- 172.21.15.49:0/26989013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600100320 0x7f06001007a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.972+0000 7f0608035700 1 --2- 172.21.15.49:0/26989013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0600104b70 0x7f0600104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.972+0000 7f0608035700 1 -- 172.21.15.49:0/26989013 >> 172.21.15.49:0/26989013 conn(0x7f0600075ff0 msgr2=0x7f06000fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.972+0000 7f0608035700 1 -- 172.21.15.49:0/26989013 shutdown_connections 2024-05-15T16:16:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.972+0000 7f0608035700 1 -- 172.21.15.49:0/26989013 wait complete. 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f0608035700 1 Processor -- start 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f0608035700 1 -- start start 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f0608035700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0600100320 0x7f0600198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f0608035700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600104b70 0x7f06001993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f0608035700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0600199aa0 con 0x7f0600104b70 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f0608035700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060019d720 con 0x7f0600100320 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f06055d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600104b70 0x7f06001993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f06055d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600104b70 0x7f06001993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42026/0 (socket says 172.21.15.49:42026) 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f06055d0700 1 -- 172.21.15.49:0/87449310 learned_addr learned my addr 172.21.15.49:0/87449310 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f0605dd1700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0600100320 0x7f0600198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f06055d0700 1 -- 172.21.15.49:0/87449310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0600100320 msgr2=0x7f0600198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f06055d0700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0600100320 0x7f0600198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f06055d0700 1 -- 172.21.15.49:0/87449310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40095f0 con 0x7f0600104b70 2024-05-15T16:16:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.973+0000 7f06055d0700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600104b70 0x7f06001993c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f05f400bd70 tx=0x7f05f400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.974+0000 7f05f2ffd700 1 -- 172.21.15.49:0/87449310 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05f401d070 con 0x7f0600104b70 2024-05-15T16:16:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.974+0000 7f05f2ffd700 1 -- 172.21.15.49:0/87449310 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05f40056e0 con 0x7f0600104b70 2024-05-15T16:16:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.974+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060019d9a0 con 0x7f0600104b70 2024-05-15T16:16:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.974+0000 7f05f2ffd700 1 -- 172.21.15.49:0/87449310 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05f4017400 con 0x7f0600104b70 2024-05-15T16:16:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.974+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060019de90 con 0x7f0600104b70 2024-05-15T16:16:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.975+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0600101330 con 0x7f0600104b70 2024-05-15T16:16:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.975+0000 7f05f2ffd700 1 -- 172.21.15.49:0/87449310 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f05f4003a60 con 0x7f0600104b70 2024-05-15T16:16:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.976+0000 7f05f2ffd700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f05ec06e200 0x7f05ec0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.976+0000 7f0605dd1700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f05ec06e200 0x7f05ec0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.976+0000 7f05f2ffd700 1 -- 172.21.15.49:0/87449310 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f05f408f5a0 con 0x7f0600104b70 2024-05-15T16:16:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.976+0000 7f0605dd1700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f05ec06e200 0x7f05ec0706c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005950 tx=0x7f05fc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:34.981+0000 7f05f2ffd700 1 -- 172.21.15.49:0/87449310 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f05f4058900 con 0x7f0600104b70 2024-05-15T16:16:35.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.209+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f060010ce60 con 0x7f05ec06e200 2024-05-15T16:16:35.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.210+0000 7f05f2ffd700 1 -- 172.21.15.49:0/87449310 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f060010ce60 con 0x7f05ec06e200 2024-05-15T16:16:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f05ec06e200 msgr2=0x7f05ec0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f05ec06e200 0x7f05ec0706c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005950 tx=0x7f05fc0058e0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600104b70 msgr2=0x7f06001993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600104b70 0x7f06001993c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f05f400bd70 tx=0x7f05f400be50 comp rx=0 tx=0).stop 2024-05-15T16:16:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 shutdown_connections 2024-05-15T16:16:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f05ec06e200 0x7f05ec0706c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0600100320 0x7f0600198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 --2- 172.21.15.49:0/87449310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0600104b70 0x7f06001993c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 >> 172.21.15.49:0/87449310 conn(0x7f0600075ff0 msgr2=0x7f060010a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 shutdown_connections 2024-05-15T16:16:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.212+0000 7f0608035700 1 -- 172.21.15.49:0/87449310 wait complete. 2024-05-15T16:16:35.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.321+0000 7fde455c0700 1 -- 172.21.15.49:0/2117686027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 msgr2=0x7fde4010ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.321+0000 7fde455c0700 1 --2- 172.21.15.49:0/2117686027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 0x7fde4010ac30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fde28009910 tx=0x7fde28009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.321+0000 7fde455c0700 1 -- 172.21.15.49:0/2117686027 shutdown_connections 2024-05-15T16:16:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.321+0000 7fde455c0700 1 --2- 172.21.15.49:0/2117686027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 0x7fde4010ac30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.321+0000 7fde455c0700 1 --2- 172.21.15.49:0/2117686027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde40101e20 0x7fde40102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.321+0000 7fde455c0700 1 -- 172.21.15.49:0/2117686027 >> 172.21.15.49:0/2117686027 conn(0x7fde40076800 msgr2=0x7fde400fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.321+0000 7fde455c0700 1 -- 172.21.15.49:0/2117686027 shutdown_connections 2024-05-15T16:16:35.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.321+0000 7fde455c0700 1 -- 172.21.15.49:0/2117686027 wait complete. 2024-05-15T16:16:35.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde455c0700 1 Processor -- start 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde455c0700 1 -- start start 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde455c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde40101e20 0x7fde400ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde455c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 0x7fde400fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde455c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40100500 con 0x7fde40102740 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde455c0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40100640 con 0x7fde40101e20 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde3e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 0x7fde400fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde3e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 0x7fde400fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42040/0 (socket says 172.21.15.49:42040) 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde3e7fc700 1 -- 172.21.15.49:0/2540906116 learned_addr learned my addr 172.21.15.49:0/2540906116 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.322+0000 7fde3effd700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde40101e20 0x7fde400ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.323+0000 7fde3e7fc700 1 -- 172.21.15.49:0/2540906116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde40101e20 msgr2=0x7fde400ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.323+0000 7fde3e7fc700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde40101e20 0x7fde400ffa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.323+0000 7fde3e7fc700 1 -- 172.21.15.49:0/2540906116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280095f0 con 0x7fde40102740 2024-05-15T16:16:35.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.323+0000 7fde3e7fc700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 0x7fde400fffc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fde280092e0 tx=0x7fde2800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.323+0000 7fde37fff700 1 -- 172.21.15.49:0/2540906116 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde2801d070 con 0x7fde40102740 2024-05-15T16:16:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.323+0000 7fde37fff700 1 -- 172.21.15.49:0/2540906116 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde28003d50 con 0x7fde40102740 2024-05-15T16:16:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.323+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde401008c0 con 0x7fde40102740 2024-05-15T16:16:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.323+0000 7fde37fff700 1 -- 172.21.15.49:0/2540906116 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde28017400 con 0x7fde40102740 2024-05-15T16:16:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.323+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde401a3040 con 0x7fde40102740 2024-05-15T16:16:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.324+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde40068f60 con 0x7fde40102740 2024-05-15T16:16:35.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.325+0000 7fde37fff700 1 -- 172.21.15.49:0/2540906116 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fde2800f6f0 con 0x7fde40102740 2024-05-15T16:16:35.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.325+0000 7fde37fff700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde2c06e210 0x7fde2c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:35.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.325+0000 7fde37fff700 1 -- 172.21.15.49:0/2540906116 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fde2808f520 con 0x7fde40102740 2024-05-15T16:16:35.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.325+0000 7fde3effd700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde2c06e210 0x7fde2c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:35.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.326+0000 7fde3effd700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde2c06e210 0x7fde2c0706d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fde300060b0 tx=0x7fde3000c040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:35.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.330+0000 7fde37fff700 1 -- 172.21.15.49:0/2540906116 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fde28058800 con 0x7fde40102740 2024-05-15T16:16:35.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.549+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde4010de80 con 0x7fde2c06e210 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.560+0000 7fde37fff700 1 -- 172.21.15.49:0/2540906116 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fde4010de80 con 0x7fde2c06e210 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (19m) 13m ago 31m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (31m) 13m ago 31m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (30m) 12m ago 30m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (17m) 13m ago 31m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (17m) 12m ago 30m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (19m) 13m ago 31m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (13m) 13m ago 25m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:16:35.561 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (13m) 13m ago 25m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (12m) 12m ago 25m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (25m) 12m ago 25m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (22m) 13m ago 32m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (21m) 12m ago 30m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (18m) 13m ago 32m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (17m) 12m ago 30m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (20m) 13m ago 31m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (20m) 12m ago 30m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (16m) 13m ago 29m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (16m) 13m ago 29m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (15m) 13m ago 28m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (14m) 12m ago 28m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (14m) 12m ago 27m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (13m) 12m ago 27m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:16:35.562 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (20m) 13m ago 30m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.563+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde2c06e210 msgr2=0x7fde2c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.563+0000 7fde455c0700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde2c06e210 0x7fde2c0706d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fde300060b0 tx=0x7fde3000c040 comp rx=0 tx=0).stop 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.563+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 msgr2=0x7fde400fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.563+0000 7fde455c0700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 0x7fde400fffc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fde280092e0 tx=0x7fde2800bd70 comp rx=0 tx=0).stop 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.563+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 shutdown_connections 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.563+0000 7fde455c0700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde2c06e210 0x7fde2c0706d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.563+0000 7fde455c0700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde40101e20 0x7fde400ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.563+0000 7fde455c0700 1 --2- 172.21.15.49:0/2540906116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde40102740 0x7fde400fffc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.563+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 >> 172.21.15.49:0/2540906116 conn(0x7fde40076800 msgr2=0x7fde40103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.564+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 shutdown_connections 2024-05-15T16:16:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.564+0000 7fde455c0700 1 -- 172.21.15.49:0/2540906116 wait complete. 2024-05-15T16:16:35.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.670+0000 7f7e1294f700 1 -- 172.21.15.49:0/509254642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c1007a0 msgr2=0x7f7e0c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.670+0000 7f7e1294f700 1 --2- 172.21.15.49:0/509254642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c1007a0 0x7f7e0c1047f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009910 tx=0x7f7e00009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:35.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.670+0000 7f7e1294f700 1 -- 172.21.15.49:0/509254642 shutdown_connections 2024-05-15T16:16:35.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.670+0000 7f7e1294f700 1 --2- 172.21.15.49:0/509254642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c1007a0 0x7f7e0c1047f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.670+0000 7f7e1294f700 1 --2- 172.21.15.49:0/509254642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e0c0ffdf0 0x7f7e0c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.670+0000 7f7e1294f700 1 -- 172.21.15.49:0/509254642 >> 172.21.15.49:0/509254642 conn(0x7f7e0c0fb6a0 msgr2=0x7f7e0c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:35.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.671+0000 7f7e1294f700 1 -- 172.21.15.49:0/509254642 shutdown_connections 2024-05-15T16:16:35.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.671+0000 7f7e1294f700 1 -- 172.21.15.49:0/509254642 wait complete. 2024-05-15T16:16:35.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.671+0000 7f7e1294f700 1 Processor -- start 2024-05-15T16:16:35.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.671+0000 7f7e1294f700 1 -- start start 2024-05-15T16:16:35.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e1294f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c0ffdf0 0x7f7e0c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e1294f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e0c1007a0 0x7f7e0c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e1294f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c1986a0 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e1294f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c198810 con 0x7f7e0c1007a0 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e0bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c0ffdf0 0x7f7e0c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e0bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c0ffdf0 0x7f7e0c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42056/0 (socket says 172.21.15.49:42056) 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e0bfff700 1 -- 172.21.15.49:0/53380250 learned_addr learned my addr 172.21.15.49:0/53380250 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e0b7fe700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e0c1007a0 0x7f7e0c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e0bfff700 1 -- 172.21.15.49:0/53380250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e0c1007a0 msgr2=0x7f7e0c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e0bfff700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e0c1007a0 0x7f7e0c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e0bfff700 1 -- 172.21.15.49:0/53380250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e000095f0 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e0bfff700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c0ffdf0 0x7f7e0c197b90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00e920 tx=0x7f7dfc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e097fa700 1 -- 172.21.15.49:0/53380250 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dfc00c9d0 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e097fa700 1 -- 172.21.15.49:0/53380250 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dfc010430 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c1a0c00 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.672+0000 7f7e097fa700 1 -- 172.21.15.49:0/53380250 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dfc018630 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.673+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c1a1070 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.673+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c1081f0 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.674+0000 7f7e097fa700 1 -- 172.21.15.49:0/53380250 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7dfc0105a0 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.674+0000 7f7e097fa700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7df406e200 0x7f7df40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:35.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.675+0000 7f7e097fa700 1 -- 172.21.15.49:0/53380250 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7dfc014070 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.675+0000 7f7e0b7fe700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7df406e200 0x7f7df40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:35.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.675+0000 7f7e0b7fe700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7df406e200 0x7f7df40706c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7e000059d0 tx=0x7f7e00005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:35.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.679+0000 7f7e097fa700 1 -- 172.21.15.49:0/53380250 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7dfc053850 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:35 smithi049 ceph-mon[112142]: pgmap v586: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:35 smithi049 ceph-mon[112142]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:16:35.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.976+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7e0c069790 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.977+0000 7f7e097fa700 1 -- 172.21.15.49:0/53380250 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f7dfc057750 con 0x7f7e0c0ffdf0 2024-05-15T16:16:35.977 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:16:35.977 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:16:35.977 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:16:35.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:16:35.977 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:16:35.977 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:16:35.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:16:35.977 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:16:35.978 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7df406e200 msgr2=0x7f7df40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7df406e200 0x7f7df40706c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7e000059d0 tx=0x7f7e00005940 comp rx=0 tx=0).stop 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c0ffdf0 msgr2=0x7f7e0c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c0ffdf0 0x7f7e0c197b90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00e920 tx=0x7f7dfc00ec30 comp rx=0 tx=0).stop 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 shutdown_connections 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7df406e200 0x7f7df40706c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e0c0ffdf0 0x7f7e0c197b90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 --2- 172.21.15.49:0/53380250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e0c1007a0 0x7f7e0c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 >> 172.21.15.49:0/53380250 conn(0x7f7e0c0fb6a0 msgr2=0x7f7e0c0fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 shutdown_connections 2024-05-15T16:16:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:35.979+0000 7f7e1294f700 1 -- 172.21.15.49:0/53380250 wait complete. 2024-05-15T16:16:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.087+0000 7fca7443e700 1 -- 172.21.15.49:0/1422431253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c100770 msgr2=0x7fca6c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.087+0000 7fca7443e700 1 --2- 172.21.15.49:0/1422431253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c100770 0x7fca6c1047c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fca68009910 tx=0x7fca68009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.087+0000 7fca7443e700 1 -- 172.21.15.49:0/1422431253 shutdown_connections 2024-05-15T16:16:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.087+0000 7fca7443e700 1 --2- 172.21.15.49:0/1422431253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c100770 0x7fca6c1047c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.087+0000 7fca7443e700 1 --2- 172.21.15.49:0/1422431253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6c0ffdc0 0x7fca6c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.087+0000 7fca7443e700 1 -- 172.21.15.49:0/1422431253 >> 172.21.15.49:0/1422431253 conn(0x7fca6c076800 msgr2=0x7fca6c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.087+0000 7fca7443e700 1 -- 172.21.15.49:0/1422431253 shutdown_connections 2024-05-15T16:16:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.087+0000 7fca7443e700 1 -- 172.21.15.49:0/1422431253 wait complete. 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.088+0000 7fca7443e700 1 Processor -- start 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.088+0000 7fca7443e700 1 -- start start 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.088+0000 7fca7443e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c0ffdc0 0x7fca6c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.088+0000 7fca7443e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6c100770 0x7fca6c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.088+0000 7fca7443e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c19d410 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.088+0000 7fca7443e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c196a40 con 0x7fca6c100770 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.088+0000 7fca721da700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c0ffdc0 0x7fca6c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.088+0000 7fca721da700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c0ffdc0 0x7fca6c19c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42068/0 (socket says 172.21.15.49:42068) 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.088+0000 7fca721da700 1 -- 172.21.15.49:0/1529307245 learned_addr learned my addr 172.21.15.49:0/1529307245 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca719d9700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6c100770 0x7fca6c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca721da700 1 -- 172.21.15.49:0/1529307245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6c100770 msgr2=0x7fca6c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca721da700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6c100770 0x7fca6c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca721da700 1 -- 172.21.15.49:0/1529307245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca680095f0 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca721da700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c0ffdc0 0x7fca6c19c8b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d900 tx=0x7fca6000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca5f7fe700 1 -- 172.21.15.49:0/1529307245 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca600041f0 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca5f7fe700 1 -- 172.21.15.49:0/1529307245 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca60004cf0 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c196d20 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca5f7fe700 1 -- 172.21.15.49:0/1529307245 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca6000b750 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.089+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6c197270 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.090+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6c1012a0 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.091+0000 7fca5f7fe700 1 -- 172.21.15.49:0/1529307245 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fca60004750 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.091+0000 7fca5f7fe700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca5806e200 0x7fca580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.091+0000 7fca5f7fe700 1 -- 172.21.15.49:0/1529307245 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fca6008e070 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.091+0000 7fca719d9700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca5806e200 0x7fca580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.092+0000 7fca719d9700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca5806e200 0x7fca580706c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca6c197bb0 tx=0x7fca6801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.096+0000 7fca5f7fe700 1 -- 172.21.15.49:0/1529307245 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fca6005ae80 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:35 smithi144 ceph-mon[86635]: pgmap v586: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:35 smithi144 ceph-mon[86635]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:16:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.357+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fca6c10cd60 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.357+0000 7fca5f7fe700 1 -- 172.21.15.49:0/1529307245 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7fca6005aa10 con 0x7fca6c0ffdc0 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:16:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:16:36.359 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:16:36.360 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:16:36.361 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:16:36.361 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:16:36.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.360+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca5806e200 msgr2=0x7fca580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.360+0000 7fca7443e700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca5806e200 0x7fca580706c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca6c197bb0 tx=0x7fca6801a040 comp rx=0 tx=0).stop 2024-05-15T16:16:36.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.360+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c0ffdc0 msgr2=0x7fca6c19c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.360+0000 7fca7443e700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c0ffdc0 0x7fca6c19c8b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d900 tx=0x7fca6000dc10 comp rx=0 tx=0).stop 2024-05-15T16:16:36.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.361+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 shutdown_connections 2024-05-15T16:16:36.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.361+0000 7fca7443e700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca5806e200 0x7fca580706c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.361+0000 7fca7443e700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca6c0ffdc0 0x7fca6c19c8b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.361+0000 7fca7443e700 1 --2- 172.21.15.49:0/1529307245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6c100770 0x7fca6c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.361+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 >> 172.21.15.49:0/1529307245 conn(0x7fca6c076800 msgr2=0x7fca6c103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:36.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.361+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 shutdown_connections 2024-05-15T16:16:36.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.361+0000 7fca7443e700 1 -- 172.21.15.49:0/1529307245 wait complete. 2024-05-15T16:16:36.362 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:16:36.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.472+0000 7fc581fe3700 1 -- 172.21.15.49:0/2542774658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c0ff6f0 msgr2=0x7fc57c0ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.472+0000 7fc581fe3700 1 --2- 172.21.15.49:0/2542774658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c0ff6f0 0x7fc57c0ffad0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009910 tx=0x7fc56c009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:36.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.472+0000 7fc581fe3700 1 -- 172.21.15.49:0/2542774658 shutdown_connections 2024-05-15T16:16:36.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.472+0000 7fc581fe3700 1 --2- 172.21.15.49:0/2542774658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc57c1000a0 0x7fc57c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.472+0000 7fc581fe3700 1 --2- 172.21.15.49:0/2542774658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c0ff6f0 0x7fc57c0ffad0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.472+0000 7fc581fe3700 1 -- 172.21.15.49:0/2542774658 >> 172.21.15.49:0/2542774658 conn(0x7fc57c074e50 msgr2=0x7fc57c075260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:36.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.472+0000 7fc581fe3700 1 -- 172.21.15.49:0/2542774658 shutdown_connections 2024-05-15T16:16:36.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.472+0000 7fc581fe3700 1 -- 172.21.15.49:0/2542774658 wait complete. 2024-05-15T16:16:36.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.472+0000 7fc581fe3700 1 Processor -- start 2024-05-15T16:16:36.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc581fe3700 1 -- start start 2024-05-15T16:16:36.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc581fe3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc57c0ff6f0 0x7fc57c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:36.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc581fe3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c1000a0 0x7fc57c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:36.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc581fe3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c19c360 con 0x7fc57c1000a0 2024-05-15T16:16:36.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc581fe3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c195940 con 0x7fc57c0ff6f0 2024-05-15T16:16:36.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc57affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c1000a0 0x7fc57c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:36.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc57affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c1000a0 0x7fc57c19bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42092/0 (socket says 172.21.15.49:42092) 2024-05-15T16:16:36.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc57affd700 1 -- 172.21.15.49:0/844767655 learned_addr learned my addr 172.21.15.49:0/844767655 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc57b7fe700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc57c0ff6f0 0x7fc57c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc57affd700 1 -- 172.21.15.49:0/844767655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc57c0ff6f0 msgr2=0x7fc57c19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc57affd700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc57c0ff6f0 0x7fc57c19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc57affd700 1 -- 172.21.15.49:0/844767655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c0095f0 con 0x7fc57c1000a0 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.473+0000 7fc57b7fe700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc57c0ff6f0 0x7fc57c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.474+0000 7fc57affd700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c1000a0 0x7fc57c19bd40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc57000d900 tx=0x7fc57000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.474+0000 7fc578ff9700 1 -- 172.21.15.49:0/844767655 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc570009900 con 0x7fc57c1000a0 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.474+0000 7fc578ff9700 1 -- 172.21.15.49:0/844767655 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc570010460 con 0x7fc57c1000a0 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.474+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c195c20 con 0x7fc57c1000a0 2024-05-15T16:16:36.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.474+0000 7fc578ff9700 1 -- 172.21.15.49:0/844767655 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc57000f5f0 con 0x7fc57c1000a0 2024-05-15T16:16:36.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.474+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c196170 con 0x7fc57c1000a0 2024-05-15T16:16:36.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.475+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c100a70 con 0x7fc57c1000a0 2024-05-15T16:16:36.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.476+0000 7fc578ff9700 1 -- 172.21.15.49:0/844767655 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc5700105d0 con 0x7fc57c1000a0 2024-05-15T16:16:36.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.476+0000 7fc578ff9700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc56406e200 0x7fc5640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:36.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.476+0000 7fc578ff9700 1 -- 172.21.15.49:0/844767655 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc57008e020 con 0x7fc57c1000a0 2024-05-15T16:16:36.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.476+0000 7fc57b7fe700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc56406e200 0x7fc5640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:36.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.477+0000 7fc57b7fe700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc56406e200 0x7fc5640706c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0059d0 tx=0x7fc56c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:36.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.481+0000 7fc578ff9700 1 -- 172.21.15.49:0/844767655 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc57005ae30 con 0x7fc57c1000a0 2024-05-15T16:16:36.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.717+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc57c10c5c0 con 0x7fc56406e200 2024-05-15T16:16:36.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.718+0000 7fc578ff9700 1 -- 172.21.15.49:0/844767655 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc57c10c5c0 con 0x7fc56406e200 2024-05-15T16:16:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:16:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.720+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc56406e200 msgr2=0x7fc5640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.720+0000 7fc581fe3700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc56406e200 0x7fc5640706c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0059d0 tx=0x7fc56c01a040 comp rx=0 tx=0).stop 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.720+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c1000a0 msgr2=0x7fc57c19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.720+0000 7fc581fe3700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c1000a0 0x7fc57c19bd40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc57000d900 tx=0x7fc57000dcc0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.720+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 shutdown_connections 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.720+0000 7fc581fe3700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc56406e200 0x7fc5640706c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.720+0000 7fc581fe3700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc57c0ff6f0 0x7fc57c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.720+0000 7fc581fe3700 1 --2- 172.21.15.49:0/844767655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc57c1000a0 0x7fc57c19bd40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.720+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 >> 172.21.15.49:0/844767655 conn(0x7fc57c074e50 msgr2=0x7fc57c102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:36.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.721+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 shutdown_connections 2024-05-15T16:16:36.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.721+0000 7fc581fe3700 1 -- 172.21.15.49:0/844767655 wait complete. 2024-05-15T16:16:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.829+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/1327310825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 msgr2=0x7fb3e41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.829+0000 7fb3e8ee3700 1 --2- 172.21.15.49:0/1327310825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 0x7fb3e41012c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009910 tx=0x7fb3d4009c20 comp rx=0 tx=0).stop 2024-05-15T16:16:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.829+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/1327310825 shutdown_connections 2024-05-15T16:16:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.829+0000 7fb3e8ee3700 1 --2- 172.21.15.49:0/1327310825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e4101890 0x7fb3e41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.829+0000 7fb3e8ee3700 1 --2- 172.21.15.49:0/1327310825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 0x7fb3e41012c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.829+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/1327310825 >> 172.21.15.49:0/1327310825 conn(0x7fb3e40fc790 msgr2=0x7fb3e40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.829+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/1327310825 shutdown_connections 2024-05-15T16:16:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.829+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/1327310825 wait complete. 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.830+0000 7fb3e8ee3700 1 Processor -- start 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.830+0000 7fb3e8ee3700 1 -- start start 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.830+0000 7fb3e8ee3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 0x7fb3e4198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.830+0000 7fb3e8ee3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e4101890 0x7fb3e4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.830+0000 7fb3e8ee3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41999c0 con 0x7fb3e4100ee0 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.830+0000 7fb3e8ee3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4199b00 con 0x7fb3e4101890 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 0x7fb3e4198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 0x7fb3e4198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42118/0 (socket says 172.21.15.49:42118) 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e259c700 1 -- 172.21.15.49:0/3815289170 learned_addr learned my addr 172.21.15.49:0/3815289170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e1d9b700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e4101890 0x7fb3e4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e259c700 1 -- 172.21.15.49:0/3815289170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e4101890 msgr2=0x7fb3e4199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e259c700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e4101890 0x7fb3e4199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e259c700 1 -- 172.21.15.49:0/3815289170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d40095f0 con 0x7fb3e4100ee0 2024-05-15T16:16:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e259c700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 0x7fb3e4198e30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40092e0 tx=0x7fb3d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3d37fe700 1 -- 172.21.15.49:0/3815289170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3d401d070 con 0x7fb3e4100ee0 2024-05-15T16:16:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3d37fe700 1 -- 172.21.15.49:0/3815289170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3d400f460 con 0x7fb3e4100ee0 2024-05-15T16:16:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e419d7e0 con 0x7fb3e4100ee0 2024-05-15T16:16:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3d37fe700 1 -- 172.21.15.49:0/3815289170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3d4017630 con 0x7fb3e4100ee0 2024-05-15T16:16:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.831+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e419dcd0 con 0x7fb3e4100ee0 2024-05-15T16:16:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.832+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e41023c0 con 0x7fb3e4100ee0 2024-05-15T16:16:36.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.833+0000 7fb3d37fe700 1 -- 172.21.15.49:0/3815289170 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb3d400f5d0 con 0x7fb3e4100ee0 2024-05-15T16:16:36.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.833+0000 7fb3d37fe700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb3cc06e210 0x7fb3cc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:16:36.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.833+0000 7fb3d37fe700 1 -- 172.21.15.49:0/3815289170 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb3d408f660 con 0x7fb3e4100ee0 2024-05-15T16:16:36.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.834+0000 7fb3e1d9b700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb3cc06e210 0x7fb3cc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:16:36.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.834+0000 7fb3e1d9b700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb3cc06e210 0x7fb3cc0706d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8005ea0 tx=0x7fb3d8005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:16:36.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:36.838+0000 7fb3d37fe700 1 -- 172.21.15.49:0/3815289170 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb3d4058920 con 0x7fb3e4100ee0 2024-05-15T16:16:37.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.129+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb3e410df70 con 0x7fb3e4100ee0 2024-05-15T16:16:37.130 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:36 smithi049 ceph-mon[112142]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:37.130 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:36 smithi049 ceph-mon[112142]: from='client.35110 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:37.130 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:36 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/53380250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:16:37.130 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:36 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1529307245' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:16:37.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.129+0000 7fb3d37fe700 1 -- 172.21.15.49:0/3815289170 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fb3d405bfe0 con 0x7fb3e4100ee0 2024-05-15T16:16:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:16:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:16:37.131 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:16:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:16:37.131 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:16:37.131 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:16:37.131 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:16:37.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb3cc06e210 msgr2=0x7fb3cc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:37.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb3cc06e210 0x7fb3cc0706d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8005ea0 tx=0x7fb3d8005e30 comp rx=0 tx=0).stop 2024-05-15T16:16:37.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 msgr2=0x7fb3e4198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:16:37.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 0x7fb3e4198e30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40092e0 tx=0x7fb3d400be20 comp rx=0 tx=0).stop 2024-05-15T16:16:37.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 shutdown_connections 2024-05-15T16:16:37.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb3cc06e210 0x7fb3cc0706d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:37.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3e4100ee0 0x7fb3e4198e30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:37.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 --2- 172.21.15.49:0/3815289170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e4101890 0x7fb3e4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:16:37.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 >> 172.21.15.49:0/3815289170 conn(0x7fb3e40fc790 msgr2=0x7fb3e4074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:16:37.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 shutdown_connections 2024-05-15T16:16:37.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:16:37.133+0000 7fb3e8ee3700 1 -- 172.21.15.49:0/3815289170 wait complete. 2024-05-15T16:16:37.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:36 smithi144 ceph-mon[86635]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:37.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:36 smithi144 ceph-mon[86635]: from='client.35110 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:37.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:36 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/53380250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:16:37.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:36 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1529307245' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:16:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:37 smithi144 ceph-mon[86635]: pgmap v587: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:37 smithi144 ceph-mon[86635]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:37 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3815289170' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:16:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:37 smithi049 ceph-mon[112142]: pgmap v587: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:37 smithi049 ceph-mon[112142]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:16:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:37 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3815289170' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:16:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:39 smithi144 ceph-mon[86635]: pgmap v588: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:39 smithi049 ceph-mon[112142]: pgmap v588: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:41 smithi144 ceph-mon[86635]: pgmap v589: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:41 smithi049 ceph-mon[112142]: pgmap v589: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:43 smithi144 ceph-mon[86635]: pgmap v590: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:43 smithi049 ceph-mon[112142]: pgmap v590: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:45 smithi144 ceph-mon[86635]: pgmap v591: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:45 smithi049 ceph-mon[112142]: pgmap v591: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:47 smithi144 ceph-mon[86635]: pgmap v592: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:47 smithi049 ceph-mon[112142]: pgmap v592: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:49 smithi144 ceph-mon[86635]: pgmap v593: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:49 smithi049 ceph-mon[112142]: pgmap v593: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:16:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:16:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:51 smithi144 ceph-mon[86635]: pgmap v594: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:51 smithi049 ceph-mon[112142]: pgmap v594: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:53 smithi144 ceph-mon[86635]: pgmap v595: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:53 smithi049 ceph-mon[112142]: pgmap v595: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:55 smithi144 ceph-mon[86635]: pgmap v596: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:55 smithi049 ceph-mon[112142]: pgmap v596: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:57 smithi144 ceph-mon[86635]: pgmap v597: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:16:58.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:57 smithi049 ceph-mon[112142]: pgmap v597: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:16:59 smithi144 ceph-mon[86635]: pgmap v598: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:16:59 smithi049 ceph-mon[112142]: pgmap v598: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:01 smithi144 ceph-mon[86635]: pgmap v599: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:01 smithi049 ceph-mon[112142]: pgmap v599: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:03 smithi144 ceph-mon[86635]: pgmap v600: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:03 smithi049 ceph-mon[112142]: pgmap v600: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:05 smithi144 ceph-mon[86635]: pgmap v601: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:17:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:05 smithi049 ceph-mon[112142]: pgmap v601: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:17:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.256+0000 7f0b41cbc700 1 -- 172.21.15.49:0/94790187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c100ed0 msgr2=0x7f0b3c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.256+0000 7f0b41cbc700 1 --2- 172.21.15.49:0/94790187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c100ed0 0x7f0b3c1012b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009910 tx=0x7f0b2c009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.256+0000 7f0b41cbc700 1 -- 172.21.15.49:0/94790187 shutdown_connections 2024-05-15T16:17:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.256+0000 7f0b41cbc700 1 --2- 172.21.15.49:0/94790187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b3c101880 0x7f0b3c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.256+0000 7f0b41cbc700 1 --2- 172.21.15.49:0/94790187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c100ed0 0x7f0b3c1012b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.256+0000 7f0b41cbc700 1 -- 172.21.15.49:0/94790187 >> 172.21.15.49:0/94790187 conn(0x7f0b3c0fc7a0 msgr2=0x7f0b3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:07.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.257+0000 7f0b41cbc700 1 -- 172.21.15.49:0/94790187 shutdown_connections 2024-05-15T16:17:07.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.257+0000 7f0b41cbc700 1 -- 172.21.15.49:0/94790187 wait complete. 2024-05-15T16:17:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.257+0000 7f0b41cbc700 1 Processor -- start 2024-05-15T16:17:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.257+0000 7f0b41cbc700 1 -- start start 2024-05-15T16:17:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b41cbc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b3c100ed0 0x7f0b3c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b41cbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c101880 0x7f0b3c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b41cbc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c19d500 con 0x7f0b3c101880 2024-05-15T16:17:07.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b41cbc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c196b30 con 0x7f0b3c100ed0 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b3affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c101880 0x7f0b3c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b3affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c101880 0x7f0b3c19cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54956/0 (socket says 172.21.15.49:54956) 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b3affd700 1 -- 172.21.15.49:0/1576468917 learned_addr learned my addr 172.21.15.49:0/1576468917 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b3b7fe700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b3c100ed0 0x7f0b3c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b3affd700 1 -- 172.21.15.49:0/1576468917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b3c100ed0 msgr2=0x7f0b3c19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b3affd700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b3c100ed0 0x7f0b3c19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b3affd700 1 -- 172.21.15.49:0/1576468917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c0095f0 con 0x7f0b3c101880 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.258+0000 7f0b3affd700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c101880 0x7f0b3c19cee0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000b480 tx=0x7f0b3000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.259+0000 7f0b38ff9700 1 -- 172.21.15.49:0/1576468917 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b300118c0 con 0x7f0b3c101880 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.259+0000 7f0b38ff9700 1 -- 172.21.15.49:0/1576468917 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b30010380 con 0x7f0b3c101880 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.259+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c196e10 con 0x7f0b3c101880 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.259+0000 7f0b38ff9700 1 -- 172.21.15.49:0/1576468917 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b3000f4f0 con 0x7f0b3c101880 2024-05-15T16:17:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.259+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c197360 con 0x7f0b3c101880 2024-05-15T16:17:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.260+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c109270 con 0x7f0b3c101880 2024-05-15T16:17:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.260+0000 7f0b38ff9700 1 -- 172.21.15.49:0/1576468917 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0b300104f0 con 0x7f0b3c101880 2024-05-15T16:17:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.261+0000 7f0b38ff9700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0b2406e210 0x7f0b240706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.261+0000 7f0b38ff9700 1 -- 172.21.15.49:0/1576468917 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0b3008dd20 con 0x7f0b3c101880 2024-05-15T16:17:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.261+0000 7f0b3b7fe700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0b2406e210 0x7f0b240706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.261+0000 7f0b3b7fe700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0b2406e210 0x7f0b240706d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00b5c0 tx=0x7f0b2c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:07.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.266+0000 7f0b38ff9700 1 -- 172.21.15.49:0/1576468917 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0b3005abb0 con 0x7f0b3c101880 2024-05-15T16:17:07.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.495+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b3c198010 con 0x7f0b2406e210 2024-05-15T16:17:07.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.496+0000 7f0b38ff9700 1 -- 172.21.15.49:0/1576468917 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0b3c198010 con 0x7f0b2406e210 2024-05-15T16:17:07.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0b2406e210 msgr2=0x7f0b240706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0b2406e210 0x7f0b240706d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00b5c0 tx=0x7f0b2c01a040 comp rx=0 tx=0).stop 2024-05-15T16:17:07.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c101880 msgr2=0x7f0b3c19cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c101880 0x7f0b3c19cee0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000b480 tx=0x7f0b3000b840 comp rx=0 tx=0).stop 2024-05-15T16:17:07.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 shutdown_connections 2024-05-15T16:17:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0b2406e210 0x7f0b240706d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b3c100ed0 0x7f0b3c19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 --2- 172.21.15.49:0/1576468917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c101880 0x7f0b3c19cee0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 >> 172.21.15.49:0/1576468917 conn(0x7f0b3c0fc7a0 msgr2=0x7f0b3c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 shutdown_connections 2024-05-15T16:17:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.499+0000 7f0b41cbc700 1 -- 172.21.15.49:0/1576468917 wait complete. 2024-05-15T16:17:07.510 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:17:07.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.609+0000 7f2d9359e700 1 -- 172.21.15.49:0/3219749524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94101c70 msgr2=0x7f2d94102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.609+0000 7f2d9359e700 1 --2- 172.21.15.49:0/3219749524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94101c70 0x7f2d94102050 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009910 tx=0x7f2d84009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:07.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.609+0000 7f2d9359e700 1 -- 172.21.15.49:0/3219749524 shutdown_connections 2024-05-15T16:17:07.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.609+0000 7f2d9359e700 1 --2- 172.21.15.49:0/3219749524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d94102590 0x7f2d9410aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.609+0000 7f2d9359e700 1 --2- 172.21.15.49:0/3219749524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94101c70 0x7f2d94102050 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.609+0000 7f2d9359e700 1 -- 172.21.15.49:0/3219749524 >> 172.21.15.49:0/3219749524 conn(0x7f2d940fb500 msgr2=0x7f2d940fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:07.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.609+0000 7f2d9359e700 1 -- 172.21.15.49:0/3219749524 shutdown_connections 2024-05-15T16:17:07.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.609+0000 7f2d9359e700 1 -- 172.21.15.49:0/3219749524 wait complete. 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d9359e700 1 Processor -- start 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d9359e700 1 -- start start 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d9359e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d94101c70 0x7f2d940ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d9359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94102590 0x7f2d940ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d9359e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d94100350 con 0x7f2d94102590 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d9359e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d94100490 con 0x7f2d94101c70 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d91d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94102590 0x7f2d940ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d91d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94102590 0x7f2d940ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54976/0 (socket says 172.21.15.49:54976) 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d91d9b700 1 -- 172.21.15.49:0/2295914027 learned_addr learned my addr 172.21.15.49:0/2295914027 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.610+0000 7f2d9259c700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d94101c70 0x7f2d940ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.611+0000 7f2d91d9b700 1 -- 172.21.15.49:0/2295914027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d94101c70 msgr2=0x7f2d940ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.611+0000 7f2d91d9b700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d94101c70 0x7f2d940ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.611+0000 7f2d91d9b700 1 -- 172.21.15.49:0/2295914027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840095f0 con 0x7f2d94102590 2024-05-15T16:17:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.611+0000 7f2d91d9b700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94102590 0x7f2d940ffe10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800e940 tx=0x7f2d8800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.611+0000 7f2d837fe700 1 -- 172.21.15.49:0/2295914027 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d880096e0 con 0x7f2d94102590 2024-05-15T16:17:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.611+0000 7f2d837fe700 1 -- 172.21.15.49:0/2295914027 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d8800c7c0 con 0x7f2d94102590 2024-05-15T16:17:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.611+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d941a2bb0 con 0x7f2d94102590 2024-05-15T16:17:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.611+0000 7f2d837fe700 1 -- 172.21.15.49:0/2295914027 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d88005580 con 0x7f2d94102590 2024-05-15T16:17:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.611+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d941a2f10 con 0x7f2d94102590 2024-05-15T16:17:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.612+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d940697f0 con 0x7f2d94102590 2024-05-15T16:17:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.613+0000 7f2d837fe700 1 -- 172.21.15.49:0/2295914027 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2d8800c930 con 0x7f2d94102590 2024-05-15T16:17:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.613+0000 7f2d837fe700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d7c06e200 0x7f2d7c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.613+0000 7f2d837fe700 1 -- 172.21.15.49:0/2295914027 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d88014070 con 0x7f2d94102590 2024-05-15T16:17:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.613+0000 7f2d9259c700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d7c06e200 0x7f2d7c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.614+0000 7f2d9259c700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d7c06e200 0x7f2d7c0706c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2d840059d0 tx=0x7f2d84005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:07.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.618+0000 7f2d837fe700 1 -- 172.21.15.49:0/2295914027 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2d88057650 con 0x7f2d94102590 2024-05-15T16:17:07.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.845+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d941053c0 con 0x7f2d7c06e200 2024-05-15T16:17:07.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.847+0000 7f2d837fe700 1 -- 172.21.15.49:0/2295914027 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2d8401e040 con 0x7f2d7c06e200 2024-05-15T16:17:07.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d7c06e200 msgr2=0x7f2d7c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d7c06e200 0x7f2d7c0706c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2d840059d0 tx=0x7f2d84005940 comp rx=0 tx=0).stop 2024-05-15T16:17:07.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94102590 msgr2=0x7f2d940ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94102590 0x7f2d940ffe10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800e940 tx=0x7f2d8800ec50 comp rx=0 tx=0).stop 2024-05-15T16:17:07.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 shutdown_connections 2024-05-15T16:17:07.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2d7c06e200 0x7f2d7c0706c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d94101c70 0x7f2d940ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 --2- 172.21.15.49:0/2295914027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d94102590 0x7f2d940ffe10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 >> 172.21.15.49:0/2295914027 conn(0x7f2d940fb500 msgr2=0x7f2d94102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:07.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 shutdown_connections 2024-05-15T16:17:07.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.849+0000 7f2d9359e700 1 -- 172.21.15.49:0/2295914027 wait complete. 2024-05-15T16:17:07.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.958+0000 7f3070ac6700 1 -- 172.21.15.49:0/1196712444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 msgr2=0x7f306c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.958+0000 7f3070ac6700 1 --2- 172.21.15.49:0/1196712444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 0x7f306c1047c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3054009910 tx=0x7f3054009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:07.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.958+0000 7f3070ac6700 1 -- 172.21.15.49:0/1196712444 shutdown_connections 2024-05-15T16:17:07.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.958+0000 7f3070ac6700 1 --2- 172.21.15.49:0/1196712444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 0x7f306c1047c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.958+0000 7f3070ac6700 1 --2- 172.21.15.49:0/1196712444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f306c0ffdc0 0x7f306c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.958+0000 7f3070ac6700 1 -- 172.21.15.49:0/1196712444 >> 172.21.15.49:0/1196712444 conn(0x7f306c076800 msgr2=0x7f306c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:07.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.958+0000 7f3070ac6700 1 -- 172.21.15.49:0/1196712444 shutdown_connections 2024-05-15T16:17:07.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.958+0000 7f3070ac6700 1 -- 172.21.15.49:0/1196712444 wait complete. 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.959+0000 7f3070ac6700 1 Processor -- start 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.959+0000 7f3070ac6700 1 -- start start 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.959+0000 7f3070ac6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f306c0ffdc0 0x7f306c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.959+0000 7f3070ac6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 0x7f306c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.959+0000 7f3070ac6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c198480 con 0x7f306c100770 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.959+0000 7f3070ac6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c1985c0 con 0x7f306c0ffdc0 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3061d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 0x7f306c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3061d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 0x7f306c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55000/0 (socket says 172.21.15.49:55000) 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3061d9b700 1 -- 172.21.15.49:0/574177949 learned_addr learned my addr 172.21.15.49:0/574177949 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f306a59c700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f306c0ffdc0 0x7f306c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:07.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3061d9b700 1 -- 172.21.15.49:0/574177949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f306c0ffdc0 msgr2=0x7f306c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3061d9b700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f306c0ffdc0 0x7f306c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3061d9b700 1 -- 172.21.15.49:0/574177949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30540095f0 con 0x7f306c100770 2024-05-15T16:17:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3061d9b700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 0x7f306c197f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3054005b40 tx=0x7f305400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3063fff700 1 -- 172.21.15.49:0/574177949 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f305401d070 con 0x7f306c100770 2024-05-15T16:17:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c198840 con 0x7f306c100770 2024-05-15T16:17:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3063fff700 1 -- 172.21.15.49:0/574177949 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f305400f460 con 0x7f306c100770 2024-05-15T16:17:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3063fff700 1 -- 172.21.15.49:0/574177949 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3054005720 con 0x7f306c100770 2024-05-15T16:17:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.960+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c1a0d50 con 0x7f306c100770 2024-05-15T16:17:07.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.961+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c1081c0 con 0x7f306c100770 2024-05-15T16:17:07.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.962+0000 7f3063fff700 1 -- 172.21.15.49:0/574177949 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f305400f5d0 con 0x7f306c100770 2024-05-15T16:17:07.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.962+0000 7f3063fff700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f305806e210 0x7f30580706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:07.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.962+0000 7f306a59c700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f305806e210 0x7f30580706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:07.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.962+0000 7f3063fff700 1 -- 172.21.15.49:0/574177949 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f305408f5f0 con 0x7f306c100770 2024-05-15T16:17:07.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.963+0000 7f306a59c700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f305806e210 0x7f30580706d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f305c00ac50 tx=0x7f305c00a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:07.967+0000 7f3063fff700 1 -- 172.21.15.49:0/574177949 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f30540588d0 con 0x7f306c100770 2024-05-15T16:17:08.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.185+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f306c10cd60 con 0x7f305806e210 2024-05-15T16:17:08.186 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:07 smithi049 ceph-mon[112142]: pgmap v602: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:08.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.196+0000 7f3063fff700 1 -- 172.21.15.49:0/574177949 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f306c10cd60 con 0x7f305806e210 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (20m) 13m ago 31m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (32m) 13m ago 32m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (31m) 13m ago 31m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (17m) 13m ago 32m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (17m) 13m ago 31m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (19m) 13m ago 31m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (14m) 13m ago 26m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (13m) 13m ago 26m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (13m) 13m ago 26m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (26m) 13m ago 26m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (22m) 13m ago 33m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:17:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (21m) 13m ago 30m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (18m) 13m ago 33m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (18m) 13m ago 30m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (21m) 13m ago 32m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (21m) 13m ago 30m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (17m) 13m ago 30m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (16m) 13m ago 29m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (16m) 13m ago 29m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (15m) 13m ago 28m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (14m) 13m ago 28m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (14m) 13m ago 27m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:17:08.198 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (20m) 13m ago 31m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:17:08.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:07 smithi144 ceph-mon[86635]: pgmap v602: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:08.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f305806e210 msgr2=0x7f30580706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f305806e210 0x7f30580706d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f305c00ac50 tx=0x7f305c00a380 comp rx=0 tx=0).stop 2024-05-15T16:17:08.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 msgr2=0x7f306c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 0x7f306c197f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3054005b40 tx=0x7f305400bd70 comp rx=0 tx=0).stop 2024-05-15T16:17:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 shutdown_connections 2024-05-15T16:17:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f305806e210 0x7f30580706d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f306c0ffdc0 0x7f306c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 --2- 172.21.15.49:0/574177949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306c100770 0x7f306c197f40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 >> 172.21.15.49:0/574177949 conn(0x7f306c076800 msgr2=0x7f306c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 shutdown_connections 2024-05-15T16:17:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.198+0000 7f3070ac6700 1 -- 172.21.15.49:0/574177949 wait complete. 2024-05-15T16:17:08.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 -- 172.21.15.49:0/418761665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c101eb0 msgr2=0x7f9b4c102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 --2- 172.21.15.49:0/418761665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c101eb0 0x7f9b4c102290 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009940 tx=0x7f9b3c009c50 comp rx=0 tx=0).stop 2024-05-15T16:17:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 -- 172.21.15.49:0/418761665 shutdown_connections 2024-05-15T16:17:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 --2- 172.21.15.49:0/418761665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4c1027d0 0x7f9b4c10acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 --2- 172.21.15.49:0/418761665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c101eb0 0x7f9b4c102290 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 -- 172.21.15.49:0/418761665 >> 172.21.15.49:0/418761665 conn(0x7f9b4c0fb6a0 msgr2=0x7f9b4c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 -- 172.21.15.49:0/418761665 shutdown_connections 2024-05-15T16:17:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 -- 172.21.15.49:0/418761665 wait complete. 2024-05-15T16:17:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 Processor -- start 2024-05-15T16:17:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.306+0000 7f9b51934700 1 -- start start 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b51934700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4c101eb0 0x7f9b4c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b51934700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c1027d0 0x7f9b4c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b51934700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c19d450 con 0x7f9b4c1027d0 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b51934700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c196a30 con 0x7f9b4c101eb0 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b427fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c1027d0 0x7f9b4c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b427fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c1027d0 0x7f9b4c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55020/0 (socket says 172.21.15.49:55020) 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b427fc700 1 -- 172.21.15.49:0/2576050208 learned_addr learned my addr 172.21.15.49:0/2576050208 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b4affd700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4c101eb0 0x7f9b4c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b427fc700 1 -- 172.21.15.49:0/2576050208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4c101eb0 msgr2=0x7f9b4c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b427fc700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4c101eb0 0x7f9b4c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b427fc700 1 -- 172.21.15.49:0/2576050208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0095f0 con 0x7f9b4c1027d0 2024-05-15T16:17:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.307+0000 7f9b427fc700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c1027d0 0x7f9b4c19ce30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400d900 tx=0x7f9b3400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.308+0000 7f9b50932700 1 -- 172.21.15.49:0/2576050208 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b34009900 con 0x7f9b4c1027d0 2024-05-15T16:17:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.308+0000 7f9b50932700 1 -- 172.21.15.49:0/2576050208 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b34010460 con 0x7f9b4c1027d0 2024-05-15T16:17:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.308+0000 7f9b50932700 1 -- 172.21.15.49:0/2576050208 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3400f5f0 con 0x7f9b4c1027d0 2024-05-15T16:17:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.308+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c196d10 con 0x7f9b4c1027d0 2024-05-15T16:17:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.308+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4c197260 con 0x7f9b4c1027d0 2024-05-15T16:17:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.309+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4c1055d0 con 0x7f9b4c1027d0 2024-05-15T16:17:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.309+0000 7f9b50932700 1 -- 172.21.15.49:0/2576050208 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9b340105d0 con 0x7f9b4c1027d0 2024-05-15T16:17:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.310+0000 7f9b50932700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9b3806e210 0x7f9b380706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.310+0000 7f9b50932700 1 -- 172.21.15.49:0/2576050208 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9b3408dc40 con 0x7f9b4c1027d0 2024-05-15T16:17:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.310+0000 7f9b4affd700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9b3806e210 0x7f9b380706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.310+0000 7f9b4affd700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9b3806e210 0x7f9b380706d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c006010 tx=0x7f9b3c005ad0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:08.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.314+0000 7f9b50932700 1 -- 172.21.15.49:0/2576050208 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9b3405aad0 con 0x7f9b4c1027d0 2024-05-15T16:17:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.607+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9b4c197ef0 con 0x7f9b4c1027d0 2024-05-15T16:17:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.607+0000 7f9b50932700 1 -- 172.21.15.49:0/2576050208 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f9b34017360 con 0x7f9b4c1027d0 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:17:08.608 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:17:08.609 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:17:08.609 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:17:08.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:17:08.609 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:17:08.609 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:17:08.609 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:17:08.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:17:08.609 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:17:08.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9b3806e210 msgr2=0x7f9b380706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9b3806e210 0x7f9b380706d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c006010 tx=0x7f9b3c005ad0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c1027d0 msgr2=0x7f9b4c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c1027d0 0x7f9b4c19ce30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400d900 tx=0x7f9b3400dc10 comp rx=0 tx=0).stop 2024-05-15T16:17:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 shutdown_connections 2024-05-15T16:17:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9b3806e210 0x7f9b380706d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4c101eb0 0x7f9b4c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 --2- 172.21.15.49:0/2576050208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b4c1027d0 0x7f9b4c19ce30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 >> 172.21.15.49:0/2576050208 conn(0x7f9b4c0fb6a0 msgr2=0x7f9b4c1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 shutdown_connections 2024-05-15T16:17:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.610+0000 7f9b51934700 1 -- 172.21.15.49:0/2576050208 wait complete. 2024-05-15T16:17:08.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 -- 172.21.15.49:0/308900398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c101e70 msgr2=0x7f813c102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 --2- 172.21.15.49:0/308900398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c101e70 0x7f813c102250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8130009910 tx=0x7f8130009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:08.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 -- 172.21.15.49:0/308900398 shutdown_connections 2024-05-15T16:17:08.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 --2- 172.21.15.49:0/308900398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f813c102790 0x7f813c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 --2- 172.21.15.49:0/308900398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c101e70 0x7f813c102250 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 -- 172.21.15.49:0/308900398 >> 172.21.15.49:0/308900398 conn(0x7f813c0fb6a0 msgr2=0x7f813c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:08.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 -- 172.21.15.49:0/308900398 shutdown_connections 2024-05-15T16:17:08.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 -- 172.21.15.49:0/308900398 wait complete. 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 Processor -- start 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.719+0000 7f81431b1700 1 -- start start 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f81431b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f813c101e70 0x7f813c19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f81431b1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c102790 0x7f813c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f81431b1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c19d530 con 0x7f813c102790 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f81431b1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c196b10 con 0x7f813c101e70 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f813bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c102790 0x7f813c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f813bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c102790 0x7f813c19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55032/0 (socket says 172.21.15.49:55032) 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f813bfff700 1 -- 172.21.15.49:0/2530786911 learned_addr learned my addr 172.21.15.49:0/2530786911 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f8140f4d700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f813c101e70 0x7f813c19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f813bfff700 1 -- 172.21.15.49:0/2530786911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f813c101e70 msgr2=0x7f813c19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f813bfff700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f813c101e70 0x7f813c19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f813bfff700 1 -- 172.21.15.49:0/2530786911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300095f0 con 0x7f813c102790 2024-05-15T16:17:08.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f8140f4d700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f813c101e70 0x7f813c19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:17:08.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f813bfff700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c102790 0x7f813c19cf10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f812800b480 tx=0x7f812800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:08.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f8139ffb700 1 -- 172.21.15.49:0/2530786911 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81280118c0 con 0x7f813c102790 2024-05-15T16:17:08.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.720+0000 7f8139ffb700 1 -- 172.21.15.49:0/2530786911 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8128010380 con 0x7f813c102790 2024-05-15T16:17:08.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.721+0000 7f8139ffb700 1 -- 172.21.15.49:0/2530786911 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f812800f4f0 con 0x7f813c102790 2024-05-15T16:17:08.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.721+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813c196df0 con 0x7f813c102790 2024-05-15T16:17:08.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.721+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c197340 con 0x7f813c102790 2024-05-15T16:17:08.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.722+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813c1055d0 con 0x7f813c102790 2024-05-15T16:17:08.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.722+0000 7f8139ffb700 1 -- 172.21.15.49:0/2530786911 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f81280104f0 con 0x7f813c102790 2024-05-15T16:17:08.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.723+0000 7f8139ffb700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f812c06e200 0x7f812c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:08.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.723+0000 7f8139ffb700 1 -- 172.21.15.49:0/2530786911 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f812808dd10 con 0x7f813c102790 2024-05-15T16:17:08.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.723+0000 7f8140f4d700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f812c06e200 0x7f812c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:08.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.723+0000 7f8140f4d700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f812c06e200 0x7f812c0706c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8130005f50 tx=0x7f8130005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:08.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.727+0000 7f8139ffb700 1 -- 172.21.15.49:0/2530786911 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f812805aba0 con 0x7f813c102790 2024-05-15T16:17:08.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:08 smithi049 ceph-mon[112142]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:08.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:08 smithi049 ceph-mon[112142]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:08.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:08 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2576050208' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:17:08.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.980+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f813c069a90 con 0x7f813c102790 2024-05-15T16:17:08.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.981+0000 7f8139ffb700 1 -- 172.21.15.49:0/2530786911 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f8128015760 con 0x7f813c102790 2024-05-15T16:17:08.981 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:17:08.981 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:17:08.981 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:17:08.981 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:17:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:17:08.982 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:17:08.983 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:17:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:17:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:17:08.984 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:17:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:17:08.984 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:17:08.984 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:17:08.984 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:17:08.984 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f812c06e200 msgr2=0x7f812c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f812c06e200 0x7f812c0706c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8130005f50 tx=0x7f8130005dc0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c102790 msgr2=0x7f813c19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c102790 0x7f813c19cf10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f812800b480 tx=0x7f812800b840 comp rx=0 tx=0).stop 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 shutdown_connections 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f812c06e200 0x7f812c0706c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f813c101e70 0x7f813c19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 --2- 172.21.15.49:0/2530786911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f813c102790 0x7f813c19cf10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 >> 172.21.15.49:0/2530786911 conn(0x7f813c0fb6a0 msgr2=0x7f813c103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:08.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 shutdown_connections 2024-05-15T16:17:08.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:08.983+0000 7f81431b1700 1 -- 172.21.15.49:0/2530786911 wait complete. 2024-05-15T16:17:08.986 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:17:09.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.094+0000 7fdc73e3c700 1 -- 172.21.15.49:0/456514736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 msgr2=0x7fdc6c0ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:09.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.094+0000 7fdc73e3c700 1 --2- 172.21.15.49:0/456514736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 0x7fdc6c0ffdc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009910 tx=0x7fdc68009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:09.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.094+0000 7fdc73e3c700 1 -- 172.21.15.49:0/456514736 shutdown_connections 2024-05-15T16:17:09.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.094+0000 7fdc73e3c700 1 --2- 172.21.15.49:0/456514736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc6c100390 0x7fdc6c10d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.094+0000 7fdc73e3c700 1 --2- 172.21.15.49:0/456514736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 0x7fdc6c0ffdc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.094+0000 7fdc73e3c700 1 -- 172.21.15.49:0/456514736 >> 172.21.15.49:0/456514736 conn(0x7fdc6c0fb6a0 msgr2=0x7fdc6c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:09.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.094+0000 7fdc73e3c700 1 -- 172.21.15.49:0/456514736 shutdown_connections 2024-05-15T16:17:09.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.094+0000 7fdc73e3c700 1 -- 172.21.15.49:0/456514736 wait complete. 2024-05-15T16:17:09.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.094+0000 7fdc73e3c700 1 Processor -- start 2024-05-15T16:17:09.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc73e3c700 1 -- start start 2024-05-15T16:17:09.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc73e3c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 0x7fdc6c198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:09.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc73e3c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc6c100390 0x7fdc6c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc73e3c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c199970 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc73e3c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c199ab0 con 0x7fdc6c100390 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc71bd8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 0x7fdc6c198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc71bd8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 0x7fdc6c198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55052/0 (socket says 172.21.15.49:55052) 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc71bd8700 1 -- 172.21.15.49:0/2163149754 learned_addr learned my addr 172.21.15.49:0/2163149754 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc71bd8700 1 -- 172.21.15.49:0/2163149754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc6c100390 msgr2=0x7fdc6c199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc71bd8700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc6c100390 0x7fdc6c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc71bd8700 1 -- 172.21.15.49:0/2163149754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680095f0 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc71bd8700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 0x7fdc6c198de0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800b850 tx=0x7fdc680056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.095+0000 7fdc62ffd700 1 -- 172.21.15.49:0/2163149754 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc6801d070 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.096+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c19d790 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.096+0000 7fdc62ffd700 1 -- 172.21.15.49:0/2163149754 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc68003d50 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.096+0000 7fdc62ffd700 1 -- 172.21.15.49:0/2163149754 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc68021410 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.096+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c19dce0 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.097+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c10aed0 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.097+0000 7fdc62ffd700 1 -- 172.21.15.49:0/2163149754 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdc6802b440 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.098+0000 7fdc62ffd700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc5806e200 0x7fdc580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:09.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.098+0000 7fdc62ffd700 1 -- 172.21.15.49:0/2163149754 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdc6808f490 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.098+0000 7fdc713d7700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc5806e200 0x7fdc580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:09.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.098+0000 7fdc713d7700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc5806e200 0x7fdc580706c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005fd0 tx=0x7fdc5c005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:09.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.103+0000 7fdc62ffd700 1 -- 172.21.15.49:0/2163149754 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdc68058770 con 0x7fdc6c0ff9e0 2024-05-15T16:17:09.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:08 smithi144 ceph-mon[86635]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:09.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:08 smithi144 ceph-mon[86635]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:09.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:08 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2576050208' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:17:09.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.332+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc6c19a0a0 con 0x7fdc5806e200 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.333+0000 7fdc62ffd700 1 -- 172.21.15.49:0/2163149754 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdc6c19a0a0 con 0x7fdc5806e200 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:17:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:17:09.335 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:17:09.335 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:17:09.335 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc5806e200 msgr2=0x7fdc580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc5806e200 0x7fdc580706c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005fd0 tx=0x7fdc5c005d00 comp rx=0 tx=0).stop 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 msgr2=0x7fdc6c198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 0x7fdc6c198de0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800b850 tx=0x7fdc680056a0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 shutdown_connections 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc5806e200 0x7fdc580706c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc6c0ff9e0 0x7fdc6c198de0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 --2- 172.21.15.49:0/2163149754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc6c100390 0x7fdc6c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 >> 172.21.15.49:0/2163149754 conn(0x7fdc6c0fb6a0 msgr2=0x7fdc6c108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.336+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 shutdown_connections 2024-05-15T16:17:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.337+0000 7fdc73e3c700 1 -- 172.21.15.49:0/2163149754 wait complete. 2024-05-15T16:17:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.444+0000 7f02cefa4700 1 -- 172.21.15.49:0/1011544815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 msgr2=0x7f02c810ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.444+0000 7f02cefa4700 1 --2- 172.21.15.49:0/1011544815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 0x7f02c810ad00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f02b800b0d0 tx=0x7f02b800b3e0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.444+0000 7f02cefa4700 1 -- 172.21.15.49:0/1011544815 shutdown_connections 2024-05-15T16:17:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.444+0000 7f02cefa4700 1 --2- 172.21.15.49:0/1011544815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 0x7f02c810ad00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.444+0000 7f02cefa4700 1 --2- 172.21.15.49:0/1011544815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c8101ef0 0x7f02c81022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.444+0000 7f02cefa4700 1 -- 172.21.15.49:0/1011544815 >> 172.21.15.49:0/1011544815 conn(0x7f02c80fb700 msgr2=0x7f02c80fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.444+0000 7f02cefa4700 1 -- 172.21.15.49:0/1011544815 shutdown_connections 2024-05-15T16:17:09.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.444+0000 7f02cefa4700 1 -- 172.21.15.49:0/1011544815 wait complete. 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.445+0000 7f02cefa4700 1 Processor -- start 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.445+0000 7f02cefa4700 1 -- start start 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.445+0000 7f02cefa4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c8101ef0 0x7f02c819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.445+0000 7f02cefa4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 0x7f02c819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.445+0000 7f02cefa4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c819d490 con 0x7f02c8102810 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.445+0000 7f02cefa4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c8196a70 con 0x7f02c8101ef0 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.445+0000 7f02c7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 0x7f02c819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02c7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 0x7f02c819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55074/0 (socket says 172.21.15.49:55074) 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02c7fff700 1 -- 172.21.15.49:0/316565869 learned_addr learned my addr 172.21.15.49:0/316565869 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02ccd40700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c8101ef0 0x7f02c819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:09.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02c7fff700 1 -- 172.21.15.49:0/316565869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c8101ef0 msgr2=0x7f02c819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:09.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02c7fff700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c8101ef0 0x7f02c819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02c7fff700 1 -- 172.21.15.49:0/316565869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b8009d00 con 0x7f02c8102810 2024-05-15T16:17:09.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02ccd40700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c8101ef0 0x7f02c819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:17:09.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02c7fff700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 0x7f02c819ce70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f02b800b820 tx=0x7f02b8003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:09.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02c5ffb700 1 -- 172.21.15.49:0/316565869 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02b800e040 con 0x7f02c8102810 2024-05-15T16:17:09.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02c5ffb700 1 -- 172.21.15.49:0/316565869 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02b8004040 con 0x7f02c8102810 2024-05-15T16:17:09.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02c5ffb700 1 -- 172.21.15.49:0/316565869 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02b801d6b0 con 0x7f02c8102810 2024-05-15T16:17:09.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c8196cf0 con 0x7f02c8102810 2024-05-15T16:17:09.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.446+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c81971e0 con 0x7f02c8102810 2024-05-15T16:17:09.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.447+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c8105680 con 0x7f02c8102810 2024-05-15T16:17:09.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.448+0000 7f02c5ffb700 1 -- 172.21.15.49:0/316565869 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f02b80041b0 con 0x7f02c8102810 2024-05-15T16:17:09.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.448+0000 7f02c5ffb700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f02b406e200 0x7f02b40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:09.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.448+0000 7f02c5ffb700 1 -- 172.21.15.49:0/316565869 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f02b808f8f0 con 0x7f02c8102810 2024-05-15T16:17:09.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.449+0000 7f02ccd40700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f02b406e200 0x7f02b40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:09.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.449+0000 7f02ccd40700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f02b406e200 0x7f02b40706c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f02b0005fd0 tx=0x7f02b0005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:09.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.453+0000 7f02c5ffb700 1 -- 172.21.15.49:0/316565869 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f02b805c6a0 con 0x7f02c8102810 2024-05-15T16:17:09.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.748+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f02c8069a30 con 0x7f02c8102810 2024-05-15T16:17:09.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.749+0000 7f02c5ffb700 1 -- 172.21.15.49:0/316565869 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f02b805c230 con 0x7f02c8102810 2024-05-15T16:17:09.749 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:17:09.749 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:17:09.749 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:17:09.750 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:17:09.750 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:17:09.750 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:17:09.750 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:17:09.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f02b406e200 msgr2=0x7f02b40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:09.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f02b406e200 0x7f02b40706c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f02b0005fd0 tx=0x7f02b0005ee0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 msgr2=0x7f02c819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 0x7f02c819ce70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f02b800b820 tx=0x7f02b8003cc0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 shutdown_connections 2024-05-15T16:17:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f02b406e200 0x7f02b40706c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c8101ef0 0x7f02c819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 --2- 172.21.15.49:0/316565869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02c8102810 0x7f02c819ce70 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 >> 172.21.15.49:0/316565869 conn(0x7f02c80fb700 msgr2=0x7f02c81031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 shutdown_connections 2024-05-15T16:17:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:09.751+0000 7f02cefa4700 1 -- 172.21.15.49:0/316565869 wait complete. 2024-05-15T16:17:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:09 smithi144 ceph-mon[86635]: from='client.35138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:09 smithi144 ceph-mon[86635]: pgmap v603: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:09 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2530786911' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:17:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:09 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/316565869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:17:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:09 smithi049 ceph-mon[112142]: from='client.35138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:09 smithi049 ceph-mon[112142]: pgmap v603: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:09 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2530786911' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:17:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:09 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/316565869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:17:11.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:10 smithi049 ceph-mon[112142]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:11.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:10 smithi144 ceph-mon[86635]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:11 smithi144 ceph-mon[86635]: pgmap v604: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:11 smithi049 ceph-mon[112142]: pgmap v604: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:13 smithi144 ceph-mon[86635]: pgmap v605: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:13 smithi049 ceph-mon[112142]: pgmap v605: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:15 smithi144 ceph-mon[86635]: pgmap v606: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:15 smithi049 ceph-mon[112142]: pgmap v606: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:17.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:17 smithi144 ceph-mon[86635]: pgmap v607: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:17 smithi049 ceph-mon[112142]: pgmap v607: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:19 smithi144 ceph-mon[86635]: pgmap v608: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:19 smithi049 ceph-mon[112142]: pgmap v608: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:21.161 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:17:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:17:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:21 smithi144 ceph-mon[86635]: pgmap v609: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:21 smithi049 ceph-mon[112142]: pgmap v609: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:23 smithi144 ceph-mon[86635]: pgmap v610: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:23 smithi049 ceph-mon[112142]: pgmap v610: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:25 smithi144 ceph-mon[86635]: pgmap v611: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:25 smithi049 ceph-mon[112142]: pgmap v611: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:27 smithi144 ceph-mon[86635]: pgmap v612: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:27 smithi049 ceph-mon[112142]: pgmap v612: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:29 smithi144 ceph-mon[86635]: pgmap v613: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:29 smithi049 ceph-mon[112142]: pgmap v613: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:31 smithi144 ceph-mon[86635]: pgmap v614: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:31 smithi049 ceph-mon[112142]: pgmap v614: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:33 smithi144 ceph-mon[86635]: pgmap v615: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:33 smithi049 ceph-mon[112142]: pgmap v615: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:35 smithi144 ceph-mon[86635]: pgmap v616: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:17:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:35 smithi049 ceph-mon[112142]: pgmap v616: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:17:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:37 smithi144 ceph-mon[86635]: pgmap v617: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:37 smithi049 ceph-mon[112142]: pgmap v617: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:39.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.875+0000 7fde5bcf0700 1 -- 172.21.15.49:0/1292654248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 msgr2=0x7fde541012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:39.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.875+0000 7fde5bcf0700 1 --2- 172.21.15.49:0/1292654248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 0x7fde541012c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fde50009910 tx=0x7fde50009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:39.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.875+0000 7fde5bcf0700 1 -- 172.21.15.49:0/1292654248 shutdown_connections 2024-05-15T16:17:39.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.875+0000 7fde5bcf0700 1 --2- 172.21.15.49:0/1292654248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde54101890 0x7fde541058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:39.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.875+0000 7fde5bcf0700 1 --2- 172.21.15.49:0/1292654248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 0x7fde541012c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:39.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.875+0000 7fde5bcf0700 1 -- 172.21.15.49:0/1292654248 >> 172.21.15.49:0/1292654248 conn(0x7fde540fc790 msgr2=0x7fde540febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:39.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.875+0000 7fde5bcf0700 1 -- 172.21.15.49:0/1292654248 shutdown_connections 2024-05-15T16:17:39.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.875+0000 7fde5bcf0700 1 -- 172.21.15.49:0/1292654248 wait complete. 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.876+0000 7fde5bcf0700 1 Processor -- start 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.876+0000 7fde5bcf0700 1 -- start start 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.876+0000 7fde5bcf0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 0x7fde5419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.876+0000 7fde5bcf0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde54101890 0x7fde5419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.876+0000 7fde5bcf0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5419d460 con 0x7fde54100ee0 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.876+0000 7fde5bcf0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde54196a90 con 0x7fde54101890 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde59a8c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 0x7fde5419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde59a8c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 0x7fde5419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36832/0 (socket says 172.21.15.49:36832) 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde59a8c700 1 -- 172.21.15.49:0/4275135650 learned_addr learned my addr 172.21.15.49:0/4275135650 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde5928b700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde54101890 0x7fde5419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:39.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde59a8c700 1 -- 172.21.15.49:0/4275135650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde54101890 msgr2=0x7fde5419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:39.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde59a8c700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde54101890 0x7fde5419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:39.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde59a8c700 1 -- 172.21.15.49:0/4275135650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500095f0 con 0x7fde54100ee0 2024-05-15T16:17:39.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde59a8c700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 0x7fde5419c900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fde5000bd70 tx=0x7fde5000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:39.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde4affd700 1 -- 172.21.15.49:0/4275135650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde5001d070 con 0x7fde54100ee0 2024-05-15T16:17:39.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde54196d10 con 0x7fde54100ee0 2024-05-15T16:17:39.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde4affd700 1 -- 172.21.15.49:0/4275135650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde500056e0 con 0x7fde54100ee0 2024-05-15T16:17:39.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde4affd700 1 -- 172.21.15.49:0/4275135650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde5000fe40 con 0x7fde54100ee0 2024-05-15T16:17:39.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.877+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde54197200 con 0x7fde54100ee0 2024-05-15T16:17:39.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.878+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde54109280 con 0x7fde54100ee0 2024-05-15T16:17:39.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.879+0000 7fde4affd700 1 -- 172.21.15.49:0/4275135650 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fde5000f680 con 0x7fde54100ee0 2024-05-15T16:17:39.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.879+0000 7fde4affd700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde4006e1a0 0x7fde40070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:39.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.880+0000 7fde4affd700 1 -- 172.21.15.49:0/4275135650 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fde5008f580 con 0x7fde54100ee0 2024-05-15T16:17:39.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.880+0000 7fde5928b700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde4006e1a0 0x7fde40070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:39.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.880+0000 7fde5928b700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde4006e1a0 0x7fde40070660 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde54197c00 tx=0x7fde4400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:39.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:39.884+0000 7fde4affd700 1 -- 172.21.15.49:0/4275135650 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fde5005c390 con 0x7fde54100ee0 2024-05-15T16:17:40.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.114+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde54197f00 con 0x7fde4006e1a0 2024-05-15T16:17:40.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.116+0000 7fde4affd700 1 -- 172.21.15.49:0/4275135650 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fde54197f00 con 0x7fde4006e1a0 2024-05-15T16:17:40.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde4006e1a0 msgr2=0x7fde40070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde4006e1a0 0x7fde40070660 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde54197c00 tx=0x7fde4400b410 comp rx=0 tx=0).stop 2024-05-15T16:17:40.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 msgr2=0x7fde5419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 0x7fde5419c900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fde5000bd70 tx=0x7fde5000bda0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 shutdown_connections 2024-05-15T16:17:40.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde4006e1a0 0x7fde40070660 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde54100ee0 0x7fde5419c900 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 --2- 172.21.15.49:0/4275135650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde54101890 0x7fde5419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 >> 172.21.15.49:0/4275135650 conn(0x7fde540fc790 msgr2=0x7fde540fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:40.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 shutdown_connections 2024-05-15T16:17:40.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.118+0000 7fde5bcf0700 1 -- 172.21.15.49:0/4275135650 wait complete. 2024-05-15T16:17:40.129 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:17:40.129 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:39 smithi049 ceph-mon[112142]: pgmap v618: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:39 smithi144 ceph-mon[86635]: pgmap v618: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.228+0000 7f0922f20700 1 -- 172.21.15.49:0/601407037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c1007c0 msgr2=0x7f091c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.228+0000 7f0922f20700 1 --2- 172.21.15.49:0/601407037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c1007c0 0x7f091c104810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0908009910 tx=0x7f0908009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.228+0000 7f0922f20700 1 -- 172.21.15.49:0/601407037 shutdown_connections 2024-05-15T16:17:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.228+0000 7f0922f20700 1 --2- 172.21.15.49:0/601407037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c1007c0 0x7f091c104810 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.228+0000 7f0922f20700 1 --2- 172.21.15.49:0/601407037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f091c0ffe10 0x7f091c1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.228+0000 7f0922f20700 1 -- 172.21.15.49:0/601407037 >> 172.21.15.49:0/601407037 conn(0x7f091c0fb700 msgr2=0x7f091c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.228+0000 7f0922f20700 1 -- 172.21.15.49:0/601407037 shutdown_connections 2024-05-15T16:17:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.228+0000 7f0922f20700 1 -- 172.21.15.49:0/601407037 wait complete. 2024-05-15T16:17:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.229+0000 7f0922f20700 1 Processor -- start 2024-05-15T16:17:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.229+0000 7f0922f20700 1 -- start start 2024-05-15T16:17:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0922f20700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c0ffe10 0x7f091c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0922f20700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f091c1007c0 0x7f091c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0922f20700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c1985a0 con 0x7f091c0ffe10 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0922f20700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c1986e0 con 0x7f091c1007c0 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0920cbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c0ffe10 0x7f091c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0920cbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c0ffe10 0x7f091c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36850/0 (socket says 172.21.15.49:36850) 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0920cbc700 1 -- 172.21.15.49:0/3020848178 learned_addr learned my addr 172.21.15.49:0/3020848178 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f091bfff700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f091c1007c0 0x7f091c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0920cbc700 1 -- 172.21.15.49:0/3020848178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f091c1007c0 msgr2=0x7f091c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0920cbc700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f091c1007c0 0x7f091c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0920cbc700 1 -- 172.21.15.49:0/3020848178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09080095f0 con 0x7f091c0ffe10 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0920cbc700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c0ffe10 0x7f091c197a90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f091000d8d0 tx=0x7f091000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0919ffb700 1 -- 172.21.15.49:0/3020848178 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0910009940 con 0x7f091c0ffe10 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c1a0b00 con 0x7f091c0ffe10 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0919ffb700 1 -- 172.21.15.49:0/3020848178 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0910010460 con 0x7f091c0ffe10 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.230+0000 7f0919ffb700 1 -- 172.21.15.49:0/3020848178 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f091000f5f0 con 0x7f091c0ffe10 2024-05-15T16:17:40.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.231+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c1a0f10 con 0x7f091c0ffe10 2024-05-15T16:17:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.232+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c108210 con 0x7f091c0ffe10 2024-05-15T16:17:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.232+0000 7f0919ffb700 1 -- 172.21.15.49:0/3020848178 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f09100105d0 con 0x7f091c0ffe10 2024-05-15T16:17:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.233+0000 7f0919ffb700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f090c06e210 0x7f090c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.233+0000 7f0919ffb700 1 -- 172.21.15.49:0/3020848178 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f091008e1a0 con 0x7f091c0ffe10 2024-05-15T16:17:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.233+0000 7f091bfff700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f090c06e210 0x7f090c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.233+0000 7f091bfff700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f090c06e210 0x7f090c0706d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f091c0699c0 tx=0x7f090801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:40.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.237+0000 7f0919ffb700 1 -- 172.21.15.49:0/3020848178 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0910057480 con 0x7f091c0ffe10 2024-05-15T16:17:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.466+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f091c10cdb0 con 0x7f090c06e210 2024-05-15T16:17:40.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.467+0000 7f0919ffb700 1 -- 172.21.15.49:0/3020848178 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f091c10cdb0 con 0x7f090c06e210 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f090c06e210 msgr2=0x7f090c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f090c06e210 0x7f090c0706d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f091c0699c0 tx=0x7f090801a040 comp rx=0 tx=0).stop 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c0ffe10 msgr2=0x7f091c197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c0ffe10 0x7f091c197a90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f091000d8d0 tx=0x7f091000dc90 comp rx=0 tx=0).stop 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 shutdown_connections 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f090c06e210 0x7f090c0706d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091c0ffe10 0x7f091c197a90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 --2- 172.21.15.49:0/3020848178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f091c1007c0 0x7f091c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 >> 172.21.15.49:0/3020848178 conn(0x7f091c0fb700 msgr2=0x7f091c0fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 shutdown_connections 2024-05-15T16:17:40.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.469+0000 7f0922f20700 1 -- 172.21.15.49:0/3020848178 wait complete. 2024-05-15T16:17:40.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.577+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/1463199808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 msgr2=0x7fbe181001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.578+0000 7fbe1d1c4700 1 --2- 172.21.15.49:0/1463199808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 0x7fbe181001d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009910 tx=0x7fbe08009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:40.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.578+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/1463199808 shutdown_connections 2024-05-15T16:17:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.578+0000 7fbe1d1c4700 1 --2- 172.21.15.49:0/1463199808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe181007a0 0x7fbe181047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.578+0000 7fbe1d1c4700 1 --2- 172.21.15.49:0/1463199808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 0x7fbe181001d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.578+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/1463199808 >> 172.21.15.49:0/1463199808 conn(0x7fbe180fb6c0 msgr2=0x7fbe180fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.578+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/1463199808 shutdown_connections 2024-05-15T16:17:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.578+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/1463199808 wait complete. 2024-05-15T16:17:40.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.578+0000 7fbe1d1c4700 1 Processor -- start 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe1d1c4700 1 -- start start 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe1d1c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 0x7fbe18101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe1d1c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe181007a0 0x7fbe18101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe1d1c4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe18102310 con 0x7fbe180ffdf0 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe1d1c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe18102480 con 0x7fbe181007a0 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe16d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 0x7fbe18101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe16d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 0x7fbe18101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36876/0 (socket says 172.21.15.49:36876) 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe16d9d700 1 -- 172.21.15.49:0/911911640 learned_addr learned my addr 172.21.15.49:0/911911640 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe1659c700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe181007a0 0x7fbe18101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe16d9d700 1 -- 172.21.15.49:0/911911640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe181007a0 msgr2=0x7fbe18101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe16d9d700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe181007a0 0x7fbe18101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.579+0000 7fbe16d9d700 1 -- 172.21.15.49:0/911911640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe080095f0 con 0x7fbe180ffdf0 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.580+0000 7fbe16d9d700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 0x7fbe18101890 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbe08006010 tx=0x7fbe0800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.580+0000 7fbdfffff700 1 -- 172.21.15.49:0/911911640 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0801d070 con 0x7fbe180ffdf0 2024-05-15T16:17:40.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.580+0000 7fbdfffff700 1 -- 172.21.15.49:0/911911640 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe0800f460 con 0x7fbe180ffdf0 2024-05-15T16:17:40.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.580+0000 7fbdfffff700 1 -- 172.21.15.49:0/911911640 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe080175f0 con 0x7fbe180ffdf0 2024-05-15T16:17:40.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.580+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe18102700 con 0x7fbe180ffdf0 2024-05-15T16:17:40.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.580+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe181a2fc0 con 0x7fbe180ffdf0 2024-05-15T16:17:40.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.581+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe181012d0 con 0x7fbe180ffdf0 2024-05-15T16:17:40.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.581+0000 7fbdfffff700 1 -- 172.21.15.49:0/911911640 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbe0800f5d0 con 0x7fbe180ffdf0 2024-05-15T16:17:40.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.582+0000 7fbdfffff700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe0006e200 0x7fbe000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:40.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.582+0000 7fbdfffff700 1 -- 172.21.15.49:0/911911640 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbe0808f460 con 0x7fbe180ffdf0 2024-05-15T16:17:40.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.582+0000 7fbe1659c700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe0006e200 0x7fbe000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:40.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.583+0000 7fbe1659c700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe0006e200 0x7fbe000706c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbe18102fc0 tx=0x7fbe0c00a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:40.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.587+0000 7fbdfffff700 1 -- 172.21.15.49:0/911911640 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbe08058740 con 0x7fbe180ffdf0 2024-05-15T16:17:40.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.801+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe18102700 con 0x7fbe0006e200 2024-05-15T16:17:40.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.813+0000 7fbdfffff700 1 -- 172.21.15.49:0/911911640 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbe18102700 con 0x7fbe0006e200 2024-05-15T16:17:40.813 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:17:40.813 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (20m) 14m ago 32m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:17:40.813 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (32m) 14m ago 32m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:17:40.813 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (31m) 13m ago 31m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:17:40.813 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (18m) 14m ago 32m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:17:40.813 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (18m) 13m ago 31m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (20m) 14m ago 32m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (14m) 14m ago 26m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (14m) 14m ago 26m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (13m) 13m ago 26m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (26m) 13m ago 26m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (23m) 14m ago 33m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (22m) 13m ago 31m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (19m) 14m ago 34m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (19m) 13m ago 31m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (21m) 14m ago 32m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (21m) 13m ago 31m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (17m) 14m ago 30m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (17m) 14m ago 30m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:17:40.814 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (16m) 14m ago 29m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:17:40.815 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (16m) 13m ago 29m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:17:40.815 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (15m) 13m ago 28m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:17:40.815 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (14m) 13m ago 28m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:17:40.815 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (21m) 14m ago 31m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:17:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe0006e200 msgr2=0x7fbe000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe0006e200 0x7fbe000706c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbe18102fc0 tx=0x7fbe0c00a380 comp rx=0 tx=0).stop 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 msgr2=0x7fbe18101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 0x7fbe18101890 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbe08006010 tx=0x7fbe0800bd70 comp rx=0 tx=0).stop 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 shutdown_connections 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbe0006e200 0x7fbe000706c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe180ffdf0 0x7fbe18101890 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 --2- 172.21.15.49:0/911911640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe181007a0 0x7fbe18101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 >> 172.21.15.49:0/911911640 conn(0x7fbe180fb6c0 msgr2=0x7fbe18069630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 shutdown_connections 2024-05-15T16:17:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.815+0000 7fbe1d1c4700 1 -- 172.21.15.49:0/911911640 wait complete. 2024-05-15T16:17:40.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.925+0000 7f6924511700 1 -- 172.21.15.49:0/2189378367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073c60 msgr2=0x7f691c111770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.925+0000 7f6924511700 1 --2- 172.21.15.49:0/2189378367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073c60 0x7f691c111770 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f690c009910 tx=0x7f690c009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:40.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.925+0000 7f6924511700 1 -- 172.21.15.49:0/2189378367 shutdown_connections 2024-05-15T16:17:40.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.925+0000 7f6924511700 1 --2- 172.21.15.49:0/2189378367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073c60 0x7f691c111770 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.925+0000 7f6924511700 1 --2- 172.21.15.49:0/2189378367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c073340 0x7f691c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.925+0000 7f6924511700 1 -- 172.21.15.49:0/2189378367 >> 172.21.15.49:0/2189378367 conn(0x7f691c0fc7d0 msgr2=0x7f691c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.925+0000 7f6924511700 1 -- 172.21.15.49:0/2189378367 shutdown_connections 2024-05-15T16:17:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.925+0000 7f6924511700 1 -- 172.21.15.49:0/2189378367 wait complete. 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.925+0000 7f6924511700 1 Processor -- start 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f6924511700 1 -- start start 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f6924511700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073340 0x7f691c19d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f6924511700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c073c60 0x7f691c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f6924511700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c19dd70 con 0x7f691c073340 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f6924511700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c1a1a40 con 0x7f691c073c60 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f69222ad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073340 0x7f691c19d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f69222ad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073340 0x7f691c19d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36892/0 (socket says 172.21.15.49:36892) 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f69222ad700 1 -- 172.21.15.49:0/1820674178 learned_addr learned my addr 172.21.15.49:0/1820674178 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f6921aac700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c073c60 0x7f691c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f69222ad700 1 -- 172.21.15.49:0/1820674178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c073c60 msgr2=0x7f691c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f69222ad700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c073c60 0x7f691c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:40.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f69222ad700 1 -- 172.21.15.49:0/1820674178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0095f0 con 0x7f691c073340 2024-05-15T16:17:40.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f6921aac700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c073c60 0x7f691c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:17:40.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.926+0000 7f69222ad700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073340 0x7f691c19d150 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f691800c8a0 tx=0x7f691800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:40.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.927+0000 7f69137fe700 1 -- 172.21.15.49:0/1820674178 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691800cea0 con 0x7f691c073340 2024-05-15T16:17:40.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.927+0000 7f69137fe700 1 -- 172.21.15.49:0/1820674178 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6918004830 con 0x7f691c073340 2024-05-15T16:17:40.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.927+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c1a1d20 con 0x7f691c073340 2024-05-15T16:17:40.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.927+0000 7f69137fe700 1 -- 172.21.15.49:0/1820674178 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69180055b0 con 0x7f691c073340 2024-05-15T16:17:40.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.927+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691c1a2270 con 0x7f691c073340 2024-05-15T16:17:40.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.928+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691c10eef0 con 0x7f691c073340 2024-05-15T16:17:40.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.928+0000 7f69137fe700 1 -- 172.21.15.49:0/1820674178 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f69180049a0 con 0x7f691c073340 2024-05-15T16:17:40.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.929+0000 7f69137fe700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f690806e210 0x7f69080706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:40.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.929+0000 7f69137fe700 1 -- 172.21.15.49:0/1820674178 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f691808de40 con 0x7f691c073340 2024-05-15T16:17:40.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.929+0000 7f6921aac700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f690806e210 0x7f69080706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:40.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.929+0000 7f6921aac700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f690806e210 0x7f69080706d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f690c006010 tx=0x7f690c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:40.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:40.933+0000 7f69137fe700 1 -- 172.21.15.49:0/1820674178 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6918057120 con 0x7f691c073340 2024-05-15T16:17:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.225+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f691c19e500 con 0x7f691c073340 2024-05-15T16:17:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.226+0000 7f69137fe700 1 -- 172.21.15.49:0/1820674178 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f691801d360 con 0x7f691c073340 2024-05-15T16:17:41.226 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:17:41.226 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:17:41.226 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:17:41.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:17:41.227 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:17:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f690806e210 msgr2=0x7f69080706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f690806e210 0x7f69080706d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f690c006010 tx=0x7f690c00b540 comp rx=0 tx=0).stop 2024-05-15T16:17:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073340 msgr2=0x7f691c19d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073340 0x7f691c19d150 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f691800c8a0 tx=0x7f691800cc60 comp rx=0 tx=0).stop 2024-05-15T16:17:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 shutdown_connections 2024-05-15T16:17:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f690806e210 0x7f69080706d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f691c073340 0x7f691c19d150 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 --2- 172.21.15.49:0/1820674178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c073c60 0x7f691c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 >> 172.21.15.49:0/1820674178 conn(0x7f691c0fc7d0 msgr2=0x7f691c104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 shutdown_connections 2024-05-15T16:17:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.228+0000 7f6924511700 1 -- 172.21.15.49:0/1820674178 wait complete. 2024-05-15T16:17:41.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 -- 172.21.15.49:0/4106433989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 msgr2=0x7fa3a01022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 --2- 172.21.15.49:0/4106433989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 0x7fa3a01022f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009910 tx=0x7fa39c009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 -- 172.21.15.49:0/4106433989 shutdown_connections 2024-05-15T16:17:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 --2- 172.21.15.49:0/4106433989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0102830 0x7fa3a010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 --2- 172.21.15.49:0/4106433989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 0x7fa3a01022f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 -- 172.21.15.49:0/4106433989 >> 172.21.15.49:0/4106433989 conn(0x7fa3a00fb6a0 msgr2=0x7fa3a00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 -- 172.21.15.49:0/4106433989 shutdown_connections 2024-05-15T16:17:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 -- 172.21.15.49:0/4106433989 wait complete. 2024-05-15T16:17:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 Processor -- start 2024-05-15T16:17:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.338+0000 7fa3a7c79700 1 -- start start 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a7c79700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 0x7fa3a019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a7c79700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0102830 0x7fa3a019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a7c79700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a019d490 con 0x7fa3a0101f10 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a7c79700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a0196a70 con 0x7fa3a0102830 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a5a15700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 0x7fa3a019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a5a15700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 0x7fa3a019c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36904/0 (socket says 172.21.15.49:36904) 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a5a15700 1 -- 172.21.15.49:0/614476589 learned_addr learned my addr 172.21.15.49:0/614476589 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a5214700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0102830 0x7fa3a019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a5a15700 1 -- 172.21.15.49:0/614476589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0102830 msgr2=0x7fa3a019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a5a15700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0102830 0x7fa3a019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a5a15700 1 -- 172.21.15.49:0/614476589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c0095f0 con 0x7fa3a0101f10 2024-05-15T16:17:41.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.339+0000 7fa3a5a15700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 0x7fa3a019c930 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00bd70 tx=0x7fa39c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:41.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.340+0000 7fa396ffd700 1 -- 172.21.15.49:0/614476589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa39c01d070 con 0x7fa3a0101f10 2024-05-15T16:17:41.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.340+0000 7fa396ffd700 1 -- 172.21.15.49:0/614476589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa39c0056e0 con 0x7fa3a0101f10 2024-05-15T16:17:41.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.340+0000 7fa396ffd700 1 -- 172.21.15.49:0/614476589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa39c00fe40 con 0x7fa3a0101f10 2024-05-15T16:17:41.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.340+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a0196cf0 con 0x7fa3a0101f10 2024-05-15T16:17:41.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.340+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a0197190 con 0x7fa3a0101f10 2024-05-15T16:17:41.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.341+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a0105630 con 0x7fa3a0101f10 2024-05-15T16:17:41.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.341+0000 7fa396ffd700 1 -- 172.21.15.49:0/614476589 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa39c00f680 con 0x7fa3a0101f10 2024-05-15T16:17:41.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.342+0000 7fa396ffd700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa38c06e200 0x7fa38c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:41.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.342+0000 7fa396ffd700 1 -- 172.21.15.49:0/614476589 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa39c08f4f0 con 0x7fa3a0101f10 2024-05-15T16:17:41.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.342+0000 7fa3a5214700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa38c06e200 0x7fa38c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:41.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.342+0000 7fa3a5214700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa38c06e200 0x7fa38c0706c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0197c30 tx=0x7fa39000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:41.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.347+0000 7fa396ffd700 1 -- 172.21.15.49:0/614476589 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa39c05c380 con 0x7fa3a0101f10 2024-05-15T16:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.597+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa3a0197f30 con 0x7fa3a0101f10 2024-05-15T16:17:41.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.598+0000 7fa396ffd700 1 -- 172.21.15.49:0/614476589 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7fa39c05bf10 con 0x7fa3a0101f10 2024-05-15T16:17:41.598 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:17:41.598 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:17:41.598 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:17:41.598 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:17:41.599 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:17:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:17:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:17:41.601 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:17:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:17:41.601 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:17:41.601 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:17:41.601 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:17:41.601 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa38c06e200 msgr2=0x7fa38c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa38c06e200 0x7fa38c0706c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0197c30 tx=0x7fa39000b410 comp rx=0 tx=0).stop 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 msgr2=0x7fa3a019c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 0x7fa3a019c930 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00bd70 tx=0x7fa39c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 shutdown_connections 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa38c06e200 0x7fa38c0706c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3a0101f10 0x7fa3a019c930 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 --2- 172.21.15.49:0/614476589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0102830 0x7fa3a019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 >> 172.21.15.49:0/614476589 conn(0x7fa3a00fb6a0 msgr2=0x7fa3a0103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 shutdown_connections 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.600+0000 7fa3a7c79700 1 -- 172.21.15.49:0/614476589 wait complete. 2024-05-15T16:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:17:41.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.710+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/3397745191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8102770 msgr2=0x7fcdc810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.710+0000 7fcdcfbbe700 1 --2- 172.21.15.49:0/3397745191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8102770 0x7fcdc810ac60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009910 tx=0x7fcdb8009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:41.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.710+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/3397745191 shutdown_connections 2024-05-15T16:17:41.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.710+0000 7fcdcfbbe700 1 --2- 172.21.15.49:0/3397745191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8102770 0x7fcdc810ac60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.710+0000 7fcdcfbbe700 1 --2- 172.21.15.49:0/3397745191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc8101e50 0x7fcdc8102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.710+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/3397745191 >> 172.21.15.49:0/3397745191 conn(0x7fcdc80fb6a0 msgr2=0x7fcdc80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.710+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/3397745191 shutdown_connections 2024-05-15T16:17:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.710+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/3397745191 wait complete. 2024-05-15T16:17:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.710+0000 7fcdcfbbe700 1 Processor -- start 2024-05-15T16:17:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcfbbe700 1 -- start start 2024-05-15T16:17:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcfbbe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8101e50 0x7fcdc80ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcfbbe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc8102770 0x7fcdc80ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcfbbe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8100530 con 0x7fcdc8101e50 2024-05-15T16:17:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcfbbe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8100670 con 0x7fcdc8102770 2024-05-15T16:17:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcd95a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8101e50 0x7fcdc80ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcd95a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8101e50 0x7fcdc80ffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36926/0 (socket says 172.21.15.49:36926) 2024-05-15T16:17:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcd95a700 1 -- 172.21.15.49:0/1583419445 learned_addr learned my addr 172.21.15.49:0/1583419445 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcd159700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc8102770 0x7fcdc80ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcd95a700 1 -- 172.21.15.49:0/1583419445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc8102770 msgr2=0x7fcdc80ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcd95a700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc8102770 0x7fcdc80ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcd95a700 1 -- 172.21.15.49:0/1583419445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb80095f0 con 0x7fcdc8101e50 2024-05-15T16:17:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcd159700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc8102770 0x7fcdc80ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:17:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.711+0000 7fcdcd95a700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8101e50 0x7fcdc80ffab0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400d900 tx=0x7fcdc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.712+0000 7fcdbeffd700 1 -- 172.21.15.49:0/1583419445 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc40041f0 con 0x7fcdc8101e50 2024-05-15T16:17:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.712+0000 7fcdbeffd700 1 -- 172.21.15.49:0/1583419445 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdc4004cf0 con 0x7fcdc8101e50 2024-05-15T16:17:41.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.712+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc81a2e10 con 0x7fcdc8101e50 2024-05-15T16:17:41.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.712+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc81a3170 con 0x7fcdc8101e50 2024-05-15T16:17:41.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.712+0000 7fcdbeffd700 1 -- 172.21.15.49:0/1583419445 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc400b750 con 0x7fcdc8101e50 2024-05-15T16:17:41.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.713+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc8069af0 con 0x7fcdc8101e50 2024-05-15T16:17:41.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.713+0000 7fcdbeffd700 1 -- 172.21.15.49:0/1583419445 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcdc4004350 con 0x7fcdc8101e50 2024-05-15T16:17:41.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.714+0000 7fcdbeffd700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdb406e200 0x7fcdb40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:41.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.714+0000 7fcdbeffd700 1 -- 172.21.15.49:0/1583419445 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcdc408e2c0 con 0x7fcdc8101e50 2024-05-15T16:17:41.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.714+0000 7fcdcd159700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdb406e200 0x7fcdb40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:41.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.714+0000 7fcdcd159700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdb406e200 0x7fcdb40706c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8006010 tx=0x7fcdb800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:41.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.719+0000 7fcdbeffd700 1 -- 172.21.15.49:0/1583419445 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcdc40575a0 con 0x7fcdc8101e50 2024-05-15T16:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.949+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdc810deb0 con 0x7fcdb406e200 2024-05-15T16:17:41.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.950+0000 7fcdbeffd700 1 -- 172.21.15.49:0/1583419445 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcdc810deb0 con 0x7fcdb406e200 2024-05-15T16:17:41.954 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:17:41.954 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:17:41.954 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:17:41.955 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:17:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.952+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdb406e200 msgr2=0x7fcdb40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.952+0000 7fcdcfbbe700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdb406e200 0x7fcdb40706c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8006010 tx=0x7fcdb800b540 comp rx=0 tx=0).stop 2024-05-15T16:17:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.952+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8101e50 msgr2=0x7fcdc80ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.952+0000 7fcdcfbbe700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8101e50 0x7fcdc80ffab0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400d900 tx=0x7fcdc400dcc0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.952+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 shutdown_connections 2024-05-15T16:17:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.952+0000 7fcdcfbbe700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcdb406e200 0x7fcdb40706c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.952+0000 7fcdcfbbe700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc8101e50 0x7fcdc80ffab0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.952+0000 7fcdcfbbe700 1 --2- 172.21.15.49:0/1583419445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc8102770 0x7fcdc80ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:41.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.953+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 >> 172.21.15.49:0/1583419445 conn(0x7fcdc80fb6a0 msgr2=0x7fcdc8103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:41.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.953+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 shutdown_connections 2024-05-15T16:17:41.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:41.953+0000 7fcdcfbbe700 1 -- 172.21.15.49:0/1583419445 wait complete. 2024-05-15T16:17:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.064+0000 7f697531d700 1 -- 172.21.15.49:0/2901231967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69700ffe10 msgr2=0x7f69701001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.064+0000 7f697531d700 1 --2- 172.21.15.49:0/2901231967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69700ffe10 0x7f69701001f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6960009910 tx=0x7f6960009c20 comp rx=0 tx=0).stop 2024-05-15T16:17:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.064+0000 7f697531d700 1 -- 172.21.15.49:0/2901231967 shutdown_connections 2024-05-15T16:17:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.064+0000 7f697531d700 1 --2- 172.21.15.49:0/2901231967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69701007c0 0x7f6970104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.064+0000 7f697531d700 1 --2- 172.21.15.49:0/2901231967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69700ffe10 0x7f69701001f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.064+0000 7f697531d700 1 -- 172.21.15.49:0/2901231967 >> 172.21.15.49:0/2901231967 conn(0x7f69700fb700 msgr2=0x7f69700fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.064+0000 7f697531d700 1 -- 172.21.15.49:0/2901231967 shutdown_connections 2024-05-15T16:17:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.064+0000 7f697531d700 1 -- 172.21.15.49:0/2901231967 wait complete. 2024-05-15T16:17:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.065+0000 7f697531d700 1 Processor -- start 2024-05-15T16:17:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.065+0000 7f697531d700 1 -- start start 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.065+0000 7f697531d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69700ffe10 0x7f6970198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.065+0000 7f697531d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69701007c0 0x7f6970199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.065+0000 7f697531d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6970199b00 con 0x7f69701007c0 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.065+0000 7f697531d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697019d7d0 con 0x7f69700ffe10 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f696e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69701007c0 0x7f6970199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f696e7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69701007c0 0x7f6970199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36942/0 (socket says 172.21.15.49:36942) 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f696e7fc700 1 -- 172.21.15.49:0/1356161235 learned_addr learned my addr 172.21.15.49:0/1356161235 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f696effd700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69700ffe10 0x7f6970198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f696e7fc700 1 -- 172.21.15.49:0/1356161235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69700ffe10 msgr2=0x7f6970198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f696e7fc700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69700ffe10 0x7f6970198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f696e7fc700 1 -- 172.21.15.49:0/1356161235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69600095f0 con 0x7f69701007c0 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f696e7fc700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69701007c0 0x7f6970199420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f696400b480 tx=0x7f696400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:42.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f6957fff700 1 -- 172.21.15.49:0/1356161235 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6964011800 con 0x7f69701007c0 2024-05-15T16:17:42.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697019dab0 con 0x7f69701007c0 2024-05-15T16:17:42.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f6957fff700 1 -- 172.21.15.49:0/1356161235 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6964011e20 con 0x7f69701007c0 2024-05-15T16:17:42.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f6957fff700 1 -- 172.21.15.49:0/1356161235 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6964010ea0 con 0x7f69701007c0 2024-05-15T16:17:42.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.066+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697019e000 con 0x7f69701007c0 2024-05-15T16:17:42.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.067+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69701012f0 con 0x7f69701007c0 2024-05-15T16:17:42.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.068+0000 7f6957fff700 1 -- 172.21.15.49:0/1356161235 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6964011960 con 0x7f69701007c0 2024-05-15T16:17:42.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.068+0000 7f6957fff700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f695806e200 0x7f69580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:17:42.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.069+0000 7f6957fff700 1 -- 172.21.15.49:0/1356161235 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f696408dcf0 con 0x7f69701007c0 2024-05-15T16:17:42.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.069+0000 7f696effd700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f695806e200 0x7f69580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:17:42.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.069+0000 7f696effd700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f695806e200 0x7f69580706c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6960006010 tx=0x7f696001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:17:42.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.076+0000 7f6957fff700 1 -- 172.21.15.49:0/1356161235 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6964056f30 con 0x7f69701007c0 2024-05-15T16:17:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:41 smithi144 ceph-mon[86635]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:41 smithi144 ceph-mon[86635]: pgmap v619: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:41 smithi144 ceph-mon[86635]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:41 smithi144 ceph-mon[86635]: from='client.35166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:41 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1820674178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:17:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:41 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/614476589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:17:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:41 smithi049 ceph-mon[112142]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:41 smithi049 ceph-mon[112142]: pgmap v619: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:41 smithi049 ceph-mon[112142]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:41 smithi049 ceph-mon[112142]: from='client.35166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:41 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1820674178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:17:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:41 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/614476589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:17:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.374+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f697004ec60 con 0x7f69701007c0 2024-05-15T16:17:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.374+0000 7f6957fff700 1 -- 172.21.15.49:0/1356161235 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f696405a5f0 con 0x7f69701007c0 2024-05-15T16:17:42.375 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:17:42.375 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:17:42.375 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:17:42.375 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:17:42.375 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:17:42.375 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:17:42.375 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:17:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f695806e200 msgr2=0x7f69580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f695806e200 0x7f69580706c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6960006010 tx=0x7f696001a040 comp rx=0 tx=0).stop 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69701007c0 msgr2=0x7f6970199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69701007c0 0x7f6970199420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f696400b480 tx=0x7f696400b790 comp rx=0 tx=0).stop 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 shutdown_connections 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f695806e200 0x7f69580706c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69700ffe10 0x7f6970198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 --2- 172.21.15.49:0/1356161235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69701007c0 0x7f6970199420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 >> 172.21.15.49:0/1356161235 conn(0x7f69700fb700 msgr2=0x7f69700fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 shutdown_connections 2024-05-15T16:17:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:17:42.376+0000 7f697531d700 1 -- 172.21.15.49:0/1356161235 wait complete. 2024-05-15T16:17:43.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:42 smithi144 ceph-mon[86635]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:43.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:42 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1356161235' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:17:43.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:42 smithi049 ceph-mon[112142]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:17:43.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:42 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1356161235' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:17:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:43 smithi144 ceph-mon[86635]: pgmap v620: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:43 smithi049 ceph-mon[112142]: pgmap v620: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:45 smithi144 ceph-mon[86635]: pgmap v621: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:45 smithi049 ceph-mon[112142]: pgmap v621: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:47 smithi144 ceph-mon[86635]: pgmap v622: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:47 smithi049 ceph-mon[112142]: pgmap v622: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:49 smithi144 ceph-mon[86635]: pgmap v623: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:49 smithi049 ceph-mon[112142]: pgmap v623: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:17:51.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:17:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:51 smithi144 ceph-mon[86635]: pgmap v624: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:51 smithi049 ceph-mon[112142]: pgmap v624: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:53 smithi144 ceph-mon[86635]: pgmap v625: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:53 smithi049 ceph-mon[112142]: pgmap v625: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:55 smithi144 ceph-mon[86635]: pgmap v626: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:55 smithi049 ceph-mon[112142]: pgmap v626: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:57 smithi144 ceph-mon[86635]: pgmap v627: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:17:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:57 smithi049 ceph-mon[112142]: pgmap v627: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:17:59 smithi144 ceph-mon[86635]: pgmap v628: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:17:59 smithi049 ceph-mon[112142]: pgmap v628: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:01 smithi144 ceph-mon[86635]: pgmap v629: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:01 smithi049 ceph-mon[112142]: pgmap v629: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:03 smithi144 ceph-mon[86635]: pgmap v630: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:03 smithi049 ceph-mon[112142]: pgmap v630: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:05 smithi144 ceph-mon[86635]: pgmap v631: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:18:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:05 smithi049 ceph-mon[112142]: pgmap v631: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:18:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:07 smithi049 ceph-mon[112142]: pgmap v632: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:08.214 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:07 smithi144 ceph-mon[86635]: pgmap v632: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:09 smithi049 ceph-mon[112142]: pgmap v633: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:09 smithi144 ceph-mon[86635]: pgmap v633: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:12.362 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:11 smithi049 ceph-mon[112142]: pgmap v634: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:11 smithi144 ceph-mon[86635]: pgmap v634: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.496+0000 7f451e835700 1 -- 172.21.15.49:0/1583411345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4518100780 msgr2=0x7f45181047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.496+0000 7f451e835700 1 --2- 172.21.15.49:0/1583411345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4518100780 0x7f45181047d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4500009910 tx=0x7f4500009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.496+0000 7f451e835700 1 -- 172.21.15.49:0/1583411345 shutdown_connections 2024-05-15T16:18:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.496+0000 7f451e835700 1 --2- 172.21.15.49:0/1583411345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4518100780 0x7f45181047d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.496+0000 7f451e835700 1 --2- 172.21.15.49:0/1583411345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45180ffdd0 0x7f45181001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.496+0000 7f451e835700 1 -- 172.21.15.49:0/1583411345 >> 172.21.15.49:0/1583411345 conn(0x7f45180fb6a0 msgr2=0x7f45180fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.496+0000 7f451e835700 1 -- 172.21.15.49:0/1583411345 shutdown_connections 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.496+0000 7f451e835700 1 -- 172.21.15.49:0/1583411345 wait complete. 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.497+0000 7f451e835700 1 Processor -- start 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.497+0000 7f451e835700 1 -- start start 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.497+0000 7f451e835700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45180ffdd0 0x7f4518198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.497+0000 7f451e835700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518100780 0x7f4518199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.497+0000 7f451e835700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4518199a00 con 0x7f45180ffdd0 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.497+0000 7f451e835700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451819d6d0 con 0x7f4518100780 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.497+0000 7f4517fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45180ffdd0 0x7f4518198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f4517fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45180ffdd0 0x7f4518198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54574/0 (socket says 172.21.15.49:54574) 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f4517fff700 1 -- 172.21.15.49:0/2612969333 learned_addr learned my addr 172.21.15.49:0/2612969333 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f450ffff700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518100780 0x7f4518199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f4517fff700 1 -- 172.21.15.49:0/2612969333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518100780 msgr2=0x7f4518199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:12.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f4517fff700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518100780 0x7f4518199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:12.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f4517fff700 1 -- 172.21.15.49:0/2612969333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45000095f0 con 0x7f45180ffdd0 2024-05-15T16:18:12.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f4517fff700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45180ffdd0 0x7f4518198de0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f450800c8a0 tx=0x7f450800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f4515ffb700 1 -- 172.21.15.49:0/2612969333 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f450800cea0 con 0x7f45180ffdd0 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f4515ffb700 1 -- 172.21.15.49:0/2612969333 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4508004830 con 0x7f45180ffdd0 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f4515ffb700 1 -- 172.21.15.49:0/2612969333 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45080055b0 con 0x7f45180ffdd0 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451819d9b0 con 0x7f45180ffdd0 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.498+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451819df00 con 0x7f45180ffdd0 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.499+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45181012b0 con 0x7f45180ffdd0 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.500+0000 7f4515ffb700 1 -- 172.21.15.49:0/2612969333 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f45080049a0 con 0x7f45180ffdd0 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.500+0000 7f4515ffb700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f450406e200 0x7f45040706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.501+0000 7f4515ffb700 1 -- 172.21.15.49:0/2612969333 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f450808dee0 con 0x7f45180ffdd0 2024-05-15T16:18:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.501+0000 7f450ffff700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f450406e200 0x7f45040706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:12.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.501+0000 7f450ffff700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f450406e200 0x7f45040706c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f451819a340 tx=0x7f450001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:12.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.505+0000 7f4515ffb700 1 -- 172.21.15.49:0/2612969333 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4508057120 con 0x7f45180ffdd0 2024-05-15T16:18:12.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.733+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f451810cd70 con 0x7f450406e200 2024-05-15T16:18:12.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.734+0000 7f4515ffb700 1 -- 172.21.15.49:0/2612969333 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f451810cd70 con 0x7f450406e200 2024-05-15T16:18:12.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f450406e200 msgr2=0x7f45040706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f450406e200 0x7f45040706c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f451819a340 tx=0x7f450001a040 comp rx=0 tx=0).stop 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45180ffdd0 msgr2=0x7f4518198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45180ffdd0 0x7f4518198de0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f450800c8a0 tx=0x7f450800cc60 comp rx=0 tx=0).stop 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 shutdown_connections 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f450406e200 0x7f45040706c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45180ffdd0 0x7f4518198de0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 --2- 172.21.15.49:0/2612969333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4518100780 0x7f4518199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 >> 172.21.15.49:0/2612969333 conn(0x7f45180fb6a0 msgr2=0x7f45180fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 shutdown_connections 2024-05-15T16:18:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.736+0000 7f451e835700 1 -- 172.21.15.49:0/2612969333 wait complete. 2024-05-15T16:18:12.748 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:18:12.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.846+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3185527823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4041007a0 msgr2=0x7fd4041047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:12.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.846+0000 7fd40bdc5700 1 --2- 172.21.15.49:0/3185527823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4041007a0 0x7fd4041047f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009910 tx=0x7fd3f4009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:12.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.846+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3185527823 shutdown_connections 2024-05-15T16:18:12.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.846+0000 7fd40bdc5700 1 --2- 172.21.15.49:0/3185527823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4041007a0 0x7fd4041047f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:12.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.846+0000 7fd40bdc5700 1 --2- 172.21.15.49:0/3185527823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4040ffdf0 0x7fd4041001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:12.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.846+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3185527823 >> 172.21.15.49:0/3185527823 conn(0x7fd4040fb6a0 msgr2=0x7fd4040fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:12.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.846+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3185527823 shutdown_connections 2024-05-15T16:18:12.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.846+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3185527823 wait complete. 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd40bdc5700 1 Processor -- start 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd40bdc5700 1 -- start start 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd40bdc5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4040ffdf0 0x7fd40419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd40bdc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4041007a0 0x7fd40419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd40bdc5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40419d470 con 0x7fd4040ffdf0 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd40bdc5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd404196a50 con 0x7fd4041007a0 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd409b61700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4040ffdf0 0x7fd40419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd409b61700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4040ffdf0 0x7fd40419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54588/0 (socket says 172.21.15.49:54588) 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd409b61700 1 -- 172.21.15.49:0/3179743109 learned_addr learned my addr 172.21.15.49:0/3179743109 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd409360700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4041007a0 0x7fd40419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd409b61700 1 -- 172.21.15.49:0/3179743109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4041007a0 msgr2=0x7fd40419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd409b61700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4041007a0 0x7fd40419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.847+0000 7fd409b61700 1 -- 172.21.15.49:0/3179743109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f40095f0 con 0x7fd4040ffdf0 2024-05-15T16:18:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.848+0000 7fd409b61700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4040ffdf0 0x7fd40419c910 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d8d0 tx=0x7fd40000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.848+0000 7fd3faffd700 1 -- 172.21.15.49:0/3179743109 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd400009940 con 0x7fd4040ffdf0 2024-05-15T16:18:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.848+0000 7fd3faffd700 1 -- 172.21.15.49:0/3179743109 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd400010460 con 0x7fd4040ffdf0 2024-05-15T16:18:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.848+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd404196d30 con 0x7fd4040ffdf0 2024-05-15T16:18:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.848+0000 7fd3faffd700 1 -- 172.21.15.49:0/3179743109 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40000f5f0 con 0x7fd4040ffdf0 2024-05-15T16:18:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.848+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd404197280 con 0x7fd4040ffdf0 2024-05-15T16:18:12.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.849+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd404109200 con 0x7fd4040ffdf0 2024-05-15T16:18:12.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.850+0000 7fd3faffd700 1 -- 172.21.15.49:0/3179743109 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd4000105d0 con 0x7fd4040ffdf0 2024-05-15T16:18:12.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.850+0000 7fd3faffd700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd3f006e210 0x7fd3f00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:12.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.850+0000 7fd3faffd700 1 -- 172.21.15.49:0/3179743109 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd40008e090 con 0x7fd4040ffdf0 2024-05-15T16:18:12.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.850+0000 7fd409360700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd3f006e210 0x7fd3f00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:12.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.851+0000 7fd409360700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd3f006e210 0x7fd3f00706d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd404197c10 tx=0x7fd3f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:12.855+0000 7fd3faffd700 1 -- 172.21.15.49:0/3179743109 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd40005af20 con 0x7fd4040ffdf0 2024-05-15T16:18:13.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.081+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd404197f80 con 0x7fd3f006e210 2024-05-15T16:18:13.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.083+0000 7fd3faffd700 1 -- 172.21.15.49:0/3179743109 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd404197f80 con 0x7fd3f006e210 2024-05-15T16:18:13.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd3f006e210 msgr2=0x7fd3f00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd3f006e210 0x7fd3f00706d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd404197c10 tx=0x7fd3f401a040 comp rx=0 tx=0).stop 2024-05-15T16:18:13.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4040ffdf0 msgr2=0x7fd40419c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4040ffdf0 0x7fd40419c910 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d8d0 tx=0x7fd40000dc90 comp rx=0 tx=0).stop 2024-05-15T16:18:13.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 shutdown_connections 2024-05-15T16:18:13.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd3f006e210 0x7fd3f00706d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4040ffdf0 0x7fd40419c910 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 --2- 172.21.15.49:0/3179743109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4041007a0 0x7fd40419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 >> 172.21.15.49:0/3179743109 conn(0x7fd4040fb6a0 msgr2=0x7fd4040697c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:13.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 shutdown_connections 2024-05-15T16:18:13.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.085+0000 7fd40bdc5700 1 -- 172.21.15.49:0/3179743109 wait complete. 2024-05-15T16:18:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.193+0000 7f8362a6a700 1 -- 172.21.15.49:0/1662002484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c0ffdc0 msgr2=0x7f835c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.193+0000 7f8362a6a700 1 --2- 172.21.15.49:0/1662002484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c0ffdc0 0x7f835c1001a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f834c009910 tx=0x7f834c009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.193+0000 7f8362a6a700 1 -- 172.21.15.49:0/1662002484 shutdown_connections 2024-05-15T16:18:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.193+0000 7f8362a6a700 1 --2- 172.21.15.49:0/1662002484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f835c100770 0x7f835c1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.193+0000 7f8362a6a700 1 --2- 172.21.15.49:0/1662002484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c0ffdc0 0x7f835c1001a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.193+0000 7f8362a6a700 1 -- 172.21.15.49:0/1662002484 >> 172.21.15.49:0/1662002484 conn(0x7f835c076800 msgr2=0x7f835c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.193+0000 7f8362a6a700 1 -- 172.21.15.49:0/1662002484 shutdown_connections 2024-05-15T16:18:13.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.193+0000 7f8362a6a700 1 -- 172.21.15.49:0/1662002484 wait complete. 2024-05-15T16:18:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f8362a6a700 1 Processor -- start 2024-05-15T16:18:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f8362a6a700 1 -- start start 2024-05-15T16:18:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f8362a6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f835c0ffdc0 0x7f835c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f8362a6a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c100770 0x7f835c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f8362a6a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1984d0 con 0x7f835c100770 2024-05-15T16:18:13.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f8362a6a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c198610 con 0x7f835c0ffdc0 2024-05-15T16:18:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f835bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c100770 0x7f835c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f835bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c100770 0x7f835c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54608/0 (socket says 172.21.15.49:54608) 2024-05-15T16:18:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f835bfff700 1 -- 172.21.15.49:0/1933926951 learned_addr learned my addr 172.21.15.49:0/1933926951 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f8360806700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f835c0ffdc0 0x7f835c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f835bfff700 1 -- 172.21.15.49:0/1933926951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f835c0ffdc0 msgr2=0x7f835c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f835bfff700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f835c0ffdc0 0x7f835c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.194+0000 7f835bfff700 1 -- 172.21.15.49:0/1933926951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c0095f0 con 0x7f835c100770 2024-05-15T16:18:13.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.195+0000 7f835bfff700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c100770 0x7f835c197f00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8344007e90 tx=0x7f834400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:13.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.195+0000 7f8359ffb700 1 -- 172.21.15.49:0/1933926951 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834400dda0 con 0x7f835c100770 2024-05-15T16:18:13.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.195+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c1a0a30 con 0x7f835c100770 2024-05-15T16:18:13.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.195+0000 7f8359ffb700 1 -- 172.21.15.49:0/1933926951 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f834400f040 con 0x7f835c100770 2024-05-15T16:18:13.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.195+0000 7f8359ffb700 1 -- 172.21.15.49:0/1933926951 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83440125f0 con 0x7f835c100770 2024-05-15T16:18:13.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.195+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c1a0ea0 con 0x7f835c100770 2024-05-15T16:18:13.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.196+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835c108160 con 0x7f835c100770 2024-05-15T16:18:13.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.196+0000 7f8359ffb700 1 -- 172.21.15.49:0/1933926951 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f83440133f0 con 0x7f835c100770 2024-05-15T16:18:13.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.197+0000 7f8359ffb700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f834806e200 0x7f83480706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:13.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.197+0000 7f8359ffb700 1 -- 172.21.15.49:0/1933926951 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f834408db00 con 0x7f835c100770 2024-05-15T16:18:13.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.197+0000 7f8360806700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f834806e200 0x7f83480706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:13.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.197+0000 7f8360806700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f834806e200 0x7f83480706c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f834c006010 tx=0x7f834c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:13.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.202+0000 7f8359ffb700 1 -- 172.21.15.49:0/1933926951 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8344056d40 con 0x7f835c100770 2024-05-15T16:18:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.420+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f835c068cf0 con 0x7f834806e200 2024-05-15T16:18:13.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.431+0000 7f8359ffb700 1 -- 172.21.15.49:0/1933926951 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f835c068cf0 con 0x7f834806e200 2024-05-15T16:18:13.432 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:18:13.432 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (21m) 14m ago 32m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:18:13.432 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (33m) 14m ago 33m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:18:13.432 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (32m) 14m ago 32m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:18:13.432 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (18m) 14m ago 33m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (18m) 14m ago 32m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (21m) 14m ago 32m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (15m) 14m ago 27m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (14m) 14m ago 27m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (14m) 14m ago 27m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (27m) 14m ago 27m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (23m) 14m ago 34m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (22m) 14m ago 31m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (20m) 14m ago 34m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (19m) 14m ago 31m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (22m) 14m ago 33m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (22m) 14m ago 32m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (18m) 14m ago 31m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (17m) 14m ago 30m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:18:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (17m) 14m ago 30m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:18:13.434 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (16m) 14m ago 29m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:18:13.434 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (16m) 14m ago 29m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:18:13.434 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (15m) 14m ago 28m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:18:13.434 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (21m) 14m ago 32m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f834806e200 msgr2=0x7f83480706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f834806e200 0x7f83480706c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f834c006010 tx=0x7f834c01a040 comp rx=0 tx=0).stop 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c100770 msgr2=0x7f835c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c100770 0x7f835c197f00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8344007e90 tx=0x7f834400d3b0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 shutdown_connections 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f834806e200 0x7f83480706c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f835c0ffdc0 0x7f835c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 --2- 172.21.15.49:0/1933926951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c100770 0x7f835c197f00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 >> 172.21.15.49:0/1933926951 conn(0x7f835c076800 msgr2=0x7f835c103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 shutdown_connections 2024-05-15T16:18:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.433+0000 7f8362a6a700 1 -- 172.21.15.49:0/1933926951 wait complete. 2024-05-15T16:18:13.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.541+0000 7f95a31aa700 1 -- 172.21.15.49:0/3206279707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c101ee0 msgr2=0x7f959c1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.541+0000 7f95a31aa700 1 --2- 172.21.15.49:0/3206279707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c101ee0 0x7f959c1022c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9590009910 tx=0x7f9590009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:13.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.541+0000 7f95a31aa700 1 -- 172.21.15.49:0/3206279707 shutdown_connections 2024-05-15T16:18:13.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.541+0000 7f95a31aa700 1 --2- 172.21.15.49:0/3206279707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c102800 0x7f959c10acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.541+0000 7f95a31aa700 1 --2- 172.21.15.49:0/3206279707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c101ee0 0x7f959c1022c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.541+0000 7f95a31aa700 1 -- 172.21.15.49:0/3206279707 >> 172.21.15.49:0/3206279707 conn(0x7f959c076800 msgr2=0x7f959c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:13.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.541+0000 7f95a31aa700 1 -- 172.21.15.49:0/3206279707 shutdown_connections 2024-05-15T16:18:13.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.541+0000 7f95a31aa700 1 -- 172.21.15.49:0/3206279707 wait complete. 2024-05-15T16:18:13.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.542+0000 7f95a31aa700 1 Processor -- start 2024-05-15T16:18:13.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.542+0000 7f95a31aa700 1 -- start start 2024-05-15T16:18:13.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.542+0000 7f95a31aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c101ee0 0x7f959c19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:13.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.542+0000 7f95a31aa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c102800 0x7f959c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:13.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.542+0000 7f95a31aa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c19d430 con 0x7f959c102800 2024-05-15T16:18:13.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.542+0000 7f95a31aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c196a10 con 0x7f959c101ee0 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.542+0000 7f959bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c102800 0x7f959c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f959bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c102800 0x7f959c19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54620/0 (socket says 172.21.15.49:54620) 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f959bfff700 1 -- 172.21.15.49:0/1071762942 learned_addr learned my addr 172.21.15.49:0/1071762942 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f95a0f46700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c101ee0 0x7f959c19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f959bfff700 1 -- 172.21.15.49:0/1071762942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c101ee0 msgr2=0x7f959c19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f959bfff700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c101ee0 0x7f959c19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f959bfff700 1 -- 172.21.15.49:0/1071762942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900095f0 con 0x7f959c102800 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f959bfff700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c102800 0x7f959c19ce10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f958800d900 tx=0x7f958800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f9599ffb700 1 -- 172.21.15.49:0/1071762942 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9588009900 con 0x7f959c102800 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f9599ffb700 1 -- 172.21.15.49:0/1071762942 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9588010460 con 0x7f959c102800 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c196cf0 con 0x7f959c102800 2024-05-15T16:18:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f9599ffb700 1 -- 172.21.15.49:0/1071762942 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f958800f5f0 con 0x7f959c102800 2024-05-15T16:18:13.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.543+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c197240 con 0x7f959c102800 2024-05-15T16:18:13.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.545+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c105600 con 0x7f959c102800 2024-05-15T16:18:13.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.545+0000 7f9599ffb700 1 -- 172.21.15.49:0/1071762942 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f95880105d0 con 0x7f959c102800 2024-05-15T16:18:13.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.546+0000 7f9599ffb700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f958c06e210 0x7f958c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:13.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.546+0000 7f9599ffb700 1 -- 172.21.15.49:0/1071762942 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f958808dd60 con 0x7f959c102800 2024-05-15T16:18:13.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.546+0000 7f95a0f46700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f958c06e210 0x7f958c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:13.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.546+0000 7f95a0f46700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f958c06e210 0x7f958c0706d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9590006010 tx=0x7f959001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:13.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.550+0000 7f9599ffb700 1 -- 172.21.15.49:0/1071762942 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f958805ab70 con 0x7f959c102800 2024-05-15T16:18:13.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.844+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f959c197ed0 con 0x7f959c102800 2024-05-15T16:18:13.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.845+0000 7f9599ffb700 1 -- 172.21.15.49:0/1071762942 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f958805a700 con 0x7f959c102800 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:18:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:18:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:18:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:18:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:18:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:18:13.848 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f958c06e210 msgr2=0x7f958c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f958c06e210 0x7f958c0706d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9590006010 tx=0x7f959001a040 comp rx=0 tx=0).stop 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c102800 msgr2=0x7f959c19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c102800 0x7f959c19ce10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f958800d900 tx=0x7f958800dc10 comp rx=0 tx=0).stop 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 shutdown_connections 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f958c06e210 0x7f958c0706d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c101ee0 0x7f959c19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 --2- 172.21.15.49:0/1071762942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f959c102800 0x7f959c19ce10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 >> 172.21.15.49:0/1071762942 conn(0x7f959c076800 msgr2=0x7f959c1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 shutdown_connections 2024-05-15T16:18:13.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.847+0000 7f95a31aa700 1 -- 172.21.15.49:0/1071762942 wait complete. 2024-05-15T16:18:13.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.958+0000 7f8671ff6700 1 -- 172.21.15.49:0/3552364993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c100ed0 msgr2=0x7f866c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.958+0000 7f8671ff6700 1 --2- 172.21.15.49:0/3552364993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c100ed0 0x7f866c1012b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f865c009910 tx=0x7f865c009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.959+0000 7f8671ff6700 1 -- 172.21.15.49:0/3552364993 shutdown_connections 2024-05-15T16:18:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.959+0000 7f8671ff6700 1 --2- 172.21.15.49:0/3552364993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f866c101880 0x7f866c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.959+0000 7f8671ff6700 1 --2- 172.21.15.49:0/3552364993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c100ed0 0x7f866c1012b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.959+0000 7f8671ff6700 1 -- 172.21.15.49:0/3552364993 >> 172.21.15.49:0/3552364993 conn(0x7f866c0fc7a0 msgr2=0x7f866c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.959+0000 7f8671ff6700 1 -- 172.21.15.49:0/3552364993 shutdown_connections 2024-05-15T16:18:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.959+0000 7f8671ff6700 1 -- 172.21.15.49:0/3552364993 wait complete. 2024-05-15T16:18:13.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.959+0000 7f8671ff6700 1 Processor -- start 2024-05-15T16:18:13.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f8671ff6700 1 -- start start 2024-05-15T16:18:13.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f8671ff6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f866c100ed0 0x7f866c0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f8671ff6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c101880 0x7f866c073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f8671ff6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0752a0 con 0x7f866c101880 2024-05-15T16:18:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f8671ff6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c073d40 con 0x7f866c100ed0 2024-05-15T16:18:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f866affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c101880 0x7f866c073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f866affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c101880 0x7f866c073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44192/0 (socket says 172.21.15.49:44192) 2024-05-15T16:18:13.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f866affd700 1 -- 172.21.15.49:0/3819221102 learned_addr learned my addr 172.21.15.49:0/3819221102 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f866b7fe700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f866c100ed0 0x7f866c0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f866affd700 1 -- 172.21.15.49:0/3819221102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f866c100ed0 msgr2=0x7f866c0732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f866affd700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f866c100ed0 0x7f866c0732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f866affd700 1 -- 172.21.15.49:0/3819221102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c0095f0 con 0x7f866c101880 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f866b7fe700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f866c100ed0 0x7f866c0732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.960+0000 7f866affd700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c101880 0x7f866c073800 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f866000e970 tx=0x7f866000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.961+0000 7f8668ff9700 1 -- 172.21.15.49:0/3819221102 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8660009750 con 0x7f866c101880 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.961+0000 7f8668ff9700 1 -- 172.21.15.49:0/3819221102 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8660004cf0 con 0x7f866c101880 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.961+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c074020 con 0x7f866c101880 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.961+0000 7f8668ff9700 1 -- 172.21.15.49:0/3819221102 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8660005660 con 0x7f866c101880 2024-05-15T16:18:13.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.961+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c1a30e0 con 0x7f866c101880 2024-05-15T16:18:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.962+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c109270 con 0x7f866c101880 2024-05-15T16:18:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.962+0000 7f8668ff9700 1 -- 172.21.15.49:0/3819221102 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8660004830 con 0x7f866c101880 2024-05-15T16:18:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.963+0000 7f8668ff9700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865406e210 0x7f86540706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.963+0000 7f8668ff9700 1 -- 172.21.15.49:0/3819221102 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8660014070 con 0x7f866c101880 2024-05-15T16:18:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.963+0000 7f866b7fe700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865406e210 0x7f86540706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:13.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.963+0000 7f866b7fe700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865406e210 0x7f86540706d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f865c00b760 tx=0x7f865c005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:13.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:13.967+0000 7f8668ff9700 1 -- 172.21.15.49:0/3819221102 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8660057820 con 0x7f866c101880 2024-05-15T16:18:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:13 smithi049 ceph-mon[112142]: pgmap v635: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:13 smithi049 ceph-mon[112142]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:13 smithi049 ceph-mon[112142]: from='client.35190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:13 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1071762942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.221+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f866c1023b0 con 0x7f866c101880 2024-05-15T16:18:14.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.222+0000 7f8668ff9700 1 -- 172.21.15.49:0/3819221102 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f866005aee0 con 0x7f866c101880 2024-05-15T16:18:14.223 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:18:14.223 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:18:14.224 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:18:14.225 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:18:14.226 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:18:14.226 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:18:14.226 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:18:14.226 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:18:14.226 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:18:14.226 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:18:14.226 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:18:14.226 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865406e210 msgr2=0x7f86540706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865406e210 0x7f86540706d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f865c00b760 tx=0x7f865c005a90 comp rx=0 tx=0).stop 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c101880 msgr2=0x7f866c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c101880 0x7f866c073800 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f866000e970 tx=0x7f866000ed30 comp rx=0 tx=0).stop 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 shutdown_connections 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865406e210 0x7f86540706d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f866c100ed0 0x7f866c0732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 --2- 172.21.15.49:0/3819221102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f866c101880 0x7f866c073800 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 >> 172.21.15.49:0/3819221102 conn(0x7f866c0fc7a0 msgr2=0x7f866c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 shutdown_connections 2024-05-15T16:18:14.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.225+0000 7f8671ff6700 1 -- 172.21.15.49:0/3819221102 wait complete. 2024-05-15T16:18:14.228 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:18:14.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.334+0000 7fafbaa90700 1 -- 172.21.15.49:0/4026789053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4102740 msgr2=0x7fafb410ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.334+0000 7fafbaa90700 1 --2- 172.21.15.49:0/4026789053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4102740 0x7fafb410ac30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009910 tx=0x7fafa0009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.334+0000 7fafbaa90700 1 -- 172.21.15.49:0/4026789053 shutdown_connections 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.334+0000 7fafbaa90700 1 --2- 172.21.15.49:0/4026789053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4102740 0x7fafb410ac30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 --2- 172.21.15.49:0/4026789053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb4101e20 0x7fafb4102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 -- 172.21.15.49:0/4026789053 >> 172.21.15.49:0/4026789053 conn(0x7fafb4076800 msgr2=0x7fafb40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 -- 172.21.15.49:0/4026789053 shutdown_connections 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 -- 172.21.15.49:0/4026789053 wait complete. 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 Processor -- start 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 -- start start 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4101e20 0x7fafb419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb4102740 0x7fafb419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb419d3d0 con 0x7fafb4101e20 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.335+0000 7fafbaa90700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4196a40 con 0x7fafb4102740 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb882c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4101e20 0x7fafb419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb882c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4101e20 0x7fafb419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44216/0 (socket says 172.21.15.49:44216) 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb882c700 1 -- 172.21.15.49:0/1056216615 learned_addr learned my addr 172.21.15.49:0/1056216615 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb3fff700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb4102740 0x7fafb419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb882c700 1 -- 172.21.15.49:0/1056216615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb4102740 msgr2=0x7fafb419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb882c700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb4102740 0x7fafb419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb882c700 1 -- 172.21.15.49:0/1056216615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa00095f0 con 0x7fafb4101e20 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb882c700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4101e20 0x7fafb419c900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c930 tx=0x7fafa800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb1ffb700 1 -- 172.21.15.49:0/1056216615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa8007ad0 con 0x7fafb4101e20 2024-05-15T16:18:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb1ffb700 1 -- 172.21.15.49:0/1056216615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafa8007c30 con 0x7fafb4101e20 2024-05-15T16:18:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafb1ffb700 1 -- 172.21.15.49:0/1056216615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa8018610 con 0x7fafb4101e20 2024-05-15T16:18:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb4196d20 con 0x7fafb4101e20 2024-05-15T16:18:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.336+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb4197270 con 0x7fafb4101e20 2024-05-15T16:18:14.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.337+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb4105540 con 0x7fafb4101e20 2024-05-15T16:18:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.338+0000 7fafb1ffb700 1 -- 172.21.15.49:0/1056216615 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fafa800f450 con 0x7fafb4101e20 2024-05-15T16:18:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.338+0000 7fafb1ffb700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fafa406e200 0x7fafa40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.339+0000 7fafb1ffb700 1 -- 172.21.15.49:0/1056216615 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fafa808e2a0 con 0x7fafb4101e20 2024-05-15T16:18:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.339+0000 7fafb3fff700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fafa406e200 0x7fafa40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.339+0000 7fafb3fff700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fafa406e200 0x7fafa40706c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fafb4197c00 tx=0x7fafa001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.344+0000 7fafb1ffb700 1 -- 172.21.15.49:0/1056216615 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fafa805b130 con 0x7fafb4101e20 2024-05-15T16:18:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:13 smithi144 ceph-mon[86635]: pgmap v635: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:13 smithi144 ceph-mon[86635]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:13 smithi144 ceph-mon[86635]: from='client.35190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:13 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1071762942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:14.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.573+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafb4197f70 con 0x7fafa406e200 2024-05-15T16:18:14.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.574+0000 7fafb1ffb700 1 -- 172.21.15.49:0/1056216615 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fafb4197f70 con 0x7fafa406e200 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:18:14.576 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:18:14.577 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:18:14.577 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:18:14.577 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:18:14.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fafa406e200 msgr2=0x7fafa40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fafa406e200 0x7fafa40706c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fafb4197c00 tx=0x7fafa001a040 comp rx=0 tx=0).stop 2024-05-15T16:18:14.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4101e20 msgr2=0x7fafb419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4101e20 0x7fafb419c900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c930 tx=0x7fafa800ccf0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 shutdown_connections 2024-05-15T16:18:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fafa406e200 0x7fafa40706c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafb4101e20 0x7fafb419c900 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 --2- 172.21.15.49:0/1056216615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb4102740 0x7fafb419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 >> 172.21.15.49:0/1056216615 conn(0x7fafb4076800 msgr2=0x7fafb4103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 shutdown_connections 2024-05-15T16:18:14.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.577+0000 7fafbaa90700 1 -- 172.21.15.49:0/1056216615 wait complete. 2024-05-15T16:18:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.687+0000 7f4124bba700 1 -- 172.21.15.49:0/546818051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4120100790 msgr2=0x7f41201047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.687+0000 7f4124bba700 1 --2- 172.21.15.49:0/546818051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4120100790 0x7f41201047e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4108009910 tx=0x7f4108009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.687+0000 7f4124bba700 1 -- 172.21.15.49:0/546818051 shutdown_connections 2024-05-15T16:18:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.687+0000 7f4124bba700 1 --2- 172.21.15.49:0/546818051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4120100790 0x7f41201047e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.687+0000 7f4124bba700 1 --2- 172.21.15.49:0/546818051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ffde0 0x7f41201001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.687+0000 7f4124bba700 1 -- 172.21.15.49:0/546818051 >> 172.21.15.49:0/546818051 conn(0x7f41200fb6b0 msgr2=0x7f41200fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.687+0000 7f4124bba700 1 -- 172.21.15.49:0/546818051 shutdown_connections 2024-05-15T16:18:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.687+0000 7f4124bba700 1 -- 172.21.15.49:0/546818051 wait complete. 2024-05-15T16:18:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.688+0000 7f4124bba700 1 Processor -- start 2024-05-15T16:18:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.688+0000 7f4124bba700 1 -- start start 2024-05-15T16:18:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.688+0000 7f4124bba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41200ffde0 0x7f412019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.688+0000 7f4124bba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4120100790 0x7f412019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.688+0000 7f4124bba700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412019c350 con 0x7f41200ffde0 2024-05-15T16:18:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.688+0000 7f4124bba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4120195980 con 0x7f4120100790 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.688+0000 7f411e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41200ffde0 0x7f412019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f411e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41200ffde0 0x7f412019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44234/0 (socket says 172.21.15.49:44234) 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f411e59c700 1 -- 172.21.15.49:0/2210706870 learned_addr learned my addr 172.21.15.49:0/2210706870 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f411dd9b700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4120100790 0x7f412019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f411e59c700 1 -- 172.21.15.49:0/2210706870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4120100790 msgr2=0x7f412019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f411e59c700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4120100790 0x7f412019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f411e59c700 1 -- 172.21.15.49:0/2210706870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080095f0 con 0x7f41200ffde0 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f411e59c700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41200ffde0 0x7f412019b7f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f411000d900 tx=0x7f411000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f41177fe700 1 -- 172.21.15.49:0/2210706870 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41100041f0 con 0x7f41200ffde0 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4120195c60 con 0x7f41200ffde0 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f41177fe700 1 -- 172.21.15.49:0/2210706870 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4110004cf0 con 0x7f41200ffde0 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f41177fe700 1 -- 172.21.15.49:0/2210706870 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411000b750 con 0x7f41200ffde0 2024-05-15T16:18:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.689+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41201961b0 con 0x7f41200ffde0 2024-05-15T16:18:14.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.690+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41201081e0 con 0x7f41200ffde0 2024-05-15T16:18:14.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.691+0000 7f41177fe700 1 -- 172.21.15.49:0/2210706870 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4110004750 con 0x7f41200ffde0 2024-05-15T16:18:14.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.691+0000 7f41177fe700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f410c06e200 0x7f410c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:14.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.692+0000 7f41177fe700 1 -- 172.21.15.49:0/2210706870 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f411008e070 con 0x7f41200ffde0 2024-05-15T16:18:14.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.692+0000 7f411dd9b700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f410c06e200 0x7f410c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:14.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.692+0000 7f411dd9b700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f410c06e200 0x7f410c0706c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4120196af0 tx=0x7f410801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:14.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.696+0000 7f41177fe700 1 -- 172.21.15.49:0/2210706870 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f411005ae80 con 0x7f41200ffde0 2024-05-15T16:18:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.991+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4120196df0 con 0x7f41200ffde0 2024-05-15T16:18:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.991+0000 7f41177fe700 1 -- 172.21.15.49:0/2210706870 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f411005aa10 con 0x7f41200ffde0 2024-05-15T16:18:14.992 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:18:14.992 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:18:14.992 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:18:14.992 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:18:14.992 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:18:14.992 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:18:14.992 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:18:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f410c06e200 msgr2=0x7f410c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f410c06e200 0x7f410c0706c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4120196af0 tx=0x7f410801a040 comp rx=0 tx=0).stop 2024-05-15T16:18:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41200ffde0 msgr2=0x7f412019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41200ffde0 0x7f412019b7f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f411000d900 tx=0x7f411000dc10 comp rx=0 tx=0).stop 2024-05-15T16:18:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 shutdown_connections 2024-05-15T16:18:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f410c06e200 0x7f410c0706c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41200ffde0 0x7f412019b7f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 --2- 172.21.15.49:0/2210706870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4120100790 0x7f412019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 >> 172.21.15.49:0/2210706870 conn(0x7f41200fb6b0 msgr2=0x7f41200fba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 shutdown_connections 2024-05-15T16:18:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:14.994+0000 7f4124bba700 1 -- 172.21.15.49:0/2210706870 wait complete. 2024-05-15T16:18:15.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:14 smithi144 ceph-mon[86635]: from='client.35194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:15.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:14 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3819221102' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:18:15.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:14 smithi049 ceph-mon[112142]: from='client.35194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:15.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:14 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3819221102' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:18:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:15 smithi144 ceph-mon[86635]: pgmap v636: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:15 smithi144 ceph-mon[86635]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:15 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2210706870' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:18:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:15 smithi049 ceph-mon[112142]: pgmap v636: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:15 smithi049 ceph-mon[112142]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:15 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2210706870' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:18:18.320 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:17 smithi144 ceph-mon[86635]: pgmap v637: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:17 smithi049 ceph-mon[112142]: pgmap v637: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:19 smithi144 ceph-mon[86635]: pgmap v638: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:19 smithi049 ceph-mon[112142]: pgmap v638: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:18:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:18:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:21 smithi144 ceph-mon[86635]: pgmap v639: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:21 smithi049 ceph-mon[112142]: pgmap v639: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:23 smithi144 ceph-mon[86635]: pgmap v640: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:23 smithi049 ceph-mon[112142]: pgmap v640: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:25 smithi144 ceph-mon[86635]: pgmap v641: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:25 smithi049 ceph-mon[112142]: pgmap v641: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:28.426 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:27 smithi144 ceph-mon[86635]: pgmap v642: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:27 smithi049 ceph-mon[112142]: pgmap v642: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:30.290 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:29 smithi144 ceph-mon[86635]: pgmap v643: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:29 smithi049 ceph-mon[112142]: pgmap v643: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:31 smithi144 ceph-mon[86635]: pgmap v644: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:31 smithi049 ceph-mon[112142]: pgmap v644: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:33 smithi144 ceph-mon[86635]: pgmap v645: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:33 smithi049 ceph-mon[112142]: pgmap v645: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:35 smithi144 ceph-mon[86635]: pgmap v646: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:18:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:35 smithi049 ceph-mon[112142]: pgmap v646: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:18:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:38 smithi144 ceph-mon[86635]: pgmap v647: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:38 smithi049 ceph-mon[112142]: pgmap v647: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:40 smithi144 ceph-mon[86635]: pgmap v648: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:40 smithi049 ceph-mon[112142]: pgmap v648: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:42 smithi144 ceph-mon[86635]: pgmap v649: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:42 smithi049 ceph-mon[112142]: pgmap v649: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:43.298 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:18:43.298 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:18:43.299 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:43.299 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:18:43.299 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:43.299 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:43.299 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:43.299 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:43 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:43.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:18:43.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:18:43.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:43.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:18:43.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:43.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:43.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:43.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:43 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:44.376 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:44 smithi144 ceph-mon[86635]: pgmap v650: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:44.376 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:44 smithi144 ceph-mon[86635]: pgmap v651: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:44.376 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:44 smithi144 ceph-mon[86635]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-05-15T16:18:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:44 smithi049 ceph-mon[112142]: pgmap v650: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:44 smithi049 ceph-mon[112142]: pgmap v651: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:44 smithi049 ceph-mon[112142]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-05-15T16:18:45.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.116+0000 7f8d046e4700 1 -- 172.21.15.49:0/3121045581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 msgr2=0x7f8cfc104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.116+0000 7f8d046e4700 1 --2- 172.21.15.49:0/3121045581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 0x7f8cfc104810 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009910 tx=0x7f8cec009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:45.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.116+0000 7f8d046e4700 1 -- 172.21.15.49:0/3121045581 shutdown_connections 2024-05-15T16:18:45.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.116+0000 7f8d046e4700 1 --2- 172.21.15.49:0/3121045581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 0x7f8cfc104810 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.116+0000 7f8d046e4700 1 --2- 172.21.15.49:0/3121045581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc0ffe10 0x7f8cfc1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.116+0000 7f8d046e4700 1 -- 172.21.15.49:0/3121045581 >> 172.21.15.49:0/3121045581 conn(0x7f8cfc0fb700 msgr2=0x7f8cfc0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:45.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.116+0000 7f8d046e4700 1 -- 172.21.15.49:0/3121045581 shutdown_connections 2024-05-15T16:18:45.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.116+0000 7f8d046e4700 1 -- 172.21.15.49:0/3121045581 wait complete. 2024-05-15T16:18:45.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.117+0000 7f8d046e4700 1 Processor -- start 2024-05-15T16:18:45.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.117+0000 7f8d046e4700 1 -- start start 2024-05-15T16:18:45.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.117+0000 7f8d046e4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc0ffe10 0x7f8cfc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:45.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.117+0000 7f8d046e4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 0x7f8cfc198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.117+0000 7f8d046e4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc198570 con 0x7f8cfc1007c0 2024-05-15T16:18:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.117+0000 7f8d046e4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1986b0 con 0x7f8cfc0ffe10 2024-05-15T16:18:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d01c7f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 0x7f8cfc198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d01c7f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 0x7f8cfc198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59430/0 (socket says 172.21.15.49:59430) 2024-05-15T16:18:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d01c7f700 1 -- 172.21.15.49:0/3757292825 learned_addr learned my addr 172.21.15.49:0/3757292825 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d02480700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc0ffe10 0x7f8cfc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d01c7f700 1 -- 172.21.15.49:0/3757292825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc0ffe10 msgr2=0x7f8cfc197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d01c7f700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc0ffe10 0x7f8cfc197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d01c7f700 1 -- 172.21.15.49:0/3757292825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec0095f0 con 0x7f8cfc1007c0 2024-05-15T16:18:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d01c7f700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 0x7f8cfc198030 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009fd0 tx=0x7f8cec00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8cf37fe700 1 -- 172.21.15.49:0/3757292825 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cec01d070 con 0x7f8cfc1007c0 2024-05-15T16:18:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8cf37fe700 1 -- 172.21.15.49:0/3757292825 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cec0052a0 con 0x7f8cfc1007c0 2024-05-15T16:18:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc198930 con 0x7f8cfc1007c0 2024-05-15T16:18:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8cf37fe700 1 -- 172.21.15.49:0/3757292825 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cec022470 con 0x7f8cfc1007c0 2024-05-15T16:18:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.118+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc1a0e90 con 0x7f8cfc1007c0 2024-05-15T16:18:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.119+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc108210 con 0x7f8cfc1007c0 2024-05-15T16:18:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.120+0000 7f8cf37fe700 1 -- 172.21.15.49:0/3757292825 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8cec00f6f0 con 0x7f8cfc1007c0 2024-05-15T16:18:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.120+0000 7f8cf37fe700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ce806e200 0x7f8ce80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.121+0000 7f8cf37fe700 1 -- 172.21.15.49:0/3757292825 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8cec08f450 con 0x7f8cfc1007c0 2024-05-15T16:18:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.121+0000 7f8d02480700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ce806e200 0x7f8ce80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.121+0000 7f8d02480700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ce806e200 0x7f8ce80706c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009c60 tx=0x7f8cf8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:45.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.125+0000 7f8cf37fe700 1 -- 172.21.15.49:0/3757292825 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8cec058730 con 0x7f8cfc1007c0 2024-05-15T16:18:45.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.353+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cfc069720 con 0x7f8ce806e200 2024-05-15T16:18:45.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.354+0000 7f8cf37fe700 1 -- 172.21.15.49:0/3757292825 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8cfc069720 con 0x7f8ce806e200 2024-05-15T16:18:45.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.356+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ce806e200 msgr2=0x7f8ce80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.356+0000 7f8d046e4700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ce806e200 0x7f8ce80706c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009c60 tx=0x7f8cf8009260 comp rx=0 tx=0).stop 2024-05-15T16:18:45.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.356+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 msgr2=0x7f8cfc198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.356+0000 7f8d046e4700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 0x7f8cfc198030 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009fd0 tx=0x7f8cec00b5a0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.357+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 shutdown_connections 2024-05-15T16:18:45.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.357+0000 7f8d046e4700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ce806e200 0x7f8ce80706c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.357+0000 7f8d046e4700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc0ffe10 0x7f8cfc197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.357+0000 7f8d046e4700 1 --2- 172.21.15.49:0/3757292825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cfc1007c0 0x7f8cfc198030 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.357+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 >> 172.21.15.49:0/3757292825 conn(0x7f8cfc0fb700 msgr2=0x7f8cfc0fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:45.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.357+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 shutdown_connections 2024-05-15T16:18:45.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.357+0000 7f8d046e4700 1 -- 172.21.15.49:0/3757292825 wait complete. 2024-05-15T16:18:45.369 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:18:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.467+0000 7ff25d661700 1 -- 172.21.15.49:0/371375787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258101890 msgr2=0x7ff2581058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.467+0000 7ff25d661700 1 --2- 172.21.15.49:0/371375787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258101890 0x7ff2581058e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009910 tx=0x7ff24c009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.467+0000 7ff25d661700 1 -- 172.21.15.49:0/371375787 shutdown_connections 2024-05-15T16:18:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.467+0000 7ff25d661700 1 --2- 172.21.15.49:0/371375787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258101890 0x7ff2581058e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.467+0000 7ff25d661700 1 --2- 172.21.15.49:0/371375787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff258100ee0 0x7ff2581012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.467+0000 7ff25d661700 1 -- 172.21.15.49:0/371375787 >> 172.21.15.49:0/371375787 conn(0x7ff2580fc790 msgr2=0x7ff2580febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.467+0000 7ff25d661700 1 -- 172.21.15.49:0/371375787 shutdown_connections 2024-05-15T16:18:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.467+0000 7ff25d661700 1 -- 172.21.15.49:0/371375787 wait complete. 2024-05-15T16:18:45.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.468+0000 7ff25d661700 1 Processor -- start 2024-05-15T16:18:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.468+0000 7ff25d661700 1 -- start start 2024-05-15T16:18:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff25d661700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258100ee0 0x7ff25819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff25d661700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff258101890 0x7ff25819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff25d661700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25819d440 con 0x7ff258100ee0 2024-05-15T16:18:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff25d661700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff258196a20 con 0x7ff258101890 2024-05-15T16:18:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff256ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258100ee0 0x7ff25819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff256ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258100ee0 0x7ff25819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59452/0 (socket says 172.21.15.49:59452) 2024-05-15T16:18:45.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff256ffd700 1 -- 172.21.15.49:0/1316610474 learned_addr learned my addr 172.21.15.49:0/1316610474 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff2567fc700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff258101890 0x7ff25819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff256ffd700 1 -- 172.21.15.49:0/1316610474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff258101890 msgr2=0x7ff25819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff256ffd700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff258101890 0x7ff25819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff256ffd700 1 -- 172.21.15.49:0/1316610474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c0095f0 con 0x7ff258100ee0 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff256ffd700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258100ee0 0x7ff25819c8e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff24800d8d0 tx=0x7ff24800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff23ffff700 1 -- 172.21.15.49:0/1316610474 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2480098a0 con 0x7ff258100ee0 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff258196d00 con 0x7ff258100ee0 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff23ffff700 1 -- 172.21.15.49:0/1316610474 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff248010460 con 0x7ff258100ee0 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.469+0000 7ff23ffff700 1 -- 172.21.15.49:0/1316610474 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff24800f5f0 con 0x7ff258100ee0 2024-05-15T16:18:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.470+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff258197250 con 0x7ff258100ee0 2024-05-15T16:18:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.471+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2581092e0 con 0x7ff258100ee0 2024-05-15T16:18:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.471+0000 7ff23ffff700 1 -- 172.21.15.49:0/1316610474 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff248009a00 con 0x7ff258100ee0 2024-05-15T16:18:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.471+0000 7ff23ffff700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff24006e200 0x7ff2400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.471+0000 7ff23ffff700 1 -- 172.21.15.49:0/1316610474 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff24808e0a0 con 0x7ff258100ee0 2024-05-15T16:18:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.472+0000 7ff2567fc700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff24006e200 0x7ff2400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.472+0000 7ff2567fc700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff24006e200 0x7ff2400706c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff258197be0 tx=0x7ff24c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.476+0000 7ff23ffff700 1 -- 172.21.15.49:0/1316610474 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff24805aeb0 con 0x7ff258100ee0 2024-05-15T16:18:45.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:45 smithi144 ceph-mon[86635]: Upgrade: Updating mds.cephfs.smithi144.kziuke 2024-05-15T16:18:45.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:45 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:45.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:45 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.kziuke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T16:18:45.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:45 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:18:45.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:45 smithi144 ceph-mon[86635]: Deploying daemon mds.cephfs.smithi144.kziuke on smithi144 2024-05-15T16:18:45.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:45 smithi144 ceph-mon[86635]: pgmap v652: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:45.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:45 smithi049 ceph-mon[112142]: Upgrade: Updating mds.cephfs.smithi144.kziuke 2024-05-15T16:18:45.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:45 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:45.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:45 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.kziuke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T16:18:45.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:45 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:18:45.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:45 smithi049 ceph-mon[112142]: Deploying daemon mds.cephfs.smithi144.kziuke on smithi144 2024-05-15T16:18:45.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:45 smithi049 ceph-mon[112142]: pgmap v652: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:45.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.704+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff258197f50 con 0x7ff24006e200 2024-05-15T16:18:45.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.705+0000 7ff23ffff700 1 -- 172.21.15.49:0/1316610474 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff258197f50 con 0x7ff24006e200 2024-05-15T16:18:45.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff24006e200 msgr2=0x7ff2400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff24006e200 0x7ff2400706c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff258197be0 tx=0x7ff24c01a040 comp rx=0 tx=0).stop 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258100ee0 msgr2=0x7ff25819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258100ee0 0x7ff25819c8e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff24800d8d0 tx=0x7ff24800dbe0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 shutdown_connections 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff24006e200 0x7ff2400706c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff258100ee0 0x7ff25819c8e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 --2- 172.21.15.49:0/1316610474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff258101890 0x7ff25819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 >> 172.21.15.49:0/1316610474 conn(0x7ff2580fc790 msgr2=0x7ff2580fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 shutdown_connections 2024-05-15T16:18:45.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.707+0000 7ff25d661700 1 -- 172.21.15.49:0/1316610474 wait complete. 2024-05-15T16:18:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.816+0000 7f73f9fb3700 1 -- 172.21.15.49:0/2660684273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4100770 msgr2=0x7f73f41047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.816+0000 7f73f9fb3700 1 --2- 172.21.15.49:0/2660684273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4100770 0x7f73f41047c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009910 tx=0x7f73dc009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.816+0000 7f73f9fb3700 1 -- 172.21.15.49:0/2660684273 shutdown_connections 2024-05-15T16:18:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.816+0000 7f73f9fb3700 1 --2- 172.21.15.49:0/2660684273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4100770 0x7f73f41047c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.816+0000 7f73f9fb3700 1 --2- 172.21.15.49:0/2660684273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f40ffdc0 0x7f73f41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.816+0000 7f73f9fb3700 1 -- 172.21.15.49:0/2660684273 >> 172.21.15.49:0/2660684273 conn(0x7f73f4076800 msgr2=0x7f73f40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.816+0000 7f73f9fb3700 1 -- 172.21.15.49:0/2660684273 shutdown_connections 2024-05-15T16:18:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.816+0000 7f73f9fb3700 1 -- 172.21.15.49:0/2660684273 wait complete. 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.817+0000 7f73f9fb3700 1 Processor -- start 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.817+0000 7f73f9fb3700 1 -- start start 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.817+0000 7f73f9fb3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f40ffdc0 0x7f73f40fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.817+0000 7f73f9fb3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f4100770 0x7f73f40fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.817+0000 7f73f9fb3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40fc760 con 0x7f73f40ffdc0 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.817+0000 7f73f9fb3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40fc8d0 con 0x7f73f4100770 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.817+0000 7f73f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f40ffdc0 0x7f73f40fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.817+0000 7f73f37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f40ffdc0 0x7f73f40fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59462/0 (socket says 172.21.15.49:59462) 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.817+0000 7f73f37fe700 1 -- 172.21.15.49:0/150722208 learned_addr learned my addr 172.21.15.49:0/150722208 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73eaffd700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f4100770 0x7f73f40fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73f37fe700 1 -- 172.21.15.49:0/150722208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f4100770 msgr2=0x7f73f40fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73f37fe700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f4100770 0x7f73f40fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73f37fe700 1 -- 172.21.15.49:0/150722208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc0095f0 con 0x7f73f40ffdc0 2024-05-15T16:18:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73f37fe700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f40ffdc0 0x7f73f40fbce0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f73e400e920 tx=0x7f73e400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73f17fa700 1 -- 172.21.15.49:0/150722208 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73e400c9d0 con 0x7f73f40ffdc0 2024-05-15T16:18:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73f17fa700 1 -- 172.21.15.49:0/150722208 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73e4010430 con 0x7f73f40ffdc0 2024-05-15T16:18:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f41a2d20 con 0x7f73f40ffdc0 2024-05-15T16:18:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73f17fa700 1 -- 172.21.15.49:0/150722208 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73e4018630 con 0x7f73f40ffdc0 2024-05-15T16:18:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.818+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f41a30d0 con 0x7f73f40ffdc0 2024-05-15T16:18:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.819+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f40fcb50 con 0x7f73f40ffdc0 2024-05-15T16:18:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.820+0000 7f73f17fa700 1 -- 172.21.15.49:0/150722208 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f73e40105a0 con 0x7f73f40ffdc0 2024-05-15T16:18:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.820+0000 7f73f17fa700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f73e006e210 0x7f73e00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.820+0000 7f73f17fa700 1 -- 172.21.15.49:0/150722208 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f73e4014070 con 0x7f73f40ffdc0 2024-05-15T16:18:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.820+0000 7f73eaffd700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f73e006e210 0x7f73e00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.821+0000 7f73eaffd700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f73e006e210 0x7f73e00706d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f73dc005b40 tx=0x7f73dc005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:45.825+0000 7f73f17fa700 1 -- 172.21.15.49:0/150722208 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f73e4053850 con 0x7f73f40ffdc0 2024-05-15T16:18:46.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.041+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f73f410cd60 con 0x7f73e006e210 2024-05-15T16:18:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.052+0000 7f73f17fa700 1 -- 172.21.15.49:0/150722208 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f73f410cd60 con 0x7f73e006e210 2024-05-15T16:18:46.054 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:18:46.054 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (22m) 15m ago 33m 24.5M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:18:46.054 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (33m) 15m ago 33m 9369k - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:18:46.054 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (32m) 15m ago 32m 9110k - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:18:46.054 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (19m) 15m ago 33m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:18:46.054 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (19m) 15m ago 32m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:18:46.054 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (21m) 15m ago 33m 78.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (15m) 15m ago 28m 14.8M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (15m) 15m ago 27m 11.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (15m) 15m ago 27m 12.7M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (27m) 15m ago 27m 11.9M - 18.2.2-1343-g2965081f a04a73ffe079 b5c808a36e19 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (24m) 15m ago 35m 512M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (23m) 15m ago 32m 454M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (20m) 15m ago 35m 55.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (20m) 15m ago 32m 45.9M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (22m) 15m ago 33m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (22m) 15m ago 32m 15.6M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (18m) 15m ago 31m 87.6M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (18m) 15m ago 31m 87.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (17m) 15m ago 30m 84.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:18:46.055 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (17m) 15m ago 30m 99.7M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:18:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (16m) 15m ago 29m 85.6M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:18:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (16m) 15m ago 29m 80.3M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:18:46.056 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (22m) 15m ago 33m 77.2M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:18:46.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f73e006e210 msgr2=0x7f73e00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f73e006e210 0x7f73e00706d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f73dc005b40 tx=0x7f73dc005a90 comp rx=0 tx=0).stop 2024-05-15T16:18:46.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f40ffdc0 msgr2=0x7f73f40fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f40ffdc0 0x7f73f40fbce0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f73e400e920 tx=0x7f73e400ec30 comp rx=0 tx=0).stop 2024-05-15T16:18:46.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 shutdown_connections 2024-05-15T16:18:46.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f73e006e210 0x7f73e00706d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f40ffdc0 0x7f73f40fbce0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 --2- 172.21.15.49:0/150722208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f4100770 0x7f73f40fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 >> 172.21.15.49:0/150722208 conn(0x7f73f4076800 msgr2=0x7f73f40688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:46.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 shutdown_connections 2024-05-15T16:18:46.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.055+0000 7f73f9fb3700 1 -- 172.21.15.49:0/150722208 wait complete. 2024-05-15T16:18:46.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.162+0000 7f4d480a3700 1 -- 172.21.15.49:0/4223902607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d400ffdc0 msgr2=0x7f4d401001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.162+0000 7f4d480a3700 1 --2- 172.21.15.49:0/4223902607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d400ffdc0 0x7f4d401001a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009910 tx=0x7f4d3c009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:46.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.162+0000 7f4d480a3700 1 -- 172.21.15.49:0/4223902607 shutdown_connections 2024-05-15T16:18:46.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.162+0000 7f4d480a3700 1 --2- 172.21.15.49:0/4223902607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d40100770 0x7f4d401047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.162+0000 7f4d480a3700 1 --2- 172.21.15.49:0/4223902607 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d400ffdc0 0x7f4d401001a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.163+0000 7f4d480a3700 1 -- 172.21.15.49:0/4223902607 >> 172.21.15.49:0/4223902607 conn(0x7f4d40076800 msgr2=0x7f4d400fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:46.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.163+0000 7f4d480a3700 1 -- 172.21.15.49:0/4223902607 shutdown_connections 2024-05-15T16:18:46.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.163+0000 7f4d480a3700 1 -- 172.21.15.49:0/4223902607 wait complete. 2024-05-15T16:18:46.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.163+0000 7f4d480a3700 1 Processor -- start 2024-05-15T16:18:46.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.163+0000 7f4d480a3700 1 -- start start 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.163+0000 7f4d480a3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d400ffdc0 0x7f4d40198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.163+0000 7f4d480a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40100770 0x7f4d40199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.163+0000 7f4d480a3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40199980 con 0x7f4d40100770 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.163+0000 7f4d480a3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40199ac0 con 0x7f4d400ffdc0 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d4563e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40100770 0x7f4d40199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d4563e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40100770 0x7f4d40199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59492/0 (socket says 172.21.15.49:59492) 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d4563e700 1 -- 172.21.15.49:0/1764824748 learned_addr learned my addr 172.21.15.49:0/1764824748 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d45e3f700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d400ffdc0 0x7f4d40198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d4563e700 1 -- 172.21.15.49:0/1764824748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d400ffdc0 msgr2=0x7f4d40198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d4563e700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d400ffdc0 0x7f4d40198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d4563e700 1 -- 172.21.15.49:0/1764824748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0095f0 con 0x7f4d40100770 2024-05-15T16:18:46.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d4563e700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40100770 0x7f4d40199330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000c960 tx=0x7f4d3000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:46.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d36ffd700 1 -- 172.21.15.49:0/1764824748 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d30007a70 con 0x7f4d40100770 2024-05-15T16:18:46.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d36ffd700 1 -- 172.21.15.49:0/1764824748 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d30007bd0 con 0x7f4d40100770 2024-05-15T16:18:46.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4019d850 con 0x7f4d40100770 2024-05-15T16:18:46.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d36ffd700 1 -- 172.21.15.49:0/1764824748 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d30018630 con 0x7f4d40100770 2024-05-15T16:18:46.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.164+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4019dda0 con 0x7f4d40100770 2024-05-15T16:18:46.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.165+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d401012a0 con 0x7f4d40100770 2024-05-15T16:18:46.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.166+0000 7f4d36ffd700 1 -- 172.21.15.49:0/1764824748 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4d3000f450 con 0x7f4d40100770 2024-05-15T16:18:46.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.167+0000 7f4d36ffd700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d2c06e200 0x7f4d2c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:46.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.167+0000 7f4d36ffd700 1 -- 172.21.15.49:0/1764824748 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4d3008e100 con 0x7f4d40100770 2024-05-15T16:18:46.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.167+0000 7f4d45e3f700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d2c06e200 0x7f4d2c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:46.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.167+0000 7f4d45e3f700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d2c06e200 0x7f4d2c0706c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c006010 tx=0x7f4d3c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:46.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.171+0000 7f4d36ffd700 1 -- 172.21.15.49:0/1764824748 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4d300573e0 con 0x7f4d40100770 2024-05-15T16:18:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.465+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4d4004ec60 con 0x7f4d40100770 2024-05-15T16:18:46.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.466+0000 7f4d36ffd700 1 -- 172.21.15.49:0/1764824748 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f4d3005aaa0 con 0x7f4d40100770 2024-05-15T16:18:46.467 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:18:46.467 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:18:46.467 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:18:46.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:18:46.467 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 3 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 18.2.2-1343-g2965081f (2965081fb1a343a09e00ceeede998525f5c6cb5e) reef (stable)": 1, 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 13 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:18:46.468 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:18:46.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d2c06e200 msgr2=0x7f4d2c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d2c06e200 0x7f4d2c0706c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c006010 tx=0x7f4d3c01a040 comp rx=0 tx=0).stop 2024-05-15T16:18:46.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40100770 msgr2=0x7f4d40199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40100770 0x7f4d40199330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000c960 tx=0x7f4d3000cc70 comp rx=0 tx=0).stop 2024-05-15T16:18:46.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 shutdown_connections 2024-05-15T16:18:46.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d2c06e200 0x7f4d2c0706c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d400ffdc0 0x7f4d40198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 --2- 172.21.15.49:0/1764824748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d40100770 0x7f4d40199330 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 >> 172.21.15.49:0/1764824748 conn(0x7f4d40076800 msgr2=0x7f4d40068a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:46.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 shutdown_connections 2024-05-15T16:18:46.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.468+0000 7f4d480a3700 1 -- 172.21.15.49:0/1764824748 wait complete. 2024-05-15T16:18:46.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.577+0000 7fbcc361d700 1 -- 172.21.15.49:0/1047559004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc100390 msgr2=0x7fbcbc10d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.577+0000 7fbcc361d700 1 --2- 172.21.15.49:0/1047559004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc100390 0x7fbcbc10d050 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009910 tx=0x7fbcac009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:46.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.577+0000 7fbcc361d700 1 -- 172.21.15.49:0/1047559004 shutdown_connections 2024-05-15T16:18:46.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.577+0000 7fbcc361d700 1 --2- 172.21.15.49:0/1047559004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc100390 0x7fbcbc10d050 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.577+0000 7fbcc361d700 1 --2- 172.21.15.49:0/1047559004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcbc0ff9e0 0x7fbcbc0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.577+0000 7fbcc361d700 1 -- 172.21.15.49:0/1047559004 >> 172.21.15.49:0/1047559004 conn(0x7fbcbc0fb6a0 msgr2=0x7fbcbc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:46.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.577+0000 7fbcc361d700 1 -- 172.21.15.49:0/1047559004 shutdown_connections 2024-05-15T16:18:46.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.577+0000 7fbcc361d700 1 -- 172.21.15.49:0/1047559004 wait complete. 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc361d700 1 Processor -- start 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc361d700 1 -- start start 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc361d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc0ff9e0 0x7fbcbc198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc361d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcbc100390 0x7fbcbc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc361d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc199a30 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc361d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc19d700 con 0x7fbcbc100390 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc13b9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc0ff9e0 0x7fbcbc198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc13b9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc0ff9e0 0x7fbcbc198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59500/0 (socket says 172.21.15.49:59500) 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc13b9700 1 -- 172.21.15.49:0/2533332526 learned_addr learned my addr 172.21.15.49:0/2533332526 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc0bb8700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcbc100390 0x7fbcbc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc13b9700 1 -- 172.21.15.49:0/2533332526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcbc100390 msgr2=0x7fbcbc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc13b9700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcbc100390 0x7fbcbc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc13b9700 1 -- 172.21.15.49:0/2533332526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0095f0 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.578+0000 7fbcc13b9700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc0ff9e0 0x7fbcbc198e10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800e920 tx=0x7fbcb800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.579+0000 7fbcb27fc700 1 -- 172.21.15.49:0/2533332526 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcb800c9d0 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.579+0000 7fbcb27fc700 1 -- 172.21.15.49:0/2533332526 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcb8010430 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.579+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc19d9e0 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.579+0000 7fbcb27fc700 1 -- 172.21.15.49:0/2533332526 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcb8018630 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.579+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc19df30 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.580+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc10aed0 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.580+0000 7fbcb27fc700 1 -- 172.21.15.49:0/2533332526 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbcb80105a0 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.581+0000 7fbcb27fc700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbca806e200 0x7fbca80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:46.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.581+0000 7fbcb27fc700 1 -- 172.21.15.49:0/2533332526 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbcb8014070 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.581+0000 7fbcc0bb8700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbca806e200 0x7fbca80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:46.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.581+0000 7fbcc0bb8700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbca806e200 0x7fbca80706c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc19a370 tx=0x7fbcac009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:46.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.585+0000 7fbcb27fc700 1 -- 172.21.15.49:0/2533332526 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbcb8053850 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:46 smithi144 ceph-mon[86635]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:46.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:46 smithi144 ceph-mon[86635]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:46.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:46 smithi144 ceph-mon[86635]: from='client.35222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:46.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:46 smithi049 ceph-mon[112142]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:46.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:46 smithi049 ceph-mon[112142]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:46.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:46 smithi049 ceph-mon[112142]: from='client.35222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:46.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.837+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbcbc19a1c0 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.838+0000 7fbcb27fc700 1 -- 172.21.15.49:0/2533332526 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1871 (secure 0 0 0) 0x7fbcb8057750 con 0x7fbcbc0ff9e0 2024-05-15T16:18:46.839 INFO:teuthology.orchestra.run.smithi049.stdout:e26 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:18:46.840 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:18:46.841 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/334969158,v1:172.21.15.144:6827/334969158] compat {c=[1],r=[1],i=[7ff]}] 2024-05-15T16:18:46.842 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:18:46.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbca806e200 msgr2=0x7fbca80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbca806e200 0x7fbca80706c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc19a370 tx=0x7fbcac009f70 comp rx=0 tx=0).stop 2024-05-15T16:18:46.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc0ff9e0 msgr2=0x7fbcbc198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc0ff9e0 0x7fbcbc198e10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800e920 tx=0x7fbcb800ec30 comp rx=0 tx=0).stop 2024-05-15T16:18:46.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 shutdown_connections 2024-05-15T16:18:46.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbca806e200 0x7fbca80706c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcbc0ff9e0 0x7fbcbc198e10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 --2- 172.21.15.49:0/2533332526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcbc100390 0x7fbcbc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 >> 172.21.15.49:0/2533332526 conn(0x7fbcbc0fb6a0 msgr2=0x7fbcbc108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:46.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 shutdown_connections 2024-05-15T16:18:46.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.840+0000 7fbcc361d700 1 -- 172.21.15.49:0/2533332526 wait complete. 2024-05-15T16:18:46.844 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:18:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.949+0000 7f2884045700 1 -- 172.21.15.49:0/3642134297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 msgr2=0x7f287c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.949+0000 7f2884045700 1 --2- 172.21.15.49:0/3642134297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 0x7f287c1047f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2870009910 tx=0x7f2870009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.950+0000 7f2884045700 1 -- 172.21.15.49:0/3642134297 shutdown_connections 2024-05-15T16:18:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.950+0000 7f2884045700 1 --2- 172.21.15.49:0/3642134297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 0x7f287c1047f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.950+0000 7f2884045700 1 --2- 172.21.15.49:0/3642134297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f287c0ffdf0 0x7f287c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.950+0000 7f2884045700 1 -- 172.21.15.49:0/3642134297 >> 172.21.15.49:0/3642134297 conn(0x7f287c0fb6c0 msgr2=0x7f287c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.950+0000 7f2884045700 1 -- 172.21.15.49:0/3642134297 shutdown_connections 2024-05-15T16:18:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.950+0000 7f2884045700 1 -- 172.21.15.49:0/3642134297 wait complete. 2024-05-15T16:18:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.950+0000 7f2884045700 1 Processor -- start 2024-05-15T16:18:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.950+0000 7f2884045700 1 -- start start 2024-05-15T16:18:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f2884045700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f287c0ffdf0 0x7f287c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f2884045700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 0x7f287c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f2884045700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c19d450 con 0x7f287c1007a0 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f2884045700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c196a80 con 0x7f287c0ffdf0 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f28815e0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 0x7f287c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f28815e0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 0x7f287c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59530/0 (socket says 172.21.15.49:59530) 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f28815e0700 1 -- 172.21.15.49:0/3080919947 learned_addr learned my addr 172.21.15.49:0/3080919947 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f2881de1700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f287c0ffdf0 0x7f287c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f28815e0700 1 -- 172.21.15.49:0/3080919947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f287c0ffdf0 msgr2=0x7f287c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f28815e0700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f287c0ffdf0 0x7f287c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f28815e0700 1 -- 172.21.15.49:0/3080919947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28700095f0 con 0x7f287c1007a0 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f28815e0700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 0x7f287c19ce30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2870009fd0 tx=0x7f287000b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f286effd700 1 -- 172.21.15.49:0/3080919947 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f287001d070 con 0x7f287c1007a0 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c196d00 con 0x7f287c1007a0 2024-05-15T16:18:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f286effd700 1 -- 172.21.15.49:0/3080919947 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28700052a0 con 0x7f287c1007a0 2024-05-15T16:18:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.951+0000 7f286effd700 1 -- 172.21.15.49:0/3080919947 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2870022470 con 0x7f287c1007a0 2024-05-15T16:18:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.952+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c1971f0 con 0x7f287c1007a0 2024-05-15T16:18:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.953+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c1081f0 con 0x7f287c1007a0 2024-05-15T16:18:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.953+0000 7f286effd700 1 -- 172.21.15.49:0/3080919947 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f287000f6f0 con 0x7f287c1007a0 2024-05-15T16:18:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.954+0000 7f286effd700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f286806e200 0x7f28680706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.954+0000 7f286effd700 1 -- 172.21.15.49:0/3080919947 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f287008f450 con 0x7f287c1007a0 2024-05-15T16:18:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.954+0000 7f2881de1700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f286806e200 0x7f28680706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.954+0000 7f2881de1700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f286806e200 0x7f28680706c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2878009c60 tx=0x7f2878009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:46.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:46.959+0000 7f286effd700 1 -- 172.21.15.49:0/3080919947 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f287005c260 con 0x7f287c1007a0 2024-05-15T16:18:47.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.190+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f287c197ef0 con 0x7f286806e200 2024-05-15T16:18:47.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.191+0000 7f286effd700 1 -- 172.21.15.49:0/3080919947 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f287c197ef0 con 0x7f286806e200 2024-05-15T16:18:47.192 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "15/23 daemons upgraded", 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:18:47.193 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:18:47.194 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f286806e200 msgr2=0x7f28680706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f286806e200 0x7f28680706c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2878009c60 tx=0x7f2878009260 comp rx=0 tx=0).stop 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 msgr2=0x7f287c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 0x7f287c19ce30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2870009fd0 tx=0x7f287000b5a0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 shutdown_connections 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f286806e200 0x7f28680706c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f287c0ffdf0 0x7f287c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 --2- 172.21.15.49:0/3080919947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c1007a0 0x7f287c19ce30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 >> 172.21.15.49:0/3080919947 conn(0x7f287c0fb6c0 msgr2=0x7f287c0fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 shutdown_connections 2024-05-15T16:18:47.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.193+0000 7f2884045700 1 -- 172.21.15.49:0/3080919947 wait complete. 2024-05-15T16:18:47.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.302+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/3257257039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 msgr2=0x7f4cac1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.302+0000 7f4cb3d0d700 1 --2- 172.21.15.49:0/3257257039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 0x7f4cac1047c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009910 tx=0x7f4c9c009c20 comp rx=0 tx=0).stop 2024-05-15T16:18:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.302+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/3257257039 shutdown_connections 2024-05-15T16:18:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.302+0000 7f4cb3d0d700 1 --2- 172.21.15.49:0/3257257039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 0x7f4cac1047c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.302+0000 7f4cb3d0d700 1 --2- 172.21.15.49:0/3257257039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cac0ffdc0 0x7f4cac1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.302+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/3257257039 >> 172.21.15.49:0/3257257039 conn(0x7f4cac076800 msgr2=0x7f4cac0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.302+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/3257257039 shutdown_connections 2024-05-15T16:18:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.302+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/3257257039 wait complete. 2024-05-15T16:18:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb3d0d700 1 Processor -- start 2024-05-15T16:18:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb3d0d700 1 -- start start 2024-05-15T16:18:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb3d0d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cac0ffdc0 0x7f4cac1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb3d0d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 0x7f4cac197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb3d0d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac198420 con 0x7f4cac100770 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb3d0d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac198560 con 0x7f4cac0ffdc0 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb12a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 0x7f4cac197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb12a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 0x7f4cac197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59552/0 (socket says 172.21.15.49:59552) 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb12a8700 1 -- 172.21.15.49:0/721446619 learned_addr learned my addr 172.21.15.49:0/721446619 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb1aa9700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cac0ffdc0 0x7f4cac1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb12a8700 1 -- 172.21.15.49:0/721446619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cac0ffdc0 msgr2=0x7f4cac1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb12a8700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cac0ffdc0 0x7f4cac1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb12a8700 1 -- 172.21.15.49:0/721446619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0095f0 con 0x7f4cac100770 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.303+0000 7f4cb1aa9700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cac0ffdc0 0x7f4cac1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.304+0000 7f4cb12a8700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 0x7f4cac197ee0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0092e0 tx=0x7f4c9c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.304+0000 7f4ca2ffd700 1 -- 172.21.15.49:0/721446619 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c01d070 con 0x7f4cac100770 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.304+0000 7f4ca2ffd700 1 -- 172.21.15.49:0/721446619 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c9c00f460 con 0x7f4cac100770 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.304+0000 7f4ca2ffd700 1 -- 172.21.15.49:0/721446619 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c017630 con 0x7f4cac100770 2024-05-15T16:18:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.304+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cac1987e0 con 0x7f4cac100770 2024-05-15T16:18:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.304+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac1a0d40 con 0x7f4cac100770 2024-05-15T16:18:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.305+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cac1081c0 con 0x7f4cac100770 2024-05-15T16:18:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.306+0000 7f4ca2ffd700 1 -- 172.21.15.49:0/721446619 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4c9c00f5d0 con 0x7f4cac100770 2024-05-15T16:18:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.306+0000 7f4ca2ffd700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c9806e200 0x7f4c980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:18:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.306+0000 7f4ca2ffd700 1 -- 172.21.15.49:0/721446619 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4c9c08f550 con 0x7f4cac100770 2024-05-15T16:18:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.306+0000 7f4cb1aa9700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c9806e200 0x7f4c980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:18:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.306+0000 7f4cb1aa9700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c9806e200 0x7f4c980706c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8005fd0 tx=0x7f4ca8005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:18:47.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.311+0000 7f4ca2ffd700 1 -- 172.21.15.49:0/721446619 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4c9c0588b0 con 0x7f4cac100770 2024-05-15T16:18:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.605+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4cac1012a0 con 0x7f4cac100770 2024-05-15T16:18:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.606+0000 7f4ca2ffd700 1 -- 172.21.15.49:0/721446619 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f4c9c005740 con 0x7f4cac100770 2024-05-15T16:18:47.607 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:18:47.607 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:18:47.607 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:18:47.607 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:18:47.607 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:18:47.608 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:18:47.609 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:18:47.609 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:18:47.609 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:18:47.609 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:18:47.609 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:18:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c9806e200 msgr2=0x7f4c980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c9806e200 0x7f4c980706c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8005fd0 tx=0x7f4ca8005e20 comp rx=0 tx=0).stop 2024-05-15T16:18:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 msgr2=0x7f4cac197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:18:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 0x7f4cac197ee0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0092e0 tx=0x7f4c9c00be20 comp rx=0 tx=0).stop 2024-05-15T16:18:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 shutdown_connections 2024-05-15T16:18:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c9806e200 0x7f4c980706c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cac0ffdc0 0x7f4cac1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 --2- 172.21.15.49:0/721446619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cac100770 0x7f4cac197ee0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:18:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 >> 172.21.15.49:0/721446619 conn(0x7f4cac076800 msgr2=0x7f4cac103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:18:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 shutdown_connections 2024-05-15T16:18:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:18:47.608+0000 7f4cb3d0d700 1 -- 172.21.15.49:0/721446619 wait complete. 2024-05-15T16:18:47.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:47 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1764824748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:47.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:47 smithi049 ceph-mon[112142]: pgmap v653: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:47.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:47 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2533332526' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:18:47.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:47 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1764824748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:18:47.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:47 smithi144 ceph-mon[86635]: pgmap v653: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:47.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:47 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2533332526' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:18:48.633 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:48 smithi144 ceph-mon[86635]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:48.633 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:48 smithi144 ceph-mon[86635]: osdmap e87: 6 total, 6 up, 6 in 2024-05-15T16:18:48.633 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:48 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-15T16:18:48.633 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:48 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/721446619' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:18:48.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:48 smithi049 ceph-mon[112142]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:18:48.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:48 smithi049 ceph-mon[112142]: osdmap e87: 6 total, 6 up, 6 in 2024-05-15T16:18:48.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:48 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-15T16:18:48.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:48 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/721446619' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:18:49.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:49 smithi144 ceph-mon[86635]: pgmap v655: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:49.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:49 smithi049 ceph-mon[112142]: pgmap v655: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:18:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:18:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] up:boot 2024-05-15T16:18:50.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:18:50.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T16:18:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:18:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:18:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] up:boot 2024-05-15T16:18:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-15T16:18:50.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T16:18:51.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:51 smithi144 ceph-mon[86635]: pgmap v656: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:51.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:51 smithi049 ceph-mon[112142]: pgmap v656: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:54 smithi144 ceph-mon[86635]: pgmap v657: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:54 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:54 smithi049 ceph-mon[112142]: pgmap v657: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:54 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:55.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:55.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:55.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:55 smithi144 ceph-mon[86635]: pgmap v658: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:55.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:55.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:55.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:55 smithi049 ceph-mon[112142]: pgmap v658: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:56.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:56.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:56 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:56.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:56.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:56 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:18:57.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:57 smithi144 ceph-mon[86635]: pgmap v659: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:57.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:57 smithi049 ceph-mon[112142]: pgmap v659: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:18:59 smithi144 ceph-mon[86635]: pgmap v660: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:18:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:18:59 smithi049 ceph-mon[112142]: pgmap v660: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:01.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:01 smithi144 ceph-mon[86635]: pgmap v661: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:01.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:01 smithi049 ceph-mon[112142]: pgmap v661: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:03.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:03 smithi144 ceph-mon[86635]: pgmap v662: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:03.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:03 smithi049 ceph-mon[112142]: pgmap v662: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:05.947 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:05 smithi144 ceph-mon[86635]: pgmap v663: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:19:05.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:05 smithi049 ceph-mon[112142]: pgmap v663: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:05.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:19:07.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:07 smithi144 ceph-mon[86635]: pgmap v664: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:07.948 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:07 smithi049 ceph-mon[112142]: pgmap v664: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:09.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:09 smithi144 ceph-mon[86635]: pgmap v665: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:09.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:09 smithi049 ceph-mon[112142]: pgmap v665: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:11.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:11 smithi144 ceph-mon[86635]: pgmap v666: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:11 smithi049 ceph-mon[112142]: pgmap v666: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:13.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:13 smithi144 ceph-mon[86635]: pgmap v667: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:13.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:13 smithi049 ceph-mon[112142]: pgmap v667: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:15.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:15 smithi144 ceph-mon[86635]: pgmap v668: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:15.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:15 smithi049 ceph-mon[112142]: pgmap v668: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.732+0000 7f27afca4700 1 -- 172.21.15.49:0/2540537411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 msgr2=0x7f27a81001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.732+0000 7f27afca4700 1 --2- 172.21.15.49:0/2540537411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 0x7f27a81001f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009910 tx=0x7f27a4009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.732+0000 7f27afca4700 1 -- 172.21.15.49:0/2540537411 shutdown_connections 2024-05-15T16:19:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.732+0000 7f27afca4700 1 --2- 172.21.15.49:0/2540537411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a81007c0 0x7f27a8104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.732+0000 7f27afca4700 1 --2- 172.21.15.49:0/2540537411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 0x7f27a81001f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.732+0000 7f27afca4700 1 -- 172.21.15.49:0/2540537411 >> 172.21.15.49:0/2540537411 conn(0x7f27a80fb700 msgr2=0x7f27a80fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.732+0000 7f27afca4700 1 -- 172.21.15.49:0/2540537411 shutdown_connections 2024-05-15T16:19:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.732+0000 7f27afca4700 1 -- 172.21.15.49:0/2540537411 wait complete. 2024-05-15T16:19:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27afca4700 1 Processor -- start 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27afca4700 1 -- start start 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27afca4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 0x7f27a8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27afca4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a81007c0 0x7f27a8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27afca4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a8198570 con 0x7f27a80ffe10 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27afca4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a81986b0 con 0x7f27a81007c0 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27ada40700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 0x7f27a8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27ada40700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 0x7f27a8197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52552/0 (socket says 172.21.15.49:52552) 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27ada40700 1 -- 172.21.15.49:0/2974157416 learned_addr learned my addr 172.21.15.49:0/2974157416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.733+0000 7f27ad23f700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a81007c0 0x7f27a8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.734+0000 7f27ada40700 1 -- 172.21.15.49:0/2974157416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a81007c0 msgr2=0x7f27a8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.734+0000 7f27ada40700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a81007c0 0x7f27a8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.734+0000 7f27ada40700 1 -- 172.21.15.49:0/2974157416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a40095f0 con 0x7f27a80ffe10 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.734+0000 7f27ada40700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 0x7f27a8197a60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f27a4006010 tx=0x7f27a400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:17.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.734+0000 7f279effd700 1 -- 172.21.15.49:0/2974157416 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27a401d070 con 0x7f27a80ffe10 2024-05-15T16:19:17.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.734+0000 7f279effd700 1 -- 172.21.15.49:0/2974157416 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27a400f460 con 0x7f27a80ffe10 2024-05-15T16:19:17.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.734+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a81a0ad0 con 0x7f27a80ffe10 2024-05-15T16:19:17.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.734+0000 7f279effd700 1 -- 172.21.15.49:0/2974157416 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27a4017630 con 0x7f27a80ffe10 2024-05-15T16:19:17.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.734+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a81a0e30 con 0x7f27a80ffe10 2024-05-15T16:19:17.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.735+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a81081b0 con 0x7f27a80ffe10 2024-05-15T16:19:17.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.736+0000 7f279effd700 1 -- 172.21.15.49:0/2974157416 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f27a400f5d0 con 0x7f27a80ffe10 2024-05-15T16:19:17.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.736+0000 7f279effd700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f279406e210 0x7f27940706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:17.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.736+0000 7f279effd700 1 -- 172.21.15.49:0/2974157416 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f27a408f5f0 con 0x7f27a80ffe10 2024-05-15T16:19:17.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.736+0000 7f27ad23f700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f279406e210 0x7f27940706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:17.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.737+0000 7f27ad23f700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f279406e210 0x7f27940706d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27a80699c0 tx=0x7f279800a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:17.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.742+0000 7f279effd700 1 -- 172.21.15.49:0/2974157416 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f27a4058990 con 0x7f27a80ffe10 2024-05-15T16:19:17.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:17 smithi144 ceph-mon[86635]: pgmap v669: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:17.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:17 smithi049 ceph-mon[112142]: pgmap v669: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:17.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.970+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27a8069850 con 0x7f279406e210 2024-05-15T16:19:17.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.971+0000 7f279effd700 1 -- 172.21.15.49:0/2974157416 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f27a8069850 con 0x7f279406e210 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f279406e210 msgr2=0x7f27940706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f279406e210 0x7f27940706d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27a80699c0 tx=0x7f279800a300 comp rx=0 tx=0).stop 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 msgr2=0x7f27a8197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 0x7f27a8197a60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f27a4006010 tx=0x7f27a400bd70 comp rx=0 tx=0).stop 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 shutdown_connections 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f279406e210 0x7f27940706d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27a80ffe10 0x7f27a8197a60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 --2- 172.21.15.49:0/2974157416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a81007c0 0x7f27a8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 >> 172.21.15.49:0/2974157416 conn(0x7f27a80fb700 msgr2=0x7f27a80fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 shutdown_connections 2024-05-15T16:19:17.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:17.973+0000 7f27afca4700 1 -- 172.21.15.49:0/2974157416 wait complete. 2024-05-15T16:19:17.985 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:19:18.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.085+0000 7f44b8f4f700 1 -- 172.21.15.49:0/35733619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b4100790 msgr2=0x7f44b41047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.085+0000 7f44b8f4f700 1 --2- 172.21.15.49:0/35733619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b4100790 0x7f44b41047e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009910 tx=0x7f44a8009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.085+0000 7f44b8f4f700 1 -- 172.21.15.49:0/35733619 shutdown_connections 2024-05-15T16:19:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.085+0000 7f44b8f4f700 1 --2- 172.21.15.49:0/35733619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b4100790 0x7f44b41047e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.085+0000 7f44b8f4f700 1 --2- 172.21.15.49:0/35733619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b40ffde0 0x7f44b41001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.085+0000 7f44b8f4f700 1 -- 172.21.15.49:0/35733619 >> 172.21.15.49:0/35733619 conn(0x7f44b40fb690 msgr2=0x7f44b40fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.085+0000 7f44b8f4f700 1 -- 172.21.15.49:0/35733619 shutdown_connections 2024-05-15T16:19:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.085+0000 7f44b8f4f700 1 -- 172.21.15.49:0/35733619 wait complete. 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b8f4f700 1 Processor -- start 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b8f4f700 1 -- start start 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b8f4f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b40ffde0 0x7f44b419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b8f4f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b4100790 0x7f44b419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b8f4f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b419d3b0 con 0x7f44b40ffde0 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b8f4f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b41969a0 con 0x7f44b4100790 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b40ffde0 0x7f44b419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b40ffde0 0x7f44b419c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52564/0 (socket says 172.21.15.49:52564) 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b259c700 1 -- 172.21.15.49:0/400460434 learned_addr learned my addr 172.21.15.49:0/400460434 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.086+0000 7f44b1d9b700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b4100790 0x7f44b419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.087+0000 7f44b259c700 1 -- 172.21.15.49:0/400460434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b4100790 msgr2=0x7f44b419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.087+0000 7f44b259c700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b4100790 0x7f44b419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.087+0000 7f44b259c700 1 -- 172.21.15.49:0/400460434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80095f0 con 0x7f44b40ffde0 2024-05-15T16:19:18.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.087+0000 7f44b259c700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b40ffde0 0x7f44b419c790 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f44a400e920 tx=0x7f44a400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:18.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.087+0000 7f44a37fe700 1 -- 172.21.15.49:0/400460434 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44a400c9d0 con 0x7f44b40ffde0 2024-05-15T16:19:18.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.087+0000 7f44a37fe700 1 -- 172.21.15.49:0/400460434 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44a4010430 con 0x7f44b40ffde0 2024-05-15T16:19:18.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.087+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b4196c80 con 0x7f44b40ffde0 2024-05-15T16:19:18.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.087+0000 7f44a37fe700 1 -- 172.21.15.49:0/400460434 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44a4018630 con 0x7f44b40ffde0 2024-05-15T16:19:18.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.087+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b41971a0 con 0x7f44b40ffde0 2024-05-15T16:19:18.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.088+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b41091f0 con 0x7f44b40ffde0 2024-05-15T16:19:18.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.089+0000 7f44a37fe700 1 -- 172.21.15.49:0/400460434 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f44a40105a0 con 0x7f44b40ffde0 2024-05-15T16:19:18.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.089+0000 7f44a37fe700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f449c06e200 0x7f449c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:18.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.089+0000 7f44a37fe700 1 -- 172.21.15.49:0/400460434 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f44a4014070 con 0x7f44b40ffde0 2024-05-15T16:19:18.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.090+0000 7f44b1d9b700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f449c06e200 0x7f449c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:18.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.090+0000 7f44b1d9b700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f449c06e200 0x7f449c0706c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f44a80059d0 tx=0x7f44a8005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:18.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.094+0000 7f44a37fe700 1 -- 172.21.15.49:0/400460434 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f44a4053850 con 0x7f44b40ffde0 2024-05-15T16:19:18.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.321+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44b4197f90 con 0x7f449c06e200 2024-05-15T16:19:18.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.322+0000 7f44a37fe700 1 -- 172.21.15.49:0/400460434 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f44b4197f90 con 0x7f449c06e200 2024-05-15T16:19:18.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f449c06e200 msgr2=0x7f449c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f449c06e200 0x7f449c0706c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f44a80059d0 tx=0x7f44a8005940 comp rx=0 tx=0).stop 2024-05-15T16:19:18.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b40ffde0 msgr2=0x7f44b419c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b40ffde0 0x7f44b419c790 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f44a400e920 tx=0x7f44a400ec30 comp rx=0 tx=0).stop 2024-05-15T16:19:18.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 shutdown_connections 2024-05-15T16:19:18.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f449c06e200 0x7f449c0706c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44b40ffde0 0x7f44b419c790 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 --2- 172.21.15.49:0/400460434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b4100790 0x7f44b419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 >> 172.21.15.49:0/400460434 conn(0x7f44b40fb690 msgr2=0x7f44b40696e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:18.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 shutdown_connections 2024-05-15T16:19:18.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.325+0000 7f44b8f4f700 1 -- 172.21.15.49:0/400460434 wait complete. 2024-05-15T16:19:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.434+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/643326068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec106b70 msgr2=0x7fa3ec106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.434+0000 7fa3f1b9e700 1 --2- 172.21.15.49:0/643326068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec106b70 0x7fa3ec106f50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009910 tx=0x7fa3e0009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:18.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.434+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/643326068 shutdown_connections 2024-05-15T16:19:18.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.434+0000 7fa3f1b9e700 1 --2- 172.21.15.49:0/643326068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec100ae0 0x7fa3ec100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.434+0000 7fa3f1b9e700 1 --2- 172.21.15.49:0/643326068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec106b70 0x7fa3ec106f50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.434+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/643326068 >> 172.21.15.49:0/643326068 conn(0x7fa3ec0fc5f0 msgr2=0x7fa3ec0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:18.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.434+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/643326068 shutdown_connections 2024-05-15T16:19:18.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.434+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/643326068 wait complete. 2024-05-15T16:19:18.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f1b9e700 1 Processor -- start 2024-05-15T16:19:18.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f1b9e700 1 -- start start 2024-05-15T16:19:18.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f1b9e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec100ae0 0x7fa3ec0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f1b9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec106b70 0x7fa3ec0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f1b9e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec075220 con 0x7fa3ec100ae0 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f1b9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec073d20 con 0x7fa3ec106b70 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f0b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec100ae0 0x7fa3ec0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f0b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec100ae0 0x7fa3ec0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52592/0 (socket says 172.21.15.49:52592) 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f0b9c700 1 -- 172.21.15.49:0/2955781924 learned_addr learned my addr 172.21.15.49:0/2955781924 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3ebfff700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec106b70 0x7fa3ec0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f0b9c700 1 -- 172.21.15.49:0/2955781924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec106b70 msgr2=0x7fa3ec0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f0b9c700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec106b70 0x7fa3ec0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f0b9c700 1 -- 172.21.15.49:0/2955781924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00095f0 con 0x7fa3ec100ae0 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.435+0000 7fa3f0b9c700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec100ae0 0x7fa3ec0732a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0006010 tx=0x7fa3e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.436+0000 7fa3e9ffb700 1 -- 172.21.15.49:0/2955781924 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3e001d070 con 0x7fa3ec100ae0 2024-05-15T16:19:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.436+0000 7fa3e9ffb700 1 -- 172.21.15.49:0/2955781924 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3e000f460 con 0x7fa3ec100ae0 2024-05-15T16:19:18.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.436+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec073fa0 con 0x7fa3ec100ae0 2024-05-15T16:19:18.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.436+0000 7fa3e9ffb700 1 -- 172.21.15.49:0/2955781924 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3e0017630 con 0x7fa3ec100ae0 2024-05-15T16:19:18.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.436+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec1a2df0 con 0x7fa3ec100ae0 2024-05-15T16:19:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.437+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec1090e0 con 0x7fa3ec100ae0 2024-05-15T16:19:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.437+0000 7fa3e9ffb700 1 -- 172.21.15.49:0/2955781924 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa3e000f5d0 con 0x7fa3ec100ae0 2024-05-15T16:19:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.438+0000 7fa3e9ffb700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3dc06e200 0x7fa3dc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.438+0000 7fa3e9ffb700 1 -- 172.21.15.49:0/2955781924 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa3e008f620 con 0x7fa3ec100ae0 2024-05-15T16:19:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.438+0000 7fa3ebfff700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3dc06e200 0x7fa3dc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.438+0000 7fa3ebfff700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3dc06e200 0x7fa3dc0706c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005ea0 tx=0x7fa3d8005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:18.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.443+0000 7fa3e9ffb700 1 -- 172.21.15.49:0/2955781924 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa3e0058a40 con 0x7fa3ec100ae0 2024-05-15T16:19:18.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.658+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3ec0747f0 con 0x7fa3dc06e200 2024-05-15T16:19:18.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.669+0000 7fa3e9ffb700 1 -- 172.21.15.49:0/2955781924 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa3ec0747f0 con 0x7fa3dc06e200 2024-05-15T16:19:18.670 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:19:18.670 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (22m) 23s ago 34m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:19:18.670 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (34m) 23s ago 34m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:19:18.670 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (33m) 25s ago 33m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (19m) 23s ago 34m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (19m) 25s ago 33m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (22m) 23s ago 33m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (16m) 23s ago 28m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (15m) 23s ago 28m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (15m) 25s ago 28m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (29s) 25s ago 28m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (24m) 23s ago 35m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (23m) 25s ago 33m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (21m) 23s ago 35m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (20m) 25s ago 32m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (23m) 23s ago 34m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (23m) 25s ago 33m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (19m) 23s ago 32m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:19:18.671 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (18m) 23s ago 31m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:19:18.672 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (18m) 23s ago 31m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:19:18.672 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (17m) 25s ago 30m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:19:18.672 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (17m) 25s ago 30m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:19:18.672 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (16m) 25s ago 30m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:19:18.672 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (22m) 23s ago 33m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:19:18.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3dc06e200 msgr2=0x7fa3dc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3dc06e200 0x7fa3dc0706c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005ea0 tx=0x7fa3d8005e30 comp rx=0 tx=0).stop 2024-05-15T16:19:18.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec100ae0 msgr2=0x7fa3ec0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec100ae0 0x7fa3ec0732a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0006010 tx=0x7fa3e000be20 comp rx=0 tx=0).stop 2024-05-15T16:19:18.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 shutdown_connections 2024-05-15T16:19:18.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa3dc06e200 0x7fa3dc0706c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3ec100ae0 0x7fa3ec0732a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 --2- 172.21.15.49:0/2955781924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec106b70 0x7fa3ec0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 >> 172.21.15.49:0/2955781924 conn(0x7fa3ec0fc5f0 msgr2=0x7fa3ec0fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:18.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 shutdown_connections 2024-05-15T16:19:18.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.672+0000 7fa3f1b9e700 1 -- 172.21.15.49:0/2955781924 wait complete. 2024-05-15T16:19:18.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.780+0000 7f20d8b28700 1 -- 172.21.15.49:0/809298270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 msgr2=0x7f20d41001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.780+0000 7f20d8b28700 1 --2- 172.21.15.49:0/809298270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 0x7f20d41001b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009910 tx=0x7f20c4009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.780+0000 7f20d8b28700 1 -- 172.21.15.49:0/809298270 shutdown_connections 2024-05-15T16:19:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.780+0000 7f20d8b28700 1 --2- 172.21.15.49:0/809298270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d4100780 0x7f20d41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.780+0000 7f20d8b28700 1 --2- 172.21.15.49:0/809298270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 0x7f20d41001b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.780+0000 7f20d8b28700 1 -- 172.21.15.49:0/809298270 >> 172.21.15.49:0/809298270 conn(0x7f20d40fb6a0 msgr2=0x7f20d40fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.780+0000 7f20d8b28700 1 -- 172.21.15.49:0/809298270 shutdown_connections 2024-05-15T16:19:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.780+0000 7f20d8b28700 1 -- 172.21.15.49:0/809298270 wait complete. 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.781+0000 7f20d8b28700 1 Processor -- start 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.781+0000 7f20d8b28700 1 -- start start 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.781+0000 7f20d8b28700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 0x7f20d4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.781+0000 7f20d8b28700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d4100780 0x7f20d41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.781+0000 7f20d8b28700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41986a0 con 0x7f20d40ffdd0 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.781+0000 7f20d8b28700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41987e0 con 0x7f20d4100780 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.781+0000 7f20d259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 0x7f20d4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.781+0000 7f20d259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 0x7f20d4197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52596/0 (socket says 172.21.15.49:52596) 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.781+0000 7f20d259c700 1 -- 172.21.15.49:0/1763035051 learned_addr learned my addr 172.21.15.49:0/1763035051 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20d1d9b700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d4100780 0x7f20d41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20d259c700 1 -- 172.21.15.49:0/1763035051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d4100780 msgr2=0x7f20d41980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20d259c700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d4100780 0x7f20d41980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20d259c700 1 -- 172.21.15.49:0/1763035051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40095f0 con 0x7f20d40ffdd0 2024-05-15T16:19:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20d1d9b700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d4100780 0x7f20d41980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:19:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20d259c700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 0x7f20d4197b90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f20c400bd70 tx=0x7f20c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20c37fe700 1 -- 172.21.15.49:0/1763035051 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20c401d070 con 0x7f20d40ffdd0 2024-05-15T16:19:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d4198a00 con 0x7f20d40ffdd0 2024-05-15T16:19:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20c37fe700 1 -- 172.21.15.49:0/1763035051 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20c400f460 con 0x7f20d40ffdd0 2024-05-15T16:19:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20c37fe700 1 -- 172.21.15.49:0/1763035051 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20c4017630 con 0x7f20d40ffdd0 2024-05-15T16:19:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.782+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d41a0f60 con 0x7f20d40ffdd0 2024-05-15T16:19:18.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.783+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d41012b0 con 0x7f20d40ffdd0 2024-05-15T16:19:18.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.784+0000 7f20c37fe700 1 -- 172.21.15.49:0/1763035051 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f20c400f5d0 con 0x7f20d40ffdd0 2024-05-15T16:19:18.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.784+0000 7f20c37fe700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20bc06e200 0x7f20bc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:18.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.784+0000 7f20c37fe700 1 -- 172.21.15.49:0/1763035051 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f20c408f450 con 0x7f20d40ffdd0 2024-05-15T16:19:18.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.784+0000 7f20d1d9b700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20bc06e200 0x7f20bc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:18.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.785+0000 7f20d1d9b700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20bc06e200 0x7f20bc0706c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f20d4069720 tx=0x7f20c8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:18.789+0000 7f20c37fe700 1 -- 172.21.15.49:0/1763035051 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f20c40587b0 con 0x7f20d40ffdd0 2024-05-15T16:19:18.943 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:18 smithi144 ceph-mon[86635]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:18.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:18 smithi049 ceph-mon[112142]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:19.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.088+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f20d41989a0 con 0x7f20d40ffdd0 2024-05-15T16:19:19.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.088+0000 7f20c37fe700 1 -- 172.21.15.49:0/1763035051 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f20c405be70 con 0x7f20d40ffdd0 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:19:19.090 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:19:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:19:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:19:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:19:19.091 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:19:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:19:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20bc06e200 msgr2=0x7f20bc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20bc06e200 0x7f20bc0706c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f20d4069720 tx=0x7f20c8008040 comp rx=0 tx=0).stop 2024-05-15T16:19:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 msgr2=0x7f20d4197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 0x7f20d4197b90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f20c400bd70 tx=0x7f20c400be50 comp rx=0 tx=0).stop 2024-05-15T16:19:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 shutdown_connections 2024-05-15T16:19:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f20bc06e200 0x7f20bc0706c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20d40ffdd0 0x7f20d4197b90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 --2- 172.21.15.49:0/1763035051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d4100780 0x7f20d41980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 >> 172.21.15.49:0/1763035051 conn(0x7f20d40fb6a0 msgr2=0x7f20d40fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:19.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 shutdown_connections 2024-05-15T16:19:19.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.091+0000 7f20d8b28700 1 -- 172.21.15.49:0/1763035051 wait complete. 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.199+0000 7fd309fa7700 1 -- 172.21.15.49:0/3241077711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304101e70 msgr2=0x7fd304102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.199+0000 7fd309fa7700 1 --2- 172.21.15.49:0/3241077711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304101e70 0x7fd304102250 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009910 tx=0x7fd2f4009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.199+0000 7fd309fa7700 1 -- 172.21.15.49:0/3241077711 shutdown_connections 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.199+0000 7fd309fa7700 1 --2- 172.21.15.49:0/3241077711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd304102790 0x7fd30410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.199+0000 7fd309fa7700 1 --2- 172.21.15.49:0/3241077711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304101e70 0x7fd304102250 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.199+0000 7fd309fa7700 1 -- 172.21.15.49:0/3241077711 >> 172.21.15.49:0/3241077711 conn(0x7fd3040fb700 msgr2=0x7fd3040fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.199+0000 7fd309fa7700 1 -- 172.21.15.49:0/3241077711 shutdown_connections 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.199+0000 7fd309fa7700 1 -- 172.21.15.49:0/3241077711 wait complete. 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.199+0000 7fd309fa7700 1 Processor -- start 2024-05-15T16:19:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd309fa7700 1 -- start start 2024-05-15T16:19:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd309fa7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd304101e70 0x7fd3040ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd309fa7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304102790 0x7fd304100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd309fa7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304100550 con 0x7fd304102790 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd309fa7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304100690 con 0x7fd304101e70 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd302ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304102790 0x7fd304100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd302ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304102790 0x7fd304100010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52622/0 (socket says 172.21.15.49:52622) 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd302ffd700 1 -- 172.21.15.49:0/122137069 learned_addr learned my addr 172.21.15.49:0/122137069 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd3037fe700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd304101e70 0x7fd3040ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd302ffd700 1 -- 172.21.15.49:0/122137069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd304101e70 msgr2=0x7fd3040ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd302ffd700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd304101e70 0x7fd3040ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd302ffd700 1 -- 172.21.15.49:0/122137069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40095f0 con 0x7fd304102790 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.200+0000 7fd302ffd700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304102790 0x7fd304100010 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800d6d0 tx=0x7fd2f800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.201+0000 7fd300ff9700 1 -- 172.21.15.49:0/122137069 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f800f860 con 0x7fd304102790 2024-05-15T16:19:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.201+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3041a2e10 con 0x7fd304102790 2024-05-15T16:19:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.201+0000 7fd300ff9700 1 -- 172.21.15.49:0/122137069 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2f800fe80 con 0x7fd304102790 2024-05-15T16:19:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.201+0000 7fd300ff9700 1 -- 172.21.15.49:0/122137069 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f8010ea0 con 0x7fd304102790 2024-05-15T16:19:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.201+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3041a3170 con 0x7fd304102790 2024-05-15T16:19:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.202+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd304069ab0 con 0x7fd304102790 2024-05-15T16:19:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.202+0000 7fd300ff9700 1 -- 172.21.15.49:0/122137069 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd2f8010460 con 0x7fd304102790 2024-05-15T16:19:19.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.203+0000 7fd300ff9700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2ec06e200 0x7fd2ec0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:19.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.203+0000 7fd300ff9700 1 -- 172.21.15.49:0/122137069 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd2f808ded0 con 0x7fd304102790 2024-05-15T16:19:19.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.203+0000 7fd3037fe700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2ec06e200 0x7fd2ec0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:19.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.203+0000 7fd3037fe700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2ec06e200 0x7fd2ec0706c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4000c00 tx=0x7fd2f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.207+0000 7fd300ff9700 1 -- 172.21.15.49:0/122137069 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd2f80571b0 con 0x7fd304102790 2024-05-15T16:19:19.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.458+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd30404ec60 con 0x7fd304102790 2024-05-15T16:19:19.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.458+0000 7fd300ff9700 1 -- 172.21.15.49:0/122137069 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fd2f805a870 con 0x7fd304102790 2024-05-15T16:19:19.460 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:19:19.460 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:19:19.461 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:19:19.462 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:19:19.463 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:19:19.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.460+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2ec06e200 msgr2=0x7fd2ec0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.460+0000 7fd309fa7700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2ec06e200 0x7fd2ec0706c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4000c00 tx=0x7fd2f401a040 comp rx=0 tx=0).stop 2024-05-15T16:19:19.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.460+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304102790 msgr2=0x7fd304100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.461+0000 7fd309fa7700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304102790 0x7fd304100010 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800d6d0 tx=0x7fd2f800d9e0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.461+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 shutdown_connections 2024-05-15T16:19:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.461+0000 7fd309fa7700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2ec06e200 0x7fd2ec0706c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.461+0000 7fd309fa7700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd304101e70 0x7fd3040ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.461+0000 7fd309fa7700 1 --2- 172.21.15.49:0/122137069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd304102790 0x7fd304100010 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.461+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 >> 172.21.15.49:0/122137069 conn(0x7fd3040fb700 msgr2=0x7fd304103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.461+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 shutdown_connections 2024-05-15T16:19:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.461+0000 7fd309fa7700 1 -- 172.21.15.49:0/122137069 wait complete. 2024-05-15T16:19:19.465 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:19:19.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.570+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/2989516616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 msgr2=0x7fb1c410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.570+0000 7fb1cb7a9700 1 --2- 172.21.15.49:0/2989516616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 0x7fb1c410d070 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009910 tx=0x7fb1b4009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:19.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.570+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/2989516616 shutdown_connections 2024-05-15T16:19:19.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.570+0000 7fb1cb7a9700 1 --2- 172.21.15.49:0/2989516616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 0x7fb1c410d070 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.570+0000 7fb1cb7a9700 1 --2- 172.21.15.49:0/2989516616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c40ffa00 0x7fb1c40ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.570+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/2989516616 >> 172.21.15.49:0/2989516616 conn(0x7fb1c40fb700 msgr2=0x7fb1c40fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:19.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.570+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/2989516616 shutdown_connections 2024-05-15T16:19:19.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.570+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/2989516616 wait complete. 2024-05-15T16:19:19.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.570+0000 7fb1cb7a9700 1 Processor -- start 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1cb7a9700 1 -- start start 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1cb7a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c40ffa00 0x7fb1c4198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1cb7a9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 0x7fb1c41993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1cb7a9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4199a00 con 0x7fb1c41003b0 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1cb7a9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4199b40 con 0x7fb1c40ffa00 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1c8d44700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 0x7fb1c41993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1c8d44700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 0x7fb1c41993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52636/0 (socket says 172.21.15.49:52636) 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1c8d44700 1 -- 172.21.15.49:0/706253215 learned_addr learned my addr 172.21.15.49:0/706253215 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1c9545700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c40ffa00 0x7fb1c4198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1c8d44700 1 -- 172.21.15.49:0/706253215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c40ffa00 msgr2=0x7fb1c4198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1c8d44700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c40ffa00 0x7fb1c4198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1c8d44700 1 -- 172.21.15.49:0/706253215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40095f0 con 0x7fb1c41003b0 2024-05-15T16:19:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.571+0000 7fb1c8d44700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 0x7fb1c41993b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b850 tx=0x7fb1b40055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.572+0000 7fb1ba7fc700 1 -- 172.21.15.49:0/706253215 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b401d070 con 0x7fb1c41003b0 2024-05-15T16:19:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.572+0000 7fb1ba7fc700 1 -- 172.21.15.49:0/706253215 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1b4003d50 con 0x7fb1c41003b0 2024-05-15T16:19:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.572+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c419d870 con 0x7fb1c41003b0 2024-05-15T16:19:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.572+0000 7fb1ba7fc700 1 -- 172.21.15.49:0/706253215 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b4021410 con 0x7fb1c41003b0 2024-05-15T16:19:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.572+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c419ddc0 con 0x7fb1c41003b0 2024-05-15T16:19:19.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.573+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c410aef0 con 0x7fb1c41003b0 2024-05-15T16:19:19.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.573+0000 7fb1ba7fc700 1 -- 172.21.15.49:0/706253215 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb1b400f6b0 con 0x7fb1c41003b0 2024-05-15T16:19:19.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.574+0000 7fb1ba7fc700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb1b006e200 0x7fb1b00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:19.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.574+0000 7fb1ba7fc700 1 -- 172.21.15.49:0/706253215 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb1b408f3b0 con 0x7fb1c41003b0 2024-05-15T16:19:19.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.574+0000 7fb1c9545700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb1b006e200 0x7fb1b00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:19.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.574+0000 7fb1c9545700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb1b006e200 0x7fb1b00706c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000ac50 tx=0x7fb1c000a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:19.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.579+0000 7fb1ba7fc700 1 -- 172.21.15.49:0/706253215 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb1b4058690 con 0x7fb1c41003b0 2024-05-15T16:19:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.806+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1c419a180 con 0x7fb1b006e200 2024-05-15T16:19:19.808 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:19 smithi049 ceph-mon[112142]: from='client.35246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:19.808 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:19 smithi049 ceph-mon[112142]: pgmap v670: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:19.808 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:19 smithi049 ceph-mon[112142]: from='client.35250 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:19.808 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:19 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1763035051' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:19:19.808 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:19 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/122137069' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.808+0000 7fb1ba7fc700 1 -- 172.21.15.49:0/706253215 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb1c419a180 con 0x7fb1b006e200 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:19:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:19:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:19:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:19:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:19:19.810 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:19:19.810 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb1b006e200 msgr2=0x7fb1b00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb1b006e200 0x7fb1b00706c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000ac50 tx=0x7fb1c000a380 comp rx=0 tx=0).stop 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 msgr2=0x7fb1c41993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 0x7fb1c41993b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b850 tx=0x7fb1b40055f0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 shutdown_connections 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb1b006e200 0x7fb1b00706c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c40ffa00 0x7fb1c4198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 --2- 172.21.15.49:0/706253215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1c41003b0 0x7fb1c41993b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 >> 172.21.15.49:0/706253215 conn(0x7fb1c40fb700 msgr2=0x7fb1c4108030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 shutdown_connections 2024-05-15T16:19:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.810+0000 7fb1cb7a9700 1 -- 172.21.15.49:0/706253215 wait complete. 2024-05-15T16:19:19.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.919+0000 7f22bde54700 1 -- 172.21.15.49:0/2632225033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 msgr2=0x7f22b810acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.919+0000 7f22bde54700 1 --2- 172.21.15.49:0/2632225033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 0x7f22b810acc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009940 tx=0x7f22ac009c50 comp rx=0 tx=0).stop 2024-05-15T16:19:19.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.919+0000 7f22bde54700 1 -- 172.21.15.49:0/2632225033 shutdown_connections 2024-05-15T16:19:19.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.919+0000 7f22bde54700 1 --2- 172.21.15.49:0/2632225033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 0x7f22b810acc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.919+0000 7f22bde54700 1 --2- 172.21.15.49:0/2632225033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b8101eb0 0x7f22b8102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.919+0000 7f22bde54700 1 -- 172.21.15.49:0/2632225033 >> 172.21.15.49:0/2632225033 conn(0x7f22b80fb6a0 msgr2=0x7f22b80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:19.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.919+0000 7f22bde54700 1 -- 172.21.15.49:0/2632225033 shutdown_connections 2024-05-15T16:19:19.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.919+0000 7f22bde54700 1 -- 172.21.15.49:0/2632225033 wait complete. 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22bde54700 1 Processor -- start 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22bde54700 1 -- start start 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22bde54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b8101eb0 0x7f22b819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22bde54700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 0x7f22b819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22bde54700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b819d470 con 0x7f22b81027d0 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22bde54700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b8196a50 con 0x7f22b8101eb0 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22b6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 0x7f22b819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22b6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 0x7f22b819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52648/0 (socket says 172.21.15.49:52648) 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22b6ffd700 1 -- 172.21.15.49:0/1751739224 learned_addr learned my addr 172.21.15.49:0/1751739224 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22b77fe700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b8101eb0 0x7f22b819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.920+0000 7f22b6ffd700 1 -- 172.21.15.49:0/1751739224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b8101eb0 msgr2=0x7f22b819c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.921+0000 7f22b6ffd700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b8101eb0 0x7f22b819c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.921+0000 7f22b6ffd700 1 -- 172.21.15.49:0/1751739224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac0095f0 con 0x7f22b81027d0 2024-05-15T16:19:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.921+0000 7f22b6ffd700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 0x7f22b819ce50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0055f0 tx=0x7f22ac005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.921+0000 7f22b4ff9700 1 -- 172.21.15.49:0/1751739224 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22ac01d070 con 0x7f22b81027d0 2024-05-15T16:19:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.921+0000 7f22b4ff9700 1 -- 172.21.15.49:0/1751739224 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22ac003d50 con 0x7f22b81027d0 2024-05-15T16:19:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.921+0000 7f22b4ff9700 1 -- 172.21.15.49:0/1751739224 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22ac021410 con 0x7f22b81027d0 2024-05-15T16:19:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.921+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b8196cd0 con 0x7f22b81027d0 2024-05-15T16:19:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.921+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b8197170 con 0x7f22b81027d0 2024-05-15T16:19:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.922+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b8105640 con 0x7f22b81027d0 2024-05-15T16:19:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.923+0000 7f22b4ff9700 1 -- 172.21.15.49:0/1751739224 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f22ac00f6d0 con 0x7f22b81027d0 2024-05-15T16:19:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.923+0000 7f22b4ff9700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f22a006e200 0x7f22a00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.923+0000 7f22b4ff9700 1 -- 172.21.15.49:0/1751739224 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f22ac08f3f0 con 0x7f22b81027d0 2024-05-15T16:19:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.923+0000 7f22b77fe700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f22a006e200 0x7f22a00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.924+0000 7f22b77fe700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f22a006e200 0x7f22a00706c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005950 tx=0x7f22a800a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:19.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:19.928+0000 7f22b4ff9700 1 -- 172.21.15.49:0/1751739224 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f22ac05c200 con 0x7f22b81027d0 2024-05-15T16:19:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:19 smithi144 ceph-mon[86635]: from='client.35246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:19 smithi144 ceph-mon[86635]: pgmap v670: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:19 smithi144 ceph-mon[86635]: from='client.35250 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:19 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1763035051' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:19:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:19 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/122137069' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:19:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.237+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f22b8069a50 con 0x7f22b81027d0 2024-05-15T16:19:20.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.237+0000 7f22b4ff9700 1 -- 172.21.15.49:0/1751739224 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f22ac05bd90 con 0x7f22b81027d0 2024-05-15T16:19:20.239 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:19:20.239 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:19:20.239 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:19:20.239 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:19:20.239 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:19:20.240 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:19:20.241 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:19:20.241 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:19:20.241 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:19:20.241 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:19:20.241 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:19:20.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.239+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f22a006e200 msgr2=0x7f22a00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.239+0000 7f22bde54700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f22a006e200 0x7f22a00706c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005950 tx=0x7f22a800a300 comp rx=0 tx=0).stop 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.239+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 msgr2=0x7f22b819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.239+0000 7f22bde54700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 0x7f22b819ce50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0055f0 tx=0x7f22ac005620 comp rx=0 tx=0).stop 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.239+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 shutdown_connections 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.239+0000 7f22bde54700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f22a006e200 0x7f22a00706c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.240+0000 7f22bde54700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b8101eb0 0x7f22b819c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.240+0000 7f22bde54700 1 --2- 172.21.15.49:0/1751739224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22b81027d0 0x7f22b819ce50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.240+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 >> 172.21.15.49:0/1751739224 conn(0x7f22b80fb6a0 msgr2=0x7f22b81031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.240+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 shutdown_connections 2024-05-15T16:19:20.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:20.240+0000 7f22bde54700 1 -- 172.21.15.49:0/1751739224 wait complete. 2024-05-15T16:19:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:20 smithi144 ceph-mon[86635]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:19:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:20 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1751739224' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:19:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:20 smithi049 ceph-mon[112142]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:19:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:20 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1751739224' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:19:21.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:21 smithi049 ceph-mon[112142]: pgmap v671: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:21 smithi144 ceph-mon[86635]: pgmap v671: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:23.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:23 smithi049 ceph-mon[112142]: pgmap v672: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:23.996 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:23 smithi144 ceph-mon[86635]: pgmap v672: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:25.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:25 smithi049 ceph-mon[112142]: pgmap v673: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:25 smithi144 ceph-mon[86635]: pgmap v673: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:28.166 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:27 smithi049 ceph-mon[112142]: pgmap v674: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:27 smithi144 ceph-mon[86635]: pgmap v674: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:29 smithi144 ceph-mon[86635]: pgmap v675: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:29 smithi049 ceph-mon[112142]: pgmap v675: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:31 smithi144 ceph-mon[86635]: pgmap v676: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:31 smithi049 ceph-mon[112142]: pgmap v676: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:34.000 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:33 smithi144 ceph-mon[86635]: pgmap v677: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:33 smithi049 ceph-mon[112142]: pgmap v677: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:36.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:35 smithi144 ceph-mon[86635]: pgmap v678: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:36.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:19:36.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:35 smithi049 ceph-mon[112142]: pgmap v678: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:36.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:19:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:37 smithi144 ceph-mon[86635]: pgmap v679: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:37 smithi049 ceph-mon[112142]: pgmap v679: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:39 smithi144 ceph-mon[86635]: pgmap v680: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:39 smithi049 ceph-mon[112142]: pgmap v680: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:41 smithi144 ceph-mon[86635]: pgmap v681: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:41 smithi049 ceph-mon[112142]: pgmap v681: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:43 smithi144 ceph-mon[86635]: pgmap v682: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:43 smithi049 ceph-mon[112142]: pgmap v682: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:45 smithi144 ceph-mon[86635]: pgmap v683: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:45 smithi049 ceph-mon[112142]: pgmap v683: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:47 smithi144 ceph-mon[86635]: pgmap v684: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:47 smithi049 ceph-mon[112142]: pgmap v684: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:49 smithi144 ceph-mon[86635]: pgmap v685: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:49 smithi049 ceph-mon[112142]: pgmap v685: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.362+0000 7f001cd67700 1 -- 172.21.15.49:0/3170545092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018101890 msgr2=0x7f00181058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.362+0000 7f001cd67700 1 --2- 172.21.15.49:0/3170545092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018101890 0x7f00181058e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f000c009910 tx=0x7f000c009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.362+0000 7f001cd67700 1 -- 172.21.15.49:0/3170545092 shutdown_connections 2024-05-15T16:19:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.362+0000 7f001cd67700 1 --2- 172.21.15.49:0/3170545092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018101890 0x7f00181058e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.362+0000 7f001cd67700 1 --2- 172.21.15.49:0/3170545092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0018100ee0 0x7f00181012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.362+0000 7f001cd67700 1 -- 172.21.15.49:0/3170545092 >> 172.21.15.49:0/3170545092 conn(0x7f00180fc790 msgr2=0x7f00180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.362+0000 7f001cd67700 1 -- 172.21.15.49:0/3170545092 shutdown_connections 2024-05-15T16:19:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.362+0000 7f001cd67700 1 -- 172.21.15.49:0/3170545092 wait complete. 2024-05-15T16:19:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f001cd67700 1 Processor -- start 2024-05-15T16:19:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f001cd67700 1 -- start start 2024-05-15T16:19:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f001cd67700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100ee0 0x7f001819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f001cd67700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0018101890 0x7f001819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f001cd67700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001819d420 con 0x7f0018100ee0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f001cd67700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018196a00 con 0x7f0018101890 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f001659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100ee0 0x7f001819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f001659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100ee0 0x7f001819c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46636/0 (socket says 172.21.15.49:46636) 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f001659c700 1 -- 172.21.15.49:0/337692228 learned_addr learned my addr 172.21.15.49:0/337692228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.363+0000 7f0015d9b700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0018101890 0x7f001819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.364+0000 7f001659c700 1 -- 172.21.15.49:0/337692228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0018101890 msgr2=0x7f001819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.364+0000 7f001659c700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0018101890 0x7f001819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.364+0000 7f001659c700 1 -- 172.21.15.49:0/337692228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0095f0 con 0x7f0018100ee0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.364+0000 7f001659c700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100ee0 0x7f001819c8c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f000800e920 tx=0x7f000800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.364+0000 7f00077fe700 1 -- 172.21.15.49:0/337692228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f000800c9d0 con 0x7f0018100ee0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.364+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0018196ce0 con 0x7f0018100ee0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.364+0000 7f00077fe700 1 -- 172.21.15.49:0/337692228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0008010430 con 0x7f0018100ee0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.364+0000 7f00077fe700 1 -- 172.21.15.49:0/337692228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0008018630 con 0x7f0018100ee0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.364+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0018197200 con 0x7f0018100ee0 2024-05-15T16:19:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.365+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00181092e0 con 0x7f0018100ee0 2024-05-15T16:19:50.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.366+0000 7f00077fe700 1 -- 172.21.15.49:0/337692228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f00080105a0 con 0x7f0018100ee0 2024-05-15T16:19:50.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.366+0000 7f00077fe700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f000006e200 0x7f00000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:50.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.366+0000 7f00077fe700 1 -- 172.21.15.49:0/337692228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0008014070 con 0x7f0018100ee0 2024-05-15T16:19:50.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.366+0000 7f0015d9b700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f000006e200 0x7f00000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:50.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.367+0000 7f0015d9b700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f000006e200 0x7f00000706c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f000c0059d0 tx=0x7f000c005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:50.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.371+0000 7f00077fe700 1 -- 172.21.15.49:0/337692228 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0008053850 con 0x7f0018100ee0 2024-05-15T16:19:50.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.600+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0018197f30 con 0x7f000006e200 2024-05-15T16:19:50.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.601+0000 7f00077fe700 1 -- 172.21.15.49:0/337692228 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0018197f30 con 0x7f000006e200 2024-05-15T16:19:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f000006e200 msgr2=0x7f00000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f000006e200 0x7f00000706c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f000c0059d0 tx=0x7f000c005940 comp rx=0 tx=0).stop 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100ee0 msgr2=0x7f001819c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100ee0 0x7f001819c8c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f000800e920 tx=0x7f000800ec30 comp rx=0 tx=0).stop 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 shutdown_connections 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f000006e200 0x7f00000706c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100ee0 0x7f001819c8c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 --2- 172.21.15.49:0/337692228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0018101890 0x7f001819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 >> 172.21.15.49:0/337692228 conn(0x7f00180fc790 msgr2=0x7f00180fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 shutdown_connections 2024-05-15T16:19:50.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.603+0000 7f001cd67700 1 -- 172.21.15.49:0/337692228 wait complete. 2024-05-15T16:19:50.615 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:19:50.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.714+0000 7ffb36c67700 1 -- 172.21.15.49:0/757331313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb301000a0 msgr2=0x7ffb30103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:50.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.714+0000 7ffb36c67700 1 --2- 172.21.15.49:0/757331313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb301000a0 0x7ffb30103f90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009910 tx=0x7ffb24009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.714+0000 7ffb36c67700 1 -- 172.21.15.49:0/757331313 shutdown_connections 2024-05-15T16:19:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.714+0000 7ffb36c67700 1 --2- 172.21.15.49:0/757331313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb301000a0 0x7ffb30103f90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.714+0000 7ffb36c67700 1 --2- 172.21.15.49:0/757331313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb300ff6f0 0x7ffb300ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.714+0000 7ffb36c67700 1 -- 172.21.15.49:0/757331313 >> 172.21.15.49:0/757331313 conn(0x7ffb30074e50 msgr2=0x7ffb30075260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.714+0000 7ffb36c67700 1 -- 172.21.15.49:0/757331313 shutdown_connections 2024-05-15T16:19:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.714+0000 7ffb36c67700 1 -- 172.21.15.49:0/757331313 wait complete. 2024-05-15T16:19:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb36c67700 1 Processor -- start 2024-05-15T16:19:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb36c67700 1 -- start start 2024-05-15T16:19:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb36c67700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb300ff6f0 0x7ffb300fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb36c67700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb301000a0 0x7ffb300fd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb36c67700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300fd7d0 con 0x7ffb300ff6f0 2024-05-15T16:19:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb36c67700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300fd910 con 0x7ffb301000a0 2024-05-15T16:19:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb34a03700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb300ff6f0 0x7ffb300fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb34a03700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb300ff6f0 0x7ffb300fcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46646/0 (socket says 172.21.15.49:46646) 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb34a03700 1 -- 172.21.15.49:0/2698403218 learned_addr learned my addr 172.21.15.49:0/2698403218 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb2ffff700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb301000a0 0x7ffb300fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb34a03700 1 -- 172.21.15.49:0/2698403218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb301000a0 msgr2=0x7ffb300fd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb34a03700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb301000a0 0x7ffb300fd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb34a03700 1 -- 172.21.15.49:0/2698403218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb240095f0 con 0x7ffb300ff6f0 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.715+0000 7ffb2ffff700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb301000a0 0x7ffb300fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.716+0000 7ffb34a03700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb300ff6f0 0x7ffb300fcd50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c8a0 tx=0x7ffb2000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.716+0000 7ffb2dffb700 1 -- 172.21.15.49:0/2698403218 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb2000cea0 con 0x7ffb300ff6f0 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.716+0000 7ffb2dffb700 1 -- 172.21.15.49:0/2698403218 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb20004830 con 0x7ffb300ff6f0 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.716+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb301a2dc0 con 0x7ffb300ff6f0 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.716+0000 7ffb2dffb700 1 -- 172.21.15.49:0/2698403218 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb200055b0 con 0x7ffb300ff6f0 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.716+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb301a3120 con 0x7ffb300ff6f0 2024-05-15T16:19:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.717+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb30100a70 con 0x7ffb300ff6f0 2024-05-15T16:19:50.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.717+0000 7ffb2dffb700 1 -- 172.21.15.49:0/2698403218 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffb200049a0 con 0x7ffb300ff6f0 2024-05-15T16:19:50.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.718+0000 7ffb2dffb700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffb1806e200 0x7ffb180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:50.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.718+0000 7ffb2dffb700 1 -- 172.21.15.49:0/2698403218 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ffb2008dd60 con 0x7ffb300ff6f0 2024-05-15T16:19:50.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.718+0000 7ffb2ffff700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffb1806e200 0x7ffb180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:50.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.718+0000 7ffb2ffff700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffb1806e200 0x7ffb180706c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffb24006010 tx=0x7ffb2400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:50.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.722+0000 7ffb2dffb700 1 -- 172.21.15.49:0/2698403218 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ffb200570c0 con 0x7ffb300ff6f0 2024-05-15T16:19:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:19:50.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.953+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb3010c5f0 con 0x7ffb1806e200 2024-05-15T16:19:50.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.954+0000 7ffb2dffb700 1 -- 172.21.15.49:0/2698403218 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ffb3010c5f0 con 0x7ffb1806e200 2024-05-15T16:19:50.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffb1806e200 msgr2=0x7ffb180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffb1806e200 0x7ffb180706c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffb24006010 tx=0x7ffb2400b540 comp rx=0 tx=0).stop 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb300ff6f0 msgr2=0x7ffb300fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb300ff6f0 0x7ffb300fcd50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c8a0 tx=0x7ffb2000cc60 comp rx=0 tx=0).stop 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 shutdown_connections 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ffb1806e200 0x7ffb180706c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb300ff6f0 0x7ffb300fcd50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 --2- 172.21.15.49:0/2698403218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb301000a0 0x7ffb300fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 >> 172.21.15.49:0/2698403218 conn(0x7ffb30074e50 msgr2=0x7ffb30103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 shutdown_connections 2024-05-15T16:19:50.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:50.956+0000 7ffb36c67700 1 -- 172.21.15.49:0/2698403218 wait complete. 2024-05-15T16:19:51.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.066+0000 7fe768e6c700 1 -- 172.21.15.49:0/3596209246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7641018b0 msgr2=0x7fe764105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.066+0000 7fe768e6c700 1 --2- 172.21.15.49:0/3596209246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7641018b0 0x7fe764105900 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe758009910 tx=0x7fe758009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.066+0000 7fe768e6c700 1 -- 172.21.15.49:0/3596209246 shutdown_connections 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.066+0000 7fe768e6c700 1 --2- 172.21.15.49:0/3596209246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7641018b0 0x7fe764105900 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.066+0000 7fe768e6c700 1 --2- 172.21.15.49:0/3596209246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe764100f00 0x7fe7641012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.066+0000 7fe768e6c700 1 -- 172.21.15.49:0/3596209246 >> 172.21.15.49:0/3596209246 conn(0x7fe7640fc7d0 msgr2=0x7fe7640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.066+0000 7fe768e6c700 1 -- 172.21.15.49:0/3596209246 shutdown_connections 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.066+0000 7fe768e6c700 1 -- 172.21.15.49:0/3596209246 wait complete. 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe768e6c700 1 Processor -- start 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe768e6c700 1 -- start start 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe768e6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764100f00 0x7fe76419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe768e6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7641018b0 0x7fe76419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe768e6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76419d440 con 0x7fe764100f00 2024-05-15T16:19:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe768e6c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe764196a20 con 0x7fe7641018b0 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe76259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764100f00 0x7fe76419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe76259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764100f00 0x7fe76419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46670/0 (socket says 172.21.15.49:46670) 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe76259c700 1 -- 172.21.15.49:0/3683686629 learned_addr learned my addr 172.21.15.49:0/3683686629 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.067+0000 7fe761d9b700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7641018b0 0x7fe76419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe76259c700 1 -- 172.21.15.49:0/3683686629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7641018b0 msgr2=0x7fe76419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe76259c700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7641018b0 0x7fe76419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe76259c700 1 -- 172.21.15.49:0/3683686629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7580095f0 con 0x7fe764100f00 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe761d9b700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7641018b0 0x7fe76419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe76259c700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764100f00 0x7fe76419c8e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d8d0 tx=0x7fe75400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe7537fe700 1 -- 172.21.15.49:0/3683686629 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe754009940 con 0x7fe764100f00 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe7537fe700 1 -- 172.21.15.49:0/3683686629 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe754010460 con 0x7fe764100f00 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe764196d00 con 0x7fe764100f00 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe7537fe700 1 -- 172.21.15.49:0/3683686629 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe75400f5f0 con 0x7fe764100f00 2024-05-15T16:19:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.068+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe764197250 con 0x7fe764100f00 2024-05-15T16:19:51.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.069+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe764109300 con 0x7fe764100f00 2024-05-15T16:19:51.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.070+0000 7fe7537fe700 1 -- 172.21.15.49:0/3683686629 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe7540105d0 con 0x7fe764100f00 2024-05-15T16:19:51.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.070+0000 7fe7537fe700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe74c06e210 0x7fe74c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:51.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.070+0000 7fe7537fe700 1 -- 172.21.15.49:0/3683686629 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe75408e090 con 0x7fe764100f00 2024-05-15T16:19:51.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.070+0000 7fe761d9b700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe74c06e210 0x7fe74c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:51.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.071+0000 7fe761d9b700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe74c06e210 0x7fe74c0706d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe764197be0 tx=0x7fe75801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:51.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.075+0000 7fe7537fe700 1 -- 172.21.15.49:0/3683686629 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe75405af20 con 0x7fe764100f00 2024-05-15T16:19:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:19:51.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.293+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe764197f50 con 0x7fe74c06e210 2024-05-15T16:19:51.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.305+0000 7fe7537fe700 1 -- 172.21.15.49:0/3683686629 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe764197f50 con 0x7fe74c06e210 2024-05-15T16:19:51.306 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (23m) 55s ago 34m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (34m) 55s ago 34m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (33m) 58s ago 33m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (20m) 55s ago 34m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (20m) 58s ago 33m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (22m) 55s ago 34m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (16m) 55s ago 29m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (16m) 55s ago 28m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (16m) 58s ago 29m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (62s) 58s ago 28m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (25m) 55s ago 36m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (24m) 58s ago 33m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (21m) 55s ago 36m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (21m) 58s ago 33m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:19:51.307 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (23m) 55s ago 34m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:19:51.308 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (23m) 58s ago 33m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:19:51.308 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (20m) 55s ago 32m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:19:51.308 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (19m) 55s ago 32m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:19:51.308 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (18m) 55s ago 31m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:19:51.308 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (18m) 58s ago 31m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:19:51.308 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (17m) 58s ago 31m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:19:51.308 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (17m) 58s ago 30m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:19:51.308 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (23m) 55s ago 34m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.307+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe74c06e210 msgr2=0x7fe74c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.307+0000 7fe768e6c700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe74c06e210 0x7fe74c0706d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe764197be0 tx=0x7fe75801a040 comp rx=0 tx=0).stop 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.307+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764100f00 msgr2=0x7fe76419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.307+0000 7fe768e6c700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764100f00 0x7fe76419c8e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d8d0 tx=0x7fe75400dc90 comp rx=0 tx=0).stop 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.307+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 shutdown_connections 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.307+0000 7fe768e6c700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe74c06e210 0x7fe74c0706d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.307+0000 7fe768e6c700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe764100f00 0x7fe76419c8e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.307+0000 7fe768e6c700 1 --2- 172.21.15.49:0/3683686629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7641018b0 0x7fe76419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.307+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 >> 172.21.15.49:0/3683686629 conn(0x7fe7640fc7d0 msgr2=0x7fe7640fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.308+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 shutdown_connections 2024-05-15T16:19:51.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.308+0000 7fe768e6c700 1 -- 172.21.15.49:0/3683686629 wait complete. 2024-05-15T16:19:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.414+0000 7f298549e700 1 -- 172.21.15.49:0/173344219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 msgr2=0x7f2980102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.414+0000 7f298549e700 1 --2- 172.21.15.49:0/173344219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 0x7f2980102260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2970009940 tx=0x7f2970009c50 comp rx=0 tx=0).stop 2024-05-15T16:19:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.414+0000 7f298549e700 1 -- 172.21.15.49:0/173344219 shutdown_connections 2024-05-15T16:19:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.414+0000 7f298549e700 1 --2- 172.21.15.49:0/173344219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29801027a0 0x7f298010ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.414+0000 7f298549e700 1 --2- 172.21.15.49:0/173344219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 0x7f2980102260 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.414+0000 7f298549e700 1 -- 172.21.15.49:0/173344219 >> 172.21.15.49:0/173344219 conn(0x7f2980076800 msgr2=0x7f29800fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.414+0000 7f298549e700 1 -- 172.21.15.49:0/173344219 shutdown_connections 2024-05-15T16:19:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.414+0000 7f298549e700 1 -- 172.21.15.49:0/173344219 wait complete. 2024-05-15T16:19:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.415+0000 7f298549e700 1 Processor -- start 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.415+0000 7f298549e700 1 -- start start 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.415+0000 7f298549e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 0x7f298019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.415+0000 7f298549e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29801027a0 0x7f298019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.415+0000 7f298549e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298019d5a0 con 0x7f2980101e80 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.415+0000 7f298549e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2980196b80 con 0x7f29801027a0 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.415+0000 7f297effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 0x7f298019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.415+0000 7f297effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 0x7f298019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46680/0 (socket says 172.21.15.49:46680) 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.415+0000 7f297effd700 1 -- 172.21.15.49:0/1963894132 learned_addr learned my addr 172.21.15.49:0/1963894132 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f29767fc700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29801027a0 0x7f298019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f297effd700 1 -- 172.21.15.49:0/1963894132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29801027a0 msgr2=0x7f298019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f297effd700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29801027a0 0x7f298019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f297effd700 1 -- 172.21.15.49:0/1963894132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29700095f0 con 0x7f2980101e80 2024-05-15T16:19:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f297effd700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 0x7f298019ca40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2970005620 tx=0x7f2970005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f297cff9700 1 -- 172.21.15.49:0/1963894132 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297001d070 con 0x7f2980101e80 2024-05-15T16:19:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f297cff9700 1 -- 172.21.15.49:0/1963894132 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2970003d50 con 0x7f2980101e80 2024-05-15T16:19:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f297cff9700 1 -- 172.21.15.49:0/1963894132 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2970021410 con 0x7f2980101e80 2024-05-15T16:19:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2980196e00 con 0x7f2980101e80 2024-05-15T16:19:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.416+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29801972f0 con 0x7f2980101e80 2024-05-15T16:19:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.417+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298010a970 con 0x7f2980101e80 2024-05-15T16:19:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.418+0000 7f297cff9700 1 -- 172.21.15.49:0/1963894132 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f297000f810 con 0x7f2980101e80 2024-05-15T16:19:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.418+0000 7f297cff9700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f296c06e200 0x7f296c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.419+0000 7f297cff9700 1 -- 172.21.15.49:0/1963894132 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f297008f280 con 0x7f2980101e80 2024-05-15T16:19:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.419+0000 7f29767fc700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f296c06e200 0x7f296c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.419+0000 7f29767fc700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f296c06e200 0x7f296c0706c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2980198220 tx=0x7f2968009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:51.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.423+0000 7f297cff9700 1 -- 172.21.15.49:0/1963894132 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f297005c110 con 0x7f2980101e80 2024-05-15T16:19:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.716+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2980197fd0 con 0x7f2980101e80 2024-05-15T16:19:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.717+0000 7f297cff9700 1 -- 172.21.15.49:0/1963894132 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f297005bca0 con 0x7f2980101e80 2024-05-15T16:19:51.718 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:19:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:19:51.720 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f296c06e200 msgr2=0x7f296c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f296c06e200 0x7f296c0706c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2980198220 tx=0x7f2968009190 comp rx=0 tx=0).stop 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 msgr2=0x7f298019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 0x7f298019ca40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2970005620 tx=0x7f2970005650 comp rx=0 tx=0).stop 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 shutdown_connections 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f296c06e200 0x7f296c0706c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2980101e80 0x7f298019ca40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 --2- 172.21.15.49:0/1963894132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29801027a0 0x7f298019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 >> 172.21.15.49:0/1963894132 conn(0x7f2980076800 msgr2=0x7f2980103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.719+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 shutdown_connections 2024-05-15T16:19:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.720+0000 7f298549e700 1 -- 172.21.15.49:0/1963894132 wait complete. 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.827+0000 7f8cbaae0700 1 -- 172.21.15.49:0/2737101103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 msgr2=0x7f8cb41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.828+0000 7f8cbaae0700 1 --2- 172.21.15.49:0/2737101103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 0x7f8cb41058e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009910 tx=0x7f8ca0009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.828+0000 7f8cbaae0700 1 -- 172.21.15.49:0/2737101103 shutdown_connections 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.828+0000 7f8cbaae0700 1 --2- 172.21.15.49:0/2737101103 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 0x7f8cb41058e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.828+0000 7f8cbaae0700 1 --2- 172.21.15.49:0/2737101103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb4100ee0 0x7f8cb41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.828+0000 7f8cbaae0700 1 -- 172.21.15.49:0/2737101103 >> 172.21.15.49:0/2737101103 conn(0x7f8cb40fc790 msgr2=0x7f8cb40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.828+0000 7f8cbaae0700 1 -- 172.21.15.49:0/2737101103 shutdown_connections 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.828+0000 7f8cbaae0700 1 -- 172.21.15.49:0/2737101103 wait complete. 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.828+0000 7f8cbaae0700 1 Processor -- start 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.828+0000 7f8cbaae0700 1 -- start start 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cbaae0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb4100ee0 0x7f8cb419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:51.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cbaae0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 0x7f8cb419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:51.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cbaae0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb419d3b0 con 0x7f8cb4101890 2024-05-15T16:19:51.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cbaae0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb4196a20 con 0x7f8cb4100ee0 2024-05-15T16:19:51.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cb3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 0x7f8cb419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:51.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cb3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 0x7f8cb419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46702/0 (socket says 172.21.15.49:46702) 2024-05-15T16:19:51.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cb3fff700 1 -- 172.21.15.49:0/551135803 learned_addr learned my addr 172.21.15.49:0/551135803 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:51.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cb887c700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb4100ee0 0x7f8cb419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:51.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cb3fff700 1 -- 172.21.15.49:0/551135803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb4100ee0 msgr2=0x7f8cb419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:51.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cb3fff700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb4100ee0 0x7f8cb419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:51.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cb3fff700 1 -- 172.21.15.49:0/551135803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca00095f0 con 0x7f8cb4101890 2024-05-15T16:19:51.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cb3fff700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 0x7f8cb419ce20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00092e0 tx=0x7f8ca000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:51.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.829+0000 7f8cb1ffb700 1 -- 172.21.15.49:0/551135803 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ca001d070 con 0x7f8cb4101890 2024-05-15T16:19:51.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.830+0000 7f8cb1ffb700 1 -- 172.21.15.49:0/551135803 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ca000f460 con 0x7f8cb4101890 2024-05-15T16:19:51.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.830+0000 7f8cb1ffb700 1 -- 172.21.15.49:0/551135803 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ca0017630 con 0x7f8cb4101890 2024-05-15T16:19:51.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.830+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb4196ca0 con 0x7f8cb4101890 2024-05-15T16:19:51.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.830+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb4197190 con 0x7f8cb4101890 2024-05-15T16:19:51.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.831+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb41092e0 con 0x7f8cb4101890 2024-05-15T16:19:51.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.831+0000 7f8cb1ffb700 1 -- 172.21.15.49:0/551135803 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8ca000f5d0 con 0x7f8cb4101890 2024-05-15T16:19:51.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.832+0000 7f8cb1ffb700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ca406e200 0x7f8ca40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:51.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.832+0000 7f8cb887c700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ca406e200 0x7f8ca40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:51.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.832+0000 7f8cb1ffb700 1 -- 172.21.15.49:0/551135803 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8ca008f570 con 0x7f8cb4101890 2024-05-15T16:19:51.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.832+0000 7f8cb887c700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ca406e200 0x7f8ca40706c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005ea0 tx=0x7f8ca8005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:51.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:51.837+0000 7f8cb1ffb700 1 -- 172.21.15.49:0/551135803 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8ca005c400 con 0x7f8cb4101890 2024-05-15T16:19:52.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.087+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8cb41023c0 con 0x7f8cb4101890 2024-05-15T16:19:52.089 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:51 smithi049 ceph-mon[112142]: pgmap v686: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:52.090 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:51 smithi049 ceph-mon[112142]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:52.090 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:51 smithi049 ceph-mon[112142]: from='client.35274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:52.090 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:51 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1963894132' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:19:52.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.087+0000 7f8cb1ffb700 1 -- 172.21.15.49:0/551135803 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f8ca0005740 con 0x7f8cb4101890 2024-05-15T16:19:52.091 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:19:52.091 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:19:52.091 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:19:52.091 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:19:52.091 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:19:52.092 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:19:52.092 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:19:52.092 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:19:52.092 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:19:52.092 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:19:52.092 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:19:52.093 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:19:52.093 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:19:52.093 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:19:52.093 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:19:52.093 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:19:52.093 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:19:52.094 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:19:52.094 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:19:52.094 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:19:52.094 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:19:52.094 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:19:52.094 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:19:52.095 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:19:52.095 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:19:52.095 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:19:52.095 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:19:52.095 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:19:52.095 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:19:52.096 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:19:52.096 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:19:52.096 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:19:52.096 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:19:52.097 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:19:52.097 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:19:52.097 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:19:52.097 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:19:52.097 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:19:52.097 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:19:52.098 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:19:52.098 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:19:52.098 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:19:52.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ca406e200 msgr2=0x7f8ca40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ca406e200 0x7f8ca40706c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005ea0 tx=0x7f8ca8005e30 comp rx=0 tx=0).stop 2024-05-15T16:19:52.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 msgr2=0x7f8cb419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 0x7f8cb419ce20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00092e0 tx=0x7f8ca000be20 comp rx=0 tx=0).stop 2024-05-15T16:19:52.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 shutdown_connections 2024-05-15T16:19:52.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8ca406e200 0x7f8ca40706c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb4100ee0 0x7f8cb419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 --2- 172.21.15.49:0/551135803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb4101890 0x7f8cb419ce20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 >> 172.21.15.49:0/551135803 conn(0x7f8cb40fc790 msgr2=0x7f8cb40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:52.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 shutdown_connections 2024-05-15T16:19:52.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.090+0000 7f8cbaae0700 1 -- 172.21.15.49:0/551135803 wait complete. 2024-05-15T16:19:52.101 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:19:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:51 smithi144 ceph-mon[86635]: pgmap v686: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:51 smithi144 ceph-mon[86635]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:51 smithi144 ceph-mon[86635]: from='client.35274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:51 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1963894132' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:19:52.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.201+0000 7f004f4b7700 1 -- 172.21.15.49:0/4126964259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ff3d0 msgr2=0x7f00480ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.201+0000 7f004f4b7700 1 --2- 172.21.15.49:0/4126964259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ff3d0 0x7f00480ff7b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f004400b0a0 tx=0x7f004400b3b0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.201+0000 7f004f4b7700 1 -- 172.21.15.49:0/4126964259 shutdown_connections 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.201+0000 7f004f4b7700 1 --2- 172.21.15.49:0/4126964259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00480ffd80 0x7f0048105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.201+0000 7f004f4b7700 1 --2- 172.21.15.49:0/4126964259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ff3d0 0x7f00480ff7b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.201+0000 7f004f4b7700 1 -- 172.21.15.49:0/4126964259 >> 172.21.15.49:0/4126964259 conn(0x7f0048075ff0 msgr2=0x7f00480fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.201+0000 7f004f4b7700 1 -- 172.21.15.49:0/4126964259 shutdown_connections 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.201+0000 7f004f4b7700 1 -- 172.21.15.49:0/4126964259 wait complete. 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.202+0000 7f004f4b7700 1 Processor -- start 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.202+0000 7f004f4b7700 1 -- start start 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.202+0000 7f004f4b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00480ff3d0 0x7f0048100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:52.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.202+0000 7f004f4b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ffd80 0x7f0048100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.202+0000 7f004f4b7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0048100dd0 con 0x7f00480ffd80 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.202+0000 7f004f4b7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0048100f10 con 0x7f00480ff3d0 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.202+0000 7f004ca52700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ffd80 0x7f0048100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.202+0000 7f004ca52700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ffd80 0x7f0048100890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46716/0 (socket says 172.21.15.49:46716) 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.202+0000 7f004ca52700 1 -- 172.21.15.49:0/3262878940 learned_addr learned my addr 172.21.15.49:0/3262878940 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f004d253700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00480ff3d0 0x7f0048100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f004ca52700 1 -- 172.21.15.49:0/3262878940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00480ff3d0 msgr2=0x7f0048100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f004ca52700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00480ff3d0 0x7f0048100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f004ca52700 1 -- 172.21.15.49:0/3262878940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0044009d00 con 0x7f00480ffd80 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f004ca52700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ffd80 0x7f0048100890 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f003800d8d0 tx=0x7f003800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f003e7fc700 1 -- 172.21.15.49:0/3262878940 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0038009900 con 0x7f00480ffd80 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f003e7fc700 1 -- 172.21.15.49:0/3262878940 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0038010460 con 0x7f00480ffd80 2024-05-15T16:19:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f003e7fc700 1 -- 172.21.15.49:0/3262878940 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f003800f5f0 con 0x7f00480ffd80 2024-05-15T16:19:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00481a2dd0 con 0x7f00480ffd80 2024-05-15T16:19:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.203+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00481a3130 con 0x7f00480ffd80 2024-05-15T16:19:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.204+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00481092b0 con 0x7f00480ffd80 2024-05-15T16:19:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.205+0000 7f003e7fc700 1 -- 172.21.15.49:0/3262878940 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f00380105d0 con 0x7f00480ffd80 2024-05-15T16:19:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.205+0000 7f003e7fc700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f003406e200 0x7f00340706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.205+0000 7f004d253700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f003406e200 0x7f00340706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.205+0000 7f003e7fc700 1 -- 172.21.15.49:0/3262878940 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f003808dd50 con 0x7f00480ffd80 2024-05-15T16:19:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.206+0000 7f004d253700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f003406e200 0x7f00340706c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f00440062a0 tx=0x7f00440061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:52.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.210+0000 7f003e7fc700 1 -- 172.21.15.49:0/3262878940 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f00380570b0 con 0x7f00480ffd80 2024-05-15T16:19:52.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.438+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0048102460 con 0x7f003406e200 2024-05-15T16:19:52.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.439+0000 7f003e7fc700 1 -- 172.21.15.49:0/3262878940 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0048102460 con 0x7f003406e200 2024-05-15T16:19:52.441 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:19:52.441 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:19:52.441 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:19:52.441 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:19:52.441 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:19:52.441 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:19:52.441 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:19:52.441 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:19:52.442 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:19:52.442 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:19:52.442 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:19:52.442 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:19:52.442 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:19:52.442 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:19:52.442 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:19:52.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f003406e200 msgr2=0x7f00340706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f003406e200 0x7f00340706c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f00440062a0 tx=0x7f00440061f0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ffd80 msgr2=0x7f0048100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ffd80 0x7f0048100890 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f003800d8d0 tx=0x7f003800dc90 comp rx=0 tx=0).stop 2024-05-15T16:19:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 shutdown_connections 2024-05-15T16:19:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f003406e200 0x7f00340706c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00480ff3d0 0x7f0048100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 --2- 172.21.15.49:0/3262878940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00480ffd80 0x7f0048100890 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 >> 172.21.15.49:0/3262878940 conn(0x7f0048075ff0 msgr2=0x7f0048104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 shutdown_connections 2024-05-15T16:19:52.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.442+0000 7f004f4b7700 1 -- 172.21.15.49:0/3262878940 wait complete. 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.551+0000 7f91b0fcc700 1 -- 172.21.15.49:0/3929597304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 msgr2=0x7f91ac103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.551+0000 7f91b0fcc700 1 --2- 172.21.15.49:0/3929597304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 0x7f91ac103fa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009910 tx=0x7f91a0009c20 comp rx=0 tx=0).stop 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.551+0000 7f91b0fcc700 1 -- 172.21.15.49:0/3929597304 shutdown_connections 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.551+0000 7f91b0fcc700 1 --2- 172.21.15.49:0/3929597304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 0x7f91ac103fa0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.551+0000 7f91b0fcc700 1 --2- 172.21.15.49:0/3929597304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.551+0000 7f91b0fcc700 1 -- 172.21.15.49:0/3929597304 >> 172.21.15.49:0/3929597304 conn(0x7f91ac076000 msgr2=0x7f91ac0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.551+0000 7f91b0fcc700 1 -- 172.21.15.49:0/3929597304 shutdown_connections 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.551+0000 7f91b0fcc700 1 -- 172.21.15.49:0/3929597304 wait complete. 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91b0fcc700 1 Processor -- start 2024-05-15T16:19:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91b0fcc700 1 -- start start 2024-05-15T16:19:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91b0fcc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91b0fcc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 0x7f91ac197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91b0fcc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac198560 con 0x7f91ac0fff50 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91b0fcc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac1986a0 con 0x7f91ac0ff5a0 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91a9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 0x7f91ac197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91a9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 0x7f91ac197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46734/0 (socket says 172.21.15.49:46734) 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91a9d9b700 1 -- 172.21.15.49:0/4119711309 learned_addr learned my addr 172.21.15.49:0/4119711309 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91aa59c700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91a9d9b700 1 -- 172.21.15.49:0/4119711309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ac0ff5a0 msgr2=0x7f91ac197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91a9d9b700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91a9d9b700 1 -- 172.21.15.49:0/4119711309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a00095f0 con 0x7f91ac0fff50 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.552+0000 7f91aa59c700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.553+0000 7f91a9d9b700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 0x7f91ac197f90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f91a000bd70 tx=0x7f91a000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.553+0000 7f919b7fe700 1 -- 172.21.15.49:0/4119711309 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91a001d070 con 0x7f91ac0fff50 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.553+0000 7f919b7fe700 1 -- 172.21.15.49:0/4119711309 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91a000f460 con 0x7f91ac0fff50 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.553+0000 7f919b7fe700 1 -- 172.21.15.49:0/4119711309 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91a0017630 con 0x7f91ac0fff50 2024-05-15T16:19:52.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.553+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ac1a0ac0 con 0x7f91ac0fff50 2024-05-15T16:19:52.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.553+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac1a0e20 con 0x7f91ac0fff50 2024-05-15T16:19:52.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.554+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ac100a80 con 0x7f91ac0fff50 2024-05-15T16:19:52.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.554+0000 7f919b7fe700 1 -- 172.21.15.49:0/4119711309 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f91a000f5d0 con 0x7f91ac0fff50 2024-05-15T16:19:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.555+0000 7f919b7fe700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f919406e200 0x7f91940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:19:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.555+0000 7f919b7fe700 1 -- 172.21.15.49:0/4119711309 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f91a008f560 con 0x7f91ac0fff50 2024-05-15T16:19:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.555+0000 7f91aa59c700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f919406e200 0x7f91940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:19:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.555+0000 7f91aa59c700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f919406e200 0x7f91940706c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f919c005fd0 tx=0x7f919c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:19:52.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.560+0000 7f919b7fe700 1 -- 172.21.15.49:0/4119711309 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f91a00588c0 con 0x7f91ac0fff50 2024-05-15T16:19:52.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.854+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f91ac10c540 con 0x7f91ac0fff50 2024-05-15T16:19:52.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.855+0000 7f919b7fe700 1 -- 172.21.15.49:0/4119711309 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f91a0005740 con 0x7f91ac0fff50 2024-05-15T16:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:19:52.856 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:19:52.857 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:19:52.858 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:19:52.858 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:19:52.858 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:19:52.858 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:19:52.858 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:19:52.858 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:19:52.858 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:19:52.858 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f919406e200 msgr2=0x7f91940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f919406e200 0x7f91940706c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f919c005fd0 tx=0x7f919c005dc0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 msgr2=0x7f91ac197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 0x7f91ac197f90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f91a000bd70 tx=0x7f91a000be50 comp rx=0 tx=0).stop 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 shutdown_connections 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f919406e200 0x7f91940706c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ac0ff5a0 0x7f91ac197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 --2- 172.21.15.49:0/4119711309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91ac0fff50 0x7f91ac197f90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 >> 172.21.15.49:0/4119711309 conn(0x7f91ac076000 msgr2=0x7f91ac0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 shutdown_connections 2024-05-15T16:19:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:19:52.857+0000 7f91b0fcc700 1 -- 172.21.15.49:0/4119711309 wait complete. 2024-05-15T16:19:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:52 smithi144 ceph-mon[86635]: from='client.35278 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:52 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/551135803' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:19:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:52 smithi049 ceph-mon[112142]: from='client.35278 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:52 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/551135803' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:19:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:53 smithi144 ceph-mon[86635]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:53 smithi144 ceph-mon[86635]: pgmap v687: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:53 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4119711309' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:19:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:53 smithi049 ceph-mon[112142]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:19:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:53 smithi049 ceph-mon[112142]: pgmap v687: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:53 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4119711309' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:19:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:55 smithi144 ceph-mon[86635]: pgmap v688: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:55 smithi049 ceph-mon[112142]: pgmap v688: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:57 smithi144 ceph-mon[86635]: pgmap v689: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:19:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:57 smithi049 ceph-mon[112142]: pgmap v689: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:19:59 smithi144 ceph-mon[86635]: pgmap v690: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:19:59 smithi049 ceph-mon[112142]: pgmap v690: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-05-15T16:20:00.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:20:00.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:20:00.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:20:00.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:19:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-05-15T16:20:01.108 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:20:01.108 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:20:01.108 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:20:01.108 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: /usr/bin/podman: timeout after 895 seconds 2024-05-15T16:20:01.108 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:01.108 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: Traceback (most recent call last): 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: return _run_code(code, main_globals, None, 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: exec(code, run_globals) 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:01.109 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:20:01.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: fs cephfs is degraded 2024-05-15T16:20:01.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:20:01.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:20:01.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:20:01.110 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:00 smithi049 ceph-mon[112142]: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:20:01.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:20:01.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: /usr/bin/podman: timeout after 895 seconds 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: Traceback (most recent call last): 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: return _run_code(code, main_globals, None, 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: exec(code, run_globals) 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:20:01.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:20:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:20:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:20:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: fs cephfs is degraded 2024-05-15T16:20:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:20:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:20:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:20:01.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:00 smithi144 ceph-mon[86635]: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:20:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:01 smithi144 ceph-mon[86635]: pgmap v691: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:01 smithi049 ceph-mon[112142]: pgmap v691: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:03.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:03 smithi049 ceph-mon[112142]: pgmap v692: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:03 smithi144 ceph-mon[86635]: pgmap v692: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:05 smithi144 ceph-mon[86635]: pgmap v693: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:20:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:05 smithi049 ceph-mon[112142]: pgmap v693: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:20:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:07 smithi144 ceph-mon[86635]: pgmap v694: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:07 smithi049 ceph-mon[112142]: pgmap v694: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:09 smithi144 ceph-mon[86635]: pgmap v695: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:09 smithi049 ceph-mon[112142]: pgmap v695: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:11 smithi144 ceph-mon[86635]: pgmap v696: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:11 smithi049 ceph-mon[112142]: pgmap v696: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:13.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:13 smithi049 ceph-mon[112142]: pgmap v697: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:13 smithi144 ceph-mon[86635]: pgmap v697: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:15 smithi144 ceph-mon[86635]: pgmap v698: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:15 smithi049 ceph-mon[112142]: pgmap v698: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:17 smithi144 ceph-mon[86635]: pgmap v699: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:17 smithi049 ceph-mon[112142]: pgmap v699: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:19 smithi144 ceph-mon[86635]: pgmap v700: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:19 smithi049 ceph-mon[112142]: pgmap v700: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:21.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:20:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:20:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:21 smithi144 ceph-mon[86635]: pgmap v701: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:21 smithi049 ceph-mon[112142]: pgmap v701: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:22.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.980+0000 7ff8e0b68700 1 -- 172.21.15.49:0/1176819427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc0ffdd0 msgr2=0x7ff8dc1001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:22.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.980+0000 7ff8e0b68700 1 --2- 172.21.15.49:0/1176819427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc0ffdd0 0x7ff8dc1001b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009910 tx=0x7ff8cc009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:22.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.980+0000 7ff8e0b68700 1 -- 172.21.15.49:0/1176819427 shutdown_connections 2024-05-15T16:20:22.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.980+0000 7ff8e0b68700 1 --2- 172.21.15.49:0/1176819427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc100780 0x7ff8dc1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:22.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.980+0000 7ff8e0b68700 1 --2- 172.21.15.49:0/1176819427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc0ffdd0 0x7ff8dc1001b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:22.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.980+0000 7ff8e0b68700 1 -- 172.21.15.49:0/1176819427 >> 172.21.15.49:0/1176819427 conn(0x7ff8dc0fb6a0 msgr2=0x7ff8dc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:22.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.980+0000 7ff8e0b68700 1 -- 172.21.15.49:0/1176819427 shutdown_connections 2024-05-15T16:20:22.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.981+0000 7ff8e0b68700 1 -- 172.21.15.49:0/1176819427 wait complete. 2024-05-15T16:20:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.981+0000 7ff8e0b68700 1 Processor -- start 2024-05-15T16:20:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.981+0000 7ff8e0b68700 1 -- start start 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8e0b68700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc0ffdd0 0x7ff8dc198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8e0b68700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc100780 0x7ff8dc1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8e0b68700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc1999c0 con 0x7ff8dc100780 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8e0b68700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc19d690 con 0x7ff8dc0ffdd0 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8d9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc100780 0x7ff8dc1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8d9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc100780 0x7ff8dc1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49886/0 (socket says 172.21.15.49:49886) 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8d9d9b700 1 -- 172.21.15.49:0/3324528173 learned_addr learned my addr 172.21.15.49:0/3324528173 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8da59c700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc0ffdd0 0x7ff8dc198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8d9d9b700 1 -- 172.21.15.49:0/3324528173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc0ffdd0 msgr2=0x7ff8dc198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8d9d9b700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc0ffdd0 0x7ff8dc198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8d9d9b700 1 -- 172.21.15.49:0/3324528173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc0095f0 con 0x7ff8dc100780 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8d9d9b700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc100780 0x7ff8dc1992e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000d900 tx=0x7ff8d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.982+0000 7ff8cb7fe700 1 -- 172.21.15.49:0/3324528173 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d0009900 con 0x7ff8dc100780 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.983+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8dc19d970 con 0x7ff8dc100780 2024-05-15T16:20:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.983+0000 7ff8cb7fe700 1 -- 172.21.15.49:0/3324528173 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8d0010460 con 0x7ff8dc100780 2024-05-15T16:20:22.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.983+0000 7ff8cb7fe700 1 -- 172.21.15.49:0/3324528173 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d000f5f0 con 0x7ff8dc100780 2024-05-15T16:20:22.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.983+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc19dec0 con 0x7ff8dc100780 2024-05-15T16:20:22.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.984+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8dc1012b0 con 0x7ff8dc100780 2024-05-15T16:20:22.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.984+0000 7ff8cb7fe700 1 -- 172.21.15.49:0/3324528173 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff8d00105d0 con 0x7ff8dc100780 2024-05-15T16:20:22.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.985+0000 7ff8cb7fe700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff8c406e210 0x7ff8c40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:22.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.985+0000 7ff8cb7fe700 1 -- 172.21.15.49:0/3324528173 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff8d008dff0 con 0x7ff8dc100780 2024-05-15T16:20:22.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.985+0000 7ff8da59c700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff8c406e210 0x7ff8c40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:22.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.985+0000 7ff8da59c700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff8c406e210 0x7ff8c40706d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00b5c0 tx=0x7ff8cc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:22.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:22.990+0000 7ff8cb7fe700 1 -- 172.21.15.49:0/3324528173 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff8d00572d0 con 0x7ff8dc100780 2024-05-15T16:20:23.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.215+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8dc10ce00 con 0x7ff8c406e210 2024-05-15T16:20:23.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.217+0000 7ff8cb7fe700 1 -- 172.21.15.49:0/3324528173 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff8dc10ce00 con 0x7ff8c406e210 2024-05-15T16:20:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff8c406e210 msgr2=0x7ff8c40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff8c406e210 0x7ff8c40706d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00b5c0 tx=0x7ff8cc01a040 comp rx=0 tx=0).stop 2024-05-15T16:20:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc100780 msgr2=0x7ff8dc1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc100780 0x7ff8dc1992e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000d900 tx=0x7ff8d000dcc0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 shutdown_connections 2024-05-15T16:20:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff8c406e210 0x7ff8c40706d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc0ffdd0 0x7ff8dc198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 --2- 172.21.15.49:0/3324528173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8dc100780 0x7ff8dc1992e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 >> 172.21.15.49:0/3324528173 conn(0x7ff8dc0fb6a0 msgr2=0x7ff8dc0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 shutdown_connections 2024-05-15T16:20:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.219+0000 7ff8e0b68700 1 -- 172.21.15.49:0/3324528173 wait complete. 2024-05-15T16:20:23.231 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:20:23.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.329+0000 7ff46699d700 1 -- 172.21.15.49:0/3865239726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460106b80 msgr2=0x7ff460106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.329+0000 7ff46699d700 1 --2- 172.21.15.49:0/3865239726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460106b80 0x7ff460106f60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009910 tx=0x7ff45c009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:23.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.329+0000 7ff46699d700 1 -- 172.21.15.49:0/3865239726 shutdown_connections 2024-05-15T16:20:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.329+0000 7ff46699d700 1 --2- 172.21.15.49:0/3865239726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff460100af0 0x7ff460100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.329+0000 7ff46699d700 1 --2- 172.21.15.49:0/3865239726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460106b80 0x7ff460106f60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.329+0000 7ff46699d700 1 -- 172.21.15.49:0/3865239726 >> 172.21.15.49:0/3865239726 conn(0x7ff4600fc5e0 msgr2=0x7ff4600fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.330+0000 7ff46699d700 1 -- 172.21.15.49:0/3865239726 shutdown_connections 2024-05-15T16:20:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.330+0000 7ff46699d700 1 -- 172.21.15.49:0/3865239726 wait complete. 2024-05-15T16:20:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.330+0000 7ff46699d700 1 Processor -- start 2024-05-15T16:20:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.330+0000 7ff46699d700 1 -- start start 2024-05-15T16:20:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46699d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460100af0 0x7ff4600732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46699d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff460106b80 0x7ff4600737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46699d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460073d20 con 0x7ff460100af0 2024-05-15T16:20:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46699d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460073e60 con 0x7ff460106b80 2024-05-15T16:20:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46599b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460100af0 0x7ff4600732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46599b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460100af0 0x7ff4600732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49904/0 (socket says 172.21.15.49:49904) 2024-05-15T16:20:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46599b700 1 -- 172.21.15.49:0/2487834465 learned_addr learned my addr 172.21.15.49:0/2487834465 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46519a700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff460106b80 0x7ff4600737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46599b700 1 -- 172.21.15.49:0/2487834465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff460106b80 msgr2=0x7ff4600737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46599b700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff460106b80 0x7ff4600737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46599b700 1 -- 172.21.15.49:0/2487834465 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0095f0 con 0x7ff460100af0 2024-05-15T16:20:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46599b700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460100af0 0x7ff4600732a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff45c005f10 tx=0x7ff45c00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff456ffd700 1 -- 172.21.15.49:0/2487834465 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff45c01d070 con 0x7ff460100af0 2024-05-15T16:20:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff456ffd700 1 -- 172.21.15.49:0/2487834465 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff45c0052a0 con 0x7ff460100af0 2024-05-15T16:20:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff456ffd700 1 -- 172.21.15.49:0/2487834465 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff45c003e50 con 0x7ff460100af0 2024-05-15T16:20:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.331+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4600740e0 con 0x7ff460100af0 2024-05-15T16:20:23.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.332+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4601a2e50 con 0x7ff460100af0 2024-05-15T16:20:23.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.333+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4601090f0 con 0x7ff460100af0 2024-05-15T16:20:23.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.333+0000 7ff456ffd700 1 -- 172.21.15.49:0/2487834465 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff45c00f6f0 con 0x7ff460100af0 2024-05-15T16:20:23.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.334+0000 7ff456ffd700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44c06e200 0x7ff44c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:23.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.334+0000 7ff456ffd700 1 -- 172.21.15.49:0/2487834465 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff45c08f420 con 0x7ff460100af0 2024-05-15T16:20:23.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.334+0000 7ff46519a700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44c06e200 0x7ff44c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:23.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.334+0000 7ff46519a700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44c06e200 0x7ff44c0706c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff4600749d0 tx=0x7ff450009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:23.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.339+0000 7ff456ffd700 1 -- 172.21.15.49:0/2487834465 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff45c058780 con 0x7ff460100af0 2024-05-15T16:20:23.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.563+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff460074860 con 0x7ff44c06e200 2024-05-15T16:20:23.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.564+0000 7ff456ffd700 1 -- 172.21.15.49:0/2487834465 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff460074860 con 0x7ff44c06e200 2024-05-15T16:20:23.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.566+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44c06e200 msgr2=0x7ff44c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.566+0000 7ff46699d700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44c06e200 0x7ff44c0706c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff4600749d0 tx=0x7ff450009260 comp rx=0 tx=0).stop 2024-05-15T16:20:23.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.566+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460100af0 msgr2=0x7ff4600732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.566+0000 7ff46699d700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460100af0 0x7ff4600732a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff45c005f10 tx=0x7ff45c00b5a0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.566+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 shutdown_connections 2024-05-15T16:20:23.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.566+0000 7ff46699d700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff44c06e200 0x7ff44c0706c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.566+0000 7ff46699d700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff460100af0 0x7ff4600732a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.566+0000 7ff46699d700 1 --2- 172.21.15.49:0/2487834465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff460106b80 0x7ff4600737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.566+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 >> 172.21.15.49:0/2487834465 conn(0x7ff4600fc5e0 msgr2=0x7ff4600fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:23.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.567+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 shutdown_connections 2024-05-15T16:20:23.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.567+0000 7ff46699d700 1 -- 172.21.15.49:0/2487834465 wait complete. 2024-05-15T16:20:23.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.675+0000 7ff129f48700 1 -- 172.21.15.49:0/3125555678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 msgr2=0x7ff124104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.675+0000 7ff129f48700 1 --2- 172.21.15.49:0/3125555678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 0x7ff124104870 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009910 tx=0x7ff10c009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:23.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.675+0000 7ff129f48700 1 -- 172.21.15.49:0/3125555678 shutdown_connections 2024-05-15T16:20:23.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.675+0000 7ff129f48700 1 --2- 172.21.15.49:0/3125555678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 0x7ff124104870 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.675+0000 7ff129f48700 1 --2- 172.21.15.49:0/3125555678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1240ffe70 0x7ff124100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.675+0000 7ff129f48700 1 -- 172.21.15.49:0/3125555678 >> 172.21.15.49:0/3125555678 conn(0x7ff1240fb760 msgr2=0x7ff1240fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:23.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.675+0000 7ff129f48700 1 -- 172.21.15.49:0/3125555678 shutdown_connections 2024-05-15T16:20:23.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.675+0000 7ff129f48700 1 -- 172.21.15.49:0/3125555678 wait complete. 2024-05-15T16:20:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff129f48700 1 Processor -- start 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff129f48700 1 -- start start 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff129f48700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1240ffe70 0x7ff124198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff129f48700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 0x7ff1241993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff129f48700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff124199a80 con 0x7ff124100820 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff129f48700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12419d750 con 0x7ff1240ffe70 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff122ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 0x7ff1241993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff122ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 0x7ff1241993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53766/0 (socket says 172.21.15.49:53766) 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff122ffd700 1 -- 172.21.15.49:0/3869866437 learned_addr learned my addr 172.21.15.49:0/3869866437 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.676+0000 7ff1237fe700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1240ffe70 0x7ff124198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff122ffd700 1 -- 172.21.15.49:0/3869866437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1240ffe70 msgr2=0x7ff124198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff122ffd700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1240ffe70 0x7ff124198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff122ffd700 1 -- 172.21.15.49:0/3869866437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c0095f0 con 0x7ff124100820 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff1237fe700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1240ffe70 0x7ff124198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T16:20:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff122ffd700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 0x7ff1241993a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00bd70 tx=0x7ff10c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff120ff9700 1 -- 172.21.15.49:0/3869866437 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff10c01d070 con 0x7ff124100820 2024-05-15T16:20:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff120ff9700 1 -- 172.21.15.49:0/3869866437 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff10c0056e0 con 0x7ff124100820 2024-05-15T16:20:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12419d9d0 con 0x7ff124100820 2024-05-15T16:20:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff120ff9700 1 -- 172.21.15.49:0/3869866437 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff10c00fe50 con 0x7ff124100820 2024-05-15T16:20:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.677+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12419dec0 con 0x7ff124100820 2024-05-15T16:20:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.679+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff124101350 con 0x7ff124100820 2024-05-15T16:20:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.679+0000 7ff120ff9700 1 -- 172.21.15.49:0/3869866437 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff10c00f680 con 0x7ff124100820 2024-05-15T16:20:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.679+0000 7ff120ff9700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff11006e200 0x7ff1100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.680+0000 7ff120ff9700 1 -- 172.21.15.49:0/3869866437 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff10c08f570 con 0x7ff124100820 2024-05-15T16:20:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.680+0000 7ff1237fe700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff11006e200 0x7ff1100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.680+0000 7ff1237fe700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff11006e200 0x7ff1100706c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff114009fd0 tx=0x7ff1140091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.684+0000 7ff120ff9700 1 -- 172.21.15.49:0/3869866437 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff10c0587b0 con 0x7ff124100820 2024-05-15T16:20:23.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.903+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff12410ce10 con 0x7ff11006e200 2024-05-15T16:20:23.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.914+0000 7ff120ff9700 1 -- 172.21.15.49:0/3869866437 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff12410ce10 con 0x7ff11006e200 2024-05-15T16:20:23.916 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:20:23.916 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (23m) 88s ago 35m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:20:23.916 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (35m) 88s ago 35m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:20:23.916 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (34m) 90s ago 34m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:20:23.916 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (21m) 88s ago 35m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:20:23.916 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (20m) 90s ago 34m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:20:23.916 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (23m) 88s ago 34m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:20:23.916 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (17m) 88s ago 29m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (16m) 88s ago 29m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (16m) 90s ago 29m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (94s) 90s ago 29m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (25m) 88s ago 36m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (24m) 90s ago 34m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (22m) 88s ago 36m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (21m) 90s ago 34m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (24m) 88s ago 35m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (24m) 90s ago 34m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (20m) 88s ago 33m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (19m) 88s ago 32m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (19m) 88s ago 32m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (18m) 90s ago 31m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (18m) 90s ago 31m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:20:23.917 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (17m) 90s ago 31m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:20:23.918 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (23m) 88s ago 34m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:20:23.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff11006e200 msgr2=0x7ff1100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff11006e200 0x7ff1100706c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff114009fd0 tx=0x7ff1140091b0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 msgr2=0x7ff1241993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 0x7ff1241993a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00bd70 tx=0x7ff10c00bda0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 shutdown_connections 2024-05-15T16:20:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff11006e200 0x7ff1100706c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1240ffe70 0x7ff124198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 --2- 172.21.15.49:0/3869866437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff124100820 0x7ff1241993a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 >> 172.21.15.49:0/3869866437 conn(0x7ff1240fb760 msgr2=0x7ff1240fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 shutdown_connections 2024-05-15T16:20:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:23.917+0000 7ff129f48700 1 -- 172.21.15.49:0/3869866437 wait complete. 2024-05-15T16:20:23.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:23 smithi049 ceph-mon[112142]: pgmap v702: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:24.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.025+0000 7ff2ecace700 1 -- 172.21.15.49:0/2085167668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 msgr2=0x7ff2e81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.025+0000 7ff2ecace700 1 --2- 172.21.15.49:0/2085167668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 0x7ff2e81012e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009910 tx=0x7ff2d8009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.025+0000 7ff2ecace700 1 -- 172.21.15.49:0/2085167668 shutdown_connections 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.025+0000 7ff2ecace700 1 --2- 172.21.15.49:0/2085167668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e81018b0 0x7ff2e8105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.025+0000 7ff2ecace700 1 --2- 172.21.15.49:0/2085167668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 0x7ff2e81012e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.025+0000 7ff2ecace700 1 -- 172.21.15.49:0/2085167668 >> 172.21.15.49:0/2085167668 conn(0x7ff2e80fc7d0 msgr2=0x7ff2e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.025+0000 7ff2ecace700 1 -- 172.21.15.49:0/2085167668 shutdown_connections 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.025+0000 7ff2ecace700 1 -- 172.21.15.49:0/2085167668 wait complete. 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.026+0000 7ff2ecace700 1 Processor -- start 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.026+0000 7ff2ecace700 1 -- start start 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.026+0000 7ff2ecace700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 0x7ff2e80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.026+0000 7ff2ecace700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e81018b0 0x7ff2e80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:24.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.026+0000 7ff2ecace700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e8073d20 con 0x7ff2e8100f00 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.026+0000 7ff2ecace700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e8073e60 con 0x7ff2e81018b0 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.026+0000 7ff2e659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 0x7ff2e80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.026+0000 7ff2e659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 0x7ff2e80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53788/0 (socket says 172.21.15.49:53788) 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.026+0000 7ff2e659c700 1 -- 172.21.15.49:0/736565660 learned_addr learned my addr 172.21.15.49:0/736565660 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2e5d9b700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e81018b0 0x7ff2e80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2e659c700 1 -- 172.21.15.49:0/736565660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e81018b0 msgr2=0x7ff2e80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2e659c700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e81018b0 0x7ff2e80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2e659c700 1 -- 172.21.15.49:0/736565660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80095f0 con 0x7ff2e8100f00 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2e5d9b700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e81018b0 0x7ff2e80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2e659c700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 0x7ff2e80732a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800bd70 tx=0x7ff2d800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2d77fe700 1 -- 172.21.15.49:0/736565660 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2d801d070 con 0x7ff2e8100f00 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e80740e0 con 0x7ff2e8100f00 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2d77fe700 1 -- 172.21.15.49:0/736565660 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2d800f460 con 0x7ff2e8100f00 2024-05-15T16:20:24.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2d77fe700 1 -- 172.21.15.49:0/736565660 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2d8017630 con 0x7ff2e8100f00 2024-05-15T16:20:24.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.027+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e81a3040 con 0x7ff2e8100f00 2024-05-15T16:20:24.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.028+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e81092a0 con 0x7ff2e8100f00 2024-05-15T16:20:24.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.029+0000 7ff2d77fe700 1 -- 172.21.15.49:0/736565660 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff2d800f5d0 con 0x7ff2e8100f00 2024-05-15T16:20:24.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.029+0000 7ff2d77fe700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff2d006e200 0x7ff2d00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:24.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.029+0000 7ff2d77fe700 1 -- 172.21.15.49:0/736565660 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff2d808f670 con 0x7ff2e8100f00 2024-05-15T16:20:24.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.029+0000 7ff2e5d9b700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff2d006e200 0x7ff2d00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:24.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.030+0000 7ff2e5d9b700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff2d006e200 0x7ff2d00706c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8074980 tx=0x7ff2dc005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:24.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.034+0000 7ff2d77fe700 1 -- 172.21.15.49:0/736565660 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff2d8058950 con 0x7ff2e8100f00 2024-05-15T16:20:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:23 smithi144 ceph-mon[86635]: pgmap v702: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:24.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.325+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff2e80747f0 con 0x7ff2e8100f00 2024-05-15T16:20:24.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.325+0000 7ff2d77fe700 1 -- 172.21.15.49:0/736565660 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff2d805c010 con 0x7ff2e8100f00 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:20:24.327 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:20:24.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:20:24.328 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:20:24.328 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:20:24.328 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:20:24.328 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:20:24.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff2d006e200 msgr2=0x7ff2d00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff2d006e200 0x7ff2d00706c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8074980 tx=0x7ff2dc005f50 comp rx=0 tx=0).stop 2024-05-15T16:20:24.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 msgr2=0x7ff2e80732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 0x7ff2e80732a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800bd70 tx=0x7ff2d800be50 comp rx=0 tx=0).stop 2024-05-15T16:20:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 shutdown_connections 2024-05-15T16:20:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff2d006e200 0x7ff2d00706c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e8100f00 0x7ff2e80732a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 --2- 172.21.15.49:0/736565660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e81018b0 0x7ff2e80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 >> 172.21.15.49:0/736565660 conn(0x7ff2e80fc7d0 msgr2=0x7ff2e80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 shutdown_connections 2024-05-15T16:20:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.328+0000 7ff2ecace700 1 -- 172.21.15.49:0/736565660 wait complete. 2024-05-15T16:20:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.436+0000 7fb09a886700 1 -- 172.21.15.49:0/1528269511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 msgr2=0x7fb094103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.436+0000 7fb09a886700 1 --2- 172.21.15.49:0/1528269511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 0x7fb094103f60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb088009910 tx=0x7fb088009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.436+0000 7fb09a886700 1 -- 172.21.15.49:0/1528269511 shutdown_connections 2024-05-15T16:20:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.436+0000 7fb09a886700 1 --2- 172.21.15.49:0/1528269511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 0x7fb094103f60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.436+0000 7fb09a886700 1 --2- 172.21.15.49:0/1528269511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0940ff560 0x7fb0940ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.436+0000 7fb09a886700 1 -- 172.21.15.49:0/1528269511 >> 172.21.15.49:0/1528269511 conn(0x7fb094075ff0 msgr2=0x7fb0940fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.436+0000 7fb09a886700 1 -- 172.21.15.49:0/1528269511 shutdown_connections 2024-05-15T16:20:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.436+0000 7fb09a886700 1 -- 172.21.15.49:0/1528269511 wait complete. 2024-05-15T16:20:24.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.437+0000 7fb09a886700 1 Processor -- start 2024-05-15T16:20:24.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.437+0000 7fb09a886700 1 -- start start 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.437+0000 7fb09a886700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0940ff560 0x7fb09419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.437+0000 7fb09a886700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 0x7fb09419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.437+0000 7fb09a886700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09419d440 con 0x7fb0940fff10 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.437+0000 7fb09a886700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094196a20 con 0x7fb0940ff560 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0937fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 0x7fb09419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0937fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 0x7fb09419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53792/0 (socket says 172.21.15.49:53792) 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0937fe700 1 -- 172.21.15.49:0/986882516 learned_addr learned my addr 172.21.15.49:0/986882516 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb093fff700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0940ff560 0x7fb09419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0937fe700 1 -- 172.21.15.49:0/986882516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0940ff560 msgr2=0x7fb09419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0937fe700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0940ff560 0x7fb09419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0937fe700 1 -- 172.21.15.49:0/986882516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0880095f0 con 0x7fb0940fff10 2024-05-15T16:20:24.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0937fe700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 0x7fb09419ce20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb088000c00 tx=0x7fb08800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0917fa700 1 -- 172.21.15.49:0/986882516 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb08801d070 con 0x7fb0940fff10 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb094196ca0 con 0x7fb0940fff10 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0917fa700 1 -- 172.21.15.49:0/986882516 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb088005700 con 0x7fb0940fff10 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb0917fa700 1 -- 172.21.15.49:0/986882516 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb08800fda0 con 0x7fb0940fff10 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.438+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb094197190 con 0x7fb0940fff10 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.440+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb094100a40 con 0x7fb0940fff10 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.440+0000 7fb0917fa700 1 -- 172.21.15.49:0/986882516 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb08800f5d0 con 0x7fb0940fff10 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.440+0000 7fb0917fa700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb07c06e210 0x7fb07c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.441+0000 7fb093fff700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb07c06e210 0x7fb07c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:24.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.441+0000 7fb0917fa700 1 -- 172.21.15.49:0/986882516 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb08808f510 con 0x7fb0940fff10 2024-05-15T16:20:24.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.441+0000 7fb093fff700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb07c06e210 0x7fb07c0706d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb084004630 tx=0x7fb08400b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:24.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.445+0000 7fb0917fa700 1 -- 172.21.15.49:0/986882516 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb08805c320 con 0x7fb0940fff10 2024-05-15T16:20:24.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.713+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb09404ec60 con 0x7fb0940fff10 2024-05-15T16:20:24.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.714+0000 7fb0917fa700 1 -- 172.21.15.49:0/986882516 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fb08805beb0 con 0x7fb0940fff10 2024-05-15T16:20:24.715 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:20:24.716 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:20:24.717 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:20:24.718 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:20:24.719 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:20:24.719 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:20:24.719 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:20:24.719 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:20:24.719 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:20:24.719 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:20:24.719 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:20:24.719 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:20:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb07c06e210 msgr2=0x7fb07c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb07c06e210 0x7fb07c0706d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb084004630 tx=0x7fb08400b3c0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 msgr2=0x7fb09419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 0x7fb09419ce20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb088000c00 tx=0x7fb08800bd70 comp rx=0 tx=0).stop 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 shutdown_connections 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb07c06e210 0x7fb07c0706d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0940ff560 0x7fb09419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 --2- 172.21.15.49:0/986882516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0940fff10 0x7fb09419ce20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 >> 172.21.15.49:0/986882516 conn(0x7fb094075ff0 msgr2=0x7fb09410ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 shutdown_connections 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.716+0000 7fb09a886700 1 -- 172.21.15.49:0/986882516 wait complete. 2024-05-15T16:20:24.721 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:20:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.826+0000 7fb987e68700 1 -- 172.21.15.49:0/556270571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9801027d0 msgr2=0x7fb98010acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.826+0000 7fb987e68700 1 --2- 172.21.15.49:0/556270571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9801027d0 0x7fb98010acc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb970009910 tx=0x7fb970009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.826+0000 7fb987e68700 1 -- 172.21.15.49:0/556270571 shutdown_connections 2024-05-15T16:20:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.826+0000 7fb987e68700 1 --2- 172.21.15.49:0/556270571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9801027d0 0x7fb98010acc0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.826+0000 7fb987e68700 1 --2- 172.21.15.49:0/556270571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb980101eb0 0x7fb980102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.826+0000 7fb987e68700 1 -- 172.21.15.49:0/556270571 >> 172.21.15.49:0/556270571 conn(0x7fb9800fb6a0 msgr2=0x7fb9800fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.826+0000 7fb987e68700 1 -- 172.21.15.49:0/556270571 shutdown_connections 2024-05-15T16:20:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.826+0000 7fb987e68700 1 -- 172.21.15.49:0/556270571 wait complete. 2024-05-15T16:20:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.827+0000 7fb987e68700 1 Processor -- start 2024-05-15T16:20:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.827+0000 7fb987e68700 1 -- start start 2024-05-15T16:20:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.827+0000 7fb987e68700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb980101eb0 0x7fb9800ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.827+0000 7fb987e68700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801027d0 0x7fb980100050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.827+0000 7fb987e68700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb980100590 con 0x7fb980101eb0 2024-05-15T16:20:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.827+0000 7fb987e68700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801006d0 con 0x7fb9801027d0 2024-05-15T16:20:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.827+0000 7fb985c04700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb980101eb0 0x7fb9800ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:24.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.827+0000 7fb985c04700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb980101eb0 0x7fb9800ffb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53812/0 (socket says 172.21.15.49:53812) 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.827+0000 7fb985c04700 1 -- 172.21.15.49:0/1109947250 learned_addr learned my addr 172.21.15.49:0/1109947250 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb985403700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801027d0 0x7fb980100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb985c04700 1 -- 172.21.15.49:0/1109947250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801027d0 msgr2=0x7fb980100050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb985c04700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801027d0 0x7fb980100050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb985c04700 1 -- 172.21.15.49:0/1109947250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9700095f0 con 0x7fb980101eb0 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb985c04700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb980101eb0 0x7fb9800ffb10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00d900 tx=0x7fb97c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb976ffd700 1 -- 172.21.15.49:0/1109947250 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97c0041f0 con 0x7fb980101eb0 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb976ffd700 1 -- 172.21.15.49:0/1109947250 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb97c004cf0 con 0x7fb980101eb0 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9801a2d70 con 0x7fb980101eb0 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb976ffd700 1 -- 172.21.15.49:0/1109947250 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97c00b750 con 0x7fb980101eb0 2024-05-15T16:20:24.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.828+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9801a30d0 con 0x7fb980101eb0 2024-05-15T16:20:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.829+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb980069770 con 0x7fb980101eb0 2024-05-15T16:20:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.830+0000 7fb976ffd700 1 -- 172.21.15.49:0/1109947250 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb97c004750 con 0x7fb980101eb0 2024-05-15T16:20:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.830+0000 7fb976ffd700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb96c06e200 0x7fb96c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.830+0000 7fb976ffd700 1 -- 172.21.15.49:0/1109947250 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb97c08dfd0 con 0x7fb980101eb0 2024-05-15T16:20:24.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.830+0000 7fb985403700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb96c06e200 0x7fb96c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:24.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.831+0000 7fb985403700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb96c06e200 0x7fb96c0706c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb980101240 tx=0x7fb97001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:24.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:24.835+0000 7fb976ffd700 1 -- 172.21.15.49:0/1109947250 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb97c057330 con 0x7fb980101eb0 2024-05-15T16:20:24.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:24 smithi144 ceph-mon[86635]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:24.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:24 smithi144 ceph-mon[86635]: from='client.35302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:24.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:24 smithi144 ceph-mon[86635]: from='client.35306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:24.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:24 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/736565660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:20:24.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:24 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/986882516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:20:25.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.062+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb98010de70 con 0x7fb96c06e200 2024-05-15T16:20:25.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:24 smithi049 ceph-mon[112142]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:25.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:24 smithi049 ceph-mon[112142]: from='client.35302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:25.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:24 smithi049 ceph-mon[112142]: from='client.35306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:25.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:24 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/736565660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:20:25.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:24 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/986882516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.064+0000 7fb976ffd700 1 -- 172.21.15.49:0/1109947250 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb98010de70 con 0x7fb96c06e200 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:20:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:20:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:20:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:20:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:20:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:20:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:20:25.066 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:20:25.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb96c06e200 msgr2=0x7fb96c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb96c06e200 0x7fb96c0706c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb980101240 tx=0x7fb97001a040 comp rx=0 tx=0).stop 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb980101eb0 msgr2=0x7fb9800ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb980101eb0 0x7fb9800ffb10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00d900 tx=0x7fb97c00dc10 comp rx=0 tx=0).stop 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 shutdown_connections 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb96c06e200 0x7fb96c0706c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb980101eb0 0x7fb9800ffb10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 --2- 172.21.15.49:0/1109947250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801027d0 0x7fb980100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 >> 172.21.15.49:0/1109947250 conn(0x7fb9800fb6a0 msgr2=0x7fb980103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 shutdown_connections 2024-05-15T16:20:25.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.066+0000 7fb987e68700 1 -- 172.21.15.49:0/1109947250 wait complete. 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 -- 172.21.15.49:0/3193142351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 msgr2=0x7f061c1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 --2- 172.21.15.49:0/3193142351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 0x7f061c1022d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f060c009910 tx=0x7f060c009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 -- 172.21.15.49:0/3193142351 shutdown_connections 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 --2- 172.21.15.49:0/3193142351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c102810 0x7f061c10ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 --2- 172.21.15.49:0/3193142351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 0x7f061c1022d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 -- 172.21.15.49:0/3193142351 >> 172.21.15.49:0/3193142351 conn(0x7f061c0fb6a0 msgr2=0x7f061c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 -- 172.21.15.49:0/3193142351 shutdown_connections 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 -- 172.21.15.49:0/3193142351 wait complete. 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 Processor -- start 2024-05-15T16:20:25.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.175+0000 7f062180d700 1 -- start start 2024-05-15T16:20:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f062180d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 0x7f061c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f062180d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c102810 0x7f061c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f062180d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c19d3c0 con 0x7f061c101ef0 2024-05-15T16:20:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f062180d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c196a30 con 0x7f061c102810 2024-05-15T16:20:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f061affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 0x7f061c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:25.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f061affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 0x7f061c19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53830/0 (socket says 172.21.15.49:53830) 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f061affd700 1 -- 172.21.15.49:0/31268706 learned_addr learned my addr 172.21.15.49:0/31268706 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f06127fc700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c102810 0x7f061c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f061affd700 1 -- 172.21.15.49:0/31268706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c102810 msgr2=0x7f061c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f061affd700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c102810 0x7f061c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f061affd700 1 -- 172.21.15.49:0/31268706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0095f0 con 0x7f061c101ef0 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.176+0000 7f061affd700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 0x7f061c19c8f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f060c006010 tx=0x7f060c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.177+0000 7f062080b700 1 -- 172.21.15.49:0/31268706 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c01d070 con 0x7f061c101ef0 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.177+0000 7f062080b700 1 -- 172.21.15.49:0/31268706 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f060c00f460 con 0x7f061c101ef0 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.177+0000 7f062080b700 1 -- 172.21.15.49:0/31268706 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c017630 con 0x7f061c101ef0 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.177+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c196cb0 con 0x7f061c101ef0 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.177+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c197100 con 0x7f061c101ef0 2024-05-15T16:20:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.178+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c1055a0 con 0x7f061c101ef0 2024-05-15T16:20:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.179+0000 7f062080b700 1 -- 172.21.15.49:0/31268706 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f060c00f5d0 con 0x7f061c101ef0 2024-05-15T16:20:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.179+0000 7f062080b700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f060806e200 0x7f06080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.179+0000 7f06127fc700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f060806e200 0x7f06080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.179+0000 7f062080b700 1 -- 172.21.15.49:0/31268706 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f060c08f620 con 0x7f061c101ef0 2024-05-15T16:20:25.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.180+0000 7f06127fc700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f060806e200 0x7f06080706c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0604005ea0 tx=0x7f0604005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.184+0000 7f062080b700 1 -- 172.21.15.49:0/31268706 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f060c05c570 con 0x7f061c101ef0 2024-05-15T16:20:25.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.475+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f061c069a50 con 0x7f061c101ef0 2024-05-15T16:20:25.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.476+0000 7f062080b700 1 -- 172.21.15.49:0/31268706 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f061c069a50 con 0x7f061c101ef0 2024-05-15T16:20:25.477 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:20:25.477 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:20:25.477 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:20:25.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:20:25.477 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:25.478 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:20:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:20:25.479 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:20:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:20:25.479 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:20:25.479 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:20:25.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f060806e200 msgr2=0x7f06080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f060806e200 0x7f06080706c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0604005ea0 tx=0x7f0604005e30 comp rx=0 tx=0).stop 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 msgr2=0x7f061c19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 0x7f061c19c8f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f060c006010 tx=0x7f060c00be20 comp rx=0 tx=0).stop 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 shutdown_connections 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f060806e200 0x7f06080706c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f061c101ef0 0x7f061c19c8f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 --2- 172.21.15.49:0/31268706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c102810 0x7f061c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 >> 172.21.15.49:0/31268706 conn(0x7f061c0fb6a0 msgr2=0x7f061c1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 shutdown_connections 2024-05-15T16:20:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:25.478+0000 7f062180d700 1 -- 172.21.15.49:0/31268706 wait complete. 2024-05-15T16:20:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:25 smithi144 ceph-mon[86635]: pgmap v703: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:25 smithi144 ceph-mon[86635]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:25 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/31268706' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:20:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:25 smithi049 ceph-mon[112142]: pgmap v703: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:25 smithi049 ceph-mon[112142]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:25 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/31268706' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:20:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:27 smithi144 ceph-mon[86635]: pgmap v704: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:27 smithi049 ceph-mon[112142]: pgmap v704: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:29.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:29 smithi144 ceph-mon[86635]: pgmap v705: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:29 smithi049 ceph-mon[112142]: pgmap v705: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:31 smithi144 ceph-mon[86635]: pgmap v706: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:31 smithi049 ceph-mon[112142]: pgmap v706: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:33 smithi144 ceph-mon[86635]: pgmap v707: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:33 smithi049 ceph-mon[112142]: pgmap v707: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:35 smithi144 ceph-mon[86635]: pgmap v708: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:20:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:35 smithi049 ceph-mon[112142]: pgmap v708: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:20:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:37 smithi144 ceph-mon[86635]: pgmap v709: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:37 smithi049 ceph-mon[112142]: pgmap v709: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:39 smithi144 ceph-mon[86635]: pgmap v710: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:39 smithi049 ceph-mon[112142]: pgmap v710: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:41 smithi144 ceph-mon[86635]: pgmap v711: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:41 smithi049 ceph-mon[112142]: pgmap v711: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:43 smithi144 ceph-mon[86635]: pgmap v712: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:43 smithi049 ceph-mon[112142]: pgmap v712: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:45 smithi144 ceph-mon[86635]: pgmap v713: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:45 smithi049 ceph-mon[112142]: pgmap v713: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:47 smithi144 ceph-mon[86635]: pgmap v714: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:47 smithi049 ceph-mon[112142]: pgmap v714: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:49 smithi144 ceph-mon[86635]: pgmap v715: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:50.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:49 smithi049 ceph-mon[112142]: pgmap v715: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:20:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:20:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:51 smithi144 ceph-mon[86635]: pgmap v716: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:51 smithi049 ceph-mon[112142]: pgmap v716: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:53 smithi144 ceph-mon[86635]: pgmap v717: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:53 smithi049 ceph-mon[112142]: pgmap v717: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:55.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.601+0000 7f2d13627700 1 -- 172.21.15.49:0/3883618243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 msgr2=0x7f2d0c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:55.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.601+0000 7f2d13627700 1 --2- 172.21.15.49:0/3883618243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 0x7f2d0c1012b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00b0a0 tx=0x7f2cfc00b3b0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.601+0000 7f2d13627700 1 -- 172.21.15.49:0/3883618243 shutdown_connections 2024-05-15T16:20:55.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.601+0000 7f2d13627700 1 --2- 172.21.15.49:0/3883618243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d0c101880 0x7f2d0c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.601+0000 7f2d13627700 1 --2- 172.21.15.49:0/3883618243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 0x7f2d0c1012b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.601+0000 7f2d13627700 1 -- 172.21.15.49:0/3883618243 >> 172.21.15.49:0/3883618243 conn(0x7f2d0c0fc7a0 msgr2=0x7f2d0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.601+0000 7f2d13627700 1 -- 172.21.15.49:0/3883618243 shutdown_connections 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.602+0000 7f2d13627700 1 -- 172.21.15.49:0/3883618243 wait complete. 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.602+0000 7f2d13627700 1 Processor -- start 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.602+0000 7f2d13627700 1 -- start start 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.602+0000 7f2d13627700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 0x7f2d0c196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.602+0000 7f2d13627700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d0c101880 0x7f2d0c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.602+0000 7f2d13627700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c197850 con 0x7f2d0c100ed0 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.602+0000 7f2d13627700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c19b460 con 0x7f2d0c101880 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d113c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 0x7f2d0c196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d113c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 0x7f2d0c196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35626/0 (socket says 172.21.15.49:35626) 2024-05-15T16:20:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d113c3700 1 -- 172.21.15.49:0/1588453429 learned_addr learned my addr 172.21.15.49:0/1588453429 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d10bc2700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d0c101880 0x7f2d0c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d113c3700 1 -- 172.21.15.49:0/1588453429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d0c101880 msgr2=0x7f2d0c197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d113c3700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d0c101880 0x7f2d0c197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d113c3700 1 -- 172.21.15.49:0/1588453429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc009d00 con 0x7f2d0c100ed0 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d10bc2700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d0c101880 0x7f2d0c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d113c3700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 0x7f2d0c196c30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc000f80 tx=0x7f2cfc007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d0a7fc700 1 -- 172.21.15.49:0/1588453429 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc00e040 con 0x7f2d0c100ed0 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d0a7fc700 1 -- 172.21.15.49:0/1588453429 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cfc003c10 con 0x7f2d0c100ed0 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d0a7fc700 1 -- 172.21.15.49:0/1588453429 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc01d6b0 con 0x7f2d0c100ed0 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c19b6e0 con 0x7f2d0c100ed0 2024-05-15T16:20:55.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.603+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c19bbd0 con 0x7f2d0c100ed0 2024-05-15T16:20:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.604+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c109270 con 0x7f2d0c100ed0 2024-05-15T16:20:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.605+0000 7f2d0a7fc700 1 -- 172.21.15.49:0/1588453429 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2cfc004040 con 0x7f2d0c100ed0 2024-05-15T16:20:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.605+0000 7f2d0a7fc700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cf406e200 0x7f2cf40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.606+0000 7f2d0a7fc700 1 -- 172.21.15.49:0/1588453429 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2cfc08f940 con 0x7f2d0c100ed0 2024-05-15T16:20:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.606+0000 7f2d10bc2700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cf406e200 0x7f2cf40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.606+0000 7f2d10bc2700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cf406e200 0x7f2cf40706c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2d00005fd0 tx=0x7f2d00005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:55.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.610+0000 7f2d0a7fc700 1 -- 172.21.15.49:0/1588453429 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2cfc058c20 con 0x7f2d0c100ed0 2024-05-15T16:20:55.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.836+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d0c197ef0 con 0x7f2cf406e200 2024-05-15T16:20:55.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.837+0000 7f2d0a7fc700 1 -- 172.21.15.49:0/1588453429 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2d0c197ef0 con 0x7f2cf406e200 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.839+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cf406e200 msgr2=0x7f2cf40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.839+0000 7f2d13627700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cf406e200 0x7f2cf40706c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2d00005fd0 tx=0x7f2d00005ee0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.839+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 msgr2=0x7f2d0c196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.839+0000 7f2d13627700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 0x7f2d0c196c30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc000f80 tx=0x7f2cfc007bd0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.839+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 shutdown_connections 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.839+0000 7f2d13627700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2cf406e200 0x7f2cf40706c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.839+0000 7f2d13627700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d0c100ed0 0x7f2d0c196c30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.839+0000 7f2d13627700 1 --2- 172.21.15.49:0/1588453429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d0c101880 0x7f2d0c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.839+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 >> 172.21.15.49:0/1588453429 conn(0x7f2d0c0fc7a0 msgr2=0x7f2d0c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.840+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 shutdown_connections 2024-05-15T16:20:55.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.840+0000 7f2d13627700 1 -- 172.21.15.49:0/1588453429 wait complete. 2024-05-15T16:20:55.851 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:20:55.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.950+0000 7f5aa2533700 1 -- 172.21.15.49:0/3032931869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c101e20 msgr2=0x7f5a9c102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:55.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.950+0000 7f5aa2533700 1 --2- 172.21.15.49:0/3032931869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c101e20 0x7f5a9c102200 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009910 tx=0x7f5a8c009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:55.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.950+0000 7f5aa2533700 1 -- 172.21.15.49:0/3032931869 shutdown_connections 2024-05-15T16:20:55.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.950+0000 7f5aa2533700 1 --2- 172.21.15.49:0/3032931869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c102740 0x7f5a9c10ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.950+0000 7f5aa2533700 1 --2- 172.21.15.49:0/3032931869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c101e20 0x7f5a9c102200 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.950+0000 7f5aa2533700 1 -- 172.21.15.49:0/3032931869 >> 172.21.15.49:0/3032931869 conn(0x7f5a9c076800 msgr2=0x7f5a9c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:55.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.950+0000 7f5aa2533700 1 -- 172.21.15.49:0/3032931869 shutdown_connections 2024-05-15T16:20:55.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.950+0000 7f5aa2533700 1 -- 172.21.15.49:0/3032931869 wait complete. 2024-05-15T16:20:55.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.951+0000 7f5aa2533700 1 Processor -- start 2024-05-15T16:20:55.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.951+0000 7f5aa2533700 1 -- start start 2024-05-15T16:20:55.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.951+0000 7f5aa2533700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c101e20 0x7f5a9c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:55.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.951+0000 7f5aa2533700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c102740 0x7f5a9c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:55.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.951+0000 7f5aa2533700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c19d420 con 0x7f5a9c102740 2024-05-15T16:20:55.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.951+0000 7f5aa2533700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c196a00 con 0x7f5a9c101e20 2024-05-15T16:20:55.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.951+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c102740 0x7f5a9c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:55.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.951+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c102740 0x7f5a9c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35644/0 (socket says 172.21.15.49:35644) 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.951+0000 7f5a93fff700 1 -- 172.21.15.49:0/1727679252 learned_addr learned my addr 172.21.15.49:0/1727679252 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5a9bfff700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c101e20 0x7f5a9c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5a93fff700 1 -- 172.21.15.49:0/1727679252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c101e20 msgr2=0x7f5a9c19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5a93fff700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c101e20 0x7f5a9c19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5a93fff700 1 -- 172.21.15.49:0/1727679252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0095f0 con 0x7f5a9c102740 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5a93fff700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c102740 0x7f5a9c19ce00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400d6d0 tx=0x7f5a8400d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5a99ffb700 1 -- 172.21.15.49:0/1727679252 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8400f860 con 0x7f5a9c102740 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5a99ffb700 1 -- 172.21.15.49:0/1727679252 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a8400fe80 con 0x7f5a9c102740 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c196ce0 con 0x7f5a9c102740 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5a99ffb700 1 -- 172.21.15.49:0/1727679252 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a84010ea0 con 0x7f5a9c102740 2024-05-15T16:20:55.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.952+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c197230 con 0x7f5a9c102740 2024-05-15T16:20:55.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.953+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c105540 con 0x7f5a9c102740 2024-05-15T16:20:55.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.954+0000 7f5a99ffb700 1 -- 172.21.15.49:0/1727679252 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5a84010460 con 0x7f5a9c102740 2024-05-15T16:20:55.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.954+0000 7f5a99ffb700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a8806e210 0x7f5a880706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:55.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.954+0000 7f5a99ffb700 1 -- 172.21.15.49:0/1727679252 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5a8408df30 con 0x7f5a9c102740 2024-05-15T16:20:55.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.954+0000 7f5a9bfff700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a8806e210 0x7f5a880706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:55.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.955+0000 7f5a9bfff700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a8806e210 0x7f5a880706d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c006010 tx=0x7f5a8c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:55.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:55.959+0000 7f5a99ffb700 1 -- 172.21.15.49:0/1727679252 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5a8405ace0 con 0x7f5a9c102740 2024-05-15T16:20:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.188+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a9c197f30 con 0x7f5a8806e210 2024-05-15T16:20:56.190 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:55 smithi049 ceph-mon[112142]: pgmap v718: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.190+0000 7f5a99ffb700 1 -- 172.21.15.49:0/1727679252 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a9c197f30 con 0x7f5a8806e210 2024-05-15T16:20:56.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a8806e210 msgr2=0x7f5a880706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a8806e210 0x7f5a880706d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c006010 tx=0x7f5a8c01a040 comp rx=0 tx=0).stop 2024-05-15T16:20:56.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c102740 msgr2=0x7f5a9c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c102740 0x7f5a9c19ce00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400d6d0 tx=0x7f5a8400d9e0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 shutdown_connections 2024-05-15T16:20:56.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a8806e210 0x7f5a880706d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c101e20 0x7f5a9c19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 --2- 172.21.15.49:0/1727679252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a9c102740 0x7f5a9c19ce00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 >> 172.21.15.49:0/1727679252 conn(0x7f5a9c076800 msgr2=0x7f5a9c103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:56.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 shutdown_connections 2024-05-15T16:20:56.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.192+0000 7f5aa2533700 1 -- 172.21.15.49:0/1727679252 wait complete. 2024-05-15T16:20:56.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:55 smithi144 ceph-mon[86635]: pgmap v718: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.301+0000 7fef2fefe700 1 -- 172.21.15.49:0/1377238724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef281098c0 msgr2=0x7fef280ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.301+0000 7fef2fefe700 1 --2- 172.21.15.49:0/1377238724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef281098c0 0x7fef280ff430 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009910 tx=0x7fef1c009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.301+0000 7fef2fefe700 1 -- 172.21.15.49:0/1377238724 shutdown_connections 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.301+0000 7fef2fefe700 1 --2- 172.21.15.49:0/1377238724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef281098c0 0x7fef280ff430 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.301+0000 7fef2fefe700 1 --2- 172.21.15.49:0/1377238724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef28108fa0 0x7fef28109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.301+0000 7fef2fefe700 1 -- 172.21.15.49:0/1377238724 >> 172.21.15.49:0/1377238724 conn(0x7fef28074eb0 msgr2=0x7fef280752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.301+0000 7fef2fefe700 1 -- 172.21.15.49:0/1377238724 shutdown_connections 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.301+0000 7fef2fefe700 1 -- 172.21.15.49:0/1377238724 wait complete. 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2fefe700 1 Processor -- start 2024-05-15T16:20:56.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2fefe700 1 -- start start 2024-05-15T16:20:56.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2fefe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef28108fa0 0x7fef28198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:56.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2fefe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef281098c0 0x7fef28199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:56.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2fefe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef28199b00 con 0x7fef28108fa0 2024-05-15T16:20:56.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2fefe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2819d780 con 0x7fef281098c0 2024-05-15T16:20:56.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2dc9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef28108fa0 0x7fef28198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:56.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2dc9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef28108fa0 0x7fef28198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35654/0 (socket says 172.21.15.49:35654) 2024-05-15T16:20:56.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2dc9a700 1 -- 172.21.15.49:0/497020534 learned_addr learned my addr 172.21.15.49:0/497020534 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:56.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2d499700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef281098c0 0x7fef28199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2dc9a700 1 -- 172.21.15.49:0/497020534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef281098c0 msgr2=0x7fef28199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2dc9a700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef281098c0 0x7fef28199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2dc9a700 1 -- 172.21.15.49:0/497020534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c0095f0 con 0x7fef28108fa0 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.302+0000 7fef2dc9a700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef28108fa0 0x7fef28198ee0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b9c0 tx=0x7fef1800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.303+0000 7fef26ffd700 1 -- 172.21.15.49:0/497020534 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef1800c980 con 0x7fef28108fa0 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.303+0000 7fef26ffd700 1 -- 172.21.15.49:0/497020534 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef18014440 con 0x7fef28108fa0 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.303+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2819da60 con 0x7fef28108fa0 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.303+0000 7fef26ffd700 1 -- 172.21.15.49:0/497020534 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef18012570 con 0x7fef28108fa0 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.303+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2819e030 con 0x7fef28108fa0 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.304+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2810b440 con 0x7fef28108fa0 2024-05-15T16:20:56.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.304+0000 7fef26ffd700 1 -- 172.21.15.49:0/497020534 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fef1800cae0 con 0x7fef28108fa0 2024-05-15T16:20:56.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.305+0000 7fef26ffd700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef1006e200 0x7fef100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:56.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.305+0000 7fef26ffd700 1 -- 172.21.15.49:0/497020534 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fef18006e40 con 0x7fef28108fa0 2024-05-15T16:20:56.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.305+0000 7fef2d499700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef1006e200 0x7fef100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:56.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.305+0000 7fef2d499700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef1006e200 0x7fef100706c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fef2819a3f0 tx=0x7fef1c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:56.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.309+0000 7fef26ffd700 1 -- 172.21.15.49:0/497020534 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fef1805a7e0 con 0x7fef28108fa0 2024-05-15T16:20:56.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.530+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fef2819a210 con 0x7fef1006e200 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.540+0000 7fef26ffd700 1 -- 172.21.15.49:0/497020534 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fef2819a210 con 0x7fef1006e200 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (24m) 2m ago 35m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (36m) 2m ago 36m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (34m) 2m ago 34m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (21m) 2m ago 35m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (21m) 2m ago 34m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (23m) 2m ago 35m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (17m) 2m ago 30m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:20:56.542 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (17m) 2m ago 30m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (17m) 2m ago 30m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (2m) 2m ago 30m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (26m) 2m ago 37m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (25m) 2m ago 34m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (22m) 2m ago 37m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (22m) 2m ago 34m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (25m) 2m ago 35m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (24m) 2m ago 34m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (21m) 2m ago 33m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (20m) 2m ago 33m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (19m) 2m ago 32m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (19m) 2m ago 32m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (18m) 2m ago 32m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:20:56.543 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (18m) 2m ago 31m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:20:56.544 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (24m) 2m ago 35m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:20:56.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.542+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef1006e200 msgr2=0x7fef100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.542+0000 7fef2fefe700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef1006e200 0x7fef100706c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fef2819a3f0 tx=0x7fef1c01a040 comp rx=0 tx=0).stop 2024-05-15T16:20:56.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.542+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef28108fa0 msgr2=0x7fef28198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.542+0000 7fef2fefe700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef28108fa0 0x7fef28198ee0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b9c0 tx=0x7fef1800bd80 comp rx=0 tx=0).stop 2024-05-15T16:20:56.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.543+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 shutdown_connections 2024-05-15T16:20:56.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.543+0000 7fef2fefe700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fef1006e200 0x7fef100706c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.543+0000 7fef2fefe700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef28108fa0 0x7fef28198ee0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.543+0000 7fef2fefe700 1 --2- 172.21.15.49:0/497020534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef281098c0 0x7fef28199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.543+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 >> 172.21.15.49:0/497020534 conn(0x7fef28074eb0 msgr2=0x7fef28107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:56.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.543+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 shutdown_connections 2024-05-15T16:20:56.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.543+0000 7fef2fefe700 1 -- 172.21.15.49:0/497020534 wait complete. 2024-05-15T16:20:56.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.650+0000 7fb2efe99700 1 -- 172.21.15.49:0/678645623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 msgr2=0x7fb2e8105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.650+0000 7fb2efe99700 1 --2- 172.21.15.49:0/678645623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 0x7fb2e8105900 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009910 tx=0x7fb2d8009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:56.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.650+0000 7fb2efe99700 1 -- 172.21.15.49:0/678645623 shutdown_connections 2024-05-15T16:20:56.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.650+0000 7fb2efe99700 1 --2- 172.21.15.49:0/678645623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 0x7fb2e8105900 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.650+0000 7fb2efe99700 1 --2- 172.21.15.49:0/678645623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e8100f00 0x7fb2e81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.650+0000 7fb2efe99700 1 -- 172.21.15.49:0/678645623 >> 172.21.15.49:0/678645623 conn(0x7fb2e80fc7d0 msgr2=0x7fb2e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:56.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.650+0000 7fb2efe99700 1 -- 172.21.15.49:0/678645623 shutdown_connections 2024-05-15T16:20:56.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.650+0000 7fb2efe99700 1 -- 172.21.15.49:0/678645623 wait complete. 2024-05-15T16:20:56.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2efe99700 1 Processor -- start 2024-05-15T16:20:56.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2efe99700 1 -- start start 2024-05-15T16:20:56.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2efe99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e8100f00 0x7fb2e80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2efe99700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 0x7fb2e80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2efe99700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8073d20 con 0x7fb2e81018b0 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2efe99700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8073e60 con 0x7fb2e8100f00 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2ed434700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 0x7fb2e80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2ed434700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 0x7fb2e80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35670/0 (socket says 172.21.15.49:35670) 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2ed434700 1 -- 172.21.15.49:0/2243453712 learned_addr learned my addr 172.21.15.49:0/2243453712 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.651+0000 7fb2edc35700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e8100f00 0x7fb2e80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.652+0000 7fb2ed434700 1 -- 172.21.15.49:0/2243453712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e8100f00 msgr2=0x7fb2e80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.652+0000 7fb2ed434700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e8100f00 0x7fb2e80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.652+0000 7fb2ed434700 1 -- 172.21.15.49:0/2243453712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80095f0 con 0x7fb2e81018b0 2024-05-15T16:20:56.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.652+0000 7fb2ed434700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 0x7fb2e80737e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800bd70 tx=0x7fb2d800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:56.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.652+0000 7fb2deffd700 1 -- 172.21.15.49:0/2243453712 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d801d070 con 0x7fb2e81018b0 2024-05-15T16:20:56.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.652+0000 7fb2deffd700 1 -- 172.21.15.49:0/2243453712 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2d80056e0 con 0x7fb2e81018b0 2024-05-15T16:20:56.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.652+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e80740e0 con 0x7fb2e81018b0 2024-05-15T16:20:56.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.652+0000 7fb2deffd700 1 -- 172.21.15.49:0/2243453712 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d800fe50 con 0x7fb2e81018b0 2024-05-15T16:20:56.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.652+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e81a3020 con 0x7fb2e81018b0 2024-05-15T16:20:56.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.653+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e8109300 con 0x7fb2e81018b0 2024-05-15T16:20:56.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.654+0000 7fb2deffd700 1 -- 172.21.15.49:0/2243453712 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb2d800f680 con 0x7fb2e81018b0 2024-05-15T16:20:56.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.654+0000 7fb2deffd700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb2d406e200 0x7fb2d40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:56.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.654+0000 7fb2deffd700 1 -- 172.21.15.49:0/2243453712 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb2d808f4b0 con 0x7fb2e81018b0 2024-05-15T16:20:56.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.654+0000 7fb2edc35700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb2d406e200 0x7fb2d40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:56.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.655+0000 7fb2edc35700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb2d406e200 0x7fb2d40706c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009550 tx=0x7fb2e40091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:56.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.659+0000 7fb2deffd700 1 -- 172.21.15.49:0/2243453712 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb2d8058790 con 0x7fb2e81018b0 2024-05-15T16:20:56.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.952+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb2e81023e0 con 0x7fb2e81018b0 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.952+0000 7fb2deffd700 1 -- 172.21.15.49:0/2243453712 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb2d805be50 con 0x7fb2e81018b0 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:20:56.954 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:20:56.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:20:56.955 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:20:56.955 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:20:56.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:20:56.955 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:20:56.955 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:20:56.955 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:20:56.955 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:20:56.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb2d406e200 msgr2=0x7fb2d40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb2d406e200 0x7fb2d40706c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009550 tx=0x7fb2e40091b0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 msgr2=0x7fb2e80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:56.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 0x7fb2e80737e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800bd70 tx=0x7fb2d800bda0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 shutdown_connections 2024-05-15T16:20:56.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb2d406e200 0x7fb2d40706c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e8100f00 0x7fb2e80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 --2- 172.21.15.49:0/2243453712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e81018b0 0x7fb2e80737e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:56.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 >> 172.21.15.49:0/2243453712 conn(0x7fb2e80fc7d0 msgr2=0x7fb2e80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:56.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 shutdown_connections 2024-05-15T16:20:56.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:56.955+0000 7fb2efe99700 1 -- 172.21.15.49:0/2243453712 wait complete. 2024-05-15T16:20:57.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.063+0000 7f75c067e700 1 -- 172.21.15.49:0/4072925682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8102800 msgr2=0x7f75b810acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.063+0000 7f75c067e700 1 --2- 172.21.15.49:0/4072925682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8102800 0x7f75b810acf0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009910 tx=0x7f75a8009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:57.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.063+0000 7f75c067e700 1 -- 172.21.15.49:0/4072925682 shutdown_connections 2024-05-15T16:20:57.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.063+0000 7f75c067e700 1 --2- 172.21.15.49:0/4072925682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8102800 0x7f75b810acf0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.063+0000 7f75c067e700 1 --2- 172.21.15.49:0/4072925682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b8101ee0 0x7f75b81022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.063+0000 7f75c067e700 1 -- 172.21.15.49:0/4072925682 >> 172.21.15.49:0/4072925682 conn(0x7f75b8076800 msgr2=0x7f75b80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:57.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.063+0000 7f75c067e700 1 -- 172.21.15.49:0/4072925682 shutdown_connections 2024-05-15T16:20:57.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.063+0000 7f75c067e700 1 -- 172.21.15.49:0/4072925682 wait complete. 2024-05-15T16:20:57.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.064+0000 7f75c067e700 1 Processor -- start 2024-05-15T16:20:57.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.064+0000 7f75c067e700 1 -- start start 2024-05-15T16:20:57.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.064+0000 7f75c067e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8101ee0 0x7f75b819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:57.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.064+0000 7f75c067e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b8102800 0x7f75b819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:57.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.064+0000 7f75c067e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b8196a90 con 0x7f75b8101ee0 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.064+0000 7f75c067e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b8196c00 con 0x7f75b8102800 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.064+0000 7f75be41a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8101ee0 0x7f75b819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.064+0000 7f75be41a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8101ee0 0x7f75b819c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35692/0 (socket says 172.21.15.49:35692) 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.064+0000 7f75be41a700 1 -- 172.21.15.49:0/4106798122 learned_addr learned my addr 172.21.15.49:0/4106798122 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75bdc19700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b8102800 0x7f75b819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75be41a700 1 -- 172.21.15.49:0/4106798122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b8102800 msgr2=0x7f75b819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75be41a700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b8102800 0x7f75b819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75be41a700 1 -- 172.21.15.49:0/4106798122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80095f0 con 0x7f75b8101ee0 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75bdc19700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b8102800 0x7f75b819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75be41a700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8101ee0 0x7f75b819c9e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f75b400d900 tx=0x7f75b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75af7fe700 1 -- 172.21.15.49:0/4106798122 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75b40041f0 con 0x7f75b8101ee0 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75af7fe700 1 -- 172.21.15.49:0/4106798122 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75b4004cf0 con 0x7f75b8101ee0 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b8196ee0 con 0x7f75b8101ee0 2024-05-15T16:20:57.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75af7fe700 1 -- 172.21.15.49:0/4106798122 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75b400b750 con 0x7f75b8101ee0 2024-05-15T16:20:57.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.065+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b8197430 con 0x7f75b8101ee0 2024-05-15T16:20:57.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.066+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b8068ea0 con 0x7f75b8101ee0 2024-05-15T16:20:57.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.067+0000 7f75af7fe700 1 -- 172.21.15.49:0/4106798122 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f75b4004350 con 0x7f75b8101ee0 2024-05-15T16:20:57.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.067+0000 7f75af7fe700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f75a406e200 0x7f75a40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:57.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.067+0000 7f75af7fe700 1 -- 172.21.15.49:0/4106798122 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f75b408e210 con 0x7f75b8101ee0 2024-05-15T16:20:57.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.067+0000 7f75bdc19700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f75a406e200 0x7f75a40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:57.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.068+0000 7f75bdc19700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f75a406e200 0x7f75a40706c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f75b8198200 tx=0x7f75a800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:57.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.072+0000 7f75af7fe700 1 -- 172.21.15.49:0/4106798122 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f75b4057570 con 0x7f75b8101ee0 2024-05-15T16:20:57.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:56 smithi144 ceph-mon[86635]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:57.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:56 smithi049 ceph-mon[112142]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.323+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f75b8197fe0 con 0x7f75b8101ee0 2024-05-15T16:20:57.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.323+0000 7f75af7fe700 1 -- 172.21.15.49:0/4106798122 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f75b4016760 con 0x7f75b8101ee0 2024-05-15T16:20:57.325 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:20:57.325 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:20:57.325 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:20:57.325 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:20:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:20:57.325 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:20:57.325 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:20:57.325 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:20:57.326 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:20:57.327 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:20:57.328 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:20:57.328 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:20:57.328 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:20:57.328 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:20:57.328 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f75a406e200 msgr2=0x7f75a40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f75a406e200 0x7f75a40706c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f75b8198200 tx=0x7f75a800b540 comp rx=0 tx=0).stop 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8101ee0 msgr2=0x7f75b819c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8101ee0 0x7f75b819c9e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f75b400d900 tx=0x7f75b400dcc0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 shutdown_connections 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f75a406e200 0x7f75a40706c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75b8101ee0 0x7f75b819c9e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 --2- 172.21.15.49:0/4106798122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b8102800 0x7f75b819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 >> 172.21.15.49:0/4106798122 conn(0x7f75b8076800 msgr2=0x7f75b81031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:57.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 shutdown_connections 2024-05-15T16:20:57.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.326+0000 7f75c067e700 1 -- 172.21.15.49:0/4106798122 wait complete. 2024-05-15T16:20:57.330 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:20:57.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.435+0000 7f37881d0700 1 -- 172.21.15.49:0/2430333951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 msgr2=0x7f3780104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.435+0000 7f37881d0700 1 --2- 172.21.15.49:0/2430333951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 0x7f3780104880 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3770009910 tx=0x7f3770009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:57.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.435+0000 7f37881d0700 1 -- 172.21.15.49:0/2430333951 shutdown_connections 2024-05-15T16:20:57.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.435+0000 7f37881d0700 1 --2- 172.21.15.49:0/2430333951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 0x7f3780104880 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.435+0000 7f37881d0700 1 --2- 172.21.15.49:0/2430333951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37800ff690 0x7f37800ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.435+0000 7f37881d0700 1 -- 172.21.15.49:0/2430333951 >> 172.21.15.49:0/2430333951 conn(0x7f3780074eb0 msgr2=0x7f37800752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:57.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.435+0000 7f37881d0700 1 -- 172.21.15.49:0/2430333951 shutdown_connections 2024-05-15T16:20:57.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.435+0000 7f37881d0700 1 -- 172.21.15.49:0/2430333951 wait complete. 2024-05-15T16:20:57.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f37881d0700 1 Processor -- start 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f37881d0700 1 -- start start 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f37881d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37800ff690 0x7f378019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f37881d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 0x7f378019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f37881d0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378019d430 con 0x7f3780100040 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f37881d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3780196a10 con 0x7f37800ff690 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f378576b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 0x7f378019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f378576b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 0x7f378019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35716/0 (socket says 172.21.15.49:35716) 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f378576b700 1 -- 172.21.15.49:0/3908564411 learned_addr learned my addr 172.21.15.49:0/3908564411 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.436+0000 7f3785f6c700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37800ff690 0x7f378019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:57.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.437+0000 7f378576b700 1 -- 172.21.15.49:0/3908564411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37800ff690 msgr2=0x7f378019c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.437+0000 7f378576b700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37800ff690 0x7f378019c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.437+0000 7f378576b700 1 -- 172.21.15.49:0/3908564411 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37700095f0 con 0x7f3780100040 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.437+0000 7f378576b700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 0x7f378019ce10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3770006010 tx=0x7f377000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.437+0000 7f3776ffd700 1 -- 172.21.15.49:0/3908564411 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f377001d070 con 0x7f3780100040 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.437+0000 7f3776ffd700 1 -- 172.21.15.49:0/3908564411 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37700056e0 con 0x7f3780100040 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.437+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3780196c90 con 0x7f3780100040 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.437+0000 7f3776ffd700 1 -- 172.21.15.49:0/3908564411 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f377000fe80 con 0x7f3780100040 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.437+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3780197180 con 0x7f3780100040 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.438+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37801092b0 con 0x7f3780100040 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.439+0000 7f3776ffd700 1 -- 172.21.15.49:0/3908564411 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f377000f6f0 con 0x7f3780100040 2024-05-15T16:20:57.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.439+0000 7f3776ffd700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f376c06e1a0 0x7f376c070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:57.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.439+0000 7f3776ffd700 1 -- 172.21.15.49:0/3908564411 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f377008f5b0 con 0x7f3780100040 2024-05-15T16:20:57.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.439+0000 7f3785f6c700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f376c06e1a0 0x7f376c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:57.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.440+0000 7f3785f6c700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f376c06e1a0 0x7f376c070660 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f377c005950 tx=0x7f377c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:57.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.444+0000 7f3776ffd700 1 -- 172.21.15.49:0/3908564411 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f377005c360 con 0x7f3780100040 2024-05-15T16:20:57.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.671+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3780197ed0 con 0x7f376c06e1a0 2024-05-15T16:20:57.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.672+0000 7f3776ffd700 1 -- 172.21.15.49:0/3908564411 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3780197ed0 con 0x7f376c06e1a0 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:20:57.674 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:20:57.675 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:20:57.675 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:20:57.675 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:20:57.675 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:20:57.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f376c06e1a0 msgr2=0x7f376c070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f376c06e1a0 0x7f376c070660 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f377c005950 tx=0x7f377c00b410 comp rx=0 tx=0).stop 2024-05-15T16:20:57.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 msgr2=0x7f378019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 0x7f378019ce10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3770006010 tx=0x7f377000bd70 comp rx=0 tx=0).stop 2024-05-15T16:20:57.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 shutdown_connections 2024-05-15T16:20:57.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f376c06e1a0 0x7f376c070660 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37800ff690 0x7f378019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 --2- 172.21.15.49:0/3908564411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780100040 0x7f378019ce10 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 >> 172.21.15.49:0/3908564411 conn(0x7f3780074eb0 msgr2=0x7f3780102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:57.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 shutdown_connections 2024-05-15T16:20:57.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.675+0000 7f37881d0700 1 -- 172.21.15.49:0/3908564411 wait complete. 2024-05-15T16:20:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3415061218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c300ffdf0 msgr2=0x7f2c301001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 --2- 172.21.15.49:0/3415061218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c300ffdf0 0x7f2c301001d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009910 tx=0x7f2c18009c20 comp rx=0 tx=0).stop 2024-05-15T16:20:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3415061218 shutdown_connections 2024-05-15T16:20:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 --2- 172.21.15.49:0/3415061218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c301007a0 0x7f2c301047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 --2- 172.21.15.49:0/3415061218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c300ffdf0 0x7f2c301001d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3415061218 >> 172.21.15.49:0/3415061218 conn(0x7f2c300fb6a0 msgr2=0x7f2c300fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3415061218 shutdown_connections 2024-05-15T16:20:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3415061218 wait complete. 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 Processor -- start 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.784+0000 7f2c36ce3700 1 -- start start 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c36ce3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c300ffdf0 0x7f2c30197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c36ce3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c301007a0 0x7f2c30197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c36ce3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c301984f0 con 0x7f2c301007a0 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c36ce3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c30198630 con 0x7f2c300ffdf0 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c2ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c301007a0 0x7f2c30197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c2ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c301007a0 0x7f2c30197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35734/0 (socket says 172.21.15.49:35734) 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c2ffff700 1 -- 172.21.15.49:0/3384925659 learned_addr learned my addr 172.21.15.49:0/3384925659 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c34a7f700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c300ffdf0 0x7f2c30197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c2ffff700 1 -- 172.21.15.49:0/3384925659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c300ffdf0 msgr2=0x7f2c30197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c2ffff700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c300ffdf0 0x7f2c30197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c2ffff700 1 -- 172.21.15.49:0/3384925659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c180095f0 con 0x7f2c301007a0 2024-05-15T16:20:57.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c2ffff700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c301007a0 0x7f2c30197fb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000efc0 tx=0x7f2c2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:57.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.785+0000 7f2c2dffb700 1 -- 172.21.15.49:0/3384925659 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c2000c9f0 con 0x7f2c301007a0 2024-05-15T16:20:57.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.786+0000 7f2c2dffb700 1 -- 172.21.15.49:0/3384925659 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c2000f040 con 0x7f2c301007a0 2024-05-15T16:20:57.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.786+0000 7f2c2dffb700 1 -- 172.21.15.49:0/3384925659 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c20014480 con 0x7f2c301007a0 2024-05-15T16:20:57.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.786+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c301988b0 con 0x7f2c301007a0 2024-05-15T16:20:57.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.786+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c301a0e70 con 0x7f2c301007a0 2024-05-15T16:20:57.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.787+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c30108190 con 0x7f2c301007a0 2024-05-15T16:20:57.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.787+0000 7f2c2dffb700 1 -- 172.21.15.49:0/3384925659 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2c2000cb50 con 0x7f2c301007a0 2024-05-15T16:20:57.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.788+0000 7f2c2dffb700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2c1c06e200 0x7f2c1c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:20:57.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.788+0000 7f2c2dffb700 1 -- 172.21.15.49:0/3384925659 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2c2008e5b0 con 0x7f2c301007a0 2024-05-15T16:20:57.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.788+0000 7f2c34a7f700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2c1c06e200 0x7f2c1c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:20:57.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.788+0000 7f2c34a7f700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2c1c06e200 0x7f2c1c0706c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2c180059d0 tx=0x7f2c18005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:20:57.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:57.792+0000 7f2c2dffb700 1 -- 172.21.15.49:0/3384925659 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2c200577f0 con 0x7f2c301007a0 2024-05-15T16:20:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:57 smithi049 ceph-mon[112142]: from='client.35330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:57 smithi049 ceph-mon[112142]: from='client.35334 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:57 smithi049 ceph-mon[112142]: pgmap v719: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:57 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2243453712' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:20:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:57 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4106798122' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:20:58.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.089+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2c301012d0 con 0x7f2c301007a0 2024-05-15T16:20:58.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.090+0000 7f2c2dffb700 1 -- 172.21.15.49:0/3384925659 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f2c2005aeb0 con 0x7f2c301007a0 2024-05-15T16:20:58.091 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:20:58.091 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:20:58.092 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:20:58.093 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:20:58.093 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:20:58.093 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:20:58.093 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:20:58.093 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:20:58.093 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:20:58.093 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:20:58.093 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:20:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2c1c06e200 msgr2=0x7f2c1c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2c1c06e200 0x7f2c1c0706c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2c180059d0 tx=0x7f2c18005940 comp rx=0 tx=0).stop 2024-05-15T16:20:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c301007a0 msgr2=0x7f2c30197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:20:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c301007a0 0x7f2c30197fb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000efc0 tx=0x7f2c2000c3a0 comp rx=0 tx=0).stop 2024-05-15T16:20:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 shutdown_connections 2024-05-15T16:20:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2c1c06e200 0x7f2c1c0706c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c300ffdf0 0x7f2c30197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 --2- 172.21.15.49:0/3384925659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c301007a0 0x7f2c30197fb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:20:58.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 >> 172.21.15.49:0/3384925659 conn(0x7f2c300fb6a0 msgr2=0x7f2c300fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:20:58.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 shutdown_connections 2024-05-15T16:20:58.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:20:58.093+0000 7f2c36ce3700 1 -- 172.21.15.49:0/3384925659 wait complete. 2024-05-15T16:20:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:57 smithi144 ceph-mon[86635]: from='client.35330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:57 smithi144 ceph-mon[86635]: from='client.35334 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:57 smithi144 ceph-mon[86635]: pgmap v719: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:20:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:57 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2243453712' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:20:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:57 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4106798122' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:20:59.128 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:58 smithi049 ceph-mon[112142]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:59.128 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:58 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3384925659' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:20:59.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:58 smithi144 ceph-mon[86635]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:20:59.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:58 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3384925659' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:21:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:20:59 smithi144 ceph-mon[86635]: pgmap v720: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:20:59 smithi049 ceph-mon[112142]: pgmap v720: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:01 smithi144 ceph-mon[86635]: pgmap v721: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:01 smithi049 ceph-mon[112142]: pgmap v721: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:04.182 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:03 smithi049 ceph-mon[112142]: pgmap v722: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:04.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:03 smithi144 ceph-mon[86635]: pgmap v722: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:05 smithi144 ceph-mon[86635]: pgmap v723: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:21:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:05 smithi049 ceph-mon[112142]: pgmap v723: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:21:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:07 smithi144 ceph-mon[86635]: pgmap v724: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:07 smithi049 ceph-mon[112142]: pgmap v724: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:09 smithi144 ceph-mon[86635]: pgmap v725: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:09 smithi049 ceph-mon[112142]: pgmap v725: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:11 smithi144 ceph-mon[86635]: pgmap v726: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:11 smithi049 ceph-mon[112142]: pgmap v726: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:13 smithi144 ceph-mon[86635]: pgmap v727: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:13 smithi049 ceph-mon[112142]: pgmap v727: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:15 smithi144 ceph-mon[86635]: pgmap v728: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:15 smithi049 ceph-mon[112142]: pgmap v728: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:17 smithi144 ceph-mon[86635]: pgmap v729: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:17 smithi049 ceph-mon[112142]: pgmap v729: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:19 smithi144 ceph-mon[86635]: pgmap v730: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:19 smithi049 ceph-mon[112142]: pgmap v730: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:21:21.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:21:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:21 smithi144 ceph-mon[86635]: pgmap v731: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:21 smithi049 ceph-mon[112142]: pgmap v731: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:23 smithi144 ceph-mon[86635]: pgmap v732: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:23 smithi049 ceph-mon[112142]: pgmap v732: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:25 smithi144 ceph-mon[86635]: pgmap v733: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:25 smithi049 ceph-mon[112142]: pgmap v733: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:27 smithi144 ceph-mon[86635]: pgmap v734: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:27 smithi049 ceph-mon[112142]: pgmap v734: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.217+0000 7f0cea816700 1 -- 172.21.15.49:0/682254931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4101f10 msgr2=0x7f0ce41022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.217+0000 7f0cea816700 1 --2- 172.21.15.49:0/682254931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4101f10 0x7f0ce41022f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009910 tx=0x7f0cd4009c20 comp rx=0 tx=0).stop 2024-05-15T16:21:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.217+0000 7f0cea816700 1 -- 172.21.15.49:0/682254931 shutdown_connections 2024-05-15T16:21:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.217+0000 7f0cea816700 1 --2- 172.21.15.49:0/682254931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ce4102830 0x7f0ce410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.217+0000 7f0cea816700 1 --2- 172.21.15.49:0/682254931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4101f10 0x7f0ce41022f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.217+0000 7f0cea816700 1 -- 172.21.15.49:0/682254931 >> 172.21.15.49:0/682254931 conn(0x7f0ce40fb6a0 msgr2=0x7f0ce40fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.218+0000 7f0cea816700 1 -- 172.21.15.49:0/682254931 shutdown_connections 2024-05-15T16:21:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.218+0000 7f0cea816700 1 -- 172.21.15.49:0/682254931 wait complete. 2024-05-15T16:21:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.218+0000 7f0cea816700 1 Processor -- start 2024-05-15T16:21:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.218+0000 7f0cea816700 1 -- start start 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0cea816700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ce4101f10 0x7f0ce419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0cea816700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4102830 0x7f0ce419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0cea816700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce419d550 con 0x7f0ce4102830 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0cea816700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4196b30 con 0x7f0ce4101f10 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0ce37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4102830 0x7f0ce419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0ce37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4102830 0x7f0ce419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44550/0 (socket says 172.21.15.49:44550) 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0ce37fe700 1 -- 172.21.15.49:0/1502469284 learned_addr learned my addr 172.21.15.49:0/1502469284 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0ce3fff700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ce4101f10 0x7f0ce419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0ce37fe700 1 -- 172.21.15.49:0/1502469284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ce4101f10 msgr2=0x7f0ce419c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0ce37fe700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ce4101f10 0x7f0ce419c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0ce37fe700 1 -- 172.21.15.49:0/1502469284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40095f0 con 0x7f0ce4102830 2024-05-15T16:21:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0ce3fff700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ce4101f10 0x7f0ce419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:21:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.219+0000 7f0ce37fe700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4102830 0x7f0ce419cf30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00b480 tx=0x7f0ccc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.220+0000 7f0ce17fa700 1 -- 172.21.15.49:0/1502469284 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ccc0118c0 con 0x7f0ce4102830 2024-05-15T16:21:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.220+0000 7f0ce17fa700 1 -- 172.21.15.49:0/1502469284 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ccc010380 con 0x7f0ce4102830 2024-05-15T16:21:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.220+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce4196e10 con 0x7f0ce4102830 2024-05-15T16:21:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.220+0000 7f0ce17fa700 1 -- 172.21.15.49:0/1502469284 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ccc00f4f0 con 0x7f0ce4102830 2024-05-15T16:21:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.220+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce4197360 con 0x7f0ce4102830 2024-05-15T16:21:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.221+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce41056a0 con 0x7f0ce4102830 2024-05-15T16:21:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.221+0000 7f0ce17fa700 1 -- 172.21.15.49:0/1502469284 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0ccc0104f0 con 0x7f0ce4102830 2024-05-15T16:21:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.222+0000 7f0ce17fa700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0cd006e200 0x7f0cd00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.222+0000 7f0ce17fa700 1 -- 172.21.15.49:0/1502469284 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0ccc08dde0 con 0x7f0ce4102830 2024-05-15T16:21:28.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.222+0000 7f0ce3fff700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0cd006e200 0x7f0cd00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:28.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.223+0000 7f0ce3fff700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0cd006e200 0x7f0cd00706c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40059d0 tx=0x7f0cd401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.227+0000 7f0ce17fa700 1 -- 172.21.15.49:0/1502469284 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0ccc05abf0 con 0x7f0ce4102830 2024-05-15T16:21:28.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.455+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ce4197ff0 con 0x7f0cd006e200 2024-05-15T16:21:28.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.456+0000 7f0ce17fa700 1 -- 172.21.15.49:0/1502469284 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0ce4197ff0 con 0x7f0cd006e200 2024-05-15T16:21:28.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.458+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0cd006e200 msgr2=0x7f0cd00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.458+0000 7f0cea816700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0cd006e200 0x7f0cd00706c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40059d0 tx=0x7f0cd401a040 comp rx=0 tx=0).stop 2024-05-15T16:21:28.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.458+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4102830 msgr2=0x7f0ce419cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.458+0000 7f0cea816700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4102830 0x7f0ce419cf30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00b480 tx=0x7f0ccc00b840 comp rx=0 tx=0).stop 2024-05-15T16:21:28.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.458+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 shutdown_connections 2024-05-15T16:21:28.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.458+0000 7f0cea816700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0cd006e200 0x7f0cd00706c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.458+0000 7f0cea816700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ce4101f10 0x7f0ce419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.458+0000 7f0cea816700 1 --2- 172.21.15.49:0/1502469284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ce4102830 0x7f0ce419cf30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.458+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 >> 172.21.15.49:0/1502469284 conn(0x7f0ce40fb6a0 msgr2=0x7f0ce4103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:28.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.459+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 shutdown_connections 2024-05-15T16:21:28.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.459+0000 7f0cea816700 1 -- 172.21.15.49:0/1502469284 wait complete. 2024-05-15T16:21:28.469 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.568+0000 7fd284f46700 1 -- 172.21.15.49:0/4161565958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2801007b0 msgr2=0x7fd280104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.568+0000 7fd284f46700 1 --2- 172.21.15.49:0/4161565958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2801007b0 0x7fd280104800 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd268009910 tx=0x7fd268009c20 comp rx=0 tx=0).stop 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.569+0000 7fd284f46700 1 -- 172.21.15.49:0/4161565958 shutdown_connections 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.569+0000 7fd284f46700 1 --2- 172.21.15.49:0/4161565958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2801007b0 0x7fd280104800 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.569+0000 7fd284f46700 1 --2- 172.21.15.49:0/4161565958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2800ffe00 0x7fd2801001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.569+0000 7fd284f46700 1 -- 172.21.15.49:0/4161565958 >> 172.21.15.49:0/4161565958 conn(0x7fd2800fb6d0 msgr2=0x7fd2800fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.569+0000 7fd284f46700 1 -- 172.21.15.49:0/4161565958 shutdown_connections 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.569+0000 7fd284f46700 1 -- 172.21.15.49:0/4161565958 wait complete. 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.569+0000 7fd284f46700 1 Processor -- start 2024-05-15T16:21:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.569+0000 7fd284f46700 1 -- start start 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd284f46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2800ffe00 0x7fd2801018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd284f46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007b0 0x7fd280101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd284f46700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280102320 con 0x7fd2800ffe00 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd284f46700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280102460 con 0x7fd2801007b0 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd27e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2800ffe00 0x7fd2801018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd27e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2800ffe00 0x7fd2801018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44564/0 (socket says 172.21.15.49:44564) 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd27e59c700 1 -- 172.21.15.49:0/518047926 learned_addr learned my addr 172.21.15.49:0/518047926 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd27dd9b700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007b0 0x7fd280101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd27e59c700 1 -- 172.21.15.49:0/518047926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007b0 msgr2=0x7fd280101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd27e59c700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007b0 0x7fd280101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd27e59c700 1 -- 172.21.15.49:0/518047926 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2680095f0 con 0x7fd2800ffe00 2024-05-15T16:21:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd27e59c700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2800ffe00 0x7fd2801018a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd27000c8a0 tx=0x7fd27000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.570+0000 7fd2777fe700 1 -- 172.21.15.49:0/518047926 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd27000cea0 con 0x7fd2800ffe00 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.571+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2801a2d80 con 0x7fd2800ffe00 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.571+0000 7fd2777fe700 1 -- 172.21.15.49:0/518047926 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd270004830 con 0x7fd2800ffe00 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.571+0000 7fd2777fe700 1 -- 172.21.15.49:0/518047926 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2700055b0 con 0x7fd2800ffe00 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.571+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2801a30e0 con 0x7fd2800ffe00 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.572+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2801012e0 con 0x7fd2800ffe00 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.572+0000 7fd2777fe700 1 -- 172.21.15.49:0/518047926 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd2700049a0 con 0x7fd2800ffe00 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.573+0000 7fd2777fe700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd26c06e200 0x7fd26c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.573+0000 7fd2777fe700 1 -- 172.21.15.49:0/518047926 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd27008de40 con 0x7fd2800ffe00 2024-05-15T16:21:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.573+0000 7fd27dd9b700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd26c06e200 0x7fd26c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.573+0000 7fd27dd9b700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd26c06e200 0x7fd26c0706c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd280102fd0 tx=0x7fd26801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.577+0000 7fd2777fe700 1 -- 172.21.15.49:0/518047926 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd270057120 con 0x7fd2800ffe00 2024-05-15T16:21:28.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.807+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd28010cf20 con 0x7fd26c06e200 2024-05-15T16:21:28.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.808+0000 7fd2777fe700 1 -- 172.21.15.49:0/518047926 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd28010cf20 con 0x7fd26c06e200 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd26c06e200 msgr2=0x7fd26c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd26c06e200 0x7fd26c0706c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd280102fd0 tx=0x7fd26801a040 comp rx=0 tx=0).stop 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2800ffe00 msgr2=0x7fd2801018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2800ffe00 0x7fd2801018a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd27000c8a0 tx=0x7fd27000cc60 comp rx=0 tx=0).stop 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 shutdown_connections 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd26c06e200 0x7fd26c0706c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2800ffe00 0x7fd2801018a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 --2- 172.21.15.49:0/518047926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007b0 0x7fd280101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 >> 172.21.15.49:0/518047926 conn(0x7fd2800fb6d0 msgr2=0x7fd2800fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:28.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 shutdown_connections 2024-05-15T16:21:28.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.810+0000 7fd284f46700 1 -- 172.21.15.49:0/518047926 wait complete. 2024-05-15T16:21:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.918+0000 7f782125e700 1 -- 172.21.15.49:0/1115027203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c100770 msgr2=0x7f781c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.918+0000 7f782125e700 1 --2- 172.21.15.49:0/1115027203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c100770 0x7f781c1047c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7810009910 tx=0x7f7810009c20 comp rx=0 tx=0).stop 2024-05-15T16:21:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.918+0000 7f782125e700 1 -- 172.21.15.49:0/1115027203 shutdown_connections 2024-05-15T16:21:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.918+0000 7f782125e700 1 --2- 172.21.15.49:0/1115027203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c100770 0x7f781c1047c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.918+0000 7f782125e700 1 --2- 172.21.15.49:0/1115027203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f781c0ffdc0 0x7f781c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.918+0000 7f782125e700 1 -- 172.21.15.49:0/1115027203 >> 172.21.15.49:0/1115027203 conn(0x7f781c076800 msgr2=0x7f781c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.919+0000 7f782125e700 1 -- 172.21.15.49:0/1115027203 shutdown_connections 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.919+0000 7f782125e700 1 -- 172.21.15.49:0/1115027203 wait complete. 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.919+0000 7f782125e700 1 Processor -- start 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f782125e700 1 -- start start 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f782125e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c0ffdc0 0x7f781c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f782125e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f781c100770 0x7f781c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f782125e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c198510 con 0x7f781c0ffdc0 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f782125e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c198650 con 0x7f781c100770 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f781ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c0ffdc0 0x7f781c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f781ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c0ffdc0 0x7f781c197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44590/0 (socket says 172.21.15.49:44590) 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f781ad9d700 1 -- 172.21.15.49:0/3007695188 learned_addr learned my addr 172.21.15.49:0/3007695188 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:21:28.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f781a59c700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f781c100770 0x7f781c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f781ad9d700 1 -- 172.21.15.49:0/3007695188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f781c100770 msgr2=0x7f781c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f781ad9d700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f781c100770 0x7f781c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.920+0000 7f781ad9d700 1 -- 172.21.15.49:0/3007695188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100095f0 con 0x7f781c0ffdc0 2024-05-15T16:21:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.921+0000 7f781ad9d700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c0ffdc0 0x7f781c197a00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f780c00d8d0 tx=0x7f780c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.921+0000 7f7803fff700 1 -- 172.21.15.49:0/3007695188 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f780c0098a0 con 0x7f781c0ffdc0 2024-05-15T16:21:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.921+0000 7f7803fff700 1 -- 172.21.15.49:0/3007695188 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f780c010460 con 0x7f781c0ffdc0 2024-05-15T16:21:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.921+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781c1a0a70 con 0x7f781c0ffdc0 2024-05-15T16:21:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.921+0000 7f7803fff700 1 -- 172.21.15.49:0/3007695188 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f780c00f5f0 con 0x7f781c0ffdc0 2024-05-15T16:21:28.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.921+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781c1a0e80 con 0x7f781c0ffdc0 2024-05-15T16:21:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.922+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781c1081c0 con 0x7f781c0ffdc0 2024-05-15T16:21:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.923+0000 7f7803fff700 1 -- 172.21.15.49:0/3007695188 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f780c009a00 con 0x7f781c0ffdc0 2024-05-15T16:21:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.923+0000 7f7803fff700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f780406e210 0x7f78040706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.923+0000 7f781a59c700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f780406e210 0x7f78040706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.923+0000 7f7803fff700 1 -- 172.21.15.49:0/3007695188 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f780c08e070 con 0x7f781c0ffdc0 2024-05-15T16:21:28.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.924+0000 7f781a59c700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f780406e210 0x7f78040706d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7810009fd0 tx=0x7f781001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:28.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:28.928+0000 7f7803fff700 1 -- 172.21.15.49:0/3007695188 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f780c057350 con 0x7f781c0ffdc0 2024-05-15T16:21:29.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.145+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f781c068d30 con 0x7f780406e210 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.156+0000 7f7803fff700 1 -- 172.21.15.49:0/3007695188 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f781c068d30 con 0x7f780406e210 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (24m) 2m ago 36m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (36m) 2m ago 36m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (35m) 2m ago 35m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (22m) 2m ago 36m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (22m) 2m ago 35m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (24m) 2m ago 35m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (18m) 2m ago 30m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:21:29.157 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (18m) 2m ago 30m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (17m) 2m ago 30m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (2m) 2m ago 30m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (27m) 2m ago 37m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (25m) 2m ago 35m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (23m) 2m ago 37m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (22m) 2m ago 35m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (25m) 2m ago 36m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (25m) 2m ago 35m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (21m) 2m ago 34m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (21m) 2m ago 33m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:21:29.158 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (20m) 2m ago 33m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:21:29.159 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (19m) 2m ago 33m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:21:29.159 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (19m) 2m ago 32m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:21:29.159 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (18m) 2m ago 32m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:21:29.159 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (25m) 2m ago 35m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:21:29.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f780406e210 msgr2=0x7f78040706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f780406e210 0x7f78040706d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7810009fd0 tx=0x7f781001a040 comp rx=0 tx=0).stop 2024-05-15T16:21:29.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c0ffdc0 msgr2=0x7f781c197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c0ffdc0 0x7f781c197a00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f780c00d8d0 tx=0x7f780c00dbe0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 shutdown_connections 2024-05-15T16:21:29.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f780406e210 0x7f78040706d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f781c0ffdc0 0x7f781c197a00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 --2- 172.21.15.49:0/3007695188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f781c100770 0x7f781c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 >> 172.21.15.49:0/3007695188 conn(0x7f781c076800 msgr2=0x7f781c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:29.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 shutdown_connections 2024-05-15T16:21:29.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.159+0000 7f782125e700 1 -- 172.21.15.49:0/3007695188 wait complete. 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.266+0000 7f01d83f0700 1 -- 172.21.15.49:0/3317714479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 msgr2=0x7f01d010ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.266+0000 7f01d83f0700 1 --2- 172.21.15.49:0/3317714479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 0x7f01d010ac30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009910 tx=0x7f01c0009c20 comp rx=0 tx=0).stop 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.266+0000 7f01d83f0700 1 -- 172.21.15.49:0/3317714479 shutdown_connections 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.266+0000 7f01d83f0700 1 --2- 172.21.15.49:0/3317714479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 0x7f01d010ac30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.266+0000 7f01d83f0700 1 --2- 172.21.15.49:0/3317714479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d0101e20 0x7f01d0102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.266+0000 7f01d83f0700 1 -- 172.21.15.49:0/3317714479 >> 172.21.15.49:0/3317714479 conn(0x7f01d0076800 msgr2=0x7f01d00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.266+0000 7f01d83f0700 1 -- 172.21.15.49:0/3317714479 shutdown_connections 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.266+0000 7f01d83f0700 1 -- 172.21.15.49:0/3317714479 wait complete. 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d83f0700 1 Processor -- start 2024-05-15T16:21:29.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d83f0700 1 -- start start 2024-05-15T16:21:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d83f0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d0101e20 0x7f01d019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d83f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 0x7f01d019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d83f0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d019d3d0 con 0x7f01d0102740 2024-05-15T16:21:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d83f0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0196a40 con 0x7f01d0101e20 2024-05-15T16:21:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d598b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 0x7f01d019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d598b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 0x7f01d019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44610/0 (socket says 172.21.15.49:44610) 2024-05-15T16:21:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d598b700 1 -- 172.21.15.49:0/1265095149 learned_addr learned my addr 172.21.15.49:0/1265095149 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:21:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d618c700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d0101e20 0x7f01d019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d598b700 1 -- 172.21.15.49:0/1265095149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d0101e20 msgr2=0x7f01d019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d598b700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d0101e20 0x7f01d019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.267+0000 7f01d598b700 1 -- 172.21.15.49:0/1265095149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c00095f0 con 0x7f01d0102740 2024-05-15T16:21:29.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.268+0000 7f01d598b700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 0x7f01d019ce40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f01c00092e0 tx=0x7f01c000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:29.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.268+0000 7f01c77fe700 1 -- 172.21.15.49:0/1265095149 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01c001d070 con 0x7f01d0102740 2024-05-15T16:21:29.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.268+0000 7f01c77fe700 1 -- 172.21.15.49:0/1265095149 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01c0003d50 con 0x7f01d0102740 2024-05-15T16:21:29.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.268+0000 7f01c77fe700 1 -- 172.21.15.49:0/1265095149 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01c0017400 con 0x7f01d0102740 2024-05-15T16:21:29.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.268+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d0196cc0 con 0x7f01d0102740 2024-05-15T16:21:29.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.268+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d01971b0 con 0x7f01d0102740 2024-05-15T16:21:29.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.269+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d0105540 con 0x7f01d0102740 2024-05-15T16:21:29.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.269+0000 7f01c77fe700 1 -- 172.21.15.49:0/1265095149 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f01c0003890 con 0x7f01d0102740 2024-05-15T16:21:29.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.270+0000 7f01c77fe700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f01bc06e200 0x7f01bc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:29.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.270+0000 7f01c77fe700 1 -- 172.21.15.49:0/1265095149 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f01c008f4a0 con 0x7f01d0102740 2024-05-15T16:21:29.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.270+0000 7f01d618c700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f01bc06e200 0x7f01bc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:29.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.270+0000 7f01d618c700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f01bc06e200 0x7f01bc0706c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f01cc005ea0 tx=0x7f01cc005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:29.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.275+0000 7f01c77fe700 1 -- 172.21.15.49:0/1265095149 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f01c005c330 con 0x7f01d0102740 2024-05-15T16:21:29.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.573+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f01d0068f60 con 0x7f01d0102740 2024-05-15T16:21:29.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.573+0000 7f01c77fe700 1 -- 172.21.15.49:0/1265095149 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f01c005bec0 con 0x7f01d0102740 2024-05-15T16:21:29.574 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:21:29.574 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:21:29.574 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:21:29.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:21:29.574 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:21:29.574 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:21:29.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:21:29.575 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:21:29.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f01bc06e200 msgr2=0x7f01bc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f01bc06e200 0x7f01bc0706c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f01cc005ea0 tx=0x7f01cc005e30 comp rx=0 tx=0).stop 2024-05-15T16:21:29.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 msgr2=0x7f01d019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 0x7f01d019ce40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f01c00092e0 tx=0x7f01c000be20 comp rx=0 tx=0).stop 2024-05-15T16:21:29.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 shutdown_connections 2024-05-15T16:21:29.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f01bc06e200 0x7f01bc0706c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d0101e20 0x7f01d019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 --2- 172.21.15.49:0/1265095149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d0102740 0x7f01d019ce40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 >> 172.21.15.49:0/1265095149 conn(0x7f01d0076800 msgr2=0x7f01d0103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 shutdown_connections 2024-05-15T16:21:29.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.576+0000 7f01d83f0700 1 -- 172.21.15.49:0/1265095149 wait complete. 2024-05-15T16:21:29.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.684+0000 7f8ab38e3700 1 -- 172.21.15.49:0/1239251975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 msgr2=0x7f8aac102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.684+0000 7f8ab38e3700 1 --2- 172.21.15.49:0/1239251975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 0x7f8aac102200 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009910 tx=0x7f8aa8009c20 comp rx=0 tx=0).stop 2024-05-15T16:21:29.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.684+0000 7f8ab38e3700 1 -- 172.21.15.49:0/1239251975 shutdown_connections 2024-05-15T16:21:29.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.684+0000 7f8ab38e3700 1 --2- 172.21.15.49:0/1239251975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac102740 0x7f8aac10ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.684+0000 7f8ab38e3700 1 --2- 172.21.15.49:0/1239251975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 0x7f8aac102200 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.684+0000 7f8ab38e3700 1 -- 172.21.15.49:0/1239251975 >> 172.21.15.49:0/1239251975 conn(0x7f8aac076800 msgr2=0x7f8aac0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:29.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.684+0000 7f8ab38e3700 1 -- 172.21.15.49:0/1239251975 shutdown_connections 2024-05-15T16:21:29.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.684+0000 7f8ab38e3700 1 -- 172.21.15.49:0/1239251975 wait complete. 2024-05-15T16:21:29.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab38e3700 1 Processor -- start 2024-05-15T16:21:29.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab38e3700 1 -- start start 2024-05-15T16:21:29.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab38e3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 0x7f8aac0ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:29.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab38e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac102740 0x7f8aac0fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab38e3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac100500 con 0x7f8aac101e20 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab38e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac100640 con 0x7f8aac102740 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab167f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 0x7f8aac0ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab167f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 0x7f8aac0ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44616/0 (socket says 172.21.15.49:44616) 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab167f700 1 -- 172.21.15.49:0/2876544942 learned_addr learned my addr 172.21.15.49:0/2876544942 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab0e7e700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac102740 0x7f8aac0fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab167f700 1 -- 172.21.15.49:0/2876544942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac102740 msgr2=0x7f8aac0fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab167f700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac102740 0x7f8aac0fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab167f700 1 -- 172.21.15.49:0/2876544942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa80095f0 con 0x7f8aac101e20 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.685+0000 7f8ab0e7e700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac102740 0x7f8aac0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:21:29.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.686+0000 7f8ab167f700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 0x7f8aac0ffa80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8006010 tx=0x7f8aa800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:29.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.686+0000 7f8aa27fc700 1 -- 172.21.15.49:0/2876544942 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aa801d070 con 0x7f8aac101e20 2024-05-15T16:21:29.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.686+0000 7f8aa27fc700 1 -- 172.21.15.49:0/2876544942 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8aa800f460 con 0x7f8aac101e20 2024-05-15T16:21:29.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.686+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac1008c0 con 0x7f8aac101e20 2024-05-15T16:21:29.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.686+0000 7f8aa27fc700 1 -- 172.21.15.49:0/2876544942 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aa8017630 con 0x7f8aac101e20 2024-05-15T16:21:29.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.686+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac1a3000 con 0x7f8aac101e20 2024-05-15T16:21:29.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.687+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac068f60 con 0x7f8aac101e20 2024-05-15T16:21:29.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.688+0000 7f8aa27fc700 1 -- 172.21.15.49:0/2876544942 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8aa800f5d0 con 0x7f8aac101e20 2024-05-15T16:21:29.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.688+0000 7f8aa27fc700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9806e200 0x7f8a980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:29.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.688+0000 7f8aa27fc700 1 -- 172.21.15.49:0/2876544942 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8aa808f610 con 0x7f8aac101e20 2024-05-15T16:21:29.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.688+0000 7f8ab0e7e700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9806e200 0x7f8a980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:29.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.689+0000 7f8ab0e7e700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9806e200 0x7f8a980706c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8aac1011b0 tx=0x7f8a9c00a2b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:29.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.693+0000 7f8aa27fc700 1 -- 172.21.15.49:0/2876544942 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8aa8058a30 con 0x7f8aac101e20 2024-05-15T16:21:29.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.944+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8aac10dea0 con 0x7f8aac101e20 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.944+0000 7f8aa27fc700 1 -- 172.21.15.49:0/2876544942 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f8aa805c0f0 con 0x7f8aac101e20 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:21:29.945 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:21:29.946 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:21:29.947 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:21:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:21:29.948 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:21:29.948 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:21:29.948 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:21:29.948 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.946+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9806e200 msgr2=0x7f8a980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.946+0000 7f8ab38e3700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9806e200 0x7f8a980706c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8aac1011b0 tx=0x7f8a9c00a2b0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.946+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 msgr2=0x7f8aac0ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.946+0000 7f8ab38e3700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 0x7f8aac0ffa80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8006010 tx=0x7f8aa800be20 comp rx=0 tx=0).stop 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.947+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 shutdown_connections 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.947+0000 7f8ab38e3700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9806e200 0x7f8a980706c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.947+0000 7f8ab38e3700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aac101e20 0x7f8aac0ffa80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.947+0000 7f8ab38e3700 1 --2- 172.21.15.49:0/2876544942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac102740 0x7f8aac0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.947+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 >> 172.21.15.49:0/2876544942 conn(0x7f8aac076800 msgr2=0x7f8aac103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.947+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 shutdown_connections 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:29.947+0000 7f8ab38e3700 1 -- 172.21.15.49:0/2876544942 wait complete. 2024-05-15T16:21:29.949 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:21:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.056+0000 7f14318d2700 1 -- 172.21.15.49:0/1411549996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 msgr2=0x7f142c1001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.056+0000 7f14318d2700 1 --2- 172.21.15.49:0/1411549996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 0x7f142c1001f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f141c009910 tx=0x7f141c009c20 comp rx=0 tx=0).stop 2024-05-15T16:21:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.056+0000 7f14318d2700 1 -- 172.21.15.49:0/1411549996 shutdown_connections 2024-05-15T16:21:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.056+0000 7f14318d2700 1 --2- 172.21.15.49:0/1411549996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f142c1007c0 0x7f142c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.056+0000 7f14318d2700 1 --2- 172.21.15.49:0/1411549996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 0x7f142c1001f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.056+0000 7f14318d2700 1 -- 172.21.15.49:0/1411549996 >> 172.21.15.49:0/1411549996 conn(0x7f142c0fb700 msgr2=0x7f142c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.056+0000 7f14318d2700 1 -- 172.21.15.49:0/1411549996 shutdown_connections 2024-05-15T16:21:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.056+0000 7f14318d2700 1 -- 172.21.15.49:0/1411549996 wait complete. 2024-05-15T16:21:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.057+0000 7f14318d2700 1 Processor -- start 2024-05-15T16:21:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.057+0000 7f14318d2700 1 -- start start 2024-05-15T16:21:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.057+0000 7f14318d2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 0x7f142c1018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.057+0000 7f14318d2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f142c1007c0 0x7f142c101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.057+0000 7f14318d2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c102330 con 0x7f142c0ffe10 2024-05-15T16:21:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.057+0000 7f14318d2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1024a0 con 0x7f142c1007c0 2024-05-15T16:21:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.057+0000 7f142affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 0x7f142c1018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.057+0000 7f142affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 0x7f142c1018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44642/0 (socket says 172.21.15.49:44642) 2024-05-15T16:21:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.057+0000 7f142affd700 1 -- 172.21.15.49:0/2074307378 learned_addr learned my addr 172.21.15.49:0/2074307378 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f142a7fc700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f142c1007c0 0x7f142c101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f142affd700 1 -- 172.21.15.49:0/2074307378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f142c1007c0 msgr2=0x7f142c101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f142affd700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f142c1007c0 0x7f142c101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f142affd700 1 -- 172.21.15.49:0/2074307378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0095f0 con 0x7f142c0ffe10 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f142affd700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 0x7f142c1018b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f141c0092e0 tx=0x7f141c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f14308d0700 1 -- 172.21.15.49:0/2074307378 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141c01d070 con 0x7f142c0ffe10 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f14308d0700 1 -- 172.21.15.49:0/2074307378 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f141c00f460 con 0x7f142c0ffe10 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c102720 con 0x7f142c0ffe10 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f14308d0700 1 -- 172.21.15.49:0/2074307378 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141c017630 con 0x7f142c0ffe10 2024-05-15T16:21:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.058+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c1a3010 con 0x7f142c0ffe10 2024-05-15T16:21:30.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.059+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c1012f0 con 0x7f142c0ffe10 2024-05-15T16:21:30.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.060+0000 7f14308d0700 1 -- 172.21.15.49:0/2074307378 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f141c00f5d0 con 0x7f142c0ffe10 2024-05-15T16:21:30.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.060+0000 7f14308d0700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f141806e210 0x7f14180706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:30.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.061+0000 7f142a7fc700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f141806e210 0x7f14180706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:30.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.061+0000 7f14308d0700 1 -- 172.21.15.49:0/2074307378 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f141c08f380 con 0x7f142c0ffe10 2024-05-15T16:21:30.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.061+0000 7f142a7fc700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f141806e210 0x7f14180706d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f142c102fe0 tx=0x7f141400a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:30.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.065+0000 7f14308d0700 1 -- 172.21.15.49:0/2074307378 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f141c0586e0 con 0x7f142c0ffe10 2024-05-15T16:21:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:29 smithi144 ceph-mon[86635]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:21:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:29 smithi144 ceph-mon[86635]: pgmap v735: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:29 smithi144 ceph-mon[86635]: from='client.35358 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:21:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:29 smithi144 ceph-mon[86635]: from='client.35362 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:21:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:29 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1265095149' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:21:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:29 smithi049 ceph-mon[112142]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:21:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:29 smithi049 ceph-mon[112142]: pgmap v735: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:29 smithi049 ceph-mon[112142]: from='client.35358 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:21:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:29 smithi049 ceph-mon[112142]: from='client.35362 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:21:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:29 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1265095149' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:21:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.292+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f142c102720 con 0x7f141806e210 2024-05-15T16:21:30.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.293+0000 7f14308d0700 1 -- 172.21.15.49:0/2074307378 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f142c102720 con 0x7f141806e210 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:21:30.295 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:21:30.296 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:21:30.296 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.295+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f141806e210 msgr2=0x7f14180706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.295+0000 7f14318d2700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f141806e210 0x7f14180706d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f142c102fe0 tx=0x7f141400a380 comp rx=0 tx=0).stop 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.295+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 msgr2=0x7f142c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.295+0000 7f14318d2700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 0x7f142c1018b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f141c0092e0 tx=0x7f141c00bd70 comp rx=0 tx=0).stop 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.295+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 shutdown_connections 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.295+0000 7f14318d2700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f141806e210 0x7f14180706d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.296+0000 7f14318d2700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f142c0ffe10 0x7f142c1018b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.296+0000 7f14318d2700 1 --2- 172.21.15.49:0/2074307378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f142c1007c0 0x7f142c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.296+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 >> 172.21.15.49:0/2074307378 conn(0x7f142c0fb700 msgr2=0x7f142c069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.296+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 shutdown_connections 2024-05-15T16:21:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.296+0000 7f14318d2700 1 -- 172.21.15.49:0/2074307378 wait complete. 2024-05-15T16:21:30.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.405+0000 7f97a92d9700 1 -- 172.21.15.49:0/749874611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 msgr2=0x7f97a41001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:30.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.405+0000 7f97a92d9700 1 --2- 172.21.15.49:0/749874611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 0x7f97a41001e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9794009910 tx=0x7f9794009c20 comp rx=0 tx=0).stop 2024-05-15T16:21:30.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.406+0000 7f97a92d9700 1 -- 172.21.15.49:0/749874611 shutdown_connections 2024-05-15T16:21:30.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.406+0000 7f97a92d9700 1 --2- 172.21.15.49:0/749874611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a41007b0 0x7f97a4104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.406+0000 7f97a92d9700 1 --2- 172.21.15.49:0/749874611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 0x7f97a41001e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.406+0000 7f97a92d9700 1 -- 172.21.15.49:0/749874611 >> 172.21.15.49:0/749874611 conn(0x7f97a40fb6d0 msgr2=0x7f97a40fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:30.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.406+0000 7f97a92d9700 1 -- 172.21.15.49:0/749874611 shutdown_connections 2024-05-15T16:21:30.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.406+0000 7f97a92d9700 1 -- 172.21.15.49:0/749874611 wait complete. 2024-05-15T16:21:30.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.406+0000 7f97a92d9700 1 Processor -- start 2024-05-15T16:21:30.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.406+0000 7f97a92d9700 1 -- start start 2024-05-15T16:21:30.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a92d9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 0x7f97a419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a92d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a41007b0 0x7f97a419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a92d9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a419d450 con 0x7f97a40ffe00 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a92d9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a4196a30 con 0x7f97a41007b0 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 0x7f97a419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 0x7f97a419c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44654/0 (socket says 172.21.15.49:44654) 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a2ffd700 1 -- 172.21.15.49:0/1538565473 learned_addr learned my addr 172.21.15.49:0/1538565473 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a2ffd700 1 -- 172.21.15.49:0/1538565473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a41007b0 msgr2=0x7f97a419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a2ffd700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a41007b0 0x7f97a419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a2ffd700 1 -- 172.21.15.49:0/1538565473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940095f0 con 0x7f97a40ffe00 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f97a2ffd700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 0x7f97a419c8f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f97940092e0 tx=0x7f979400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.407+0000 7f978bfff700 1 -- 172.21.15.49:0/1538565473 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979401d070 con 0x7f97a40ffe00 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.408+0000 7f978bfff700 1 -- 172.21.15.49:0/1538565473 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97940056e0 con 0x7f97a40ffe00 2024-05-15T16:21:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.408+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a4196cb0 con 0x7f97a40ffe00 2024-05-15T16:21:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.408+0000 7f978bfff700 1 -- 172.21.15.49:0/1538565473 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979400fe80 con 0x7f97a40ffe00 2024-05-15T16:21:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.408+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a41971a0 con 0x7f97a40ffe00 2024-05-15T16:21:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.408+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a41012e0 con 0x7f97a40ffe00 2024-05-15T16:21:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.409+0000 7f978bfff700 1 -- 172.21.15.49:0/1538565473 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f979400f6f0 con 0x7f97a40ffe00 2024-05-15T16:21:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.410+0000 7f978bfff700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f978c06e210 0x7f978c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:21:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.410+0000 7f978bfff700 1 -- 172.21.15.49:0/1538565473 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f979408f5d0 con 0x7f97a40ffe00 2024-05-15T16:21:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.410+0000 7f97a27fc700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f978c06e210 0x7f978c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:21:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.410+0000 7f97a27fc700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f978c06e210 0x7f978c0706d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9798005fd0 tx=0x7f9798005f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:21:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.414+0000 7f978bfff700 1 -- 172.21.15.49:0/1538565473 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f979405c3e0 con 0x7f97a40ffe00 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.711+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f97a410ce90 con 0x7f97a40ffe00 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.711+0000 7f978bfff700 1 -- 172.21.15.49:0/1538565473 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f979405bf70 con 0x7f97a40ffe00 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:21:30.713 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:21:30.714 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:21:30.715 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:21:30.715 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:21:30.715 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:21:30.715 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:21:30.715 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:21:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.713+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f978c06e210 msgr2=0x7f978c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.713+0000 7f97a92d9700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f978c06e210 0x7f978c0706d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9798005fd0 tx=0x7f9798005f60 comp rx=0 tx=0).stop 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.713+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 msgr2=0x7f97a419c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.713+0000 7f97a92d9700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 0x7f97a419c8f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f97940092e0 tx=0x7f979400be20 comp rx=0 tx=0).stop 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.713+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 shutdown_connections 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.713+0000 7f97a92d9700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f978c06e210 0x7f978c0706d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.713+0000 7f97a92d9700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a40ffe00 0x7f97a419c8f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.713+0000 7f97a92d9700 1 --2- 172.21.15.49:0/1538565473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a41007b0 0x7f97a419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.713+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 >> 172.21.15.49:0/1538565473 conn(0x7f97a40fb6d0 msgr2=0x7f97a40694d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.714+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 shutdown_connections 2024-05-15T16:21:30.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:21:30.714+0000 7f97a92d9700 1 -- 172.21.15.49:0/1538565473 wait complete. 2024-05-15T16:21:31.161 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:30 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2876544942' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:21:31.161 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:30 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1538565473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:21:31.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:30 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2876544942' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:21:31.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:30 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1538565473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:21:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:31 smithi144 ceph-mon[86635]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:21:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:31 smithi144 ceph-mon[86635]: pgmap v736: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:31 smithi049 ceph-mon[112142]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:21:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:31 smithi049 ceph-mon[112142]: pgmap v736: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:33 smithi144 ceph-mon[86635]: pgmap v737: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:33 smithi049 ceph-mon[112142]: pgmap v737: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:35 smithi144 ceph-mon[86635]: pgmap v738: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:21:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:35 smithi049 ceph-mon[112142]: pgmap v738: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:21:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:37 smithi144 ceph-mon[86635]: pgmap v739: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:37 smithi049 ceph-mon[112142]: pgmap v739: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:39 smithi049 ceph-mon[112142]: pgmap v740: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:40.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:39 smithi144 ceph-mon[86635]: pgmap v740: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:41 smithi049 ceph-mon[112142]: pgmap v741: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:41 smithi144 ceph-mon[86635]: pgmap v741: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:43 smithi144 ceph-mon[86635]: pgmap v742: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:43 smithi049 ceph-mon[112142]: pgmap v742: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:45 smithi144 ceph-mon[86635]: pgmap v743: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:45 smithi049 ceph-mon[112142]: pgmap v743: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:47 smithi144 ceph-mon[86635]: pgmap v744: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:48.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:47 smithi049 ceph-mon[112142]: pgmap v744: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:49 smithi144 ceph-mon[86635]: pgmap v745: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:50.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:49 smithi049 ceph-mon[112142]: pgmap v745: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:21:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:21:52.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:51 smithi144 ceph-mon[86635]: pgmap v746: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:51 smithi049 ceph-mon[112142]: pgmap v746: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:53 smithi144 ceph-mon[86635]: pgmap v747: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:53 smithi049 ceph-mon[112142]: pgmap v747: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:55 smithi144 ceph-mon[86635]: pgmap v748: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:55 smithi049 ceph-mon[112142]: pgmap v748: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:57 smithi144 ceph-mon[86635]: pgmap v749: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:21:58.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:57 smithi049 ceph-mon[112142]: pgmap v749: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:21:59 smithi049 ceph-mon[112142]: pgmap v750: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:21:59 smithi144 ceph-mon[86635]: pgmap v750: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.838+0000 7fbf76a6d700 1 -- 172.21.15.49:0/2628915928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 msgr2=0x7fbf701001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.838+0000 7fbf76a6d700 1 --2- 172.21.15.49:0/2628915928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 0x7fbf701001a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009910 tx=0x7fbf60009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.838+0000 7fbf76a6d700 1 -- 172.21.15.49:0/2628915928 shutdown_connections 2024-05-15T16:22:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.838+0000 7fbf76a6d700 1 --2- 172.21.15.49:0/2628915928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf70100770 0x7fbf701047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:00.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.838+0000 7fbf76a6d700 1 --2- 172.21.15.49:0/2628915928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 0x7fbf701001a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:00.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.838+0000 7fbf76a6d700 1 -- 172.21.15.49:0/2628915928 >> 172.21.15.49:0/2628915928 conn(0x7fbf70076800 msgr2=0x7fbf700fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:00.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.838+0000 7fbf76a6d700 1 -- 172.21.15.49:0/2628915928 shutdown_connections 2024-05-15T16:22:00.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.838+0000 7fbf76a6d700 1 -- 172.21.15.49:0/2628915928 wait complete. 2024-05-15T16:22:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.839+0000 7fbf76a6d700 1 Processor -- start 2024-05-15T16:22:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.839+0000 7fbf76a6d700 1 -- start start 2024-05-15T16:22:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.839+0000 7fbf76a6d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 0x7fbf70198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.839+0000 7fbf76a6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf70100770 0x7fbf70199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.839+0000 7fbf76a6d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70199b50 con 0x7fbf700ffdc0 2024-05-15T16:22:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.839+0000 7fbf76a6d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7019d7d0 con 0x7fbf70100770 2024-05-15T16:22:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf74809700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 0x7fbf70198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf74809700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 0x7fbf70198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38076/0 (socket says 172.21.15.49:38076) 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf74809700 1 -- 172.21.15.49:0/3862937422 learned_addr learned my addr 172.21.15.49:0/3862937422 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf6ffff700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf70100770 0x7fbf70199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf74809700 1 -- 172.21.15.49:0/3862937422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf70100770 msgr2=0x7fbf70199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf74809700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf70100770 0x7fbf70199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf74809700 1 -- 172.21.15.49:0/3862937422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf600095f0 con 0x7fbf700ffdc0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf6ffff700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf70100770 0x7fbf70199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf74809700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 0x7fbf70198f30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006010 tx=0x7fbf6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf6dffb700 1 -- 172.21.15.49:0/3862937422 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf6001d070 con 0x7fbf700ffdc0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf6dffb700 1 -- 172.21.15.49:0/3862937422 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf6000f460 con 0x7fbf700ffdc0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7019da50 con 0x7fbf700ffdc0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.840+0000 7fbf6dffb700 1 -- 172.21.15.49:0/3862937422 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf60017630 con 0x7fbf700ffdc0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.841+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7019df40 con 0x7fbf700ffdc0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.842+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf701012a0 con 0x7fbf700ffdc0 2024-05-15T16:22:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.842+0000 7fbf6dffb700 1 -- 172.21.15.49:0/3862937422 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbf6000f5d0 con 0x7fbf700ffdc0 2024-05-15T16:22:00.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.843+0000 7fbf6dffb700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf5806e210 0x7fbf580706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:00.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.843+0000 7fbf6dffb700 1 -- 172.21.15.49:0/3862937422 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbf6008f5a0 con 0x7fbf700ffdc0 2024-05-15T16:22:00.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.843+0000 7fbf6ffff700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf5806e210 0x7fbf580706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:00.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.843+0000 7fbf6ffff700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf5806e210 0x7fbf580706d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf7019a440 tx=0x7fbf64008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:00.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:00.848+0000 7fbf6dffb700 1 -- 172.21.15.49:0/3862937422 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbf600589c0 con 0x7fbf700ffdc0 2024-05-15T16:22:01.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.073+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf7010ce00 con 0x7fbf5806e210 2024-05-15T16:22:01.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.074+0000 7fbf6dffb700 1 -- 172.21.15.49:0/3862937422 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbf7010ce00 con 0x7fbf5806e210 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf5806e210 msgr2=0x7fbf580706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf5806e210 0x7fbf580706d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf7019a440 tx=0x7fbf64008040 comp rx=0 tx=0).stop 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 msgr2=0x7fbf70198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 0x7fbf70198f30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006010 tx=0x7fbf6000be20 comp rx=0 tx=0).stop 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 shutdown_connections 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbf5806e210 0x7fbf580706d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf700ffdc0 0x7fbf70198f30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 --2- 172.21.15.49:0/3862937422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf70100770 0x7fbf70199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 >> 172.21.15.49:0/3862937422 conn(0x7fbf70076800 msgr2=0x7fbf70068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 shutdown_connections 2024-05-15T16:22:01.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.076+0000 7fbf76a6d700 1 -- 172.21.15.49:0/3862937422 wait complete. 2024-05-15T16:22:01.087 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:22:01.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.188+0000 7f08738f7700 1 -- 172.21.15.49:0/490397752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c100ee0 msgr2=0x7f086c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.188+0000 7f08738f7700 1 --2- 172.21.15.49:0/490397752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c100ee0 0x7f086c1012c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0868009910 tx=0x7f0868009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:01.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.188+0000 7f08738f7700 1 -- 172.21.15.49:0/490397752 shutdown_connections 2024-05-15T16:22:01.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.188+0000 7f08738f7700 1 --2- 172.21.15.49:0/490397752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f086c101890 0x7f086c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.188+0000 7f08738f7700 1 --2- 172.21.15.49:0/490397752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c100ee0 0x7f086c1012c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.188+0000 7f08738f7700 1 -- 172.21.15.49:0/490397752 >> 172.21.15.49:0/490397752 conn(0x7f086c0fc790 msgr2=0x7f086c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.188+0000 7f08738f7700 1 -- 172.21.15.49:0/490397752 shutdown_connections 2024-05-15T16:22:01.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.188+0000 7f08738f7700 1 -- 172.21.15.49:0/490397752 wait complete. 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.188+0000 7f08738f7700 1 Processor -- start 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f08738f7700 1 -- start start 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f08738f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f086c100ee0 0x7f086c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f08738f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c101890 0x7f086c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f08738f7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c19d3d0 con 0x7f086c101890 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f08738f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c196a90 con 0x7f086c100ee0 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f0870e92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c101890 0x7f086c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f0870e92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c101890 0x7f086c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38096/0 (socket says 172.21.15.49:38096) 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f0870e92700 1 -- 172.21.15.49:0/793655537 learned_addr learned my addr 172.21.15.49:0/793655537 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f0871693700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f086c100ee0 0x7f086c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f0870e92700 1 -- 172.21.15.49:0/793655537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f086c100ee0 msgr2=0x7f086c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f0870e92700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f086c100ee0 0x7f086c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f0870e92700 1 -- 172.21.15.49:0/793655537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680095f0 con 0x7f086c101890 2024-05-15T16:22:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.189+0000 7f0870e92700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c101890 0x7f086c19ce40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f085c00c960 tx=0x7f085c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.190+0000 7f08627fc700 1 -- 172.21.15.49:0/793655537 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f085c007a70 con 0x7f086c101890 2024-05-15T16:22:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.190+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c196d70 con 0x7f086c101890 2024-05-15T16:22:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.190+0000 7f08627fc700 1 -- 172.21.15.49:0/793655537 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f085c007bd0 con 0x7f086c101890 2024-05-15T16:22:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.190+0000 7f08627fc700 1 -- 172.21.15.49:0/793655537 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f085c0186c0 con 0x7f086c101890 2024-05-15T16:22:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.190+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086c1972c0 con 0x7f086c101890 2024-05-15T16:22:01.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.191+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086c109280 con 0x7f086c101890 2024-05-15T16:22:01.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.191+0000 7f08627fc700 1 -- 172.21.15.49:0/793655537 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f085c00faa0 con 0x7f086c101890 2024-05-15T16:22:01.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.192+0000 7f08627fc700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f085806e200 0x7f08580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:01.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.192+0000 7f08627fc700 1 -- 172.21.15.49:0/793655537 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f085c08e1d0 con 0x7f086c101890 2024-05-15T16:22:01.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.192+0000 7f0871693700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f085806e200 0x7f08580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:01.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.192+0000 7f0871693700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f085806e200 0x7f08580706c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0868005f50 tx=0x7f086801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:01.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.196+0000 7f08627fc700 1 -- 172.21.15.49:0/793655537 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f085c05ad50 con 0x7f086c101890 2024-05-15T16:22:01.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.425+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f086c197f70 con 0x7f085806e200 2024-05-15T16:22:01.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.427+0000 7f08627fc700 1 -- 172.21.15.49:0/793655537 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f086c197f70 con 0x7f085806e200 2024-05-15T16:22:01.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f085806e200 msgr2=0x7f08580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f085806e200 0x7f08580706c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0868005f50 tx=0x7f086801a040 comp rx=0 tx=0).stop 2024-05-15T16:22:01.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c101890 msgr2=0x7f086c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c101890 0x7f086c19ce40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f085c00c960 tx=0x7f085c00cd20 comp rx=0 tx=0).stop 2024-05-15T16:22:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 shutdown_connections 2024-05-15T16:22:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f085806e200 0x7f08580706c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f086c100ee0 0x7f086c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 --2- 172.21.15.49:0/793655537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f086c101890 0x7f086c19ce40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 >> 172.21.15.49:0/793655537 conn(0x7f086c0fc790 msgr2=0x7f086c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 shutdown_connections 2024-05-15T16:22:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.429+0000 7f08738f7700 1 -- 172.21.15.49:0/793655537 wait complete. 2024-05-15T16:22:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.537+0000 7fa6e85ea700 1 -- 172.21.15.49:0/868213242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 msgr2=0x7fa6e01047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.537+0000 7fa6e85ea700 1 --2- 172.21.15.49:0/868213242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 0x7fa6e01047c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009910 tx=0x7fa6d4009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.537+0000 7fa6e85ea700 1 -- 172.21.15.49:0/868213242 shutdown_connections 2024-05-15T16:22:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.537+0000 7fa6e85ea700 1 --2- 172.21.15.49:0/868213242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 0x7fa6e01047c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.537+0000 7fa6e85ea700 1 --2- 172.21.15.49:0/868213242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e00ffdc0 0x7fa6e01001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.537+0000 7fa6e85ea700 1 -- 172.21.15.49:0/868213242 >> 172.21.15.49:0/868213242 conn(0x7fa6e0076800 msgr2=0x7fa6e00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.537+0000 7fa6e85ea700 1 -- 172.21.15.49:0/868213242 shutdown_connections 2024-05-15T16:22:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.537+0000 7fa6e85ea700 1 -- 172.21.15.49:0/868213242 wait complete. 2024-05-15T16:22:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e85ea700 1 Processor -- start 2024-05-15T16:22:01.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e85ea700 1 -- start start 2024-05-15T16:22:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e85ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e00ffdc0 0x7fa6e00fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e85ea700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 0x7fa6e00fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e85ea700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e00fc760 con 0x7fa6e0100770 2024-05-15T16:22:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e85ea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e00fc8a0 con 0x7fa6e00ffdc0 2024-05-15T16:22:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e5b85700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 0x7fa6e00fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e5b85700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 0x7fa6e00fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38120/0 (socket says 172.21.15.49:38120) 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e5b85700 1 -- 172.21.15.49:0/2456175431 learned_addr learned my addr 172.21.15.49:0/2456175431 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.538+0000 7fa6e6386700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e00ffdc0 0x7fa6e00fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6e5b85700 1 -- 172.21.15.49:0/2456175431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e00ffdc0 msgr2=0x7fa6e00fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6e5b85700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e00ffdc0 0x7fa6e00fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6e5b85700 1 -- 172.21.15.49:0/2456175431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d40095f0 con 0x7fa6e0100770 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6e6386700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e00ffdc0 0x7fa6e00fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6e5b85700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 0x7fa6e00fc220 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400bd70 tx=0x7fa6d400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6d37fe700 1 -- 172.21.15.49:0/2456175431 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d401d070 con 0x7fa6e0100770 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6d37fe700 1 -- 172.21.15.49:0/2456175431 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6d400f460 con 0x7fa6e0100770 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6d37fe700 1 -- 172.21.15.49:0/2456175431 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d4017630 con 0x7fa6e0100770 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e00fcb20 con 0x7fa6e0100770 2024-05-15T16:22:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.539+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e01a2fe0 con 0x7fa6e0100770 2024-05-15T16:22:01.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.540+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e01012a0 con 0x7fa6e0100770 2024-05-15T16:22:01.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.541+0000 7fa6d37fe700 1 -- 172.21.15.49:0/2456175431 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa6d400f5d0 con 0x7fa6e0100770 2024-05-15T16:22:01.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.541+0000 7fa6d37fe700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa6cc06e200 0x7fa6cc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.541+0000 7fa6d37fe700 1 -- 172.21.15.49:0/2456175431 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa6d408f5d0 con 0x7fa6e0100770 2024-05-15T16:22:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.541+0000 7fa6e6386700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa6cc06e200 0x7fa6cc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.542+0000 7fa6e6386700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa6cc06e200 0x7fa6cc0706c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005950 tx=0x7fa6dc005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.546+0000 7fa6d37fe700 1 -- 172.21.15.49:0/2456175431 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa6d40588b0 con 0x7fa6e0100770 2024-05-15T16:22:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.766+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6e010cec0 con 0x7fa6cc06e200 2024-05-15T16:22:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.777+0000 7fa6d37fe700 1 -- 172.21.15.49:0/2456175431 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa6e010cec0 con 0x7fa6cc06e200 2024-05-15T16:22:01.777 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:22:01.777 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (25m) 3m ago 36m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (37m) 3m ago 37m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (35m) 3m ago 35m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (22m) 3m ago 37m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (22m) 3m ago 35m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (24m) 3m ago 36m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (18m) 3m ago 31m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (18m) 3m ago 31m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (18m) 3m ago 31m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (3m) 3m ago 31m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (27m) 3m ago 38m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (26m) 3m ago 35m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (23m) 3m ago 38m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:22:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (23m) 3m ago 35m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:22:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (26m) 3m ago 36m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:22:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (26m) 3m ago 35m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:22:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (22m) 3m ago 34m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:22:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (21m) 3m ago 34m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:22:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (20m) 3m ago 34m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:22:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (20m) 3m ago 33m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:22:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (19m) 3m ago 33m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:22:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (19m) 3m ago 32m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:22:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (25m) 3m ago 36m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:22:01.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa6cc06e200 msgr2=0x7fa6cc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa6cc06e200 0x7fa6cc0706c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005950 tx=0x7fa6dc005f50 comp rx=0 tx=0).stop 2024-05-15T16:22:01.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 msgr2=0x7fa6e00fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 0x7fa6e00fc220 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400bd70 tx=0x7fa6d400be50 comp rx=0 tx=0).stop 2024-05-15T16:22:01.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 shutdown_connections 2024-05-15T16:22:01.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa6cc06e200 0x7fa6cc0706c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e00ffdc0 0x7fa6e00fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 --2- 172.21.15.49:0/2456175431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6e0100770 0x7fa6e00fc220 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 >> 172.21.15.49:0/2456175431 conn(0x7fa6e0076800 msgr2=0x7fa6e0068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:01.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 shutdown_connections 2024-05-15T16:22:01.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.780+0000 7fa6e85ea700 1 -- 172.21.15.49:0/2456175431 wait complete. 2024-05-15T16:22:01.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.887+0000 7fa7e20b6700 1 -- 172.21.15.49:0/4222220574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 msgr2=0x7fa7dc1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.887+0000 7fa7e20b6700 1 --2- 172.21.15.49:0/4222220574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 0x7fa7dc1001d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009910 tx=0x7fa7cc009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:01.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.887+0000 7fa7e20b6700 1 -- 172.21.15.49:0/4222220574 shutdown_connections 2024-05-15T16:22:01.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.887+0000 7fa7e20b6700 1 --2- 172.21.15.49:0/4222220574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1007a0 0x7fa7dc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.887+0000 7fa7e20b6700 1 --2- 172.21.15.49:0/4222220574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 0x7fa7dc1001d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.887+0000 7fa7e20b6700 1 -- 172.21.15.49:0/4222220574 >> 172.21.15.49:0/4222220574 conn(0x7fa7dc0fb6a0 msgr2=0x7fa7dc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.887+0000 7fa7e20b6700 1 -- 172.21.15.49:0/4222220574 shutdown_connections 2024-05-15T16:22:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.887+0000 7fa7e20b6700 1 -- 172.21.15.49:0/4222220574 wait complete. 2024-05-15T16:22:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7e20b6700 1 Processor -- start 2024-05-15T16:22:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7e20b6700 1 -- start start 2024-05-15T16:22:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7e20b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 0x7fa7dc101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7e20b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1007a0 0x7fa7dc101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:01.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7e20b6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc102310 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7e20b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc102450 con 0x7fa7dc1007a0 2024-05-15T16:22:01.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7db7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 0x7fa7dc101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:01.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7db7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 0x7fa7dc101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38146/0 (socket says 172.21.15.49:38146) 2024-05-15T16:22:01.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7db7fe700 1 -- 172.21.15.49:0/1487487841 learned_addr learned my addr 172.21.15.49:0/1487487841 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:01.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7daffd700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1007a0 0x7fa7dc101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:01.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7db7fe700 1 -- 172.21.15.49:0/1487487841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1007a0 msgr2=0x7fa7dc101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:01.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7db7fe700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1007a0 0x7fa7dc101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:01.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.888+0000 7fa7db7fe700 1 -- 172.21.15.49:0/1487487841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc0095f0 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.889+0000 7fa7db7fe700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 0x7fa7dc101890 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:01.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.889+0000 7fa7d8ff9700 1 -- 172.21.15.49:0/1487487841 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7cc01d070 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.889+0000 7fa7d8ff9700 1 -- 172.21.15.49:0/1487487841 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7cc0056e0 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.889+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc1026d0 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.889+0000 7fa7d8ff9700 1 -- 172.21.15.49:0/1487487841 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7cc00fe40 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.889+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc1a3060 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.890+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7dc108190 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.890+0000 7fa7d8ff9700 1 -- 172.21.15.49:0/1487487841 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa7cc00f680 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.891+0000 7fa7d8ff9700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa7c406e210 0x7fa7c40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.891+0000 7fa7d8ff9700 1 -- 172.21.15.49:0/1487487841 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa7cc08f460 con 0x7fa7dc0ffdf0 2024-05-15T16:22:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.891+0000 7fa7daffd700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa7c406e210 0x7fa7c40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.891+0000 7fa7daffd700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa7c406e210 0x7fa7c40706d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc102fc0 tx=0x7fa7d000b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:01.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:01.896+0000 7fa7d8ff9700 1 -- 172.21.15.49:0/1487487841 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa7cc0587c0 con 0x7fa7dc0ffdf0 2024-05-15T16:22:02.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.190+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa7dc10cdd0 con 0x7fa7dc0ffdf0 2024-05-15T16:22:02.190 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:01 smithi049 ceph-mon[112142]: pgmap v751: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:02.190 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:01 smithi049 ceph-mon[112142]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:02.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.190+0000 7fa7d8ff9700 1 -- 172.21.15.49:0/1487487841 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa7cc05be80 con 0x7fa7dc0ffdf0 2024-05-15T16:22:02.191 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:22:02.191 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:22:02.191 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:22:02.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:22:02.191 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:22:02.191 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:22:02.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:22:02.192 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:22:02.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa7c406e210 msgr2=0x7fa7c40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa7c406e210 0x7fa7c40706d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc102fc0 tx=0x7fa7d000b3c0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 msgr2=0x7fa7dc101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 0x7fa7dc101890 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc00bd70 comp rx=0 tx=0).stop 2024-05-15T16:22:02.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 shutdown_connections 2024-05-15T16:22:02.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa7c406e210 0x7fa7c40706d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7dc0ffdf0 0x7fa7dc101890 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 --2- 172.21.15.49:0/1487487841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1007a0 0x7fa7dc101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 >> 172.21.15.49:0/1487487841 conn(0x7fa7dc0fb6a0 msgr2=0x7fa7dc109f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:02.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 shutdown_connections 2024-05-15T16:22:02.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.193+0000 7fa7e20b6700 1 -- 172.21.15.49:0/1487487841 wait complete. 2024-05-15T16:22:02.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.302+0000 7f097ae96700 1 -- 172.21.15.49:0/1891055302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 msgr2=0x7f097410ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.302+0000 7f097ae96700 1 --2- 172.21.15.49:0/1891055302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 0x7f097410ad20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0968009910 tx=0x7f0968009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:02.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.302+0000 7f097ae96700 1 -- 172.21.15.49:0/1891055302 shutdown_connections 2024-05-15T16:22:02.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.302+0000 7f097ae96700 1 --2- 172.21.15.49:0/1891055302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 0x7f097410ad20 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.302+0000 7f097ae96700 1 --2- 172.21.15.49:0/1891055302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0974101f10 0x7f09741022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.302+0000 7f097ae96700 1 -- 172.21.15.49:0/1891055302 >> 172.21.15.49:0/1891055302 conn(0x7f09740fb700 msgr2=0x7f09740fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:02.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.302+0000 7f097ae96700 1 -- 172.21.15.49:0/1891055302 shutdown_connections 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.302+0000 7f097ae96700 1 -- 172.21.15.49:0/1891055302 wait complete. 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f097ae96700 1 Processor -- start 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f097ae96700 1 -- start start 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f097ae96700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0974101f10 0x7f09740ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f097ae96700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 0x7f09741000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f097ae96700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741005f0 con 0x7f0974102830 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f097ae96700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0974100730 con 0x7f0974101f10 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f0973fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 0x7f09741000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f0973fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 0x7f09741000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38158/0 (socket says 172.21.15.49:38158) 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f0973fff700 1 -- 172.21.15.49:0/2251994821 learned_addr learned my addr 172.21.15.49:0/2251994821 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f0973fff700 1 -- 172.21.15.49:0/2251994821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0974101f10 msgr2=0x7f09740ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f0978c32700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0974101f10 0x7f09740ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f0973fff700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0974101f10 0x7f09740ffb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f0973fff700 1 -- 172.21.15.49:0/2251994821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09680095f0 con 0x7f0974102830 2024-05-15T16:22:02.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.303+0000 7f0978c32700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0974101f10 0x7f09740ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:22:02.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.304+0000 7f0973fff700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 0x7f09741000b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0968009fd0 tx=0x7f0968005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:02.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.304+0000 7f0971ffb700 1 -- 172.21.15.49:0/2251994821 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f096801d070 con 0x7f0974102830 2024-05-15T16:22:02.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.304+0000 7f0971ffb700 1 -- 172.21.15.49:0/2251994821 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f096800fe30 con 0x7f0974102830 2024-05-15T16:22:02.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.304+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09741009b0 con 0x7f0974102830 2024-05-15T16:22:02.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.304+0000 7f0971ffb700 1 -- 172.21.15.49:0/2251994821 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0968017630 con 0x7f0974102830 2024-05-15T16:22:02.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.304+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09741a3010 con 0x7f0974102830 2024-05-15T16:22:02.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.305+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0974069a10 con 0x7f0974102830 2024-05-15T16:22:02.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.306+0000 7f0971ffb700 1 -- 172.21.15.49:0/2251994821 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0968005080 con 0x7f0974102830 2024-05-15T16:22:02.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.306+0000 7f0971ffb700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f095c06e210 0x7f095c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:02.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.306+0000 7f0971ffb700 1 -- 172.21.15.49:0/2251994821 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f096808f5c0 con 0x7f0974102830 2024-05-15T16:22:02.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.306+0000 7f0978c32700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f095c06e210 0x7f095c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:02.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.307+0000 7f0978c32700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f095c06e210 0x7f095c0706d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0964009be0 tx=0x7f0964009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:02.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.311+0000 7f0971ffb700 1 -- 172.21.15.49:0/2251994821 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0968058920 con 0x7f0974102830 2024-05-15T16:22:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:01 smithi144 ceph-mon[86635]: pgmap v751: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:01 smithi144 ceph-mon[86635]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:02.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.565+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f097410ded0 con 0x7f0974102830 2024-05-15T16:22:02.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.566+0000 7f0971ffb700 1 -- 172.21.15.49:0/2251994821 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f096805bfe0 con 0x7f0974102830 2024-05-15T16:22:02.566 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:22:02.566 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:22:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:22:02.568 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:22:02.569 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:22:02.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.568+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f095c06e210 msgr2=0x7f095c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.568+0000 7f097ae96700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f095c06e210 0x7f095c0706d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0964009be0 tx=0x7f0964009260 comp rx=0 tx=0).stop 2024-05-15T16:22:02.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.568+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 msgr2=0x7f09741000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.568+0000 7f097ae96700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 0x7f09741000b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0968009fd0 tx=0x7f0968005f00 comp rx=0 tx=0).stop 2024-05-15T16:22:02.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.568+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 shutdown_connections 2024-05-15T16:22:02.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.568+0000 7f097ae96700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f095c06e210 0x7f095c0706d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.568+0000 7f097ae96700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0974101f10 0x7f09740ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.568+0000 7f097ae96700 1 --2- 172.21.15.49:0/2251994821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0974102830 0x7f09741000b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.568+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 >> 172.21.15.49:0/2251994821 conn(0x7f09740fb700 msgr2=0x7f0974103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:02.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.569+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 shutdown_connections 2024-05-15T16:22:02.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.569+0000 7f097ae96700 1 -- 172.21.15.49:0/2251994821 wait complete. 2024-05-15T16:22:02.571 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:22:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.679+0000 7fb138895700 1 -- 172.21.15.49:0/2306702770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1341007a0 msgr2=0x7fb1341047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.679+0000 7fb138895700 1 --2- 172.21.15.49:0/2306702770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1341007a0 0x7fb1341047f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009910 tx=0x7fb11c009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.679+0000 7fb138895700 1 -- 172.21.15.49:0/2306702770 shutdown_connections 2024-05-15T16:22:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.679+0000 7fb138895700 1 --2- 172.21.15.49:0/2306702770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1341007a0 0x7fb1341047f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.679+0000 7fb138895700 1 --2- 172.21.15.49:0/2306702770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1340ffdf0 0x7fb1341001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.679+0000 7fb138895700 1 -- 172.21.15.49:0/2306702770 >> 172.21.15.49:0/2306702770 conn(0x7fb1340fb6a0 msgr2=0x7fb1340fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.679+0000 7fb138895700 1 -- 172.21.15.49:0/2306702770 shutdown_connections 2024-05-15T16:22:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.679+0000 7fb138895700 1 -- 172.21.15.49:0/2306702770 wait complete. 2024-05-15T16:22:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.680+0000 7fb138895700 1 Processor -- start 2024-05-15T16:22:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.680+0000 7fb138895700 1 -- start start 2024-05-15T16:22:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.680+0000 7fb138895700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1340ffdf0 0x7fb13419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.680+0000 7fb138895700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1341007a0 0x7fb13419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.680+0000 7fb138895700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1341969f0 con 0x7fb1340ffdf0 2024-05-15T16:22:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.680+0000 7fb138895700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134196b60 con 0x7fb1341007a0 2024-05-15T16:22:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.680+0000 7fb13259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1340ffdf0 0x7fb13419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.680+0000 7fb13259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1340ffdf0 0x7fb13419c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38172/0 (socket says 172.21.15.49:38172) 2024-05-15T16:22:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb13259c700 1 -- 172.21.15.49:0/382192837 learned_addr learned my addr 172.21.15.49:0/382192837 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb12bfff700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1341007a0 0x7fb13419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb13259c700 1 -- 172.21.15.49:0/382192837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1341007a0 msgr2=0x7fb13419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb13259c700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1341007a0 0x7fb13419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb13259c700 1 -- 172.21.15.49:0/382192837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0095f0 con 0x7fb1340ffdf0 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb13259c700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1340ffdf0 0x7fb13419c940 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb12400d8d0 tx=0x7fb12400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb12b7fe700 1 -- 172.21.15.49:0/382192837 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1240098a0 con 0x7fb1340ffdf0 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb12b7fe700 1 -- 172.21.15.49:0/382192837 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb124010460 con 0x7fb1340ffdf0 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb12b7fe700 1 -- 172.21.15.49:0/382192837 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12400f5f0 con 0x7fb1340ffdf0 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb134196e40 con 0x7fb1340ffdf0 2024-05-15T16:22:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.681+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb134197360 con 0x7fb1340ffdf0 2024-05-15T16:22:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.682+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1341012d0 con 0x7fb1340ffdf0 2024-05-15T16:22:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.683+0000 7fb12b7fe700 1 -- 172.21.15.49:0/382192837 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb124009a00 con 0x7fb1340ffdf0 2024-05-15T16:22:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.683+0000 7fb12b7fe700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb12006e200 0x7fb1200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.683+0000 7fb12b7fe700 1 -- 172.21.15.49:0/382192837 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb12408df70 con 0x7fb1340ffdf0 2024-05-15T16:22:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.683+0000 7fb12bfff700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb12006e200 0x7fb1200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.684+0000 7fb12bfff700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb12006e200 0x7fb1200706c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb1341981f0 tx=0x7fb11c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:02.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.688+0000 7fb12b7fe700 1 -- 172.21.15.49:0/382192837 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb1240c2850 con 0x7fb1340ffdf0 2024-05-15T16:22:02.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.914+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb13410cd90 con 0x7fb12006e200 2024-05-15T16:22:02.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.916+0000 7fb12b7fe700 1 -- 172.21.15.49:0/382192837 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb13410cd90 con 0x7fb12006e200 2024-05-15T16:22:02.916 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:22:02.916 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:22:02.916 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:22:02.916 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:22:02.917 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:22:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.918+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb12006e200 msgr2=0x7fb1200706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.918+0000 7fb138895700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb12006e200 0x7fb1200706c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb1341981f0 tx=0x7fb11c01a040 comp rx=0 tx=0).stop 2024-05-15T16:22:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.918+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1340ffdf0 msgr2=0x7fb13419c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.918+0000 7fb138895700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1340ffdf0 0x7fb13419c940 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb12400d8d0 tx=0x7fb12400dbe0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.918+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 shutdown_connections 2024-05-15T16:22:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.918+0000 7fb138895700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb12006e200 0x7fb1200706c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.918+0000 7fb138895700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1340ffdf0 0x7fb13419c940 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.918+0000 7fb138895700 1 --2- 172.21.15.49:0/382192837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1341007a0 0x7fb13419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.918+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 >> 172.21.15.49:0/382192837 conn(0x7fb1340fb6a0 msgr2=0x7fb134069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.919+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 shutdown_connections 2024-05-15T16:22:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:02.919+0000 7fb138895700 1 -- 172.21.15.49:0/382192837 wait complete. 2024-05-15T16:22:03.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.028+0000 7f1dace31700 1 -- 172.21.15.49:0/3261887179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 msgr2=0x7f1da81058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.028+0000 7f1dace31700 1 --2- 172.21.15.49:0/3261887179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 0x7f1da81058e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009910 tx=0x7f1d9c009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.028+0000 7f1dace31700 1 -- 172.21.15.49:0/3261887179 shutdown_connections 2024-05-15T16:22:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.028+0000 7f1dace31700 1 --2- 172.21.15.49:0/3261887179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 0x7f1da81058e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.028+0000 7f1dace31700 1 --2- 172.21.15.49:0/3261887179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da8100ee0 0x7f1da81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.028+0000 7f1dace31700 1 -- 172.21.15.49:0/3261887179 >> 172.21.15.49:0/3261887179 conn(0x7f1da80fc790 msgr2=0x7f1da80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.028+0000 7f1dace31700 1 -- 172.21.15.49:0/3261887179 shutdown_connections 2024-05-15T16:22:03.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.028+0000 7f1dace31700 1 -- 172.21.15.49:0/3261887179 wait complete. 2024-05-15T16:22:03.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1dace31700 1 Processor -- start 2024-05-15T16:22:03.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1dace31700 1 -- start start 2024-05-15T16:22:03.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1dace31700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da8100ee0 0x7f1da819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:03.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1dace31700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 0x7f1da819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:03.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1dace31700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da819d380 con 0x7f1da8101890 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1dace31700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da8196a00 con 0x7f1da8100ee0 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1da5d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 0x7f1da819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1da5d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 0x7f1da819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38196/0 (socket says 172.21.15.49:38196) 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1da5d9b700 1 -- 172.21.15.49:0/2831481014 learned_addr learned my addr 172.21.15.49:0/2831481014 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.029+0000 7f1da659c700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da8100ee0 0x7f1da819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.030+0000 7f1da5d9b700 1 -- 172.21.15.49:0/2831481014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da8100ee0 msgr2=0x7f1da819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.030+0000 7f1da5d9b700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da8100ee0 0x7f1da819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.030+0000 7f1da5d9b700 1 -- 172.21.15.49:0/2831481014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c0095f0 con 0x7f1da8101890 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.030+0000 7f1da5d9b700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 0x7f1da819cd30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0092e0 tx=0x7f1d9c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.030+0000 7f1d977fe700 1 -- 172.21.15.49:0/2831481014 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9c01d070 con 0x7f1da8101890 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.030+0000 7f1d977fe700 1 -- 172.21.15.49:0/2831481014 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d9c00f460 con 0x7f1da8101890 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.030+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da8196c80 con 0x7f1da8101890 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.030+0000 7f1d977fe700 1 -- 172.21.15.49:0/2831481014 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9c017630 con 0x7f1da8101890 2024-05-15T16:22:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.030+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da8197170 con 0x7f1da8101890 2024-05-15T16:22:03.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.031+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da81092e0 con 0x7f1da8101890 2024-05-15T16:22:03.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.032+0000 7f1d977fe700 1 -- 172.21.15.49:0/2831481014 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1d9c00f5d0 con 0x7f1da8101890 2024-05-15T16:22:03.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.032+0000 7f1d977fe700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d9006e200 0x7f1d900706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.032+0000 7f1d977fe700 1 -- 172.21.15.49:0/2831481014 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1d9c08f480 con 0x7f1da8101890 2024-05-15T16:22:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.032+0000 7f1da659c700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d9006e200 0x7f1d900706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.033+0000 7f1da659c700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d9006e200 0x7f1d900706c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1da80ff040 tx=0x7f1d9800a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.037+0000 7f1d977fe700 1 -- 172.21.15.49:0/2831481014 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1d9c05c310 con 0x7f1da8101890 2024-05-15T16:22:03.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:02 smithi049 ceph-mon[112142]: from='client.35386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:03.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:02 smithi049 ceph-mon[112142]: from='client.35390 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:03.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:02 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1487487841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:22:03.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:02 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2251994821' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:22:03.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.332+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1da8197ee0 con 0x7f1da8101890 2024-05-15T16:22:03.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.332+0000 7f1d977fe700 1 -- 172.21.15.49:0/2831481014 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f1d9c005740 con 0x7f1da8101890 2024-05-15T16:22:03.333 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:22:03.333 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:22:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:22:03.333 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:22:03.333 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:22:03.333 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:22:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:22:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:22:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:22:03.335 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:22:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:22:03.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d9006e200 msgr2=0x7f1d900706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:03.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d9006e200 0x7f1d900706c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1da80ff040 tx=0x7f1d9800a380 comp rx=0 tx=0).stop 2024-05-15T16:22:03.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 msgr2=0x7f1da819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:03.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 0x7f1da819cd30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0092e0 tx=0x7f1d9c00bd70 comp rx=0 tx=0).stop 2024-05-15T16:22:03.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 shutdown_connections 2024-05-15T16:22:03.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d9006e200 0x7f1d900706c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:03.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da8100ee0 0x7f1da819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:03.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 --2- 172.21.15.49:0/2831481014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da8101890 0x7f1da819cd30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:03.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 >> 172.21.15.49:0/2831481014 conn(0x7f1da80fc790 msgr2=0x7f1da80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:03.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 shutdown_connections 2024-05-15T16:22:03.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:03.335+0000 7f1dace31700 1 -- 172.21.15.49:0/2831481014 wait complete. 2024-05-15T16:22:03.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:02 smithi144 ceph-mon[86635]: from='client.35386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:02 smithi144 ceph-mon[86635]: from='client.35390 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:02 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1487487841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:22:03.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:02 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2251994821' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:22:04.349 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:03 smithi144 ceph-mon[86635]: pgmap v752: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:04.349 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:03 smithi144 ceph-mon[86635]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:04.349 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:03 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2831481014' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:22:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:03 smithi049 ceph-mon[112142]: pgmap v752: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:03 smithi049 ceph-mon[112142]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:03 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2831481014' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:22:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:05 smithi144 ceph-mon[86635]: pgmap v753: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:22:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:05 smithi049 ceph-mon[112142]: pgmap v753: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:22:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:07 smithi144 ceph-mon[86635]: pgmap v754: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:07 smithi049 ceph-mon[112142]: pgmap v754: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:10 smithi049 ceph-mon[112142]: pgmap v755: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:10 smithi144 ceph-mon[86635]: pgmap v755: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:12 smithi144 ceph-mon[86635]: pgmap v756: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:12.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:12 smithi049 ceph-mon[112142]: pgmap v756: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:14 smithi144 ceph-mon[86635]: pgmap v757: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:14.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:14 smithi049 ceph-mon[112142]: pgmap v757: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:16 smithi144 ceph-mon[86635]: pgmap v758: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:16 smithi049 ceph-mon[112142]: pgmap v758: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:18 smithi144 ceph-mon[86635]: pgmap v759: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:18 smithi049 ceph-mon[112142]: pgmap v759: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:20 smithi144 ceph-mon[86635]: pgmap v760: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:20 smithi049 ceph-mon[112142]: pgmap v760: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:22:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:22:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:22 smithi144 ceph-mon[86635]: pgmap v761: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:22 smithi049 ceph-mon[112142]: pgmap v761: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:24 smithi144 ceph-mon[86635]: pgmap v762: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:24 smithi049 ceph-mon[112142]: pgmap v762: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:26 smithi144 ceph-mon[86635]: pgmap v763: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:26 smithi049 ceph-mon[112142]: pgmap v763: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:28 smithi144 ceph-mon[86635]: pgmap v764: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:28 smithi049 ceph-mon[112142]: pgmap v764: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:30 smithi144 ceph-mon[86635]: pgmap v765: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:30 smithi049 ceph-mon[112142]: pgmap v765: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:32 smithi144 ceph-mon[86635]: pgmap v766: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:32 smithi049 ceph-mon[112142]: pgmap v766: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:33.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.458+0000 7f5c53221700 1 -- 172.21.15.49:0/1315728738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c1027f0 msgr2=0x7f5c4c10ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.458+0000 7f5c53221700 1 --2- 172.21.15.49:0/1315728738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c1027f0 0x7f5c4c10ace0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009910 tx=0x7f5c38009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.458+0000 7f5c53221700 1 -- 172.21.15.49:0/1315728738 shutdown_connections 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.458+0000 7f5c53221700 1 --2- 172.21.15.49:0/1315728738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c1027f0 0x7f5c4c10ace0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.458+0000 7f5c53221700 1 --2- 172.21.15.49:0/1315728738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c4c101ed0 0x7f5c4c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.458+0000 7f5c53221700 1 -- 172.21.15.49:0/1315728738 >> 172.21.15.49:0/1315728738 conn(0x7f5c4c0fb6a0 msgr2=0x7f5c4c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.458+0000 7f5c53221700 1 -- 172.21.15.49:0/1315728738 shutdown_connections 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.458+0000 7f5c53221700 1 -- 172.21.15.49:0/1315728738 wait complete. 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c53221700 1 Processor -- start 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c53221700 1 -- start start 2024-05-15T16:22:33.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c53221700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c101ed0 0x7f5c4c0ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c53221700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c4c1027f0 0x7f5c4c100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c53221700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c1005b0 con 0x7f5c4c101ed0 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c53221700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c1006f0 con 0x7f5c4c1027f0 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c50fbd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c101ed0 0x7f5c4c0ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c50fbd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c101ed0 0x7f5c4c0ffb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59834/0 (socket says 172.21.15.49:59834) 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c50fbd700 1 -- 172.21.15.49:0/3047880479 learned_addr learned my addr 172.21.15.49:0/3047880479 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c4bfff700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c4c1027f0 0x7f5c4c100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c50fbd700 1 -- 172.21.15.49:0/3047880479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c4c1027f0 msgr2=0x7f5c4c100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c50fbd700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c4c1027f0 0x7f5c4c100070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.459+0000 7f5c50fbd700 1 -- 172.21.15.49:0/3047880479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380095f0 con 0x7f5c4c101ed0 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.460+0000 7f5c50fbd700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c101ed0 0x7f5c4c0ffb30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c8a0 tx=0x7f5c4000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.460+0000 7f5c49ffb700 1 -- 172.21.15.49:0/3047880479 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c4000cea0 con 0x7f5c4c101ed0 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.460+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c1a2e00 con 0x7f5c4c101ed0 2024-05-15T16:22:33.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.460+0000 7f5c49ffb700 1 -- 172.21.15.49:0/3047880479 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c40004830 con 0x7f5c4c101ed0 2024-05-15T16:22:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.460+0000 7f5c49ffb700 1 -- 172.21.15.49:0/3047880479 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c400055b0 con 0x7f5c4c101ed0 2024-05-15T16:22:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.460+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c1a3160 con 0x7f5c4c101ed0 2024-05-15T16:22:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.461+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c069a30 con 0x7f5c4c101ed0 2024-05-15T16:22:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.461+0000 7f5c49ffb700 1 -- 172.21.15.49:0/3047880479 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5c400049a0 con 0x7f5c4c101ed0 2024-05-15T16:22:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.462+0000 7f5c49ffb700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c3c06e200 0x7f5c3c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.462+0000 7f5c49ffb700 1 -- 172.21.15.49:0/3047880479 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5c4008de40 con 0x7f5c4c101ed0 2024-05-15T16:22:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.462+0000 7f5c4bfff700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c3c06e200 0x7f5c3c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:33.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.462+0000 7f5c4bfff700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c3c06e200 0x7f5c3c0706c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5c380059d0 tx=0x7f5c38005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:33.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.467+0000 7f5c49ffb700 1 -- 172.21.15.49:0/3047880479 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5c40057120 con 0x7f5c4c101ed0 2024-05-15T16:22:33.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.695+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c4c1008b0 con 0x7f5c3c06e200 2024-05-15T16:22:33.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.696+0000 7f5c49ffb700 1 -- 172.21.15.49:0/3047880479 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5c4c1008b0 con 0x7f5c3c06e200 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.698+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c3c06e200 msgr2=0x7f5c3c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.698+0000 7f5c53221700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c3c06e200 0x7f5c3c0706c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5c380059d0 tx=0x7f5c38005960 comp rx=0 tx=0).stop 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.698+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c101ed0 msgr2=0x7f5c4c0ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.698+0000 7f5c53221700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c101ed0 0x7f5c4c0ffb30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c8a0 tx=0x7f5c4000cc60 comp rx=0 tx=0).stop 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.698+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 shutdown_connections 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.698+0000 7f5c53221700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c3c06e200 0x7f5c3c0706c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.698+0000 7f5c53221700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4c101ed0 0x7f5c4c0ffb30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.698+0000 7f5c53221700 1 --2- 172.21.15.49:0/3047880479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c4c1027f0 0x7f5c4c100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.698+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 >> 172.21.15.49:0/3047880479 conn(0x7f5c4c0fb6a0 msgr2=0x7f5c4c1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.699+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 shutdown_connections 2024-05-15T16:22:33.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.699+0000 7f5c53221700 1 -- 172.21.15.49:0/3047880479 wait complete. 2024-05-15T16:22:33.709 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:22:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.808+0000 7f9848b53700 1 -- 172.21.15.49:0/1902123067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98441027d0 msgr2=0x7f984410acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.808+0000 7f9848b53700 1 --2- 172.21.15.49:0/1902123067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98441027d0 0x7f984410acc0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9838009910 tx=0x7f9838009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.808+0000 7f9848b53700 1 -- 172.21.15.49:0/1902123067 shutdown_connections 2024-05-15T16:22:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.808+0000 7f9848b53700 1 --2- 172.21.15.49:0/1902123067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98441027d0 0x7f984410acc0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.808+0000 7f9848b53700 1 --2- 172.21.15.49:0/1902123067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9844101eb0 0x7f9844102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.808+0000 7f9848b53700 1 -- 172.21.15.49:0/1902123067 >> 172.21.15.49:0/1902123067 conn(0x7f98440fb740 msgr2=0x7f98440fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.808+0000 7f9848b53700 1 -- 172.21.15.49:0/1902123067 shutdown_connections 2024-05-15T16:22:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.808+0000 7f9848b53700 1 -- 172.21.15.49:0/1902123067 wait complete. 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f9848b53700 1 Processor -- start 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f9848b53700 1 -- start start 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f9848b53700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844101eb0 0x7f984419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f9848b53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98441027d0 0x7f984419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f9848b53700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984419d4f0 con 0x7f9844101eb0 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f9848b53700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9844196ad0 con 0x7f98441027d0 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f984259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844101eb0 0x7f984419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f984259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844101eb0 0x7f984419c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33976/0 (socket says 172.21.15.49:33976) 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f984259c700 1 -- 172.21.15.49:0/728182841 learned_addr learned my addr 172.21.15.49:0/728182841 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.809+0000 7f9841d9b700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98441027d0 0x7f984419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.810+0000 7f984259c700 1 -- 172.21.15.49:0/728182841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98441027d0 msgr2=0x7f984419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.810+0000 7f984259c700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98441027d0 0x7f984419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.810+0000 7f984259c700 1 -- 172.21.15.49:0/728182841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380095f0 con 0x7f9844101eb0 2024-05-15T16:22:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.810+0000 7f984259c700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844101eb0 0x7f984419c990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f983400d8d0 tx=0x7f983400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.810+0000 7f98337fe700 1 -- 172.21.15.49:0/728182841 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98340098a0 con 0x7f9844101eb0 2024-05-15T16:22:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.810+0000 7f98337fe700 1 -- 172.21.15.49:0/728182841 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9834010460 con 0x7f9844101eb0 2024-05-15T16:22:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.810+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9844196db0 con 0x7f9844101eb0 2024-05-15T16:22:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.810+0000 7f98337fe700 1 -- 172.21.15.49:0/728182841 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983400f5f0 con 0x7f9844101eb0 2024-05-15T16:22:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.810+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9844197300 con 0x7f9844101eb0 2024-05-15T16:22:33.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.811+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98441055d0 con 0x7f9844101eb0 2024-05-15T16:22:33.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.812+0000 7f98337fe700 1 -- 172.21.15.49:0/728182841 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9834009a00 con 0x7f9844101eb0 2024-05-15T16:22:33.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.812+0000 7f98337fe700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f982c06e200 0x7f982c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:33.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.812+0000 7f98337fe700 1 -- 172.21.15.49:0/728182841 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f983408e070 con 0x7f9844101eb0 2024-05-15T16:22:33.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.812+0000 7f9841d9b700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f982c06e200 0x7f982c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.813+0000 7f9841d9b700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f982c06e200 0x7f982c0706c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9844197c90 tx=0x7f983801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:33.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:33.817+0000 7f98337fe700 1 -- 172.21.15.49:0/728182841 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f983405ae80 con 0x7f9844101eb0 2024-05-15T16:22:34.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.044+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9844197fe0 con 0x7f982c06e200 2024-05-15T16:22:34.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.046+0000 7f98337fe700 1 -- 172.21.15.49:0/728182841 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9844197fe0 con 0x7f982c06e200 2024-05-15T16:22:34.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f982c06e200 msgr2=0x7f982c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f982c06e200 0x7f982c0706c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9844197c90 tx=0x7f983801a040 comp rx=0 tx=0).stop 2024-05-15T16:22:34.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844101eb0 msgr2=0x7f984419c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844101eb0 0x7f984419c990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f983400d8d0 tx=0x7f983400dbe0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 shutdown_connections 2024-05-15T16:22:34.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f982c06e200 0x7f982c0706c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9844101eb0 0x7f984419c990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 --2- 172.21.15.49:0/728182841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98441027d0 0x7f984419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 >> 172.21.15.49:0/728182841 conn(0x7f98440fb740 msgr2=0x7f98441031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:34.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 shutdown_connections 2024-05-15T16:22:34.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.048+0000 7f9848b53700 1 -- 172.21.15.49:0/728182841 wait complete. 2024-05-15T16:22:34.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.156+0000 7f33a934c700 1 -- 172.21.15.49:0/4223291755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 msgr2=0x7f33a4106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.156+0000 7f33a934c700 1 --2- 172.21.15.49:0/4223291755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 0x7f33a4106110 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f339400b0a0 tx=0x7f339400b3b0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.156+0000 7f33a934c700 1 -- 172.21.15.49:0/4223291755 shutdown_connections 2024-05-15T16:22:34.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.156+0000 7f33a934c700 1 --2- 172.21.15.49:0/4223291755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33a40ff150 0x7f33a40ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.156+0000 7f33a934c700 1 --2- 172.21.15.49:0/4223291755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 0x7f33a4106110 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.156+0000 7f33a934c700 1 -- 172.21.15.49:0/4223291755 >> 172.21.15.49:0/4223291755 conn(0x7f33a4075fc0 msgr2=0x7f33a40fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:34.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.156+0000 7f33a934c700 1 -- 172.21.15.49:0/4223291755 shutdown_connections 2024-05-15T16:22:34.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.156+0000 7f33a934c700 1 -- 172.21.15.49:0/4223291755 wait complete. 2024-05-15T16:22:34.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.157+0000 7f33a934c700 1 Processor -- start 2024-05-15T16:22:34.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.157+0000 7f33a934c700 1 -- start start 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.157+0000 7f33a934c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33a40ff150 0x7f33a4100330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.157+0000 7f33a934c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 0x7f33a4100870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.157+0000 7f33a934c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4100db0 con 0x7f33a4105d30 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.157+0000 7f33a934c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4100ef0 con 0x7f33a40ff150 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.157+0000 7f33a27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 0x7f33a4100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.157+0000 7f33a27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 0x7f33a4100870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33990/0 (socket says 172.21.15.49:33990) 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.157+0000 7f33a27fc700 1 -- 172.21.15.49:0/2085561124 learned_addr learned my addr 172.21.15.49:0/2085561124 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f33a2ffd700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33a40ff150 0x7f33a4100330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f33a27fc700 1 -- 172.21.15.49:0/2085561124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33a40ff150 msgr2=0x7f33a4100330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f33a27fc700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33a40ff150 0x7f33a4100330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f33a27fc700 1 -- 172.21.15.49:0/2085561124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3394009d00 con 0x7f33a4105d30 2024-05-15T16:22:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f33a2ffd700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33a40ff150 0x7f33a4100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:22:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f33a27fc700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 0x7f33a4100870 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f338c00d8d0 tx=0x7f338c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f339bfff700 1 -- 172.21.15.49:0/2085561124 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338c009940 con 0x7f33a4105d30 2024-05-15T16:22:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f339bfff700 1 -- 172.21.15.49:0/2085561124 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f338c010460 con 0x7f33a4105d30 2024-05-15T16:22:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a41a2e60 con 0x7f33a4105d30 2024-05-15T16:22:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f339bfff700 1 -- 172.21.15.49:0/2085561124 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338c00f5f0 con 0x7f33a4105d30 2024-05-15T16:22:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.158+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a41a31c0 con 0x7f33a4105d30 2024-05-15T16:22:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.159+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a4109300 con 0x7f33a4105d30 2024-05-15T16:22:34.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.160+0000 7f339bfff700 1 -- 172.21.15.49:0/2085561124 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f338c0105d0 con 0x7f33a4105d30 2024-05-15T16:22:34.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.160+0000 7f339bfff700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f339006e200 0x7f33900706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:34.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.160+0000 7f339bfff700 1 -- 172.21.15.49:0/2085561124 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f338c08e170 con 0x7f33a4105d30 2024-05-15T16:22:34.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.160+0000 7f33a2ffd700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f339006e200 0x7f33900706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:34.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.161+0000 7f33a2ffd700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f339006e200 0x7f33900706c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f339400b830 tx=0x7f3394006210 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:34.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.165+0000 7f339bfff700 1 -- 172.21.15.49:0/2085561124 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f338c057450 con 0x7f33a4105d30 2024-05-15T16:22:34.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.380+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f33a4101810 con 0x7f339006e200 2024-05-15T16:22:34.381 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:34 smithi049 ceph-mon[112142]: pgmap v767: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.391+0000 7f339bfff700 1 -- 172.21.15.49:0/2085561124 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f33a4101810 con 0x7f339006e200 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (25m) 3m ago 37m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (37m) 3m ago 37m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (36m) 3m ago 36m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (23m) 3m ago 37m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (23m) 3m ago 36m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (25m) 3m ago 37m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (19m) 3m ago 31m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (19m) 3m ago 31m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (18m) 3m ago 31m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (3m) 3m ago 31m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (28m) 3m ago 38m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:22:34.392 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (26m) 3m ago 36m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (24m) 3m ago 38m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (23m) 3m ago 36m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (26m) 3m ago 37m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (26m) 3m ago 36m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (22m) 3m ago 35m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (22m) 3m ago 35m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (21m) 3m ago 34m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (20m) 3m ago 34m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (20m) 3m ago 33m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (19m) 3m ago 33m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:22:34.393 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (26m) 3m ago 36m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.393+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f339006e200 msgr2=0x7f33900706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.393+0000 7f33a934c700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f339006e200 0x7f33900706c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f339400b830 tx=0x7f3394006210 comp rx=0 tx=0).stop 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.393+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 msgr2=0x7f33a4100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.393+0000 7f33a934c700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 0x7f33a4100870 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f338c00d8d0 tx=0x7f338c00dc90 comp rx=0 tx=0).stop 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.393+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 shutdown_connections 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.393+0000 7f33a934c700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f339006e200 0x7f33900706c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.393+0000 7f33a934c700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33a40ff150 0x7f33a4100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.393+0000 7f33a934c700 1 --2- 172.21.15.49:0/2085561124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a4105d30 0x7f33a4100870 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.393+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 >> 172.21.15.49:0/2085561124 conn(0x7f33a4075fc0 msgr2=0x7f33a40fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.394+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 shutdown_connections 2024-05-15T16:22:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.394+0000 7f33a934c700 1 -- 172.21.15.49:0/2085561124 wait complete. 2024-05-15T16:22:34.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:34 smithi144 ceph-mon[86635]: pgmap v767: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.503+0000 7f556412b700 1 -- 172.21.15.49:0/1797763475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c0ffdd0 msgr2=0x7f555c1001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.503+0000 7f556412b700 1 --2- 172.21.15.49:0/1797763475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c0ffdd0 0x7f555c1001b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f554c00b0a0 tx=0x7f554c00b3b0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.503+0000 7f556412b700 1 -- 172.21.15.49:0/1797763475 shutdown_connections 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.503+0000 7f556412b700 1 --2- 172.21.15.49:0/1797763475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f555c100780 0x7f555c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.503+0000 7f556412b700 1 --2- 172.21.15.49:0/1797763475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c0ffdd0 0x7f555c1001b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.503+0000 7f556412b700 1 -- 172.21.15.49:0/1797763475 >> 172.21.15.49:0/1797763475 conn(0x7f555c0fb6a0 msgr2=0x7f555c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.503+0000 7f556412b700 1 -- 172.21.15.49:0/1797763475 shutdown_connections 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.503+0000 7f556412b700 1 -- 172.21.15.49:0/1797763475 wait complete. 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f556412b700 1 Processor -- start 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f556412b700 1 -- start start 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f556412b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f555c0ffdd0 0x7f555c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f556412b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c100780 0x7f555c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f556412b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c198640 con 0x7f555c100780 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f556412b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c198780 con 0x7f555c0ffdd0 2024-05-15T16:22:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f55616c6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c100780 0x7f555c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f55616c6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c100780 0x7f555c198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34002/0 (socket says 172.21.15.49:34002) 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f55616c6700 1 -- 172.21.15.49:0/481477932 learned_addr learned my addr 172.21.15.49:0/481477932 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.504+0000 7f5561ec7700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f555c0ffdd0 0x7f555c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.505+0000 7f55616c6700 1 -- 172.21.15.49:0/481477932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f555c0ffdd0 msgr2=0x7f555c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.505+0000 7f55616c6700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f555c0ffdd0 0x7f555c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.505+0000 7f55616c6700 1 -- 172.21.15.49:0/481477932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c009d00 con 0x7f555c100780 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.505+0000 7f55616c6700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c100780 0x7f555c198070 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f555000b480 tx=0x7f555000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.505+0000 7f555affd700 1 -- 172.21.15.49:0/481477932 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5550011800 con 0x7f555c100780 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.505+0000 7f555affd700 1 -- 172.21.15.49:0/481477932 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5550011e20 con 0x7f555c100780 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.505+0000 7f555affd700 1 -- 172.21.15.49:0/481477932 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5550010ea0 con 0x7f555c100780 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.505+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c1a0ba0 con 0x7f555c100780 2024-05-15T16:22:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.505+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c1a1010 con 0x7f555c100780 2024-05-15T16:22:34.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.506+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c1012b0 con 0x7f555c100780 2024-05-15T16:22:34.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.506+0000 7f555affd700 1 -- 172.21.15.49:0/481477932 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5550010380 con 0x7f555c100780 2024-05-15T16:22:34.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.507+0000 7f555affd700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f554406e210 0x7f55440706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:34.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.507+0000 7f555affd700 1 -- 172.21.15.49:0/481477932 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f555008dbe0 con 0x7f555c100780 2024-05-15T16:22:34.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.507+0000 7f5561ec7700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f554406e210 0x7f55440706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:34.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.508+0000 7f5561ec7700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f554406e210 0x7f55440706d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f554c000f80 tx=0x7f554c0097e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:34.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.512+0000 7f555affd700 1 -- 172.21.15.49:0/481477932 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5550056f40 con 0x7f555c100780 2024-05-15T16:22:34.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.805+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f555c04ec60 con 0x7f555c100780 2024-05-15T16:22:34.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.805+0000 7f555affd700 1 -- 172.21.15.49:0/481477932 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f555005a600 con 0x7f555c100780 2024-05-15T16:22:34.806 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:22:34.806 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:22:34.807 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:22:34.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:22:34.807 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:22:34.807 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:22:34.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:22:34.807 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:22:34.807 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:22:34.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:22:34.808 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:22:34.808 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:22:34.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:22:34.808 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:22:34.808 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:22:34.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:22:34.808 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:22:34.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f554406e210 msgr2=0x7f55440706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f554406e210 0x7f55440706d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f554c000f80 tx=0x7f554c0097e0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c100780 msgr2=0x7f555c198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c100780 0x7f555c198070 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f555000b480 tx=0x7f555000b790 comp rx=0 tx=0).stop 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 shutdown_connections 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f554406e210 0x7f55440706d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f555c0ffdd0 0x7f555c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 --2- 172.21.15.49:0/481477932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f555c100780 0x7f555c198070 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 >> 172.21.15.49:0/481477932 conn(0x7f555c0fb6a0 msgr2=0x7f555c0fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 shutdown_connections 2024-05-15T16:22:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.808+0000 7f556412b700 1 -- 172.21.15.49:0/481477932 wait complete. 2024-05-15T16:22:34.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.916+0000 7f325f32d700 1 -- 172.21.15.49:0/2367648186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 msgr2=0x7f32580ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.916+0000 7f325f32d700 1 --2- 172.21.15.49:0/2367648186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 0x7f32580ffad0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3254009910 tx=0x7f3254009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:34.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.916+0000 7f325f32d700 1 -- 172.21.15.49:0/2367648186 shutdown_connections 2024-05-15T16:22:34.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.916+0000 7f325f32d700 1 --2- 172.21.15.49:0/2367648186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32581000a0 0x7f3258103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.916+0000 7f325f32d700 1 --2- 172.21.15.49:0/2367648186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 0x7f32580ffad0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.916+0000 7f325f32d700 1 -- 172.21.15.49:0/2367648186 >> 172.21.15.49:0/2367648186 conn(0x7f3258074e50 msgr2=0x7f3258075260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:34.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.917+0000 7f325f32d700 1 -- 172.21.15.49:0/2367648186 shutdown_connections 2024-05-15T16:22:34.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.917+0000 7f325f32d700 1 -- 172.21.15.49:0/2367648186 wait complete. 2024-05-15T16:22:34.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.917+0000 7f325f32d700 1 Processor -- start 2024-05-15T16:22:34.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.917+0000 7f325f32d700 1 -- start start 2024-05-15T16:22:34.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325f32d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 0x7f325819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:34.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325f32d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32581000a0 0x7f325819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:34.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325f32d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3258196a50 con 0x7f32580ff6f0 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325f32d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3258196bc0 con 0x7f32581000a0 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325d0c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 0x7f325819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325d0c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 0x7f325819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34012/0 (socket says 172.21.15.49:34012) 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325d0c9700 1 -- 172.21.15.49:0/3969712546 learned_addr learned my addr 172.21.15.49:0/3969712546 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325c8c8700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32581000a0 0x7f325819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325d0c9700 1 -- 172.21.15.49:0/3969712546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32581000a0 msgr2=0x7f325819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325d0c9700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32581000a0 0x7f325819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325d0c9700 1 -- 172.21.15.49:0/3969712546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540095f0 con 0x7f32580ff6f0 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325c8c8700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32581000a0 0x7f325819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325d0c9700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 0x7f325819c9a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3254005f50 tx=0x7f325400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f324a7fc700 1 -- 172.21.15.49:0/3969712546 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325401d070 con 0x7f32580ff6f0 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f324a7fc700 1 -- 172.21.15.49:0/3969712546 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32540052a0 con 0x7f32580ff6f0 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f324a7fc700 1 -- 172.21.15.49:0/3969712546 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3254017400 con 0x7f32580ff6f0 2024-05-15T16:22:34.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3258196e40 con 0x7f32580ff6f0 2024-05-15T16:22:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.918+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3258197330 con 0x7f32580ff6f0 2024-05-15T16:22:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.920+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3258109940 con 0x7f32580ff6f0 2024-05-15T16:22:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.920+0000 7f324a7fc700 1 -- 172.21.15.49:0/3969712546 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f325400f7a0 con 0x7f32580ff6f0 2024-05-15T16:22:34.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.921+0000 7f324a7fc700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f324406e210 0x7f32440706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.921+0000 7f324a7fc700 1 -- 172.21.15.49:0/3969712546 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f325408f540 con 0x7f32580ff6f0 2024-05-15T16:22:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.921+0000 7f325c8c8700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f324406e210 0x7f32440706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:34.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.921+0000 7f325c8c8700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f324406e210 0x7f32440706d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f32581981c0 tx=0x7f324c00a370 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:34.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:34.925+0000 7f324a7fc700 1 -- 172.21.15.49:0/3969712546 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f32540588a0 con 0x7f32580ff6f0 2024-05-15T16:22:35.149 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:35 smithi049 ceph-mon[112142]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:35.149 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:35 smithi049 ceph-mon[112142]: from='client.35414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:35.149 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:35 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/481477932' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:22:35.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.180+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3258197f30 con 0x7f32580ff6f0 2024-05-15T16:22:35.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.180+0000 7f324a7fc700 1 -- 172.21.15.49:0/3969712546 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f3254058070 con 0x7f32580ff6f0 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:22:35.181 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:22:35.182 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:22:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:22:35.184 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:22:35.184 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:22:35.184 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:22:35.184 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:22:35.184 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.183+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f324406e210 msgr2=0x7f32440706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.183+0000 7f325f32d700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f324406e210 0x7f32440706d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f32581981c0 tx=0x7f324c00a370 comp rx=0 tx=0).stop 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.183+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 msgr2=0x7f325819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.183+0000 7f325f32d700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 0x7f325819c9a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3254005f50 tx=0x7f325400b650 comp rx=0 tx=0).stop 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.183+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 shutdown_connections 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.183+0000 7f325f32d700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f324406e210 0x7f32440706d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.183+0000 7f325f32d700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32580ff6f0 0x7f325819c9a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.183+0000 7f325f32d700 1 --2- 172.21.15.49:0/3969712546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32581000a0 0x7f325819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.183+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 >> 172.21.15.49:0/3969712546 conn(0x7f3258074e50 msgr2=0x7f32580fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.184+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 shutdown_connections 2024-05-15T16:22:35.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.184+0000 7f325f32d700 1 -- 172.21.15.49:0/3969712546 wait complete. 2024-05-15T16:22:35.186 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:22:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.294+0000 7f57668ae700 1 -- 172.21.15.49:0/2454706035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760101890 msgr2=0x7f57601058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.294+0000 7f57668ae700 1 --2- 172.21.15.49:0/2454706035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760101890 0x7f57601058e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5754009910 tx=0x7f5754009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.294+0000 7f57668ae700 1 -- 172.21.15.49:0/2454706035 shutdown_connections 2024-05-15T16:22:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.294+0000 7f57668ae700 1 --2- 172.21.15.49:0/2454706035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760101890 0x7f57601058e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.294+0000 7f57668ae700 1 --2- 172.21.15.49:0/2454706035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760100ee0 0x7f57601012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.294+0000 7f57668ae700 1 -- 172.21.15.49:0/2454706035 >> 172.21.15.49:0/2454706035 conn(0x7f57600fc790 msgr2=0x7f57600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.294+0000 7f57668ae700 1 -- 172.21.15.49:0/2454706035 shutdown_connections 2024-05-15T16:22:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.294+0000 7f57668ae700 1 -- 172.21.15.49:0/2454706035 wait complete. 2024-05-15T16:22:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f57668ae700 1 Processor -- start 2024-05-15T16:22:35.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f57668ae700 1 -- start start 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f57668ae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760100ee0 0x7f57600732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f57668ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760101890 0x7f57600737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f57668ae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5760073d20 con 0x7f5760100ee0 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f57668ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5760073e60 con 0x7f5760101890 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f575ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760100ee0 0x7f57600732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f575ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760100ee0 0x7f57600732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34036/0 (socket says 172.21.15.49:34036) 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f575ffff700 1 -- 172.21.15.49:0/2584392511 learned_addr learned my addr 172.21.15.49:0/2584392511 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.295+0000 7f575f7fe700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760101890 0x7f57600737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.296+0000 7f575ffff700 1 -- 172.21.15.49:0/2584392511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760101890 msgr2=0x7f57600737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.296+0000 7f575ffff700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760101890 0x7f57600737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.296+0000 7f575ffff700 1 -- 172.21.15.49:0/2584392511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57540095f0 con 0x7f5760100ee0 2024-05-15T16:22:35.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.296+0000 7f575ffff700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760100ee0 0x7f57600732a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f575000c8a0 tx=0x7f575000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:35.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.296+0000 7f575d7fa700 1 -- 172.21.15.49:0/2584392511 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57500078e0 con 0x7f5760100ee0 2024-05-15T16:22:35.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.296+0000 7f575d7fa700 1 -- 172.21.15.49:0/2584392511 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f575000f450 con 0x7f5760100ee0 2024-05-15T16:22:35.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.296+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57601a2da0 con 0x7f5760100ee0 2024-05-15T16:22:35.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.296+0000 7f575d7fa700 1 -- 172.21.15.49:0/2584392511 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f575000e5e0 con 0x7f5760100ee0 2024-05-15T16:22:35.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.296+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57601a3100 con 0x7f5760100ee0 2024-05-15T16:22:35.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.297+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57601092e0 con 0x7f5760100ee0 2024-05-15T16:22:35.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.298+0000 7f575d7fa700 1 -- 172.21.15.49:0/2584392511 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f575000f5c0 con 0x7f5760100ee0 2024-05-15T16:22:35.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.298+0000 7f575d7fa700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f574806e200 0x7f57480706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:35.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.299+0000 7f575f7fe700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f574806e200 0x7f57480706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:35.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.299+0000 7f575d7fa700 1 -- 172.21.15.49:0/2584392511 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f575008e130 con 0x7f5760100ee0 2024-05-15T16:22:35.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.299+0000 7f575f7fe700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f574806e200 0x7f57480706c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57600749d0 tx=0x7f575401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:35.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.303+0000 7f575d7fa700 1 -- 172.21.15.49:0/2584392511 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5750057410 con 0x7f5760100ee0 2024-05-15T16:22:35.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:35 smithi144 ceph-mon[86635]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:35.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:35 smithi144 ceph-mon[86635]: from='client.35414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:35.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:35 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/481477932' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:22:35.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.529+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f576010de80 con 0x7f574806e200 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.531+0000 7f575d7fa700 1 -- 172.21.15.49:0/2584392511 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f576010de80 con 0x7f574806e200 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:22:35.531 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:22:35.532 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:22:35.532 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:22:35.532 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:22:35.532 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:22:35.532 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:22:35.532 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:22:35.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f574806e200 msgr2=0x7f57480706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f574806e200 0x7f57480706c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57600749d0 tx=0x7f575401a040 comp rx=0 tx=0).stop 2024-05-15T16:22:35.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760100ee0 msgr2=0x7f57600732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760100ee0 0x7f57600732a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f575000c8a0 tx=0x7f575000cbb0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 shutdown_connections 2024-05-15T16:22:35.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f574806e200 0x7f57480706c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5760100ee0 0x7f57600732a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 --2- 172.21.15.49:0/2584392511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760101890 0x7f57600737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 >> 172.21.15.49:0/2584392511 conn(0x7f57600fc790 msgr2=0x7f57600fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:35.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 shutdown_connections 2024-05-15T16:22:35.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.533+0000 7f57668ae700 1 -- 172.21.15.49:0/2584392511 wait complete. 2024-05-15T16:22:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.641+0000 7ff6364df700 1 -- 172.21.15.49:0/2732339375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073c60 msgr2=0x7ff630111770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.641+0000 7ff6364df700 1 --2- 172.21.15.49:0/2732339375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073c60 0x7ff630111770 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff618009910 tx=0x7ff618009c20 comp rx=0 tx=0).stop 2024-05-15T16:22:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.641+0000 7ff6364df700 1 -- 172.21.15.49:0/2732339375 shutdown_connections 2024-05-15T16:22:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.641+0000 7ff6364df700 1 --2- 172.21.15.49:0/2732339375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073c60 0x7ff630111770 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.641+0000 7ff6364df700 1 --2- 172.21.15.49:0/2732339375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff630073340 0x7ff630073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.641+0000 7ff6364df700 1 -- 172.21.15.49:0/2732339375 >> 172.21.15.49:0/2732339375 conn(0x7ff6300fc7d0 msgr2=0x7ff6300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.641+0000 7ff6364df700 1 -- 172.21.15.49:0/2732339375 shutdown_connections 2024-05-15T16:22:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.641+0000 7ff6364df700 1 -- 172.21.15.49:0/2732339375 wait complete. 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff6364df700 1 Processor -- start 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff6364df700 1 -- start start 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff6364df700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073340 0x7ff63019d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff6364df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff630073c60 0x7ff63019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff6364df700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63019dd70 con 0x7ff630073340 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff6364df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6301a1a40 con 0x7ff630073c60 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff62ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073340 0x7ff63019d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff62ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073340 0x7ff63019d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34054/0 (socket says 172.21.15.49:34054) 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff62ffff700 1 -- 172.21.15.49:0/2656431893 learned_addr learned my addr 172.21.15.49:0/2656431893 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff62ffff700 1 -- 172.21.15.49:0/2656431893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff630073c60 msgr2=0x7ff63019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff627fff700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff630073c60 0x7ff63019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff62ffff700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff630073c60 0x7ff63019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.642+0000 7ff62ffff700 1 -- 172.21.15.49:0/2656431893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180095f0 con 0x7ff630073340 2024-05-15T16:22:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.643+0000 7ff62ffff700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073340 0x7ff63019d150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff62000d8d0 tx=0x7ff62000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.643+0000 7ff62dffb700 1 -- 172.21.15.49:0/2656431893 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff620009940 con 0x7ff630073340 2024-05-15T16:22:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.643+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6301a1d20 con 0x7ff630073340 2024-05-15T16:22:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.643+0000 7ff62dffb700 1 -- 172.21.15.49:0/2656431893 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff620010460 con 0x7ff630073340 2024-05-15T16:22:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.643+0000 7ff62dffb700 1 -- 172.21.15.49:0/2656431893 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff62000f5f0 con 0x7ff630073340 2024-05-15T16:22:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.643+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6301a2270 con 0x7ff630073340 2024-05-15T16:22:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.644+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63010eef0 con 0x7ff630073340 2024-05-15T16:22:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.645+0000 7ff62dffb700 1 -- 172.21.15.49:0/2656431893 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff6200105d0 con 0x7ff630073340 2024-05-15T16:22:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.645+0000 7ff62dffb700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61c06e200 0x7ff61c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:22:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.645+0000 7ff62dffb700 1 -- 172.21.15.49:0/2656431893 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff62008e090 con 0x7ff630073340 2024-05-15T16:22:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.645+0000 7ff627fff700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61c06e200 0x7ff61c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:22:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.646+0000 7ff627fff700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61c06e200 0x7ff61c0706c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff61800b5c0 tx=0x7ff6180052a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:22:35.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.650+0000 7ff62dffb700 1 -- 172.21.15.49:0/2656431893 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff6200573f0 con 0x7ff630073340 2024-05-15T16:22:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.942+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff630074a80 con 0x7ff630073340 2024-05-15T16:22:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.942+0000 7ff62dffb700 1 -- 172.21.15.49:0/2656431893 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7ff62005aab0 con 0x7ff630073340 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:22:35.943 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:22:35.944 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:22:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61c06e200 msgr2=0x7ff61c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61c06e200 0x7ff61c0706c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff61800b5c0 tx=0x7ff6180052a0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073340 msgr2=0x7ff63019d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:22:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073340 0x7ff63019d150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff62000d8d0 tx=0x7ff62000dc90 comp rx=0 tx=0).stop 2024-05-15T16:22:35.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 shutdown_connections 2024-05-15T16:22:35.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff61c06e200 0x7ff61c0706c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff630073340 0x7ff63019d150 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 --2- 172.21.15.49:0/2656431893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff630073c60 0x7ff63019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:22:35.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 >> 172.21.15.49:0/2656431893 conn(0x7ff6300fc7d0 msgr2=0x7ff630104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:22:35.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 shutdown_connections 2024-05-15T16:22:35.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:22:35.945+0000 7ff6364df700 1 -- 172.21.15.49:0/2656431893 wait complete. 2024-05-15T16:22:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:36 smithi049 ceph-mon[112142]: from='client.35418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:36 smithi049 ceph-mon[112142]: pgmap v768: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:36 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3969712546' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:22:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:22:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:36 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2656431893' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:22:36.254 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:36 smithi144 ceph-mon[86635]: from='client.35418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:36.254 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:36 smithi144 ceph-mon[86635]: pgmap v768: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:36.254 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:36 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3969712546' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:22:36.255 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:22:36.255 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:36 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2656431893' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:22:37.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:37 smithi144 ceph-mon[86635]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:37.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:37 smithi049 ceph-mon[112142]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:22:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:38 smithi144 ceph-mon[86635]: pgmap v769: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:38 smithi049 ceph-mon[112142]: pgmap v769: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:40 smithi144 ceph-mon[86635]: pgmap v770: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:40 smithi049 ceph-mon[112142]: pgmap v770: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:42 smithi144 ceph-mon[86635]: pgmap v771: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:42 smithi049 ceph-mon[112142]: pgmap v771: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:44 smithi144 ceph-mon[86635]: pgmap v772: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:44 smithi049 ceph-mon[112142]: pgmap v772: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:46 smithi144 ceph-mon[86635]: pgmap v773: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:46 smithi049 ceph-mon[112142]: pgmap v773: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:48 smithi144 ceph-mon[86635]: pgmap v774: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:48.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:48 smithi049 ceph-mon[112142]: pgmap v774: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:50 smithi144 ceph-mon[86635]: pgmap v775: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:50.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:50 smithi049 ceph-mon[112142]: pgmap v775: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:22:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:22:52.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:52 smithi144 ceph-mon[86635]: pgmap v776: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:52 smithi049 ceph-mon[112142]: pgmap v776: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:54 smithi144 ceph-mon[86635]: pgmap v777: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:54 smithi049 ceph-mon[112142]: pgmap v777: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:56 smithi144 ceph-mon[86635]: pgmap v778: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:56 smithi049 ceph-mon[112142]: pgmap v778: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:22:58 smithi144 ceph-mon[86635]: pgmap v779: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:22:58.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:22:58 smithi049 ceph-mon[112142]: pgmap v779: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:00.416 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:00 smithi049 ceph-mon[112142]: pgmap v780: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:00 smithi144 ceph-mon[86635]: pgmap v780: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:02 smithi144 ceph-mon[86635]: pgmap v781: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:02 smithi049 ceph-mon[112142]: pgmap v781: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:04.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:04 smithi144 ceph-mon[86635]: pgmap v782: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:04 smithi049 ceph-mon[112142]: pgmap v782: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.067+0000 7f38456a6700 1 -- 172.21.15.49:0/662293814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 msgr2=0x7f38400ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.067+0000 7f38456a6700 1 --2- 172.21.15.49:0/662293814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 0x7f38400ffe30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3830009910 tx=0x7f3830009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.067+0000 7f38456a6700 1 -- 172.21.15.49:0/662293814 shutdown_connections 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.067+0000 7f38456a6700 1 --2- 172.21.15.49:0/662293814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3840100400 0x7f384010d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.067+0000 7f38456a6700 1 --2- 172.21.15.49:0/662293814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 0x7f38400ffe30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.067+0000 7f38456a6700 1 -- 172.21.15.49:0/662293814 >> 172.21.15.49:0/662293814 conn(0x7f38400fb690 msgr2=0x7f38400fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.067+0000 7f38456a6700 1 -- 172.21.15.49:0/662293814 shutdown_connections 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.067+0000 7f38456a6700 1 -- 172.21.15.49:0/662293814 wait complete. 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.067+0000 7f38456a6700 1 Processor -- start 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f38456a6700 1 -- start start 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f38456a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 0x7f3840198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f38456a6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3840100400 0x7f38401993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f38456a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3840199ab0 con 0x7f38400ffa50 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f38456a6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384019d780 con 0x7f3840100400 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f383effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 0x7f3840198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f383effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 0x7f3840198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39758/0 (socket says 172.21.15.49:39758) 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f383effd700 1 -- 172.21.15.49:0/2966034792 learned_addr learned my addr 172.21.15.49:0/2966034792 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f38367fc700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3840100400 0x7f38401993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f383effd700 1 -- 172.21.15.49:0/2966034792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3840100400 msgr2=0x7f38401993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f383effd700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3840100400 0x7f38401993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f383effd700 1 -- 172.21.15.49:0/2966034792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38300095f0 con 0x7f38400ffa50 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.068+0000 7f38367fc700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3840100400 0x7f38401993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.069+0000 7f383effd700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 0x7f3840198e90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f383000b850 tx=0x7f38300056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.069+0000 7f383cff9700 1 -- 172.21.15.49:0/2966034792 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f383001d070 con 0x7f38400ffa50 2024-05-15T16:23:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.069+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384019da00 con 0x7f38400ffa50 2024-05-15T16:23:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.069+0000 7f383cff9700 1 -- 172.21.15.49:0/2966034792 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3830003d50 con 0x7f38400ffa50 2024-05-15T16:23:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.069+0000 7f383cff9700 1 -- 172.21.15.49:0/2966034792 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3830021410 con 0x7f38400ffa50 2024-05-15T16:23:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.069+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384019df50 con 0x7f38400ffa50 2024-05-15T16:23:06.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.070+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384010af40 con 0x7f38400ffa50 2024-05-15T16:23:06.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.071+0000 7f383cff9700 1 -- 172.21.15.49:0/2966034792 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3830003890 con 0x7f38400ffa50 2024-05-15T16:23:06.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.071+0000 7f383cff9700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f382c06e200 0x7f382c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:06.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.071+0000 7f383cff9700 1 -- 172.21.15.49:0/2966034792 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f383008f4e0 con 0x7f38400ffa50 2024-05-15T16:23:06.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.071+0000 7f38367fc700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f382c06e200 0x7f382c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:06.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.072+0000 7f38367fc700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f382c06e200 0x7f382c0706c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f384019a3f0 tx=0x7f3828008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.077+0000 7f383cff9700 1 -- 172.21.15.49:0/2966034792 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f38300587c0 con 0x7f38400ffa50 2024-05-15T16:23:06.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.304+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f384019a1a0 con 0x7f382c06e200 2024-05-15T16:23:06.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.305+0000 7f383cff9700 1 -- 172.21.15.49:0/2966034792 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f384019a1a0 con 0x7f382c06e200 2024-05-15T16:23:06.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.307+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f382c06e200 msgr2=0x7f382c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.307+0000 7f38456a6700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f382c06e200 0x7f382c0706c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f384019a3f0 tx=0x7f3828008040 comp rx=0 tx=0).stop 2024-05-15T16:23:06.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.307+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 msgr2=0x7f3840198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.307+0000 7f38456a6700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 0x7f3840198e90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f383000b850 tx=0x7f38300056a0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.308+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 shutdown_connections 2024-05-15T16:23:06.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.308+0000 7f38456a6700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f382c06e200 0x7f382c0706c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.308+0000 7f38456a6700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38400ffa50 0x7f3840198e90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.308+0000 7f38456a6700 1 --2- 172.21.15.49:0/2966034792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3840100400 0x7f38401993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.308+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 >> 172.21.15.49:0/2966034792 conn(0x7f38400fb690 msgr2=0x7f3840108080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:06.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.308+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 shutdown_connections 2024-05-15T16:23:06.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.308+0000 7f38456a6700 1 -- 172.21.15.49:0/2966034792 wait complete. 2024-05-15T16:23:06.319 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:23:06.319 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:06 smithi049 ceph-mon[112142]: pgmap v783: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:06.319 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:23:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.419+0000 7f5980699700 1 -- 172.21.15.49:0/698317627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 msgr2=0x7f59781022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.419+0000 7f5980699700 1 --2- 172.21.15.49:0/698317627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 0x7f59781022a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f597000b0a0 tx=0x7f597000b3b0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.419+0000 7f5980699700 1 -- 172.21.15.49:0/698317627 shutdown_connections 2024-05-15T16:23:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.419+0000 7f5980699700 1 --2- 172.21.15.49:0/698317627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59781027e0 0x7f597810acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.419+0000 7f5980699700 1 --2- 172.21.15.49:0/698317627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 0x7f59781022a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.419+0000 7f5980699700 1 -- 172.21.15.49:0/698317627 >> 172.21.15.49:0/698317627 conn(0x7f5978076800 msgr2=0x7f59780fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.419+0000 7f5980699700 1 -- 172.21.15.49:0/698317627 shutdown_connections 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.419+0000 7f5980699700 1 -- 172.21.15.49:0/698317627 wait complete. 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f5980699700 1 Processor -- start 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f5980699700 1 -- start start 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f5980699700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 0x7f59780ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f5980699700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59781027e0 0x7f5978100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f5980699700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59781005a0 con 0x7f5978101ec0 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f5980699700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59781006e0 con 0x7f59781027e0 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f597e435700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 0x7f59780ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f597e435700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 0x7f59780ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39776/0 (socket says 172.21.15.49:39776) 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f597e435700 1 -- 172.21.15.49:0/2073033671 learned_addr learned my addr 172.21.15.49:0/2073033671 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f597e435700 1 -- 172.21.15.49:0/2073033671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59781027e0 msgr2=0x7f5978100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.420+0000 7f597e435700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59781027e0 0x7f5978100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.421+0000 7f597e435700 1 -- 172.21.15.49:0/2073033671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970009d00 con 0x7f5978101ec0 2024-05-15T16:23:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.421+0000 7f597e435700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 0x7f59780ffb20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5970015040 tx=0x7f5970003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.421+0000 7f59777fe700 1 -- 172.21.15.49:0/2073033671 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f597000e040 con 0x7f5978101ec0 2024-05-15T16:23:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.421+0000 7f59777fe700 1 -- 172.21.15.49:0/2073033671 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5970004040 con 0x7f5978101ec0 2024-05-15T16:23:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.421+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5978100960 con 0x7f5978101ec0 2024-05-15T16:23:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.421+0000 7f59777fe700 1 -- 172.21.15.49:0/2073033671 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f597001d6b0 con 0x7f5978101ec0 2024-05-15T16:23:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.421+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59781a3000 con 0x7f5978101ec0 2024-05-15T16:23:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.422+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5978068ec0 con 0x7f5978101ec0 2024-05-15T16:23:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.423+0000 7f59777fe700 1 -- 172.21.15.49:0/2073033671 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f59700041b0 con 0x7f5978101ec0 2024-05-15T16:23:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.423+0000 7f59777fe700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596006e200 0x7f59600706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.423+0000 7f59777fe700 1 -- 172.21.15.49:0/2073033671 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f597008f910 con 0x7f5978101ec0 2024-05-15T16:23:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.423+0000 7f597dc34700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596006e200 0x7f59600706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.424+0000 7f597dc34700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596006e200 0x7f59600706c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f596c005950 tx=0x7f596c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:06.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.428+0000 7f59777fe700 1 -- 172.21.15.49:0/2073033671 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5970058bf0 con 0x7f5978101ec0 2024-05-15T16:23:06.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:06 smithi144 ceph-mon[86635]: pgmap v783: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:23:06.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.654+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f597810dea0 con 0x7f596006e200 2024-05-15T16:23:06.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.656+0000 7f59777fe700 1 -- 172.21.15.49:0/2073033671 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f597810dea0 con 0x7f596006e200 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596006e200 msgr2=0x7f59600706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596006e200 0x7f59600706c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f596c005950 tx=0x7f596c0058e0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 msgr2=0x7f59780ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 0x7f59780ffb20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5970015040 tx=0x7f5970003ce0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 shutdown_connections 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596006e200 0x7f59600706c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5978101ec0 0x7f59780ffb20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 --2- 172.21.15.49:0/2073033671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59781027e0 0x7f5978100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 >> 172.21.15.49:0/2073033671 conn(0x7f5978076800 msgr2=0x7f59781031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 shutdown_connections 2024-05-15T16:23:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.658+0000 7f5980699700 1 -- 172.21.15.49:0/2073033671 wait complete. 2024-05-15T16:23:06.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.766+0000 7f0a164df700 1 -- 172.21.15.49:0/170211276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 msgr2=0x7f0a10107210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.766+0000 7f0a164df700 1 --2- 172.21.15.49:0/170211276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 0x7f0a10107210 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009910 tx=0x7f0a00009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:06.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.767+0000 7f0a164df700 1 -- 172.21.15.49:0/170211276 shutdown_connections 2024-05-15T16:23:06.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.767+0000 7f0a164df700 1 --2- 172.21.15.49:0/170211276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a100ff490 0x7f0a100ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.767+0000 7f0a164df700 1 --2- 172.21.15.49:0/170211276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 0x7f0a10107210 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.767+0000 7f0a164df700 1 -- 172.21.15.49:0/170211276 >> 172.21.15.49:0/170211276 conn(0x7f0a10074e50 msgr2=0x7f0a10075260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:06.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.767+0000 7f0a164df700 1 -- 172.21.15.49:0/170211276 shutdown_connections 2024-05-15T16:23:06.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.767+0000 7f0a164df700 1 -- 172.21.15.49:0/170211276 wait complete. 2024-05-15T16:23:06.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.767+0000 7f0a164df700 1 Processor -- start 2024-05-15T16:23:06.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.767+0000 7f0a164df700 1 -- start start 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a164df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a100ff490 0x7f0a10100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a164df700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 0x7f0a10100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a164df700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101010f0 con 0x7f0a10106e30 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a164df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10101230 con 0x7f0a100ff490 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a0f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 0x7f0a10100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a0f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 0x7f0a10100bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39798/0 (socket says 172.21.15.49:39798) 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a0f7fe700 1 -- 172.21.15.49:0/2716955759 learned_addr learned my addr 172.21.15.49:0/2716955759 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a0ffff700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a100ff490 0x7f0a10100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a0f7fe700 1 -- 172.21.15.49:0/2716955759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a100ff490 msgr2=0x7f0a10100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a0f7fe700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a100ff490 0x7f0a10100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a0f7fe700 1 -- 172.21.15.49:0/2716955759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a000095f0 con 0x7f0a10106e30 2024-05-15T16:23:06.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a0f7fe700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 0x7f0a10100bb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400d8d0 tx=0x7f0a0400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:06.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.768+0000 7f0a0d7fa700 1 -- 172.21.15.49:0/2716955759 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a040098a0 con 0x7f0a10106e30 2024-05-15T16:23:06.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.769+0000 7f0a0d7fa700 1 -- 172.21.15.49:0/2716955759 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a04010460 con 0x7f0a10106e30 2024-05-15T16:23:06.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.769+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a101a2e80 con 0x7f0a10106e30 2024-05-15T16:23:06.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.769+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a101a31e0 con 0x7f0a10106e30 2024-05-15T16:23:06.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.769+0000 7f0a0d7fa700 1 -- 172.21.15.49:0/2716955759 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a0400f5f0 con 0x7f0a10106e30 2024-05-15T16:23:06.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.769+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc0052f0 con 0x7f0a10106e30 2024-05-15T16:23:06.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.770+0000 7f0a0d7fa700 1 -- 172.21.15.49:0/2716955759 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0a04009a00 con 0x7f0a10106e30 2024-05-15T16:23:06.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.770+0000 7f0a0d7fa700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09f806e200 0x7f09f80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:06.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.771+0000 7f0a0d7fa700 1 -- 172.21.15.49:0/2716955759 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0a0408df80 con 0x7f0a10106e30 2024-05-15T16:23:06.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.771+0000 7f0a0ffff700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09f806e200 0x7f09f80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:06.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.771+0000 7f0a0ffff700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09f806e200 0x7f09f80706c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000b5c0 tx=0x7f0a0001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:06.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.775+0000 7f0a0d7fa700 1 -- 172.21.15.49:0/2716955759 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0a04057260 con 0x7f0a10106e30 2024-05-15T16:23:06.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:06.995+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f09fc000bc0 con 0x7f09f806e200 2024-05-15T16:23:07.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.006+0000 7f0a0d7fa700 1 -- 172.21.15.49:0/2716955759 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f09fc000bc0 con 0x7f09f806e200 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (26m) 4m ago 37m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (38m) 4m ago 38m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (37m) 4m ago 37m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (23m) 4m ago 38m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (23m) 4m ago 36m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (25m) 4m ago 37m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (20m) 4m ago 32m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (19m) 4m ago 32m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (19m) 4m ago 32m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (4m) 4m ago 32m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:23:07.007 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (28m) 4m ago 39m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (27m) 4m ago 36m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (24m) 4m ago 39m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (24m) 4m ago 36m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (27m) 4m ago 38m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (27m) 4m ago 36m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (23m) 4m ago 36m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (22m) 4m ago 35m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (22m) 4m ago 35m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (21m) 4m ago 34m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (20m) 4m ago 34m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (20m) 4m ago 33m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:23:07.008 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (26m) 4m ago 37m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:23:07.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.008+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09f806e200 msgr2=0x7f09f80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.008+0000 7f0a164df700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09f806e200 0x7f09f80706c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000b5c0 tx=0x7f0a0001a040 comp rx=0 tx=0).stop 2024-05-15T16:23:07.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.008+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 msgr2=0x7f0a10100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.008+0000 7f0a164df700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 0x7f0a10100bb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400d8d0 tx=0x7f0a0400dbe0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.009+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 shutdown_connections 2024-05-15T16:23:07.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.009+0000 7f0a164df700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09f806e200 0x7f09f80706c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.009+0000 7f0a164df700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a100ff490 0x7f0a10100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.009+0000 7f0a164df700 1 --2- 172.21.15.49:0/2716955759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a10106e30 0x7f0a10100bb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.009+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 >> 172.21.15.49:0/2716955759 conn(0x7f0a10074e50 msgr2=0x7f0a100fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:07.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.009+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 shutdown_connections 2024-05-15T16:23:07.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.009+0000 7f0a164df700 1 -- 172.21.15.49:0/2716955759 wait complete. 2024-05-15T16:23:07.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3461338998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 msgr2=0x7fd6ec1001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 --2- 172.21.15.49:0/3461338998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 0x7fd6ec1001f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009910 tx=0x7fd6e0009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:07.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3461338998 shutdown_connections 2024-05-15T16:23:07.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 --2- 172.21.15.49:0/3461338998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ec1007c0 0x7fd6ec104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 --2- 172.21.15.49:0/3461338998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 0x7fd6ec1001f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3461338998 >> 172.21.15.49:0/3461338998 conn(0x7fd6ec0fb6a0 msgr2=0x7fd6ec0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:07.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3461338998 shutdown_connections 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3461338998 wait complete. 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 Processor -- start 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.117+0000 7fd6f2b54700 1 -- start start 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f2b54700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 0x7fd6ec198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f2b54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ec1007c0 0x7fd6ec199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f2b54700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1999e0 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f2b54700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec19d6b0 con 0x7fd6ec1007c0 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f08f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 0x7fd6ec198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f08f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 0x7fd6ec198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39812/0 (socket says 172.21.15.49:39812) 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f08f0700 1 -- 172.21.15.49:0/3265863541 learned_addr learned my addr 172.21.15.49:0/3265863541 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6ebfff700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ec1007c0 0x7fd6ec199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f08f0700 1 -- 172.21.15.49:0/3265863541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ec1007c0 msgr2=0x7fd6ec199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f08f0700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ec1007c0 0x7fd6ec199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f08f0700 1 -- 172.21.15.49:0/3265863541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00095f0 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6f08f0700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 0x7fd6ec198dc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006010 tx=0x7fd6e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:07.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.118+0000 7fd6e9ffb700 1 -- 172.21.15.49:0/3265863541 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e001d070 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.119+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec19d930 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.119+0000 7fd6e9ffb700 1 -- 172.21.15.49:0/3265863541 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6e00056e0 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.119+0000 7fd6e9ffb700 1 -- 172.21.15.49:0/3265863541 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e000fdf0 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.119+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec19de20 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.120+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec1081b0 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.120+0000 7fd6e9ffb700 1 -- 172.21.15.49:0/3265863541 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd6e000f5f0 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.121+0000 7fd6e9ffb700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd6dc06e200 0x7fd6dc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:07.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.121+0000 7fd6e9ffb700 1 -- 172.21.15.49:0/3265863541 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd6e008f4d0 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.121+0000 7fd6ebfff700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd6dc06e200 0x7fd6dc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:07.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.121+0000 7fd6ebfff700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd6dc06e200 0x7fd6dc0706c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec19a320 tx=0x7fd6d800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:07.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.125+0000 7fd6e9ffb700 1 -- 172.21.15.49:0/3265863541 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd6e00587b0 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.418+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd6ec19a0d0 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.418+0000 7fd6e9ffb700 1 -- 172.21.15.49:0/3265863541 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd6e005be70 con 0x7fd6ec0ffe10 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:23:07.419 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:23:07.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:23:07.420 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:23:07.420 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:23:07.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:23:07.420 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:23:07.420 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:23:07.420 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:23:07.420 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:23:07.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.421+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd6dc06e200 msgr2=0x7fd6dc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd6dc06e200 0x7fd6dc0706c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec19a320 tx=0x7fd6d800b410 comp rx=0 tx=0).stop 2024-05-15T16:23:07.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 msgr2=0x7fd6ec198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 0x7fd6ec198dc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006010 tx=0x7fd6e000bd70 comp rx=0 tx=0).stop 2024-05-15T16:23:07.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 shutdown_connections 2024-05-15T16:23:07.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd6dc06e200 0x7fd6dc0706c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6ec0ffe10 0x7fd6ec198dc0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 --2- 172.21.15.49:0/3265863541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6ec1007c0 0x7fd6ec199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 >> 172.21.15.49:0/3265863541 conn(0x7fd6ec0fb6a0 msgr2=0x7fd6ec0fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:07.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 shutdown_connections 2024-05-15T16:23:07.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.422+0000 7fd6f2b54700 1 -- 172.21.15.49:0/3265863541 wait complete. 2024-05-15T16:23:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.530+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/647691336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c04101ed0 msgr2=0x7f3c041022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.530+0000 7f3c0ac8b700 1 --2- 172.21.15.49:0/647691336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c04101ed0 0x7f3c041022b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8009940 tx=0x7f3bf8009c50 comp rx=0 tx=0).stop 2024-05-15T16:23:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.530+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/647691336 shutdown_connections 2024-05-15T16:23:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.530+0000 7f3c0ac8b700 1 --2- 172.21.15.49:0/647691336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c041027f0 0x7f3c0410ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.530+0000 7f3c0ac8b700 1 --2- 172.21.15.49:0/647691336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c04101ed0 0x7f3c041022b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.530+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/647691336 >> 172.21.15.49:0/647691336 conn(0x7f3c040fb700 msgr2=0x7f3c040fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.530+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/647691336 shutdown_connections 2024-05-15T16:23:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.530+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/647691336 wait complete. 2024-05-15T16:23:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.531+0000 7f3c0ac8b700 1 Processor -- start 2024-05-15T16:23:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.531+0000 7f3c0ac8b700 1 -- start start 2024-05-15T16:23:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.531+0000 7f3c0ac8b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c04101ed0 0x7f3c0419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.531+0000 7f3c0ac8b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c041027f0 0x7f3c0419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.531+0000 7f3c0ac8b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0419d420 con 0x7f3c041027f0 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.531+0000 7f3c0ac8b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c04196a90 con 0x7f3c04101ed0 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c03fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c041027f0 0x7f3c0419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c03fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c041027f0 0x7f3c0419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39832/0 (socket says 172.21.15.49:39832) 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c03fff700 1 -- 172.21.15.49:0/236225099 learned_addr learned my addr 172.21.15.49:0/236225099 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c08a27700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c04101ed0 0x7f3c0419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c03fff700 1 -- 172.21.15.49:0/236225099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c04101ed0 msgr2=0x7f3c0419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c03fff700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c04101ed0 0x7f3c0419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c03fff700 1 -- 172.21.15.49:0/236225099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf80095f0 con 0x7f3c041027f0 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c03fff700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c041027f0 0x7f3c0419ce90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000c960 tx=0x7f3bf000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c01ffb700 1 -- 172.21.15.49:0/236225099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bf0007a70 con 0x7f3c041027f0 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c01ffb700 1 -- 172.21.15.49:0/236225099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bf0007bd0 con 0x7f3c041027f0 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c04196d70 con 0x7f3c041027f0 2024-05-15T16:23:07.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.532+0000 7f3c01ffb700 1 -- 172.21.15.49:0/236225099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bf0018630 con 0x7f3c041027f0 2024-05-15T16:23:07.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.533+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c041972c0 con 0x7f3c041027f0 2024-05-15T16:23:07.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.534+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c041055f0 con 0x7f3c041027f0 2024-05-15T16:23:07.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.534+0000 7f3c01ffb700 1 -- 172.21.15.49:0/236225099 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3bf000f450 con 0x7f3c041027f0 2024-05-15T16:23:07.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.535+0000 7f3c01ffb700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:07.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.535+0000 7f3c01ffb700 1 -- 172.21.15.49:0/236225099 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3bf008e100 con 0x7f3c041027f0 2024-05-15T16:23:07.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.535+0000 7f3c08a27700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:07.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.535+0000 7f3c08a27700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8006010 tx=0x7f3bf800b8f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:07.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.539+0000 7f3c01ffb700 1 -- 172.21.15.49:0/236225099 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3bf005af10 con 0x7f3c041027f0 2024-05-15T16:23:07.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.791+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3c04069a50 con 0x7f3c041027f0 2024-05-15T16:23:07.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.791+0000 7f3c01ffb700 1 -- 172.21.15.49:0/236225099 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f3bf005aaa0 con 0x7f3c041027f0 2024-05-15T16:23:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:23:07.792 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:23:07.793 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:23:07.794 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:23:07.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.793+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 msgr2=0x7f3bf40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8006010 tx=0x7f3bf800b8f0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c041027f0 msgr2=0x7f3c0419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c041027f0 0x7f3c0419ce90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000c960 tx=0x7f3bf000cc70 comp rx=0 tx=0).stop 2024-05-15T16:23:07.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 shutdown_connections 2024-05-15T16:23:07.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f3bf406e200 0x7f3bf40706c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c04101ed0 0x7f3c0419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 --2- 172.21.15.49:0/236225099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c041027f0 0x7f3c0419ce90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 >> 172.21.15.49:0/236225099 conn(0x7f3c040fb700 msgr2=0x7f3c041031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:07.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 shutdown_connections 2024-05-15T16:23:07.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.794+0000 7f3c0ac8b700 1 -- 172.21.15.49:0/236225099 wait complete. 2024-05-15T16:23:07.796 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:23:07.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.903+0000 7f995a461700 1 -- 172.21.15.49:0/920027476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99540ffdc0 msgr2=0x7f99541001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.903+0000 7f995a461700 1 --2- 172.21.15.49:0/920027476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99540ffdc0 0x7f99541001a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9944009910 tx=0x7f9944009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:07.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.903+0000 7f995a461700 1 -- 172.21.15.49:0/920027476 shutdown_connections 2024-05-15T16:23:07.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.903+0000 7f995a461700 1 --2- 172.21.15.49:0/920027476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9954100770 0x7f99541047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.903+0000 7f995a461700 1 --2- 172.21.15.49:0/920027476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99540ffdc0 0x7f99541001a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.903+0000 7f995a461700 1 -- 172.21.15.49:0/920027476 >> 172.21.15.49:0/920027476 conn(0x7f9954076800 msgr2=0x7f99540fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:07.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.903+0000 7f995a461700 1 -- 172.21.15.49:0/920027476 shutdown_connections 2024-05-15T16:23:07.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.903+0000 7f995a461700 1 -- 172.21.15.49:0/920027476 wait complete. 2024-05-15T16:23:07.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.904+0000 7f995a461700 1 Processor -- start 2024-05-15T16:23:07.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.904+0000 7f995a461700 1 -- start start 2024-05-15T16:23:07.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.904+0000 7f995a461700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99540ffdc0 0x7f99540fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:07.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.904+0000 7f995a461700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954100770 0x7f99540fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:07.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.904+0000 7f995a461700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540fc760 con 0x7f9954100770 2024-05-15T16:23:07.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.904+0000 7f995a461700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540fc8d0 con 0x7f99540ffdc0 2024-05-15T16:23:07.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.904+0000 7f994bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954100770 0x7f99540fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:07.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.904+0000 7f994bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954100770 0x7f99540fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39852/0 (socket says 172.21.15.49:39852) 2024-05-15T16:23:07.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.904+0000 7f994bfff700 1 -- 172.21.15.49:0/1927648481 learned_addr learned my addr 172.21.15.49:0/1927648481 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:07.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f9953fff700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99540ffdc0 0x7f99540fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:07.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f994bfff700 1 -- 172.21.15.49:0/1927648481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99540ffdc0 msgr2=0x7f99540fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:07.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f994bfff700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99540ffdc0 0x7f99540fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:07.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f994bfff700 1 -- 172.21.15.49:0/1927648481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99440095f0 con 0x7f9954100770 2024-05-15T16:23:07.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f994bfff700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954100770 0x7f99540fc220 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f993c00b480 tx=0x7f993c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:07.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f9951ffb700 1 -- 172.21.15.49:0/1927648481 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f993c011800 con 0x7f9954100770 2024-05-15T16:23:07.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f9951ffb700 1 -- 172.21.15.49:0/1927648481 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f993c011e20 con 0x7f9954100770 2024-05-15T16:23:07.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99541a2ee0 con 0x7f9954100770 2024-05-15T16:23:07.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f9951ffb700 1 -- 172.21.15.49:0/1927648481 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f993c010ea0 con 0x7f9954100770 2024-05-15T16:23:07.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.905+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99541a3290 con 0x7f9954100770 2024-05-15T16:23:07.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.906+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99541012a0 con 0x7f9954100770 2024-05-15T16:23:07.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.907+0000 7f9951ffb700 1 -- 172.21.15.49:0/1927648481 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f993c010380 con 0x7f9954100770 2024-05-15T16:23:07.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.907+0000 7f9951ffb700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f994006e210 0x7f99400706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:07.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.907+0000 7f9951ffb700 1 -- 172.21.15.49:0/1927648481 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f993c08dc90 con 0x7f9954100770 2024-05-15T16:23:07.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.907+0000 7f9953fff700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f994006e210 0x7f99400706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:07.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.908+0000 7f9953fff700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f994006e210 0x7f99400706d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9944006010 tx=0x7f994401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:07.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:07.912+0000 7f9951ffb700 1 -- 172.21.15.49:0/1927648481 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f993c056f70 con 0x7f9954100770 2024-05-15T16:23:08.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.139+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f99540fca90 con 0x7f994006e210 2024-05-15T16:23:08.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.140+0000 7f9951ffb700 1 -- 172.21.15.49:0/1927648481 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f99540fca90 con 0x7f994006e210 2024-05-15T16:23:08.140 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:23:08.140 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:23:08.140 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:23:08.140 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:23:08.140 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:23:08.140 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:23:08.141 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:23:08.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:23:08.141 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:23:08.141 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:23:08.141 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:23:08.141 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:23:08.141 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:23:08.141 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:23:08.141 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:23:08.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f994006e210 msgr2=0x7f99400706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f994006e210 0x7f99400706d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9944006010 tx=0x7f994401a040 comp rx=0 tx=0).stop 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954100770 msgr2=0x7f99540fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954100770 0x7f99540fc220 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f993c00b480 tx=0x7f993c00b790 comp rx=0 tx=0).stop 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 shutdown_connections 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f994006e210 0x7f99400706d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99540ffdc0 0x7f99540fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 --2- 172.21.15.49:0/1927648481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9954100770 0x7f99540fc220 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 >> 172.21.15.49:0/1927648481 conn(0x7f9954076800 msgr2=0x7f99540fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 shutdown_connections 2024-05-15T16:23:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.142+0000 7f995a461700 1 -- 172.21.15.49:0/1927648481 wait complete. 2024-05-15T16:23:08.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.251+0000 7efc3c1a6700 1 -- 172.21.15.49:0/1818202523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 msgr2=0x7efc341047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.251+0000 7efc3c1a6700 1 --2- 172.21.15.49:0/1818202523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 0x7efc341047c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efc28009910 tx=0x7efc28009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.251+0000 7efc3c1a6700 1 -- 172.21.15.49:0/1818202523 shutdown_connections 2024-05-15T16:23:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.251+0000 7efc3c1a6700 1 --2- 172.21.15.49:0/1818202523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 0x7efc341047c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.251+0000 7efc3c1a6700 1 --2- 172.21.15.49:0/1818202523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc340ffdc0 0x7efc341001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.251+0000 7efc3c1a6700 1 -- 172.21.15.49:0/1818202523 >> 172.21.15.49:0/1818202523 conn(0x7efc34076800 msgr2=0x7efc340fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.251+0000 7efc3c1a6700 1 -- 172.21.15.49:0/1818202523 shutdown_connections 2024-05-15T16:23:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.251+0000 7efc3c1a6700 1 -- 172.21.15.49:0/1818202523 wait complete. 2024-05-15T16:23:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc3c1a6700 1 Processor -- start 2024-05-15T16:23:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc3c1a6700 1 -- start start 2024-05-15T16:23:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc3c1a6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc340ffdc0 0x7efc340fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc3c1a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 0x7efc340fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc3c1a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc340fc760 con 0x7efc34100770 2024-05-15T16:23:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc3c1a6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc340fc8a0 con 0x7efc340ffdc0 2024-05-15T16:23:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc39741700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 0x7efc340fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc39741700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 0x7efc340fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39874/0 (socket says 172.21.15.49:39874) 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc39741700 1 -- 172.21.15.49:0/4094980085 learned_addr learned my addr 172.21.15.49:0/4094980085 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.252+0000 7efc39f42700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc340ffdc0 0x7efc340fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.253+0000 7efc39741700 1 -- 172.21.15.49:0/4094980085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc340ffdc0 msgr2=0x7efc340fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.253+0000 7efc39741700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc340ffdc0 0x7efc340fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.253+0000 7efc39741700 1 -- 172.21.15.49:0/4094980085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280095f0 con 0x7efc34100770 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.253+0000 7efc39741700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 0x7efc340fc220 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efc28005b40 tx=0x7efc2800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.253+0000 7efc32ffd700 1 -- 172.21.15.49:0/4094980085 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2801d070 con 0x7efc34100770 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.253+0000 7efc32ffd700 1 -- 172.21.15.49:0/4094980085 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc28005700 con 0x7efc34100770 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.253+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc340fcb20 con 0x7efc34100770 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.253+0000 7efc32ffd700 1 -- 172.21.15.49:0/4094980085 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2800fd60 con 0x7efc34100770 2024-05-15T16:23:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.253+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc341a2fa0 con 0x7efc34100770 2024-05-15T16:23:08.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.254+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc341012a0 con 0x7efc34100770 2024-05-15T16:23:08.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.255+0000 7efc32ffd700 1 -- 172.21.15.49:0/4094980085 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efc2800f560 con 0x7efc34100770 2024-05-15T16:23:08.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.255+0000 7efc32ffd700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efc1c06e210 0x7efc1c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:08.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.256+0000 7efc32ffd700 1 -- 172.21.15.49:0/4094980085 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efc2808f5b0 con 0x7efc34100770 2024-05-15T16:23:08.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.256+0000 7efc39f42700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efc1c06e210 0x7efc1c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:08.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.256+0000 7efc39f42700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efc1c06e210 0x7efc1c0706d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efc24009550 tx=0x7efc240091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:08.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.260+0000 7efc32ffd700 1 -- 172.21.15.49:0/4094980085 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efc28058890 con 0x7efc34100770 2024-05-15T16:23:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:08 smithi144 ceph-mon[86635]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:08 smithi144 ceph-mon[86635]: pgmap v784: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:08 smithi144 ceph-mon[86635]: from='client.35442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:08 smithi144 ceph-mon[86635]: from='client.35446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:08 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3265863541' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:23:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:08 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/236225099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:23:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:08 smithi049 ceph-mon[112142]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:08 smithi049 ceph-mon[112142]: pgmap v784: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:08 smithi049 ceph-mon[112142]: from='client.35442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:08 smithi049 ceph-mon[112142]: from='client.35446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:08 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3265863541' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:23:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:08 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/236225099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:23:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.552+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efc3410cd60 con 0x7efc34100770 2024-05-15T16:23:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.553+0000 7efc32ffd700 1 -- 172.21.15.49:0/4094980085 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7efc2805bf50 con 0x7efc34100770 2024-05-15T16:23:08.553 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:23:08.553 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:23:08.553 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:23:08.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:23:08.553 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:23:08.553 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:23:08.554 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:23:08.555 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:23:08.555 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:23:08.555 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:23:08.555 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:23:08.555 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:23:08.555 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:23:08.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efc1c06e210 msgr2=0x7efc1c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efc1c06e210 0x7efc1c0706d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efc24009550 tx=0x7efc240091b0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 msgr2=0x7efc340fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 0x7efc340fc220 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efc28005b40 tx=0x7efc2800bd70 comp rx=0 tx=0).stop 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 shutdown_connections 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efc1c06e210 0x7efc1c0706d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc340ffdc0 0x7efc340fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 --2- 172.21.15.49:0/4094980085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc34100770 0x7efc340fc220 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 >> 172.21.15.49:0/4094980085 conn(0x7efc34076800 msgr2=0x7efc34068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 shutdown_connections 2024-05-15T16:23:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:08.555+0000 7efc3c1a6700 1 -- 172.21.15.49:0/4094980085 wait complete. 2024-05-15T16:23:09.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:09 smithi144 ceph-mon[86635]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:09.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:09 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4094980085' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:23:09.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:09 smithi049 ceph-mon[112142]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:09.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:09 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4094980085' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:23:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:10 smithi144 ceph-mon[86635]: pgmap v785: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:10 smithi049 ceph-mon[112142]: pgmap v785: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:12 smithi144 ceph-mon[86635]: pgmap v786: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:12.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:12 smithi049 ceph-mon[112142]: pgmap v786: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:14 smithi144 ceph-mon[86635]: pgmap v787: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:14.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:14 smithi049 ceph-mon[112142]: pgmap v787: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:16.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:16 smithi144 ceph-mon[86635]: pgmap v788: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:16 smithi049 ceph-mon[112142]: pgmap v788: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:18 smithi144 ceph-mon[86635]: pgmap v789: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:18 smithi049 ceph-mon[112142]: pgmap v789: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:20 smithi144 ceph-mon[86635]: pgmap v790: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:20 smithi049 ceph-mon[112142]: pgmap v790: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:23:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:23:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:22 smithi144 ceph-mon[86635]: pgmap v791: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:22 smithi049 ceph-mon[112142]: pgmap v791: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:24 smithi144 ceph-mon[86635]: pgmap v792: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:24 smithi049 ceph-mon[112142]: pgmap v792: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:26 smithi144 ceph-mon[86635]: pgmap v793: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:26 smithi049 ceph-mon[112142]: pgmap v793: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:28 smithi144 ceph-mon[86635]: pgmap v794: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:28 smithi049 ceph-mon[112142]: pgmap v794: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:30 smithi144 ceph-mon[86635]: pgmap v795: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:30 smithi049 ceph-mon[112142]: pgmap v795: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:32 smithi144 ceph-mon[86635]: pgmap v796: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:32 smithi049 ceph-mon[112142]: pgmap v796: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:34 smithi144 ceph-mon[86635]: pgmap v797: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:34 smithi049 ceph-mon[112142]: pgmap v797: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:36 smithi144 ceph-mon[86635]: pgmap v798: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:23:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:36 smithi049 ceph-mon[112142]: pgmap v798: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:23:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:38 smithi144 ceph-mon[86635]: pgmap v799: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:38 smithi049 ceph-mon[112142]: pgmap v799: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:38.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.678+0000 7fd7f44e2700 1 -- 172.21.15.49:0/597911864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec100770 msgr2=0x7fd7ec1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:38.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.678+0000 7fd7f44e2700 1 --2- 172.21.15.49:0/597911864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec100770 0x7fd7ec1047c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009910 tx=0x7fd7e0009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:38.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.678+0000 7fd7f44e2700 1 -- 172.21.15.49:0/597911864 shutdown_connections 2024-05-15T16:23:38.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.678+0000 7fd7f44e2700 1 --2- 172.21.15.49:0/597911864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec100770 0x7fd7ec1047c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:38.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.678+0000 7fd7f44e2700 1 --2- 172.21.15.49:0/597911864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ec0ffdc0 0x7fd7ec1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:38.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.678+0000 7fd7f44e2700 1 -- 172.21.15.49:0/597911864 >> 172.21.15.49:0/597911864 conn(0x7fd7ec076800 msgr2=0x7fd7ec0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:38.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.678+0000 7fd7f44e2700 1 -- 172.21.15.49:0/597911864 shutdown_connections 2024-05-15T16:23:38.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.678+0000 7fd7f44e2700 1 -- 172.21.15.49:0/597911864 wait complete. 2024-05-15T16:23:38.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.679+0000 7fd7f44e2700 1 Processor -- start 2024-05-15T16:23:38.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.679+0000 7fd7f44e2700 1 -- start start 2024-05-15T16:23:38.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.679+0000 7fd7f44e2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec0ffdc0 0x7fd7ec0fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:38.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.679+0000 7fd7f44e2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ec100770 0x7fd7ec0fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:38.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.679+0000 7fd7f44e2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0fc760 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.679+0000 7fd7f44e2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0fc8a0 con 0x7fd7ec100770 2024-05-15T16:23:38.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.679+0000 7fd7f227e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec0ffdc0 0x7fd7ec0fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:38.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7f227e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec0ffdc0 0x7fd7ec0fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51904/0 (socket says 172.21.15.49:51904) 2024-05-15T16:23:38.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7f227e700 1 -- 172.21.15.49:0/4002751211 learned_addr learned my addr 172.21.15.49:0/4002751211 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:38.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7f1a7d700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ec100770 0x7fd7ec0fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:38.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7f227e700 1 -- 172.21.15.49:0/4002751211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ec100770 msgr2=0x7fd7ec0fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:38.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7f227e700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ec100770 0x7fd7ec0fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:38.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7f227e700 1 -- 172.21.15.49:0/4002751211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e00095f0 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7f227e700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec0ffdc0 0x7fd7ec0fbce0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0060b0 tx=0x7fd7dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:38.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7eb7fe700 1 -- 172.21.15.49:0/4002751211 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7dc00dda0 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7eb7fe700 1 -- 172.21.15.49:0/4002751211 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7dc00f040 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ec1a2d70 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7eb7fe700 1 -- 172.21.15.49:0/4002751211 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7dc014790 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.680+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ec1a30d0 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.681+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ec1012a0 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.682+0000 7fd7eb7fe700 1 -- 172.21.15.49:0/4002751211 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd7dc014980 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.682+0000 7fd7eb7fe700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd7d406e200 0x7fd7d40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:38.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.683+0000 7fd7f1a7d700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd7d406e200 0x7fd7d40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:38.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.683+0000 7fd7eb7fe700 1 -- 172.21.15.49:0/4002751211 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd7dc08dbe0 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.683+0000 7fd7f1a7d700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd7d406e200 0x7fd7d40706c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0fd3c0 tx=0x7fd7e001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:38.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.688+0000 7fd7eb7fe700 1 -- 172.21.15.49:0/4002751211 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd7dc056f70 con 0x7fd7ec0ffdc0 2024-05-15T16:23:38.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.915+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7ec10cdd0 con 0x7fd7d406e200 2024-05-15T16:23:38.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.916+0000 7fd7eb7fe700 1 -- 172.21.15.49:0/4002751211 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd7ec10cdd0 con 0x7fd7d406e200 2024-05-15T16:23:38.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd7d406e200 msgr2=0x7fd7d40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:38.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd7d406e200 0x7fd7d40706c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0fd3c0 tx=0x7fd7e001a040 comp rx=0 tx=0).stop 2024-05-15T16:23:38.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec0ffdc0 msgr2=0x7fd7ec0fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:38.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec0ffdc0 0x7fd7ec0fbce0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0060b0 tx=0x7fd7dc00d3b0 comp rx=0 tx=0).stop 2024-05-15T16:23:38.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 shutdown_connections 2024-05-15T16:23:38.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd7d406e200 0x7fd7d40706c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:38.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7ec0ffdc0 0x7fd7ec0fbce0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:38.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 --2- 172.21.15.49:0/4002751211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ec100770 0x7fd7ec0fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:38.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 >> 172.21.15.49:0/4002751211 conn(0x7fd7ec076800 msgr2=0x7fd7ec068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:38.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.918+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 shutdown_connections 2024-05-15T16:23:38.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:38.919+0000 7fd7f44e2700 1 -- 172.21.15.49:0/4002751211 wait complete. 2024-05-15T16:23:38.930 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:23:39.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.027+0000 7f4e5d862700 1 -- 172.21.15.49:0/2049454082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 msgr2=0x7f4e581001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.027+0000 7f4e5d862700 1 --2- 172.21.15.49:0/2049454082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 0x7f4e581001d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800b0a0 tx=0x7f4e4800b3b0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.027+0000 7f4e5d862700 1 -- 172.21.15.49:0/2049454082 shutdown_connections 2024-05-15T16:23:39.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.027+0000 7f4e5d862700 1 --2- 172.21.15.49:0/2049454082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e581007a0 0x7f4e581047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.027+0000 7f4e5d862700 1 --2- 172.21.15.49:0/2049454082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 0x7f4e581001d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.027+0000 7f4e5d862700 1 -- 172.21.15.49:0/2049454082 >> 172.21.15.49:0/2049454082 conn(0x7f4e580fb6a0 msgr2=0x7f4e580fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:39.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.027+0000 7f4e5d862700 1 -- 172.21.15.49:0/2049454082 shutdown_connections 2024-05-15T16:23:39.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.027+0000 7f4e5d862700 1 -- 172.21.15.49:0/2049454082 wait complete. 2024-05-15T16:23:39.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.028+0000 7f4e5d862700 1 Processor -- start 2024-05-15T16:23:39.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.028+0000 7f4e5d862700 1 -- start start 2024-05-15T16:23:39.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.028+0000 7f4e5d862700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 0x7f4e5819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:39.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.028+0000 7f4e5d862700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e581007a0 0x7f4e5819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:39.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.028+0000 7f4e5d862700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5819d3f0 con 0x7f4e580ffdf0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.028+0000 7f4e5d862700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e581969d0 con 0x7f4e581007a0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e56ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 0x7f4e5819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e56ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 0x7f4e5819c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51928/0 (socket says 172.21.15.49:51928) 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e56ffd700 1 -- 172.21.15.49:0/949886572 learned_addr learned my addr 172.21.15.49:0/949886572 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e567fc700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e581007a0 0x7f4e5819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e56ffd700 1 -- 172.21.15.49:0/949886572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e581007a0 msgr2=0x7f4e5819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e56ffd700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e581007a0 0x7f4e5819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e56ffd700 1 -- 172.21.15.49:0/949886572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e48009d00 con 0x7f4e580ffdf0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e56ffd700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 0x7f4e5819c890 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000f80 tx=0x7f4e48007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e5c860700 1 -- 172.21.15.49:0/949886572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e4800e040 con 0x7f4e580ffdf0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e5c860700 1 -- 172.21.15.49:0/949886572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e48003c10 con 0x7f4e580ffdf0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e58196c50 con 0x7f4e580ffdf0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e5c860700 1 -- 172.21.15.49:0/949886572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e4801d6b0 con 0x7f4e580ffdf0 2024-05-15T16:23:39.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.029+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e58197140 con 0x7f4e580ffdf0 2024-05-15T16:23:39.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.031+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e581012d0 con 0x7f4e580ffdf0 2024-05-15T16:23:39.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.031+0000 7f4e5c860700 1 -- 172.21.15.49:0/949886572 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4e48004040 con 0x7f4e580ffdf0 2024-05-15T16:23:39.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.032+0000 7f4e5c860700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e4406e200 0x7f4e440706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:39.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.032+0000 7f4e5c860700 1 -- 172.21.15.49:0/949886572 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4e4808f880 con 0x7f4e580ffdf0 2024-05-15T16:23:39.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.032+0000 7f4e567fc700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e4406e200 0x7f4e440706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:39.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.032+0000 7f4e567fc700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e4406e200 0x7f4e440706c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4e40005950 tx=0x7f4e400058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:39.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.037+0000 7f4e5c860700 1 -- 172.21.15.49:0/949886572 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4e4805c710 con 0x7f4e580ffdf0 2024-05-15T16:23:39.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.264+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e5810ce50 con 0x7f4e4406e200 2024-05-15T16:23:39.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.266+0000 7f4e5c860700 1 -- 172.21.15.49:0/949886572 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4e5810ce50 con 0x7f4e4406e200 2024-05-15T16:23:39.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e4406e200 msgr2=0x7f4e440706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e4406e200 0x7f4e440706c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4e40005950 tx=0x7f4e400058e0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 msgr2=0x7f4e5819c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 0x7f4e5819c890 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000f80 tx=0x7f4e48007bd0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 shutdown_connections 2024-05-15T16:23:39.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4e4406e200 0x7f4e440706c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e580ffdf0 0x7f4e5819c890 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 --2- 172.21.15.49:0/949886572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e581007a0 0x7f4e5819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 >> 172.21.15.49:0/949886572 conn(0x7f4e580fb6a0 msgr2=0x7f4e58069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:39.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 shutdown_connections 2024-05-15T16:23:39.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.268+0000 7f4e5d862700 1 -- 172.21.15.49:0/949886572 wait complete. 2024-05-15T16:23:39.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.376+0000 7efe120c7700 1 -- 172.21.15.49:0/3663319764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 msgr2=0x7efe0c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.376+0000 7efe120c7700 1 --2- 172.21.15.49:0/3663319764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 0x7efe0c1047f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efe00009910 tx=0x7efe00009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:39.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.376+0000 7efe120c7700 1 -- 172.21.15.49:0/3663319764 shutdown_connections 2024-05-15T16:23:39.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.376+0000 7efe120c7700 1 --2- 172.21.15.49:0/3663319764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 0x7efe0c1047f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.376+0000 7efe120c7700 1 --2- 172.21.15.49:0/3663319764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0c0ffdf0 0x7efe0c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.376+0000 7efe120c7700 1 -- 172.21.15.49:0/3663319764 >> 172.21.15.49:0/3663319764 conn(0x7efe0c0fb6a0 msgr2=0x7efe0c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:39.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.376+0000 7efe120c7700 1 -- 172.21.15.49:0/3663319764 shutdown_connections 2024-05-15T16:23:39.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.376+0000 7efe120c7700 1 -- 172.21.15.49:0/3663319764 wait complete. 2024-05-15T16:23:39.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.376+0000 7efe120c7700 1 Processor -- start 2024-05-15T16:23:39.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe120c7700 1 -- start start 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe120c7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0c0ffdf0 0x7efe0c101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe120c7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 0x7efe0c101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe120c7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c102310 con 0x7efe0c1007a0 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe120c7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c102450 con 0x7efe0c0ffdf0 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe0affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 0x7efe0c101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe0affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 0x7efe0c101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51948/0 (socket says 172.21.15.49:51948) 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe0affd700 1 -- 172.21.15.49:0/1931455216 learned_addr learned my addr 172.21.15.49:0/1931455216 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe0b7fe700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0c0ffdf0 0x7efe0c101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe0affd700 1 -- 172.21.15.49:0/1931455216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0c0ffdf0 msgr2=0x7efe0c101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe0affd700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0c0ffdf0 0x7efe0c101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.377+0000 7efe0affd700 1 -- 172.21.15.49:0/1931455216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000095f0 con 0x7efe0c1007a0 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.378+0000 7efe0affd700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 0x7efe0c101dd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe0000bd70 tx=0x7efe0000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.378+0000 7efe08ff9700 1 -- 172.21.15.49:0/1931455216 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0001d070 con 0x7efe0c1007a0 2024-05-15T16:23:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.378+0000 7efe08ff9700 1 -- 172.21.15.49:0/1931455216 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe000056e0 con 0x7efe0c1007a0 2024-05-15T16:23:39.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.378+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c1026d0 con 0x7efe0c1007a0 2024-05-15T16:23:39.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.378+0000 7efe08ff9700 1 -- 172.21.15.49:0/1931455216 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0000fe50 con 0x7efe0c1007a0 2024-05-15T16:23:39.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.378+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c1a3050 con 0x7efe0c1007a0 2024-05-15T16:23:39.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.379+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c109200 con 0x7efe0c1007a0 2024-05-15T16:23:39.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.380+0000 7efe08ff9700 1 -- 172.21.15.49:0/1931455216 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efe0000f680 con 0x7efe0c1007a0 2024-05-15T16:23:39.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.380+0000 7efe08ff9700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdf406e200 0x7efdf40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:39.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.380+0000 7efe0b7fe700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdf406e200 0x7efdf40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:39.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.380+0000 7efe08ff9700 1 -- 172.21.15.49:0/1931455216 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efe0008f400 con 0x7efe0c1007a0 2024-05-15T16:23:39.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.381+0000 7efe0b7fe700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdf406e200 0x7efdf40706c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005950 tx=0x7efdfc00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:39.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.385+0000 7efe08ff9700 1 -- 172.21.15.49:0/1931455216 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efe00058760 con 0x7efe0c1007a0 2024-05-15T16:23:39.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.598+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efe0c10cd90 con 0x7efdf406e200 2024-05-15T16:23:39.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.609+0000 7efe08ff9700 1 -- 172.21.15.49:0/1931455216 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7efe0c10cd90 con 0x7efdf406e200 2024-05-15T16:23:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (26m) 4m ago 38m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (38m) 4m ago 38m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (37m) 4m ago 37m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (24m) 4m ago 38m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (24m) 4m ago 37m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (26m) 4m ago 38m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (20m) 4m ago 32m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (20m) 4m ago 32m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (19m) 4m ago 32m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (4m) 4m ago 32m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (29m) 4m ago 39m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (28m) 4m ago 37m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (25m) 4m ago 40m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (25m) 4m ago 37m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:23:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (27m) 4m ago 38m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:23:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (27m) 4m ago 37m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:23:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (23m) 4m ago 36m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:23:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (23m) 4m ago 36m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:23:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (22m) 4m ago 35m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:23:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (22m) 4m ago 35m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:23:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (21m) 4m ago 34m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:23:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (20m) 4m ago 34m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:23:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (27m) 4m ago 37m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.611+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdf406e200 msgr2=0x7efdf40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.611+0000 7efe120c7700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdf406e200 0x7efdf40706c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005950 tx=0x7efdfc00b410 comp rx=0 tx=0).stop 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.611+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 msgr2=0x7efe0c101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.611+0000 7efe120c7700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 0x7efe0c101dd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe0000bd70 tx=0x7efe0000bda0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.611+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 shutdown_connections 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.611+0000 7efe120c7700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efdf406e200 0x7efdf40706c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.611+0000 7efe120c7700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0c0ffdf0 0x7efe0c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.611+0000 7efe120c7700 1 --2- 172.21.15.49:0/1931455216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe0c1007a0 0x7efe0c101dd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.611+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 >> 172.21.15.49:0/1931455216 conn(0x7efe0c0fb6a0 msgr2=0x7efe0c0697c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.612+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 shutdown_connections 2024-05-15T16:23:39.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.612+0000 7efe120c7700 1 -- 172.21.15.49:0/1931455216 wait complete. 2024-05-15T16:23:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.719+0000 7ff29bbca700 1 -- 172.21.15.49:0/2699694335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff294100580 msgr2=0x7ff294104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.719+0000 7ff29bbca700 1 --2- 172.21.15.49:0/2699694335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff294100580 0x7ff294104a50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff284009910 tx=0x7ff284009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.719+0000 7ff29bbca700 1 -- 172.21.15.49:0/2699694335 shutdown_connections 2024-05-15T16:23:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.719+0000 7ff29bbca700 1 --2- 172.21.15.49:0/2699694335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff294100580 0x7ff294104a50 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.719+0000 7ff29bbca700 1 --2- 172.21.15.49:0/2699694335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2940ffc60 0x7ff294100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.719+0000 7ff29bbca700 1 -- 172.21.15.49:0/2699694335 >> 172.21.15.49:0/2699694335 conn(0x7ff2940fb700 msgr2=0x7ff2940fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.719+0000 7ff29bbca700 1 -- 172.21.15.49:0/2699694335 shutdown_connections 2024-05-15T16:23:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.719+0000 7ff29bbca700 1 -- 172.21.15.49:0/2699694335 wait complete. 2024-05-15T16:23:39.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.719+0000 7ff29bbca700 1 Processor -- start 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff29bbca700 1 -- start start 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff29bbca700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2940ffc60 0x7ff29419d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff29bbca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff294100580 0x7ff29419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff29bbca700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29419de20 con 0x7ff2940ffc60 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff29bbca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2941a1af0 con 0x7ff294100580 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff299966700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2940ffc60 0x7ff29419d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff299966700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2940ffc60 0x7ff29419d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51962/0 (socket says 172.21.15.49:51962) 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff299966700 1 -- 172.21.15.49:0/1104069508 learned_addr learned my addr 172.21.15.49:0/1104069508 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff299165700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff294100580 0x7ff29419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff299966700 1 -- 172.21.15.49:0/1104069508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff294100580 msgr2=0x7ff29419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:39.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff299966700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff294100580 0x7ff29419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:39.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff299966700 1 -- 172.21.15.49:0/1104069508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840095f0 con 0x7ff2940ffc60 2024-05-15T16:23:39.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.720+0000 7ff299966700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2940ffc60 0x7ff29419d200 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c8a0 tx=0x7ff29000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:39.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.721+0000 7ff28affd700 1 -- 172.21.15.49:0/1104069508 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2900078e0 con 0x7ff2940ffc60 2024-05-15T16:23:39.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.721+0000 7ff28affd700 1 -- 172.21.15.49:0/1104069508 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff29000f450 con 0x7ff2940ffc60 2024-05-15T16:23:39.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.721+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2941a1dd0 con 0x7ff2940ffc60 2024-05-15T16:23:39.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.721+0000 7ff28affd700 1 -- 172.21.15.49:0/1104069508 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff29000e5e0 con 0x7ff2940ffc60 2024-05-15T16:23:39.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.721+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2941a2320 con 0x7ff2940ffc60 2024-05-15T16:23:39.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.722+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff294069ab0 con 0x7ff2940ffc60 2024-05-15T16:23:39.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.722+0000 7ff28affd700 1 -- 172.21.15.49:0/1104069508 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff29000f5c0 con 0x7ff2940ffc60 2024-05-15T16:23:39.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.723+0000 7ff28affd700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff28006e200 0x7ff2800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:39.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.723+0000 7ff28affd700 1 -- 172.21.15.49:0/1104069508 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff29008e010 con 0x7ff2940ffc60 2024-05-15T16:23:39.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.723+0000 7ff299165700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff28006e200 0x7ff2800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:39.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.723+0000 7ff299165700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff28006e200 0x7ff2800706c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff29419e760 tx=0x7ff28401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:39.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:39.727+0000 7ff28affd700 1 -- 172.21.15.49:0/1104069508 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff290057370 con 0x7ff2940ffc60 2024-05-15T16:23:40.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.022+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff294114570 con 0x7ff2940ffc60 2024-05-15T16:23:40.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.023+0000 7ff28affd700 1 -- 172.21.15.49:0/1104069508 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff29005aa30 con 0x7ff2940ffc60 2024-05-15T16:23:40.024 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:23:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:23:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:23:40.025 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:23:40.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff28006e200 msgr2=0x7ff2800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff28006e200 0x7ff2800706c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff29419e760 tx=0x7ff28401a040 comp rx=0 tx=0).stop 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2940ffc60 msgr2=0x7ff29419d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2940ffc60 0x7ff29419d200 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c8a0 tx=0x7ff29000cbb0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 shutdown_connections 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff28006e200 0x7ff2800706c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2940ffc60 0x7ff29419d200 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 --2- 172.21.15.49:0/1104069508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff294100580 0x7ff29419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 >> 172.21.15.49:0/1104069508 conn(0x7ff2940fb700 msgr2=0x7ff29410e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 shutdown_connections 2024-05-15T16:23:40.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.025+0000 7ff29bbca700 1 -- 172.21.15.49:0/1104069508 wait complete. 2024-05-15T16:23:40.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.134+0000 7f371e769700 1 -- 172.21.15.49:0/1085815815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37181007a0 msgr2=0x7f37181047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.134+0000 7f371e769700 1 --2- 172.21.15.49:0/1085815815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37181007a0 0x7f37181047f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f370c009910 tx=0x7f370c009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.134+0000 7f371e769700 1 -- 172.21.15.49:0/1085815815 shutdown_connections 2024-05-15T16:23:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.134+0000 7f371e769700 1 --2- 172.21.15.49:0/1085815815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37181007a0 0x7f37181047f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.134+0000 7f371e769700 1 --2- 172.21.15.49:0/1085815815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37180ffdf0 0x7f37181001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.134+0000 7f371e769700 1 -- 172.21.15.49:0/1085815815 >> 172.21.15.49:0/1085815815 conn(0x7f37180fb6a0 msgr2=0x7f37180fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.134+0000 7f371e769700 1 -- 172.21.15.49:0/1085815815 shutdown_connections 2024-05-15T16:23:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.134+0000 7f371e769700 1 -- 172.21.15.49:0/1085815815 wait complete. 2024-05-15T16:23:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.134+0000 7f371e769700 1 Processor -- start 2024-05-15T16:23:40.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f371e769700 1 -- start start 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f371e769700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37180ffdf0 0x7f3718197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f371e769700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37181007a0 0x7f3718198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f371e769700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3718198660 con 0x7f37180ffdf0 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f371e769700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37181987d0 con 0x7f37181007a0 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f3717fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37180ffdf0 0x7f3718197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f3717fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37180ffdf0 0x7f3718197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51966/0 (socket says 172.21.15.49:51966) 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f3717fff700 1 -- 172.21.15.49:0/1010867003 learned_addr learned my addr 172.21.15.49:0/1010867003 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f37177fe700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37181007a0 0x7f3718198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f3717fff700 1 -- 172.21.15.49:0/1010867003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37181007a0 msgr2=0x7f3718198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f3717fff700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37181007a0 0x7f3718198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.135+0000 7f3717fff700 1 -- 172.21.15.49:0/1010867003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0095f0 con 0x7f37180ffdf0 2024-05-15T16:23:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.136+0000 7f3717fff700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37180ffdf0 0x7f3718197b50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f370800e920 tx=0x7f370800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.136+0000 7f37157fa700 1 -- 172.21.15.49:0/1010867003 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f370800c9d0 con 0x7f37180ffdf0 2024-05-15T16:23:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.136+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37181a0bc0 con 0x7f37180ffdf0 2024-05-15T16:23:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.136+0000 7f37157fa700 1 -- 172.21.15.49:0/1010867003 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3708010430 con 0x7f37180ffdf0 2024-05-15T16:23:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.136+0000 7f37157fa700 1 -- 172.21.15.49:0/1010867003 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3708018630 con 0x7f37180ffdf0 2024-05-15T16:23:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.136+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37181a1030 con 0x7f37180ffdf0 2024-05-15T16:23:40.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.137+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37181081f0 con 0x7f37180ffdf0 2024-05-15T16:23:40.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.137+0000 7f37157fa700 1 -- 172.21.15.49:0/1010867003 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f37080105a0 con 0x7f37180ffdf0 2024-05-15T16:23:40.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.138+0000 7f37157fa700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f370006e200 0x7f37000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:40.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.138+0000 7f37157fa700 1 -- 172.21.15.49:0/1010867003 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3708014070 con 0x7f37180ffdf0 2024-05-15T16:23:40.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.138+0000 7f37177fe700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f370006e200 0x7f37000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:40.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.138+0000 7f37177fe700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f370006e200 0x7f37000706c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f370c0059d0 tx=0x7f370c005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:40.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.143+0000 7f37157fa700 1 -- 172.21.15.49:0/1010867003 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3708053850 con 0x7f37180ffdf0 2024-05-15T16:23:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.396+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f37180697d0 con 0x7f37180ffdf0 2024-05-15T16:23:40.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:40 smithi049 ceph-mon[112142]: pgmap v800: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:40.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:40 smithi049 ceph-mon[112142]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:40.397 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:40 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1104069508' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:23:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.396+0000 7f37157fa700 1 -- 172.21.15.49:0/1010867003 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f3708057730 con 0x7f37180ffdf0 2024-05-15T16:23:40.398 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:23:40.398 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:23:40.398 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:23:40.398 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:23:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:23:40.398 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:23:40.398 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:23:40.399 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:23:40.400 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:23:40.401 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:23:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:23:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:23:40.401 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:23:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:23:40.401 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:23:40.401 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:23:40.401 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:23:40.401 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f370006e200 msgr2=0x7f37000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f370006e200 0x7f37000706c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f370c0059d0 tx=0x7f370c005940 comp rx=0 tx=0).stop 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37180ffdf0 msgr2=0x7f3718197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37180ffdf0 0x7f3718197b50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f370800e920 tx=0x7f370800ec30 comp rx=0 tx=0).stop 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 shutdown_connections 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f370006e200 0x7f37000706c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37180ffdf0 0x7f3718197b50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 --2- 172.21.15.49:0/1010867003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37181007a0 0x7f3718198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 >> 172.21.15.49:0/1010867003 conn(0x7f37180fb6a0 msgr2=0x7f37180fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 shutdown_connections 2024-05-15T16:23:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.398+0000 7f371e769700 1 -- 172.21.15.49:0/1010867003 wait complete. 2024-05-15T16:23:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:23:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:40 smithi144 ceph-mon[86635]: pgmap v800: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:40 smithi144 ceph-mon[86635]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:40 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1104069508' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:23:40.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.508+0000 7f70445ce700 1 -- 172.21.15.49:0/1700953078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c100770 msgr2=0x7f703c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.508+0000 7f70445ce700 1 --2- 172.21.15.49:0/1700953078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c100770 0x7f703c1047c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f702c009910 tx=0x7f702c009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.508+0000 7f70445ce700 1 -- 172.21.15.49:0/1700953078 shutdown_connections 2024-05-15T16:23:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.508+0000 7f70445ce700 1 --2- 172.21.15.49:0/1700953078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c100770 0x7f703c1047c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.508+0000 7f70445ce700 1 --2- 172.21.15.49:0/1700953078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c0ffdc0 0x7f703c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.508+0000 7f70445ce700 1 -- 172.21.15.49:0/1700953078 >> 172.21.15.49:0/1700953078 conn(0x7f703c076800 msgr2=0x7f703c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.508+0000 7f70445ce700 1 -- 172.21.15.49:0/1700953078 shutdown_connections 2024-05-15T16:23:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.508+0000 7f70445ce700 1 -- 172.21.15.49:0/1700953078 wait complete. 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f70445ce700 1 Processor -- start 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f70445ce700 1 -- start start 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f70445ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c0ffdc0 0x7f703c0fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f70445ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c100770 0x7f703c0fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f70445ce700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0fc760 con 0x7f703c0ffdc0 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f70445ce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0fc8d0 con 0x7f703c100770 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f704236a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c0ffdc0 0x7f703c0fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f704236a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c0ffdc0 0x7f703c0fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51978/0 (socket says 172.21.15.49:51978) 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f704236a700 1 -- 172.21.15.49:0/2066628685 learned_addr learned my addr 172.21.15.49:0/2066628685 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.509+0000 7f7041b69700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c100770 0x7f703c0fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f704236a700 1 -- 172.21.15.49:0/2066628685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c100770 msgr2=0x7f703c0fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f704236a700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c100770 0x7f703c0fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f704236a700 1 -- 172.21.15.49:0/2066628685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c0095f0 con 0x7f703c0ffdc0 2024-05-15T16:23:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f7041b69700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c100770 0x7f703c0fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:23:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f704236a700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c0ffdc0 0x7f703c0fbce0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f703800ada0 tx=0x7f703800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f70337fe700 1 -- 172.21.15.49:0/2066628685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f703800f070 con 0x7f703c0ffdc0 2024-05-15T16:23:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f70337fe700 1 -- 172.21.15.49:0/2066628685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7038007bf0 con 0x7f703c0ffdc0 2024-05-15T16:23:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f70337fe700 1 -- 172.21.15.49:0/2066628685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7038005690 con 0x7f703c0ffdc0 2024-05-15T16:23:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703c1a2d30 con 0x7f703c0ffdc0 2024-05-15T16:23:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.510+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703c1a30e0 con 0x7f703c0ffdc0 2024-05-15T16:23:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.511+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703c1012a0 con 0x7f703c0ffdc0 2024-05-15T16:23:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.512+0000 7f70337fe700 1 -- 172.21.15.49:0/2066628685 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7038005850 con 0x7f703c0ffdc0 2024-05-15T16:23:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.512+0000 7f70337fe700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f702806e200 0x7f70280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.512+0000 7f70337fe700 1 -- 172.21.15.49:0/2066628685 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f703808dae0 con 0x7f703c0ffdc0 2024-05-15T16:23:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.512+0000 7f7041b69700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f702806e200 0x7f70280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.513+0000 7f7041b69700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f702806e200 0x7f70280706c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f703c0fd790 tx=0x7f702c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.517+0000 7f70337fe700 1 -- 172.21.15.49:0/2066628685 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7038056e40 con 0x7f703c0ffdc0 2024-05-15T16:23:40.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.746+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f703c10cd60 con 0x7f702806e200 2024-05-15T16:23:40.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.748+0000 7f70337fe700 1 -- 172.21.15.49:0/2066628685 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f703c10cd60 con 0x7f702806e200 2024-05-15T16:23:40.750 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:23:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:23:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:23:40.751 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:23:40.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f702806e200 msgr2=0x7f70280706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f702806e200 0x7f70280706c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f703c0fd790 tx=0x7f702c01a040 comp rx=0 tx=0).stop 2024-05-15T16:23:40.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c0ffdc0 msgr2=0x7f703c0fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c0ffdc0 0x7f703c0fbce0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f703800ada0 tx=0x7f703800c330 comp rx=0 tx=0).stop 2024-05-15T16:23:40.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 shutdown_connections 2024-05-15T16:23:40.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f702806e200 0x7f70280706c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f703c0ffdc0 0x7f703c0fbce0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 --2- 172.21.15.49:0/2066628685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c100770 0x7f703c0fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 >> 172.21.15.49:0/2066628685 conn(0x7f703c076800 msgr2=0x7f703c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:40.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 shutdown_connections 2024-05-15T16:23:40.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.750+0000 7f70445ce700 1 -- 172.21.15.49:0/2066628685 wait complete. 2024-05-15T16:23:40.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.860+0000 7f6a149e6700 1 -- 172.21.15.49:0/4037038642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ff470 msgr2=0x7f6a100ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.860+0000 7f6a149e6700 1 --2- 172.21.15.49:0/4037038642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ff470 0x7f6a100ff850 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009910 tx=0x7f6a00009c20 comp rx=0 tx=0).stop 2024-05-15T16:23:40.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.860+0000 7f6a149e6700 1 -- 172.21.15.49:0/4037038642 shutdown_connections 2024-05-15T16:23:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.860+0000 7f6a149e6700 1 --2- 172.21.15.49:0/4037038642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a100ffe20 0x7f6a10105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.860+0000 7f6a149e6700 1 --2- 172.21.15.49:0/4037038642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ff470 0x7f6a100ff850 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.860+0000 7f6a149e6700 1 -- 172.21.15.49:0/4037038642 >> 172.21.15.49:0/4037038642 conn(0x7f6a10075fc0 msgr2=0x7f6a100fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.860+0000 7f6a149e6700 1 -- 172.21.15.49:0/4037038642 shutdown_connections 2024-05-15T16:23:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.860+0000 7f6a149e6700 1 -- 172.21.15.49:0/4037038642 wait complete. 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a149e6700 1 Processor -- start 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a149e6700 1 -- start start 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a149e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a100ff470 0x7f6a1019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a149e6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ffe20 0x7f6a1019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a149e6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1019d490 con 0x7f6a100ffe20 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a149e6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10196a70 con 0x7f6a100ff470 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a0dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ffe20 0x7f6a1019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a0dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ffe20 0x7f6a1019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52008/0 (socket says 172.21.15.49:52008) 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a0dd9b700 1 -- 172.21.15.49:0/3152784241 learned_addr learned my addr 172.21.15.49:0/3152784241 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.861+0000 7f6a0e59c700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a100ff470 0x7f6a1019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.862+0000 7f6a0dd9b700 1 -- 172.21.15.49:0/3152784241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a100ff470 msgr2=0x7f6a1019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.862+0000 7f6a0dd9b700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a100ff470 0x7f6a1019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.862+0000 7f6a0dd9b700 1 -- 172.21.15.49:0/3152784241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a000095f0 con 0x7f6a100ffe20 2024-05-15T16:23:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.862+0000 7f6a0dd9b700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ffe20 0x7f6a1019ce70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400d900 tx=0x7f6a0400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.862+0000 7f69ff7fe700 1 -- 172.21.15.49:0/3152784241 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a04009900 con 0x7f6a100ffe20 2024-05-15T16:23:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.862+0000 7f69ff7fe700 1 -- 172.21.15.49:0/3152784241 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a04010460 con 0x7f6a100ffe20 2024-05-15T16:23:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.862+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a10196d50 con 0x7f6a100ffe20 2024-05-15T16:23:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.862+0000 7f69ff7fe700 1 -- 172.21.15.49:0/3152784241 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a0400f5f0 con 0x7f6a100ffe20 2024-05-15T16:23:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.862+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a101972a0 con 0x7f6a100ffe20 2024-05-15T16:23:40.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.863+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a10109300 con 0x7f6a100ffe20 2024-05-15T16:23:40.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.864+0000 7f69ff7fe700 1 -- 172.21.15.49:0/3152784241 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6a040105d0 con 0x7f6a100ffe20 2024-05-15T16:23:40.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.864+0000 7f69ff7fe700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f69f806e200 0x7f69f80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:23:40.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.864+0000 7f69ff7fe700 1 -- 172.21.15.49:0/3152784241 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6a0408dc40 con 0x7f6a100ffe20 2024-05-15T16:23:40.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.864+0000 7f6a0e59c700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f69f806e200 0x7f69f80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:23:40.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.865+0000 7f6a0e59c700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f69f806e200 0x7f69f80706c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a00000c00 tx=0x7f6a0001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:23:40.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:40.871+0000 7f69ff7fe700 1 -- 172.21.15.49:0/3152784241 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6a0405aad0 con 0x7f6a100ffe20 2024-05-15T16:23:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.168+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6a10197f30 con 0x7f6a100ffe20 2024-05-15T16:23:41.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.168+0000 7f69ff7fe700 1 -- 172.21.15.49:0/3152784241 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f6a04017090 con 0x7f6a100ffe20 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:23:41.170 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:23:41.171 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:23:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.170+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f69f806e200 msgr2=0x7f69f80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.170+0000 7f6a149e6700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f69f806e200 0x7f69f80706c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a00000c00 tx=0x7f6a0001a040 comp rx=0 tx=0).stop 2024-05-15T16:23:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.170+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ffe20 msgr2=0x7f6a1019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:23:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.170+0000 7f6a149e6700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ffe20 0x7f6a1019ce70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400d900 tx=0x7f6a0400dc10 comp rx=0 tx=0).stop 2024-05-15T16:23:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.170+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 shutdown_connections 2024-05-15T16:23:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.170+0000 7f6a149e6700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f69f806e200 0x7f69f80706c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.170+0000 7f6a149e6700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a100ff470 0x7f6a1019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:41.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.171+0000 7f6a149e6700 1 --2- 172.21.15.49:0/3152784241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a100ffe20 0x7f6a1019ce70 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:23:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.171+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 >> 172.21.15.49:0/3152784241 conn(0x7f6a10075fc0 msgr2=0x7f6a10104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:23:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.171+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 shutdown_connections 2024-05-15T16:23:41.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:23:41.171+0000 7f6a149e6700 1 -- 172.21.15.49:0/3152784241 wait complete. 2024-05-15T16:23:41.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:41 smithi144 ceph-mon[86635]: from='client.35470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:41.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:41 smithi144 ceph-mon[86635]: from='client.35474 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:41.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:41 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1010867003' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:23:41.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:41 smithi049 ceph-mon[112142]: from='client.35470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:41.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:41 smithi049 ceph-mon[112142]: from='client.35474 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:41.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:41 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1010867003' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:23:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:42 smithi144 ceph-mon[86635]: pgmap v801: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:42 smithi144 ceph-mon[86635]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:42 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3152784241' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:23:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:42 smithi049 ceph-mon[112142]: pgmap v801: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:42 smithi049 ceph-mon[112142]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:23:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:42 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3152784241' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:23:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:44 smithi144 ceph-mon[86635]: pgmap v802: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:44 smithi049 ceph-mon[112142]: pgmap v802: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:46 smithi144 ceph-mon[86635]: pgmap v803: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:46 smithi049 ceph-mon[112142]: pgmap v803: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:48 smithi144 ceph-mon[86635]: pgmap v804: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:48.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:48 smithi049 ceph-mon[112142]: pgmap v804: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:50 smithi144 ceph-mon[86635]: pgmap v805: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:50.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:50 smithi049 ceph-mon[112142]: pgmap v805: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:23:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:23:52.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:52 smithi144 ceph-mon[86635]: pgmap v806: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:52 smithi049 ceph-mon[112142]: pgmap v806: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:53.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:53 smithi144 ceph-mon[86635]: pgmap v807: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:53.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:53 smithi049 ceph-mon[112142]: pgmap v807: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:56.008 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:55 smithi049 ceph-mon[112142]: pgmap v808: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:55 smithi144 ceph-mon[86635]: pgmap v808: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:57 smithi144 ceph-mon[86635]: pgmap v809: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:23:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:57 smithi049 ceph-mon[112142]: pgmap v809: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:23:59 smithi144 ceph-mon[86635]: pgmap v810: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:23:59 smithi049 ceph-mon[112142]: pgmap v810: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:01 smithi144 ceph-mon[86635]: pgmap v811: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:01 smithi049 ceph-mon[112142]: pgmap v811: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:04.000 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:03 smithi144 ceph-mon[86635]: pgmap v812: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:03 smithi049 ceph-mon[112142]: pgmap v812: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:06.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:05 smithi049 ceph-mon[112142]: pgmap v813: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:06.116 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:24:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:05 smithi144 ceph-mon[86635]: pgmap v813: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:24:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:07 smithi144 ceph-mon[86635]: pgmap v814: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:07 smithi049 ceph-mon[112142]: pgmap v814: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:09 smithi144 ceph-mon[86635]: pgmap v815: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:09 smithi049 ceph-mon[112142]: pgmap v815: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.293+0000 7f426776e700 1 -- 172.21.15.49:0/812980076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42601030f0 msgr2=0x7f42601034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.293+0000 7f426776e700 1 --2- 172.21.15.49:0/812980076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42601030f0 0x7f42601034d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f425c009910 tx=0x7f425c009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.293+0000 7f426776e700 1 -- 172.21.15.49:0/812980076 shutdown_connections 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.293+0000 7f426776e700 1 --2- 172.21.15.49:0/812980076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4260103aa0 0x7f4260107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.293+0000 7f426776e700 1 --2- 172.21.15.49:0/812980076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42601030f0 0x7f42601034d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.293+0000 7f426776e700 1 -- 172.21.15.49:0/812980076 >> 172.21.15.49:0/812980076 conn(0x7f42600fe9a0 msgr2=0x7f4260100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.293+0000 7f426776e700 1 -- 172.21.15.49:0/812980076 shutdown_connections 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.293+0000 7f426776e700 1 -- 172.21.15.49:0/812980076 wait complete. 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f426776e700 1 Processor -- start 2024-05-15T16:24:11.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f426776e700 1 -- start start 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f426776e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42601030f0 0x7f4260198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f426776e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260103aa0 0x7f42601993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f426776e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260199a90 con 0x7f4260103aa0 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f426776e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426019d6a0 con 0x7f42601030f0 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f4264d09700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260103aa0 0x7f42601993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f4264d09700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260103aa0 0x7f42601993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42766/0 (socket says 172.21.15.49:42766) 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f4264d09700 1 -- 172.21.15.49:0/2653605370 learned_addr learned my addr 172.21.15.49:0/2653605370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.294+0000 7f426550a700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42601030f0 0x7f4260198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.295+0000 7f4264d09700 1 -- 172.21.15.49:0/2653605370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42601030f0 msgr2=0x7f4260198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.295+0000 7f4264d09700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42601030f0 0x7f4260198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.295+0000 7f4264d09700 1 -- 172.21.15.49:0/2653605370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0095f0 con 0x7f4260103aa0 2024-05-15T16:24:11.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.295+0000 7f4264d09700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260103aa0 0x7f42601993b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f425000d900 tx=0x7f425000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:11.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.295+0000 7f42567fc700 1 -- 172.21.15.49:0/2653605370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4250009900 con 0x7f4260103aa0 2024-05-15T16:24:11.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.295+0000 7f42567fc700 1 -- 172.21.15.49:0/2653605370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4250010460 con 0x7f4260103aa0 2024-05-15T16:24:11.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.295+0000 7f42567fc700 1 -- 172.21.15.49:0/2653605370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f425000f5f0 con 0x7f4260103aa0 2024-05-15T16:24:11.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.295+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426019d980 con 0x7f4260103aa0 2024-05-15T16:24:11.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.295+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426019ded0 con 0x7f4260103aa0 2024-05-15T16:24:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.296+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426010b490 con 0x7f4260103aa0 2024-05-15T16:24:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.297+0000 7f42567fc700 1 -- 172.21.15.49:0/2653605370 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f42500105d0 con 0x7f4260103aa0 2024-05-15T16:24:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.297+0000 7f42567fc700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f424c06e200 0x7f424c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.297+0000 7f426550a700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f424c06e200 0x7f424c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.297+0000 7f42567fc700 1 -- 172.21.15.49:0/2653605370 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f425008dd60 con 0x7f4260103aa0 2024-05-15T16:24:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.298+0000 7f426550a700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f424c06e200 0x7f424c0706c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f425c000c00 tx=0x7f425c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.302+0000 7f42567fc700 1 -- 172.21.15.49:0/2653605370 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4250057040 con 0x7f4260103aa0 2024-05-15T16:24:11.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.530+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f426019a0c0 con 0x7f424c06e200 2024-05-15T16:24:11.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.531+0000 7f42567fc700 1 -- 172.21.15.49:0/2653605370 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f426019a0c0 con 0x7f424c06e200 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f424c06e200 msgr2=0x7f424c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f424c06e200 0x7f424c0706c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f425c000c00 tx=0x7f425c01a040 comp rx=0 tx=0).stop 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260103aa0 msgr2=0x7f42601993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260103aa0 0x7f42601993b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f425000d900 tx=0x7f425000dc10 comp rx=0 tx=0).stop 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 shutdown_connections 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f424c06e200 0x7f424c0706c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42601030f0 0x7f4260198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 --2- 172.21.15.49:0/2653605370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4260103aa0 0x7f42601993b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 >> 172.21.15.49:0/2653605370 conn(0x7f42600fe9a0 msgr2=0x7f42600ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 shutdown_connections 2024-05-15T16:24:11.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.533+0000 7f426776e700 1 -- 172.21.15.49:0/2653605370 wait complete. 2024-05-15T16:24:11.545 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:24:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.643+0000 7fa89d3df700 1 -- 172.21.15.49:0/1618210170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898100ee0 msgr2=0x7fa8981012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.643+0000 7fa89d3df700 1 --2- 172.21.15.49:0/1618210170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898100ee0 0x7fa8981012c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa88800b0a0 tx=0x7fa88800b3b0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.643+0000 7fa89d3df700 1 -- 172.21.15.49:0/1618210170 shutdown_connections 2024-05-15T16:24:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.643+0000 7fa89d3df700 1 --2- 172.21.15.49:0/1618210170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa898101890 0x7fa8981058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.643+0000 7fa89d3df700 1 --2- 172.21.15.49:0/1618210170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898100ee0 0x7fa8981012c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.643+0000 7fa89d3df700 1 -- 172.21.15.49:0/1618210170 >> 172.21.15.49:0/1618210170 conn(0x7fa8980fc790 msgr2=0x7fa8980febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.643+0000 7fa89d3df700 1 -- 172.21.15.49:0/1618210170 shutdown_connections 2024-05-15T16:24:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.643+0000 7fa89d3df700 1 -- 172.21.15.49:0/1618210170 wait complete. 2024-05-15T16:24:11.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa89d3df700 1 Processor -- start 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa89d3df700 1 -- start start 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa89d3df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa898100ee0 0x7fa8980732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa89d3df700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898101890 0x7fa8980737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa89d3df700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa898073d20 con 0x7fa898101890 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa89d3df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa898073e60 con 0x7fa898100ee0 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa8967fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898101890 0x7fa8980737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa8967fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898101890 0x7fa8980737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42774/0 (socket says 172.21.15.49:42774) 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa8967fc700 1 -- 172.21.15.49:0/2847727111 learned_addr learned my addr 172.21.15.49:0/2847727111 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.644+0000 7fa896ffd700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa898100ee0 0x7fa8980732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:11.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.645+0000 7fa8967fc700 1 -- 172.21.15.49:0/2847727111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa898100ee0 msgr2=0x7fa8980732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:11.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.645+0000 7fa8967fc700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa898100ee0 0x7fa8980732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.645+0000 7fa8967fc700 1 -- 172.21.15.49:0/2847727111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa888009d00 con 0x7fa898101890 2024-05-15T16:24:11.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.645+0000 7fa8967fc700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898101890 0x7fa8980737e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e8f0 tx=0x7fa88000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:11.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.645+0000 7fa88ffff700 1 -- 172.21.15.49:0/2847727111 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa880009790 con 0x7fa898101890 2024-05-15T16:24:11.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.645+0000 7fa88ffff700 1 -- 172.21.15.49:0/2847727111 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa88000c810 con 0x7fa898101890 2024-05-15T16:24:11.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.645+0000 7fa88ffff700 1 -- 172.21.15.49:0/2847727111 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa880005580 con 0x7fa898101890 2024-05-15T16:24:11.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.645+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8981a2dd0 con 0x7fa898101890 2024-05-15T16:24:11.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.645+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8981a3130 con 0x7fa898101890 2024-05-15T16:24:11.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.646+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa898109280 con 0x7fa898101890 2024-05-15T16:24:11.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.647+0000 7fa88ffff700 1 -- 172.21.15.49:0/2847727111 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa88000c980 con 0x7fa898101890 2024-05-15T16:24:11.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.647+0000 7fa88ffff700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa88406e200 0x7fa8840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:11.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.647+0000 7fa896ffd700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa88406e200 0x7fa8840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:11.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.648+0000 7fa88ffff700 1 -- 172.21.15.49:0/2847727111 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa880014070 con 0x7fa898101890 2024-05-15T16:24:11.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.648+0000 7fa896ffd700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa88406e200 0x7fa8840706c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa888000f80 tx=0x7fa8880060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:11.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.652+0000 7fa88ffff700 1 -- 172.21.15.49:0/2847727111 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa8800576e0 con 0x7fa898101890 2024-05-15T16:24:11.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.885+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa89810deb0 con 0x7fa88406e200 2024-05-15T16:24:11.886 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:11 smithi049 ceph-mon[112142]: pgmap v816: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:11.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.886+0000 7fa88ffff700 1 -- 172.21.15.49:0/2847727111 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa89810deb0 con 0x7fa88406e200 2024-05-15T16:24:11.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa88406e200 msgr2=0x7fa8840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:11.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa88406e200 0x7fa8840706c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa888000f80 tx=0x7fa8880060c0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898101890 msgr2=0x7fa8980737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:11.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898101890 0x7fa8980737e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e8f0 tx=0x7fa88000ec00 comp rx=0 tx=0).stop 2024-05-15T16:24:11.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 shutdown_connections 2024-05-15T16:24:11.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa88406e200 0x7fa8840706c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa898100ee0 0x7fa8980732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 --2- 172.21.15.49:0/2847727111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa898101890 0x7fa8980737e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 >> 172.21.15.49:0/2847727111 conn(0x7fa8980fc790 msgr2=0x7fa8980fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:11.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 shutdown_connections 2024-05-15T16:24:11.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.888+0000 7fa89d3df700 1 -- 172.21.15.49:0/2847727111 wait complete. 2024-05-15T16:24:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.997+0000 7f7499d50700 1 -- 172.21.15.49:0/251998785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494101e40 msgr2=0x7f7494102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.997+0000 7f7499d50700 1 --2- 172.21.15.49:0/251998785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494101e40 0x7f7494102220 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7484009910 tx=0x7f7484009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.997+0000 7f7499d50700 1 -- 172.21.15.49:0/251998785 shutdown_connections 2024-05-15T16:24:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.997+0000 7f7499d50700 1 --2- 172.21.15.49:0/251998785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7494102760 0x7f749410ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.997+0000 7f7499d50700 1 --2- 172.21.15.49:0/251998785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494101e40 0x7f7494102220 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.997+0000 7f7499d50700 1 -- 172.21.15.49:0/251998785 >> 172.21.15.49:0/251998785 conn(0x7f74940fb6b0 msgr2=0x7f74940fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.997+0000 7f7499d50700 1 -- 172.21.15.49:0/251998785 shutdown_connections 2024-05-15T16:24:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.997+0000 7f7499d50700 1 -- 172.21.15.49:0/251998785 wait complete. 2024-05-15T16:24:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f7499d50700 1 Processor -- start 2024-05-15T16:24:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f7499d50700 1 -- start start 2024-05-15T16:24:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f7499d50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7494101e40 0x7f7494198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f7499d50700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494102760 0x7f7494199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f7499d50700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74941999e0 con 0x7f7494102760 2024-05-15T16:24:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f7499d50700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749419d6b0 con 0x7f7494101e40 2024-05-15T16:24:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f7492ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494102760 0x7f7494199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f7492ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494102760 0x7f7494199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42796/0 (socket says 172.21.15.49:42796) 2024-05-15T16:24:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f7492ffd700 1 -- 172.21.15.49:0/1763134251 learned_addr learned my addr 172.21.15.49:0/1763134251 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.998+0000 7f74937fe700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7494101e40 0x7f7494198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.999+0000 7f7492ffd700 1 -- 172.21.15.49:0/1763134251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7494101e40 msgr2=0x7f7494198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.999+0000 7f7492ffd700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7494101e40 0x7f7494198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.999+0000 7f7492ffd700 1 -- 172.21.15.49:0/1763134251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840095f0 con 0x7f7494102760 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.999+0000 7f7492ffd700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494102760 0x7f7494199300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f748800d6d0 tx=0x7f748800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.999+0000 7f7490ff9700 1 -- 172.21.15.49:0/1763134251 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748800f860 con 0x7f7494102760 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.999+0000 7f7490ff9700 1 -- 172.21.15.49:0/1763134251 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f748800fe80 con 0x7f7494102760 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.999+0000 7f7490ff9700 1 -- 172.21.15.49:0/1763134251 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7488010ea0 con 0x7f7494102760 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.999+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749419d990 con 0x7f7494102760 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:11.999+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749419dee0 con 0x7f7494102760 2024-05-15T16:24:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.000+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74941097b0 con 0x7f7494102760 2024-05-15T16:24:12.003 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:11 smithi144 ceph-mon[86635]: pgmap v816: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.001+0000 7f7490ff9700 1 -- 172.21.15.49:0/1763134251 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7488010460 con 0x7f7494102760 2024-05-15T16:24:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.001+0000 7f7490ff9700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f747c06e200 0x7f747c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.001+0000 7f7490ff9700 1 -- 172.21.15.49:0/1763134251 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f748808ddd0 con 0x7f7494102760 2024-05-15T16:24:12.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.001+0000 7f74937fe700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f747c06e200 0x7f747c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:12.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.002+0000 7f74937fe700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f747c06e200 0x7f747c0706c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7484000c00 tx=0x7f748401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:12.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.006+0000 7f7490ff9700 1 -- 172.21.15.49:0/1763134251 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7488057130 con 0x7f7494102760 2024-05-15T16:24:12.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.222+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f749419a140 con 0x7f747c06e200 2024-05-15T16:24:12.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.233+0000 7f7490ff9700 1 -- 172.21.15.49:0/1763134251 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f749419a140 con 0x7f747c06e200 2024-05-15T16:24:12.234 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:24:12.234 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (27m) 5m ago 38m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:24:12.234 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (39m) 5m ago 39m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:24:12.234 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (38m) 5m ago 38m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:24:12.234 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (24m) 5m ago 39m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (24m) 5m ago 38m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (27m) 5m ago 38m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (21m) 5m ago 33m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (20m) 5m ago 33m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (20m) 5m ago 33m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (5m) 5m ago 33m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (29m) 5m ago 40m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (28m) 5m ago 37m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (26m) 5m ago 40m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (25m) 5m ago 37m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (28m) 5m ago 39m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (28m) 5m ago 37m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (24m) 5m ago 37m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (23m) 5m ago 36m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:24:12.235 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (23m) 5m ago 36m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:24:12.236 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (22m) 5m ago 35m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:24:12.236 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (22m) 5m ago 35m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:24:12.236 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (21m) 5m ago 34m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:24:12.236 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (27m) 5m ago 38m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:24:12.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f747c06e200 msgr2=0x7f747c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f747c06e200 0x7f747c0706c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7484000c00 tx=0x7f748401a040 comp rx=0 tx=0).stop 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494102760 msgr2=0x7f7494199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494102760 0x7f7494199300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f748800d6d0 tx=0x7f748800d9e0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 shutdown_connections 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f747c06e200 0x7f747c0706c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7494101e40 0x7f7494198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 --2- 172.21.15.49:0/1763134251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7494102760 0x7f7494199300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 >> 172.21.15.49:0/1763134251 conn(0x7f74940fb6b0 msgr2=0x7f749410a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 shutdown_connections 2024-05-15T16:24:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.235+0000 7f7499d50700 1 -- 172.21.15.49:0/1763134251 wait complete. 2024-05-15T16:24:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.343+0000 7f6b8ef57700 1 -- 172.21.15.49:0/1740560727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 msgr2=0x7f6b881012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.343+0000 7f6b8ef57700 1 --2- 172.21.15.49:0/1740560727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 0x7f6b881012c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009910 tx=0x7f6b7c009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.343+0000 7f6b8ef57700 1 -- 172.21.15.49:0/1740560727 shutdown_connections 2024-05-15T16:24:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.343+0000 7f6b8ef57700 1 --2- 172.21.15.49:0/1740560727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b88101890 0x7f6b881058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.343+0000 7f6b8ef57700 1 --2- 172.21.15.49:0/1740560727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 0x7f6b881012c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.343+0000 7f6b8ef57700 1 -- 172.21.15.49:0/1740560727 >> 172.21.15.49:0/1740560727 conn(0x7f6b880fc790 msgr2=0x7f6b880febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.343+0000 7f6b8ef57700 1 -- 172.21.15.49:0/1740560727 shutdown_connections 2024-05-15T16:24:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.343+0000 7f6b8ef57700 1 -- 172.21.15.49:0/1740560727 wait complete. 2024-05-15T16:24:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.343+0000 7f6b8ef57700 1 Processor -- start 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ef57700 1 -- start start 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ef57700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 0x7f6b8819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ef57700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b88101890 0x7f6b8819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ef57700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8819d460 con 0x7f6b88100ee0 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ef57700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b88196a90 con 0x7f6b88101890 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ccf3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 0x7f6b8819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ccf3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 0x7f6b8819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42816/0 (socket says 172.21.15.49:42816) 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ccf3700 1 -- 172.21.15.49:0/2510947291 learned_addr learned my addr 172.21.15.49:0/2510947291 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b87fff700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b88101890 0x7f6b8819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ccf3700 1 -- 172.21.15.49:0/2510947291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b88101890 msgr2=0x7f6b8819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ccf3700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b88101890 0x7f6b8819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ccf3700 1 -- 172.21.15.49:0/2510947291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c0095f0 con 0x7f6b88100ee0 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.344+0000 7f6b8ccf3700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 0x7f6b8819c900 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00bd70 tx=0x7f6b7c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.345+0000 7f6b85ffb700 1 -- 172.21.15.49:0/2510947291 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7c01d070 con 0x7f6b88100ee0 2024-05-15T16:24:12.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.345+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b88196d10 con 0x7f6b88100ee0 2024-05-15T16:24:12.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.345+0000 7f6b85ffb700 1 -- 172.21.15.49:0/2510947291 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b7c00f460 con 0x7f6b88100ee0 2024-05-15T16:24:12.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.345+0000 7f6b85ffb700 1 -- 172.21.15.49:0/2510947291 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7c017630 con 0x7f6b88100ee0 2024-05-15T16:24:12.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.345+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b88197200 con 0x7f6b88100ee0 2024-05-15T16:24:12.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.346+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b88109280 con 0x7f6b88100ee0 2024-05-15T16:24:12.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.346+0000 7f6b85ffb700 1 -- 172.21.15.49:0/2510947291 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6b7c00f5d0 con 0x7f6b88100ee0 2024-05-15T16:24:12.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.347+0000 7f6b85ffb700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b7806e200 0x7f6b780706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:12.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.347+0000 7f6b85ffb700 1 -- 172.21.15.49:0/2510947291 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6b7c08f6d0 con 0x7f6b88100ee0 2024-05-15T16:24:12.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.347+0000 7f6b87fff700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b7806e200 0x7f6b780706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:12.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.347+0000 7f6b87fff700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b7806e200 0x7f6b780706c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6b74005fd0 tx=0x7f6b7400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:12.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.352+0000 7f6b85ffb700 1 -- 172.21.15.49:0/2510947291 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6b7c05c4e0 con 0x7f6b88100ee0 2024-05-15T16:24:12.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.645+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b88197f00 con 0x7f6b88100ee0 2024-05-15T16:24:12.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.646+0000 7f6b85ffb700 1 -- 172.21.15.49:0/2510947291 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6b7c05c070 con 0x7f6b88100ee0 2024-05-15T16:24:12.647 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:24:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:24:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:24:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:24:12.648 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:24:12.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b7806e200 msgr2=0x7f6b780706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:12.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b7806e200 0x7f6b780706c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6b74005fd0 tx=0x7f6b7400a560 comp rx=0 tx=0).stop 2024-05-15T16:24:12.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 msgr2=0x7f6b8819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:12.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 0x7f6b8819c900 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00bd70 tx=0x7f6b7c00be50 comp rx=0 tx=0).stop 2024-05-15T16:24:12.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 shutdown_connections 2024-05-15T16:24:12.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b7806e200 0x7f6b780706c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b88100ee0 0x7f6b8819c900 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 --2- 172.21.15.49:0/2510947291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b88101890 0x7f6b8819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 >> 172.21.15.49:0/2510947291 conn(0x7f6b880fc790 msgr2=0x7f6b880fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:12.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 shutdown_connections 2024-05-15T16:24:12.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.649+0000 7f6b8ef57700 1 -- 172.21.15.49:0/2510947291 wait complete. 2024-05-15T16:24:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.758+0000 7f4d21968700 1 -- 172.21.15.49:0/2697154136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 msgr2=0x7f4d1c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.758+0000 7f4d21968700 1 --2- 172.21.15.49:0/2697154136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 0x7f4d1c1047f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009910 tx=0x7f4d10009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.758+0000 7f4d21968700 1 -- 172.21.15.49:0/2697154136 shutdown_connections 2024-05-15T16:24:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.758+0000 7f4d21968700 1 --2- 172.21.15.49:0/2697154136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 0x7f4d1c1047f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.758+0000 7f4d21968700 1 --2- 172.21.15.49:0/2697154136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ffdf0 0x7f4d1c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.759+0000 7f4d21968700 1 -- 172.21.15.49:0/2697154136 >> 172.21.15.49:0/2697154136 conn(0x7f4d1c0fb6a0 msgr2=0x7f4d1c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.759+0000 7f4d21968700 1 -- 172.21.15.49:0/2697154136 shutdown_connections 2024-05-15T16:24:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.759+0000 7f4d21968700 1 -- 172.21.15.49:0/2697154136 wait complete. 2024-05-15T16:24:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.759+0000 7f4d21968700 1 Processor -- start 2024-05-15T16:24:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.759+0000 7f4d21968700 1 -- start start 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d21968700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ffdf0 0x7f4d1c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d21968700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 0x7f4d1c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d21968700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1986c0 con 0x7f4d1c1007a0 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d21968700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c198800 con 0x7f4d1c0ffdf0 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d1a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 0x7f4d1c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d1a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 0x7f4d1c1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42836/0 (socket says 172.21.15.49:42836) 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d1a7fc700 1 -- 172.21.15.49:0/876848787 learned_addr learned my addr 172.21.15.49:0/876848787 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d1affd700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ffdf0 0x7f4d1c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d1a7fc700 1 -- 172.21.15.49:0/876848787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ffdf0 msgr2=0x7f4d1c197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d1a7fc700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ffdf0 0x7f4d1c197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d1a7fc700 1 -- 172.21.15.49:0/876848787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100095f0 con 0x7f4d1c1007a0 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d1a7fc700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 0x7f4d1c1980f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000bd70 tx=0x7f4d1000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.760+0000 7f4d20966700 1 -- 172.21.15.49:0/876848787 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1001d070 con 0x7f4d1c1007a0 2024-05-15T16:24:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.761+0000 7f4d20966700 1 -- 172.21.15.49:0/876848787 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d1000f460 con 0x7f4d1c1007a0 2024-05-15T16:24:12.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.761+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c1a0c20 con 0x7f4d1c1007a0 2024-05-15T16:24:12.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.761+0000 7f4d20966700 1 -- 172.21.15.49:0/876848787 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d10017630 con 0x7f4d1c1007a0 2024-05-15T16:24:12.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.761+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c1a0fd0 con 0x7f4d1c1007a0 2024-05-15T16:24:12.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.762+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c1081f0 con 0x7f4d1c1007a0 2024-05-15T16:24:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.762+0000 7f4d20966700 1 -- 172.21.15.49:0/876848787 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4d1000f5d0 con 0x7f4d1c1007a0 2024-05-15T16:24:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.763+0000 7f4d20966700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d0406e200 0x7f4d040706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.763+0000 7f4d20966700 1 -- 172.21.15.49:0/876848787 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4d1008f580 con 0x7f4d1c1007a0 2024-05-15T16:24:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.763+0000 7f4d1affd700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d0406e200 0x7f4d040706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.763+0000 7f4d1affd700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d0406e200 0x7f4d040706c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c005fd0 tx=0x7f4d0c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:12.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:12.767+0000 7f4d20966700 1 -- 172.21.15.49:0/876848787 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4d100588e0 con 0x7f4d1c1007a0 2024-05-15T16:24:12.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:12 smithi049 ceph-mon[112142]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:12.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:12 smithi049 ceph-mon[112142]: from='client.35498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:12.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:12 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2510947291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:24:13.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.020+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d1c1012d0 con 0x7f4d1c1007a0 2024-05-15T16:24:13.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.021+0000 7f4d20966700 1 -- 172.21.15.49:0/876848787 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f4d10005740 con 0x7f4d1c1007a0 2024-05-15T16:24:13.022 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:24:13.022 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:24:13.022 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:24:13.022 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:24:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:24:13.022 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:24:13.022 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:24:13.023 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:24:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:24:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:24:13.025 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:24:13.025 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:24:13.025 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:24:13.025 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d0406e200 msgr2=0x7f4d040706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d0406e200 0x7f4d040706c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c005fd0 tx=0x7f4d0c00a560 comp rx=0 tx=0).stop 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 msgr2=0x7f4d1c1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 0x7f4d1c1980f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000bd70 tx=0x7f4d1000be50 comp rx=0 tx=0).stop 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 shutdown_connections 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4d0406e200 0x7f4d040706c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ffdf0 0x7f4d1c197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 --2- 172.21.15.49:0/876848787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d1c1007a0 0x7f4d1c1980f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 >> 172.21.15.49:0/876848787 conn(0x7f4d1c0fb6a0 msgr2=0x7f4d1c0fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 shutdown_connections 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.023+0000 7f4d21968700 1 -- 172.21.15.49:0/876848787 wait complete. 2024-05-15T16:24:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:24:13.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.133+0000 7fb5fd006700 1 -- 172.21.15.49:0/3759247770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 msgr2=0x7fb5f81058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.133+0000 7fb5fd006700 1 --2- 172.21.15.49:0/3759247770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 0x7fb5f81058e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009910 tx=0x7fb5ec009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:13.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.133+0000 7fb5fd006700 1 -- 172.21.15.49:0/3759247770 shutdown_connections 2024-05-15T16:24:13.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.133+0000 7fb5fd006700 1 --2- 172.21.15.49:0/3759247770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 0x7fb5f81058e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.133+0000 7fb5fd006700 1 --2- 172.21.15.49:0/3759247770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5f8100ee0 0x7fb5f81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.133+0000 7fb5fd006700 1 -- 172.21.15.49:0/3759247770 >> 172.21.15.49:0/3759247770 conn(0x7fb5f80fc790 msgr2=0x7fb5f80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:13.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.133+0000 7fb5fd006700 1 -- 172.21.15.49:0/3759247770 shutdown_connections 2024-05-15T16:24:13.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.134+0000 7fb5fd006700 1 -- 172.21.15.49:0/3759247770 wait complete. 2024-05-15T16:24:13.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.134+0000 7fb5fd006700 1 Processor -- start 2024-05-15T16:24:13.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.134+0000 7fb5fd006700 1 -- start start 2024-05-15T16:24:13.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5fd006700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5f8100ee0 0x7fb5f80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:13.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5fd006700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 0x7fb5f80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5fd006700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8073d20 con 0x7fb5f8101890 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5fd006700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8073e60 con 0x7fb5f8100ee0 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5f659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 0x7fb5f80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5f659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 0x7fb5f80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42868/0 (socket says 172.21.15.49:42868) 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5f659c700 1 -- 172.21.15.49:0/322448154 learned_addr learned my addr 172.21.15.49:0/322448154 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5f6d9d700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5f8100ee0 0x7fb5f80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5f659c700 1 -- 172.21.15.49:0/322448154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5f8100ee0 msgr2=0x7fb5f80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5f659c700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5f8100ee0 0x7fb5f80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5f659c700 1 -- 172.21.15.49:0/322448154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0095f0 con 0x7fb5f8101890 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5f6d9d700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5f8100ee0 0x7fb5f80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5f659c700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 0x7fb5f80737e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00bd70 tx=0x7fb5ec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:13.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.135+0000 7fb5dffff700 1 -- 172.21.15.49:0/322448154 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5ec01d070 con 0x7fb5f8101890 2024-05-15T16:24:13.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.136+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f80740e0 con 0x7fb5f8101890 2024-05-15T16:24:13.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.136+0000 7fb5dffff700 1 -- 172.21.15.49:0/322448154 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5ec00f460 con 0x7fb5f8101890 2024-05-15T16:24:13.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.136+0000 7fb5dffff700 1 -- 172.21.15.49:0/322448154 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5ec017610 con 0x7fb5f8101890 2024-05-15T16:24:13.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.136+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f81a3020 con 0x7fb5f8101890 2024-05-15T16:24:13.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.137+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f81092e0 con 0x7fb5f8101890 2024-05-15T16:24:13.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.137+0000 7fb5dffff700 1 -- 172.21.15.49:0/322448154 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb5ec00f5d0 con 0x7fb5f8101890 2024-05-15T16:24:13.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.138+0000 7fb5dffff700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb5e006e200 0x7fb5e00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:13.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.138+0000 7fb5f6d9d700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb5e006e200 0x7fb5e00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:13.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.138+0000 7fb5dffff700 1 -- 172.21.15.49:0/322448154 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb5ec08f6b0 con 0x7fb5f8101890 2024-05-15T16:24:13.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.138+0000 7fb5f6d9d700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb5e006e200 0x7fb5e00706c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8005fd0 tx=0x7fb5e8005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:13.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.142+0000 7fb5dffff700 1 -- 172.21.15.49:0/322448154 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb5ec058990 con 0x7fb5f8101890 2024-05-15T16:24:13.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:12 smithi144 ceph-mon[86635]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:13.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:12 smithi144 ceph-mon[86635]: from='client.35498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:13.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:12 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2510947291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:24:13.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.372+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5f810dea0 con 0x7fb5e006e200 2024-05-15T16:24:13.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.373+0000 7fb5dffff700 1 -- 172.21.15.49:0/322448154 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb5f810dea0 con 0x7fb5e006e200 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:24:13.375 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:24:13.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb5e006e200 msgr2=0x7fb5e00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb5e006e200 0x7fb5e00706c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8005fd0 tx=0x7fb5e8005dc0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 msgr2=0x7fb5f80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 0x7fb5f80737e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00bd70 tx=0x7fb5ec00be50 comp rx=0 tx=0).stop 2024-05-15T16:24:13.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 shutdown_connections 2024-05-15T16:24:13.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb5e006e200 0x7fb5e00706c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5f8100ee0 0x7fb5f80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 --2- 172.21.15.49:0/322448154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5f8101890 0x7fb5f80737e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 >> 172.21.15.49:0/322448154 conn(0x7fb5f80fc790 msgr2=0x7fb5f80fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:13.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 shutdown_connections 2024-05-15T16:24:13.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.376+0000 7fb5fd006700 1 -- 172.21.15.49:0/322448154 wait complete. 2024-05-15T16:24:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 -- 172.21.15.49:0/2970582125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958101e50 msgr2=0x7f2958102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 --2- 172.21.15.49:0/2970582125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958101e50 0x7f2958102230 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2954009910 tx=0x7f2954009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 -- 172.21.15.49:0/2970582125 shutdown_connections 2024-05-15T16:24:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 --2- 172.21.15.49:0/2970582125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2958102770 0x7f295810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 --2- 172.21.15.49:0/2970582125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958101e50 0x7f2958102230 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 -- 172.21.15.49:0/2970582125 >> 172.21.15.49:0/2970582125 conn(0x7f29580fb6c0 msgr2=0x7f29580fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:13.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 -- 172.21.15.49:0/2970582125 shutdown_connections 2024-05-15T16:24:13.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 -- 172.21.15.49:0/2970582125 wait complete. 2024-05-15T16:24:13.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 Processor -- start 2024-05-15T16:24:13.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.485+0000 7f296047c700 1 -- start start 2024-05-15T16:24:13.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f296047c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2958101e50 0x7f29580ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:13.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f296047c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958102770 0x7f29580ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:13.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f296047c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2958100530 con 0x7f2958102770 2024-05-15T16:24:13.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f296047c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2958100670 con 0x7f2958101e50 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f295da17700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958102770 0x7f29580ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f295da17700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958102770 0x7f29580ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42890/0 (socket says 172.21.15.49:42890) 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f295da17700 1 -- 172.21.15.49:0/239182976 learned_addr learned my addr 172.21.15.49:0/239182976 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f295e218700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2958101e50 0x7f29580ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f295da17700 1 -- 172.21.15.49:0/239182976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2958101e50 msgr2=0x7f29580ffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f295da17700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2958101e50 0x7f29580ffab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f295da17700 1 -- 172.21.15.49:0/239182976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29540095f0 con 0x7f2958102770 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f295da17700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958102770 0x7f29580ffff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f294800c8f0 tx=0x7f294800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f294f7fe700 1 -- 172.21.15.49:0/239182976 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2948007ad0 con 0x7f2958102770 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f294f7fe700 1 -- 172.21.15.49:0/239182976 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f294800ce90 con 0x7f2958102770 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.486+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29581a2e50 con 0x7f2958102770 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.487+0000 7f294f7fe700 1 -- 172.21.15.49:0/239182976 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2948018630 con 0x7f2958102770 2024-05-15T16:24:13.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.487+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29581a31b0 con 0x7f2958102770 2024-05-15T16:24:13.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.488+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2958069af0 con 0x7f2958102770 2024-05-15T16:24:13.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.488+0000 7f294f7fe700 1 -- 172.21.15.49:0/239182976 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2948007c30 con 0x7f2958102770 2024-05-15T16:24:13.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.489+0000 7f294f7fe700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f294406e210 0x7f29440706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:13.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.489+0000 7f295e218700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f294406e210 0x7f29440706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:13.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.489+0000 7f294f7fe700 1 -- 172.21.15.49:0/239182976 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f294808e190 con 0x7f2958102770 2024-05-15T16:24:13.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.489+0000 7f295e218700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f294406e210 0x7f29440706d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f295400b5c0 tx=0x7f295401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:13.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.494+0000 7f294f7fe700 1 -- 172.21.15.49:0/239182976 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2948057470 con 0x7f2958102770 2024-05-15T16:24:13.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.787+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f295810ded0 con 0x7f2958102770 2024-05-15T16:24:13.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.787+0000 7f294f7fe700 1 -- 172.21.15.49:0/239182976 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f294805ab30 con 0x7f2958102770 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:24:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:24:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:24:13.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f294406e210 msgr2=0x7f29440706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f294406e210 0x7f29440706d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f295400b5c0 tx=0x7f295401a040 comp rx=0 tx=0).stop 2024-05-15T16:24:13.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958102770 msgr2=0x7f29580ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:13.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958102770 0x7f29580ffff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f294800c8f0 tx=0x7f294800ccb0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 shutdown_connections 2024-05-15T16:24:13.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f294406e210 0x7f29440706d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2958101e50 0x7f29580ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 --2- 172.21.15.49:0/239182976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2958102770 0x7f29580ffff0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:13.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 >> 172.21.15.49:0/239182976 conn(0x7f29580fb6c0 msgr2=0x7f2958103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:13.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 shutdown_connections 2024-05-15T16:24:13.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:13.790+0000 7f296047c700 1 -- 172.21.15.49:0/239182976 wait complete. 2024-05-15T16:24:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:13 smithi144 ceph-mon[86635]: from='client.35502 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:13 smithi144 ceph-mon[86635]: pgmap v817: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:13 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/876848787' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:24:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:13 smithi049 ceph-mon[112142]: from='client.35502 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:13 smithi049 ceph-mon[112142]: pgmap v817: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:13 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/876848787' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:24:15.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:14 smithi144 ceph-mon[86635]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:15.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:14 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/239182976' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:24:15.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:14 smithi049 ceph-mon[112142]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:15.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:14 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/239182976' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:24:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:15 smithi144 ceph-mon[86635]: pgmap v818: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:15 smithi049 ceph-mon[112142]: pgmap v818: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:17 smithi144 ceph-mon[86635]: pgmap v819: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:17 smithi049 ceph-mon[112142]: pgmap v819: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:19 smithi144 ceph-mon[86635]: pgmap v820: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:19 smithi049 ceph-mon[112142]: pgmap v820: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:21.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:24:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:24:22.105 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:21 smithi144 ceph-mon[86635]: pgmap v821: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:21 smithi049 ceph-mon[112142]: pgmap v821: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:23 smithi144 ceph-mon[86635]: pgmap v822: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:23 smithi049 ceph-mon[112142]: pgmap v822: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:25 smithi144 ceph-mon[86635]: pgmap v823: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:25 smithi049 ceph-mon[112142]: pgmap v823: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:27 smithi144 ceph-mon[86635]: pgmap v824: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:27 smithi049 ceph-mon[112142]: pgmap v824: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:29 smithi144 ceph-mon[86635]: pgmap v825: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:29 smithi049 ceph-mon[112142]: pgmap v825: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:31 smithi144 ceph-mon[86635]: pgmap v826: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:31 smithi049 ceph-mon[112142]: pgmap v826: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:33 smithi144 ceph-mon[86635]: pgmap v827: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:33 smithi049 ceph-mon[112142]: pgmap v827: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:35 smithi144 ceph-mon[86635]: pgmap v828: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:24:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:35 smithi049 ceph-mon[112142]: pgmap v828: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:24:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:37 smithi144 ceph-mon[86635]: pgmap v829: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:37 smithi049 ceph-mon[112142]: pgmap v829: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:39 smithi144 ceph-mon[86635]: pgmap v830: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:39 smithi049 ceph-mon[112142]: pgmap v830: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:42.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:41 smithi144 ceph-mon[86635]: pgmap v831: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:41 smithi049 ceph-mon[112142]: pgmap v831: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.913+0000 7f5e398a9700 1 -- 172.21.15.49:0/3248636007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 msgr2=0x7f5e341058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.913+0000 7f5e398a9700 1 --2- 172.21.15.49:0/3248636007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 0x7f5e341058b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009910 tx=0x7f5e28009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.913+0000 7f5e398a9700 1 -- 172.21.15.49:0/3248636007 shutdown_connections 2024-05-15T16:24:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.913+0000 7f5e398a9700 1 --2- 172.21.15.49:0/3248636007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 0x7f5e341058b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.913+0000 7f5e398a9700 1 --2- 172.21.15.49:0/3248636007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34100eb0 0x7f5e34101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.913+0000 7f5e398a9700 1 -- 172.21.15.49:0/3248636007 >> 172.21.15.49:0/3248636007 conn(0x7f5e340fc760 msgr2=0x7f5e340feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.913+0000 7f5e398a9700 1 -- 172.21.15.49:0/3248636007 shutdown_connections 2024-05-15T16:24:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.913+0000 7f5e398a9700 1 -- 172.21.15.49:0/3248636007 wait complete. 2024-05-15T16:24:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e398a9700 1 Processor -- start 2024-05-15T16:24:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e398a9700 1 -- start start 2024-05-15T16:24:43.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e398a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34100eb0 0x7f5e340732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e398a9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 0x7f5e340737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e398a9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34075270 con 0x7f5e34101860 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e398a9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34073d20 con 0x7f5e34100eb0 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e327fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 0x7f5e340737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e327fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 0x7f5e340737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42680/0 (socket says 172.21.15.49:42680) 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e327fc700 1 -- 172.21.15.49:0/3739198391 learned_addr learned my addr 172.21.15.49:0/3739198391 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.914+0000 7f5e32ffd700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34100eb0 0x7f5e340732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.915+0000 7f5e327fc700 1 -- 172.21.15.49:0/3739198391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34100eb0 msgr2=0x7f5e340732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.915+0000 7f5e327fc700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34100eb0 0x7f5e340732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.915+0000 7f5e327fc700 1 -- 172.21.15.49:0/3739198391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e280095f0 con 0x7f5e34101860 2024-05-15T16:24:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.915+0000 7f5e327fc700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 0x7f5e340737e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5e280092e0 tx=0x7f5e2800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.915+0000 7f5e388a7700 1 -- 172.21.15.49:0/3739198391 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e2801d070 con 0x7f5e34101860 2024-05-15T16:24:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.915+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e34073f70 con 0x7f5e34101860 2024-05-15T16:24:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.915+0000 7f5e388a7700 1 -- 172.21.15.49:0/3739198391 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e2800f460 con 0x7f5e34101860 2024-05-15T16:24:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.915+0000 7f5e388a7700 1 -- 172.21.15.49:0/3739198391 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e28017630 con 0x7f5e34101860 2024-05-15T16:24:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.915+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e341a2f60 con 0x7f5e34101860 2024-05-15T16:24:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.916+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e341092b0 con 0x7f5e34101860 2024-05-15T16:24:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.917+0000 7f5e388a7700 1 -- 172.21.15.49:0/3739198391 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5e2800f5d0 con 0x7f5e34101860 2024-05-15T16:24:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.917+0000 7f5e388a7700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1c06e200 0x7f5e1c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.917+0000 7f5e388a7700 1 -- 172.21.15.49:0/3739198391 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5e2808f480 con 0x7f5e34101860 2024-05-15T16:24:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.917+0000 7f5e32ffd700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1c06e200 0x7f5e1c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.918+0000 7f5e32ffd700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1c06e200 0x7f5e1c0706c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400ac50 tx=0x7f5e2400a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:43.922+0000 7f5e388a7700 1 -- 172.21.15.49:0/3739198391 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5e280587e0 con 0x7f5e34101860 2024-05-15T16:24:44.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.150+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e340747f0 con 0x7f5e1c06e200 2024-05-15T16:24:44.151 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:43 smithi049 ceph-mon[112142]: pgmap v832: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:44.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.151+0000 7f5e388a7700 1 -- 172.21.15.49:0/3739198391 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5e340747f0 con 0x7f5e1c06e200 2024-05-15T16:24:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1c06e200 msgr2=0x7f5e1c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1c06e200 0x7f5e1c0706c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400ac50 tx=0x7f5e2400a380 comp rx=0 tx=0).stop 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 msgr2=0x7f5e340737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 0x7f5e340737e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5e280092e0 tx=0x7f5e2800bd70 comp rx=0 tx=0).stop 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 shutdown_connections 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5e1c06e200 0x7f5e1c0706c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34100eb0 0x7f5e340732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 --2- 172.21.15.49:0/3739198391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e34101860 0x7f5e340737e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 >> 172.21.15.49:0/3739198391 conn(0x7f5e340fc760 msgr2=0x7f5e340fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 shutdown_connections 2024-05-15T16:24:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.153+0000 7f5e398a9700 1 -- 172.21.15.49:0/3739198391 wait complete. 2024-05-15T16:24:44.165 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:24:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:43 smithi144 ceph-mon[86635]: pgmap v832: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:44.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.263+0000 7fca18b6c700 1 -- 172.21.15.49:0/1610022622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073340 msgr2=0x7fca14073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.263+0000 7fca18b6c700 1 --2- 172.21.15.49:0/1610022622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073340 0x7fca14073720 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fca04009910 tx=0x7fca04009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:44.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.263+0000 7fca18b6c700 1 -- 172.21.15.49:0/1610022622 shutdown_connections 2024-05-15T16:24:44.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.263+0000 7fca18b6c700 1 --2- 172.21.15.49:0/1610022622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca14073c60 0x7fca14111770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.263+0000 7fca18b6c700 1 --2- 172.21.15.49:0/1610022622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073340 0x7fca14073720 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.263+0000 7fca18b6c700 1 -- 172.21.15.49:0/1610022622 >> 172.21.15.49:0/1610022622 conn(0x7fca140fc7d0 msgr2=0x7fca140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.263+0000 7fca18b6c700 1 -- 172.21.15.49:0/1610022622 shutdown_connections 2024-05-15T16:24:44.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.263+0000 7fca18b6c700 1 -- 172.21.15.49:0/1610022622 wait complete. 2024-05-15T16:24:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca18b6c700 1 Processor -- start 2024-05-15T16:24:44.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca18b6c700 1 -- start start 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca18b6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca14073340 0x7fca1419d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca18b6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073c60 0x7fca1419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca18b6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1419ddb0 con 0x7fca14073c60 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca18b6c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca141a1a80 con 0x7fca14073340 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca11d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073c60 0x7fca1419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca11d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073c60 0x7fca1419d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42706/0 (socket says 172.21.15.49:42706) 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca11d9b700 1 -- 172.21.15.49:0/1510802767 learned_addr learned my addr 172.21.15.49:0/1510802767 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca1259c700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca14073340 0x7fca1419d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca11d9b700 1 -- 172.21.15.49:0/1510802767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca14073340 msgr2=0x7fca1419d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca11d9b700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca14073340 0x7fca1419d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca11d9b700 1 -- 172.21.15.49:0/1510802767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040095f0 con 0x7fca14073c60 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.264+0000 7fca1259c700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca14073340 0x7fca1419d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.265+0000 7fca11d9b700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073c60 0x7fca1419d6d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fca0800d8d0 tx=0x7fca0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.265+0000 7fca037fe700 1 -- 172.21.15.49:0/1510802767 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca08009940 con 0x7fca14073c60 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.265+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca141a1d60 con 0x7fca14073c60 2024-05-15T16:24:44.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.265+0000 7fca037fe700 1 -- 172.21.15.49:0/1510802767 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca08010460 con 0x7fca14073c60 2024-05-15T16:24:44.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.265+0000 7fca037fe700 1 -- 172.21.15.49:0/1510802767 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca0800f5f0 con 0x7fca14073c60 2024-05-15T16:24:44.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.265+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca141a22b0 con 0x7fca14073c60 2024-05-15T16:24:44.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.266+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca14074a80 con 0x7fca14073c60 2024-05-15T16:24:44.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.266+0000 7fca037fe700 1 -- 172.21.15.49:0/1510802767 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fca080105d0 con 0x7fca14073c60 2024-05-15T16:24:44.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.267+0000 7fca037fe700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc9fc06e200 0x7fc9fc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:44.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.267+0000 7fca037fe700 1 -- 172.21.15.49:0/1510802767 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fca0808e1a0 con 0x7fca14073c60 2024-05-15T16:24:44.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.267+0000 7fca1259c700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc9fc06e200 0x7fc9fc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:44.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.267+0000 7fca1259c700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc9fc06e200 0x7fc9fc0706c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fca04006010 tx=0x7fca0400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.271+0000 7fca037fe700 1 -- 172.21.15.49:0/1510802767 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fca08057480 con 0x7fca14073c60 2024-05-15T16:24:44.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.496+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca141144b0 con 0x7fc9fc06e200 2024-05-15T16:24:44.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.497+0000 7fca037fe700 1 -- 172.21.15.49:0/1510802767 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fca141144b0 con 0x7fc9fc06e200 2024-05-15T16:24:44.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc9fc06e200 msgr2=0x7fc9fc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc9fc06e200 0x7fc9fc0706c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fca04006010 tx=0x7fca0400b540 comp rx=0 tx=0).stop 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073c60 msgr2=0x7fca1419d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073c60 0x7fca1419d6d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fca0800d8d0 tx=0x7fca0800dc90 comp rx=0 tx=0).stop 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 shutdown_connections 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc9fc06e200 0x7fc9fc0706c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca14073340 0x7fca1419d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 --2- 172.21.15.49:0/1510802767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca14073c60 0x7fca1419d6d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 >> 172.21.15.49:0/1510802767 conn(0x7fca140fc7d0 msgr2=0x7fca14100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 shutdown_connections 2024-05-15T16:24:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.500+0000 7fca18b6c700 1 -- 172.21.15.49:0/1510802767 wait complete. 2024-05-15T16:24:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.608+0000 7fe4beb76700 1 -- 172.21.15.49:0/1897293873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 msgr2=0x7fe4b8100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.608+0000 7fe4beb76700 1 --2- 172.21.15.49:0/1897293873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 0x7fe4b8100760 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009910 tx=0x7fe4a4009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.608+0000 7fe4beb76700 1 -- 172.21.15.49:0/1897293873 shutdown_connections 2024-05-15T16:24:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.608+0000 7fe4beb76700 1 --2- 172.21.15.49:0/1897293873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 0x7fe4b8100760 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.608+0000 7fe4beb76700 1 --2- 172.21.15.49:0/1897293873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8104bf0 0x7fe4b8104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.608+0000 7fe4beb76700 1 -- 172.21.15.49:0/1897293873 >> 172.21.15.49:0/1897293873 conn(0x7fe4b8076000 msgr2=0x7fe4b80fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:44.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.608+0000 7fe4beb76700 1 -- 172.21.15.49:0/1897293873 shutdown_connections 2024-05-15T16:24:44.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.608+0000 7fe4beb76700 1 -- 172.21.15.49:0/1897293873 wait complete. 2024-05-15T16:24:44.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.609+0000 7fe4beb76700 1 Processor -- start 2024-05-15T16:24:44.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.609+0000 7fe4beb76700 1 -- start start 2024-05-15T16:24:44.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.609+0000 7fe4beb76700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 0x7fe4b819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:44.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.609+0000 7fe4beb76700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8104bf0 0x7fe4b819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:44.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.609+0000 7fe4beb76700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b819d3a0 con 0x7fe4b81002e0 2024-05-15T16:24:44.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.609+0000 7fe4beb76700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b8196980 con 0x7fe4b8104bf0 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.609+0000 7fe4bc912700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 0x7fe4b819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4bc912700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 0x7fe4b819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42716/0 (socket says 172.21.15.49:42716) 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4bc912700 1 -- 172.21.15.49:0/2320660208 learned_addr learned my addr 172.21.15.49:0/2320660208 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4b7fff700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8104bf0 0x7fe4b819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4bc912700 1 -- 172.21.15.49:0/2320660208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8104bf0 msgr2=0x7fe4b819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4bc912700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8104bf0 0x7fe4b819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4bc912700 1 -- 172.21.15.49:0/2320660208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a40095f0 con 0x7fe4b81002e0 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4b7fff700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8104bf0 0x7fe4b819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4bc912700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 0x7fe4b819c840 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00d900 tx=0x7fe4ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4b5ffb700 1 -- 172.21.15.49:0/2320660208 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4ac004a00 con 0x7fe4b81002e0 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b8196c60 con 0x7fe4b81002e0 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4b5ffb700 1 -- 172.21.15.49:0/2320660208 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4ac0054e0 con 0x7fe4b81002e0 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4b5ffb700 1 -- 172.21.15.49:0/2320660208 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4ac009d60 con 0x7fe4b81002e0 2024-05-15T16:24:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.610+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b8197230 con 0x7fe4b81002e0 2024-05-15T16:24:44.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.611+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b8101500 con 0x7fe4b81002e0 2024-05-15T16:24:44.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.612+0000 7fe4b5ffb700 1 -- 172.21.15.49:0/2320660208 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe4ac005020 con 0x7fe4b81002e0 2024-05-15T16:24:44.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.612+0000 7fe4b5ffb700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe4a806e200 0x7fe4a80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:44.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.613+0000 7fe4b5ffb700 1 -- 172.21.15.49:0/2320660208 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe4ac08e3a0 con 0x7fe4b81002e0 2024-05-15T16:24:44.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.613+0000 7fe4b7fff700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe4a806e200 0x7fe4a80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:44.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.613+0000 7fe4b7fff700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe4a806e200 0x7fe4a80706c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40059d0 tx=0x7fe4a401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:44.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.617+0000 7fe4b5ffb700 1 -- 172.21.15.49:0/2320660208 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe4ac05b1b0 con 0x7fe4b81002e0 2024-05-15T16:24:44.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.836+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4b810ce60 con 0x7fe4a806e200 2024-05-15T16:24:44.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.847+0000 7fe4b5ffb700 1 -- 172.21.15.49:0/2320660208 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe4b810ce60 con 0x7fe4a806e200 2024-05-15T16:24:44.848 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (28m) 5m ago 39m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (39m) 5m ago 39m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (38m) 5m ago 38m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (25m) 5m ago 39m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (25m) 5m ago 38m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (27m) 5m ago 39m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (21m) 5m ago 33m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (21m) 5m ago 33m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (21m) 5m ago 33m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (5m) 5m ago 33m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (30m) 5m ago 41m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (29m) 5m ago 38m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (26m) 5m ago 41m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:24:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (26m) 5m ago 38m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:24:44.850 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (28m) 5m ago 39m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:24:44.850 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (28m) 5m ago 38m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:24:44.850 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (24m) 5m ago 37m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:24:44.850 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (24m) 5m ago 37m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:24:44.850 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (23m) 5m ago 36m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:24:44.850 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (23m) 5m ago 36m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:24:44.850 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (22m) 5m ago 35m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:24:44.850 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (21m) 5m ago 35m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:24:44.850 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (28m) 5m ago 39m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:24:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe4a806e200 msgr2=0x7fe4a80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe4a806e200 0x7fe4a80706c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40059d0 tx=0x7fe4a401a040 comp rx=0 tx=0).stop 2024-05-15T16:24:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 msgr2=0x7fe4b819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 0x7fe4b819c840 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00d900 tx=0x7fe4ac00dcc0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 shutdown_connections 2024-05-15T16:24:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe4a806e200 0x7fe4a80706c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4b81002e0 0x7fe4b819c840 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 --2- 172.21.15.49:0/2320660208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b8104bf0 0x7fe4b819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 >> 172.21.15.49:0/2320660208 conn(0x7fe4b8076000 msgr2=0x7fe4b80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.849+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 shutdown_connections 2024-05-15T16:24:44.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.850+0000 7fe4beb76700 1 -- 172.21.15.49:0/2320660208 wait complete. 2024-05-15T16:24:44.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.957+0000 7f0f9f97d700 1 -- 172.21.15.49:0/669614612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 msgr2=0x7f0f9810d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.957+0000 7f0f9f97d700 1 --2- 172.21.15.49:0/669614612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 0x7f0f9810d050 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009910 tx=0x7f0f88009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.957+0000 7f0f9f97d700 1 -- 172.21.15.49:0/669614612 shutdown_connections 2024-05-15T16:24:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.957+0000 7f0f9f97d700 1 --2- 172.21.15.49:0/669614612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 0x7f0f9810d050 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.957+0000 7f0f9f97d700 1 --2- 172.21.15.49:0/669614612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f980ff9e0 0x7f0f980ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.957+0000 7f0f9f97d700 1 -- 172.21.15.49:0/669614612 >> 172.21.15.49:0/669614612 conn(0x7f0f980fb6a0 msgr2=0x7f0f980fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.957+0000 7f0f9f97d700 1 -- 172.21.15.49:0/669614612 shutdown_connections 2024-05-15T16:24:44.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.957+0000 7f0f9f97d700 1 -- 172.21.15.49:0/669614612 wait complete. 2024-05-15T16:24:44.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9f97d700 1 Processor -- start 2024-05-15T16:24:44.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9f97d700 1 -- start start 2024-05-15T16:24:44.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9f97d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f980ff9e0 0x7f0f98198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:44.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9f97d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 0x7f0f981992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:44.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9f97d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f981999b0 con 0x7f0f98100390 2024-05-15T16:24:44.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9f97d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9819d680 con 0x7f0f980ff9e0 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9cf18700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 0x7f0f981992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9cf18700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 0x7f0f981992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42722/0 (socket says 172.21.15.49:42722) 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9cf18700 1 -- 172.21.15.49:0/194795977 learned_addr learned my addr 172.21.15.49:0/194795977 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9d719700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f980ff9e0 0x7f0f98198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9cf18700 1 -- 172.21.15.49:0/194795977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f980ff9e0 msgr2=0x7f0f98198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.958+0000 7f0f9cf18700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f980ff9e0 0x7f0f98198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.959+0000 7f0f9cf18700 1 -- 172.21.15.49:0/194795977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f880095f0 con 0x7f0f98100390 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.959+0000 7f0f9d719700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f980ff9e0 0x7f0f98198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.959+0000 7f0f9cf18700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 0x7f0f981992d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006010 tx=0x7f0f880056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.959+0000 7f0f8e7fc700 1 -- 172.21.15.49:0/194795977 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f8801d070 con 0x7f0f98100390 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.959+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9819d900 con 0x7f0f98100390 2024-05-15T16:24:44.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.959+0000 7f0f8e7fc700 1 -- 172.21.15.49:0/194795977 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f88003d50 con 0x7f0f98100390 2024-05-15T16:24:44.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.959+0000 7f0f8e7fc700 1 -- 172.21.15.49:0/194795977 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f88021410 con 0x7f0f98100390 2024-05-15T16:24:44.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.959+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9819de50 con 0x7f0f98100390 2024-05-15T16:24:44.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.960+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9810aed0 con 0x7f0f98100390 2024-05-15T16:24:44.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.961+0000 7f0f8e7fc700 1 -- 172.21.15.49:0/194795977 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0f8800f6f0 con 0x7f0f98100390 2024-05-15T16:24:44.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.961+0000 7f0f8e7fc700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f8406e200 0x7f0f840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:44.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.961+0000 7f0f8e7fc700 1 -- 172.21.15.49:0/194795977 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0f8808f370 con 0x7f0f98100390 2024-05-15T16:24:44.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.961+0000 7f0f9d719700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f8406e200 0x7f0f840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:44.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.962+0000 7f0f9d719700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f8406e200 0x7f0f840706c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0f94005fd0 tx=0x7f0f94005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:44.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:44.966+0000 7f0f8e7fc700 1 -- 172.21.15.49:0/194795977 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0f88058650 con 0x7f0f98100390 2024-05-15T16:24:45.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:44 smithi144 ceph-mon[86635]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:45.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:44 smithi049 ceph-mon[112142]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:45.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.262+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f9819a0a0 con 0x7f0f98100390 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.263+0000 7f0f8e7fc700 1 -- 172.21.15.49:0/194795977 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0f8805bd10 con 0x7f0f98100390 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:24:45.264 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:24:45.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:24:45.265 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:24:45.265 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:24:45.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:24:45.265 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:24:45.265 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:24:45.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:24:45.265 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f8406e200 msgr2=0x7f0f840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f8406e200 0x7f0f840706c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0f94005fd0 tx=0x7f0f94005ee0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 msgr2=0x7f0f981992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 0x7f0f981992d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006010 tx=0x7f0f880056a0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 shutdown_connections 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f8406e200 0x7f0f840706c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f980ff9e0 0x7f0f98198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 --2- 172.21.15.49:0/194795977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f98100390 0x7f0f981992d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 >> 172.21.15.49:0/194795977 conn(0x7f0f980fb6a0 msgr2=0x7f0f98108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 shutdown_connections 2024-05-15T16:24:45.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.265+0000 7f0f9f97d700 1 -- 172.21.15.49:0/194795977 wait complete. 2024-05-15T16:24:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.374+0000 7faab163a700 1 -- 172.21.15.49:0/2590770863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 msgr2=0x7faaac1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.374+0000 7faab163a700 1 --2- 172.21.15.49:0/2590770863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 0x7faaac1022b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009910 tx=0x7faa9c009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.374+0000 7faab163a700 1 -- 172.21.15.49:0/2590770863 shutdown_connections 2024-05-15T16:24:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.374+0000 7faab163a700 1 --2- 172.21.15.49:0/2590770863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaac1027f0 0x7faaac10ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.374+0000 7faab163a700 1 --2- 172.21.15.49:0/2590770863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 0x7faaac1022b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.374+0000 7faab163a700 1 -- 172.21.15.49:0/2590770863 >> 172.21.15.49:0/2590770863 conn(0x7faaac0fb760 msgr2=0x7faaac0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.374+0000 7faab163a700 1 -- 172.21.15.49:0/2590770863 shutdown_connections 2024-05-15T16:24:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.374+0000 7faab163a700 1 -- 172.21.15.49:0/2590770863 wait complete. 2024-05-15T16:24:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.374+0000 7faab163a700 1 Processor -- start 2024-05-15T16:24:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faab163a700 1 -- start start 2024-05-15T16:24:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faab163a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 0x7faaac19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faab163a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaac1027f0 0x7faaac19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faab163a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac19d4b0 con 0x7faaac101ed0 2024-05-15T16:24:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faab163a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac196a90 con 0x7faaac1027f0 2024-05-15T16:24:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faaaaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 0x7faaac19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faaaaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 0x7faaac19c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42742/0 (socket says 172.21.15.49:42742) 2024-05-15T16:24:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faaaaffd700 1 -- 172.21.15.49:0/867161035 learned_addr learned my addr 172.21.15.49:0/867161035 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faaaa7fc700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaac1027f0 0x7faaac19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faaaaffd700 1 -- 172.21.15.49:0/867161035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaac1027f0 msgr2=0x7faaac19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faaaaffd700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaac1027f0 0x7faaac19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.375+0000 7faaaaffd700 1 -- 172.21.15.49:0/867161035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c0095f0 con 0x7faaac101ed0 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.376+0000 7faaaaffd700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 0x7faaac19c950 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006010 tx=0x7faa9c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.376+0000 7faa93fff700 1 -- 172.21.15.49:0/867161035 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9c01d070 con 0x7faaac101ed0 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.376+0000 7faa93fff700 1 -- 172.21.15.49:0/867161035 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa9c00f460 con 0x7faaac101ed0 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.376+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac196d10 con 0x7faaac101ed0 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.376+0000 7faa93fff700 1 -- 172.21.15.49:0/867161035 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9c017630 con 0x7faaac101ed0 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.376+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac197200 con 0x7faaac101ed0 2024-05-15T16:24:45.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.377+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac105660 con 0x7faaac101ed0 2024-05-15T16:24:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.378+0000 7faa93fff700 1 -- 172.21.15.49:0/867161035 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faa9c00f5d0 con 0x7faaac101ed0 2024-05-15T16:24:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.378+0000 7faa93fff700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa9406e210 0x7faa940706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.378+0000 7faa93fff700 1 -- 172.21.15.49:0/867161035 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faa9c08f5e0 con 0x7faaac101ed0 2024-05-15T16:24:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.378+0000 7faaaa7fc700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa9406e210 0x7faa940706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:45.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.379+0000 7faaaa7fc700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa9406e210 0x7faa940706d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faaac197c50 tx=0x7faaa0005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:45.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.383+0000 7faa93fff700 1 -- 172.21.15.49:0/867161035 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7faa9c05c530 con 0x7faaac101ed0 2024-05-15T16:24:45.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.637+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faaac197f80 con 0x7faaac101ed0 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.637+0000 7faa93fff700 1 -- 172.21.15.49:0/867161035 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7faa9c05c0c0 con 0x7faaac101ed0 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:24:45.639 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:24:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:24:45.641 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:24:45.642 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:24:45.642 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:24:45.642 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:24:45.642 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa9406e210 msgr2=0x7faa940706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa9406e210 0x7faa940706d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faaac197c50 tx=0x7faaa0005f90 comp rx=0 tx=0).stop 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 msgr2=0x7faaac19c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 0x7faaac19c950 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006010 tx=0x7faa9c00be20 comp rx=0 tx=0).stop 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 shutdown_connections 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa9406e210 0x7faa940706d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac101ed0 0x7faaac19c950 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 --2- 172.21.15.49:0/867161035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaac1027f0 0x7faaac19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 >> 172.21.15.49:0/867161035 conn(0x7faaac0fb760 msgr2=0x7faaac103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 shutdown_connections 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.640+0000 7faab163a700 1 -- 172.21.15.49:0/867161035 wait complete. 2024-05-15T16:24:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:24:45.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.749+0000 7f058c103700 1 -- 172.21.15.49:0/966779928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05840ff9b0 msgr2=0x7f05840ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.749+0000 7f058c103700 1 --2- 172.21.15.49:0/966779928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05840ff9b0 0x7f05840ffd90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0580009910 tx=0x7f0580009c20 comp rx=0 tx=0).stop 2024-05-15T16:24:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.749+0000 7f058c103700 1 -- 172.21.15.49:0/966779928 shutdown_connections 2024-05-15T16:24:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.749+0000 7f058c103700 1 --2- 172.21.15.49:0/966779928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0584100360 0x7f058410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.749+0000 7f058c103700 1 --2- 172.21.15.49:0/966779928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05840ff9b0 0x7f05840ffd90 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.749+0000 7f058c103700 1 -- 172.21.15.49:0/966779928 >> 172.21.15.49:0/966779928 conn(0x7f0584076800 msgr2=0x7f05840fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.749+0000 7f058c103700 1 -- 172.21.15.49:0/966779928 shutdown_connections 2024-05-15T16:24:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.749+0000 7f058c103700 1 -- 172.21.15.49:0/966779928 wait complete. 2024-05-15T16:24:45.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058c103700 1 Processor -- start 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058c103700 1 -- start start 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058c103700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05840ff9b0 0x7f0584198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058c103700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0584100360 0x7f05841992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058c103700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0584199990 con 0x7f0584100360 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058c103700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058419d660 con 0x7f05840ff9b0 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058969e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0584100360 0x7f05841992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058969e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0584100360 0x7f05841992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42758/0 (socket says 172.21.15.49:42758) 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058969e700 1 -- 172.21.15.49:0/2502380754 learned_addr learned my addr 172.21.15.49:0/2502380754 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f0589e9f700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05840ff9b0 0x7f0584198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058969e700 1 -- 172.21.15.49:0/2502380754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05840ff9b0 msgr2=0x7f0584198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058969e700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05840ff9b0 0x7f0584198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.750+0000 7f058969e700 1 -- 172.21.15.49:0/2502380754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800095f0 con 0x7f0584100360 2024-05-15T16:24:45.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.751+0000 7f058969e700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0584100360 0x7f05841992b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f057400d900 tx=0x7f057400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.751+0000 7f057affd700 1 -- 172.21.15.49:0/2502380754 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05740041f0 con 0x7f0584100360 2024-05-15T16:24:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.751+0000 7f057affd700 1 -- 172.21.15.49:0/2502380754 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0574004cf0 con 0x7f0584100360 2024-05-15T16:24:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.751+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058419d940 con 0x7f0584100360 2024-05-15T16:24:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.751+0000 7f057affd700 1 -- 172.21.15.49:0/2502380754 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f057400b750 con 0x7f0584100360 2024-05-15T16:24:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.751+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058419de90 con 0x7f0584100360 2024-05-15T16:24:45.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.752+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0584068f60 con 0x7f0584100360 2024-05-15T16:24:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.752+0000 7f057affd700 1 -- 172.21.15.49:0/2502380754 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0574004750 con 0x7f0584100360 2024-05-15T16:24:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.753+0000 7f057affd700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f057006e200 0x7f05700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.753+0000 7f057affd700 1 -- 172.21.15.49:0/2502380754 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f057408e070 con 0x7f0584100360 2024-05-15T16:24:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.753+0000 7f0589e9f700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f057006e200 0x7f05700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.753+0000 7f0589e9f700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f057006e200 0x7f05700706c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0580006010 tx=0x7f058000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:45.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.758+0000 7f057affd700 1 -- 172.21.15.49:0/2502380754 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0574057350 con 0x7f0584100360 2024-05-15T16:24:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:45 smithi049 ceph-mon[112142]: from='client.35526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:45 smithi049 ceph-mon[112142]: pgmap v833: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:45 smithi049 ceph-mon[112142]: from='client.35530 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:45 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/194795977' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:24:45.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:45 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/867161035' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:24:45.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.985+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f05841078c0 con 0x7f057006e200 2024-05-15T16:24:45.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.987+0000 7f057affd700 1 -- 172.21.15.49:0/2502380754 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f05841078c0 con 0x7f057006e200 2024-05-15T16:24:45.988 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:24:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:24:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:24:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:24:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:24:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:24:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:24:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:24:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:24:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:24:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:24:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:24:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:24:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:24:45.989 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:24:45.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f057006e200 msgr2=0x7f05700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f057006e200 0x7f05700706c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0580006010 tx=0x7f058000b540 comp rx=0 tx=0).stop 2024-05-15T16:24:45.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0584100360 msgr2=0x7f05841992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:45.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0584100360 0x7f05841992b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f057400d900 tx=0x7f057400dc10 comp rx=0 tx=0).stop 2024-05-15T16:24:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 shutdown_connections 2024-05-15T16:24:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f057006e200 0x7f05700706c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05840ff9b0 0x7f0584198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 --2- 172.21.15.49:0/2502380754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0584100360 0x7f05841992b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 >> 172.21.15.49:0/2502380754 conn(0x7f0584076800 msgr2=0x7f05840fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 shutdown_connections 2024-05-15T16:24:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:45.989+0000 7f058c103700 1 -- 172.21.15.49:0/2502380754 wait complete. 2024-05-15T16:24:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.097+0000 7faa22ad7700 1 -- 172.21.15.49:0/2746762835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c1027d0 msgr2=0x7faa1c10acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.097+0000 7faa22ad7700 1 --2- 172.21.15.49:0/2746762835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c1027d0 0x7faa1c10acc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00b0d0 tx=0x7faa0c00b3e0 comp rx=0 tx=0).stop 2024-05-15T16:24:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.097+0000 7faa22ad7700 1 -- 172.21.15.49:0/2746762835 shutdown_connections 2024-05-15T16:24:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.097+0000 7faa22ad7700 1 --2- 172.21.15.49:0/2746762835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c1027d0 0x7faa1c10acc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.097+0000 7faa22ad7700 1 --2- 172.21.15.49:0/2746762835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c101eb0 0x7faa1c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.097+0000 7faa22ad7700 1 -- 172.21.15.49:0/2746762835 >> 172.21.15.49:0/2746762835 conn(0x7faa1c0fb6c0 msgr2=0x7faa1c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.098+0000 7faa22ad7700 1 -- 172.21.15.49:0/2746762835 shutdown_connections 2024-05-15T16:24:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.098+0000 7faa22ad7700 1 -- 172.21.15.49:0/2746762835 wait complete. 2024-05-15T16:24:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.098+0000 7faa22ad7700 1 Processor -- start 2024-05-15T16:24:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.098+0000 7faa22ad7700 1 -- start start 2024-05-15T16:24:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.098+0000 7faa22ad7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c101eb0 0x7faa1c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.098+0000 7faa22ad7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c1027d0 0x7faa1c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.098+0000 7faa22ad7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c19d3c0 con 0x7faa1c101eb0 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.098+0000 7faa22ad7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c196a30 con 0x7faa1c1027d0 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.098+0000 7faa20873700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c101eb0 0x7faa1c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa20873700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c101eb0 0x7faa1c19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42780/0 (socket says 172.21.15.49:42780) 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa20873700 1 -- 172.21.15.49:0/3006232202 learned_addr learned my addr 172.21.15.49:0/3006232202 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa1bfff700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c1027d0 0x7faa1c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa20873700 1 -- 172.21.15.49:0/3006232202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c1027d0 msgr2=0x7faa1c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa20873700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c1027d0 0x7faa1c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa20873700 1 -- 172.21.15.49:0/3006232202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c009d00 con 0x7faa1c101eb0 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa20873700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c101eb0 0x7faa1c19c8f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faa0400c930 tx=0x7faa0400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa19ffb700 1 -- 172.21.15.49:0/3006232202 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa04007ad0 con 0x7faa1c101eb0 2024-05-15T16:24:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa19ffb700 1 -- 172.21.15.49:0/3006232202 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa04007c30 con 0x7faa1c101eb0 2024-05-15T16:24:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c196d10 con 0x7faa1c101eb0 2024-05-15T16:24:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa19ffb700 1 -- 172.21.15.49:0/3006232202 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa04018610 con 0x7faa1c101eb0 2024-05-15T16:24:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.099+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c197230 con 0x7faa1c101eb0 2024-05-15T16:24:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.100+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c1055d0 con 0x7faa1c101eb0 2024-05-15T16:24:46.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.101+0000 7faa19ffb700 1 -- 172.21.15.49:0/3006232202 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faa0400f450 con 0x7faa1c101eb0 2024-05-15T16:24:46.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.101+0000 7faa19ffb700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa0806e200 0x7faa080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:24:46.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.101+0000 7faa19ffb700 1 -- 172.21.15.49:0/3006232202 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faa0408e3a0 con 0x7faa1c101eb0 2024-05-15T16:24:46.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.101+0000 7faa1bfff700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa0806e200 0x7faa080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:24:46.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.102+0000 7faa1bfff700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa0806e200 0x7faa080706c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00b820 tx=0x7faa0c0097e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:24:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.106+0000 7faa19ffb700 1 -- 172.21.15.49:0/3006232202 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7faa0405b1b0 con 0x7faa1c101eb0 2024-05-15T16:24:46.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:45 smithi144 ceph-mon[86635]: from='client.35526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:45 smithi144 ceph-mon[86635]: pgmap v833: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:45 smithi144 ceph-mon[86635]: from='client.35530 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:45 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/194795977' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:24:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:45 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/867161035' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.396+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faa1c197ef0 con 0x7faa1c101eb0 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.396+0000 7faa19ffb700 1 -- 172.21.15.49:0/3006232202 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7faa0405ad40 con 0x7faa1c101eb0 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:24:46.398 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:24:46.399 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.399+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa0806e200 msgr2=0x7faa080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.399+0000 7faa22ad7700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa0806e200 0x7faa080706c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00b820 tx=0x7faa0c0097e0 comp rx=0 tx=0).stop 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.399+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c101eb0 msgr2=0x7faa1c19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.399+0000 7faa22ad7700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c101eb0 0x7faa1c19c8f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faa0400c930 tx=0x7faa0400ccf0 comp rx=0 tx=0).stop 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.399+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 shutdown_connections 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.399+0000 7faa22ad7700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faa0806e200 0x7faa080706c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.399+0000 7faa22ad7700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa1c101eb0 0x7faa1c19c8f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.399+0000 7faa22ad7700 1 --2- 172.21.15.49:0/3006232202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c1027d0 0x7faa1c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.399+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 >> 172.21.15.49:0/3006232202 conn(0x7faa1c0fb6c0 msgr2=0x7faa1c1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.400+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 shutdown_connections 2024-05-15T16:24:46.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:24:46.400+0000 7faa22ad7700 1 -- 172.21.15.49:0/3006232202 wait complete. 2024-05-15T16:24:47.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:46 smithi144 ceph-mon[86635]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:47.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:46 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3006232202' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:24:47.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:46 smithi049 ceph-mon[112142]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:24:47.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:46 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3006232202' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:24:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:47 smithi144 ceph-mon[86635]: pgmap v834: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:47 smithi049 ceph-mon[112142]: pgmap v834: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:49 smithi144 ceph-mon[86635]: pgmap v835: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:49 smithi049 ceph-mon[112142]: pgmap v835: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:24:51.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:24:51.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:51 smithi049 ceph-mon[112142]: pgmap v836: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:51 smithi144 ceph-mon[86635]: pgmap v836: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:53 smithi144 ceph-mon[86635]: pgmap v837: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:53 smithi049 ceph-mon[112142]: pgmap v837: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:55 smithi144 ceph-mon[86635]: pgmap v838: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:55 smithi049 ceph-mon[112142]: pgmap v838: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:57 smithi144 ceph-mon[86635]: pgmap v839: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:24:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:57 smithi049 ceph-mon[112142]: pgmap v839: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:24:59 smithi144 ceph-mon[86635]: pgmap v840: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:24:59 smithi049 ceph-mon[112142]: pgmap v840: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:01 smithi144 ceph-mon[86635]: pgmap v841: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:01 smithi049 ceph-mon[112142]: pgmap v841: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:03 smithi144 ceph-mon[86635]: pgmap v842: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:03 smithi049 ceph-mon[112142]: pgmap v842: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:05 smithi144 ceph-mon[86635]: pgmap v843: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:25:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:05 smithi049 ceph-mon[112142]: pgmap v843: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:25:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:07 smithi144 ceph-mon[86635]: pgmap v844: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:07 smithi049 ceph-mon[112142]: pgmap v844: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:09 smithi144 ceph-mon[86635]: pgmap v845: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:09 smithi049 ceph-mon[112142]: pgmap v845: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:11 smithi144 ceph-mon[86635]: pgmap v846: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:11 smithi049 ceph-mon[112142]: pgmap v846: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:13 smithi144 ceph-mon[86635]: pgmap v847: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:13 smithi049 ceph-mon[112142]: pgmap v847: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:15 smithi144 ceph-mon[86635]: pgmap v848: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:15 smithi049 ceph-mon[112142]: pgmap v848: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:16.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.523+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3917739491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c0ff9b0 msgr2=0x7fbb6c0ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.523+0000 7fbb70ba1700 1 --2- 172.21.15.49:0/3917739491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c0ff9b0 0x7fbb6c0ffe30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009910 tx=0x7fbb54009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.523+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3917739491 shutdown_connections 2024-05-15T16:25:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.523+0000 7fbb70ba1700 1 --2- 172.21.15.49:0/3917739491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c0ff9b0 0x7fbb6c0ffe30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.523+0000 7fbb70ba1700 1 --2- 172.21.15.49:0/3917739491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb6c1059b0 0x7fbb6c105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.523+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3917739491 >> 172.21.15.49:0/3917739491 conn(0x7fbb6c0fb500 msgr2=0x7fbb6c0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.523+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3917739491 shutdown_connections 2024-05-15T16:25:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.523+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3917739491 wait complete. 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.524+0000 7fbb70ba1700 1 Processor -- start 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.524+0000 7fbb70ba1700 1 -- start start 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.524+0000 7fbb70ba1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb6c0ff9b0 0x7fbb6c19b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.524+0000 7fbb70ba1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c1059b0 0x7fbb6c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb70ba1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c19c100 con 0x7fbb6c1059b0 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb70ba1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c195730 con 0x7fbb6c0ff9b0 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb62ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c1059b0 0x7fbb6c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb62ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c1059b0 0x7fbb6c19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34328/0 (socket says 172.21.15.49:34328) 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb62ffd700 1 -- 172.21.15.49:0/3998721580 learned_addr learned my addr 172.21.15.49:0/3998721580 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb6b7fe700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb6c0ff9b0 0x7fbb6c19b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb62ffd700 1 -- 172.21.15.49:0/3998721580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb6c0ff9b0 msgr2=0x7fbb6c19b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb62ffd700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb6c0ff9b0 0x7fbb6c19b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb62ffd700 1 -- 172.21.15.49:0/3998721580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540095f0 con 0x7fbb6c1059b0 2024-05-15T16:25:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb62ffd700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c1059b0 0x7fbb6c19bae0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005f10 tx=0x7fbb5400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb697fa700 1 -- 172.21.15.49:0/3998721580 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5401d070 con 0x7fbb6c1059b0 2024-05-15T16:25:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb697fa700 1 -- 172.21.15.49:0/3998721580 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb540052a0 con 0x7fbb6c1059b0 2024-05-15T16:25:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb697fa700 1 -- 172.21.15.49:0/3998721580 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb54022470 con 0x7fbb6c1059b0 2024-05-15T16:25:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.525+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c1959b0 con 0x7fbb6c1059b0 2024-05-15T16:25:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.526+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c195e50 con 0x7fbb6c1059b0 2024-05-15T16:25:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.526+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c107fd0 con 0x7fbb6c1059b0 2024-05-15T16:25:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.527+0000 7fbb697fa700 1 -- 172.21.15.49:0/3998721580 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbb54003780 con 0x7fbb6c1059b0 2024-05-15T16:25:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.528+0000 7fbb697fa700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5806e200 0x7fbb580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.528+0000 7fbb697fa700 1 -- 172.21.15.49:0/3998721580 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbb5408f5c0 con 0x7fbb6c1059b0 2024-05-15T16:25:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.528+0000 7fbb6b7fe700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5806e200 0x7fbb580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:16.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.528+0000 7fbb6b7fe700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5806e200 0x7fbb580706c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009bf0 tx=0x7fbb5c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:16.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.533+0000 7fbb697fa700 1 -- 172.21.15.49:0/3998721580 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbb5405c3d0 con 0x7fbb6c1059b0 2024-05-15T16:25:16.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.763+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb6c10cbb0 con 0x7fbb5806e200 2024-05-15T16:25:16.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.764+0000 7fbb697fa700 1 -- 172.21.15.49:0/3998721580 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbb6c10cbb0 con 0x7fbb5806e200 2024-05-15T16:25:16.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5806e200 msgr2=0x7fbb580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:16.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5806e200 0x7fbb580706c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009bf0 tx=0x7fbb5c009260 comp rx=0 tx=0).stop 2024-05-15T16:25:16.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c1059b0 msgr2=0x7fbb6c19bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:16.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c1059b0 0x7fbb6c19bae0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005f10 tx=0x7fbb5400b650 comp rx=0 tx=0).stop 2024-05-15T16:25:16.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 shutdown_connections 2024-05-15T16:25:16.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbb5806e200 0x7fbb580706c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:16.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb6c0ff9b0 0x7fbb6c19b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:16.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 --2- 172.21.15.49:0/3998721580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb6c1059b0 0x7fbb6c19bae0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:16.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 >> 172.21.15.49:0/3998721580 conn(0x7fbb6c0fb500 msgr2=0x7fbb6c0fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:16.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 shutdown_connections 2024-05-15T16:25:16.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.767+0000 7fbb70ba1700 1 -- 172.21.15.49:0/3998721580 wait complete. 2024-05-15T16:25:16.779 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:25:16.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.878+0000 7f8627286700 1 -- 172.21.15.49:0/2507461614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100780 msgr2=0x7f86201047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.878+0000 7f8627286700 1 --2- 172.21.15.49:0/2507461614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100780 0x7f86201047d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8610009910 tx=0x7f8610009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.878+0000 7f8627286700 1 -- 172.21.15.49:0/2507461614 shutdown_connections 2024-05-15T16:25:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.878+0000 7f8627286700 1 --2- 172.21.15.49:0/2507461614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8620100780 0x7f86201047d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.878+0000 7f8627286700 1 --2- 172.21.15.49:0/2507461614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86200ffdd0 0x7f86201001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.878+0000 7f8627286700 1 -- 172.21.15.49:0/2507461614 >> 172.21.15.49:0/2507461614 conn(0x7f86200fb6a0 msgr2=0x7f86200fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.878+0000 7f8627286700 1 -- 172.21.15.49:0/2507461614 shutdown_connections 2024-05-15T16:25:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.878+0000 7f8627286700 1 -- 172.21.15.49:0/2507461614 wait complete. 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.879+0000 7f8627286700 1 Processor -- start 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.879+0000 7f8627286700 1 -- start start 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.879+0000 7f8627286700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86200ffdd0 0x7f862019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.879+0000 7f8627286700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8620100780 0x7f862019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.879+0000 7f8627286700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862019c350 con 0x7f86200ffdd0 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.879+0000 7f8627286700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8620195930 con 0x7f8620100780 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.879+0000 7f8625022700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86200ffdd0 0x7f862019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.879+0000 7f8625022700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86200ffdd0 0x7f862019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34344/0 (socket says 172.21.15.49:34344) 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.879+0000 7f8625022700 1 -- 172.21.15.49:0/867729874 learned_addr learned my addr 172.21.15.49:0/867729874 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f8624821700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8620100780 0x7f862019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f8625022700 1 -- 172.21.15.49:0/867729874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8620100780 msgr2=0x7f862019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f8625022700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8620100780 0x7f862019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f8625022700 1 -- 172.21.15.49:0/867729874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86100095f0 con 0x7f86200ffdd0 2024-05-15T16:25:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f8625022700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86200ffdd0 0x7f862019b7f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f861c009970 tx=0x7f861c009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f86167fc700 1 -- 172.21.15.49:0/867729874 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f861c012070 con 0x7f86200ffdd0 2024-05-15T16:25:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8620195bb0 con 0x7f86200ffdd0 2024-05-15T16:25:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f86167fc700 1 -- 172.21.15.49:0/867729874 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f861c007af0 con 0x7f86200ffdd0 2024-05-15T16:25:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f86167fc700 1 -- 172.21.15.49:0/867729874 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f861c017bf0 con 0x7f86200ffdd0 2024-05-15T16:25:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.880+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86201960d0 con 0x7f86200ffdd0 2024-05-15T16:25:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.881+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86201012b0 con 0x7f86200ffdd0 2024-05-15T16:25:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.882+0000 7f86167fc700 1 -- 172.21.15.49:0/867729874 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f861c007c60 con 0x7f86200ffdd0 2024-05-15T16:25:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.882+0000 7f86167fc700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f860c06e210 0x7f860c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.882+0000 7f86167fc700 1 -- 172.21.15.49:0/867729874 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f861c08f6a0 con 0x7f86200ffdd0 2024-05-15T16:25:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.883+0000 7f8624821700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f860c06e210 0x7f860c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.883+0000 7f8624821700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f860c06e210 0x7f860c0706d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8610009fd0 tx=0x7f861001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:16.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:16.887+0000 7f86167fc700 1 -- 172.21.15.49:0/867729874 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f861c05c560 con 0x7f86200ffdd0 2024-05-15T16:25:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.113+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f862010ce30 con 0x7f860c06e210 2024-05-15T16:25:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.114+0000 7f86167fc700 1 -- 172.21.15.49:0/867729874 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f862010ce30 con 0x7f860c06e210 2024-05-15T16:25:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f860c06e210 msgr2=0x7f860c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f860c06e210 0x7f860c0706d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8610009fd0 tx=0x7f861001a040 comp rx=0 tx=0).stop 2024-05-15T16:25:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86200ffdd0 msgr2=0x7f862019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86200ffdd0 0x7f862019b7f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f861c009970 tx=0x7f861c009c80 comp rx=0 tx=0).stop 2024-05-15T16:25:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 shutdown_connections 2024-05-15T16:25:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f860c06e210 0x7f860c0706d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86200ffdd0 0x7f862019b7f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 --2- 172.21.15.49:0/867729874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8620100780 0x7f862019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 >> 172.21.15.49:0/867729874 conn(0x7f86200fb6a0 msgr2=0x7f86200fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 shutdown_connections 2024-05-15T16:25:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.116+0000 7f8627286700 1 -- 172.21.15.49:0/867729874 wait complete. 2024-05-15T16:25:17.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.224+0000 7fbbc350c700 1 -- 172.21.15.49:0/3874723030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 msgr2=0x7fbbbc1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.224+0000 7fbbc350c700 1 --2- 172.21.15.49:0/3874723030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 0x7fbbbc1058e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009910 tx=0x7fbbac009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:17.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.224+0000 7fbbc350c700 1 -- 172.21.15.49:0/3874723030 shutdown_connections 2024-05-15T16:25:17.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.224+0000 7fbbc350c700 1 --2- 172.21.15.49:0/3874723030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 0x7fbbbc1058e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.224+0000 7fbbc350c700 1 --2- 172.21.15.49:0/3874723030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbbc100ee0 0x7fbbbc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.224+0000 7fbbc350c700 1 -- 172.21.15.49:0/3874723030 >> 172.21.15.49:0/3874723030 conn(0x7fbbbc0fc790 msgr2=0x7fbbbc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.224+0000 7fbbc350c700 1 -- 172.21.15.49:0/3874723030 shutdown_connections 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.224+0000 7fbbc350c700 1 -- 172.21.15.49:0/3874723030 wait complete. 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.225+0000 7fbbc350c700 1 Processor -- start 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.225+0000 7fbbc350c700 1 -- start start 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.225+0000 7fbbc350c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbbc100ee0 0x7fbbbc19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.225+0000 7fbbc350c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 0x7fbbbc19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.225+0000 7fbbc350c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc19d440 con 0x7fbbbc101890 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.225+0000 7fbbc350c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc196a20 con 0x7fbbbc100ee0 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.225+0000 7fbbc0aa7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 0x7fbbbc19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.225+0000 7fbbc0aa7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 0x7fbbbc19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34358/0 (socket says 172.21.15.49:34358) 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.225+0000 7fbbc0aa7700 1 -- 172.21.15.49:0/3172357576 learned_addr learned my addr 172.21.15.49:0/3172357576 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbc12a8700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbbc100ee0 0x7fbbbc19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:17.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbc0aa7700 1 -- 172.21.15.49:0/3172357576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbbc100ee0 msgr2=0x7fbbbc19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbc0aa7700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbbc100ee0 0x7fbbbc19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbc0aa7700 1 -- 172.21.15.49:0/3172357576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac0095f0 con 0x7fbbbc101890 2024-05-15T16:25:17.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbc0aa7700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 0x7fbbbc19ce20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00bd70 tx=0x7fbbac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:17.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbb27fc700 1 -- 172.21.15.49:0/3172357576 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbac01d070 con 0x7fbbbc101890 2024-05-15T16:25:17.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbbc196ca0 con 0x7fbbbc101890 2024-05-15T16:25:17.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbb27fc700 1 -- 172.21.15.49:0/3172357576 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbac0056e0 con 0x7fbbbc101890 2024-05-15T16:25:17.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbb27fc700 1 -- 172.21.15.49:0/3172357576 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbac017400 con 0x7fbbbc101890 2024-05-15T16:25:17.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.226+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbbc197190 con 0x7fbbbc101890 2024-05-15T16:25:17.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.227+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbbc1092e0 con 0x7fbbbc101890 2024-05-15T16:25:17.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.228+0000 7fbbb27fc700 1 -- 172.21.15.49:0/3172357576 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbbac00f460 con 0x7fbbbc101890 2024-05-15T16:25:17.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.228+0000 7fbbb27fc700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbba806e200 0x7fbba80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:17.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.228+0000 7fbbb27fc700 1 -- 172.21.15.49:0/3172357576 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbbac08f5b0 con 0x7fbbbc101890 2024-05-15T16:25:17.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.228+0000 7fbbc12a8700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbba806e200 0x7fbba80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:17.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.229+0000 7fbbc12a8700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbba806e200 0x7fbba80706c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8005950 tx=0x7fbbb80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:17.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.233+0000 7fbbb27fc700 1 -- 172.21.15.49:0/3172357576 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbbac05c480 con 0x7fbbbc101890 2024-05-15T16:25:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.449+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbbc197ee0 con 0x7fbba806e200 2024-05-15T16:25:17.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.461+0000 7fbbb27fc700 1 -- 172.21.15.49:0/3172357576 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbbbc197ee0 con 0x7fbba806e200 2024-05-15T16:25:17.462 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:25:17.462 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (28m) 6m ago 40m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (40m) 6m ago 40m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (39m) 6m ago 39m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (25m) 6m ago 40m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (25m) 6m ago 39m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (28m) 6m ago 39m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (22m) 6m ago 34m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (21m) 6m ago 34m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (21m) 6m ago 34m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (6m) 6m ago 34m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (30m) 6m ago 41m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (29m) 6m ago 39m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (27m) 6m ago 41m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (26m) 6m ago 38m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (29m) 6m ago 40m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:25:17.463 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (29m) 6m ago 39m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:25:17.464 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (25m) 6m ago 38m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:25:17.464 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (24m) 6m ago 37m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:25:17.464 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (24m) 6m ago 37m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:25:17.464 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (23m) 6m ago 36m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:25:17.464 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (23m) 6m ago 36m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:25:17.464 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (22m) 6m ago 35m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:25:17.464 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (28m) 6m ago 39m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.463+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbba806e200 msgr2=0x7fbba80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbba806e200 0x7fbba80706c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8005950 tx=0x7fbbb80058e0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 msgr2=0x7fbbbc19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 0x7fbbbc19ce20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00bd70 tx=0x7fbbac00be50 comp rx=0 tx=0).stop 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 shutdown_connections 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fbba806e200 0x7fbba80706c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbbc100ee0 0x7fbbbc19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 --2- 172.21.15.49:0/3172357576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc101890 0x7fbbbc19ce20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 >> 172.21.15.49:0/3172357576 conn(0x7fbbbc0fc790 msgr2=0x7fbbbc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 shutdown_connections 2024-05-15T16:25:17.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.464+0000 7fbbc350c700 1 -- 172.21.15.49:0/3172357576 wait complete. 2024-05-15T16:25:17.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.571+0000 7f5a109a6700 1 -- 172.21.15.49:0/519507296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c1018a0 msgr2=0x7f5a0c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.571+0000 7f5a109a6700 1 --2- 172.21.15.49:0/519507296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c1018a0 0x7f5a0c1058f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009910 tx=0x7f59f4009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:17.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.571+0000 7f5a109a6700 1 -- 172.21.15.49:0/519507296 shutdown_connections 2024-05-15T16:25:17.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.571+0000 7f5a109a6700 1 --2- 172.21.15.49:0/519507296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c1018a0 0x7f5a0c1058f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.571+0000 7f5a109a6700 1 --2- 172.21.15.49:0/519507296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a0c100ef0 0x7f5a0c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.571+0000 7f5a109a6700 1 -- 172.21.15.49:0/519507296 >> 172.21.15.49:0/519507296 conn(0x7f5a0c0fc7c0 msgr2=0x7f5a0c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:17.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.571+0000 7f5a109a6700 1 -- 172.21.15.49:0/519507296 shutdown_connections 2024-05-15T16:25:17.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.571+0000 7f5a109a6700 1 -- 172.21.15.49:0/519507296 wait complete. 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.572+0000 7f5a109a6700 1 Processor -- start 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.572+0000 7f5a109a6700 1 -- start start 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.572+0000 7f5a109a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c100ef0 0x7f5a0c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.572+0000 7f5a109a6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a0c1018a0 0x7f5a0c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.572+0000 7f5a109a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c19d4c0 con 0x7f5a0c100ef0 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.572+0000 7f5a109a6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c196a30 con 0x7f5a0c1018a0 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.572+0000 7f5a0a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c100ef0 0x7f5a0c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.572+0000 7f5a0a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c100ef0 0x7f5a0c19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34388/0 (socket says 172.21.15.49:34388) 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.572+0000 7f5a0a59c700 1 -- 172.21.15.49:0/2384739649 learned_addr learned my addr 172.21.15.49:0/2384739649 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a09d9b700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a0c1018a0 0x7f5a0c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a0a59c700 1 -- 172.21.15.49:0/2384739649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a0c1018a0 msgr2=0x7f5a0c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a0a59c700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a0c1018a0 0x7f5a0c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a0a59c700 1 -- 172.21.15.49:0/2384739649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f40095f0 con 0x7f5a0c100ef0 2024-05-15T16:25:17.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a0a59c700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c100ef0 0x7f5a0c19c8f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f59fc007e90 tx=0x7f59fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:17.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a037fe700 1 -- 172.21.15.49:0/2384739649 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59fc00de60 con 0x7f5a0c100ef0 2024-05-15T16:25:17.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a037fe700 1 -- 172.21.15.49:0/2384739649 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59fc00f040 con 0x7f5a0c100ef0 2024-05-15T16:25:17.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c196d10 con 0x7f5a0c100ef0 2024-05-15T16:25:17.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a037fe700 1 -- 172.21.15.49:0/2384739649 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59fc004520 con 0x7f5a0c100ef0 2024-05-15T16:25:17.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.573+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c197260 con 0x7f5a0c100ef0 2024-05-15T16:25:17.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.574+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0c1092f0 con 0x7f5a0c100ef0 2024-05-15T16:25:17.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.575+0000 7f5a037fe700 1 -- 172.21.15.49:0/2384739649 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f59fc0046a0 con 0x7f5a0c100ef0 2024-05-15T16:25:17.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.576+0000 7f5a037fe700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e200 0x7f59f80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:17.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.576+0000 7f5a037fe700 1 -- 172.21.15.49:0/2384739649 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f59fc08dcc0 con 0x7f5a0c100ef0 2024-05-15T16:25:17.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.576+0000 7f5a09d9b700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e200 0x7f59f80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:17.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.576+0000 7f5a09d9b700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e200 0x7f59f80706c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0fed90 tx=0x7f59f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:17.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.580+0000 7f5a037fe700 1 -- 172.21.15.49:0/2384739649 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f59fc05ab50 con 0x7f5a0c100ef0 2024-05-15T16:25:17.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.874+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5a0c197f20 con 0x7f5a0c100ef0 2024-05-15T16:25:17.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.874+0000 7f5a037fe700 1 -- 172.21.15.49:0/2384739649 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f59fc019070 con 0x7f5a0c100ef0 2024-05-15T16:25:17.875 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:25:17.875 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:25:17.876 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.876+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e200 msgr2=0x7f59f80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.876+0000 7f5a109a6700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e200 0x7f59f80706c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0fed90 tx=0x7f59f401a040 comp rx=0 tx=0).stop 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.876+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c100ef0 msgr2=0x7f5a0c19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.876+0000 7f5a109a6700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c100ef0 0x7f5a0c19c8f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f59fc007e90 tx=0x7f59fc00d3b0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.876+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 shutdown_connections 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.876+0000 7f5a109a6700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e200 0x7f59f80706c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.876+0000 7f5a109a6700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0c100ef0 0x7f5a0c19c8f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.876+0000 7f5a109a6700 1 --2- 172.21.15.49:0/2384739649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a0c1018a0 0x7f5a0c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.876+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 >> 172.21.15.49:0/2384739649 conn(0x7f5a0c0fc7c0 msgr2=0x7f5a0c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.877+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 shutdown_connections 2024-05-15T16:25:17.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.877+0000 7f5a109a6700 1 -- 172.21.15.49:0/2384739649 wait complete. 2024-05-15T16:25:17.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:17 smithi144 ceph-mon[86635]: pgmap v849: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:17.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:17 smithi144 ceph-mon[86635]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:17.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:17 smithi144 ceph-mon[86635]: from='client.35554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.985+0000 7f8e92257700 1 -- 172.21.15.49:0/1816182546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 msgr2=0x7f8e8c10ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.985+0000 7f8e92257700 1 --2- 172.21.15.49:0/1816182546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 0x7f8e8c10ace0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009940 tx=0x7f8e80009c50 comp rx=0 tx=0).stop 2024-05-15T16:25:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.985+0000 7f8e92257700 1 -- 172.21.15.49:0/1816182546 shutdown_connections 2024-05-15T16:25:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.985+0000 7f8e92257700 1 --2- 172.21.15.49:0/1816182546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 0x7f8e8c10ace0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.985+0000 7f8e92257700 1 --2- 172.21.15.49:0/1816182546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101ed0 0x7f8e8c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.985+0000 7f8e92257700 1 -- 172.21.15.49:0/1816182546 >> 172.21.15.49:0/1816182546 conn(0x7f8e8c0fb700 msgr2=0x7f8e8c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.985+0000 7f8e92257700 1 -- 172.21.15.49:0/1816182546 shutdown_connections 2024-05-15T16:25:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.985+0000 7f8e92257700 1 -- 172.21.15.49:0/1816182546 wait complete. 2024-05-15T16:25:17.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e92257700 1 Processor -- start 2024-05-15T16:25:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e92257700 1 -- start start 2024-05-15T16:25:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e92257700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101ed0 0x7f8e8c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e92257700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 0x7f8e8c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e92257700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c19d580 con 0x7f8e8c1027f0 2024-05-15T16:25:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e92257700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c196af0 con 0x7f8e8c101ed0 2024-05-15T16:25:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e8affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 0x7f8e8c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:17.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e8affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 0x7f8e8c19cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34404/0 (socket says 172.21.15.49:34404) 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e8affd700 1 -- 172.21.15.49:0/2795366211 learned_addr learned my addr 172.21.15.49:0/2795366211 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.986+0000 7f8e8b7fe700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101ed0 0x7f8e8c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e8affd700 1 -- 172.21.15.49:0/2795366211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101ed0 msgr2=0x7f8e8c19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e8affd700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101ed0 0x7f8e8c19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e8affd700 1 -- 172.21.15.49:0/2795366211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e800095f0 con 0x7f8e8c1027f0 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e8b7fe700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101ed0 0x7f8e8c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e8affd700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 0x7f8e8c19cef0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8e800055f0 tx=0x7f8e800056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e88ff9700 1 -- 172.21.15.49:0/2795366211 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e8001d070 con 0x7f8e8c1027f0 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e88ff9700 1 -- 172.21.15.49:0/2795366211 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e8000f460 con 0x7f8e8c1027f0 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8c196d70 con 0x7f8e8c1027f0 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e88ff9700 1 -- 172.21.15.49:0/2795366211 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e80021640 con 0x7f8e8c1027f0 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.987+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8c197260 con 0x7f8e8c1027f0 2024-05-15T16:25:17.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.988+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8c069a50 con 0x7f8e8c1027f0 2024-05-15T16:25:17.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.989+0000 7f8e88ff9700 1 -- 172.21.15.49:0/2795366211 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8e8000fa40 con 0x7f8e8c1027f0 2024-05-15T16:25:17.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.989+0000 7f8e88ff9700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e7406e200 0x7f8e740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:17.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.989+0000 7f8e88ff9700 1 -- 172.21.15.49:0/2795366211 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8e8008f2a0 con 0x7f8e8c1027f0 2024-05-15T16:25:17.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.989+0000 7f8e8b7fe700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e7406e200 0x7f8e740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:17.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.990+0000 7f8e8b7fe700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e7406e200 0x7f8e740706c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c005fd0 tx=0x7f8e7c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:17.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:17.994+0000 7f8e88ff9700 1 -- 172.21.15.49:0/2795366211 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8e8005c0b0 con 0x7f8e8c1027f0 2024-05-15T16:25:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:17 smithi049 ceph-mon[112142]: pgmap v849: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:17 smithi049 ceph-mon[112142]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:17 smithi049 ceph-mon[112142]: from='client.35554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:18.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.245+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8e8c10ded0 con 0x7f8e8c1027f0 2024-05-15T16:25:18.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.246+0000 7f8e88ff9700 1 -- 172.21.15.49:0/2795366211 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f8e8005bc40 con 0x7f8e8c1027f0 2024-05-15T16:25:18.247 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:25:18.247 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:25:18.247 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:25:18.247 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:25:18.247 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:25:18.247 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:25:18.247 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:25:18.248 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:25:18.249 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:25:18.250 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:25:18.250 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:25:18.250 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:25:18.250 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:25:18.250 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:25:18.250 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:25:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e7406e200 msgr2=0x7f8e740706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e7406e200 0x7f8e740706c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c005fd0 tx=0x7f8e7c005ee0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 msgr2=0x7f8e8c19cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 0x7f8e8c19cef0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8e800055f0 tx=0x7f8e800056d0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 shutdown_connections 2024-05-15T16:25:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e7406e200 0x7f8e740706c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101ed0 0x7f8e8c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 --2- 172.21.15.49:0/2795366211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c1027f0 0x7f8e8c19cef0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 >> 172.21.15.49:0/2795366211 conn(0x7f8e8c0fb700 msgr2=0x7f8e8c1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 shutdown_connections 2024-05-15T16:25:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.248+0000 7f8e92257700 1 -- 172.21.15.49:0/2795366211 wait complete. 2024-05-15T16:25:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:25:18.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.356+0000 7fc25c7ed700 1 -- 172.21.15.49:0/2498468513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 msgr2=0x7fc2541001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:18.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.356+0000 7fc25c7ed700 1 --2- 172.21.15.49:0/2498468513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 0x7fc2541001f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc250009910 tx=0x7fc250009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:18.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.357+0000 7fc25c7ed700 1 -- 172.21.15.49:0/2498468513 shutdown_connections 2024-05-15T16:25:18.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.357+0000 7fc25c7ed700 1 --2- 172.21.15.49:0/2498468513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2541007c0 0x7fc254104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.357+0000 7fc25c7ed700 1 --2- 172.21.15.49:0/2498468513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 0x7fc2541001f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.357+0000 7fc25c7ed700 1 -- 172.21.15.49:0/2498468513 >> 172.21.15.49:0/2498468513 conn(0x7fc2540fb700 msgr2=0x7fc2540fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:18.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.357+0000 7fc25c7ed700 1 -- 172.21.15.49:0/2498468513 shutdown_connections 2024-05-15T16:25:18.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.357+0000 7fc25c7ed700 1 -- 172.21.15.49:0/2498468513 wait complete. 2024-05-15T16:25:18.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.357+0000 7fc25c7ed700 1 Processor -- start 2024-05-15T16:25:18.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.357+0000 7fc25c7ed700 1 -- start start 2024-05-15T16:25:18.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25c7ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 0x7fc25419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:18.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25c7ed700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2541007c0 0x7fc25419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:18.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25c7ed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc254196a70 con 0x7fc2540ffe10 2024-05-15T16:25:18.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25c7ed700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc254196be0 con 0x7fc2541007c0 2024-05-15T16:25:18.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25a589700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 0x7fc25419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25a589700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 0x7fc25419c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34422/0 (socket says 172.21.15.49:34422) 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25a589700 1 -- 172.21.15.49:0/3061089195 learned_addr learned my addr 172.21.15.49:0/3061089195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc259d88700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2541007c0 0x7fc25419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25a589700 1 -- 172.21.15.49:0/3061089195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2541007c0 msgr2=0x7fc25419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25a589700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2541007c0 0x7fc25419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25a589700 1 -- 172.21.15.49:0/3061089195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500095f0 con 0x7fc2540ffe10 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25a589700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 0x7fc25419c970 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc25000bd70 tx=0x7fc25000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc24b7fe700 1 -- 172.21.15.49:0/3061089195 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc25001d070 con 0x7fc2540ffe10 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc24b7fe700 1 -- 172.21.15.49:0/3061089195 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2500056e0 con 0x7fc2540ffe10 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc254196e60 con 0x7fc2540ffe10 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.358+0000 7fc24b7fe700 1 -- 172.21.15.49:0/3061089195 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc25000fe40 con 0x7fc2540ffe10 2024-05-15T16:25:18.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.359+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc254197350 con 0x7fc2540ffe10 2024-05-15T16:25:18.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.359+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2541012f0 con 0x7fc2540ffe10 2024-05-15T16:25:18.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.360+0000 7fc24b7fe700 1 -- 172.21.15.49:0/3061089195 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc25000f680 con 0x7fc2540ffe10 2024-05-15T16:25:18.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.360+0000 7fc24b7fe700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc24006e1a0 0x7fc240070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:18.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.361+0000 7fc24b7fe700 1 -- 172.21.15.49:0/3061089195 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc25008f5a0 con 0x7fc2540ffe10 2024-05-15T16:25:18.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.361+0000 7fc259d88700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc24006e1a0 0x7fc240070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:18.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.361+0000 7fc259d88700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc24006e1a0 0x7fc240070660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc254198190 tx=0x7fc24400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:18.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.365+0000 7fc24b7fe700 1 -- 172.21.15.49:0/3061089195 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc250058880 con 0x7fc2540ffe10 2024-05-15T16:25:18.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.592+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc254197fa0 con 0x7fc24006e1a0 2024-05-15T16:25:18.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.593+0000 7fc24b7fe700 1 -- 172.21.15.49:0/3061089195 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc254197fa0 con 0x7fc24006e1a0 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:25:18.595 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:25:18.596 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:25:18.596 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.595+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc24006e1a0 msgr2=0x7fc240070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.595+0000 7fc25c7ed700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc24006e1a0 0x7fc240070660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc254198190 tx=0x7fc24400b410 comp rx=0 tx=0).stop 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.595+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 msgr2=0x7fc25419c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.595+0000 7fc25c7ed700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 0x7fc25419c970 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc25000bd70 tx=0x7fc25000bda0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.595+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 shutdown_connections 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.595+0000 7fc25c7ed700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc24006e1a0 0x7fc240070660 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.595+0000 7fc25c7ed700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2540ffe10 0x7fc25419c970 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.595+0000 7fc25c7ed700 1 --2- 172.21.15.49:0/3061089195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2541007c0 0x7fc25419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.595+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 >> 172.21.15.49:0/3061089195 conn(0x7fc2540fb700 msgr2=0x7fc254069670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.596+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 shutdown_connections 2024-05-15T16:25:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.596+0000 7fc25c7ed700 1 -- 172.21.15.49:0/3061089195 wait complete. 2024-05-15T16:25:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.705+0000 7fcfd549d700 1 -- 172.21.15.49:0/3504705253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd01018b0 msgr2=0x7fcfd0105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.705+0000 7fcfd549d700 1 --2- 172.21.15.49:0/3504705253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd01018b0 0x7fcfd0105900 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009910 tx=0x7fcfb8009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.705+0000 7fcfd549d700 1 -- 172.21.15.49:0/3504705253 shutdown_connections 2024-05-15T16:25:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.705+0000 7fcfd549d700 1 --2- 172.21.15.49:0/3504705253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd01018b0 0x7fcfd0105900 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.705+0000 7fcfd549d700 1 --2- 172.21.15.49:0/3504705253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfd0100f00 0x7fcfd01012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.705+0000 7fcfd549d700 1 -- 172.21.15.49:0/3504705253 >> 172.21.15.49:0/3504705253 conn(0x7fcfd00fc7d0 msgr2=0x7fcfd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.705+0000 7fcfd549d700 1 -- 172.21.15.49:0/3504705253 shutdown_connections 2024-05-15T16:25:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.705+0000 7fcfd549d700 1 -- 172.21.15.49:0/3504705253 wait complete. 2024-05-15T16:25:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.705+0000 7fcfd549d700 1 Processor -- start 2024-05-15T16:25:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfd549d700 1 -- start start 2024-05-15T16:25:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfd549d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd0100f00 0x7fcfd019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfd549d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfd01018b0 0x7fcfd019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfd549d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd019d390 con 0x7fcfd0100f00 2024-05-15T16:25:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfd549d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0196a00 con 0x7fcfd01018b0 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd0100f00 0x7fcfd019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd0100f00 0x7fcfd019c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34428/0 (socket says 172.21.15.49:34428) 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfceffd700 1 -- 172.21.15.49:0/3393092664 learned_addr learned my addr 172.21.15.49:0/3393092664 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfceffd700 1 -- 172.21.15.49:0/3393092664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfd01018b0 msgr2=0x7fcfd019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfceffd700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfd01018b0 0x7fcfd019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfceffd700 1 -- 172.21.15.49:0/3393092664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80095f0 con 0x7fcfd0100f00 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.706+0000 7fcfceffd700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd0100f00 0x7fcfd019c8c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000cfd0 tx=0x7fcfc000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.707+0000 7fcfc7fff700 1 -- 172.21.15.49:0/3393092664 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc000ad70 con 0x7fcfd0100f00 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.707+0000 7fcfc7fff700 1 -- 172.21.15.49:0/3393092664 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfc00128e0 con 0x7fcfd0100f00 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.707+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd0196ce0 con 0x7fcfd0100f00 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.707+0000 7fcfc7fff700 1 -- 172.21.15.49:0/3393092664 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc0011880 con 0x7fcfd0100f00 2024-05-15T16:25:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.707+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd01971e0 con 0x7fcfd0100f00 2024-05-15T16:25:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.708+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd0109300 con 0x7fcfd0100f00 2024-05-15T16:25:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.708+0000 7fcfc7fff700 1 -- 172.21.15.49:0/3393092664 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcfc0012420 con 0x7fcfd0100f00 2024-05-15T16:25:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.709+0000 7fcfc7fff700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcfbc06e200 0x7fcfbc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.709+0000 7fcfc7fff700 1 -- 172.21.15.49:0/3393092664 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcfc008e2f0 con 0x7fcfd0100f00 2024-05-15T16:25:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.709+0000 7fcfce7fc700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcfbc06e200 0x7fcfbc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:18.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.710+0000 7fcfce7fc700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcfbc06e200 0x7fcfbc0706c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800b5c0 tx=0x7fcfb8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:18.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:18.714+0000 7fcfc7fff700 1 -- 172.21.15.49:0/3393092664 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcfc005b100 con 0x7fcfd0100f00 2024-05-15T16:25:19.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.011+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcfd0197ec0 con 0x7fcfd0100f00 2024-05-15T16:25:19.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.011+0000 7fcfc7fff700 1 -- 172.21.15.49:0/3393092664 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fcfc005ac90 con 0x7fcfd0100f00 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:25:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:25:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:25:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.013+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcfbc06e200 msgr2=0x7fcfbc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.013+0000 7fcfd549d700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcfbc06e200 0x7fcfbc0706c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800b5c0 tx=0x7fcfb8005960 comp rx=0 tx=0).stop 2024-05-15T16:25:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.013+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd0100f00 msgr2=0x7fcfd019c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.013+0000 7fcfd549d700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd0100f00 0x7fcfd019c8c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000cfd0 tx=0x7fcfc000a340 comp rx=0 tx=0).stop 2024-05-15T16:25:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.013+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 shutdown_connections 2024-05-15T16:25:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.013+0000 7fcfd549d700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcfbc06e200 0x7fcfbc0706c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.013+0000 7fcfd549d700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd0100f00 0x7fcfd019c8c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.013+0000 7fcfd549d700 1 --2- 172.21.15.49:0/3393092664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfd01018b0 0x7fcfd019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.013+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 >> 172.21.15.49:0/3393092664 conn(0x7fcfd00fc7d0 msgr2=0x7fcfd00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.014+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 shutdown_connections 2024-05-15T16:25:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:19.014+0000 7fcfd549d700 1 -- 172.21.15.49:0/3393092664 wait complete. 2024-05-15T16:25:19.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:18 smithi144 ceph-mon[86635]: from='client.35558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:19.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:18 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2384739649' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:25:19.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:18 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2795366211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:25:19.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:18 smithi049 ceph-mon[112142]: from='client.35558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:19.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:18 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2384739649' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:25:19.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:18 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2795366211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:25:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:19 smithi144 ceph-mon[86635]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:19 smithi144 ceph-mon[86635]: pgmap v850: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:19 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3393092664' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:25:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:19 smithi049 ceph-mon[112142]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:19 smithi049 ceph-mon[112142]: pgmap v850: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:19 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3393092664' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:25:21.161 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:25:21.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:25:22.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:21 smithi144 ceph-mon[86635]: pgmap v851: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:21 smithi049 ceph-mon[112142]: pgmap v851: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:24.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:23 smithi144 ceph-mon[86635]: pgmap v852: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:23 smithi049 ceph-mon[112142]: pgmap v852: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:25 smithi144 ceph-mon[86635]: pgmap v853: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:25 smithi049 ceph-mon[112142]: pgmap v853: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:27 smithi144 ceph-mon[86635]: pgmap v854: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:28.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:27 smithi049 ceph-mon[112142]: pgmap v854: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:29 smithi144 ceph-mon[86635]: pgmap v855: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:29 smithi049 ceph-mon[112142]: pgmap v855: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:31 smithi144 ceph-mon[86635]: pgmap v856: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:31 smithi049 ceph-mon[112142]: pgmap v856: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:33 smithi144 ceph-mon[86635]: pgmap v857: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:33 smithi049 ceph-mon[112142]: pgmap v857: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:35 smithi144 ceph-mon[86635]: pgmap v858: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:25:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:35 smithi049 ceph-mon[112142]: pgmap v858: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:25:38.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:37 smithi144 ceph-mon[86635]: pgmap v859: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:37 smithi049 ceph-mon[112142]: pgmap v859: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:40.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:39 smithi144 ceph-mon[86635]: pgmap v860: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:39 smithi049 ceph-mon[112142]: pgmap v860: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:41 smithi144 ceph-mon[86635]: pgmap v861: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:41 smithi049 ceph-mon[112142]: pgmap v861: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:43 smithi144 ceph-mon[86635]: pgmap v862: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:43 smithi049 ceph-mon[112142]: pgmap v862: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:45 smithi144 ceph-mon[86635]: pgmap v863: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:45 smithi049 ceph-mon[112142]: pgmap v863: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:47 smithi144 ceph-mon[86635]: pgmap v864: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:47 smithi049 ceph-mon[112142]: pgmap v864: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:49.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.137+0000 7f6d93a0c700 1 -- 172.21.15.49:0/2712777085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 msgr2=0x7f6d8c100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.137+0000 7f6d93a0c700 1 --2- 172.21.15.49:0/2712777085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 0x7f6d8c100250 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b0a0 tx=0x7f6d8800b3b0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.137+0000 7f6d93a0c700 1 -- 172.21.15.49:0/2712777085 shutdown_connections 2024-05-15T16:25:49.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.137+0000 7f6d93a0c700 1 --2- 172.21.15.49:0/2712777085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c100820 0x7f6d8c104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.137+0000 7f6d93a0c700 1 --2- 172.21.15.49:0/2712777085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 0x7f6d8c100250 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.137+0000 7f6d93a0c700 1 -- 172.21.15.49:0/2712777085 >> 172.21.15.49:0/2712777085 conn(0x7f6d8c0fb760 msgr2=0x7f6d8c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:49.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.137+0000 7f6d93a0c700 1 -- 172.21.15.49:0/2712777085 shutdown_connections 2024-05-15T16:25:49.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.137+0000 7f6d93a0c700 1 -- 172.21.15.49:0/2712777085 wait complete. 2024-05-15T16:25:49.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.138+0000 7f6d93a0c700 1 Processor -- start 2024-05-15T16:25:49.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.138+0000 7f6d93a0c700 1 -- start start 2024-05-15T16:25:49.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.138+0000 7f6d93a0c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 0x7f6d8c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:49.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.138+0000 7f6d93a0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c100820 0x7f6d8c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:49.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.138+0000 7f6d93a0c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1985e0 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.138+0000 7f6d93a0c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c198720 con 0x7f6d8c100820 2024-05-15T16:25:49.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d917a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 0x7f6d8c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:49.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d917a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 0x7f6d8c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40556/0 (socket says 172.21.15.49:40556) 2024-05-15T16:25:49.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d917a8700 1 -- 172.21.15.49:0/4126061098 learned_addr learned my addr 172.21.15.49:0/4126061098 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d90fa7700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c100820 0x7f6d8c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d917a8700 1 -- 172.21.15.49:0/4126061098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c100820 msgr2=0x7f6d8c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d917a8700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c100820 0x7f6d8c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d917a8700 1 -- 172.21.15.49:0/4126061098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88009d00 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d90fa7700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c100820 0x7f6d8c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d917a8700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 0x7f6d8c197ad0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000f80 tx=0x7f6d88007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d827fc700 1 -- 172.21.15.49:0/4126061098 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d8800e040 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d827fc700 1 -- 172.21.15.49:0/4126061098 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d88003de0 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c1a0b40 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d827fc700 1 -- 172.21.15.49:0/4126061098 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d8801d720 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.139+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c1a0ef0 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.141+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c108210 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.141+0000 7f6d827fc700 1 -- 172.21.15.49:0/4126061098 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6d88004690 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.142+0000 7f6d827fc700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d7806e200 0x7f6d780706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:49.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.142+0000 7f6d90fa7700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d7806e200 0x7f6d780706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:49.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.142+0000 7f6d827fc700 1 -- 172.21.15.49:0/4126061098 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6d8808fa10 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.142+0000 7f6d90fa7700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d7806e200 0x7f6d780706c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0699c0 tx=0x7f6d7c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:49.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.147+0000 7f6d827fc700 1 -- 172.21.15.49:0/4126061098 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6d88058cf0 con 0x7f6d8c0ffe70 2024-05-15T16:25:49.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.375+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d8c10ceb0 con 0x7f6d7806e200 2024-05-15T16:25:49.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.377+0000 7f6d827fc700 1 -- 172.21.15.49:0/4126061098 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6d8c10ceb0 con 0x7f6d7806e200 2024-05-15T16:25:49.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d7806e200 msgr2=0x7f6d780706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d7806e200 0x7f6d780706c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0699c0 tx=0x7f6d7c008040 comp rx=0 tx=0).stop 2024-05-15T16:25:49.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 msgr2=0x7f6d8c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 0x7f6d8c197ad0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000f80 tx=0x7f6d88007bd0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 shutdown_connections 2024-05-15T16:25:49.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6d7806e200 0x7f6d780706c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d8c0ffe70 0x7f6d8c197ad0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 --2- 172.21.15.49:0/4126061098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c100820 0x7f6d8c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 >> 172.21.15.49:0/4126061098 conn(0x7f6d8c0fb760 msgr2=0x7f6d8c0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:49.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 shutdown_connections 2024-05-15T16:25:49.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.379+0000 7f6d93a0c700 1 -- 172.21.15.49:0/4126061098 wait complete. 2024-05-15T16:25:49.391 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.488+0000 7fde56c66700 1 -- 172.21.15.49:0/3480002674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde501018a0 msgr2=0x7fde501058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.488+0000 7fde56c66700 1 --2- 172.21.15.49:0/3480002674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde501018a0 0x7fde501058f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fde38009910 tx=0x7fde38009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.488+0000 7fde56c66700 1 -- 172.21.15.49:0/3480002674 shutdown_connections 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.488+0000 7fde56c66700 1 --2- 172.21.15.49:0/3480002674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde501018a0 0x7fde501058f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.488+0000 7fde56c66700 1 --2- 172.21.15.49:0/3480002674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde50100ef0 0x7fde501012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.488+0000 7fde56c66700 1 -- 172.21.15.49:0/3480002674 >> 172.21.15.49:0/3480002674 conn(0x7fde500fc7c0 msgr2=0x7fde500febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.488+0000 7fde56c66700 1 -- 172.21.15.49:0/3480002674 shutdown_connections 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.488+0000 7fde56c66700 1 -- 172.21.15.49:0/3480002674 wait complete. 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde56c66700 1 Processor -- start 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde56c66700 1 -- start start 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde56c66700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde50100ef0 0x7fde50196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde56c66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde501018a0 0x7fde50197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde56c66700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde501978d0 con 0x7fde50100ef0 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde56c66700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5019a540 con 0x7fde501018a0 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde54a02700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde50100ef0 0x7fde50196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde54a02700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde50100ef0 0x7fde50196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40568/0 (socket says 172.21.15.49:40568) 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde54a02700 1 -- 172.21.15.49:0/319701625 learned_addr learned my addr 172.21.15.49:0/319701625 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.489+0000 7fde4ffff700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde501018a0 0x7fde50197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.490+0000 7fde54a02700 1 -- 172.21.15.49:0/319701625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde501018a0 msgr2=0x7fde50197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.490+0000 7fde54a02700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde501018a0 0x7fde50197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.490+0000 7fde54a02700 1 -- 172.21.15.49:0/319701625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde380095f0 con 0x7fde50100ef0 2024-05-15T16:25:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.490+0000 7fde54a02700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde50100ef0 0x7fde50196d00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fde4000ba50 tx=0x7fde4000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.490+0000 7fde4dffb700 1 -- 172.21.15.49:0/319701625 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde4000ca00 con 0x7fde50100ef0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.490+0000 7fde4dffb700 1 -- 172.21.15.49:0/319701625 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde4000cb60 con 0x7fde50100ef0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.490+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5019a820 con 0x7fde50100ef0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.490+0000 7fde4dffb700 1 -- 172.21.15.49:0/319701625 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde400075a0 con 0x7fde50100ef0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.490+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5019ad40 con 0x7fde50100ef0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.491+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde50109240 con 0x7fde50100ef0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.491+0000 7fde4dffb700 1 -- 172.21.15.49:0/319701625 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fde40014440 con 0x7fde50100ef0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.492+0000 7fde4dffb700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde3c06e200 0x7fde3c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.492+0000 7fde4dffb700 1 -- 172.21.15.49:0/319701625 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fde40006e40 con 0x7fde50100ef0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.492+0000 7fde4ffff700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde3c06e200 0x7fde3c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.493+0000 7fde4ffff700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde3c06e200 0x7fde3c0706c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fde501982c0 tx=0x7fde3801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:49.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.497+0000 7fde4dffb700 1 -- 172.21.15.49:0/319701625 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fde40056c20 con 0x7fde50100ef0 2024-05-15T16:25:49.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.722+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde50198040 con 0x7fde3c06e200 2024-05-15T16:25:49.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.724+0000 7fde4dffb700 1 -- 172.21.15.49:0/319701625 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fde50198040 con 0x7fde3c06e200 2024-05-15T16:25:49.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde3c06e200 msgr2=0x7fde3c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde3c06e200 0x7fde3c0706c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fde501982c0 tx=0x7fde3801a040 comp rx=0 tx=0).stop 2024-05-15T16:25:49.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde50100ef0 msgr2=0x7fde50196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde50100ef0 0x7fde50196d00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fde4000ba50 tx=0x7fde4000be10 comp rx=0 tx=0).stop 2024-05-15T16:25:49.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 shutdown_connections 2024-05-15T16:25:49.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde3c06e200 0x7fde3c0706c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde50100ef0 0x7fde50196d00 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 --2- 172.21.15.49:0/319701625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde501018a0 0x7fde50197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 >> 172.21.15.49:0/319701625 conn(0x7fde500fc7c0 msgr2=0x7fde500fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:49.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 shutdown_connections 2024-05-15T16:25:49.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.726+0000 7fde56c66700 1 -- 172.21.15.49:0/319701625 wait complete. 2024-05-15T16:25:49.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.835+0000 7f1e07415700 1 -- 172.21.15.49:0/3330305783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 msgr2=0x7f1e001001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.835+0000 7f1e07415700 1 --2- 172.21.15.49:0/3330305783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 0x7f1e001001d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009910 tx=0x7f1dfc009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:49.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.835+0000 7f1e07415700 1 -- 172.21.15.49:0/3330305783 shutdown_connections 2024-05-15T16:25:49.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.835+0000 7f1e07415700 1 --2- 172.21.15.49:0/3330305783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e001007a0 0x7f1e001047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.835+0000 7f1e07415700 1 --2- 172.21.15.49:0/3330305783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 0x7f1e001001d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.835+0000 7f1e07415700 1 -- 172.21.15.49:0/3330305783 >> 172.21.15.49:0/3330305783 conn(0x7f1e000fb6a0 msgr2=0x7f1e000fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:49.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.835+0000 7f1e07415700 1 -- 172.21.15.49:0/3330305783 shutdown_connections 2024-05-15T16:25:49.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.835+0000 7f1e07415700 1 -- 172.21.15.49:0/3330305783 wait complete. 2024-05-15T16:25:49.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.835+0000 7f1e07415700 1 Processor -- start 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e07415700 1 -- start start 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e07415700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 0x7f1e0019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e07415700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e001007a0 0x7f1e0019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e07415700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0019d4c0 con 0x7f1e000ffdf0 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e07415700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00196aa0 con 0x7f1e001007a0 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e051b1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 0x7f1e0019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e051b1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 0x7f1e0019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40590/0 (socket says 172.21.15.49:40590) 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e051b1700 1 -- 172.21.15.49:0/1943218118 learned_addr learned my addr 172.21.15.49:0/1943218118 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e049b0700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e001007a0 0x7f1e0019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e051b1700 1 -- 172.21.15.49:0/1943218118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e001007a0 msgr2=0x7f1e0019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e051b1700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e001007a0 0x7f1e0019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e051b1700 1 -- 172.21.15.49:0/1943218118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc0095f0 con 0x7f1e000ffdf0 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.836+0000 7f1e051b1700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 0x7f1e0019c960 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00bd70 tx=0x7f1dfc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.837+0000 7f1df27fc700 1 -- 172.21.15.49:0/1943218118 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dfc01d070 con 0x7f1e000ffdf0 2024-05-15T16:25:49.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.837+0000 7f1df27fc700 1 -- 172.21.15.49:0/1943218118 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dfc00f460 con 0x7f1e000ffdf0 2024-05-15T16:25:49.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.837+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e00196d20 con 0x7f1e000ffdf0 2024-05-15T16:25:49.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.837+0000 7f1df27fc700 1 -- 172.21.15.49:0/1943218118 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dfc017630 con 0x7f1e000ffdf0 2024-05-15T16:25:49.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.837+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e00197210 con 0x7f1e000ffdf0 2024-05-15T16:25:49.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.838+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e00108190 con 0x7f1e000ffdf0 2024-05-15T16:25:49.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.838+0000 7f1df27fc700 1 -- 172.21.15.49:0/1943218118 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1dfc00f5d0 con 0x7f1e000ffdf0 2024-05-15T16:25:49.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.839+0000 7f1df27fc700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1dec06e200 0x7f1dec0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:49.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.839+0000 7f1df27fc700 1 -- 172.21.15.49:0/1943218118 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1dfc08f6d0 con 0x7f1e000ffdf0 2024-05-15T16:25:49.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.839+0000 7f1e049b0700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1dec06e200 0x7f1dec0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:49.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.839+0000 7f1e049b0700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1dec06e200 0x7f1dec0706c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e00197c60 tx=0x7f1df4005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:49.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:49.844+0000 7f1df27fc700 1 -- 172.21.15.49:0/1943218118 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1dfc05c4e0 con 0x7f1e000ffdf0 2024-05-15T16:25:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.061+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e00197fd0 con 0x7f1dec06e200 2024-05-15T16:25:50.063 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:49 smithi049 ceph-mon[112142]: pgmap v865: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:50.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.073+0000 7f1df27fc700 1 -- 172.21.15.49:0/1943218118 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1e00197fd0 con 0x7f1dec06e200 2024-05-15T16:25:50.074 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:25:50.074 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (29m) 6m ago 40m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:25:50.074 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (40m) 6m ago 40m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:25:50.074 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (39m) 6m ago 39m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:25:50.074 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (26m) 6m ago 40m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:25:50.074 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (26m) 6m ago 39m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:25:50.074 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (28m) 6m ago 40m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:25:50.074 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (22m) 6m ago 35m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (22m) 6m ago 34m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (22m) 6m ago 35m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (7m) 6m ago 34m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (31m) 6m ago 42m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (30m) 6m ago 39m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (27m) 6m ago 42m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (27m) 6m ago 39m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (29m) 6m ago 40m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (29m) 6m ago 39m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (26m) 6m ago 38m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (25m) 6m ago 38m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (24m) 6m ago 37m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (24m) 6m ago 37m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (23m) 6m ago 36m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:25:50.075 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (23m) 6m ago 36m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:25:50.076 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (29m) 6m ago 40m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.075+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1dec06e200 msgr2=0x7f1dec0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.075+0000 7f1e07415700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1dec06e200 0x7f1dec0706c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e00197c60 tx=0x7f1df4005cf0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.075+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 msgr2=0x7f1e0019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.075+0000 7f1e07415700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 0x7f1e0019c960 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00bd70 tx=0x7f1dfc00be50 comp rx=0 tx=0).stop 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.076+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 shutdown_connections 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.076+0000 7f1e07415700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1dec06e200 0x7f1dec0706c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.076+0000 7f1e07415700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e000ffdf0 0x7f1e0019c960 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.076+0000 7f1e07415700 1 --2- 172.21.15.49:0/1943218118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e001007a0 0x7f1e0019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.076+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 >> 172.21.15.49:0/1943218118 conn(0x7f1e000fb6a0 msgr2=0x7f1e000fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.076+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 shutdown_connections 2024-05-15T16:25:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.076+0000 7f1e07415700 1 -- 172.21.15.49:0/1943218118 wait complete. 2024-05-15T16:25:50.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.183+0000 7f3109ec2700 1 -- 172.21.15.49:0/298472574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3104100f00 msgr2=0x7f31041012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.183+0000 7f3109ec2700 1 --2- 172.21.15.49:0/298472574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3104100f00 0x7f31041012e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009910 tx=0x7f30f4009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:50.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.183+0000 7f3109ec2700 1 -- 172.21.15.49:0/298472574 shutdown_connections 2024-05-15T16:25:50.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.183+0000 7f3109ec2700 1 --2- 172.21.15.49:0/298472574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31041018b0 0x7f3104105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.183+0000 7f3109ec2700 1 --2- 172.21.15.49:0/298472574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3104100f00 0x7f31041012e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.183+0000 7f3109ec2700 1 -- 172.21.15.49:0/298472574 >> 172.21.15.49:0/298472574 conn(0x7f31040fc7d0 msgr2=0x7f31040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:50.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.183+0000 7f3109ec2700 1 -- 172.21.15.49:0/298472574 shutdown_connections 2024-05-15T16:25:50.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.183+0000 7f3109ec2700 1 -- 172.21.15.49:0/298472574 wait complete. 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.184+0000 7f3109ec2700 1 Processor -- start 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.184+0000 7f3109ec2700 1 -- start start 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.184+0000 7f3109ec2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3104100f00 0x7f310419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.184+0000 7f3109ec2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31041018b0 0x7f310419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.184+0000 7f3109ec2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310419d460 con 0x7f31041018b0 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.184+0000 7f3109ec2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3104196a90 con 0x7f3104100f00 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.184+0000 7f3102ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31041018b0 0x7f310419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.184+0000 7f3102ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31041018b0 0x7f310419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40616/0 (socket says 172.21.15.49:40616) 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.184+0000 7f3102ffd700 1 -- 172.21.15.49:0/4005435800 learned_addr learned my addr 172.21.15.49:0/4005435800 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f31037fe700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3104100f00 0x7f310419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f3102ffd700 1 -- 172.21.15.49:0/4005435800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3104100f00 msgr2=0x7f310419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f3102ffd700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3104100f00 0x7f310419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f3102ffd700 1 -- 172.21.15.49:0/4005435800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40095f0 con 0x7f31041018b0 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f31037fe700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3104100f00 0x7f310419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f3102ffd700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31041018b0 0x7f310419ce40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00d900 tx=0x7f30ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f3100ff9700 1 -- 172.21.15.49:0/4005435800 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec009900 con 0x7f31041018b0 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3104196d70 con 0x7f31041018b0 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f3100ff9700 1 -- 172.21.15.49:0/4005435800 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30ec010460 con 0x7f31041018b0 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f3100ff9700 1 -- 172.21.15.49:0/4005435800 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec00f5f0 con 0x7f31041018b0 2024-05-15T16:25:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.185+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31041972c0 con 0x7f31041018b0 2024-05-15T16:25:50.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.186+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31041092a0 con 0x7f31041018b0 2024-05-15T16:25:50.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.187+0000 7f3100ff9700 1 -- 172.21.15.49:0/4005435800 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f30ec0105d0 con 0x7f31041018b0 2024-05-15T16:25:50.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.187+0000 7f3100ff9700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30f006e200 0x7f30f00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:50.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.188+0000 7f3100ff9700 1 -- 172.21.15.49:0/4005435800 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f30ec08e020 con 0x7f31041018b0 2024-05-15T16:25:50.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.188+0000 7f31037fe700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30f006e200 0x7f30f00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:50.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.188+0000 7f31037fe700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30f006e200 0x7f30f00706c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f30f40059d0 tx=0x7f30f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:50.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.192+0000 7f3100ff9700 1 -- 172.21.15.49:0/4005435800 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f30ec05ae30 con 0x7f31041018b0 2024-05-15T16:25:50.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:49 smithi144 ceph-mon[86635]: pgmap v865: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:50.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.489+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f31041023e0 con 0x7f31041018b0 2024-05-15T16:25:50.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.490+0000 7f3100ff9700 1 -- 172.21.15.49:0/4005435800 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f30ec05a9c0 con 0x7f31041018b0 2024-05-15T16:25:50.491 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:25:50.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:25:50.491 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:25:50.492 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:25:50.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30f006e200 msgr2=0x7f30f00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30f006e200 0x7f30f00706c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f30f40059d0 tx=0x7f30f401a040 comp rx=0 tx=0).stop 2024-05-15T16:25:50.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31041018b0 msgr2=0x7f310419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31041018b0 0x7f310419ce40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00d900 tx=0x7f30ec00dcc0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 shutdown_connections 2024-05-15T16:25:50.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f30f006e200 0x7f30f00706c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3104100f00 0x7f310419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 --2- 172.21.15.49:0/4005435800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f31041018b0 0x7f310419ce40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 >> 172.21.15.49:0/4005435800 conn(0x7f31040fc7d0 msgr2=0x7f31040fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:50.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 shutdown_connections 2024-05-15T16:25:50.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.492+0000 7f3109ec2700 1 -- 172.21.15.49:0/4005435800 wait complete. 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.602+0000 7fde7b261700 1 -- 172.21.15.49:0/3542414426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74102780 msgr2=0x7fde7410ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.602+0000 7fde7b261700 1 --2- 172.21.15.49:0/3542414426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74102780 0x7fde7410ac70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009910 tx=0x7fde5c009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.602+0000 7fde7b261700 1 -- 172.21.15.49:0/3542414426 shutdown_connections 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.602+0000 7fde7b261700 1 --2- 172.21.15.49:0/3542414426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74102780 0x7fde7410ac70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.602+0000 7fde7b261700 1 --2- 172.21.15.49:0/3542414426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74101e60 0x7fde74102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.602+0000 7fde7b261700 1 -- 172.21.15.49:0/3542414426 >> 172.21.15.49:0/3542414426 conn(0x7fde740fb6d0 msgr2=0x7fde740fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.602+0000 7fde7b261700 1 -- 172.21.15.49:0/3542414426 shutdown_connections 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.602+0000 7fde7b261700 1 -- 172.21.15.49:0/3542414426 wait complete. 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde7b261700 1 Processor -- start 2024-05-15T16:25:50.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde7b261700 1 -- start start 2024-05-15T16:25:50.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde7b261700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74101e60 0x7fde7419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:50.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde7b261700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74102780 0x7fde7419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:50.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde7b261700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7419d480 con 0x7fde74101e60 2024-05-15T16:25:50.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde7b261700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde74196a60 con 0x7fde74102780 2024-05-15T16:25:50.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde78ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74101e60 0x7fde7419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:50.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde78ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74101e60 0x7fde7419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40630/0 (socket says 172.21.15.49:40630) 2024-05-15T16:25:50.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde78ffd700 1 -- 172.21.15.49:0/1323662825 learned_addr learned my addr 172.21.15.49:0/1323662825 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde78ffd700 1 -- 172.21.15.49:0/1323662825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74102780 msgr2=0x7fde7419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde73fff700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74102780 0x7fde7419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde78ffd700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74102780 0x7fde7419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.603+0000 7fde78ffd700 1 -- 172.21.15.49:0/1323662825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0095f0 con 0x7fde74101e60 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.604+0000 7fde78ffd700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74101e60 0x7fde7419c920 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fde640060b0 tx=0x7fde6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.604+0000 7fde71ffb700 1 -- 172.21.15.49:0/1323662825 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde6400dda0 con 0x7fde74101e60 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.604+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde74196d40 con 0x7fde74101e60 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.604+0000 7fde71ffb700 1 -- 172.21.15.49:0/1323662825 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde6400f040 con 0x7fde74101e60 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.604+0000 7fde71ffb700 1 -- 172.21.15.49:0/1323662825 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde64014790 con 0x7fde74101e60 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.604+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde74197290 con 0x7fde74101e60 2024-05-15T16:25:50.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.605+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde74105580 con 0x7fde74101e60 2024-05-15T16:25:50.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.606+0000 7fde71ffb700 1 -- 172.21.15.49:0/1323662825 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fde64014980 con 0x7fde74101e60 2024-05-15T16:25:50.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.606+0000 7fde71ffb700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde6006e200 0x7fde600706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:50.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.606+0000 7fde71ffb700 1 -- 172.21.15.49:0/1323662825 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fde6408dc30 con 0x7fde74101e60 2024-05-15T16:25:50.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.606+0000 7fde73fff700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde6006e200 0x7fde600706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:50.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.607+0000 7fde73fff700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde6006e200 0x7fde600706c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00b5c0 tx=0x7fde5c0052a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:50.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.611+0000 7fde71ffb700 1 -- 172.21.15.49:0/1323662825 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fde6405aaf0 con 0x7fde74101e60 2024-05-15T16:25:50.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.864+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fde74197f20 con 0x7fde74101e60 2024-05-15T16:25:50.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.865+0000 7fde71ffb700 1 -- 172.21.15.49:0/1323662825 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fde6401a760 con 0x7fde74101e60 2024-05-15T16:25:50.866 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:25:50.866 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:25:50.866 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:25:50.866 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:25:50.866 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:25:50.867 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:25:50.868 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:25:50.869 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:25:50.869 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:25:50.869 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:25:50.869 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:25:50.869 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:25:50.869 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:25:50.869 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde6006e200 msgr2=0x7fde600706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde6006e200 0x7fde600706c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00b5c0 tx=0x7fde5c0052a0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74101e60 msgr2=0x7fde7419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74101e60 0x7fde7419c920 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fde640060b0 tx=0x7fde6400d3b0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 shutdown_connections 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fde6006e200 0x7fde600706c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde74101e60 0x7fde7419c920 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 --2- 172.21.15.49:0/1323662825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74102780 0x7fde7419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 >> 172.21.15.49:0/1323662825 conn(0x7fde740fb6d0 msgr2=0x7fde74103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 shutdown_connections 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.867+0000 7fde7b261700 1 -- 172.21.15.49:0/1323662825 wait complete. 2024-05-15T16:25:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:25:50.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.978+0000 7f888f58d700 1 -- 172.21.15.49:0/2078851286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8888100770 msgr2=0x7f88881047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.978+0000 7f888f58d700 1 --2- 172.21.15.49:0/2078851286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8888100770 0x7f88881047c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8878009910 tx=0x7f8878009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:50.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.978+0000 7f888f58d700 1 -- 172.21.15.49:0/2078851286 shutdown_connections 2024-05-15T16:25:50.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.978+0000 7f888f58d700 1 --2- 172.21.15.49:0/2078851286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8888100770 0x7f88881047c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.978+0000 7f888f58d700 1 --2- 172.21.15.49:0/2078851286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88880ffdc0 0x7f88881001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.978+0000 7f888f58d700 1 -- 172.21.15.49:0/2078851286 >> 172.21.15.49:0/2078851286 conn(0x7f8888076800 msgr2=0x7f88880fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:50.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.978+0000 7f888f58d700 1 -- 172.21.15.49:0/2078851286 shutdown_connections 2024-05-15T16:25:50.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.978+0000 7f888f58d700 1 -- 172.21.15.49:0/2078851286 wait complete. 2024-05-15T16:25:50.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.979+0000 7f888f58d700 1 Processor -- start 2024-05-15T16:25:50.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.979+0000 7f888f58d700 1 -- start start 2024-05-15T16:25:50.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.979+0000 7f888f58d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88880ffdc0 0x7f888819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:50.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.979+0000 7f888f58d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8888100770 0x7f888819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:50.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.979+0000 7f888f58d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888819d3c0 con 0x7f88880ffdc0 2024-05-15T16:25:50.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.979+0000 7f888f58d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88881969f0 con 0x7f8888100770 2024-05-15T16:25:50.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.979+0000 7f888d329700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88880ffdc0 0x7f888819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.979+0000 7f888d329700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88880ffdc0 0x7f888819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40646/0 (socket says 172.21.15.49:40646) 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.979+0000 7f888d329700 1 -- 172.21.15.49:0/900889446 learned_addr learned my addr 172.21.15.49:0/900889446 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f888cb28700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8888100770 0x7f888819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f888d329700 1 -- 172.21.15.49:0/900889446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8888100770 msgr2=0x7f888819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f888d329700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8888100770 0x7f888819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f888d329700 1 -- 172.21.15.49:0/900889446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88780095f0 con 0x7f88880ffdc0 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f888cb28700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8888100770 0x7f888819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f888d329700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88880ffdc0 0x7f888819c860 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f88840060b0 tx=0x7f888400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f887e7fc700 1 -- 172.21.15.49:0/900889446 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f888400dda0 con 0x7f88880ffdc0 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f887e7fc700 1 -- 172.21.15.49:0/900889446 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f888400f040 con 0x7f88880ffdc0 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8888196cd0 con 0x7f88880ffdc0 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f887e7fc700 1 -- 172.21.15.49:0/900889446 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8884014790 con 0x7f88880ffdc0 2024-05-15T16:25:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.980+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8888197220 con 0x7f88880ffdc0 2024-05-15T16:25:50.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.981+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88881012a0 con 0x7f88880ffdc0 2024-05-15T16:25:50.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.982+0000 7f887e7fc700 1 -- 172.21.15.49:0/900889446 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f88840148f0 con 0x7f88880ffdc0 2024-05-15T16:25:50.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.982+0000 7f887e7fc700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e210 0x7f88740706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:50.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.982+0000 7f887e7fc700 1 -- 172.21.15.49:0/900889446 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f888408db40 con 0x7f88880ffdc0 2024-05-15T16:25:50.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.982+0000 7f888cb28700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e210 0x7f88740706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:50.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.983+0000 7f888cb28700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e210 0x7f88740706d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8888197b60 tx=0x7f887801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:50.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:50.987+0000 7f887e7fc700 1 -- 172.21.15.49:0/900889446 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f888405aa80 con 0x7f88880ffdc0 2024-05-15T16:25:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:50 smithi049 ceph-mon[112142]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:50 smithi049 ceph-mon[112142]: from='client.35582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:50 smithi049 ceph-mon[112142]: from='client.35586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:25:51.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:50 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4005435800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:25:51.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:50 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1323662825' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:25:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:50 smithi144 ceph-mon[86635]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:50 smithi144 ceph-mon[86635]: from='client.35582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:50 smithi144 ceph-mon[86635]: from='client.35586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:25:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:50 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4005435800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:25:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:50 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1323662825' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:25:51.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.213+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f888810ce20 con 0x7f887406e210 2024-05-15T16:25:51.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.214+0000 7f887e7fc700 1 -- 172.21.15.49:0/900889446 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f888810ce20 con 0x7f887406e210 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:25:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:25:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:25:51.217 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:25:51.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e210 msgr2=0x7f88740706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:51.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e210 0x7f88740706d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8888197b60 tx=0x7f887801a040 comp rx=0 tx=0).stop 2024-05-15T16:25:51.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88880ffdc0 msgr2=0x7f888819c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:51.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88880ffdc0 0x7f888819c860 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f88840060b0 tx=0x7f888400d3b0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 shutdown_connections 2024-05-15T16:25:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e210 0x7f88740706d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88880ffdc0 0x7f888819c860 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 --2- 172.21.15.49:0/900889446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8888100770 0x7f888819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 >> 172.21.15.49:0/900889446 conn(0x7f8888076800 msgr2=0x7f8888068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 shutdown_connections 2024-05-15T16:25:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.217+0000 7f888f58d700 1 -- 172.21.15.49:0/900889446 wait complete. 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.325+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1101723902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918100ee0 msgr2=0x7fc9181012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.325+0000 7fc91f0a8700 1 --2- 172.21.15.49:0/1101723902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918100ee0 0x7fc9181012c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009910 tx=0x7fc90c009c20 comp rx=0 tx=0).stop 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.325+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1101723902 shutdown_connections 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.325+0000 7fc91f0a8700 1 --2- 172.21.15.49:0/1101723902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc918101890 0x7fc9181058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.325+0000 7fc91f0a8700 1 --2- 172.21.15.49:0/1101723902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918100ee0 0x7fc9181012c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.325+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1101723902 >> 172.21.15.49:0/1101723902 conn(0x7fc9180fc790 msgr2=0x7fc9180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.326+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1101723902 shutdown_connections 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.326+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1101723902 wait complete. 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.326+0000 7fc91f0a8700 1 Processor -- start 2024-05-15T16:25:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.326+0000 7fc91f0a8700 1 -- start start 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.326+0000 7fc91f0a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc918100ee0 0x7fc918196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.326+0000 7fc91f0a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918101890 0x7fc918197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.326+0000 7fc91f0a8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9181978c0 con 0x7fc918101890 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.326+0000 7fc91f0a8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91819a530 con 0x7fc918100ee0 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.326+0000 7fc917fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918101890 0x7fc918197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc917fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918101890 0x7fc918197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40678/0 (socket says 172.21.15.49:40678) 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc917fff700 1 -- 172.21.15.49:0/1447110306 learned_addr learned my addr 172.21.15.49:0/1447110306 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc91ce44700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc918100ee0 0x7fc918196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc917fff700 1 -- 172.21.15.49:0/1447110306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc918100ee0 msgr2=0x7fc918196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc917fff700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc918100ee0 0x7fc918196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc917fff700 1 -- 172.21.15.49:0/1447110306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c0095f0 con 0x7fc918101890 2024-05-15T16:25:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc91ce44700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc918100ee0 0x7fc918196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:25:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc917fff700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918101890 0x7fc918197230 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc90400b480 tx=0x7fc90400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc915ffb700 1 -- 172.21.15.49:0/1447110306 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9040118c0 con 0x7fc918101890 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc915ffb700 1 -- 172.21.15.49:0/1447110306 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc904010380 con 0x7fc918101890 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91819a810 con 0x7fc918101890 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc915ffb700 1 -- 172.21.15.49:0/1447110306 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc90400f4f0 con 0x7fc918101890 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.327+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91819ad60 con 0x7fc918101890 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.328+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc918109280 con 0x7fc918101890 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.329+0000 7fc915ffb700 1 -- 172.21.15.49:0/1447110306 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc9040104f0 con 0x7fc918101890 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.329+0000 7fc915ffb700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc90806e200 0x7fc9080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.330+0000 7fc915ffb700 1 -- 172.21.15.49:0/1447110306 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc90408dd20 con 0x7fc918101890 2024-05-15T16:25:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.330+0000 7fc91ce44700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc90806e200 0x7fc9080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:25:51.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.330+0000 7fc91ce44700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc90806e200 0x7fc9080706c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0059d0 tx=0x7fc90c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:25:51.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.334+0000 7fc915ffb700 1 -- 172.21.15.49:0/1447110306 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc904057080 con 0x7fc918101890 2024-05-15T16:25:51.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.627+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc9181023c0 con 0x7fc918101890 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.627+0000 7fc915ffb700 1 -- 172.21.15.49:0/1447110306 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fc90405a740 con 0x7fc918101890 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:25:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:25:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:25:51.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc90806e200 msgr2=0x7fc9080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:51.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc90806e200 0x7fc9080706c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0059d0 tx=0x7fc90c01a040 comp rx=0 tx=0).stop 2024-05-15T16:25:51.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918101890 msgr2=0x7fc918197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:25:51.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918101890 0x7fc918197230 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc90400b480 tx=0x7fc90400b840 comp rx=0 tx=0).stop 2024-05-15T16:25:51.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 shutdown_connections 2024-05-15T16:25:51.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc90806e200 0x7fc9080706c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc918100ee0 0x7fc918196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 --2- 172.21.15.49:0/1447110306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc918101890 0x7fc918197230 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:25:51.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 >> 172.21.15.49:0/1447110306 conn(0x7fc9180fc790 msgr2=0x7fc9180fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:25:51.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 shutdown_connections 2024-05-15T16:25:51.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:25:51.630+0000 7fc91f0a8700 1 -- 172.21.15.49:0/1447110306 wait complete. 2024-05-15T16:25:52.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:51 smithi144 ceph-mon[86635]: pgmap v866: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:51 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1447110306' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:25:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:51 smithi049 ceph-mon[112142]: pgmap v866: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:51 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1447110306' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:25:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:52 smithi144 ceph-mon[86635]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:52 smithi049 ceph-mon[112142]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:25:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:53 smithi144 ceph-mon[86635]: pgmap v867: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:53 smithi049 ceph-mon[112142]: pgmap v867: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:55 smithi144 ceph-mon[86635]: pgmap v868: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:55 smithi049 ceph-mon[112142]: pgmap v868: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:57 smithi144 ceph-mon[86635]: pgmap v869: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:25:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:57 smithi049 ceph-mon[112142]: pgmap v869: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:00.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:25:59 smithi144 ceph-mon[86635]: pgmap v870: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:25:59 smithi049 ceph-mon[112142]: pgmap v870: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:02.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:01 smithi144 ceph-mon[86635]: pgmap v871: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:01 smithi049 ceph-mon[112142]: pgmap v871: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:03 smithi144 ceph-mon[86635]: pgmap v872: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:03 smithi049 ceph-mon[112142]: pgmap v872: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:05 smithi144 ceph-mon[86635]: pgmap v873: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:26:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:05 smithi049 ceph-mon[112142]: pgmap v873: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:26:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:07 smithi144 ceph-mon[86635]: pgmap v874: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:07 smithi049 ceph-mon[112142]: pgmap v874: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:10.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:09 smithi144 ceph-mon[86635]: pgmap v875: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:09 smithi049 ceph-mon[112142]: pgmap v875: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:11 smithi049 ceph-mon[112142]: pgmap v876: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:11 smithi144 ceph-mon[86635]: pgmap v876: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:13 smithi049 ceph-mon[112142]: pgmap v877: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:13 smithi144 ceph-mon[86635]: pgmap v877: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:15 smithi144 ceph-mon[86635]: pgmap v878: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:15 smithi049 ceph-mon[112142]: pgmap v878: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:18.315 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:17 smithi144 ceph-mon[86635]: pgmap v879: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:17 smithi049 ceph-mon[112142]: pgmap v879: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:19 smithi144 ceph-mon[86635]: pgmap v880: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:19 smithi049 ceph-mon[112142]: pgmap v880: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:26:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:26:21.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.750+0000 7f5fa4173700 1 -- 172.21.15.49:0/659560526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c101e70 msgr2=0x7f5f9c102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.750+0000 7f5fa4173700 1 --2- 172.21.15.49:0/659560526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c101e70 0x7f5f9c102250 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009910 tx=0x7f5f98009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.750+0000 7f5fa4173700 1 -- 172.21.15.49:0/659560526 shutdown_connections 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.750+0000 7f5fa4173700 1 --2- 172.21.15.49:0/659560526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f9c102790 0x7f5f9c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.750+0000 7f5fa4173700 1 --2- 172.21.15.49:0/659560526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c101e70 0x7f5f9c102250 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.750+0000 7f5fa4173700 1 -- 172.21.15.49:0/659560526 >> 172.21.15.49:0/659560526 conn(0x7f5f9c0fb700 msgr2=0x7f5f9c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.750+0000 7f5fa4173700 1 -- 172.21.15.49:0/659560526 shutdown_connections 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.750+0000 7f5fa4173700 1 -- 172.21.15.49:0/659560526 wait complete. 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa4173700 1 Processor -- start 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa4173700 1 -- start start 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa4173700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f9c101e70 0x7f5f9c0ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:21.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa4173700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c102790 0x7f5f9c100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa4173700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c100550 con 0x7f5f9c102790 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa4173700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c100690 con 0x7f5f9c101e70 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa170e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c102790 0x7f5f9c100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa170e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c102790 0x7f5f9c100010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46606/0 (socket says 172.21.15.49:46606) 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa170e700 1 -- 172.21.15.49:0/952832155 learned_addr learned my addr 172.21.15.49:0/952832155 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.751+0000 7f5fa1f0f700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f9c101e70 0x7f5f9c0ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5fa170e700 1 -- 172.21.15.49:0/952832155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f9c101e70 msgr2=0x7f5f9c0ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5fa170e700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f9c101e70 0x7f5f9c0ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5fa170e700 1 -- 172.21.15.49:0/952832155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f980095f0 con 0x7f5f9c102790 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5fa1f0f700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f9c101e70 0x7f5f9c0ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5fa170e700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c102790 0x7f5f9c100010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00b3d0 tx=0x7f5f8c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:21.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5f92ffd700 1 -- 172.21.15.49:0/952832155 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f8c011800 con 0x7f5f9c102790 2024-05-15T16:26:21.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5f92ffd700 1 -- 172.21.15.49:0/952832155 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f8c011e20 con 0x7f5f9c102790 2024-05-15T16:26:21.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c1a2ef0 con 0x7f5f9c102790 2024-05-15T16:26:21.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5f92ffd700 1 -- 172.21.15.49:0/952832155 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f8c010ea0 con 0x7f5f9c102790 2024-05-15T16:26:21.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.752+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c1a3250 con 0x7f5f9c102790 2024-05-15T16:26:21.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.753+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c069ab0 con 0x7f5f9c102790 2024-05-15T16:26:21.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.754+0000 7f5f92ffd700 1 -- 172.21.15.49:0/952832155 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5f8c010380 con 0x7f5f9c102790 2024-05-15T16:26:21.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.754+0000 7f5f92ffd700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f8806e200 0x7f5f880706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:21.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.754+0000 7f5f92ffd700 1 -- 172.21.15.49:0/952832155 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f8c08dc60 con 0x7f5f9c102790 2024-05-15T16:26:21.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.754+0000 7f5fa1f0f700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f8806e200 0x7f5f880706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:21.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.755+0000 7f5fa1f0f700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f8806e200 0x7f5f880706c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5f98006010 tx=0x7f5f9800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.759+0000 7f5f92ffd700 1 -- 172.21.15.49:0/952832155 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5f8c056f40 con 0x7f5f9c102790 2024-05-15T16:26:21.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.985+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f9c10df70 con 0x7f5f8806e200 2024-05-15T16:26:21.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.986+0000 7f5f92ffd700 1 -- 172.21.15.49:0/952832155 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5f9c10df70 con 0x7f5f8806e200 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.988+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f8806e200 msgr2=0x7f5f880706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.988+0000 7f5fa4173700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f8806e200 0x7f5f880706c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5f98006010 tx=0x7f5f9800b540 comp rx=0 tx=0).stop 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.988+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c102790 msgr2=0x7f5f9c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.988+0000 7f5fa4173700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c102790 0x7f5f9c100010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00b3d0 tx=0x7f5f8c00b790 comp rx=0 tx=0).stop 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.989+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 shutdown_connections 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.989+0000 7f5fa4173700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5f8806e200 0x7f5f880706c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.989+0000 7f5fa4173700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f9c101e70 0x7f5f9c0ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.989+0000 7f5fa4173700 1 --2- 172.21.15.49:0/952832155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f9c102790 0x7f5f9c100010 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.989+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 >> 172.21.15.49:0/952832155 conn(0x7f5f9c0fb700 msgr2=0x7f5f9c103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.989+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 shutdown_connections 2024-05-15T16:26:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:21.989+0000 7f5fa4173700 1 -- 172.21.15.49:0/952832155 wait complete. 2024-05-15T16:26:22.000 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:26:22.001 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:21 smithi049 ceph-mon[112142]: pgmap v881: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.097+0000 7fc84d715700 1 -- 172.21.15.49:0/580138491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480fff50 msgr2=0x7fc848103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.097+0000 7fc84d715700 1 --2- 172.21.15.49:0/580138491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480fff50 0x7fc848103fa0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00aa20 tx=0x7fc83c00ad30 comp rx=0 tx=0).stop 2024-05-15T16:26:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.098+0000 7fc84d715700 1 -- 172.21.15.49:0/580138491 shutdown_connections 2024-05-15T16:26:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.098+0000 7fc84d715700 1 --2- 172.21.15.49:0/580138491 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480fff50 0x7fc848103fa0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.098+0000 7fc84d715700 1 --2- 172.21.15.49:0/580138491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8480ff5a0 0x7fc8480ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.098+0000 7fc84d715700 1 -- 172.21.15.49:0/580138491 >> 172.21.15.49:0/580138491 conn(0x7fc848076000 msgr2=0x7fc8480fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.098+0000 7fc84d715700 1 -- 172.21.15.49:0/580138491 shutdown_connections 2024-05-15T16:26:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.098+0000 7fc84d715700 1 -- 172.21.15.49:0/580138491 wait complete. 2024-05-15T16:26:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.098+0000 7fc84d715700 1 Processor -- start 2024-05-15T16:26:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.098+0000 7fc84d715700 1 -- start start 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc84d715700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480ff5a0 0x7fc8480fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc84d715700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8480fff50 0x7fc8480fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc84d715700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480fbec0 con 0x7fc8480ff5a0 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc84d715700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480fc000 con 0x7fc8480fff50 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc846ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480ff5a0 0x7fc8480fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc846ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480ff5a0 0x7fc8480fb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46618/0 (socket says 172.21.15.49:46618) 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc846ffd700 1 -- 172.21.15.49:0/3552346122 learned_addr learned my addr 172.21.15.49:0/3552346122 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc8467fc700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8480fff50 0x7fc8480fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc846ffd700 1 -- 172.21.15.49:0/3552346122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8480fff50 msgr2=0x7fc8480fb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc846ffd700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8480fff50 0x7fc8480fb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc846ffd700 1 -- 172.21.15.49:0/3552346122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c00a700 con 0x7fc8480ff5a0 2024-05-15T16:26:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc8467fc700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8480fff50 0x7fc8480fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:26:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc846ffd700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480ff5a0 0x7fc8480fb440 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c8a0 tx=0x7fc83800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc82ffff700 1 -- 172.21.15.49:0/3552346122 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83800cea0 con 0x7fc8480ff5a0 2024-05-15T16:26:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc82ffff700 1 -- 172.21.15.49:0/3552346122 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc838004830 con 0x7fc8480ff5a0 2024-05-15T16:26:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8481a2dd0 con 0x7fc8480ff5a0 2024-05-15T16:26:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc82ffff700 1 -- 172.21.15.49:0/3552346122 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8380055b0 con 0x7fc8480ff5a0 2024-05-15T16:26:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.099+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8481a3130 con 0x7fc8480ff5a0 2024-05-15T16:26:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.101+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc848100a80 con 0x7fc8480ff5a0 2024-05-15T16:26:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.101+0000 7fc82ffff700 1 -- 172.21.15.49:0/3552346122 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc8380049a0 con 0x7fc8480ff5a0 2024-05-15T16:26:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.102+0000 7fc82ffff700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc83006e200 0x7fc8300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.102+0000 7fc82ffff700 1 -- 172.21.15.49:0/3552346122 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc83808de40 con 0x7fc8480ff5a0 2024-05-15T16:26:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.102+0000 7fc8467fc700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc83006e200 0x7fc8300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.102+0000 7fc8467fc700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc83006e200 0x7fc8300706c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00afd0 tx=0x7fc83c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:22.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.106+0000 7fc82ffff700 1 -- 172.21.15.49:0/3552346122 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc838057120 con 0x7fc8480ff5a0 2024-05-15T16:26:22.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.336+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc84810bdc0 con 0x7fc83006e200 2024-05-15T16:26:22.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.337+0000 7fc82ffff700 1 -- 172.21.15.49:0/3552346122 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc84810bdc0 con 0x7fc83006e200 2024-05-15T16:26:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc83006e200 msgr2=0x7fc8300706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc83006e200 0x7fc8300706c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00afd0 tx=0x7fc83c005dc0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480ff5a0 msgr2=0x7fc8480fb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480ff5a0 0x7fc8480fb440 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c8a0 tx=0x7fc83800cc60 comp rx=0 tx=0).stop 2024-05-15T16:26:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 shutdown_connections 2024-05-15T16:26:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc83006e200 0x7fc8300706c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8480ff5a0 0x7fc8480fb440 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 --2- 172.21.15.49:0/3552346122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8480fff50 0x7fc8480fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 >> 172.21.15.49:0/3552346122 conn(0x7fc848076000 msgr2=0x7fc848109850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:22.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 shutdown_connections 2024-05-15T16:26:22.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.340+0000 7fc84d715700 1 -- 172.21.15.49:0/3552346122 wait complete. 2024-05-15T16:26:22.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:21 smithi144 ceph-mon[86635]: pgmap v881: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:22.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.448+0000 7fa480285700 1 -- 172.21.15.49:0/2967098904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 msgr2=0x7fa4781012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.448+0000 7fa480285700 1 --2- 172.21.15.49:0/2967098904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 0x7fa4781012e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa474009910 tx=0x7fa474009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:22.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.448+0000 7fa480285700 1 -- 172.21.15.49:0/2967098904 shutdown_connections 2024-05-15T16:26:22.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.448+0000 7fa480285700 1 --2- 172.21.15.49:0/2967098904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4781018b0 0x7fa478105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.448+0000 7fa480285700 1 --2- 172.21.15.49:0/2967098904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 0x7fa4781012e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.448+0000 7fa480285700 1 -- 172.21.15.49:0/2967098904 >> 172.21.15.49:0/2967098904 conn(0x7fa4780fc7d0 msgr2=0x7fa4780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:22.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.448+0000 7fa480285700 1 -- 172.21.15.49:0/2967098904 shutdown_connections 2024-05-15T16:26:22.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.448+0000 7fa480285700 1 -- 172.21.15.49:0/2967098904 wait complete. 2024-05-15T16:26:22.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa480285700 1 Processor -- start 2024-05-15T16:26:22.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa480285700 1 -- start start 2024-05-15T16:26:22.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa480285700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 0x7fa47819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:22.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa480285700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4781018b0 0x7fa47819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:22.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa480285700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47819d460 con 0x7fa478100f00 2024-05-15T16:26:22.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa480285700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa478196a90 con 0x7fa4781018b0 2024-05-15T16:26:22.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa47e021700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 0x7fa47819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:22.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa47e021700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 0x7fa47819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46644/0 (socket says 172.21.15.49:46644) 2024-05-15T16:26:22.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa47e021700 1 -- 172.21.15.49:0/3785960956 learned_addr learned my addr 172.21.15.49:0/3785960956 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa47e021700 1 -- 172.21.15.49:0/3785960956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4781018b0 msgr2=0x7fa47819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa47d820700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4781018b0 0x7fa47819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa47e021700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4781018b0 0x7fa47819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa47e021700 1 -- 172.21.15.49:0/3785960956 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740095f0 con 0x7fa478100f00 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.449+0000 7fa47d820700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4781018b0 0x7fa47819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.450+0000 7fa47e021700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 0x7fa47819c900 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa474000c00 tx=0x7fa47400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.450+0000 7fa46f7fe700 1 -- 172.21.15.49:0/3785960956 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47401d070 con 0x7fa478100f00 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.450+0000 7fa46f7fe700 1 -- 172.21.15.49:0/3785960956 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4740056e0 con 0x7fa478100f00 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.450+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa478196d10 con 0x7fa478100f00 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.450+0000 7fa46f7fe700 1 -- 172.21.15.49:0/3785960956 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47400fe80 con 0x7fa478100f00 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.450+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa478197200 con 0x7fa478100f00 2024-05-15T16:26:22.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.451+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4781092a0 con 0x7fa478100f00 2024-05-15T16:26:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.451+0000 7fa46f7fe700 1 -- 172.21.15.49:0/3785960956 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa47400f6f0 con 0x7fa478100f00 2024-05-15T16:26:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.452+0000 7fa46f7fe700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa46406e210 0x7fa4640706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.452+0000 7fa46f7fe700 1 -- 172.21.15.49:0/3785960956 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa47408f5f0 con 0x7fa478100f00 2024-05-15T16:26:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.452+0000 7fa47d820700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa46406e210 0x7fa4640706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:22.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.452+0000 7fa47d820700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa46406e210 0x7fa4640706d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa468005fd0 tx=0x7fa468005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:22.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.457+0000 7fa46f7fe700 1 -- 172.21.15.49:0/3785960956 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa47405c400 con 0x7fa478100f00 2024-05-15T16:26:22.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.679+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa478197f00 con 0x7fa46406e210 2024-05-15T16:26:22.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.690+0000 7fa46f7fe700 1 -- 172.21.15.49:0/3785960956 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa478197f00 con 0x7fa46406e210 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (29m) 7m ago 41m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (41m) 7m ago 41m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (40m) 7m ago 40m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (27m) 7m ago 41m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (26m) 7m ago 40m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (29m) 7m ago 40m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (23m) 7m ago 35m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (22m) 7m ago 35m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (22m) 7m ago 35m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (7m) 7m ago 35m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (31m) 7m ago 42m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (30m) 7m ago 40m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:26:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (28m) 7m ago 42m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (27m) 7m ago 40m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (30m) 7m ago 41m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (30m) 7m ago 40m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (26m) 7m ago 39m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (25m) 7m ago 38m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (25m) 7m ago 38m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (24m) 7m ago 37m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (24m) 7m ago 37m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (23m) 7m ago 37m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:26:22.693 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (29m) 7m ago 40m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:26:22.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa46406e210 msgr2=0x7fa4640706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa46406e210 0x7fa4640706d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa468005fd0 tx=0x7fa468005ee0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 msgr2=0x7fa47819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 0x7fa47819c900 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa474000c00 tx=0x7fa47400be20 comp rx=0 tx=0).stop 2024-05-15T16:26:22.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 shutdown_connections 2024-05-15T16:26:22.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa46406e210 0x7fa4640706d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa478100f00 0x7fa47819c900 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 --2- 172.21.15.49:0/3785960956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4781018b0 0x7fa47819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 >> 172.21.15.49:0/3785960956 conn(0x7fa4780fc7d0 msgr2=0x7fa4780fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:22.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 shutdown_connections 2024-05-15T16:26:22.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.693+0000 7fa480285700 1 -- 172.21.15.49:0/3785960956 wait complete. 2024-05-15T16:26:22.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.800+0000 7f6b7d834700 1 -- 172.21.15.49:0/1208118028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 msgr2=0x7f6b781047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.800+0000 7f6b7d834700 1 --2- 172.21.15.49:0/1208118028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 0x7f6b781047f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009910 tx=0x7f6b60009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:22.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.800+0000 7f6b7d834700 1 -- 172.21.15.49:0/1208118028 shutdown_connections 2024-05-15T16:26:22.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.800+0000 7f6b7d834700 1 --2- 172.21.15.49:0/1208118028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 0x7f6b781047f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.800+0000 7f6b7d834700 1 --2- 172.21.15.49:0/1208118028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b780ffdf0 0x7f6b781001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.800+0000 7f6b7d834700 1 -- 172.21.15.49:0/1208118028 >> 172.21.15.49:0/1208118028 conn(0x7f6b780fb6a0 msgr2=0x7f6b780fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:22.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.800+0000 7f6b7d834700 1 -- 172.21.15.49:0/1208118028 shutdown_connections 2024-05-15T16:26:22.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.800+0000 7f6b7d834700 1 -- 172.21.15.49:0/1208118028 wait complete. 2024-05-15T16:26:22.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b7d834700 1 Processor -- start 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b7d834700 1 -- start start 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b7d834700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b780ffdf0 0x7f6b7819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b7d834700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 0x7f6b7819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b7d834700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78196ab0 con 0x7f6b781007a0 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b7d834700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78196c20 con 0x7f6b780ffdf0 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b767fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 0x7f6b7819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b767fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 0x7f6b7819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46664/0 (socket says 172.21.15.49:46664) 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b767fc700 1 -- 172.21.15.49:0/2187301353 learned_addr learned my addr 172.21.15.49:0/2187301353 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.801+0000 7f6b76ffd700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b780ffdf0 0x7f6b7819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b767fc700 1 -- 172.21.15.49:0/2187301353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b780ffdf0 msgr2=0x7f6b7819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:22.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b767fc700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b780ffdf0 0x7f6b7819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:22.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b767fc700 1 -- 172.21.15.49:0/2187301353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600095f0 con 0x7f6b781007a0 2024-05-15T16:26:22.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b76ffd700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b780ffdf0 0x7f6b7819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:26:22.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b767fc700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 0x7f6b7819cf40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6b600092e0 tx=0x7f6b6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:22.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b7c832700 1 -- 172.21.15.49:0/2187301353 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b6001d070 con 0x7f6b781007a0 2024-05-15T16:26:22.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b7c832700 1 -- 172.21.15.49:0/2187301353 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b6000f460 con 0x7f6b781007a0 2024-05-15T16:26:22.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b78196ea0 con 0x7f6b781007a0 2024-05-15T16:26:22.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b7c832700 1 -- 172.21.15.49:0/2187301353 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b60017610 con 0x7f6b781007a0 2024-05-15T16:26:22.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.802+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b78197390 con 0x7f6b781007a0 2024-05-15T16:26:22.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.803+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b781012d0 con 0x7f6b781007a0 2024-05-15T16:26:22.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.804+0000 7f6b7c832700 1 -- 172.21.15.49:0/2187301353 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6b6000f5d0 con 0x7f6b781007a0 2024-05-15T16:26:22.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.804+0000 7f6b7c832700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b6406e210 0x7f6b640706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:22.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.805+0000 7f6b76ffd700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b6406e210 0x7f6b640706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:22.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.805+0000 7f6b7c832700 1 -- 172.21.15.49:0/2187301353 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6b6008f5c0 con 0x7f6b781007a0 2024-05-15T16:26:22.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.805+0000 7f6b76ffd700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b6406e210 0x7f6b640706d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6b680060b0 tx=0x7f6b680076a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:22.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:22.809+0000 7f6b7c832700 1 -- 172.21.15.49:0/2187301353 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6b60058920 con 0x7f6b781007a0 2024-05-15T16:26:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.106+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b78197f90 con 0x7f6b781007a0 2024-05-15T16:26:23.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.106+0000 7f6b7c832700 1 -- 172.21.15.49:0/2187301353 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6b6005bfe0 con 0x7f6b781007a0 2024-05-15T16:26:23.107 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:26:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:26:23.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:26:23.109 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.108+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b6406e210 msgr2=0x7f6b640706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.108+0000 7f6b7d834700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b6406e210 0x7f6b640706d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6b680060b0 tx=0x7f6b680076a0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.108+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 msgr2=0x7f6b7819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.108+0000 7f6b7d834700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 0x7f6b7819cf40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6b600092e0 tx=0x7f6b6000be20 comp rx=0 tx=0).stop 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.108+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 shutdown_connections 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.108+0000 7f6b7d834700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6b6406e210 0x7f6b640706d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.108+0000 7f6b7d834700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b780ffdf0 0x7f6b7819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.108+0000 7f6b7d834700 1 --2- 172.21.15.49:0/2187301353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b781007a0 0x7f6b7819cf40 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.108+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 >> 172.21.15.49:0/2187301353 conn(0x7f6b780fb6a0 msgr2=0x7f6b78069590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.109+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 shutdown_connections 2024-05-15T16:26:23.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.109+0000 7f6b7d834700 1 -- 172.21.15.49:0/2187301353 wait complete. 2024-05-15T16:26:23.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.217+0000 7f00700b5700 1 -- 172.21.15.49:0/2328188376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100ed0 msgr2=0x7f0068104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.217+0000 7f00700b5700 1 --2- 172.21.15.49:0/2328188376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100ed0 0x7f0068104a00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0058009910 tx=0x7f0058009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.217+0000 7f00700b5700 1 -- 172.21.15.49:0/2328188376 shutdown_connections 2024-05-15T16:26:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.217+0000 7f00700b5700 1 --2- 172.21.15.49:0/2328188376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100ed0 0x7f0068104a00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.217+0000 7f00700b5700 1 --2- 172.21.15.49:0/2328188376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0068100520 0x7f0068100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.217+0000 7f00700b5700 1 -- 172.21.15.49:0/2328188376 >> 172.21.15.49:0/2328188376 conn(0x7f0068075ff0 msgr2=0x7f00680fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.217+0000 7f00700b5700 1 -- 172.21.15.49:0/2328188376 shutdown_connections 2024-05-15T16:26:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.217+0000 7f00700b5700 1 -- 172.21.15.49:0/2328188376 wait complete. 2024-05-15T16:26:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f00700b5700 1 Processor -- start 2024-05-15T16:26:23.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f00700b5700 1 -- start start 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f00700b5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100520 0x7f0068197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f00700b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0068100ed0 0x7f0068198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f00700b5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068198640 con 0x7f0068100520 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f00700b5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068198780 con 0x7f0068100ed0 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f006de51700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100520 0x7f0068197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f006de51700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100520 0x7f0068197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46684/0 (socket says 172.21.15.49:46684) 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f006de51700 1 -- 172.21.15.49:0/3150359637 learned_addr learned my addr 172.21.15.49:0/3150359637 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f006d650700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0068100ed0 0x7f0068198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f006de51700 1 -- 172.21.15.49:0/3150359637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0068100ed0 msgr2=0x7f0068198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f006de51700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0068100ed0 0x7f0068198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.218+0000 7f006de51700 1 -- 172.21.15.49:0/3150359637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00580095f0 con 0x7f0068100520 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.219+0000 7f006de51700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100520 0x7f0068197bc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f006400c930 tx=0x7f006400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:23.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.219+0000 7f005effd700 1 -- 172.21.15.49:0/3150359637 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0064007ad0 con 0x7f0068100520 2024-05-15T16:26:23.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.219+0000 7f005effd700 1 -- 172.21.15.49:0/3150359637 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f006400ce80 con 0x7f0068100520 2024-05-15T16:26:23.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.219+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00681a0c30 con 0x7f0068100520 2024-05-15T16:26:23.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.219+0000 7f005effd700 1 -- 172.21.15.49:0/3150359637 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0064018630 con 0x7f0068100520 2024-05-15T16:26:23.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.219+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00681a1040 con 0x7f0068100520 2024-05-15T16:26:23.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.220+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0068108400 con 0x7f0068100520 2024-05-15T16:26:23.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.220+0000 7f005effd700 1 -- 172.21.15.49:0/3150359637 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0064007c30 con 0x7f0068100520 2024-05-15T16:26:23.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.221+0000 7f005effd700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f005406e200 0x7f00540706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:23.223 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:22 smithi049 ceph-mon[112142]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:23.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.221+0000 7f005effd700 1 -- 172.21.15.49:0/3150359637 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f006408e1e0 con 0x7f0068100520 2024-05-15T16:26:23.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.221+0000 7f006d650700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f005406e200 0x7f00540706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:23.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.221+0000 7f006d650700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f005406e200 0x7f00540706c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f00680ffa00 tx=0x7f005801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:23.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.226+0000 7f005effd700 1 -- 172.21.15.49:0/3150359637 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f00640574c0 con 0x7f0068100520 2024-05-15T16:26:23.368 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:22 smithi144 ceph-mon[86635]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.484+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0068101570 con 0x7f0068100520 2024-05-15T16:26:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.484+0000 7f005effd700 1 -- 172.21.15.49:0/3150359637 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f006405ab80 con 0x7f0068100520 2024-05-15T16:26:23.485 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:26:23.485 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:26:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:26:23.487 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:26:23.488 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:26:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f005406e200 msgr2=0x7f00540706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f005406e200 0x7f00540706c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f00680ffa00 tx=0x7f005801a040 comp rx=0 tx=0).stop 2024-05-15T16:26:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100520 msgr2=0x7f0068197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100520 0x7f0068197bc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f006400c930 tx=0x7f006400cc40 comp rx=0 tx=0).stop 2024-05-15T16:26:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 shutdown_connections 2024-05-15T16:26:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f005406e200 0x7f00540706c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0068100520 0x7f0068197bc0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 --2- 172.21.15.49:0/3150359637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0068100ed0 0x7f0068198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 >> 172.21.15.49:0/3150359637 conn(0x7f0068075ff0 msgr2=0x7f0068103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:23.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 shutdown_connections 2024-05-15T16:26:23.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.486+0000 7f00700b5700 1 -- 172.21.15.49:0/3150359637 wait complete. 2024-05-15T16:26:23.490 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.596+0000 7fdfdc789700 1 -- 172.21.15.49:0/3378789754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 msgr2=0x7fdfd4105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.596+0000 7fdfdc789700 1 --2- 172.21.15.49:0/3378789754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 0x7fdfd4105900 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009910 tx=0x7fdfc4009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.596+0000 7fdfdc789700 1 -- 172.21.15.49:0/3378789754 shutdown_connections 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.596+0000 7fdfdc789700 1 --2- 172.21.15.49:0/3378789754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 0x7fdfd4105900 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.596+0000 7fdfdc789700 1 --2- 172.21.15.49:0/3378789754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4100f00 0x7fdfd41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.596+0000 7fdfdc789700 1 -- 172.21.15.49:0/3378789754 >> 172.21.15.49:0/3378789754 conn(0x7fdfd40fc7d0 msgr2=0x7fdfd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.596+0000 7fdfdc789700 1 -- 172.21.15.49:0/3378789754 shutdown_connections 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.596+0000 7fdfdc789700 1 -- 172.21.15.49:0/3378789754 wait complete. 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfdc789700 1 Processor -- start 2024-05-15T16:26:23.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfdc789700 1 -- start start 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfdc789700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4100f00 0x7fdfd419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfdc789700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 0x7fdfd419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfdc789700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd419d440 con 0x7fdfd41018b0 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfdc789700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd4196a20 con 0x7fdfd4100f00 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfd9d24700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 0x7fdfd419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfd9d24700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 0x7fdfd419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55144/0 (socket says 172.21.15.49:55144) 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfda525700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4100f00 0x7fdfd419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfd9d24700 1 -- 172.21.15.49:0/3507234199 learned_addr learned my addr 172.21.15.49:0/3507234199 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfd9d24700 1 -- 172.21.15.49:0/3507234199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4100f00 msgr2=0x7fdfd419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfd9d24700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4100f00 0x7fdfd419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.597+0000 7fdfd9d24700 1 -- 172.21.15.49:0/3507234199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc40095f0 con 0x7fdfd41018b0 2024-05-15T16:26:23.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.598+0000 7fdfd9d24700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 0x7fdfd419ce20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400bd70 tx=0x7fdfc400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:23.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.598+0000 7fdfcb7fe700 1 -- 172.21.15.49:0/3507234199 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfc401d070 con 0x7fdfd41018b0 2024-05-15T16:26:23.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.598+0000 7fdfcb7fe700 1 -- 172.21.15.49:0/3507234199 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfc40056e0 con 0x7fdfd41018b0 2024-05-15T16:26:23.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.598+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd4196ca0 con 0x7fdfd41018b0 2024-05-15T16:26:23.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.598+0000 7fdfcb7fe700 1 -- 172.21.15.49:0/3507234199 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfc400fe50 con 0x7fdfd41018b0 2024-05-15T16:26:23.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.598+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd4197190 con 0x7fdfd41018b0 2024-05-15T16:26:23.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.599+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd4109300 con 0x7fdfd41018b0 2024-05-15T16:26:23.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.600+0000 7fdfcb7fe700 1 -- 172.21.15.49:0/3507234199 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdfc400f680 con 0x7fdfd41018b0 2024-05-15T16:26:23.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.600+0000 7fdfcb7fe700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdfc006e200 0x7fdfc00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:23.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.600+0000 7fdfcb7fe700 1 -- 172.21.15.49:0/3507234199 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdfc408f590 con 0x7fdfd41018b0 2024-05-15T16:26:23.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.600+0000 7fdfda525700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdfc006e200 0x7fdfc00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:23.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.601+0000 7fdfda525700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdfc006e200 0x7fdfc00706c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0005950 tx=0x7fdfd000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:23.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.605+0000 7fdfcb7fe700 1 -- 172.21.15.49:0/3507234199 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdfc405c420 con 0x7fdfd41018b0 2024-05-15T16:26:23.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.833+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfd4197ee0 con 0x7fdfc006e200 2024-05-15T16:26:23.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.834+0000 7fdfcb7fe700 1 -- 172.21.15.49:0/3507234199 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdfd4197ee0 con 0x7fdfc006e200 2024-05-15T16:26:23.835 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:26:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:26:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:26:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:26:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:26:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:26:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:26:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:26:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:26:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:26:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:26:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:26:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:26:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:26:23.836 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:26:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdfc006e200 msgr2=0x7fdfc00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdfc006e200 0x7fdfc00706c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0005950 tx=0x7fdfd000b410 comp rx=0 tx=0).stop 2024-05-15T16:26:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 msgr2=0x7fdfd419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 0x7fdfd419ce20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400bd70 tx=0x7fdfc400bda0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 shutdown_connections 2024-05-15T16:26:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdfc006e200 0x7fdfc00706c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4100f00 0x7fdfd419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 --2- 172.21.15.49:0/3507234199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfd41018b0 0x7fdfd419ce20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 >> 172.21.15.49:0/3507234199 conn(0x7fdfd40fc7d0 msgr2=0x7fdfd40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 shutdown_connections 2024-05-15T16:26:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.837+0000 7fdfdc789700 1 -- 172.21.15.49:0/3507234199 wait complete. 2024-05-15T16:26:23.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.946+0000 7f2385a41700 1 -- 172.21.15.49:0/3320718348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23800ffdf0 msgr2=0x7f23801001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.946+0000 7f2385a41700 1 --2- 172.21.15.49:0/3320718348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23800ffdf0 0x7f23801001d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2370009910 tx=0x7f2370009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:23.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.947+0000 7f2385a41700 1 -- 172.21.15.49:0/3320718348 shutdown_connections 2024-05-15T16:26:23.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.947+0000 7f2385a41700 1 --2- 172.21.15.49:0/3320718348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23801007a0 0x7f23801047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.947+0000 7f2385a41700 1 --2- 172.21.15.49:0/3320718348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23800ffdf0 0x7f23801001d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.947+0000 7f2385a41700 1 -- 172.21.15.49:0/3320718348 >> 172.21.15.49:0/3320718348 conn(0x7f23800fb6a0 msgr2=0x7f23800fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:23.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.947+0000 7f2385a41700 1 -- 172.21.15.49:0/3320718348 shutdown_connections 2024-05-15T16:26:23.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.947+0000 7f2385a41700 1 -- 172.21.15.49:0/3320718348 wait complete. 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.947+0000 7f2385a41700 1 Processor -- start 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.947+0000 7f2385a41700 1 -- start start 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f2385a41700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23800ffdf0 0x7f2380197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f2385a41700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23801007a0 0x7f2380197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f2385a41700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23801984f0 con 0x7f23801007a0 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f2385a41700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2380198630 con 0x7f23800ffdf0 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f23767fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23801007a0 0x7f2380197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f23767fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23801007a0 0x7f2380197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55156/0 (socket says 172.21.15.49:55156) 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f23767fc700 1 -- 172.21.15.49:0/4182954400 learned_addr learned my addr 172.21.15.49:0/4182954400 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:23.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f237effd700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23800ffdf0 0x7f2380197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f23767fc700 1 -- 172.21.15.49:0/4182954400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23800ffdf0 msgr2=0x7f2380197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f23767fc700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23800ffdf0 0x7f2380197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f23767fc700 1 -- 172.21.15.49:0/4182954400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23700095f0 con 0x7f23801007a0 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f237effd700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23800ffdf0 0x7f2380197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f23767fc700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23801007a0 0x7f2380197fb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f236800c960 tx=0x7f236800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f2384a3f700 1 -- 172.21.15.49:0/4182954400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2368007a70 con 0x7f23801007a0 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f2384a3f700 1 -- 172.21.15.49:0/4182954400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2368007bd0 con 0x7f23801007a0 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23801a0ae0 con 0x7f23801007a0 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f2384a3f700 1 -- 172.21.15.49:0/4182954400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23680186c0 con 0x7f23801007a0 2024-05-15T16:26:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.948+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23801a0e40 con 0x7f23801007a0 2024-05-15T16:26:23.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.949+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2380108190 con 0x7f23801007a0 2024-05-15T16:26:23.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.950+0000 7f2384a3f700 1 -- 172.21.15.49:0/4182954400 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f236800faa0 con 0x7f23801007a0 2024-05-15T16:26:23.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.950+0000 7f2384a3f700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f236c06e200 0x7f236c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:23.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.950+0000 7f2384a3f700 1 -- 172.21.15.49:0/4182954400 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f236808e140 con 0x7f23801007a0 2024-05-15T16:26:23.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.951+0000 7f237effd700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f236c06e200 0x7f236c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:23.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.951+0000 7f237effd700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f236c06e200 0x7f236c0706c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2370006010 tx=0x7f237001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:23.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:23.955+0000 7f2384a3f700 1 -- 172.21.15.49:0/4182954400 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f236805afd0 con 0x7f23801007a0 2024-05-15T16:26:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:23 smithi049 ceph-mon[112142]: from='client.35610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:23 smithi049 ceph-mon[112142]: from='client.35614 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:23 smithi049 ceph-mon[112142]: pgmap v882: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:23 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2187301353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:26:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:23 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3150359637' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:26:24.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.247+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f23801012d0 con 0x7f23801007a0 2024-05-15T16:26:24.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.247+0000 7f2384a3f700 1 -- 172.21.15.49:0/4182954400 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f236805ab60 con 0x7f23801007a0 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:26:24.249 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:26:24.250 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:26:24.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.249+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f236c06e200 msgr2=0x7f236c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:24.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.249+0000 7f2385a41700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f236c06e200 0x7f236c0706c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2370006010 tx=0x7f237001a040 comp rx=0 tx=0).stop 2024-05-15T16:26:24.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.249+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23801007a0 msgr2=0x7f2380197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:24.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.249+0000 7f2385a41700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23801007a0 0x7f2380197fb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f236800c960 tx=0x7f236800cd20 comp rx=0 tx=0).stop 2024-05-15T16:26:24.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.250+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 shutdown_connections 2024-05-15T16:26:24.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.250+0000 7f2385a41700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f236c06e200 0x7f236c0706c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:24.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.250+0000 7f2385a41700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23800ffdf0 0x7f2380197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:24.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.250+0000 7f2385a41700 1 --2- 172.21.15.49:0/4182954400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23801007a0 0x7f2380197fb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:24.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.250+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 >> 172.21.15.49:0/4182954400 conn(0x7f23800fb6a0 msgr2=0x7f23800fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:24.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.250+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 shutdown_connections 2024-05-15T16:26:24.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:24.250+0000 7f2385a41700 1 -- 172.21.15.49:0/4182954400 wait complete. 2024-05-15T16:26:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:23 smithi144 ceph-mon[86635]: from='client.35610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:23 smithi144 ceph-mon[86635]: from='client.35614 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:23 smithi144 ceph-mon[86635]: pgmap v882: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:23 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2187301353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:26:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:23 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3150359637' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:26:25.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:24 smithi144 ceph-mon[86635]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:25.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:24 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4182954400' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:26:25.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:24 smithi049 ceph-mon[112142]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:25.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:24 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4182954400' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:26:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:25 smithi144 ceph-mon[86635]: pgmap v883: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:25 smithi049 ceph-mon[112142]: pgmap v883: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:28.420 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:27 smithi144 ceph-mon[86635]: pgmap v884: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:27 smithi049 ceph-mon[112142]: pgmap v884: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:30.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:29 smithi144 ceph-mon[86635]: pgmap v885: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:29 smithi049 ceph-mon[112142]: pgmap v885: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:32.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:31 smithi144 ceph-mon[86635]: pgmap v886: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:31 smithi049 ceph-mon[112142]: pgmap v886: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:33 smithi144 ceph-mon[86635]: pgmap v887: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:33 smithi049 ceph-mon[112142]: pgmap v887: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:36.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:35 smithi144 ceph-mon[86635]: pgmap v888: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:36.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:26:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:35 smithi049 ceph-mon[112142]: pgmap v888: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:36.469 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:26:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:37 smithi144 ceph-mon[86635]: pgmap v889: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:37 smithi049 ceph-mon[112142]: pgmap v889: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:40 smithi144 ceph-mon[86635]: pgmap v890: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:40 smithi049 ceph-mon[112142]: pgmap v890: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:42 smithi144 ceph-mon[86635]: pgmap v891: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:42 smithi049 ceph-mon[112142]: pgmap v891: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:44 smithi144 ceph-mon[86635]: pgmap v892: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:44 smithi049 ceph-mon[112142]: pgmap v892: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:46 smithi144 ceph-mon[86635]: pgmap v893: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:46 smithi049 ceph-mon[112142]: pgmap v893: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:48 smithi049 ceph-mon[112142]: pgmap v894: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:48 smithi144 ceph-mon[86635]: pgmap v894: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:50 smithi144 ceph-mon[86635]: pgmap v895: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:50.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:50 smithi049 ceph-mon[112142]: pgmap v895: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:26:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:26:52.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:52 smithi144 ceph-mon[86635]: pgmap v896: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:52 smithi049 ceph-mon[112142]: pgmap v896: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.374+0000 7fad865a0700 1 -- 172.21.15.49:0/2149723626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80106b70 msgr2=0x7fad80106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:54.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.374+0000 7fad865a0700 1 --2- 172.21.15.49:0/2149723626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80106b70 0x7fad80106f50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009910 tx=0x7fad7c009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.374+0000 7fad865a0700 1 -- 172.21.15.49:0/2149723626 shutdown_connections 2024-05-15T16:26:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.374+0000 7fad865a0700 1 --2- 172.21.15.49:0/2149723626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad80100ae0 0x7fad80100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.374+0000 7fad865a0700 1 --2- 172.21.15.49:0/2149723626 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80106b70 0x7fad80106f50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.374+0000 7fad865a0700 1 -- 172.21.15.49:0/2149723626 >> 172.21.15.49:0/2149723626 conn(0x7fad800fc5f0 msgr2=0x7fad800fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:54.375 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:54 smithi049 ceph-mon[112142]: pgmap v897: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:54.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.374+0000 7fad865a0700 1 -- 172.21.15.49:0/2149723626 shutdown_connections 2024-05-15T16:26:54.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.374+0000 7fad865a0700 1 -- 172.21.15.49:0/2149723626 wait complete. 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad865a0700 1 Processor -- start 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad865a0700 1 -- start start 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad865a0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80100ae0 0x7fad800732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad865a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad80106b70 0x7fad800737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad865a0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80073d20 con 0x7fad80100ae0 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad865a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80073e60 con 0x7fad80106b70 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad8559e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80100ae0 0x7fad800732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad8559e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80100ae0 0x7fad800732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33608/0 (socket says 172.21.15.49:33608) 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad8559e700 1 -- 172.21.15.49:0/1690475812 learned_addr learned my addr 172.21.15.49:0/1690475812 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad8559e700 1 -- 172.21.15.49:0/1690475812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad80106b70 msgr2=0x7fad800737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad84d9d700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad80106b70 0x7fad800737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad8559e700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad80106b70 0x7fad800737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad8559e700 1 -- 172.21.15.49:0/1690475812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c0095f0 con 0x7fad80100ae0 2024-05-15T16:26:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.375+0000 7fad84d9d700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad80106b70 0x7fad800737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:26:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.376+0000 7fad8559e700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80100ae0 0x7fad800732a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006010 tx=0x7fad7c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.376+0000 7fad767fc700 1 -- 172.21.15.49:0/1690475812 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad7c01d070 con 0x7fad80100ae0 2024-05-15T16:26:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.376+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad800740e0 con 0x7fad80100ae0 2024-05-15T16:26:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.376+0000 7fad767fc700 1 -- 172.21.15.49:0/1690475812 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad7c00f460 con 0x7fad80100ae0 2024-05-15T16:26:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.376+0000 7fad767fc700 1 -- 172.21.15.49:0/1690475812 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad7c017630 con 0x7fad80100ae0 2024-05-15T16:26:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.376+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad801a2e50 con 0x7fad80100ae0 2024-05-15T16:26:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.377+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad801090e0 con 0x7fad80100ae0 2024-05-15T16:26:54.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.377+0000 7fad767fc700 1 -- 172.21.15.49:0/1690475812 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fad7c00f5d0 con 0x7fad80100ae0 2024-05-15T16:26:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.378+0000 7fad767fc700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad6c06e200 0x7fad6c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.378+0000 7fad767fc700 1 -- 172.21.15.49:0/1690475812 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fad7c08f640 con 0x7fad80100ae0 2024-05-15T16:26:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.378+0000 7fad84d9d700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad6c06e200 0x7fad6c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:54.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.379+0000 7fad84d9d700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad6c06e200 0x7fad6c0706c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fad70005fd0 tx=0x7fad70005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:54.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.383+0000 7fad767fc700 1 -- 172.21.15.49:0/1690475812 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fad7c0589e0 con 0x7fad80100ae0 2024-05-15T16:26:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:54 smithi144 ceph-mon[86635]: pgmap v897: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:54.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.625+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad8010dce0 con 0x7fad6c06e200 2024-05-15T16:26:54.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.627+0000 7fad767fc700 1 -- 172.21.15.49:0/1690475812 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fad8010dce0 con 0x7fad6c06e200 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad6c06e200 msgr2=0x7fad6c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad6c06e200 0x7fad6c0706c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fad70005fd0 tx=0x7fad70005dc0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80100ae0 msgr2=0x7fad800732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80100ae0 0x7fad800732a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006010 tx=0x7fad7c00be20 comp rx=0 tx=0).stop 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 shutdown_connections 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad6c06e200 0x7fad6c0706c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad80100ae0 0x7fad800732a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 --2- 172.21.15.49:0/1690475812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad80106b70 0x7fad800737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 >> 172.21.15.49:0/1690475812 conn(0x7fad800fc5f0 msgr2=0x7fad800fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 shutdown_connections 2024-05-15T16:26:54.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.629+0000 7fad865a0700 1 -- 172.21.15.49:0/1690475812 wait complete. 2024-05-15T16:26:54.640 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:26:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.739+0000 7f34dd139700 1 -- 172.21.15.49:0/830245587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d80ffe10 msgr2=0x7f34d81001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.739+0000 7f34dd139700 1 --2- 172.21.15.49:0/830245587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d80ffe10 0x7f34d81001f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009910 tx=0x7f34c8009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.739+0000 7f34dd139700 1 -- 172.21.15.49:0/830245587 shutdown_connections 2024-05-15T16:26:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.739+0000 7f34dd139700 1 --2- 172.21.15.49:0/830245587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34d81007c0 0x7f34d8104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.739+0000 7f34dd139700 1 --2- 172.21.15.49:0/830245587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d80ffe10 0x7f34d81001f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.739+0000 7f34dd139700 1 -- 172.21.15.49:0/830245587 >> 172.21.15.49:0/830245587 conn(0x7f34d80fb700 msgr2=0x7f34d80fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.739+0000 7f34dd139700 1 -- 172.21.15.49:0/830245587 shutdown_connections 2024-05-15T16:26:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.739+0000 7f34dd139700 1 -- 172.21.15.49:0/830245587 wait complete. 2024-05-15T16:26:54.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.740+0000 7f34dd139700 1 Processor -- start 2024-05-15T16:26:54.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.740+0000 7f34dd139700 1 -- start start 2024-05-15T16:26:54.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.740+0000 7f34dd139700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34d80ffe10 0x7f34d819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:54.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.740+0000 7f34dd139700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d81007c0 0x7f34d819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.740+0000 7f34dd139700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d819d430 con 0x7f34d81007c0 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.740+0000 7f34dd139700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d8196a70 con 0x7f34d80ffe10 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34cffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d81007c0 0x7f34d819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34cffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d81007c0 0x7f34d819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33620/0 (socket says 172.21.15.49:33620) 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34cffff700 1 -- 172.21.15.49:0/951915729 learned_addr learned my addr 172.21.15.49:0/951915729 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34d6d9d700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34d80ffe10 0x7f34d819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34cffff700 1 -- 172.21.15.49:0/951915729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34d80ffe10 msgr2=0x7f34d819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34cffff700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34d80ffe10 0x7f34d819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34cffff700 1 -- 172.21.15.49:0/951915729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c80095f0 con 0x7f34d81007c0 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34cffff700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d81007c0 0x7f34d819cd50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f34c000d900 tx=0x7f34c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34d4d99700 1 -- 172.21.15.49:0/951915729 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c0009900 con 0x7f34d81007c0 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34d4d99700 1 -- 172.21.15.49:0/951915729 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34c0010460 con 0x7f34d81007c0 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d8196d50 con 0x7f34d81007c0 2024-05-15T16:26:54.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34d4d99700 1 -- 172.21.15.49:0/951915729 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c000f5f0 con 0x7f34d81007c0 2024-05-15T16:26:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.741+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d81972a0 con 0x7f34d81007c0 2024-05-15T16:26:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.742+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d81012f0 con 0x7f34d81007c0 2024-05-15T16:26:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.743+0000 7f34d4d99700 1 -- 172.21.15.49:0/951915729 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f34c00105d0 con 0x7f34d81007c0 2024-05-15T16:26:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.743+0000 7f34d4d99700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f34c406e210 0x7f34c40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.743+0000 7f34d4d99700 1 -- 172.21.15.49:0/951915729 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f34c008dc40 con 0x7f34d81007c0 2024-05-15T16:26:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.743+0000 7f34d6d9d700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f34c406e210 0x7f34c40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.744+0000 7f34d6d9d700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f34c406e210 0x7f34c40706d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009fd0 tx=0x7f34c801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:54.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.748+0000 7f34d4d99700 1 -- 172.21.15.49:0/951915729 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f34c005aad0 con 0x7f34d81007c0 2024-05-15T16:26:54.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.976+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34d810cde0 con 0x7f34c406e210 2024-05-15T16:26:54.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.977+0000 7f34d4d99700 1 -- 172.21.15.49:0/951915729 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f34d810cde0 con 0x7f34c406e210 2024-05-15T16:26:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f34c406e210 msgr2=0x7f34c40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f34c406e210 0x7f34c40706d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009fd0 tx=0x7f34c801a040 comp rx=0 tx=0).stop 2024-05-15T16:26:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d81007c0 msgr2=0x7f34d819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d81007c0 0x7f34d819cd50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f34c000d900 tx=0x7f34c000dc10 comp rx=0 tx=0).stop 2024-05-15T16:26:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 shutdown_connections 2024-05-15T16:26:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f34c406e210 0x7f34c40706d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34d80ffe10 0x7f34d819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 --2- 172.21.15.49:0/951915729 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34d81007c0 0x7f34d819cd50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 >> 172.21.15.49:0/951915729 conn(0x7f34d80fb700 msgr2=0x7f34d8069670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:54.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 shutdown_connections 2024-05-15T16:26:54.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:54.980+0000 7f34dd139700 1 -- 172.21.15.49:0/951915729 wait complete. 2024-05-15T16:26:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.089+0000 7f9a8ea68700 1 -- 172.21.15.49:0/649671838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 msgr2=0x7f9a881060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.089+0000 7f9a8ea68700 1 --2- 172.21.15.49:0/649671838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 0x7f9a881060a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009910 tx=0x7f9a78009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.089+0000 7f9a8ea68700 1 -- 172.21.15.49:0/649671838 shutdown_connections 2024-05-15T16:26:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.089+0000 7f9a8ea68700 1 --2- 172.21.15.49:0/649671838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a880ff1f0 0x7f9a880ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.089+0000 7f9a8ea68700 1 --2- 172.21.15.49:0/649671838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 0x7f9a881060a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.089+0000 7f9a8ea68700 1 -- 172.21.15.49:0/649671838 >> 172.21.15.49:0/649671838 conn(0x7f9a88076000 msgr2=0x7f9a880fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.089+0000 7f9a8ea68700 1 -- 172.21.15.49:0/649671838 shutdown_connections 2024-05-15T16:26:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.089+0000 7f9a8ea68700 1 -- 172.21.15.49:0/649671838 wait complete. 2024-05-15T16:26:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.089+0000 7f9a8ea68700 1 Processor -- start 2024-05-15T16:26:55.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a8ea68700 1 -- start start 2024-05-15T16:26:55.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a8ea68700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a880ff1f0 0x7f9a8819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:55.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a8ea68700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 0x7f9a8819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a8ea68700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8819d4c0 con 0x7f9a88105cc0 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a8ea68700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88196aa0 con 0x7f9a880ff1f0 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a87fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 0x7f9a8819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a87fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 0x7f9a8819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33648/0 (socket says 172.21.15.49:33648) 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a87fff700 1 -- 172.21.15.49:0/2494024990 learned_addr learned my addr 172.21.15.49:0/2494024990 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a8c804700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a880ff1f0 0x7f9a8819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a87fff700 1 -- 172.21.15.49:0/2494024990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a880ff1f0 msgr2=0x7f9a8819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a87fff700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a880ff1f0 0x7f9a8819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a87fff700 1 -- 172.21.15.49:0/2494024990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780095f0 con 0x7f9a88105cc0 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a8c804700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a880ff1f0 0x7f9a8819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.090+0000 7f9a87fff700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 0x7f9a8819cea0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00e920 tx=0x7f9a7c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.091+0000 7f9a85ffb700 1 -- 172.21.15.49:0/2494024990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a7c009790 con 0x7f9a88105cc0 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.091+0000 7f9a85ffb700 1 -- 172.21.15.49:0/2494024990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a7c00cba0 con 0x7f9a88105cc0 2024-05-15T16:26:55.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.091+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a88196d80 con 0x7f9a88105cc0 2024-05-15T16:26:55.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.091+0000 7f9a85ffb700 1 -- 172.21.15.49:0/2494024990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a7c018650 con 0x7f9a88105cc0 2024-05-15T16:26:55.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.091+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a881972d0 con 0x7f9a88105cc0 2024-05-15T16:26:55.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.092+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a88109200 con 0x7f9a88105cc0 2024-05-15T16:26:55.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.092+0000 7f9a85ffb700 1 -- 172.21.15.49:0/2494024990 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9a7c00cd10 con 0x7f9a88105cc0 2024-05-15T16:26:55.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.093+0000 7f9a85ffb700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9a7006e200 0x7f9a700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:55.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.093+0000 7f9a85ffb700 1 -- 172.21.15.49:0/2494024990 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9a7c014070 con 0x7f9a88105cc0 2024-05-15T16:26:55.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.093+0000 7f9a8c804700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9a7006e200 0x7f9a700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:55.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.094+0000 7f9a8c804700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9a7006e200 0x7f9a700706c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9a78006010 tx=0x7f9a78009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:55.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.098+0000 7f9a85ffb700 1 -- 172.21.15.49:0/2494024990 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9a7c053850 con 0x7f9a88105cc0 2024-05-15T16:26:55.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.313+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a88197f60 con 0x7f9a7006e200 2024-05-15T16:26:55.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.325+0000 7f9a85ffb700 1 -- 172.21.15.49:0/2494024990 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9a88197f60 con 0x7f9a7006e200 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (30m) 7m ago 41m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (41m) 7m ago 41m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (40m) 8m ago 40m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (27m) 7m ago 41m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (27m) 8m ago 40m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (29m) 7m ago 41m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (23m) 7m ago 36m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (23m) 7m ago 36m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (23m) 8m ago 36m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (8m) 8m ago 36m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:26:55.326 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (32m) 7m ago 43m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (31m) 8m ago 40m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (28m) 7m ago 43m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (28m) 8m ago 40m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (31m) 7m ago 41m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (30m) 8m ago 40m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (27m) 7m ago 39m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (26m) 7m ago 39m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (25m) 7m ago 38m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (25m) 8m ago 38m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (24m) 8m ago 38m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (24m) 8m ago 37m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:26:55.327 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (30m) 7m ago 41m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:26:55.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9a7006e200 msgr2=0x7f9a700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9a7006e200 0x7f9a700706c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9a78006010 tx=0x7f9a78009f70 comp rx=0 tx=0).stop 2024-05-15T16:26:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 msgr2=0x7f9a8819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 0x7f9a8819cea0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00e920 tx=0x7f9a7c00ece0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 shutdown_connections 2024-05-15T16:26:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9a7006e200 0x7f9a700706c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a880ff1f0 0x7f9a8819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 --2- 172.21.15.49:0/2494024990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a88105cc0 0x7f9a8819cea0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 >> 172.21.15.49:0/2494024990 conn(0x7f9a88076000 msgr2=0x7f9a880fb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 shutdown_connections 2024-05-15T16:26:55.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.328+0000 7f9a8ea68700 1 -- 172.21.15.49:0/2494024990 wait complete. 2024-05-15T16:26:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.435+0000 7f1b4632e700 1 -- 172.21.15.49:0/899664080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b40101e80 msgr2=0x7f1b40102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.435+0000 7f1b4632e700 1 --2- 172.21.15.49:0/899664080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b40101e80 0x7f1b40102260 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009940 tx=0x7f1b30009c50 comp rx=0 tx=0).stop 2024-05-15T16:26:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.435+0000 7f1b4632e700 1 -- 172.21.15.49:0/899664080 shutdown_connections 2024-05-15T16:26:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.435+0000 7f1b4632e700 1 --2- 172.21.15.49:0/899664080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b401027a0 0x7f1b4010ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.435+0000 7f1b4632e700 1 --2- 172.21.15.49:0/899664080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b40101e80 0x7f1b40102260 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.436+0000 7f1b4632e700 1 -- 172.21.15.49:0/899664080 >> 172.21.15.49:0/899664080 conn(0x7f1b40076800 msgr2=0x7f1b400fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.436+0000 7f1b4632e700 1 -- 172.21.15.49:0/899664080 shutdown_connections 2024-05-15T16:26:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.436+0000 7f1b4632e700 1 -- 172.21.15.49:0/899664080 wait complete. 2024-05-15T16:26:55.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.436+0000 7f1b4632e700 1 Processor -- start 2024-05-15T16:26:55.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.436+0000 7f1b4632e700 1 -- start start 2024-05-15T16:26:55.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.436+0000 7f1b4632e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b40101e80 0x7f1b400ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.436+0000 7f1b4632e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b401027a0 0x7f1b40100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.436+0000 7f1b4632e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b40100560 con 0x7f1b401027a0 2024-05-15T16:26:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.436+0000 7f1b4632e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b401006a0 con 0x7f1b40101e80 2024-05-15T16:26:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b401027a0 0x7f1b40100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b401027a0 0x7f1b40100020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33656/0 (socket says 172.21.15.49:33656) 2024-05-15T16:26:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b37fff700 1 -- 172.21.15.49:0/1078178990 learned_addr learned my addr 172.21.15.49:0/1078178990 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b3ffff700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b40101e80 0x7f1b400ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b37fff700 1 -- 172.21.15.49:0/1078178990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b40101e80 msgr2=0x7f1b400ffae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b37fff700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b40101e80 0x7f1b400ffae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b37fff700 1 -- 172.21.15.49:0/1078178990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b300095f0 con 0x7f1b401027a0 2024-05-15T16:26:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b37fff700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b401027a0 0x7f1b40100020 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e8f0 tx=0x7f1b2800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b3dffb700 1 -- 172.21.15.49:0/1078178990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b28009790 con 0x7f1b401027a0 2024-05-15T16:26:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b3dffb700 1 -- 172.21.15.49:0/1078178990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b2800c810 con 0x7f1b401027a0 2024-05-15T16:26:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b401a2d80 con 0x7f1b401027a0 2024-05-15T16:26:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b3dffb700 1 -- 172.21.15.49:0/1078178990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b28005580 con 0x7f1b401027a0 2024-05-15T16:26:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.437+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b401a30e0 con 0x7f1b401027a0 2024-05-15T16:26:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.438+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b40068f00 con 0x7f1b401027a0 2024-05-15T16:26:55.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.439+0000 7f1b3dffb700 1 -- 172.21.15.49:0/1078178990 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1b2800c980 con 0x7f1b401027a0 2024-05-15T16:26:55.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.439+0000 7f1b3dffb700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1b2c06e210 0x7f1b2c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:55.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.439+0000 7f1b3dffb700 1 -- 172.21.15.49:0/1078178990 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1b28014070 con 0x7f1b401027a0 2024-05-15T16:26:55.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.439+0000 7f1b3ffff700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1b2c06e210 0x7f1b2c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:55.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.440+0000 7f1b3ffff700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1b2c06e210 0x7f1b2c0706d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1b30006010 tx=0x7f1b30005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:55.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.444+0000 7f1b3dffb700 1 -- 172.21.15.49:0/1078178990 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1b280576f0 con 0x7f1b401027a0 2024-05-15T16:26:55.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.741+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1b40100860 con 0x7f1b401027a0 2024-05-15T16:26:55.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.742+0000 7f1b3dffb700 1 -- 172.21.15.49:0/1078178990 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1b2805adb0 con 0x7f1b401027a0 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:26:55.743 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:26:55.744 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:26:55.744 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:26:55.744 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:26:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.744+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1b2c06e210 msgr2=0x7f1b2c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.744+0000 7f1b4632e700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1b2c06e210 0x7f1b2c0706d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1b30006010 tx=0x7f1b30005ab0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.744+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b401027a0 msgr2=0x7f1b40100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.744+0000 7f1b4632e700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b401027a0 0x7f1b40100020 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e8f0 tx=0x7f1b2800ec00 comp rx=0 tx=0).stop 2024-05-15T16:26:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.744+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 shutdown_connections 2024-05-15T16:26:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.744+0000 7f1b4632e700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1b2c06e210 0x7f1b2c0706d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.744+0000 7f1b4632e700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b40101e80 0x7f1b400ffae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.744+0000 7f1b4632e700 1 --2- 172.21.15.49:0/1078178990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b401027a0 0x7f1b40100020 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.745+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 >> 172.21.15.49:0/1078178990 conn(0x7f1b40076800 msgr2=0x7f1b40103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:55.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.745+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 shutdown_connections 2024-05-15T16:26:55.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.745+0000 7f1b4632e700 1 -- 172.21.15.49:0/1078178990 wait complete. 2024-05-15T16:26:55.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.853+0000 7fcbe02b6700 1 -- 172.21.15.49:0/2214801218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd8101e80 msgr2=0x7fcbd8102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.853+0000 7fcbe02b6700 1 --2- 172.21.15.49:0/2214801218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd8101e80 0x7fcbd8102260 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009940 tx=0x7fcbd4009c50 comp rx=0 tx=0).stop 2024-05-15T16:26:55.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.853+0000 7fcbe02b6700 1 -- 172.21.15.49:0/2214801218 shutdown_connections 2024-05-15T16:26:55.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.853+0000 7fcbe02b6700 1 --2- 172.21.15.49:0/2214801218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd81027a0 0x7fcbd810ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.853+0000 7fcbe02b6700 1 --2- 172.21.15.49:0/2214801218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd8101e80 0x7fcbd8102260 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.853+0000 7fcbe02b6700 1 -- 172.21.15.49:0/2214801218 >> 172.21.15.49:0/2214801218 conn(0x7fcbd8076800 msgr2=0x7fcbd80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.853+0000 7fcbe02b6700 1 -- 172.21.15.49:0/2214801218 shutdown_connections 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.853+0000 7fcbe02b6700 1 -- 172.21.15.49:0/2214801218 wait complete. 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbe02b6700 1 Processor -- start 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbe02b6700 1 -- start start 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbe02b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd8101e80 0x7fcbd819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbe02b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81027a0 0x7fcbd819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbe02b6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd819d420 con 0x7fcbd81027a0 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbe02b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8196a00 con 0x7fcbd8101e80 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbdd851700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81027a0 0x7fcbd819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbdd851700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81027a0 0x7fcbd819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33666/0 (socket says 172.21.15.49:33666) 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbdd851700 1 -- 172.21.15.49:0/1316801509 learned_addr learned my addr 172.21.15.49:0/1316801509 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbde052700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd8101e80 0x7fcbd819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:55.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbdd851700 1 -- 172.21.15.49:0/1316801509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd8101e80 msgr2=0x7fcbd819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.854+0000 7fcbdd851700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd8101e80 0x7fcbd819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.855+0000 7fcbdd851700 1 -- 172.21.15.49:0/1316801509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40095f0 con 0x7fcbd81027a0 2024-05-15T16:26:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.855+0000 7fcbde052700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd8101e80 0x7fcbd819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:26:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.855+0000 7fcbdd851700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81027a0 0x7fcbd819ce00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800d900 tx=0x7fcbc800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.855+0000 7fcbcf7fe700 1 -- 172.21.15.49:0/1316801509 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbc8009900 con 0x7fcbd81027a0 2024-05-15T16:26:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.855+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd8196ce0 con 0x7fcbd81027a0 2024-05-15T16:26:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.855+0000 7fcbcf7fe700 1 -- 172.21.15.49:0/1316801509 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbc8010460 con 0x7fcbd81027a0 2024-05-15T16:26:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.855+0000 7fcbcf7fe700 1 -- 172.21.15.49:0/1316801509 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbc800f5f0 con 0x7fcbd81027a0 2024-05-15T16:26:55.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.855+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd8197230 con 0x7fcbd81027a0 2024-05-15T16:26:55.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.856+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd81055a0 con 0x7fcbd81027a0 2024-05-15T16:26:55.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.856+0000 7fcbcf7fe700 1 -- 172.21.15.49:0/1316801509 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcbc80105d0 con 0x7fcbd81027a0 2024-05-15T16:26:55.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.857+0000 7fcbcf7fe700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcbc406e200 0x7fcbc40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:55.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.857+0000 7fcbcf7fe700 1 -- 172.21.15.49:0/1316801509 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcbc808e020 con 0x7fcbd81027a0 2024-05-15T16:26:55.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.857+0000 7fcbde052700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcbc406e200 0x7fcbc40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:55.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.857+0000 7fcbde052700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcbc406e200 0x7fcbc40706c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4006010 tx=0x7fcbd4005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:55.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:55.862+0000 7fcbcf7fe700 1 -- 172.21.15.49:0/1316801509 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcbc805ae30 con 0x7fcbd81027a0 2024-05-15T16:26:56.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.114+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcbd8197ec0 con 0x7fcbd81027a0 2024-05-15T16:26:56.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.115+0000 7fcbcf7fe700 1 -- 172.21.15.49:0/1316801509 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fcbc805a9c0 con 0x7fcbd81027a0 2024-05-15T16:26:56.115 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:26:56.115 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:26:56.115 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:26:56.115 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:26:56.115 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:26:56.115 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:26:56.115 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:26:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:26:56.117 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:26:56.118 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:26:56.118 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:26:56.118 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:26:56.118 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:26:56.118 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:26:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcbc406e200 msgr2=0x7fcbc40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcbc406e200 0x7fcbc40706c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4006010 tx=0x7fcbd4005960 comp rx=0 tx=0).stop 2024-05-15T16:26:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81027a0 msgr2=0x7fcbd819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81027a0 0x7fcbd819ce00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800d900 tx=0x7fcbc800dcc0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 shutdown_connections 2024-05-15T16:26:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fcbc406e200 0x7fcbc40706c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd8101e80 0x7fcbd819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 --2- 172.21.15.49:0/1316801509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcbd81027a0 0x7fcbd819ce00 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 >> 172.21.15.49:0/1316801509 conn(0x7fcbd8076800 msgr2=0x7fcbd8103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:56.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 shutdown_connections 2024-05-15T16:26:56.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.117+0000 7fcbe02b6700 1 -- 172.21.15.49:0/1316801509 wait complete. 2024-05-15T16:26:56.120 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:26:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.226+0000 7fdc218c7700 1 -- 172.21.15.49:0/4204717630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c0732c0 msgr2=0x7fdc1c0736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.226+0000 7fdc218c7700 1 --2- 172.21.15.49:0/4204717630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c0732c0 0x7fdc1c0736a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b0a0 tx=0x7fdc0c00b3b0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.226+0000 7fdc218c7700 1 -- 172.21.15.49:0/4204717630 shutdown_connections 2024-05-15T16:26:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.226+0000 7fdc218c7700 1 --2- 172.21.15.49:0/4204717630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c073be0 0x7fdc1c10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.226+0000 7fdc218c7700 1 --2- 172.21.15.49:0/4204717630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c0732c0 0x7fdc1c0736a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.226+0000 7fdc218c7700 1 -- 172.21.15.49:0/4204717630 >> 172.21.15.49:0/4204717630 conn(0x7fdc1c0fc790 msgr2=0x7fdc1c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.226+0000 7fdc218c7700 1 -- 172.21.15.49:0/4204717630 shutdown_connections 2024-05-15T16:26:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.226+0000 7fdc218c7700 1 -- 172.21.15.49:0/4204717630 wait complete. 2024-05-15T16:26:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.227+0000 7fdc218c7700 1 Processor -- start 2024-05-15T16:26:56.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.227+0000 7fdc218c7700 1 -- start start 2024-05-15T16:26:56.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.227+0000 7fdc218c7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c0732c0 0x7fdc1c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:56.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.227+0000 7fdc218c7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c073be0 0x7fdc1c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:56.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.227+0000 7fdc218c7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c199b10 con 0x7fdc1c073be0 2024-05-15T16:26:56.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.227+0000 7fdc218c7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c19d7e0 con 0x7fdc1c0732c0 2024-05-15T16:26:56.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.227+0000 7fdc1a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c073be0 0x7fdc1c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:56.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.227+0000 7fdc1a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c073be0 0x7fdc1c199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33670/0 (socket says 172.21.15.49:33670) 2024-05-15T16:26:56.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.227+0000 7fdc1a7fc700 1 -- 172.21.15.49:0/475037725 learned_addr learned my addr 172.21.15.49:0/475037725 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:56.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc1affd700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c0732c0 0x7fdc1c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc1a7fc700 1 -- 172.21.15.49:0/475037725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c0732c0 msgr2=0x7fdc1c198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc1a7fc700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c0732c0 0x7fdc1c198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc1a7fc700 1 -- 172.21.15.49:0/475037725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c009d00 con 0x7fdc1c073be0 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc1a7fc700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c073be0 0x7fdc1c199430 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400b4b0 tx=0x7fdc0400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc208c5700 1 -- 172.21.15.49:0/475037725 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc04011800 con 0x7fdc1c073be0 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c19dac0 con 0x7fdc1c073be0 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc208c5700 1 -- 172.21.15.49:0/475037725 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc04011e20 con 0x7fdc1c073be0 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc208c5700 1 -- 172.21.15.49:0/475037725 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc0400f2e0 con 0x7fdc1c073be0 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.228+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c19e010 con 0x7fdc1c073be0 2024-05-15T16:26:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.229+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c074a00 con 0x7fdc1c073be0 2024-05-15T16:26:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.230+0000 7fdc208c5700 1 -- 172.21.15.49:0/475037725 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdc04010380 con 0x7fdc1c073be0 2024-05-15T16:26:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.230+0000 7fdc208c5700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc0806e210 0x7fdc080706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.230+0000 7fdc208c5700 1 -- 172.21.15.49:0/475037725 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdc0408dd50 con 0x7fdc1c073be0 2024-05-15T16:26:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.230+0000 7fdc1affd700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc0806e210 0x7fdc080706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.231+0000 7fdc1affd700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc0806e210 0x7fdc080706d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009480 tx=0x7fdc0c000f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:56.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.235+0000 7fdc208c5700 1 -- 172.21.15.49:0/475037725 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdc04057030 con 0x7fdc1c073be0 2024-05-15T16:26:56.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:56 smithi144 ceph-mon[86635]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:56 smithi144 ceph-mon[86635]: pgmap v898: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:56 smithi144 ceph-mon[86635]: from='client.35638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:56 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1078178990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:26:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:56 smithi049 ceph-mon[112142]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:56 smithi049 ceph-mon[112142]: pgmap v898: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:56 smithi049 ceph-mon[112142]: from='client.35638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:56 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1078178990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:26:56.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.466+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc1c110100 con 0x7fdc0806e210 2024-05-15T16:26:56.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.467+0000 7fdc208c5700 1 -- 172.21.15.49:0/475037725 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdc1c110100 con 0x7fdc0806e210 2024-05-15T16:26:56.467 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:26:56.467 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:26:56.467 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:26:56.467 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:26:56.467 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:26:56.467 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:26:56.467 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:26:56.468 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:26:56.468 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:26:56.468 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:26:56.468 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:26:56.468 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:26:56.468 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:26:56.468 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:26:56.468 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:26:56.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc0806e210 msgr2=0x7fdc080706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc0806e210 0x7fdc080706d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009480 tx=0x7fdc0c000f40 comp rx=0 tx=0).stop 2024-05-15T16:26:56.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c073be0 msgr2=0x7fdc1c199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c073be0 0x7fdc1c199430 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400b4b0 tx=0x7fdc0400b870 comp rx=0 tx=0).stop 2024-05-15T16:26:56.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 shutdown_connections 2024-05-15T16:26:56.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc0806e210 0x7fdc080706d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c0732c0 0x7fdc1c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 --2- 172.21.15.49:0/475037725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc1c073be0 0x7fdc1c199430 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 >> 172.21.15.49:0/475037725 conn(0x7fdc1c0fc790 msgr2=0x7fdc1c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:56.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 shutdown_connections 2024-05-15T16:26:56.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.469+0000 7fdc218c7700 1 -- 172.21.15.49:0/475037725 wait complete. 2024-05-15T16:26:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.577+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2389847780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc0ff0e0 msgr2=0x7fd2dc0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.577+0000 7fd2e1d34700 1 --2- 172.21.15.49:0/2389847780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc0ff0e0 0x7fd2dc0ff560 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009910 tx=0x7fd2c4009c20 comp rx=0 tx=0).stop 2024-05-15T16:26:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.577+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2389847780 shutdown_connections 2024-05-15T16:26:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.577+0000 7fd2e1d34700 1 --2- 172.21.15.49:0/2389847780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc0ff0e0 0x7fd2dc0ff560 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.577+0000 7fd2e1d34700 1 --2- 172.21.15.49:0/2389847780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc105cc0 0x7fd2dc1060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.577+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2389847780 >> 172.21.15.49:0/2389847780 conn(0x7fd2dc075f90 msgr2=0x7fd2dc0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.577+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2389847780 shutdown_connections 2024-05-15T16:26:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.577+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2389847780 wait complete. 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.578+0000 7fd2e1d34700 1 Processor -- start 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.578+0000 7fd2e1d34700 1 -- start start 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.578+0000 7fd2e1d34700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff0e0 0x7fd2dc19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.578+0000 7fd2e1d34700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc105cc0 0x7fd2dc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.578+0000 7fd2e1d34700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc19d520 con 0x7fd2dc105cc0 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.578+0000 7fd2e1d34700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc196b10 con 0x7fd2dc0ff0e0 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.578+0000 7fd2d2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc105cc0 0x7fd2dc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.578+0000 7fd2d2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc105cc0 0x7fd2dc19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33682/0 (socket says 172.21.15.49:33682) 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2d2ffd700 1 -- 172.21.15.49:0/2776211552 learned_addr learned my addr 172.21.15.49:0/2776211552 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2db7fe700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff0e0 0x7fd2dc19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2d2ffd700 1 -- 172.21.15.49:0/2776211552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff0e0 msgr2=0x7fd2dc19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2d2ffd700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff0e0 0x7fd2dc19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2d2ffd700 1 -- 172.21.15.49:0/2776211552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c40095f0 con 0x7fd2dc105cc0 2024-05-15T16:26:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2db7fe700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff0e0 0x7fd2dc19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:26:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2d2ffd700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc105cc0 0x7fd2dc19ce40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400bda0 tx=0x7fd2c400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2d97fa700 1 -- 172.21.15.49:0/2776211552 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c401d070 con 0x7fd2dc105cc0 2024-05-15T16:26:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc196d90 con 0x7fd2dc105cc0 2024-05-15T16:26:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2d97fa700 1 -- 172.21.15.49:0/2776211552 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2c400f460 con 0x7fd2dc105cc0 2024-05-15T16:26:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2d97fa700 1 -- 172.21.15.49:0/2776211552 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c4017630 con 0x7fd2dc105cc0 2024-05-15T16:26:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.579+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc197280 con 0x7fd2dc105cc0 2024-05-15T16:26:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.580+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2dc109290 con 0x7fd2dc105cc0 2024-05-15T16:26:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.581+0000 7fd2d97fa700 1 -- 172.21.15.49:0/2776211552 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd2c400fad0 con 0x7fd2dc105cc0 2024-05-15T16:26:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.581+0000 7fd2d97fa700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2c806e210 0x7fd2c80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:26:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.581+0000 7fd2d97fa700 1 -- 172.21.15.49:0/2776211552 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd2c408f590 con 0x7fd2dc105cc0 2024-05-15T16:26:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.581+0000 7fd2db7fe700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2c806e210 0x7fd2c80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:26:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.582+0000 7fd2db7fe700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2c806e210 0x7fd2c80706d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc103970 tx=0x7fd2cc009590 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:26:56.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.586+0000 7fd2d97fa700 1 -- 172.21.15.49:0/2776211552 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd2c405c420 con 0x7fd2dc105cc0 2024-05-15T16:26:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.881+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd2dc102400 con 0x7fd2dc105cc0 2024-05-15T16:26:56.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.881+0000 7fd2d97fa700 1 -- 172.21.15.49:0/2776211552 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fd2c405bfb0 con 0x7fd2dc105cc0 2024-05-15T16:26:56.881 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:26:56.881 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:26:56.882 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:26:56.883 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:26:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2c806e210 msgr2=0x7fd2c80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2c806e210 0x7fd2c80706d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc103970 tx=0x7fd2cc009590 comp rx=0 tx=0).stop 2024-05-15T16:26:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc105cc0 msgr2=0x7fd2dc19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:26:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc105cc0 0x7fd2dc19ce40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400bda0 tx=0x7fd2c400be80 comp rx=0 tx=0).stop 2024-05-15T16:26:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 shutdown_connections 2024-05-15T16:26:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd2c806e210 0x7fd2c80706d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff0e0 0x7fd2dc19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 --2- 172.21.15.49:0/2776211552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2dc105cc0 0x7fd2dc19ce40 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:26:56.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 >> 172.21.15.49:0/2776211552 conn(0x7fd2dc075f90 msgr2=0x7fd2dc0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:26:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 shutdown_connections 2024-05-15T16:26:56.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:26:56.883+0000 7fd2e1d34700 1 -- 172.21.15.49:0/2776211552 wait complete. 2024-05-15T16:26:57.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:57 smithi049 ceph-mon[112142]: from='client.35642 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:57.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:57 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1316801509' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:26:57.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:57 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2776211552' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:26:57.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:57 smithi144 ceph-mon[86635]: from='client.35642 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:57.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:57 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1316801509' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:26:57.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:57 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2776211552' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:26:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:58 smithi049 ceph-mon[112142]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:26:58 smithi049 ceph-mon[112142]: pgmap v899: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:26:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:58 smithi144 ceph-mon[86635]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:26:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:26:58 smithi144 ceph-mon[86635]: pgmap v899: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:00 smithi144 ceph-mon[86635]: pgmap v900: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:00 smithi049 ceph-mon[112142]: pgmap v900: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:02 smithi144 ceph-mon[86635]: pgmap v901: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:02 smithi049 ceph-mon[112142]: pgmap v901: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:04.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:04 smithi144 ceph-mon[86635]: pgmap v902: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:04 smithi049 ceph-mon[112142]: pgmap v902: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:06 smithi144 ceph-mon[86635]: pgmap v903: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:27:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:06 smithi049 ceph-mon[112142]: pgmap v903: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:27:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:08 smithi144 ceph-mon[86635]: pgmap v904: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:08 smithi049 ceph-mon[112142]: pgmap v904: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:10 smithi144 ceph-mon[86635]: pgmap v905: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:10 smithi049 ceph-mon[112142]: pgmap v905: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:12 smithi144 ceph-mon[86635]: pgmap v906: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:12.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:12 smithi049 ceph-mon[112142]: pgmap v906: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:14 smithi144 ceph-mon[86635]: pgmap v907: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:14.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:14 smithi049 ceph-mon[112142]: pgmap v907: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:16 smithi144 ceph-mon[86635]: pgmap v908: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:16 smithi049 ceph-mon[112142]: pgmap v908: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:18 smithi144 ceph-mon[86635]: pgmap v909: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:18 smithi049 ceph-mon[112142]: pgmap v909: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:20 smithi144 ceph-mon[86635]: pgmap v910: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:20 smithi049 ceph-mon[112142]: pgmap v910: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:27:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:27:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:22 smithi144 ceph-mon[86635]: pgmap v911: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:22 smithi049 ceph-mon[112142]: pgmap v911: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:24 smithi144 ceph-mon[86635]: pgmap v912: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:24 smithi049 ceph-mon[112142]: pgmap v912: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:26 smithi144 ceph-mon[86635]: pgmap v913: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:26 smithi049 ceph-mon[112142]: pgmap v913: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:27.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.006+0000 7f860bebd700 1 -- 172.21.15.49:0/2414716436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 msgr2=0x7f8604105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.006+0000 7f860bebd700 1 --2- 172.21.15.49:0/2414716436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 0x7f8604105900 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009910 tx=0x7f85f8009c20 comp rx=0 tx=0).stop 2024-05-15T16:27:27.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.006+0000 7f860bebd700 1 -- 172.21.15.49:0/2414716436 shutdown_connections 2024-05-15T16:27:27.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.006+0000 7f860bebd700 1 --2- 172.21.15.49:0/2414716436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 0x7f8604105900 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.006+0000 7f860bebd700 1 --2- 172.21.15.49:0/2414716436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8604100f00 0x7f86041012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.006+0000 7f860bebd700 1 -- 172.21.15.49:0/2414716436 >> 172.21.15.49:0/2414716436 conn(0x7f86040fc7d0 msgr2=0x7f86040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:27.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.006+0000 7f860bebd700 1 -- 172.21.15.49:0/2414716436 shutdown_connections 2024-05-15T16:27:27.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.006+0000 7f860bebd700 1 -- 172.21.15.49:0/2414716436 wait complete. 2024-05-15T16:27:27.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.007+0000 7f860bebd700 1 Processor -- start 2024-05-15T16:27:27.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.007+0000 7f860bebd700 1 -- start start 2024-05-15T16:27:27.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.007+0000 7f860bebd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8604100f00 0x7f86040732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:27.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.007+0000 7f860bebd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 0x7f86040737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:27.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.007+0000 7f860bebd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8604073d20 con 0x7f86041018b0 2024-05-15T16:27:27.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.007+0000 7f860bebd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8604073e60 con 0x7f8604100f00 2024-05-15T16:27:27.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8609458700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 0x7f86040737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:27.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8609458700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 0x7f86040737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53022/0 (socket says 172.21.15.49:53022) 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8609458700 1 -- 172.21.15.49:0/2790316782 learned_addr learned my addr 172.21.15.49:0/2790316782 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8609c59700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8604100f00 0x7f86040732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8609458700 1 -- 172.21.15.49:0/2790316782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8604100f00 msgr2=0x7f86040732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8609458700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8604100f00 0x7f86040732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8609458700 1 -- 172.21.15.49:0/2790316782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f80095f0 con 0x7f86041018b0 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8609458700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 0x7f86040737e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f85f80092e0 tx=0x7f85f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8602ffd700 1 -- 172.21.15.49:0/2790316782 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f801d070 con 0x7f86041018b0 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86040740e0 con 0x7f86041018b0 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8602ffd700 1 -- 172.21.15.49:0/2790316782 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85f8005700 con 0x7f86041018b0 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.008+0000 7f8602ffd700 1 -- 172.21.15.49:0/2790316782 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f8017400 con 0x7f86041018b0 2024-05-15T16:27:27.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.009+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86041a3020 con 0x7f86041018b0 2024-05-15T16:27:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.010+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8604109300 con 0x7f86041018b0 2024-05-15T16:27:27.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.010+0000 7f8602ffd700 1 -- 172.21.15.49:0/2790316782 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f85f8003a60 con 0x7f86041018b0 2024-05-15T16:27:27.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.011+0000 7f8602ffd700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85ec06e200 0x7f85ec0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:27.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.011+0000 7f8602ffd700 1 -- 172.21.15.49:0/2790316782 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f85f808f560 con 0x7f86041018b0 2024-05-15T16:27:27.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.011+0000 7f8609c59700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85ec06e200 0x7f85ec0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:27.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.011+0000 7f8609c59700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85ec06e200 0x7f85ec0706c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d10 tx=0x7f85f4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:27.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.015+0000 7f8602ffd700 1 -- 172.21.15.49:0/2790316782 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f85f8058840 con 0x7f86041018b0 2024-05-15T16:27:27.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.245+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f860410dea0 con 0x7f85ec06e200 2024-05-15T16:27:27.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.246+0000 7f8602ffd700 1 -- 172.21.15.49:0/2790316782 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f860410dea0 con 0x7f85ec06e200 2024-05-15T16:27:27.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85ec06e200 msgr2=0x7f85ec0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85ec06e200 0x7f85ec0706c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d10 tx=0x7f85f4009040 comp rx=0 tx=0).stop 2024-05-15T16:27:27.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 msgr2=0x7f86040737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 0x7f86040737e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f85f80092e0 tx=0x7f85f800be20 comp rx=0 tx=0).stop 2024-05-15T16:27:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 shutdown_connections 2024-05-15T16:27:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f85ec06e200 0x7f85ec0706c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8604100f00 0x7f86040732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 --2- 172.21.15.49:0/2790316782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86041018b0 0x7f86040737e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 >> 172.21.15.49:0/2790316782 conn(0x7f86040fc7d0 msgr2=0x7f86040fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 shutdown_connections 2024-05-15T16:27:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.248+0000 7f860bebd700 1 -- 172.21.15.49:0/2790316782 wait complete. 2024-05-15T16:27:27.259 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:27:27.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.358+0000 7f7ff265f700 1 -- 172.21.15.49:0/2705681807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec101880 msgr2=0x7f7fec1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.358+0000 7f7ff265f700 1 --2- 172.21.15.49:0/2705681807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec101880 0x7f7fec1058d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009910 tx=0x7f7fe0009c20 comp rx=0 tx=0).stop 2024-05-15T16:27:27.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.358+0000 7f7ff265f700 1 -- 172.21.15.49:0/2705681807 shutdown_connections 2024-05-15T16:27:27.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.358+0000 7f7ff265f700 1 --2- 172.21.15.49:0/2705681807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec101880 0x7f7fec1058d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.358+0000 7f7ff265f700 1 --2- 172.21.15.49:0/2705681807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fec100ed0 0x7f7fec1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.358+0000 7f7ff265f700 1 -- 172.21.15.49:0/2705681807 >> 172.21.15.49:0/2705681807 conn(0x7f7fec0fc7a0 msgr2=0x7f7fec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:27.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.358+0000 7f7ff265f700 1 -- 172.21.15.49:0/2705681807 shutdown_connections 2024-05-15T16:27:27.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.358+0000 7f7ff265f700 1 -- 172.21.15.49:0/2705681807 wait complete. 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.359+0000 7f7ff265f700 1 Processor -- start 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.359+0000 7f7ff265f700 1 -- start start 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.359+0000 7f7ff265f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec100ed0 0x7f7fec0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.359+0000 7f7ff265f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fec101880 0x7f7fec0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.359+0000 7f7ff265f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec073d20 con 0x7f7fec100ed0 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.359+0000 7f7ff265f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec073e60 con 0x7f7fec101880 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7febfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec100ed0 0x7f7fec0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7febfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec100ed0 0x7f7fec0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53036/0 (socket says 172.21.15.49:53036) 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7febfff700 1 -- 172.21.15.49:0/4202625938 learned_addr learned my addr 172.21.15.49:0/4202625938 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:27:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7feb7fe700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fec101880 0x7f7fec0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7febfff700 1 -- 172.21.15.49:0/4202625938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fec101880 msgr2=0x7f7fec0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7febfff700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fec101880 0x7f7fec0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7febfff700 1 -- 172.21.15.49:0/4202625938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe00095f0 con 0x7f7fec100ed0 2024-05-15T16:27:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7febfff700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec100ed0 0x7f7fec0732a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00d8d0 tx=0x7f7fdc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7fe97fa700 1 -- 172.21.15.49:0/4202625938 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fdc0098a0 con 0x7f7fec100ed0 2024-05-15T16:27:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7fe97fa700 1 -- 172.21.15.49:0/4202625938 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fdc010460 con 0x7f7fec100ed0 2024-05-15T16:27:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7fe97fa700 1 -- 172.21.15.49:0/4202625938 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fdc00f5f0 con 0x7f7fec100ed0 2024-05-15T16:27:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fec1a2df0 con 0x7f7fec100ed0 2024-05-15T16:27:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.360+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec1a3150 con 0x7f7fec100ed0 2024-05-15T16:27:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.361+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fec1092d0 con 0x7f7fec100ed0 2024-05-15T16:27:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.362+0000 7f7fe97fa700 1 -- 172.21.15.49:0/4202625938 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7fdc009a00 con 0x7f7fec100ed0 2024-05-15T16:27:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.362+0000 7f7fe97fa700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7fd406e210 0x7f7fd40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.363+0000 7f7fe97fa700 1 -- 172.21.15.49:0/4202625938 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7fdc08df70 con 0x7f7fec100ed0 2024-05-15T16:27:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.363+0000 7f7feb7fe700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7fd406e210 0x7f7fd40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:27.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.363+0000 7f7feb7fe700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7fd406e210 0x7f7fd40706d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009fd0 tx=0x7f7fe001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:27.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.367+0000 7f7fe97fa700 1 -- 172.21.15.49:0/4202625938 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7fdc0c2850 con 0x7f7fec100ed0 2024-05-15T16:27:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.593+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fec074820 con 0x7f7fd406e210 2024-05-15T16:27:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.594+0000 7f7fe97fa700 1 -- 172.21.15.49:0/4202625938 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7fec074820 con 0x7f7fd406e210 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7fd406e210 msgr2=0x7f7fd40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7fd406e210 0x7f7fd40706d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009fd0 tx=0x7f7fe001a040 comp rx=0 tx=0).stop 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec100ed0 msgr2=0x7f7fec0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec100ed0 0x7f7fec0732a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00d8d0 tx=0x7f7fdc00dbe0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 shutdown_connections 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7fd406e210 0x7f7fd40706d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fec100ed0 0x7f7fec0732a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 --2- 172.21.15.49:0/4202625938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fec101880 0x7f7fec0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 >> 172.21.15.49:0/4202625938 conn(0x7f7fec0fc7a0 msgr2=0x7f7fec0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 shutdown_connections 2024-05-15T16:27:27.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.597+0000 7f7ff265f700 1 -- 172.21.15.49:0/4202625938 wait complete. 2024-05-15T16:27:27.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.709+0000 7fc28f186700 1 -- 172.21.15.49:0/933439856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc288101eb0 msgr2=0x7fc288102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.709+0000 7fc28f186700 1 --2- 172.21.15.49:0/933439856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc288101eb0 0x7fc288102290 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009940 tx=0x7fc27c009c50 comp rx=0 tx=0).stop 2024-05-15T16:27:27.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.709+0000 7fc28f186700 1 -- 172.21.15.49:0/933439856 shutdown_connections 2024-05-15T16:27:27.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.709+0000 7fc28f186700 1 --2- 172.21.15.49:0/933439856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2881027d0 0x7fc28810acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.709+0000 7fc28f186700 1 --2- 172.21.15.49:0/933439856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc288101eb0 0x7fc288102290 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.709+0000 7fc28f186700 1 -- 172.21.15.49:0/933439856 >> 172.21.15.49:0/933439856 conn(0x7fc2880fb6a0 msgr2=0x7fc2880fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:27.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.709+0000 7fc28f186700 1 -- 172.21.15.49:0/933439856 shutdown_connections 2024-05-15T16:27:27.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.709+0000 7fc28f186700 1 -- 172.21.15.49:0/933439856 wait complete. 2024-05-15T16:27:27.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc28f186700 1 Processor -- start 2024-05-15T16:27:27.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc28f186700 1 -- start start 2024-05-15T16:27:27.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc28f186700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc288101eb0 0x7fc28819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:27.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc28f186700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2881027d0 0x7fc28819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:27.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc28f186700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288196b00 con 0x7fc2881027d0 2024-05-15T16:27:27.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc28f186700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288196c70 con 0x7fc288101eb0 2024-05-15T16:27:27.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc287fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2881027d0 0x7fc28819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc287fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2881027d0 0x7fc28819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53060/0 (socket says 172.21.15.49:53060) 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc287fff700 1 -- 172.21.15.49:0/675123838 learned_addr learned my addr 172.21.15.49:0/675123838 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc28cf22700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc288101eb0 0x7fc28819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc287fff700 1 -- 172.21.15.49:0/675123838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc288101eb0 msgr2=0x7fc28819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc287fff700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc288101eb0 0x7fc28819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc287fff700 1 -- 172.21.15.49:0/675123838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c0095f0 con 0x7fc2881027d0 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.710+0000 7fc28cf22700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc288101eb0 0x7fc28819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.711+0000 7fc287fff700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2881027d0 0x7fc28819cf90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc27400e970 tx=0x7fc27400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.711+0000 7fc285ffb700 1 -- 172.21.15.49:0/675123838 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc274009750 con 0x7fc2881027d0 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.711+0000 7fc285ffb700 1 -- 172.21.15.49:0/675123838 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc274004cf0 con 0x7fc2881027d0 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.711+0000 7fc285ffb700 1 -- 172.21.15.49:0/675123838 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc274005660 con 0x7fc2881027d0 2024-05-15T16:27:27.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.711+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc288196f50 con 0x7fc2881027d0 2024-05-15T16:27:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.711+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2881974a0 con 0x7fc2881027d0 2024-05-15T16:27:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.712+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288069a50 con 0x7fc2881027d0 2024-05-15T16:27:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.712+0000 7fc285ffb700 1 -- 172.21.15.49:0/675123838 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc274004750 con 0x7fc2881027d0 2024-05-15T16:27:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.713+0000 7fc285ffb700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc27806e210 0x7fc2780706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.713+0000 7fc285ffb700 1 -- 172.21.15.49:0/675123838 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc274014070 con 0x7fc2881027d0 2024-05-15T16:27:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.713+0000 7fc28cf22700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc27806e210 0x7fc2780706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.714+0000 7fc28cf22700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc27806e210 0x7fc2780706d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00b780 tx=0x7fc27c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:27.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.717+0000 7fc285ffb700 1 -- 172.21.15.49:0/675123838 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc274057820 con 0x7fc2881027d0 2024-05-15T16:27:27.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.937+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc288198080 con 0x7fc27806e210 2024-05-15T16:27:27.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.948+0000 7fc285ffb700 1 -- 172.21.15.49:0/675123838 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc288198080 con 0x7fc27806e210 2024-05-15T16:27:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:27:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (30m) 8m ago 42m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:27:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (42m) 8m ago 42m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:27:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (41m) 8m ago 41m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:27:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (28m) 8m ago 42m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:27:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (28m) 8m ago 41m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:27:27.948 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (30m) 8m ago 41m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (24m) 8m ago 36m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (24m) 8m ago 36m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (23m) 8m ago 36m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (8m) 8m ago 36m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (33m) 8m ago 43m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (31m) 8m ago 41m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (29m) 8m ago 43m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (28m) 8m ago 41m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (31m) 8m ago 42m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (31m) 8m ago 41m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (27m) 8m ago 40m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (27m) 8m ago 39m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (26m) 8m ago 39m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:27:27.949 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (25m) 8m ago 39m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:27:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (25m) 8m ago 38m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:27:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (24m) 8m ago 38m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:27:27.950 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (31m) 8m ago 41m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:27:27.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc27806e210 msgr2=0x7fc2780706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc27806e210 0x7fc2780706d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00b780 tx=0x7fc27c0058e0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2881027d0 msgr2=0x7fc28819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2881027d0 0x7fc28819cf90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc27400e970 tx=0x7fc27400ed30 comp rx=0 tx=0).stop 2024-05-15T16:27:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 shutdown_connections 2024-05-15T16:27:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc27806e210 0x7fc2780706d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc288101eb0 0x7fc28819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 --2- 172.21.15.49:0/675123838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2881027d0 0x7fc28819cf90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 >> 172.21.15.49:0/675123838 conn(0x7fc2880fb6a0 msgr2=0x7fc2881031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 shutdown_connections 2024-05-15T16:27:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:27.950+0000 7fc28f186700 1 -- 172.21.15.49:0/675123838 wait complete. 2024-05-15T16:27:28.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.057+0000 7f4a19e21700 1 -- 172.21.15.49:0/2587961923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14101890 msgr2=0x7f4a141058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.057+0000 7f4a19e21700 1 --2- 172.21.15.49:0/2587961923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14101890 0x7f4a141058e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009910 tx=0x7f4a08009c20 comp rx=0 tx=0).stop 2024-05-15T16:27:28.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.057+0000 7f4a19e21700 1 -- 172.21.15.49:0/2587961923 shutdown_connections 2024-05-15T16:27:28.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.057+0000 7f4a19e21700 1 --2- 172.21.15.49:0/2587961923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14101890 0x7f4a141058e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.057+0000 7f4a19e21700 1 --2- 172.21.15.49:0/2587961923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a14100ee0 0x7f4a141012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.057+0000 7f4a19e21700 1 -- 172.21.15.49:0/2587961923 >> 172.21.15.49:0/2587961923 conn(0x7f4a140fc790 msgr2=0x7f4a140febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:28.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.057+0000 7f4a19e21700 1 -- 172.21.15.49:0/2587961923 shutdown_connections 2024-05-15T16:27:28.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.057+0000 7f4a19e21700 1 -- 172.21.15.49:0/2587961923 wait complete. 2024-05-15T16:27:28.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a19e21700 1 Processor -- start 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a19e21700 1 -- start start 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a19e21700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14100ee0 0x7f4a1419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a19e21700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a14101890 0x7f4a1419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a19e21700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1419d440 con 0x7f4a14100ee0 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a19e21700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a14196a20 con 0x7f4a14101890 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14100ee0 0x7f4a1419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14100ee0 0x7f4a1419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53074/0 (socket says 172.21.15.49:53074) 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a137fe700 1 -- 172.21.15.49:0/2985167214 learned_addr learned my addr 172.21.15.49:0/2985167214 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.058+0000 7f4a12ffd700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a14101890 0x7f4a1419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:28.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.059+0000 7f4a137fe700 1 -- 172.21.15.49:0/2985167214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a14101890 msgr2=0x7f4a1419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.059+0000 7f4a137fe700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a14101890 0x7f4a1419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.059+0000 7f4a137fe700 1 -- 172.21.15.49:0/2985167214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a080095f0 con 0x7f4a14100ee0 2024-05-15T16:27:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.059+0000 7f4a137fe700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14100ee0 0x7f4a1419c8e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c8a0 tx=0x7f4a0400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.059+0000 7f4a10ff9700 1 -- 172.21.15.49:0/2985167214 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a040078e0 con 0x7f4a14100ee0 2024-05-15T16:27:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.059+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a14196d00 con 0x7f4a14100ee0 2024-05-15T16:27:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.059+0000 7f4a10ff9700 1 -- 172.21.15.49:0/2985167214 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a0400f450 con 0x7f4a14100ee0 2024-05-15T16:27:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.059+0000 7f4a10ff9700 1 -- 172.21.15.49:0/2985167214 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a0400e5e0 con 0x7f4a14100ee0 2024-05-15T16:27:28.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.059+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a14197220 con 0x7f4a14100ee0 2024-05-15T16:27:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.060+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a141092e0 con 0x7f4a14100ee0 2024-05-15T16:27:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.061+0000 7f4a10ff9700 1 -- 172.21.15.49:0/2985167214 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4a0400f5c0 con 0x7f4a14100ee0 2024-05-15T16:27:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.061+0000 7f4a10ff9700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f49fc06e210 0x7f49fc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.061+0000 7f4a10ff9700 1 -- 172.21.15.49:0/2985167214 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4a0408e1b0 con 0x7f4a14100ee0 2024-05-15T16:27:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.061+0000 7f4a12ffd700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f49fc06e210 0x7f49fc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:28.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.062+0000 7f4a12ffd700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f49fc06e210 0x7f49fc0706d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009fd0 tx=0x7f4a0801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:28.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.066+0000 7f4a10ff9700 1 -- 172.21.15.49:0/2985167214 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4a0405afc0 con 0x7f4a14100ee0 2024-05-15T16:27:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:28 smithi049 ceph-mon[112142]: pgmap v914: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:28.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.366+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4a14197ee0 con 0x7f4a14100ee0 2024-05-15T16:27:28.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.366+0000 7f4a10ff9700 1 -- 172.21.15.49:0/2985167214 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4a0405ab50 con 0x7f4a14100ee0 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:27:28.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:27:28.368 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:27:28.368 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:27:28.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:27:28.368 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.368+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f49fc06e210 msgr2=0x7f49fc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.368+0000 7f4a19e21700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f49fc06e210 0x7f49fc0706d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009fd0 tx=0x7f4a0801a040 comp rx=0 tx=0).stop 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.368+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14100ee0 msgr2=0x7f4a1419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.368+0000 7f4a19e21700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14100ee0 0x7f4a1419c8e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c8a0 tx=0x7f4a0400cbb0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.368+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 shutdown_connections 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.368+0000 7f4a19e21700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f49fc06e210 0x7f49fc0706d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.369+0000 7f4a19e21700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a14100ee0 0x7f4a1419c8e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.369+0000 7f4a19e21700 1 --2- 172.21.15.49:0/2985167214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a14101890 0x7f4a1419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.369+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 >> 172.21.15.49:0/2985167214 conn(0x7f4a140fc790 msgr2=0x7f4a140fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.369+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 shutdown_connections 2024-05-15T16:27:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.369+0000 7f4a19e21700 1 -- 172.21.15.49:0/2985167214 wait complete. 2024-05-15T16:27:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:28 smithi144 ceph-mon[86635]: pgmap v914: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:28.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.477+0000 7f057a8f3700 1 -- 172.21.15.49:0/3488545727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 msgr2=0x7f0574102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.477+0000 7f057a8f3700 1 --2- 172.21.15.49:0/3488545727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 0x7f0574102200 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0564009910 tx=0x7f0564009c20 comp rx=0 tx=0).stop 2024-05-15T16:27:28.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.477+0000 7f057a8f3700 1 -- 172.21.15.49:0/3488545727 shutdown_connections 2024-05-15T16:27:28.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.477+0000 7f057a8f3700 1 --2- 172.21.15.49:0/3488545727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0574102740 0x7f057410ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.477+0000 7f057a8f3700 1 --2- 172.21.15.49:0/3488545727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 0x7f0574102200 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.477+0000 7f057a8f3700 1 -- 172.21.15.49:0/3488545727 >> 172.21.15.49:0/3488545727 conn(0x7f0574076800 msgr2=0x7f05740fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.477+0000 7f057a8f3700 1 -- 172.21.15.49:0/3488545727 shutdown_connections 2024-05-15T16:27:28.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.477+0000 7f057a8f3700 1 -- 172.21.15.49:0/3488545727 wait complete. 2024-05-15T16:27:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f057a8f3700 1 Processor -- start 2024-05-15T16:27:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f057a8f3700 1 -- start start 2024-05-15T16:27:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f057a8f3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 0x7f05740ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f057a8f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0574102740 0x7f05740fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f057a8f3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0574100500 con 0x7f0574101e20 2024-05-15T16:27:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f057a8f3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0574100640 con 0x7f0574102740 2024-05-15T16:27:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f0573fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 0x7f05740ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f0573fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 0x7f05740ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53096/0 (socket says 172.21.15.49:53096) 2024-05-15T16:27:28.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f0573fff700 1 -- 172.21.15.49:0/1664569252 learned_addr learned my addr 172.21.15.49:0/1664569252 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f056bfff700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0574102740 0x7f05740fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f0573fff700 1 -- 172.21.15.49:0/1664569252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0574102740 msgr2=0x7f05740fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f0573fff700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0574102740 0x7f05740fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f0573fff700 1 -- 172.21.15.49:0/1664569252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640095f0 con 0x7f0574101e20 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f056bfff700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0574102740 0x7f05740fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.478+0000 7f0573fff700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 0x7f05740ffa80 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0564005f10 tx=0x7f056400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.479+0000 7f0571ffb700 1 -- 172.21.15.49:0/1664569252 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f056401d070 con 0x7f0574101e20 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.479+0000 7f0571ffb700 1 -- 172.21.15.49:0/1664569252 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05640052a0 con 0x7f0574101e20 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.479+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05741008c0 con 0x7f0574101e20 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.479+0000 7f0571ffb700 1 -- 172.21.15.49:0/1664569252 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0564022470 con 0x7f0574101e20 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.479+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05741a3000 con 0x7f0574101e20 2024-05-15T16:27:28.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.480+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0574068f60 con 0x7f0574101e20 2024-05-15T16:27:28.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.480+0000 7f0571ffb700 1 -- 172.21.15.49:0/1664569252 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0564003780 con 0x7f0574101e20 2024-05-15T16:27:28.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.481+0000 7f0571ffb700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f056006e200 0x7f05600706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:28.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.481+0000 7f0571ffb700 1 -- 172.21.15.49:0/1664569252 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f056408f4b0 con 0x7f0574101e20 2024-05-15T16:27:28.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.481+0000 7f056bfff700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f056006e200 0x7f05600706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:28.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.481+0000 7f056bfff700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f056006e200 0x7f05600706c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f05741011b0 tx=0x7f055c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:28.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.486+0000 7f0571ffb700 1 -- 172.21.15.49:0/1664569252 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0564058810 con 0x7f0574101e20 2024-05-15T16:27:28.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.742+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0574100f60 con 0x7f0574101e20 2024-05-15T16:27:28.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.742+0000 7f0571ffb700 1 -- 172.21.15.49:0/1664569252 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f056405bed0 con 0x7f0574101e20 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:27:28.743 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:27:28.744 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:27:28.745 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:27:28.746 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:27:28.746 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:27:28.746 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:27:28.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.744+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f056006e200 msgr2=0x7f05600706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.744+0000 7f057a8f3700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f056006e200 0x7f05600706c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f05741011b0 tx=0x7f055c009260 comp rx=0 tx=0).stop 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.744+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 msgr2=0x7f05740ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.744+0000 7f057a8f3700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 0x7f05740ffa80 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0564005f10 tx=0x7f056400b650 comp rx=0 tx=0).stop 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.744+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 shutdown_connections 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.744+0000 7f057a8f3700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f056006e200 0x7f05600706c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.744+0000 7f057a8f3700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0574101e20 0x7f05740ffa80 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.745+0000 7f057a8f3700 1 --2- 172.21.15.49:0/1664569252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0574102740 0x7f05740fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.745+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 >> 172.21.15.49:0/1664569252 conn(0x7f0574076800 msgr2=0x7f0574103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.745+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 shutdown_connections 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.745+0000 7f057a8f3700 1 -- 172.21.15.49:0/1664569252 wait complete. 2024-05-15T16:27:28.747 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:27:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.853+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1783435533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6781007a0 msgr2=0x7fc6781047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.854+0000 7fc67ed1f700 1 --2- 172.21.15.49:0/1783435533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6781007a0 0x7fc6781047f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc664009910 tx=0x7fc664009c20 comp rx=0 tx=0).stop 2024-05-15T16:27:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.854+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1783435533 shutdown_connections 2024-05-15T16:27:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.854+0000 7fc67ed1f700 1 --2- 172.21.15.49:0/1783435533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6781007a0 0x7fc6781047f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.854+0000 7fc67ed1f700 1 --2- 172.21.15.49:0/1783435533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6780ffdf0 0x7fc6781001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.854+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1783435533 >> 172.21.15.49:0/1783435533 conn(0x7fc6780fb6a0 msgr2=0x7fc6780fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.854+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1783435533 shutdown_connections 2024-05-15T16:27:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.854+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1783435533 wait complete. 2024-05-15T16:27:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.854+0000 7fc67ed1f700 1 Processor -- start 2024-05-15T16:27:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.854+0000 7fc67ed1f700 1 -- start start 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67ed1f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6780ffdf0 0x7fc67819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67ed1f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6781007a0 0x7fc67819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67ed1f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67819d590 con 0x7fc6780ffdf0 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67ed1f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc678196ab0 con 0x7fc6781007a0 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67cabb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6780ffdf0 0x7fc67819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67cabb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6780ffdf0 0x7fc67819ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53110/0 (socket says 172.21.15.49:53110) 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67cabb700 1 -- 172.21.15.49:0/1203381621 learned_addr learned my addr 172.21.15.49:0/1203381621 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67cabb700 1 -- 172.21.15.49:0/1203381621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6781007a0 msgr2=0x7fc67819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67cabb700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6781007a0 0x7fc67819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67cabb700 1 -- 172.21.15.49:0/1203381621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6640095f0 con 0x7fc6780ffdf0 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67cabb700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6780ffdf0 0x7fc67819ca00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c930 tx=0x7fc66c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc675ffb700 1 -- 172.21.15.49:0/1203381621 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc66c007ad0 con 0x7fc6780ffdf0 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc678196d90 con 0x7fc6780ffdf0 2024-05-15T16:27:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc675ffb700 1 -- 172.21.15.49:0/1203381621 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc66c007c30 con 0x7fc6780ffdf0 2024-05-15T16:27:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.855+0000 7fc675ffb700 1 -- 172.21.15.49:0/1203381621 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc66c018610 con 0x7fc6780ffdf0 2024-05-15T16:27:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.856+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6781972e0 con 0x7fc6780ffdf0 2024-05-15T16:27:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.857+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6781012d0 con 0x7fc6780ffdf0 2024-05-15T16:27:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.857+0000 7fc675ffb700 1 -- 172.21.15.49:0/1203381621 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc66c00f450 con 0x7fc6780ffdf0 2024-05-15T16:27:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.857+0000 7fc675ffb700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc66806e200 0x7fc6680706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.858+0000 7fc675ffb700 1 -- 172.21.15.49:0/1203381621 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc66c08e3a0 con 0x7fc6780ffdf0 2024-05-15T16:27:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.858+0000 7fc677fff700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc66806e200 0x7fc6680706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.858+0000 7fc677fff700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc66806e200 0x7fc6680706c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc6640059d0 tx=0x7fc664005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:28.862+0000 7fc675ffb700 1 -- 172.21.15.49:0/1203381621 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc66c057680 con 0x7fc6780ffdf0 2024-05-15T16:27:29.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.087+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc67810cd90 con 0x7fc66806e200 2024-05-15T16:27:29.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.089+0000 7fc675ffb700 1 -- 172.21.15.49:0/1203381621 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc67810cd90 con 0x7fc66806e200 2024-05-15T16:27:29.089 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:27:29.089 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:27:29.089 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:27:29.089 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:27:29.089 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:27:29.090 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:27:29.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc66806e200 msgr2=0x7fc6680706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:29.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc66806e200 0x7fc6680706c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc6640059d0 tx=0x7fc664005960 comp rx=0 tx=0).stop 2024-05-15T16:27:29.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6780ffdf0 msgr2=0x7fc67819ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:29.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6780ffdf0 0x7fc67819ca00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c930 tx=0x7fc66c00ccf0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 shutdown_connections 2024-05-15T16:27:29.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc66806e200 0x7fc6680706c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6780ffdf0 0x7fc67819ca00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 --2- 172.21.15.49:0/1203381621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6781007a0 0x7fc67819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 >> 172.21.15.49:0/1203381621 conn(0x7fc6780fb6a0 msgr2=0x7fc678069590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:29.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 shutdown_connections 2024-05-15T16:27:29.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.092+0000 7fc67ed1f700 1 -- 172.21.15.49:0/1203381621 wait complete. 2024-05-15T16:27:29.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.200+0000 7f23b4ef2700 1 -- 172.21.15.49:0/2885953350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b01027f0 msgr2=0x7f23b010ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:29.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.200+0000 7f23b4ef2700 1 --2- 172.21.15.49:0/2885953350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b01027f0 0x7f23b010ace0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009940 tx=0x7f23a4009c50 comp rx=0 tx=0).stop 2024-05-15T16:27:29.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.200+0000 7f23b4ef2700 1 -- 172.21.15.49:0/2885953350 shutdown_connections 2024-05-15T16:27:29.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.200+0000 7f23b4ef2700 1 --2- 172.21.15.49:0/2885953350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b01027f0 0x7f23b010ace0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.200+0000 7f23b4ef2700 1 --2- 172.21.15.49:0/2885953350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b0101ed0 0x7f23b01022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.200+0000 7f23b4ef2700 1 -- 172.21.15.49:0/2885953350 >> 172.21.15.49:0/2885953350 conn(0x7f23b00fb700 msgr2=0x7f23b00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:29.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.200+0000 7f23b4ef2700 1 -- 172.21.15.49:0/2885953350 shutdown_connections 2024-05-15T16:27:29.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.200+0000 7f23b4ef2700 1 -- 172.21.15.49:0/2885953350 wait complete. 2024-05-15T16:27:29.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23b4ef2700 1 Processor -- start 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23b4ef2700 1 -- start start 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23b4ef2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b0101ed0 0x7f23b019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23b4ef2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b01027f0 0x7f23b019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23b4ef2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b019d4f0 con 0x7f23b0101ed0 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23b4ef2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0196af0 con 0x7f23b01027f0 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23ae59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b0101ed0 0x7f23b019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23ae59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b0101ed0 0x7f23b019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53132/0 (socket says 172.21.15.49:53132) 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23ae59c700 1 -- 172.21.15.49:0/3973970957 learned_addr learned my addr 172.21.15.49:0/3973970957 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23add9b700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b01027f0 0x7f23b019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23ae59c700 1 -- 172.21.15.49:0/3973970957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b01027f0 msgr2=0x7f23b019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23ae59c700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b01027f0 0x7f23b019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23ae59c700 1 -- 172.21.15.49:0/3973970957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a40095f0 con 0x7f23b0101ed0 2024-05-15T16:27:29.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.201+0000 7f23add9b700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b01027f0 0x7f23b019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:27:29.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.202+0000 7f23ae59c700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b0101ed0 0x7f23b019c9b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f23a000c930 tx=0x7f23a000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:29.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.202+0000 7f239f7fe700 1 -- 172.21.15.49:0/3973970957 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23a0007ad0 con 0x7f23b0101ed0 2024-05-15T16:27:29.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.202+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b0196dd0 con 0x7f23b0101ed0 2024-05-15T16:27:29.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.202+0000 7f239f7fe700 1 -- 172.21.15.49:0/3973970957 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23a0007c30 con 0x7f23b0101ed0 2024-05-15T16:27:29.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.202+0000 7f239f7fe700 1 -- 172.21.15.49:0/3973970957 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23a0018610 con 0x7f23b0101ed0 2024-05-15T16:27:29.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.202+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b0197320 con 0x7f23b0101ed0 2024-05-15T16:27:29.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.203+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b0069a50 con 0x7f23b0101ed0 2024-05-15T16:27:29.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.203+0000 7f239f7fe700 1 -- 172.21.15.49:0/3973970957 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f23a000f450 con 0x7f23b0101ed0 2024-05-15T16:27:29.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.204+0000 7f239f7fe700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f239806e200 0x7f23980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:29.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.204+0000 7f239f7fe700 1 -- 172.21.15.49:0/3973970957 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f23a008e360 con 0x7f23b0101ed0 2024-05-15T16:27:29.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.204+0000 7f23add9b700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f239806e200 0x7f23980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:29.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.204+0000 7f23add9b700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f239806e200 0x7f23980706c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f23a4006010 tx=0x7f23a4005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:29.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.209+0000 7f239f7fe700 1 -- 172.21.15.49:0/3973970957 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f23a005b170 con 0x7f23b0101ed0 2024-05-15T16:27:29.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:29 smithi144 ceph-mon[86635]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:27:29.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:29 smithi144 ceph-mon[86635]: from='client.35666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:27:29.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:29 smithi144 ceph-mon[86635]: from='client.35670 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:27:29.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:29 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2985167214' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:27:29.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:29 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1664569252' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:27:29.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:29 smithi049 ceph-mon[112142]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:27:29.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:29 smithi049 ceph-mon[112142]: from='client.35666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:27:29.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:29 smithi049 ceph-mon[112142]: from='client.35670 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:27:29.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:29 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2985167214' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:27:29.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:29 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1664569252' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:27:29.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.500+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f23b004ec60 con 0x7f23b0101ed0 2024-05-15T16:27:29.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.500+0000 7f239f7fe700 1 -- 172.21.15.49:0/3973970957 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f23a005ad00 con 0x7f23b0101ed0 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:27:29.501 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:27:29.502 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:27:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.502+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f239806e200 msgr2=0x7f23980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.502+0000 7f23b4ef2700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f239806e200 0x7f23980706c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f23a4006010 tx=0x7f23a4005960 comp rx=0 tx=0).stop 2024-05-15T16:27:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.502+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b0101ed0 msgr2=0x7f23b019c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.502+0000 7f23b4ef2700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b0101ed0 0x7f23b019c9b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f23a000c930 tx=0x7f23a000ccf0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.502+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 shutdown_connections 2024-05-15T16:27:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.502+0000 7f23b4ef2700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f239806e200 0x7f23980706c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.502+0000 7f23b4ef2700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23b0101ed0 0x7f23b019c9b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.502+0000 7f23b4ef2700 1 --2- 172.21.15.49:0/3973970957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b01027f0 0x7f23b019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:29.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.503+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 >> 172.21.15.49:0/3973970957 conn(0x7f23b00fb700 msgr2=0x7f23b01031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:29.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.503+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 shutdown_connections 2024-05-15T16:27:29.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:29.503+0000 7f23b4ef2700 1 -- 172.21.15.49:0/3973970957 wait complete. 2024-05-15T16:27:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:30 smithi144 ceph-mon[86635]: pgmap v915: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:30 smithi144 ceph-mon[86635]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:27:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:30 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3973970957' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:27:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:30 smithi049 ceph-mon[112142]: pgmap v915: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:30 smithi049 ceph-mon[112142]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:27:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:30 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3973970957' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:27:32.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:32 smithi144 ceph-mon[86635]: pgmap v916: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:32 smithi049 ceph-mon[112142]: pgmap v916: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:34 smithi144 ceph-mon[86635]: pgmap v917: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:34.453 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:34 smithi049 ceph-mon[112142]: pgmap v917: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:36 smithi144 ceph-mon[86635]: pgmap v918: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:27:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:36 smithi049 ceph-mon[112142]: pgmap v918: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:27:38.373 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:38 smithi049 ceph-mon[112142]: pgmap v919: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:38 smithi144 ceph-mon[86635]: pgmap v919: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:40 smithi144 ceph-mon[86635]: pgmap v920: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:40 smithi049 ceph-mon[112142]: pgmap v920: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:42 smithi144 ceph-mon[86635]: pgmap v921: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:42 smithi049 ceph-mon[112142]: pgmap v921: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:44 smithi144 ceph-mon[86635]: pgmap v922: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:44 smithi049 ceph-mon[112142]: pgmap v922: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:46 smithi144 ceph-mon[86635]: pgmap v923: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:46 smithi049 ceph-mon[112142]: pgmap v923: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:48 smithi144 ceph-mon[86635]: pgmap v924: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:48.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:48 smithi049 ceph-mon[112142]: pgmap v924: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:50 smithi144 ceph-mon[86635]: pgmap v925: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:50.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:50 smithi049 ceph-mon[112142]: pgmap v925: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:51.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:27:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:27:52.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:52 smithi144 ceph-mon[86635]: pgmap v926: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:52 smithi049 ceph-mon[112142]: pgmap v926: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:54 smithi144 ceph-mon[86635]: pgmap v927: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:54 smithi049 ceph-mon[112142]: pgmap v927: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:56 smithi144 ceph-mon[86635]: pgmap v928: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:56 smithi049 ceph-mon[112142]: pgmap v928: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:27:58 smithi144 ceph-mon[86635]: pgmap v929: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:58.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:27:58 smithi049 ceph-mon[112142]: pgmap v929: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:27:59.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.626+0000 7f5b196cc700 1 -- 172.21.15.49:0/2894389653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 msgr2=0x7f5b1410acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.626+0000 7f5b196cc700 1 --2- 172.21.15.49:0/2894389653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 0x7f5b1410acc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009940 tx=0x7f5afc009c50 comp rx=0 tx=0).stop 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.626+0000 7f5b196cc700 1 -- 172.21.15.49:0/2894389653 shutdown_connections 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.626+0000 7f5b196cc700 1 --2- 172.21.15.49:0/2894389653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 0x7f5b1410acc0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.626+0000 7f5b196cc700 1 --2- 172.21.15.49:0/2894389653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b14101eb0 0x7f5b14102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.626+0000 7f5b196cc700 1 -- 172.21.15.49:0/2894389653 >> 172.21.15.49:0/2894389653 conn(0x7f5b140fb6a0 msgr2=0x7f5b140fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.626+0000 7f5b196cc700 1 -- 172.21.15.49:0/2894389653 shutdown_connections 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.626+0000 7f5b196cc700 1 -- 172.21.15.49:0/2894389653 wait complete. 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.626+0000 7f5b196cc700 1 Processor -- start 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.627+0000 7f5b196cc700 1 -- start start 2024-05-15T16:27:59.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.627+0000 7f5b196cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b14101eb0 0x7f5b140ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:59.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.627+0000 7f5b196cc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 0x7f5b14100050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:59.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.627+0000 7f5b196cc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14100590 con 0x7f5b141027d0 2024-05-15T16:27:59.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.627+0000 7f5b196cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141006d0 con 0x7f5b14101eb0 2024-05-15T16:27:59.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.627+0000 7f5b127fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 0x7f5b14100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:59.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.627+0000 7f5b127fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 0x7f5b14100050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32906/0 (socket says 172.21.15.49:32906) 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.627+0000 7f5b127fc700 1 -- 172.21.15.49:0/2627124705 learned_addr learned my addr 172.21.15.49:0/2627124705 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.627+0000 7f5b12ffd700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b14101eb0 0x7f5b140ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.628+0000 7f5b127fc700 1 -- 172.21.15.49:0/2627124705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b14101eb0 msgr2=0x7f5b140ffb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.628+0000 7f5b127fc700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b14101eb0 0x7f5b140ffb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.628+0000 7f5b127fc700 1 -- 172.21.15.49:0/2627124705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0095f0 con 0x7f5b141027d0 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.628+0000 7f5b127fc700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 0x7f5b14100050 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00b580 tx=0x7f5afc0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.628+0000 7f5b0bfff700 1 -- 172.21.15.49:0/2627124705 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5afc01d070 con 0x7f5b141027d0 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.628+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b14100950 con 0x7f5b141027d0 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.628+0000 7f5b0bfff700 1 -- 172.21.15.49:0/2627124705 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5afc003d50 con 0x7f5b141027d0 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.628+0000 7f5b0bfff700 1 -- 172.21.15.49:0/2627124705 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5afc021410 con 0x7f5b141027d0 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.628+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b141a3050 con 0x7f5b141027d0 2024-05-15T16:27:59.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.629+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b14069a50 con 0x7f5b141027d0 2024-05-15T16:27:59.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.630+0000 7f5b0bfff700 1 -- 172.21.15.49:0/2627124705 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5afc00f6f0 con 0x7f5b141027d0 2024-05-15T16:27:59.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.630+0000 7f5b0bfff700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b0006e210 0x7f5b000706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:59.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.630+0000 7f5b0bfff700 1 -- 172.21.15.49:0/2627124705 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5afc08f2c0 con 0x7f5b141027d0 2024-05-15T16:27:59.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.630+0000 7f5b12ffd700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b0006e210 0x7f5b000706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:59.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.631+0000 7f5b12ffd700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b0006e210 0x7f5b000706d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5b04005950 tx=0x7f5b0400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:59.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.635+0000 7f5b0bfff700 1 -- 172.21.15.49:0/2627124705 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5afc0585a0 con 0x7f5b141027d0 2024-05-15T16:27:59.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.864+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b1410deb0 con 0x7f5b0006e210 2024-05-15T16:27:59.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.865+0000 7f5b0bfff700 1 -- 172.21.15.49:0/2627124705 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5b1410deb0 con 0x7f5b0006e210 2024-05-15T16:27:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.867+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b0006e210 msgr2=0x7f5b000706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.867+0000 7f5b196cc700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b0006e210 0x7f5b000706d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5b04005950 tx=0x7f5b0400b410 comp rx=0 tx=0).stop 2024-05-15T16:27:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.867+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 msgr2=0x7f5b14100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.867+0000 7f5b196cc700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 0x7f5b14100050 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00b580 tx=0x7f5afc0055f0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.868+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 shutdown_connections 2024-05-15T16:27:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.868+0000 7f5b196cc700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5b0006e210 0x7f5b000706d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.868+0000 7f5b196cc700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b14101eb0 0x7f5b140ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.868+0000 7f5b196cc700 1 --2- 172.21.15.49:0/2627124705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b141027d0 0x7f5b14100050 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.868+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 >> 172.21.15.49:0/2627124705 conn(0x7f5b140fb6a0 msgr2=0x7f5b141031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.868+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 shutdown_connections 2024-05-15T16:27:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.868+0000 7f5b196cc700 1 -- 172.21.15.49:0/2627124705 wait complete. 2024-05-15T16:27:59.878 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:27:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.976+0000 7fe9ed11e700 1 -- 172.21.15.49:0/643386084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 msgr2=0x7fe9e8105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.976+0000 7fe9ed11e700 1 --2- 172.21.15.49:0/643386084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 0x7fe9e8105900 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009910 tx=0x7fe9dc009c20 comp rx=0 tx=0).stop 2024-05-15T16:27:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.976+0000 7fe9ed11e700 1 -- 172.21.15.49:0/643386084 shutdown_connections 2024-05-15T16:27:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.976+0000 7fe9ed11e700 1 --2- 172.21.15.49:0/643386084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 0x7fe9e8105900 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.976+0000 7fe9ed11e700 1 --2- 172.21.15.49:0/643386084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8100f00 0x7fe9e81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.976+0000 7fe9ed11e700 1 -- 172.21.15.49:0/643386084 >> 172.21.15.49:0/643386084 conn(0x7fe9e80fc7d0 msgr2=0x7fe9e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:27:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.977+0000 7fe9ed11e700 1 -- 172.21.15.49:0/643386084 shutdown_connections 2024-05-15T16:27:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.977+0000 7fe9ed11e700 1 -- 172.21.15.49:0/643386084 wait complete. 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.977+0000 7fe9ed11e700 1 Processor -- start 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9ed11e700 1 -- start start 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9ed11e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8100f00 0x7fe9e80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9ed11e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 0x7fe9e80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9ed11e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e8073d20 con 0x7fe9e81018b0 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9ed11e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e8073e60 con 0x7fe9e8100f00 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9e659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 0x7fe9e80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9e659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 0x7fe9e80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32926/0 (socket says 172.21.15.49:32926) 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9e659c700 1 -- 172.21.15.49:0/3689690878 learned_addr learned my addr 172.21.15.49:0/3689690878 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:27:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9e6d9d700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8100f00 0x7fe9e80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:59.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9e659c700 1 -- 172.21.15.49:0/3689690878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8100f00 msgr2=0x7fe9e80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:27:59.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9e659c700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8100f00 0x7fe9e80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:27:59.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9e659c700 1 -- 172.21.15.49:0/3689690878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc0095f0 con 0x7fe9e81018b0 2024-05-15T16:27:59.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9e659c700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 0x7fe9e80737e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0092e0 tx=0x7fe9dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:59.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.978+0000 7fe9cffff700 1 -- 172.21.15.49:0/3689690878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9dc01d070 con 0x7fe9e81018b0 2024-05-15T16:27:59.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.979+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e80740e0 con 0x7fe9e81018b0 2024-05-15T16:27:59.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.979+0000 7fe9cffff700 1 -- 172.21.15.49:0/3689690878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9dc00f460 con 0x7fe9e81018b0 2024-05-15T16:27:59.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.979+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e81a3120 con 0x7fe9e81018b0 2024-05-15T16:27:59.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.979+0000 7fe9cffff700 1 -- 172.21.15.49:0/3689690878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9dc017630 con 0x7fe9e81018b0 2024-05-15T16:27:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.980+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e8109250 con 0x7fe9e81018b0 2024-05-15T16:27:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.980+0000 7fe9cffff700 1 -- 172.21.15.49:0/3689690878 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe9dc00f5d0 con 0x7fe9e81018b0 2024-05-15T16:27:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.981+0000 7fe9cffff700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe9d006e200 0x7fe9d00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:27:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.981+0000 7fe9cffff700 1 -- 172.21.15.49:0/3689690878 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe9dc08f5b0 con 0x7fe9e81018b0 2024-05-15T16:27:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.981+0000 7fe9e6d9d700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe9d006e200 0x7fe9d00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:27:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.981+0000 7fe9e6d9d700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe9d006e200 0x7fe9d00706c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006fd0 tx=0x7fe9d8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:27:59.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:27:59.985+0000 7fe9cffff700 1 -- 172.21.15.49:0/3689690878 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe9dc058890 con 0x7fe9e81018b0 2024-05-15T16:28:00.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.212+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9e8074780 con 0x7fe9d006e200 2024-05-15T16:28:00.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.213+0000 7fe9cffff700 1 -- 172.21.15.49:0/3689690878 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe9e8074780 con 0x7fe9d006e200 2024-05-15T16:28:00.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe9d006e200 msgr2=0x7fe9d00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe9d006e200 0x7fe9d00706c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006fd0 tx=0x7fe9d8008040 comp rx=0 tx=0).stop 2024-05-15T16:28:00.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 msgr2=0x7fe9e80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 0x7fe9e80737e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0092e0 tx=0x7fe9dc00be20 comp rx=0 tx=0).stop 2024-05-15T16:28:00.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 shutdown_connections 2024-05-15T16:28:00.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe9d006e200 0x7fe9d00706c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8100f00 0x7fe9e80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 --2- 172.21.15.49:0/3689690878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9e81018b0 0x7fe9e80737e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 >> 172.21.15.49:0/3689690878 conn(0x7fe9e80fc7d0 msgr2=0x7fe9e80fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:00.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 shutdown_connections 2024-05-15T16:28:00.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.215+0000 7fe9ed11e700 1 -- 172.21.15.49:0/3689690878 wait complete. 2024-05-15T16:28:00.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 -- 172.21.15.49:0/190330724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 msgr2=0x7f6eb8073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 --2- 172.21.15.49:0/190330724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 0x7f6eb8073720 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009910 tx=0x7f6eb4009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:00.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 -- 172.21.15.49:0/190330724 shutdown_connections 2024-05-15T16:28:00.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 --2- 172.21.15.49:0/190330724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb8073c60 0x7f6eb8111750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 --2- 172.21.15.49:0/190330724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 0x7f6eb8073720 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 -- 172.21.15.49:0/190330724 >> 172.21.15.49:0/190330724 conn(0x7f6eb80fc790 msgr2=0x7f6eb80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:00.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 -- 172.21.15.49:0/190330724 shutdown_connections 2024-05-15T16:28:00.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 -- 172.21.15.49:0/190330724 wait complete. 2024-05-15T16:28:00.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 Processor -- start 2024-05-15T16:28:00.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.324+0000 7f6ebf85a700 1 -- start start 2024-05-15T16:28:00.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebf85a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 0x7f6eb819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebf85a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb8073c60 0x7f6eb819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebf85a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb819dd70 con 0x7f6eb8073340 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebf85a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb81a1a40 con 0x7f6eb8073c60 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebd5f6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 0x7f6eb819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebd5f6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 0x7f6eb819d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32942/0 (socket says 172.21.15.49:32942) 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebd5f6700 1 -- 172.21.15.49:0/2692101260 learned_addr learned my addr 172.21.15.49:0/2692101260 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebcdf5700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb8073c60 0x7f6eb819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebd5f6700 1 -- 172.21.15.49:0/2692101260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb8073c60 msgr2=0x7f6eb819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebd5f6700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb8073c60 0x7f6eb819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebd5f6700 1 -- 172.21.15.49:0/2692101260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40095f0 con 0x7f6eb8073340 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.325+0000 7f6ebd5f6700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 0x7f6eb819d150 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005f10 tx=0x7f6eb400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.326+0000 7f6eae7fc700 1 -- 172.21.15.49:0/2692101260 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb401d070 con 0x7f6eb8073340 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.326+0000 7f6eae7fc700 1 -- 172.21.15.49:0/2692101260 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eb40052a0 con 0x7f6eb8073340 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.326+0000 7f6eae7fc700 1 -- 172.21.15.49:0/2692101260 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb4003e50 con 0x7f6eb8073340 2024-05-15T16:28:00.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.326+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb81a1cc0 con 0x7f6eb8073340 2024-05-15T16:28:00.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.326+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb81a2210 con 0x7f6eb8073340 2024-05-15T16:28:00.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.327+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb8074a80 con 0x7f6eb8073340 2024-05-15T16:28:00.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.327+0000 7f6eae7fc700 1 -- 172.21.15.49:0/2692101260 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6eb400f6f0 con 0x7f6eb8073340 2024-05-15T16:28:00.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.328+0000 7f6eae7fc700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ea406e200 0x7f6ea40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:00.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.328+0000 7f6eae7fc700 1 -- 172.21.15.49:0/2692101260 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6eb408f490 con 0x7f6eb8073340 2024-05-15T16:28:00.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.328+0000 7f6ebcdf5700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ea406e200 0x7f6ea40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:00.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.328+0000 7f6ebcdf5700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ea406e200 0x7f6ea40706c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6eb819e6b0 tx=0x7f6ea8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:00.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.332+0000 7f6eae7fc700 1 -- 172.21.15.49:0/2692101260 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6eb40587f0 con 0x7f6eb8073340 2024-05-15T16:28:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:00 smithi144 ceph-mon[86635]: pgmap v930: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:00 smithi049 ceph-mon[112142]: pgmap v930: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:00.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.549+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6eb81144e0 con 0x7f6ea406e200 2024-05-15T16:28:00.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.560+0000 7f6eae7fc700 1 -- 172.21.15.49:0/2692101260 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6eb81144e0 con 0x7f6ea406e200 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (31m) 9m ago 42m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (43m) 9m ago 43m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (41m) 9m ago 41m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (28m) 9m ago 43m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (28m) 9m ago 41m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (30m) 9m ago 42m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (24m) 9m ago 37m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (24m) 9m ago 37m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (24m) 9m ago 37m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (9m) 9m ago 37m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (33m) 9m ago 44m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:28:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (32m) 9m ago 41m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (29m) 9m ago 44m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (29m) 9m ago 41m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (32m) 9m ago 42m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (31m) 9m ago 41m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (28m) 9m ago 40m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (27m) 9m ago 40m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (26m) 9m ago 39m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (26m) 9m ago 39m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (25m) 9m ago 39m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (25m) 9m ago 38m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:28:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (31m) 9m ago 42m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:28:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ea406e200 msgr2=0x7f6ea40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ea406e200 0x7f6ea40706c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6eb819e6b0 tx=0x7f6ea8009260 comp rx=0 tx=0).stop 2024-05-15T16:28:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 msgr2=0x7f6eb819d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 0x7f6eb819d150 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005f10 tx=0x7f6eb400b5a0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 shutdown_connections 2024-05-15T16:28:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6ea406e200 0x7f6ea40706c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb8073340 0x7f6eb819d150 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 --2- 172.21.15.49:0/2692101260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb8073c60 0x7f6eb819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 >> 172.21.15.49:0/2692101260 conn(0x7f6eb80fc790 msgr2=0x7f6eb81009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:00.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 shutdown_connections 2024-05-15T16:28:00.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.563+0000 7f6ebf85a700 1 -- 172.21.15.49:0/2692101260 wait complete. 2024-05-15T16:28:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.670+0000 7f597c853700 1 -- 172.21.15.49:0/2866179272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59780ffdf0 msgr2=0x7f59781001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.670+0000 7f597c853700 1 --2- 172.21.15.49:0/2866179272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59780ffdf0 0x7f59781001d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5968009910 tx=0x7f5968009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.670+0000 7f597c853700 1 -- 172.21.15.49:0/2866179272 shutdown_connections 2024-05-15T16:28:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.670+0000 7f597c853700 1 --2- 172.21.15.49:0/2866179272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59781007a0 0x7f59781047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.670+0000 7f597c853700 1 --2- 172.21.15.49:0/2866179272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59780ffdf0 0x7f59781001d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.670+0000 7f597c853700 1 -- 172.21.15.49:0/2866179272 >> 172.21.15.49:0/2866179272 conn(0x7f59780fb6a0 msgr2=0x7f59780fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.670+0000 7f597c853700 1 -- 172.21.15.49:0/2866179272 shutdown_connections 2024-05-15T16:28:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.670+0000 7f597c853700 1 -- 172.21.15.49:0/2866179272 wait complete. 2024-05-15T16:28:00.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.671+0000 7f597c853700 1 Processor -- start 2024-05-15T16:28:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.671+0000 7f597c853700 1 -- start start 2024-05-15T16:28:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.671+0000 7f597c853700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59780ffdf0 0x7f597819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.671+0000 7f597c853700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59781007a0 0x7f597819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:00.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.671+0000 7f597c853700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5978196b00 con 0x7f59781007a0 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.671+0000 7f597c853700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5978196c70 con 0x7f59780ffdf0 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59781007a0 0x7f597819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59781007a0 0x7f597819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32974/0 (socket says 172.21.15.49:32974) 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596dd9b700 1 -- 172.21.15.49:0/4244774675 learned_addr learned my addr 172.21.15.49:0/4244774675 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f597659c700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59780ffdf0 0x7f597819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596dd9b700 1 -- 172.21.15.49:0/4244774675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59780ffdf0 msgr2=0x7f597819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596dd9b700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59780ffdf0 0x7f597819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596dd9b700 1 -- 172.21.15.49:0/4244774675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59680095f0 con 0x7f59781007a0 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f597659c700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59780ffdf0 0x7f597819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596dd9b700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59781007a0 0x7f597819cf90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f596000b480 tx=0x7f596000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596ffff700 1 -- 172.21.15.49:0/4244774675 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59600118c0 con 0x7f59781007a0 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596ffff700 1 -- 172.21.15.49:0/4244774675 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5960010380 con 0x7f59781007a0 2024-05-15T16:28:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f596ffff700 1 -- 172.21.15.49:0/4244774675 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f596000f4f0 con 0x7f59781007a0 2024-05-15T16:28:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5978196f50 con 0x7f59781007a0 2024-05-15T16:28:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.672+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5978197450 con 0x7f59781007a0 2024-05-15T16:28:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.674+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59781012d0 con 0x7f59781007a0 2024-05-15T16:28:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.674+0000 7f596ffff700 1 -- 172.21.15.49:0/4244774675 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f59600104f0 con 0x7f59781007a0 2024-05-15T16:28:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.674+0000 7f596ffff700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596406e200 0x7f59640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.675+0000 7f596ffff700 1 -- 172.21.15.49:0/4244774675 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f596008dde0 con 0x7f59781007a0 2024-05-15T16:28:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.675+0000 7f597659c700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596406e200 0x7f59640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.675+0000 7f597659c700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596406e200 0x7f59640706c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5968006010 tx=0x7f596800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:00.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.679+0000 7f596ffff700 1 -- 172.21.15.49:0/4244774675 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f59600570c0 con 0x7f59781007a0 2024-05-15T16:28:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.973+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f597810ce30 con 0x7f59781007a0 2024-05-15T16:28:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.974+0000 7f596ffff700 1 -- 172.21.15.49:0/4244774675 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f596005a780 con 0x7f59781007a0 2024-05-15T16:28:00.974 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:28:00.974 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:28:00.974 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:28:00.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:28:00.974 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:28:00.974 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:28:00.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:28:00.974 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:28:00.975 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:28:00.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:28:00.975 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:28:00.975 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:28:00.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:28:00.975 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:28:00.975 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:28:00.975 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:28:00.975 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596406e200 msgr2=0x7f59640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596406e200 0x7f59640706c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5968006010 tx=0x7f596800b540 comp rx=0 tx=0).stop 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59781007a0 msgr2=0x7f597819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59781007a0 0x7f597819cf90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f596000b480 tx=0x7f596000b840 comp rx=0 tx=0).stop 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 shutdown_connections 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f596406e200 0x7f59640706c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59780ffdf0 0x7f597819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 --2- 172.21.15.49:0/4244774675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59781007a0 0x7f597819cf90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 >> 172.21.15.49:0/4244774675 conn(0x7f59780fb6a0 msgr2=0x7f59780695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 shutdown_connections 2024-05-15T16:28:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:00.976+0000 7f597c853700 1 -- 172.21.15.49:0/4244774675 wait complete. 2024-05-15T16:28:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.084+0000 7fa49cc58700 1 -- 172.21.15.49:0/286829199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa498100770 msgr2=0x7fa4981047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.084+0000 7fa49cc58700 1 --2- 172.21.15.49:0/286829199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa498100770 0x7fa4981047c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009910 tx=0x7fa48c009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.084+0000 7fa49cc58700 1 -- 172.21.15.49:0/286829199 shutdown_connections 2024-05-15T16:28:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.084+0000 7fa49cc58700 1 --2- 172.21.15.49:0/286829199 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa498100770 0x7fa4981047c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.084+0000 7fa49cc58700 1 --2- 172.21.15.49:0/286829199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4980ffdc0 0x7fa4981001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.084+0000 7fa49cc58700 1 -- 172.21.15.49:0/286829199 >> 172.21.15.49:0/286829199 conn(0x7fa498076800 msgr2=0x7fa4980fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.084+0000 7fa49cc58700 1 -- 172.21.15.49:0/286829199 shutdown_connections 2024-05-15T16:28:01.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.084+0000 7fa49cc58700 1 -- 172.21.15.49:0/286829199 wait complete. 2024-05-15T16:28:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.085+0000 7fa49cc58700 1 Processor -- start 2024-05-15T16:28:01.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.085+0000 7fa49cc58700 1 -- start start 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49cc58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4980ffdc0 0x7fa4981979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49cc58700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa498100770 0x7fa498197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49cc58700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4981984f0 con 0x7fa4980ffdc0 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49cc58700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa498198630 con 0x7fa498100770 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4980ffdc0 0x7fa4981979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4980ffdc0 0x7fa4981979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32984/0 (socket says 172.21.15.49:32984) 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49659c700 1 -- 172.21.15.49:0/3226527866 learned_addr learned my addr 172.21.15.49:0/3226527866 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa495d9b700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa498100770 0x7fa498197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49659c700 1 -- 172.21.15.49:0/3226527866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa498100770 msgr2=0x7fa498197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49659c700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa498100770 0x7fa498197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49659c700 1 -- 172.21.15.49:0/3226527866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0095f0 con 0x7fa4980ffdc0 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa495d9b700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa498100770 0x7fa498197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:28:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa49659c700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4980ffdc0 0x7fa4981979e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa48800d8d0 tx=0x7fa48800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa4877fe700 1 -- 172.21.15.49:0/3226527866 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa488009940 con 0x7fa4980ffdc0 2024-05-15T16:28:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.086+0000 7fa4877fe700 1 -- 172.21.15.49:0/3226527866 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa488010460 con 0x7fa4980ffdc0 2024-05-15T16:28:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.087+0000 7fa4877fe700 1 -- 172.21.15.49:0/3226527866 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa48800f5f0 con 0x7fa4980ffdc0 2024-05-15T16:28:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.087+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4981a0a50 con 0x7fa4980ffdc0 2024-05-15T16:28:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.087+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4981a0e10 con 0x7fa4980ffdc0 2024-05-15T16:28:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.088+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4981081c0 con 0x7fa4980ffdc0 2024-05-15T16:28:01.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.088+0000 7fa4877fe700 1 -- 172.21.15.49:0/3226527866 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa4880105d0 con 0x7fa4980ffdc0 2024-05-15T16:28:01.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.088+0000 7fa4877fe700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa48006e200 0x7fa4800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:01.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.089+0000 7fa4877fe700 1 -- 172.21.15.49:0/3226527866 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa48808e1a0 con 0x7fa4980ffdc0 2024-05-15T16:28:01.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.089+0000 7fa495d9b700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa48006e200 0x7fa4800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:01.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.089+0000 7fa495d9b700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa48006e200 0x7fa4800706c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa498068e90 tx=0x7fa48c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.093+0000 7fa4877fe700 1 -- 172.21.15.49:0/3226527866 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa488057480 con 0x7fa4980ffdc0 2024-05-15T16:28:01.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.348+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa4981012a0 con 0x7fa4980ffdc0 2024-05-15T16:28:01.349 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:01 smithi049 ceph-mon[112142]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:01.349 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:01 smithi049 ceph-mon[112142]: from='client.35694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:01.349 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:01 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4244774675' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:28:01.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.348+0000 7fa4877fe700 1 -- 172.21.15.49:0/3226527866 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fa48805ab40 con 0x7fa4980ffdc0 2024-05-15T16:28:01.349 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:28:01.349 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:28:01.349 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:28:01.349 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:28:01.350 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:28:01.351 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:28:01.352 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:28:01.352 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:28:01.352 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:28:01.352 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:28:01.352 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:28:01.352 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:28:01.352 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:28:01.352 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:28:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.350+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa48006e200 msgr2=0x7fa4800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.350+0000 7fa49cc58700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa48006e200 0x7fa4800706c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa498068e90 tx=0x7fa48c01a040 comp rx=0 tx=0).stop 2024-05-15T16:28:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.351+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4980ffdc0 msgr2=0x7fa4981979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.351+0000 7fa49cc58700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4980ffdc0 0x7fa4981979e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa48800d8d0 tx=0x7fa48800dc90 comp rx=0 tx=0).stop 2024-05-15T16:28:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.351+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 shutdown_connections 2024-05-15T16:28:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.351+0000 7fa49cc58700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa48006e200 0x7fa4800706c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.351+0000 7fa49cc58700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4980ffdc0 0x7fa4981979e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.351+0000 7fa49cc58700 1 --2- 172.21.15.49:0/3226527866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa498100770 0x7fa498197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.351+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 >> 172.21.15.49:0/3226527866 conn(0x7fa498076800 msgr2=0x7fa498103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:01.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.351+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 shutdown_connections 2024-05-15T16:28:01.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.351+0000 7fa49cc58700 1 -- 172.21.15.49:0/3226527866 wait complete. 2024-05-15T16:28:01.354 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:28:01.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:01 smithi144 ceph-mon[86635]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:01.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:01 smithi144 ceph-mon[86635]: from='client.35694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:01.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:01 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4244774675' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:28:01.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.460+0000 7f2499f92700 1 -- 172.21.15.49:0/3694081703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494101890 msgr2=0x7f24941058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.460+0000 7f2499f92700 1 --2- 172.21.15.49:0/3694081703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494101890 0x7f24941058e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f247c009910 tx=0x7f247c009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.460+0000 7f2499f92700 1 -- 172.21.15.49:0/3694081703 shutdown_connections 2024-05-15T16:28:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.460+0000 7f2499f92700 1 --2- 172.21.15.49:0/3694081703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494101890 0x7f24941058e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.460+0000 7f2499f92700 1 --2- 172.21.15.49:0/3694081703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2494100ee0 0x7f24941012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.460+0000 7f2499f92700 1 -- 172.21.15.49:0/3694081703 >> 172.21.15.49:0/3694081703 conn(0x7f24940fc790 msgr2=0x7f24940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.460+0000 7f2499f92700 1 -- 172.21.15.49:0/3694081703 shutdown_connections 2024-05-15T16:28:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.460+0000 7f2499f92700 1 -- 172.21.15.49:0/3694081703 wait complete. 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.461+0000 7f2499f92700 1 Processor -- start 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.461+0000 7f2499f92700 1 -- start start 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.461+0000 7f2499f92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494100ee0 0x7f24940732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.461+0000 7f2499f92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2494101890 0x7f24940737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.461+0000 7f2499f92700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2494073d20 con 0x7f2494100ee0 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.461+0000 7f2499f92700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2494073e60 con 0x7f2494101890 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.461+0000 7f24937fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494100ee0 0x7f24940732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.461+0000 7f24937fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494100ee0 0x7f24940732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32998/0 (socket says 172.21.15.49:32998) 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.461+0000 7f24937fe700 1 -- 172.21.15.49:0/163234982 learned_addr learned my addr 172.21.15.49:0/163234982 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f248affd700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2494101890 0x7f24940737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f24937fe700 1 -- 172.21.15.49:0/163234982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2494101890 msgr2=0x7f24940737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f24937fe700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2494101890 0x7f24940737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f24937fe700 1 -- 172.21.15.49:0/163234982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c0095f0 con 0x7f2494100ee0 2024-05-15T16:28:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f24937fe700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494100ee0 0x7f24940732a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f248400c8a0 tx=0x7f248400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f24917fa700 1 -- 172.21.15.49:0/163234982 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248400cea0 con 0x7f2494100ee0 2024-05-15T16:28:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f24917fa700 1 -- 172.21.15.49:0/163234982 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2484004830 con 0x7f2494100ee0 2024-05-15T16:28:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24941a2da0 con 0x7f2494100ee0 2024-05-15T16:28:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f24917fa700 1 -- 172.21.15.49:0/163234982 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24840055b0 con 0x7f2494100ee0 2024-05-15T16:28:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.462+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24941a3100 con 0x7f2494100ee0 2024-05-15T16:28:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.463+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24941092e0 con 0x7f2494100ee0 2024-05-15T16:28:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.464+0000 7f24917fa700 1 -- 172.21.15.49:0/163234982 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f24840049a0 con 0x7f2494100ee0 2024-05-15T16:28:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.464+0000 7f24917fa700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f248006e210 0x7f24800706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.464+0000 7f24917fa700 1 -- 172.21.15.49:0/163234982 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f248408ddb0 con 0x7f2494100ee0 2024-05-15T16:28:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.464+0000 7f248affd700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f248006e210 0x7f24800706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.465+0000 7f248affd700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f248006e210 0x7f24800706d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24940749d0 tx=0x7f247c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.469+0000 7f24917fa700 1 -- 172.21.15.49:0/163234982 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2484057110 con 0x7f2494100ee0 2024-05-15T16:28:01.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.694+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f249410de80 con 0x7f248006e210 2024-05-15T16:28:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.695+0000 7f24917fa700 1 -- 172.21.15.49:0/163234982 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f249410de80 con 0x7f248006e210 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:28:01.696 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:28:01.697 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:28:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f248006e210 msgr2=0x7f24800706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f248006e210 0x7f24800706d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24940749d0 tx=0x7f247c01a040 comp rx=0 tx=0).stop 2024-05-15T16:28:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494100ee0 msgr2=0x7f24940732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494100ee0 0x7f24940732a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f248400c8a0 tx=0x7f248400cc60 comp rx=0 tx=0).stop 2024-05-15T16:28:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 shutdown_connections 2024-05-15T16:28:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f248006e210 0x7f24800706d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2494100ee0 0x7f24940732a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 --2- 172.21.15.49:0/163234982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2494101890 0x7f24940737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 >> 172.21.15.49:0/163234982 conn(0x7f24940fc790 msgr2=0x7f24940fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 shutdown_connections 2024-05-15T16:28:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.698+0000 7f2499f92700 1 -- 172.21.15.49:0/163234982 wait complete. 2024-05-15T16:28:01.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.806+0000 7f811f2f3700 1 -- 172.21.15.49:0/2666039270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118101890 msgr2=0x7f81181058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.806+0000 7f811f2f3700 1 --2- 172.21.15.49:0/2666039270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118101890 0x7f81181058e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f810c009910 tx=0x7f810c009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:01.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.806+0000 7f811f2f3700 1 -- 172.21.15.49:0/2666039270 shutdown_connections 2024-05-15T16:28:01.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.806+0000 7f811f2f3700 1 --2- 172.21.15.49:0/2666039270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118101890 0x7f81181058e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.806+0000 7f811f2f3700 1 --2- 172.21.15.49:0/2666039270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8118100ee0 0x7f81181012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.806+0000 7f811f2f3700 1 -- 172.21.15.49:0/2666039270 >> 172.21.15.49:0/2666039270 conn(0x7f81180fc790 msgr2=0x7f81180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:01.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.806+0000 7f811f2f3700 1 -- 172.21.15.49:0/2666039270 shutdown_connections 2024-05-15T16:28:01.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.806+0000 7f811f2f3700 1 -- 172.21.15.49:0/2666039270 wait complete. 2024-05-15T16:28:01.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.806+0000 7f811f2f3700 1 Processor -- start 2024-05-15T16:28:01.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811f2f3700 1 -- start start 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811f2f3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118100ee0 0x7f811819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811f2f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8118101890 0x7f811819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811f2f3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811819d440 con 0x7f8118100ee0 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811f2f3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8118196a20 con 0x7f8118101890 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811d08f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118100ee0 0x7f811819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811d08f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118100ee0 0x7f811819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33012/0 (socket says 172.21.15.49:33012) 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811d08f700 1 -- 172.21.15.49:0/2969076425 learned_addr learned my addr 172.21.15.49:0/2969076425 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811c88e700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8118101890 0x7f811819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811d08f700 1 -- 172.21.15.49:0/2969076425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8118101890 msgr2=0x7f811819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811d08f700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8118101890 0x7f811819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811d08f700 1 -- 172.21.15.49:0/2969076425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c0095f0 con 0x7f8118100ee0 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.807+0000 7f811d08f700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118100ee0 0x7f811819c8e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8108007e90 tx=0x7f810800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:01.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.808+0000 7f81167fc700 1 -- 172.21.15.49:0/2969076425 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f810800de60 con 0x7f8118100ee0 2024-05-15T16:28:01.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.808+0000 7f81167fc700 1 -- 172.21.15.49:0/2969076425 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f810800f040 con 0x7f8118100ee0 2024-05-15T16:28:01.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.808+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8118196d00 con 0x7f8118100ee0 2024-05-15T16:28:01.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.808+0000 7f81167fc700 1 -- 172.21.15.49:0/2969076425 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81080045b0 con 0x7f8118100ee0 2024-05-15T16:28:01.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.808+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8118197250 con 0x7f8118100ee0 2024-05-15T16:28:01.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.809+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81181092e0 con 0x7f8118100ee0 2024-05-15T16:28:01.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.809+0000 7f81167fc700 1 -- 172.21.15.49:0/2969076425 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8108004ad0 con 0x7f8118100ee0 2024-05-15T16:28:01.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.810+0000 7f81167fc700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f810006e200 0x7f81000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:01.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.810+0000 7f81167fc700 1 -- 172.21.15.49:0/2969076425 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f810808dd30 con 0x7f8118100ee0 2024-05-15T16:28:01.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.810+0000 7f811c88e700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f810006e200 0x7f81000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:01.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.810+0000 7f811c88e700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f810006e200 0x7f81000706c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f810c0059d0 tx=0x7f810c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:01.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:01.815+0000 7f81167fc700 1 -- 172.21.15.49:0/2969076425 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f810805ab40 con 0x7f8118100ee0 2024-05-15T16:28:02.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.106+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f81181023c0 con 0x7f8118100ee0 2024-05-15T16:28:02.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.107+0000 7f81167fc700 1 -- 172.21.15.49:0/2969076425 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f8108019090 con 0x7f8118100ee0 2024-05-15T16:28:02.107 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:28:02.107 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:28:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:28:02.107 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:28:02.107 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:28:02.107 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:28:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:28:02.108 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:28:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:28:02.109 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:28:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:28:02.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f810006e200 msgr2=0x7f81000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f810006e200 0x7f81000706c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f810c0059d0 tx=0x7f810c005960 comp rx=0 tx=0).stop 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118100ee0 msgr2=0x7f811819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118100ee0 0x7f811819c8e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8108007e90 tx=0x7f810800d3b0 comp rx=0 tx=0).stop 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 shutdown_connections 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f810006e200 0x7f81000706c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8118100ee0 0x7f811819c8e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 --2- 172.21.15.49:0/2969076425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8118101890 0x7f811819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 >> 172.21.15.49:0/2969076425 conn(0x7f81180fc790 msgr2=0x7f81180fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 shutdown_connections 2024-05-15T16:28:02.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:02.109+0000 7f811f2f3700 1 -- 172.21.15.49:0/2969076425 wait complete. 2024-05-15T16:28:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:02 smithi144 ceph-mon[86635]: from='client.35698 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:02 smithi144 ceph-mon[86635]: pgmap v931: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:02 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3226527866' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:28:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:02 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2969076425' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:28:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:02 smithi049 ceph-mon[112142]: from='client.35698 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:02 smithi049 ceph-mon[112142]: pgmap v931: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:02 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3226527866' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:28:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:02 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2969076425' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:28:03.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:03 smithi144 ceph-mon[86635]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:03.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:03 smithi049 ceph-mon[112142]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:04.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:04 smithi144 ceph-mon[86635]: pgmap v932: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:04 smithi049 ceph-mon[112142]: pgmap v932: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:06 smithi144 ceph-mon[86635]: pgmap v933: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:28:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:06 smithi049 ceph-mon[112142]: pgmap v933: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:28:08.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:08 smithi144 ceph-mon[86635]: pgmap v934: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:08 smithi049 ceph-mon[112142]: pgmap v934: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:10 smithi144 ceph-mon[86635]: pgmap v935: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:10 smithi049 ceph-mon[112142]: pgmap v935: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:12 smithi144 ceph-mon[86635]: pgmap v936: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:12.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:12 smithi049 ceph-mon[112142]: pgmap v936: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:14 smithi144 ceph-mon[86635]: pgmap v937: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:14.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:14 smithi049 ceph-mon[112142]: pgmap v937: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:16 smithi144 ceph-mon[86635]: pgmap v938: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:16 smithi049 ceph-mon[112142]: pgmap v938: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:18 smithi144 ceph-mon[86635]: pgmap v939: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:18 smithi049 ceph-mon[112142]: pgmap v939: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:20 smithi144 ceph-mon[86635]: pgmap v940: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:20 smithi049 ceph-mon[112142]: pgmap v940: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:28:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:28:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:22 smithi144 ceph-mon[86635]: pgmap v941: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:22 smithi049 ceph-mon[112142]: pgmap v941: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:24 smithi144 ceph-mon[86635]: pgmap v942: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:24 smithi049 ceph-mon[112142]: pgmap v942: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:26 smithi144 ceph-mon[86635]: pgmap v943: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:26 smithi049 ceph-mon[112142]: pgmap v943: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:28 smithi144 ceph-mon[86635]: pgmap v944: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:28.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:28 smithi049 ceph-mon[112142]: pgmap v944: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:30 smithi144 ceph-mon[86635]: pgmap v945: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:30 smithi049 ceph-mon[112142]: pgmap v945: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:32.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.234+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/447727564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38101e50 msgr2=0x7f0a38102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.234+0000 7f0a3ebdc700 1 --2- 172.21.15.49:0/447727564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38101e50 0x7f0a38102230 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009910 tx=0x7f0a2c009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.234+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/447727564 shutdown_connections 2024-05-15T16:28:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.234+0000 7f0a3ebdc700 1 --2- 172.21.15.49:0/447727564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a38102770 0x7f0a3810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.234+0000 7f0a3ebdc700 1 --2- 172.21.15.49:0/447727564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38101e50 0x7f0a38102230 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.234+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/447727564 >> 172.21.15.49:0/447727564 conn(0x7f0a380fb6a0 msgr2=0x7f0a380fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.234+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/447727564 shutdown_connections 2024-05-15T16:28:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.234+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/447727564 wait complete. 2024-05-15T16:28:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a3ebdc700 1 Processor -- start 2024-05-15T16:28:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a3ebdc700 1 -- start start 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a3ebdc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a38101e50 0x7f0a380ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a3ebdc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38102770 0x7f0a380ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a3ebdc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38100530 con 0x7f0a38102770 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a3ebdc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38100670 con 0x7f0a38101e50 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38102770 0x7f0a380ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38102770 0x7f0a380ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46848/0 (socket says 172.21.15.49:46848) 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a37fff700 1 -- 172.21.15.49:0/3068642834 learned_addr learned my addr 172.21.15.49:0/3068642834 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a3c978700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a38101e50 0x7f0a380ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a37fff700 1 -- 172.21.15.49:0/3068642834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a38101e50 msgr2=0x7f0a380ffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a37fff700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a38101e50 0x7f0a380ffab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.235+0000 7f0a37fff700 1 -- 172.21.15.49:0/3068642834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c0095f0 con 0x7f0a38102770 2024-05-15T16:28:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.236+0000 7f0a37fff700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38102770 0x7f0a380ffff0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400d900 tx=0x7f0a2400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.236+0000 7f0a35ffb700 1 -- 172.21.15.49:0/3068642834 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a24009900 con 0x7f0a38102770 2024-05-15T16:28:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.236+0000 7f0a35ffb700 1 -- 172.21.15.49:0/3068642834 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a24010460 con 0x7f0a38102770 2024-05-15T16:28:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.236+0000 7f0a35ffb700 1 -- 172.21.15.49:0/3068642834 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a2400f5f0 con 0x7f0a38102770 2024-05-15T16:28:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.236+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a381a2d90 con 0x7f0a38102770 2024-05-15T16:28:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.236+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381a30f0 con 0x7f0a38102770 2024-05-15T16:28:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.237+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a38069ab0 con 0x7f0a38102770 2024-05-15T16:28:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.237+0000 7f0a35ffb700 1 -- 172.21.15.49:0/3068642834 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0a240105d0 con 0x7f0a38102770 2024-05-15T16:28:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.238+0000 7f0a35ffb700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a2806e200 0x7f0a280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.238+0000 7f0a35ffb700 1 -- 172.21.15.49:0/3068642834 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0a2408dc30 con 0x7f0a38102770 2024-05-15T16:28:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.238+0000 7f0a3c978700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a2806e200 0x7f0a280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.239+0000 7f0a3c978700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a2806e200 0x7f0a280706c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c000c00 tx=0x7f0a2c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.243+0000 7f0a35ffb700 1 -- 172.21.15.49:0/3068642834 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0a24056f90 con 0x7f0a38102770 2024-05-15T16:28:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:32 smithi049 ceph-mon[112142]: pgmap v946: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:32.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.475+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a3810ded0 con 0x7f0a2806e200 2024-05-15T16:28:32.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.476+0000 7f0a35ffb700 1 -- 172.21.15.49:0/3068642834 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0a3810ded0 con 0x7f0a2806e200 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.478+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a2806e200 msgr2=0x7f0a280706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.478+0000 7f0a3ebdc700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a2806e200 0x7f0a280706c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c000c00 tx=0x7f0a2c01a040 comp rx=0 tx=0).stop 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.478+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38102770 msgr2=0x7f0a380ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.478+0000 7f0a3ebdc700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38102770 0x7f0a380ffff0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400d900 tx=0x7f0a2400dc10 comp rx=0 tx=0).stop 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.479+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 shutdown_connections 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.479+0000 7f0a3ebdc700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0a2806e200 0x7f0a280706c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.479+0000 7f0a3ebdc700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a38101e50 0x7f0a380ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.479+0000 7f0a3ebdc700 1 --2- 172.21.15.49:0/3068642834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a38102770 0x7f0a380ffff0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.479+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 >> 172.21.15.49:0/3068642834 conn(0x7f0a380fb6a0 msgr2=0x7f0a381030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.479+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 shutdown_connections 2024-05-15T16:28:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.479+0000 7f0a3ebdc700 1 -- 172.21.15.49:0/3068642834 wait complete. 2024-05-15T16:28:32.490 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:28:32.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.588+0000 7fd8c4895700 1 -- 172.21.15.49:0/1933018885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c00ffe10 msgr2=0x7fd8c01001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.588+0000 7fd8c4895700 1 --2- 172.21.15.49:0/1933018885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c00ffe10 0x7fd8c01001f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0009910 tx=0x7fd8b0009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:32.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.588+0000 7fd8c4895700 1 -- 172.21.15.49:0/1933018885 shutdown_connections 2024-05-15T16:28:32.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.588+0000 7fd8c4895700 1 --2- 172.21.15.49:0/1933018885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c01007c0 0x7fd8c0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.588+0000 7fd8c4895700 1 --2- 172.21.15.49:0/1933018885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c00ffe10 0x7fd8c01001f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.588+0000 7fd8c4895700 1 -- 172.21.15.49:0/1933018885 >> 172.21.15.49:0/1933018885 conn(0x7fd8c00fb700 msgr2=0x7fd8c00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:32.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.588+0000 7fd8c4895700 1 -- 172.21.15.49:0/1933018885 shutdown_connections 2024-05-15T16:28:32.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.588+0000 7fd8c4895700 1 -- 172.21.15.49:0/1933018885 wait complete. 2024-05-15T16:28:32.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.589+0000 7fd8c4895700 1 Processor -- start 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.589+0000 7fd8c4895700 1 -- start start 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.589+0000 7fd8c4895700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c00ffe10 0x7fd8c019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.589+0000 7fd8c4895700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c01007c0 0x7fd8c019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.589+0000 7fd8c4895700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0196a70 con 0x7fd8c01007c0 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.589+0000 7fd8c4895700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0196be0 con 0x7fd8c00ffe10 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8bdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c01007c0 0x7fd8c019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8bdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c01007c0 0x7fd8c019ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46864/0 (socket says 172.21.15.49:46864) 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8bdd9b700 1 -- 172.21.15.49:0/306430714 learned_addr learned my addr 172.21.15.49:0/306430714 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8be59c700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c00ffe10 0x7fd8c019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8bdd9b700 1 -- 172.21.15.49:0/306430714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c00ffe10 msgr2=0x7fd8c019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8bdd9b700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c00ffe10 0x7fd8c019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8bdd9b700 1 -- 172.21.15.49:0/306430714 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b00095f0 con 0x7fd8c01007c0 2024-05-15T16:28:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8be59c700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c00ffe10 0x7fd8c019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:28:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8bdd9b700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c01007c0 0x7fd8c019ceb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400c8f0 tx=0x7fd8b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8af7fe700 1 -- 172.21.15.49:0/306430714 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8b4007a50 con 0x7fd8c01007c0 2024-05-15T16:28:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8af7fe700 1 -- 172.21.15.49:0/306430714 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8b4007bb0 con 0x7fd8c01007c0 2024-05-15T16:28:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8af7fe700 1 -- 172.21.15.49:0/306430714 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8b4018630 con 0x7fd8c01007c0 2024-05-15T16:28:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c0196ec0 con 0x7fd8c01007c0 2024-05-15T16:28:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.590+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c0197410 con 0x7fd8c01007c0 2024-05-15T16:28:32.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.591+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c01012f0 con 0x7fd8c01007c0 2024-05-15T16:28:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.592+0000 7fd8af7fe700 1 -- 172.21.15.49:0/306430714 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd8b4007d20 con 0x7fd8c01007c0 2024-05-15T16:28:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.592+0000 7fd8af7fe700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd8a806e200 0x7fd8a80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.592+0000 7fd8af7fe700 1 -- 172.21.15.49:0/306430714 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd8b408e0a0 con 0x7fd8c01007c0 2024-05-15T16:28:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.592+0000 7fd8be59c700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd8a806e200 0x7fd8a80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.593+0000 7fd8be59c700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd8a806e200 0x7fd8a80706c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00059d0 tx=0x7fd8b001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:32.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.597+0000 7fd8af7fe700 1 -- 172.21.15.49:0/306430714 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd8b4057400 con 0x7fd8c01007c0 2024-05-15T16:28:32.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:32 smithi144 ceph-mon[86635]: pgmap v946: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:32.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.825+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8c01976e0 con 0x7fd8a806e200 2024-05-15T16:28:32.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.826+0000 7fd8af7fe700 1 -- 172.21.15.49:0/306430714 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd8c01976e0 con 0x7fd8a806e200 2024-05-15T16:28:32.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd8a806e200 msgr2=0x7fd8a80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd8a806e200 0x7fd8a80706c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00059d0 tx=0x7fd8b001a040 comp rx=0 tx=0).stop 2024-05-15T16:28:32.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c01007c0 msgr2=0x7fd8c019ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c01007c0 0x7fd8c019ceb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400c8f0 tx=0x7fd8b400ccb0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 shutdown_connections 2024-05-15T16:28:32.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd8a806e200 0x7fd8a80706c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8c00ffe10 0x7fd8c019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 --2- 172.21.15.49:0/306430714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8c01007c0 0x7fd8c019ceb0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 >> 172.21.15.49:0/306430714 conn(0x7fd8c00fb700 msgr2=0x7fd8c0069670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:32.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 shutdown_connections 2024-05-15T16:28:32.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.829+0000 7fd8c4895700 1 -- 172.21.15.49:0/306430714 wait complete. 2024-05-15T16:28:32.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.938+0000 7fad91ac2700 1 -- 172.21.15.49:0/702147847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c102790 msgr2=0x7fad8c10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.938+0000 7fad91ac2700 1 --2- 172.21.15.49:0/702147847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c102790 0x7fad8c10ac80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fad80009910 tx=0x7fad80009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:32.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.938+0000 7fad91ac2700 1 -- 172.21.15.49:0/702147847 shutdown_connections 2024-05-15T16:28:32.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.938+0000 7fad91ac2700 1 --2- 172.21.15.49:0/702147847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c102790 0x7fad8c10ac80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.938+0000 7fad91ac2700 1 --2- 172.21.15.49:0/702147847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8c101e70 0x7fad8c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.938+0000 7fad91ac2700 1 -- 172.21.15.49:0/702147847 >> 172.21.15.49:0/702147847 conn(0x7fad8c0fb700 msgr2=0x7fad8c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:32.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.938+0000 7fad91ac2700 1 -- 172.21.15.49:0/702147847 shutdown_connections 2024-05-15T16:28:32.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.938+0000 7fad91ac2700 1 -- 172.21.15.49:0/702147847 wait complete. 2024-05-15T16:28:32.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.939+0000 7fad91ac2700 1 Processor -- start 2024-05-15T16:28:32.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.939+0000 7fad91ac2700 1 -- start start 2024-05-15T16:28:32.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.939+0000 7fad91ac2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c101e70 0x7fad8c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:32.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.939+0000 7fad91ac2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8c102790 0x7fad8c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:32.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.939+0000 7fad91ac2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c19d490 con 0x7fad8c101e70 2024-05-15T16:28:32.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.939+0000 7fad91ac2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c196a70 con 0x7fad8c102790 2024-05-15T16:28:32.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.939+0000 7fad8b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c101e70 0x7fad8c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:32.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.939+0000 7fad8b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c101e70 0x7fad8c19c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46882/0 (socket says 172.21.15.49:46882) 2024-05-15T16:28:32.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.939+0000 7fad8b7fe700 1 -- 172.21.15.49:0/1823912308 learned_addr learned my addr 172.21.15.49:0/1823912308 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:32.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad8affd700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8c102790 0x7fad8c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad8b7fe700 1 -- 172.21.15.49:0/1823912308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8c102790 msgr2=0x7fad8c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad8b7fe700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8c102790 0x7fad8c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad8b7fe700 1 -- 172.21.15.49:0/1823912308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad800095f0 con 0x7fad8c101e70 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad8affd700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8c102790 0x7fad8c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad8b7fe700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c101e70 0x7fad8c19c930 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d8d0 tx=0x7fad7c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad88ff9700 1 -- 172.21.15.49:0/1823912308 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad7c009940 con 0x7fad8c101e70 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad88ff9700 1 -- 172.21.15.49:0/1823912308 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad7c010460 con 0x7fad8c101e70 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad88ff9700 1 -- 172.21.15.49:0/1823912308 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad7c00f5f0 con 0x7fad8c101e70 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c196d50 con 0x7fad8c101e70 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.940+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c1972a0 con 0x7fad8c101e70 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.941+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c105590 con 0x7fad8c101e70 2024-05-15T16:28:32.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.942+0000 7fad88ff9700 1 -- 172.21.15.49:0/1823912308 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fad7c0105d0 con 0x7fad8c101e70 2024-05-15T16:28:32.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.942+0000 7fad88ff9700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad7406e200 0x7fad740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:32.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.942+0000 7fad88ff9700 1 -- 172.21.15.49:0/1823912308 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fad7c08e090 con 0x7fad8c101e70 2024-05-15T16:28:32.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.942+0000 7fad8affd700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad7406e200 0x7fad740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:32.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.943+0000 7fad8affd700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad7406e200 0x7fad740706c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fad80006010 tx=0x7fad8000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:32.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:32.947+0000 7fad88ff9700 1 -- 172.21.15.49:0/1823912308 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fad7c05af20 con 0x7fad8c101e70 2024-05-15T16:28:33.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.162+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fad8c197fa0 con 0x7fad7406e200 2024-05-15T16:28:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.173+0000 7fad88ff9700 1 -- 172.21.15.49:0/1823912308 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fad8c197fa0 con 0x7fad7406e200 2024-05-15T16:28:33.174 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (31m) 9m ago 43m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (43m) 9m ago 43m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (42m) 9m ago 42m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (29m) 9m ago 43m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (29m) 9m ago 42m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (31m) 9m ago 43m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (25m) 9m ago 37m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (25m) 9m ago 37m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (24m) 9m ago 37m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (9m) 9m ago 37m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (34m) 9m ago 44m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (32m) 9m ago 42m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:28:33.175 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (30m) 9m ago 44m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (29m) 9m ago 42m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (32m) 9m ago 43m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (32m) 9m ago 42m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (28m) 9m ago 41m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (28m) 9m ago 41m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (27m) 9m ago 40m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (26m) 9m ago 40m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (26m) 9m ago 39m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (25m) 9m ago 39m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:28:33.176 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (32m) 9m ago 42m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad7406e200 msgr2=0x7fad740706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad7406e200 0x7fad740706c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fad80006010 tx=0x7fad8000b540 comp rx=0 tx=0).stop 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c101e70 msgr2=0x7fad8c19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c101e70 0x7fad8c19c930 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d8d0 tx=0x7fad7c00dc90 comp rx=0 tx=0).stop 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 shutdown_connections 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad7406e200 0x7fad740706c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad8c101e70 0x7fad8c19c930 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 --2- 172.21.15.49:0/1823912308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8c102790 0x7fad8c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 >> 172.21.15.49:0/1823912308 conn(0x7fad8c0fb700 msgr2=0x7fad8c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 shutdown_connections 2024-05-15T16:28:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.175+0000 7fad91ac2700 1 -- 172.21.15.49:0/1823912308 wait complete. 2024-05-15T16:28:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.283+0000 7f539b58f700 1 -- 172.21.15.49:0/857241455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394101890 msgr2=0x7f53941058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.283+0000 7f539b58f700 1 --2- 172.21.15.49:0/857241455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394101890 0x7f53941058e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5388009910 tx=0x7f5388009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.283+0000 7f539b58f700 1 -- 172.21.15.49:0/857241455 shutdown_connections 2024-05-15T16:28:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.283+0000 7f539b58f700 1 --2- 172.21.15.49:0/857241455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394101890 0x7f53941058e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.283+0000 7f539b58f700 1 --2- 172.21.15.49:0/857241455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5394100ee0 0x7f53941012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.283+0000 7f539b58f700 1 -- 172.21.15.49:0/857241455 >> 172.21.15.49:0/857241455 conn(0x7f53940fc790 msgr2=0x7f53940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.283+0000 7f539b58f700 1 -- 172.21.15.49:0/857241455 shutdown_connections 2024-05-15T16:28:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.283+0000 7f539b58f700 1 -- 172.21.15.49:0/857241455 wait complete. 2024-05-15T16:28:33.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539b58f700 1 Processor -- start 2024-05-15T16:28:33.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539b58f700 1 -- start start 2024-05-15T16:28:33.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539b58f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394100ee0 0x7f53940732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:33.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539b58f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5394101890 0x7f53940737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:33.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539b58f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5394073d20 con 0x7f5394100ee0 2024-05-15T16:28:33.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539b58f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5394073e60 con 0x7f5394101890 2024-05-15T16:28:33.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539932b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394100ee0 0x7f53940732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539932b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394100ee0 0x7f53940732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46886/0 (socket says 172.21.15.49:46886) 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539932b700 1 -- 172.21.15.49:0/3526259784 learned_addr learned my addr 172.21.15.49:0/3526259784 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f5398b2a700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5394101890 0x7f53940737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539932b700 1 -- 172.21.15.49:0/3526259784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5394101890 msgr2=0x7f53940737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539932b700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5394101890 0x7f53940737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f539932b700 1 -- 172.21.15.49:0/3526259784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880095f0 con 0x7f5394100ee0 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.284+0000 7f5398b2a700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5394101890 0x7f53940737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.285+0000 7f539932b700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394100ee0 0x7f53940732a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f53840060b0 tx=0x7f538400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.285+0000 7f53927fc700 1 -- 172.21.15.49:0/3526259784 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f538400dda0 con 0x7f5394100ee0 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.285+0000 7f53927fc700 1 -- 172.21.15.49:0/3526259784 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f538400f040 con 0x7f5394100ee0 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.285+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53941a2da0 con 0x7f5394100ee0 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.285+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53941a3100 con 0x7f5394100ee0 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.285+0000 7f53927fc700 1 -- 172.21.15.49:0/3526259784 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5384014790 con 0x7f5394100ee0 2024-05-15T16:28:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.286+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53800052f0 con 0x7f5394100ee0 2024-05-15T16:28:33.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.286+0000 7f53927fc700 1 -- 172.21.15.49:0/3526259784 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5384015400 con 0x7f5394100ee0 2024-05-15T16:28:33.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.287+0000 7f53927fc700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f537c06e200 0x7f537c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:33.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.287+0000 7f53927fc700 1 -- 172.21.15.49:0/3526259784 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f538408dad0 con 0x7f5394100ee0 2024-05-15T16:28:33.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.287+0000 7f5398b2a700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f537c06e200 0x7f537c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:33.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.287+0000 7f5398b2a700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f537c06e200 0x7f537c0706c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f53940749d0 tx=0x7f538801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:33.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.291+0000 7f53927fc700 1 -- 172.21.15.49:0/3526259784 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5384056ee0 con 0x7f5394100ee0 2024-05-15T16:28:33.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.586+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5380005db0 con 0x7f5394100ee0 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.587+0000 7f53927fc700 1 -- 172.21.15.49:0/3526259784 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f538405a5a0 con 0x7f5394100ee0 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:28:33.588 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:28:33.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:28:33.589 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:28:33.589 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:28:33.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:28:33.589 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:28:33.589 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:28:33.589 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:28:33.589 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:28:33.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f537c06e200 msgr2=0x7f537c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f537c06e200 0x7f537c0706c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f53940749d0 tx=0x7f538801a040 comp rx=0 tx=0).stop 2024-05-15T16:28:33.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394100ee0 msgr2=0x7f53940732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394100ee0 0x7f53940732a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f53840060b0 tx=0x7f538400d3b0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 shutdown_connections 2024-05-15T16:28:33.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f537c06e200 0x7f537c0706c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5394100ee0 0x7f53940732a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 --2- 172.21.15.49:0/3526259784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5394101890 0x7f53940737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 >> 172.21.15.49:0/3526259784 conn(0x7f53940fc790 msgr2=0x7f53940fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:33.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 shutdown_connections 2024-05-15T16:28:33.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.589+0000 7f539b58f700 1 -- 172.21.15.49:0/3526259784 wait complete. 2024-05-15T16:28:33.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.697+0000 7f426b0a7700 1 -- 172.21.15.49:0/3142139569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264100ee0 msgr2=0x7f42641012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.697+0000 7f426b0a7700 1 --2- 172.21.15.49:0/3142139569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264100ee0 0x7f42641012c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4258009910 tx=0x7f4258009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:33.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.697+0000 7f426b0a7700 1 -- 172.21.15.49:0/3142139569 shutdown_connections 2024-05-15T16:28:33.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.697+0000 7f426b0a7700 1 --2- 172.21.15.49:0/3142139569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4264101890 0x7f42641058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.697+0000 7f426b0a7700 1 --2- 172.21.15.49:0/3142139569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264100ee0 0x7f42641012c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.697+0000 7f426b0a7700 1 -- 172.21.15.49:0/3142139569 >> 172.21.15.49:0/3142139569 conn(0x7f42640fc790 msgr2=0x7f42640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.697+0000 7f426b0a7700 1 -- 172.21.15.49:0/3142139569 shutdown_connections 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.697+0000 7f426b0a7700 1 -- 172.21.15.49:0/3142139569 wait complete. 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.697+0000 7f426b0a7700 1 Processor -- start 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f426b0a7700 1 -- start start 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f426b0a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4264100ee0 0x7f426419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f426b0a7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264101890 0x7f426419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f426b0a7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426419d460 con 0x7f4264101890 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f426b0a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4264196a90 con 0x7f4264100ee0 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f4263fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264101890 0x7f426419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f4263fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264101890 0x7f426419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42968/0 (socket says 172.21.15.49:42968) 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f4263fff700 1 -- 172.21.15.49:0/272237763 learned_addr learned my addr 172.21.15.49:0/272237763 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f4268e43700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4264100ee0 0x7f426419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f4263fff700 1 -- 172.21.15.49:0/272237763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4264100ee0 msgr2=0x7f426419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f4263fff700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4264100ee0 0x7f426419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f4263fff700 1 -- 172.21.15.49:0/272237763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42580095f0 con 0x7f4264101890 2024-05-15T16:28:33.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f4263fff700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264101890 0x7f426419ce40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f425000e8f0 tx=0x7f425000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:33.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.698+0000 7f4261ffb700 1 -- 172.21.15.49:0/272237763 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4250009790 con 0x7f4264101890 2024-05-15T16:28:33.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.699+0000 7f4261ffb700 1 -- 172.21.15.49:0/272237763 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4250004cf0 con 0x7f4264101890 2024-05-15T16:28:33.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.699+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4264196d70 con 0x7f4264101890 2024-05-15T16:28:33.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.699+0000 7f4261ffb700 1 -- 172.21.15.49:0/272237763 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4250005690 con 0x7f4264101890 2024-05-15T16:28:33.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.699+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42641972c0 con 0x7f4264101890 2024-05-15T16:28:33.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.700+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4264109280 con 0x7f4264101890 2024-05-15T16:28:33.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.700+0000 7f4261ffb700 1 -- 172.21.15.49:0/272237763 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4250004750 con 0x7f4264101890 2024-05-15T16:28:33.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.701+0000 7f4261ffb700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f425406e200 0x7f42540706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:33.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.701+0000 7f4261ffb700 1 -- 172.21.15.49:0/272237763 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4250014070 con 0x7f4264101890 2024-05-15T16:28:33.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.701+0000 7f4268e43700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f425406e200 0x7f42540706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:33.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.701+0000 7f4268e43700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f425406e200 0x7f42540706c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f425800b760 tx=0x7f4258005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:33.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.705+0000 7f4261ffb700 1 -- 172.21.15.49:0/272237763 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f425005b3c0 con 0x7f4264101890 2024-05-15T16:28:33.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.955+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f42641023c0 con 0x7f4264101890 2024-05-15T16:28:33.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.956+0000 7f4261ffb700 1 -- 172.21.15.49:0/272237763 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f425005af50 con 0x7f4264101890 2024-05-15T16:28:33.957 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:28:33.957 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:28:33.957 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:28:33.958 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:28:33.959 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:28:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:28:33.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f425406e200 msgr2=0x7f42540706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f425406e200 0x7f42540706c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f425800b760 tx=0x7f4258005940 comp rx=0 tx=0).stop 2024-05-15T16:28:33.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264101890 msgr2=0x7f426419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:33.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264101890 0x7f426419ce40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f425000e8f0 tx=0x7f425000ecb0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 shutdown_connections 2024-05-15T16:28:33.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f425406e200 0x7f42540706c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4264100ee0 0x7f426419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 --2- 172.21.15.49:0/272237763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4264101890 0x7f426419ce40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:33.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 >> 172.21.15.49:0/272237763 conn(0x7f42640fc790 msgr2=0x7f42640fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:33.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 shutdown_connections 2024-05-15T16:28:33.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:33.958+0000 7f426b0a7700 1 -- 172.21.15.49:0/272237763 wait complete. 2024-05-15T16:28:33.962 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:28:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.074+0000 7f24cffff700 1 -- 172.21.15.49:0/871903066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0106b70 msgr2=0x7f24d0106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.074+0000 7f24cffff700 1 --2- 172.21.15.49:0/871903066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0106b70 0x7f24d0106f50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009910 tx=0x7f24c0009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.074+0000 7f24cffff700 1 -- 172.21.15.49:0/871903066 shutdown_connections 2024-05-15T16:28:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.074+0000 7f24cffff700 1 --2- 172.21.15.49:0/871903066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d0100ae0 0x7f24d0100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.074+0000 7f24cffff700 1 --2- 172.21.15.49:0/871903066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0106b70 0x7f24d0106f50 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.074+0000 7f24cffff700 1 -- 172.21.15.49:0/871903066 >> 172.21.15.49:0/871903066 conn(0x7f24d00fc5f0 msgr2=0x7f24d00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.074+0000 7f24cffff700 1 -- 172.21.15.49:0/871903066 shutdown_connections 2024-05-15T16:28:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.074+0000 7f24cffff700 1 -- 172.21.15.49:0/871903066 wait complete. 2024-05-15T16:28:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.075+0000 7f24cffff700 1 Processor -- start 2024-05-15T16:28:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.075+0000 7f24cffff700 1 -- start start 2024-05-15T16:28:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.075+0000 7f24cffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0100ae0 0x7f24d019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.075+0000 7f24cffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d0106b70 0x7f24d019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.075+0000 7f24cffff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d019f470 con 0x7f24d0100ae0 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.075+0000 7f24cffff700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0198a50 con 0x7f24d0106b70 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24ceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0100ae0 0x7f24d019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24ceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0100ae0 0x7f24d019e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42980/0 (socket says 172.21.15.49:42980) 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24ceffd700 1 -- 172.21.15.49:0/611814086 learned_addr learned my addr 172.21.15.49:0/611814086 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24c67fc700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d0106b70 0x7f24d019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24ceffd700 1 -- 172.21.15.49:0/611814086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d0106b70 msgr2=0x7f24d019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24ceffd700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d0106b70 0x7f24d019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24ceffd700 1 -- 172.21.15.49:0/611814086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00095f0 con 0x7f24d0100ae0 2024-05-15T16:28:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24ceffd700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0100ae0 0x7f24d019e910 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f24c000bd70 tx=0x7f24c000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24d49fd700 1 -- 172.21.15.49:0/611814086 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c001d070 con 0x7f24d0100ae0 2024-05-15T16:28:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24d49fd700 1 -- 172.21.15.49:0/611814086 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24c000f460 con 0x7f24d0100ae0 2024-05-15T16:28:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d0198cd0 con 0x7f24d0100ae0 2024-05-15T16:28:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24d49fd700 1 -- 172.21.15.49:0/611814086 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c0017630 con 0x7f24d0100ae0 2024-05-15T16:28:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.076+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01991c0 con 0x7f24d0100ae0 2024-05-15T16:28:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.077+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d0104e20 con 0x7f24d0100ae0 2024-05-15T16:28:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.078+0000 7f24d49fd700 1 -- 172.21.15.49:0/611814086 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f24c000f5d0 con 0x7f24d0100ae0 2024-05-15T16:28:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.078+0000 7f24d49fd700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f24bc06e200 0x7f24bc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.078+0000 7f24d49fd700 1 -- 172.21.15.49:0/611814086 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f24c008f6d0 con 0x7f24d0100ae0 2024-05-15T16:28:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.078+0000 7f24c67fc700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f24bc06e200 0x7f24bc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.079+0000 7f24c67fc700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f24bc06e200 0x7f24bc0706c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f24b8005fd0 tx=0x7f24b800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.083+0000 7f24d49fd700 1 -- 172.21.15.49:0/611814086 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f24c005c4e0 con 0x7f24d0100ae0 2024-05-15T16:28:34.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.310+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24d0199f10 con 0x7f24bc06e200 2024-05-15T16:28:34.311 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:34 smithi049 ceph-mon[112142]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:34.311 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:34 smithi049 ceph-mon[112142]: pgmap v947: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:34.311 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:34 smithi049 ceph-mon[112142]: from='client.35722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:34.311 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:34 smithi049 ceph-mon[112142]: from='client.35726 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:34.311 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:34 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3526259784' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:28:34.311 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:34 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/272237763' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:28:34.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.311+0000 7f24d49fd700 1 -- 172.21.15.49:0/611814086 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f24d0199f10 con 0x7f24bc06e200 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:28:34.312 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:28:34.313 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:28:34.313 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:28:34.313 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:28:34.313 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:28:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f24bc06e200 msgr2=0x7f24bc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f24bc06e200 0x7f24bc0706c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f24b8005fd0 tx=0x7f24b800a560 comp rx=0 tx=0).stop 2024-05-15T16:28:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0100ae0 msgr2=0x7f24d019e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0100ae0 0x7f24d019e910 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f24c000bd70 tx=0x7f24c000be50 comp rx=0 tx=0).stop 2024-05-15T16:28:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 shutdown_connections 2024-05-15T16:28:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f24bc06e200 0x7f24bc0706c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24d0100ae0 0x7f24d019e910 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 --2- 172.21.15.49:0/611814086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d0106b70 0x7f24d019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 >> 172.21.15.49:0/611814086 conn(0x7f24d00fc5f0 msgr2=0x7f24d0073410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:34.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 shutdown_connections 2024-05-15T16:28:34.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.314+0000 7f24cffff700 1 -- 172.21.15.49:0/611814086 wait complete. 2024-05-15T16:28:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.423+0000 7f89a44ef700 1 -- 172.21.15.49:0/1139010124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c101890 msgr2=0x7f899c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.423+0000 7f89a44ef700 1 --2- 172.21.15.49:0/1139010124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c101890 0x7f899c1058e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8998009910 tx=0x7f8998009c20 comp rx=0 tx=0).stop 2024-05-15T16:28:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.423+0000 7f89a44ef700 1 -- 172.21.15.49:0/1139010124 shutdown_connections 2024-05-15T16:28:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.423+0000 7f89a44ef700 1 --2- 172.21.15.49:0/1139010124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c101890 0x7f899c1058e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.423+0000 7f89a44ef700 1 --2- 172.21.15.49:0/1139010124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f899c100ee0 0x7f899c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.423+0000 7f89a44ef700 1 -- 172.21.15.49:0/1139010124 >> 172.21.15.49:0/1139010124 conn(0x7f899c0fc790 msgr2=0x7f899c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.423+0000 7f89a44ef700 1 -- 172.21.15.49:0/1139010124 shutdown_connections 2024-05-15T16:28:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.423+0000 7f89a44ef700 1 -- 172.21.15.49:0/1139010124 wait complete. 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.424+0000 7f89a44ef700 1 Processor -- start 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.424+0000 7f89a44ef700 1 -- start start 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.424+0000 7f89a44ef700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c100ee0 0x7f899c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.424+0000 7f89a44ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f899c101890 0x7f899c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.424+0000 7f89a44ef700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c073d20 con 0x7f899c100ee0 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.424+0000 7f89a44ef700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c073e60 con 0x7f899c101890 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.424+0000 7f89a228b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c100ee0 0x7f899c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.424+0000 7f89a228b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c100ee0 0x7f899c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43000/0 (socket says 172.21.15.49:43000) 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.424+0000 7f89a228b700 1 -- 172.21.15.49:0/2040964469 learned_addr learned my addr 172.21.15.49:0/2040964469 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f89a1a8a700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f899c101890 0x7f899c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f89a228b700 1 -- 172.21.15.49:0/2040964469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f899c101890 msgr2=0x7f899c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f89a228b700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f899c101890 0x7f899c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f89a228b700 1 -- 172.21.15.49:0/2040964469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980095f0 con 0x7f899c100ee0 2024-05-15T16:28:34.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f89a1a8a700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f899c101890 0x7f899c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:28:34.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f89a228b700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c100ee0 0x7f899c0732a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f89900060b0 tx=0x7f899000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:34.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f898f7fe700 1 -- 172.21.15.49:0/2040964469 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f899000dda0 con 0x7f899c100ee0 2024-05-15T16:28:34.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f898f7fe700 1 -- 172.21.15.49:0/2040964469 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f899000f040 con 0x7f899c100ee0 2024-05-15T16:28:34.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c1a2da0 con 0x7f899c100ee0 2024-05-15T16:28:34.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f898f7fe700 1 -- 172.21.15.49:0/2040964469 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8990014790 con 0x7f899c100ee0 2024-05-15T16:28:34.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.425+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c1a3100 con 0x7f899c100ee0 2024-05-15T16:28:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.426+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c1092e0 con 0x7f899c100ee0 2024-05-15T16:28:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.427+0000 7f898f7fe700 1 -- 172.21.15.49:0/2040964469 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8990014980 con 0x7f899c100ee0 2024-05-15T16:28:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.427+0000 7f898f7fe700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f898806e210 0x7f89880706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:28:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.427+0000 7f898f7fe700 1 -- 172.21.15.49:0/2040964469 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f899008dbe0 con 0x7f899c100ee0 2024-05-15T16:28:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.427+0000 7f89a1a8a700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f898806e210 0x7f89880706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:28:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.428+0000 7f89a1a8a700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f898806e210 0x7f89880706d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f899c0749d0 tx=0x7f899801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:28:34.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.432+0000 7f898f7fe700 1 -- 172.21.15.49:0/2040964469 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8990056f70 con 0x7f899c100ee0 2024-05-15T16:28:34.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:34 smithi144 ceph-mon[86635]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:34.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:34 smithi144 ceph-mon[86635]: pgmap v947: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:34.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:34 smithi144 ceph-mon[86635]: from='client.35722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:34.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:34 smithi144 ceph-mon[86635]: from='client.35726 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:34.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:34 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3526259784' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:28:34.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:34 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/272237763' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:28:34.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.724+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f899c1023c0 con 0x7f899c100ee0 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.725+0000 7f898f7fe700 1 -- 172.21.15.49:0/2040964469 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f899005a630 con 0x7f899c100ee0 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:28:34.725 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:28:34.726 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f898806e210 msgr2=0x7f89880706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f898806e210 0x7f89880706d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f899c0749d0 tx=0x7f899801a040 comp rx=0 tx=0).stop 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c100ee0 msgr2=0x7f899c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c100ee0 0x7f899c0732a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f89900060b0 tx=0x7f899000d3b0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 shutdown_connections 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f898806e210 0x7f89880706d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f899c100ee0 0x7f899c0732a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 --2- 172.21.15.49:0/2040964469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f899c101890 0x7f899c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 >> 172.21.15.49:0/2040964469 conn(0x7f899c0fc790 msgr2=0x7f899c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:28:34.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 shutdown_connections 2024-05-15T16:28:34.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:28:34.727+0000 7f89a44ef700 1 -- 172.21.15.49:0/2040964469 wait complete. 2024-05-15T16:28:35.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:35 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2040964469' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:28:35.705 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:35 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2040964469' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:28:36.599 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:36 smithi144 ceph-mon[86635]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:36.599 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:36 smithi144 ceph-mon[86635]: pgmap v948: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:36.599 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:28:36.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:36 smithi049 ceph-mon[112142]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:28:36.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:36 smithi049 ceph-mon[112142]: pgmap v948: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:36.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:28:38.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:38 smithi144 ceph-mon[86635]: pgmap v949: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:38.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:38 smithi049 ceph-mon[112142]: pgmap v949: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:40.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:40 smithi144 ceph-mon[86635]: pgmap v950: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:40.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:40 smithi049 ceph-mon[112142]: pgmap v950: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:42 smithi144 ceph-mon[86635]: pgmap v951: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:42.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:42 smithi049 ceph-mon[112142]: pgmap v951: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:44 smithi049 ceph-mon[112142]: pgmap v952: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:44.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:44 smithi144 ceph-mon[86635]: pgmap v952: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:45.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:45 smithi144 ceph-mon[86635]: pgmap v953: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:45.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:45 smithi049 ceph-mon[112142]: pgmap v953: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:47 smithi144 ceph-mon[86635]: pgmap v954: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:47 smithi049 ceph-mon[112142]: pgmap v954: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:49 smithi144 ceph-mon[86635]: pgmap v955: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:49 smithi049 ceph-mon[112142]: pgmap v955: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:28:51.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:28:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:51 smithi144 ceph-mon[86635]: pgmap v956: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:51 smithi049 ceph-mon[112142]: pgmap v956: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:54.177 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:53 smithi049 ceph-mon[112142]: pgmap v957: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:54.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:53 smithi144 ceph-mon[86635]: pgmap v957: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:55 smithi144 ceph-mon[86635]: pgmap v958: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:56.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:55 smithi049 ceph-mon[112142]: pgmap v958: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:57 smithi144 ceph-mon[86635]: pgmap v959: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:28:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:57 smithi049 ceph-mon[112142]: pgmap v959: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:28:59 smithi144 ceph-mon[86635]: pgmap v960: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:28:59 smithi049 ceph-mon[112142]: pgmap v960: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:01 smithi144 ceph-mon[86635]: pgmap v961: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:01 smithi049 ceph-mon[112142]: pgmap v961: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:03 smithi144 ceph-mon[86635]: pgmap v962: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:03 smithi049 ceph-mon[112142]: pgmap v962: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.850+0000 7f03e0366700 1 -- 172.21.15.49:0/3414419035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d81018b0 msgr2=0x7f03d8105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.850+0000 7f03e0366700 1 --2- 172.21.15.49:0/3414419035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d81018b0 0x7f03d8105900 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009910 tx=0x7f03c8009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.850+0000 7f03e0366700 1 -- 172.21.15.49:0/3414419035 shutdown_connections 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.850+0000 7f03e0366700 1 --2- 172.21.15.49:0/3414419035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d81018b0 0x7f03d8105900 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.850+0000 7f03e0366700 1 --2- 172.21.15.49:0/3414419035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03d8100f00 0x7f03d81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.850+0000 7f03e0366700 1 -- 172.21.15.49:0/3414419035 >> 172.21.15.49:0/3414419035 conn(0x7f03d80fc7d0 msgr2=0x7f03d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.850+0000 7f03e0366700 1 -- 172.21.15.49:0/3414419035 shutdown_connections 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.850+0000 7f03e0366700 1 -- 172.21.15.49:0/3414419035 wait complete. 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03e0366700 1 Processor -- start 2024-05-15T16:29:04.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03e0366700 1 -- start start 2024-05-15T16:29:04.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03e0366700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d8100f00 0x7f03d819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:04.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03e0366700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03d81018b0 0x7f03d819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:04.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03e0366700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d819d520 con 0x7f03d8100f00 2024-05-15T16:29:04.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03e0366700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d8196ae0 con 0x7f03d81018b0 2024-05-15T16:29:04.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03de102700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d8100f00 0x7f03d819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:04.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03de102700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d8100f00 0x7f03d819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33234/0 (socket says 172.21.15.49:33234) 2024-05-15T16:29:04.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03de102700 1 -- 172.21.15.49:0/2110472998 learned_addr learned my addr 172.21.15.49:0/2110472998 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:04.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03dd901700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03d81018b0 0x7f03d819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:04.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03de102700 1 -- 172.21.15.49:0/2110472998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03d81018b0 msgr2=0x7f03d819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:04.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03de102700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03d81018b0 0x7f03d819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:04.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.851+0000 7f03de102700 1 -- 172.21.15.49:0/2110472998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c80095f0 con 0x7f03d8100f00 2024-05-15T16:29:04.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.852+0000 7f03de102700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d8100f00 0x7f03d819c950 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f03d400d900 tx=0x7f03d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:04.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.852+0000 7f03cf7fe700 1 -- 172.21.15.49:0/2110472998 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d4004a00 con 0x7f03d8100f00 2024-05-15T16:29:04.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.852+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d8196dc0 con 0x7f03d8100f00 2024-05-15T16:29:04.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.852+0000 7f03cf7fe700 1 -- 172.21.15.49:0/2110472998 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03d40054e0 con 0x7f03d8100f00 2024-05-15T16:29:04.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.852+0000 7f03cf7fe700 1 -- 172.21.15.49:0/2110472998 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d4009d00 con 0x7f03d8100f00 2024-05-15T16:29:04.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.852+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d8197310 con 0x7f03d8100f00 2024-05-15T16:29:04.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.853+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d8109300 con 0x7f03d8100f00 2024-05-15T16:29:04.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.854+0000 7f03cf7fe700 1 -- 172.21.15.49:0/2110472998 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f03d4005020 con 0x7f03d8100f00 2024-05-15T16:29:04.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.854+0000 7f03cf7fe700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03c406e200 0x7f03c40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:04.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.854+0000 7f03cf7fe700 1 -- 172.21.15.49:0/2110472998 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f03d408e100 con 0x7f03d8100f00 2024-05-15T16:29:04.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.854+0000 7f03dd901700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03c406e200 0x7f03c40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:04.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.855+0000 7f03dd901700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03c406e200 0x7f03c40706c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f03d80fcbd0 tx=0x7f03c801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:04.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:04.859+0000 7f03cf7fe700 1 -- 172.21.15.49:0/2110472998 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f03d405af10 con 0x7f03d8100f00 2024-05-15T16:29:05.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.086+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03d810dec0 con 0x7f03c406e200 2024-05-15T16:29:05.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.088+0000 7f03cf7fe700 1 -- 172.21.15.49:0/2110472998 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f03d810dec0 con 0x7f03c406e200 2024-05-15T16:29:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03c406e200 msgr2=0x7f03c40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03c406e200 0x7f03c40706c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f03d80fcbd0 tx=0x7f03c801a040 comp rx=0 tx=0).stop 2024-05-15T16:29:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d8100f00 msgr2=0x7f03d819c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d8100f00 0x7f03d819c950 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f03d400d900 tx=0x7f03d400dc10 comp rx=0 tx=0).stop 2024-05-15T16:29:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 shutdown_connections 2024-05-15T16:29:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03c406e200 0x7f03c40706c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03d8100f00 0x7f03d819c950 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 --2- 172.21.15.49:0/2110472998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03d81018b0 0x7f03d819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 >> 172.21.15.49:0/2110472998 conn(0x7f03d80fc7d0 msgr2=0x7f03d80fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 shutdown_connections 2024-05-15T16:29:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.090+0000 7f03e0366700 1 -- 172.21.15.49:0/2110472998 wait complete. 2024-05-15T16:29:05.102 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:29:05.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.200+0000 7eff1b920700 1 -- 172.21.15.49:0/1080139052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 msgr2=0x7eff14102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.200+0000 7eff1b920700 1 --2- 172.21.15.49:0/1080139052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 0x7eff14102250 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7eff0400b0a0 tx=0x7eff0400b3b0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.200+0000 7eff1b920700 1 -- 172.21.15.49:0/1080139052 shutdown_connections 2024-05-15T16:29:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.200+0000 7eff1b920700 1 --2- 172.21.15.49:0/1080139052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff14102790 0x7eff1410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.200+0000 7eff1b920700 1 --2- 172.21.15.49:0/1080139052 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 0x7eff14102250 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.200+0000 7eff1b920700 1 -- 172.21.15.49:0/1080139052 >> 172.21.15.49:0/1080139052 conn(0x7eff140fb700 msgr2=0x7eff140fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.200+0000 7eff1b920700 1 -- 172.21.15.49:0/1080139052 shutdown_connections 2024-05-15T16:29:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.200+0000 7eff1b920700 1 -- 172.21.15.49:0/1080139052 wait complete. 2024-05-15T16:29:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff1b920700 1 Processor -- start 2024-05-15T16:29:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff1b920700 1 -- start start 2024-05-15T16:29:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff1b920700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 0x7eff1419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff1b920700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff14102790 0x7eff1419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff1b920700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1419d420 con 0x7eff14101e70 2024-05-15T16:29:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff1b920700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff14196a90 con 0x7eff14102790 2024-05-15T16:29:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff196bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 0x7eff1419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff196bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 0x7eff1419c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33254/0 (socket says 172.21.15.49:33254) 2024-05-15T16:29:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff196bc700 1 -- 172.21.15.49:0/1864586655 learned_addr learned my addr 172.21.15.49:0/1864586655 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff18ebb700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff14102790 0x7eff1419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff196bc700 1 -- 172.21.15.49:0/1864586655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff14102790 msgr2=0x7eff1419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff196bc700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff14102790 0x7eff1419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff196bc700 1 -- 172.21.15.49:0/1864586655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff04009d00 con 0x7eff14101e70 2024-05-15T16:29:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.201+0000 7eff196bc700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 0x7eff1419c950 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7eff04015040 tx=0x7eff04003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.202+0000 7eff127fc700 1 -- 172.21.15.49:0/1864586655 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff0400e040 con 0x7eff14101e70 2024-05-15T16:29:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.202+0000 7eff127fc700 1 -- 172.21.15.49:0/1864586655 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff04007cd0 con 0x7eff14101e70 2024-05-15T16:29:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.202+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff14196d10 con 0x7eff14101e70 2024-05-15T16:29:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.202+0000 7eff127fc700 1 -- 172.21.15.49:0/1864586655 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff0401d6b0 con 0x7eff14101e70 2024-05-15T16:29:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.202+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff14197200 con 0x7eff14101e70 2024-05-15T16:29:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.203+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff14105600 con 0x7eff14101e70 2024-05-15T16:29:05.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.203+0000 7eff127fc700 1 -- 172.21.15.49:0/1864586655 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7eff04004040 con 0x7eff14101e70 2024-05-15T16:29:05.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.204+0000 7eff127fc700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efefc06e200 0x7efefc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:05.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.204+0000 7eff18ebb700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efefc06e200 0x7efefc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:05.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.204+0000 7eff127fc700 1 -- 172.21.15.49:0/1864586655 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7eff0408f800 con 0x7eff14101e70 2024-05-15T16:29:05.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.204+0000 7eff18ebb700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efefc06e200 0x7efefc0706c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7eff08005ea0 tx=0x7eff08005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:05.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.209+0000 7eff127fc700 1 -- 172.21.15.49:0/1864586655 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7eff0405c690 con 0x7eff14101e70 2024-05-15T16:29:05.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.446+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff14197f50 con 0x7efefc06e200 2024-05-15T16:29:05.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.447+0000 7eff127fc700 1 -- 172.21.15.49:0/1864586655 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7eff14197f50 con 0x7efefc06e200 2024-05-15T16:29:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.449+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efefc06e200 msgr2=0x7efefc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efefc06e200 0x7efefc0706c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7eff08005ea0 tx=0x7eff08005e30 comp rx=0 tx=0).stop 2024-05-15T16:29:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 msgr2=0x7eff1419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 0x7eff1419c950 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7eff04015040 tx=0x7eff04003ce0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 shutdown_connections 2024-05-15T16:29:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efefc06e200 0x7efefc0706c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff14101e70 0x7eff1419c950 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 --2- 172.21.15.49:0/1864586655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff14102790 0x7eff1419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 >> 172.21.15.49:0/1864586655 conn(0x7eff140fb700 msgr2=0x7eff14103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 shutdown_connections 2024-05-15T16:29:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.450+0000 7eff1b920700 1 -- 172.21.15.49:0/1864586655 wait complete. 2024-05-15T16:29:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.558+0000 7fa949dad700 1 -- 172.21.15.49:0/3349661842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 msgr2=0x7fa9441001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.558+0000 7fa949dad700 1 --2- 172.21.15.49:0/3349661842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 0x7fa9441001f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa934009910 tx=0x7fa934009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.558+0000 7fa949dad700 1 -- 172.21.15.49:0/3349661842 shutdown_connections 2024-05-15T16:29:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.558+0000 7fa949dad700 1 --2- 172.21.15.49:0/3349661842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9441007c0 0x7fa944104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.558+0000 7fa949dad700 1 --2- 172.21.15.49:0/3349661842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 0x7fa9441001f0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.558+0000 7fa949dad700 1 -- 172.21.15.49:0/3349661842 >> 172.21.15.49:0/3349661842 conn(0x7fa9440fb700 msgr2=0x7fa9440fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.558+0000 7fa949dad700 1 -- 172.21.15.49:0/3349661842 shutdown_connections 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.558+0000 7fa949dad700 1 -- 172.21.15.49:0/3349661842 wait complete. 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.558+0000 7fa949dad700 1 Processor -- start 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa949dad700 1 -- start start 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa949dad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 0x7fa944198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa949dad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9441007c0 0x7fa944199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa949dad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa944199b20 con 0x7fa9440ffe10 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa949dad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94419d7a0 con 0x7fa9441007c0 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa9437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 0x7fa944198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa9437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 0x7fa944198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33286/0 (socket says 172.21.15.49:33286) 2024-05-15T16:29:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa9437fe700 1 -- 172.21.15.49:0/2270233688 learned_addr learned my addr 172.21.15.49:0/2270233688 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa9437fe700 1 -- 172.21.15.49:0/2270233688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9441007c0 msgr2=0x7fa944199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa942ffd700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9441007c0 0x7fa944199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa9437fe700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9441007c0 0x7fa944199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.559+0000 7fa9437fe700 1 -- 172.21.15.49:0/2270233688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9340095f0 con 0x7fa9440ffe10 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.560+0000 7fa9437fe700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 0x7fa944198f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa93400bd70 tx=0x7fa93400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.560+0000 7fa940ff9700 1 -- 172.21.15.49:0/2270233688 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa93401d070 con 0x7fa9440ffe10 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.560+0000 7fa940ff9700 1 -- 172.21.15.49:0/2270233688 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa93400f460 con 0x7fa9440ffe10 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.560+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94419da20 con 0x7fa9440ffe10 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.560+0000 7fa940ff9700 1 -- 172.21.15.49:0/2270233688 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa934017630 con 0x7fa9440ffe10 2024-05-15T16:29:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.560+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94419df10 con 0x7fa9440ffe10 2024-05-15T16:29:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.561+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9441012f0 con 0x7fa9440ffe10 2024-05-15T16:29:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.562+0000 7fa940ff9700 1 -- 172.21.15.49:0/2270233688 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa93400f5d0 con 0x7fa9440ffe10 2024-05-15T16:29:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.562+0000 7fa940ff9700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa92c06e200 0x7fa92c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.562+0000 7fa940ff9700 1 -- 172.21.15.49:0/2270233688 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa93408f610 con 0x7fa9440ffe10 2024-05-15T16:29:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.562+0000 7fa942ffd700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa92c06e200 0x7fa92c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.563+0000 7fa942ffd700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa92c06e200 0x7fa92c0706c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa938006fd0 tx=0x7fa938008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:05.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.567+0000 7fa940ff9700 1 -- 172.21.15.49:0/2270233688 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa934058970 con 0x7fa9440ffe10 2024-05-15T16:29:05.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.784+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa94410cec0 con 0x7fa92c06e200 2024-05-15T16:29:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.794+0000 7fa940ff9700 1 -- 172.21.15.49:0/2270233688 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa94410cec0 con 0x7fa92c06e200 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (32m) 10m ago 43m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (44m) 10m ago 44m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (43m) 10m ago 43m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (29m) 10m ago 44m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (29m) 10m ago 42m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (31m) 10m ago 43m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (26m) 10m ago 38m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (25m) 10m ago 38m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (25m) 10m ago 38m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (10m) 10m ago 38m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:29:05.796 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (34m) 10m ago 45m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (33m) 10m ago 42m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (30m) 10m ago 45m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (30m) 10m ago 42m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (33m) 10m ago 44m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (33m) 10m ago 42m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (29m) 10m ago 42m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (28m) 10m ago 41m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (28m) 10m ago 41m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (27m) 10m ago 40m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (26m) 10m ago 40m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (26m) 10m ago 39m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:29:05.797 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (32m) 10m ago 43m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:29:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa92c06e200 msgr2=0x7fa92c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa92c06e200 0x7fa92c0706c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa938006fd0 tx=0x7fa938008040 comp rx=0 tx=0).stop 2024-05-15T16:29:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 msgr2=0x7fa944198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 0x7fa944198f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa93400bd70 tx=0x7fa93400be50 comp rx=0 tx=0).stop 2024-05-15T16:29:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 shutdown_connections 2024-05-15T16:29:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa92c06e200 0x7fa92c0706c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9440ffe10 0x7fa944198f00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 --2- 172.21.15.49:0/2270233688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9441007c0 0x7fa944199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 >> 172.21.15.49:0/2270233688 conn(0x7fa9440fb700 msgr2=0x7fa9440fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:05.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 shutdown_connections 2024-05-15T16:29:05.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.797+0000 7fa949dad700 1 -- 172.21.15.49:0/2270233688 wait complete. 2024-05-15T16:29:05.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.904+0000 7f33a42e2700 1 -- 172.21.15.49:0/2464993003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 msgr2=0x7f339c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.904+0000 7f33a42e2700 1 --2- 172.21.15.49:0/2464993003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 0x7f339c1047c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3390009910 tx=0x7f3390009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:05.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.904+0000 7f33a42e2700 1 -- 172.21.15.49:0/2464993003 shutdown_connections 2024-05-15T16:29:05.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.904+0000 7f33a42e2700 1 --2- 172.21.15.49:0/2464993003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 0x7f339c1047c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.904+0000 7f33a42e2700 1 --2- 172.21.15.49:0/2464993003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339c0ffdc0 0x7f339c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.904+0000 7f33a42e2700 1 -- 172.21.15.49:0/2464993003 >> 172.21.15.49:0/2464993003 conn(0x7f339c076800 msgr2=0x7f339c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:05.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.904+0000 7f33a42e2700 1 -- 172.21.15.49:0/2464993003 shutdown_connections 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.904+0000 7f33a42e2700 1 -- 172.21.15.49:0/2464993003 wait complete. 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a42e2700 1 Processor -- start 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a42e2700 1 -- start start 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a42e2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339c0ffdc0 0x7f339c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a42e2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 0x7f339c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a42e2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c19d470 con 0x7f339c100770 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a42e2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c1969e0 con 0x7f339c0ffdc0 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a187d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 0x7f339c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a187d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 0x7f339c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33312/0 (socket says 172.21.15.49:33312) 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a187d700 1 -- 172.21.15.49:0/2125784685 learned_addr learned my addr 172.21.15.49:0/2125784685 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a207e700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339c0ffdc0 0x7f339c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a187d700 1 -- 172.21.15.49:0/2125784685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339c0ffdc0 msgr2=0x7f339c19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a187d700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339c0ffdc0 0x7f339c19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.905+0000 7f33a187d700 1 -- 172.21.15.49:0/2125784685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900095f0 con 0x7f339c100770 2024-05-15T16:29:05.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.906+0000 7f33a187d700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 0x7f339c19cde0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f33900092e0 tx=0x7f339000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:05.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.906+0000 7f339b7fe700 1 -- 172.21.15.49:0/2125784685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f339001d070 con 0x7f339c100770 2024-05-15T16:29:05.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.906+0000 7f339b7fe700 1 -- 172.21.15.49:0/2125784685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f339000f460 con 0x7f339c100770 2024-05-15T16:29:05.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.906+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339c196c60 con 0x7f339c100770 2024-05-15T16:29:05.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.906+0000 7f339b7fe700 1 -- 172.21.15.49:0/2125784685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3390017610 con 0x7f339c100770 2024-05-15T16:29:05.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.906+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339c197150 con 0x7f339c100770 2024-05-15T16:29:05.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.907+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339c1012a0 con 0x7f339c100770 2024-05-15T16:29:05.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.907+0000 7f339b7fe700 1 -- 172.21.15.49:0/2125784685 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f339000f5d0 con 0x7f339c100770 2024-05-15T16:29:05.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.908+0000 7f339b7fe700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f338406e210 0x7f33840706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:05.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.908+0000 7f339b7fe700 1 -- 172.21.15.49:0/2125784685 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f339008f5c0 con 0x7f339c100770 2024-05-15T16:29:05.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.908+0000 7f33a207e700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f338406e210 0x7f33840706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:05.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.908+0000 7f33a207e700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f338406e210 0x7f33840706d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f338c0060b0 tx=0x7f338c007b80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:05.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:05.913+0000 7f339b7fe700 1 -- 172.21.15.49:0/2125784685 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f339005c450 con 0x7f339c100770 2024-05-15T16:29:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:05 smithi144 ceph-mon[86635]: pgmap v963: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:05 smithi144 ceph-mon[86635]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:29:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:05 smithi049 ceph-mon[112142]: pgmap v963: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:05 smithi049 ceph-mon[112142]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:29:06.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.210+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f339c04ec60 con 0x7f339c100770 2024-05-15T16:29:06.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.211+0000 7f339b7fe700 1 -- 172.21.15.49:0/2125784685 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f339005bfe0 con 0x7f339c100770 2024-05-15T16:29:06.212 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:29:06.213 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:29:06.213 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:29:06.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:29:06.213 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:29:06.213 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:29:06.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:29:06.214 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:29:06.214 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:29:06.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:29:06.214 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:29:06.214 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:29:06.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:29:06.215 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:29:06.215 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:29:06.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:29:06.215 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:29:06.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f338406e210 msgr2=0x7f33840706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f338406e210 0x7f33840706d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f338c0060b0 tx=0x7f338c007b80 comp rx=0 tx=0).stop 2024-05-15T16:29:06.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 msgr2=0x7f339c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 0x7f339c19cde0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f33900092e0 tx=0x7f339000be20 comp rx=0 tx=0).stop 2024-05-15T16:29:06.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 shutdown_connections 2024-05-15T16:29:06.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f338406e210 0x7f33840706d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f339c0ffdc0 0x7f339c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 --2- 172.21.15.49:0/2125784685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f339c100770 0x7f339c19cde0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 >> 172.21.15.49:0/2125784685 conn(0x7f339c076800 msgr2=0x7f339c068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:06.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 shutdown_connections 2024-05-15T16:29:06.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.213+0000 7f33a42e2700 1 -- 172.21.15.49:0/2125784685 wait complete. 2024-05-15T16:29:06.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.321+0000 7f1efcf14700 1 -- 172.21.15.49:0/1255902695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef81018b0 msgr2=0x7f1ef8105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.321+0000 7f1efcf14700 1 --2- 172.21.15.49:0/1255902695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef81018b0 0x7f1ef8105900 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009910 tx=0x7f1ee0009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:06.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.321+0000 7f1efcf14700 1 -- 172.21.15.49:0/1255902695 shutdown_connections 2024-05-15T16:29:06.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.321+0000 7f1efcf14700 1 --2- 172.21.15.49:0/1255902695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef81018b0 0x7f1ef8105900 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.321+0000 7f1efcf14700 1 --2- 172.21.15.49:0/1255902695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef8100f00 0x7f1ef81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.321+0000 7f1efcf14700 1 -- 172.21.15.49:0/1255902695 >> 172.21.15.49:0/1255902695 conn(0x7f1ef80fc7d0 msgr2=0x7f1ef80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:06.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.321+0000 7f1efcf14700 1 -- 172.21.15.49:0/1255902695 shutdown_connections 2024-05-15T16:29:06.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.321+0000 7f1efcf14700 1 -- 172.21.15.49:0/1255902695 wait complete. 2024-05-15T16:29:06.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.322+0000 7f1efcf14700 1 Processor -- start 2024-05-15T16:29:06.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.322+0000 7f1efcf14700 1 -- start start 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.322+0000 7f1efcf14700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef8100f00 0x7f1ef80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1efcf14700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef81018b0 0x7f1ef80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1efcf14700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef8073d20 con 0x7f1ef8100f00 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1efcf14700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef8073e60 con 0x7f1ef81018b0 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1ef659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef8100f00 0x7f1ef80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1ef659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef8100f00 0x7f1ef80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33328/0 (socket says 172.21.15.49:33328) 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1ef659c700 1 -- 172.21.15.49:0/4176925119 learned_addr learned my addr 172.21.15.49:0/4176925119 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1eeffff700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef81018b0 0x7f1ef80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1ef659c700 1 -- 172.21.15.49:0/4176925119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef81018b0 msgr2=0x7f1ef80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1ef659c700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef81018b0 0x7f1ef80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1ef659c700 1 -- 172.21.15.49:0/4176925119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00095f0 con 0x7f1ef8100f00 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1ef659c700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef8100f00 0x7f1ef80732a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d8d0 tx=0x7f1ee800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:06.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1eef7fe700 1 -- 172.21.15.49:0/4176925119 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ee80098a0 con 0x7f1ef8100f00 2024-05-15T16:29:06.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1eef7fe700 1 -- 172.21.15.49:0/4176925119 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ee8010460 con 0x7f1ef8100f00 2024-05-15T16:29:06.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef81a2da0 con 0x7f1ef8100f00 2024-05-15T16:29:06.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1eef7fe700 1 -- 172.21.15.49:0/4176925119 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ee800f5f0 con 0x7f1ef8100f00 2024-05-15T16:29:06.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.323+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef81a3100 con 0x7f1ef8100f00 2024-05-15T16:29:06.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.324+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef8109300 con 0x7f1ef8100f00 2024-05-15T16:29:06.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.325+0000 7f1eef7fe700 1 -- 172.21.15.49:0/4176925119 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1ee8009a00 con 0x7f1ef8100f00 2024-05-15T16:29:06.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.326+0000 7f1eef7fe700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1ee406e210 0x7f1ee40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:06.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.326+0000 7f1eef7fe700 1 -- 172.21.15.49:0/4176925119 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1ee808df80 con 0x7f1ef8100f00 2024-05-15T16:29:06.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.326+0000 7f1eeffff700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1ee406e210 0x7f1ee40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:06.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.326+0000 7f1eeffff700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1ee406e210 0x7f1ee40706d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009fd0 tx=0x7f1ee001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:06.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.330+0000 7f1eef7fe700 1 -- 172.21.15.49:0/4176925119 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1ee80572e0 con 0x7f1ef8100f00 2024-05-15T16:29:06.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.582+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1ef8074780 con 0x7f1ef8100f00 2024-05-15T16:29:06.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.583+0000 7f1eef7fe700 1 -- 172.21.15.49:0/4176925119 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f1ee8016760 con 0x7f1ef8100f00 2024-05-15T16:29:06.584 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:29:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:29:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:29:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:29:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:29:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:29:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:29:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:29:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:29:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:29:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:29:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:29:06.588 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:29:06.588 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:29:06.588 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:29:06.588 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:29:06.588 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:29:06.588 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:29:06.588 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:29:06.588 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1ee406e210 msgr2=0x7f1ee40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1ee406e210 0x7f1ee40706d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009fd0 tx=0x7f1ee001a040 comp rx=0 tx=0).stop 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef8100f00 msgr2=0x7f1ef80732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef8100f00 0x7f1ef80732a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d8d0 tx=0x7f1ee800dbe0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 shutdown_connections 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1ee406e210 0x7f1ee40706d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ef8100f00 0x7f1ef80732a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 --2- 172.21.15.49:0/4176925119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef81018b0 0x7f1ef80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 >> 172.21.15.49:0/4176925119 conn(0x7f1ef80fc7d0 msgr2=0x7f1ef80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 shutdown_connections 2024-05-15T16:29:06.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.585+0000 7f1efcf14700 1 -- 172.21.15.49:0/4176925119 wait complete. 2024-05-15T16:29:06.590 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:29:06.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.696+0000 7f4779fd7700 1 -- 172.21.15.49:0/777036592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774101890 msgr2=0x7f47741058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.696+0000 7f4779fd7700 1 --2- 172.21.15.49:0/777036592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774101890 0x7f47741058e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4768009910 tx=0x7f4768009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:06.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.696+0000 7f4779fd7700 1 -- 172.21.15.49:0/777036592 shutdown_connections 2024-05-15T16:29:06.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.696+0000 7f4779fd7700 1 --2- 172.21.15.49:0/777036592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774101890 0x7f47741058e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.696+0000 7f4779fd7700 1 --2- 172.21.15.49:0/777036592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774100ee0 0x7f47741012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.696+0000 7f4779fd7700 1 -- 172.21.15.49:0/777036592 >> 172.21.15.49:0/777036592 conn(0x7f47740fc790 msgr2=0x7f47740febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.696+0000 7f4779fd7700 1 -- 172.21.15.49:0/777036592 shutdown_connections 2024-05-15T16:29:06.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.696+0000 7f4779fd7700 1 -- 172.21.15.49:0/777036592 wait complete. 2024-05-15T16:29:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f4779fd7700 1 Processor -- start 2024-05-15T16:29:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f4779fd7700 1 -- start start 2024-05-15T16:29:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f4779fd7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774100ee0 0x7f477419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f4779fd7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774101890 0x7f477419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:06.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f4779fd7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477419d440 con 0x7f4774100ee0 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f4779fd7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4774196a20 con 0x7f4774101890 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f47737fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774100ee0 0x7f477419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f47737fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774100ee0 0x7f477419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33348/0 (socket says 172.21.15.49:33348) 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f47737fe700 1 -- 172.21.15.49:0/4175309367 learned_addr learned my addr 172.21.15.49:0/4175309367 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f4772ffd700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774101890 0x7f477419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f47737fe700 1 -- 172.21.15.49:0/4175309367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774101890 msgr2=0x7f477419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f47737fe700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774101890 0x7f477419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.697+0000 7f47737fe700 1 -- 172.21.15.49:0/4175309367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680095f0 con 0x7f4774100ee0 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.698+0000 7f47737fe700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774100ee0 0x7f477419c8e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f476400c8a0 tx=0x7f476400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.698+0000 7f4770ff9700 1 -- 172.21.15.49:0/4175309367 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47640078e0 con 0x7f4774100ee0 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.698+0000 7f4770ff9700 1 -- 172.21.15.49:0/4175309367 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f476400f450 con 0x7f4774100ee0 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.698+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4774196d00 con 0x7f4774100ee0 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.698+0000 7f4770ff9700 1 -- 172.21.15.49:0/4175309367 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f476400e5e0 con 0x7f4774100ee0 2024-05-15T16:29:06.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.698+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4774197250 con 0x7f4774100ee0 2024-05-15T16:29:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.699+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47741092e0 con 0x7f4774100ee0 2024-05-15T16:29:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.699+0000 7f4770ff9700 1 -- 172.21.15.49:0/4175309367 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f476400f5c0 con 0x7f4774100ee0 2024-05-15T16:29:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.700+0000 7f4770ff9700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f475c06e200 0x7f475c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:06.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.700+0000 7f4770ff9700 1 -- 172.21.15.49:0/4175309367 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f476408e1b0 con 0x7f4774100ee0 2024-05-15T16:29:06.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.700+0000 7f4772ffd700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f475c06e200 0x7f475c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:06.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.700+0000 7f4772ffd700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f475c06e200 0x7f475c0706c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4774197be0 tx=0x7f476801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:06.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.705+0000 7f4770ff9700 1 -- 172.21.15.49:0/4175309367 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f476405afc0 con 0x7f4774100ee0 2024-05-15T16:29:06.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.935+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4774197f50 con 0x7f475c06e200 2024-05-15T16:29:06.936 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:06 smithi049 ceph-mon[112142]: from='client.35750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:06.936 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:06 smithi049 ceph-mon[112142]: from='client.35754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:06.936 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:06 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2125784685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:29:06.936 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:06 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4176925119' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:29:06.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.936+0000 7f4770ff9700 1 -- 172.21.15.49:0/4175309367 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4774197f50 con 0x7f475c06e200 2024-05-15T16:29:06.937 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:29:06.938 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:29:06.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.938+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f475c06e200 msgr2=0x7f475c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.938+0000 7f4779fd7700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f475c06e200 0x7f475c0706c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4774197be0 tx=0x7f476801a040 comp rx=0 tx=0).stop 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.938+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774100ee0 msgr2=0x7f477419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.938+0000 7f4779fd7700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774100ee0 0x7f477419c8e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f476400c8a0 tx=0x7f476400cbb0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.938+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 shutdown_connections 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.938+0000 7f4779fd7700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f475c06e200 0x7f475c0706c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.938+0000 7f4779fd7700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4774100ee0 0x7f477419c8e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.938+0000 7f4779fd7700 1 --2- 172.21.15.49:0/4175309367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774101890 0x7f477419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.938+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 >> 172.21.15.49:0/4175309367 conn(0x7f47740fc790 msgr2=0x7f47740fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.939+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 shutdown_connections 2024-05-15T16:29:06.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:06.939+0000 7f4779fd7700 1 -- 172.21.15.49:0/4175309367 wait complete. 2024-05-15T16:29:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.047+0000 7f0f7d43a700 1 -- 172.21.15.49:0/2748035951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f780ffde0 msgr2=0x7f0f781001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.047+0000 7f0f7d43a700 1 --2- 172.21.15.49:0/2748035951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f780ffde0 0x7f0f781001c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009910 tx=0x7f0f68009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.047+0000 7f0f7d43a700 1 -- 172.21.15.49:0/2748035951 shutdown_connections 2024-05-15T16:29:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.047+0000 7f0f7d43a700 1 --2- 172.21.15.49:0/2748035951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f78100790 0x7f0f781047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.047+0000 7f0f7d43a700 1 --2- 172.21.15.49:0/2748035951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f780ffde0 0x7f0f781001c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.047+0000 7f0f7d43a700 1 -- 172.21.15.49:0/2748035951 >> 172.21.15.49:0/2748035951 conn(0x7f0f780fb690 msgr2=0x7f0f780fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.047+0000 7f0f7d43a700 1 -- 172.21.15.49:0/2748035951 shutdown_connections 2024-05-15T16:29:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.047+0000 7f0f7d43a700 1 -- 172.21.15.49:0/2748035951 wait complete. 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f7d43a700 1 Processor -- start 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f7d43a700 1 -- start start 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f7d43a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f780ffde0 0x7f0f7819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f7d43a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f78100790 0x7f0f7819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f7d43a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7819d410 con 0x7f0f78100790 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f7d43a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f781969f0 con 0x7f0f780ffde0 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f6ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f78100790 0x7f0f7819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f6ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f78100790 0x7f0f7819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33364/0 (socket says 172.21.15.49:33364) 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f6ffff700 1 -- 172.21.15.49:0/148615299 learned_addr learned my addr 172.21.15.49:0/148615299 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f76ffd700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f780ffde0 0x7f0f7819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f6ffff700 1 -- 172.21.15.49:0/148615299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f780ffde0 msgr2=0x7f0f7819c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f6ffff700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f780ffde0 0x7f0f7819c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.048+0000 7f0f6ffff700 1 -- 172.21.15.49:0/148615299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f680095f0 con 0x7f0f78100790 2024-05-15T16:29:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.049+0000 7f0f6ffff700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f78100790 0x7f0f7819cdf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000d900 tx=0x7f0f6000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.049+0000 7f0f74ff9700 1 -- 172.21.15.49:0/148615299 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f60009900 con 0x7f0f78100790 2024-05-15T16:29:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.049+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f78196cd0 con 0x7f0f78100790 2024-05-15T16:29:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.049+0000 7f0f74ff9700 1 -- 172.21.15.49:0/148615299 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f60010460 con 0x7f0f78100790 2024-05-15T16:29:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.049+0000 7f0f74ff9700 1 -- 172.21.15.49:0/148615299 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f6000f5f0 con 0x7f0f78100790 2024-05-15T16:29:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.049+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f78197220 con 0x7f0f78100790 2024-05-15T16:29:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.050+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f78108180 con 0x7f0f78100790 2024-05-15T16:29:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.050+0000 7f0f74ff9700 1 -- 172.21.15.49:0/148615299 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0f600105d0 con 0x7f0f78100790 2024-05-15T16:29:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.051+0000 7f0f74ff9700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f6406e210 0x7f0f640706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.051+0000 7f0f74ff9700 1 -- 172.21.15.49:0/148615299 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0f6008ddc0 con 0x7f0f78100790 2024-05-15T16:29:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.051+0000 7f0f76ffd700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f6406e210 0x7f0f640706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.051+0000 7f0f76ffd700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f6406e210 0x7f0f640706d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0f68000c00 tx=0x7f0f6801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:07.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.056+0000 7f0f74ff9700 1 -- 172.21.15.49:0/148615299 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0f6005ab70 con 0x7f0f78100790 2024-05-15T16:29:07.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:06 smithi144 ceph-mon[86635]: from='client.35750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:07.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:06 smithi144 ceph-mon[86635]: from='client.35754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:07.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:06 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2125784685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:29:07.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:06 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4176925119' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:29:07.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.347+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0f78197f10 con 0x7f0f78100790 2024-05-15T16:29:07.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.347+0000 7f0f74ff9700 1 -- 172.21.15.49:0/148615299 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f0f6005a700 con 0x7f0f78100790 2024-05-15T16:29:07.348 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:29:07.349 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:29:07.350 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:29:07.350 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:29:07.350 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:29:07.350 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:29:07.350 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:29:07.350 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:29:07.350 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:29:07.350 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:29:07.350 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f6406e210 msgr2=0x7f0f640706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f6406e210 0x7f0f640706d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0f68000c00 tx=0x7f0f6801a040 comp rx=0 tx=0).stop 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f78100790 msgr2=0x7f0f7819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f78100790 0x7f0f7819cdf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000d900 tx=0x7f0f6000dc10 comp rx=0 tx=0).stop 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 shutdown_connections 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0f6406e210 0x7f0f640706d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f780ffde0 0x7f0f7819c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 --2- 172.21.15.49:0/148615299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f78100790 0x7f0f7819cdf0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 >> 172.21.15.49:0/148615299 conn(0x7f0f780fb690 msgr2=0x7f0f78109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 shutdown_connections 2024-05-15T16:29:07.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:07.349+0000 7f0f7d43a700 1 -- 172.21.15.49:0/148615299 wait complete. 2024-05-15T16:29:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:07 smithi144 ceph-mon[86635]: pgmap v964: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:07 smithi144 ceph-mon[86635]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:07 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/148615299' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:29:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:07 smithi049 ceph-mon[112142]: pgmap v964: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:07 smithi049 ceph-mon[112142]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:07 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/148615299' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:29:10.109 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:09 smithi144 ceph-mon[86635]: pgmap v965: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:09 smithi049 ceph-mon[112142]: pgmap v965: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:11 smithi144 ceph-mon[86635]: pgmap v966: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:11 smithi049 ceph-mon[112142]: pgmap v966: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:13 smithi144 ceph-mon[86635]: pgmap v967: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:13 smithi049 ceph-mon[112142]: pgmap v967: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:15 smithi144 ceph-mon[86635]: pgmap v968: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:15 smithi049 ceph-mon[112142]: pgmap v968: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:17 smithi144 ceph-mon[86635]: pgmap v969: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:17 smithi049 ceph-mon[112142]: pgmap v969: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:19 smithi144 ceph-mon[86635]: pgmap v970: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:19 smithi049 ceph-mon[112142]: pgmap v970: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:21.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:29:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:29:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:21 smithi144 ceph-mon[86635]: pgmap v971: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:21 smithi049 ceph-mon[112142]: pgmap v971: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:23 smithi144 ceph-mon[86635]: pgmap v972: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:23 smithi049 ceph-mon[112142]: pgmap v972: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:25 smithi144 ceph-mon[86635]: pgmap v973: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:25 smithi049 ceph-mon[112142]: pgmap v973: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:27 smithi144 ceph-mon[86635]: pgmap v974: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:27 smithi049 ceph-mon[112142]: pgmap v974: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:29 smithi144 ceph-mon[86635]: pgmap v975: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:29 smithi049 ceph-mon[112142]: pgmap v975: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:31 smithi144 ceph-mon[86635]: pgmap v976: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:31 smithi049 ceph-mon[112142]: pgmap v976: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:33 smithi144 ceph-mon[86635]: pgmap v977: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:33 smithi049 ceph-mon[112142]: pgmap v977: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:35 smithi144 ceph-mon[86635]: pgmap v978: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:29:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:35 smithi049 ceph-mon[112142]: pgmap v978: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:29:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.472+0000 7fc81050f700 1 -- 172.21.15.49:0/3377217008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 msgr2=0x7fc80810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.472+0000 7fc81050f700 1 --2- 172.21.15.49:0/3377217008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 0x7fc80810d020 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009910 tx=0x7fc7fc009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.472+0000 7fc81050f700 1 -- 172.21.15.49:0/3377217008 shutdown_connections 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.472+0000 7fc81050f700 1 --2- 172.21.15.49:0/3377217008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 0x7fc80810d020 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.472+0000 7fc81050f700 1 --2- 172.21.15.49:0/3377217008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8080732c0 0x7fc8080736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.472+0000 7fc81050f700 1 -- 172.21.15.49:0/3377217008 >> 172.21.15.49:0/3377217008 conn(0x7fc8080fc7d0 msgr2=0x7fc8080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.472+0000 7fc81050f700 1 -- 172.21.15.49:0/3377217008 shutdown_connections 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.472+0000 7fc81050f700 1 -- 172.21.15.49:0/3377217008 wait complete. 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.473+0000 7fc81050f700 1 Processor -- start 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.473+0000 7fc81050f700 1 -- start start 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.473+0000 7fc81050f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8080732c0 0x7fc808198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.473+0000 7fc81050f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 0x7fc808199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.473+0000 7fc81050f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc808199a00 con 0x7fc808073be0 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.473+0000 7fc81050f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80819d680 con 0x7fc8080732c0 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.473+0000 7fc80daaa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 0x7fc808199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.473+0000 7fc80daaa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 0x7fc808199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44054/0 (socket says 172.21.15.49:44054) 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.473+0000 7fc80daaa700 1 -- 172.21.15.49:0/2449693800 learned_addr learned my addr 172.21.15.49:0/2449693800 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc80e2ab700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8080732c0 0x7fc808198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc80daaa700 1 -- 172.21.15.49:0/2449693800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8080732c0 msgr2=0x7fc808198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc80daaa700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8080732c0 0x7fc808198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc80daaa700 1 -- 172.21.15.49:0/2449693800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc0095f0 con 0x7fc808073be0 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc80daaa700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 0x7fc808199320 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00bd70 tx=0x7fc7fc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc8077fe700 1 -- 172.21.15.49:0/2449693800 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7fc01d070 con 0x7fc808073be0 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc8077fe700 1 -- 172.21.15.49:0/2449693800 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7fc0056e0 con 0x7fc808073be0 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80819d900 con 0x7fc808073be0 2024-05-15T16:29:37.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc8077fe700 1 -- 172.21.15.49:0/2449693800 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7fc00fe40 con 0x7fc808073be0 2024-05-15T16:29:37.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.474+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80819ddf0 con 0x7fc808073be0 2024-05-15T16:29:37.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.475+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80810a7c0 con 0x7fc808073be0 2024-05-15T16:29:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.476+0000 7fc8077fe700 1 -- 172.21.15.49:0/2449693800 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc7fc00f680 con 0x7fc808073be0 2024-05-15T16:29:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.476+0000 7fc8077fe700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc7f006e200 0x7fc7f00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.476+0000 7fc8077fe700 1 -- 172.21.15.49:0/2449693800 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc7fc08f520 con 0x7fc808073be0 2024-05-15T16:29:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.476+0000 7fc80e2ab700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc7f006e200 0x7fc7f00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.477+0000 7fc80e2ab700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc7f006e200 0x7fc7f00706c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009550 tx=0x7fc7f80091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:37.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.481+0000 7fc8077fe700 1 -- 172.21.15.49:0/2449693800 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc7fc058880 con 0x7fc808073be0 2024-05-15T16:29:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.708+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc80819a110 con 0x7fc7f006e200 2024-05-15T16:29:37.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.709+0000 7fc8077fe700 1 -- 172.21.15.49:0/2449693800 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc80819a110 con 0x7fc7f006e200 2024-05-15T16:29:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc7f006e200 msgr2=0x7fc7f00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc7f006e200 0x7fc7f00706c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009550 tx=0x7fc7f80091b0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 msgr2=0x7fc808199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 0x7fc808199320 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00bd70 tx=0x7fc7fc00bda0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 shutdown_connections 2024-05-15T16:29:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc7f006e200 0x7fc7f00706c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8080732c0 0x7fc808198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 --2- 172.21.15.49:0/2449693800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc808073be0 0x7fc808199320 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 >> 172.21.15.49:0/2449693800 conn(0x7fc8080fc7d0 msgr2=0x7fc808107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 shutdown_connections 2024-05-15T16:29:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.711+0000 7fc81050f700 1 -- 172.21.15.49:0/2449693800 wait complete. 2024-05-15T16:29:37.723 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:29:37.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.821+0000 7f66afc09700 1 -- 172.21.15.49:0/851458748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 msgr2=0x7f66a81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:37.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.821+0000 7f66afc09700 1 --2- 172.21.15.49:0/851458748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 0x7f66a81012c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009910 tx=0x7f66a4009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:37.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.821+0000 7f66afc09700 1 -- 172.21.15.49:0/851458748 shutdown_connections 2024-05-15T16:29:37.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.821+0000 7f66afc09700 1 --2- 172.21.15.49:0/851458748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a8101890 0x7f66a81058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.821+0000 7f66afc09700 1 --2- 172.21.15.49:0/851458748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 0x7f66a81012c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.821+0000 7f66afc09700 1 -- 172.21.15.49:0/851458748 >> 172.21.15.49:0/851458748 conn(0x7f66a80fc790 msgr2=0x7f66a80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:37.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.821+0000 7f66afc09700 1 -- 172.21.15.49:0/851458748 shutdown_connections 2024-05-15T16:29:37.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.821+0000 7f66afc09700 1 -- 172.21.15.49:0/851458748 wait complete. 2024-05-15T16:29:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66afc09700 1 Processor -- start 2024-05-15T16:29:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66afc09700 1 -- start start 2024-05-15T16:29:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66afc09700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 0x7f66a819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66afc09700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a8101890 0x7f66a819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66afc09700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a819d3d0 con 0x7f66a8100ee0 2024-05-15T16:29:37.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66afc09700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a8196a90 con 0x7f66a8101890 2024-05-15T16:29:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66ad9a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 0x7f66a819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66ad9a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 0x7f66a819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44078/0 (socket says 172.21.15.49:44078) 2024-05-15T16:29:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66ad9a5700 1 -- 172.21.15.49:0/2464192887 learned_addr learned my addr 172.21.15.49:0/2464192887 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.822+0000 7f66ad1a4700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a8101890 0x7f66a819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:37.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.823+0000 7f66ad9a5700 1 -- 172.21.15.49:0/2464192887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a8101890 msgr2=0x7f66a819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:37.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.823+0000 7f66ad9a5700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a8101890 0x7f66a819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:37.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.823+0000 7f66ad9a5700 1 -- 172.21.15.49:0/2464192887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a40095f0 con 0x7f66a8100ee0 2024-05-15T16:29:37.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.823+0000 7f66ad9a5700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 0x7f66a819c900 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f66a4006010 tx=0x7f66a400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:37.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.823+0000 7f669effd700 1 -- 172.21.15.49:0/2464192887 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66a401d070 con 0x7f66a8100ee0 2024-05-15T16:29:37.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.823+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a8196d10 con 0x7f66a8100ee0 2024-05-15T16:29:37.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.823+0000 7f669effd700 1 -- 172.21.15.49:0/2464192887 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66a400f460 con 0x7f66a8100ee0 2024-05-15T16:29:37.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.823+0000 7f669effd700 1 -- 172.21.15.49:0/2464192887 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66a40175f0 con 0x7f66a8100ee0 2024-05-15T16:29:37.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.823+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a8197200 con 0x7f66a8100ee0 2024-05-15T16:29:37.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.824+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a8109280 con 0x7f66a8100ee0 2024-05-15T16:29:37.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.825+0000 7f669effd700 1 -- 172.21.15.49:0/2464192887 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f66a400f5d0 con 0x7f66a8100ee0 2024-05-15T16:29:37.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.826+0000 7f669effd700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f669406e200 0x7f66940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:37.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.826+0000 7f669effd700 1 -- 172.21.15.49:0/2464192887 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f66a408f620 con 0x7f66a8100ee0 2024-05-15T16:29:37.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.826+0000 7f66ad1a4700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f669406e200 0x7f66940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:37.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.826+0000 7f66ad1a4700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f669406e200 0x7f66940706c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f66a8197c00 tx=0x7f669800a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:37.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:37.830+0000 7f669effd700 1 -- 172.21.15.49:0/2464192887 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f66a405c430 con 0x7f66a8100ee0 2024-05-15T16:29:38.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.061+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66a8197f70 con 0x7f669406e200 2024-05-15T16:29:38.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.063+0000 7f669effd700 1 -- 172.21.15.49:0/2464192887 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f66a8197f70 con 0x7f669406e200 2024-05-15T16:29:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f669406e200 msgr2=0x7f66940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f669406e200 0x7f66940706c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f66a8197c00 tx=0x7f669800a380 comp rx=0 tx=0).stop 2024-05-15T16:29:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 msgr2=0x7f66a819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 0x7f66a819c900 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f66a4006010 tx=0x7f66a400bd70 comp rx=0 tx=0).stop 2024-05-15T16:29:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 shutdown_connections 2024-05-15T16:29:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f669406e200 0x7f66940706c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66a8100ee0 0x7f66a819c900 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 --2- 172.21.15.49:0/2464192887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a8101890 0x7f66a819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 >> 172.21.15.49:0/2464192887 conn(0x7f66a80fc790 msgr2=0x7f66a80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:38.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 shutdown_connections 2024-05-15T16:29:38.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.065+0000 7f66afc09700 1 -- 172.21.15.49:0/2464192887 wait complete. 2024-05-15T16:29:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.173+0000 7f52ba97e700 1 -- 172.21.15.49:0/3921967924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 msgr2=0x7f52b41047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.173+0000 7f52ba97e700 1 --2- 172.21.15.49:0/3921967924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 0x7f52b41047c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f529c009910 tx=0x7f529c009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.173+0000 7f52ba97e700 1 -- 172.21.15.49:0/3921967924 shutdown_connections 2024-05-15T16:29:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.173+0000 7f52ba97e700 1 --2- 172.21.15.49:0/3921967924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 0x7f52b41047c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.173+0000 7f52ba97e700 1 --2- 172.21.15.49:0/3921967924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b40ffdc0 0x7f52b41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.173+0000 7f52ba97e700 1 -- 172.21.15.49:0/3921967924 >> 172.21.15.49:0/3921967924 conn(0x7f52b4076800 msgr2=0x7f52b40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:38.175 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:37 smithi049 ceph-mon[112142]: pgmap v979: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.174+0000 7f52ba97e700 1 -- 172.21.15.49:0/3921967924 shutdown_connections 2024-05-15T16:29:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.174+0000 7f52ba97e700 1 -- 172.21.15.49:0/3921967924 wait complete. 2024-05-15T16:29:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.174+0000 7f52ba97e700 1 Processor -- start 2024-05-15T16:29:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.174+0000 7f52ba97e700 1 -- start start 2024-05-15T16:29:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52ba97e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b40ffdc0 0x7f52b40fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52ba97e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 0x7f52b40fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52ba97e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40fdcb0 con 0x7f52b4100770 2024-05-15T16:29:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52ba97e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40fc760 con 0x7f52b40ffdc0 2024-05-15T16:29:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52abfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 0x7f52b40fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52abfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 0x7f52b40fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44090/0 (socket says 172.21.15.49:44090) 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52abfff700 1 -- 172.21.15.49:0/3261009232 learned_addr learned my addr 172.21.15.49:0/3261009232 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52b3fff700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b40ffdc0 0x7f52b40fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52abfff700 1 -- 172.21.15.49:0/3261009232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b40ffdc0 msgr2=0x7f52b40fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52abfff700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b40ffdc0 0x7f52b40fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52abfff700 1 -- 172.21.15.49:0/3261009232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c0095f0 con 0x7f52b4100770 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52abfff700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 0x7f52b40fc220 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f529c0092e0 tx=0x7f529c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.175+0000 7f52b1ffb700 1 -- 172.21.15.49:0/3261009232 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f529c01d070 con 0x7f52b4100770 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.176+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b40fc9b0 con 0x7f52b4100770 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.176+0000 7f52b1ffb700 1 -- 172.21.15.49:0/3261009232 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f529c00f460 con 0x7f52b4100770 2024-05-15T16:29:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.176+0000 7f52b1ffb700 1 -- 172.21.15.49:0/3261009232 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f529c017630 con 0x7f52b4100770 2024-05-15T16:29:38.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.176+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b41a2f30 con 0x7f52b4100770 2024-05-15T16:29:38.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.177+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b41012a0 con 0x7f52b4100770 2024-05-15T16:29:38.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.177+0000 7f52b1ffb700 1 -- 172.21.15.49:0/3261009232 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f529c00f5d0 con 0x7f52b4100770 2024-05-15T16:29:38.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.178+0000 7f52b1ffb700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52a006e200 0x7f52a00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:38.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.178+0000 7f52b1ffb700 1 -- 172.21.15.49:0/3261009232 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f529c08f5f0 con 0x7f52b4100770 2024-05-15T16:29:38.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.178+0000 7f52b3fff700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52a006e200 0x7f52a00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:38.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.178+0000 7f52b3fff700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52a006e200 0x7f52a00706c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f52a4005ea0 tx=0x7f52a4005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.183+0000 7f52b1ffb700 1 -- 172.21.15.49:0/3261009232 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f529c0588d0 con 0x7f52b4100770 2024-05-15T16:29:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:37 smithi144 ceph-mon[86635]: pgmap v979: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.400+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f52b410cdd0 con 0x7f52a006e200 2024-05-15T16:29:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.411+0000 7f52b1ffb700 1 -- 172.21.15.49:0/3261009232 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f52b410cdd0 con 0x7f52a006e200 2024-05-15T16:29:38.412 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:29:38.412 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (32m) 10m ago 44m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:29:38.412 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (44m) 10m ago 44m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:29:38.412 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (43m) 10m ago 43m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (30m) 10m ago 44m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (30m) 10m ago 43m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (32m) 10m ago 44m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (26m) 10m ago 38m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (26m) 10m ago 38m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (25m) 10m ago 38m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (10m) 10m ago 38m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (35m) 10m ago 45m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (34m) 10m ago 43m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (31m) 10m ago 46m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (31m) 10m ago 43m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (33m) 10m ago 44m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (33m) 10m ago 43m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (29m) 10m ago 42m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:29:38.413 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (29m) 10m ago 42m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:29:38.414 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (28m) 10m ago 41m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:29:38.414 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (28m) 10m ago 41m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:29:38.414 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (27m) 10m ago 40m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:29:38.414 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (26m) 10m ago 40m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:29:38.414 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (33m) 10m ago 43m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:29:38.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52a006e200 msgr2=0x7f52a00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52a006e200 0x7f52a00706c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f52a4005ea0 tx=0x7f52a4005e30 comp rx=0 tx=0).stop 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 msgr2=0x7f52b40fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 0x7f52b40fc220 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f529c0092e0 tx=0x7f529c00be20 comp rx=0 tx=0).stop 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 shutdown_connections 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f52a006e200 0x7f52a00706c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b40ffdc0 0x7f52b40fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 --2- 172.21.15.49:0/3261009232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52b4100770 0x7f52b40fc220 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 >> 172.21.15.49:0/3261009232 conn(0x7f52b4076800 msgr2=0x7f52b4068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 shutdown_connections 2024-05-15T16:29:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.413+0000 7f52ba97e700 1 -- 172.21.15.49:0/3261009232 wait complete. 2024-05-15T16:29:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.521+0000 7f88912f6700 1 -- 172.21.15.49:0/870014424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c101890 msgr2=0x7f888c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.521+0000 7f88912f6700 1 --2- 172.21.15.49:0/870014424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c101890 0x7f888c1058e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8880009910 tx=0x7f8880009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.521+0000 7f88912f6700 1 -- 172.21.15.49:0/870014424 shutdown_connections 2024-05-15T16:29:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.521+0000 7f88912f6700 1 --2- 172.21.15.49:0/870014424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c101890 0x7f888c1058e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.521+0000 7f88912f6700 1 --2- 172.21.15.49:0/870014424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c100ee0 0x7f888c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.521+0000 7f88912f6700 1 -- 172.21.15.49:0/870014424 >> 172.21.15.49:0/870014424 conn(0x7f888c0fc790 msgr2=0x7f888c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.521+0000 7f88912f6700 1 -- 172.21.15.49:0/870014424 shutdown_connections 2024-05-15T16:29:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.522+0000 7f88912f6700 1 -- 172.21.15.49:0/870014424 wait complete. 2024-05-15T16:29:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.522+0000 7f88912f6700 1 Processor -- start 2024-05-15T16:29:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.522+0000 7f88912f6700 1 -- start start 2024-05-15T16:29:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f88912f6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c100ee0 0x7f888c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f88912f6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101890 0x7f888c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f88912f6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c075270 con 0x7f888c100ee0 2024-05-15T16:29:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f88912f6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c073d20 con 0x7f888c101890 2024-05-15T16:29:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f888affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c100ee0 0x7f888c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f888affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c100ee0 0x7f888c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44098/0 (socket says 172.21.15.49:44098) 2024-05-15T16:29:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f888affd700 1 -- 172.21.15.49:0/3446266798 learned_addr learned my addr 172.21.15.49:0/3446266798 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f888a7fc700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101890 0x7f888c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f888affd700 1 -- 172.21.15.49:0/3446266798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101890 msgr2=0x7f888c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f888affd700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101890 0x7f888c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f888affd700 1 -- 172.21.15.49:0/3446266798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88800095f0 con 0x7f888c100ee0 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f888affd700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c100ee0 0x7f888c0732a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c930 tx=0x7f887c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f8873fff700 1 -- 172.21.15.49:0/3446266798 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f887c007ad0 con 0x7f888c100ee0 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f8873fff700 1 -- 172.21.15.49:0/3446266798 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f887c00ce80 con 0x7f888c100ee0 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c073fd0 con 0x7f888c100ee0 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f8873fff700 1 -- 172.21.15.49:0/3446266798 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f887c018630 con 0x7f888c100ee0 2024-05-15T16:29:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.523+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c1a3020 con 0x7f888c100ee0 2024-05-15T16:29:38.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.524+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888c1092e0 con 0x7f888c100ee0 2024-05-15T16:29:38.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.525+0000 7f8873fff700 1 -- 172.21.15.49:0/3446266798 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f887c007c30 con 0x7f888c100ee0 2024-05-15T16:29:38.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.526+0000 7f8873fff700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e200 0x7f88740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:38.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.526+0000 7f888a7fc700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e200 0x7f88740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:38.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.526+0000 7f8873fff700 1 -- 172.21.15.49:0/3446266798 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f887c08e110 con 0x7f888c100ee0 2024-05-15T16:29:38.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.526+0000 7f888a7fc700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e200 0x7f88740706c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f888c0749d0 tx=0x7f888001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:38.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.530+0000 7f8873fff700 1 -- 172.21.15.49:0/3446266798 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f887c0573f0 con 0x7f888c100ee0 2024-05-15T16:29:38.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.828+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f888c074810 con 0x7f888c100ee0 2024-05-15T16:29:38.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.828+0000 7f8873fff700 1 -- 172.21.15.49:0/3446266798 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f887c05aab0 con 0x7f888c100ee0 2024-05-15T16:29:38.830 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:29:38.830 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:29:38.830 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:29:38.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:29:38.830 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:29:38.830 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:29:38.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:29:38.830 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:29:38.831 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:29:38.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:29:38.831 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:29:38.831 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:29:38.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:29:38.831 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:29:38.831 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:29:38.831 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:29:38.831 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:29:38.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e200 msgr2=0x7f88740706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e200 0x7f88740706c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f888c0749d0 tx=0x7f888001a040 comp rx=0 tx=0).stop 2024-05-15T16:29:38.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c100ee0 msgr2=0x7f888c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c100ee0 0x7f888c0732a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c930 tx=0x7f887c00cc40 comp rx=0 tx=0).stop 2024-05-15T16:29:38.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 shutdown_connections 2024-05-15T16:29:38.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f887406e200 0x7f88740706c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f888c100ee0 0x7f888c0732a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 --2- 172.21.15.49:0/3446266798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101890 0x7f888c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 >> 172.21.15.49:0/3446266798 conn(0x7f888c0fc790 msgr2=0x7f888c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:38.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 shutdown_connections 2024-05-15T16:29:38.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.831+0000 7f88912f6700 1 -- 172.21.15.49:0/3446266798 wait complete. 2024-05-15T16:29:38.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.941+0000 7f0a00211700 1 -- 172.21.15.49:0/19486128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8100ee0 msgr2=0x7f09f81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.941+0000 7f0a00211700 1 --2- 172.21.15.49:0/19486128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8100ee0 0x7f09f81012c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009910 tx=0x7f09f4009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:38.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.941+0000 7f0a00211700 1 -- 172.21.15.49:0/19486128 shutdown_connections 2024-05-15T16:29:38.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.941+0000 7f0a00211700 1 --2- 172.21.15.49:0/19486128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8101890 0x7f09f81058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.941+0000 7f0a00211700 1 --2- 172.21.15.49:0/19486128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8100ee0 0x7f09f81012c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.941+0000 7f0a00211700 1 -- 172.21.15.49:0/19486128 >> 172.21.15.49:0/19486128 conn(0x7f09f80fc790 msgr2=0x7f09f80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:38.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.941+0000 7f0a00211700 1 -- 172.21.15.49:0/19486128 shutdown_connections 2024-05-15T16:29:38.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.941+0000 7f0a00211700 1 -- 172.21.15.49:0/19486128 wait complete. 2024-05-15T16:29:38.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f0a00211700 1 Processor -- start 2024-05-15T16:29:38.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f0a00211700 1 -- start start 2024-05-15T16:29:38.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f0a00211700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8100ee0 0x7f09f80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:38.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f0a00211700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8101890 0x7f09f80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:38.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f0a00211700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f8073d20 con 0x7f09f8101890 2024-05-15T16:29:38.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f0a00211700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f8073e60 con 0x7f09f8100ee0 2024-05-15T16:29:38.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f09fd7ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8101890 0x7f09f80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f09fd7ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8101890 0x7f09f80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44122/0 (socket says 172.21.15.49:44122) 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f09fd7ac700 1 -- 172.21.15.49:0/2679516391 learned_addr learned my addr 172.21.15.49:0/2679516391 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.942+0000 7f09fdfad700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8100ee0 0x7f09f80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.943+0000 7f09fd7ac700 1 -- 172.21.15.49:0/2679516391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8100ee0 msgr2=0x7f09f80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.943+0000 7f09fd7ac700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8100ee0 0x7f09f80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.943+0000 7f09fd7ac700 1 -- 172.21.15.49:0/2679516391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f40095f0 con 0x7f09f8101890 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.943+0000 7f09fd7ac700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8101890 0x7f09f80737e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f09e800c8f0 tx=0x7f09e800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.943+0000 7f09eeffd700 1 -- 172.21.15.49:0/2679516391 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e80078a0 con 0x7f09f8101890 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.943+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f81a2e10 con 0x7f09f8101890 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.943+0000 7f09eeffd700 1 -- 172.21.15.49:0/2679516391 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09e800f450 con 0x7f09f8101890 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.943+0000 7f09eeffd700 1 -- 172.21.15.49:0/2679516391 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e8018610 con 0x7f09f8101890 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.943+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f81a3170 con 0x7f09f8101890 2024-05-15T16:29:38.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.944+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f8109280 con 0x7f09f8101890 2024-05-15T16:29:38.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.945+0000 7f09eeffd700 1 -- 172.21.15.49:0/2679516391 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f09e800f5c0 con 0x7f09f8101890 2024-05-15T16:29:38.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.945+0000 7f09eeffd700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09e406e210 0x7f09e40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:38.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.945+0000 7f09eeffd700 1 -- 172.21.15.49:0/2679516391 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f09e808e0e0 con 0x7f09f8101890 2024-05-15T16:29:38.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.945+0000 7f09fdfad700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09e406e210 0x7f09e40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:38.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.946+0000 7f09fdfad700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09e406e210 0x7f09e40706d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f09f4006010 tx=0x7f09f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:38.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:38.950+0000 7f09eeffd700 1 -- 172.21.15.49:0/2679516391 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f09e80573c0 con 0x7f09f8101890 2024-05-15T16:29:39.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:38 smithi144 ceph-mon[86635]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:39.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:38 smithi144 ceph-mon[86635]: from='client.35778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:39.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:38 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3446266798' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:29:39.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:38 smithi049 ceph-mon[112142]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:39.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:38 smithi049 ceph-mon[112142]: from='client.35778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:39.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:38 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3446266798' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:29:39.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.204+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f09f81023c0 con 0x7f09f8101890 2024-05-15T16:29:39.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.205+0000 7f09eeffd700 1 -- 172.21.15.49:0/2679516391 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f09e805aa80 con 0x7f09f8101890 2024-05-15T16:29:39.206 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:29:39.206 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:29:39.206 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:29:39.206 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:29:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:29:39.206 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:29:39.207 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:29:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:29:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:29:39.209 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:29:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:29:39.209 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:29:39.209 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:29:39.209 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:29:39.209 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09e406e210 msgr2=0x7f09e40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09e406e210 0x7f09e40706d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f09f4006010 tx=0x7f09f401a040 comp rx=0 tx=0).stop 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8101890 msgr2=0x7f09f80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8101890 0x7f09f80737e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f09e800c8f0 tx=0x7f09e800cc00 comp rx=0 tx=0).stop 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 shutdown_connections 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f09e406e210 0x7f09e40706d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8100ee0 0x7f09f80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 --2- 172.21.15.49:0/2679516391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f8101890 0x7f09f80737e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 >> 172.21.15.49:0/2679516391 conn(0x7f09f80fc790 msgr2=0x7f09f80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 shutdown_connections 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.207+0000 7f0a00211700 1 -- 172.21.15.49:0/2679516391 wait complete. 2024-05-15T16:29:39.210 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:29:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.317+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/4253018393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 msgr2=0x7f2e44102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.317+0000 7f2e4a2cb700 1 --2- 172.21.15.49:0/4253018393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 0x7f2e44102250 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009910 tx=0x7f2e34009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.317+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/4253018393 shutdown_connections 2024-05-15T16:29:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.317+0000 7f2e4a2cb700 1 --2- 172.21.15.49:0/4253018393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e44102790 0x7f2e4410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.317+0000 7f2e4a2cb700 1 --2- 172.21.15.49:0/4253018393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 0x7f2e44102250 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.317+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/4253018393 >> 172.21.15.49:0/4253018393 conn(0x7f2e440fb700 msgr2=0x7f2e440fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.317+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/4253018393 shutdown_connections 2024-05-15T16:29:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.317+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/4253018393 wait complete. 2024-05-15T16:29:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e4a2cb700 1 Processor -- start 2024-05-15T16:29:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e4a2cb700 1 -- start start 2024-05-15T16:29:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e4a2cb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 0x7f2e440ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e4a2cb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e44102790 0x7f2e44100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e4a2cb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44101aa0 con 0x7f2e44101e70 2024-05-15T16:29:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e4a2cb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44100550 con 0x7f2e44102790 2024-05-15T16:29:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e43fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 0x7f2e440ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e43fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 0x7f2e440ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44140/0 (socket says 172.21.15.49:44140) 2024-05-15T16:29:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e43fff700 1 -- 172.21.15.49:0/651657584 learned_addr learned my addr 172.21.15.49:0/651657584 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e43fff700 1 -- 172.21.15.49:0/651657584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e44102790 msgr2=0x7f2e44100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e437fe700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e44102790 0x7f2e44100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e43fff700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e44102790 0x7f2e44100010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.318+0000 7f2e43fff700 1 -- 172.21.15.49:0/651657584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340095f0 con 0x7f2e44101e70 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.319+0000 7f2e43fff700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 0x7f2e440ffad0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2e340092e0 tx=0x7f2e3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.319+0000 7f2e417fa700 1 -- 172.21.15.49:0/651657584 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3401d070 con 0x7f2e44101e70 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.319+0000 7f2e417fa700 1 -- 172.21.15.49:0/651657584 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e3400f460 con 0x7f2e44101e70 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.319+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e441007a0 con 0x7f2e44101e70 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.319+0000 7f2e417fa700 1 -- 172.21.15.49:0/651657584 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e34017630 con 0x7f2e44101e70 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.319+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e441a2fe0 con 0x7f2e44101e70 2024-05-15T16:29:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.320+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e44069ab0 con 0x7f2e44101e70 2024-05-15T16:29:39.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.320+0000 7f2e417fa700 1 -- 172.21.15.49:0/651657584 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2e3400f5d0 con 0x7f2e44101e70 2024-05-15T16:29:39.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.321+0000 7f2e417fa700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2e2c06e210 0x7f2e2c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.321+0000 7f2e417fa700 1 -- 172.21.15.49:0/651657584 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2e3408f5d0 con 0x7f2e44101e70 2024-05-15T16:29:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.321+0000 7f2e437fe700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2e2c06e210 0x7f2e2c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.322+0000 7f2e437fe700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2e2c06e210 0x7f2e2c0706d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2e44101200 tx=0x7f2e38005df0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:39.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.326+0000 7f2e417fa700 1 -- 172.21.15.49:0/651657584 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2e34058930 con 0x7f2e44101e70 2024-05-15T16:29:39.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.549+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e441055e0 con 0x7f2e2c06e210 2024-05-15T16:29:39.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.551+0000 7f2e417fa700 1 -- 172.21.15.49:0/651657584 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2e441055e0 con 0x7f2e2c06e210 2024-05-15T16:29:39.552 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:29:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:29:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:29:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:29:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:29:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:29:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:29:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:29:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:29:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:29:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:29:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:29:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:29:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:29:39.555 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:29:39.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2e2c06e210 msgr2=0x7f2e2c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2e2c06e210 0x7f2e2c0706d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2e44101200 tx=0x7f2e38005df0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 msgr2=0x7f2e440ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 0x7f2e440ffad0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2e340092e0 tx=0x7f2e3400be20 comp rx=0 tx=0).stop 2024-05-15T16:29:39.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 shutdown_connections 2024-05-15T16:29:39.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2e2c06e210 0x7f2e2c0706d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44101e70 0x7f2e440ffad0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 --2- 172.21.15.49:0/651657584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e44102790 0x7f2e44100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 >> 172.21.15.49:0/651657584 conn(0x7f2e440fb700 msgr2=0x7f2e44103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:39.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 shutdown_connections 2024-05-15T16:29:39.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.553+0000 7f2e4a2cb700 1 -- 172.21.15.49:0/651657584 wait complete. 2024-05-15T16:29:39.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.662+0000 7ff48a69c700 1 -- 172.21.15.49:0/2317921037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4840ffdf0 msgr2=0x7ff4841001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.662+0000 7ff48a69c700 1 --2- 172.21.15.49:0/2317921037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4840ffdf0 0x7ff4841001d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff474009910 tx=0x7ff474009c20 comp rx=0 tx=0).stop 2024-05-15T16:29:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.662+0000 7ff48a69c700 1 -- 172.21.15.49:0/2317921037 shutdown_connections 2024-05-15T16:29:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.662+0000 7ff48a69c700 1 --2- 172.21.15.49:0/2317921037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4841007a0 0x7ff4841047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.662+0000 7ff48a69c700 1 --2- 172.21.15.49:0/2317921037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4840ffdf0 0x7ff4841001d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.662+0000 7ff48a69c700 1 -- 172.21.15.49:0/2317921037 >> 172.21.15.49:0/2317921037 conn(0x7ff4840fb6a0 msgr2=0x7ff4840fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.662+0000 7ff48a69c700 1 -- 172.21.15.49:0/2317921037 shutdown_connections 2024-05-15T16:29:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.662+0000 7ff48a69c700 1 -- 172.21.15.49:0/2317921037 wait complete. 2024-05-15T16:29:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.663+0000 7ff48a69c700 1 Processor -- start 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.663+0000 7ff48a69c700 1 -- start start 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.663+0000 7ff48a69c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4840ffdf0 0x7ff484197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.663+0000 7ff48a69c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4841007a0 0x7ff484197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.663+0000 7ff48a69c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4841984f0 con 0x7ff4841007a0 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.663+0000 7ff48a69c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff484198630 con 0x7ff4840ffdf0 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.663+0000 7ff4837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4841007a0 0x7ff484197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.663+0000 7ff4837fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4841007a0 0x7ff484197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44166/0 (socket says 172.21.15.49:44166) 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.663+0000 7ff4837fe700 1 -- 172.21.15.49:0/1446055351 learned_addr learned my addr 172.21.15.49:0/1446055351 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff483fff700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4840ffdf0 0x7ff484197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff4837fe700 1 -- 172.21.15.49:0/1446055351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4840ffdf0 msgr2=0x7ff484197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff4837fe700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4840ffdf0 0x7ff484197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff4837fe700 1 -- 172.21.15.49:0/1446055351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740095f0 con 0x7ff4841007a0 2024-05-15T16:29:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff4837fe700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4841007a0 0x7ff484197fb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff47800d900 tx=0x7ff47800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff4817fa700 1 -- 172.21.15.49:0/1446055351 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff478009900 con 0x7ff4841007a0 2024-05-15T16:29:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff4817fa700 1 -- 172.21.15.49:0/1446055351 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff478010460 con 0x7ff4841007a0 2024-05-15T16:29:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4841a0ae0 con 0x7ff4841007a0 2024-05-15T16:29:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff4817fa700 1 -- 172.21.15.49:0/1446055351 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff47800f5f0 con 0x7ff4841007a0 2024-05-15T16:29:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.664+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4841a0ef0 con 0x7ff4841007a0 2024-05-15T16:29:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.666+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff484108190 con 0x7ff4841007a0 2024-05-15T16:29:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.666+0000 7ff4817fa700 1 -- 172.21.15.49:0/1446055351 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff478009a60 con 0x7ff4841007a0 2024-05-15T16:29:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.666+0000 7ff4817fa700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff46c06e200 0x7ff46c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:29:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.667+0000 7ff4817fa700 1 -- 172.21.15.49:0/1446055351 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff47808dc80 con 0x7ff4841007a0 2024-05-15T16:29:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.667+0000 7ff483fff700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff46c06e200 0x7ff46c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:29:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.667+0000 7ff483fff700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff46c06e200 0x7ff46c0706c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff474000c00 tx=0x7ff47401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:29:39.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.671+0000 7ff4817fa700 1 -- 172.21.15.49:0/1446055351 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff478056fe0 con 0x7ff4841007a0 2024-05-15T16:29:39.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.967+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff4841987f0 con 0x7ff4841007a0 2024-05-15T16:29:39.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.967+0000 7ff4817fa700 1 -- 172.21.15.49:0/1446055351 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7ff4841987f0 con 0x7ff4841007a0 2024-05-15T16:29:39.969 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:29:39.969 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:29:39.969 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:29:39.969 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:29:39.969 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:29:39.969 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:29:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:29:39.971 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:29:39.971 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:29:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:29:39.971 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:29:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:29:39.971 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:29:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:29:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff46c06e200 msgr2=0x7ff46c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff46c06e200 0x7ff46c0706c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff474000c00 tx=0x7ff47401a040 comp rx=0 tx=0).stop 2024-05-15T16:29:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4841007a0 msgr2=0x7ff484197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:29:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4841007a0 0x7ff484197fb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff47800d900 tx=0x7ff47800dc10 comp rx=0 tx=0).stop 2024-05-15T16:29:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 shutdown_connections 2024-05-15T16:29:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff46c06e200 0x7ff46c0706c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4840ffdf0 0x7ff484197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 --2- 172.21.15.49:0/1446055351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4841007a0 0x7ff484197fb0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:29:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 >> 172.21.15.49:0/1446055351 conn(0x7ff4840fb6a0 msgr2=0x7ff4840fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:29:39.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 shutdown_connections 2024-05-15T16:29:39.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:29:39.970+0000 7ff48a69c700 1 -- 172.21.15.49:0/1446055351 wait complete. 2024-05-15T16:29:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:39 smithi144 ceph-mon[86635]: from='client.35782 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:39 smithi144 ceph-mon[86635]: pgmap v980: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:40.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:39 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2679516391' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:29:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:39 smithi049 ceph-mon[112142]: from='client.35782 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:39 smithi049 ceph-mon[112142]: pgmap v980: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:39 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2679516391' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:29:41.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:40 smithi049 ceph-mon[112142]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:41.163 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:40 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1446055351' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:29:41.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:40 smithi144 ceph-mon[86635]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:29:41.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:40 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1446055351' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:29:42.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:41 smithi144 ceph-mon[86635]: pgmap v981: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:41 smithi049 ceph-mon[112142]: pgmap v981: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:43 smithi144 ceph-mon[86635]: pgmap v982: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:43 smithi049 ceph-mon[112142]: pgmap v982: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:45 smithi144 ceph-mon[86635]: pgmap v983: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:45 smithi049 ceph-mon[112142]: pgmap v983: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:47 smithi144 ceph-mon[86635]: pgmap v984: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:47 smithi049 ceph-mon[112142]: pgmap v984: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:49 smithi144 ceph-mon[86635]: pgmap v985: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:49 smithi049 ceph-mon[112142]: pgmap v985: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:51.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:29:51.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:29:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:51 smithi144 ceph-mon[86635]: pgmap v986: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:51 smithi049 ceph-mon[112142]: pgmap v986: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:53 smithi144 ceph-mon[86635]: pgmap v987: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:54.203 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:53 smithi049 ceph-mon[112142]: pgmap v987: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:55 smithi144 ceph-mon[86635]: pgmap v988: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:55 smithi049 ceph-mon[112142]: pgmap v988: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:57 smithi144 ceph-mon[86635]: pgmap v989: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:29:58.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:57 smithi049 ceph-mon[112142]: pgmap v989: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:29:59 smithi144 ceph-mon[86635]: pgmap v990: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:29:59 smithi049 ceph-mon[112142]: pgmap v990: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:00 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:29:59.999+0000 7f6ffd1f4700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:30:01.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:00 smithi144 ceph-mon[86635]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:30:01.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:00 smithi049 ceph-mon[112142]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:30:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:01 smithi144 ceph-mon[86635]: pgmap v991: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:02.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:01 smithi049 ceph-mon[112142]: pgmap v991: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:04.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:03 smithi144 ceph-mon[86635]: pgmap v992: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:03 smithi049 ceph-mon[112142]: pgmap v992: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:05 smithi144 ceph-mon[86635]: pgmap v993: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:30:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:05 smithi049 ceph-mon[112142]: pgmap v993: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:30:08.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:07 smithi144 ceph-mon[86635]: pgmap v994: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:07 smithi049 ceph-mon[112142]: pgmap v994: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:10.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.094+0000 7f529a54a700 1 -- 172.21.15.49:0/795074355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294102830 msgr2=0x7f529410ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.094+0000 7f529a54a700 1 --2- 172.21.15.49:0/795074355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294102830 0x7f529410ad20 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f527c009910 tx=0x7f527c009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:10.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.094+0000 7f529a54a700 1 -- 172.21.15.49:0/795074355 shutdown_connections 2024-05-15T16:30:10.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.094+0000 7f529a54a700 1 --2- 172.21.15.49:0/795074355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294102830 0x7f529410ad20 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.094+0000 7f529a54a700 1 --2- 172.21.15.49:0/795074355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5294101f10 0x7f52941022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.094+0000 7f529a54a700 1 -- 172.21.15.49:0/795074355 >> 172.21.15.49:0/795074355 conn(0x7f52940fb6a0 msgr2=0x7f52940fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.094+0000 7f529a54a700 1 -- 172.21.15.49:0/795074355 shutdown_connections 2024-05-15T16:30:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.094+0000 7f529a54a700 1 -- 172.21.15.49:0/795074355 wait complete. 2024-05-15T16:30:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.095+0000 7f529a54a700 1 Processor -- start 2024-05-15T16:30:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.095+0000 7f529a54a700 1 -- start start 2024-05-15T16:30:10.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f529a54a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294101f10 0x7f52940ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:10.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f529a54a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5294102830 0x7f52941000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:10.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f529a54a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52941005f0 con 0x7f5294101f10 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f529a54a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5294100730 con 0x7f5294102830 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5293fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294101f10 0x7f52940ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5293fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294101f10 0x7f52940ffb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60480/0 (socket says 172.21.15.49:60480) 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5293fff700 1 -- 172.21.15.49:0/1958434792 learned_addr learned my addr 172.21.15.49:0/1958434792 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f528bfff700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5294102830 0x7f52941000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5293fff700 1 -- 172.21.15.49:0/1958434792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5294102830 msgr2=0x7f52941000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5293fff700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5294102830 0x7f52941000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5293fff700 1 -- 172.21.15.49:0/1958434792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0095f0 con 0x7f5294101f10 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5293fff700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294101f10 0x7f52940ffb70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f528400d8d0 tx=0x7f528400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5291ffb700 1 -- 172.21.15.49:0/1958434792 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52840098a0 con 0x7f5294101f10 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5291ffb700 1 -- 172.21.15.49:0/1958434792 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5284010460 con 0x7f5294101f10 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52941a2e10 con 0x7f5294101f10 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.096+0000 7f5291ffb700 1 -- 172.21.15.49:0/1958434792 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528400f5f0 con 0x7f5294101f10 2024-05-15T16:30:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.097+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52941a3170 con 0x7f5294101f10 2024-05-15T16:30:10.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.097+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52940699f0 con 0x7f5294101f10 2024-05-15T16:30:10.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.098+0000 7f5291ffb700 1 -- 172.21.15.49:0/1958434792 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5284009a00 con 0x7f5294101f10 2024-05-15T16:30:10.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.099+0000 7f5291ffb700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f528006e210 0x7f52800706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:10.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.099+0000 7f5291ffb700 1 -- 172.21.15.49:0/1958434792 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f528408e070 con 0x7f5294101f10 2024-05-15T16:30:10.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.099+0000 7f528bfff700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f528006e210 0x7f52800706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:10.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.099+0000 7f528bfff700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f528006e210 0x7f52800706d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f527c009fd0 tx=0x7f527c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:10.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.103+0000 7f5291ffb700 1 -- 172.21.15.49:0/1958434792 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5284057350 con 0x7f5294101f10 2024-05-15T16:30:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:09 smithi144 ceph-mon[86635]: pgmap v995: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:09 smithi049 ceph-mon[112142]: pgmap v995: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.332+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f529410deb0 con 0x7f528006e210 2024-05-15T16:30:10.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.333+0000 7f5291ffb700 1 -- 172.21.15.49:0/1958434792 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f529410deb0 con 0x7f528006e210 2024-05-15T16:30:10.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.335+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f528006e210 msgr2=0x7f52800706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.335+0000 7f529a54a700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f528006e210 0x7f52800706d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f527c009fd0 tx=0x7f527c01a040 comp rx=0 tx=0).stop 2024-05-15T16:30:10.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.335+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294101f10 msgr2=0x7f52940ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.335+0000 7f529a54a700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294101f10 0x7f52940ffb70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f528400d8d0 tx=0x7f528400dbe0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.335+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 shutdown_connections 2024-05-15T16:30:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.335+0000 7f529a54a700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f528006e210 0x7f52800706d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.335+0000 7f529a54a700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5294101f10 0x7f52940ffb70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.335+0000 7f529a54a700 1 --2- 172.21.15.49:0/1958434792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5294102830 0x7f52941000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.335+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 >> 172.21.15.49:0/1958434792 conn(0x7f52940fb6a0 msgr2=0x7f5294103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.336+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 shutdown_connections 2024-05-15T16:30:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.336+0000 7f529a54a700 1 -- 172.21.15.49:0/1958434792 wait complete. 2024-05-15T16:30:10.347 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:30:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.447+0000 7fc68decc700 1 -- 172.21.15.49:0/1182711656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688101e70 msgr2=0x7fc688102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.447+0000 7fc68decc700 1 --2- 172.21.15.49:0/1182711656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688101e70 0x7fc688102250 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc678009910 tx=0x7fc678009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.447+0000 7fc68decc700 1 -- 172.21.15.49:0/1182711656 shutdown_connections 2024-05-15T16:30:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.447+0000 7fc68decc700 1 --2- 172.21.15.49:0/1182711656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc688102790 0x7fc68810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.447+0000 7fc68decc700 1 --2- 172.21.15.49:0/1182711656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688101e70 0x7fc688102250 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.447+0000 7fc68decc700 1 -- 172.21.15.49:0/1182711656 >> 172.21.15.49:0/1182711656 conn(0x7fc6880fb700 msgr2=0x7fc6880fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.447+0000 7fc68decc700 1 -- 172.21.15.49:0/1182711656 shutdown_connections 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.447+0000 7fc68decc700 1 -- 172.21.15.49:0/1182711656 wait complete. 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc68decc700 1 Processor -- start 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc68decc700 1 -- start start 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc68decc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc688101e70 0x7fc68819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc68decc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688102790 0x7fc68819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc68decc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68819d4b0 con 0x7fc688102790 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc68decc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc688196a90 con 0x7fc688101e70 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc686ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688102790 0x7fc68819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc686ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688102790 0x7fc68819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60506/0 (socket says 172.21.15.49:60506) 2024-05-15T16:30:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc686ffd700 1 -- 172.21.15.49:0/1431623497 learned_addr learned my addr 172.21.15.49:0/1431623497 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc6877fe700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc688101e70 0x7fc68819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc686ffd700 1 -- 172.21.15.49:0/1431623497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc688101e70 msgr2=0x7fc68819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc686ffd700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc688101e70 0x7fc68819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.448+0000 7fc686ffd700 1 -- 172.21.15.49:0/1431623497 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780095f0 con 0x7fc688102790 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.449+0000 7fc686ffd700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688102790 0x7fc68819ce90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00d6d0 tx=0x7fc67c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.449+0000 7fc684ff9700 1 -- 172.21.15.49:0/1431623497 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc67c00f860 con 0x7fc688102790 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.449+0000 7fc684ff9700 1 -- 172.21.15.49:0/1431623497 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc67c00fe80 con 0x7fc688102790 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.449+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc688196d70 con 0x7fc688102790 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.449+0000 7fc684ff9700 1 -- 172.21.15.49:0/1431623497 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc67c010ea0 con 0x7fc688102790 2024-05-15T16:30:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.449+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6881972c0 con 0x7fc688102790 2024-05-15T16:30:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.450+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc688105600 con 0x7fc688102790 2024-05-15T16:30:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.450+0000 7fc684ff9700 1 -- 172.21.15.49:0/1431623497 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc67c010460 con 0x7fc688102790 2024-05-15T16:30:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.451+0000 7fc684ff9700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc67006e210 0x7fc6700706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.451+0000 7fc684ff9700 1 -- 172.21.15.49:0/1431623497 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc67c08dd90 con 0x7fc688102790 2024-05-15T16:30:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.451+0000 7fc6877fe700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc67006e210 0x7fc6700706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.451+0000 7fc6877fe700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc67006e210 0x7fc6700706d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc678006010 tx=0x7fc67800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:10.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.456+0000 7fc684ff9700 1 -- 172.21.15.49:0/1431623497 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc67c05aba0 con 0x7fc688102790 2024-05-15T16:30:10.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.682+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc688197fc0 con 0x7fc67006e210 2024-05-15T16:30:10.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.683+0000 7fc684ff9700 1 -- 172.21.15.49:0/1431623497 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc688197fc0 con 0x7fc67006e210 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.685+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc67006e210 msgr2=0x7fc6700706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.685+0000 7fc68decc700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc67006e210 0x7fc6700706d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc678006010 tx=0x7fc67800b540 comp rx=0 tx=0).stop 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.685+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688102790 msgr2=0x7fc68819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.685+0000 7fc68decc700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688102790 0x7fc68819ce90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00d6d0 tx=0x7fc67c00da90 comp rx=0 tx=0).stop 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.685+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 shutdown_connections 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.685+0000 7fc68decc700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc67006e210 0x7fc6700706d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.685+0000 7fc68decc700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc688101e70 0x7fc68819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.685+0000 7fc68decc700 1 --2- 172.21.15.49:0/1431623497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc688102790 0x7fc68819ce90 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.685+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 >> 172.21.15.49:0/1431623497 conn(0x7fc6880fb700 msgr2=0x7fc688103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.686+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 shutdown_connections 2024-05-15T16:30:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.686+0000 7fc68decc700 1 -- 172.21.15.49:0/1431623497 wait complete. 2024-05-15T16:30:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.793+0000 7f241447f700 1 -- 172.21.15.49:0/1137400167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c101890 msgr2=0x7f240c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.793+0000 7f241447f700 1 --2- 172.21.15.49:0/1137400167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c101890 0x7f240c1058e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2400009910 tx=0x7f2400009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.793+0000 7f241447f700 1 -- 172.21.15.49:0/1137400167 shutdown_connections 2024-05-15T16:30:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.793+0000 7f241447f700 1 --2- 172.21.15.49:0/1137400167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c101890 0x7f240c1058e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.793+0000 7f241447f700 1 --2- 172.21.15.49:0/1137400167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f240c100ee0 0x7f240c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.793+0000 7f241447f700 1 -- 172.21.15.49:0/1137400167 >> 172.21.15.49:0/1137400167 conn(0x7f240c0fc790 msgr2=0x7f240c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.793+0000 7f241447f700 1 -- 172.21.15.49:0/1137400167 shutdown_connections 2024-05-15T16:30:10.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.793+0000 7f241447f700 1 -- 172.21.15.49:0/1137400167 wait complete. 2024-05-15T16:30:10.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.794+0000 7f241447f700 1 Processor -- start 2024-05-15T16:30:10.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.794+0000 7f241447f700 1 -- start start 2024-05-15T16:30:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.794+0000 7f241447f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c100ee0 0x7f240c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.794+0000 7f241447f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f240c101890 0x7f240c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:10.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.794+0000 7f241447f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19d440 con 0x7f240c100ee0 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.794+0000 7f241447f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c196a20 con 0x7f240c101890 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.794+0000 7f241221b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c100ee0 0x7f240c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.794+0000 7f241221b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c100ee0 0x7f240c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60518/0 (socket says 172.21.15.49:60518) 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.794+0000 7f241221b700 1 -- 172.21.15.49:0/1874093927 learned_addr learned my addr 172.21.15.49:0/1874093927 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f2411a1a700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f240c101890 0x7f240c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f241221b700 1 -- 172.21.15.49:0/1874093927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f240c101890 msgr2=0x7f240c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f241221b700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f240c101890 0x7f240c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f241221b700 1 -- 172.21.15.49:0/1874093927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24000095f0 con 0x7f240c100ee0 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f2411a1a700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f240c101890 0x7f240c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f241221b700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c100ee0 0x7f240c19c8e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f240800d8d0 tx=0x7f240800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f23ff7fe700 1 -- 172.21.15.49:0/1874093927 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2408009940 con 0x7f240c100ee0 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f23ff7fe700 1 -- 172.21.15.49:0/1874093927 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2408010460 con 0x7f240c100ee0 2024-05-15T16:30:10.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c196d00 con 0x7f240c100ee0 2024-05-15T16:30:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f23ff7fe700 1 -- 172.21.15.49:0/1874093927 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f240800f5f0 con 0x7f240c100ee0 2024-05-15T16:30:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.795+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c197250 con 0x7f240c100ee0 2024-05-15T16:30:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.796+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c1092e0 con 0x7f240c100ee0 2024-05-15T16:30:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.797+0000 7f23ff7fe700 1 -- 172.21.15.49:0/1874093927 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f24080105d0 con 0x7f240c100ee0 2024-05-15T16:30:10.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.797+0000 7f23ff7fe700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f23f806e200 0x7f23f80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:10.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.797+0000 7f23ff7fe700 1 -- 172.21.15.49:0/1874093927 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f240808e1a0 con 0x7f240c100ee0 2024-05-15T16:30:10.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.797+0000 7f2411a1a700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f23f806e200 0x7f23f80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:10.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.798+0000 7f2411a1a700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f23f806e200 0x7f23f80706c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f240c197be0 tx=0x7f240001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:10.802+0000 7f23ff7fe700 1 -- 172.21.15.49:0/1874093927 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f240805afb0 con 0x7f240c100ee0 2024-05-15T16:30:11.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.020+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f240c197f50 con 0x7f23f806e200 2024-05-15T16:30:11.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.031+0000 7f23ff7fe700 1 -- 172.21.15.49:0/1874093927 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f240c197f50 con 0x7f23f806e200 2024-05-15T16:30:11.032 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:30:11.032 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (33m) 11m ago 44m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:30:11.032 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (45m) 11m ago 45m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:30:11.032 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (44m) 11m ago 44m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (30m) 11m ago 45m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (30m) 11m ago 44m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (32m) 11m ago 44m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (27m) 11m ago 39m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (26m) 11m ago 39m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (26m) 11m ago 39m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (11m) 11m ago 39m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (35m) 11m ago 46m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (34m) 11m ago 43m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (31m) 11m ago 46m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (31m) 11m ago 43m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (34m) 11m ago 45m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (34m) 11m ago 43m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (30m) 11m ago 43m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:30:11.033 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (29m) 11m ago 42m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:30:11.034 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (29m) 11m ago 42m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:30:11.034 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (28m) 11m ago 41m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:30:11.034 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (27m) 11m ago 41m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:30:11.034 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (27m) 11m ago 40m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:30:11.034 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (33m) 11m ago 44m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:30:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f23f806e200 msgr2=0x7f23f80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f23f806e200 0x7f23f80706c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f240c197be0 tx=0x7f240001a040 comp rx=0 tx=0).stop 2024-05-15T16:30:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c100ee0 msgr2=0x7f240c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c100ee0 0x7f240c19c8e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f240800d8d0 tx=0x7f240800dc90 comp rx=0 tx=0).stop 2024-05-15T16:30:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 shutdown_connections 2024-05-15T16:30:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f23f806e200 0x7f23f80706c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f240c100ee0 0x7f240c19c8e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 --2- 172.21.15.49:0/1874093927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f240c101890 0x7f240c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 >> 172.21.15.49:0/1874093927 conn(0x7f240c0fc790 msgr2=0x7f240c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:11.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 shutdown_connections 2024-05-15T16:30:11.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.034+0000 7f241447f700 1 -- 172.21.15.49:0/1874093927 wait complete. 2024-05-15T16:30:11.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.142+0000 7f04714b9700 1 -- 172.21.15.49:0/3896082611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 msgr2=0x7f046c102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.142+0000 7f04714b9700 1 --2- 172.21.15.49:0/3896082611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 0x7f046c102200 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f045c00b0a0 tx=0x7f045c00b3b0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.142+0000 7f04714b9700 1 -- 172.21.15.49:0/3896082611 shutdown_connections 2024-05-15T16:30:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.142+0000 7f04714b9700 1 --2- 172.21.15.49:0/3896082611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c102740 0x7f046c10ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.142+0000 7f04714b9700 1 --2- 172.21.15.49:0/3896082611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 0x7f046c102200 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.142+0000 7f04714b9700 1 -- 172.21.15.49:0/3896082611 >> 172.21.15.49:0/3896082611 conn(0x7f046c076800 msgr2=0x7f046c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.142+0000 7f04714b9700 1 -- 172.21.15.49:0/3896082611 shutdown_connections 2024-05-15T16:30:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.142+0000 7f04714b9700 1 -- 172.21.15.49:0/3896082611 wait complete. 2024-05-15T16:30:11.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f04714b9700 1 Processor -- start 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f04714b9700 1 -- start start 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f04714b9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 0x7f046c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f04714b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c102740 0x7f046c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f04714b9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c19d500 con 0x7f046c101e20 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f04714b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c196ae0 con 0x7f046c102740 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f046affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 0x7f046c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f046affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 0x7f046c19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60544/0 (socket says 172.21.15.49:60544) 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f046affd700 1 -- 172.21.15.49:0/3303428426 learned_addr learned my addr 172.21.15.49:0/3303428426 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.143+0000 7f046a7fc700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c102740 0x7f046c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:11.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.144+0000 7f046affd700 1 -- 172.21.15.49:0/3303428426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c102740 msgr2=0x7f046c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.144+0000 7f046affd700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c102740 0x7f046c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.144+0000 7f046affd700 1 -- 172.21.15.49:0/3303428426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c009d00 con 0x7f046c101e20 2024-05-15T16:30:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.144+0000 7f046affd700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 0x7f046c19c9a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f045c015040 tx=0x7f045c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.144+0000 7f0463fff700 1 -- 172.21.15.49:0/3303428426 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f045c00e040 con 0x7f046c101e20 2024-05-15T16:30:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.144+0000 7f0463fff700 1 -- 172.21.15.49:0/3303428426 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f045c007cd0 con 0x7f046c101e20 2024-05-15T16:30:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.144+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046c196d60 con 0x7f046c101e20 2024-05-15T16:30:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.144+0000 7f0463fff700 1 -- 172.21.15.49:0/3303428426 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f045c01d6b0 con 0x7f046c101e20 2024-05-15T16:30:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.144+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046c197250 con 0x7f046c101e20 2024-05-15T16:30:11.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.145+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046c1055b0 con 0x7f046c101e20 2024-05-15T16:30:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.146+0000 7f0463fff700 1 -- 172.21.15.49:0/3303428426 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f045c004040 con 0x7f046c101e20 2024-05-15T16:30:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.146+0000 7f0463fff700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.146+0000 7f0463fff700 1 -- 172.21.15.49:0/3303428426 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f045c08f7f0 con 0x7f046c101e20 2024-05-15T16:30:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.146+0000 7f046a7fc700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:11.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.147+0000 7f046a7fc700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f046c197ca0 tx=0x7f04540090e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:11.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.151+0000 7f0463fff700 1 -- 172.21.15.49:0/3303428426 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f045c05c680 con 0x7f046c101e20 2024-05-15T16:30:11.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.443+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f046c068f60 con 0x7f046c101e20 2024-05-15T16:30:11.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.444+0000 7f0463fff700 1 -- 172.21.15.49:0/3303428426 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f045c05c210 con 0x7f046c101e20 2024-05-15T16:30:11.445 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:30:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:30:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:30:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:30:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:30:11.446 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:30:11.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 msgr2=0x7f04580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f046c197ca0 tx=0x7f04540090e0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 msgr2=0x7f046c19c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 0x7f046c19c9a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f045c015040 tx=0x7f045c003c30 comp rx=0 tx=0).stop 2024-05-15T16:30:11.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 shutdown_connections 2024-05-15T16:30:11.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f045806e200 0x7f04580706c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f046c101e20 0x7f046c19c9a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 --2- 172.21.15.49:0/3303428426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f046c102740 0x7f046c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 >> 172.21.15.49:0/3303428426 conn(0x7f046c076800 msgr2=0x7f046c1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:11.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 shutdown_connections 2024-05-15T16:30:11.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.446+0000 7f04714b9700 1 -- 172.21.15.49:0/3303428426 wait complete. 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.555+0000 7f4916d5c700 1 -- 172.21.15.49:0/72862719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 msgr2=0x7f49101058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.555+0000 7f4916d5c700 1 --2- 172.21.15.49:0/72862719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 0x7f49101058e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009910 tx=0x7f48fc009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.555+0000 7f4916d5c700 1 -- 172.21.15.49:0/72862719 shutdown_connections 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.555+0000 7f4916d5c700 1 --2- 172.21.15.49:0/72862719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 0x7f49101058e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.555+0000 7f4916d5c700 1 --2- 172.21.15.49:0/72862719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4910100ee0 0x7f49101012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.555+0000 7f4916d5c700 1 -- 172.21.15.49:0/72862719 >> 172.21.15.49:0/72862719 conn(0x7f49100fc790 msgr2=0x7f49100febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.555+0000 7f4916d5c700 1 -- 172.21.15.49:0/72862719 shutdown_connections 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.555+0000 7f4916d5c700 1 -- 172.21.15.49:0/72862719 wait complete. 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f4916d5c700 1 Processor -- start 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f4916d5c700 1 -- start start 2024-05-15T16:30:11.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f4916d5c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4910100ee0 0x7f491019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f4916d5c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 0x7f491019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f4916d5c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491019d440 con 0x7f4910101890 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f4916d5c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4910196a20 con 0x7f4910100ee0 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f490ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 0x7f491019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f490ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 0x7f491019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60550/0 (socket says 172.21.15.49:60550) 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f490ffff700 1 -- 172.21.15.49:0/3968835381 learned_addr learned my addr 172.21.15.49:0/3968835381 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f4914af8700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4910100ee0 0x7f491019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f490ffff700 1 -- 172.21.15.49:0/3968835381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4910100ee0 msgr2=0x7f491019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f490ffff700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4910100ee0 0x7f491019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f490ffff700 1 -- 172.21.15.49:0/3968835381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0095f0 con 0x7f4910101890 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.556+0000 7f4914af8700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4910100ee0 0x7f491019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.557+0000 7f490ffff700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 0x7f491019ce20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00bd70 tx=0x7f48fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:11.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.557+0000 7f490dffb700 1 -- 172.21.15.49:0/3968835381 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48fc01d070 con 0x7f4910101890 2024-05-15T16:30:11.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.557+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4910196ca0 con 0x7f4910101890 2024-05-15T16:30:11.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.557+0000 7f490dffb700 1 -- 172.21.15.49:0/3968835381 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48fc00f460 con 0x7f4910101890 2024-05-15T16:30:11.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.557+0000 7f490dffb700 1 -- 172.21.15.49:0/3968835381 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48fc017630 con 0x7f4910101890 2024-05-15T16:30:11.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.557+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4910197190 con 0x7f4910101890 2024-05-15T16:30:11.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.558+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49101092e0 con 0x7f4910101890 2024-05-15T16:30:11.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.559+0000 7f490dffb700 1 -- 172.21.15.49:0/3968835381 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f48fc00f5d0 con 0x7f4910101890 2024-05-15T16:30:11.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.559+0000 7f490dffb700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f490006e200 0x7f49000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:11.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.559+0000 7f490dffb700 1 -- 172.21.15.49:0/3968835381 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f48fc08f640 con 0x7f4910101890 2024-05-15T16:30:11.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.559+0000 7f4914af8700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f490006e200 0x7f49000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:11.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.560+0000 7f4914af8700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f490006e200 0x7f49000706c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4904005950 tx=0x7f4904005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:11.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.564+0000 7f490dffb700 1 -- 172.21.15.49:0/3968835381 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f48fc05c450 con 0x7f4910101890 2024-05-15T16:30:11.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.821+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f49101023c0 con 0x7f4910101890 2024-05-15T16:30:11.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.822+0000 7f490dffb700 1 -- 172.21.15.49:0/3968835381 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f48fc005740 con 0x7f4910101890 2024-05-15T16:30:11.823 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:30:11.823 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:30:11.823 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:30:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:30:11.825 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:30:11.826 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:30:11.826 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:30:11.826 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:30:11.826 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:30:11.826 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:30:11.826 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:30:11.826 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:30:11.826 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:30:11.826 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:30:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f490006e200 msgr2=0x7f49000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f490006e200 0x7f49000706c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4904005950 tx=0x7f4904005f50 comp rx=0 tx=0).stop 2024-05-15T16:30:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 msgr2=0x7f491019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 0x7f491019ce20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00bd70 tx=0x7f48fc00be50 comp rx=0 tx=0).stop 2024-05-15T16:30:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 shutdown_connections 2024-05-15T16:30:11.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f490006e200 0x7f49000706c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4910100ee0 0x7f491019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 --2- 172.21.15.49:0/3968835381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4910101890 0x7f491019ce20 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 >> 172.21.15.49:0/3968835381 conn(0x7f49100fc790 msgr2=0x7f49100fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:11.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.824+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 shutdown_connections 2024-05-15T16:30:11.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.825+0000 7f4916d5c700 1 -- 172.21.15.49:0/3968835381 wait complete. 2024-05-15T16:30:11.828 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:30:11.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 -- 172.21.15.49:0/534710664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440ff560 msgr2=0x7f5a440ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 --2- 172.21.15.49:0/534710664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440ff560 0x7f5a440ff940 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009910 tx=0x7f5a34009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:11.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 -- 172.21.15.49:0/534710664 shutdown_connections 2024-05-15T16:30:11.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 --2- 172.21.15.49:0/534710664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440fff10 0x7f5a44103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 --2- 172.21.15.49:0/534710664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440ff560 0x7f5a440ff940 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 -- 172.21.15.49:0/534710664 >> 172.21.15.49:0/534710664 conn(0x7f5a44075ff0 msgr2=0x7f5a440fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:11.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 -- 172.21.15.49:0/534710664 shutdown_connections 2024-05-15T16:30:11.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 -- 172.21.15.49:0/534710664 wait complete. 2024-05-15T16:30:11.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 Processor -- start 2024-05-15T16:30:11.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.934+0000 7f5a4a47b700 1 -- start start 2024-05-15T16:30:11.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a4a47b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440ff560 0x7f5a4419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:11.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a4a47b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440fff10 0x7f5a4419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:11.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a4a47b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4419d510 con 0x7f5a440fff10 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a4a47b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a44196af0 con 0x7f5a440ff560 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440fff10 0x7f5a4419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440fff10 0x7f5a4419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60570/0 (socket says 172.21.15.49:60570) 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a437fe700 1 -- 172.21.15.49:0/1615750069 learned_addr learned my addr 172.21.15.49:0/1615750069 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a43fff700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440ff560 0x7f5a4419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a437fe700 1 -- 172.21.15.49:0/1615750069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440ff560 msgr2=0x7f5a4419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a437fe700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440ff560 0x7f5a4419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a437fe700 1 -- 172.21.15.49:0/1615750069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a340095f0 con 0x7f5a440fff10 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.935+0000 7f5a437fe700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440fff10 0x7f5a4419cef0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00b480 tx=0x7f5a2c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.936+0000 7f5a417fa700 1 -- 172.21.15.49:0/1615750069 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a2c011800 con 0x7f5a440fff10 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.936+0000 7f5a417fa700 1 -- 172.21.15.49:0/1615750069 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a2c011e20 con 0x7f5a440fff10 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.936+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a44196dd0 con 0x7f5a440fff10 2024-05-15T16:30:11.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.936+0000 7f5a417fa700 1 -- 172.21.15.49:0/1615750069 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a2c010ea0 con 0x7f5a440fff10 2024-05-15T16:30:11.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.936+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a44197320 con 0x7f5a440fff10 2024-05-15T16:30:11.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.937+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a44100a40 con 0x7f5a440fff10 2024-05-15T16:30:11.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.937+0000 7f5a417fa700 1 -- 172.21.15.49:0/1615750069 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5a2c010380 con 0x7f5a440fff10 2024-05-15T16:30:11.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.938+0000 7f5a417fa700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a3006e210 0x7f5a300706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:11.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.938+0000 7f5a417fa700 1 -- 172.21.15.49:0/1615750069 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5a2c08dbf0 con 0x7f5a440fff10 2024-05-15T16:30:11.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.938+0000 7f5a43fff700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a3006e210 0x7f5a300706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:11.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.938+0000 7f5a43fff700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a3006e210 0x7f5a300706d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5a34006010 tx=0x7f5a3401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:11.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:11.943+0000 7f5a417fa700 1 -- 172.21.15.49:0/1615750069 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5a2c05aa80 con 0x7f5a440fff10 2024-05-15T16:30:12.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.167+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a4410c6b0 con 0x7f5a3006e210 2024-05-15T16:30:12.169 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:11 smithi049 ceph-mon[112142]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:12.169 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:11 smithi049 ceph-mon[112142]: from='client.35806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:12.169 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:11 smithi049 ceph-mon[112142]: pgmap v996: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:12.169 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:11 smithi049 ceph-mon[112142]: from='client.35810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:12.169 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:11 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3303428426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:30:12.169 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:11 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3968835381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:30:12.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.169+0000 7f5a417fa700 1 -- 172.21.15.49:0/1615750069 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a4410c6b0 con 0x7f5a3006e210 2024-05-15T16:30:12.170 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:30:12.170 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:30:12.170 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:30:12.170 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:30:12.171 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:30:12.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a3006e210 msgr2=0x7f5a300706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:12.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a3006e210 0x7f5a300706d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5a34006010 tx=0x7f5a3401a040 comp rx=0 tx=0).stop 2024-05-15T16:30:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440fff10 msgr2=0x7f5a4419cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440fff10 0x7f5a4419cef0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00b480 tx=0x7f5a2c00b790 comp rx=0 tx=0).stop 2024-05-15T16:30:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 shutdown_connections 2024-05-15T16:30:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5a3006e210 0x7f5a300706d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440ff560 0x7f5a4419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 --2- 172.21.15.49:0/1615750069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a440fff10 0x7f5a4419cef0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 >> 172.21.15.49:0/1615750069 conn(0x7f5a44075ff0 msgr2=0x7f5a440fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 shutdown_connections 2024-05-15T16:30:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.171+0000 7f5a4a47b700 1 -- 172.21.15.49:0/1615750069 wait complete. 2024-05-15T16:30:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:11 smithi144 ceph-mon[86635]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:11 smithi144 ceph-mon[86635]: from='client.35806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:11 smithi144 ceph-mon[86635]: pgmap v996: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:11 smithi144 ceph-mon[86635]: from='client.35810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:11 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3303428426' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:30:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:11 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3968835381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:30:12.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.280+0000 7fd71a743700 1 -- 172.21.15.49:0/872912855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 msgr2=0x7fd7141047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:12.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.280+0000 7fd71a743700 1 --2- 172.21.15.49:0/872912855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 0x7fd7141047c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009910 tx=0x7fd6fc009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:12.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.280+0000 7fd71a743700 1 -- 172.21.15.49:0/872912855 shutdown_connections 2024-05-15T16:30:12.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.280+0000 7fd71a743700 1 --2- 172.21.15.49:0/872912855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 0x7fd7141047c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:12.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.280+0000 7fd71a743700 1 --2- 172.21.15.49:0/872912855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7140ffdc0 0x7fd7141001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:12.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.280+0000 7fd71a743700 1 -- 172.21.15.49:0/872912855 >> 172.21.15.49:0/872912855 conn(0x7fd714076800 msgr2=0x7fd7140fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:12.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.280+0000 7fd71a743700 1 -- 172.21.15.49:0/872912855 shutdown_connections 2024-05-15T16:30:12.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.280+0000 7fd71a743700 1 -- 172.21.15.49:0/872912855 wait complete. 2024-05-15T16:30:12.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.281+0000 7fd71a743700 1 Processor -- start 2024-05-15T16:30:12.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.281+0000 7fd71a743700 1 -- start start 2024-05-15T16:30:12.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.281+0000 7fd71a743700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7140ffdc0 0x7fd71419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:12.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.281+0000 7fd71a743700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 0x7fd71419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:12.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.281+0000 7fd71a743700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419d470 con 0x7fd714100770 2024-05-15T16:30:12.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.281+0000 7fd71a743700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7141969e0 con 0x7fd7140ffdc0 2024-05-15T16:30:12.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd70bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 0x7fd71419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:12.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd70bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 0x7fd71419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60588/0 (socket says 172.21.15.49:60588) 2024-05-15T16:30:12.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd70bfff700 1 -- 172.21.15.49:0/1109882661 learned_addr learned my addr 172.21.15.49:0/1109882661 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd713fff700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7140ffdc0 0x7fd71419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd70bfff700 1 -- 172.21.15.49:0/1109882661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7140ffdc0 msgr2=0x7fd71419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd70bfff700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7140ffdc0 0x7fd71419c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd70bfff700 1 -- 172.21.15.49:0/1109882661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0095f0 con 0x7fd714100770 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd70bfff700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 0x7fd71419cde0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc000c00 tx=0x7fd6fc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd711ffb700 1 -- 172.21.15.49:0/1109882661 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6fc01d070 con 0x7fd714100770 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd711ffb700 1 -- 172.21.15.49:0/1109882661 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6fc005700 con 0x7fd714100770 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd714196c60 con 0x7fd714100770 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd711ffb700 1 -- 172.21.15.49:0/1109882661 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6fc00fda0 con 0x7fd714100770 2024-05-15T16:30:12.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.282+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd714197150 con 0x7fd714100770 2024-05-15T16:30:12.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.283+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7141012a0 con 0x7fd714100770 2024-05-15T16:30:12.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.284+0000 7fd711ffb700 1 -- 172.21.15.49:0/1109882661 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd6fc00f5d0 con 0x7fd714100770 2024-05-15T16:30:12.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.285+0000 7fd711ffb700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd70006e210 0x7fd7000706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:12.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.285+0000 7fd711ffb700 1 -- 172.21.15.49:0/1109882661 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd6fc08f540 con 0x7fd714100770 2024-05-15T16:30:12.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.285+0000 7fd713fff700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd70006e210 0x7fd7000706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:12.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.285+0000 7fd713fff700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd70006e210 0x7fd7000706d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd704005950 tx=0x7fd70400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:12.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.289+0000 7fd711ffb700 1 -- 172.21.15.49:0/1109882661 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd6fc05c350 con 0x7fd714100770 2024-05-15T16:30:12.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.580+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd71404ec60 con 0x7fd714100770 2024-05-15T16:30:12.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.580+0000 7fd711ffb700 1 -- 172.21.15.49:0/1109882661 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fd6fc05bee0 con 0x7fd714100770 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:30:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:30:12.583 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:30:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:30:12.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd70006e210 msgr2=0x7fd7000706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:12.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd70006e210 0x7fd7000706d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd704005950 tx=0x7fd70400b410 comp rx=0 tx=0).stop 2024-05-15T16:30:12.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 msgr2=0x7fd71419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:12.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 0x7fd71419cde0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc000c00 tx=0x7fd6fc00bd70 comp rx=0 tx=0).stop 2024-05-15T16:30:12.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 shutdown_connections 2024-05-15T16:30:12.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd70006e210 0x7fd7000706d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:12.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7140ffdc0 0x7fd71419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:12.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 --2- 172.21.15.49:0/1109882661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd714100770 0x7fd71419cde0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:12.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 >> 172.21.15.49:0/1109882661 conn(0x7fd714076800 msgr2=0x7fd714068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:12.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 shutdown_connections 2024-05-15T16:30:12.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:12.583+0000 7fd71a743700 1 -- 172.21.15.49:0/1109882661 wait complete. 2024-05-15T16:30:13.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:12 smithi049 ceph-mon[112142]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:13.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:12 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1109882661' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:30:13.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:12 smithi144 ceph-mon[86635]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:13.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:12 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1109882661' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:30:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:13 smithi049 ceph-mon[112142]: pgmap v997: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:13 smithi144 ceph-mon[86635]: pgmap v997: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:15 smithi144 ceph-mon[86635]: pgmap v998: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:15 smithi049 ceph-mon[112142]: pgmap v998: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:17 smithi144 ceph-mon[86635]: pgmap v999: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:17 smithi049 ceph-mon[112142]: pgmap v999: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:19 smithi144 ceph-mon[86635]: pgmap v1000: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:19 smithi049 ceph-mon[112142]: pgmap v1000: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:30:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:30:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:21 smithi144 ceph-mon[86635]: pgmap v1001: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:21 smithi049 ceph-mon[112142]: pgmap v1001: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:23 smithi144 ceph-mon[86635]: pgmap v1002: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:23 smithi049 ceph-mon[112142]: pgmap v1002: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:25 smithi144 ceph-mon[86635]: pgmap v1003: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:25 smithi049 ceph-mon[112142]: pgmap v1003: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:27 smithi144 ceph-mon[86635]: pgmap v1004: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:27 smithi049 ceph-mon[112142]: pgmap v1004: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:29 smithi144 ceph-mon[86635]: pgmap v1005: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:29 smithi049 ceph-mon[112142]: pgmap v1005: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:31 smithi144 ceph-mon[86635]: pgmap v1006: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:31 smithi049 ceph-mon[112142]: pgmap v1006: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:33 smithi144 ceph-mon[86635]: pgmap v1007: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:33 smithi049 ceph-mon[112142]: pgmap v1007: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:35 smithi144 ceph-mon[86635]: pgmap v1008: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:30:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:35 smithi049 ceph-mon[112142]: pgmap v1008: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:30:38.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:37 smithi144 ceph-mon[86635]: pgmap v1009: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:37 smithi049 ceph-mon[112142]: pgmap v1009: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:40.306 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:39 smithi049 ceph-mon[112142]: pgmap v1010: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:39 smithi144 ceph-mon[86635]: pgmap v1010: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:42.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:41 smithi144 ceph-mon[86635]: pgmap v1011: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:41 smithi049 ceph-mon[112142]: pgmap v1011: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.706+0000 7fad3cd46700 1 -- 172.21.15.49:0/1339827871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38101e70 msgr2=0x7fad38102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.706+0000 7fad3cd46700 1 --2- 172.21.15.49:0/1339827871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38101e70 0x7fad38102250 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fad28009910 tx=0x7fad28009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.706+0000 7fad3cd46700 1 -- 172.21.15.49:0/1339827871 shutdown_connections 2024-05-15T16:30:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.706+0000 7fad3cd46700 1 --2- 172.21.15.49:0/1339827871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad38102790 0x7fad3810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.706+0000 7fad3cd46700 1 --2- 172.21.15.49:0/1339827871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38101e70 0x7fad38102250 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.706+0000 7fad3cd46700 1 -- 172.21.15.49:0/1339827871 >> 172.21.15.49:0/1339827871 conn(0x7fad380fb700 msgr2=0x7fad380fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.706+0000 7fad3cd46700 1 -- 172.21.15.49:0/1339827871 shutdown_connections 2024-05-15T16:30:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.706+0000 7fad3cd46700 1 -- 172.21.15.49:0/1339827871 wait complete. 2024-05-15T16:30:42.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.707+0000 7fad3cd46700 1 Processor -- start 2024-05-15T16:30:42.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.707+0000 7fad3cd46700 1 -- start start 2024-05-15T16:30:42.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad3cd46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad38101e70 0x7fad3819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:42.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad3cd46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38102790 0x7fad3819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad3cd46700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3819d4c0 con 0x7fad38102790 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad3cd46700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad38196aa0 con 0x7fad38101e70 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad35d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38102790 0x7fad3819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad35d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38102790 0x7fad3819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39460/0 (socket says 172.21.15.49:39460) 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad35d9b700 1 -- 172.21.15.49:0/3380925050 learned_addr learned my addr 172.21.15.49:0/3380925050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad3659c700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad38101e70 0x7fad3819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad35d9b700 1 -- 172.21.15.49:0/3380925050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad38101e70 msgr2=0x7fad3819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad35d9b700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad38101e70 0x7fad3819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad35d9b700 1 -- 172.21.15.49:0/3380925050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280095f0 con 0x7fad38102790 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad35d9b700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38102790 0x7fad3819cea0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00d6a0 tx=0x7fad2c00d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.708+0000 7fad277fe700 1 -- 172.21.15.49:0/3380925050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad2c004a00 con 0x7fad38102790 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.709+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad38196d80 con 0x7fad38102790 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.709+0000 7fad277fe700 1 -- 172.21.15.49:0/3380925050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad2c0054e0 con 0x7fad38102790 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.709+0000 7fad277fe700 1 -- 172.21.15.49:0/3380925050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad2c00fca0 con 0x7fad38102790 2024-05-15T16:30:42.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.709+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad381972d0 con 0x7fad38102790 2024-05-15T16:30:42.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.710+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad38105520 con 0x7fad38102790 2024-05-15T16:30:42.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.710+0000 7fad277fe700 1 -- 172.21.15.49:0/3380925050 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fad2c005020 con 0x7fad38102790 2024-05-15T16:30:42.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.711+0000 7fad277fe700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad2006e200 0x7fad200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:42.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.711+0000 7fad277fe700 1 -- 172.21.15.49:0/3380925050 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fad2c08dad0 con 0x7fad38102790 2024-05-15T16:30:42.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.711+0000 7fad3659c700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad2006e200 0x7fad200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:42.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.711+0000 7fad3659c700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad2006e200 0x7fad200706c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fad28000c00 tx=0x7fad2801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:42.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.716+0000 7fad277fe700 1 -- 172.21.15.49:0/3380925050 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fad2c05a960 con 0x7fad38102790 2024-05-15T16:30:42.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.944+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad38197fd0 con 0x7fad2006e200 2024-05-15T16:30:42.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.945+0000 7fad277fe700 1 -- 172.21.15.49:0/3380925050 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fad38197fd0 con 0x7fad2006e200 2024-05-15T16:30:42.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad2006e200 msgr2=0x7fad200706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad2006e200 0x7fad200706c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fad28000c00 tx=0x7fad2801a040 comp rx=0 tx=0).stop 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38102790 msgr2=0x7fad3819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38102790 0x7fad3819cea0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00d6a0 tx=0x7fad2c00d9b0 comp rx=0 tx=0).stop 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 shutdown_connections 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fad2006e200 0x7fad200706c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad38101e70 0x7fad3819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 --2- 172.21.15.49:0/3380925050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad38102790 0x7fad3819cea0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 >> 172.21.15.49:0/3380925050 conn(0x7fad380fb700 msgr2=0x7fad38103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 shutdown_connections 2024-05-15T16:30:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:42.947+0000 7fad3cd46700 1 -- 172.21.15.49:0/3380925050 wait complete. 2024-05-15T16:30:42.959 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:30:43.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.057+0000 7fde0538f700 1 -- 172.21.15.49:0/515270846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 msgr2=0x7fde001047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.057+0000 7fde0538f700 1 --2- 172.21.15.49:0/515270846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 0x7fde001047c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009910 tx=0x7fdde8009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.057+0000 7fde0538f700 1 -- 172.21.15.49:0/515270846 shutdown_connections 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.057+0000 7fde0538f700 1 --2- 172.21.15.49:0/515270846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 0x7fde001047c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.057+0000 7fde0538f700 1 --2- 172.21.15.49:0/515270846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde000ffdc0 0x7fde001001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.057+0000 7fde0538f700 1 -- 172.21.15.49:0/515270846 >> 172.21.15.49:0/515270846 conn(0x7fde00076800 msgr2=0x7fde000fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.057+0000 7fde0538f700 1 -- 172.21.15.49:0/515270846 shutdown_connections 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.057+0000 7fde0538f700 1 -- 172.21.15.49:0/515270846 wait complete. 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fde0538f700 1 Processor -- start 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fde0538f700 1 -- start start 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fde0538f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde000ffdc0 0x7fde000fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fde0538f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 0x7fde000fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fde0538f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000fc760 con 0x7fde00100770 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fde0538f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000fc8d0 con 0x7fde000ffdc0 2024-05-15T16:30:43.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fddf67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 0x7fde000fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:43.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fddf67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 0x7fde000fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39488/0 (socket says 172.21.15.49:39488) 2024-05-15T16:30:43.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fddf67fc700 1 -- 172.21.15.49:0/2310489299 learned_addr learned my addr 172.21.15.49:0/2310489299 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:43.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.058+0000 7fddfeffd700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde000ffdc0 0x7fde000fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:43.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.059+0000 7fddf67fc700 1 -- 172.21.15.49:0/2310489299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde000ffdc0 msgr2=0x7fde000fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.059+0000 7fddf67fc700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde000ffdc0 0x7fde000fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.059+0000 7fddf67fc700 1 -- 172.21.15.49:0/2310489299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde80095f0 con 0x7fde00100770 2024-05-15T16:30:43.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.059+0000 7fddf67fc700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 0x7fde000fc220 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdde8000c00 tx=0x7fdde800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:43.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.059+0000 7fddfcff9700 1 -- 172.21.15.49:0/2310489299 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde801d070 con 0x7fde00100770 2024-05-15T16:30:43.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.059+0000 7fddfcff9700 1 -- 172.21.15.49:0/2310489299 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdde8005700 con 0x7fde00100770 2024-05-15T16:30:43.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.059+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde000fcb50 con 0x7fde00100770 2024-05-15T16:30:43.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.059+0000 7fddfcff9700 1 -- 172.21.15.49:0/2310489299 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde800fda0 con 0x7fde00100770 2024-05-15T16:30:43.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.059+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde001a3020 con 0x7fde00100770 2024-05-15T16:30:43.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.060+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde001012a0 con 0x7fde00100770 2024-05-15T16:30:43.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.061+0000 7fddfcff9700 1 -- 172.21.15.49:0/2310489299 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdde800f5d0 con 0x7fde00100770 2024-05-15T16:30:43.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.061+0000 7fddfcff9700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddec06e210 0x7fddec0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:43.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.061+0000 7fddfcff9700 1 -- 172.21.15.49:0/2310489299 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdde808f370 con 0x7fde00100770 2024-05-15T16:30:43.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.061+0000 7fddfeffd700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddec06e210 0x7fddec0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:43.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.062+0000 7fddfeffd700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddec06e210 0x7fddec0706d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fddf0005950 tx=0x7fddf000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:43.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.066+0000 7fddfcff9700 1 -- 172.21.15.49:0/2310489299 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdde8058650 con 0x7fde00100770 2024-05-15T16:30:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.292+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde000fcb50 con 0x7fddec06e210 2024-05-15T16:30:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.294+0000 7fddfcff9700 1 -- 172.21.15.49:0/2310489299 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fde000fcb50 con 0x7fddec06e210 2024-05-15T16:30:43.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddec06e210 msgr2=0x7fddec0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddec06e210 0x7fddec0706d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fddf0005950 tx=0x7fddf000b410 comp rx=0 tx=0).stop 2024-05-15T16:30:43.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 msgr2=0x7fde000fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 0x7fde000fc220 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdde8000c00 tx=0x7fdde800bd70 comp rx=0 tx=0).stop 2024-05-15T16:30:43.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 shutdown_connections 2024-05-15T16:30:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fddec06e210 0x7fddec0706d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde000ffdc0 0x7fde000fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 --2- 172.21.15.49:0/2310489299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde00100770 0x7fde000fc220 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 >> 172.21.15.49:0/2310489299 conn(0x7fde00076800 msgr2=0x7fde000688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 shutdown_connections 2024-05-15T16:30:43.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.296+0000 7fde0538f700 1 -- 172.21.15.49:0/2310489299 wait complete. 2024-05-15T16:30:43.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.405+0000 7f80af571700 1 -- 172.21.15.49:0/3405534861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 msgr2=0x7f80a8101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.405+0000 7f80af571700 1 --2- 172.21.15.49:0/3405534861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 0x7f80a8101290 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009910 tx=0x7f80a4009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.405+0000 7f80af571700 1 -- 172.21.15.49:0/3405534861 shutdown_connections 2024-05-15T16:30:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.405+0000 7f80af571700 1 --2- 172.21.15.49:0/3405534861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a8101860 0x7f80a81058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.405+0000 7f80af571700 1 --2- 172.21.15.49:0/3405534861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 0x7f80a8101290 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.405+0000 7f80af571700 1 -- 172.21.15.49:0/3405534861 >> 172.21.15.49:0/3405534861 conn(0x7f80a80fc760 msgr2=0x7f80a80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.405+0000 7f80af571700 1 -- 172.21.15.49:0/3405534861 shutdown_connections 2024-05-15T16:30:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.405+0000 7f80af571700 1 -- 172.21.15.49:0/3405534861 wait complete. 2024-05-15T16:30:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.405+0000 7f80af571700 1 Processor -- start 2024-05-15T16:30:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80af571700 1 -- start start 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80af571700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 0x7f80a819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80af571700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a8101860 0x7f80a819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80af571700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a819d410 con 0x7f80a8100eb0 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80af571700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8196a40 con 0x7f80a8101860 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80ad30d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 0x7f80a819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80ad30d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 0x7f80a819c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39504/0 (socket says 172.21.15.49:39504) 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80ad30d700 1 -- 172.21.15.49:0/1198817427 learned_addr learned my addr 172.21.15.49:0/1198817427 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80acb0c700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a8101860 0x7f80a819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80ad30d700 1 -- 172.21.15.49:0/1198817427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a8101860 msgr2=0x7f80a819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80ad30d700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a8101860 0x7f80a819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80ad30d700 1 -- 172.21.15.49:0/1198817427 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40095f0 con 0x7f80a8100eb0 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80acb0c700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a8101860 0x7f80a819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:30:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.406+0000 7f80ad30d700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 0x7f80a819c8b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000c00 tx=0x7f80a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.407+0000 7f809e7fc700 1 -- 172.21.15.49:0/1198817427 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a401d070 con 0x7f80a8100eb0 2024-05-15T16:30:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.407+0000 7f809e7fc700 1 -- 172.21.15.49:0/1198817427 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80a40056e0 con 0x7f80a8100eb0 2024-05-15T16:30:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.407+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a8196cc0 con 0x7f80a8100eb0 2024-05-15T16:30:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.407+0000 7f809e7fc700 1 -- 172.21.15.49:0/1198817427 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a400fe80 con 0x7f80a8100eb0 2024-05-15T16:30:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.407+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a81971b0 con 0x7f80a8100eb0 2024-05-15T16:30:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.408+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a8109250 con 0x7f80a8100eb0 2024-05-15T16:30:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.408+0000 7f809e7fc700 1 -- 172.21.15.49:0/1198817427 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f80a400f6f0 con 0x7f80a8100eb0 2024-05-15T16:30:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.409+0000 7f809e7fc700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f809406e210 0x7f80940706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.409+0000 7f809e7fc700 1 -- 172.21.15.49:0/1198817427 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f80a408f5f0 con 0x7f80a8100eb0 2024-05-15T16:30:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.409+0000 7f80acb0c700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f809406e210 0x7f80940706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.409+0000 7f80acb0c700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f809406e210 0x7f80940706d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8098005950 tx=0x7f80980058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:43.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.414+0000 7f809e7fc700 1 -- 172.21.15.49:0/1198817427 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f80a405c400 con 0x7f80a8100eb0 2024-05-15T16:30:43.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.630+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80a8197f20 con 0x7f809406e210 2024-05-15T16:30:43.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.641+0000 7f809e7fc700 1 -- 172.21.15.49:0/1198817427 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f80a8197f20 con 0x7f809406e210 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (33m) 11m ago 45m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (45m) 11m ago 45m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (44m) 11m ago 44m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (31m) 11m ago 45m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (31m) 11m ago 44m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (33m) 11m ago 45m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (27m) 11m ago 39m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (27m) 11m ago 39m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (27m) 11m ago 39m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (11m) 11m ago 39m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (36m) 11m ago 47m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:30:43.643 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (35m) 11m ago 44m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (32m) 11m ago 47m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (32m) 11m ago 44m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (34m) 11m ago 45m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (34m) 11m ago 44m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (30m) 11m ago 43m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (30m) 11m ago 43m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (29m) 11m ago 42m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (29m) 11m ago 42m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (28m) 11m ago 41m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (27m) 11m ago 41m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:30:43.644 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (34m) 11m ago 45m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:30:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f809406e210 msgr2=0x7f80940706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f809406e210 0x7f80940706d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8098005950 tx=0x7f80980058e0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 msgr2=0x7f80a819c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 0x7f80a819c8b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000c00 tx=0x7f80a400be20 comp rx=0 tx=0).stop 2024-05-15T16:30:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 shutdown_connections 2024-05-15T16:30:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f809406e210 0x7f80940706d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80a8100eb0 0x7f80a819c8b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 --2- 172.21.15.49:0/1198817427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a8101860 0x7f80a819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 >> 172.21.15.49:0/1198817427 conn(0x7f80a80fc760 msgr2=0x7f80a80fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 shutdown_connections 2024-05-15T16:30:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.644+0000 7f80af571700 1 -- 172.21.15.49:0/1198817427 wait complete. 2024-05-15T16:30:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.750+0000 7f03c4edf700 1 -- 172.21.15.49:0/1603206344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 msgr2=0x7f03c0102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.750+0000 7f03c4edf700 1 --2- 172.21.15.49:0/1603206344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 0x7f03c0102230 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009910 tx=0x7f03b0009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.750+0000 7f03c4edf700 1 -- 172.21.15.49:0/1603206344 shutdown_connections 2024-05-15T16:30:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.750+0000 7f03c4edf700 1 --2- 172.21.15.49:0/1603206344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c0102770 0x7f03c010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.750+0000 7f03c4edf700 1 --2- 172.21.15.49:0/1603206344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 0x7f03c0102230 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.750+0000 7f03c4edf700 1 -- 172.21.15.49:0/1603206344 >> 172.21.15.49:0/1603206344 conn(0x7f03c00fb6a0 msgr2=0x7f03c00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.750+0000 7f03c4edf700 1 -- 172.21.15.49:0/1603206344 shutdown_connections 2024-05-15T16:30:43.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.750+0000 7f03c4edf700 1 -- 172.21.15.49:0/1603206344 wait complete. 2024-05-15T16:30:43.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.751+0000 7f03c4edf700 1 Processor -- start 2024-05-15T16:30:43.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.751+0000 7f03c4edf700 1 -- start start 2024-05-15T16:30:43.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03c4edf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 0x7f03c019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:43.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03c4edf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c0102770 0x7f03c019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:43.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03c4edf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c019d430 con 0x7f03c0101e50 2024-05-15T16:30:43.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03c4edf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c0196a10 con 0x7f03c0102770 2024-05-15T16:30:43.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03be59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 0x7f03c019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:43.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03be59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 0x7f03c019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45396/0 (socket says 172.21.15.49:45396) 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03be59c700 1 -- 172.21.15.49:0/3646299281 learned_addr learned my addr 172.21.15.49:0/3646299281 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03b5d9b700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c0102770 0x7f03c019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03be59c700 1 -- 172.21.15.49:0/3646299281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c0102770 msgr2=0x7f03c019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03be59c700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c0102770 0x7f03c019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03be59c700 1 -- 172.21.15.49:0/3646299281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00095f0 con 0x7f03c0101e50 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03be59c700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 0x7f03c019c8d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f03b000bd70 tx=0x7f03b000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.752+0000 7f03b7fff700 1 -- 172.21.15.49:0/3646299281 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b001d070 con 0x7f03c0101e50 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.753+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c0196c90 con 0x7f03c0101e50 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.753+0000 7f03b7fff700 1 -- 172.21.15.49:0/3646299281 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03b000f460 con 0x7f03c0101e50 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.753+0000 7f03b7fff700 1 -- 172.21.15.49:0/3646299281 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b0017630 con 0x7f03c0101e50 2024-05-15T16:30:43.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.753+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c0197180 con 0x7f03c0101e50 2024-05-15T16:30:43.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.753+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c0105570 con 0x7f03c0101e50 2024-05-15T16:30:43.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.754+0000 7f03b7fff700 1 -- 172.21.15.49:0/3646299281 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f03b000f5d0 con 0x7f03c0101e50 2024-05-15T16:30:43.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.755+0000 7f03b7fff700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03ac06e200 0x7f03ac0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:43.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.755+0000 7f03b7fff700 1 -- 172.21.15.49:0/3646299281 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f03b008f690 con 0x7f03c0101e50 2024-05-15T16:30:43.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.755+0000 7f03b5d9b700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03ac06e200 0x7f03ac0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:43.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.755+0000 7f03b5d9b700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03ac06e200 0x7f03ac0706c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005fd0 tx=0x7f03a800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:43.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:43.759+0000 7f03b7fff700 1 -- 172.21.15.49:0/3646299281 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f03b005c4a0 con 0x7f03c0101e50 2024-05-15T16:30:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.052+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f03c0197ed0 con 0x7f03c0101e50 2024-05-15T16:30:44.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.053+0000 7f03b7fff700 1 -- 172.21.15.49:0/3646299281 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f03b005c030 con 0x7f03c0101e50 2024-05-15T16:30:44.054 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:30:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:30:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:30:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:30:44.055 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:30:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03ac06e200 msgr2=0x7f03ac0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03ac06e200 0x7f03ac0706c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005fd0 tx=0x7f03a800a560 comp rx=0 tx=0).stop 2024-05-15T16:30:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 msgr2=0x7f03c019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 0x7f03c019c8d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f03b000bd70 tx=0x7f03b000be50 comp rx=0 tx=0).stop 2024-05-15T16:30:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 shutdown_connections 2024-05-15T16:30:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f03ac06e200 0x7f03ac0706c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03c0101e50 0x7f03c019c8d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 --2- 172.21.15.49:0/3646299281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c0102770 0x7f03c019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 >> 172.21.15.49:0/3646299281 conn(0x7f03c00fb6a0 msgr2=0x7f03c01030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 shutdown_connections 2024-05-15T16:30:44.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.055+0000 7f03c4edf700 1 -- 172.21.15.49:0/3646299281 wait complete. 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.163+0000 7f9fef6a2700 1 -- 172.21.15.49:0/1335137965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe81027a0 msgr2=0x7f9fe810ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.163+0000 7f9fef6a2700 1 --2- 172.21.15.49:0/1335137965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe81027a0 0x7f9fe810ac90 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009940 tx=0x7f9fd8009c50 comp rx=0 tx=0).stop 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.163+0000 7f9fef6a2700 1 -- 172.21.15.49:0/1335137965 shutdown_connections 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.163+0000 7f9fef6a2700 1 --2- 172.21.15.49:0/1335137965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe81027a0 0x7f9fe810ac90 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.163+0000 7f9fef6a2700 1 --2- 172.21.15.49:0/1335137965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fe8101e80 0x7f9fe8102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.163+0000 7f9fef6a2700 1 -- 172.21.15.49:0/1335137965 >> 172.21.15.49:0/1335137965 conn(0x7f9fe8076800 msgr2=0x7f9fe80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.163+0000 7f9fef6a2700 1 -- 172.21.15.49:0/1335137965 shutdown_connections 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.163+0000 7f9fef6a2700 1 -- 172.21.15.49:0/1335137965 wait complete. 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fef6a2700 1 Processor -- start 2024-05-15T16:30:44.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fef6a2700 1 -- start start 2024-05-15T16:30:44.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fef6a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe8101e80 0x7f9fe80ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:44.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fef6a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fe81027a0 0x7f9fe8100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:44.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fef6a2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8101ab0 con 0x7f9fe8101e80 2024-05-15T16:30:44.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fef6a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8100560 con 0x7f9fe81027a0 2024-05-15T16:30:44.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fed43e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe8101e80 0x7f9fe80ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:44.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fed43e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe8101e80 0x7f9fe80ffae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45412/0 (socket says 172.21.15.49:45412) 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fed43e700 1 -- 172.21.15.49:0/946222933 learned_addr learned my addr 172.21.15.49:0/946222933 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.164+0000 7f9fecc3d700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fe81027a0 0x7f9fe8100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fed43e700 1 -- 172.21.15.49:0/946222933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fe81027a0 msgr2=0x7f9fe8100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fed43e700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fe81027a0 0x7f9fe8100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fed43e700 1 -- 172.21.15.49:0/946222933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd80095f0 con 0x7f9fe8101e80 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fecc3d700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fe81027a0 0x7f9fe8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fed43e700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe8101e80 0x7f9fe80ffae0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c930 tx=0x7f9fe400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fde7fc700 1 -- 172.21.15.49:0/946222933 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe4007ad0 con 0x7f9fe8101e80 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fde7fc700 1 -- 172.21.15.49:0/946222933 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9fe4007c30 con 0x7f9fe8101e80 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe8100810 con 0x7f9fe8101e80 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fde7fc700 1 -- 172.21.15.49:0/946222933 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe4018610 con 0x7f9fe8101e80 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.165+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe81a2fe0 con 0x7f9fe8101e80 2024-05-15T16:30:44.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.166+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe8068f00 con 0x7f9fe8101e80 2024-05-15T16:30:44.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.167+0000 7f9fde7fc700 1 -- 172.21.15.49:0/946222933 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9fe400f450 con 0x7f9fe8101e80 2024-05-15T16:30:44.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.167+0000 7f9fde7fc700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fd406e200 0x7f9fd40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:44.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.167+0000 7f9fde7fc700 1 -- 172.21.15.49:0/946222933 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9fe408e2c0 con 0x7f9fe8101e80 2024-05-15T16:30:44.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.167+0000 7f9fecc3d700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fd406e200 0x7f9fd40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:44.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.168+0000 7f9fecc3d700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fd406e200 0x7f9fd40706c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009910 tx=0x7f9fd8005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:44.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.172+0000 7f9fde7fc700 1 -- 172.21.15.49:0/946222933 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9fe4057620 con 0x7f9fe8101e80 2024-05-15T16:30:44.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.425+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9fe804ec60 con 0x7f9fe8101e80 2024-05-15T16:30:44.426 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:44 smithi049 ceph-mon[112142]: pgmap v1012: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:44.427 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:44 smithi049 ceph-mon[112142]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:44.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.425+0000 7f9fde7fc700 1 -- 172.21.15.49:0/946222933 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f9fe405ace0 con 0x7f9fe8101e80 2024-05-15T16:30:44.427 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:30:44.427 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:30:44.427 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:30:44.427 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:30:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:30:44.428 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:30:44.429 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:30:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:30:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:30:44.430 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:30:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:30:44.430 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:30:44.430 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:30:44.430 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:30:44.430 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:30:44.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fd406e200 msgr2=0x7f9fd40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fd406e200 0x7f9fd40706c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009910 tx=0x7f9fd8005940 comp rx=0 tx=0).stop 2024-05-15T16:30:44.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe8101e80 msgr2=0x7f9fe80ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe8101e80 0x7f9fe80ffae0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c930 tx=0x7f9fe400ccf0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 shutdown_connections 2024-05-15T16:30:44.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9fd406e200 0x7f9fd40706c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fe8101e80 0x7f9fe80ffae0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 --2- 172.21.15.49:0/946222933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fe81027a0 0x7f9fe8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 >> 172.21.15.49:0/946222933 conn(0x7f9fe8076800 msgr2=0x7f9fe8103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:44.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 shutdown_connections 2024-05-15T16:30:44.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.428+0000 7f9fef6a2700 1 -- 172.21.15.49:0/946222933 wait complete. 2024-05-15T16:30:44.432 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:30:44.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:44 smithi144 ceph-mon[86635]: pgmap v1012: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:44 smithi144 ceph-mon[86635]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:44.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.539+0000 7f1d4d771700 1 -- 172.21.15.49:0/1819800507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d481018b0 msgr2=0x7f1d48105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.539+0000 7f1d4d771700 1 --2- 172.21.15.49:0/1819800507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d481018b0 0x7f1d48105900 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009910 tx=0x7f1d3c009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.539+0000 7f1d4d771700 1 -- 172.21.15.49:0/1819800507 shutdown_connections 2024-05-15T16:30:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.539+0000 7f1d4d771700 1 --2- 172.21.15.49:0/1819800507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d481018b0 0x7f1d48105900 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.539+0000 7f1d4d771700 1 --2- 172.21.15.49:0/1819800507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d48100f00 0x7f1d481012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.539+0000 7f1d4d771700 1 -- 172.21.15.49:0/1819800507 >> 172.21.15.49:0/1819800507 conn(0x7f1d480fc7d0 msgr2=0x7f1d480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.539+0000 7f1d4d771700 1 -- 172.21.15.49:0/1819800507 shutdown_connections 2024-05-15T16:30:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.539+0000 7f1d4d771700 1 -- 172.21.15.49:0/1819800507 wait complete. 2024-05-15T16:30:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.539+0000 7f1d4d771700 1 Processor -- start 2024-05-15T16:30:44.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d4d771700 1 -- start start 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d4d771700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d48100f00 0x7f1d4819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d4d771700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d481018b0 0x7f1d4819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d4d771700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4819d440 con 0x7f1d48100f00 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d4d771700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d48196a20 con 0x7f1d481018b0 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d46ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d48100f00 0x7f1d4819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d46ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d48100f00 0x7f1d4819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45420/0 (socket says 172.21.15.49:45420) 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d46ffd700 1 -- 172.21.15.49:0/4104290461 learned_addr learned my addr 172.21.15.49:0/4104290461 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d467fc700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d481018b0 0x7f1d4819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d46ffd700 1 -- 172.21.15.49:0/4104290461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d481018b0 msgr2=0x7f1d4819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d46ffd700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d481018b0 0x7f1d4819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d46ffd700 1 -- 172.21.15.49:0/4104290461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c0095f0 con 0x7f1d48100f00 2024-05-15T16:30:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.540+0000 7f1d46ffd700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d48100f00 0x7f1d4819c8e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800d8d0 tx=0x7f1d3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.541+0000 7f1d2ffff700 1 -- 172.21.15.49:0/4104290461 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d38009940 con 0x7f1d48100f00 2024-05-15T16:30:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.541+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d48196d00 con 0x7f1d48100f00 2024-05-15T16:30:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.541+0000 7f1d2ffff700 1 -- 172.21.15.49:0/4104290461 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d38010460 con 0x7f1d48100f00 2024-05-15T16:30:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.541+0000 7f1d2ffff700 1 -- 172.21.15.49:0/4104290461 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d3800f5f0 con 0x7f1d48100f00 2024-05-15T16:30:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.541+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d48197250 con 0x7f1d48100f00 2024-05-15T16:30:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.542+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d48109300 con 0x7f1d48100f00 2024-05-15T16:30:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.542+0000 7f1d2ffff700 1 -- 172.21.15.49:0/4104290461 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1d380105d0 con 0x7f1d48100f00 2024-05-15T16:30:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.543+0000 7f1d2ffff700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d3006e200 0x7f1d300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.543+0000 7f1d2ffff700 1 -- 172.21.15.49:0/4104290461 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1d3808e1a0 con 0x7f1d48100f00 2024-05-15T16:30:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.543+0000 7f1d467fc700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d3006e200 0x7f1d300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.543+0000 7f1d467fc700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d3006e200 0x7f1d300706c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00b5c0 tx=0x7f1d3c0052a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:44.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.547+0000 7f1d2ffff700 1 -- 172.21.15.49:0/4104290461 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1d3805afb0 con 0x7f1d48100f00 2024-05-15T16:30:44.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.771+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d4810dea0 con 0x7f1d3006e200 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.773+0000 7f1d2ffff700 1 -- 172.21.15.49:0/4104290461 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1d4810dea0 con 0x7f1d3006e200 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:30:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:30:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:30:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:30:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:30:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:30:44.775 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:30:44.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d3006e200 msgr2=0x7f1d300706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d3006e200 0x7f1d300706c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00b5c0 tx=0x7f1d3c0052a0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d48100f00 msgr2=0x7f1d4819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d48100f00 0x7f1d4819c8e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800d8d0 tx=0x7f1d3800dc90 comp rx=0 tx=0).stop 2024-05-15T16:30:44.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 shutdown_connections 2024-05-15T16:30:44.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1d3006e200 0x7f1d300706c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d48100f00 0x7f1d4819c8e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 --2- 172.21.15.49:0/4104290461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d481018b0 0x7f1d4819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 >> 172.21.15.49:0/4104290461 conn(0x7f1d480fc7d0 msgr2=0x7f1d480fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:44.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 shutdown_connections 2024-05-15T16:30:44.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.775+0000 7f1d4d771700 1 -- 172.21.15.49:0/4104290461 wait complete. 2024-05-15T16:30:44.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.883+0000 7feb1c774700 1 -- 172.21.15.49:0/2274697881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 msgr2=0x7feb141012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.883+0000 7feb1c774700 1 --2- 172.21.15.49:0/2274697881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 0x7feb141012e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7feb10009910 tx=0x7feb10009c20 comp rx=0 tx=0).stop 2024-05-15T16:30:44.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.884+0000 7feb1c774700 1 -- 172.21.15.49:0/2274697881 shutdown_connections 2024-05-15T16:30:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.884+0000 7feb1c774700 1 --2- 172.21.15.49:0/2274697881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb141018b0 0x7feb14105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.884+0000 7feb1c774700 1 --2- 172.21.15.49:0/2274697881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 0x7feb141012e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.884+0000 7feb1c774700 1 -- 172.21.15.49:0/2274697881 >> 172.21.15.49:0/2274697881 conn(0x7feb140fc7d0 msgr2=0x7feb140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.884+0000 7feb1c774700 1 -- 172.21.15.49:0/2274697881 shutdown_connections 2024-05-15T16:30:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.884+0000 7feb1c774700 1 -- 172.21.15.49:0/2274697881 wait complete. 2024-05-15T16:30:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.884+0000 7feb1c774700 1 Processor -- start 2024-05-15T16:30:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.884+0000 7feb1c774700 1 -- start start 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1c774700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 0x7feb140732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1c774700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb141018b0 0x7feb140737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1c774700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb14073d20 con 0x7feb14100f00 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1c774700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb14073e60 con 0x7feb141018b0 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1a510700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 0x7feb140732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1a510700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 0x7feb140732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45428/0 (socket says 172.21.15.49:45428) 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1a510700 1 -- 172.21.15.49:0/1741784971 learned_addr learned my addr 172.21.15.49:0/1741784971 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb19d0f700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb141018b0 0x7feb140737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1a510700 1 -- 172.21.15.49:0/1741784971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb141018b0 msgr2=0x7feb140737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1a510700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb141018b0 0x7feb140737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1a510700 1 -- 172.21.15.49:0/1741784971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100095f0 con 0x7feb14100f00 2024-05-15T16:30:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1a510700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 0x7feb140732a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7feb1000bd70 tx=0x7feb1000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:44.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb0b7fe700 1 -- 172.21.15.49:0/1741784971 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb1001d070 con 0x7feb14100f00 2024-05-15T16:30:44.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb0b7fe700 1 -- 172.21.15.49:0/1741784971 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb100056e0 con 0x7feb14100f00 2024-05-15T16:30:44.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb140740e0 con 0x7feb14100f00 2024-05-15T16:30:44.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.885+0000 7feb0b7fe700 1 -- 172.21.15.49:0/1741784971 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb10017400 con 0x7feb14100f00 2024-05-15T16:30:44.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.886+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb141a3040 con 0x7feb14100f00 2024-05-15T16:30:44.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.887+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb141092a0 con 0x7feb14100f00 2024-05-15T16:30:44.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.887+0000 7feb0b7fe700 1 -- 172.21.15.49:0/1741784971 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feb10003a60 con 0x7feb14100f00 2024-05-15T16:30:44.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.888+0000 7feb0b7fe700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feb0006e200 0x7feb000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:30:44.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.888+0000 7feb0b7fe700 1 -- 172.21.15.49:0/1741784971 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7feb1008f5d0 con 0x7feb14100f00 2024-05-15T16:30:44.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.888+0000 7feb19d0f700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feb0006e200 0x7feb000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:30:44.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.888+0000 7feb19d0f700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feb0006e200 0x7feb000706c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7feb04005950 tx=0x7feb040058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:30:44.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:44.892+0000 7feb0b7fe700 1 -- 172.21.15.49:0/1741784971 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7feb100588b0 con 0x7feb14100f00 2024-05-15T16:30:45.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.188+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feb140747d0 con 0x7feb14100f00 2024-05-15T16:30:45.190 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:45 smithi049 ceph-mon[112142]: from='client.35834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:45.190 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:45 smithi049 ceph-mon[112142]: from='client.35838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:45.190 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:45 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3646299281' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:30:45.190 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:45 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/946222933' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:30:45.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.189+0000 7feb0b7fe700 1 -- 172.21.15.49:0/1741784971 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7feb1005bf70 con 0x7feb14100f00 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:30:45.191 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:30:45.192 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:30:45.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feb0006e200 msgr2=0x7feb000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:45.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feb0006e200 0x7feb000706c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7feb04005950 tx=0x7feb040058e0 comp rx=0 tx=0).stop 2024-05-15T16:30:45.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 msgr2=0x7feb140732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:30:45.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 0x7feb140732a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7feb1000bd70 tx=0x7feb1000be50 comp rx=0 tx=0).stop 2024-05-15T16:30:45.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 shutdown_connections 2024-05-15T16:30:45.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7feb0006e200 0x7feb000706c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:45.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb14100f00 0x7feb140732a0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:45.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 --2- 172.21.15.49:0/1741784971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb141018b0 0x7feb140737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:30:45.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 >> 172.21.15.49:0/1741784971 conn(0x7feb140fc7d0 msgr2=0x7feb140fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:30:45.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 shutdown_connections 2024-05-15T16:30:45.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:30:45.191+0000 7feb1c774700 1 -- 172.21.15.49:0/1741784971 wait complete. 2024-05-15T16:30:45.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:45 smithi144 ceph-mon[86635]: from='client.35834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:45.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:45 smithi144 ceph-mon[86635]: from='client.35838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:45.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:45 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3646299281' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:30:45.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:45 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/946222933' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:30:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:46 smithi144 ceph-mon[86635]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:46 smithi144 ceph-mon[86635]: pgmap v1013: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:46 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1741784971' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:30:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:46 smithi049 ceph-mon[112142]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:30:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:46 smithi049 ceph-mon[112142]: pgmap v1013: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:46 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1741784971' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:30:48.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:48 smithi144 ceph-mon[86635]: pgmap v1014: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:48.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:48 smithi049 ceph-mon[112142]: pgmap v1014: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:50.409 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:50 smithi049 ceph-mon[112142]: pgmap v1015: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:50 smithi144 ceph-mon[86635]: pgmap v1015: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:51.021 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:30:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:30:52.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:52 smithi144 ceph-mon[86635]: pgmap v1016: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:52.468 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:52 smithi049 ceph-mon[112142]: pgmap v1016: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:54 smithi144 ceph-mon[86635]: pgmap v1017: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:54 smithi049 ceph-mon[112142]: pgmap v1017: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:56 smithi144 ceph-mon[86635]: pgmap v1018: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:56 smithi049 ceph-mon[112142]: pgmap v1018: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:30:58 smithi144 ceph-mon[86635]: pgmap v1019: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:30:58.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:30:58 smithi049 ceph-mon[112142]: pgmap v1019: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:00.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:00 smithi144 ceph-mon[86635]: pgmap v1020: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:00 smithi049 ceph-mon[112142]: pgmap v1020: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:02 smithi144 ceph-mon[86635]: pgmap v1021: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:02 smithi049 ceph-mon[112142]: pgmap v1021: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:04.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:04 smithi144 ceph-mon[86635]: pgmap v1022: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:04 smithi049 ceph-mon[112142]: pgmap v1022: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:06.331 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:06 smithi144 ceph-mon[86635]: pgmap v1023: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:06.331 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:31:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:06 smithi049 ceph-mon[112142]: pgmap v1023: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:31:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:08 smithi144 ceph-mon[86635]: pgmap v1024: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:08 smithi049 ceph-mon[112142]: pgmap v1024: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:10 smithi144 ceph-mon[86635]: pgmap v1025: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:10 smithi049 ceph-mon[112142]: pgmap v1025: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:12.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:12 smithi144 ceph-mon[86635]: pgmap v1026: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:12.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:12 smithi049 ceph-mon[112142]: pgmap v1026: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:14 smithi144 ceph-mon[86635]: pgmap v1027: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:14.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:14 smithi049 ceph-mon[112142]: pgmap v1027: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:15.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.315+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1132082742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 msgr2=0x7fc6a8103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:15.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.315+0000 7fc6afbeb700 1 --2- 172.21.15.49:0/1132082742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 0x7fc6a8103f90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc69800aa20 tx=0x7fc69800ad30 comp rx=0 tx=0).stop 2024-05-15T16:31:15.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.315+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1132082742 shutdown_connections 2024-05-15T16:31:15.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.315+0000 7fc6afbeb700 1 --2- 172.21.15.49:0/1132082742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 0x7fc6a8103f90 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.315+0000 7fc6afbeb700 1 --2- 172.21.15.49:0/1132082742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a80ff6f0 0x7fc6a80ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.315+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1132082742 >> 172.21.15.49:0/1132082742 conn(0x7fc6a8074e50 msgr2=0x7fc6a8075260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.315+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1132082742 shutdown_connections 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.315+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1132082742 wait complete. 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6afbeb700 1 Processor -- start 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6afbeb700 1 -- start start 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6afbeb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a80ff6f0 0x7fc6a819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6afbeb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 0x7fc6a819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6afbeb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a819b300 con 0x7fc6a81000a0 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6afbeb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a8194880 con 0x7fc6a80ff6f0 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6ad186700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 0x7fc6a819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6ad186700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 0x7fc6a819ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34944/0 (socket says 172.21.15.49:34944) 2024-05-15T16:31:15.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6ad186700 1 -- 172.21.15.49:0/1045211959 learned_addr learned my addr 172.21.15.49:0/1045211959 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.316+0000 7fc6ad987700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a80ff6f0 0x7fc6a819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.317+0000 7fc6ad186700 1 -- 172.21.15.49:0/1045211959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a80ff6f0 msgr2=0x7fc6a819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.317+0000 7fc6ad186700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a80ff6f0 0x7fc6a819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.317+0000 7fc6ad186700 1 -- 172.21.15.49:0/1045211959 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69800a700 con 0x7fc6a81000a0 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.317+0000 7fc6ad186700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 0x7fc6a819ac20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc698005e20 tx=0x7fc698005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.317+0000 7fc69effd700 1 -- 172.21.15.49:0/1045211959 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc698004a90 con 0x7fc6a81000a0 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.317+0000 7fc69effd700 1 -- 172.21.15.49:0/1045211959 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc698004bf0 con 0x7fc6a81000a0 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.317+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a8194b00 con 0x7fc6a81000a0 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.317+0000 7fc69effd700 1 -- 172.21.15.49:0/1045211959 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc698016820 con 0x7fc6a81000a0 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.317+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a8194ff0 con 0x7fc6a81000a0 2024-05-15T16:31:15.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.318+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a8107990 con 0x7fc6a81000a0 2024-05-15T16:31:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.319+0000 7fc69effd700 1 -- 172.21.15.49:0/1045211959 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc69800ec30 con 0x7fc6a81000a0 2024-05-15T16:31:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.319+0000 7fc69effd700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc69406e200 0x7fc6940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.319+0000 7fc69effd700 1 -- 172.21.15.49:0/1045211959 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc698012070 con 0x7fc6a81000a0 2024-05-15T16:31:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.319+0000 7fc6ad987700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc69406e200 0x7fc6940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:15.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.320+0000 7fc6ad987700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc69406e200 0x7fc6940706c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005950 tx=0x7fc6a40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:15.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.324+0000 7fc69effd700 1 -- 172.21.15.49:0/1045211959 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc69805de70 con 0x7fc6a81000a0 2024-05-15T16:31:15.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.551+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6a8195dd0 con 0x7fc69406e200 2024-05-15T16:31:15.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.553+0000 7fc69effd700 1 -- 172.21.15.49:0/1045211959 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc6a8195dd0 con 0x7fc69406e200 2024-05-15T16:31:15.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc69406e200 msgr2=0x7fc6940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:15.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc69406e200 0x7fc6940706c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005950 tx=0x7fc6a40058e0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 msgr2=0x7fc6a819ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:15.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 0x7fc6a819ac20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc698005e20 tx=0x7fc698005ef0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 shutdown_connections 2024-05-15T16:31:15.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc69406e200 0x7fc6940706c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a80ff6f0 0x7fc6a819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 --2- 172.21.15.49:0/1045211959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6a81000a0 0x7fc6a819ac20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 >> 172.21.15.49:0/1045211959 conn(0x7fc6a8074e50 msgr2=0x7fc6a80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:15.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 shutdown_connections 2024-05-15T16:31:15.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.555+0000 7fc6afbeb700 1 -- 172.21.15.49:0/1045211959 wait complete. 2024-05-15T16:31:15.566 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:31:15.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.667+0000 7f06b3ed3700 1 -- 172.21.15.49:0/937146457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 msgr2=0x7f06ac1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:15.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.667+0000 7f06b3ed3700 1 --2- 172.21.15.49:0/937146457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 0x7f06ac1012e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f069c00b0a0 tx=0x7f069c00b3b0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.667+0000 7f06b3ed3700 1 -- 172.21.15.49:0/937146457 shutdown_connections 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.667+0000 7f06b3ed3700 1 --2- 172.21.15.49:0/937146457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac1018b0 0x7f06ac105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.667+0000 7f06b3ed3700 1 --2- 172.21.15.49:0/937146457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 0x7f06ac1012e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.667+0000 7f06b3ed3700 1 -- 172.21.15.49:0/937146457 >> 172.21.15.49:0/937146457 conn(0x7f06ac0fc7f0 msgr2=0x7f06ac0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.667+0000 7f06b3ed3700 1 -- 172.21.15.49:0/937146457 shutdown_connections 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.667+0000 7f06b3ed3700 1 -- 172.21.15.49:0/937146457 wait complete. 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b3ed3700 1 Processor -- start 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b3ed3700 1 -- start start 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b3ed3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 0x7f06ac0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b3ed3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac1018b0 0x7f06ac0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b3ed3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac075220 con 0x7f06ac100f00 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b3ed3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac073d20 con 0x7f06ac1018b0 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b1c6f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 0x7f06ac0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b1c6f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 0x7f06ac0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34966/0 (socket says 172.21.15.49:34966) 2024-05-15T16:31:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b1c6f700 1 -- 172.21.15.49:0/3476389179 learned_addr learned my addr 172.21.15.49:0/3476389179 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b146e700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac1018b0 0x7f06ac0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b1c6f700 1 -- 172.21.15.49:0/3476389179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac1018b0 msgr2=0x7f06ac0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b1c6f700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac1018b0 0x7f06ac0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.668+0000 7f06b1c6f700 1 -- 172.21.15.49:0/3476389179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c009d00 con 0x7f06ac100f00 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.669+0000 7f06b1c6f700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 0x7f06ac0732a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f069c000f80 tx=0x7f069c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.669+0000 7f06aaffd700 1 -- 172.21.15.49:0/3476389179 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f069c00e040 con 0x7f06ac100f00 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.669+0000 7f06aaffd700 1 -- 172.21.15.49:0/3476389179 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f069c003de0 con 0x7f06ac100f00 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.669+0000 7f06aaffd700 1 -- 172.21.15.49:0/3476389179 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f069c01d720 con 0x7f06ac100f00 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.669+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac073fa0 con 0x7f06ac100f00 2024-05-15T16:31:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.669+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac1a2fc0 con 0x7f06ac100f00 2024-05-15T16:31:15.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.670+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac1092a0 con 0x7f06ac100f00 2024-05-15T16:31:15.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.670+0000 7f06aaffd700 1 -- 172.21.15.49:0/3476389179 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f069c004690 con 0x7f06ac100f00 2024-05-15T16:31:15.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.671+0000 7f06aaffd700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f069406e200 0x7f06940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:15.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.671+0000 7f06aaffd700 1 -- 172.21.15.49:0/3476389179 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f069c08fa10 con 0x7f06ac100f00 2024-05-15T16:31:15.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.671+0000 7f06b146e700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f069406e200 0x7f06940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:15.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.671+0000 7f06b146e700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f069406e200 0x7f06940706c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f06a0005ea0 tx=0x7f06a0005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.676+0000 7f06aaffd700 1 -- 172.21.15.49:0/3476389179 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f069c058cf0 con 0x7f06ac100f00 2024-05-15T16:31:15.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.908+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06ac0747d0 con 0x7f069406e200 2024-05-15T16:31:15.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.910+0000 7f06aaffd700 1 -- 172.21.15.49:0/3476389179 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f06ac0747d0 con 0x7f069406e200 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f069406e200 msgr2=0x7f06940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f069406e200 0x7f06940706c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f06a0005ea0 tx=0x7f06a0005e30 comp rx=0 tx=0).stop 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 msgr2=0x7f06ac0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 0x7f06ac0732a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f069c000f80 tx=0x7f069c007bd0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 shutdown_connections 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f069406e200 0x7f06940706c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac100f00 0x7f06ac0732a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 --2- 172.21.15.49:0/3476389179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac1018b0 0x7f06ac0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 >> 172.21.15.49:0/3476389179 conn(0x7f06ac0fc7f0 msgr2=0x7f06ac0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 shutdown_connections 2024-05-15T16:31:15.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:15.912+0000 7f06b3ed3700 1 -- 172.21.15.49:0/3476389179 wait complete. 2024-05-15T16:31:16.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.020+0000 7f8168175700 1 -- 172.21.15.49:0/3400072264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 msgr2=0x7f81601012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.020+0000 7f8168175700 1 --2- 172.21.15.49:0/3400072264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 0x7f81601012e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f815c009910 tx=0x7f815c009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.020+0000 7f8168175700 1 -- 172.21.15.49:0/3400072264 shutdown_connections 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.020+0000 7f8168175700 1 --2- 172.21.15.49:0/3400072264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81601018b0 0x7f8160105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.020+0000 7f8168175700 1 --2- 172.21.15.49:0/3400072264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 0x7f81601012e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.020+0000 7f8168175700 1 -- 172.21.15.49:0/3400072264 >> 172.21.15.49:0/3400072264 conn(0x7f81600fc7d0 msgr2=0x7f81600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.021+0000 7f8168175700 1 -- 172.21.15.49:0/3400072264 shutdown_connections 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.021+0000 7f8168175700 1 -- 172.21.15.49:0/3400072264 wait complete. 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.021+0000 7f8168175700 1 Processor -- start 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.021+0000 7f8168175700 1 -- start start 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.021+0000 7f8168175700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 0x7f8160198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.021+0000 7f8168175700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81601018b0 0x7f8160199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.021+0000 7f8168175700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81601999f0 con 0x7f8160100f00 2024-05-15T16:31:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.021+0000 7f8168175700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019d6c0 con 0x7f81601018b0 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.021+0000 7f8165f11700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 0x7f8160198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8165f11700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 0x7f8160198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34992/0 (socket says 172.21.15.49:34992) 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8165f11700 1 -- 172.21.15.49:0/2656177047 learned_addr learned my addr 172.21.15.49:0/2656177047 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8165710700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81601018b0 0x7f8160199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8165f11700 1 -- 172.21.15.49:0/2656177047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81601018b0 msgr2=0x7f8160199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8165f11700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81601018b0 0x7f8160199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8165f11700 1 -- 172.21.15.49:0/2656177047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c0095f0 con 0x7f8160100f00 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8165f11700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 0x7f8160198dd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f815c000c00 tx=0x7f815c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8152ffd700 1 -- 172.21.15.49:0/2656177047 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815c01d070 con 0x7f8160100f00 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8152ffd700 1 -- 172.21.15.49:0/2656177047 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f815c0056e0 con 0x7f8160100f00 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816019d940 con 0x7f8160100f00 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8152ffd700 1 -- 172.21.15.49:0/2656177047 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815c00fe80 con 0x7f8160100f00 2024-05-15T16:31:16.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.022+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816019de30 con 0x7f8160100f00 2024-05-15T16:31:16.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.023+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8160105080 con 0x7f8160100f00 2024-05-15T16:31:16.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.024+0000 7f8152ffd700 1 -- 172.21.15.49:0/2656177047 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f815c00f6f0 con 0x7f8160100f00 2024-05-15T16:31:16.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.024+0000 7f8152ffd700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f814c06e210 0x7f814c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:16.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.024+0000 7f8152ffd700 1 -- 172.21.15.49:0/2656177047 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f815c08f550 con 0x7f8160100f00 2024-05-15T16:31:16.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.024+0000 7f8165710700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f814c06e210 0x7f814c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:16.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.025+0000 7f8165710700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f814c06e210 0x7f814c0706d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8154005950 tx=0x7f81540058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:16.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.029+0000 7f8152ffd700 1 -- 172.21.15.49:0/2656177047 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f815c0588b0 con 0x7f8160100f00 2024-05-15T16:31:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:16 smithi049 ceph-mon[112142]: pgmap v1028: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:16.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.250+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f816019a1c0 con 0x7f814c06e210 2024-05-15T16:31:16.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.261+0000 7f8152ffd700 1 -- 172.21.15.49:0/2656177047 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f816019a1c0 con 0x7f814c06e210 2024-05-15T16:31:16.262 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:31:16.262 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (34m) 12m ago 46m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:31:16.262 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (46m) 12m ago 46m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:31:16.262 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (45m) 12m ago 45m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:31:16.262 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (31m) 12m ago 46m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:31:16.262 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (31m) 12m ago 45m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (34m) 12m ago 45m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (28m) 12m ago 40m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (27m) 12m ago 40m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (27m) 12m ago 40m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (12m) 12m ago 40m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (36m) 12m ago 47m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (35m) 12m ago 44m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (33m) 12m ago 47m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (32m) 12m ago 44m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (35m) 12m ago 46m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (35m) 12m ago 45m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (31m) 12m ago 44m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (30m) 12m ago 43m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (30m) 12m ago 43m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:31:16.263 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (29m) 12m ago 42m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:31:16.264 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (29m) 12m ago 42m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:31:16.264 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (28m) 12m ago 41m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:31:16.264 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (34m) 12m ago 45m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:31:16.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f814c06e210 msgr2=0x7f814c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f814c06e210 0x7f814c0706d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8154005950 tx=0x7f81540058e0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 msgr2=0x7f8160198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 0x7f8160198dd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f815c000c00 tx=0x7f815c00be20 comp rx=0 tx=0).stop 2024-05-15T16:31:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 shutdown_connections 2024-05-15T16:31:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f814c06e210 0x7f814c0706d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8160100f00 0x7f8160198dd0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 --2- 172.21.15.49:0/2656177047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81601018b0 0x7f8160199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 >> 172.21.15.49:0/2656177047 conn(0x7f81600fc7d0 msgr2=0x7f81600fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 shutdown_connections 2024-05-15T16:31:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.263+0000 7f8168175700 1 -- 172.21.15.49:0/2656177047 wait complete. 2024-05-15T16:31:16.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.371+0000 7fe49cb4b700 1 -- 172.21.15.49:0/637900365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 msgr2=0x7fe4981012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.371+0000 7fe49cb4b700 1 --2- 172.21.15.49:0/637900365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 0x7fe4981012c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe488009910 tx=0x7fe488009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:16.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.371+0000 7fe49cb4b700 1 -- 172.21.15.49:0/637900365 shutdown_connections 2024-05-15T16:31:16.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.371+0000 7fe49cb4b700 1 --2- 172.21.15.49:0/637900365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe498101890 0x7fe4981058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.371+0000 7fe49cb4b700 1 --2- 172.21.15.49:0/637900365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 0x7fe4981012c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.371+0000 7fe49cb4b700 1 -- 172.21.15.49:0/637900365 >> 172.21.15.49:0/637900365 conn(0x7fe4980fc790 msgr2=0x7fe4980febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:16.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.371+0000 7fe49cb4b700 1 -- 172.21.15.49:0/637900365 shutdown_connections 2024-05-15T16:31:16.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.371+0000 7fe49cb4b700 1 -- 172.21.15.49:0/637900365 wait complete. 2024-05-15T16:31:16.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.371+0000 7fe49cb4b700 1 Processor -- start 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49cb4b700 1 -- start start 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49cb4b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 0x7fe4980732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49cb4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe498101890 0x7fe4980737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49cb4b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe498073d20 con 0x7fe498100ee0 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49cb4b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe498073e60 con 0x7fe498101890 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 0x7fe4980732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 0x7fe4980732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35008/0 (socket says 172.21.15.49:35008) 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49659c700 1 -- 172.21.15.49:0/3344358611 learned_addr learned my addr 172.21.15.49:0/3344358611 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe495d9b700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe498101890 0x7fe4980737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49659c700 1 -- 172.21.15.49:0/3344358611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe498101890 msgr2=0x7fe4980737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49659c700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe498101890 0x7fe4980737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49659c700 1 -- 172.21.15.49:0/3344358611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4880095f0 con 0x7fe498100ee0 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe495d9b700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe498101890 0x7fe4980737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.372+0000 7fe49659c700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 0x7fe4980732a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe48800bd70 tx=0x7fe48800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.373+0000 7fe48f7fe700 1 -- 172.21.15.49:0/3344358611 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48801d070 con 0x7fe498100ee0 2024-05-15T16:31:16.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.373+0000 7fe48f7fe700 1 -- 172.21.15.49:0/3344358611 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe48800f460 con 0x7fe498100ee0 2024-05-15T16:31:16.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.373+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4980740e0 con 0x7fe498100ee0 2024-05-15T16:31:16.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.373+0000 7fe48f7fe700 1 -- 172.21.15.49:0/3344358611 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe488017630 con 0x7fe498100ee0 2024-05-15T16:31:16.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.373+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4981a3040 con 0x7fe498100ee0 2024-05-15T16:31:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.374+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe498109280 con 0x7fe498100ee0 2024-05-15T16:31:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.374+0000 7fe48f7fe700 1 -- 172.21.15.49:0/3344358611 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe48800f5d0 con 0x7fe498100ee0 2024-05-15T16:31:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.375+0000 7fe48f7fe700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe48406e200 0x7fe4840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.375+0000 7fe48f7fe700 1 -- 172.21.15.49:0/3344358611 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe48808f5e0 con 0x7fe498100ee0 2024-05-15T16:31:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.375+0000 7fe495d9b700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe48406e200 0x7fe4840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.375+0000 7fe495d9b700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe48406e200 0x7fe4840706c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe480005fd0 tx=0x7fe480005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:16.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.379+0000 7fe48f7fe700 1 -- 172.21.15.49:0/3344358611 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe488058940 con 0x7fe498100ee0 2024-05-15T16:31:16.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:16 smithi144 ceph-mon[86635]: pgmap v1028: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.674+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe49810deb0 con 0x7fe498100ee0 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.675+0000 7fe48f7fe700 1 -- 172.21.15.49:0/3344358611 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe48805c000 con 0x7fe498100ee0 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:31:16.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:31:16.677 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:31:16.677 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:31:16.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:31:16.677 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:31:16.677 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:31:16.677 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:31:16.677 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe48406e200 msgr2=0x7fe4840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe48406e200 0x7fe4840706c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe480005fd0 tx=0x7fe480005dc0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 msgr2=0x7fe4980732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 0x7fe4980732a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe48800bd70 tx=0x7fe48800be50 comp rx=0 tx=0).stop 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 shutdown_connections 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fe48406e200 0x7fe4840706c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe498100ee0 0x7fe4980732a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 --2- 172.21.15.49:0/3344358611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe498101890 0x7fe4980737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 >> 172.21.15.49:0/3344358611 conn(0x7fe4980fc790 msgr2=0x7fe4980fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 shutdown_connections 2024-05-15T16:31:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.677+0000 7fe49cb4b700 1 -- 172.21.15.49:0/3344358611 wait complete. 2024-05-15T16:31:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.785+0000 7f967394d700 1 -- 172.21.15.49:0/2491162760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c0ffea0 msgr2=0x7f966c100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.785+0000 7f967394d700 1 --2- 172.21.15.49:0/2491162760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c0ffea0 0x7f966c100280 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9668009910 tx=0x7f9668009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.785+0000 7f967394d700 1 -- 172.21.15.49:0/2491162760 shutdown_connections 2024-05-15T16:31:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.785+0000 7f967394d700 1 --2- 172.21.15.49:0/2491162760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966c100850 0x7f966c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.785+0000 7f967394d700 1 --2- 172.21.15.49:0/2491162760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c0ffea0 0x7f966c100280 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.785+0000 7f967394d700 1 -- 172.21.15.49:0/2491162760 >> 172.21.15.49:0/2491162760 conn(0x7f966c0fb750 msgr2=0x7f966c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.786+0000 7f967394d700 1 -- 172.21.15.49:0/2491162760 shutdown_connections 2024-05-15T16:31:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.786+0000 7f967394d700 1 -- 172.21.15.49:0/2491162760 wait complete. 2024-05-15T16:31:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.786+0000 7f967394d700 1 Processor -- start 2024-05-15T16:31:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.786+0000 7f967394d700 1 -- start start 2024-05-15T16:31:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.786+0000 7f967394d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966c0ffea0 0x7f966c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f967394d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c100850 0x7f966c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f967394d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c196af0 con 0x7f966c100850 2024-05-15T16:31:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f967394d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c196c60 con 0x7f966c0ffea0 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f9670ee8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c100850 0x7f966c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f9670ee8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c100850 0x7f966c19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35034/0 (socket says 172.21.15.49:35034) 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f9670ee8700 1 -- 172.21.15.49:0/1333224381 learned_addr learned my addr 172.21.15.49:0/1333224381 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f96716e9700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966c0ffea0 0x7f966c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f9670ee8700 1 -- 172.21.15.49:0/1333224381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966c0ffea0 msgr2=0x7f966c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f9670ee8700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966c0ffea0 0x7f966c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f9670ee8700 1 -- 172.21.15.49:0/1333224381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96680095f0 con 0x7f966c100850 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f9670ee8700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c100850 0x7f966c19cf80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b480 tx=0x7f965c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f96627fc700 1 -- 172.21.15.49:0/1333224381 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f965c0118c0 con 0x7f966c100850 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f96627fc700 1 -- 172.21.15.49:0/1333224381 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f965c010380 con 0x7f966c100850 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966c196f40 con 0x7f966c100850 2024-05-15T16:31:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f96627fc700 1 -- 172.21.15.49:0/1333224381 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f965c00f4f0 con 0x7f966c100850 2024-05-15T16:31:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.787+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966c197490 con 0x7f966c100850 2024-05-15T16:31:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.788+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966c101380 con 0x7f966c100850 2024-05-15T16:31:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.789+0000 7f96627fc700 1 -- 172.21.15.49:0/1333224381 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f965c0104f0 con 0x7f966c100850 2024-05-15T16:31:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.789+0000 7f96627fc700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f965806e200 0x7f96580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.790+0000 7f96627fc700 1 -- 172.21.15.49:0/1333224381 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f965c08dd20 con 0x7f966c100850 2024-05-15T16:31:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.790+0000 7f96716e9700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f965806e200 0x7f96580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.790+0000 7f96716e9700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f965806e200 0x7f96580706c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f966800b5c0 tx=0x7f9668005380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:16.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:16.794+0000 7f96627fc700 1 -- 172.21.15.49:0/1333224381 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f965c057080 con 0x7f966c100850 2024-05-15T16:31:17.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.048+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f966c197760 con 0x7f966c100850 2024-05-15T16:31:17.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.048+0000 7f96627fc700 1 -- 172.21.15.49:0/1333224381 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f965c015760 con 0x7f966c100850 2024-05-15T16:31:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:31:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:31:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:31:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:31:17.050 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:31:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:31:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:31:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:31:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:31:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:31:17.053 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:31:17.053 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:31:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:31:17.053 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:31:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:31:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:31:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:31:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:31:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.050+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f965806e200 msgr2=0x7f96580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.050+0000 7f967394d700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f965806e200 0x7f96580706c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f966800b5c0 tx=0x7f9668005380 comp rx=0 tx=0).stop 2024-05-15T16:31:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.050+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c100850 msgr2=0x7f966c19cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.050+0000 7f967394d700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c100850 0x7f966c19cf80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b480 tx=0x7f965c00b840 comp rx=0 tx=0).stop 2024-05-15T16:31:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.050+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 shutdown_connections 2024-05-15T16:31:17.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.050+0000 7f967394d700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f965806e200 0x7f96580706c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.050+0000 7f967394d700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966c0ffea0 0x7f966c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.051+0000 7f967394d700 1 --2- 172.21.15.49:0/1333224381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f966c100850 0x7f966c19cf80 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.051+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 >> 172.21.15.49:0/1333224381 conn(0x7f966c0fb750 msgr2=0x7f966c0fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:17.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.051+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 shutdown_connections 2024-05-15T16:31:17.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.051+0000 7f967394d700 1 -- 172.21.15.49:0/1333224381 wait complete. 2024-05-15T16:31:17.055 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.161+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3661058017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e78100770 msgr2=0x7f8e781047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.161+0000 7f8e7ded9700 1 --2- 172.21.15.49:0/3661058017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e78100770 0x7f8e781047c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009910 tx=0x7f8e60009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.161+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3661058017 shutdown_connections 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.161+0000 7f8e7ded9700 1 --2- 172.21.15.49:0/3661058017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e78100770 0x7f8e781047c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.161+0000 7f8e7ded9700 1 --2- 172.21.15.49:0/3661058017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e780ffdc0 0x7f8e781001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.161+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3661058017 >> 172.21.15.49:0/3661058017 conn(0x7f8e78076800 msgr2=0x7f8e780fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.161+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3661058017 shutdown_connections 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.161+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3661058017 wait complete. 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.162+0000 7f8e7ded9700 1 Processor -- start 2024-05-15T16:31:17.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.162+0000 7f8e7ded9700 1 -- start start 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.162+0000 7f8e7ded9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e780ffdc0 0x7f8e7819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.162+0000 7f8e7ded9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e78100770 0x7f8e7819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.162+0000 7f8e7ded9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7819d3c0 con 0x7f8e780ffdc0 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.162+0000 7f8e7ded9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781969f0 con 0x7f8e78100770 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.162+0000 7f8e777fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e780ffdc0 0x7f8e7819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.162+0000 7f8e777fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e780ffdc0 0x7f8e7819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35052/0 (socket says 172.21.15.49:35052) 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.162+0000 7f8e777fe700 1 -- 172.21.15.49:0/3194539014 learned_addr learned my addr 172.21.15.49:0/3194539014 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e76ffd700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e78100770 0x7f8e7819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e777fe700 1 -- 172.21.15.49:0/3194539014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e78100770 msgr2=0x7f8e7819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e777fe700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e78100770 0x7f8e7819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e777fe700 1 -- 172.21.15.49:0/3194539014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e600095f0 con 0x7f8e780ffdc0 2024-05-15T16:31:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e777fe700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e780ffdc0 0x7f8e7819c860 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8e680060b0 tx=0x7f8e6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e74ff9700 1 -- 172.21.15.49:0/3194539014 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e6800dda0 con 0x7f8e780ffdc0 2024-05-15T16:31:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e74ff9700 1 -- 172.21.15.49:0/3194539014 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e6800f040 con 0x7f8e780ffdc0 2024-05-15T16:31:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e78196cd0 con 0x7f8e780ffdc0 2024-05-15T16:31:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e74ff9700 1 -- 172.21.15.49:0/3194539014 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e68014790 con 0x7f8e780ffdc0 2024-05-15T16:31:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.163+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e78197220 con 0x7f8e780ffdc0 2024-05-15T16:31:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.164+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e781012a0 con 0x7f8e780ffdc0 2024-05-15T16:31:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.165+0000 7f8e74ff9700 1 -- 172.21.15.49:0/3194539014 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8e68014980 con 0x7f8e780ffdc0 2024-05-15T16:31:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.165+0000 7f8e74ff9700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e6406e210 0x7f8e640706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.165+0000 7f8e74ff9700 1 -- 172.21.15.49:0/3194539014 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8e6808dc20 con 0x7f8e780ffdc0 2024-05-15T16:31:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.165+0000 7f8e76ffd700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e6406e210 0x7f8e640706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.166+0000 7f8e76ffd700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e6406e210 0x7f8e640706d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8e78197b60 tx=0x7f8e6001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:17.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.170+0000 7f8e74ff9700 1 -- 172.21.15.49:0/3194539014 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8e6805aae0 con 0x7f8e780ffdc0 2024-05-15T16:31:17.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.397+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e7810ce20 con 0x7f8e6406e210 2024-05-15T16:31:17.399 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:17 smithi049 ceph-mon[112142]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:17.399 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:17 smithi049 ceph-mon[112142]: from='client.35862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:17.399 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:17 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3344358611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:31:17.399 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:17 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1333224381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:31:17.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.399+0000 7f8e74ff9700 1 -- 172.21.15.49:0/3194539014 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8e7810ce20 con 0x7f8e6406e210 2024-05-15T16:31:17.400 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:31:17.400 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:31:17.400 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:31:17.400 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:31:17.401 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:31:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e6406e210 msgr2=0x7f8e640706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e6406e210 0x7f8e640706d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8e78197b60 tx=0x7f8e6001a040 comp rx=0 tx=0).stop 2024-05-15T16:31:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e780ffdc0 msgr2=0x7f8e7819c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e780ffdc0 0x7f8e7819c860 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8e680060b0 tx=0x7f8e6800d3b0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 shutdown_connections 2024-05-15T16:31:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e6406e210 0x7f8e640706d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e780ffdc0 0x7f8e7819c860 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 --2- 172.21.15.49:0/3194539014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e78100770 0x7f8e7819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 >> 172.21.15.49:0/3194539014 conn(0x7f8e78076800 msgr2=0x7f8e78068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 shutdown_connections 2024-05-15T16:31:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.401+0000 7f8e7ded9700 1 -- 172.21.15.49:0/3194539014 wait complete. 2024-05-15T16:31:17.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:17 smithi144 ceph-mon[86635]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:17.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:17 smithi144 ceph-mon[86635]: from='client.35862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:17.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:17 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3344358611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:31:17.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:17 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1333224381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:31:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.510+0000 7f4c98c73700 1 -- 172.21.15.49:0/1975292123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94101890 msgr2=0x7f4c941058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.510+0000 7f4c98c73700 1 --2- 172.21.15.49:0/1975292123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94101890 0x7f4c941058e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009910 tx=0x7f4c88009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.510+0000 7f4c98c73700 1 -- 172.21.15.49:0/1975292123 shutdown_connections 2024-05-15T16:31:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.510+0000 7f4c98c73700 1 --2- 172.21.15.49:0/1975292123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94101890 0x7f4c941058e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.510+0000 7f4c98c73700 1 --2- 172.21.15.49:0/1975292123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c94100ee0 0x7f4c941012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.510+0000 7f4c98c73700 1 -- 172.21.15.49:0/1975292123 >> 172.21.15.49:0/1975292123 conn(0x7f4c940fc790 msgr2=0x7f4c940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.510+0000 7f4c98c73700 1 -- 172.21.15.49:0/1975292123 shutdown_connections 2024-05-15T16:31:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.510+0000 7f4c98c73700 1 -- 172.21.15.49:0/1975292123 wait complete. 2024-05-15T16:31:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c98c73700 1 Processor -- start 2024-05-15T16:31:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c98c73700 1 -- start start 2024-05-15T16:31:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c98c73700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94100ee0 0x7f4c9419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c98c73700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c94101890 0x7f4c9419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c98c73700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9419d440 con 0x7f4c94100ee0 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c98c73700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94196a20 con 0x7f4c94101890 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c9259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94100ee0 0x7f4c9419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c9259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94100ee0 0x7f4c9419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35076/0 (socket says 172.21.15.49:35076) 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c9259c700 1 -- 172.21.15.49:0/3687374722 learned_addr learned my addr 172.21.15.49:0/3687374722 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.511+0000 7f4c91d9b700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c94101890 0x7f4c9419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.512+0000 7f4c9259c700 1 -- 172.21.15.49:0/3687374722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c94101890 msgr2=0x7f4c9419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.512+0000 7f4c9259c700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c94101890 0x7f4c9419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.512+0000 7f4c9259c700 1 -- 172.21.15.49:0/3687374722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880095f0 con 0x7f4c94100ee0 2024-05-15T16:31:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.512+0000 7f4c9259c700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94100ee0 0x7f4c9419c8e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d8d0 tx=0x7f4c8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.512+0000 7f4c837fe700 1 -- 172.21.15.49:0/3687374722 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c84009940 con 0x7f4c94100ee0 2024-05-15T16:31:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.512+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c94196d00 con 0x7f4c94100ee0 2024-05-15T16:31:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.512+0000 7f4c837fe700 1 -- 172.21.15.49:0/3687374722 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c84010460 con 0x7f4c94100ee0 2024-05-15T16:31:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.512+0000 7f4c837fe700 1 -- 172.21.15.49:0/3687374722 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8400f5f0 con 0x7f4c94100ee0 2024-05-15T16:31:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.512+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c94197250 con 0x7f4c94100ee0 2024-05-15T16:31:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.513+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c941092e0 con 0x7f4c94100ee0 2024-05-15T16:31:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.514+0000 7f4c837fe700 1 -- 172.21.15.49:0/3687374722 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4c840105d0 con 0x7f4c94100ee0 2024-05-15T16:31:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.514+0000 7f4c837fe700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c7c06e200 0x7f4c7c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.514+0000 7f4c837fe700 1 -- 172.21.15.49:0/3687374722 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4c8408e130 con 0x7f4c94100ee0 2024-05-15T16:31:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.515+0000 7f4c91d9b700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c7c06e200 0x7f4c7c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:17.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.515+0000 7f4c91d9b700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c7c06e200 0x7f4c7c0706c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c94197be0 tx=0x7f4c8801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:17.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.518+0000 7f4c837fe700 1 -- 172.21.15.49:0/3687374722 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4c8405b070 con 0x7f4c94100ee0 2024-05-15T16:31:17.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.809+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4c941023c0 con 0x7f4c94100ee0 2024-05-15T16:31:17.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.810+0000 7f4c837fe700 1 -- 172.21.15.49:0/3687374722 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f4c8405ac00 con 0x7f4c94100ee0 2024-05-15T16:31:17.811 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:31:17.811 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:31:17.811 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:31:17.812 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:31:17.813 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:31:17.813 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:31:17.813 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:31:17.813 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:31:17.813 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:31:17.813 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:31:17.813 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:31:17.813 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:31:17.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c7c06e200 msgr2=0x7f4c7c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c7c06e200 0x7f4c7c0706c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c94197be0 tx=0x7f4c8801a040 comp rx=0 tx=0).stop 2024-05-15T16:31:17.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94100ee0 msgr2=0x7f4c9419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:17.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94100ee0 0x7f4c9419c8e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d8d0 tx=0x7f4c8400dc90 comp rx=0 tx=0).stop 2024-05-15T16:31:17.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 shutdown_connections 2024-05-15T16:31:17.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c7c06e200 0x7f4c7c0706c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c94100ee0 0x7f4c9419c8e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 --2- 172.21.15.49:0/3687374722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c94101890 0x7f4c9419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:17.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 >> 172.21.15.49:0/3687374722 conn(0x7f4c940fc790 msgr2=0x7f4c940fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:17.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 shutdown_connections 2024-05-15T16:31:17.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:17.812+0000 7f4c98c73700 1 -- 172.21.15.49:0/3687374722 wait complete. 2024-05-15T16:31:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:18 smithi049 ceph-mon[112142]: from='client.35866 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:18 smithi049 ceph-mon[112142]: pgmap v1029: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:18 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3687374722' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:31:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:18 smithi144 ceph-mon[86635]: from='client.35866 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:18 smithi144 ceph-mon[86635]: pgmap v1029: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:18 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3687374722' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:31:19.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:19 smithi144 ceph-mon[86635]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:19.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:19 smithi049 ceph-mon[112142]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:20 smithi144 ceph-mon[86635]: pgmap v1030: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:20 smithi049 ceph-mon[112142]: pgmap v1030: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:31:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:31:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:22 smithi144 ceph-mon[86635]: pgmap v1031: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:22 smithi049 ceph-mon[112142]: pgmap v1031: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:24 smithi144 ceph-mon[86635]: pgmap v1032: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:24 smithi049 ceph-mon[112142]: pgmap v1032: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:26 smithi144 ceph-mon[86635]: pgmap v1033: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:26 smithi049 ceph-mon[112142]: pgmap v1033: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:28 smithi144 ceph-mon[86635]: pgmap v1034: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:28 smithi049 ceph-mon[112142]: pgmap v1034: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:30 smithi144 ceph-mon[86635]: pgmap v1035: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:30 smithi049 ceph-mon[112142]: pgmap v1035: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:32 smithi144 ceph-mon[86635]: pgmap v1036: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:32 smithi049 ceph-mon[112142]: pgmap v1036: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:34 smithi144 ceph-mon[86635]: pgmap v1037: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:34 smithi049 ceph-mon[112142]: pgmap v1037: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:36 smithi049 ceph-mon[112142]: pgmap v1038: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:31:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:36 smithi144 ceph-mon[86635]: pgmap v1038: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:31:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:38 smithi144 ceph-mon[86635]: pgmap v1039: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:38 smithi049 ceph-mon[112142]: pgmap v1039: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:40 smithi144 ceph-mon[86635]: pgmap v1040: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:40 smithi049 ceph-mon[112142]: pgmap v1040: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:42 smithi144 ceph-mon[86635]: pgmap v1041: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:42 smithi049 ceph-mon[112142]: pgmap v1041: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:44 smithi144 ceph-mon[86635]: pgmap v1042: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:44 smithi049 ceph-mon[112142]: pgmap v1042: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:46 smithi144 ceph-mon[86635]: pgmap v1043: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:46 smithi049 ceph-mon[112142]: pgmap v1043: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:47.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/2641217014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 msgr2=0x7f6a641012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 --2- 172.21.15.49:0/2641217014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 0x7f6a641012c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009910 tx=0x7f6a54009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/2641217014 shutdown_connections 2024-05-15T16:31:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 --2- 172.21.15.49:0/2641217014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64101890 0x7f6a641058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 --2- 172.21.15.49:0/2641217014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 0x7f6a641012c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/2641217014 >> 172.21.15.49:0/2641217014 conn(0x7f6a640fc790 msgr2=0x7f6a640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/2641217014 shutdown_connections 2024-05-15T16:31:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/2641217014 wait complete. 2024-05-15T16:31:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 Processor -- start 2024-05-15T16:31:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.937+0000 7f6a6a7a0700 1 -- start start 2024-05-15T16:31:47.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a6a7a0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 0x7f6a6419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a6a7a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64101890 0x7f6a6419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a6a7a0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6419d460 con 0x7f6a64100ee0 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a6a7a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a64196a90 con 0x7f6a64101890 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a63fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 0x7f6a6419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a63fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 0x7f6a6419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57246/0 (socket says 172.21.15.49:57246) 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a63fff700 1 -- 172.21.15.49:0/1157096501 learned_addr learned my addr 172.21.15.49:0/1157096501 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a637fe700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64101890 0x7f6a6419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a63fff700 1 -- 172.21.15.49:0/1157096501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64101890 msgr2=0x7f6a6419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a63fff700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64101890 0x7f6a6419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.938+0000 7f6a63fff700 1 -- 172.21.15.49:0/1157096501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a540095f0 con 0x7f6a64100ee0 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.939+0000 7f6a63fff700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 0x7f6a6419c900 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400bd70 tx=0x7f6a5400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.939+0000 7f6a617fa700 1 -- 172.21.15.49:0/1157096501 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a5401d070 con 0x7f6a64100ee0 2024-05-15T16:31:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.939+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a64196d10 con 0x7f6a64100ee0 2024-05-15T16:31:47.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.939+0000 7f6a617fa700 1 -- 172.21.15.49:0/1157096501 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a540056e0 con 0x7f6a64100ee0 2024-05-15T16:31:47.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.939+0000 7f6a617fa700 1 -- 172.21.15.49:0/1157096501 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a54017400 con 0x7f6a64100ee0 2024-05-15T16:31:47.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.939+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a64197200 con 0x7f6a64100ee0 2024-05-15T16:31:47.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.940+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a64109280 con 0x7f6a64100ee0 2024-05-15T16:31:47.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.941+0000 7f6a617fa700 1 -- 172.21.15.49:0/1157096501 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6a54003a60 con 0x7f6a64100ee0 2024-05-15T16:31:47.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.941+0000 7f6a617fa700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6a4c06e200 0x7f6a4c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:47.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.941+0000 7f6a617fa700 1 -- 172.21.15.49:0/1157096501 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6a5408f570 con 0x7f6a64100ee0 2024-05-15T16:31:47.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.941+0000 7f6a637fe700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6a4c06e200 0x7f6a4c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:47.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.942+0000 7f6a637fe700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6a4c06e200 0x7f6a4c0706c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005950 tx=0x7f6a580058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:47.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:47.946+0000 7f6a617fa700 1 -- 172.21.15.49:0/1157096501 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6a5405c400 con 0x7f6a64100ee0 2024-05-15T16:31:48.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.174+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a64197f70 con 0x7f6a4c06e200 2024-05-15T16:31:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.175+0000 7f6a617fa700 1 -- 172.21.15.49:0/1157096501 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6a64197f70 con 0x7f6a4c06e200 2024-05-15T16:31:48.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6a4c06e200 msgr2=0x7f6a4c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6a4c06e200 0x7f6a4c0706c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005950 tx=0x7f6a580058e0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 msgr2=0x7f6a6419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 0x7f6a6419c900 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400bd70 tx=0x7f6a5400be50 comp rx=0 tx=0).stop 2024-05-15T16:31:48.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 shutdown_connections 2024-05-15T16:31:48.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6a4c06e200 0x7f6a4c0706c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a64100ee0 0x7f6a6419c900 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 --2- 172.21.15.49:0/1157096501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64101890 0x7f6a6419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 >> 172.21.15.49:0/1157096501 conn(0x7f6a640fc790 msgr2=0x7f6a640fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:48.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 shutdown_connections 2024-05-15T16:31:48.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.178+0000 7f6a6a7a0700 1 -- 172.21.15.49:0/1157096501 wait complete. 2024-05-15T16:31:48.190 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:31:48.190 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:48 smithi049 ceph-mon[112142]: pgmap v1044: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.287+0000 7f13a9334700 1 -- 172.21.15.49:0/1574692843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 msgr2=0x7f13a41047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.287+0000 7f13a9334700 1 --2- 172.21.15.49:0/1574692843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 0x7f13a41047f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1398009910 tx=0x7f1398009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.287+0000 7f13a9334700 1 -- 172.21.15.49:0/1574692843 shutdown_connections 2024-05-15T16:31:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.287+0000 7f13a9334700 1 --2- 172.21.15.49:0/1574692843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 0x7f13a41047f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.287+0000 7f13a9334700 1 --2- 172.21.15.49:0/1574692843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a40ffdf0 0x7f13a41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.287+0000 7f13a9334700 1 -- 172.21.15.49:0/1574692843 >> 172.21.15.49:0/1574692843 conn(0x7f13a40fb6a0 msgr2=0x7f13a40fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.287+0000 7f13a9334700 1 -- 172.21.15.49:0/1574692843 shutdown_connections 2024-05-15T16:31:48.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.287+0000 7f13a9334700 1 -- 172.21.15.49:0/1574692843 wait complete. 2024-05-15T16:31:48.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.288+0000 7f13a9334700 1 Processor -- start 2024-05-15T16:31:48.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.288+0000 7f13a9334700 1 -- start start 2024-05-15T16:31:48.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.288+0000 7f13a9334700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a40ffdf0 0x7f13a419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:48.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.288+0000 7f13a9334700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 0x7f13a419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:48.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.288+0000 7f13a9334700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a419d3e0 con 0x7f13a41007a0 2024-05-15T16:31:48.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.288+0000 7f13a9334700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a4196a50 con 0x7f13a40ffdf0 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 0x7f13a419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 0x7f13a419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57264/0 (socket says 172.21.15.49:57264) 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a27fc700 1 -- 172.21.15.49:0/4172388025 learned_addr learned my addr 172.21.15.49:0/4172388025 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a2ffd700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a40ffdf0 0x7f13a419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a27fc700 1 -- 172.21.15.49:0/4172388025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a40ffdf0 msgr2=0x7f13a419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a27fc700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a40ffdf0 0x7f13a419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a27fc700 1 -- 172.21.15.49:0/4172388025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13980095f0 con 0x7f13a41007a0 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a27fc700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 0x7f13a419ce50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1398005b40 tx=0x7f139800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f138bfff700 1 -- 172.21.15.49:0/4172388025 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f139801d070 con 0x7f13a41007a0 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f138bfff700 1 -- 172.21.15.49:0/4172388025 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f139800f460 con 0x7f13a41007a0 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a4196cd0 con 0x7f13a41007a0 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f138bfff700 1 -- 172.21.15.49:0/4172388025 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1398017630 con 0x7f13a41007a0 2024-05-15T16:31:48.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.289+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a41971c0 con 0x7f13a41007a0 2024-05-15T16:31:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.291+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a4109200 con 0x7f13a41007a0 2024-05-15T16:31:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.291+0000 7f138bfff700 1 -- 172.21.15.49:0/4172388025 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f139800f5d0 con 0x7f13a41007a0 2024-05-15T16:31:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.291+0000 7f138bfff700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f138c06e210 0x7f138c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.292+0000 7f13a2ffd700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f138c06e210 0x7f138c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.292+0000 7f138bfff700 1 -- 172.21.15.49:0/4172388025 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f139808f6e0 con 0x7f13a41007a0 2024-05-15T16:31:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.292+0000 7f13a2ffd700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f138c06e210 0x7f138c0706d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1394005ea0 tx=0x7f1394005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:48.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.296+0000 7f138bfff700 1 -- 172.21.15.49:0/4172388025 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f139805c4f0 con 0x7f13a41007a0 2024-05-15T16:31:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:48 smithi144 ceph-mon[86635]: pgmap v1044: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:48.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.523+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13a4197f10 con 0x7f138c06e210 2024-05-15T16:31:48.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.525+0000 7f138bfff700 1 -- 172.21.15.49:0/4172388025 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f13a4197f10 con 0x7f138c06e210 2024-05-15T16:31:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f138c06e210 msgr2=0x7f138c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f138c06e210 0x7f138c0706d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1394005ea0 tx=0x7f1394005e30 comp rx=0 tx=0).stop 2024-05-15T16:31:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 msgr2=0x7f13a419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 0x7f13a419ce50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1398005b40 tx=0x7f139800be20 comp rx=0 tx=0).stop 2024-05-15T16:31:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 shutdown_connections 2024-05-15T16:31:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f138c06e210 0x7f138c0706d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a40ffdf0 0x7f13a419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 --2- 172.21.15.49:0/4172388025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13a41007a0 0x7f13a419ce50 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 >> 172.21.15.49:0/4172388025 conn(0x7f13a40fb6a0 msgr2=0x7f13a40697c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:48.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 shutdown_connections 2024-05-15T16:31:48.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.527+0000 7f13a9334700 1 -- 172.21.15.49:0/4172388025 wait complete. 2024-05-15T16:31:48.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.637+0000 7f82c7cdc700 1 -- 172.21.15.49:0/1351333753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c00ff110 msgr2=0x7f82c00ff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.637+0000 7f82c7cdc700 1 --2- 172.21.15.49:0/1351333753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c00ff110 0x7f82c00ff590 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009910 tx=0x7f82b0009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:48.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.637+0000 7f82c7cdc700 1 -- 172.21.15.49:0/1351333753 shutdown_connections 2024-05-15T16:31:48.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.637+0000 7f82c7cdc700 1 --2- 172.21.15.49:0/1351333753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c00ff110 0x7f82c00ff590 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.637+0000 7f82c7cdc700 1 --2- 172.21.15.49:0/1351333753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c0105cf0 0x7f82c01060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.637+0000 7f82c7cdc700 1 -- 172.21.15.49:0/1351333753 >> 172.21.15.49:0/1351333753 conn(0x7f82c0076000 msgr2=0x7f82c00fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:48.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.637+0000 7f82c7cdc700 1 -- 172.21.15.49:0/1351333753 shutdown_connections 2024-05-15T16:31:48.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.637+0000 7f82c7cdc700 1 -- 172.21.15.49:0/1351333753 wait complete. 2024-05-15T16:31:48.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.638+0000 7f82c7cdc700 1 Processor -- start 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.638+0000 7f82c7cdc700 1 -- start start 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.638+0000 7f82c7cdc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c00ff110 0x7f82c01002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.638+0000 7f82c7cdc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c0105cf0 0x7f82c0100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.638+0000 7f82c7cdc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0100d70 con 0x7f82c0105cf0 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.638+0000 7f82c7cdc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0100ee0 con 0x7f82c00ff110 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.638+0000 7f82c5277700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c0105cf0 0x7f82c0100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82c5277700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c0105cf0 0x7f82c0100830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57280/0 (socket says 172.21.15.49:57280) 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82c5277700 1 -- 172.21.15.49:0/633414295 learned_addr learned my addr 172.21.15.49:0/633414295 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82c5a78700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c00ff110 0x7f82c01002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82c5277700 1 -- 172.21.15.49:0/633414295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c00ff110 msgr2=0x7f82c01002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82c5277700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c00ff110 0x7f82c01002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82c5277700 1 -- 172.21.15.49:0/633414295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00095f0 con 0x7f82c0105cf0 2024-05-15T16:31:48.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82c5277700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c0105cf0 0x7f82c0100830 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f82b000bd70 tx=0x7f82b000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:48.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82b6ffd700 1 -- 172.21.15.49:0/633414295 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b001d070 con 0x7f82c0105cf0 2024-05-15T16:31:48.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82b6ffd700 1 -- 172.21.15.49:0/633414295 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82b00056e0 con 0x7f82c0105cf0 2024-05-15T16:31:48.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c0101160 con 0x7f82c0105cf0 2024-05-15T16:31:48.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82b6ffd700 1 -- 172.21.15.49:0/633414295 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b000fe40 con 0x7f82c0105cf0 2024-05-15T16:31:48.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.639+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c01a3100 con 0x7f82c0105cf0 2024-05-15T16:31:48.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.640+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c0109270 con 0x7f82c0105cf0 2024-05-15T16:31:48.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.641+0000 7f82b6ffd700 1 -- 172.21.15.49:0/633414295 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f82b000f680 con 0x7f82c0105cf0 2024-05-15T16:31:48.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.641+0000 7f82b6ffd700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f82ac06e1a0 0x7f82ac070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:48.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.641+0000 7f82b6ffd700 1 -- 172.21.15.49:0/633414295 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f82b008f3c0 con 0x7f82c0105cf0 2024-05-15T16:31:48.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.641+0000 7f82c5a78700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f82ac06e1a0 0x7f82ac070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:48.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.642+0000 7f82c5a78700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f82ac06e1a0 0x7f82ac070660 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009550 tx=0x7f82bc0091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:48.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.646+0000 7f82b6ffd700 1 -- 172.21.15.49:0/633414295 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f82b00586a0 con 0x7f82c0105cf0 2024-05-15T16:31:48.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.863+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f82c01023f0 con 0x7f82ac06e1a0 2024-05-15T16:31:48.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.876+0000 7f82b6ffd700 1 -- 172.21.15.49:0/633414295 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f82bc00f040 con 0x7f82ac06e1a0 2024-05-15T16:31:48.877 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:31:48.877 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (35m) 12m ago 46m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:31:48.877 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (46m) 12m ago 46m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:31:48.877 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (45m) 12m ago 45m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:31:48.877 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (32m) 12m ago 46m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:31:48.877 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (32m) 12m ago 45m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:31:48.877 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (34m) 12m ago 46m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:31:48.877 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (28m) 12m ago 41m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (28m) 12m ago 40m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (28m) 12m ago 41m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (12m) 12m ago 40m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (37m) 12m ago 48m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (36m) 12m ago 45m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (33m) 12m ago 48m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (33m) 12m ago 45m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (35m) 12m ago 46m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (35m) 12m ago 45m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (31m) 12m ago 44m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (31m) 12m ago 44m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (30m) 12m ago 43m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (30m) 12m ago 43m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:31:48.878 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (29m) 12m ago 42m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:31:48.879 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (29m) 12m ago 42m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:31:48.879 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (35m) 12m ago 46m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:31:48.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f82ac06e1a0 msgr2=0x7f82ac070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f82ac06e1a0 0x7f82ac070660 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009550 tx=0x7f82bc0091b0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c0105cf0 msgr2=0x7f82c0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c0105cf0 0x7f82c0100830 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f82b000bd70 tx=0x7f82b000bda0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 shutdown_connections 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f82ac06e1a0 0x7f82ac070660 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c00ff110 0x7f82c01002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 --2- 172.21.15.49:0/633414295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82c0105cf0 0x7f82c0100830 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 >> 172.21.15.49:0/633414295 conn(0x7f82c0076000 msgr2=0x7f82c00fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 shutdown_connections 2024-05-15T16:31:48.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.878+0000 7f82c7cdc700 1 -- 172.21.15.49:0/633414295 wait complete. 2024-05-15T16:31:48.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.984+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/3480661824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44100ee0 msgr2=0x7f5d441012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.984+0000 7f5d4a0b2700 1 --2- 172.21.15.49:0/3480661824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44100ee0 0x7f5d441012c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009910 tx=0x7f5d34009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.985+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/3480661824 shutdown_connections 2024-05-15T16:31:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.985+0000 7f5d4a0b2700 1 --2- 172.21.15.49:0/3480661824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d44101890 0x7f5d441058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.985+0000 7f5d4a0b2700 1 --2- 172.21.15.49:0/3480661824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44100ee0 0x7f5d441012c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.985+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/3480661824 >> 172.21.15.49:0/3480661824 conn(0x7f5d440fc790 msgr2=0x7f5d440febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.985+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/3480661824 shutdown_connections 2024-05-15T16:31:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.985+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/3480661824 wait complete. 2024-05-15T16:31:48.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.985+0000 7f5d4a0b2700 1 Processor -- start 2024-05-15T16:31:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d4a0b2700 1 -- start start 2024-05-15T16:31:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d4a0b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d44100ee0 0x7f5d4419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d4a0b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44101890 0x7f5d4419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d4a0b2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4419d460 con 0x7f5d44101890 2024-05-15T16:31:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d4a0b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d44196a90 con 0x7f5d44100ee0 2024-05-15T16:31:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d42ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44101890 0x7f5d4419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d42ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44101890 0x7f5d4419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57302/0 (socket says 172.21.15.49:57302) 2024-05-15T16:31:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d42ffd700 1 -- 172.21.15.49:0/1473677983 learned_addr learned my addr 172.21.15.49:0/1473677983 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d437fe700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d44100ee0 0x7f5d4419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d42ffd700 1 -- 172.21.15.49:0/1473677983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d44100ee0 msgr2=0x7f5d4419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d42ffd700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d44100ee0 0x7f5d4419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.986+0000 7f5d42ffd700 1 -- 172.21.15.49:0/1473677983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d340095f0 con 0x7f5d44101890 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.987+0000 7f5d42ffd700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44101890 0x7f5d4419ce40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00d900 tx=0x7f5d2c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.987+0000 7f5d40ff9700 1 -- 172.21.15.49:0/1473677983 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2c009900 con 0x7f5d44101890 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.987+0000 7f5d40ff9700 1 -- 172.21.15.49:0/1473677983 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d2c010460 con 0x7f5d44101890 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.987+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d44196d70 con 0x7f5d44101890 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.987+0000 7f5d40ff9700 1 -- 172.21.15.49:0/1473677983 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2c00f5f0 con 0x7f5d44101890 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.987+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d441972c0 con 0x7f5d44101890 2024-05-15T16:31:48.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.988+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d44109280 con 0x7f5d44101890 2024-05-15T16:31:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.988+0000 7f5d40ff9700 1 -- 172.21.15.49:0/1473677983 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5d2c0105d0 con 0x7f5d44101890 2024-05-15T16:31:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.989+0000 7f5d40ff9700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d3006e210 0x7f5d300706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.989+0000 7f5d40ff9700 1 -- 172.21.15.49:0/1473677983 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5d2c08dc40 con 0x7f5d44101890 2024-05-15T16:31:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.989+0000 7f5d437fe700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d3006e210 0x7f5d300706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:48.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.989+0000 7f5d437fe700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d3006e210 0x7f5d300706d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d34006010 tx=0x7f5d3401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:48.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:48.994+0000 7f5d40ff9700 1 -- 172.21.15.49:0/1473677983 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5d2c05aad0 con 0x7f5d44101890 2024-05-15T16:31:49.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.292+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5d441023c0 con 0x7f5d44101890 2024-05-15T16:31:49.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.292+0000 7f5d40ff9700 1 -- 172.21.15.49:0/1473677983 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5d2c017280 con 0x7f5d44101890 2024-05-15T16:31:49.293 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:31:49.293 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:31:49.294 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:31:49.295 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:31:49.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d3006e210 msgr2=0x7f5d300706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d3006e210 0x7f5d300706d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d34006010 tx=0x7f5d3401a040 comp rx=0 tx=0).stop 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44101890 msgr2=0x7f5d4419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44101890 0x7f5d4419ce40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00d900 tx=0x7f5d2c00dc10 comp rx=0 tx=0).stop 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 shutdown_connections 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d3006e210 0x7f5d300706d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d44100ee0 0x7f5d4419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 --2- 172.21.15.49:0/1473677983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44101890 0x7f5d4419ce40 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 >> 172.21.15.49:0/1473677983 conn(0x7f5d440fc790 msgr2=0x7f5d440fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 shutdown_connections 2024-05-15T16:31:49.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.294+0000 7f5d4a0b2700 1 -- 172.21.15.49:0/1473677983 wait complete. 2024-05-15T16:31:49.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.402+0000 7f40fcf9a700 1 -- 172.21.15.49:0/1083779524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 msgr2=0x7f40f81047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:49.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.402+0000 7f40fcf9a700 1 --2- 172.21.15.49:0/1083779524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 0x7f40f81047c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009910 tx=0x7f40ec009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:49.404 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:49 smithi049 ceph-mon[112142]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.402+0000 7f40fcf9a700 1 -- 172.21.15.49:0/1083779524 shutdown_connections 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.402+0000 7f40fcf9a700 1 --2- 172.21.15.49:0/1083779524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 0x7f40f81047c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.402+0000 7f40fcf9a700 1 --2- 172.21.15.49:0/1083779524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f80ffdc0 0x7f40f81001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.402+0000 7f40fcf9a700 1 -- 172.21.15.49:0/1083779524 >> 172.21.15.49:0/1083779524 conn(0x7f40f8076800 msgr2=0x7f40f80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.402+0000 7f40fcf9a700 1 -- 172.21.15.49:0/1083779524 shutdown_connections 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.402+0000 7f40fcf9a700 1 -- 172.21.15.49:0/1083779524 wait complete. 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.403+0000 7f40fcf9a700 1 Processor -- start 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.403+0000 7f40fcf9a700 1 -- start start 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40fcf9a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f80ffdc0 0x7f40f819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40fcf9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 0x7f40f819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40fcf9a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f81958c0 con 0x7f40f8100770 2024-05-15T16:31:49.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40fcf9a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f8195a30 con 0x7f40f80ffdc0 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40f5d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 0x7f40f819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40f5d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 0x7f40f819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57322/0 (socket says 172.21.15.49:57322) 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40f5d9b700 1 -- 172.21.15.49:0/4099952378 learned_addr learned my addr 172.21.15.49:0/4099952378 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40f659c700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f80ffdc0 0x7f40f819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40f5d9b700 1 -- 172.21.15.49:0/4099952378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f80ffdc0 msgr2=0x7f40f819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40f5d9b700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f80ffdc0 0x7f40f819b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40f5d9b700 1 -- 172.21.15.49:0/4099952378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0095f0 con 0x7f40f8100770 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.404+0000 7f40f5d9b700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 0x7f40f819bd50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0092e0 tx=0x7f40ec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.405+0000 7f40e77fe700 1 -- 172.21.15.49:0/4099952378 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40ec01d070 con 0x7f40f8100770 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.405+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f8195cb0 con 0x7f40f8100770 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.405+0000 7f40e77fe700 1 -- 172.21.15.49:0/4099952378 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40ec00f460 con 0x7f40f8100770 2024-05-15T16:31:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.405+0000 7f40e77fe700 1 -- 172.21.15.49:0/4099952378 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40ec017630 con 0x7f40f8100770 2024-05-15T16:31:49.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.405+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f81961a0 con 0x7f40f8100770 2024-05-15T16:31:49.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.406+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f81081c0 con 0x7f40f8100770 2024-05-15T16:31:49.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.407+0000 7f40e77fe700 1 -- 172.21.15.49:0/4099952378 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f40ec00f5d0 con 0x7f40f8100770 2024-05-15T16:31:49.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.407+0000 7f40e77fe700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f40e006e200 0x7f40e00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:49.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.407+0000 7f40e77fe700 1 -- 172.21.15.49:0/4099952378 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f40ec08f550 con 0x7f40f8100770 2024-05-15T16:31:49.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.407+0000 7f40f659c700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f40e006e200 0x7f40e00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:49.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.408+0000 7f40f659c700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f40e006e200 0x7f40e00706c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f40e8005950 tx=0x7f40e800a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:49.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.412+0000 7f40e77fe700 1 -- 172.21.15.49:0/4099952378 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f40ec058830 con 0x7f40f8100770 2024-05-15T16:31:49.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:49 smithi144 ceph-mon[86635]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:49.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.668+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f40f8196da0 con 0x7f40f8100770 2024-05-15T16:31:49.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.668+0000 7f40e77fe700 1 -- 172.21.15.49:0/4099952378 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f40ec005740 con 0x7f40f8100770 2024-05-15T16:31:49.669 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:31:49.669 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:31:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:31:49.671 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:31:49.672 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:31:49.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f40e006e200 msgr2=0x7f40e00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:49.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f40e006e200 0x7f40e00706c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f40e8005950 tx=0x7f40e800a300 comp rx=0 tx=0).stop 2024-05-15T16:31:49.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 msgr2=0x7f40f819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:49.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 0x7f40f819bd50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0092e0 tx=0x7f40ec00bd70 comp rx=0 tx=0).stop 2024-05-15T16:31:49.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 shutdown_connections 2024-05-15T16:31:49.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f40e006e200 0x7f40e00706c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f80ffdc0 0x7f40f819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 --2- 172.21.15.49:0/4099952378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40f8100770 0x7f40f819bd50 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 >> 172.21.15.49:0/4099952378 conn(0x7f40f8076800 msgr2=0x7f40f8103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:49.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 shutdown_connections 2024-05-15T16:31:49.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.670+0000 7f40fcf9a700 1 -- 172.21.15.49:0/4099952378 wait complete. 2024-05-15T16:31:49.674 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:31:49.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.780+0000 7ff81897d700 1 -- 172.21.15.49:0/2177948393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8140ffa30 msgr2=0x7ff8140ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.780+0000 7ff81897d700 1 --2- 172.21.15.49:0/2177948393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8140ffa30 0x7ff8140ffe10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff804009910 tx=0x7ff804009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.780+0000 7ff81897d700 1 -- 172.21.15.49:0/2177948393 shutdown_connections 2024-05-15T16:31:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.780+0000 7ff81897d700 1 --2- 172.21.15.49:0/2177948393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8141003e0 0x7ff81410d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.780+0000 7ff81897d700 1 --2- 172.21.15.49:0/2177948393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8140ffa30 0x7ff8140ffe10 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.780+0000 7ff81897d700 1 -- 172.21.15.49:0/2177948393 >> 172.21.15.49:0/2177948393 conn(0x7ff814076800 msgr2=0x7ff8140fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.780+0000 7ff81897d700 1 -- 172.21.15.49:0/2177948393 shutdown_connections 2024-05-15T16:31:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.780+0000 7ff81897d700 1 -- 172.21.15.49:0/2177948393 wait complete. 2024-05-15T16:31:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.781+0000 7ff81897d700 1 Processor -- start 2024-05-15T16:31:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.781+0000 7ff81897d700 1 -- start start 2024-05-15T16:31:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.781+0000 7ff81897d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8140ffa30 0x7ff814198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.781+0000 7ff81897d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8141003e0 0x7ff814199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.781+0000 7ff81897d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff814199a10 con 0x7ff8141003e0 2024-05-15T16:31:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.781+0000 7ff81897d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419d6e0 con 0x7ff8140ffa30 2024-05-15T16:31:49.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.781+0000 7ff811d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8141003e0 0x7ff814199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:49.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.781+0000 7ff811d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8141003e0 0x7ff814199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57340/0 (socket says 172.21.15.49:57340) 2024-05-15T16:31:49.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.781+0000 7ff811d9b700 1 -- 172.21.15.49:0/2938358952 learned_addr learned my addr 172.21.15.49:0/2938358952 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:49.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff81259c700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8140ffa30 0x7ff814198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:49.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff811d9b700 1 -- 172.21.15.49:0/2938358952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8140ffa30 msgr2=0x7ff814198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:49.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff811d9b700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8140ffa30 0x7ff814198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:49.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff811d9b700 1 -- 172.21.15.49:0/2938358952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040095f0 con 0x7ff8141003e0 2024-05-15T16:31:49.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff81259c700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8140ffa30 0x7ff814198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:31:49.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff811d9b700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8141003e0 0x7ff814199330 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff80800e920 tx=0x7ff80800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:49.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff8037fe700 1 -- 172.21.15.49:0/2938358952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff808009790 con 0x7ff8141003e0 2024-05-15T16:31:49.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81419d9c0 con 0x7ff8141003e0 2024-05-15T16:31:49.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff8037fe700 1 -- 172.21.15.49:0/2938358952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff80800cba0 con 0x7ff8141003e0 2024-05-15T16:31:49.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff8037fe700 1 -- 172.21.15.49:0/2938358952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff808018650 con 0x7ff8141003e0 2024-05-15T16:31:49.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.782+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81419df10 con 0x7ff8141003e0 2024-05-15T16:31:49.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.783+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff814068ee0 con 0x7ff8141003e0 2024-05-15T16:31:49.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.784+0000 7ff8037fe700 1 -- 172.21.15.49:0/2938358952 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff80800cd10 con 0x7ff8141003e0 2024-05-15T16:31:49.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.784+0000 7ff8037fe700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff7fc06e200 0x7ff7fc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:49.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.784+0000 7ff8037fe700 1 -- 172.21.15.49:0/2938358952 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff808014070 con 0x7ff8141003e0 2024-05-15T16:31:49.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.784+0000 7ff81259c700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff7fc06e200 0x7ff7fc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:49.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.785+0000 7ff81259c700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff7fc06e200 0x7ff7fc0706c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff804006010 tx=0x7ff804009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:49.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:49.789+0000 7ff8037fe700 1 -- 172.21.15.49:0/2938358952 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff808053850 con 0x7ff8141003e0 2024-05-15T16:31:50.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.016+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff814110020 con 0x7ff7fc06e200 2024-05-15T16:31:50.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.017+0000 7ff8037fe700 1 -- 172.21.15.49:0/2938358952 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff814110020 con 0x7ff7fc06e200 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:31:50.019 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:31:50.020 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:31:50.020 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:31:50.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.019+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff7fc06e200 msgr2=0x7ff7fc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:50.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.019+0000 7ff81897d700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff7fc06e200 0x7ff7fc0706c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff804006010 tx=0x7ff804009f70 comp rx=0 tx=0).stop 2024-05-15T16:31:50.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.019+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8141003e0 msgr2=0x7ff814199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:50.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.019+0000 7ff81897d700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8141003e0 0x7ff814199330 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff80800e920 tx=0x7ff80800ece0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.019+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 shutdown_connections 2024-05-15T16:31:50.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.019+0000 7ff81897d700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff7fc06e200 0x7ff7fc0706c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.019+0000 7ff81897d700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8140ffa30 0x7ff814198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.019+0000 7ff81897d700 1 --2- 172.21.15.49:0/2938358952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8141003e0 0x7ff814199330 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.019+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 >> 172.21.15.49:0/2938358952 conn(0x7ff814076800 msgr2=0x7ff8140fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:50.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.020+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 shutdown_connections 2024-05-15T16:31:50.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.020+0000 7ff81897d700 1 -- 172.21.15.49:0/2938358952 wait complete. 2024-05-15T16:31:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.127+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/1030925630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc100f00 msgr2=0x7f1fbc1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.127+0000 7f1fc0dd9700 1 --2- 172.21.15.49:0/1030925630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc100f00 0x7f1fbc1012e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009910 tx=0x7f1fac009c20 comp rx=0 tx=0).stop 2024-05-15T16:31:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.127+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/1030925630 shutdown_connections 2024-05-15T16:31:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.127+0000 7f1fc0dd9700 1 --2- 172.21.15.49:0/1030925630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fbc1018b0 0x7f1fbc105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.127+0000 7f1fc0dd9700 1 --2- 172.21.15.49:0/1030925630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc100f00 0x7f1fbc1012e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.127+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/1030925630 >> 172.21.15.49:0/1030925630 conn(0x7f1fbc0fc7d0 msgr2=0x7f1fbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:50.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.128+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/1030925630 shutdown_connections 2024-05-15T16:31:50.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.128+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/1030925630 wait complete. 2024-05-15T16:31:50.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.128+0000 7f1fc0dd9700 1 Processor -- start 2024-05-15T16:31:50.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.128+0000 7f1fc0dd9700 1 -- start start 2024-05-15T16:31:50.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fc0dd9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fbc100f00 0x7f1fbc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fc0dd9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc1018b0 0x7f1fbc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fc0dd9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc073d20 con 0x7f1fbc1018b0 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fc0dd9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc073e60 con 0x7f1fbc100f00 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fb9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc1018b0 0x7f1fbc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fb9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc1018b0 0x7f1fbc0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57368/0 (socket says 172.21.15.49:57368) 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fb9d9b700 1 -- 172.21.15.49:0/79974456 learned_addr learned my addr 172.21.15.49:0/79974456 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fba59c700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fbc100f00 0x7f1fbc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fb9d9b700 1 -- 172.21.15.49:0/79974456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fbc100f00 msgr2=0x7f1fbc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fb9d9b700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fbc100f00 0x7f1fbc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fb9d9b700 1 -- 172.21.15.49:0/79974456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac0095f0 con 0x7f1fbc1018b0 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fb9d9b700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc1018b0 0x7f1fbc0737e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400b480 tx=0x7f1fa400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.129+0000 7f1fb37fe700 1 -- 172.21.15.49:0/79974456 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fa4011800 con 0x7f1fbc1018b0 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.130+0000 7f1fb37fe700 1 -- 172.21.15.49:0/79974456 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fa4011e20 con 0x7f1fbc1018b0 2024-05-15T16:31:50.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.130+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc1a2ec0 con 0x7f1fbc1018b0 2024-05-15T16:31:50.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.130+0000 7f1fb37fe700 1 -- 172.21.15.49:0/79974456 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fa4010ea0 con 0x7f1fbc1018b0 2024-05-15T16:31:50.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.130+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc1a3220 con 0x7f1fbc1018b0 2024-05-15T16:31:50.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.130+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fbc1092a0 con 0x7f1fbc1018b0 2024-05-15T16:31:50.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.131+0000 7f1fb37fe700 1 -- 172.21.15.49:0/79974456 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1fa4010380 con 0x7f1fbc1018b0 2024-05-15T16:31:50.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.132+0000 7f1fb37fe700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1fa806e200 0x7f1fa80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:31:50.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.132+0000 7f1fb37fe700 1 -- 172.21.15.49:0/79974456 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1fa408dc60 con 0x7f1fbc1018b0 2024-05-15T16:31:50.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.132+0000 7f1fba59c700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1fa806e200 0x7f1fa80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:31:50.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.132+0000 7f1fba59c700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1fa806e200 0x7f1fa80706c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:31:50.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.136+0000 7f1fb37fe700 1 -- 172.21.15.49:0/79974456 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1fa4056f40 con 0x7f1fbc1018b0 2024-05-15T16:31:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.430+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1fbc074020 con 0x7f1fbc1018b0 2024-05-15T16:31:50.432 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:50 smithi049 ceph-mon[112142]: from='client.35890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:50.433 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:50 smithi049 ceph-mon[112142]: pgmap v1045: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:50.433 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:50 smithi049 ceph-mon[112142]: from='client.35894 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:50.433 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:50 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1473677983' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:31:50.433 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:50 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4099952378' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:31:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.431+0000 7f1fb37fe700 1 -- 172.21.15.49:0/79974456 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f1fa405a600 con 0x7f1fbc1018b0 2024-05-15T16:31:50.433 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:31:50.433 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:31:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:31:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:31:50.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.433+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1fa806e200 msgr2=0x7f1fa80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:50.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.433+0000 7f1fc0dd9700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1fa806e200 0x7f1fa80706c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac01a040 comp rx=0 tx=0).stop 2024-05-15T16:31:50.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.433+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc1018b0 msgr2=0x7f1fbc0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:31:50.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.433+0000 7f1fc0dd9700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc1018b0 0x7f1fbc0737e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400b480 tx=0x7f1fa400b790 comp rx=0 tx=0).stop 2024-05-15T16:31:50.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.433+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 shutdown_connections 2024-05-15T16:31:50.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.433+0000 7f1fc0dd9700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f1fa806e200 0x7f1fa80706c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.433+0000 7f1fc0dd9700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fbc100f00 0x7f1fbc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.433+0000 7f1fc0dd9700 1 --2- 172.21.15.49:0/79974456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fbc1018b0 0x7f1fbc0737e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:31:50.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.434+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 >> 172.21.15.49:0/79974456 conn(0x7f1fbc0fc7d0 msgr2=0x7f1fbc0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:31:50.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.434+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 shutdown_connections 2024-05-15T16:31:50.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:31:50.434+0000 7f1fc0dd9700 1 -- 172.21.15.49:0/79974456 wait complete. 2024-05-15T16:31:50.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:50 smithi144 ceph-mon[86635]: from='client.35890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:50 smithi144 ceph-mon[86635]: pgmap v1045: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:50 smithi144 ceph-mon[86635]: from='client.35894 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:50 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1473677983' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:31:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:50 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4099952378' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:31:51.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:51 smithi144 ceph-mon[86635]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:31:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:51 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/79974456' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:31:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:51 smithi049 ceph-mon[112142]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:31:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:31:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:51 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/79974456' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:31:52.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:52 smithi144 ceph-mon[86635]: pgmap v1046: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:52 smithi049 ceph-mon[112142]: pgmap v1046: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:54 smithi144 ceph-mon[86635]: pgmap v1047: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:54 smithi049 ceph-mon[112142]: pgmap v1047: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:56 smithi144 ceph-mon[86635]: pgmap v1048: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:56 smithi049 ceph-mon[112142]: pgmap v1048: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:31:58 smithi144 ceph-mon[86635]: pgmap v1049: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:31:58.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:31:58 smithi049 ceph-mon[112142]: pgmap v1049: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:00 smithi144 ceph-mon[86635]: pgmap v1050: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:00 smithi049 ceph-mon[112142]: pgmap v1050: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:02.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:02 smithi144 ceph-mon[86635]: pgmap v1051: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:02 smithi049 ceph-mon[112142]: pgmap v1051: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:04.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:04 smithi144 ceph-mon[86635]: pgmap v1052: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:04 smithi049 ceph-mon[112142]: pgmap v1052: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:06 smithi144 ceph-mon[86635]: pgmap v1053: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:32:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:06 smithi049 ceph-mon[112142]: pgmap v1053: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:32:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:08 smithi144 ceph-mon[86635]: pgmap v1054: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:08 smithi049 ceph-mon[112142]: pgmap v1054: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:10 smithi144 ceph-mon[86635]: pgmap v1055: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:10 smithi049 ceph-mon[112142]: pgmap v1055: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:12 smithi144 ceph-mon[86635]: pgmap v1056: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:12.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:12 smithi049 ceph-mon[112142]: pgmap v1056: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:14 smithi144 ceph-mon[86635]: pgmap v1057: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:14.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:14 smithi049 ceph-mon[112142]: pgmap v1057: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:16 smithi144 ceph-mon[86635]: pgmap v1058: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:16 smithi049 ceph-mon[112142]: pgmap v1058: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:18 smithi144 ceph-mon[86635]: pgmap v1059: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:18 smithi049 ceph-mon[112142]: pgmap v1059: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:20 smithi144 ceph-mon[86635]: pgmap v1060: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:20 smithi049 ceph-mon[112142]: pgmap v1060: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.558+0000 7f70ca8a5700 1 -- 172.21.15.49:0/2074528897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4101890 msgr2=0x7f70c41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.558+0000 7f70ca8a5700 1 --2- 172.21.15.49:0/2074528897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4101890 0x7f70c41058e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009910 tx=0x7f70ac009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.558+0000 7f70ca8a5700 1 -- 172.21.15.49:0/2074528897 shutdown_connections 2024-05-15T16:32:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.558+0000 7f70ca8a5700 1 --2- 172.21.15.49:0/2074528897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4101890 0x7f70c41058e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.558+0000 7f70ca8a5700 1 --2- 172.21.15.49:0/2074528897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c4100ee0 0x7f70c41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.558+0000 7f70ca8a5700 1 -- 172.21.15.49:0/2074528897 >> 172.21.15.49:0/2074528897 conn(0x7f70c40fc790 msgr2=0x7f70c40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.558+0000 7f70ca8a5700 1 -- 172.21.15.49:0/2074528897 shutdown_connections 2024-05-15T16:32:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.558+0000 7f70ca8a5700 1 -- 172.21.15.49:0/2074528897 wait complete. 2024-05-15T16:32:20.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.559+0000 7f70ca8a5700 1 Processor -- start 2024-05-15T16:32:20.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.559+0000 7f70ca8a5700 1 -- start start 2024-05-15T16:32:20.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.559+0000 7f70ca8a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4100ee0 0x7f70c419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.559+0000 7f70ca8a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c4101890 0x7f70c419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.559+0000 7f70ca8a5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c419d3b0 con 0x7f70c4100ee0 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.559+0000 7f70ca8a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c4196a20 con 0x7f70c4101890 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4100ee0 0x7f70c419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4100ee0 0x7f70c419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49396/0 (socket says 172.21.15.49:49396) 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c3fff700 1 -- 172.21.15.49:0/138595761 learned_addr learned my addr 172.21.15.49:0/138595761 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70bbfff700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c4101890 0x7f70c419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c3fff700 1 -- 172.21.15.49:0/138595761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c4101890 msgr2=0x7f70c419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c3fff700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c4101890 0x7f70c419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c3fff700 1 -- 172.21.15.49:0/138595761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac0095f0 con 0x7f70c4100ee0 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c3fff700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4100ee0 0x7f70c419c8e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c930 tx=0x7f70b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:20.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c1ffb700 1 -- 172.21.15.49:0/138595761 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b4007ad0 con 0x7f70c4100ee0 2024-05-15T16:32:20.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c1ffb700 1 -- 172.21.15.49:0/138595761 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70b400ce80 con 0x7f70c4100ee0 2024-05-15T16:32:20.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c4196d00 con 0x7f70c4100ee0 2024-05-15T16:32:20.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70c1ffb700 1 -- 172.21.15.49:0/138595761 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b4018630 con 0x7f70c4100ee0 2024-05-15T16:32:20.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.560+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c4197250 con 0x7f70c4100ee0 2024-05-15T16:32:20.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.561+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c41092e0 con 0x7f70c4100ee0 2024-05-15T16:32:20.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.562+0000 7f70c1ffb700 1 -- 172.21.15.49:0/138595761 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f70b4007c30 con 0x7f70c4100ee0 2024-05-15T16:32:20.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.563+0000 7f70c1ffb700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70b006e200 0x7f70b00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:20.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.563+0000 7f70c1ffb700 1 -- 172.21.15.49:0/138595761 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f70b408e070 con 0x7f70c4100ee0 2024-05-15T16:32:20.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.563+0000 7f70bbfff700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70b006e200 0x7f70b00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:20.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.563+0000 7f70bbfff700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70b006e200 0x7f70b00706c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f70c4197be0 tx=0x7f70ac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:20.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.567+0000 7f70c1ffb700 1 -- 172.21.15.49:0/138595761 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f70b405af00 con 0x7f70c4100ee0 2024-05-15T16:32:20.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.794+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70c4197ee0 con 0x7f70b006e200 2024-05-15T16:32:20.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.796+0000 7f70c1ffb700 1 -- 172.21.15.49:0/138595761 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f70c4197ee0 con 0x7f70b006e200 2024-05-15T16:32:20.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70b006e200 msgr2=0x7f70b00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70b006e200 0x7f70b00706c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f70c4197be0 tx=0x7f70ac01a040 comp rx=0 tx=0).stop 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4100ee0 msgr2=0x7f70c419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4100ee0 0x7f70c419c8e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c930 tx=0x7f70b400cc40 comp rx=0 tx=0).stop 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 shutdown_connections 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70b006e200 0x7f70b00706c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70c4100ee0 0x7f70c419c8e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 --2- 172.21.15.49:0/138595761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c4101890 0x7f70c419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 >> 172.21.15.49:0/138595761 conn(0x7f70c40fc790 msgr2=0x7f70c40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 shutdown_connections 2024-05-15T16:32:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.798+0000 7f70ca8a5700 1 -- 172.21.15.49:0/138595761 wait complete. 2024-05-15T16:32:20.809 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:32:20.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.907+0000 7f5de92ce700 1 -- 172.21.15.49:0/1521823632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 msgr2=0x7f5de4105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:20.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.907+0000 7f5de92ce700 1 --2- 172.21.15.49:0/1521823632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 0x7f5de4105900 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009910 tx=0x7f5dd8009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:20.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.907+0000 7f5de92ce700 1 -- 172.21.15.49:0/1521823632 shutdown_connections 2024-05-15T16:32:20.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.907+0000 7f5de92ce700 1 --2- 172.21.15.49:0/1521823632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 0x7f5de4105900 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:20.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.907+0000 7f5de92ce700 1 --2- 172.21.15.49:0/1521823632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5de4100f00 0x7f5de41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:20.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.908+0000 7f5de92ce700 1 -- 172.21.15.49:0/1521823632 >> 172.21.15.49:0/1521823632 conn(0x7f5de40fc7d0 msgr2=0x7f5de40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:20.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.908+0000 7f5de92ce700 1 -- 172.21.15.49:0/1521823632 shutdown_connections 2024-05-15T16:32:20.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.908+0000 7f5de92ce700 1 -- 172.21.15.49:0/1521823632 wait complete. 2024-05-15T16:32:20.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.908+0000 7f5de92ce700 1 Processor -- start 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.908+0000 7f5de92ce700 1 -- start start 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de92ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5de4100f00 0x7f5de419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de92ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 0x7f5de419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de92ce700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de419d440 con 0x7f5de41018b0 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de92ce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4196a20 con 0x7f5de4100f00 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 0x7f5de419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 0x7f5de419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49422/0 (socket says 172.21.15.49:49422) 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de27fc700 1 -- 172.21.15.49:0/3156193523 learned_addr learned my addr 172.21.15.49:0/3156193523 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de2ffd700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5de4100f00 0x7f5de419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de27fc700 1 -- 172.21.15.49:0/3156193523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5de4100f00 msgr2=0x7f5de419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:20.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de27fc700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5de4100f00 0x7f5de419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:20.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de27fc700 1 -- 172.21.15.49:0/3156193523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd80095f0 con 0x7f5de41018b0 2024-05-15T16:32:20.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.909+0000 7f5de27fc700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 0x7f5de419ce20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800bd70 tx=0x7f5dd800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:20.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.910+0000 7f5dcbfff700 1 -- 172.21.15.49:0/3156193523 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dd801d070 con 0x7f5de41018b0 2024-05-15T16:32:20.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.910+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de4196ca0 con 0x7f5de41018b0 2024-05-15T16:32:20.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.910+0000 7f5dcbfff700 1 -- 172.21.15.49:0/3156193523 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5dd80056e0 con 0x7f5de41018b0 2024-05-15T16:32:20.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.910+0000 7f5dcbfff700 1 -- 172.21.15.49:0/3156193523 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dd800fe50 con 0x7f5de41018b0 2024-05-15T16:32:20.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.910+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de4197190 con 0x7f5de41018b0 2024-05-15T16:32:20.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.911+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de4109300 con 0x7f5de41018b0 2024-05-15T16:32:20.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.911+0000 7f5dcbfff700 1 -- 172.21.15.49:0/3156193523 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5dd800f680 con 0x7f5de41018b0 2024-05-15T16:32:20.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.912+0000 7f5dcbfff700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5dcc06e200 0x7f5dcc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:20.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.912+0000 7f5dcbfff700 1 -- 172.21.15.49:0/3156193523 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5dd808f4d0 con 0x7f5de41018b0 2024-05-15T16:32:20.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.912+0000 7f5de2ffd700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5dcc06e200 0x7f5dcc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:20.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.912+0000 7f5de2ffd700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5dcc06e200 0x7f5dcc0706c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005950 tx=0x7f5dd400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:20.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:20.916+0000 7f5dcbfff700 1 -- 172.21.15.49:0/3156193523 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5dd805c2e0 con 0x7f5de41018b0 2024-05-15T16:32:21.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.143+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5de4197ee0 con 0x7f5dcc06e200 2024-05-15T16:32:21.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.144+0000 7f5dcbfff700 1 -- 172.21.15.49:0/3156193523 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5de4197ee0 con 0x7f5dcc06e200 2024-05-15T16:32:21.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5dcc06e200 msgr2=0x7f5dcc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5dcc06e200 0x7f5dcc0706c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005950 tx=0x7f5dd400b410 comp rx=0 tx=0).stop 2024-05-15T16:32:21.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 msgr2=0x7f5de419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 0x7f5de419ce20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800bd70 tx=0x7f5dd800bda0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 shutdown_connections 2024-05-15T16:32:21.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5dcc06e200 0x7f5dcc0706c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5de4100f00 0x7f5de419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 --2- 172.21.15.49:0/3156193523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5de41018b0 0x7f5de419ce20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 >> 172.21.15.49:0/3156193523 conn(0x7f5de40fc7d0 msgr2=0x7f5de40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:21.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 shutdown_connections 2024-05-15T16:32:21.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.146+0000 7f5de92ce700 1 -- 172.21.15.49:0/3156193523 wait complete. 2024-05-15T16:32:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.255+0000 7f8c567c5700 1 -- 172.21.15.49:0/274353535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c500ffdf0 msgr2=0x7f8c501001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.255+0000 7f8c567c5700 1 --2- 172.21.15.49:0/274353535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c500ffdf0 0x7f8c501001d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009910 tx=0x7f8c40009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.255+0000 7f8c567c5700 1 -- 172.21.15.49:0/274353535 shutdown_connections 2024-05-15T16:32:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.255+0000 7f8c567c5700 1 --2- 172.21.15.49:0/274353535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c501007a0 0x7f8c501047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.255+0000 7f8c567c5700 1 --2- 172.21.15.49:0/274353535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c500ffdf0 0x7f8c501001d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.255+0000 7f8c567c5700 1 -- 172.21.15.49:0/274353535 >> 172.21.15.49:0/274353535 conn(0x7f8c500fb6a0 msgr2=0x7f8c500fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.255+0000 7f8c567c5700 1 -- 172.21.15.49:0/274353535 shutdown_connections 2024-05-15T16:32:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.255+0000 7f8c567c5700 1 -- 172.21.15.49:0/274353535 wait complete. 2024-05-15T16:32:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c567c5700 1 Processor -- start 2024-05-15T16:32:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c567c5700 1 -- start start 2024-05-15T16:32:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c567c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c500ffdf0 0x7f8c50197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c567c5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c501007a0 0x7f8c50197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c567c5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50198490 con 0x7f8c501007a0 2024-05-15T16:32:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c567c5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c501985d0 con 0x7f8c500ffdf0 2024-05-15T16:32:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c4f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c501007a0 0x7f8c50197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c4f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c501007a0 0x7f8c50197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49436/0 (socket says 172.21.15.49:49436) 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c4f7fe700 1 -- 172.21.15.49:0/877404759 learned_addr learned my addr 172.21.15.49:0/877404759 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.256+0000 7f8c4ffff700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c500ffdf0 0x7f8c50197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.257+0000 7f8c4f7fe700 1 -- 172.21.15.49:0/877404759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c500ffdf0 msgr2=0x7f8c50197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.257+0000 7f8c4f7fe700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c500ffdf0 0x7f8c50197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.257+0000 7f8c4f7fe700 1 -- 172.21.15.49:0/877404759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c400095f0 con 0x7f8c501007a0 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.257+0000 7f8c4f7fe700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c501007a0 0x7f8c50197f50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400c960 tx=0x7f8c4400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.257+0000 7f8c4d7fa700 1 -- 172.21.15.49:0/877404759 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c44007a70 con 0x7f8c501007a0 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.257+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c501a0a80 con 0x7f8c501007a0 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.257+0000 7f8c4d7fa700 1 -- 172.21.15.49:0/877404759 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c44007bd0 con 0x7f8c501007a0 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.257+0000 7f8c4d7fa700 1 -- 172.21.15.49:0/877404759 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c44018630 con 0x7f8c501007a0 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.257+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c501a0e30 con 0x7f8c501007a0 2024-05-15T16:32:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.258+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50108190 con 0x7f8c501007a0 2024-05-15T16:32:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.259+0000 7f8c4d7fa700 1 -- 172.21.15.49:0/877404759 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8c4400f450 con 0x7f8c501007a0 2024-05-15T16:32:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.259+0000 7f8c4d7fa700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8c3806e200 0x7f8c380706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.259+0000 7f8c4d7fa700 1 -- 172.21.15.49:0/877404759 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8c4408e0d0 con 0x7f8c501007a0 2024-05-15T16:32:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.259+0000 7f8c4ffff700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8c3806e200 0x7f8c380706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.260+0000 7f8c4ffff700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8c3806e200 0x7f8c380706c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006010 tx=0x7f8c4001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:21.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.264+0000 7f8c4d7fa700 1 -- 172.21.15.49:0/877404759 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8c440573b0 con 0x7f8c501007a0 2024-05-15T16:32:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:32:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:32:21.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.486+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c50069820 con 0x7f8c3806e200 2024-05-15T16:32:21.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.497+0000 7f8c4d7fa700 1 -- 172.21.15.49:0/877404759 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8c50069820 con 0x7f8c3806e200 2024-05-15T16:32:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:32:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (35m) 13m ago 47m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:32:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (47m) 13m ago 47m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:32:21.497 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (46m) 13m ago 46m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (32m) 13m ago 47m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (32m) 13m ago 46m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (35m) 13m ago 46m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (29m) 13m ago 41m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (28m) 13m ago 41m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (28m) 13m ago 41m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (13m) 13m ago 41m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (37m) 13m ago 48m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (36m) 13m ago 46m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (34m) 13m ago 48m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (33m) 13m ago 45m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (36m) 13m ago 47m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:32:21.498 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (36m) 13m ago 46m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:32:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (32m) 13m ago 45m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:32:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (31m) 13m ago 44m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:32:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (31m) 13m ago 44m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:32:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (30m) 13m ago 43m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:32:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (30m) 13m ago 43m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:32:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (29m) 13m ago 43m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:32:21.499 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (35m) 13m ago 46m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8c3806e200 msgr2=0x7f8c380706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8c3806e200 0x7f8c380706c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006010 tx=0x7f8c4001a040 comp rx=0 tx=0).stop 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c501007a0 msgr2=0x7f8c50197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c501007a0 0x7f8c50197f50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400c960 tx=0x7f8c4400cc70 comp rx=0 tx=0).stop 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 shutdown_connections 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8c3806e200 0x7f8c380706c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c500ffdf0 0x7f8c50197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 --2- 172.21.15.49:0/877404759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c501007a0 0x7f8c50197f50 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 >> 172.21.15.49:0/877404759 conn(0x7f8c500fb6a0 msgr2=0x7f8c500fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 shutdown_connections 2024-05-15T16:32:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.499+0000 7f8c567c5700 1 -- 172.21.15.49:0/877404759 wait complete. 2024-05-15T16:32:21.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.606+0000 7f9c2e112700 1 -- 172.21.15.49:0/2193626454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 msgr2=0x7f9c281058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.606+0000 7f9c2e112700 1 --2- 172.21.15.49:0/2193626454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 0x7f9c281058e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009910 tx=0x7f9c1c009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:21.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.606+0000 7f9c2e112700 1 -- 172.21.15.49:0/2193626454 shutdown_connections 2024-05-15T16:32:21.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.606+0000 7f9c2e112700 1 --2- 172.21.15.49:0/2193626454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 0x7f9c281058e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.606+0000 7f9c2e112700 1 --2- 172.21.15.49:0/2193626454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c28100ee0 0x7f9c281012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.606+0000 7f9c2e112700 1 -- 172.21.15.49:0/2193626454 >> 172.21.15.49:0/2193626454 conn(0x7f9c280fc790 msgr2=0x7f9c280febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.606+0000 7f9c2e112700 1 -- 172.21.15.49:0/2193626454 shutdown_connections 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.606+0000 7f9c2e112700 1 -- 172.21.15.49:0/2193626454 wait complete. 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c2e112700 1 Processor -- start 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c2e112700 1 -- start start 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c2e112700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c28100ee0 0x7f9c280732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c2e112700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 0x7f9c280737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c2e112700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c28073d20 con 0x7f9c28101890 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c2e112700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c28073e60 con 0x7f9c28100ee0 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c26ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 0x7f9c280737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c26ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 0x7f9c280737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49462/0 (socket says 172.21.15.49:49462) 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c26ffd700 1 -- 172.21.15.49:0/1321195252 learned_addr learned my addr 172.21.15.49:0/1321195252 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c277fe700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c28100ee0 0x7f9c280732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c26ffd700 1 -- 172.21.15.49:0/1321195252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c28100ee0 msgr2=0x7f9c280732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c26ffd700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c28100ee0 0x7f9c280732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.607+0000 7f9c26ffd700 1 -- 172.21.15.49:0/1321195252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c0095f0 con 0x7f9c28101890 2024-05-15T16:32:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.608+0000 7f9c277fe700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c28100ee0 0x7f9c280732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:32:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.608+0000 7f9c26ffd700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 0x7f9c280737e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00bd70 tx=0x7f9c1c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.608+0000 7f9c24ff9700 1 -- 172.21.15.49:0/1321195252 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1c01d070 con 0x7f9c28101890 2024-05-15T16:32:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.608+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c280740e0 con 0x7f9c28101890 2024-05-15T16:32:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.608+0000 7f9c24ff9700 1 -- 172.21.15.49:0/1321195252 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c1c00f460 con 0x7f9c28101890 2024-05-15T16:32:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.608+0000 7f9c24ff9700 1 -- 172.21.15.49:0/1321195252 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1c017630 con 0x7f9c28101890 2024-05-15T16:32:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.608+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c281a3160 con 0x7f9c28101890 2024-05-15T16:32:21.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.609+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c28109230 con 0x7f9c28101890 2024-05-15T16:32:21.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.610+0000 7f9c24ff9700 1 -- 172.21.15.49:0/1321195252 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9c1c00f5d0 con 0x7f9c28101890 2024-05-15T16:32:21.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.610+0000 7f9c24ff9700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c1006e200 0x7f9c100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:21.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.610+0000 7f9c24ff9700 1 -- 172.21.15.49:0/1321195252 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9c1c08f600 con 0x7f9c28101890 2024-05-15T16:32:21.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.610+0000 7f9c277fe700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c1006e200 0x7f9c100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:21.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.611+0000 7f9c277fe700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c1006e200 0x7f9c100706c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9c18007900 tx=0x7f9c18008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:21.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.620+0000 7f9c24ff9700 1 -- 172.21.15.49:0/1321195252 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9c1c0588e0 con 0x7f9c28101890 2024-05-15T16:32:21.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.918+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9c2810df00 con 0x7f9c28101890 2024-05-15T16:32:21.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.919+0000 7f9c24ff9700 1 -- 172.21.15.49:0/1321195252 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9c1c005740 con 0x7f9c28101890 2024-05-15T16:32:21.919 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:32:21.919 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:32:21.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:32:21.921 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:32:21.921 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:32:21.921 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:32:21.921 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:32:21.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c1006e200 msgr2=0x7f9c100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c1006e200 0x7f9c100706c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9c18007900 tx=0x7f9c18008040 comp rx=0 tx=0).stop 2024-05-15T16:32:21.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 msgr2=0x7f9c280737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:21.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 0x7f9c280737e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00bd70 tx=0x7f9c1c00be50 comp rx=0 tx=0).stop 2024-05-15T16:32:21.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 shutdown_connections 2024-05-15T16:32:21.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c1006e200 0x7f9c100706c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c28100ee0 0x7f9c280732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 --2- 172.21.15.49:0/1321195252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c28101890 0x7f9c280737e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:21.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 >> 172.21.15.49:0/1321195252 conn(0x7f9c280fc790 msgr2=0x7f9c280fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:21.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 shutdown_connections 2024-05-15T16:32:21.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:21.921+0000 7f9c2e112700 1 -- 172.21.15.49:0/1321195252 wait complete. 2024-05-15T16:32:22.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.029+0000 7f9cb0e46700 1 -- 172.21.15.49:0/3380486533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac1007c0 msgr2=0x7f9cac104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.029+0000 7f9cb0e46700 1 --2- 172.21.15.49:0/3380486533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac1007c0 0x7f9cac104810 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009910 tx=0x7f9c94009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:22.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.030+0000 7f9cb0e46700 1 -- 172.21.15.49:0/3380486533 shutdown_connections 2024-05-15T16:32:22.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.030+0000 7f9cb0e46700 1 --2- 172.21.15.49:0/3380486533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac1007c0 0x7f9cac104810 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.030+0000 7f9cb0e46700 1 --2- 172.21.15.49:0/3380486533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac0ffe10 0x7f9cac1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.030+0000 7f9cb0e46700 1 -- 172.21.15.49:0/3380486533 >> 172.21.15.49:0/3380486533 conn(0x7f9cac0fb700 msgr2=0x7f9cac0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:22.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.030+0000 7f9cb0e46700 1 -- 172.21.15.49:0/3380486533 shutdown_connections 2024-05-15T16:32:22.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.030+0000 7f9cb0e46700 1 -- 172.21.15.49:0/3380486533 wait complete. 2024-05-15T16:32:22.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.030+0000 7f9cb0e46700 1 Processor -- start 2024-05-15T16:32:22.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.030+0000 7f9cb0e46700 1 -- start start 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9cb0e46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac0ffe10 0x7f9cac19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9cb0e46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1007c0 0x7f9cac19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9cb0e46700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac196a50 con 0x7f9cac0ffe10 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9cb0e46700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac196bc0 con 0x7f9cac1007c0 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9caa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac0ffe10 0x7f9cac19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9caa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac0ffe10 0x7f9cac19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49490/0 (socket says 172.21.15.49:49490) 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9caa59c700 1 -- 172.21.15.49:0/4029048602 learned_addr learned my addr 172.21.15.49:0/4029048602 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9ca9d9b700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1007c0 0x7f9cac19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9caa59c700 1 -- 172.21.15.49:0/4029048602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1007c0 msgr2=0x7f9cac19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9caa59c700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1007c0 0x7f9cac19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9caa59c700 1 -- 172.21.15.49:0/4029048602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940095f0 con 0x7f9cac0ffe10 2024-05-15T16:32:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9ca9d9b700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1007c0 0x7f9cac19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:32:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.031+0000 7f9caa59c700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac0ffe10 0x7f9cac19c9a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00c8f0 tx=0x7f9c9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.032+0000 7f9ca37fe700 1 -- 172.21.15.49:0/4029048602 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c9c007a70 con 0x7f9cac0ffe10 2024-05-15T16:32:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.032+0000 7f9ca37fe700 1 -- 172.21.15.49:0/4029048602 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c9c004cf0 con 0x7f9cac0ffe10 2024-05-15T16:32:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.032+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac196ea0 con 0x7f9cac0ffe10 2024-05-15T16:32:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.032+0000 7f9ca37fe700 1 -- 172.21.15.49:0/4029048602 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c9c005610 con 0x7f9cac0ffe10 2024-05-15T16:32:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.032+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac1973f0 con 0x7f9cac0ffe10 2024-05-15T16:32:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.033+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac1012f0 con 0x7f9cac0ffe10 2024-05-15T16:32:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.034+0000 7f9ca37fe700 1 -- 172.21.15.49:0/4029048602 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9c9c004750 con 0x7f9cac0ffe10 2024-05-15T16:32:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.034+0000 7f9ca37fe700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c9806e200 0x7f9c980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.034+0000 7f9ca37fe700 1 -- 172.21.15.49:0/4029048602 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9c9c013070 con 0x7f9cac0ffe10 2024-05-15T16:32:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.034+0000 7f9ca9d9b700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c9806e200 0x7f9c980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.035+0000 7f9ca9d9b700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c9806e200 0x7f9c980706c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9cac1981c0 tx=0x7f9c9400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:22.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.039+0000 7f9ca37fe700 1 -- 172.21.15.49:0/4029048602 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9c9c0572e0 con 0x7f9cac0ffe10 2024-05-15T16:32:22.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.295+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9cac10cdd0 con 0x7f9cac0ffe10 2024-05-15T16:32:22.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.295+0000 7f9ca37fe700 1 -- 172.21.15.49:0/4029048602 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f9c9c05a9a0 con 0x7f9cac0ffe10 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:32:22.296 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:32:22.297 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:32:22.298 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:32:22.299 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:32:22.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.297+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c9806e200 msgr2=0x7f9c980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.297+0000 7f9cb0e46700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c9806e200 0x7f9c980706c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9cac1981c0 tx=0x7f9c9400b540 comp rx=0 tx=0).stop 2024-05-15T16:32:22.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.297+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac0ffe10 msgr2=0x7f9cac19c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.297+0000 7f9cb0e46700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac0ffe10 0x7f9cac19c9a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00c8f0 tx=0x7f9c9c00ccb0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.298+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 shutdown_connections 2024-05-15T16:32:22.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.298+0000 7f9cb0e46700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9c9806e200 0x7f9c980706c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.298+0000 7f9cb0e46700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9cac0ffe10 0x7f9cac19c9a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.298+0000 7f9cb0e46700 1 --2- 172.21.15.49:0/4029048602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1007c0 0x7f9cac19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.298+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 >> 172.21.15.49:0/4029048602 conn(0x7f9cac0fb700 msgr2=0x7f9cac0fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:22.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.298+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 shutdown_connections 2024-05-15T16:32:22.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.298+0000 7f9cb0e46700 1 -- 172.21.15.49:0/4029048602 wait complete. 2024-05-15T16:32:22.300 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:32:22.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.407+0000 7f672b491700 1 -- 172.21.15.49:0/1837410137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 msgr2=0x7f67241047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.407+0000 7f672b491700 1 --2- 172.21.15.49:0/1837410137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 0x7f67241047f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6714009910 tx=0x7f6714009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:22.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.407+0000 7f672b491700 1 -- 172.21.15.49:0/1837410137 shutdown_connections 2024-05-15T16:32:22.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.407+0000 7f672b491700 1 --2- 172.21.15.49:0/1837410137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 0x7f67241047f0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.407+0000 7f672b491700 1 --2- 172.21.15.49:0/1837410137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67240ffdf0 0x7f67241001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.407+0000 7f672b491700 1 -- 172.21.15.49:0/1837410137 >> 172.21.15.49:0/1837410137 conn(0x7f67240fb6a0 msgr2=0x7f67240fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:22.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.407+0000 7f672b491700 1 -- 172.21.15.49:0/1837410137 shutdown_connections 2024-05-15T16:32:22.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.407+0000 7f672b491700 1 -- 172.21.15.49:0/1837410137 wait complete. 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f672b491700 1 Processor -- start 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f672b491700 1 -- start start 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f672b491700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67240ffdf0 0x7f672419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f672b491700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 0x7f672419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f672b491700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672419d420 con 0x7f67241007a0 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f672b491700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6724196a90 con 0x7f67240ffdf0 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f6728a2c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 0x7f672419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f6728a2c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 0x7f672419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49498/0 (socket says 172.21.15.49:49498) 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f6728a2c700 1 -- 172.21.15.49:0/2047055969 learned_addr learned my addr 172.21.15.49:0/2047055969 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f672922d700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67240ffdf0 0x7f672419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f6728a2c700 1 -- 172.21.15.49:0/2047055969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67240ffdf0 msgr2=0x7f672419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f6728a2c700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67240ffdf0 0x7f672419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f6728a2c700 1 -- 172.21.15.49:0/2047055969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140095f0 con 0x7f67241007a0 2024-05-15T16:32:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.408+0000 7f672922d700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67240ffdf0 0x7f672419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:32:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.409+0000 7f6728a2c700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 0x7f672419ce90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f67140092e0 tx=0x7f671400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.409+0000 7f671a7fc700 1 -- 172.21.15.49:0/2047055969 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f671401d070 con 0x7f67241007a0 2024-05-15T16:32:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.409+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6724196d10 con 0x7f67241007a0 2024-05-15T16:32:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.409+0000 7f671a7fc700 1 -- 172.21.15.49:0/2047055969 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f671400f460 con 0x7f67241007a0 2024-05-15T16:32:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.409+0000 7f671a7fc700 1 -- 172.21.15.49:0/2047055969 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6714017630 con 0x7f67241007a0 2024-05-15T16:32:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.409+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6724197200 con 0x7f67241007a0 2024-05-15T16:32:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.410+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67241012d0 con 0x7f67241007a0 2024-05-15T16:32:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.410+0000 7f671a7fc700 1 -- 172.21.15.49:0/2047055969 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f671400f5d0 con 0x7f67241007a0 2024-05-15T16:32:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.411+0000 7f671a7fc700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f671006e200 0x7f67100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.411+0000 7f671a7fc700 1 -- 172.21.15.49:0/2047055969 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f671408f620 con 0x7f67241007a0 2024-05-15T16:32:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.411+0000 7f672922d700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f671006e200 0x7f67100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.411+0000 7f672922d700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f671006e200 0x7f67100706c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6720005fd0 tx=0x7f6720005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:22.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.416+0000 7f671a7fc700 1 -- 172.21.15.49:0/2047055969 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f671405c430 con 0x7f67241007a0 2024-05-15T16:32:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:22 smithi144 ceph-mon[86635]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:22 smithi144 ceph-mon[86635]: pgmap v1061: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:22 smithi144 ceph-mon[86635]: from='client.35918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:22 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1321195252' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:32:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:22 smithi049 ceph-mon[112142]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:22 smithi049 ceph-mon[112142]: pgmap v1061: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:22 smithi049 ceph-mon[112142]: from='client.35918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:22 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1321195252' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:32:22.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.641+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f672410cd90 con 0x7f671006e200 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.643+0000 7f671a7fc700 1 -- 172.21.15.49:0/2047055969 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f672410cd90 con 0x7f671006e200 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:32:22.643 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:32:22.644 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:32:22.644 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:32:22.644 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:32:22.644 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:32:22.644 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:32:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f671006e200 msgr2=0x7f67100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f671006e200 0x7f67100706c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6720005fd0 tx=0x7f6720005e30 comp rx=0 tx=0).stop 2024-05-15T16:32:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 msgr2=0x7f672419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 0x7f672419ce90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f67140092e0 tx=0x7f671400be20 comp rx=0 tx=0).stop 2024-05-15T16:32:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 shutdown_connections 2024-05-15T16:32:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f671006e200 0x7f67100706c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67240ffdf0 0x7f672419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 --2- 172.21.15.49:0/2047055969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67241007a0 0x7f672419ce90 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 >> 172.21.15.49:0/2047055969 conn(0x7f67240fb6a0 msgr2=0x7f6724069590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 shutdown_connections 2024-05-15T16:32:22.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.645+0000 7f672b491700 1 -- 172.21.15.49:0/2047055969 wait complete. 2024-05-15T16:32:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 -- 172.21.15.49:0/3479326723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 msgr2=0x7fc43c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 --2- 172.21.15.49:0/3479326723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 0x7fc43c1047f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc430009910 tx=0x7fc430009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 -- 172.21.15.49:0/3479326723 shutdown_connections 2024-05-15T16:32:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 --2- 172.21.15.49:0/3479326723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 0x7fc43c1047f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 --2- 172.21.15.49:0/3479326723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc43c0ffdf0 0x7fc43c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 -- 172.21.15.49:0/3479326723 >> 172.21.15.49:0/3479326723 conn(0x7fc43c0fb6a0 msgr2=0x7fc43c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:22.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 -- 172.21.15.49:0/3479326723 shutdown_connections 2024-05-15T16:32:22.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 -- 172.21.15.49:0/3479326723 wait complete. 2024-05-15T16:32:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 Processor -- start 2024-05-15T16:32:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.753+0000 7fc443bb3700 1 -- start start 2024-05-15T16:32:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc443bb3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc43c0ffdf0 0x7fc43c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc443bb3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 0x7fc43c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc443bb3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c19d650 con 0x7fc43c1007a0 2024-05-15T16:32:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc443bb3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c196b70 con 0x7fc43c0ffdf0 2024-05-15T16:32:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc44114e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 0x7fc43c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc44114e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 0x7fc43c19d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49514/0 (socket says 172.21.15.49:49514) 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc44114e700 1 -- 172.21.15.49:0/873180010 learned_addr learned my addr 172.21.15.49:0/873180010 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc44194f700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc43c0ffdf0 0x7fc43c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc44114e700 1 -- 172.21.15.49:0/873180010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc43c0ffdf0 msgr2=0x7fc43c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc44114e700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc43c0ffdf0 0x7fc43c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc44114e700 1 -- 172.21.15.49:0/873180010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4300095f0 con 0x7fc43c1007a0 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc44114e700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 0x7fc43c19d000 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc4300092e0 tx=0x7fc43000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.754+0000 7fc42effd700 1 -- 172.21.15.49:0/873180010 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc43001d070 con 0x7fc43c1007a0 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.755+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c196df0 con 0x7fc43c1007a0 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.755+0000 7fc42effd700 1 -- 172.21.15.49:0/873180010 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc43000f460 con 0x7fc43c1007a0 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.755+0000 7fc42effd700 1 -- 172.21.15.49:0/873180010 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc430017630 con 0x7fc43c1007a0 2024-05-15T16:32:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.755+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c1972e0 con 0x7fc43c1007a0 2024-05-15T16:32:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.755+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c1081f0 con 0x7fc43c1007a0 2024-05-15T16:32:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.756+0000 7fc42effd700 1 -- 172.21.15.49:0/873180010 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc43000f5d0 con 0x7fc43c1007a0 2024-05-15T16:32:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.757+0000 7fc42effd700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc42806e200 0x7fc4280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.757+0000 7fc42effd700 1 -- 172.21.15.49:0/873180010 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc43008f530 con 0x7fc43c1007a0 2024-05-15T16:32:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.757+0000 7fc44194f700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc42806e200 0x7fc4280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:22.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.757+0000 7fc44194f700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc42806e200 0x7fc4280706c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc43800ac50 tx=0x7fc43800a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:22.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:22.762+0000 7fc42effd700 1 -- 172.21.15.49:0/873180010 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc430058810 con 0x7fc43c1007a0 2024-05-15T16:32:23.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.055+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc43c197fe0 con 0x7fc43c1007a0 2024-05-15T16:32:23.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.056+0000 7fc42effd700 1 -- 172.21.15.49:0/873180010 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fc430005740 con 0x7fc43c1007a0 2024-05-15T16:32:23.056 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:32:23.056 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:32:23.056 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:32:23.056 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:32:23.056 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:32:23.057 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:32:23.058 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:32:23.058 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:32:23.058 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:32:23.058 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:32:23.058 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.058+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc42806e200 msgr2=0x7fc4280706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.058+0000 7fc443bb3700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc42806e200 0x7fc4280706c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc43800ac50 tx=0x7fc43800a380 comp rx=0 tx=0).stop 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.058+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 msgr2=0x7fc43c19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.058+0000 7fc443bb3700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 0x7fc43c19d000 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc4300092e0 tx=0x7fc43000bd70 comp rx=0 tx=0).stop 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.058+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 shutdown_connections 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.058+0000 7fc443bb3700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc42806e200 0x7fc4280706c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.058+0000 7fc443bb3700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc43c0ffdf0 0x7fc43c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.058+0000 7fc443bb3700 1 --2- 172.21.15.49:0/873180010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc43c1007a0 0x7fc43c19d000 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.058+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 >> 172.21.15.49:0/873180010 conn(0x7fc43c0fb6a0 msgr2=0x7fc43c0fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.059+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 shutdown_connections 2024-05-15T16:32:23.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:23.059+0000 7fc443bb3700 1 -- 172.21.15.49:0/873180010 wait complete. 2024-05-15T16:32:23.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:23 smithi144 ceph-mon[86635]: from='client.35922 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:23.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:23 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4029048602' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:32:23.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:23 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/873180010' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:32:23.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:23 smithi049 ceph-mon[112142]: from='client.35922 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:23.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:23 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4029048602' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:32:23.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:23 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/873180010' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:32:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:24 smithi144 ceph-mon[86635]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:24 smithi144 ceph-mon[86635]: pgmap v1062: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:24 smithi049 ceph-mon[112142]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:24 smithi049 ceph-mon[112142]: pgmap v1062: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:26 smithi144 ceph-mon[86635]: pgmap v1063: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:26 smithi049 ceph-mon[112142]: pgmap v1063: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:28 smithi144 ceph-mon[86635]: pgmap v1064: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:28 smithi049 ceph-mon[112142]: pgmap v1064: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:30.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:30 smithi144 ceph-mon[86635]: pgmap v1065: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:30 smithi049 ceph-mon[112142]: pgmap v1065: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:32 smithi144 ceph-mon[86635]: pgmap v1066: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:32 smithi049 ceph-mon[112142]: pgmap v1066: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:34 smithi049 ceph-mon[112142]: pgmap v1067: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:34.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:34 smithi144 ceph-mon[86635]: pgmap v1067: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:36 smithi049 ceph-mon[112142]: pgmap v1068: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:32:36.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:36 smithi144 ceph-mon[86635]: pgmap v1068: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:36.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:32:38.599 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:38 smithi144 ceph-mon[86635]: pgmap v1069: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:38.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:38 smithi049 ceph-mon[112142]: pgmap v1069: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:40.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:40 smithi144 ceph-mon[86635]: pgmap v1070: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:40.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:40 smithi049 ceph-mon[112142]: pgmap v1070: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:42 smithi144 ceph-mon[86635]: pgmap v1071: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:42.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:42 smithi049 ceph-mon[112142]: pgmap v1071: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:44.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:44 smithi144 ceph-mon[86635]: pgmap v1072: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:44.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:44 smithi049 ceph-mon[112142]: pgmap v1072: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:46.643 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:46 smithi049 ceph-mon[112142]: pgmap v1073: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:46.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:46 smithi144 ceph-mon[86635]: pgmap v1073: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:48.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:48 smithi144 ceph-mon[86635]: pgmap v1074: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:48.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:48 smithi049 ceph-mon[112142]: pgmap v1074: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:50.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:50 smithi144 ceph-mon[86635]: pgmap v1075: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:50.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:32:50.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:50 smithi049 ceph-mon[112142]: pgmap v1075: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:50.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:32:52.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:52 smithi144 ceph-mon[86635]: pgmap v1076: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:52.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:52 smithi049 ceph-mon[112142]: pgmap v1076: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:53.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.181+0000 7f9bc596c700 1 -- 172.21.15.49:0/1716776944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 msgr2=0x7f9bc01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.181+0000 7f9bc596c700 1 --2- 172.21.15.49:0/1716776944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 0x7f9bc01012c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009910 tx=0x7f9bb0009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:53.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.182+0000 7f9bc596c700 1 -- 172.21.15.49:0/1716776944 shutdown_connections 2024-05-15T16:32:53.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.182+0000 7f9bc596c700 1 --2- 172.21.15.49:0/1716776944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc0101890 0x7f9bc01058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.182+0000 7f9bc596c700 1 --2- 172.21.15.49:0/1716776944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 0x7f9bc01012c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.182+0000 7f9bc596c700 1 -- 172.21.15.49:0/1716776944 >> 172.21.15.49:0/1716776944 conn(0x7f9bc00fc790 msgr2=0x7f9bc00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:53.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.182+0000 7f9bc596c700 1 -- 172.21.15.49:0/1716776944 shutdown_connections 2024-05-15T16:32:53.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.182+0000 7f9bc596c700 1 -- 172.21.15.49:0/1716776944 wait complete. 2024-05-15T16:32:53.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.182+0000 7f9bc596c700 1 Processor -- start 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bc596c700 1 -- start start 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bc596c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 0x7f9bc019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bc596c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc0101890 0x7f9bc019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bc596c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc019d3d0 con 0x7f9bc0100ee0 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bc596c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc0196a90 con 0x7f9bc0101890 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bbeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 0x7f9bc019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bbeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 0x7f9bc019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38818/0 (socket says 172.21.15.49:38818) 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bbeffd700 1 -- 172.21.15.49:0/2951700354 learned_addr learned my addr 172.21.15.49:0/2951700354 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bbe7fc700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc0101890 0x7f9bc019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bbeffd700 1 -- 172.21.15.49:0/2951700354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc0101890 msgr2=0x7f9bc019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bbeffd700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc0101890 0x7f9bc019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.183+0000 7f9bbeffd700 1 -- 172.21.15.49:0/2951700354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb00095f0 con 0x7f9bc0100ee0 2024-05-15T16:32:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.184+0000 7f9bbeffd700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 0x7f9bc019c900 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006010 tx=0x7f9bb000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.184+0000 7f9bc496a700 1 -- 172.21.15.49:0/2951700354 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bb001d070 con 0x7f9bc0100ee0 2024-05-15T16:32:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.184+0000 7f9bc496a700 1 -- 172.21.15.49:0/2951700354 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bb000f460 con 0x7f9bc0100ee0 2024-05-15T16:32:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.184+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc0196d10 con 0x7f9bc0100ee0 2024-05-15T16:32:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.184+0000 7f9bc496a700 1 -- 172.21.15.49:0/2951700354 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bb00175f0 con 0x7f9bc0100ee0 2024-05-15T16:32:53.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.184+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc0197200 con 0x7f9bc0100ee0 2024-05-15T16:32:53.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.185+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc0109280 con 0x7f9bc0100ee0 2024-05-15T16:32:53.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.185+0000 7f9bc496a700 1 -- 172.21.15.49:0/2951700354 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9bb000f5d0 con 0x7f9bc0100ee0 2024-05-15T16:32:53.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.186+0000 7f9bc496a700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bac06e200 0x7f9bac0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:53.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.186+0000 7f9bc496a700 1 -- 172.21.15.49:0/2951700354 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9bb008f550 con 0x7f9bc0100ee0 2024-05-15T16:32:53.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.186+0000 7f9bbe7fc700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bac06e200 0x7f9bac0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:53.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.186+0000 7f9bbe7fc700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bac06e200 0x7f9bac0706c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0197c00 tx=0x7f9ba800a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:53.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.191+0000 7f9bc496a700 1 -- 172.21.15.49:0/2951700354 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9bb005c3e0 con 0x7f9bc0100ee0 2024-05-15T16:32:53.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.418+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bc0197f00 con 0x7f9bac06e200 2024-05-15T16:32:53.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.420+0000 7f9bc496a700 1 -- 172.21.15.49:0/2951700354 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9bc0197f00 con 0x7f9bac06e200 2024-05-15T16:32:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bac06e200 msgr2=0x7f9bac0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bac06e200 0x7f9bac0706c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0197c00 tx=0x7f9ba800a380 comp rx=0 tx=0).stop 2024-05-15T16:32:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 msgr2=0x7f9bc019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 0x7f9bc019c900 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006010 tx=0x7f9bb000bd70 comp rx=0 tx=0).stop 2024-05-15T16:32:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 shutdown_connections 2024-05-15T16:32:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9bac06e200 0x7f9bac0706c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bc0100ee0 0x7f9bc019c900 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 --2- 172.21.15.49:0/2951700354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bc0101890 0x7f9bc019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 >> 172.21.15.49:0/2951700354 conn(0x7f9bc00fc790 msgr2=0x7f9bc00fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 shutdown_connections 2024-05-15T16:32:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.422+0000 7f9bc596c700 1 -- 172.21.15.49:0/2951700354 wait complete. 2024-05-15T16:32:53.433 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:32:53.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.532+0000 7f43bc58d700 1 -- 172.21.15.49:0/3186939828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4102790 msgr2=0x7f43b410ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.532+0000 7f43bc58d700 1 --2- 172.21.15.49:0/3186939828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4102790 0x7f43b410ac80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009910 tx=0x7f43a8009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.532+0000 7f43bc58d700 1 -- 172.21.15.49:0/3186939828 shutdown_connections 2024-05-15T16:32:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.532+0000 7f43bc58d700 1 --2- 172.21.15.49:0/3186939828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4102790 0x7f43b410ac80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.532+0000 7f43bc58d700 1 --2- 172.21.15.49:0/3186939828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b4101e70 0x7f43b4102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.532+0000 7f43bc58d700 1 -- 172.21.15.49:0/3186939828 >> 172.21.15.49:0/3186939828 conn(0x7f43b40fb700 msgr2=0x7f43b40fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.532+0000 7f43bc58d700 1 -- 172.21.15.49:0/3186939828 shutdown_connections 2024-05-15T16:32:53.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.532+0000 7f43bc58d700 1 -- 172.21.15.49:0/3186939828 wait complete. 2024-05-15T16:32:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43bc58d700 1 Processor -- start 2024-05-15T16:32:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43bc58d700 1 -- start start 2024-05-15T16:32:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43bc58d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4101e70 0x7f43b40ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43bc58d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b4102790 0x7f43b4100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43bc58d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4100550 con 0x7f43b4101e70 2024-05-15T16:32:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43bc58d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4100690 con 0x7f43b4102790 2024-05-15T16:32:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43ba329700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4101e70 0x7f43b40ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43ba329700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4101e70 0x7f43b40ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38842/0 (socket says 172.21.15.49:38842) 2024-05-15T16:32:53.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43ba329700 1 -- 172.21.15.49:0/2615729452 learned_addr learned my addr 172.21.15.49:0/2615729452 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43ba329700 1 -- 172.21.15.49:0/2615729452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b4102790 msgr2=0x7f43b4100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43b9b28700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b4102790 0x7f43b4100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43ba329700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b4102790 0x7f43b4100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43ba329700 1 -- 172.21.15.49:0/2615729452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a80095f0 con 0x7f43b4101e70 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.533+0000 7f43b9b28700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b4102790 0x7f43b4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.534+0000 7f43ba329700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4101e70 0x7f43b40ffad0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f43b000d8d0 tx=0x7f43b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.534+0000 7f43a77fe700 1 -- 172.21.15.49:0/2615729452 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b0009940 con 0x7f43b4101e70 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.534+0000 7f43a77fe700 1 -- 172.21.15.49:0/2615729452 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43b0010460 con 0x7f43b4101e70 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.534+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b41a2df0 con 0x7f43b4101e70 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.534+0000 7f43a77fe700 1 -- 172.21.15.49:0/2615729452 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b000f5f0 con 0x7f43b4101e70 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.534+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b41a3150 con 0x7f43b4101e70 2024-05-15T16:32:53.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.535+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b4069ab0 con 0x7f43b4101e70 2024-05-15T16:32:53.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.535+0000 7f43a77fe700 1 -- 172.21.15.49:0/2615729452 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f43b00105d0 con 0x7f43b4101e70 2024-05-15T16:32:53.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.536+0000 7f43a77fe700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f43a006e200 0x7f43a00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:53.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.536+0000 7f43a77fe700 1 -- 172.21.15.49:0/2615729452 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f43b008e170 con 0x7f43b4101e70 2024-05-15T16:32:53.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.536+0000 7f43b9b28700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f43a006e200 0x7f43a00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:53.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.536+0000 7f43b9b28700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f43a006e200 0x7f43a00706c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f43a8006010 tx=0x7f43a800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:53.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.540+0000 7f43a77fe700 1 -- 172.21.15.49:0/2615729452 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f43b0057450 con 0x7f43b4101e70 2024-05-15T16:32:53.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.768+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43b410ded0 con 0x7f43a006e200 2024-05-15T16:32:53.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.770+0000 7f43a77fe700 1 -- 172.21.15.49:0/2615729452 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f43b410ded0 con 0x7f43a006e200 2024-05-15T16:32:53.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f43a006e200 msgr2=0x7f43a00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f43a006e200 0x7f43a00706c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f43a8006010 tx=0x7f43a800b540 comp rx=0 tx=0).stop 2024-05-15T16:32:53.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4101e70 msgr2=0x7f43b40ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4101e70 0x7f43b40ffad0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f43b000d8d0 tx=0x7f43b000dc90 comp rx=0 tx=0).stop 2024-05-15T16:32:53.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 shutdown_connections 2024-05-15T16:32:53.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f43a006e200 0x7f43a00706c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43b4101e70 0x7f43b40ffad0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 --2- 172.21.15.49:0/2615729452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b4102790 0x7f43b4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 >> 172.21.15.49:0/2615729452 conn(0x7f43b40fb700 msgr2=0x7f43b4103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:53.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 shutdown_connections 2024-05-15T16:32:53.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.772+0000 7f43bc58d700 1 -- 172.21.15.49:0/2615729452 wait complete. 2024-05-15T16:32:53.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.881+0000 7fa926d24700 1 -- 172.21.15.49:0/3928738293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa920100040 msgr2=0x7fa920104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.881+0000 7fa926d24700 1 --2- 172.21.15.49:0/3928738293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa920100040 0x7fa920104880 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009910 tx=0x7fa90c009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:53.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.881+0000 7fa926d24700 1 -- 172.21.15.49:0/3928738293 shutdown_connections 2024-05-15T16:32:53.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.881+0000 7fa926d24700 1 --2- 172.21.15.49:0/3928738293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa920100040 0x7fa920104880 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.881+0000 7fa926d24700 1 --2- 172.21.15.49:0/3928738293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9200ff690 0x7fa9200ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.881+0000 7fa926d24700 1 -- 172.21.15.49:0/3928738293 >> 172.21.15.49:0/3928738293 conn(0x7fa920074eb0 msgr2=0x7fa9200752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.881+0000 7fa926d24700 1 -- 172.21.15.49:0/3928738293 shutdown_connections 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.881+0000 7fa926d24700 1 -- 172.21.15.49:0/3928738293 wait complete. 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.881+0000 7fa926d24700 1 Processor -- start 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa926d24700 1 -- start start 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa926d24700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9200ff690 0x7fa92019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa926d24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa920100040 0x7fa92019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa926d24700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92019d3a0 con 0x7fa9200ff690 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa926d24700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa920196a10 con 0x7fa920100040 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa924ac0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9200ff690 0x7fa92019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa924ac0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9200ff690 0x7fa92019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51490/0 (socket says 172.21.15.49:51490) 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa924ac0700 1 -- 172.21.15.49:0/3919122047 learned_addr learned my addr 172.21.15.49:0/3919122047 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa924ac0700 1 -- 172.21.15.49:0/3919122047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa920100040 msgr2=0x7fa92019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa924ac0700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa920100040 0x7fa92019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa924ac0700 1 -- 172.21.15.49:0/3919122047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c0095f0 con 0x7fa9200ff690 2024-05-15T16:32:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa924ac0700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9200ff690 0x7fa92019c8d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa91400c930 tx=0x7fa91400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.882+0000 7fa91dffb700 1 -- 172.21.15.49:0/3919122047 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa914007ad0 con 0x7fa9200ff690 2024-05-15T16:32:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.883+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa920196cf0 con 0x7fa9200ff690 2024-05-15T16:32:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.883+0000 7fa91dffb700 1 -- 172.21.15.49:0/3919122047 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa914007c30 con 0x7fa9200ff690 2024-05-15T16:32:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.883+0000 7fa91dffb700 1 -- 172.21.15.49:0/3919122047 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa914018610 con 0x7fa9200ff690 2024-05-15T16:32:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.883+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa920197240 con 0x7fa9200ff690 2024-05-15T16:32:53.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.884+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9201092b0 con 0x7fa9200ff690 2024-05-15T16:32:53.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.884+0000 7fa91dffb700 1 -- 172.21.15.49:0/3919122047 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa91400f450 con 0x7fa9200ff690 2024-05-15T16:32:53.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.885+0000 7fa91dffb700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa91006e200 0x7fa9100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:53.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.885+0000 7fa91dffb700 1 -- 172.21.15.49:0/3919122047 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa91408e3a0 con 0x7fa9200ff690 2024-05-15T16:32:53.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.885+0000 7fa91ffff700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa91006e200 0x7fa9100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:53.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.885+0000 7fa91ffff700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa91006e200 0x7fa9100706c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00b5c0 tx=0x7fa90c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:53.889+0000 7fa91dffb700 1 -- 172.21.15.49:0/3919122047 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa91405b1b0 con 0x7fa9200ff690 2024-05-15T16:32:54.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.104+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa920197ed0 con 0x7fa91006e200 2024-05-15T16:32:54.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.115+0000 7fa91dffb700 1 -- 172.21.15.49:0/3919122047 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa920197ed0 con 0x7fa91006e200 2024-05-15T16:32:54.115 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:32:54.115 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (36m) 13m ago 47m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:32:54.115 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (47m) 13m ago 47m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:32:54.115 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (46m) 14m ago 46m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:32:54.115 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (33m) 13m ago 47m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (33m) 14m ago 46m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (35m) 13m ago 47m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (29m) 13m ago 42m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (29m) 13m ago 42m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (29m) 14m ago 42m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (14m) 14m ago 41m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (38m) 13m ago 49m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (37m) 14m ago 46m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (34m) 13m ago 49m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (34m) 14m ago 46m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (36m) 13m ago 47m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:32:54.116 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (36m) 14m ago 46m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:32:54.117 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (33m) 13m ago 45m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:32:54.117 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (32m) 13m ago 45m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:32:54.117 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (31m) 13m ago 44m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:32:54.117 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (31m) 14m ago 44m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:32:54.117 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (30m) 14m ago 44m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:32:54.117 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (30m) 14m ago 43m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:32:54.117 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (36m) 13m ago 47m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa91006e200 msgr2=0x7fa9100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa91006e200 0x7fa9100706c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00b5c0 tx=0x7fa90c005960 comp rx=0 tx=0).stop 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9200ff690 msgr2=0x7fa92019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9200ff690 0x7fa92019c8d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa91400c930 tx=0x7fa91400ccf0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 shutdown_connections 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa91006e200 0x7fa9100706c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9200ff690 0x7fa92019c8d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 --2- 172.21.15.49:0/3919122047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa920100040 0x7fa92019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 >> 172.21.15.49:0/3919122047 conn(0x7fa920074eb0 msgr2=0x7fa920102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 shutdown_connections 2024-05-15T16:32:54.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.117+0000 7fa926d24700 1 -- 172.21.15.49:0/3919122047 wait complete. 2024-05-15T16:32:54.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.225+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2644902590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc50100f00 msgr2=0x7fdc501012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.225+0000 7fdc54f0a700 1 --2- 172.21.15.49:0/2644902590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc50100f00 0x7fdc501012e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009910 tx=0x7fdc40009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:54.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.225+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2644902590 shutdown_connections 2024-05-15T16:32:54.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.225+0000 7fdc54f0a700 1 --2- 172.21.15.49:0/2644902590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc501018b0 0x7fdc50105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.225+0000 7fdc54f0a700 1 --2- 172.21.15.49:0/2644902590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc50100f00 0x7fdc501012e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.225+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2644902590 >> 172.21.15.49:0/2644902590 conn(0x7fdc500fc7d0 msgr2=0x7fdc500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:54.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.225+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2644902590 shutdown_connections 2024-05-15T16:32:54.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.225+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2644902590 wait complete. 2024-05-15T16:32:54.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc54f0a700 1 Processor -- start 2024-05-15T16:32:54.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc54f0a700 1 -- start start 2024-05-15T16:32:54.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc54f0a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc50100f00 0x7fdc5019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:54.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc54f0a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc501018b0 0x7fdc5019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:54.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc54f0a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5019d460 con 0x7fdc501018b0 2024-05-15T16:32:54.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc54f0a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50196a90 con 0x7fdc50100f00 2024-05-15T16:32:54.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc4dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc501018b0 0x7fdc5019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:54.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc4dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc501018b0 0x7fdc5019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51498/0 (socket says 172.21.15.49:51498) 2024-05-15T16:32:54.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc4dd9b700 1 -- 172.21.15.49:0/2534497195 learned_addr learned my addr 172.21.15.49:0/2534497195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:54.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc4e59c700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc50100f00 0x7fdc5019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:54.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc4dd9b700 1 -- 172.21.15.49:0/2534497195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc50100f00 msgr2=0x7fdc5019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc4dd9b700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc50100f00 0x7fdc5019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc4dd9b700 1 -- 172.21.15.49:0/2534497195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc400095f0 con 0x7fdc501018b0 2024-05-15T16:32:54.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc4e59c700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc50100f00 0x7fdc5019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:32:54.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.226+0000 7fdc4dd9b700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc501018b0 0x7fdc5019ce40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400d900 tx=0x7fdc4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:54.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.227+0000 7fdc3f7fe700 1 -- 172.21.15.49:0/2534497195 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc44009900 con 0x7fdc501018b0 2024-05-15T16:32:54.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.227+0000 7fdc3f7fe700 1 -- 172.21.15.49:0/2534497195 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc44010460 con 0x7fdc501018b0 2024-05-15T16:32:54.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.227+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc50196d70 con 0x7fdc501018b0 2024-05-15T16:32:54.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.227+0000 7fdc3f7fe700 1 -- 172.21.15.49:0/2534497195 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4400f5f0 con 0x7fdc501018b0 2024-05-15T16:32:54.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.227+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc501972c0 con 0x7fdc501018b0 2024-05-15T16:32:54.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.228+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc501092a0 con 0x7fdc501018b0 2024-05-15T16:32:54.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.229+0000 7fdc3f7fe700 1 -- 172.21.15.49:0/2534497195 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdc440105d0 con 0x7fdc501018b0 2024-05-15T16:32:54.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.229+0000 7fdc3f7fe700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc3806e200 0x7fdc380706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:54.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.229+0000 7fdc3f7fe700 1 -- 172.21.15.49:0/2534497195 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdc4408dcf0 con 0x7fdc501018b0 2024-05-15T16:32:54.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.229+0000 7fdc4e59c700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc3806e200 0x7fdc380706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:54.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.230+0000 7fdc4e59c700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc3806e200 0x7fdc380706c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdc400059d0 tx=0x7fdc4001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:54.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.235+0000 7fdc3f7fe700 1 -- 172.21.15.49:0/2534497195 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdc4405ab80 con 0x7fdc501018b0 2024-05-15T16:32:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:54 smithi049 ceph-mon[112142]: pgmap v1077: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:54.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.533+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdc501023e0 con 0x7fdc501018b0 2024-05-15T16:32:54.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.534+0000 7fdc3f7fe700 1 -- 172.21.15.49:0/2534497195 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fdc44017280 con 0x7fdc501018b0 2024-05-15T16:32:54.534 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:32:54.534 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:32:54.535 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:32:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc3806e200 msgr2=0x7fdc380706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc3806e200 0x7fdc380706c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdc400059d0 tx=0x7fdc4001a040 comp rx=0 tx=0).stop 2024-05-15T16:32:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc501018b0 msgr2=0x7fdc5019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc501018b0 0x7fdc5019ce40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400d900 tx=0x7fdc4400dcc0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 shutdown_connections 2024-05-15T16:32:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fdc3806e200 0x7fdc380706c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc50100f00 0x7fdc5019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 --2- 172.21.15.49:0/2534497195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc501018b0 0x7fdc5019ce40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 >> 172.21.15.49:0/2534497195 conn(0x7fdc500fc7d0 msgr2=0x7fdc500fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 shutdown_connections 2024-05-15T16:32:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.536+0000 7fdc54f0a700 1 -- 172.21.15.49:0/2534497195 wait complete. 2024-05-15T16:32:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.647+0000 7fa5e16cc700 1 -- 172.21.15.49:0/23348418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 msgr2=0x7fa5dc1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.647+0000 7fa5e16cc700 1 --2- 172.21.15.49:0/23348418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 0x7fa5dc1047c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009910 tx=0x7fa5c4009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.647+0000 7fa5e16cc700 1 -- 172.21.15.49:0/23348418 shutdown_connections 2024-05-15T16:32:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.647+0000 7fa5e16cc700 1 --2- 172.21.15.49:0/23348418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 0x7fa5dc1047c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.647+0000 7fa5e16cc700 1 --2- 172.21.15.49:0/23348418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc0ffdc0 0x7fa5dc1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.647+0000 7fa5e16cc700 1 -- 172.21.15.49:0/23348418 >> 172.21.15.49:0/23348418 conn(0x7fa5dc076800 msgr2=0x7fa5dc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.647+0000 7fa5e16cc700 1 -- 172.21.15.49:0/23348418 shutdown_connections 2024-05-15T16:32:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.647+0000 7fa5e16cc700 1 -- 172.21.15.49:0/23348418 wait complete. 2024-05-15T16:32:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5e16cc700 1 Processor -- start 2024-05-15T16:32:54.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5e16cc700 1 -- start start 2024-05-15T16:32:54.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5e16cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc0ffdc0 0x7fa5dc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:54.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5e16cc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 0x7fa5dc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:54.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5e16cc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc198510 con 0x7fa5dc100770 2024-05-15T16:32:54.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5e16cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc198650 con 0x7fa5dc0ffdc0 2024-05-15T16:32:54.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5da7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 0x7fa5dc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:54.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5da7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 0x7fa5dc197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51528/0 (socket says 172.21.15.49:51528) 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5da7fc700 1 -- 172.21.15.49:0/2421519107 learned_addr learned my addr 172.21.15.49:0/2421519107 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.648+0000 7fa5daffd700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc0ffdc0 0x7fa5dc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.649+0000 7fa5da7fc700 1 -- 172.21.15.49:0/2421519107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc0ffdc0 msgr2=0x7fa5dc197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.649+0000 7fa5da7fc700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc0ffdc0 0x7fa5dc197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.649+0000 7fa5da7fc700 1 -- 172.21.15.49:0/2421519107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c40095f0 con 0x7fa5dc100770 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.649+0000 7fa5da7fc700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 0x7fa5dc197f40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009fd0 tx=0x7fa5c400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.649+0000 7fa5d3fff700 1 -- 172.21.15.49:0/2421519107 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5c401d070 con 0x7fa5dc100770 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.649+0000 7fa5d3fff700 1 -- 172.21.15.49:0/2421519107 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5c40052a0 con 0x7fa5dc100770 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.649+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc1a0a70 con 0x7fa5dc100770 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.649+0000 7fa5d3fff700 1 -- 172.21.15.49:0/2421519107 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5c4022470 con 0x7fa5dc100770 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.649+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc1a0dd0 con 0x7fa5dc100770 2024-05-15T16:32:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.650+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc1081c0 con 0x7fa5dc100770 2024-05-15T16:32:54.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.651+0000 7fa5d3fff700 1 -- 172.21.15.49:0/2421519107 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa5c400f770 con 0x7fa5dc100770 2024-05-15T16:32:54.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.651+0000 7fa5d3fff700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa5c806e200 0x7fa5c80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:54.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.651+0000 7fa5d3fff700 1 -- 172.21.15.49:0/2421519107 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa5c408f570 con 0x7fa5dc100770 2024-05-15T16:32:54.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.651+0000 7fa5daffd700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa5c806e200 0x7fa5c80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:54.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.652+0000 7fa5daffd700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa5c806e200 0x7fa5c80706c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009380 tx=0x7fa5cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:54.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.656+0000 7fa5d3fff700 1 -- 172.21.15.49:0/2421519107 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa5c4058850 con 0x7fa5dc100770 2024-05-15T16:32:54.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:54 smithi144 ceph-mon[86635]: pgmap v1077: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:54.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.907+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa5dc1012a0 con 0x7fa5dc100770 2024-05-15T16:32:54.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.908+0000 7fa5d3fff700 1 -- 172.21.15.49:0/2421519107 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fa5c405bf10 con 0x7fa5dc100770 2024-05-15T16:32:54.908 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:32:54.908 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:32:54.908 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:32:54.908 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:32:54.908 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:32:54.908 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:32:54.908 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:32:54.909 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:32:54.910 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:32:54.911 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:32:54.911 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:32:54.911 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:32:54.911 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:32:54.911 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:32:54.911 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:32:54.911 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa5c806e200 msgr2=0x7fa5c80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa5c806e200 0x7fa5c80706c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009380 tx=0x7fa5cc00b040 comp rx=0 tx=0).stop 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 msgr2=0x7fa5dc197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 0x7fa5dc197f40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009fd0 tx=0x7fa5c400b650 comp rx=0 tx=0).stop 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 shutdown_connections 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa5c806e200 0x7fa5c80706c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc0ffdc0 0x7fa5dc197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 --2- 172.21.15.49:0/2421519107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5dc100770 0x7fa5dc197f40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 >> 172.21.15.49:0/2421519107 conn(0x7fa5dc076800 msgr2=0x7fa5dc103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 shutdown_connections 2024-05-15T16:32:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:54.910+0000 7fa5e16cc700 1 -- 172.21.15.49:0/2421519107 wait complete. 2024-05-15T16:32:54.913 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:32:55.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.019+0000 7f79a6bae700 1 -- 172.21.15.49:0/1389738610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a00ffdc0 msgr2=0x7f79a01001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:55.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.019+0000 7f79a6bae700 1 --2- 172.21.15.49:0/1389738610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a00ffdc0 0x7f79a01001a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7988009910 tx=0x7f7988009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:55.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.019+0000 7f79a6bae700 1 -- 172.21.15.49:0/1389738610 shutdown_connections 2024-05-15T16:32:55.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.019+0000 7f79a6bae700 1 --2- 172.21.15.49:0/1389738610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a0100770 0x7f79a01047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.019+0000 7f79a6bae700 1 --2- 172.21.15.49:0/1389738610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a00ffdc0 0x7f79a01001a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.019+0000 7f79a6bae700 1 -- 172.21.15.49:0/1389738610 >> 172.21.15.49:0/1389738610 conn(0x7f79a0076800 msgr2=0x7f79a00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:55.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.019+0000 7f79a6bae700 1 -- 172.21.15.49:0/1389738610 shutdown_connections 2024-05-15T16:32:55.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.019+0000 7f79a6bae700 1 -- 172.21.15.49:0/1389738610 wait complete. 2024-05-15T16:32:55.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f79a6bae700 1 Processor -- start 2024-05-15T16:32:55.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f79a6bae700 1 -- start start 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f79a6bae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a00ffdc0 0x7f79a0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f79a6bae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a0100770 0x7f79a0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f79a6bae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a01984a0 con 0x7f79a0100770 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f79a6bae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a01985e0 con 0x7f79a00ffdc0 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f799ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a0100770 0x7f79a0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f799ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a0100770 0x7f79a0197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51544/0 (socket says 172.21.15.49:51544) 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f799ffff700 1 -- 172.21.15.49:0/1331170152 learned_addr learned my addr 172.21.15.49:0/1331170152 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.020+0000 7f79a494a700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a00ffdc0 0x7f79a0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.021+0000 7f799ffff700 1 -- 172.21.15.49:0/1331170152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a00ffdc0 msgr2=0x7f79a0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.021+0000 7f799ffff700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a00ffdc0 0x7f79a0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.021+0000 7f799ffff700 1 -- 172.21.15.49:0/1331170152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880095f0 con 0x7f79a0100770 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.021+0000 7f799ffff700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a0100770 0x7f79a0197f60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f799000c960 tx=0x7f799000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.021+0000 7f799dffb700 1 -- 172.21.15.49:0/1331170152 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7990007a70 con 0x7f79a0100770 2024-05-15T16:32:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.021+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a01a0a90 con 0x7f79a0100770 2024-05-15T16:32:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.021+0000 7f799dffb700 1 -- 172.21.15.49:0/1331170152 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7990007bd0 con 0x7f79a0100770 2024-05-15T16:32:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.021+0000 7f799dffb700 1 -- 172.21.15.49:0/1331170152 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7990018630 con 0x7f79a0100770 2024-05-15T16:32:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.021+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a01a0e40 con 0x7f79a0100770 2024-05-15T16:32:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.022+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a0108160 con 0x7f79a0100770 2024-05-15T16:32:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.023+0000 7f799dffb700 1 -- 172.21.15.49:0/1331170152 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f799000f450 con 0x7f79a0100770 2024-05-15T16:32:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.023+0000 7f799dffb700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f798c06e200 0x7f798c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.023+0000 7f799dffb700 1 -- 172.21.15.49:0/1331170152 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f799008e060 con 0x7f79a0100770 2024-05-15T16:32:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.023+0000 7f79a494a700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f798c06e200 0x7f798c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.024+0000 7f79a494a700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f798c06e200 0x7f798c0706c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7988006010 tx=0x7f798801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:55.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.028+0000 7f799dffb700 1 -- 172.21.15.49:0/1331170152 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7990057470 con 0x7f79a0100770 2024-05-15T16:32:55.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.255+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79a0068c90 con 0x7f798c06e200 2024-05-15T16:32:55.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.256+0000 7f799dffb700 1 -- 172.21.15.49:0/1331170152 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f79a0068c90 con 0x7f798c06e200 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:32:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:32:55.258 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:32:55.258 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:32:55.258 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:32:55.258 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:32:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f798c06e200 msgr2=0x7f798c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f798c06e200 0x7f798c0706c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7988006010 tx=0x7f798801a040 comp rx=0 tx=0).stop 2024-05-15T16:32:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a0100770 msgr2=0x7f79a0197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a0100770 0x7f79a0197f60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f799000c960 tx=0x7f799000cc70 comp rx=0 tx=0).stop 2024-05-15T16:32:55.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 shutdown_connections 2024-05-15T16:32:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f798c06e200 0x7f798c0706c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a00ffdc0 0x7f79a0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 --2- 172.21.15.49:0/1331170152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79a0100770 0x7f79a0197f60 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 >> 172.21.15.49:0/1331170152 conn(0x7f79a0076800 msgr2=0x7f79a0103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 shutdown_connections 2024-05-15T16:32:55.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.259+0000 7f79a6bae700 1 -- 172.21.15.49:0/1331170152 wait complete. 2024-05-15T16:32:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.367+0000 7f5d36576700 1 -- 172.21.15.49:0/3888813242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 msgr2=0x7f5d30103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.367+0000 7f5d36576700 1 --2- 172.21.15.49:0/3888813242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 0x7f5d30103fa0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009910 tx=0x7f5d24009c20 comp rx=0 tx=0).stop 2024-05-15T16:32:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.367+0000 7f5d36576700 1 -- 172.21.15.49:0/3888813242 shutdown_connections 2024-05-15T16:32:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.367+0000 7f5d36576700 1 --2- 172.21.15.49:0/3888813242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 0x7f5d30103fa0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.367+0000 7f5d36576700 1 --2- 172.21.15.49:0/3888813242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff5a0 0x7f5d300ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.367+0000 7f5d36576700 1 -- 172.21.15.49:0/3888813242 >> 172.21.15.49:0/3888813242 conn(0x7f5d30076000 msgr2=0x7f5d300fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:55.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.367+0000 7f5d36576700 1 -- 172.21.15.49:0/3888813242 shutdown_connections 2024-05-15T16:32:55.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.367+0000 7f5d36576700 1 -- 172.21.15.49:0/3888813242 wait complete. 2024-05-15T16:32:55.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.368+0000 7f5d36576700 1 Processor -- start 2024-05-15T16:32:55.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.368+0000 7f5d36576700 1 -- start start 2024-05-15T16:32:55.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.368+0000 7f5d36576700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff5a0 0x7f5d30198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:55.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.368+0000 7f5d36576700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 0x7f5d30199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:55.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.368+0000 7f5d36576700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30199a60 con 0x7f5d300fff50 2024-05-15T16:32:55.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.368+0000 7f5d36576700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3019d6e0 con 0x7f5d300ff5a0 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 0x7f5d30199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 0x7f5d30199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51554/0 (socket says 172.21.15.49:51554) 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2f7fe700 1 -- 172.21.15.49:0/1407246967 learned_addr learned my addr 172.21.15.49:0/1407246967 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2ffff700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff5a0 0x7f5d30198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2f7fe700 1 -- 172.21.15.49:0/1407246967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff5a0 msgr2=0x7f5d30198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2f7fe700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff5a0 0x7f5d30198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2f7fe700 1 -- 172.21.15.49:0/1407246967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d240095f0 con 0x7f5d300fff50 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2f7fe700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 0x7f5d30199380 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400bd70 tx=0x7f5d2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2d7fa700 1 -- 172.21.15.49:0/1407246967 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2401d070 con 0x7f5d300fff50 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2d7fa700 1 -- 172.21.15.49:0/1407246967 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d240056e0 con 0x7f5d300fff50 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3019d960 con 0x7f5d300fff50 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d2d7fa700 1 -- 172.21.15.49:0/1407246967 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d24017400 con 0x7f5d300fff50 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.369+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3019de50 con 0x7f5d300fff50 2024-05-15T16:32:55.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.370+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d30100a80 con 0x7f5d300fff50 2024-05-15T16:32:55.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.371+0000 7f5d2d7fa700 1 -- 172.21.15.49:0/1407246967 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5d2400f460 con 0x7f5d300fff50 2024-05-15T16:32:55.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.371+0000 7f5d2d7fa700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d1806e200 0x7f5d180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:32:55.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.371+0000 7f5d2d7fa700 1 -- 172.21.15.49:0/1407246967 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5d2408f610 con 0x7f5d300fff50 2024-05-15T16:32:55.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.371+0000 7f5d2ffff700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d1806e200 0x7f5d180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:32:55.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.372+0000 7f5d2ffff700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d1806e200 0x7f5d180706c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5d20005950 tx=0x7f5d200058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:32:55.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.376+0000 7f5d2d7fa700 1 -- 172.21.15.49:0/1407246967 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5d24058950 con 0x7f5d300fff50 2024-05-15T16:32:55.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.669+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5d3010c560 con 0x7f5d300fff50 2024-05-15T16:32:55.670 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:55 smithi049 ceph-mon[112142]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:55.670 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:55 smithi049 ceph-mon[112142]: from='client.35946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:55.670 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:55 smithi049 ceph-mon[112142]: from='client.35950 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:55.670 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:55 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2534497195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:32:55.670 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:55 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2421519107' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:32:55.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.669+0000 7f5d2d7fa700 1 -- 172.21.15.49:0/1407246967 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f5d2405c010 con 0x7f5d300fff50 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:32:55.671 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:32:55.672 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:32:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d1806e200 msgr2=0x7f5d180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d1806e200 0x7f5d180706c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5d20005950 tx=0x7f5d200058e0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 msgr2=0x7f5d30199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:32:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 0x7f5d30199380 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400bd70 tx=0x7f5d2400be50 comp rx=0 tx=0).stop 2024-05-15T16:32:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 shutdown_connections 2024-05-15T16:32:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5d1806e200 0x7f5d180706c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff5a0 0x7f5d30198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 --2- 172.21.15.49:0/1407246967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d300fff50 0x7f5d30199380 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:32:55.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 >> 172.21.15.49:0/1407246967 conn(0x7f5d30076000 msgr2=0x7f5d300fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:32:55.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 shutdown_connections 2024-05-15T16:32:55.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:32:55.672+0000 7f5d36576700 1 -- 172.21.15.49:0/1407246967 wait complete. 2024-05-15T16:32:55.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:55 smithi144 ceph-mon[86635]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:55.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:55 smithi144 ceph-mon[86635]: from='client.35946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:55.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:55 smithi144 ceph-mon[86635]: from='client.35950 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:55.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:55 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2534497195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:32:55.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:55 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2421519107' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:32:56.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:56 smithi144 ceph-mon[86635]: pgmap v1078: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:56.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:56 smithi144 ceph-mon[86635]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:56.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:56 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1407246967' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:32:56.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:56 smithi049 ceph-mon[112142]: pgmap v1078: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:56.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:56 smithi049 ceph-mon[112142]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:32:56.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:56 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1407246967' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:32:58.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:32:58 smithi144 ceph-mon[86635]: pgmap v1079: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:32:58.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:32:58 smithi049 ceph-mon[112142]: pgmap v1079: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:00.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:00 smithi144 ceph-mon[86635]: pgmap v1080: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:00.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:00 smithi049 ceph-mon[112142]: pgmap v1080: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:02.549 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:02 smithi144 ceph-mon[86635]: pgmap v1081: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:02.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:02 smithi049 ceph-mon[112142]: pgmap v1081: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:04.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:04 smithi144 ceph-mon[86635]: pgmap v1082: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:04.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:04 smithi049 ceph-mon[112142]: pgmap v1082: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:05.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:33:05.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:33:06.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:06 smithi144 ceph-mon[86635]: pgmap v1083: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:06.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:06 smithi049 ceph-mon[112142]: pgmap v1083: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:08.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:08 smithi144 ceph-mon[86635]: pgmap v1084: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:08.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:08 smithi049 ceph-mon[112142]: pgmap v1084: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:10.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:10 smithi144 ceph-mon[86635]: pgmap v1085: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:10.723 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:10 smithi049 ceph-mon[112142]: pgmap v1085: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:12.655 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:12 smithi144 ceph-mon[86635]: pgmap v1086: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:12.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:12 smithi049 ceph-mon[112142]: pgmap v1086: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:13.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:13 smithi144 ceph-mon[86635]: pgmap v1087: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:13.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:13 smithi049 ceph-mon[112142]: pgmap v1087: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:15 smithi144 ceph-mon[86635]: pgmap v1088: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:15 smithi049 ceph-mon[112142]: pgmap v1088: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:17 smithi144 ceph-mon[86635]: pgmap v1089: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:17 smithi049 ceph-mon[112142]: pgmap v1089: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:19 smithi144 ceph-mon[86635]: pgmap v1090: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:19 smithi049 ceph-mon[112142]: pgmap v1090: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:21.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:33:21.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:33:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:21 smithi144 ceph-mon[86635]: pgmap v1091: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:21 smithi049 ceph-mon[112142]: pgmap v1091: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:23 smithi144 ceph-mon[86635]: pgmap v1092: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:23 smithi049 ceph-mon[112142]: pgmap v1092: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.795+0000 7f234ead7700 1 -- 172.21.15.49:0/2018883177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 msgr2=0x7f2348105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.795+0000 7f234ead7700 1 --2- 172.21.15.49:0/2018883177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 0x7f2348105900 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2334009910 tx=0x7f2334009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.795+0000 7f234ead7700 1 -- 172.21.15.49:0/2018883177 shutdown_connections 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.795+0000 7f234ead7700 1 --2- 172.21.15.49:0/2018883177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 0x7f2348105900 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.795+0000 7f234ead7700 1 --2- 172.21.15.49:0/2018883177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2348100f00 0x7f23481012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.795+0000 7f234ead7700 1 -- 172.21.15.49:0/2018883177 >> 172.21.15.49:0/2018883177 conn(0x7f23480fc7d0 msgr2=0x7f23480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.795+0000 7f234ead7700 1 -- 172.21.15.49:0/2018883177 shutdown_connections 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.795+0000 7f234ead7700 1 -- 172.21.15.49:0/2018883177 wait complete. 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f234ead7700 1 Processor -- start 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f234ead7700 1 -- start start 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f234ead7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2348100f00 0x7f234819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f234ead7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 0x7f234819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f234ead7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234819d440 con 0x7f23481018b0 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f234ead7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2348196a20 con 0x7f2348100f00 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f2347fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 0x7f234819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f2347fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 0x7f234819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38622/0 (socket says 172.21.15.49:38622) 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f2347fff700 1 -- 172.21.15.49:0/2356084762 learned_addr learned my addr 172.21.15.49:0/2356084762 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f234c873700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2348100f00 0x7f234819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f2347fff700 1 -- 172.21.15.49:0/2356084762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2348100f00 msgr2=0x7f234819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f2347fff700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2348100f00 0x7f234819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.796+0000 7f2347fff700 1 -- 172.21.15.49:0/2356084762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340095f0 con 0x7f23481018b0 2024-05-15T16:33:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.797+0000 7f2347fff700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 0x7f234819ce20 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f233400bd70 tx=0x7f233400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.797+0000 7f2345ffb700 1 -- 172.21.15.49:0/2356084762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f233401d070 con 0x7f23481018b0 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.797+0000 7f2345ffb700 1 -- 172.21.15.49:0/2356084762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23340056e0 con 0x7f23481018b0 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.797+0000 7f2345ffb700 1 -- 172.21.15.49:0/2356084762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2334017400 con 0x7f23481018b0 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.797+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2348196ca0 con 0x7f23481018b0 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.797+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2348197190 con 0x7f23481018b0 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.798+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2348109300 con 0x7f23481018b0 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.799+0000 7f2345ffb700 1 -- 172.21.15.49:0/2356084762 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f233400f460 con 0x7f23481018b0 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.799+0000 7f2345ffb700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f233806e200 0x7f23380706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.799+0000 7f2345ffb700 1 -- 172.21.15.49:0/2356084762 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f233408f5b0 con 0x7f23481018b0 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.799+0000 7f234c873700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f233806e200 0x7f23380706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.800+0000 7f234c873700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f233806e200 0x7f23380706c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f233c005950 tx=0x7f233c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:25.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:25.804+0000 7f2345ffb700 1 -- 172.21.15.49:0/2356084762 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f233405c480 con 0x7f23481018b0 2024-05-15T16:33:26.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.032+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2348197f50 con 0x7f233806e200 2024-05-15T16:33:26.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.034+0000 7f2345ffb700 1 -- 172.21.15.49:0/2356084762 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2348197f50 con 0x7f233806e200 2024-05-15T16:33:26.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f233806e200 msgr2=0x7f23380706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f233806e200 0x7f23380706c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f233c005950 tx=0x7f233c0058e0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 msgr2=0x7f234819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 0x7f234819ce20 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f233400bd70 tx=0x7f233400be50 comp rx=0 tx=0).stop 2024-05-15T16:33:26.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 shutdown_connections 2024-05-15T16:33:26.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f233806e200 0x7f23380706c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2348100f00 0x7f234819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 --2- 172.21.15.49:0/2356084762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f23481018b0 0x7f234819ce20 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 >> 172.21.15.49:0/2356084762 conn(0x7f23480fc7d0 msgr2=0x7f23480fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:26.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 shutdown_connections 2024-05-15T16:33:26.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.036+0000 7f234ead7700 1 -- 172.21.15.49:0/2356084762 wait complete. 2024-05-15T16:33:26.047 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:33:26.047 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:25 smithi049 ceph-mon[112142]: pgmap v1093: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.145+0000 7f7236d6d700 1 -- 172.21.15.49:0/3811810818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 msgr2=0x7f7230105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.145+0000 7f7236d6d700 1 --2- 172.21.15.49:0/3811810818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 0x7f7230105900 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f721c009910 tx=0x7f721c009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.145+0000 7f7236d6d700 1 -- 172.21.15.49:0/3811810818 shutdown_connections 2024-05-15T16:33:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.145+0000 7f7236d6d700 1 --2- 172.21.15.49:0/3811810818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 0x7f7230105900 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.145+0000 7f7236d6d700 1 --2- 172.21.15.49:0/3811810818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7230100f00 0x7f72301012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.145+0000 7f7236d6d700 1 -- 172.21.15.49:0/3811810818 >> 172.21.15.49:0/3811810818 conn(0x7f72300fc7d0 msgr2=0x7f72300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.145+0000 7f7236d6d700 1 -- 172.21.15.49:0/3811810818 shutdown_connections 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.145+0000 7f7236d6d700 1 -- 172.21.15.49:0/3811810818 wait complete. 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f7236d6d700 1 Processor -- start 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f7236d6d700 1 -- start start 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f7236d6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7230100f00 0x7f7230198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f7236d6d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 0x7f7230199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f7236d6d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72301999e0 con 0x7f72301018b0 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f7236d6d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723019d6b0 con 0x7f7230100f00 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f722ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 0x7f7230199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f722ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 0x7f7230199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38636/0 (socket says 172.21.15.49:38636) 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f722ffff700 1 -- 172.21.15.49:0/1959819379 learned_addr learned my addr 172.21.15.49:0/1959819379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f7234b09700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7230100f00 0x7f7230198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f722ffff700 1 -- 172.21.15.49:0/1959819379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7230100f00 msgr2=0x7f7230198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f722ffff700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7230100f00 0x7f7230198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.146+0000 7f722ffff700 1 -- 172.21.15.49:0/1959819379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c0095f0 con 0x7f72301018b0 2024-05-15T16:33:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.147+0000 7f722ffff700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 0x7f7230199300 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f721c00bd70 tx=0x7f721c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.147+0000 7f722dffb700 1 -- 172.21.15.49:0/1959819379 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f721c01d070 con 0x7f72301018b0 2024-05-15T16:33:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.147+0000 7f722dffb700 1 -- 172.21.15.49:0/1959819379 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f721c0056e0 con 0x7f72301018b0 2024-05-15T16:33:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.147+0000 7f722dffb700 1 -- 172.21.15.49:0/1959819379 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f721c017400 con 0x7f72301018b0 2024-05-15T16:33:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.147+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723019d930 con 0x7f72301018b0 2024-05-15T16:33:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.147+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723019de20 con 0x7f72301018b0 2024-05-15T16:33:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.148+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7230105080 con 0x7f72301018b0 2024-05-15T16:33:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.149+0000 7f722dffb700 1 -- 172.21.15.49:0/1959819379 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f721c00f460 con 0x7f72301018b0 2024-05-15T16:33:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.149+0000 7f722dffb700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f722006e200 0x7f72200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.149+0000 7f722dffb700 1 -- 172.21.15.49:0/1959819379 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f721c08f4c0 con 0x7f72301018b0 2024-05-15T16:33:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.149+0000 7f7234b09700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f722006e200 0x7f72200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.150+0000 7f7234b09700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f722006e200 0x7f72200706c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7224005950 tx=0x7f72240058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.154+0000 7f722dffb700 1 -- 172.21.15.49:0/1959819379 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f721c0588e0 con 0x7f72301018b0 2024-05-15T16:33:26.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:25 smithi144 ceph-mon[86635]: pgmap v1093: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:26.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.380+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f723010dea0 con 0x7f722006e200 2024-05-15T16:33:26.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.381+0000 7f722dffb700 1 -- 172.21.15.49:0/1959819379 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f723010dea0 con 0x7f722006e200 2024-05-15T16:33:26.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f722006e200 msgr2=0x7f72200706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f722006e200 0x7f72200706c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7224005950 tx=0x7f72240058e0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 msgr2=0x7f7230199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 0x7f7230199300 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f721c00bd70 tx=0x7f721c00be50 comp rx=0 tx=0).stop 2024-05-15T16:33:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 shutdown_connections 2024-05-15T16:33:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f722006e200 0x7f72200706c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7230100f00 0x7f7230198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 --2- 172.21.15.49:0/1959819379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72301018b0 0x7f7230199300 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 >> 172.21.15.49:0/1959819379 conn(0x7f72300fc7d0 msgr2=0x7f72300fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 shutdown_connections 2024-05-15T16:33:26.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.383+0000 7f7236d6d700 1 -- 172.21.15.49:0/1959819379 wait complete. 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.492+0000 7fa4914b2700 1 -- 172.21.15.49:0/2850461711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 msgr2=0x7fa48c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.492+0000 7fa4914b2700 1 --2- 172.21.15.49:0/2850461711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 0x7fa48c1001a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009910 tx=0x7fa47c009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.492+0000 7fa4914b2700 1 -- 172.21.15.49:0/2850461711 shutdown_connections 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.492+0000 7fa4914b2700 1 --2- 172.21.15.49:0/2850461711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c100770 0x7fa48c1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.492+0000 7fa4914b2700 1 --2- 172.21.15.49:0/2850461711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 0x7fa48c1001a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.492+0000 7fa4914b2700 1 -- 172.21.15.49:0/2850461711 >> 172.21.15.49:0/2850461711 conn(0x7fa48c076800 msgr2=0x7fa48c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.492+0000 7fa4914b2700 1 -- 172.21.15.49:0/2850461711 shutdown_connections 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.492+0000 7fa4914b2700 1 -- 172.21.15.49:0/2850461711 wait complete. 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa4914b2700 1 Processor -- start 2024-05-15T16:33:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa4914b2700 1 -- start start 2024-05-15T16:33:26.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa4914b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 0x7fa48c0fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:26.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa4914b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c100770 0x7fa48c0fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:26.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa4914b2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0fc760 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa4914b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0fc8a0 con 0x7fa48c100770 2024-05-15T16:33:26.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa48affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 0x7fa48c0fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:26.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa48affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 0x7fa48c0fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38660/0 (socket says 172.21.15.49:38660) 2024-05-15T16:33:26.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa48affd700 1 -- 172.21.15.49:0/4270716774 learned_addr learned my addr 172.21.15.49:0/4270716774 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:26.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa48a7fc700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c100770 0x7fa48c0fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa48affd700 1 -- 172.21.15.49:0/4270716774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c100770 msgr2=0x7fa48c0fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.493+0000 7fa48affd700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c100770 0x7fa48c0fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.494+0000 7fa48affd700 1 -- 172.21.15.49:0/4270716774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0095f0 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.494+0000 7fa48affd700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 0x7fa48c0fbce0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa47c006010 tx=0x7fa47c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.494+0000 7fa473fff700 1 -- 172.21.15.49:0/4270716774 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47c01d070 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.494+0000 7fa473fff700 1 -- 172.21.15.49:0/4270716774 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa47c0056e0 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.494+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c0fcb20 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.494+0000 7fa473fff700 1 -- 172.21.15.49:0/4270716774 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47c017400 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.494+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c1a3020 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.495+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c1012a0 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.495+0000 7fa473fff700 1 -- 172.21.15.49:0/4270716774 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa47c00f6f0 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.496+0000 7fa473fff700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa47406e200 0x7fa4740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:26.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.496+0000 7fa48a7fc700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa47406e200 0x7fa4740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:26.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.496+0000 7fa473fff700 1 -- 172.21.15.49:0/4270716774 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa47c08f550 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.497+0000 7fa48a7fc700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa47406e200 0x7fa4740706c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa480005950 tx=0x7fa4800058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.501+0000 7fa473fff700 1 -- 172.21.15.49:0/4270716774 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa47c058830 con 0x7fa48c0ffdc0 2024-05-15T16:33:26.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.719+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa48c10ce90 con 0x7fa47406e200 2024-05-15T16:33:26.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.730+0000 7fa473fff700 1 -- 172.21.15.49:0/4270716774 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa48c10ce90 con 0x7fa47406e200 2024-05-15T16:33:26.730 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:33:26.730 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (36m) 14m ago 48m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:33:26.730 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (48m) 14m ago 48m 10.3M - 18.2.2-1343-g2965081f a04a73ffe079 d74fc8c26a96 2024-05-15T16:33:26.730 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (47m) 14m ago 47m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (34m) 14m ago 48m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (33m) 14m ago 47m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (36m) 14m ago 47m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (30m) 14m ago 42m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (30m) 14m ago 42m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (29m) 14m ago 42m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (14m) 14m ago 42m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (38m) 14m ago 49m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (37m) 14m ago 47m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (35m) 14m ago 49m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:33:26.731 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (34m) 14m ago 47m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:33:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (37m) 14m ago 48m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:33:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (37m) 14m ago 47m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:33:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (33m) 14m ago 46m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:33:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (33m) 14m ago 45m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:33:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (32m) 14m ago 45m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:33:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (31m) 14m ago 45m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:33:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (31m) 14m ago 44m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:33:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (30m) 14m ago 44m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:33:26.732 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (36m) 14m ago 47m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:33:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.732+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa47406e200 msgr2=0x7fa4740706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.732+0000 7fa4914b2700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa47406e200 0x7fa4740706c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa480005950 tx=0x7fa4800058e0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.732+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 msgr2=0x7fa48c0fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.732+0000 7fa4914b2700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 0x7fa48c0fbce0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa47c006010 tx=0x7fa47c00be20 comp rx=0 tx=0).stop 2024-05-15T16:33:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.732+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 shutdown_connections 2024-05-15T16:33:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.732+0000 7fa4914b2700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fa47406e200 0x7fa4740706c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.732+0000 7fa4914b2700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa48c0ffdc0 0x7fa48c0fbce0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.732+0000 7fa4914b2700 1 --2- 172.21.15.49:0/4270716774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c100770 0x7fa48c0fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.732+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 >> 172.21.15.49:0/4270716774 conn(0x7fa48c076800 msgr2=0x7fa48c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:26.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.733+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 shutdown_connections 2024-05-15T16:33:26.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.733+0000 7fa4914b2700 1 -- 172.21.15.49:0/4270716774 wait complete. 2024-05-15T16:33:26.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.840+0000 7f6fe8694700 1 -- 172.21.15.49:0/3822203012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe00ffe10 msgr2=0x7f6fe01001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.840+0000 7f6fe8694700 1 --2- 172.21.15.49:0/3822203012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe00ffe10 0x7f6fe01001f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009910 tx=0x7f6fdc009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.840+0000 7f6fe8694700 1 -- 172.21.15.49:0/3822203012 shutdown_connections 2024-05-15T16:33:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.840+0000 7f6fe8694700 1 --2- 172.21.15.49:0/3822203012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe01007c0 0x7f6fe0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.840+0000 7f6fe8694700 1 --2- 172.21.15.49:0/3822203012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe00ffe10 0x7f6fe01001f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.840+0000 7f6fe8694700 1 -- 172.21.15.49:0/3822203012 >> 172.21.15.49:0/3822203012 conn(0x7f6fe00fb700 msgr2=0x7f6fe00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.840+0000 7f6fe8694700 1 -- 172.21.15.49:0/3822203012 shutdown_connections 2024-05-15T16:33:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.840+0000 7f6fe8694700 1 -- 172.21.15.49:0/3822203012 wait complete. 2024-05-15T16:33:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe8694700 1 Processor -- start 2024-05-15T16:33:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe8694700 1 -- start start 2024-05-15T16:33:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe8694700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe00ffe10 0x7f6fe0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe8694700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe01007c0 0x7f6fe0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe8694700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe0198570 con 0x7f6fe01007c0 2024-05-15T16:33:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe8694700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe01986b0 con 0x7f6fe00ffe10 2024-05-15T16:33:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe5c2f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe01007c0 0x7f6fe0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe5c2f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe01007c0 0x7f6fe0197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38670/0 (socket says 172.21.15.49:38670) 2024-05-15T16:33:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe5c2f700 1 -- 172.21.15.49:0/3862489747 learned_addr learned my addr 172.21.15.49:0/3862489747 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.841+0000 7f6fe6430700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe00ffe10 0x7f6fe0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.842+0000 7f6fe5c2f700 1 -- 172.21.15.49:0/3862489747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe00ffe10 msgr2=0x7f6fe0197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.842+0000 7f6fe5c2f700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe00ffe10 0x7f6fe0197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.842+0000 7f6fe5c2f700 1 -- 172.21.15.49:0/3862489747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc0095f0 con 0x7f6fe01007c0 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.842+0000 7f6fe5c2f700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe01007c0 0x7f6fe0197fa0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000d900 tx=0x7f6fd000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.842+0000 7f6fd77fe700 1 -- 172.21.15.49:0/3862489747 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fd0009900 con 0x7f6fe01007c0 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.842+0000 7f6fd77fe700 1 -- 172.21.15.49:0/3862489747 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fd0010460 con 0x7f6fe01007c0 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.842+0000 7f6fd77fe700 1 -- 172.21.15.49:0/3862489747 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fd000f5f0 con 0x7f6fe01007c0 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.842+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe01a0ad0 con 0x7f6fe01007c0 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.842+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe01a0ee0 con 0x7f6fe01007c0 2024-05-15T16:33:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.843+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe01081b0 con 0x7f6fe01007c0 2024-05-15T16:33:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.844+0000 7f6fd77fe700 1 -- 172.21.15.49:0/3862489747 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6fd00105d0 con 0x7f6fe01007c0 2024-05-15T16:33:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.844+0000 7f6fd77fe700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6fcc06e200 0x7f6fcc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.844+0000 7f6fd77fe700 1 -- 172.21.15.49:0/3862489747 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6fd008dcf0 con 0x7f6fe01007c0 2024-05-15T16:33:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.844+0000 7f6fe6430700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6fcc06e200 0x7f6fcc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.845+0000 7f6fe6430700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6fcc06e200 0x7f6fcc0706c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00b5c0 tx=0x7f6fdc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:26.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:26.849+0000 7f6fd77fe700 1 -- 172.21.15.49:0/3862489747 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6fd0057050 con 0x7f6fe01007c0 2024-05-15T16:33:27.074 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:26 smithi049 ceph-mon[112142]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.151+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6fe0069770 con 0x7f6fe01007c0 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.151+0000 7f6fd77fe700 1 -- 172.21.15.49:0/3862489747 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6fd0017280 con 0x7f6fe01007c0 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:33:27.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:33:27.153 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:33:27.153 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:33:27.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:33:27.153 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:33:27.153 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:33:27.153 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:33:27.153 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:33:27.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6fcc06e200 msgr2=0x7f6fcc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6fcc06e200 0x7f6fcc0706c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00b5c0 tx=0x7f6fdc01a040 comp rx=0 tx=0).stop 2024-05-15T16:33:27.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe01007c0 msgr2=0x7f6fe0197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe01007c0 0x7f6fe0197fa0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000d900 tx=0x7f6fd000dcc0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 shutdown_connections 2024-05-15T16:33:27.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6fcc06e200 0x7f6fcc0706c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe00ffe10 0x7f6fe0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 --2- 172.21.15.49:0/3862489747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe01007c0 0x7f6fe0197fa0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 >> 172.21.15.49:0/3862489747 conn(0x7f6fe00fb700 msgr2=0x7f6fe00fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 shutdown_connections 2024-05-15T16:33:27.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.154+0000 7f6fe8694700 1 -- 172.21.15.49:0/3862489747 wait complete. 2024-05-15T16:33:27.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:26 smithi144 ceph-mon[86635]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:27.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.262+0000 7fd617be5700 1 -- 172.21.15.49:0/1678441454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd610100040 msgr2=0x7fd610104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.262+0000 7fd617be5700 1 --2- 172.21.15.49:0/1678441454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd610100040 0x7fd610104880 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009910 tx=0x7fd60c009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.262+0000 7fd617be5700 1 -- 172.21.15.49:0/1678441454 shutdown_connections 2024-05-15T16:33:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.262+0000 7fd617be5700 1 --2- 172.21.15.49:0/1678441454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd610100040 0x7fd610104880 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.262+0000 7fd617be5700 1 --2- 172.21.15.49:0/1678441454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6100ff690 0x7fd6100ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.262+0000 7fd617be5700 1 -- 172.21.15.49:0/1678441454 >> 172.21.15.49:0/1678441454 conn(0x7fd610074eb0 msgr2=0x7fd6100752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.262+0000 7fd617be5700 1 -- 172.21.15.49:0/1678441454 shutdown_connections 2024-05-15T16:33:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.262+0000 7fd617be5700 1 -- 172.21.15.49:0/1678441454 wait complete. 2024-05-15T16:33:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd617be5700 1 Processor -- start 2024-05-15T16:33:27.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd617be5700 1 -- start start 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd617be5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6100ff690 0x7fd61019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd617be5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd610100040 0x7fd61019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd617be5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019d4b0 con 0x7fd6100ff690 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd617be5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd610196a90 con 0x7fd610100040 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd615981700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6100ff690 0x7fd61019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd615981700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6100ff690 0x7fd61019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38680/0 (socket says 172.21.15.49:38680) 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd615981700 1 -- 172.21.15.49:0/2317385734 learned_addr learned my addr 172.21.15.49:0/2317385734 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd615981700 1 -- 172.21.15.49:0/2317385734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd610100040 msgr2=0x7fd61019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd615981700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd610100040 0x7fd61019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd615981700 1 -- 172.21.15.49:0/2317385734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c0095f0 con 0x7fd6100ff690 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd615981700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6100ff690 0x7fd61019c950 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b4b0 tx=0x7fd60000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:27.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.263+0000 7fd606ffd700 1 -- 172.21.15.49:0/2317385734 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd600011800 con 0x7fd6100ff690 2024-05-15T16:33:27.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.264+0000 7fd606ffd700 1 -- 172.21.15.49:0/2317385734 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd600011e20 con 0x7fd6100ff690 2024-05-15T16:33:27.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.264+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd610196d70 con 0x7fd6100ff690 2024-05-15T16:33:27.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.264+0000 7fd606ffd700 1 -- 172.21.15.49:0/2317385734 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60000f2e0 con 0x7fd6100ff690 2024-05-15T16:33:27.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.264+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd610197340 con 0x7fd6100ff690 2024-05-15T16:33:27.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.265+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd610109300 con 0x7fd6100ff690 2024-05-15T16:33:27.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.265+0000 7fd606ffd700 1 -- 172.21.15.49:0/2317385734 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd600010380 con 0x7fd6100ff690 2024-05-15T16:33:27.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.266+0000 7fd606ffd700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd5fc06e200 0x7fd5fc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:27.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.266+0000 7fd606ffd700 1 -- 172.21.15.49:0/2317385734 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd60008dd50 con 0x7fd6100ff690 2024-05-15T16:33:27.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.266+0000 7fd615180700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd5fc06e200 0x7fd5fc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:27.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.266+0000 7fd615180700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd5fc06e200 0x7fd5fc0706c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00b5c0 tx=0x7fd60c005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:27.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.270+0000 7fd606ffd700 1 -- 172.21.15.49:0/2317385734 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd60005ab60 con 0x7fd6100ff690 2024-05-15T16:33:27.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.525+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd610197f50 con 0x7fd6100ff690 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.525+0000 7fd606ffd700 1 -- 172.21.15.49:0/2317385734 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fd60005a6f0 con 0x7fd6100ff690 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 19 2024-05-15T16:33:27.526 INFO:teuthology.orchestra.run.smithi049.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:02:56.078027+0000 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:33:27.527 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:up {} 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:failed 0,1 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:33:27.528 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:33:27.529 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:33:27.529 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:33:27.529 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:33:27.529 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:33:27.529 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.527+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd5fc06e200 msgr2=0x7fd5fc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.527+0000 7fd617be5700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd5fc06e200 0x7fd5fc0706c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00b5c0 tx=0x7fd60c005e20 comp rx=0 tx=0).stop 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.528+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6100ff690 msgr2=0x7fd61019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.528+0000 7fd617be5700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6100ff690 0x7fd61019c950 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b4b0 tx=0x7fd60000b870 comp rx=0 tx=0).stop 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.528+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 shutdown_connections 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.528+0000 7fd617be5700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd5fc06e200 0x7fd5fc0706c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.528+0000 7fd617be5700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6100ff690 0x7fd61019c950 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.528+0000 7fd617be5700 1 --2- 172.21.15.49:0/2317385734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd610100040 0x7fd61019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.528+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 >> 172.21.15.49:0/2317385734 conn(0x7fd610074eb0 msgr2=0x7fd610102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.528+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 shutdown_connections 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.528+0000 7fd617be5700 1 -- 172.21.15.49:0/2317385734 wait complete. 2024-05-15T16:33:27.530 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:33:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.640+0000 7f888257b700 1 -- 172.21.15.49:0/1591057222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c100770 msgr2=0x7f887c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.640+0000 7f888257b700 1 --2- 172.21.15.49:0/1591057222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c100770 0x7f887c1047c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8864009910 tx=0x7f8864009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.640+0000 7f888257b700 1 -- 172.21.15.49:0/1591057222 shutdown_connections 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.640+0000 7f888257b700 1 --2- 172.21.15.49:0/1591057222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c100770 0x7f887c1047c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.640+0000 7f888257b700 1 --2- 172.21.15.49:0/1591057222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f887c0ffdc0 0x7f887c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.640+0000 7f888257b700 1 -- 172.21.15.49:0/1591057222 >> 172.21.15.49:0/1591057222 conn(0x7f887c076800 msgr2=0x7f887c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.640+0000 7f888257b700 1 -- 172.21.15.49:0/1591057222 shutdown_connections 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.640+0000 7f888257b700 1 -- 172.21.15.49:0/1591057222 wait complete. 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f888257b700 1 Processor -- start 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f888257b700 1 -- start start 2024-05-15T16:33:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f888257b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c0ffdc0 0x7f887c19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f888257b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f887c100770 0x7f887c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f888257b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c196980 con 0x7f887c0ffdc0 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f888257b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c196af0 con 0x7f887c100770 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f887bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c0ffdc0 0x7f887c19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f887bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c0ffdc0 0x7f887c19c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38694/0 (socket says 172.21.15.49:38694) 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f887bfff700 1 -- 172.21.15.49:0/1225799248 learned_addr learned my addr 172.21.15.49:0/1225799248 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f8873fff700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f887c100770 0x7f887c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f887bfff700 1 -- 172.21.15.49:0/1225799248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f887c100770 msgr2=0x7f887c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f887bfff700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f887c100770 0x7f887c19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.641+0000 7f887bfff700 1 -- 172.21.15.49:0/1225799248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88640095f0 con 0x7f887c0ffdc0 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.642+0000 7f887bfff700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c0ffdc0 0x7f887c19c8d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f886c00d8d0 tx=0x7f886c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.642+0000 7f8879ffb700 1 -- 172.21.15.49:0/1225799248 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f886c009940 con 0x7f887c0ffdc0 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.642+0000 7f8879ffb700 1 -- 172.21.15.49:0/1225799248 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f886c010460 con 0x7f887c0ffdc0 2024-05-15T16:33:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.642+0000 7f8879ffb700 1 -- 172.21.15.49:0/1225799248 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f886c00f5f0 con 0x7f887c0ffdc0 2024-05-15T16:33:27.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.642+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c196dd0 con 0x7f887c0ffdc0 2024-05-15T16:33:27.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.642+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c197320 con 0x7f887c0ffdc0 2024-05-15T16:33:27.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.643+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c1012a0 con 0x7f887c0ffdc0 2024-05-15T16:33:27.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.644+0000 7f8879ffb700 1 -- 172.21.15.49:0/1225799248 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f886c0105d0 con 0x7f887c0ffdc0 2024-05-15T16:33:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.644+0000 7f8879ffb700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f886806e200 0x7f88680706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.644+0000 7f8879ffb700 1 -- 172.21.15.49:0/1225799248 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f886c08e1a0 con 0x7f887c0ffdc0 2024-05-15T16:33:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.644+0000 7f8873fff700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f886806e200 0x7f88680706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:27.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.645+0000 7f8873fff700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f886806e200 0x7f88680706c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f887c1980f0 tx=0x7f8864005280 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:27.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.649+0000 7f8879ffb700 1 -- 172.21.15.49:0/1225799248 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f886c057480 con 0x7f887c0ffdc0 2024-05-15T16:33:27.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.874+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f887c10cd60 con 0x7f886806e200 2024-05-15T16:33:27.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.876+0000 7f8879ffb700 1 -- 172.21.15.49:0/1225799248 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f887c10cd60 con 0x7f886806e200 2024-05-15T16:33:27.876 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:33:27.876 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:33:27.876 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:33:27.876 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:33:27.876 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:33:27.876 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:33:27.876 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:33:27.876 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:33:27.877 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:33:27.877 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:33:27.877 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:33:27.877 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:33:27.877 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading mds daemons", 2024-05-15T16:33:27.877 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:33:27.877 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:33:27.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f886806e200 msgr2=0x7f88680706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f886806e200 0x7f88680706c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f887c1980f0 tx=0x7f8864005280 comp rx=0 tx=0).stop 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c0ffdc0 msgr2=0x7f887c19c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c0ffdc0 0x7f887c19c8d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f886c00d8d0 tx=0x7f886c00dc90 comp rx=0 tx=0).stop 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 shutdown_connections 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f886806e200 0x7f88680706c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f887c0ffdc0 0x7f887c19c8d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 --2- 172.21.15.49:0/1225799248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f887c100770 0x7f887c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 >> 172.21.15.49:0/1225799248 conn(0x7f887c076800 msgr2=0x7f887c0688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 shutdown_connections 2024-05-15T16:33:27.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.878+0000 7f888257b700 1 -- 172.21.15.49:0/1225799248 wait complete. 2024-05-15T16:33:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.987+0000 7ff720982700 1 -- 172.21.15.49:0/1786785512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c100770 msgr2=0x7ff71c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.987+0000 7ff720982700 1 --2- 172.21.15.49:0/1786785512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c100770 0x7ff71c1047c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff704009910 tx=0x7ff704009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.987+0000 7ff720982700 1 -- 172.21.15.49:0/1786785512 shutdown_connections 2024-05-15T16:33:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.987+0000 7ff720982700 1 --2- 172.21.15.49:0/1786785512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c100770 0x7ff71c1047c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.987+0000 7ff720982700 1 --2- 172.21.15.49:0/1786785512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c0ffdc0 0x7ff71c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.987+0000 7ff720982700 1 -- 172.21.15.49:0/1786785512 >> 172.21.15.49:0/1786785512 conn(0x7ff71c076800 msgr2=0x7ff71c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.987+0000 7ff720982700 1 -- 172.21.15.49:0/1786785512 shutdown_connections 2024-05-15T16:33:27.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.987+0000 7ff720982700 1 -- 172.21.15.49:0/1786785512 wait complete. 2024-05-15T16:33:27.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.988+0000 7ff720982700 1 Processor -- start 2024-05-15T16:33:27.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.988+0000 7ff720982700 1 -- start start 2024-05-15T16:33:27.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.988+0000 7ff720982700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c0ffdc0 0x7ff71c0fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:27.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.988+0000 7ff720982700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c100770 0x7ff71c0fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:27.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.988+0000 7ff720982700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0fc760 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.988+0000 7ff720982700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0fc8a0 con 0x7ff71c100770 2024-05-15T16:33:27.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.988+0000 7ff71a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c0ffdc0 0x7ff71c0fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:27.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff71a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c0ffdc0 0x7ff71c0fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38712/0 (socket says 172.21.15.49:38712) 2024-05-15T16:33:27.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff71a59c700 1 -- 172.21.15.49:0/2049870809 learned_addr learned my addr 172.21.15.49:0/2049870809 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:27.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff719d9b700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c100770 0x7ff71c0fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:27.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff71a59c700 1 -- 172.21.15.49:0/2049870809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c100770 msgr2=0x7ff71c0fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:27.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff71a59c700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c100770 0x7ff71c0fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:27.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff71a59c700 1 -- 172.21.15.49:0/2049870809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7040095f0 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff71a59c700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c0ffdc0 0x7ff71c0fbce0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00ba50 tx=0x7ff70c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:27.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff7137fe700 1 -- 172.21.15.49:0/2049870809 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff70c00c900 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71c1a2e20 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff7137fe700 1 -- 172.21.15.49:0/2049870809 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff70c014440 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff7137fe700 1 -- 172.21.15.49:0/2049870809 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff70c0075a0 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.989+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c1a3180 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.990+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71c1012a0 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.991+0000 7ff7137fe700 1 -- 172.21.15.49:0/2049870809 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff70c0145b0 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.991+0000 7ff7137fe700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff70806e200 0x7ff7080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:27.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.992+0000 7ff7137fe700 1 -- 172.21.15.49:0/2049870809 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff70c006e40 con 0x7ff71c0ffdc0 2024-05-15T16:33:27.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.992+0000 7ff719d9b700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff70806e200 0x7ff7080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:27.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.992+0000 7ff719d9b700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff70806e200 0x7ff7080706c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0fd410 tx=0x7ff70401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:27.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:27.996+0000 7ff7137fe700 1 -- 172.21.15.49:0/2049870809 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff70c056c10 con 0x7ff71c0ffdc0 2024-05-15T16:33:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:27 smithi144 ceph-mon[86635]: from='client.35974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:27 smithi144 ceph-mon[86635]: from='client.35978 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:27 smithi144 ceph-mon[86635]: pgmap v1094: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:27 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3862489747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:27 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2317385734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:33:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:27 smithi049 ceph-mon[112142]: from='client.35974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:27 smithi049 ceph-mon[112142]: from='client.35978 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:27 smithi049 ceph-mon[112142]: pgmap v1094: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:27 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3862489747' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:27 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2317385734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:33:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.292+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff71c0fca60 con 0x7ff71c0ffdc0 2024-05-15T16:33:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.293+0000 7ff7137fe700 1 -- 172.21.15.49:0/2049870809 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7ff70c05a2d0 con 0x7ff71c0ffdc0 2024-05-15T16:33:28.293 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-15T16:33:28.293 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:33:28.293 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:33:28.293 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:33:28.293 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:33:28.294 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp5wey4ju7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:33:28.295 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:33:28.295 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:33:28.295 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:33:28.295 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:33:28.295 INFO:teuthology.orchestra.run.smithi049.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-15T16:33:28.295 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-15T16:33:28.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.295+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff70806e200 msgr2=0x7ff7080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:28.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff70806e200 0x7ff7080706c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0fd410 tx=0x7ff70401a040 comp rx=0 tx=0).stop 2024-05-15T16:33:28.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c0ffdc0 msgr2=0x7ff71c0fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:28.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c0ffdc0 0x7ff71c0fbce0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00ba50 tx=0x7ff70c00bd60 comp rx=0 tx=0).stop 2024-05-15T16:33:28.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 shutdown_connections 2024-05-15T16:33:28.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff70806e200 0x7ff7080706c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:28.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff71c0ffdc0 0x7ff71c0fbce0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:28.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 --2- 172.21.15.49:0/2049870809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c100770 0x7ff71c0fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:28.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 >> 172.21.15.49:0/2049870809 conn(0x7ff71c076800 msgr2=0x7ff71c068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:28.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 shutdown_connections 2024-05-15T16:33:28.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:28.296+0000 7ff720982700 1 -- 172.21.15.49:0/2049870809 wait complete. 2024-05-15T16:33:29.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:28 smithi144 ceph-mon[86635]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:29.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:28 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2049870809' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:33:29.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:28 smithi049 ceph-mon[112142]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:29.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:28 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2049870809' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:33:30.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:29 smithi144 ceph-mon[86635]: pgmap v1095: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:29 smithi049 ceph-mon[112142]: pgmap v1095: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:31 smithi144 ceph-mon[86635]: pgmap v1096: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:31 smithi049 ceph-mon[112142]: pgmap v1096: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:33 smithi144 ceph-mon[86635]: pgmap v1097: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:33 smithi049 ceph-mon[112142]: pgmap v1097: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:35 smithi144 ceph-mon[86635]: pgmap v1098: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:33:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:35 smithi049 ceph-mon[112142]: pgmap v1098: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:33:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:37 smithi144 ceph-mon[86635]: pgmap v1099: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:37 smithi049 ceph-mon[112142]: pgmap v1099: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:39 smithi144 ceph-mon[86635]: pgmap v1100: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:39 smithi049 ceph-mon[112142]: pgmap v1100: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:41 smithi144 ceph-mon[86635]: pgmap v1101: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:41 smithi049 ceph-mon[112142]: pgmap v1101: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:44.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:43 smithi144 ceph-mon[86635]: pgmap v1102: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:43 smithi049 ceph-mon[112142]: pgmap v1102: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:45 smithi144 ceph-mon[86635]: pgmap v1103: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:45 smithi049 ceph-mon[112142]: pgmap v1103: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:47 smithi144 ceph-mon[86635]: pgmap v1104: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:47 smithi049 ceph-mon[112142]: pgmap v1104: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:49 smithi144 ceph-mon[86635]: pgmap v1105: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:49 smithi049 ceph-mon[112142]: pgmap v1105: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:33:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:33:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:33:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:51.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:33:51.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:33:51.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:33:51.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: pgmap v1106: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: pgmap v1107: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: pgmap v1108: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:33:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all mds 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi049.abbagx"}]': finished 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi049.gocadf"}]': finished 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.jgodzs"}]': finished 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.kziuke"}]': finished 2024-05-15T16:33:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: Upgrade: Setting filesystem cephfs Joinable 2024-05-15T16:33:52.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-15T16:33:52.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: Standby daemon mds.cephfs.smithi049.abbagx assigned to filesystem cephfs as rank 0 2024-05-15T16:33:52.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: Standby daemon mds.cephfs.smithi049.gocadf assigned to filesystem cephfs as rank 1 2024-05-15T16:33:52.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-15T16:33:52.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-15T16:33:52.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:replay,1=cephfs.smithi049.gocadf=up:replay} 2 up:standby 2024-05-15T16:33:52.200 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-15T16:33:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: pgmap v1106: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: pgmap v1107: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: pgmap v1108: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail 2024-05-15T16:33:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:33:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:52.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all mds 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi049.abbagx"}]: dispatch 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi049.abbagx"}]': finished 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi049.gocadf"}]: dispatch 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi049.gocadf"}]': finished 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.jgodzs"}]: dispatch 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.jgodzs"}]': finished 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.kziuke"}]: dispatch 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.kziuke"}]': finished 2024-05-15T16:33:52.206 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: Upgrade: Setting filesystem cephfs Joinable 2024-05-15T16:33:52.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-15T16:33:52.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: Standby daemon mds.cephfs.smithi049.abbagx assigned to filesystem cephfs as rank 0 2024-05-15T16:33:52.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: Standby daemon mds.cephfs.smithi049.gocadf assigned to filesystem cephfs as rank 1 2024-05-15T16:33:52.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-15T16:33:52.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-15T16:33:52.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:replay,1=cephfs.smithi049.gocadf=up:replay} 2 up:standby 2024-05-15T16:33:52.207 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:resolve 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:replay,1=cephfs.smithi049.gocadf=up:resolve} 2 up:standby 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:52 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-15T16:33:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-15T16:33:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:resolve 2024-05-15T16:33:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:replay,1=cephfs.smithi049.gocadf=up:resolve} 2 up:standby 2024-05-15T16:33:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:53.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:53.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:53.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:52 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:33:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:53 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all rgw 2024-05-15T16:33:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:53 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all rbd-mirror 2024-05-15T16:33:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:53 smithi144 ceph-mon[86635]: pgmap v1109: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 2 op/s 2024-05-15T16:33:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:53 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all rgw 2024-05-15T16:33:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:53 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all rbd-mirror 2024-05-15T16:33:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:53 smithi049 ceph-mon[112142]: pgmap v1109: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 2 op/s 2024-05-15T16:33:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:55 smithi144 ceph-mon[86635]: Upgrade: Updating ceph-exporter.smithi049 (1/2) 2024-05-15T16:33:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T16:33:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:55 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:33:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:55 smithi144 ceph-mon[86635]: Deploying daemon ceph-exporter.smithi049 on smithi049 2024-05-15T16:33:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:55 smithi144 ceph-mon[86635]: pgmap v1110: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 8.9 MiB/s rd, 4 op/s 2024-05-15T16:33:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:55 smithi049 ceph-mon[112142]: Upgrade: Updating ceph-exporter.smithi049 (1/2) 2024-05-15T16:33:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi049", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T16:33:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:55 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:33:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:55 smithi049 ceph-mon[112142]: Deploying daemon ceph-exporter.smithi049 on smithi049 2024-05-15T16:33:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:55 smithi049 ceph-mon[112142]: pgmap v1110: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 8.9 MiB/s rd, 4 op/s 2024-05-15T16:33:58.333 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:57 smithi049 ceph-mon[112142]: pgmap v1111: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 6 op/s 2024-05-15T16:33:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.410+0000 7fb0ec41c700 1 -- 172.21.15.49:0/3440535010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 msgr2=0x7fb0e410d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.410+0000 7fb0ec41c700 1 --2- 172.21.15.49:0/3440535010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 0x7fb0e410d0a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009910 tx=0x7fb0d4009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.410+0000 7fb0ec41c700 1 -- 172.21.15.49:0/3440535010 shutdown_connections 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.410+0000 7fb0ec41c700 1 --2- 172.21.15.49:0/3440535010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 0x7fb0e410d0a0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.410+0000 7fb0ec41c700 1 --2- 172.21.15.49:0/3440535010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0e40ffa30 0x7fb0e40ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.410+0000 7fb0ec41c700 1 -- 172.21.15.49:0/3440535010 >> 172.21.15.49:0/3440535010 conn(0x7fb0e4076800 msgr2=0x7fb0e40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.410+0000 7fb0ec41c700 1 -- 172.21.15.49:0/3440535010 shutdown_connections 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.410+0000 7fb0ec41c700 1 -- 172.21.15.49:0/3440535010 wait complete. 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0ec41c700 1 Processor -- start 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0ec41c700 1 -- start start 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0ec41c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0e40ffa30 0x7fb0e4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0ec41c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 0x7fb0e41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0ec41c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e41999a0 con 0x7fb0e41003e0 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0ec41c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e419d670 con 0x7fb0e40ffa30 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0e99b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 0x7fb0e41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0e99b7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 0x7fb0e41992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55558/0 (socket says 172.21.15.49:55558) 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0e99b7700 1 -- 172.21.15.49:0/2669603984 learned_addr learned my addr 172.21.15.49:0/2669603984 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.411+0000 7fb0ea1b8700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0e40ffa30 0x7fb0e4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.412+0000 7fb0e99b7700 1 -- 172.21.15.49:0/2669603984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0e40ffa30 msgr2=0x7fb0e4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.412+0000 7fb0e99b7700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0e40ffa30 0x7fb0e4198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.412+0000 7fb0e99b7700 1 -- 172.21.15.49:0/2669603984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d40095f0 con 0x7fb0e41003e0 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.412+0000 7fb0e99b7700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 0x7fb0e41992c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4006010 tx=0x7fb0d40055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.412+0000 7fb0db7fe700 1 -- 172.21.15.49:0/2669603984 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d401d070 con 0x7fb0e41003e0 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.412+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e419d8f0 con 0x7fb0e41003e0 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.412+0000 7fb0db7fe700 1 -- 172.21.15.49:0/2669603984 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0d400bea0 con 0x7fb0e41003e0 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.412+0000 7fb0db7fe700 1 -- 172.21.15.49:0/2669603984 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d400fe90 con 0x7fb0e41003e0 2024-05-15T16:33:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.412+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e419de40 con 0x7fb0e41003e0 2024-05-15T16:33:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.414+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e4068ee0 con 0x7fb0e41003e0 2024-05-15T16:33:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.414+0000 7fb0db7fe700 1 -- 172.21.15.49:0/2669603984 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb0d400f6f0 con 0x7fb0e41003e0 2024-05-15T16:33:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.414+0000 7fb0db7fe700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb0d006e210 0x7fb0d00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:58.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.414+0000 7fb0db7fe700 1 -- 172.21.15.49:0/2669603984 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb0d408f180 con 0x7fb0e41003e0 2024-05-15T16:33:58.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.415+0000 7fb0ea1b8700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb0d006e210 0x7fb0d00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:58.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.415+0000 7fb0ea1b8700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb0d006e210 0x7fb0d00706d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005950 tx=0x7fb0e000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:58.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.419+0000 7fb0db7fe700 1 -- 172.21.15.49:0/2669603984 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb0d40585a0 con 0x7fb0e41003e0 2024-05-15T16:33:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:57 smithi144 ceph-mon[86635]: pgmap v1111: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 6 op/s 2024-05-15T16:33:58.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.712+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0e4110030 con 0x7fb0d006e210 2024-05-15T16:33:58.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.714+0000 7fb0db7fe700 1 -- 172.21.15.49:0/2669603984 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fb0e4110030 con 0x7fb0d006e210 2024-05-15T16:33:58.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.716+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb0d006e210 msgr2=0x7fb0d00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:58.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.716+0000 7fb0ec41c700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb0d006e210 0x7fb0d00706d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005950 tx=0x7fb0e000b410 comp rx=0 tx=0).stop 2024-05-15T16:33:58.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.716+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 msgr2=0x7fb0e41992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:58.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.716+0000 7fb0ec41c700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 0x7fb0e41992c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4006010 tx=0x7fb0d40055f0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.717+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 shutdown_connections 2024-05-15T16:33:58.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.717+0000 7fb0ec41c700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fb0d006e210 0x7fb0d00706d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.717+0000 7fb0ec41c700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0e40ffa30 0x7fb0e4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.717+0000 7fb0ec41c700 1 --2- 172.21.15.49:0/2669603984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0e41003e0 0x7fb0e41992c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.717+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 >> 172.21.15.49:0/2669603984 conn(0x7fb0e4076800 msgr2=0x7fb0e40fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:58.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.717+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 shutdown_connections 2024-05-15T16:33:58.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.717+0000 7fb0ec41c700 1 -- 172.21.15.49:0/2669603984 wait complete. 2024-05-15T16:33:58.730 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:33:58.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.838+0000 7f6c2c294700 1 -- 172.21.15.49:0/3141803320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 msgr2=0x7f6c24102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:58.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.838+0000 7f6c2c294700 1 --2- 172.21.15.49:0/3141803320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 0x7f6c24102200 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009910 tx=0x7f6c20009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:58.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.839+0000 7f6c2c294700 1 -- 172.21.15.49:0/3141803320 shutdown_connections 2024-05-15T16:33:58.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.839+0000 7f6c2c294700 1 --2- 172.21.15.49:0/3141803320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c24102740 0x7f6c2410ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.839+0000 7f6c2c294700 1 --2- 172.21.15.49:0/3141803320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 0x7f6c24102200 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.839+0000 7f6c2c294700 1 -- 172.21.15.49:0/3141803320 >> 172.21.15.49:0/3141803320 conn(0x7f6c24076800 msgr2=0x7f6c240fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:58.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.839+0000 7f6c2c294700 1 -- 172.21.15.49:0/3141803320 shutdown_connections 2024-05-15T16:33:58.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.839+0000 7f6c2c294700 1 -- 172.21.15.49:0/3141803320 wait complete. 2024-05-15T16:33:58.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.839+0000 7f6c2c294700 1 Processor -- start 2024-05-15T16:33:58.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.839+0000 7f6c2c294700 1 -- start start 2024-05-15T16:33:58.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2c294700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 0x7f6c2419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:58.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2c294700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c24102740 0x7f6c2419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:58.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2c294700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419d420 con 0x7f6c24101e20 2024-05-15T16:33:58.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2c294700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c24196a00 con 0x7f6c24102740 2024-05-15T16:33:58.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2a030700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 0x7f6c2419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:58.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2a030700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 0x7f6c2419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55586/0 (socket says 172.21.15.49:55586) 2024-05-15T16:33:58.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2a030700 1 -- 172.21.15.49:0/1701537039 learned_addr learned my addr 172.21.15.49:0/1701537039 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2982f700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c24102740 0x7f6c2419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2a030700 1 -- 172.21.15.49:0/1701537039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c24102740 msgr2=0x7f6c2419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2a030700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c24102740 0x7f6c2419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2a030700 1 -- 172.21.15.49:0/1701537039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200095f0 con 0x7f6c24101e20 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2982f700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c24102740 0x7f6c2419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.840+0000 7f6c2a030700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 0x7f6c2419c8c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005f10 tx=0x7f6c2000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.841+0000 7f6c1b7fe700 1 -- 172.21.15.49:0/1701537039 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2001d070 con 0x7f6c24101e20 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.841+0000 7f6c1b7fe700 1 -- 172.21.15.49:0/1701537039 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c200052a0 con 0x7f6c24101e20 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.841+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c24196c80 con 0x7f6c24101e20 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.841+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c24197170 con 0x7f6c24101e20 2024-05-15T16:33:58.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.841+0000 7f6c1b7fe700 1 -- 172.21.15.49:0/1701537039 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c20022470 con 0x7f6c24101e20 2024-05-15T16:33:58.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.842+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c24105540 con 0x7f6c24101e20 2024-05-15T16:33:58.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.843+0000 7f6c1b7fe700 1 -- 172.21.15.49:0/1701537039 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6c20003780 con 0x7f6c24101e20 2024-05-15T16:33:58.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.843+0000 7f6c1b7fe700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c1006e200 0x7f6c100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:58.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.844+0000 7f6c1b7fe700 1 -- 172.21.15.49:0/1701537039 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6c2008f5c0 con 0x7f6c24101e20 2024-05-15T16:33:58.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.844+0000 7f6c2982f700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c1006e200 0x7f6c100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:58.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.844+0000 7f6c2982f700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c1006e200 0x7f6c100706c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6c24197bc0 tx=0x7f6c14009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:58.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:58.853+0000 7f6c1b7fe700 1 -- 172.21.15.49:0/1701537039 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6c2005c3d0 con 0x7f6c24101e20 2024-05-15T16:33:59.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.085+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c24197ec0 con 0x7f6c1006e200 2024-05-15T16:33:59.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.087+0000 7f6c1b7fe700 1 -- 172.21.15.49:0/1701537039 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f6c24197ec0 con 0x7f6c1006e200 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c1006e200 msgr2=0x7f6c100706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c1006e200 0x7f6c100706c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6c24197bc0 tx=0x7f6c14009260 comp rx=0 tx=0).stop 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 msgr2=0x7f6c2419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 0x7f6c2419c8c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005f10 tx=0x7f6c2000b650 comp rx=0 tx=0).stop 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 shutdown_connections 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6c1006e200 0x7f6c100706c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c24101e20 0x7f6c2419c8c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 --2- 172.21.15.49:0/1701537039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c24102740 0x7f6c2419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 >> 172.21.15.49:0/1701537039 conn(0x7f6c24076800 msgr2=0x7f6c24103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 shutdown_connections 2024-05-15T16:33:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.089+0000 7f6c2c294700 1 -- 172.21.15.49:0/1701537039 wait complete. 2024-05-15T16:33:59.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.201+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/1390400937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e480ffa10 msgr2=0x7f8e480ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.201+0000 7f8e4e7b6700 1 --2- 172.21.15.49:0/1390400937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e480ffa10 0x7f8e480ffdf0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009910 tx=0x7f8e38009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.201+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/1390400937 shutdown_connections 2024-05-15T16:33:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.201+0000 7f8e4e7b6700 1 --2- 172.21.15.49:0/1390400937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e481003c0 0x7f8e4810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.201+0000 7f8e4e7b6700 1 --2- 172.21.15.49:0/1390400937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e480ffa10 0x7f8e480ffdf0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.201+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/1390400937 >> 172.21.15.49:0/1390400937 conn(0x7f8e48076800 msgr2=0x7f8e480fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.201+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/1390400937 shutdown_connections 2024-05-15T16:33:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.201+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/1390400937 wait complete. 2024-05-15T16:33:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.202+0000 7f8e4e7b6700 1 Processor -- start 2024-05-15T16:33:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.202+0000 7f8e4e7b6700 1 -- start start 2024-05-15T16:33:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.202+0000 7f8e4e7b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e480ffa10 0x7f8e4819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.202+0000 7f8e4e7b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e481003c0 0x7f8e4819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.202+0000 7f8e4e7b6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4819f5c0 con 0x7f8e481003c0 2024-05-15T16:33:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.202+0000 7f8e4e7b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48198ba0 con 0x7f8e480ffa10 2024-05-15T16:33:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e477fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e481003c0 0x7f8e4819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e477fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e481003c0 0x7f8e4819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55616/0 (socket says 172.21.15.49:55616) 2024-05-15T16:33:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e47fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e480ffa10 0x7f8e4819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e477fe700 1 -- 172.21.15.49:0/4089191371 learned_addr learned my addr 172.21.15.49:0/4089191371 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e47fff700 1 -- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e481003c0 msgr2=0x7f8e4819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e47fff700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e481003c0 0x7f8e4819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e47fff700 1 -- 172.21.15.49:0/4089191371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380095f0 con 0x7f8e480ffa10 2024-05-15T16:33:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e477fe700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e481003c0 0x7f8e4819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T16:33:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e47fff700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e480ffa10 0x7f8e4819ea60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009fd0 tx=0x7f8e38005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:59.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e457fa700 1 -- 172.21.15.49:0/4089191371 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3801d070 con 0x7f8e480ffa10 2024-05-15T16:33:59.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.203+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e48198e80 con 0x7f8e480ffa10 2024-05-15T16:33:59.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.204+0000 7f8e457fa700 1 -- 172.21.15.49:0/4089191371 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e38003b40 con 0x7f8e480ffa10 2024-05-15T16:33:59.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.204+0000 7f8e457fa700 1 -- 172.21.15.49:0/4089191371 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3800fd10 con 0x7f8e480ffa10 2024-05-15T16:33:59.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.204+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e481993d0 con 0x7f8e480ffa10 2024-05-15T16:33:59.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.205+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e48107990 con 0x7f8e480ffa10 2024-05-15T16:33:59.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.205+0000 7f8e457fa700 1 -- 172.21.15.49:0/4089191371 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8e3802b440 con 0x7f8e480ffa10 2024-05-15T16:33:59.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.206+0000 7f8e457fa700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e3006e200 0x7f8e300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:59.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.206+0000 7f8e457fa700 1 -- 172.21.15.49:0/4089191371 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8e3808f240 con 0x7f8e480ffa10 2024-05-15T16:33:59.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.206+0000 7f8e477fe700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e3006e200 0x7f8e300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:59.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.206+0000 7f8e477fe700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e3006e200 0x7f8e300706c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8e4819a240 tx=0x7f8e3c00b480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:59.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.211+0000 7f8e457fa700 1 -- 172.21.15.49:0/4089191371 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8e3805c020 con 0x7f8e480ffa10 2024-05-15T16:33:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.429+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e48199ff0 con 0x7f8e3006e200 2024-05-15T16:33:59.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.441+0000 7f8e457fa700 1 -- 172.21.15.49:0/4089191371 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8e48199ff0 con 0x7f8e3006e200 2024-05-15T16:33:59.441 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (37m) 15m ago 48m 24.0M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 starting - - - - 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (47m) 15m ago 47m 10.1M - 18.2.2-1343-g2965081f a04a73ffe079 a09271dc127b 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (34m) 15m ago 48m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (34m) 15m ago 47m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (36m) 15m ago 48m 79.2M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (30m) 15m ago 43m 21.5M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (30m) 15m ago 43m 19.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (30m) 15m ago 43m 21.4M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (15m) 15m ago 43m 12.2M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (39m) 15m ago 50m 515M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (38m) 15m ago 47m 460M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (35m) 15m ago 50m 71.7M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:33:59.442 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (35m) 15m ago 47m 53.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:33:59.443 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (38m) 15m ago 48m 15.7M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:33:59.443 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (37m) 15m ago 47m 15.8M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:33:59.443 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (34m) 15m ago 46m 90.7M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:33:59.443 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (33m) 15m ago 46m 91.5M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:33:59.443 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (32m) 15m ago 45m 88.2M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:33:59.443 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (32m) 15m ago 45m 103M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:33:59.443 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (31m) 15m ago 45m 89.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:33:59.443 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (31m) 15m ago 44m 84.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:33:59.443 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (37m) 15m ago 48m 85.9M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:33:59.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e3006e200 msgr2=0x7f8e300706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e3006e200 0x7f8e300706c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8e4819a240 tx=0x7f8e3c00b480 comp rx=0 tx=0).stop 2024-05-15T16:33:59.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e480ffa10 msgr2=0x7f8e4819ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e480ffa10 0x7f8e4819ea60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009fd0 tx=0x7f8e38005280 comp rx=0 tx=0).stop 2024-05-15T16:33:59.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 shutdown_connections 2024-05-15T16:33:59.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8e3006e200 0x7f8e300706c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e480ffa10 0x7f8e4819ea60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 --2- 172.21.15.49:0/4089191371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e481003c0 0x7f8e4819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 >> 172.21.15.49:0/4089191371 conn(0x7f8e48076800 msgr2=0x7f8e480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:59.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 shutdown_connections 2024-05-15T16:33:59.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.443+0000 7f8e4e7b6700 1 -- 172.21.15.49:0/4089191371 wait complete. 2024-05-15T16:33:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.553+0000 7fc5b29d0700 1 -- 172.21.15.49:0/274454660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5ac101880 msgr2=0x7fc5ac1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.553+0000 7fc5b29d0700 1 --2- 172.21.15.49:0/274454660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5ac101880 0x7fc5ac1058d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009910 tx=0x7fc5a0009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.553+0000 7fc5b29d0700 1 -- 172.21.15.49:0/274454660 shutdown_connections 2024-05-15T16:33:59.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.553+0000 7fc5b29d0700 1 --2- 172.21.15.49:0/274454660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5ac101880 0x7fc5ac1058d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.553+0000 7fc5b29d0700 1 --2- 172.21.15.49:0/274454660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac100ed0 0x7fc5ac1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.553+0000 7fc5b29d0700 1 -- 172.21.15.49:0/274454660 >> 172.21.15.49:0/274454660 conn(0x7fc5ac0fc7a0 msgr2=0x7fc5ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.553+0000 7fc5b29d0700 1 -- 172.21.15.49:0/274454660 shutdown_connections 2024-05-15T16:33:59.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.553+0000 7fc5b29d0700 1 -- 172.21.15.49:0/274454660 wait complete. 2024-05-15T16:33:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5b29d0700 1 Processor -- start 2024-05-15T16:33:59.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5b29d0700 1 -- start start 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5b29d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac100ed0 0x7fc5ac19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5b29d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5ac101880 0x7fc5ac19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5b29d0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac19d440 con 0x7fc5ac101880 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5b29d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac196a70 con 0x7fc5ac100ed0 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5ab7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5ac101880 0x7fc5ac19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac100ed0 0x7fc5ac19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5ab7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5ac101880 0x7fc5ac19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55646/0 (socket says 172.21.15.49:55646) 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.554+0000 7fc5ab7fe700 1 -- 172.21.15.49:0/2236868313 learned_addr learned my addr 172.21.15.49:0/2236868313 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.555+0000 7fc5abfff700 1 -- 172.21.15.49:0/2236868313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5ac101880 msgr2=0x7fc5ac19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.555+0000 7fc5abfff700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5ac101880 0x7fc5ac19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.555+0000 7fc5abfff700 1 -- 172.21.15.49:0/2236868313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a00095f0 con 0x7fc5ac100ed0 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.555+0000 7fc5abfff700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac100ed0 0x7fc5ac19c8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e970 tx=0x7fc59c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.555+0000 7fc5a97fa700 1 -- 172.21.15.49:0/2236868313 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc59c00c9d0 con 0x7fc5ac100ed0 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.555+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac196d50 con 0x7fc5ac100ed0 2024-05-15T16:33:59.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.555+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac1972a0 con 0x7fc5ac100ed0 2024-05-15T16:33:59.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.555+0000 7fc5a97fa700 1 -- 172.21.15.49:0/2236868313 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc59c010430 con 0x7fc5ac100ed0 2024-05-15T16:33:59.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.555+0000 7fc5a97fa700 1 -- 172.21.15.49:0/2236868313 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc59c018630 con 0x7fc5ac100ed0 2024-05-15T16:33:59.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.556+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac1092d0 con 0x7fc5ac100ed0 2024-05-15T16:33:59.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.557+0000 7fc5a97fa700 1 -- 172.21.15.49:0/2236868313 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc59c00cb30 con 0x7fc5ac100ed0 2024-05-15T16:33:59.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.558+0000 7fc5a97fa700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc59406e200 0x7fc5940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:59.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.558+0000 7fc5ab7fe700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc59406e200 0x7fc5940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:59.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.558+0000 7fc5a97fa700 1 -- 172.21.15.49:0/2236868313 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc59c014070 con 0x7fc5ac100ed0 2024-05-15T16:33:59.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.558+0000 7fc5ab7fe700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc59406e200 0x7fc5940706c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac197be0 tx=0x7fc5a0005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:59.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.562+0000 7fc5a97fa700 1 -- 172.21.15.49:0/2236868313 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc59c053850 con 0x7fc5ac100ed0 2024-05-15T16:33:59.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.865+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc5ac197ee0 con 0x7fc5ac100ed0 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.866+0000 7fc5a97fa700 1 -- 172.21.15.49:0/2236868313 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc59c05b1a0 con 0x7fc5ac100ed0 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:33:59.866 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:33:59.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:33:59.867 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:33:59.867 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:33:59.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:33:59.867 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:33:59.867 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:33:59.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:33:59.867 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.868+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc59406e200 msgr2=0x7fc5940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.868+0000 7fc5b29d0700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc59406e200 0x7fc5940706c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac197be0 tx=0x7fc5a0005940 comp rx=0 tx=0).stop 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.869+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac100ed0 msgr2=0x7fc5ac19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.869+0000 7fc5b29d0700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac100ed0 0x7fc5ac19c8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e970 tx=0x7fc59c00ec80 comp rx=0 tx=0).stop 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.869+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 shutdown_connections 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.869+0000 7fc5b29d0700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc59406e200 0x7fc5940706c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.869+0000 7fc5b29d0700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac100ed0 0x7fc5ac19c8e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.869+0000 7fc5b29d0700 1 --2- 172.21.15.49:0/2236868313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5ac101880 0x7fc5ac19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.869+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 >> 172.21.15.49:0/2236868313 conn(0x7fc5ac0fc7a0 msgr2=0x7fc5ac0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.869+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 shutdown_connections 2024-05-15T16:33:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.869+0000 7fc5b29d0700 1 -- 172.21.15.49:0/2236868313 wait complete. 2024-05-15T16:33:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:59 smithi144 ceph-mon[86635]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:59 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:59 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:33:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:59 smithi144 ceph-mon[86635]: pgmap v1112: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8 op/s 2024-05-15T16:33:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:33:59 smithi144 ceph-mon[86635]: from='client.36004 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:33:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.980+0000 7f5c5f039700 1 -- 172.21.15.49:0/1205927238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58102740 msgr2=0x7f5c5810ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.980+0000 7f5c5f039700 1 --2- 172.21.15.49:0/1205927238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58102740 0x7f5c5810ac30 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009910 tx=0x7f5c44009c20 comp rx=0 tx=0).stop 2024-05-15T16:33:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.980+0000 7f5c5f039700 1 -- 172.21.15.49:0/1205927238 shutdown_connections 2024-05-15T16:33:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.980+0000 7f5c5f039700 1 --2- 172.21.15.49:0/1205927238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58102740 0x7f5c5810ac30 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.980+0000 7f5c5f039700 1 --2- 172.21.15.49:0/1205927238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c58101e20 0x7f5c58102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.980+0000 7f5c5f039700 1 -- 172.21.15.49:0/1205927238 >> 172.21.15.49:0/1205927238 conn(0x7f5c58076800 msgr2=0x7f5c580fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:33:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.980+0000 7f5c5f039700 1 -- 172.21.15.49:0/1205927238 shutdown_connections 2024-05-15T16:33:59.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.980+0000 7f5c5f039700 1 -- 172.21.15.49:0/1205927238 wait complete. 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.981+0000 7f5c5f039700 1 Processor -- start 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.981+0000 7f5c5f039700 1 -- start start 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5f039700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58101e20 0x7f5c580ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5f039700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c58102740 0x7f5c580fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5f039700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58100500 con 0x7f5c58101e20 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5f039700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58100670 con 0x7f5c58102740 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5cdd5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58101e20 0x7f5c580ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5cdd5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58101e20 0x7f5c580ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55654/0 (socket says 172.21.15.49:55654) 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5cdd5700 1 -- 172.21.15.49:0/1458408949 learned_addr learned my addr 172.21.15.49:0/1458408949 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:33:59.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c57fff700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c58102740 0x7f5c580fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:59.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5cdd5700 1 -- 172.21.15.49:0/1458408949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c58102740 msgr2=0x7f5c580fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:33:59.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5cdd5700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c58102740 0x7f5c580fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:33:59.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5cdd5700 1 -- 172.21.15.49:0/1458408949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440095f0 con 0x7f5c58101e20 2024-05-15T16:33:59.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.982+0000 7f5c5cdd5700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58101e20 0x7f5c580ffa80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00b4b0 tx=0x7f5c4c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:59.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.983+0000 7f5c55ffb700 1 -- 172.21.15.49:0/1458408949 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c4c011800 con 0x7f5c58101e20 2024-05-15T16:33:59.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.983+0000 7f5c55ffb700 1 -- 172.21.15.49:0/1458408949 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c4c011e20 con 0x7f5c58101e20 2024-05-15T16:33:59.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.983+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c581a2ee0 con 0x7f5c58101e20 2024-05-15T16:33:59.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.984+0000 7f5c55ffb700 1 -- 172.21.15.49:0/1458408949 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c4c00f2e0 con 0x7f5c58101e20 2024-05-15T16:33:59.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.984+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c581a3240 con 0x7f5c58101e20 2024-05-15T16:33:59.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.984+0000 7f5c55ffb700 1 -- 172.21.15.49:0/1458408949 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5c4c011960 con 0x7f5c58101e20 2024-05-15T16:33:59.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.985+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c58068f60 con 0x7f5c58101e20 2024-05-15T16:33:59.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.985+0000 7f5c55ffb700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e200 0x7f5c480706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:33:59.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.985+0000 7f5c55ffb700 1 -- 172.21.15.49:0/1458408949 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5c4c08ceb0 con 0x7f5c58101e20 2024-05-15T16:33:59.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.985+0000 7f5c57fff700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e200 0x7f5c480706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:33:59.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.986+0000 7f5c57fff700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e200 0x7f5c480706c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5c58101530 tx=0x7f5c4401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:33:59.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:33:59.990+0000 7f5c55ffb700 1 -- 172.21.15.49:0/1458408949 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5c4c092060 con 0x7f5c58101e20 2024-05-15T16:34:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:59 smithi049 ceph-mon[112142]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:59 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:59 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:59 smithi049 ceph-mon[112142]: pgmap v1112: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8 op/s 2024-05-15T16:34:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:33:59 smithi049 ceph-mon[112142]: from='client.36004 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:00.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.266+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5c5804ec60 con 0x7f5c58101e20 2024-05-15T16:34:00.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.266+0000 7f5c55ffb700 1 -- 172.21.15.49:0/1458408949 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1957 (secure 0 0 0) 0x7f5c4c0157c0 con 0x7f5c58101e20 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:e30 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 30 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:33:51.834694+0000 2024-05-15T16:34:00.267 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=34420,1=34424} 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:34:00.268 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 0 members: 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:34420} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{1:34424} state up:resolve seq 458 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:34:00.269 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{-1:44891} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:34:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e200 msgr2=0x7f5c480706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e200 0x7f5c480706c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5c58101530 tx=0x7f5c4401a040 comp rx=0 tx=0).stop 2024-05-15T16:34:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58101e20 msgr2=0x7f5c580ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58101e20 0x7f5c580ffa80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00b4b0 tx=0x7f5c4c00b7c0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 shutdown_connections 2024-05-15T16:34:00.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e200 0x7f5c480706c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c58101e20 0x7f5c580ffa80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 --2- 172.21.15.49:0/1458408949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c58102740 0x7f5c580fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 >> 172.21.15.49:0/1458408949 conn(0x7f5c58076800 msgr2=0x7f5c58103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:00.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 shutdown_connections 2024-05-15T16:34:00.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.269+0000 7f5c5f039700 1 -- 172.21.15.49:0/1458408949 wait complete. 2024-05-15T16:34:00.271 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 30 2024-05-15T16:34:00.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.385+0000 7f8486c5b700 1 -- 172.21.15.49:0/3928314437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 msgr2=0x7f84801012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:00.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.385+0000 7f8486c5b700 1 --2- 172.21.15.49:0/3928314437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 0x7f84801012b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8474009910 tx=0x7f8474009c20 comp rx=0 tx=0).stop 2024-05-15T16:34:00.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.385+0000 7f8486c5b700 1 -- 172.21.15.49:0/3928314437 shutdown_connections 2024-05-15T16:34:00.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.385+0000 7f8486c5b700 1 --2- 172.21.15.49:0/3928314437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8480101880 0x7f84801058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.385+0000 7f8486c5b700 1 --2- 172.21.15.49:0/3928314437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 0x7f84801012b0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.385+0000 7f8486c5b700 1 -- 172.21.15.49:0/3928314437 >> 172.21.15.49:0/3928314437 conn(0x7f84800fc7a0 msgr2=0x7f84800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.385+0000 7f8486c5b700 1 -- 172.21.15.49:0/3928314437 shutdown_connections 2024-05-15T16:34:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.385+0000 7f8486c5b700 1 -- 172.21.15.49:0/3928314437 wait complete. 2024-05-15T16:34:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.386+0000 7f8486c5b700 1 Processor -- start 2024-05-15T16:34:00.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.386+0000 7f8486c5b700 1 -- start start 2024-05-15T16:34:00.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.386+0000 7f8486c5b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 0x7f8480196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:00.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.386+0000 7f8486c5b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8480101880 0x7f8480197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:00.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f8486c5b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8480197850 con 0x7f8480100ed0 2024-05-15T16:34:00.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f8486c5b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848019b460 con 0x7f8480101880 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f84849f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 0x7f8480196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f84849f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 0x7f8480196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55670/0 (socket says 172.21.15.49:55670) 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f84849f7700 1 -- 172.21.15.49:0/1735044349 learned_addr learned my addr 172.21.15.49:0/1735044349 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f847ffff700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8480101880 0x7f8480197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f84849f7700 1 -- 172.21.15.49:0/1735044349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8480101880 msgr2=0x7f8480197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f84849f7700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8480101880 0x7f8480197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f84849f7700 1 -- 172.21.15.49:0/1735044349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84740095f0 con 0x7f8480100ed0 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f84849f7700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 0x7f8480196c30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8474000c00 tx=0x7f847400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f847dffb700 1 -- 172.21.15.49:0/1735044349 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f847401d070 con 0x7f8480100ed0 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f847dffb700 1 -- 172.21.15.49:0/1735044349 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84740056e0 con 0x7f8480100ed0 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848019b6e0 con 0x7f8480100ed0 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.387+0000 7f847dffb700 1 -- 172.21.15.49:0/1735044349 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f847400fe40 con 0x7f8480100ed0 2024-05-15T16:34:00.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.388+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848019bbd0 con 0x7f8480100ed0 2024-05-15T16:34:00.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.389+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8480109270 con 0x7f8480100ed0 2024-05-15T16:34:00.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.389+0000 7f847dffb700 1 -- 172.21.15.49:0/1735044349 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f847400f680 con 0x7f8480100ed0 2024-05-15T16:34:00.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.390+0000 7f847dffb700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f847006e210 0x7f84700706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:00.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.390+0000 7f847dffb700 1 -- 172.21.15.49:0/1735044349 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f847408f480 con 0x7f8480100ed0 2024-05-15T16:34:00.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.390+0000 7f847ffff700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f847006e210 0x7f84700706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:00.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.391+0000 7f847ffff700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f847006e210 0x7f84700706d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f84801029c0 tx=0x7f846c0073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:00.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.394+0000 7f847dffb700 1 -- 172.21.15.49:0/1735044349 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f84740587e0 con 0x7f8480100ed0 2024-05-15T16:34:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.661+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8480197e80 con 0x7f847006e210 2024-05-15T16:34:00.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.662+0000 7f847dffb700 1 -- 172.21.15.49:0/1735044349 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f8480197e80 con 0x7f847006e210 2024-05-15T16:34:00.662 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a", 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": true, 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [ 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "mds", 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "osd", 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr", 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "mon" 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": "16/23 daemons upgraded", 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:34:00.663 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:34:00.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f847006e210 msgr2=0x7f84700706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:00.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f847006e210 0x7f84700706d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f84801029c0 tx=0x7f846c0073d0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 msgr2=0x7f8480196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:00.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 0x7f8480196c30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8474000c00 tx=0x7f847400bd70 comp rx=0 tx=0).stop 2024-05-15T16:34:00.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 shutdown_connections 2024-05-15T16:34:00.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f847006e210 0x7f84700706d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8480100ed0 0x7f8480196c30 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 --2- 172.21.15.49:0/1735044349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8480101880 0x7f8480197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 >> 172.21.15.49:0/1735044349 conn(0x7f84800fc7a0 msgr2=0x7f84800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:00.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 shutdown_connections 2024-05-15T16:34:00.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.665+0000 7f8486c5b700 1 -- 172.21.15.49:0/1735044349 wait complete. 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.778+0000 7f6744608700 1 -- 172.21.15.49:0/2835721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c1030e0 msgr2=0x7f673c1034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.778+0000 7f6744608700 1 --2- 172.21.15.49:0/2835721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c1030e0 0x7f673c1034c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6738009910 tx=0x7f6738009c20 comp rx=0 tx=0).stop 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.778+0000 7f6744608700 1 -- 172.21.15.49:0/2835721296 shutdown_connections 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.778+0000 7f6744608700 1 --2- 172.21.15.49:0/2835721296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f673c103a90 0x7f673c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.778+0000 7f6744608700 1 --2- 172.21.15.49:0/2835721296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c1030e0 0x7f673c1034c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.778+0000 7f6744608700 1 -- 172.21.15.49:0/2835721296 >> 172.21.15.49:0/2835721296 conn(0x7f673c0fe990 msgr2=0x7f673c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.778+0000 7f6744608700 1 -- 172.21.15.49:0/2835721296 shutdown_connections 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.778+0000 7f6744608700 1 -- 172.21.15.49:0/2835721296 wait complete. 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.779+0000 7f6744608700 1 Processor -- start 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.779+0000 7f6744608700 1 -- start start 2024-05-15T16:34:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f6744608700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f673c1030e0 0x7f673c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f6744608700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c103a90 0x7f673c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f6744608700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c199a40 con 0x7f673c103a90 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f6744608700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c19d6c0 con 0x7f673c1030e0 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f6741ba3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c103a90 0x7f673c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f6741ba3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c103a90 0x7f673c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55686/0 (socket says 172.21.15.49:55686) 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f6741ba3700 1 -- 172.21.15.49:0/3323379856 learned_addr learned my addr 172.21.15.49:0/3323379856 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f67423a4700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f673c1030e0 0x7f673c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f67423a4700 1 -- 172.21.15.49:0/3323379856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c103a90 msgr2=0x7f673c199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f67423a4700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c103a90 0x7f673c199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f67423a4700 1 -- 172.21.15.49:0/3323379856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67380095f0 con 0x7f673c1030e0 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.780+0000 7f67423a4700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f673c1030e0 0x7f673c198e20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6738009fd0 tx=0x7f673800b5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.781+0000 7f67337fe700 1 -- 172.21.15.49:0/3323379856 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f673801d070 con 0x7f673c1030e0 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.781+0000 7f67337fe700 1 -- 172.21.15.49:0/3323379856 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67380052a0 con 0x7f673c1030e0 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.781+0000 7f67337fe700 1 -- 172.21.15.49:0/3323379856 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6738022470 con 0x7f673c1030e0 2024-05-15T16:34:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.781+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c19d940 con 0x7f673c1030e0 2024-05-15T16:34:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.781+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c19dd70 con 0x7f673c1030e0 2024-05-15T16:34:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.782+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c10b480 con 0x7f673c1030e0 2024-05-15T16:34:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.783+0000 7f67337fe700 1 -- 172.21.15.49:0/3323379856 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f673800f6f0 con 0x7f673c1030e0 2024-05-15T16:34:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.784+0000 7f67337fe700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f672806e210 0x7f67280706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.784+0000 7f67337fe700 1 -- 172.21.15.49:0/3323379856 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f673808f440 con 0x7f673c1030e0 2024-05-15T16:34:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.784+0000 7f6741ba3700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f672806e210 0x7f67280706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.785+0000 7f6741ba3700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f672806e210 0x7f67280706d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f673c19a330 tx=0x7f672c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:00.788+0000 7f67337fe700 1 -- 172.21.15.49:0/3323379856 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6738058700 con 0x7f673c1030e0 2024-05-15T16:34:00.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:00 smithi049 ceph-mon[112142]: from='client.45395 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:00.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:00 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2236868313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:00.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:00 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1458408949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:34:00.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:00.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T16:34:00.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:00 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:34:01.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:00 smithi144 ceph-mon[86635]: from='client.45395 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:01.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:00 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2236868313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:01.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:00 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1458408949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:34:01.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:01.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-15T16:34:01.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:00 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:34:01.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.158+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f673c1045c0 con 0x7f673c1030e0 2024-05-15T16:34:01.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.159+0000 7f67337fe700 1 -- 172.21.15.49:0/3323379856 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4367 (secure 0 0 0) 0x7f673805bdc0 con 0x7f673c1030e0 2024-05-15T16:34:01.159 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-05-15T16:34:01.159 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-05-15T16:34:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: host smithi144 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi144/config 2024-05-15T16:34:01.159 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/podman: timeout after 895 seconds 2024-05-15T16:34:01.159 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbzc2g19v:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:34:01.159 INFO:teuthology.orchestra.run.smithi049.stdout:Traceback (most recent call last): 2024-05-15T16:34:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: return _run_code(code, main_globals, None, 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: exec(code, run_globals) 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5618, in 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 5606, in main 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 408, in _infer_config 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 323, in _infer_fsid 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 436, in _infer_image 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 310, in _validate_fsid 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/__main__.py", line 3359, in command_ceph_volume 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: File "/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/cephadm.0274d6ff5b9f2aae0b3541aacf95498bc9ef10781b3c093549ab0705b6baf2bc/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a -e NODE_NAME=smithi144 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbzc2g19v:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a inventory --format=json-pretty --filter-for-batch 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs is degraded 2024-05-15T16:34:01.160 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:34:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:34:01.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.162+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f672806e210 msgr2=0x7f67280706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.162+0000 7f6744608700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f672806e210 0x7f67280706d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f673c19a330 tx=0x7f672c009260 comp rx=0 tx=0).stop 2024-05-15T16:34:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.162+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f673c1030e0 msgr2=0x7f673c198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:01.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.162+0000 7f6744608700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f673c1030e0 0x7f673c198e20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6738009fd0 tx=0x7f673800b5a0 comp rx=0 tx=0).stop 2024-05-15T16:34:01.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.162+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 shutdown_connections 2024-05-15T16:34:01.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.162+0000 7f6744608700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f672806e210 0x7f67280706d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:01.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.162+0000 7f6744608700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f673c1030e0 0x7f673c198e20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:01.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.162+0000 7f6744608700 1 --2- 172.21.15.49:0/3323379856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f673c103a90 0x7f673c199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:01.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.162+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 >> 172.21.15.49:0/3323379856 conn(0x7f673c0fe990 msgr2=0x7f673c0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:01.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.163+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 shutdown_connections 2024-05-15T16:34:01.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:01.163+0000 7f6744608700 1 -- 172.21.15.49:0/3323379856 wait complete. 2024-05-15T16:34:02.041 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:01 smithi049 ceph-mon[112142]: Upgrade: Updating ceph-exporter.smithi144 (2/2) 2024-05-15T16:34:02.041 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:01 smithi049 ceph-mon[112142]: Deploying daemon ceph-exporter.smithi144 on smithi144 2024-05-15T16:34:02.041 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:01 smithi049 ceph-mon[112142]: from='client.36020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:02.041 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:01 smithi049 ceph-mon[112142]: pgmap v1113: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-05-15T16:34:02.041 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:01 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3323379856' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:34:02.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:01 smithi144 ceph-mon[86635]: Upgrade: Updating ceph-exporter.smithi144 (2/2) 2024-05-15T16:34:02.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:01 smithi144 ceph-mon[86635]: Deploying daemon ceph-exporter.smithi144 on smithi144 2024-05-15T16:34:02.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:01 smithi144 ceph-mon[86635]: from='client.36020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:02.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:01 smithi144 ceph-mon[86635]: pgmap v1113: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-05-15T16:34:02.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:01 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3323379856' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:34:02.838 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:02 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:resolve 2024-05-15T16:34:02.839 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:02 smithi144 ceph-mon[86635]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:resolve,1=cephfs.smithi049.gocadf=up:resolve} 2 up:standby 2024-05-15T16:34:03.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:02 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:resolve 2024-05-15T16:34:03.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:02 smithi049 ceph-mon[112142]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:resolve,1=cephfs.smithi049.gocadf=up:resolve} 2 up:standby 2024-05-15T16:34:03.899 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:03 smithi144 ceph-mon[86635]: reconnect by client.24353 192.168.0.1:0/2186632469 after 0.000999972 2024-05-15T16:34:03.899 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:03 smithi144 ceph-mon[86635]: reconnect by client.24353 192.168.0.1:0/2186632469 after 0.000999972 2024-05-15T16:34:03.899 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:03 smithi144 ceph-mon[86635]: reconnect by client.14534 192.168.0.1:0/1252206647 after 0.000999972 2024-05-15T16:34:03.899 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:03 smithi144 ceph-mon[86635]: reconnect by client.14534 192.168.0.1:0/1252206647 after 0.00199994 2024-05-15T16:34:03.899 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:03 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:reconnect 2024-05-15T16:34:03.899 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:03 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:reconnect 2024-05-15T16:34:03.900 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:03 smithi144 ceph-mon[86635]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:reconnect,1=cephfs.smithi049.gocadf=up:reconnect} 2 up:standby 2024-05-15T16:34:03.900 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:03 smithi144 ceph-mon[86635]: pgmap v1114: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-05-15T16:34:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:03 smithi049 ceph-mon[112142]: reconnect by client.24353 192.168.0.1:0/2186632469 after 0.000999972 2024-05-15T16:34:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:03 smithi049 ceph-mon[112142]: reconnect by client.24353 192.168.0.1:0/2186632469 after 0.000999972 2024-05-15T16:34:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:03 smithi049 ceph-mon[112142]: reconnect by client.14534 192.168.0.1:0/1252206647 after 0.000999972 2024-05-15T16:34:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:03 smithi049 ceph-mon[112142]: reconnect by client.14534 192.168.0.1:0/1252206647 after 0.00199994 2024-05-15T16:34:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:03 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:reconnect 2024-05-15T16:34:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:03 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:reconnect 2024-05-15T16:34:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:03 smithi049 ceph-mon[112142]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:reconnect,1=cephfs.smithi049.gocadf=up:reconnect} 2 up:standby 2024-05-15T16:34:04.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:03 smithi049 ceph-mon[112142]: pgmap v1114: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-05-15T16:34:05.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:rejoin 2024-05-15T16:34:05.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:rejoin 2024-05-15T16:34:05.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:rejoin,1=cephfs.smithi049.gocadf=up:rejoin} 2 up:standby 2024-05-15T16:34:05.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: daemon mds.cephfs.smithi049.gocadf is now active in filesystem cephfs as rank 1 2024-05-15T16:34:05.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: daemon mds.cephfs.smithi049.abbagx is now active in filesystem cephfs as rank 0 2024-05-15T16:34:05.148 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.148 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.148 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:34:05.148 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.148 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.148 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.148 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:04 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:rejoin 2024-05-15T16:34:05.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:rejoin 2024-05-15T16:34:05.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: fsmap cephfs:2/2 {0=cephfs.smithi049.abbagx=up:rejoin,1=cephfs.smithi049.gocadf=up:rejoin} 2 up:standby 2024-05-15T16:34:05.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: daemon mds.cephfs.smithi049.gocadf is now active in filesystem cephfs as rank 1 2024-05-15T16:34:05.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: daemon mds.cephfs.smithi049.abbagx is now active in filesystem cephfs as rank 0 2024-05-15T16:34:05.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:34:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:05.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:04 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:06.068 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:05 smithi144 ceph-mon[86635]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-15T16:34:06.068 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:05 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:active 2024-05-15T16:34:06.068 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:05 smithi144 ceph-mon[86635]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:active 2024-05-15T16:34:06.068 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:05 smithi144 ceph-mon[86635]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi049.gocadf=up:active} 2 up:standby 2024-05-15T16:34:06.069 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:05 smithi144 ceph-mon[86635]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi049.gocadf=up:active} 2 up:standby-replay 2024-05-15T16:34:06.069 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:05 smithi144 ceph-mon[86635]: pgmap v1115: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 85 B/s wr, 5 op/s 2024-05-15T16:34:06.069 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:06.069 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:34:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:05 smithi049 ceph-mon[112142]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-15T16:34:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:05 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] up:active 2024-05-15T16:34:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:05 smithi049 ceph-mon[112142]: mds.? [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] up:active 2024-05-15T16:34:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:05 smithi049 ceph-mon[112142]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi049.gocadf=up:active} 2 up:standby 2024-05-15T16:34:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:05 smithi049 ceph-mon[112142]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi049.gocadf=up:active} 2 up:standby-replay 2024-05-15T16:34:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:05 smithi049 ceph-mon[112142]: pgmap v1115: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 85 B/s wr, 5 op/s 2024-05-15T16:34:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:06.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:34:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:07 smithi144 ceph-mon[86635]: pgmap v1116: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-15T16:34:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:07 smithi049 ceph-mon[112142]: pgmap v1116: 65 pgs: 65 active+clean; 254 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-15T16:34:10.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:10 smithi144 ceph-mon[86635]: pgmap v1117: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-05-15T16:34:10.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:10 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:10.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:10 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:10.699 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:10 smithi049 ceph-mon[112142]: pgmap v1117: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-05-15T16:34:10.699 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:10 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:10.699 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:10 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:11.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:11.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:11 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:11.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:11 smithi144 ceph-mon[86635]: pgmap v1118: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4.8 KiB/s wr, 16 op/s 2024-05-15T16:34:11.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:11.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:11 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:11.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:11 smithi049 ceph-mon[112142]: pgmap v1118: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4.8 KiB/s wr, 16 op/s 2024-05-15T16:34:14.035 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:13 smithi144 ceph-mon[86635]: pgmap v1119: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-05-15T16:34:14.035 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:14.035 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:13 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:13 smithi049 ceph-mon[112142]: pgmap v1119: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-05-15T16:34:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:13 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:15.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:15.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:15.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:15.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:15 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:15.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:15.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:15.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:15.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:15 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:16 smithi049 ceph-mon[112142]: pgmap v1120: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-05-15T16:34:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:16 smithi144 ceph-mon[86635]: pgmap v1120: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-05-15T16:34:18.442 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:18 smithi049 ceph-mon[112142]: pgmap v1121: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-05-15T16:34:18.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:18 smithi144 ceph-mon[86635]: pgmap v1121: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: pgmap v1122: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 352 B/s wr, 14 op/s 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: Upgrade: Setting filesystem cephfs Joinable 2024-05-15T16:34:19.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:19 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-15T16:34:19.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:19.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:19.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:34:19.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:34:19.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: pgmap v1122: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 352 B/s wr, 14 op/s 2024-05-15T16:34:19.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:34:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: Upgrade: Setting filesystem cephfs Joinable 2024-05-15T16:34:19.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:19 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-15T16:34:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi049.gocadf=up:active} 2 up:standby-replay 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all ceph-exporter 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi049"}]: dispatch 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi049"}]': finished 2024-05-15T16:34:20.949 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi144"}]: dispatch 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi144"}]': finished 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all iscsi 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all nfs 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: Upgrade: Setting container_image for all nvmeof 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.950 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: Upgrade: Finalizing container_image settings 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-15T16:34:20.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.952 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: Upgrade: Complete! 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.953 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:34:20.954 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: fsmap cephfs:2 {0=cephfs.smithi049.abbagx=up:active,1=cephfs.smithi049.gocadf=up:active} 2 up:standby-replay 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all ceph-exporter 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi049"}]: dispatch 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi049"}]': finished 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi144"}]: dispatch 2024-05-15T16:34:20.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi144"}]': finished 2024-05-15T16:34:20.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all iscsi 2024-05-15T16:34:20.956 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all nfs 2024-05-15T16:34:20.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: Upgrade: Setting container_image for all nvmeof 2024-05-15T16:34:20.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.960 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: Upgrade: Finalizing container_image settings 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-15T16:34:20.961 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-15T16:34:20.962 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-15T16:34:20.963 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-15T16:34:20.963 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-15T16:34:20.963 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-15T16:34:20.963 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-15T16:34:20.963 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-15T16:34:20.963 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.963 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.963 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.964 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.964 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.964 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-15T16:34:20.964 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: Upgrade: Complete! 2024-05-15T16:34:20.964 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-15T16:34:20.964 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-15T16:34:20.964 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:34:20.965 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:34:21.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:21 smithi144 ceph-mon[86635]: pgmap v1123: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 10 op/s 2024-05-15T16:34:21.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:21 smithi049 ceph-mon[112142]: pgmap v1123: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 10 op/s 2024-05-15T16:34:23.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:23 smithi144 ceph-mon[86635]: pgmap v1124: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 6 op/s 2024-05-15T16:34:23.949 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:23 smithi049 ceph-mon[112142]: pgmap v1124: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 6 op/s 2024-05-15T16:34:25.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:25 smithi144 ceph-mon[86635]: pgmap v1125: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 3 op/s 2024-05-15T16:34:25.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:25 smithi049 ceph-mon[112142]: pgmap v1125: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 3 op/s 2024-05-15T16:34:27.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:27 smithi144 ceph-mon[86635]: pgmap v1126: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 615 KiB/s rd, 4 op/s 2024-05-15T16:34:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:27 smithi049 ceph-mon[112142]: pgmap v1126: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 615 KiB/s rd, 4 op/s 2024-05-15T16:34:29.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:29 smithi144 ceph-mon[86635]: pgmap v1127: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-15T16:34:29.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:29 smithi049 ceph-mon[112142]: pgmap v1127: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-15T16:34:31.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.291+0000 7f80f3ef9700 1 -- 172.21.15.49:0/2212385837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec1027c0 msgr2=0x7f80ec10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.291+0000 7f80f3ef9700 1 --2- 172.21.15.49:0/2212385837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec1027c0 0x7f80ec10acb0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009940 tx=0x7f80e0009c50 comp rx=0 tx=0).stop 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.291+0000 7f80f3ef9700 1 -- 172.21.15.49:0/2212385837 shutdown_connections 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.291+0000 7f80f3ef9700 1 --2- 172.21.15.49:0/2212385837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec1027c0 0x7f80ec10acb0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.291+0000 7f80f3ef9700 1 --2- 172.21.15.49:0/2212385837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80ec101ea0 0x7f80ec102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.291+0000 7f80f3ef9700 1 -- 172.21.15.49:0/2212385837 >> 172.21.15.49:0/2212385837 conn(0x7f80ec076800 msgr2=0x7f80ec0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.292+0000 7f80f3ef9700 1 -- 172.21.15.49:0/2212385837 shutdown_connections 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.292+0000 7f80f3ef9700 1 -- 172.21.15.49:0/2212385837 wait complete. 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.292+0000 7f80f3ef9700 1 Processor -- start 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.292+0000 7f80f3ef9700 1 -- start start 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f3ef9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec101ea0 0x7f80ec0ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f3ef9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80ec1027c0 0x7f80ec100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f3ef9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec100580 con 0x7f80ec101ea0 2024-05-15T16:34:31.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f3ef9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1006c0 con 0x7f80ec1027c0 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f1c95700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec101ea0 0x7f80ec0ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f1c95700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec101ea0 0x7f80ec0ffb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49144/0 (socket says 172.21.15.49:49144) 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f1c95700 1 -- 172.21.15.49:0/3452377946 learned_addr learned my addr 172.21.15.49:0/3452377946 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f1494700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80ec1027c0 0x7f80ec100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f1c95700 1 -- 172.21.15.49:0/3452377946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80ec1027c0 msgr2=0x7f80ec100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f1c95700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80ec1027c0 0x7f80ec100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f1c95700 1 -- 172.21.15.49:0/3452377946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e00095f0 con 0x7f80ec101ea0 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f1494700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80ec1027c0 0x7f80ec100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f1c95700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec101ea0 0x7f80ec0ffb00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00ef10 tx=0x7f80dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80eaffd700 1 -- 172.21.15.49:0/3452377946 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80dc00ca50 con 0x7f80ec101ea0 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec100940 con 0x7f80ec101ea0 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.293+0000 7f80eaffd700 1 -- 172.21.15.49:0/3452377946 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80dc00f040 con 0x7f80ec101ea0 2024-05-15T16:34:31.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.294+0000 7f80eaffd700 1 -- 172.21.15.49:0/3452377946 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80dc0146b0 con 0x7f80ec101ea0 2024-05-15T16:34:31.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.294+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec1a3080 con 0x7f80ec101ea0 2024-05-15T16:34:31.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.294+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec068ee0 con 0x7f80ec101ea0 2024-05-15T16:34:31.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.295+0000 7f80eaffd700 1 -- 172.21.15.49:0/3452377946 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f80dc00cbb0 con 0x7f80ec101ea0 2024-05-15T16:34:31.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.296+0000 7f80eaffd700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d406e210 0x7f80d40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:31.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.296+0000 7f80eaffd700 1 -- 172.21.15.49:0/3452377946 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f80dc08e590 con 0x7f80ec101ea0 2024-05-15T16:34:31.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.296+0000 7f80f1494700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d406e210 0x7f80d40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:31.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.296+0000 7f80f1494700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d406e210 0x7f80d40706d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006010 tx=0x7f80e0005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:31.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.301+0000 7f80eaffd700 1 -- 172.21.15.49:0/3452377946 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f80dc057870 con 0x7f80ec101ea0 2024-05-15T16:34:31.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.532+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80ec10de80 con 0x7f80d406e210 2024-05-15T16:34:31.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.533+0000 7f80eaffd700 1 -- 172.21.15.49:0/3452377946 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f80ec10de80 con 0x7f80d406e210 2024-05-15T16:34:31.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d406e210 msgr2=0x7f80d40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:31.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d406e210 0x7f80d40706d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006010 tx=0x7f80e0005ab0 comp rx=0 tx=0).stop 2024-05-15T16:34:31.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec101ea0 msgr2=0x7f80ec0ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:31.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec101ea0 0x7f80ec0ffb00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00ef10 tx=0x7f80dc00c3a0 comp rx=0 tx=0).stop 2024-05-15T16:34:31.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 shutdown_connections 2024-05-15T16:34:31.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f80d406e210 0x7f80d40706d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80ec101ea0 0x7f80ec0ffb00 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 --2- 172.21.15.49:0/3452377946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80ec1027c0 0x7f80ec100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 >> 172.21.15.49:0/3452377946 conn(0x7f80ec076800 msgr2=0x7f80ec103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 shutdown_connections 2024-05-15T16:34:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:31.536+0000 7f80f3ef9700 1 -- 172.21.15.49:0/3452377946 wait complete. 2024-05-15T16:34:31.729 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:31 smithi049 ceph-mon[112142]: pgmap v1128: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:34:31.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:31 smithi144 ceph-mon[86635]: pgmap v1128: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:34:32.186 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph orch ps' 2024-05-15T16:34:32.414 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:34:32.809 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:32 smithi049 ceph-mon[112142]: from='client.36028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:32.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:32 smithi144 ceph-mon[86635]: from='client.36028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:33.881 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:33 smithi049 ceph-mon[112142]: pgmap v1129: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:33.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:33 smithi144 ceph-mon[86635]: pgmap v1129: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.127+0000 7f032eb6c700 1 -- 172.21.15.49:0/2177571948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 msgr2=0x7f03281012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.127+0000 7f032eb6c700 1 --2- 172.21.15.49:0/2177571948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 0x7f03281012c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0318009910 tx=0x7f0318009c20 comp rx=0 tx=0).stop 2024-05-15T16:34:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.127+0000 7f032eb6c700 1 -- 172.21.15.49:0/2177571948 shutdown_connections 2024-05-15T16:34:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.127+0000 7f032eb6c700 1 --2- 172.21.15.49:0/2177571948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0328101890 0x7f03281058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.127+0000 7f032eb6c700 1 --2- 172.21.15.49:0/2177571948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 0x7f03281012c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.127+0000 7f032eb6c700 1 -- 172.21.15.49:0/2177571948 >> 172.21.15.49:0/2177571948 conn(0x7f03280fc790 msgr2=0x7f03280febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.127+0000 7f032eb6c700 1 -- 172.21.15.49:0/2177571948 shutdown_connections 2024-05-15T16:34:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.127+0000 7f032eb6c700 1 -- 172.21.15.49:0/2177571948 wait complete. 2024-05-15T16:34:34.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.127+0000 7f032eb6c700 1 Processor -- start 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032eb6c700 1 -- start start 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032eb6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 0x7f032819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032eb6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0328101890 0x7f032819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032eb6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032819d460 con 0x7f0328100ee0 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032eb6c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0328196a90 con 0x7f0328101890 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032c908700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 0x7f032819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032c908700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 0x7f032819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46490/0 (socket says 172.21.15.49:46490) 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032c908700 1 -- 172.21.15.49:0/2323939990 learned_addr learned my addr 172.21.15.49:0/2323939990 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f0327fff700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0328101890 0x7f032819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032c908700 1 -- 172.21.15.49:0/2323939990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0328101890 msgr2=0x7f032819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032c908700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0328101890 0x7f032819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:34.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032c908700 1 -- 172.21.15.49:0/2323939990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180095f0 con 0x7f0328100ee0 2024-05-15T16:34:34.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f0327fff700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0328101890 0x7f032819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:34:34.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.128+0000 7f032c908700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 0x7f032819c900 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f031800bd70 tx=0x7f031800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:34.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.129+0000 7f0325ffb700 1 -- 172.21.15.49:0/2323939990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f031801d070 con 0x7f0328100ee0 2024-05-15T16:34:34.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.129+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0328196d10 con 0x7f0328100ee0 2024-05-15T16:34:34.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.129+0000 7f0325ffb700 1 -- 172.21.15.49:0/2323939990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03180056e0 con 0x7f0328100ee0 2024-05-15T16:34:34.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.129+0000 7f0325ffb700 1 -- 172.21.15.49:0/2323939990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0318017400 con 0x7f0328100ee0 2024-05-15T16:34:34.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.129+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0328197200 con 0x7f0328100ee0 2024-05-15T16:34:34.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.130+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0328109280 con 0x7f0328100ee0 2024-05-15T16:34:34.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.131+0000 7f0325ffb700 1 -- 172.21.15.49:0/2323939990 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0318003a60 con 0x7f0328100ee0 2024-05-15T16:34:34.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.131+0000 7f0325ffb700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f031406e200 0x7f03140706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:34.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.131+0000 7f0325ffb700 1 -- 172.21.15.49:0/2323939990 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f031808f620 con 0x7f0328100ee0 2024-05-15T16:34:34.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.131+0000 7f0327fff700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f031406e200 0x7f03140706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:34.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.132+0000 7f0327fff700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f031406e200 0x7f03140706c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f03100094f0 tx=0x7f0310009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:34.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.136+0000 7f0325ffb700 1 -- 172.21.15.49:0/2323939990 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f031805c430 con 0x7f0328100ee0 2024-05-15T16:34:34.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.351+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0328197f70 con 0x7f031406e200 2024-05-15T16:34:34.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.362+0000 7f0325ffb700 1 -- 172.21.15.49:0/2323939990 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0328197f70 con 0x7f031406e200 2024-05-15T16:34:34.363 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:34:34.363 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (37m) 20s ago 49m 24.2M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (36s) 20s ago 49m 7486k - 19.0.0-2576-gde5210d5 48bc18eb22d8 ed7b98cce7c8 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (30s) 25s ago 48m 6971k - 19.0.0-2576-gde5210d5 48bc18eb22d8 d734ad348c4b 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (35m) 20s ago 49m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (35m) 25s ago 48m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (37m) 20s ago 49m 78.3M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (31m) 20s ago 43m 165M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (31m) 20s ago 43m 25.3M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (30m) 25s ago 43m 150M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (15m) 25s ago 43m 20.9M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (40m) 20s ago 50m 519M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (38m) 25s ago 48m 461M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (36m) 20s ago 50m 87.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:34:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (35m) 25s ago 48m 61.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:34:34.365 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (38m) 20s ago 49m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:34:34.365 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (38m) 25s ago 48m 16.3M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:34:34.365 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (34m) 20s ago 47m 98.0M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:34:34.365 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (34m) 20s ago 47m 100M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:34:34.365 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (33m) 20s ago 46m 96.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:34:34.365 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (32m) 25s ago 46m 112M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:34:34.365 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (32m) 25s ago 45m 95.2M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:34:34.365 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (31m) 25s ago 45m 93.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:34:34.365 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (38m) 20s ago 48m 87.1M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:34:34.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f031406e200 msgr2=0x7f03140706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:34.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f031406e200 0x7f03140706c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f03100094f0 tx=0x7f0310009040 comp rx=0 tx=0).stop 2024-05-15T16:34:34.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 msgr2=0x7f032819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:34.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 0x7f032819c900 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f031800bd70 tx=0x7f031800be50 comp rx=0 tx=0).stop 2024-05-15T16:34:34.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 shutdown_connections 2024-05-15T16:34:34.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f031406e200 0x7f03140706c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:34.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0328100ee0 0x7f032819c900 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:34.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 --2- 172.21.15.49:0/2323939990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0328101890 0x7f032819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:34.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 >> 172.21.15.49:0/2323939990 conn(0x7f03280fc790 msgr2=0x7f03280fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:34.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 shutdown_connections 2024-05-15T16:34:34.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:34.364+0000 7f032eb6c700 1 -- 172.21.15.49:0/2323939990 wait complete. 2024-05-15T16:34:34.969 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph orch upgrade status' 2024-05-15T16:34:35.187 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:34:35.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:35 smithi144 ceph-mon[86635]: from='client.36032 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:35.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:35 smithi144 ceph-mon[86635]: pgmap v1130: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:35.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:34:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:35 smithi049 ceph-mon[112142]: from='client.36032 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:35 smithi049 ceph-mon[112142]: pgmap v1130: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:35.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:34:36.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.796+0000 7f61950d3700 1 -- 172.21.15.49:0/1812804429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 msgr2=0x7f61901047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:36.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.796+0000 7f61950d3700 1 --2- 172.21.15.49:0/1812804429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 0x7f61901047c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6184009910 tx=0x7f6184009c20 comp rx=0 tx=0).stop 2024-05-15T16:34:36.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.796+0000 7f61950d3700 1 -- 172.21.15.49:0/1812804429 shutdown_connections 2024-05-15T16:34:36.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.796+0000 7f61950d3700 1 --2- 172.21.15.49:0/1812804429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 0x7f61901047c0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:36.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.796+0000 7f61950d3700 1 --2- 172.21.15.49:0/1812804429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61900ffdc0 0x7f61901001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:36.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.796+0000 7f61950d3700 1 -- 172.21.15.49:0/1812804429 >> 172.21.15.49:0/1812804429 conn(0x7f6190076800 msgr2=0x7f61900fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:36.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.796+0000 7f61950d3700 1 -- 172.21.15.49:0/1812804429 shutdown_connections 2024-05-15T16:34:36.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.796+0000 7f61950d3700 1 -- 172.21.15.49:0/1812804429 wait complete. 2024-05-15T16:34:36.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.797+0000 7f61950d3700 1 Processor -- start 2024-05-15T16:34:36.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f61950d3700 1 -- start start 2024-05-15T16:34:36.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f61950d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61900ffdc0 0x7f619019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:36.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f61950d3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 0x7f619019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:36.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f61950d3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619019c2e0 con 0x7f6190100770 2024-05-15T16:34:36.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f61950d3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61901958e0 con 0x7f61900ffdc0 2024-05-15T16:34:36.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f618e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 0x7f619019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:36.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f618e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 0x7f619019bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46516/0 (socket says 172.21.15.49:46516) 2024-05-15T16:34:36.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f618e59c700 1 -- 172.21.15.49:0/1058906617 learned_addr learned my addr 172.21.15.49:0/1058906617 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:34:36.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f618ed9d700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61900ffdc0 0x7f619019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:36.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f618e59c700 1 -- 172.21.15.49:0/1058906617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61900ffdc0 msgr2=0x7f619019b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:36.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f618e59c700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61900ffdc0 0x7f619019b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:36.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f618e59c700 1 -- 172.21.15.49:0/1058906617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61840095f0 con 0x7f6190100770 2024-05-15T16:34:36.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f618ed9d700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61900ffdc0 0x7f619019b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:34:36.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.798+0000 7f618e59c700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 0x7f619019bce0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6184005b40 tx=0x7f618400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.799+0000 7f6177fff700 1 -- 172.21.15.49:0/1058906617 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f618401d070 con 0x7f6190100770 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.799+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6190195b60 con 0x7f6190100770 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.799+0000 7f6177fff700 1 -- 172.21.15.49:0/1058906617 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f618400f460 con 0x7f6190100770 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.799+0000 7f6177fff700 1 -- 172.21.15.49:0/1058906617 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6184017630 con 0x7f6190100770 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.799+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6190196050 con 0x7f6190100770 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.800+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61901081c0 con 0x7f6190100770 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.800+0000 7f6177fff700 1 -- 172.21.15.49:0/1058906617 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f61840178a0 con 0x7f6190100770 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.801+0000 7f6177fff700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f617806e210 0x7f61780706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.801+0000 7f6177fff700 1 -- 172.21.15.49:0/1058906617 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f618408f600 con 0x7f6190100770 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.801+0000 7f618ed9d700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f617806e210 0x7f61780706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.801+0000 7f618ed9d700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f617806e210 0x7f61780706d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6180005fd0 tx=0x7f6180005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:36.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:36.805+0000 7f6177fff700 1 -- 172.21.15.49:0/1058906617 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f618405c480 con 0x7f6190100770 2024-05-15T16:34:37.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.035+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6190196da0 con 0x7f617806e210 2024-05-15T16:34:37.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.036+0000 7f6177fff700 1 -- 172.21.15.49:0/1058906617 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f6190196da0 con 0x7f617806e210 2024-05-15T16:34:37.036 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:34:37.036 INFO:teuthology.orchestra.run.smithi049.stdout: "target_image": null, 2024-05-15T16:34:37.036 INFO:teuthology.orchestra.run.smithi049.stdout: "in_progress": false, 2024-05-15T16:34:37.036 INFO:teuthology.orchestra.run.smithi049.stdout: "which": "", 2024-05-15T16:34:37.036 INFO:teuthology.orchestra.run.smithi049.stdout: "services_complete": [], 2024-05-15T16:34:37.036 INFO:teuthology.orchestra.run.smithi049.stdout: "progress": null, 2024-05-15T16:34:37.037 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "", 2024-05-15T16:34:37.037 INFO:teuthology.orchestra.run.smithi049.stdout: "is_paused": false 2024-05-15T16:34:37.037 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f617806e210 msgr2=0x7f61780706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f617806e210 0x7f61780706d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6180005fd0 tx=0x7f6180005e20 comp rx=0 tx=0).stop 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 msgr2=0x7f619019bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 0x7f619019bce0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6184005b40 tx=0x7f618400be20 comp rx=0 tx=0).stop 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 shutdown_connections 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f617806e210 0x7f61780706d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61900ffdc0 0x7f619019b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 --2- 172.21.15.49:0/1058906617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6190100770 0x7f619019bce0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 >> 172.21.15.49:0/1058906617 conn(0x7f6190076800 msgr2=0x7f6190103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 shutdown_connections 2024-05-15T16:34:37.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:37.038+0000 7f61950d3700 1 -- 172.21.15.49:0/1058906617 wait complete. 2024-05-15T16:34:37.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph health detail' 2024-05-15T16:34:37.843 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:34:37.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:37 smithi144 ceph-mon[86635]: pgmap v1131: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:37.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:37 smithi144 ceph-mon[86635]: from='client.36036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:37.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:37 smithi049 ceph-mon[112142]: pgmap v1131: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:37.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:37 smithi049 ceph-mon[112142]: from='client.36036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.411+0000 7fca27b5e700 1 -- 172.21.15.49:0/919160850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20100ee0 msgr2=0x7fca201012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.411+0000 7fca27b5e700 1 --2- 172.21.15.49:0/919160850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20100ee0 0x7fca201012c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009910 tx=0x7fca1c009c20 comp rx=0 tx=0).stop 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.411+0000 7fca27b5e700 1 -- 172.21.15.49:0/919160850 shutdown_connections 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.411+0000 7fca27b5e700 1 --2- 172.21.15.49:0/919160850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca20101890 0x7fca201058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.411+0000 7fca27b5e700 1 --2- 172.21.15.49:0/919160850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20100ee0 0x7fca201012c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.411+0000 7fca27b5e700 1 -- 172.21.15.49:0/919160850 >> 172.21.15.49:0/919160850 conn(0x7fca200fc790 msgr2=0x7fca200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.411+0000 7fca27b5e700 1 -- 172.21.15.49:0/919160850 shutdown_connections 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.411+0000 7fca27b5e700 1 -- 172.21.15.49:0/919160850 wait complete. 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca27b5e700 1 Processor -- start 2024-05-15T16:34:39.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca27b5e700 1 -- start start 2024-05-15T16:34:39.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca27b5e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca20100ee0 0x7fca2019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:39.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca27b5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20101890 0x7fca2019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:39.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca27b5e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2019d470 con 0x7fca20101890 2024-05-15T16:34:39.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca27b5e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20196a50 con 0x7fca20100ee0 2024-05-15T16:34:39.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca250f9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20101890 0x7fca2019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:39.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca250f9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20101890 0x7fca2019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46534/0 (socket says 172.21.15.49:46534) 2024-05-15T16:34:39.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca250f9700 1 -- 172.21.15.49:0/1936283074 learned_addr learned my addr 172.21.15.49:0/1936283074 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.412+0000 7fca258fa700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca20100ee0 0x7fca2019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.413+0000 7fca250f9700 1 -- 172.21.15.49:0/1936283074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca20100ee0 msgr2=0x7fca2019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.413+0000 7fca250f9700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca20100ee0 0x7fca2019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.413+0000 7fca250f9700 1 -- 172.21.15.49:0/1936283074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c0095f0 con 0x7fca20101890 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.413+0000 7fca250f9700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20101890 0x7fca2019ce50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c8f0 tx=0x7fca1000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.413+0000 7fca16ffd700 1 -- 172.21.15.49:0/1936283074 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca100078a0 con 0x7fca20101890 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.413+0000 7fca16ffd700 1 -- 172.21.15.49:0/1936283074 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca1000f450 con 0x7fca20101890 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.413+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca20196d30 con 0x7fca20101890 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.413+0000 7fca16ffd700 1 -- 172.21.15.49:0/1936283074 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca10018610 con 0x7fca20101890 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.413+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca20197280 con 0x7fca20101890 2024-05-15T16:34:39.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.414+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca20109280 con 0x7fca20101890 2024-05-15T16:34:39.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.415+0000 7fca16ffd700 1 -- 172.21.15.49:0/1936283074 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fca1000f5c0 con 0x7fca20101890 2024-05-15T16:34:39.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.415+0000 7fca16ffd700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca0c06e200 0x7fca0c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:39.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.415+0000 7fca16ffd700 1 -- 172.21.15.49:0/1936283074 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fca1008e0e0 con 0x7fca20101890 2024-05-15T16:34:39.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.415+0000 7fca258fa700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca0c06e200 0x7fca0c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:39.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.416+0000 7fca258fa700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca0c06e200 0x7fca0c0706c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:39.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.420+0000 7fca16ffd700 1 -- 172.21.15.49:0/1936283074 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fca1005aef0 con 0x7fca20101890 2024-05-15T16:34:39.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:39 smithi049 ceph-mon[112142]: pgmap v1132: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:34:39.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.716+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fca201023c0 con 0x7fca20101890 2024-05-15T16:34:39.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.716+0000 7fca16ffd700 1 -- 172.21.15.49:0/1936283074 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fca1005aa80 con 0x7fca20101890 2024-05-15T16:34:39.717 INFO:teuthology.orchestra.run.smithi049.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T16:34:39.717 INFO:teuthology.orchestra.run.smithi049.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:34:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:34:39.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca0c06e200 msgr2=0x7fca0c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca0c06e200 0x7fca0c0706c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c01a040 comp rx=0 tx=0).stop 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20101890 msgr2=0x7fca2019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20101890 0x7fca2019ce50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c8f0 tx=0x7fca1000cc00 comp rx=0 tx=0).stop 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 shutdown_connections 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fca0c06e200 0x7fca0c0706c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca20100ee0 0x7fca2019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 --2- 172.21.15.49:0/1936283074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fca20101890 0x7fca2019ce50 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 >> 172.21.15.49:0/1936283074 conn(0x7fca200fc790 msgr2=0x7fca200fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 shutdown_connections 2024-05-15T16:34:39.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:39.718+0000 7fca27b5e700 1 -- 172.21.15.49:0/1936283074 wait complete. 2024-05-15T16:34:39.750 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:39 smithi144 ceph-mon[86635]: pgmap v1132: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:34:40.299 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph versions' 2024-05-15T16:34:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:34:40.870 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:40 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1936283074' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:34:40.947 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:40 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1936283074' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-15T16:34:41.828 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:41 smithi049 ceph-mon[112142]: pgmap v1133: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:41.947 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:41 smithi144 ceph-mon[86635]: pgmap v1133: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.126+0000 7fd26b460700 1 -- 172.21.15.49:0/1129296142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 msgr2=0x7fd2641012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.126+0000 7fd26b460700 1 --2- 172.21.15.49:0/1129296142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 0x7fd2641012c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd260009910 tx=0x7fd260009c20 comp rx=0 tx=0).stop 2024-05-15T16:34:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.126+0000 7fd26b460700 1 -- 172.21.15.49:0/1129296142 shutdown_connections 2024-05-15T16:34:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.126+0000 7fd26b460700 1 --2- 172.21.15.49:0/1129296142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd264101890 0x7fd2641058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.126+0000 7fd26b460700 1 --2- 172.21.15.49:0/1129296142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 0x7fd2641012c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.126+0000 7fd26b460700 1 -- 172.21.15.49:0/1129296142 >> 172.21.15.49:0/1129296142 conn(0x7fd2640fc790 msgr2=0x7fd2640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.126+0000 7fd26b460700 1 -- 172.21.15.49:0/1129296142 shutdown_connections 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.126+0000 7fd26b460700 1 -- 172.21.15.49:0/1129296142 wait complete. 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd26b460700 1 Processor -- start 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd26b460700 1 -- start start 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd26b460700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 0x7fd2640732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd26b460700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd264101890 0x7fd2640737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd26b460700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264073d20 con 0x7fd264100ee0 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd26b460700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264073e60 con 0x7fd264101890 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd2691fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 0x7fd2640732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd2691fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 0x7fd2640732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46550/0 (socket says 172.21.15.49:46550) 2024-05-15T16:34:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd2691fc700 1 -- 172.21.15.49:0/3690036947 learned_addr learned my addr 172.21.15.49:0/3690036947 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd2689fb700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd264101890 0x7fd2640737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd2691fc700 1 -- 172.21.15.49:0/3690036947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd264101890 msgr2=0x7fd2640737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd2691fc700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd264101890 0x7fd2640737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd2691fc700 1 -- 172.21.15.49:0/3690036947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2600095f0 con 0x7fd264100ee0 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.127+0000 7fd2689fb700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd264101890 0x7fd2640737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.128+0000 7fd2691fc700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 0x7fd2640732a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd26000bd70 tx=0x7fd26000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.128+0000 7fd25a7fc700 1 -- 172.21.15.49:0/3690036947 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26001d070 con 0x7fd264100ee0 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.128+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2640740e0 con 0x7fd264100ee0 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.128+0000 7fd25a7fc700 1 -- 172.21.15.49:0/3690036947 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2600056e0 con 0x7fd264100ee0 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.128+0000 7fd25a7fc700 1 -- 172.21.15.49:0/3690036947 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd260017400 con 0x7fd264100ee0 2024-05-15T16:34:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.128+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2641a3090 con 0x7fd264100ee0 2024-05-15T16:34:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.129+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd264109280 con 0x7fd264100ee0 2024-05-15T16:34:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.129+0000 7fd25a7fc700 1 -- 172.21.15.49:0/3690036947 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd260003a60 con 0x7fd264100ee0 2024-05-15T16:34:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.130+0000 7fd25a7fc700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd25006e200 0x7fd2500706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:34:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.130+0000 7fd25a7fc700 1 -- 172.21.15.49:0/3690036947 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd26008f5c0 con 0x7fd264100ee0 2024-05-15T16:34:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.130+0000 7fd2689fb700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd25006e200 0x7fd2500706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:34:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.130+0000 7fd2689fb700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd25006e200 0x7fd2500706c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd254005950 tx=0x7fd2540058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:34:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.135+0000 7fd25a7fc700 1 -- 172.21.15.49:0/3690036947 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd2600588a0 con 0x7fd264100ee0 2024-05-15T16:34:42.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.429+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd264074840 con 0x7fd264100ee0 2024-05-15T16:34:42.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.429+0000 7fd25a7fc700 1 -- 172.21.15.49:0/3690036947 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd260058030 con 0x7fd264100ee0 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:34:42.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:34:42.431 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:34:42.431 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:34:42.431 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:34:42.431 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:34:42.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd25006e200 msgr2=0x7fd2500706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:42.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd25006e200 0x7fd2500706c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd254005950 tx=0x7fd2540058e0 comp rx=0 tx=0).stop 2024-05-15T16:34:42.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 msgr2=0x7fd2640732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:34:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 0x7fd2640732a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd26000bd70 tx=0x7fd26000be50 comp rx=0 tx=0).stop 2024-05-15T16:34:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 shutdown_connections 2024-05-15T16:34:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd25006e200 0x7fd2500706c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd264100ee0 0x7fd2640732a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 --2- 172.21.15.49:0/3690036947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd264101890 0x7fd2640737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:34:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 >> 172.21.15.49:0/3690036947 conn(0x7fd2640fc790 msgr2=0x7fd2640fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:34:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 shutdown_connections 2024-05-15T16:34:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:34:42.432+0000 7fd26b460700 1 -- 172.21.15.49:0/3690036947 wait complete. 2024-05-15T16:34:42.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:42 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3690036947' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:42.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:42 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3690036947' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:34:43.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-05-15T16:34:43.242 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:34:43.947 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:43 smithi144 ceph-mon[86635]: pgmap v1134: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:43.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:43 smithi049 ceph-mon[112142]: pgmap v1134: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:44.884 INFO:teuthology.orchestra.run.smithi049.stdout:wait for servicemap items w/ changing names to refresh 2024-05-15T16:34:45.446 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'sleep 60' 2024-05-15T16:34:45.666 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:34:45.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:45 smithi049 ceph-mon[112142]: pgmap v1135: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:45.947 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:45 smithi144 ceph-mon[86635]: pgmap v1135: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:47.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:47 smithi144 ceph-mon[86635]: pgmap v1136: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:47.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:47 smithi049 ceph-mon[112142]: pgmap v1136: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:49.947 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:49 smithi144 ceph-mon[86635]: pgmap v1137: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:34:49.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:49 smithi049 ceph-mon[112142]: pgmap v1137: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:34:50.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:34:50.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:34:51.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:51 smithi144 ceph-mon[86635]: pgmap v1138: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:51.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:51 smithi049 ceph-mon[112142]: pgmap v1138: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:53.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:53 smithi144 ceph-mon[86635]: pgmap v1139: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:53.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:53 smithi049 ceph-mon[112142]: pgmap v1139: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:55.947 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:55 smithi144 ceph-mon[86635]: pgmap v1140: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:55.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:55 smithi049 ceph-mon[112142]: pgmap v1140: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:34:57.947 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:57 smithi144 ceph-mon[86635]: pgmap v1141: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:57 smithi049 ceph-mon[112142]: pgmap v1141: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:34:59.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:34:59 smithi144 ceph-mon[86635]: pgmap v1142: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:34:59.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:34:59 smithi049 ceph-mon[112142]: pgmap v1142: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:01.743 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:01 smithi049 ceph-mon[112142]: pgmap v1143: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:01.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:01 smithi144 ceph-mon[86635]: pgmap v1143: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:03.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:03 smithi144 ceph-mon[86635]: pgmap v1144: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:03.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:03 smithi049 ceph-mon[112142]: pgmap v1144: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:05 smithi144 ceph-mon[86635]: pgmap v1145: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:05.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:35:05.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:05 smithi049 ceph-mon[112142]: pgmap v1145: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:05.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:35:07.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:07 smithi144 ceph-mon[86635]: pgmap v1146: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:07.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:07 smithi049 ceph-mon[112142]: pgmap v1146: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:09.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:09 smithi144 ceph-mon[86635]: pgmap v1147: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:09.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:09 smithi049 ceph-mon[112142]: pgmap v1147: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:11.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:11 smithi144 ceph-mon[86635]: pgmap v1148: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:11 smithi049 ceph-mon[112142]: pgmap v1148: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:13.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:13 smithi144 ceph-mon[86635]: pgmap v1149: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:13.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:13 smithi049 ceph-mon[112142]: pgmap v1149: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:15.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:15 smithi144 ceph-mon[86635]: pgmap v1150: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:15.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:15 smithi049 ceph-mon[112142]: pgmap v1150: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:17.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:17 smithi144 ceph-mon[86635]: pgmap v1151: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:17.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:17 smithi049 ceph-mon[112142]: pgmap v1151: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:19.947 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:19 smithi144 ceph-mon[86635]: pgmap v1152: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:19.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:19 smithi049 ceph-mon[112142]: pgmap v1152: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:35:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:35:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:35:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:35:20.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:35:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:35:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:35:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:35:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:35:20.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:35:21.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:21 smithi144 ceph-mon[86635]: pgmap v1153: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:21.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:21 smithi049 ceph-mon[112142]: pgmap v1153: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:23.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:23 smithi144 ceph-mon[86635]: pgmap v1154: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:23.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:23 smithi049 ceph-mon[112142]: pgmap v1154: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:25.948 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:25 smithi144 ceph-mon[86635]: pgmap v1155: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:25.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:25 smithi049 ceph-mon[112142]: pgmap v1155: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:27.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:27 smithi049 ceph-mon[112142]: pgmap v1156: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:27 smithi144 ceph-mon[86635]: pgmap v1156: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:29.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:29 smithi049 ceph-mon[112142]: pgmap v1157: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:30.018 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:29 smithi144 ceph-mon[86635]: pgmap v1157: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:32.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:31 smithi144 ceph-mon[86635]: pgmap v1158: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:31 smithi049 ceph-mon[112142]: pgmap v1158: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:33 smithi144 ceph-mon[86635]: pgmap v1159: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:33 smithi049 ceph-mon[112142]: pgmap v1159: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:35 smithi144 ceph-mon[86635]: pgmap v1160: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:35:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:35 smithi049 ceph-mon[112142]: pgmap v1160: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:35:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:37 smithi144 ceph-mon[86635]: pgmap v1161: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:37 smithi049 ceph-mon[112142]: pgmap v1161: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:40.000 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:39 smithi144 ceph-mon[86635]: pgmap v1162: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:39 smithi049 ceph-mon[112142]: pgmap v1162: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:41 smithi144 ceph-mon[86635]: pgmap v1163: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:41 smithi049 ceph-mon[112142]: pgmap v1163: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:43 smithi144 ceph-mon[86635]: pgmap v1164: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:43 smithi049 ceph-mon[112142]: pgmap v1164: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:45 smithi144 ceph-mon[86635]: pgmap v1165: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:45 smithi049 ceph-mon[112142]: pgmap v1165: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:47.780 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph orch ps' 2024-05-15T16:35:48.008 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:35:48.130 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:47 smithi049 ceph-mon[112142]: pgmap v1166: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:47 smithi144 ceph-mon[86635]: pgmap v1166: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:49.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.650+0000 7eff741bd700 1 -- 172.21.15.49:0/110429447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 msgr2=0x7eff6c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:49.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.650+0000 7eff741bd700 1 --2- 172.21.15.49:0/110429447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 0x7eff6c1001a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7eff68009910 tx=0x7eff68009c20 comp rx=0 tx=0).stop 2024-05-15T16:35:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.650+0000 7eff741bd700 1 -- 172.21.15.49:0/110429447 shutdown_connections 2024-05-15T16:35:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.650+0000 7eff741bd700 1 --2- 172.21.15.49:0/110429447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6c100770 0x7eff6c1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.650+0000 7eff741bd700 1 --2- 172.21.15.49:0/110429447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 0x7eff6c1001a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.650+0000 7eff741bd700 1 -- 172.21.15.49:0/110429447 >> 172.21.15.49:0/110429447 conn(0x7eff6c076800 msgr2=0x7eff6c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:35:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.650+0000 7eff741bd700 1 -- 172.21.15.49:0/110429447 shutdown_connections 2024-05-15T16:35:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.650+0000 7eff741bd700 1 -- 172.21.15.49:0/110429447 wait complete. 2024-05-15T16:35:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.651+0000 7eff741bd700 1 Processor -- start 2024-05-15T16:35:49.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.651+0000 7eff741bd700 1 -- start start 2024-05-15T16:35:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.651+0000 7eff741bd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 0x7eff6c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.651+0000 7eff741bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6c100770 0x7eff6c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.651+0000 7eff741bd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c19d440 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.651+0000 7eff741bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c196a20 con 0x7eff6c100770 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.651+0000 7eff71f59700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 0x7eff6c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff71f59700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 0x7eff6c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36314/0 (socket says 172.21.15.49:36314) 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff71f59700 1 -- 172.21.15.49:0/744365976 learned_addr learned my addr 172.21.15.49:0/744365976 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff71758700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6c100770 0x7eff6c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff71f59700 1 -- 172.21.15.49:0/744365976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6c100770 msgr2=0x7eff6c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff71f59700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6c100770 0x7eff6c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff71f59700 1 -- 172.21.15.49:0/744365976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680095f0 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff71758700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6c100770 0x7eff6c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff71f59700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 0x7eff6c19c8e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7eff68006010 tx=0x7eff6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff62ffd700 1 -- 172.21.15.49:0/744365976 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff6801d070 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff62ffd700 1 -- 172.21.15.49:0/744365976 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff680056e0 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff62ffd700 1 -- 172.21.15.49:0/744365976 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff68017400 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c196ca0 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.652+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c197190 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.653+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c1012a0 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.654+0000 7eff62ffd700 1 -- 172.21.15.49:0/744365976 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7eff6800f860 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.654+0000 7eff62ffd700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff5806e200 0x7eff580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.654+0000 7eff71758700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff5806e200 0x7eff580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.654+0000 7eff62ffd700 1 -- 172.21.15.49:0/744365976 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7eff6808f490 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.655+0000 7eff71758700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff5806e200 0x7eff580706c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005d90 tx=0x7eff5c005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:35:49.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.659+0000 7eff62ffd700 1 -- 172.21.15.49:0/744365976 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7eff6805c3e0 con 0x7eff6c0ffdc0 2024-05-15T16:35:49.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.879+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 --> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7eff6c10cd80 con 0x7eff5806e200 2024-05-15T16:35:49.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.890+0000 7eff62ffd700 1 -- 172.21.15.49:0/744365976 <== mgr.34104 v2:172.21.15.49:6800/3324404093 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7eff6c10cd80 con 0x7eff5806e200 2024-05-15T16:35:49.890 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T16:35:49.890 INFO:teuthology.orchestra.run.smithi049.stdout:alertmanager.smithi049 smithi049 *:9093,9094 running (39m) 96s ago 50m 24.2M - 0.25.0 c8568f914cd2 64d15b758bec 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi049 smithi049 running (111s) 96s ago 50m 7486k - 19.0.0-2576-gde5210d5 48bc18eb22d8 ed7b98cce7c8 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-exporter.smithi144 smithi144 running (106s) 100s ago 49m 6971k - 19.0.0-2576-gde5210d5 48bc18eb22d8 d734ad348c4b 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi049 smithi049 running (36m) 96s ago 50m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 746a2e19ceeb 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:crash.smithi144 smithi144 running (36m) 100s ago 49m 6660k - 19.0.0-2576-gde5210d5 48bc18eb22d8 897e2fb02e84 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:grafana.smithi049 smithi049 *:3000 running (38m) 96s ago 50m 78.3M - 9.4.12 2bacad6d85d8 b43e345dbcaf 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.abbagx smithi049 running (32m) 96s ago 45m 165M - 19.0.0-2576-gde5210d5 48bc18eb22d8 526de94872bd 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi049.gocadf smithi049 running (32m) 96s ago 44m 25.3M - 19.0.0-2576-gde5210d5 48bc18eb22d8 2782d6f310ff 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.jgodzs smithi144 running (32m) 100s ago 45m 150M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ec0102b673d9 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:mds.cephfs.smithi144.kziuke smithi144 running (17m) 100s ago 44m 20.9M - 19.0.0-2576-gde5210d5 48bc18eb22d8 92ec31311a9e 2024-05-15T16:35:49.891 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi049.vuhzgf smithi049 *:8443,9283,8765 running (41m) 96s ago 52m 519M - 19.0.0-2576-gde5210d5 48bc18eb22d8 ce9c65c38c75 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.smithi144.whcjkr smithi144 *:8443,9283,8765 running (40m) 100s ago 49m 461M - 19.0.0-2576-gde5210d5 48bc18eb22d8 e63b4350a718 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi049 smithi049 running (37m) 96s ago 52m 87.2M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 05295c0417ca 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:mon.smithi144 smithi144 running (37m) 100s ago 49m 61.4M 2048M 19.0.0-2576-gde5210d5 48bc18eb22d8 2e8f6c238d31 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi049 smithi049 *:9100 running (39m) 96s ago 50m 15.5M - 1.5.0 0da6a335fe13 133e445096ac 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:node-exporter.smithi144 smithi144 *:9100 running (39m) 100s ago 49m 16.3M - 1.5.0 0da6a335fe13 76ae612354da 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (36m) 96s ago 48m 98.0M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 84a251f158d3 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi049 running (35m) 96s ago 48m 100M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 defc7d001185 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi049 running (34m) 96s ago 47m 96.3M 1216M 19.0.0-2576-gde5210d5 48bc18eb22d8 b2e567da235f 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi144 running (34m) 100s ago 47m 112M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 0a1e2d34dad4 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi144 running (33m) 100s ago 46m 95.2M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 d7e6dd64b07e 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi144 running (33m) 100s ago 46m 93.5M 2240M 19.0.0-2576-gde5210d5 48bc18eb22d8 eec3c5d0c5ee 2024-05-15T16:35:49.892 INFO:teuthology.orchestra.run.smithi049.stdout:prometheus.smithi049 smithi049 *:9095 running (39m) 96s ago 50m 87.1M - 2.43.0 a07b618ecd1d 863f0851bcb2 2024-05-15T16:35:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.892+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff5806e200 msgr2=0x7eff580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.892+0000 7eff741bd700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff5806e200 0x7eff580706c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005d90 tx=0x7eff5c005d00 comp rx=0 tx=0).stop 2024-05-15T16:35:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.892+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 msgr2=0x7eff6c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.892+0000 7eff741bd700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 0x7eff6c19c8e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7eff68006010 tx=0x7eff6800be20 comp rx=0 tx=0).stop 2024-05-15T16:35:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.892+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 shutdown_connections 2024-05-15T16:35:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.892+0000 7eff741bd700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7eff5806e200 0x7eff580706c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.892+0000 7eff741bd700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff6c0ffdc0 0x7eff6c19c8e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:49.893 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.892+0000 7eff741bd700 1 --2- 172.21.15.49:0/744365976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff6c100770 0x7eff6c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.892+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 >> 172.21.15.49:0/744365976 conn(0x7eff6c076800 msgr2=0x7eff6c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:35:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.893+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 shutdown_connections 2024-05-15T16:35:49.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:49.893+0000 7eff741bd700 1 -- 172.21.15.49:0/744365976 wait complete. 2024-05-15T16:35:50.191 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:49 smithi049 ceph-mon[112142]: pgmap v1167: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:49 smithi144 ceph-mon[86635]: pgmap v1167: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:35:50.639 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph versions' 2024-05-15T16:35:50.868 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:35:50.928 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:50 smithi049 ceph-mon[112142]: from='client.36048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:35:50.928 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:35:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:50 smithi144 ceph-mon[86635]: from='client.36048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T16:35:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:35:52.198 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:51 smithi049 ceph-mon[112142]: pgmap v1168: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:52.199 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:51 smithi144 ceph-mon[86635]: pgmap v1168: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:52.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 -- 172.21.15.49:0/2766333271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c101890 msgr2=0x7f490c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:52.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 --2- 172.21.15.49:0/2766333271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c101890 0x7f490c1058e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009910 tx=0x7f48f8009c20 comp rx=0 tx=0).stop 2024-05-15T16:35:52.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 -- 172.21.15.49:0/2766333271 shutdown_connections 2024-05-15T16:35:52.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 --2- 172.21.15.49:0/2766333271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c101890 0x7f490c1058e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 --2- 172.21.15.49:0/2766333271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c100ee0 0x7f490c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 -- 172.21.15.49:0/2766333271 >> 172.21.15.49:0/2766333271 conn(0x7f490c0fc790 msgr2=0x7f490c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 -- 172.21.15.49:0/2766333271 shutdown_connections 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 -- 172.21.15.49:0/2766333271 wait complete. 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 Processor -- start 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.432+0000 7f49130f8700 1 -- start start 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f49130f8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c100ee0 0x7f490c196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f49130f8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c101890 0x7f490c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f49130f8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c1978e0 con 0x7f490c100ee0 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f49130f8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c19a550 con 0x7f490c101890 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f4910e94700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c100ee0 0x7f490c196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f4910e94700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c100ee0 0x7f490c196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36340/0 (socket says 172.21.15.49:36340) 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f4910e94700 1 -- 172.21.15.49:0/523036634 learned_addr learned my addr 172.21.15.49:0/523036634 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f490bfff700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c101890 0x7f490c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f4910e94700 1 -- 172.21.15.49:0/523036634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c101890 msgr2=0x7f490c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f4910e94700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c101890 0x7f490c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f4910e94700 1 -- 172.21.15.49:0/523036634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80095f0 con 0x7f490c100ee0 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f4910e94700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c100ee0 0x7f490c196d10 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f490000b4b0 tx=0x7f490000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.433+0000 7f4909ffb700 1 -- 172.21.15.49:0/523036634 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4900011800 con 0x7f490c100ee0 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.434+0000 7f4909ffb700 1 -- 172.21.15.49:0/523036634 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4900011e20 con 0x7f490c100ee0 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.434+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c19a830 con 0x7f490c100ee0 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.434+0000 7f4909ffb700 1 -- 172.21.15.49:0/523036634 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490000f2e0 con 0x7f490c100ee0 2024-05-15T16:35:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.434+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c19ad50 con 0x7f490c100ee0 2024-05-15T16:35:52.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.435+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c109210 con 0x7f490c100ee0 2024-05-15T16:35:52.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.435+0000 7f4909ffb700 1 -- 172.21.15.49:0/523036634 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4900010380 con 0x7f490c100ee0 2024-05-15T16:35:52.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.436+0000 7f4909ffb700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f48fc06e200 0x7f48fc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:52.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.436+0000 7f4909ffb700 1 -- 172.21.15.49:0/523036634 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f490008dd30 con 0x7f490c100ee0 2024-05-15T16:35:52.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.436+0000 7f490bfff700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f48fc06e200 0x7f48fc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:52.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.436+0000 7f490bfff700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f48fc06e200 0x7f48fc0706c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f490c1982d0 tx=0x7f48f801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:35:52.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.441+0000 7f4909ffb700 1 -- 172.21.15.49:0/523036634 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4900056fb0 con 0x7f490c100ee0 2024-05-15T16:35:52.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.737+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f490c198080 con 0x7f490c100ee0 2024-05-15T16:35:52.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.737+0000 7f4909ffb700 1 -- 172.21.15.49:0/523036634 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f490005a670 con 0x7f490c100ee0 2024-05-15T16:35:52.738 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-15T16:35:52.738 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": { 2024-05-15T16:35:52.738 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:35:52.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": { 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 2 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": { 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 6 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 4 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: "overall": { 2024-05-15T16:35:52.739 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)": 14 2024-05-15T16:35:52.740 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-15T16:35:52.740 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-15T16:35:52.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.739+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f48fc06e200 msgr2=0x7f48fc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:52.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.739+0000 7f49130f8700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f48fc06e200 0x7f48fc0706c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f490c1982d0 tx=0x7f48f801a040 comp rx=0 tx=0).stop 2024-05-15T16:35:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.739+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c100ee0 msgr2=0x7f490c196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.739+0000 7f49130f8700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c100ee0 0x7f490c196d10 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f490000b4b0 tx=0x7f490000b7c0 comp rx=0 tx=0).stop 2024-05-15T16:35:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.739+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 shutdown_connections 2024-05-15T16:35:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.739+0000 7f49130f8700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f48fc06e200 0x7f48fc0706c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.739+0000 7f49130f8700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f490c100ee0 0x7f490c196d10 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.739+0000 7f49130f8700 1 --2- 172.21.15.49:0/523036634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c101890 0x7f490c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.739+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 >> 172.21.15.49:0/523036634 conn(0x7f490c0fc790 msgr2=0x7f490c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:35:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.740+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 shutdown_connections 2024-05-15T16:35:52.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:52.740+0000 7f49130f8700 1 -- 172.21.15.49:0/523036634 wait complete. 2024-05-15T16:35:53.064 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:52 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/523036634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:35:53.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:52 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/523036634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:35:53.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-05-15T16:35:53.605 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:35:53.979 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:53 smithi049 ceph-mon[112142]: pgmap v1169: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:53 smithi144 ceph-mon[86635]: pgmap v1169: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.299+0000 7f867a506700 1 -- 172.21.15.49:0/2532068109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86741007a0 msgr2=0x7f86741047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.299+0000 7f867a506700 1 --2- 172.21.15.49:0/2532068109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86741007a0 0x7f86741047f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8668009910 tx=0x7f8668009c20 comp rx=0 tx=0).stop 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.299+0000 7f867a506700 1 -- 172.21.15.49:0/2532068109 shutdown_connections 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.299+0000 7f867a506700 1 --2- 172.21.15.49:0/2532068109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86741007a0 0x7f86741047f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.299+0000 7f867a506700 1 --2- 172.21.15.49:0/2532068109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86740ffdf0 0x7f86741001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.299+0000 7f867a506700 1 -- 172.21.15.49:0/2532068109 >> 172.21.15.49:0/2532068109 conn(0x7f86740fb6a0 msgr2=0x7f86740fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.299+0000 7f867a506700 1 -- 172.21.15.49:0/2532068109 shutdown_connections 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.299+0000 7f867a506700 1 -- 172.21.15.49:0/2532068109 wait complete. 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f867a506700 1 Processor -- start 2024-05-15T16:35:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f867a506700 1 -- start start 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f867a506700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86740ffdf0 0x7f867419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f867a506700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741007a0 0x7f867419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f867a506700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8674195a70 con 0x7f86740ffdf0 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f867a506700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8674195be0 con 0x7f86741007a0 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f8673fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86740ffdf0 0x7f867419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f8673fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86740ffdf0 0x7f867419b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51926/0 (socket says 172.21.15.49:51926) 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f8673fff700 1 -- 172.21.15.49:0/2691762620 learned_addr learned my addr 172.21.15.49:0/2691762620 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.300+0000 7f86737fe700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741007a0 0x7f867419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.301+0000 7f8673fff700 1 -- 172.21.15.49:0/2691762620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741007a0 msgr2=0x7f867419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.301+0000 7f8673fff700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741007a0 0x7f867419bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.301+0000 7f8673fff700 1 -- 172.21.15.49:0/2691762620 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86680095f0 con 0x7f86740ffdf0 2024-05-15T16:35:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.301+0000 7f8673fff700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86740ffdf0 0x7f867419b9c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f866400d900 tx=0x7f866400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.301+0000 7f86717fa700 1 -- 172.21.15.49:0/2691762620 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86640041f0 con 0x7f86740ffdf0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.301+0000 7f86717fa700 1 -- 172.21.15.49:0/2691762620 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8664004cf0 con 0x7f86740ffdf0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.301+0000 7f86717fa700 1 -- 172.21.15.49:0/2691762620 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f866400b750 con 0x7f86740ffdf0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.301+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8674195ec0 con 0x7f86740ffdf0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.301+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86741963e0 con 0x7f86740ffdf0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.302+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86741081f0 con 0x7f86740ffdf0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.303+0000 7f86717fa700 1 -- 172.21.15.49:0/2691762620 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8664004750 con 0x7f86740ffdf0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.303+0000 7f86717fa700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865c06e200 0x7f865c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.303+0000 7f86717fa700 1 -- 172.21.15.49:0/2691762620 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f866408e070 con 0x7f86740ffdf0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.303+0000 7f86737fe700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865c06e200 0x7f865c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:55.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.304+0000 7f86737fe700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865c06e200 0x7f865c0706c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8674197270 tx=0x7f866801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:35:55.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.308+0000 7f86717fa700 1 -- 172.21.15.49:0/2691762620 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8664057350 con 0x7f86740ffdf0 2024-05-15T16:35:55.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.605+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f867404ecc0 con 0x7f86740ffdf0 2024-05-15T16:35:55.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.606+0000 7f86717fa700 1 -- 172.21.15.49:0/2691762620 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f866405aa10 con 0x7f86740ffdf0 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865c06e200 msgr2=0x7f865c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865c06e200 0x7f865c0706c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8674197270 tx=0x7f866801a040 comp rx=0 tx=0).stop 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86740ffdf0 msgr2=0x7f867419b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86740ffdf0 0x7f867419b9c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f866400d900 tx=0x7f866400dc10 comp rx=0 tx=0).stop 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 shutdown_connections 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f865c06e200 0x7f865c0706c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86740ffdf0 0x7f867419b9c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 --2- 172.21.15.49:0/2691762620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741007a0 0x7f867419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 >> 172.21.15.49:0/2691762620 conn(0x7f86740fb6a0 msgr2=0x7f86740fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 shutdown_connections 2024-05-15T16:35:55.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:55.608+0000 7f867a506700 1 -- 172.21.15.49:0/2691762620 wait complete. 2024-05-15T16:35:55.619 INFO:teuthology.orchestra.run.smithi049.stdout:true 2024-05-15T16:35:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:55 smithi144 ceph-mon[86635]: pgmap v1170: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:56.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:55 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2691762620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:35:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:55 smithi049 ceph-mon[112142]: pgmap v1170: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:35:56.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:55 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2691762620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:35:56.346 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-05-15T16:35:56.560 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:35:57.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:57 smithi049 ceph-mon[112142]: pgmap v1171: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:58.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:57 smithi144 ceph-mon[86635]: pgmap v1171: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:35:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.248+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/802681832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 msgr2=0x7f0c481058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.248+0000 7f0c4cbbf700 1 --2- 172.21.15.49:0/802681832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 0x7f0c481058e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009910 tx=0x7f0c30009c20 comp rx=0 tx=0).stop 2024-05-15T16:35:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.249+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/802681832 shutdown_connections 2024-05-15T16:35:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.249+0000 7f0c4cbbf700 1 --2- 172.21.15.49:0/802681832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 0x7f0c481058e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:58.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.249+0000 7f0c4cbbf700 1 --2- 172.21.15.49:0/802681832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c48100ee0 0x7f0c481012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.249+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/802681832 >> 172.21.15.49:0/802681832 conn(0x7f0c480fc790 msgr2=0x7f0c480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:35:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.249+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/802681832 shutdown_connections 2024-05-15T16:35:58.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.249+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/802681832 wait complete. 2024-05-15T16:35:58.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.249+0000 7f0c4cbbf700 1 Processor -- start 2024-05-15T16:35:58.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c4cbbf700 1 -- start start 2024-05-15T16:35:58.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c4cbbf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c48100ee0 0x7f0c480732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c4cbbf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 0x7f0c480737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c4cbbf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48073d20 con 0x7f0c48101890 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c4cbbf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48073e60 con 0x7f0c48100ee0 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c3dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 0x7f0c480737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c3dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 0x7f0c480737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51954/0 (socket says 172.21.15.49:51954) 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c3dd9b700 1 -- 172.21.15.49:0/2049514799 learned_addr learned my addr 172.21.15.49:0/2049514799 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c4659c700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c48100ee0 0x7f0c480732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c3dd9b700 1 -- 172.21.15.49:0/2049514799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c48100ee0 msgr2=0x7f0c480732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c3dd9b700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c48100ee0 0x7f0c480732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c3dd9b700 1 -- 172.21.15.49:0/2049514799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c300095f0 con 0x7f0c48101890 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.250+0000 7f0c3dd9b700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 0x7f0c480737e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0c300092e0 tx=0x7f0c3000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.251+0000 7f0c3ffff700 1 -- 172.21.15.49:0/2049514799 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c3001d070 con 0x7f0c48101890 2024-05-15T16:35:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.251+0000 7f0c3ffff700 1 -- 172.21.15.49:0/2049514799 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c30005700 con 0x7f0c48101890 2024-05-15T16:35:58.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.251+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c480740e0 con 0x7f0c48101890 2024-05-15T16:35:58.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.251+0000 7f0c3ffff700 1 -- 172.21.15.49:0/2049514799 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c3000fdb0 con 0x7f0c48101890 2024-05-15T16:35:58.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.251+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c481a3020 con 0x7f0c48101890 2024-05-15T16:35:58.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.252+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c481092e0 con 0x7f0c48101890 2024-05-15T16:35:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.252+0000 7f0c3ffff700 1 -- 172.21.15.49:0/2049514799 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0c30017450 con 0x7f0c48101890 2024-05-15T16:35:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.253+0000 7f0c3ffff700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c3406e200 0x7f0c340706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:35:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.253+0000 7f0c3ffff700 1 -- 172.21.15.49:0/2049514799 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0c3008f4b0 con 0x7f0c48101890 2024-05-15T16:35:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.253+0000 7f0c4659c700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c3406e200 0x7f0c340706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:35:58.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.253+0000 7f0c4659c700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c3406e200 0x7f0c340706c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0c38007be0 tx=0x7f0c380073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:35:58.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.258+0000 7f0c3ffff700 1 -- 172.21.15.49:0/2049514799 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0c30058790 con 0x7f0c48101890 2024-05-15T16:35:58.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.553+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c481023c0 con 0x7f0c48101890 2024-05-15T16:35:58.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.554+0000 7f0c3ffff700 1 -- 172.21.15.49:0/2049514799 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0c3005be50 con 0x7f0c48101890 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c3406e200 msgr2=0x7f0c340706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c3406e200 0x7f0c340706c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0c38007be0 tx=0x7f0c380073d0 comp rx=0 tx=0).stop 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 msgr2=0x7f0c480737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 0x7f0c480737e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0c300092e0 tx=0x7f0c3000bd70 comp rx=0 tx=0).stop 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 shutdown_connections 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f0c3406e200 0x7f0c340706c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c48100ee0 0x7f0c480732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 --2- 172.21.15.49:0/2049514799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c48101890 0x7f0c480737e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 >> 172.21.15.49:0/2049514799 conn(0x7f0c480fc790 msgr2=0x7f0c480fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 shutdown_connections 2024-05-15T16:35:58.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:35:58.556+0000 7f0c4cbbf700 1 -- 172.21.15.49:0/2049514799 wait complete. 2024-05-15T16:35:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph version 19.0.0-2576-gde5210d5 (de5210d58560a4684fdb5b238ef3e6e0285a1de8) squid (dev)" 2024-05-15T16:35:59.164 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:58 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2049514799' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:35:59.188 DEBUG:teuthology.parallel:result is None 2024-05-15T16:35:59.188 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T16:35:59.200 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T16:35:59.200 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c619ac44-12d1-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-15T16:35:59.201 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:58 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2049514799' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-15T16:35:59.418 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/mon.smithi049/config 2024-05-15T16:36:00.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:35:59 smithi144 ceph-mon[86635]: pgmap v1172: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:00.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:35:59 smithi049 ceph-mon[112142]: pgmap v1172: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:01.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.040+0000 7f50ff68e700 1 -- 172.21.15.49:0/2301871500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f80ffdc0 msgr2=0x7f50f81001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.040+0000 7f50ff68e700 1 --2- 172.21.15.49:0/2301871500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f80ffdc0 0x7f50f81001a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009910 tx=0x7f50ec009c20 comp rx=0 tx=0).stop 2024-05-15T16:36:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.040+0000 7f50ff68e700 1 -- 172.21.15.49:0/2301871500 shutdown_connections 2024-05-15T16:36:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.040+0000 7f50ff68e700 1 --2- 172.21.15.49:0/2301871500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f8100770 0x7f50f81047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.040+0000 7f50ff68e700 1 --2- 172.21.15.49:0/2301871500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f80ffdc0 0x7f50f81001a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.040+0000 7f50ff68e700 1 -- 172.21.15.49:0/2301871500 >> 172.21.15.49:0/2301871500 conn(0x7f50f8076800 msgr2=0x7f50f80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.040+0000 7f50ff68e700 1 -- 172.21.15.49:0/2301871500 shutdown_connections 2024-05-15T16:36:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.040+0000 7f50ff68e700 1 -- 172.21.15.49:0/2301871500 wait complete. 2024-05-15T16:36:01.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50ff68e700 1 Processor -- start 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50ff68e700 1 -- start start 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50ff68e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f80ffdc0 0x7f50f819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50ff68e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f8100770 0x7f50f819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50ff68e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f819d540 con 0x7f50f8100770 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50ff68e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8196b20 con 0x7f50f80ffdc0 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50fcc29700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f8100770 0x7f50f819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50fcc29700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f8100770 0x7f50f819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51980/0 (socket says 172.21.15.49:51980) 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50fcc29700 1 -- 172.21.15.49:0/305476249 learned_addr learned my addr 172.21.15.49:0/305476249 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.041+0000 7f50fd42a700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f80ffdc0 0x7f50f819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.042+0000 7f50fcc29700 1 -- 172.21.15.49:0/305476249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f80ffdc0 msgr2=0x7f50f819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.042+0000 7f50fcc29700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f80ffdc0 0x7f50f819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.042+0000 7f50fcc29700 1 -- 172.21.15.49:0/305476249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec0095f0 con 0x7f50f8100770 2024-05-15T16:36:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.042+0000 7f50fcc29700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f8100770 0x7f50f819cf20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b480 tx=0x7f50f400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.042+0000 7f50ea7fc700 1 -- 172.21.15.49:0/305476249 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50f4011800 con 0x7f50f8100770 2024-05-15T16:36:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.042+0000 7f50ea7fc700 1 -- 172.21.15.49:0/305476249 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50f4011e20 con 0x7f50f8100770 2024-05-15T16:36:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.042+0000 7f50ea7fc700 1 -- 172.21.15.49:0/305476249 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50f4010ea0 con 0x7f50f8100770 2024-05-15T16:36:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.042+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f8196e00 con 0x7f50f8100770 2024-05-15T16:36:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.042+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f8197350 con 0x7f50f8100770 2024-05-15T16:36:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.043+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f81012a0 con 0x7f50f8100770 2024-05-15T16:36:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.044+0000 7f50ea7fc700 1 -- 172.21.15.49:0/305476249 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f50f4010380 con 0x7f50f8100770 2024-05-15T16:36:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.044+0000 7f50ea7fc700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f50e406e200 0x7f50e40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.044+0000 7f50ea7fc700 1 -- 172.21.15.49:0/305476249 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f50f408dbf0 con 0x7f50f8100770 2024-05-15T16:36:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.044+0000 7f50fd42a700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f50e406e200 0x7f50e40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.045+0000 7f50fd42a700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f50e406e200 0x7f50e40706c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f50ec006010 tx=0x7f50ec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:01.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.049+0000 7f50ea7fc700 1 -- 172.21.15.49:0/305476249 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f50f405aa80 con 0x7f50f8100770 2024-05-15T16:36:01.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.299+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f50f804ec60 con 0x7f50f8100770 2024-05-15T16:36:01.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.299+0000 7f50ea7fc700 1 -- 172.21.15.49:0/305476249 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1966 (secure 0 0 0) 0x7f50f40157c0 con 0x7f50f8100770 2024-05-15T16:36:01.301 INFO:teuthology.orchestra.run.smithi049.stdout:e36 2024-05-15T16:36:01.301 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T16:36:01.301 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-15T16:36:01.301 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-05-15T16:36:01.301 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 36 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-15T15:50:40.850889+0000 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-15T16:34:18.682149+0000 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 60 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-05-15T16:36:01.302 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 83 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 2 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=34420,1=34424} 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3] 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data enabled 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 1 2024-05-15T16:36:01.303 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 34420 members: 34420,34424 2024-05-15T16:36:01.304 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.abbagx{0:34420} state up:active seq 468 join_fscid=1 addr [v2:172.21.15.49:6826/683477188,v1:172.21.15.49:6827/683477188] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:36:01.304 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.jgodzs{0:44339} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6824/2706427362,v1:172.21.15.144:6825/2706427362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:36:01.304 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi049.gocadf{1:34424} state up:active seq 463 join_fscid=1 addr [v2:172.21.15.49:6828/2031688662,v1:172.21.15.49:6829/2031688662] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:36:01.304 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.cephfs.smithi144.kziuke{1:44891} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/755343004,v1:172.21.15.144:6827/755343004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T16:36:01.304 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:01.304 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.302+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f50e406e200 msgr2=0x7f50e40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.302+0000 7f50ff68e700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f50e406e200 0x7f50e40706c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f50ec006010 tx=0x7f50ec01a040 comp rx=0 tx=0).stop 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.302+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f8100770 msgr2=0x7f50f819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.302+0000 7f50ff68e700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f8100770 0x7f50f819cf20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b480 tx=0x7f50f400b790 comp rx=0 tx=0).stop 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.302+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 shutdown_connections 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.302+0000 7f50ff68e700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f50e406e200 0x7f50e40706c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.302+0000 7f50ff68e700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f80ffdc0 0x7f50f819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.303+0000 7f50ff68e700 1 --2- 172.21.15.49:0/305476249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50f8100770 0x7f50f819cf20 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.303+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 >> 172.21.15.49:0/305476249 conn(0x7f50f8076800 msgr2=0x7f50f80fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:01.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.303+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 shutdown_connections 2024-05-15T16:36:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:01.303+0000 7f50ff68e700 1 -- 172.21.15.49:0/305476249 wait complete. 2024-05-15T16:36:01.306 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 36 2024-05-15T16:36:01.947 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:01 smithi049 ceph-mon[112142]: pgmap v1173: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:01.948 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:01 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/305476249' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:36:01.971 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-05-15T16:36:01.981 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.090+0000 7f133e37d640 1 -- 172.21.15.49:0/2389745239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 msgr2=0x7f13380ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.090+0000 7f133e37d640 1 --2- 172.21.15.49:0/2389745239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 0x7f13380ff690 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1328009880 tx=0x7f132802bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.090+0000 7f133e37d640 1 -- 172.21.15.49:0/2389745239 shutdown_connections 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.090+0000 7f133e37d640 1 --2- 172.21.15.49:0/2389745239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13380ffc60 0x7f1338103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.090+0000 7f133e37d640 1 --2- 172.21.15.49:0/2389745239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 0x7f13380ff690 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.090+0000 7f133e37d640 1 -- 172.21.15.49:0/2389745239 >> 172.21.15.49:0/2389745239 conn(0x7f1338075cf0 msgr2=0x7f13380fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.090+0000 7f133e37d640 1 -- 172.21.15.49:0/2389745239 shutdown_connections 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.090+0000 7f133e37d640 1 -- 172.21.15.49:0/2389745239 wait complete. 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f133e37d640 1 Processor -- start 2024-05-15T16:36:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f133e37d640 1 -- start start 2024-05-15T16:36:02.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f133e37d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 0x7f1338196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:02.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f133e37d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13381967a0 0x7f133819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:02.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f133e37d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1338196da0 con 0x7f13380ff2b0 2024-05-15T16:36:02.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f133e37d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1338196f10 con 0x7f13381967a0 2024-05-15T16:36:02.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f1337fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 0x7f1338196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f1337fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 0x7f1338196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51988/0 (socket says 172.21.15.49:51988) 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f1337fff640 1 -- 172.21.15.49:0/25395537 learned_addr learned my addr 172.21.15.49:0/25395537 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f1337fff640 1 -- 172.21.15.49:0/25395537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13381967a0 msgr2=0x7f133819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.091+0000 7f13377fe640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13381967a0 0x7f133819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.092+0000 7f1337fff640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13381967a0 0x7f133819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.092+0000 7f1337fff640 1 -- 172.21.15.49:0/25395537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1328009560 con 0x7f13380ff2b0 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.092+0000 7f13377fe640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13381967a0 0x7f133819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.092+0000 7f1337fff640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 0x7f1338196260 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1328036040 tx=0x7f1328004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.092+0000 7f13357fa640 1 -- 172.21.15.49:0/25395537 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13280035f0 con 0x7f13380ff2b0 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.092+0000 7f13357fa640 1 -- 172.21.15.49:0/25395537 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1328003c10 con 0x7f13380ff2b0 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.092+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133819b0e0 con 0x7f13380ff2b0 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.092+0000 7f13357fa640 1 -- 172.21.15.49:0/25395537 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1328034440 con 0x7f13380ff2b0 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.092+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133819b640 con 0x7f13380ff2b0 2024-05-15T16:36:02.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.093+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1338100780 con 0x7f13380ff2b0 2024-05-15T16:36:02.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.094+0000 7f13357fa640 1 -- 172.21.15.49:0/25395537 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1328004f60 con 0x7f13380ff2b0 2024-05-15T16:36:02.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.094+0000 7f13357fa640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f132006e170 0x7f1320070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:02.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.094+0000 7f13357fa640 1 -- 172.21.15.49:0/25395537 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f13280b0590 con 0x7f13380ff2b0 2024-05-15T16:36:02.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.094+0000 7f13377fe640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f132006e170 0x7f1320070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:02.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.095+0000 7f13377fe640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f132006e170 0x7f1320070630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f132c006fd0 tx=0x7f132c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:02.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.099+0000 7f13357fa640 1 -- 172.21.15.49:0/25395537 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1328079870 con 0x7f13380ff2b0 2024-05-15T16:36:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:01 smithi144 ceph-mon[86635]: pgmap v1173: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:02.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:01 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/305476249' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T16:36:02.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.285+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f133819b980 con 0x7f13380ff2b0 2024-05-15T16:36:02.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.286+0000 7f13357fa640 1 -- 172.21.15.49:0/25395537 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16768 (secure 0 0 0) 0x7f132807cf30 con 0x7f13380ff2b0 2024-05-15T16:36:02.287 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:02.288 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":87,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","created":"2024-05-15T15:43:35.648208+0000","modified":"2024-05-15T16:18:46.696180+0000","last_up_change":"2024-05-15T16:02:52.176866+0000","last_in_change":"2024-05-15T15:49:04.722203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T15:48:09.899234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-15T15:50:39.462287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-15T15:50:40.285269+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25d1eb06-bee2-48a6-8d86-30b5b3b348d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6803","nonce":1701263964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6805","nonce":1701263964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6809","nonce":1701263964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6807","nonce":1701263964}]},"public_addr":"172.21.15.49:6803/1701263964","cluster_addr":"172.21.15.49:6805/1701263964","heartbeat_back_addr":"172.21.15.49:6809/1701263964","heartbeat_front_addr":"172.21.15.49:6807/1701263964","state":["exists","up"]},{"osd":1,"uuid":"c0e717cf-ae5c-4fa4-a30d-d2235ff27315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6811","nonce":3720875621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6813","nonce":3720875621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6817","nonce":3720875621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6815","nonce":3720875621}]},"public_addr":"172.21.15.49:6811/3720875621","cluster_addr":"172.21.15.49:6813/3720875621","heartbeat_back_addr":"172.21.15.49:6817/3720875621","heartbeat_front_addr":"172.21.15.49:6815/3720875621","state":["exists","up"]},{"osd":2,"uuid":"d7752daf-dd5b-49cc-abc2-ca953968e542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6819","nonce":3306250339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6821","nonce":3306250339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6825","nonce":3306250339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6823","nonce":3306250339}]},"public_addr":"172.21.15.49:6819/3306250339","cluster_addr":"172.21.15.49:6821/3306250339","heartbeat_back_addr":"172.21.15.49:6825/3306250339","heartbeat_front_addr":"172.21.15.49:6823/3306250339","state":["exists","up"]},{"osd":3,"uuid":"2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6801","nonce":4258967767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6803","nonce":4258967767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6807","nonce":4258967767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6805","nonce":4258967767}]},"public_addr":"172.21.15.144:6801/4258967767","cluster_addr":"172.21.15.144:6803/4258967767","heartbeat_back_addr":"172.21.15.144:6807/4258967767","heartbeat_front_addr":"172.21.15.144:6805/4258967767","state":["exists","up"]},{"osd":4,"uuid":"6659631f-da39-4a2c-9f75-e7b4a06b602f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6809","nonce":691212832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6811","nonce":691212832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6815","nonce":691212832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6813","nonce":691212832}]},"public_addr":"172.21.15.144:6809/691212832","cluster_addr":"172.21.15.144:6811/691212832","heartbeat_back_addr":"172.21.15.144:6815/691212832","heartbeat_front_addr":"172.21.15.144:6813/691212832","state":["exists","up"]},{"osd":5,"uuid":"51ea2d85-6cf7-4a96-8c74-612958083a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6817","nonce":255063313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6819","nonce":255063313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6823","nonce":255063313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6821","nonce":255063313}]},"public_addr":"172.21.15.144:6817/255063313","cluster_addr":"172.21.15.144:6819/255063313","heartbeat_back_addr":"172.21.15.144:6823/255063313","heartbeat_front_addr":"172.21.15.144:6821/255063313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:10.108689+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:37.458152+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:05.915664+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:30.194669+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:55.748301+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:49:22.635948+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6828/2512292397":"2024-05-16T16:02:56.077800+0000","172.21.15.144:6825/2552333341":"2024-05-16T16:02:56.077816+0000","172.21.15.49:6801/1448243308":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/75769186":"2024-05-16T15:44:39.328329+0000","172.21.15.144:0/2391504982":"2024-05-16T15:54:47.414074+0000","172.21.15.49:0/287650046":"2024-05-16T15:58:20.089143+0000","172.21.15.49:6801/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/644218315":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/1515356048":"2024-05-16T15:58:20.089143+0000","172.21.15.49:6800/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.144:0/2959282915":"2024-05-16T15:54:47.414074+0000","172.21.15.144:6828/177983287":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6800/1448243308":"2024-05-16T15:53:24.658899+0000","172.21.15.144:6829/177983287":"2024-05-16T15:54:47.414074+0000","172.21.15.49:0/1173274751":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/4024404689":"2024-05-16T15:44:07.361150+0000","172.21.15.144:0/3162546694":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6826/2475999224":"2024-05-16T15:50:56.872336+0000","172.21.15.49:0/3014161738":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6800/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/1166053004":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/2163344179":"2024-05-16T15:44:07.361150+0000","172.21.15.144:0/2251902137":"2024-05-16T15:54:47.414074+0000","172.21.15.49:0/2464608729":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/4010845594":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6800/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/1904525014":"2024-05-16T15:53:24.658899+0000","172.21.15.49:6801/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.144:6824/2552333341":"2024-05-16T16:02:56.077816+0000","172.21.15.144:0/4014499308":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6801/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/2543812499":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/3452717560":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6827/2475999224":"2024-05-16T15:50:56.872336+0000","172.21.15.49:6829/2512292397":"2024-05-16T16:02:56.077800+0000","172.21.15.49:0/940476764":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/133547858":"2024-05-16T15:58:20.089143+0000","172.21.15.49:0/1530195241":"2024-05-16T15:58:20.089143+0000","172.21.15.49:6800/70490983":"2024-05-16T15:58:20.089143+0000","172.21.15.144:0/1791319417":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6801/70490983":"2024-05-16T15:58:20.089143+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T16:36:02.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f132006e170 msgr2=0x7f1320070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f132006e170 0x7f1320070630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f132c006fd0 tx=0x7f132c008040 comp rx=0 tx=0).stop 2024-05-15T16:36:02.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 msgr2=0x7f1338196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 0x7f1338196260 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1328036040 tx=0x7f1328004040 comp rx=0 tx=0).stop 2024-05-15T16:36:02.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 shutdown_connections 2024-05-15T16:36:02.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f132006e170 0x7f1320070630 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13381967a0 0x7f133819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 --2- 172.21.15.49:0/25395537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13380ff2b0 0x7f1338196260 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 >> 172.21.15.49:0/25395537 conn(0x7f1338075cf0 msgr2=0x7f1338067790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:02.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 shutdown_connections 2024-05-15T16:36:02.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.288+0000 7f133e37d640 1 -- 172.21.15.49:0/25395537 wait complete. 2024-05-15T16:36:02.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-15T15:48:09.899234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-15T15:50:39.462287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-15T15:50:40.285269+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-15T16:36:02.301 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-15T16:36:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.407+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2190776767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 msgr2=0x7f94f8101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.407+0000 7f94fc9d7640 1 --2- 172.21.15.49:0/2190776767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 0x7f94f8101710 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009880 tx=0x7f94e802bd70 comp rx=0 tx=0).stop 2024-05-15T16:36:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.408+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2190776767 shutdown_connections 2024-05-15T16:36:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.408+0000 7f94fc9d7640 1 --2- 172.21.15.49:0/2190776767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f8101c50 0x7f94f810a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.408+0000 7f94fc9d7640 1 --2- 172.21.15.49:0/2190776767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 0x7f94f8101710 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.408+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2190776767 >> 172.21.15.49:0/2190776767 conn(0x7f94f80fab70 msgr2=0x7f94f80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.408+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2190776767 shutdown_connections 2024-05-15T16:36:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.408+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2190776767 wait complete. 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94fc9d7640 1 Processor -- start 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94fc9d7640 1 -- start start 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94fc9d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 0x7f94f81961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94fc9d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f8101c50 0x7f94f8196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94fc9d7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f8196d90 con 0x7f94f8101330 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94fc9d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f819aaa0 con 0x7f94f8101c50 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94f659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 0x7f94f81961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94f659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 0x7f94f81961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52010/0 (socket says 172.21.15.49:52010) 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94f659c640 1 -- 172.21.15.49:0/2905107948 learned_addr learned my addr 172.21.15.49:0/2905107948 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:02.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94f5d9b640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f8101c50 0x7f94f8196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94f659c640 1 -- 172.21.15.49:0/2905107948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f8101c50 msgr2=0x7f94f8196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94f659c640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f8101c50 0x7f94f8196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.409+0000 7f94f659c640 1 -- 172.21.15.49:0/2905107948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e8009560 con 0x7f94f8101330 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.410+0000 7f94f659c640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 0x7f94f81961c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f94e802c2b0 tx=0x7f94e802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.410+0000 7f94e77fe640 1 -- 172.21.15.49:0/2905107948 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e8004520 con 0x7f94f8101330 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.410+0000 7f94e77fe640 1 -- 172.21.15.49:0/2905107948 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94e8003ab0 con 0x7f94f8101330 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.410+0000 7f94e77fe640 1 -- 172.21.15.49:0/2905107948 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e8005500 con 0x7f94f8101330 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.410+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f819ad20 con 0x7f94f8101330 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.410+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f819b280 con 0x7f94f8101330 2024-05-15T16:36:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.411+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f8068ed0 con 0x7f94f8101330 2024-05-15T16:36:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.411+0000 7f94e77fe640 1 -- 172.21.15.49:0/2905107948 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f94e8003c20 con 0x7f94f8101330 2024-05-15T16:36:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.412+0000 7f94e77fe640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f94e006e180 0x7f94e0070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.412+0000 7f94e77fe640 1 -- 172.21.15.49:0/2905107948 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f94e80b0b90 con 0x7f94f8101330 2024-05-15T16:36:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.412+0000 7f94f5d9b640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f94e006e180 0x7f94e0070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.412+0000 7f94f5d9b640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f94e006e180 0x7f94e0070640 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f94f8197710 tx=0x7f94ec00b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:02.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.417+0000 7f94e77fe640 1 -- 172.21.15.49:0/2905107948 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f94e8079f90 con 0x7f94f8101330 2024-05-15T16:36:02.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.600+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94f8104940 con 0x7f94f8101330 2024-05-15T16:36:02.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.600+0000 7f94e77fe640 1 -- 172.21.15.49:0/2905107948 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94e8043020 con 0x7f94f8101330 2024-05-15T16:36:02.602 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-05-15T16:36:02.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f94e006e180 msgr2=0x7f94e0070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f94e006e180 0x7f94e0070640 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f94f8197710 tx=0x7f94ec00b390 comp rx=0 tx=0).stop 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 msgr2=0x7f94f81961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 0x7f94f81961c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f94e802c2b0 tx=0x7f94e802c850 comp rx=0 tx=0).stop 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 shutdown_connections 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f94e006e180 0x7f94e0070640 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f8101c50 0x7f94f8196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 --2- 172.21.15.49:0/2905107948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94f8101330 0x7f94f81961c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 >> 172.21.15.49:0/2905107948 conn(0x7f94f80fab70 msgr2=0x7f94f81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 shutdown_connections 2024-05-15T16:36:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.602+0000 7f94fc9d7640 1 -- 172.21.15.49:0/2905107948 wait complete. 2024-05-15T16:36:02.614 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-15T16:36:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.720+0000 7efcbe955640 1 -- 172.21.15.49:0/340926781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 msgr2=0x7efcb81016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.720+0000 7efcbe955640 1 --2- 172.21.15.49:0/340926781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 0x7efcb81016d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7efca8009880 tx=0x7efca802bd70 comp rx=0 tx=0).stop 2024-05-15T16:36:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.720+0000 7efcbe955640 1 -- 172.21.15.49:0/340926781 shutdown_connections 2024-05-15T16:36:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.720+0000 7efcbe955640 1 --2- 172.21.15.49:0/340926781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcb8101c10 0x7efcb810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.720+0000 7efcbe955640 1 --2- 172.21.15.49:0/340926781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 0x7efcb81016d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.720+0000 7efcbe955640 1 -- 172.21.15.49:0/340926781 >> 172.21.15.49:0/340926781 conn(0x7efcb80fab10 msgr2=0x7efcb80fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.720+0000 7efcbe955640 1 -- 172.21.15.49:0/340926781 shutdown_connections 2024-05-15T16:36:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.720+0000 7efcbe955640 1 -- 172.21.15.49:0/340926781 wait complete. 2024-05-15T16:36:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcbe955640 1 Processor -- start 2024-05-15T16:36:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcbe955640 1 -- start start 2024-05-15T16:36:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcbe955640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 0x7efcb81961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcbe955640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcb8101c10 0x7efcb81966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcbe955640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8196d80 con 0x7efcb81012f0 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcbe955640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb819aa90 con 0x7efcb8101c10 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcb7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 0x7efcb81961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcb7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 0x7efcb81961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52022/0 (socket says 172.21.15.49:52022) 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcb7fff640 1 -- 172.21.15.49:0/745648290 learned_addr learned my addr 172.21.15.49:0/745648290 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.721+0000 7efcb77fe640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcb8101c10 0x7efcb81966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.722+0000 7efcb7fff640 1 -- 172.21.15.49:0/745648290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcb8101c10 msgr2=0x7efcb81966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.722+0000 7efcb7fff640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcb8101c10 0x7efcb81966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.722+0000 7efcb7fff640 1 -- 172.21.15.49:0/745648290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca8009560 con 0x7efcb81012f0 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.722+0000 7efcb7fff640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 0x7efcb81961b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7efca802c2b0 tx=0x7efca802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.722+0000 7efcb57fa640 1 -- 172.21.15.49:0/745648290 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efca80035f0 con 0x7efcb81012f0 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.722+0000 7efcb57fa640 1 -- 172.21.15.49:0/745648290 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efca8003c10 con 0x7efcb81012f0 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.722+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb819ad10 con 0x7efcb81012f0 2024-05-15T16:36:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.722+0000 7efcb57fa640 1 -- 172.21.15.49:0/745648290 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efca8005690 con 0x7efcb81012f0 2024-05-15T16:36:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.722+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb819b270 con 0x7efcb81012f0 2024-05-15T16:36:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.723+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb8068f10 con 0x7efcb81012f0 2024-05-15T16:36:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.724+0000 7efcb57fa640 1 -- 172.21.15.49:0/745648290 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efca8004ed0 con 0x7efcb81012f0 2024-05-15T16:36:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.724+0000 7efcb57fa640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efca006e120 0x7efca00705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.724+0000 7efcb57fa640 1 -- 172.21.15.49:0/745648290 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efca80b0c90 con 0x7efcb81012f0 2024-05-15T16:36:02.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.724+0000 7efcb77fe640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efca006e120 0x7efca00705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:02.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.725+0000 7efcb77fe640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efca006e120 0x7efca00705e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efcb8197700 tx=0x7efcac005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:02.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.729+0000 7efcb57fa640 1 -- 172.21.15.49:0/745648290 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efca807a090 con 0x7efcb81012f0 2024-05-15T16:36:02.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.915+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7efcb804e210 con 0x7efcb81012f0 2024-05-15T16:36:02.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.916+0000 7efcb57fa640 1 -- 172.21.15.49:0/745648290 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7efca80430b0 con 0x7efcb81012f0 2024-05-15T16:36:02.917 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-15T16:36:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.917+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efca006e120 msgr2=0x7efca00705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.917+0000 7efcbe955640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efca006e120 0x7efca00705e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efcb8197700 tx=0x7efcac005f50 comp rx=0 tx=0).stop 2024-05-15T16:36:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.917+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 msgr2=0x7efcb81961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.917+0000 7efcbe955640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 0x7efcb81961b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7efca802c2b0 tx=0x7efca802c8e0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.918+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 shutdown_connections 2024-05-15T16:36:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.918+0000 7efcbe955640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7efca006e120 0x7efca00705e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.918+0000 7efcbe955640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcb8101c10 0x7efcb81966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.918+0000 7efcbe955640 1 --2- 172.21.15.49:0/745648290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcb81012f0 0x7efcb81961b0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.918+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 >> 172.21.15.49:0/745648290 conn(0x7efcb80fab10 msgr2=0x7efcb81025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.918+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 shutdown_connections 2024-05-15T16:36:02.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:02.918+0000 7efcbe955640 1 -- 172.21.15.49:0/745648290 wait complete. 2024-05-15T16:36:02.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-15T16:36:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.036+0000 7fd184a4a640 1 -- 172.21.15.49:0/1197902753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180100d50 msgr2=0x7fd180104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.036+0000 7fd184a4a640 1 --2- 172.21.15.49:0/1197902753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180100d50 0x7fd180104d90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd168009880 tx=0x7fd16802bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.036+0000 7fd184a4a640 1 -- 172.21.15.49:0/1197902753 shutdown_connections 2024-05-15T16:36:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.036+0000 7fd184a4a640 1 --2- 172.21.15.49:0/1197902753 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180100d50 0x7fd180104d90 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.036+0000 7fd184a4a640 1 --2- 172.21.15.49:0/1197902753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1801003a0 0x7fd180100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.036+0000 7fd184a4a640 1 -- 172.21.15.49:0/1197902753 >> 172.21.15.49:0/1197902753 conn(0x7fd1800fbc20 msgr2=0x7fd1800fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.036+0000 7fd184a4a640 1 -- 172.21.15.49:0/1197902753 shutdown_connections 2024-05-15T16:36:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.036+0000 7fd184a4a640 1 -- 172.21.15.49:0/1197902753 wait complete. 2024-05-15T16:36:03.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.037+0000 7fd184a4a640 1 Processor -- start 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.037+0000 7fd184a4a640 1 -- start start 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.037+0000 7fd184a4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1801003a0 0x7fd1801961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.037+0000 7fd184a4a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180196720 0x7fd18019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.037+0000 7fd184a4a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180196d20 con 0x7fd1801003a0 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.037+0000 7fd184a4a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180196e90 con 0x7fd180196720 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.037+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1801003a0 0x7fd1801961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.037+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1801003a0 0x7fd1801961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52046/0 (socket says 172.21.15.49:52046) 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.037+0000 7fd17e59c640 1 -- 172.21.15.49:0/2030447473 learned_addr learned my addr 172.21.15.49:0/2030447473 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd17e59c640 1 -- 172.21.15.49:0/2030447473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180196720 msgr2=0x7fd18019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd17dd9b640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180196720 0x7fd18019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd17e59c640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180196720 0x7fd18019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd17e59c640 1 -- 172.21.15.49:0/2030447473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009560 con 0x7fd1801003a0 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd17dd9b640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180196720 0x7fd18019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd17e59c640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1801003a0 0x7fd1801961e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd17800ea60 tx=0x7fd17800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd1777fe640 1 -- 172.21.15.49:0/2030447473 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd17800cdb0 con 0x7fd1801003a0 2024-05-15T16:36:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18019b070 con 0x7fd1801003a0 2024-05-15T16:36:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd1777fe640 1 -- 172.21.15.49:0/2030447473 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd178011930 con 0x7fd1801003a0 2024-05-15T16:36:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd1777fe640 1 -- 172.21.15.49:0/2030447473 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd17801aa30 con 0x7fd1801003a0 2024-05-15T16:36:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.038+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18019b5d0 con 0x7fd1801003a0 2024-05-15T16:36:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.039+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd180101870 con 0x7fd1801003a0 2024-05-15T16:36:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.039+0000 7fd1777fe640 1 -- 172.21.15.49:0/2030447473 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd17801aca0 con 0x7fd1801003a0 2024-05-15T16:36:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.040+0000 7fd1777fe640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd16c06e170 0x7fd16c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.040+0000 7fd17dd9b640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd16c06e170 0x7fd16c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.040+0000 7fd1777fe640 1 -- 172.21.15.49:0/2030447473 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd17808e990 con 0x7fd1801003a0 2024-05-15T16:36:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.041+0000 7fd17dd9b640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd16c06e170 0x7fd16c070630 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd168036040 tx=0x7fd168000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.045+0000 7fd1777fe640 1 -- 172.21.15.49:0/2030447473 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd178057c70 con 0x7fd1801003a0 2024-05-15T16:36:03.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:02 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/25395537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T16:36:03.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:02 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2905107948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T16:36:03.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:02 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/25395537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T16:36:03.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:02 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2905107948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T16:36:03.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.229+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fd18019b850 con 0x7fd1801003a0 2024-05-15T16:36:03.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.229+0000 7fd1777fe640 1 -- 172.21.15.49:0/2030447473 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd17805b330 con 0x7fd1801003a0 2024-05-15T16:36:03.231 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-15T16:36:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd16c06e170 msgr2=0x7fd16c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd16c06e170 0x7fd16c070630 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd168036040 tx=0x7fd168000bc0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1801003a0 msgr2=0x7fd1801961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1801003a0 0x7fd1801961e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd17800ea60 tx=0x7fd17800ef20 comp rx=0 tx=0).stop 2024-05-15T16:36:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 shutdown_connections 2024-05-15T16:36:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd16c06e170 0x7fd16c070630 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180196720 0x7fd18019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 --2- 172.21.15.49:0/2030447473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1801003a0 0x7fd1801961e0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 >> 172.21.15.49:0/2030447473 conn(0x7fd1800fbc20 msgr2=0x7fd1800fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:03.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 shutdown_connections 2024-05-15T16:36:03.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.231+0000 7fd184a4a640 1 -- 172.21.15.49:0/2030447473 wait complete. 2024-05-15T16:36:03.244 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-15T16:36:03.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.347+0000 7f22809e2640 1 -- 172.21.15.49:0/3539911206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c0ff2d0 msgr2=0x7f227c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.347+0000 7f22809e2640 1 --2- 172.21.15.49:0/3539911206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c0ff2d0 0x7f227c0ff6b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f226c009880 tx=0x7f226c02bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.347+0000 7f22809e2640 1 -- 172.21.15.49:0/3539911206 shutdown_connections 2024-05-15T16:36:03.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.347+0000 7f22809e2640 1 --2- 172.21.15.49:0/3539911206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227c0ffc80 0x7f227c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.347+0000 7f22809e2640 1 --2- 172.21.15.49:0/3539911206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c0ff2d0 0x7f227c0ff6b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.347+0000 7f22809e2640 1 -- 172.21.15.49:0/3539911206 >> 172.21.15.49:0/3539911206 conn(0x7f227c0fab70 msgr2=0x7f227c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:03.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.347+0000 7f22809e2640 1 -- 172.21.15.49:0/3539911206 shutdown_connections 2024-05-15T16:36:03.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.347+0000 7f22809e2640 1 -- 172.21.15.49:0/3539911206 wait complete. 2024-05-15T16:36:03.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.348+0000 7f22809e2640 1 Processor -- start 2024-05-15T16:36:03.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.348+0000 7f22809e2640 1 -- start start 2024-05-15T16:36:03.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.348+0000 7f22809e2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227c0ff2d0 0x7f227c1951b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.348+0000 7f22809e2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c1956f0 0x7f227c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.348+0000 7f22809e2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c195cf0 con 0x7f227c1956f0 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.348+0000 7f22809e2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c195e60 con 0x7f227c0ff2d0 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.348+0000 7f2271d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c1956f0 0x7f227c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.348+0000 7f2271d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c1956f0 0x7f227c199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52072/0 (socket says 172.21.15.49:52072) 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.348+0000 7f2271d9b640 1 -- 172.21.15.49:0/610260681 learned_addr learned my addr 172.21.15.49:0/610260681 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f227a59c640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227c0ff2d0 0x7f227c1951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f2271d9b640 1 -- 172.21.15.49:0/610260681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227c0ff2d0 msgr2=0x7f227c1951b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f2271d9b640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227c0ff2d0 0x7f227c1951b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f2271d9b640 1 -- 172.21.15.49:0/610260681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c009560 con 0x7f227c1956f0 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f227a59c640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227c0ff2d0 0x7f227c1951b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f2271d9b640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c1956f0 0x7f227c199a90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f226400b400 tx=0x7f226400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f2273fff640 1 -- 172.21.15.49:0/610260681 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2264011800 con 0x7f227c1956f0 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c19a090 con 0x7f227c1956f0 2024-05-15T16:36:03.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f2273fff640 1 -- 172.21.15.49:0/610260681 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2264011e20 con 0x7f227c1956f0 2024-05-15T16:36:03.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f2273fff640 1 -- 172.21.15.49:0/610260681 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f226400f2e0 con 0x7f227c1956f0 2024-05-15T16:36:03.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.349+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c19a5f0 con 0x7f227c1956f0 2024-05-15T16:36:03.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.350+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c1007a0 con 0x7f227c1956f0 2024-05-15T16:36:03.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.351+0000 7f2273fff640 1 -- 172.21.15.49:0/610260681 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2264010380 con 0x7f227c1956f0 2024-05-15T16:36:03.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.351+0000 7f2273fff640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f226806e170 0x7f2268070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.351+0000 7f2273fff640 1 -- 172.21.15.49:0/610260681 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f226408dec0 con 0x7f227c1956f0 2024-05-15T16:36:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.351+0000 7f227a59c640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f226806e170 0x7f2268070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:03.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.352+0000 7f227a59c640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f226806e170 0x7f2268070630 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f226c000c00 tx=0x7f226c02cb60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:03.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.356+0000 7f2273fff640 1 -- 172.21.15.49:0/610260681 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f22640571a0 con 0x7f227c1956f0 2024-05-15T16:36:03.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.578+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f227c19a900 con 0x7f227c1956f0 2024-05-15T16:36:03.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.578+0000 7f2273fff640 1 -- 172.21.15.49:0/610260681 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5248 (secure 0 0 0) 0x7f227c19a900 con 0x7f227c1956f0 2024-05-15T16:36:03.580 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:03.580 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:34:18.682149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":0,"incarnation":29,"state":"up:active","state_seq":468,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":1,"incarnation":29,"state":"up:active","state_seq":463,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44339":{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44891":{"gid":44891,"name":"cephfs.smithi144.kziuke","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6827/755343004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":755343004},{"type":"v1","addr":"172.21.15.144:6827","nonce":755343004}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34420,"qdb_cluster":[34420,34424]},"id":1}]} 2024-05-15T16:36:03.580 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 36 2024-05-15T16:36:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.580+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f226806e170 msgr2=0x7f2268070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.580+0000 7f22809e2640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f226806e170 0x7f2268070630 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f226c000c00 tx=0x7f226c02cb60 comp rx=0 tx=0).stop 2024-05-15T16:36:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.580+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c1956f0 msgr2=0x7f227c199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.580+0000 7f22809e2640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c1956f0 0x7f227c199a90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f226400b400 tx=0x7f226400b8c0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.580+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 shutdown_connections 2024-05-15T16:36:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.580+0000 7f22809e2640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f226806e170 0x7f2268070630 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.580+0000 7f22809e2640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f227c1956f0 0x7f227c199a90 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.580+0000 7f22809e2640 1 --2- 172.21.15.49:0/610260681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227c0ff2d0 0x7f227c1951b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.580+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 >> 172.21.15.49:0/610260681 conn(0x7f227c0fab70 msgr2=0x7f227c0fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:03.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.581+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 shutdown_connections 2024-05-15T16:36:03.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.581+0000 7f22809e2640 1 -- 172.21.15.49:0/610260681 wait complete. 2024-05-15T16:36:03.593 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 50} 2024-05-15T16:36:03.594 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-05-15T16:36:03.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.698+0000 7f947c9d7640 1 -- 172.21.15.49:0/874152290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101320 msgr2=0x7f9478101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.698+0000 7f947c9d7640 1 --2- 172.21.15.49:0/874152290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101320 0x7f9478101700 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9468009880 tx=0x7f946802bd70 comp rx=0 tx=0).stop 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.698+0000 7f947c9d7640 1 -- 172.21.15.49:0/874152290 shutdown_connections 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.698+0000 7f947c9d7640 1 --2- 172.21.15.49:0/874152290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478101c40 0x7f947810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.698+0000 7f947c9d7640 1 --2- 172.21.15.49:0/874152290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101320 0x7f9478101700 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.698+0000 7f947c9d7640 1 -- 172.21.15.49:0/874152290 >> 172.21.15.49:0/874152290 conn(0x7f94780fab40 msgr2=0x7f94780fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.698+0000 7f947c9d7640 1 -- 172.21.15.49:0/874152290 shutdown_connections 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.698+0000 7f947c9d7640 1 -- 172.21.15.49:0/874152290 wait complete. 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f947c9d7640 1 Processor -- start 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f947c9d7640 1 -- start start 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f947c9d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478101320 0x7f9478196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f947c9d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101c40 0x7f9478196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f947c9d7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9478196df0 con 0x7f9478101c40 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f947c9d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947819aab0 con 0x7f9478101320 2024-05-15T16:36:03.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f9475d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101c40 0x7f9478196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f9475d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101c40 0x7f9478196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53836/0 (socket says 172.21.15.49:53836) 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f9475d9b640 1 -- 172.21.15.49:0/904772615 learned_addr learned my addr 172.21.15.49:0/904772615 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.699+0000 7f947659c640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478101320 0x7f9478196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.700+0000 7f9475d9b640 1 -- 172.21.15.49:0/904772615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478101320 msgr2=0x7f9478196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.700+0000 7f9475d9b640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478101320 0x7f9478196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.700+0000 7f9475d9b640 1 -- 172.21.15.49:0/904772615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468009560 con 0x7f9478101c40 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.700+0000 7f9475d9b640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101c40 0x7f9478196760 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c8f0 tx=0x7f946c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.700+0000 7f94677fe640 1 -- 172.21.15.49:0/904772615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946c007b80 con 0x7f9478101c40 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.700+0000 7f94677fe640 1 -- 172.21.15.49:0/904772615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f946c00ae50 con 0x7f9478101c40 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.700+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947819ad90 con 0x7f9478101c40 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.700+0000 7f94677fe640 1 -- 172.21.15.49:0/904772615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946c004e50 con 0x7f9478101c40 2024-05-15T16:36:03.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.700+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947819b2f0 con 0x7f9478101c40 2024-05-15T16:36:03.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.701+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9478068f10 con 0x7f9478101c40 2024-05-15T16:36:03.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.702+0000 7f94677fe640 1 -- 172.21.15.49:0/904772615 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f946c005020 con 0x7f9478101c40 2024-05-15T16:36:03.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.702+0000 7f94677fe640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f946006e120 0x7f94600705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:03.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.702+0000 7f94677fe640 1 -- 172.21.15.49:0/904772615 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f946c08df60 con 0x7f9478101c40 2024-05-15T16:36:03.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.702+0000 7f947659c640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f946006e120 0x7f94600705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:03.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.703+0000 7f947659c640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f946006e120 0x7f94600705e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9468000c00 tx=0x7f946802cbd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:03.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.707+0000 7f94677fe640 1 -- 172.21.15.49:0/904772615 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f946c0c2850 con 0x7f9478101c40 2024-05-15T16:36:03.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.931+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f947804e210 con 0x7f9478101c40 2024-05-15T16:36:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.932+0000 7f94677fe640 1 -- 172.21.15.49:0/904772615 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v36) v1 ==== 107+0+3334 (secure 0 0 0) 0x7f946c01dd20 con 0x7f9478101c40 2024-05-15T16:36:03.933 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:03.933 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:55.915045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14484,"mds_1":24303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi049.gocadf","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6829/2512292397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2512292397},{"type":"v1","addr":"172.21.15.49:6829","nonce":2512292397}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24303":{"gid":24303,"name":"cephfs.smithi144.jgodzs","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6825/2552333341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2552333341},{"type":"v1","addr":"172.21.15.144:6825","nonce":2552333341}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14484,"qdb_cluster":[14484,24303]},"id":1}]} 2024-05-15T16:36:03.934 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 18 2024-05-15T16:36:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f946006e120 msgr2=0x7f94600705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f946006e120 0x7f94600705e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9468000c00 tx=0x7f946802cbd0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101c40 msgr2=0x7f9478196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101c40 0x7f9478196760 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c8f0 tx=0x7f946c00cdb0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 shutdown_connections 2024-05-15T16:36:03.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f946006e120 0x7f94600705e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9478101c40 0x7f9478196760 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 --2- 172.21.15.49:0/904772615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478101320 0x7f9478196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:03.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 >> 172.21.15.49:0/904772615 conn(0x7f94780fab40 msgr2=0x7f9478102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:03.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 shutdown_connections 2024-05-15T16:36:03.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:03.934+0000 7f947c9d7640 1 -- 172.21.15.49:0/904772615 wait complete. 2024-05-15T16:36:03.946 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-05-15T16:36:03.946 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-05-15T16:36:04.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.051+0000 7f522e496640 1 -- 172.21.15.49:0/2051050520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 msgr2=0x7f5228067aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.051+0000 7f522e496640 1 --2- 172.21.15.49:0/2051050520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 0x7f5228067aa0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5218009880 tx=0x7f521802bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.051+0000 7f522e496640 1 -- 172.21.15.49:0/2051050520 shutdown_connections 2024-05-15T16:36:04.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.051+0000 7f522e496640 1 --2- 172.21.15.49:0/2051050520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5228068070 0x7f5228104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.051+0000 7f522e496640 1 --2- 172.21.15.49:0/2051050520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 0x7f5228067aa0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.051+0000 7f522e496640 1 -- 172.21.15.49:0/2051050520 >> 172.21.15.49:0/2051050520 conn(0x7f5228074b50 msgr2=0x7f5228074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.051+0000 7f522e496640 1 -- 172.21.15.49:0/2051050520 shutdown_connections 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.051+0000 7f522e496640 1 -- 172.21.15.49:0/2051050520 wait complete. 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.051+0000 7f522e496640 1 Processor -- start 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f522e496640 1 -- start start 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f522e496640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 0x7f5228196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f522e496640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5228196760 0x7f522819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f522e496640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5228196d60 con 0x7f52280676c0 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f522e496640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5228196ed0 con 0x7f5228196760 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f5227fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 0x7f5228196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f5227fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 0x7f5228196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53848/0 (socket says 172.21.15.49:53848) 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f5227fff640 1 -- 172.21.15.49:0/1007724983 learned_addr learned my addr 172.21.15.49:0/1007724983 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f521ffff640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5228196760 0x7f522819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f5227fff640 1 -- 172.21.15.49:0/1007724983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5228196760 msgr2=0x7f522819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f5227fff640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5228196760 0x7f522819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f5227fff640 1 -- 172.21.15.49:0/1007724983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5218009560 con 0x7f52280676c0 2024-05-15T16:36:04.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.052+0000 7f5227fff640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 0x7f5228196220 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5218006010 tx=0x7f521802cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:04.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.053+0000 7f5225ffb640 1 -- 172.21.15.49:0/1007724983 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52180044a0 con 0x7f52280676c0 2024-05-15T16:36:04.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.053+0000 7f5225ffb640 1 -- 172.21.15.49:0/1007724983 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5218003ab0 con 0x7f52280676c0 2024-05-15T16:36:04.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.053+0000 7f5225ffb640 1 -- 172.21.15.49:0/1007724983 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5218005590 con 0x7f52280676c0 2024-05-15T16:36:04.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.053+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522819b0a0 con 0x7f52280676c0 2024-05-15T16:36:04.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.053+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522819b5b0 con 0x7f52280676c0 2024-05-15T16:36:04.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.054+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5228101890 con 0x7f52280676c0 2024-05-15T16:36:04.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.054+0000 7f5225ffb640 1 -- 172.21.15.49:0/1007724983 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5218004dd0 con 0x7f52280676c0 2024-05-15T16:36:04.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.055+0000 7f5225ffb640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f521406e170 0x7f5214070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:04.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.055+0000 7f5225ffb640 1 -- 172.21.15.49:0/1007724983 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f52180b05b0 con 0x7f52280676c0 2024-05-15T16:36:04.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.055+0000 7f521ffff640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f521406e170 0x7f5214070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:04.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.055+0000 7f521ffff640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f521406e170 0x7f5214070630 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5210005fd0 tx=0x7f5210005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:04.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.059+0000 7f5225ffb640 1 -- 172.21.15.49:0/1007724983 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5218079890 con 0x7f52280676c0 2024-05-15T16:36:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:03 smithi144 ceph-mon[86635]: pgmap v1174: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:03 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/745648290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-15T16:36:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:03 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2030447473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-15T16:36:04.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:03 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/610260681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T16:36:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:03 smithi049 ceph-mon[112142]: pgmap v1174: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:03 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/745648290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-15T16:36:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:03 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2030447473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-15T16:36:04.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:03 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/610260681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T16:36:04.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.282+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f522819b820 con 0x7f52280676c0 2024-05-15T16:36:04.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.282+0000 7f5225ffb640 1 -- 172.21.15.49:0/1007724983 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v36) v1 ==== 107+0+3295 (secure 0 0 0) 0x7f5218041cf0 con 0x7f52280676c0 2024-05-15T16:36:04.284 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:04.284 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi049.abbagx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6827/3089875224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":3089875224},{"type":"v1","addr":"172.21.15.49:6827","nonce":3089875224}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44269,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/334969158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":334969158},{"type":"v1","addr":"172.21.15.144:6827","nonce":334969158}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 19 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f521406e170 msgr2=0x7f5214070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f521406e170 0x7f5214070630 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5210005fd0 tx=0x7f5210005850 comp rx=0 tx=0).stop 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 msgr2=0x7f5228196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 0x7f5228196220 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5218006010 tx=0x7f521802cea0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 shutdown_connections 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f521406e170 0x7f5214070630 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5228196760 0x7f522819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 --2- 172.21.15.49:0/1007724983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52280676c0 0x7f5228196220 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 >> 172.21.15.49:0/1007724983 conn(0x7f5228074b50 msgr2=0x7f5228103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 shutdown_connections 2024-05-15T16:36:04.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.284+0000 7f522e496640 1 -- 172.21.15.49:0/1007724983 wait complete. 2024-05-15T16:36:04.297 DEBUG:tasks.fs:max_mds not reduced in epoch 19 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:04.297 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-05-15T16:36:04.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-05-15T16:36:04.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 -- 172.21.15.49:0/528328781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c0ffd00 msgr2=0x7f935c103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 --2- 172.21.15.49:0/528328781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c0ffd00 0x7f935c103c90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f934c009880 tx=0x7f934c02bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 -- 172.21.15.49:0/528328781 shutdown_connections 2024-05-15T16:36:04.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 --2- 172.21.15.49:0/528328781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c0ffd00 0x7f935c103c90 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 --2- 172.21.15.49:0/528328781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f935c0ff350 0x7f935c0ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 -- 172.21.15.49:0/528328781 >> 172.21.15.49:0/528328781 conn(0x7f935c074b50 msgr2=0x7f935c074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 -- 172.21.15.49:0/528328781 shutdown_connections 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 -- 172.21.15.49:0/528328781 wait complete. 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 Processor -- start 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.402+0000 7f9362feb640 1 -- start start 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f9362feb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f935c0ff350 0x7f935c195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f9362feb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c195680 0x7f935c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f9362feb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c195c80 con 0x7f935c195680 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f9362feb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c195df0 con 0x7f935c0ff350 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f935bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c195680 0x7f935c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f935bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c195680 0x7f935c1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53864/0 (socket says 172.21.15.49:53864) 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f935bfff640 1 -- 172.21.15.49:0/3362834253 learned_addr learned my addr 172.21.15.49:0/3362834253 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f9360d87640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f935c0ff350 0x7f935c195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f935bfff640 1 -- 172.21.15.49:0/3362834253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f935c0ff350 msgr2=0x7f935c195140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f935bfff640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f935c0ff350 0x7f935c195140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f935bfff640 1 -- 172.21.15.49:0/3362834253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c009560 con 0x7f935c195680 2024-05-15T16:36:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f9360d87640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f935c0ff350 0x7f935c195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f935bfff640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c195680 0x7f935c1999d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f934c036040 tx=0x7f934c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.403+0000 7f9359ffb640 1 -- 172.21.15.49:0/3362834253 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f934c0035f0 con 0x7f935c195680 2024-05-15T16:36:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.404+0000 7f9359ffb640 1 -- 172.21.15.49:0/3362834253 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f934c003c10 con 0x7f935c195680 2024-05-15T16:36:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.404+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c199f70 con 0x7f935c195680 2024-05-15T16:36:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.404+0000 7f9359ffb640 1 -- 172.21.15.49:0/3362834253 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f934c034440 con 0x7f935c195680 2024-05-15T16:36:04.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.404+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c19a4d0 con 0x7f935c195680 2024-05-15T16:36:04.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.405+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c100770 con 0x7f935c195680 2024-05-15T16:36:04.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.405+0000 7f9359ffb640 1 -- 172.21.15.49:0/3362834253 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f934c004f60 con 0x7f935c195680 2024-05-15T16:36:04.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.406+0000 7f9359ffb640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f934806e170 0x7f9348070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:04.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.406+0000 7f9359ffb640 1 -- 172.21.15.49:0/3362834253 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f934c0b0560 con 0x7f935c195680 2024-05-15T16:36:04.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.406+0000 7f9360d87640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f934806e170 0x7f9348070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:04.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.406+0000 7f9360d87640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f934806e170 0x7f9348070630 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f93440058c0 tx=0x7f9344005d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:04.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.410+0000 7f9359ffb640 1 -- 172.21.15.49:0/3362834253 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f934c079840 con 0x7f935c195680 2024-05-15T16:36:04.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.636+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f935c04e210 con 0x7f935c195680 2024-05-15T16:36:04.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.636+0000 7f9359ffb640 1 -- 172.21.15.49:0/3362834253 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v36) v1 ==== 107+0+4853 (secure 0 0 0) 0x7f934c07cf00 con 0x7f935c195680 2024-05-15T16:36:04.638 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:04.638 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi049.abbagx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6827/3089875224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":3089875224},{"type":"v1","addr":"172.21.15.49:6827","nonce":3089875224}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":34388,"name":"cephfs.smithi049.gocadf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6829/210029258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":210029258},{"type":"v1","addr":"172.21.15.49:6829","nonce":210029258}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":44269,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/334969158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":334969158},{"type":"v1","addr":"172.21.15.144:6827","nonce":334969158}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44277,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/1712325654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1712325654},{"type":"v1","addr":"172.21.15.144:6825","nonce":1712325654}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:04.638 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 20 2024-05-15T16:36:04.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f934806e170 msgr2=0x7f9348070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f934806e170 0x7f9348070630 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f93440058c0 tx=0x7f9344005d30 comp rx=0 tx=0).stop 2024-05-15T16:36:04.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c195680 msgr2=0x7f935c1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c195680 0x7f935c1999d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f934c036040 tx=0x7f934c004040 comp rx=0 tx=0).stop 2024-05-15T16:36:04.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 shutdown_connections 2024-05-15T16:36:04.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f934806e170 0x7f9348070630 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f935c195680 0x7f935c1999d0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 --2- 172.21.15.49:0/3362834253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f935c0ff350 0x7f935c195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 >> 172.21.15.49:0/3362834253 conn(0x7f935c074b50 msgr2=0x7f935c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:04.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 shutdown_connections 2024-05-15T16:36:04.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.638+0000 7f9362feb640 1 -- 172.21.15.49:0/3362834253 wait complete. 2024-05-15T16:36:04.651 DEBUG:tasks.fs:max_mds not reduced in epoch 20 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:04.651 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-05-15T16:36:04.651 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-05-15T16:36:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.756+0000 7fd61dfa7640 1 -- 172.21.15.49:0/3707217368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6181003a0 msgr2=0x7fd618100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.756+0000 7fd61dfa7640 1 --2- 172.21.15.49:0/3707217368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6181003a0 0x7fd618100780 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd608009880 tx=0x7fd60802bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.757+0000 7fd61dfa7640 1 -- 172.21.15.49:0/3707217368 shutdown_connections 2024-05-15T16:36:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.757+0000 7fd61dfa7640 1 --2- 172.21.15.49:0/3707217368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618100d50 0x7fd618104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.757+0000 7fd61dfa7640 1 --2- 172.21.15.49:0/3707217368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6181003a0 0x7fd618100780 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.757+0000 7fd61dfa7640 1 -- 172.21.15.49:0/3707217368 >> 172.21.15.49:0/3707217368 conn(0x7fd6180fbc20 msgr2=0x7fd6180fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:04.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.757+0000 7fd61dfa7640 1 -- 172.21.15.49:0/3707217368 shutdown_connections 2024-05-15T16:36:04.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.757+0000 7fd61dfa7640 1 -- 172.21.15.49:0/3707217368 wait complete. 2024-05-15T16:36:04.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.757+0000 7fd61dfa7640 1 Processor -- start 2024-05-15T16:36:04.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.757+0000 7fd61dfa7640 1 -- start start 2024-05-15T16:36:04.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd61dfa7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6181003a0 0x7fd618196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:04.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd61dfa7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd618196760 0x7fd61819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd61dfa7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618196d60 con 0x7fd618196760 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd61dfa7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618196ed0 con 0x7fd6181003a0 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd616ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd618196760 0x7fd61819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd616ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd618196760 0x7fd61819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53868/0 (socket says 172.21.15.49:53868) 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd616ffd640 1 -- 172.21.15.49:0/1610227237 learned_addr learned my addr 172.21.15.49:0/1610227237 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd6177fe640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6181003a0 0x7fd618196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd616ffd640 1 -- 172.21.15.49:0/1610227237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6181003a0 msgr2=0x7fd618196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd616ffd640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6181003a0 0x7fd618196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd616ffd640 1 -- 172.21.15.49:0/1610227237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd608009560 con 0x7fd618196760 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd616ffd640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd618196760 0x7fd61819aab0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00d6f0 tx=0x7fd60c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd614ff9640 1 -- 172.21.15.49:0/1610227237 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60c0051f0 con 0x7fd618196760 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd614ff9640 1 -- 172.21.15.49:0/1610227237 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd60c009ce0 con 0x7fd618196760 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61819b0b0 con 0x7fd618196760 2024-05-15T16:36:04.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd614ff9640 1 -- 172.21.15.49:0/1610227237 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60c004e80 con 0x7fd618196760 2024-05-15T16:36:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.758+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61819b610 con 0x7fd618196760 2024-05-15T16:36:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.759+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61819b8e0 con 0x7fd618196760 2024-05-15T16:36:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.760+0000 7fd614ff9640 1 -- 172.21.15.49:0/1610227237 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd60c0046c0 con 0x7fd618196760 2024-05-15T16:36:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.760+0000 7fd614ff9640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd60006e120 0x7fd6000705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.760+0000 7fd614ff9640 1 -- 172.21.15.49:0/1610227237 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd60c08dc40 con 0x7fd618196760 2024-05-15T16:36:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.760+0000 7fd6177fe640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd60006e120 0x7fd6000705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.761+0000 7fd6177fe640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd60006e120 0x7fd6000705e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd608036040 tx=0x7fd608000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:04.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.765+0000 7fd614ff9640 1 -- 172.21.15.49:0/1610227237 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd60c056f20 con 0x7fd618196760 2024-05-15T16:36:04.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:04 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/904772615' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-05-15T16:36:04.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:04 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1007724983' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-15T16:36:04.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:04 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3362834253' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-15T16:36:04.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.988+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fd6181974d0 con 0x7fd618196760 2024-05-15T16:36:04.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.989+0000 7fd614ff9640 1 -- 172.21.15.49:0/1610227237 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v36) v1 ==== 107+0+4074 (secure 0 0 0) 0x7fd60c017090 con 0x7fd618196760 2024-05-15T16:36:04.990 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:04.991 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi049.gocadf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6829/210029258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":210029258},{"type":"v1","addr":"172.21.15.49:6829","nonce":210029258}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":44269,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/334969158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":334969158},{"type":"v1","addr":"172.21.15.144:6827","nonce":334969158}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44277,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/1712325654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1712325654},{"type":"v1","addr":"172.21.15.144:6825","nonce":1712325654}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:04.991 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 21 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd60006e120 msgr2=0x7fd6000705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd60006e120 0x7fd6000705e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd608036040 tx=0x7fd608000bc0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd618196760 msgr2=0x7fd61819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd618196760 0x7fd61819aab0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00d6f0 tx=0x7fd60c00dbb0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 shutdown_connections 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fd60006e120 0x7fd6000705e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd618196760 0x7fd61819aab0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 --2- 172.21.15.49:0/1610227237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6181003a0 0x7fd618196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 >> 172.21.15.49:0/1610227237 conn(0x7fd6180fbc20 msgr2=0x7fd6180fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 shutdown_connections 2024-05-15T16:36:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:04.991+0000 7fd61dfa7640 1 -- 172.21.15.49:0/1610227237 wait complete. 2024-05-15T16:36:05.004 DEBUG:tasks.fs:max_mds not reduced in epoch 21 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:05.004 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-05-15T16:36:05.004 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-05-15T16:36:05.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.109+0000 7f5a174fc640 1 -- 172.21.15.49:0/3511637612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a100676c0 msgr2=0x7f5a10067aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.109+0000 7f5a174fc640 1 --2- 172.21.15.49:0/3511637612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a100676c0 0x7f5a10067aa0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009c20 tx=0x7f5a0002c1b0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.109+0000 7f5a174fc640 1 -- 172.21.15.49:0/3511637612 shutdown_connections 2024-05-15T16:36:05.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.109+0000 7f5a174fc640 1 --2- 172.21.15.49:0/3511637612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a10068070 0x7f5a10104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.109+0000 7f5a174fc640 1 --2- 172.21.15.49:0/3511637612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a100676c0 0x7f5a10067aa0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.109+0000 7f5a174fc640 1 -- 172.21.15.49:0/3511637612 >> 172.21.15.49:0/3511637612 conn(0x7f5a10074b50 msgr2=0x7f5a10074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:05.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.109+0000 7f5a174fc640 1 -- 172.21.15.49:0/3511637612 shutdown_connections 2024-05-15T16:36:05.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.109+0000 7f5a174fc640 1 -- 172.21.15.49:0/3511637612 wait complete. 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a174fc640 1 Processor -- start 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a174fc640 1 -- start start 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a174fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a100676c0 0x7f5a10196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a174fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a10196760 0x7f5a1019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a174fc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10196d60 con 0x7f5a10196760 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a174fc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10196ed0 con 0x7f5a100676c0 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a14a97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a10196760 0x7f5a1019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a14a97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a10196760 0x7f5a1019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53884/0 (socket says 172.21.15.49:53884) 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a14a97640 1 -- 172.21.15.49:0/2582594988 learned_addr learned my addr 172.21.15.49:0/2582594988 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a15298640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a100676c0 0x7f5a10196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a14a97640 1 -- 172.21.15.49:0/2582594988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a100676c0 msgr2=0x7f5a10196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a14a97640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a100676c0 0x7f5a10196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a14a97640 1 -- 172.21.15.49:0/2582594988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a00009900 con 0x7f5a10196760 2024-05-15T16:36:05.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.110+0000 7f5a15298640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a100676c0 0x7f5a10196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:05.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.111+0000 7f5a14a97640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a10196760 0x7f5a1019ab00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c920 tx=0x7f5a0400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:05.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.111+0000 7f5a0e7fc640 1 -- 172.21.15.49:0/2582594988 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a040041f0 con 0x7f5a10196760 2024-05-15T16:36:05.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.111+0000 7f5a0e7fc640 1 -- 172.21.15.49:0/2582594988 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a04007e70 con 0x7f5a10196760 2024-05-15T16:36:05.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.111+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1019b100 con 0x7f5a10196760 2024-05-15T16:36:05.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.111+0000 7f5a0e7fc640 1 -- 172.21.15.49:0/2582594988 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a040049d0 con 0x7f5a10196760 2024-05-15T16:36:05.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.111+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1019b660 con 0x7f5a10196760 2024-05-15T16:36:05.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.112+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1019b940 con 0x7f5a10196760 2024-05-15T16:36:05.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.112+0000 7f5a0e7fc640 1 -- 172.21.15.49:0/2582594988 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5a040050b0 con 0x7f5a10196760 2024-05-15T16:36:05.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.113+0000 7f5a0e7fc640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e170 0x7f59f8070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:05.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.113+0000 7f5a15298640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e170 0x7f59f8070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:05.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.113+0000 7f5a0e7fc640 1 -- 172.21.15.49:0/2582594988 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5a0408e000 con 0x7f5a10196760 2024-05-15T16:36:05.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.114+0000 7f5a15298640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e170 0x7f59f8070630 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5a00006010 tx=0x7f5a000093f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:05.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.118+0000 7f5a0e7fc640 1 -- 172.21.15.49:0/2582594988 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5a040572e0 con 0x7f5a10196760 2024-05-15T16:36:05.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:04 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/904772615' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-05-15T16:36:05.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:04 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1007724983' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-15T16:36:05.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:04 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3362834253' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-15T16:36:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.374+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f5a10197520 con 0x7f5a10196760 2024-05-15T16:36:05.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.374+0000 7f5a0e7fc640 1 -- 172.21.15.49:0/2582594988 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v36) v1 ==== 107+0+4918 (secure 0 0 0) 0x7f5a0405a9a0 con 0x7f5a10196760 2024-05-15T16:36:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:05.377 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi049.gocadf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6829/210029258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":210029258},{"type":"v1","addr":"172.21.15.49:6829","nonce":210029258}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":44269,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/334969158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":334969158},{"type":"v1","addr":"172.21.15.144:6827","nonce":334969158}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44277,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/1712325654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1712325654},{"type":"v1","addr":"172.21.15.144:6825","nonce":1712325654}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:05.377 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 22 2024-05-15T16:36:05.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.376+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e170 msgr2=0x7f59f8070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.376+0000 7f5a174fc640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e170 0x7f59f8070630 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5a00006010 tx=0x7f5a000093f0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.376+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a10196760 msgr2=0x7f5a1019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.376+0000 7f5a174fc640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a10196760 0x7f5a1019ab00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c920 tx=0x7f5a0400cde0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.376+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 shutdown_connections 2024-05-15T16:36:05.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.376+0000 7f5a174fc640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f59f806e170 0x7f59f8070630 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.376+0000 7f5a174fc640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a10196760 0x7f5a1019ab00 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.376+0000 7f5a174fc640 1 --2- 172.21.15.49:0/2582594988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a100676c0 0x7f5a10196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.376+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 >> 172.21.15.49:0/2582594988 conn(0x7f5a10074b50 msgr2=0x7f5a10103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:05.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.377+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 shutdown_connections 2024-05-15T16:36:05.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.377+0000 7f5a174fc640 1 -- 172.21.15.49:0/2582594988 wait complete. 2024-05-15T16:36:05.390 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:05.390 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-05-15T16:36:05.390 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-05-15T16:36:05.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2285179421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f841003c0 msgr2=0x7f7f841007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 --2- 172.21.15.49:0/2285179421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f841003c0 0x7f7f841007a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009c20 tx=0x7f7f7402c1b0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2285179421 shutdown_connections 2024-05-15T16:36:05.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 --2- 172.21.15.49:0/2285179421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f84100d70 0x7f7f84104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 --2- 172.21.15.49:0/2285179421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f841003c0 0x7f7f841007a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2285179421 >> 172.21.15.49:0/2285179421 conn(0x7f7f840fbc40 msgr2=0x7f7f840fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:05.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2285179421 shutdown_connections 2024-05-15T16:36:05.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2285179421 wait complete. 2024-05-15T16:36:05.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 Processor -- start 2024-05-15T16:36:05.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.495+0000 7f7f8ad73640 1 -- start start 2024-05-15T16:36:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f8ad73640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f841003c0 0x7f7f84196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f8ad73640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f84196780 0x7f7f8419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f8ad73640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f84196d80 con 0x7f7f84196780 2024-05-15T16:36:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f8ad73640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f84196ef0 con 0x7f7f841003c0 2024-05-15T16:36:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f83fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f84196780 0x7f7f8419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f83fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f84196780 0x7f7f8419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53900/0 (socket says 172.21.15.49:53900) 2024-05-15T16:36:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f83fff640 1 -- 172.21.15.49:0/2886399802 learned_addr learned my addr 172.21.15.49:0/2886399802 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f88b0f640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f841003c0 0x7f7f84196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f83fff640 1 -- 172.21.15.49:0/2886399802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f841003c0 msgr2=0x7f7f84196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f83fff640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f841003c0 0x7f7f84196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f83fff640 1 -- 172.21.15.49:0/2886399802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74009900 con 0x7f7f84196780 2024-05-15T16:36:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f88b0f640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f841003c0 0x7f7f84196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f83fff640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f84196780 0x7f7f8419aad0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00c920 tx=0x7f7f6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f81ffb640 1 -- 172.21.15.49:0/2886399802 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f6c007b80 con 0x7f7f84196780 2024-05-15T16:36:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f81ffb640 1 -- 172.21.15.49:0/2886399802 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f6c007ce0 con 0x7f7f84196780 2024-05-15T16:36:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8419b0d0 con 0x7f7f84196780 2024-05-15T16:36:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.496+0000 7f7f81ffb640 1 -- 172.21.15.49:0/2886399802 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f6c0187d0 con 0x7f7f84196780 2024-05-15T16:36:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.497+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8419b630 con 0x7f7f84196780 2024-05-15T16:36:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.497+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f84101890 con 0x7f7f84196780 2024-05-15T16:36:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.498+0000 7f7f81ffb640 1 -- 172.21.15.49:0/2886399802 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7f6c00f450 con 0x7f7f84196780 2024-05-15T16:36:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.498+0000 7f7f81ffb640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f7006e170 0x7f7f70070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.498+0000 7f7f81ffb640 1 -- 172.21.15.49:0/2886399802 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7f6c08e260 con 0x7f7f84196780 2024-05-15T16:36:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.498+0000 7f7f88b0f640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f7006e170 0x7f7f70070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.499+0000 7f7f88b0f640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f7006e170 0x7f7f70070630 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006010 tx=0x7f7f740093f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:05.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.503+0000 7f7f81ffb640 1 -- 172.21.15.49:0/2886399802 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7f6c0575c0 con 0x7f7f84196780 2024-05-15T16:36:05.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.726+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f7f84100d70 con 0x7f7f84196780 2024-05-15T16:36:05.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.726+0000 7f7f81ffb640 1 -- 172.21.15.49:0/2886399802 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v36) v1 ==== 107+0+4142 (secure 0 0 0) 0x7f7f6c05ac80 con 0x7f7f84196780 2024-05-15T16:36:05.728 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:05.728 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":44269,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/334969158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":334969158},{"type":"v1","addr":"172.21.15.144:6827","nonce":334969158}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44277,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/1712325654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1712325654},{"type":"v1","addr":"172.21.15.144:6825","nonce":1712325654}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:05.728 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2024-05-15T16:36:05.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.728+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f7006e170 msgr2=0x7f7f70070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.728+0000 7f7f8ad73640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f7006e170 0x7f7f70070630 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006010 tx=0x7f7f740093f0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.728+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f84196780 msgr2=0x7f7f8419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.728+0000 7f7f8ad73640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f84196780 0x7f7f8419aad0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00c920 tx=0x7f7f6c00cde0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.728+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 shutdown_connections 2024-05-15T16:36:05.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.728+0000 7f7f8ad73640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7f7006e170 0x7f7f70070630 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.728+0000 7f7f8ad73640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f84196780 0x7f7f8419aad0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.728+0000 7f7f8ad73640 1 --2- 172.21.15.49:0/2886399802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f841003c0 0x7f7f84196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.728+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 >> 172.21.15.49:0/2886399802 conn(0x7f7f840fbc40 msgr2=0x7f7f840fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.729+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 shutdown_connections 2024-05-15T16:36:05.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.729+0000 7f7f8ad73640 1 -- 172.21.15.49:0/2886399802 wait complete. 2024-05-15T16:36:05.741 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:05.741 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-05-15T16:36:05.741 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-05-15T16:36:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.846+0000 7f6eb34c7640 1 -- 172.21.15.49:0/2309496818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 msgr2=0x7f6eac10a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.846+0000 7f6eb34c7640 1 --2- 172.21.15.49:0/2309496818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 0x7f6eac10a180 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0098b0 tx=0x7f6e9c02bd60 comp rx=0 tx=0).stop 2024-05-15T16:36:05.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.846+0000 7f6eb34c7640 1 -- 172.21.15.49:0/2309496818 shutdown_connections 2024-05-15T16:36:05.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.846+0000 7f6eb34c7640 1 --2- 172.21.15.49:0/2309496818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 0x7f6eac10a180 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.846+0000 7f6eb34c7640 1 --2- 172.21.15.49:0/2309496818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eac101380 0x7f6eac101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.846+0000 7f6eb34c7640 1 -- 172.21.15.49:0/2309496818 >> 172.21.15.49:0/2309496818 conn(0x7f6eac0fab40 msgr2=0x7f6eac0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:05.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.846+0000 7f6eb34c7640 1 -- 172.21.15.49:0/2309496818 shutdown_connections 2024-05-15T16:36:05.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.846+0000 7f6eb34c7640 1 -- 172.21.15.49:0/2309496818 wait complete. 2024-05-15T16:36:05.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb34c7640 1 Processor -- start 2024-05-15T16:36:05.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb34c7640 1 -- start start 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb34c7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eac101380 0x7f6eac196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb34c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 0x7f6eac1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb34c7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac196e30 con 0x7f6eac101ca0 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb34c7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac19ab40 con 0x7f6eac101380 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb0a62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 0x7f6eac1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb0a62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 0x7f6eac1967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53918/0 (socket says 172.21.15.49:53918) 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb0a62640 1 -- 172.21.15.49:0/4164430752 learned_addr learned my addr 172.21.15.49:0/4164430752 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.847+0000 7f6eb1263640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eac101380 0x7f6eac196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.848+0000 7f6eb0a62640 1 -- 172.21.15.49:0/4164430752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eac101380 msgr2=0x7f6eac196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.848+0000 7f6eb0a62640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eac101380 0x7f6eac196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.848+0000 7f6eb0a62640 1 -- 172.21.15.49:0/4164430752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c009560 con 0x7f6eac101ca0 2024-05-15T16:36:05.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.848+0000 7f6eb0a62640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 0x7f6eac1967a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006010 tx=0x7f6e9c02c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.848+0000 7f6ea27fc640 1 -- 172.21.15.49:0/4164430752 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e9c0044a0 con 0x7f6eac101ca0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.848+0000 7f6ea27fc640 1 -- 172.21.15.49:0/4164430752 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e9c003ab0 con 0x7f6eac101ca0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.848+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac19adc0 con 0x7f6eac101ca0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.848+0000 7f6ea27fc640 1 -- 172.21.15.49:0/4164430752 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e9c0055a0 con 0x7f6eac101ca0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.848+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac19b320 con 0x7f6eac101ca0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.849+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac068eb0 con 0x7f6eac101ca0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.849+0000 7f6ea27fc640 1 -- 172.21.15.49:0/4164430752 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6e9c004dd0 con 0x7f6eac101ca0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.850+0000 7f6ea27fc640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6e9806e120 0x7f6e980705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.850+0000 7f6ea27fc640 1 -- 172.21.15.49:0/4164430752 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6e9c0b0bf0 con 0x7f6eac101ca0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.850+0000 7f6eb1263640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6e9806e120 0x7f6e980705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:05.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.851+0000 7f6eb1263640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6e9806e120 0x7f6e980705e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005fd0 tx=0x7f6ea800b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:05.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:05.855+0000 7f6ea27fc640 1 -- 172.21.15.49:0/4164430752 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6e9c079ed0 con 0x7f6eac101ca0 2024-05-15T16:36:06.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.080+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f6eac04e210 con 0x7f6eac101ca0 2024-05-15T16:36:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.081+0000 7f6ea27fc640 1 -- 172.21.15.49:0/4164430752 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v36) v1 ==== 107+0+4989 (secure 0 0 0) 0x7f6e9c084070 con 0x7f6eac101ca0 2024-05-15T16:36:06.083 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:06.083 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44269,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/334969158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":334969158},{"type":"v1","addr":"172.21.15.144:6827","nonce":334969158}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44277,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/1712325654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1712325654},{"type":"v1","addr":"172.21.15.144:6825","nonce":1712325654}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:06.084 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 24 2024-05-15T16:36:06.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6e9806e120 msgr2=0x7f6e980705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6e9806e120 0x7f6e980705e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005fd0 tx=0x7f6ea800b390 comp rx=0 tx=0).stop 2024-05-15T16:36:06.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 msgr2=0x7f6eac1967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 0x7f6eac1967a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006010 tx=0x7f6e9c02c770 comp rx=0 tx=0).stop 2024-05-15T16:36:06.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 shutdown_connections 2024-05-15T16:36:06.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f6e9806e120 0x7f6e980705e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eac101ca0 0x7f6eac1967a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 --2- 172.21.15.49:0/4164430752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eac101380 0x7f6eac196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 >> 172.21.15.49:0/4164430752 conn(0x7f6eac0fab40 msgr2=0x7f6eac102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:06.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 shutdown_connections 2024-05-15T16:36:06.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.083+0000 7f6eb34c7640 1 -- 172.21.15.49:0/4164430752 wait complete. 2024-05-15T16:36:06.094 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:05 smithi049 ceph-mon[112142]: pgmap v1175: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:06.095 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:05 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1610227237' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-15T16:36:06.095 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:36:06.095 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:05 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2582594988' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-15T16:36:06.095 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:05 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2886399802' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-15T16:36:06.096 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:06.096 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-05-15T16:36:06.096 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-05-15T16:36:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:05 smithi144 ceph-mon[86635]: pgmap v1175: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:05 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1610227237' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-15T16:36:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:36:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:05 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2582594988' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-15T16:36:06.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:05 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2886399802' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-15T16:36:06.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.200+0000 7f70bdc70640 1 -- 172.21.15.49:0/1819524636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 msgr2=0x7f70b8067aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.200+0000 7f70bdc70640 1 --2- 172.21.15.49:0/1819524636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 0x7f70b8067aa0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009880 tx=0x7f70a802bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.200+0000 7f70bdc70640 1 -- 172.21.15.49:0/1819524636 shutdown_connections 2024-05-15T16:36:06.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.200+0000 7f70bdc70640 1 --2- 172.21.15.49:0/1819524636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b8068070 0x7f70b8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.200+0000 7f70bdc70640 1 --2- 172.21.15.49:0/1819524636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 0x7f70b8067aa0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.200+0000 7f70bdc70640 1 -- 172.21.15.49:0/1819524636 >> 172.21.15.49:0/1819524636 conn(0x7f70b8074b50 msgr2=0x7f70b8074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:06.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.201+0000 7f70bdc70640 1 -- 172.21.15.49:0/1819524636 shutdown_connections 2024-05-15T16:36:06.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.201+0000 7f70bdc70640 1 -- 172.21.15.49:0/1819524636 wait complete. 2024-05-15T16:36:06.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.201+0000 7f70bdc70640 1 Processor -- start 2024-05-15T16:36:06.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.201+0000 7f70bdc70640 1 -- start start 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70bdc70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 0x7f70b80ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70bdc70640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b80ff6f0 0x7f70b80ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70bdc70640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b8101110 con 0x7f70b80676c0 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70bdc70640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b81a23e0 con 0x7f70b80ff6f0 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 0x7f70b80ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 0x7f70b80ff1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53944/0 (socket says 172.21.15.49:53944) 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b77fe640 1 -- 172.21.15.49:0/2776586029 learned_addr learned my addr 172.21.15.49:0/2776586029 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b6ffd640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b80ff6f0 0x7f70b80ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b77fe640 1 -- 172.21.15.49:0/2776586029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b80ff6f0 msgr2=0x7f70b80ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b77fe640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b80ff6f0 0x7f70b80ffb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b77fe640 1 -- 172.21.15.49:0/2776586029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8009560 con 0x7f70b80676c0 2024-05-15T16:36:06.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b77fe640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 0x7f70b80ff1b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f70a802c220 tx=0x7f70a802cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b4ff9640 1 -- 172.21.15.49:0/2776586029 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a80044a0 con 0x7f70b80676c0 2024-05-15T16:36:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b4ff9640 1 -- 172.21.15.49:0/2776586029 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70a8003ab0 con 0x7f70b80676c0 2024-05-15T16:36:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b81a2580 con 0x7f70b80676c0 2024-05-15T16:36:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70b4ff9640 1 -- 172.21.15.49:0/2776586029 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a8005590 con 0x7f70b80676c0 2024-05-15T16:36:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.202+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b81a2ab0 con 0x7f70b80676c0 2024-05-15T16:36:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.203+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b8101890 con 0x7f70b80676c0 2024-05-15T16:36:06.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.204+0000 7f70b4ff9640 1 -- 172.21.15.49:0/2776586029 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f70a8004dd0 con 0x7f70b80676c0 2024-05-15T16:36:06.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.204+0000 7f70b4ff9640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70a006e170 0x7f70a0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:06.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.204+0000 7f70b4ff9640 1 -- 172.21.15.49:0/2776586029 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f70a80b05f0 con 0x7f70b80676c0 2024-05-15T16:36:06.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.204+0000 7f70b6ffd640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70a006e170 0x7f70a0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:06.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.205+0000 7f70b6ffd640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70a006e170 0x7f70a0070630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f70ac005fd0 tx=0x7f70ac005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:06.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.209+0000 7f70b4ff9640 1 -- 172.21.15.49:0/2776586029 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f70a80798d0 con 0x7f70b80676c0 2024-05-15T16:36:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.430+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f70b804e210 con 0x7f70b80676c0 2024-05-15T16:36:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.431+0000 7f70b4ff9640 1 -- 172.21.15.49:0/2776586029 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v36) v1 ==== 107+0+4207 (secure 0 0 0) 0x7f70a807cf90 con 0x7f70b80676c0 2024-05-15T16:36:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44269,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/334969158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":334969158},{"type":"v1","addr":"172.21.15.144:6827","nonce":334969158}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:06.433 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 25 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70a006e170 msgr2=0x7f70a0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70a006e170 0x7f70a0070630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f70ac005fd0 tx=0x7f70ac005850 comp rx=0 tx=0).stop 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 msgr2=0x7f70b80ff1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 0x7f70b80ff1b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f70a802c220 tx=0x7f70a802cea0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 shutdown_connections 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f70a006e170 0x7f70a0070630 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b80ff6f0 0x7f70b80ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 --2- 172.21.15.49:0/2776586029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70b80676c0 0x7f70b80ff1b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 >> 172.21.15.49:0/2776586029 conn(0x7f70b8074b50 msgr2=0x7f70b8103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 shutdown_connections 2024-05-15T16:36:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.433+0000 7f70bdc70640 1 -- 172.21.15.49:0/2776586029 wait complete. 2024-05-15T16:36:06.446 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:06.446 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-05-15T16:36:06.446 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-05-15T16:36:06.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.550+0000 7f7d328d5640 1 -- 172.21.15.49:0/1727526676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c0ffc70 msgr2=0x7f7d2c103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.550+0000 7f7d328d5640 1 --2- 172.21.15.49:0/1727526676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c0ffc70 0x7f7d2c103cb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009880 tx=0x7f7d2002bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.550+0000 7f7d328d5640 1 -- 172.21.15.49:0/1727526676 shutdown_connections 2024-05-15T16:36:06.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.550+0000 7f7d328d5640 1 --2- 172.21.15.49:0/1727526676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c0ffc70 0x7f7d2c103cb0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.550+0000 7f7d328d5640 1 --2- 172.21.15.49:0/1727526676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2c0ff2c0 0x7f7d2c0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.550+0000 7f7d328d5640 1 -- 172.21.15.49:0/1727526676 >> 172.21.15.49:0/1727526676 conn(0x7f7d2c0fab40 msgr2=0x7f7d2c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:06.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.550+0000 7f7d328d5640 1 -- 172.21.15.49:0/1727526676 shutdown_connections 2024-05-15T16:36:06.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.550+0000 7f7d328d5640 1 -- 172.21.15.49:0/1727526676 wait complete. 2024-05-15T16:36:06.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d328d5640 1 Processor -- start 2024-05-15T16:36:06.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d328d5640 1 -- start start 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d328d5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2c0ff2c0 0x7f7d2c195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d328d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c195680 0x7f7d2c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d328d5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c195c80 con 0x7f7d2c195680 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d328d5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c195df0 con 0x7f7d2c0ff2c0 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d2b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c195680 0x7f7d2c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d2b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c195680 0x7f7d2c199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53960/0 (socket says 172.21.15.49:53960) 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d2b7fe640 1 -- 172.21.15.49:0/88090352 learned_addr learned my addr 172.21.15.49:0/88090352 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d2bfff640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2c0ff2c0 0x7f7d2c195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d2b7fe640 1 -- 172.21.15.49:0/88090352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2c0ff2c0 msgr2=0x7f7d2c195140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d2b7fe640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2c0ff2c0 0x7f7d2c195140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d2b7fe640 1 -- 172.21.15.49:0/88090352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20009560 con 0x7f7d2c195680 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.551+0000 7f7d2bfff640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2c0ff2c0 0x7f7d2c195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.552+0000 7f7d2b7fe640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c195680 0x7f7d2c199a20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006010 tx=0x7f7d2002cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.552+0000 7f7d297fa640 1 -- 172.21.15.49:0/88090352 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d200044a0 con 0x7f7d2c195680 2024-05-15T16:36:06.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.552+0000 7f7d297fa640 1 -- 172.21.15.49:0/88090352 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d20003ab0 con 0x7f7d2c195680 2024-05-15T16:36:06.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.552+0000 7f7d297fa640 1 -- 172.21.15.49:0/88090352 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d20005590 con 0x7f7d2c195680 2024-05-15T16:36:06.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.552+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c199fc0 con 0x7f7d2c195680 2024-05-15T16:36:06.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.552+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c19a520 con 0x7f7d2c195680 2024-05-15T16:36:06.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.553+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c100790 con 0x7f7d2c195680 2024-05-15T16:36:06.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.553+0000 7f7d297fa640 1 -- 172.21.15.49:0/88090352 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7d20004dd0 con 0x7f7d2c195680 2024-05-15T16:36:06.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.554+0000 7f7d297fa640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d1406e170 0x7f7d14070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:06.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.554+0000 7f7d297fa640 1 -- 172.21.15.49:0/88090352 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7d200b0580 con 0x7f7d2c195680 2024-05-15T16:36:06.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.554+0000 7f7d2bfff640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d1406e170 0x7f7d14070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:06.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.554+0000 7f7d2bfff640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d1406e170 0x7f7d14070630 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0058c0 tx=0x7f7d1c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:06.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.558+0000 7f7d297fa640 1 -- 172.21.15.49:0/88090352 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7d20079860 con 0x7f7d2c195680 2024-05-15T16:36:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.783+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f7d2c04e210 con 0x7f7d2c195680 2024-05-15T16:36:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.783+0000 7f7d297fa640 1 -- 172.21.15.49:0/88090352 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v36) v1 ==== 107+0+5057 (secure 0 0 0) 0x7f7d2009d090 con 0x7f7d2c195680 2024-05-15T16:36:06.785 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:06.785 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44269,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/334969158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":334969158},{"type":"v1","addr":"172.21.15.144:6827","nonce":334969158}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 26 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.785+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d1406e170 msgr2=0x7f7d14070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.785+0000 7f7d328d5640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d1406e170 0x7f7d14070630 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0058c0 tx=0x7f7d1c005850 comp rx=0 tx=0).stop 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.785+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c195680 msgr2=0x7f7d2c199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.785+0000 7f7d328d5640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c195680 0x7f7d2c199a20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006010 tx=0x7f7d2002cea0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.785+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 shutdown_connections 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.785+0000 7f7d328d5640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f7d1406e170 0x7f7d14070630 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.785+0000 7f7d328d5640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d2c195680 0x7f7d2c199a20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.785+0000 7f7d328d5640 1 --2- 172.21.15.49:0/88090352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2c0ff2c0 0x7f7d2c195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.786+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 >> 172.21.15.49:0/88090352 conn(0x7f7d2c0fab40 msgr2=0x7f7d2c0fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.786+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 shutdown_connections 2024-05-15T16:36:06.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.786+0000 7f7d328d5640 1 -- 172.21.15.49:0/88090352 wait complete. 2024-05-15T16:36:06.798 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:06.798 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-05-15T16:36:06.798 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-05-15T16:36:06.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 -- 172.21.15.49:0/3946828834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ffd00 msgr2=0x7f8aa4103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 --2- 172.21.15.49:0/3946828834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ffd00 0x7f8aa4103c90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009880 tx=0x7f8a8c02bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 -- 172.21.15.49:0/3946828834 shutdown_connections 2024-05-15T16:36:06.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 --2- 172.21.15.49:0/3946828834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ffd00 0x7f8aa4103c90 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 --2- 172.21.15.49:0/3946828834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aa40ff350 0x7f8aa40ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 -- 172.21.15.49:0/3946828834 >> 172.21.15.49:0/3946828834 conn(0x7f8aa4074b50 msgr2=0x7f8aa4074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:06.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 -- 172.21.15.49:0/3946828834 shutdown_connections 2024-05-15T16:36:06.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 -- 172.21.15.49:0/3946828834 wait complete. 2024-05-15T16:36:06.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 Processor -- start 2024-05-15T16:36:06.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.906+0000 7f8aa9816640 1 -- start start 2024-05-15T16:36:06.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa9816640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ff350 0x7f8aa4197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:06.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa9816640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aa41975a0 0x7f8aa4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:06.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa9816640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa4068420 con 0x7f8aa40ff350 2024-05-15T16:36:06.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa9816640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa41a00d0 con 0x7f8aa41975a0 2024-05-15T16:36:06.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ff350 0x7f8aa4197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:06.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ff350 0x7f8aa4197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53966/0 (socket says 172.21.15.49:53966) 2024-05-15T16:36:06.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa2ffd640 1 -- 172.21.15.49:0/1017864436 learned_addr learned my addr 172.21.15.49:0/1017864436 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:06.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa2ffd640 1 -- 172.21.15.49:0/1017864436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aa41975a0 msgr2=0x7f8aa4197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8a9a7fc640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aa41975a0 0x7f8aa4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa2ffd640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aa41975a0 0x7f8aa4197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa2ffd640 1 -- 172.21.15.49:0/1017864436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c009560 con 0x7f8aa40ff350 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8a9a7fc640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aa41975a0 0x7f8aa4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa2ffd640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ff350 0x7f8aa4197060 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400d900 tx=0x7f8a9400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa8814640 1 -- 172.21.15.49:0/1017864436 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a94009c00 con 0x7f8aa40ff350 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa8814640 1 -- 172.21.15.49:0/1017864436 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a94009d60 con 0x7f8aa40ff350 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.907+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa41a0330 con 0x7f8aa40ff350 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.908+0000 7f8aa8814640 1 -- 172.21.15.49:0/1017864436 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a940049d0 con 0x7f8aa40ff350 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.908+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa41a0890 con 0x7f8aa40ff350 2024-05-15T16:36:06.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.908+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa4100770 con 0x7f8aa40ff350 2024-05-15T16:36:06.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.909+0000 7f8aa8814640 1 -- 172.21.15.49:0/1017864436 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8a94004020 con 0x7f8aa40ff350 2024-05-15T16:36:06.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.910+0000 7f8aa8814640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9006e170 0x7f8a90070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:06.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.910+0000 7f8aa8814640 1 -- 172.21.15.49:0/1017864436 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8a9408de10 con 0x7f8aa40ff350 2024-05-15T16:36:06.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.910+0000 7f8a9a7fc640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9006e170 0x7f8a90070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:06.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.910+0000 7f8a9a7fc640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9006e170 0x7f8a90070630 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c036040 tx=0x7f8a8c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:06.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:06.914+0000 7f8aa8814640 1 -- 172.21.15.49:0/1017864436 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8a94057170 con 0x7f8aa40ff350 2024-05-15T16:36:07.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.136+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f8aa40ffd60 con 0x7f8aa40ff350 2024-05-15T16:36:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.137+0000 7f8aa8814640 1 -- 172.21.15.49:0/1017864436 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v36) v1 ==== 107+0+4278 (secure 0 0 0) 0x7f8a94017090 con 0x7f8aa40ff350 2024-05-15T16:36:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:07.138 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:07.139 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 27 2024-05-15T16:36:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9006e170 msgr2=0x7f8a90070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9006e170 0x7f8a90070630 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c036040 tx=0x7f8a8c000bc0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ff350 msgr2=0x7f8aa4197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ff350 0x7f8aa4197060 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400d900 tx=0x7f8a9400ddc0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 shutdown_connections 2024-05-15T16:36:07.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f8a9006e170 0x7f8a90070630 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aa41975a0 0x7f8aa4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 --2- 172.21.15.49:0/1017864436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8aa40ff350 0x7f8aa4197060 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 >> 172.21.15.49:0/1017864436 conn(0x7f8aa4074b50 msgr2=0x7f8aa4102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:07.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 shutdown_connections 2024-05-15T16:36:07.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.139+0000 7f8aa9816640 1 -- 172.21.15.49:0/1017864436 wait complete. 2024-05-15T16:36:07.150 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:06 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4164430752' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-15T16:36:07.151 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:06 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2776586029' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-15T16:36:07.151 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:06 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/88090352' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-15T16:36:07.152 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:07.152 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-05-15T16:36:07.152 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-05-15T16:36:07.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:06 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4164430752' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-15T16:36:07.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:06 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2776586029' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-15T16:36:07.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:06 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/88090352' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-15T16:36:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.256+0000 7f9f408db640 1 -- 172.21.15.49:0/1284463440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c0ff920 msgr2=0x7f9f3c0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.256+0000 7f9f408db640 1 --2- 172.21.15.49:0/1284463440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c0ff920 0x7f9f3c0ffda0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009880 tx=0x7f9f3002bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 -- 172.21.15.49:0/1284463440 shutdown_connections 2024-05-15T16:36:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 --2- 172.21.15.49:0/1284463440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c0ff920 0x7f9f3c0ffda0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 --2- 172.21.15.49:0/1284463440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c1051b0 0x7f9f3c105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 -- 172.21.15.49:0/1284463440 >> 172.21.15.49:0/1284463440 conn(0x7f9f3c075380 msgr2=0x7f9f3c075790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 -- 172.21.15.49:0/1284463440 shutdown_connections 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 -- 172.21.15.49:0/1284463440 wait complete. 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 Processor -- start 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 -- start start 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c0ff920 0x7f9f3c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.257+0000 7f9f408db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c1051b0 0x7f9f3c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f408db640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c197b20 con 0x7f9f3c1051b0 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f408db640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c197c90 con 0x7f9f3c0ff920 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f39d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c1051b0 0x7f9f3c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f39d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c1051b0 0x7f9f3c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53994/0 (socket says 172.21.15.49:53994) 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f39d9b640 1 -- 172.21.15.49:0/2495643940 learned_addr learned my addr 172.21.15.49:0/2495643940 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f3a59c640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c0ff920 0x7f9f3c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f39d9b640 1 -- 172.21.15.49:0/2495643940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c0ff920 msgr2=0x7f9f3c1970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f39d9b640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c0ff920 0x7f9f3c1970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f39d9b640 1 -- 172.21.15.49:0/2495643940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f30009560 con 0x7f9f3c1051b0 2024-05-15T16:36:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f39d9b640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c1051b0 0x7f9f3c1975e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f9f3002c220 tx=0x7f9f30004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f2b7fe640 1 -- 172.21.15.49:0/2495643940 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f30004420 con 0x7f9f3c1051b0 2024-05-15T16:36:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f2b7fe640 1 -- 172.21.15.49:0/2495643940 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f30003ab0 con 0x7f9f3c1051b0 2024-05-15T16:36:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3c197f10 con 0x7f9f3c1051b0 2024-05-15T16:36:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f2b7fe640 1 -- 172.21.15.49:0/2495643940 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f30005520 con 0x7f9f3c1051b0 2024-05-15T16:36:07.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.258+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3c1a0400 con 0x7f9f3c1051b0 2024-05-15T16:36:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.260+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3c100930 con 0x7f9f3c1051b0 2024-05-15T16:36:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.260+0000 7f9f2b7fe640 1 -- 172.21.15.49:0/2495643940 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9f30003c20 con 0x7f9f3c1051b0 2024-05-15T16:36:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.260+0000 7f9f2b7fe640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9f2406e170 0x7f9f24070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.260+0000 7f9f2b7fe640 1 -- 172.21.15.49:0/2495643940 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9f300b01f0 con 0x7f9f3c1051b0 2024-05-15T16:36:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.261+0000 7f9f3a59c640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9f2406e170 0x7f9f24070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.261+0000 7f9f3a59c640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9f2406e170 0x7f9f24070630 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009bd0 tx=0x7f9f2c005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:07.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.265+0000 7f9f2b7fe640 1 -- 172.21.15.49:0/2495643940 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9f30079580 con 0x7f9f3c1051b0 2024-05-15T16:36:07.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.487+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f9f3c197f10 con 0x7f9f3c1051b0 2024-05-15T16:36:07.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.488+0000 7f9f2b7fe640 1 -- 172.21.15.49:0/2495643940 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v36) v1 ==== 107+0+5125 (secure 0 0 0) 0x7f9f3c197f10 con 0x7f9f3c1051b0 2024-05-15T16:36:07.489 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:07.490 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44891,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/755343004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":755343004},{"type":"v1","addr":"172.21.15.144:6827","nonce":755343004}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:02:56.078027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:07.490 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-05-15T16:36:07.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9f2406e170 msgr2=0x7f9f24070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9f2406e170 0x7f9f24070630 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009bd0 tx=0x7f9f2c005f90 comp rx=0 tx=0).stop 2024-05-15T16:36:07.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c1051b0 msgr2=0x7f9f3c1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c1051b0 0x7f9f3c1975e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f9f3002c220 tx=0x7f9f30004a10 comp rx=0 tx=0).stop 2024-05-15T16:36:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 shutdown_connections 2024-05-15T16:36:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f9f2406e170 0x7f9f24070630 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f3c1051b0 0x7f9f3c1975e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 --2- 172.21.15.49:0/2495643940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c0ff920 0x7f9f3c1970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 >> 172.21.15.49:0/2495643940 conn(0x7f9f3c075380 msgr2=0x7f9f3c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 shutdown_connections 2024-05-15T16:36:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.490+0000 7f9f408db640 1 -- 172.21.15.49:0/2495643940 wait complete. 2024-05-15T16:36:07.503 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-15T16:36:07.503 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-05-15T16:36:07.503 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-05-15T16:36:07.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.608+0000 7f2871fd7640 1 -- 172.21.15.49:0/139306091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101280 msgr2=0x7f286c101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.608+0000 7f2871fd7640 1 --2- 172.21.15.49:0/139306091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101280 0x7f286c101660 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f285c009c20 tx=0x7f285c02c230 comp rx=0 tx=0).stop 2024-05-15T16:36:07.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.608+0000 7f2871fd7640 1 -- 172.21.15.49:0/139306091 shutdown_connections 2024-05-15T16:36:07.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.608+0000 7f2871fd7640 1 --2- 172.21.15.49:0/139306091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c101ba0 0x7f286c10a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.608+0000 7f2871fd7640 1 --2- 172.21.15.49:0/139306091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101280 0x7f286c101660 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.608+0000 7f2871fd7640 1 -- 172.21.15.49:0/139306091 >> 172.21.15.49:0/139306091 conn(0x7f286c075c80 msgr2=0x7f286c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.608+0000 7f2871fd7640 1 -- 172.21.15.49:0/139306091 shutdown_connections 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.608+0000 7f2871fd7640 1 -- 172.21.15.49:0/139306091 wait complete. 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.608+0000 7f2871fd7640 1 Processor -- start 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f2871fd7640 1 -- start start 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f2871fd7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c101280 0x7f286c196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f2871fd7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101ba0 0x7f286c196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f2871fd7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c196d10 con 0x7f286c101ba0 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f2871fd7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c19aa20 con 0x7f286c101280 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f286affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101ba0 0x7f286c196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f286affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101ba0 0x7f286c196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54012/0 (socket says 172.21.15.49:54012) 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f286affd640 1 -- 172.21.15.49:0/563875469 learned_addr learned my addr 172.21.15.49:0/563875469 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:07.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f286b7fe640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c101280 0x7f286c196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f286affd640 1 -- 172.21.15.49:0/563875469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c101280 msgr2=0x7f286c196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f286affd640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c101280 0x7f286c196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f286affd640 1 -- 172.21.15.49:0/563875469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c009900 con 0x7f286c101ba0 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f286b7fe640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c101280 0x7f286c196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f286affd640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101ba0 0x7f286c196680 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f285400d6d0 tx=0x7f285400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f2868ff9640 1 -- 172.21.15.49:0/563875469 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28540051f0 con 0x7f286c101ba0 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c19ad00 con 0x7f286c101ba0 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f2868ff9640 1 -- 172.21.15.49:0/563875469 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2854009ce0 con 0x7f286c101ba0 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.609+0000 7f2868ff9640 1 -- 172.21.15.49:0/563875469 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2854004db0 con 0x7f286c101ba0 2024-05-15T16:36:07.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.610+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c19b260 con 0x7f286c101ba0 2024-05-15T16:36:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.610+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286c067d30 con 0x7f286c101ba0 2024-05-15T16:36:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.611+0000 7f2868ff9640 1 -- 172.21.15.49:0/563875469 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2854009820 con 0x7f286c101ba0 2024-05-15T16:36:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.611+0000 7f2868ff9640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f285806e120 0x7f28580705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:07.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.612+0000 7f2868ff9640 1 -- 172.21.15.49:0/563875469 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f285408dc00 con 0x7f286c101ba0 2024-05-15T16:36:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.612+0000 7f286b7fe640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f285806e120 0x7f28580705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:07.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.612+0000 7f286b7fe640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f285806e120 0x7f28580705e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f285c037040 tx=0x7f285c0093f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:07.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.616+0000 7f2868ff9640 1 -- 172.21.15.49:0/563875469 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2854093020 con 0x7f286c101ba0 2024-05-15T16:36:07.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.844+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f286c04e210 con 0x7f286c101ba0 2024-05-15T16:36:07.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.845+0000 7f2868ff9640 1 -- 172.21.15.49:0/563875469 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v36) v1 ==== 107+0+5215 (secure 0 0 0) 0x7f2854017090 con 0x7f286c101ba0 2024-05-15T16:36:07.846 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:07.846 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44891,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/755343004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":755343004},{"type":"v1","addr":"172.21.15.144:6827","nonce":755343004}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:33:51.822001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":1,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:07.847 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 29 2024-05-15T16:36:07.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f285806e120 msgr2=0x7f28580705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f285806e120 0x7f28580705e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f285c037040 tx=0x7f285c0093f0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101ba0 msgr2=0x7f286c196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101ba0 0x7f286c196680 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f285400d6d0 tx=0x7f285400db90 comp rx=0 tx=0).stop 2024-05-15T16:36:07.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 shutdown_connections 2024-05-15T16:36:07.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f285806e120 0x7f28580705e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f286c101ba0 0x7f286c196680 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 --2- 172.21.15.49:0/563875469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286c101280 0x7f286c196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 >> 172.21.15.49:0/563875469 conn(0x7f286c075c80 msgr2=0x7f286c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:07.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 shutdown_connections 2024-05-15T16:36:07.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.847+0000 7f2871fd7640 1 -- 172.21.15.49:0/563875469 wait complete. 2024-05-15T16:36:07.859 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-05-15T16:36:07.860 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-05-15T16:36:07.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.963+0000 7ff5de069640 1 -- 172.21.15.49:0/104720233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 msgr2=0x7ff5d8100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.963+0000 7ff5de069640 1 --2- 172.21.15.49:0/104720233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 0x7ff5d8100780 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009880 tx=0x7ff5c802bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.963+0000 7ff5de069640 1 -- 172.21.15.49:0/104720233 shutdown_connections 2024-05-15T16:36:07.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.963+0000 7ff5de069640 1 --2- 172.21.15.49:0/104720233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d8100d50 0x7ff5d8104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.963+0000 7ff5de069640 1 --2- 172.21.15.49:0/104720233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 0x7ff5d8100780 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.963+0000 7ff5de069640 1 -- 172.21.15.49:0/104720233 >> 172.21.15.49:0/104720233 conn(0x7ff5d80fbc20 msgr2=0x7ff5d80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.963+0000 7ff5de069640 1 -- 172.21.15.49:0/104720233 shutdown_connections 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.963+0000 7ff5de069640 1 -- 172.21.15.49:0/104720233 wait complete. 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5de069640 1 Processor -- start 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5de069640 1 -- start start 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5de069640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 0x7ff5d81962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5de069640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d8196820 0x7ff5d819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5de069640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d8196e20 con 0x7ff5d81003a0 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5de069640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d8196f90 con 0x7ff5d8196820 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 0x7ff5d81962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 0x7ff5d81962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54036/0 (socket says 172.21.15.49:54036) 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5d77fe640 1 -- 172.21.15.49:0/1231441777 learned_addr learned my addr 172.21.15.49:0/1231441777 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.964+0000 7ff5d6ffd640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d8196820 0x7ff5d819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.965+0000 7ff5d77fe640 1 -- 172.21.15.49:0/1231441777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d8196820 msgr2=0x7ff5d819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.965+0000 7ff5d77fe640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d8196820 0x7ff5d819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.965+0000 7ff5d77fe640 1 -- 172.21.15.49:0/1231441777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8009560 con 0x7ff5d81003a0 2024-05-15T16:36:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.965+0000 7ff5d77fe640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 0x7ff5d81962e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8036040 tx=0x7ff5c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.965+0000 7ff5d4ff9640 1 -- 172.21.15.49:0/1231441777 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c80035f0 con 0x7ff5d81003a0 2024-05-15T16:36:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.965+0000 7ff5d4ff9640 1 -- 172.21.15.49:0/1231441777 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5c8003c10 con 0x7ff5d81003a0 2024-05-15T16:36:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.965+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d819b160 con 0x7ff5d81003a0 2024-05-15T16:36:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.965+0000 7ff5d4ff9640 1 -- 172.21.15.49:0/1231441777 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c8034440 con 0x7ff5d81003a0 2024-05-15T16:36:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.965+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d819b6c0 con 0x7ff5d81003a0 2024-05-15T16:36:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.966+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d8101870 con 0x7ff5d81003a0 2024-05-15T16:36:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.967+0000 7ff5d4ff9640 1 -- 172.21.15.49:0/1231441777 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff5c8004f60 con 0x7ff5d81003a0 2024-05-15T16:36:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.967+0000 7ff5d4ff9640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff5c006e170 0x7ff5c0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.967+0000 7ff5d4ff9640 1 -- 172.21.15.49:0/1231441777 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff5c80b05b0 con 0x7ff5d81003a0 2024-05-15T16:36:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.967+0000 7ff5d6ffd640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff5c006e170 0x7ff5c0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.968+0000 7ff5d6ffd640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff5c006e170 0x7ff5c0070630 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80696f0 tx=0x7ff5cc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:07.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:07.972+0000 7ff5d4ff9640 1 -- 172.21.15.49:0/1231441777 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff5c8079890 con 0x7ff5d81003a0 2024-05-15T16:36:08.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.195+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7ff5d819b8e0 con 0x7ff5d81003a0 2024-05-15T16:36:08.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.196+0000 7ff5d4ff9640 1 -- 172.21.15.49:0/1231441777 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v36) v1 ==== 107+0+5217 (secure 0 0 0) 0x7ff5c807cf50 con 0x7ff5d81003a0 2024-05-15T16:36:08.197 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:08.197 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44891,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/755343004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":755343004},{"type":"v1","addr":"172.21.15.144:6827","nonce":755343004}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:33:51.834694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":1,"incarnation":29,"state":"up:resolve","state_seq":458,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:08.198 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 30 2024-05-15T16:36:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:07 smithi144 ceph-mon[86635]: pgmap v1176: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:07 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1017864436' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-15T16:36:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:07 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2495643940' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-15T16:36:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:07 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/563875469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-15T16:36:08.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff5c006e170 msgr2=0x7ff5c0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff5c006e170 0x7ff5c0070630 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80696f0 tx=0x7ff5cc008040 comp rx=0 tx=0).stop 2024-05-15T16:36:08.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 msgr2=0x7ff5d81962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 0x7ff5d81962e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8036040 tx=0x7ff5c8004040 comp rx=0 tx=0).stop 2024-05-15T16:36:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 shutdown_connections 2024-05-15T16:36:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7ff5c006e170 0x7ff5c0070630 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d8196820 0x7ff5d819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 --2- 172.21.15.49:0/1231441777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5d81003a0 0x7ff5d81962e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 >> 172.21.15.49:0/1231441777 conn(0x7ff5d80fbc20 msgr2=0x7ff5d80fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 shutdown_connections 2024-05-15T16:36:08.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.198+0000 7ff5de069640 1 -- 172.21.15.49:0/1231441777 wait complete. 2024-05-15T16:36:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:07 smithi049 ceph-mon[112142]: pgmap v1176: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:07 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1017864436' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-15T16:36:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:07 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2495643940' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-15T16:36:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:07 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/563875469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-15T16:36:08.211 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-05-15T16:36:08.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.315+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/1138188752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e00ffd00 msgr2=0x7fc3e0103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.315+0000 7fc3e4b4c640 1 --2- 172.21.15.49:0/1138188752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e00ffd00 0x7fc3e0103c90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009880 tx=0x7fc3d002bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.315+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/1138188752 shutdown_connections 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.315+0000 7fc3e4b4c640 1 --2- 172.21.15.49:0/1138188752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e00ffd00 0x7fc3e0103c90 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.315+0000 7fc3e4b4c640 1 --2- 172.21.15.49:0/1138188752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e00ff350 0x7fc3e00ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.315+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/1138188752 >> 172.21.15.49:0/1138188752 conn(0x7fc3e0074b50 msgr2=0x7fc3e0074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.315+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/1138188752 shutdown_connections 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.315+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/1138188752 wait complete. 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3e4b4c640 1 Processor -- start 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3e4b4c640 1 -- start start 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3e4b4c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e00ff350 0x7fc3e0197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3e4b4c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e0197580 0x7fc3e01979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3e4b4c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e01a00b0 con 0x7fc3e0197580 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3e4b4c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e01a01f0 con 0x7fc3e00ff350 2024-05-15T16:36:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3ddd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e0197580 0x7fc3e01979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3ddd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e0197580 0x7fc3e01979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54044/0 (socket says 172.21.15.49:54044) 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3ddd9b640 1 -- 172.21.15.49:0/4119125863 learned_addr learned my addr 172.21.15.49:0/4119125863 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3de59c640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e00ff350 0x7fc3e0197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3ddd9b640 1 -- 172.21.15.49:0/4119125863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e00ff350 msgr2=0x7fc3e0197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3ddd9b640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e00ff350 0x7fc3e0197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.316+0000 7fc3ddd9b640 1 -- 172.21.15.49:0/4119125863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0009560 con 0x7fc3e0197580 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.317+0000 7fc3ddd9b640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e0197580 0x7fc3e01979e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0006010 tx=0x7fc3d0004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.317+0000 7fc3d77fe640 1 -- 172.21.15.49:0/4119125863 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3d0004420 con 0x7fc3e0197580 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.317+0000 7fc3d77fe640 1 -- 172.21.15.49:0/4119125863 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3d0003ab0 con 0x7fc3e0197580 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.317+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e01a0330 con 0x7fc3e0197580 2024-05-15T16:36:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.317+0000 7fc3d77fe640 1 -- 172.21.15.49:0/4119125863 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3d0005520 con 0x7fc3e0197580 2024-05-15T16:36:08.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.317+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e01a0830 con 0x7fc3e0197580 2024-05-15T16:36:08.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.318+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e0100770 con 0x7fc3e0197580 2024-05-15T16:36:08.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.318+0000 7fc3d77fe640 1 -- 172.21.15.49:0/4119125863 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc3d0003c20 con 0x7fc3e0197580 2024-05-15T16:36:08.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.319+0000 7fc3d77fe640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3cc06e170 0x7fc3cc070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:08.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.319+0000 7fc3d77fe640 1 -- 172.21.15.49:0/4119125863 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc3d00b0270 con 0x7fc3e0197580 2024-05-15T16:36:08.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.319+0000 7fc3de59c640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3cc06e170 0x7fc3cc070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:08.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.319+0000 7fc3de59c640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3cc06e170 0x7fc3cc070630 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009bd0 tx=0x7fc3c8005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:08.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.323+0000 7fc3d77fe640 1 -- 172.21.15.49:0/4119125863 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc3d0079680 con 0x7fc3e0197580 2024-05-15T16:36:08.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.546+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fc3e01075a0 con 0x7fc3e0197580 2024-05-15T16:36:08.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.546+0000 7fc3d77fe640 1 -- 172.21.15.49:0/4119125863 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v36) v1 ==== 107+0+5220 (secure 0 0 0) 0x7fc3d007cd40 con 0x7fc3e0197580 2024-05-15T16:36:08.548 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:08.548 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44891,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/755343004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":755343004},{"type":"v1","addr":"172.21.15.144:6827","nonce":755343004}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:34:01.599864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":0,"incarnation":29,"state":"up:resolve","state_seq":465,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":1,"incarnation":29,"state":"up:resolve","state_seq":458,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 31 2024-05-15T16:36:08.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.548+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3cc06e170 msgr2=0x7fc3cc070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.548+0000 7fc3e4b4c640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3cc06e170 0x7fc3cc070630 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009bd0 tx=0x7fc3c8005f90 comp rx=0 tx=0).stop 2024-05-15T16:36:08.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.548+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e0197580 msgr2=0x7fc3e01979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.548+0000 7fc3e4b4c640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e0197580 0x7fc3e01979e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0006010 tx=0x7fc3d0004a10 comp rx=0 tx=0).stop 2024-05-15T16:36:08.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.549+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 shutdown_connections 2024-05-15T16:36:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.549+0000 7fc3e4b4c640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fc3cc06e170 0x7fc3cc070630 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.549+0000 7fc3e4b4c640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3e0197580 0x7fc3e01979e0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.549+0000 7fc3e4b4c640 1 --2- 172.21.15.49:0/4119125863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3e00ff350 0x7fc3e0197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.549+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 >> 172.21.15.49:0/4119125863 conn(0x7fc3e0074b50 msgr2=0x7fc3e00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.549+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 shutdown_connections 2024-05-15T16:36:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.549+0000 7fc3e4b4c640 1 -- 172.21.15.49:0/4119125863 wait complete. 2024-05-15T16:36:08.562 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.667+0000 7f1545b11640 1 -- 172.21.15.49:0/1438332342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540100d50 msgr2=0x7f1540104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.667+0000 7f1545b11640 1 --2- 172.21.15.49:0/1438332342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540100d50 0x7f1540104d90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1534009880 tx=0x7f153402bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.667+0000 7f1545b11640 1 -- 172.21.15.49:0/1438332342 shutdown_connections 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.667+0000 7f1545b11640 1 --2- 172.21.15.49:0/1438332342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540100d50 0x7f1540104d90 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.667+0000 7f1545b11640 1 --2- 172.21.15.49:0/1438332342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15401003a0 0x7f1540100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.667+0000 7f1545b11640 1 -- 172.21.15.49:0/1438332342 >> 172.21.15.49:0/1438332342 conn(0x7f15400fbc20 msgr2=0x7f15400fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.667+0000 7f1545b11640 1 -- 172.21.15.49:0/1438332342 shutdown_connections 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.667+0000 7f1545b11640 1 -- 172.21.15.49:0/1438332342 wait complete. 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f1545b11640 1 Processor -- start 2024-05-15T16:36:08.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f1545b11640 1 -- start start 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f1545b11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15401003a0 0x7f15401961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f1545b11640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1540196720 0x7f154019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f1545b11640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1540196c90 con 0x7f15401003a0 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f1545b11640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1540196e00 con 0x7f1540196720 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f153f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15401003a0 0x7f15401961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f153f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15401003a0 0x7f15401961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54056/0 (socket says 172.21.15.49:54056) 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f153f7fe640 1 -- 172.21.15.49:0/786766910 learned_addr learned my addr 172.21.15.49:0/786766910 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f153effd640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1540196720 0x7f154019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f153f7fe640 1 -- 172.21.15.49:0/786766910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1540196720 msgr2=0x7f154019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f153f7fe640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1540196720 0x7f154019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f153f7fe640 1 -- 172.21.15.49:0/786766910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1534009560 con 0x7f15401003a0 2024-05-15T16:36:08.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.668+0000 7f153effd640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1540196720 0x7f154019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:08.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.669+0000 7f153f7fe640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15401003a0 0x7f15401961e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f153000cc60 tx=0x7f153000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:08.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.669+0000 7f153cff9640 1 -- 172.21.15.49:0/786766910 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f153000f450 con 0x7f15401003a0 2024-05-15T16:36:08.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.669+0000 7f153cff9640 1 -- 172.21.15.49:0/786766910 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f153000fa70 con 0x7f15401003a0 2024-05-15T16:36:08.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.669+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154019b010 con 0x7f15401003a0 2024-05-15T16:36:08.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.669+0000 7f153cff9640 1 -- 172.21.15.49:0/786766910 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1530018b80 con 0x7f15401003a0 2024-05-15T16:36:08.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.669+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154019b520 con 0x7f15401003a0 2024-05-15T16:36:08.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.670+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1540101870 con 0x7f15401003a0 2024-05-15T16:36:08.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.670+0000 7f153cff9640 1 -- 172.21.15.49:0/786766910 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1530018ce0 con 0x7f15401003a0 2024-05-15T16:36:08.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.671+0000 7f153cff9640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f152806e170 0x7f1528070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:08.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.671+0000 7f153effd640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f152806e170 0x7f1528070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:08.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.671+0000 7f153cff9640 1 -- 172.21.15.49:0/786766910 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f153008e680 con 0x7f15401003a0 2024-05-15T16:36:08.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.672+0000 7f153effd640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f152806e170 0x7f1528070630 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1534036040 tx=0x7f1534000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:08.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.676+0000 7f153cff9640 1 -- 172.21.15.49:0/786766910 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1530057a90 con 0x7f15401003a0 2024-05-15T16:36:08.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.899+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f15401086a0 con 0x7f15401003a0 2024-05-15T16:36:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.900+0000 7f153cff9640 1 -- 172.21.15.49:0/786766910 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v36) v1 ==== 107+0+5224 (secure 0 0 0) 0x7f153005b150 con 0x7f15401003a0 2024-05-15T16:36:08.901 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44891,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/755343004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":755343004},{"type":"v1","addr":"172.21.15.144:6827","nonce":755343004}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:34:01.820556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":0,"incarnation":29,"state":"up:reconnect","state_seq":466,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":1,"incarnation":29,"state":"up:reconnect","state_seq":461,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 32 2024-05-15T16:36:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f152806e170 msgr2=0x7f1528070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f152806e170 0x7f1528070630 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1534036040 tx=0x7f1534000bc0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15401003a0 msgr2=0x7f15401961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15401003a0 0x7f15401961e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f153000cc60 tx=0x7f153000a620 comp rx=0 tx=0).stop 2024-05-15T16:36:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 shutdown_connections 2024-05-15T16:36:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f152806e170 0x7f1528070630 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1540196720 0x7f154019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 --2- 172.21.15.49:0/786766910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15401003a0 0x7f15401961e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:08.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 >> 172.21.15.49:0/786766910 conn(0x7f15400fbc20 msgr2=0x7f15400fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:08.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 shutdown_connections 2024-05-15T16:36:08.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:08.902+0000 7f1545b11640 1 -- 172.21.15.49:0/786766910 wait complete. 2024-05-15T16:36:08.915 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.019+0000 7f5c6137a640 1 -- 172.21.15.49:0/4277867015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101320 msgr2=0x7f5c5c101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.019+0000 7f5c6137a640 1 --2- 172.21.15.49:0/4277867015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101320 0x7f5c5c101700 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009880 tx=0x7f5c4c02bd70 comp rx=0 tx=0).stop 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.019+0000 7f5c6137a640 1 -- 172.21.15.49:0/4277867015 shutdown_connections 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.019+0000 7f5c6137a640 1 --2- 172.21.15.49:0/4277867015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101c40 0x7f5c5c10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.019+0000 7f5c6137a640 1 --2- 172.21.15.49:0/4277867015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101320 0x7f5c5c101700 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.019+0000 7f5c6137a640 1 -- 172.21.15.49:0/4277867015 >> 172.21.15.49:0/4277867015 conn(0x7f5c5c0fab40 msgr2=0x7f5c5c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.019+0000 7f5c6137a640 1 -- 172.21.15.49:0/4277867015 shutdown_connections 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.019+0000 7f5c6137a640 1 -- 172.21.15.49:0/4277867015 wait complete. 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c6137a640 1 Processor -- start 2024-05-15T16:36:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c6137a640 1 -- start start 2024-05-15T16:36:09.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c6137a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101320 0x7f5c5c1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:09.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c6137a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101c40 0x7f5c5c196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:09.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c6137a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c196e90 con 0x7f5c5c101c40 2024-05-15T16:36:09.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c6137a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c19aba0 con 0x7f5c5c101320 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c5a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101c40 0x7f5c5c196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c5a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101c40 0x7f5c5c196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54074/0 (socket says 172.21.15.49:54074) 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c5a7fc640 1 -- 172.21.15.49:0/2430387706 learned_addr learned my addr 172.21.15.49:0/2430387706 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c5affd640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101320 0x7f5c5c1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c5a7fc640 1 -- 172.21.15.49:0/2430387706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101320 msgr2=0x7f5c5c1962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c5a7fc640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101320 0x7f5c5c1962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c5a7fc640 1 -- 172.21.15.49:0/2430387706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c009560 con 0x7f5c5c101c40 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.020+0000 7f5c5affd640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101320 0x7f5c5c1962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.021+0000 7f5c5a7fc640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101c40 0x7f5c5c196800 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400b370 tx=0x7f5c4400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.021+0000 7f5c53fff640 1 -- 172.21.15.49:0/2430387706 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c44011800 con 0x7f5c5c101c40 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.021+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5c19ae80 con 0x7f5c5c101c40 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.021+0000 7f5c53fff640 1 -- 172.21.15.49:0/2430387706 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c44011e20 con 0x7f5c5c101c40 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.021+0000 7f5c53fff640 1 -- 172.21.15.49:0/2430387706 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c4400f2e0 con 0x7f5c5c101c40 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.021+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5c19b3e0 con 0x7f5c5c101c40 2024-05-15T16:36:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.022+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5c068f10 con 0x7f5c5c101c40 2024-05-15T16:36:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.022+0000 7f5c53fff640 1 -- 172.21.15.49:0/2430387706 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5c44011960 con 0x7f5c5c101c40 2024-05-15T16:36:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.023+0000 7f5c53fff640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e170 0x7f5c48070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.023+0000 7f5c53fff640 1 -- 172.21.15.49:0/2430387706 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5c4408dd40 con 0x7f5c5c101c40 2024-05-15T16:36:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.023+0000 7f5c5affd640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e170 0x7f5c48070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.023+0000 7f5c5affd640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e170 0x7f5c48070630 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02ca80 tx=0x7f5c4c02cbd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:09.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.027+0000 7f5c53fff640 1 -- 172.21.15.49:0/2430387706 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5c440570a0 con 0x7f5c5c101c40 2024-05-15T16:36:09.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:08 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1231441777' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-15T16:36:09.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:08 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4119125863' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-15T16:36:09.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:08 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/786766910' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-15T16:36:09.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:08 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1231441777' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-15T16:36:09.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:08 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4119125863' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-15T16:36:09.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:08 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/786766910' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-15T16:36:09.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.256+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f5c5c04e210 con 0x7f5c5c101c40 2024-05-15T16:36:09.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.256+0000 7f5c53fff640 1 -- 172.21.15.49:0/2430387706 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v36) v1 ==== 107+0+5218 (secure 0 0 0) 0x7f5c4405a760 con 0x7f5c5c101c40 2024-05-15T16:36:09.258 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:09.258 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44891,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/755343004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":755343004},{"type":"v1","addr":"172.21.15.144:6827","nonce":755343004}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:34:02.780028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":0,"incarnation":29,"state":"up:rejoin","state_seq":467,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":1,"incarnation":29,"state":"up:rejoin","state_seq":462,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-15T16:36:09.258 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 33 2024-05-15T16:36:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.258+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e170 msgr2=0x7f5c48070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.258+0000 7f5c6137a640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e170 0x7f5c48070630 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02ca80 tx=0x7f5c4c02cbd0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.258+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101c40 msgr2=0x7f5c5c196800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.258+0000 7f5c6137a640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101c40 0x7f5c5c196800 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400b370 tx=0x7f5c4400b830 comp rx=0 tx=0).stop 2024-05-15T16:36:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.258+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 shutdown_connections 2024-05-15T16:36:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.258+0000 7f5c6137a640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f5c4806e170 0x7f5c48070630 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.258+0000 7f5c6137a640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c5c101c40 0x7f5c5c196800 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.258+0000 7f5c6137a640 1 --2- 172.21.15.49:0/2430387706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101320 0x7f5c5c1962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.258+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 >> 172.21.15.49:0/2430387706 conn(0x7f5c5c0fab40 msgr2=0x7f5c5c102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:09.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.259+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 shutdown_connections 2024-05-15T16:36:09.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.259+0000 7f5c6137a640 1 -- 172.21.15.49:0/2430387706 wait complete. 2024-05-15T16:36:09.271 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-05-15T16:36:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.376+0000 7fab66e0b640 1 -- 172.21.15.49:0/3539915499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab60068070 msgr2=0x7fab60104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.376+0000 7fab66e0b640 1 --2- 172.21.15.49:0/3539915499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab60068070 0x7fab60104db0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fab50009880 tx=0x7fab5002bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.376+0000 7fab66e0b640 1 -- 172.21.15.49:0/3539915499 shutdown_connections 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.376+0000 7fab66e0b640 1 --2- 172.21.15.49:0/3539915499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab60068070 0x7fab60104db0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.376+0000 7fab66e0b640 1 --2- 172.21.15.49:0/3539915499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab600676c0 0x7fab60067aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.376+0000 7fab66e0b640 1 -- 172.21.15.49:0/3539915499 >> 172.21.15.49:0/3539915499 conn(0x7fab60074b50 msgr2=0x7fab60074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.376+0000 7fab66e0b640 1 -- 172.21.15.49:0/3539915499 shutdown_connections 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.376+0000 7fab66e0b640 1 -- 172.21.15.49:0/3539915499 wait complete. 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab66e0b640 1 Processor -- start 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab66e0b640 1 -- start start 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab66e0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab600676c0 0x7fab60196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab66e0b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60196780 0x7fab6019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab66e0b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab60196d80 con 0x7fab600676c0 2024-05-15T16:36:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab66e0b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab60196ef0 con 0x7fab60196780 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab64ba7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab600676c0 0x7fab60196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab64ba7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab600676c0 0x7fab60196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54090/0 (socket says 172.21.15.49:54090) 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab64ba7640 1 -- 172.21.15.49:0/2309407495 learned_addr learned my addr 172.21.15.49:0/2309407495 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab5ffff640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60196780 0x7fab6019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab64ba7640 1 -- 172.21.15.49:0/2309407495 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60196780 msgr2=0x7fab6019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab64ba7640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60196780 0x7fab6019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.377+0000 7fab64ba7640 1 -- 172.21.15.49:0/2309407495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50009560 con 0x7fab600676c0 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.378+0000 7fab64ba7640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab600676c0 0x7fab60196240 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fab4800d720 tx=0x7fab4800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.378+0000 7fab5dffb640 1 -- 172.21.15.49:0/2309407495 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab480051f0 con 0x7fab600676c0 2024-05-15T16:36:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.378+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6019b120 con 0x7fab600676c0 2024-05-15T16:36:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.378+0000 7fab5dffb640 1 -- 172.21.15.49:0/2309407495 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab48009ce0 con 0x7fab600676c0 2024-05-15T16:36:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.378+0000 7fab5dffb640 1 -- 172.21.15.49:0/2309407495 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab48010460 con 0x7fab600676c0 2024-05-15T16:36:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.378+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6019b680 con 0x7fab600676c0 2024-05-15T16:36:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.379+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab60101890 con 0x7fab600676c0 2024-05-15T16:36:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.380+0000 7fab5dffb640 1 -- 172.21.15.49:0/2309407495 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fab48005350 con 0x7fab600676c0 2024-05-15T16:36:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.380+0000 7fab5dffb640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fab4c06e170 0x7fab4c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.380+0000 7fab5dffb640 1 -- 172.21.15.49:0/2309407495 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fab4808dc70 con 0x7fab600676c0 2024-05-15T16:36:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.380+0000 7fab5ffff640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fab4c06e170 0x7fab4c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.380+0000 7fab5ffff640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fab4c06e170 0x7fab4c070630 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fab50000c00 tx=0x7fab50005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:09.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.385+0000 7fab5dffb640 1 -- 172.21.15.49:0/2309407495 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fab48057000 con 0x7fab600676c0 2024-05-15T16:36:09.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.607+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fab6019b900 con 0x7fab600676c0 2024-05-15T16:36:09.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.607+0000 7fab5dffb640 1 -- 172.21.15.49:0/2309407495 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v36) v1 ==== 107+0+5233 (secure 0 0 0) 0x7fab48017090 con 0x7fab600676c0 2024-05-15T16:36:09.609 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:09.609 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44891,"name":"cephfs.smithi144.kziuke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/755343004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":755343004},{"type":"v1","addr":"172.21.15.144:6827","nonce":755343004}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:34:04.781255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":0,"incarnation":29,"state":"up:active","state_seq":468,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":1,"incarnation":29,"state":"up:active","state_seq":463,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34420,"qdb_cluster":[34420,34424]},"id":1}]} 2024-05-15T16:36:09.609 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 34 2024-05-15T16:36:09.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.609+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fab4c06e170 msgr2=0x7fab4c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.609+0000 7fab66e0b640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fab4c06e170 0x7fab4c070630 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fab50000c00 tx=0x7fab50005fd0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.610+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab600676c0 msgr2=0x7fab60196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.610+0000 7fab66e0b640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab600676c0 0x7fab60196240 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fab4800d720 tx=0x7fab4800dbe0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.610+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 shutdown_connections 2024-05-15T16:36:09.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.610+0000 7fab66e0b640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7fab4c06e170 0x7fab4c070630 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.610+0000 7fab66e0b640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60196780 0x7fab6019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.610+0000 7fab66e0b640 1 --2- 172.21.15.49:0/2309407495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fab600676c0 0x7fab60196240 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.610+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 >> 172.21.15.49:0/2309407495 conn(0x7fab60074b50 msgr2=0x7fab60103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:09.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.610+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 shutdown_connections 2024-05-15T16:36:09.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.610+0000 7fab66e0b640 1 -- 172.21.15.49:0/2309407495 wait complete. 2024-05-15T16:36:09.625 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-05-15T16:36:09.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.729+0000 7f4c3aba7640 1 -- 172.21.15.49:0/1192594571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 msgr2=0x7f4c34101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.729+0000 7f4c3aba7640 1 --2- 172.21.15.49:0/1192594571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 0x7f4c34101700 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009c20 tx=0x7f4c2402c230 comp rx=0 tx=0).stop 2024-05-15T16:36:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.729+0000 7f4c3aba7640 1 -- 172.21.15.49:0/1192594571 shutdown_connections 2024-05-15T16:36:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.729+0000 7f4c3aba7640 1 --2- 172.21.15.49:0/1192594571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c34101c40 0x7f4c3410a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.729+0000 7f4c3aba7640 1 --2- 172.21.15.49:0/1192594571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 0x7f4c34101700 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.729+0000 7f4c3aba7640 1 -- 172.21.15.49:0/1192594571 >> 172.21.15.49:0/1192594571 conn(0x7f4c340fab40 msgr2=0x7f4c340fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.729+0000 7f4c3aba7640 1 -- 172.21.15.49:0/1192594571 shutdown_connections 2024-05-15T16:36:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.729+0000 7f4c3aba7640 1 -- 172.21.15.49:0/1192594571 wait complete. 2024-05-15T16:36:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c3aba7640 1 Processor -- start 2024-05-15T16:36:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c3aba7640 1 -- start start 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c3aba7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 0x7f4c34196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c3aba7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c34101c40 0x7f4c34196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c3aba7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34196d40 con 0x7f4c34101320 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c3aba7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34196eb0 con 0x7f4c34101c40 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c38943640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 0x7f4c34196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c38943640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 0x7f4c34196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54102/0 (socket says 172.21.15.49:54102) 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c38943640 1 -- 172.21.15.49:0/3344258650 learned_addr learned my addr 172.21.15.49:0/3344258650 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c33fff640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c34101c40 0x7f4c34196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c38943640 1 -- 172.21.15.49:0/3344258650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c34101c40 msgr2=0x7f4c34196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c38943640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c34101c40 0x7f4c34196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c38943640 1 -- 172.21.15.49:0/3344258650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c24009900 con 0x7f4c34101320 2024-05-15T16:36:09.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c33fff640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c34101c40 0x7f4c34196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T16:36:09.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.730+0000 7f4c38943640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 0x7f4c34196200 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4c24037040 tx=0x7f4c24034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:09.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.731+0000 7f4c31ffb640 1 -- 172.21.15.49:0/3344258650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c240326b0 con 0x7f4c34101320 2024-05-15T16:36:09.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.731+0000 7f4c31ffb640 1 -- 172.21.15.49:0/3344258650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c24032e60 con 0x7f4c34101320 2024-05-15T16:36:09.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.731+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3419abd0 con 0x7f4c34101320 2024-05-15T16:36:09.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.731+0000 7f4c31ffb640 1 -- 172.21.15.49:0/3344258650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c24004890 con 0x7f4c34101320 2024-05-15T16:36:09.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.731+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3419b0d0 con 0x7f4c34101320 2024-05-15T16:36:09.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.732+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c34068f10 con 0x7f4c34101320 2024-05-15T16:36:09.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.732+0000 7f4c31ffb640 1 -- 172.21.15.49:0/3344258650 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4c2403b040 con 0x7f4c34101320 2024-05-15T16:36:09.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.733+0000 7f4c31ffb640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c2006e120 0x7f4c200705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:09.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.733+0000 7f4c33fff640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c2006e120 0x7f4c200705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:09.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.733+0000 7f4c31ffb640 1 -- 172.21.15.49:0/3344258650 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4c240b0e80 con 0x7f4c34101320 2024-05-15T16:36:09.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.733+0000 7f4c33fff640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c2006e120 0x7f4c200705e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4c34197700 tx=0x7f4c1c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:09.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.738+0000 7f4c31ffb640 1 -- 172.21.15.49:0/3344258650 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4c2407a160 con 0x7f4c34101320 2024-05-15T16:36:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.961+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f4c34104ae0 con 0x7f4c34101320 2024-05-15T16:36:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.961+0000 7f4c31ffb640 1 -- 172.21.15.49:0/3344258650 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v36) v1 ==== 107+0+5248 (secure 0 0 0) 0x7f4c2407d820 con 0x7f4c34101320 2024-05-15T16:36:09.963 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:09.963 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T15:50:40.850889+0000","modified":"2024-05-15T16:34:04.786022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi049.abbagx","rank":0,"incarnation":29,"state":"up:active","state_seq":468,"addr":"172.21.15.49:6827/683477188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":683477188},{"type":"v1","addr":"172.21.15.49:6827","nonce":683477188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi049.gocadf","rank":1,"incarnation":29,"state":"up:active","state_seq":463,"addr":"172.21.15.49:6829/2031688662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2031688662},{"type":"v1","addr":"172.21.15.49:6829","nonce":2031688662}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44339":{"gid":44339,"name":"cephfs.smithi144.jgodzs","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6825/2706427362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2706427362},{"type":"v1","addr":"172.21.15.144:6825","nonce":2706427362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44891":{"gid":44891,"name":"cephfs.smithi144.kziuke","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6827/755343004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":755343004},{"type":"v1","addr":"172.21.15.144:6827","nonce":755343004}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34420,"qdb_cluster":[34424,34420]},"id":1}]} 2024-05-15T16:36:09.963 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 35 2024-05-15T16:36:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c2006e120 msgr2=0x7f4c200705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c2006e120 0x7f4c200705e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4c34197700 tx=0x7f4c1c005e30 comp rx=0 tx=0).stop 2024-05-15T16:36:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 msgr2=0x7f4c34196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 0x7f4c34196200 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4c24037040 tx=0x7f4c24034670 comp rx=0 tx=0).stop 2024-05-15T16:36:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 shutdown_connections 2024-05-15T16:36:09.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f4c2006e120 0x7f4c200705e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c34101c40 0x7f4c34196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 --2- 172.21.15.49:0/3344258650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c34101320 0x7f4c34196200 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 >> 172.21.15.49:0/3344258650 conn(0x7f4c340fab40 msgr2=0x7f4c34102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 shutdown_connections 2024-05-15T16:36:09.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:09.964+0000 7f4c3aba7640 1 -- 172.21.15.49:0/3344258650 wait complete. 2024-05-15T16:36:09.979 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-15T16:36:09.990 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-15T16:36:09.991 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T16:36:09.991 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-15T16:36:10.009 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T16:36:10.009 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-15T16:36:10.067 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-15T16:36:10.172 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:09 smithi049 ceph-mon[112142]: pgmap v1177: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:10.172 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:09 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2430387706' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-15T16:36:10.172 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:09 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/2309407495' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-15T16:36:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:09 smithi144 ceph-mon[86635]: pgmap v1177: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:09 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2430387706' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-15T16:36:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:09 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/2309407495' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-15T16:36:10.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.211+0000 7f222bbd7640 1 -- 172.21.15.49:0/2116037864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ffc60 msgr2=0x7f2224103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.211+0000 7f222bbd7640 1 --2- 172.21.15.49:0/2116037864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ffc60 0x7f2224103ca0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2214009880 tx=0x7f221402bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.211+0000 7f222bbd7640 1 -- 172.21.15.49:0/2116037864 shutdown_connections 2024-05-15T16:36:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.211+0000 7f222bbd7640 1 --2- 172.21.15.49:0/2116037864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ffc60 0x7f2224103ca0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.211+0000 7f222bbd7640 1 --2- 172.21.15.49:0/2116037864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22240ff2b0 0x7f22240ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.211+0000 7f222bbd7640 1 -- 172.21.15.49:0/2116037864 >> 172.21.15.49:0/2116037864 conn(0x7f2224075cf0 msgr2=0x7f22240fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.211+0000 7f222bbd7640 1 -- 172.21.15.49:0/2116037864 shutdown_connections 2024-05-15T16:36:10.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.211+0000 7f222bbd7640 1 -- 172.21.15.49:0/2116037864 wait complete. 2024-05-15T16:36:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f222bbd7640 1 Processor -- start 2024-05-15T16:36:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f222bbd7640 1 -- start start 2024-05-15T16:36:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f222bbd7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ff2b0 0x7f2224196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f222bbd7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2224197510 0x7f2224197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f222bbd7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22241a0040 con 0x7f22240ff2b0 2024-05-15T16:36:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f222bbd7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22241a0180 con 0x7f2224197510 2024-05-15T16:36:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f2229973640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ff2b0 0x7f2224196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f2229973640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ff2b0 0x7f2224196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54116/0 (socket says 172.21.15.49:54116) 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f2229973640 1 -- 172.21.15.49:0/3683734717 learned_addr learned my addr 172.21.15.49:0/3683734717 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f2229172640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2224197510 0x7f2224197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f2229973640 1 -- 172.21.15.49:0/3683734717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2224197510 msgr2=0x7f2224197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f2229973640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2224197510 0x7f2224197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.212+0000 7f2229973640 1 -- 172.21.15.49:0/3683734717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2214009560 con 0x7f22240ff2b0 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.213+0000 7f2229973640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ff2b0 0x7f2224196fd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f222000e9a0 tx=0x7f222000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.213+0000 7f221affd640 1 -- 172.21.15.49:0/3683734717 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f222000cd50 con 0x7f22240ff2b0 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.213+0000 7f221affd640 1 -- 172.21.15.49:0/3683734717 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f222000ceb0 con 0x7f22240ff2b0 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.213+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22241a0380 con 0x7f22240ff2b0 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.213+0000 7f221affd640 1 -- 172.21.15.49:0/3683734717 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2220010660 con 0x7f22240ff2b0 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.213+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22241a08b0 con 0x7f22240ff2b0 2024-05-15T16:36:10.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.214+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2224100780 con 0x7f22240ff2b0 2024-05-15T16:36:10.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.215+0000 7f221affd640 1 -- 172.21.15.49:0/3683734717 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f22200048a0 con 0x7f22240ff2b0 2024-05-15T16:36:10.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.215+0000 7f221affd640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f221006e170 0x7f2210070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:10.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.215+0000 7f2229172640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f221006e170 0x7f2210070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:10.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.215+0000 7f221affd640 1 -- 172.21.15.49:0/3683734717 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2220014070 con 0x7f22240ff2b0 2024-05-15T16:36:10.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.216+0000 7f2229172640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f221006e170 0x7f2210070630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2214009230 tx=0x7f2214005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:10.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.220+0000 7f221affd640 1 -- 172.21.15.49:0/3683734717 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f222005b4e0 con 0x7f22240ff2b0 2024-05-15T16:36:10.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.409+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f222404e210 con 0x7f22240ff2b0 2024-05-15T16:36:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.411+0000 7f221affd640 1 -- 172.21.15.49:0/3683734717 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16768 (secure 0 0 0) 0x7f222005b070 con 0x7f22240ff2b0 2024-05-15T16:36:10.412 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:10.412 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":87,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","created":"2024-05-15T15:43:35.648208+0000","modified":"2024-05-15T16:18:46.696180+0000","last_up_change":"2024-05-15T16:02:52.176866+0000","last_in_change":"2024-05-15T15:49:04.722203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T15:48:09.899234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-15T15:50:39.462287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-15T15:50:40.285269+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25d1eb06-bee2-48a6-8d86-30b5b3b348d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6803","nonce":1701263964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6805","nonce":1701263964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6809","nonce":1701263964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6807","nonce":1701263964}]},"public_addr":"172.21.15.49:6803/1701263964","cluster_addr":"172.21.15.49:6805/1701263964","heartbeat_back_addr":"172.21.15.49:6809/1701263964","heartbeat_front_addr":"172.21.15.49:6807/1701263964","state":["exists","up"]},{"osd":1,"uuid":"c0e717cf-ae5c-4fa4-a30d-d2235ff27315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6811","nonce":3720875621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6813","nonce":3720875621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6817","nonce":3720875621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6815","nonce":3720875621}]},"public_addr":"172.21.15.49:6811/3720875621","cluster_addr":"172.21.15.49:6813/3720875621","heartbeat_back_addr":"172.21.15.49:6817/3720875621","heartbeat_front_addr":"172.21.15.49:6815/3720875621","state":["exists","up"]},{"osd":2,"uuid":"d7752daf-dd5b-49cc-abc2-ca953968e542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6819","nonce":3306250339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6821","nonce":3306250339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6825","nonce":3306250339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6823","nonce":3306250339}]},"public_addr":"172.21.15.49:6819/3306250339","cluster_addr":"172.21.15.49:6821/3306250339","heartbeat_back_addr":"172.21.15.49:6825/3306250339","heartbeat_front_addr":"172.21.15.49:6823/3306250339","state":["exists","up"]},{"osd":3,"uuid":"2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6801","nonce":4258967767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6803","nonce":4258967767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6807","nonce":4258967767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6805","nonce":4258967767}]},"public_addr":"172.21.15.144:6801/4258967767","cluster_addr":"172.21.15.144:6803/4258967767","heartbeat_back_addr":"172.21.15.144:6807/4258967767","heartbeat_front_addr":"172.21.15.144:6805/4258967767","state":["exists","up"]},{"osd":4,"uuid":"6659631f-da39-4a2c-9f75-e7b4a06b602f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6809","nonce":691212832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6811","nonce":691212832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6815","nonce":691212832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6813","nonce":691212832}]},"public_addr":"172.21.15.144:6809/691212832","cluster_addr":"172.21.15.144:6811/691212832","heartbeat_back_addr":"172.21.15.144:6815/691212832","heartbeat_front_addr":"172.21.15.144:6813/691212832","state":["exists","up"]},{"osd":5,"uuid":"51ea2d85-6cf7-4a96-8c74-612958083a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6817","nonce":255063313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6819","nonce":255063313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6823","nonce":255063313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6821","nonce":255063313}]},"public_addr":"172.21.15.144:6817/255063313","cluster_addr":"172.21.15.144:6819/255063313","heartbeat_back_addr":"172.21.15.144:6823/255063313","heartbeat_front_addr":"172.21.15.144:6821/255063313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:10.108689+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:37.458152+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:05.915664+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:30.194669+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:55.748301+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:49:22.635948+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6828/2512292397":"2024-05-16T16:02:56.077800+0000","172.21.15.144:6825/2552333341":"2024-05-16T16:02:56.077816+0000","172.21.15.49:6801/1448243308":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/75769186":"2024-05-16T15:44:39.328329+0000","172.21.15.144:0/2391504982":"2024-05-16T15:54:47.414074+0000","172.21.15.49:0/287650046":"2024-05-16T15:58:20.089143+0000","172.21.15.49:6801/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/644218315":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/1515356048":"2024-05-16T15:58:20.089143+0000","172.21.15.49:6800/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.144:0/2959282915":"2024-05-16T15:54:47.414074+0000","172.21.15.144:6828/177983287":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6800/1448243308":"2024-05-16T15:53:24.658899+0000","172.21.15.144:6829/177983287":"2024-05-16T15:54:47.414074+0000","172.21.15.49:0/1173274751":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/4024404689":"2024-05-16T15:44:07.361150+0000","172.21.15.144:0/3162546694":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6826/2475999224":"2024-05-16T15:50:56.872336+0000","172.21.15.49:0/3014161738":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6800/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/1166053004":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/2163344179":"2024-05-16T15:44:07.361150+0000","172.21.15.144:0/2251902137":"2024-05-16T15:54:47.414074+0000","172.21.15.49:0/2464608729":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/4010845594":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6800/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/1904525014":"2024-05-16T15:53:24.658899+0000","172.21.15.49:6801/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.144:6824/2552333341":"2024-05-16T16:02:56.077816+0000","172.21.15.144:0/4014499308":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6801/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/2543812499":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/3452717560":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6827/2475999224":"2024-05-16T15:50:56.872336+0000","172.21.15.49:6829/2512292397":"2024-05-16T16:02:56.077800+0000","172.21.15.49:0/940476764":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/133547858":"2024-05-16T15:58:20.089143+0000","172.21.15.49:0/1530195241":"2024-05-16T15:58:20.089143+0000","172.21.15.49:6800/70490983":"2024-05-16T15:58:20.089143+0000","172.21.15.144:0/1791319417":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6801/70490983":"2024-05-16T15:58:20.089143+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T16:36:10.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f221006e170 msgr2=0x7f2210070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f221006e170 0x7f2210070630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2214009230 tx=0x7f2214005fd0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ff2b0 msgr2=0x7f2224196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ff2b0 0x7f2224196fd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f222000e9a0 tx=0x7f222000eda0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 shutdown_connections 2024-05-15T16:36:10.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f221006e170 0x7f2210070630 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2224197510 0x7f2224197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 --2- 172.21.15.49:0/3683734717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22240ff2b0 0x7f2224196fd0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 >> 172.21.15.49:0/3683734717 conn(0x7f2224075cf0 msgr2=0x7f2224102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:10.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 shutdown_connections 2024-05-15T16:36:10.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.413+0000 7f222bbd7640 1 -- 172.21.15.49:0/3683734717 wait complete. 2024-05-15T16:36:10.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-15T15:48:09.899234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-15T15:50:39.462287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-15T15:50:40.285269+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-15T16:36:10.426 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-15T16:36:10.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.532+0000 7f1379ad1640 1 -- 172.21.15.49:0/962815331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 msgr2=0x7f13741007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.532+0000 7f1379ad1640 1 --2- 172.21.15.49:0/962815331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 0x7f13741007a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1364009880 tx=0x7f136402bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.532+0000 7f1379ad1640 1 -- 172.21.15.49:0/962815331 shutdown_connections 2024-05-15T16:36:10.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.532+0000 7f1379ad1640 1 --2- 172.21.15.49:0/962815331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374100d70 0x7f1374104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.532+0000 7f1379ad1640 1 --2- 172.21.15.49:0/962815331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 0x7f13741007a0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.532+0000 7f1379ad1640 1 -- 172.21.15.49:0/962815331 >> 172.21.15.49:0/962815331 conn(0x7f13740fbc80 msgr2=0x7f13740fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.533+0000 7f1379ad1640 1 -- 172.21.15.49:0/962815331 shutdown_connections 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.533+0000 7f1379ad1640 1 -- 172.21.15.49:0/962815331 wait complete. 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.533+0000 7f1379ad1640 1 Processor -- start 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.533+0000 7f1379ad1640 1 -- start start 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1379ad1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 0x7f1374196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1379ad1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374196740 0x7f137419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1379ad1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374196d40 con 0x7f13741003c0 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1379ad1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374196eb0 con 0x7f1374196740 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f13737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 0x7f1374196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f13737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 0x7f1374196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54120/0 (socket says 172.21.15.49:54120) 2024-05-15T16:36:10.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f13737fe640 1 -- 172.21.15.49:0/1800706584 learned_addr learned my addr 172.21.15.49:0/1800706584 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1372ffd640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374196740 0x7f137419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f13737fe640 1 -- 172.21.15.49:0/1800706584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374196740 msgr2=0x7f137419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f13737fe640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374196740 0x7f137419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f13737fe640 1 -- 172.21.15.49:0/1800706584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1364009560 con 0x7f13741003c0 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f13737fe640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 0x7f1374196200 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1364006010 tx=0x7f136402cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1370ff9640 1 -- 172.21.15.49:0/1800706584 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13640044a0 con 0x7f13741003c0 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1370ff9640 1 -- 172.21.15.49:0/1800706584 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1364003ab0 con 0x7f13741003c0 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137419b030 con 0x7f13741003c0 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1370ff9640 1 -- 172.21.15.49:0/1800706584 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1364005590 con 0x7f13741003c0 2024-05-15T16:36:10.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.534+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137419b590 con 0x7f13741003c0 2024-05-15T16:36:10.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.535+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1374101890 con 0x7f13741003c0 2024-05-15T16:36:10.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.536+0000 7f1370ff9640 1 -- 172.21.15.49:0/1800706584 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f13640035f0 con 0x7f13741003c0 2024-05-15T16:36:10.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.536+0000 7f1370ff9640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f135c06e180 0x7f135c070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:10.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.536+0000 7f1370ff9640 1 -- 172.21.15.49:0/1800706584 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f13640b05b0 con 0x7f13741003c0 2024-05-15T16:36:10.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.536+0000 7f1372ffd640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f135c06e180 0x7f135c070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:10.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.537+0000 7f1372ffd640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f135c06e180 0x7f135c070640 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1368005fd0 tx=0x7f1368005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:10.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.541+0000 7f1370ff9640 1 -- 172.21.15.49:0/1800706584 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1364079890 con 0x7f13741003c0 2024-05-15T16:36:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.725+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13741086c0 con 0x7f13741003c0 2024-05-15T16:36:10.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.725+0000 7f1370ff9640 1 -- 172.21.15.49:0/1800706584 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f136407cf50 con 0x7f13741003c0 2024-05-15T16:36:10.727 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-05-15T16:36:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f135c06e180 msgr2=0x7f135c070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f135c06e180 0x7f135c070640 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1368005fd0 tx=0x7f1368005850 comp rx=0 tx=0).stop 2024-05-15T16:36:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 msgr2=0x7f1374196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 0x7f1374196200 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1364006010 tx=0x7f136402cea0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 shutdown_connections 2024-05-15T16:36:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f135c06e180 0x7f135c070640 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374196740 0x7f137419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 --2- 172.21.15.49:0/1800706584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13741003c0 0x7f1374196200 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 >> 172.21.15.49:0/1800706584 conn(0x7f13740fbc80 msgr2=0x7f13740fc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 shutdown_connections 2024-05-15T16:36:10.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.727+0000 7f1379ad1640 1 -- 172.21.15.49:0/1800706584 wait complete. 2024-05-15T16:36:10.740 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.846+0000 7f2b65edb640 1 -- 172.21.15.49:0/1436360661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 msgr2=0x7f2b60101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.846+0000 7f2b65edb640 1 --2- 172.21.15.49:0/1436360661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 0x7f2b60101700 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009880 tx=0x7f2b5002bd70 comp rx=0 tx=0).stop 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.847+0000 7f2b65edb640 1 -- 172.21.15.49:0/1436360661 shutdown_connections 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.847+0000 7f2b65edb640 1 --2- 172.21.15.49:0/1436360661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b60101c40 0x7f2b6010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.847+0000 7f2b65edb640 1 --2- 172.21.15.49:0/1436360661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 0x7f2b60101700 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.847+0000 7f2b65edb640 1 -- 172.21.15.49:0/1436360661 >> 172.21.15.49:0/1436360661 conn(0x7f2b600fab40 msgr2=0x7f2b600fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.847+0000 7f2b65edb640 1 -- 172.21.15.49:0/1436360661 shutdown_connections 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.847+0000 7f2b65edb640 1 -- 172.21.15.49:0/1436360661 wait complete. 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.847+0000 7f2b65edb640 1 Processor -- start 2024-05-15T16:36:10.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.847+0000 7f2b65edb640 1 -- start start 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b65edb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 0x7f2b60196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b65edb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b60101c40 0x7f2b60196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b65edb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60196df0 con 0x7f2b60101320 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b65edb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6019aab0 con 0x7f2b60101c40 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 0x7f2b60196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 0x7f2b60196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54144/0 (socket says 172.21.15.49:54144) 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5f7fe640 1 -- 172.21.15.49:0/1522772217 learned_addr learned my addr 172.21.15.49:0/1522772217 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5effd640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b60101c40 0x7f2b60196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5f7fe640 1 -- 172.21.15.49:0/1522772217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b60101c40 msgr2=0x7f2b60196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5f7fe640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b60101c40 0x7f2b60196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5f7fe640 1 -- 172.21.15.49:0/1522772217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b50009560 con 0x7f2b60101320 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5f7fe640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 0x7f2b60196220 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2b5002c2b0 tx=0x7f2b5002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5cff9640 1 -- 172.21.15.49:0/1522772217 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b50004520 con 0x7f2b60101320 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5cff9640 1 -- 172.21.15.49:0/1522772217 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b50003ab0 con 0x7f2b60101320 2024-05-15T16:36:10.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b5cff9640 1 -- 172.21.15.49:0/1522772217 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b50005500 con 0x7f2b60101320 2024-05-15T16:36:10.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.848+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6019ad30 con 0x7f2b60101320 2024-05-15T16:36:10.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.849+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6019b240 con 0x7f2b60101320 2024-05-15T16:36:10.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.849+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b60068f10 con 0x7f2b60101320 2024-05-15T16:36:10.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.850+0000 7f2b5cff9640 1 -- 172.21.15.49:0/1522772217 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2b50003c20 con 0x7f2b60101320 2024-05-15T16:36:10.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.850+0000 7f2b5cff9640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b4806e170 0x7f2b48070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:10.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.850+0000 7f2b5cff9640 1 -- 172.21.15.49:0/1522772217 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2b500b0b90 con 0x7f2b60101320 2024-05-15T16:36:10.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.851+0000 7f2b5effd640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b4806e170 0x7f2b48070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:10.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.851+0000 7f2b5effd640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b4806e170 0x7f2b48070630 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2b60197720 tx=0x7f2b5400b2e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:10.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:10.855+0000 7f2b5cff9640 1 -- 172.21.15.49:0/1522772217 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2b50079f90 con 0x7f2b60101320 2024-05-15T16:36:11.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.042+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f2b6004e210 con 0x7f2b60101320 2024-05-15T16:36:11.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.043+0000 7f2b5cff9640 1 -- 172.21.15.49:0/1522772217 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2b50043020 con 0x7f2b60101320 2024-05-15T16:36:11.044 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-15T16:36:11.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b4806e170 msgr2=0x7f2b48070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b4806e170 0x7f2b48070630 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2b60197720 tx=0x7f2b5400b2e0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 msgr2=0x7f2b60196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 0x7f2b60196220 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2b5002c2b0 tx=0x7f2b5002c850 comp rx=0 tx=0).stop 2024-05-15T16:36:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 shutdown_connections 2024-05-15T16:36:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2b4806e170 0x7f2b48070630 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b60101c40 0x7f2b60196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 --2- 172.21.15.49:0/1522772217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b60101320 0x7f2b60196220 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 >> 172.21.15.49:0/1522772217 conn(0x7f2b600fab40 msgr2=0x7f2b60102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 shutdown_connections 2024-05-15T16:36:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.045+0000 7f2b65edb640 1 -- 172.21.15.49:0/1522772217 wait complete. 2024-05-15T16:36:11.056 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:10 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3344258650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-15T16:36:11.056 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:10 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3683734717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T16:36:11.056 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:10 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1800706584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T16:36:11.057 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 -- 172.21.15.49:0/4077895562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924100da0 msgr2=0x7f5924104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 --2- 172.21.15.49:0/4077895562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924100da0 0x7f5924104de0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f590c009880 tx=0x7f590c02bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 -- 172.21.15.49:0/4077895562 shutdown_connections 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 --2- 172.21.15.49:0/4077895562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924100da0 0x7f5924104de0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 --2- 172.21.15.49:0/4077895562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59241003f0 0x7f59241007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 -- 172.21.15.49:0/4077895562 >> 172.21.15.49:0/4077895562 conn(0x7f59240fbc70 msgr2=0x7f59240fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 -- 172.21.15.49:0/4077895562 shutdown_connections 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 -- 172.21.15.49:0/4077895562 wait complete. 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 Processor -- start 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 -- start start 2024-05-15T16:36:11.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59241003f0 0x7f59241962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924196810 0x7f592419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5924196e10 con 0x7f5924196810 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.165+0000 7f592a7b9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5924196f80 con 0x7f59241003f0 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f591bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924196810 0x7f592419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f591bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924196810 0x7f592419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54160/0 (socket says 172.21.15.49:54160) 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f591bfff640 1 -- 172.21.15.49:0/3057551835 learned_addr learned my addr 172.21.15.49:0/3057551835 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f5923fff640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59241003f0 0x7f59241962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f591bfff640 1 -- 172.21.15.49:0/3057551835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59241003f0 msgr2=0x7f59241962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f591bfff640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59241003f0 0x7f59241962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f591bfff640 1 -- 172.21.15.49:0/3057551835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c009560 con 0x7f5924196810 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f591bfff640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924196810 0x7f592419abb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f590c036040 tx=0x7f590c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f5921ffb640 1 -- 172.21.15.49:0/3057551835 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590c0035f0 con 0x7f5924196810 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f5921ffb640 1 -- 172.21.15.49:0/3057551835 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f590c003c10 con 0x7f5924196810 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592419b150 con 0x7f5924196810 2024-05-15T16:36:11.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f5921ffb640 1 -- 172.21.15.49:0/3057551835 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590c034440 con 0x7f5924196810 2024-05-15T16:36:11.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.166+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592419b6b0 con 0x7f5924196810 2024-05-15T16:36:11.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.167+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59241018c0 con 0x7f5924196810 2024-05-15T16:36:11.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.168+0000 7f5921ffb640 1 -- 172.21.15.49:0/3057551835 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f590c004f60 con 0x7f5924196810 2024-05-15T16:36:11.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.168+0000 7f5921ffb640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f591006e170 0x7f5910070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:11.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.168+0000 7f5923fff640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f591006e170 0x7f5910070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:11.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.168+0000 7f5921ffb640 1 -- 172.21.15.49:0/3057551835 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f590c0b0560 con 0x7f5924196810 2024-05-15T16:36:11.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.169+0000 7f5923fff640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f591006e170 0x7f5910070630 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5914009890 tx=0x7f5914008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:11.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.173+0000 7f5921ffb640 1 -- 172.21.15.49:0/3057551835 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f590c079840 con 0x7f5924196810 2024-05-15T16:36:11.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:10 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3344258650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-15T16:36:11.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:10 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3683734717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T16:36:11.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:10 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1800706584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T16:36:11.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.357+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f592419b8d0 con 0x7f5924196810 2024-05-15T16:36:11.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.358+0000 7f5921ffb640 1 -- 172.21.15.49:0/3057551835 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f590c07cf00 con 0x7f5924196810 2024-05-15T16:36:11.359 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-15T16:36:11.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f591006e170 msgr2=0x7f5910070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f591006e170 0x7f5910070630 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5914009890 tx=0x7f5914008040 comp rx=0 tx=0).stop 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924196810 msgr2=0x7f592419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924196810 0x7f592419abb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f590c036040 tx=0x7f590c004040 comp rx=0 tx=0).stop 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 shutdown_connections 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f591006e170 0x7f5910070630 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5924196810 0x7f592419abb0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 --2- 172.21.15.49:0/3057551835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59241003f0 0x7f59241962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 >> 172.21.15.49:0/3057551835 conn(0x7f59240fbc70 msgr2=0x7f59240fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 shutdown_connections 2024-05-15T16:36:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.360+0000 7f592a7b9640 1 -- 172.21.15.49:0/3057551835 wait complete. 2024-05-15T16:36:11.374 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-15T16:36:11.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.479+0000 7f56de74c640 1 -- 172.21.15.49:0/3779645802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 msgr2=0x7f56d8101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.479+0000 7f56de74c640 1 --2- 172.21.15.49:0/3779645802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 0x7f56d8101710 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009880 tx=0x7f56c802bd70 comp rx=0 tx=0).stop 2024-05-15T16:36:11.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.480+0000 7f56de74c640 1 -- 172.21.15.49:0/3779645802 shutdown_connections 2024-05-15T16:36:11.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.480+0000 7f56de74c640 1 --2- 172.21.15.49:0/3779645802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d8101c50 0x7f56d810a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.480+0000 7f56de74c640 1 --2- 172.21.15.49:0/3779645802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 0x7f56d8101710 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.480+0000 7f56de74c640 1 -- 172.21.15.49:0/3779645802 >> 172.21.15.49:0/3779645802 conn(0x7f56d80fab70 msgr2=0x7f56d80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:11.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.480+0000 7f56de74c640 1 -- 172.21.15.49:0/3779645802 shutdown_connections 2024-05-15T16:36:11.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.480+0000 7f56de74c640 1 -- 172.21.15.49:0/3779645802 wait complete. 2024-05-15T16:36:11.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.480+0000 7f56de74c640 1 Processor -- start 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.480+0000 7f56de74c640 1 -- start start 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56de74c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 0x7f56d8196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56de74c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d8101c50 0x7f56d81966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56de74c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8196ca0 con 0x7f56d8101330 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56de74c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8196e10 con 0x7f56d8101c50 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 0x7f56d8196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 0x7f56d8196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54178/0 (socket says 172.21.15.49:54178) 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d7fff640 1 -- 172.21.15.49:0/1533469780 learned_addr learned my addr 172.21.15.49:0/1533469780 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d7fff640 1 -- 172.21.15.49:0/1533469780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d8101c50 msgr2=0x7f56d81966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d7fff640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d8101c50 0x7f56d81966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d7fff640 1 -- 172.21.15.49:0/1533469780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c8009560 con 0x7f56d8101330 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d7fff640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 0x7f56d8196160 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f56c802c2b0 tx=0x7f56c802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d57fa640 1 -- 172.21.15.49:0/1533469780 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c80035f0 con 0x7f56d8101330 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d57fa640 1 -- 172.21.15.49:0/1533469780 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56c8003c10 con 0x7f56d8101330 2024-05-15T16:36:11.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d819ab80 con 0x7f56d8101330 2024-05-15T16:36:11.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56d57fa640 1 -- 172.21.15.49:0/1533469780 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c8005690 con 0x7f56d8101330 2024-05-15T16:36:11.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.481+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d819b0e0 con 0x7f56d8101330 2024-05-15T16:36:11.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.482+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d8068ed0 con 0x7f56d8101330 2024-05-15T16:36:11.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.483+0000 7f56d57fa640 1 -- 172.21.15.49:0/1533469780 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f56c8004ed0 con 0x7f56d8101330 2024-05-15T16:36:11.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.483+0000 7f56d57fa640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56c006e170 0x7f56c0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:11.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.484+0000 7f56d57fa640 1 -- 172.21.15.49:0/1533469780 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f56c80b0c40 con 0x7f56d8101330 2024-05-15T16:36:11.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.484+0000 7f56d77fe640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56c006e170 0x7f56c0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:11.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.484+0000 7f56d77fe640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56c006e170 0x7f56c0070630 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f56d81976b0 tx=0x7f56cc005c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:11.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.488+0000 7f56d57fa640 1 -- 172.21.15.49:0/1533469780 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f56c8079fa0 con 0x7f56d8101330 2024-05-15T16:36:11.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.686+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f56d804e210 con 0x7f56d8101330 2024-05-15T16:36:11.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.686+0000 7f56d57fa640 1 -- 172.21.15.49:0/1533469780 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v87) v1 ==== 81+0+2428 (secure 0 0 0) 0x7f56c8043080 con 0x7f56d8101330 2024-05-15T16:36:11.688 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6828/2512292397 2024-05-16T16:02:56.077800+0000 2024-05-15T16:36:11.688 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:6825/2552333341 2024-05-16T16:02:56.077816+0000 2024-05-15T16:36:11.688 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/1448243308 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:11.688 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/75769186 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:11.688 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/2391504982 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:11.688 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/287650046 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:11.688 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/821763420 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:11.688 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/644218315 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1515356048 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/821763420 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/2959282915 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:6828/177983287 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/1448243308 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:6829/177983287 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1173274751 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/4024404689 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/3162546694 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6826/2475999224 2024-05-16T15:50:56.872336+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3014161738 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/2885879516 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:11.689 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1166053004 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2163344179 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/2251902137 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2464608729 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/4010845594 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/1508146783 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1904525014 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/2885879516 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:6824/2552333341 2024-05-16T16:02:56.077816+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/4014499308 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/1508146783 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2543812499 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3452717560 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6827/2475999224 2024-05-16T15:50:56.872336+0000 2024-05-15T16:36:11.690 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6829/2512292397 2024-05-16T16:02:56.077800+0000 2024-05-15T16:36:11.691 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/940476764 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:11.691 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/133547858 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:11.691 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1530195241 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:11.691 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/70490983 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:11.691 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/1791319417 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:11.691 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/70490983 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:listed 41 entries 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.688+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56c006e170 msgr2=0x7f56c0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.688+0000 7f56de74c640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56c006e170 0x7f56c0070630 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f56d81976b0 tx=0x7f56cc005c70 comp rx=0 tx=0).stop 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.688+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 msgr2=0x7f56d8196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.688+0000 7f56de74c640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 0x7f56d8196160 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f56c802c2b0 tx=0x7f56c802c8e0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.689+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 shutdown_connections 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.689+0000 7f56de74c640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f56c006e170 0x7f56c0070630 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.689+0000 7f56de74c640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d8101c50 0x7f56d81966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.689+0000 7f56de74c640 1 --2- 172.21.15.49:0/1533469780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56d8101330 0x7f56d8196160 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.689+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 >> 172.21.15.49:0/1533469780 conn(0x7f56d80fab70 msgr2=0x7f56d8102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.689+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 shutdown_connections 2024-05-15T16:36:11.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.689+0000 7f56de74c640 1 -- 172.21.15.49:0/1533469780 wait complete. 2024-05-15T16:36:11.701 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-15T16:36:11.701 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-15T16:36:11.719 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-15T16:36:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.863+0000 7f44df973640 1 -- 172.21.15.49:0/2350654001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 msgr2=0x7f44d810a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.863+0000 7f44df973640 1 --2- 172.21.15.49:0/2350654001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 0x7f44d810a1d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009880 tx=0x7f44c802bd70 comp rx=0 tx=0).stop 2024-05-15T16:36:11.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.863+0000 7f44df973640 1 -- 172.21.15.49:0/2350654001 shutdown_connections 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.863+0000 7f44df973640 1 --2- 172.21.15.49:0/2350654001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 0x7f44d810a1d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.863+0000 7f44df973640 1 --2- 172.21.15.49:0/2350654001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d81013d0 0x7f44d81017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.863+0000 7f44df973640 1 -- 172.21.15.49:0/2350654001 >> 172.21.15.49:0/2350654001 conn(0x7f44d80fab70 msgr2=0x7f44d80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.863+0000 7f44df973640 1 -- 172.21.15.49:0/2350654001 shutdown_connections 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.863+0000 7f44df973640 1 -- 172.21.15.49:0/2350654001 wait complete. 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44df973640 1 Processor -- start 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44df973640 1 -- start start 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44df973640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d81013d0 0x7f44d80ff020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44df973640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 0x7f44d80ff560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44df973640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d8100fb0 con 0x7f44d8101cf0 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44df973640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d80ffaa0 con 0x7f44d81013d0 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44dcf0e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 0x7f44d80ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44dcf0e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 0x7f44d80ff560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54204/0 (socket says 172.21.15.49:54204) 2024-05-15T16:36:11.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44dcf0e640 1 -- 172.21.15.49:0/3136548323 learned_addr learned my addr 172.21.15.49:0/3136548323 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44dd70f640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d81013d0 0x7f44d80ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44dcf0e640 1 -- 172.21.15.49:0/3136548323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d81013d0 msgr2=0x7f44d80ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44dcf0e640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d81013d0 0x7f44d80ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44dcf0e640 1 -- 172.21.15.49:0/3136548323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c8009560 con 0x7f44d8101cf0 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.864+0000 7f44dd70f640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d81013d0 0x7f44d80ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.865+0000 7f44dcf0e640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 0x7f44d80ff560 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f44c802c2b0 tx=0x7f44c8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.865+0000 7f44ce7fc640 1 -- 172.21.15.49:0/3136548323 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44c8004520 con 0x7f44d8101cf0 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.865+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d80ffd20 con 0x7f44d8101cf0 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.865+0000 7f44ce7fc640 1 -- 172.21.15.49:0/3136548323 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44c8003ab0 con 0x7f44d8101cf0 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.865+0000 7f44ce7fc640 1 -- 172.21.15.49:0/3136548323 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44c8005690 con 0x7f44d8101cf0 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.865+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d81a2600 con 0x7f44d8101cf0 2024-05-15T16:36:11.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.866+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d8068e30 con 0x7f44d8101cf0 2024-05-15T16:36:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.867+0000 7f44ce7fc640 1 -- 172.21.15.49:0/3136548323 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f44c8004ed0 con 0x7f44d8101cf0 2024-05-15T16:36:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.867+0000 7f44ce7fc640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44c406e110 0x7f44c40705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.867+0000 7f44ce7fc640 1 -- 172.21.15.49:0/3136548323 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f44c80b0ce0 con 0x7f44d8101cf0 2024-05-15T16:36:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.867+0000 7f44dd70f640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44c406e110 0x7f44c40705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:11.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.868+0000 7f44dd70f640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44c406e110 0x7f44c40705d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f44d400abe0 tx=0x7f44d400a5c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:11.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:11.872+0000 7f44ce7fc640 1 -- 172.21.15.49:0/3136548323 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f44c8079fc0 con 0x7f44d8101cf0 2024-05-15T16:36:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:11 smithi049 ceph-mon[112142]: pgmap v1178: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:11 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1522772217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-15T16:36:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:11 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3057551835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-15T16:36:11.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:11 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1533469780' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-15T16:36:12.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.059+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44d804e210 con 0x7f44d8101cf0 2024-05-15T16:36:12.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.060+0000 7f44ce7fc640 1 -- 172.21.15.49:0/3136548323 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16768 (secure 0 0 0) 0x7f44c807d680 con 0x7f44d8101cf0 2024-05-15T16:36:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-15T16:36:12.061 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":87,"fsid":"c619ac44-12d1-11ef-bc98-c7b262605968","created":"2024-05-15T15:43:35.648208+0000","modified":"2024-05-15T16:18:46.696180+0000","last_up_change":"2024-05-15T16:02:52.176866+0000","last_in_change":"2024-05-15T15:49:04.722203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T15:48:09.899234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-15T15:50:39.462287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-15T15:50:40.285269+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25d1eb06-bee2-48a6-8d86-30b5b3b348d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6803","nonce":1701263964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6805","nonce":1701263964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6809","nonce":1701263964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1701263964},{"type":"v1","addr":"172.21.15.49:6807","nonce":1701263964}]},"public_addr":"172.21.15.49:6803/1701263964","cluster_addr":"172.21.15.49:6805/1701263964","heartbeat_back_addr":"172.21.15.49:6809/1701263964","heartbeat_front_addr":"172.21.15.49:6807/1701263964","state":["exists","up"]},{"osd":1,"uuid":"c0e717cf-ae5c-4fa4-a30d-d2235ff27315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6811","nonce":3720875621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6813","nonce":3720875621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6817","nonce":3720875621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3720875621},{"type":"v1","addr":"172.21.15.49:6815","nonce":3720875621}]},"public_addr":"172.21.15.49:6811/3720875621","cluster_addr":"172.21.15.49:6813/3720875621","heartbeat_back_addr":"172.21.15.49:6817/3720875621","heartbeat_front_addr":"172.21.15.49:6815/3720875621","state":["exists","up"]},{"osd":2,"uuid":"d7752daf-dd5b-49cc-abc2-ca953968e542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6819","nonce":3306250339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6821","nonce":3306250339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6825","nonce":3306250339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3306250339},{"type":"v1","addr":"172.21.15.49:6823","nonce":3306250339}]},"public_addr":"172.21.15.49:6819/3306250339","cluster_addr":"172.21.15.49:6821/3306250339","heartbeat_back_addr":"172.21.15.49:6825/3306250339","heartbeat_front_addr":"172.21.15.49:6823/3306250339","state":["exists","up"]},{"osd":3,"uuid":"2eecc6b1-2c0f-416e-ba0b-6cb21cb67e99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6801","nonce":4258967767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6803","nonce":4258967767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6807","nonce":4258967767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4258967767},{"type":"v1","addr":"172.21.15.144:6805","nonce":4258967767}]},"public_addr":"172.21.15.144:6801/4258967767","cluster_addr":"172.21.15.144:6803/4258967767","heartbeat_back_addr":"172.21.15.144:6807/4258967767","heartbeat_front_addr":"172.21.15.144:6805/4258967767","state":["exists","up"]},{"osd":4,"uuid":"6659631f-da39-4a2c-9f75-e7b4a06b602f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6809","nonce":691212832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6811","nonce":691212832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6815","nonce":691212832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":691212832},{"type":"v1","addr":"172.21.15.144:6813","nonce":691212832}]},"public_addr":"172.21.15.144:6809/691212832","cluster_addr":"172.21.15.144:6811/691212832","heartbeat_back_addr":"172.21.15.144:6815/691212832","heartbeat_front_addr":"172.21.15.144:6813/691212832","state":["exists","up"]},{"osd":5,"uuid":"51ea2d85-6cf7-4a96-8c74-612958083a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6817","nonce":255063313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6819","nonce":255063313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6823","nonce":255063313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":255063313},{"type":"v1","addr":"172.21.15.144:6821","nonce":255063313}]},"public_addr":"172.21.15.144:6817/255063313","cluster_addr":"172.21.15.144:6819/255063313","heartbeat_back_addr":"172.21.15.144:6823/255063313","heartbeat_front_addr":"172.21.15.144:6821/255063313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:10.108689+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:47:37.458152+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:05.915664+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:30.194669+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:48:55.748301+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T15:49:22.635948+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6828/2512292397":"2024-05-16T16:02:56.077800+0000","172.21.15.144:6825/2552333341":"2024-05-16T16:02:56.077816+0000","172.21.15.49:6801/1448243308":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/75769186":"2024-05-16T15:44:39.328329+0000","172.21.15.144:0/2391504982":"2024-05-16T15:54:47.414074+0000","172.21.15.49:0/287650046":"2024-05-16T15:58:20.089143+0000","172.21.15.49:6801/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/644218315":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/1515356048":"2024-05-16T15:58:20.089143+0000","172.21.15.49:6800/821763420":"2024-05-16T15:44:39.328329+0000","172.21.15.144:0/2959282915":"2024-05-16T15:54:47.414074+0000","172.21.15.144:6828/177983287":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6800/1448243308":"2024-05-16T15:53:24.658899+0000","172.21.15.144:6829/177983287":"2024-05-16T15:54:47.414074+0000","172.21.15.49:0/1173274751":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/4024404689":"2024-05-16T15:44:07.361150+0000","172.21.15.144:0/3162546694":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6826/2475999224":"2024-05-16T15:50:56.872336+0000","172.21.15.49:0/3014161738":"2024-05-16T15:45:51.392671+0000","172.21.15.49:6800/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.49:0/1166053004":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/2163344179":"2024-05-16T15:44:07.361150+0000","172.21.15.144:0/2251902137":"2024-05-16T15:54:47.414074+0000","172.21.15.49:0/2464608729":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/4010845594":"2024-05-16T15:44:07.361150+0000","172.21.15.49:6800/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/1904525014":"2024-05-16T15:53:24.658899+0000","172.21.15.49:6801/2885879516":"2024-05-16T15:44:07.361150+0000","172.21.15.144:6824/2552333341":"2024-05-16T16:02:56.077816+0000","172.21.15.144:0/4014499308":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6801/1508146783":"2024-05-16T15:45:51.392671+0000","172.21.15.49:0/2543812499":"2024-05-16T15:53:24.658899+0000","172.21.15.49:0/3452717560":"2024-05-16T15:44:39.328329+0000","172.21.15.49:6827/2475999224":"2024-05-16T15:50:56.872336+0000","172.21.15.49:6829/2512292397":"2024-05-16T16:02:56.077800+0000","172.21.15.49:0/940476764":"2024-05-16T15:44:39.328329+0000","172.21.15.49:0/133547858":"2024-05-16T15:58:20.089143+0000","172.21.15.49:0/1530195241":"2024-05-16T15:58:20.089143+0000","172.21.15.49:6800/70490983":"2024-05-16T15:58:20.089143+0000","172.21.15.144:0/1791319417":"2024-05-16T15:54:47.414074+0000","172.21.15.49:6801/70490983":"2024-05-16T15:58:20.089143+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T16:36:12.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44c406e110 msgr2=0x7f44c40705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44c406e110 0x7f44c40705d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f44d400abe0 tx=0x7f44d400a5c0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 msgr2=0x7f44d80ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 0x7f44d80ff560 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f44c802c2b0 tx=0x7f44c8005e50 comp rx=0 tx=0).stop 2024-05-15T16:36:12.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 shutdown_connections 2024-05-15T16:36:12.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f44c406e110 0x7f44c40705d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d8101cf0 0x7f44d80ff560 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 --2- 172.21.15.49:0/3136548323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d81013d0 0x7f44d80ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 >> 172.21.15.49:0/3136548323 conn(0x7f44d80fab70 msgr2=0x7f44d81026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:12.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 shutdown_connections 2024-05-15T16:36:12.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.062+0000 7f44df973640 1 -- 172.21.15.49:0/3136548323 wait complete. 2024-05-15T16:36:12.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-15T15:48:09.899234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-15T15:50:39.462287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-15T15:50:40.285269+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-15T16:36:12.075 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-15T16:36:12.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.181+0000 7febeaae2640 1 -- 172.21.15.49:0/4263882240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 msgr2=0x7febe410a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.181+0000 7febeaae2640 1 --2- 172.21.15.49:0/4263882240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 0x7febe410a100 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7febd0009880 tx=0x7febd002bd70 comp rx=0 tx=0).stop 2024-05-15T16:36:12.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.181+0000 7febeaae2640 1 -- 172.21.15.49:0/4263882240 shutdown_connections 2024-05-15T16:36:12.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.181+0000 7febeaae2640 1 --2- 172.21.15.49:0/4263882240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 0x7febe410a100 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.181+0000 7febeaae2640 1 --2- 172.21.15.49:0/4263882240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe4101300 0x7febe41016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.181+0000 7febeaae2640 1 -- 172.21.15.49:0/4263882240 >> 172.21.15.49:0/4263882240 conn(0x7febe40fab60 msgr2=0x7febe40fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.181+0000 7febeaae2640 1 -- 172.21.15.49:0/4263882240 shutdown_connections 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.181+0000 7febeaae2640 1 -- 172.21.15.49:0/4263882240 wait complete. 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febeaae2640 1 Processor -- start 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febeaae2640 1 -- start start 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febeaae2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe4101300 0x7febe4196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febeaae2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 0x7febe41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febeaae2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe4196d60 con 0x7febe4101c20 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febeaae2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe419aa70 con 0x7febe4101300 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febe3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 0x7febe41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febe3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 0x7febe41966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54220/0 (socket says 172.21.15.49:54220) 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febe3fff640 1 -- 172.21.15.49:0/1432790007 learned_addr learned my addr 172.21.15.49:0/1432790007 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febe887e640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe4101300 0x7febe4196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febe3fff640 1 -- 172.21.15.49:0/1432790007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe4101300 msgr2=0x7febe4196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febe3fff640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe4101300 0x7febe4196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.182+0000 7febe3fff640 1 -- 172.21.15.49:0/1432790007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd0009560 con 0x7febe4101c20 2024-05-15T16:36:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.183+0000 7febe3fff640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 0x7febe41966d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7febd002c2b0 tx=0x7febd002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.183+0000 7febe1ffb640 1 -- 172.21.15.49:0/1432790007 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd0004520 con 0x7febe4101c20 2024-05-15T16:36:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.183+0000 7febe1ffb640 1 -- 172.21.15.49:0/1432790007 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febd0003ab0 con 0x7febe4101c20 2024-05-15T16:36:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.183+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe419acf0 con 0x7febe4101c20 2024-05-15T16:36:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.183+0000 7febe1ffb640 1 -- 172.21.15.49:0/1432790007 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd0005500 con 0x7febe4101c20 2024-05-15T16:36:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.183+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe419b250 con 0x7febe4101c20 2024-05-15T16:36:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.184+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe4068ed0 con 0x7febe4101c20 2024-05-15T16:36:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.184+0000 7febe1ffb640 1 -- 172.21.15.49:0/1432790007 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7febd0003c20 con 0x7febe4101c20 2024-05-15T16:36:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.185+0000 7febe1ffb640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7febd406e180 0x7febd4070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.185+0000 7febe887e640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7febd406e180 0x7febd4070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.185+0000 7febe1ffb640 1 -- 172.21.15.49:0/1432790007 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7febd00b0b90 con 0x7febe4101c20 2024-05-15T16:36:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.186+0000 7febe887e640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7febd406e180 0x7febd4070640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7febd8005fd0 tx=0x7febd800b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:12.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.190+0000 7febe1ffb640 1 -- 172.21.15.49:0/1432790007 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7febd0079f90 con 0x7febe4101c20 2024-05-15T16:36:12.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:11 smithi144 ceph-mon[86635]: pgmap v1178: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:11 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1522772217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-15T16:36:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:11 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3057551835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-15T16:36:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:11 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1533469780' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-15T16:36:12.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.374+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febe4104910 con 0x7febe4101c20 2024-05-15T16:36:12.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.374+0000 7febe1ffb640 1 -- 172.21.15.49:0/1432790007 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7febd0043020 con 0x7febe4101c20 2024-05-15T16:36:12.375 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-05-15T16:36:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7febd406e180 msgr2=0x7febd4070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7febd406e180 0x7febd4070640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7febd8005fd0 tx=0x7febd800b390 comp rx=0 tx=0).stop 2024-05-15T16:36:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 msgr2=0x7febe41966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 0x7febe41966d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7febd002c2b0 tx=0x7febd002c850 comp rx=0 tx=0).stop 2024-05-15T16:36:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 shutdown_connections 2024-05-15T16:36:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7febd406e180 0x7febd4070640 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe4101c20 0x7febe41966d0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 --2- 172.21.15.49:0/1432790007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe4101300 0x7febe4196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 >> 172.21.15.49:0/1432790007 conn(0x7febe40fab60 msgr2=0x7febe4102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:12.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.376+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 shutdown_connections 2024-05-15T16:36:12.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.377+0000 7febeaae2640 1 -- 172.21.15.49:0/1432790007 wait complete. 2024-05-15T16:36:12.389 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-15T16:36:12.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.495+0000 7f16fb07e640 1 -- 172.21.15.49:0/3836837883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f41012f0 msgr2=0x7f16f41016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.495+0000 7f16fb07e640 1 --2- 172.21.15.49:0/3836837883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f41012f0 0x7f16f41016d0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009880 tx=0x7f16e802bd70 comp rx=0 tx=0).stop 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.495+0000 7f16fb07e640 1 -- 172.21.15.49:0/3836837883 shutdown_connections 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.495+0000 7f16fb07e640 1 --2- 172.21.15.49:0/3836837883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16f4101c10 0x7f16f410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.495+0000 7f16fb07e640 1 --2- 172.21.15.49:0/3836837883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f41012f0 0x7f16f41016d0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.495+0000 7f16fb07e640 1 -- 172.21.15.49:0/3836837883 >> 172.21.15.49:0/3836837883 conn(0x7f16f40fab10 msgr2=0x7f16f40fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.495+0000 7f16fb07e640 1 -- 172.21.15.49:0/3836837883 shutdown_connections 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.495+0000 7f16fb07e640 1 -- 172.21.15.49:0/3836837883 wait complete. 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16fb07e640 1 Processor -- start 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16fb07e640 1 -- start start 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16fb07e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16f41012f0 0x7f16f4196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16fb07e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f4101c10 0x7f16f4196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16fb07e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f4196f50 con 0x7f16f4101c10 2024-05-15T16:36:12.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16fb07e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f419ab50 con 0x7f16f41012f0 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f4101c10 0x7f16f4196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f4101c10 0x7f16f4196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54228/0 (socket says 172.21.15.49:54228) 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16f3fff640 1 -- 172.21.15.49:0/3708833840 learned_addr learned my addr 172.21.15.49:0/3708833840 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16f8e1a640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16f41012f0 0x7f16f4196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16f3fff640 1 -- 172.21.15.49:0/3708833840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16f41012f0 msgr2=0x7f16f4196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16f3fff640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16f41012f0 0x7f16f4196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.496+0000 7f16f3fff640 1 -- 172.21.15.49:0/3708833840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8009560 con 0x7f16f4101c10 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.497+0000 7f16f3fff640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f4101c10 0x7f16f4196870 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f16e000b370 tx=0x7f16e000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.497+0000 7f16f1ffb640 1 -- 172.21.15.49:0/3708833840 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16e0011800 con 0x7f16f4101c10 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.497+0000 7f16f1ffb640 1 -- 172.21.15.49:0/3708833840 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16e0011e20 con 0x7f16f4101c10 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.497+0000 7f16f1ffb640 1 -- 172.21.15.49:0/3708833840 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16e000f2e0 con 0x7f16f4101c10 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.497+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f419ae30 con 0x7f16f4101c10 2024-05-15T16:36:12.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.497+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f419b340 con 0x7f16f4101c10 2024-05-15T16:36:12.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.498+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f4068eb0 con 0x7f16f4101c10 2024-05-15T16:36:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.498+0000 7f16f1ffb640 1 -- 172.21.15.49:0/3708833840 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f16e0010380 con 0x7f16f4101c10 2024-05-15T16:36:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.499+0000 7f16f1ffb640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f16e406e170 0x7f16e4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.499+0000 7f16f1ffb640 1 -- 172.21.15.49:0/3708833840 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f16e008de30 con 0x7f16f4101c10 2024-05-15T16:36:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.499+0000 7f16f8e1a640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f16e406e170 0x7f16e4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:12.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.499+0000 7f16f8e1a640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f16e406e170 0x7f16e4070630 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006010 tx=0x7f16e802cbd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:12.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.504+0000 7f16f1ffb640 1 -- 172.21.15.49:0/3708833840 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f16e0057110 con 0x7f16f4101c10 2024-05-15T16:36:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.689+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f16f4109d40 con 0x7f16f4101c10 2024-05-15T16:36:12.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.689+0000 7f16f1ffb640 1 -- 172.21.15.49:0/3708833840 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f16e005a7d0 con 0x7f16f4101c10 2024-05-15T16:36:12.691 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f16e406e170 msgr2=0x7f16e4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f16e406e170 0x7f16e4070630 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006010 tx=0x7f16e802cbd0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f4101c10 msgr2=0x7f16f4196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f4101c10 0x7f16f4196870 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f16e000b370 tx=0x7f16e000b830 comp rx=0 tx=0).stop 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 shutdown_connections 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f16e406e170 0x7f16e4070630 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16f4101c10 0x7f16f4196870 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 --2- 172.21.15.49:0/3708833840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16f41012f0 0x7f16f4196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 >> 172.21.15.49:0/3708833840 conn(0x7f16f40fab10 msgr2=0x7f16f41025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 shutdown_connections 2024-05-15T16:36:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.691+0000 7f16fb07e640 1 -- 172.21.15.49:0/3708833840 wait complete. 2024-05-15T16:36:12.704 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-15T16:36:12.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.810+0000 7f2f18106640 1 -- 172.21.15.49:0/3773859263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067450 msgr2=0x7f2f10067830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.810+0000 7f2f18106640 1 --2- 172.21.15.49:0/3773859263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067450 0x7f2f10067830 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009880 tx=0x7f2f0c02bce0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.810+0000 7f2f18106640 1 -- 172.21.15.49:0/3773859263 shutdown_connections 2024-05-15T16:36:12.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.810+0000 7f2f18106640 1 --2- 172.21.15.49:0/3773859263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f10067d70 0x7f2f1010c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.810+0000 7f2f18106640 1 --2- 172.21.15.49:0/3773859263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067450 0x7f2f10067830 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.810+0000 7f2f18106640 1 -- 172.21.15.49:0/3773859263 >> 172.21.15.49:0/3773859263 conn(0x7f2f10074b50 msgr2=0x7f2f10074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:12.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.810+0000 7f2f18106640 1 -- 172.21.15.49:0/3773859263 shutdown_connections 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.810+0000 7f2f18106640 1 -- 172.21.15.49:0/3773859263 wait complete. 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.810+0000 7f2f18106640 1 Processor -- start 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f18106640 1 -- start start 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f18106640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f10067450 0x7f2f10198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f18106640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067d70 0x7f2f101988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f18106640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10198f50 con 0x7f2f10067d70 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f18106640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1019cc10 con 0x7f2f10067450 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f156a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067d70 0x7f2f101988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f156a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067d70 0x7f2f101988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54240/0 (socket says 172.21.15.49:54240) 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f156a1640 1 -- 172.21.15.49:0/714109723 learned_addr learned my addr 172.21.15.49:0/714109723 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f156a1640 1 -- 172.21.15.49:0/714109723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f10067450 msgr2=0x7f2f10198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f15ea2640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f10067450 0x7f2f10198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f156a1640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f10067450 0x7f2f10198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:12.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f156a1640 1 -- 172.21.15.49:0/714109723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c009560 con 0x7f2f10067d70 2024-05-15T16:36:12.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f156a1640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067d70 0x7f2f101988c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d6f0 tx=0x7f2f0000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:12.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f06ffd640 1 -- 172.21.15.49:0/714109723 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f00009a30 con 0x7f2f10067d70 2024-05-15T16:36:12.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f06ffd640 1 -- 172.21.15.49:0/714109723 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f000054e0 con 0x7f2f10067d70 2024-05-15T16:36:12.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1019cef0 con 0x7f2f10067d70 2024-05-15T16:36:12.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.811+0000 7f2f06ffd640 1 -- 172.21.15.49:0/714109723 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f00010460 con 0x7f2f10067d70 2024-05-15T16:36:12.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.812+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1019d450 con 0x7f2f10067d70 2024-05-15T16:36:12.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.812+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f100ff680 con 0x7f2f10067d70 2024-05-15T16:36:12.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.813+0000 7f2f06ffd640 1 -- 172.21.15.49:0/714109723 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2f00009b90 con 0x7f2f10067d70 2024-05-15T16:36:12.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.813+0000 7f2f06ffd640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2efc06e170 0x7f2efc070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:12.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.814+0000 7f2f06ffd640 1 -- 172.21.15.49:0/714109723 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2f0008dd30 con 0x7f2f10067d70 2024-05-15T16:36:12.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.814+0000 7f2f15ea2640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2efc06e170 0x7f2efc070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:12.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.814+0000 7f2f15ea2640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2efc06e170 0x7f2efc070630 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c036040 tx=0x7f2f0c02c780 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:12.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:12.818+0000 7f2f06ffd640 1 -- 172.21.15.49:0/714109723 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2f00057010 con 0x7f2f10067d70 2024-05-15T16:36:12.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:12 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3136548323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T16:36:12.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:12 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/1432790007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T16:36:12.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:12 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/3708833840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-15T16:36:13.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.001+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f2f1010a540 con 0x7f2f10067d70 2024-05-15T16:36:13.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.002+0000 7f2f06ffd640 1 -- 172.21.15.49:0/714109723 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2f000170b0 con 0x7f2f10067d70 2024-05-15T16:36:13.003 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-15T16:36:13.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2efc06e170 msgr2=0x7f2efc070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2efc06e170 0x7f2efc070630 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c036040 tx=0x7f2f0c02c780 comp rx=0 tx=0).stop 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067d70 msgr2=0x7f2f101988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067d70 0x7f2f101988c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d6f0 tx=0x7f2f0000dbb0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 shutdown_connections 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7f2efc06e170 0x7f2efc070630 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f10067d70 0x7f2f101988c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 --2- 172.21.15.49:0/714109723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f10067450 0x7f2f10198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 >> 172.21.15.49:0/714109723 conn(0x7f2f10074b50 msgr2=0x7f2f10107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 shutdown_connections 2024-05-15T16:36:13.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.004+0000 7f2f18106640 1 -- 172.21.15.49:0/714109723 wait complete. 2024-05-15T16:36:13.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-15T16:36:13.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.120+0000 7faafec3b640 1 -- 172.21.15.49:0/151911628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf80ffc60 msgr2=0x7faaf8103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:13.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.120+0000 7faafec3b640 1 --2- 172.21.15.49:0/151911628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf80ffc60 0x7faaf8103ca0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7faae8009c20 tx=0x7faae802c1b0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.120+0000 7faafec3b640 1 -- 172.21.15.49:0/151911628 shutdown_connections 2024-05-15T16:36:13.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.120+0000 7faafec3b640 1 --2- 172.21.15.49:0/151911628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf80ffc60 0x7faaf8103ca0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.120+0000 7faafec3b640 1 --2- 172.21.15.49:0/151911628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf80ff2b0 0x7faaf80ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.120+0000 7faafec3b640 1 -- 172.21.15.49:0/151911628 >> 172.21.15.49:0/151911628 conn(0x7faaf8075cf0 msgr2=0x7faaf80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.120+0000 7faafec3b640 1 -- 172.21.15.49:0/151911628 shutdown_connections 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.120+0000 7faafec3b640 1 -- 172.21.15.49:0/151911628 wait complete. 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faafec3b640 1 Processor -- start 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faafec3b640 1 -- start start 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faafec3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf80ff2b0 0x7faaf8197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faafec3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf8197550 0x7faaf81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faafec3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80683e0 con 0x7faaf8197550 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faafec3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81a0080 con 0x7faaf80ff2b0 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faaf7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf8197550 0x7faaf81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faaf7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf8197550 0x7faaf81979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54264/0 (socket says 172.21.15.49:54264) 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faaf7fff640 1 -- 172.21.15.49:0/4199425950 learned_addr learned my addr 172.21.15.49:0/4199425950 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faaf7fff640 1 -- 172.21.15.49:0/4199425950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf80ff2b0 msgr2=0x7faaf8197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faaf7fff640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf80ff2b0 0x7faaf8197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faaf7fff640 1 -- 172.21.15.49:0/4199425950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae8009900 con 0x7faaf8197550 2024-05-15T16:36:13.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.121+0000 7faaf7fff640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf8197550 0x7faaf81979b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7faae802c6f0 tx=0x7faae8032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:13.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.122+0000 7faaf5ffb640 1 -- 172.21.15.49:0/4199425950 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae8034820 con 0x7faaf8197550 2024-05-15T16:36:13.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.122+0000 7faaf5ffb640 1 -- 172.21.15.49:0/4199425950 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faae8003e30 con 0x7faaf8197550 2024-05-15T16:36:13.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.122+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf81a0280 con 0x7faaf8197550 2024-05-15T16:36:13.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.122+0000 7faaf5ffb640 1 -- 172.21.15.49:0/4199425950 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae803f440 con 0x7faaf8197550 2024-05-15T16:36:13.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.122+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf81a0750 con 0x7faaf8197550 2024-05-15T16:36:13.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.123+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf8100780 con 0x7faaf8197550 2024-05-15T16:36:13.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.123+0000 7faaf5ffb640 1 -- 172.21.15.49:0/4199425950 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faae803b040 con 0x7faaf8197550 2024-05-15T16:36:13.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.124+0000 7faaf5ffb640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faae406e170 0x7faae4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T16:36:13.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.124+0000 7faafc9d7640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faae406e170 0x7faae4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T16:36:13.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.124+0000 7faaf5ffb640 1 -- 172.21.15.49:0/4199425950 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faae80b0f20 con 0x7faaf8197550 2024-05-15T16:36:13.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.124+0000 7faafc9d7640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faae406e170 0x7faae4070630 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faae000a990 tx=0x7faae0005c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T16:36:13.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.129+0000 7faaf5ffb640 1 -- 172.21.15.49:0/4199425950 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7faae80e5850 con 0x7faaf8197550 2024-05-15T16:36:13.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:12 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3136548323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T16:36:13.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:12 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/1432790007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T16:36:13.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:12 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/3708833840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-15T16:36:13.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.327+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faaf80679f0 con 0x7faaf8197550 2024-05-15T16:36:13.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.328+0000 7faaf5ffb640 1 -- 172.21.15.49:0/4199425950 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v87) v1 ==== 81+0+2428 (secure 0 0 0) 0x7faae807a200 con 0x7faaf8197550 2024-05-15T16:36:13.329 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6828/2512292397 2024-05-16T16:02:56.077800+0000 2024-05-15T16:36:13.329 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:6825/2552333341 2024-05-16T16:02:56.077816+0000 2024-05-15T16:36:13.329 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/1448243308 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:13.329 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/75769186 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/2391504982 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/287650046 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/821763420 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/644218315 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1515356048 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/821763420 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/2959282915 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:6828/177983287 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/1448243308 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:6829/177983287 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1173274751 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/4024404689 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:13.330 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/3162546694 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6826/2475999224 2024-05-16T15:50:56.872336+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3014161738 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/2885879516 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1166053004 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2163344179 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/2251902137 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2464608729 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/4010845594 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/1508146783 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1904525014 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/2885879516 2024-05-16T15:44:07.361150+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:6824/2552333341 2024-05-16T16:02:56.077816+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/4014499308 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:13.331 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/1508146783 2024-05-16T15:45:51.392671+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2543812499 2024-05-16T15:53:24.658899+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/3452717560 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6827/2475999224 2024-05-16T15:50:56.872336+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6829/2512292397 2024-05-16T16:02:56.077800+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/940476764 2024-05-16T15:44:39.328329+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/133547858 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1530195241 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/70490983 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.144:0/1791319417 2024-05-16T15:54:47.414074+0000 2024-05-15T16:36:13.332 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/70490983 2024-05-16T15:58:20.089143+0000 2024-05-15T16:36:13.333 INFO:teuthology.orchestra.run.smithi049.stderr:listed 41 entries 2024-05-15T16:36:13.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faae406e170 msgr2=0x7faae4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:13.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faae406e170 0x7faae4070630 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faae000a990 tx=0x7faae0005c70 comp rx=0 tx=0).stop 2024-05-15T16:36:13.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf8197550 msgr2=0x7faaf81979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T16:36:13.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf8197550 0x7faaf81979b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7faae802c6f0 tx=0x7faae8032570 comp rx=0 tx=0).stop 2024-05-15T16:36:13.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 shutdown_connections 2024-05-15T16:36:13.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:6800/3324404093,v1:172.21.15.49:6801/3324404093] conn(0x7faae406e170 0x7faae4070630 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaf8197550 0x7faaf81979b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 --2- 172.21.15.49:0/4199425950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf80ff2b0 0x7faaf8197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T16:36:13.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 >> 172.21.15.49:0/4199425950 conn(0x7faaf8075cf0 msgr2=0x7faaf80fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T16:36:13.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 shutdown_connections 2024-05-15T16:36:13.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-15T16:36:13.330+0000 7faafec3b640 1 -- 172.21.15.49:0/4199425950 wait complete. 2024-05-15T16:36:13.342 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2024-05-15T16:36:13.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T16:36:13.342 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-15T16:36:13.368 INFO:teuthology.orchestra.run:waiting for 300 2024-05-15T16:36:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:13 smithi144 ceph-mon[86635]: pgmap v1179: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:13 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/714109723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-15T16:36:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:13 smithi144 ceph-mon[86635]: from='client.? 172.21.15.49:0/4199425950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-15T16:36:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:13 smithi049 ceph-mon[112142]: pgmap v1179: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:13 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/714109723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-15T16:36:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:13 smithi049 ceph-mon[112142]: from='client.? 172.21.15.49:0/4199425950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-15T16:36:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:15 smithi144 ceph-mon[86635]: pgmap v1180: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:16.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:15 smithi049 ceph-mon[112142]: pgmap v1180: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:17 smithi144 ceph-mon[86635]: pgmap v1181: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:17 smithi049 ceph-mon[112142]: pgmap v1181: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:20.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:19 smithi144 ceph-mon[86635]: pgmap v1182: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:19 smithi049 ceph-mon[112142]: pgmap v1182: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:21.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:36:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:36:21.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:36:21.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:36:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:21 smithi144 ceph-mon[86635]: pgmap v1183: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:36:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:36:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:36:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:21 smithi049 ceph-mon[112142]: pgmap v1183: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:36:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:36:22.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:36:24.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:23 smithi144 ceph-mon[86635]: pgmap v1184: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:23 smithi049 ceph-mon[112142]: pgmap v1184: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:25 smithi144 ceph-mon[86635]: pgmap v1185: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:25 smithi049 ceph-mon[112142]: pgmap v1185: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:27 smithi144 ceph-mon[86635]: pgmap v1186: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:27 smithi049 ceph-mon[112142]: pgmap v1186: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:29 smithi144 ceph-mon[86635]: pgmap v1187: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:29 smithi049 ceph-mon[112142]: pgmap v1187: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:31 smithi049 ceph-mon[112142]: pgmap v1188: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:31 smithi144 ceph-mon[86635]: pgmap v1188: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:33 smithi049 ceph-mon[112142]: pgmap v1189: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:33 smithi144 ceph-mon[86635]: pgmap v1189: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:35 smithi144 ceph-mon[86635]: pgmap v1190: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:36:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:35 smithi049 ceph-mon[112142]: pgmap v1190: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:36:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:37 smithi144 ceph-mon[86635]: pgmap v1191: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:37 smithi049 ceph-mon[112142]: pgmap v1191: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:40.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:39 smithi144 ceph-mon[86635]: pgmap v1192: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:39 smithi049 ceph-mon[112142]: pgmap v1192: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:41 smithi144 ceph-mon[86635]: pgmap v1193: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:41 smithi049 ceph-mon[112142]: pgmap v1193: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:43 smithi144 ceph-mon[86635]: pgmap v1194: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:43 smithi049 ceph-mon[112142]: pgmap v1194: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:45 smithi144 ceph-mon[86635]: pgmap v1195: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:45 smithi049 ceph-mon[112142]: pgmap v1195: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:47 smithi144 ceph-mon[86635]: pgmap v1196: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:48.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:47 smithi049 ceph-mon[112142]: pgmap v1196: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:49 smithi144 ceph-mon[86635]: pgmap v1197: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:50.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:49 smithi049 ceph-mon[112142]: pgmap v1197: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:36:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:36:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:36:52.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:51 smithi144 ceph-mon[86635]: pgmap v1198: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:51 smithi049 ceph-mon[112142]: pgmap v1198: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:53 smithi144 ceph-mon[86635]: pgmap v1199: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:53 smithi049 ceph-mon[112142]: pgmap v1199: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:55 smithi144 ceph-mon[86635]: pgmap v1200: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:55 smithi049 ceph-mon[112142]: pgmap v1200: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:36:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:57 smithi144 ceph-mon[86635]: pgmap v1201: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:36:58.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:57 smithi049 ceph-mon[112142]: pgmap v1201: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:36:59 smithi144 ceph-mon[86635]: pgmap v1202: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:36:59 smithi049 ceph-mon[112142]: pgmap v1202: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:01 smithi144 ceph-mon[86635]: pgmap v1203: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:01 smithi049 ceph-mon[112142]: pgmap v1203: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:04.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:04 smithi144 ceph-mon[86635]: pgmap v1204: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:04 smithi049 ceph-mon[112142]: pgmap v1204: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:06 smithi049 ceph-mon[112142]: pgmap v1205: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:06.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:37:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:06 smithi144 ceph-mon[86635]: pgmap v1205: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:37:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:08 smithi144 ceph-mon[86635]: pgmap v1206: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:08 smithi049 ceph-mon[112142]: pgmap v1206: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:10 smithi144 ceph-mon[86635]: pgmap v1207: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:10 smithi049 ceph-mon[112142]: pgmap v1207: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:12 smithi144 ceph-mon[86635]: pgmap v1208: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:12.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:12 smithi049 ceph-mon[112142]: pgmap v1208: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:14 smithi144 ceph-mon[86635]: pgmap v1209: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:14.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:14 smithi049 ceph-mon[112142]: pgmap v1209: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:16.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:16 smithi049 ceph-mon[112142]: pgmap v1210: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:16 smithi144 ceph-mon[86635]: pgmap v1210: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:18 smithi144 ceph-mon[86635]: pgmap v1211: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:18 smithi049 ceph-mon[112142]: pgmap v1211: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:20 smithi144 ceph-mon[86635]: pgmap v1212: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:20 smithi049 ceph-mon[112142]: pgmap v1212: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:37:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:37:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:22 smithi144 ceph-mon[86635]: pgmap v1213: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:22 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:37:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:22 smithi049 ceph-mon[112142]: pgmap v1213: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:22 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:37:23.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:23 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:37:23.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:23 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:37:23.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:23 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:37:23.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:37:23.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:37:23.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:37:24.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:24 smithi144 ceph-mon[86635]: pgmap v1214: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:24.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:24 smithi049 ceph-mon[112142]: pgmap v1214: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:26 smithi049 ceph-mon[112142]: pgmap v1215: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:26.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:26 smithi144 ceph-mon[86635]: pgmap v1215: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:28.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:28 smithi144 ceph-mon[86635]: pgmap v1216: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:28.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:28 smithi049 ceph-mon[112142]: pgmap v1216: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:30.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:30 smithi144 ceph-mon[86635]: pgmap v1217: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:30.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:30 smithi049 ceph-mon[112142]: pgmap v1217: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:32.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:32 smithi144 ceph-mon[86635]: pgmap v1218: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:32.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:32 smithi049 ceph-mon[112142]: pgmap v1218: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:34.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:34 smithi144 ceph-mon[86635]: pgmap v1219: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:34.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:34 smithi049 ceph-mon[112142]: pgmap v1219: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:35.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:37:35.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:37:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:36 smithi049 ceph-mon[112142]: pgmap v1220: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:36.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:36 smithi144 ceph-mon[86635]: pgmap v1220: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:38.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:38 smithi144 ceph-mon[86635]: pgmap v1221: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:38.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:38 smithi049 ceph-mon[112142]: pgmap v1221: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:40.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:40 smithi144 ceph-mon[86635]: pgmap v1222: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:40.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:40 smithi049 ceph-mon[112142]: pgmap v1222: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:42 smithi144 ceph-mon[86635]: pgmap v1223: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:42.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:42 smithi049 ceph-mon[112142]: pgmap v1223: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:44.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:44 smithi144 ceph-mon[86635]: pgmap v1224: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:44.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:44 smithi049 ceph-mon[112142]: pgmap v1224: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:46.523 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:46 smithi144 ceph-mon[86635]: pgmap v1225: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:46.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:46 smithi049 ceph-mon[112142]: pgmap v1225: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:48.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:48 smithi144 ceph-mon[86635]: pgmap v1226: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:48.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:48 smithi049 ceph-mon[112142]: pgmap v1226: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:50.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:50 smithi144 ceph-mon[86635]: pgmap v1227: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:50.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:37:50.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:50 smithi049 ceph-mon[112142]: pgmap v1227: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:37:50.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:37:52.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:52 smithi144 ceph-mon[86635]: pgmap v1228: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:52.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:52 smithi049 ceph-mon[112142]: pgmap v1228: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:54.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:54 smithi144 ceph-mon[86635]: pgmap v1229: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:54.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:54 smithi049 ceph-mon[112142]: pgmap v1229: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:56.634 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:56 smithi144 ceph-mon[86635]: pgmap v1230: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:56.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:56 smithi049 ceph-mon[112142]: pgmap v1230: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:37:58.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:37:58 smithi144 ceph-mon[86635]: pgmap v1231: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:37:58.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:37:58 smithi049 ceph-mon[112142]: pgmap v1231: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:00.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:00 smithi144 ceph-mon[86635]: pgmap v1232: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:00.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:00 smithi049 ceph-mon[112142]: pgmap v1232: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:02.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:02 smithi144 ceph-mon[86635]: pgmap v1233: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:02.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:02 smithi049 ceph-mon[112142]: pgmap v1233: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:04.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:04 smithi144 ceph-mon[86635]: pgmap v1234: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:04.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:04 smithi049 ceph-mon[112142]: pgmap v1234: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:05.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:38:05.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:38:06.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:06 smithi144 ceph-mon[86635]: pgmap v1235: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:06.704 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:06 smithi049 ceph-mon[112142]: pgmap v1235: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:08.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:07 smithi144 ceph-mon[86635]: pgmap v1236: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:07 smithi049 ceph-mon[112142]: pgmap v1236: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:10.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:09 smithi144 ceph-mon[86635]: pgmap v1237: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:10.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:09 smithi049 ceph-mon[112142]: pgmap v1237: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:12.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:11 smithi144 ceph-mon[86635]: pgmap v1238: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:12.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:11 smithi049 ceph-mon[112142]: pgmap v1238: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:14.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:13 smithi144 ceph-mon[86635]: pgmap v1239: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:14.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:13 smithi049 ceph-mon[112142]: pgmap v1239: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:16.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:15 smithi144 ceph-mon[86635]: pgmap v1240: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:16.205 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:15 smithi049 ceph-mon[112142]: pgmap v1240: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:18.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:17 smithi144 ceph-mon[86635]: pgmap v1241: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:18.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:17 smithi049 ceph-mon[112142]: pgmap v1241: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:20.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:19 smithi144 ceph-mon[86635]: pgmap v1242: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:20.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:19 smithi049 ceph-mon[112142]: pgmap v1242: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:21.162 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:38:21.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:38:21.955 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:21 smithi049 ceph-mon[112142]: pgmap v1243: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:22.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:21 smithi144 ceph-mon[86635]: pgmap v1243: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:23.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:22 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:38:23.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:22 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:38:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:23 smithi144 ceph-mon[86635]: pgmap v1244: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:23 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:38:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:23 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:38:24.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:23 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:38:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:23 smithi049 ceph-mon[112142]: pgmap v1244: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:38:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:38:24.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:38:26.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:25 smithi144 ceph-mon[86635]: pgmap v1245: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:26.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:25 smithi049 ceph-mon[112142]: pgmap v1245: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:28.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:27 smithi144 ceph-mon[86635]: pgmap v1246: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:28.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:27 smithi049 ceph-mon[112142]: pgmap v1246: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:30.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:29 smithi144 ceph-mon[86635]: pgmap v1247: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:30.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:29 smithi049 ceph-mon[112142]: pgmap v1247: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:32.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:31 smithi144 ceph-mon[86635]: pgmap v1248: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:32.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:31 smithi049 ceph-mon[112142]: pgmap v1248: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:34.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:33 smithi144 ceph-mon[86635]: pgmap v1249: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:38:34.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:33 smithi049 ceph-mon[112142]: pgmap v1249: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:38:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:35 smithi144 ceph-mon[86635]: pgmap v1250: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:38:36.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:35 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:38:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:35 smithi049 ceph-mon[112142]: pgmap v1250: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:38:36.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:35 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:38:38.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:37 smithi144 ceph-mon[86635]: pgmap v1251: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:38:38.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:37 smithi049 ceph-mon[112142]: pgmap v1251: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:38:40.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:39 smithi144 ceph-mon[86635]: pgmap v1252: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-15T16:38:40.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:39 smithi049 ceph-mon[112142]: pgmap v1252: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-15T16:38:42.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:41 smithi144 ceph-mon[86635]: pgmap v1253: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:38:42.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:41 smithi049 ceph-mon[112142]: pgmap v1253: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-05-15T16:38:44.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:43 smithi144 ceph-mon[86635]: pgmap v1254: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:38:44.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:43 smithi049 ceph-mon[112142]: pgmap v1254: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-05-15T16:38:46.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:45 smithi144 ceph-mon[86635]: pgmap v1255: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:46.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:45 smithi049 ceph-mon[112142]: pgmap v1255: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:47 smithi144 ceph-mon[86635]: pgmap v1256: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:48.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:47 smithi049 ceph-mon[112142]: pgmap v1256: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:50.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:49 smithi144 ceph-mon[86635]: pgmap v1257: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:50.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:49 smithi049 ceph-mon[112142]: pgmap v1257: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:38:51.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:50 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:38:51.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:50 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:38:52.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:51 smithi144 ceph-mon[86635]: pgmap v1258: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:52.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:51 smithi049 ceph-mon[112142]: pgmap v1258: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:54.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:53 smithi144 ceph-mon[86635]: pgmap v1259: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:54.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:53 smithi049 ceph-mon[112142]: pgmap v1259: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:55 smithi144 ceph-mon[86635]: pgmap v1260: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:55 smithi049 ceph-mon[112142]: pgmap v1260: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:38:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:57 smithi144 ceph-mon[86635]: pgmap v1261: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:38:58.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:57 smithi049 ceph-mon[112142]: pgmap v1261: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:38:59 smithi144 ceph-mon[86635]: pgmap v1262: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:38:59 smithi049 ceph-mon[112142]: pgmap v1262: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:02.365 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:01 smithi144 ceph-mon[86635]: pgmap v1263: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:01 smithi049 ceph-mon[112142]: pgmap v1263: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:04.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:03 smithi144 ceph-mon[86635]: pgmap v1264: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:03 smithi049 ceph-mon[112142]: pgmap v1264: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:05 smithi144 ceph-mon[86635]: pgmap v1265: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:05 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:39:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:05 smithi049 ceph-mon[112142]: pgmap v1265: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:05 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:39:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:07 smithi144 ceph-mon[86635]: pgmap v1266: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:07 smithi049 ceph-mon[112142]: pgmap v1266: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:10.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:09 smithi144 ceph-mon[86635]: pgmap v1267: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:09 smithi049 ceph-mon[112142]: pgmap v1267: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:12.243 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:11 smithi049 ceph-mon[112142]: pgmap v1268: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:11 smithi144 ceph-mon[86635]: pgmap v1268: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:13 smithi144 ceph-mon[86635]: pgmap v1269: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:14.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:13 smithi049 ceph-mon[112142]: pgmap v1269: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:15 smithi144 ceph-mon[86635]: pgmap v1270: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:15 smithi049 ceph-mon[112142]: pgmap v1270: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:17 smithi144 ceph-mon[86635]: pgmap v1271: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:17 smithi049 ceph-mon[112142]: pgmap v1271: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:19 smithi144 ceph-mon[86635]: pgmap v1272: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:19 smithi049 ceph-mon[112142]: pgmap v1272: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:20 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:39:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:20 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:39:22.359 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:21 smithi049 ceph-mon[112142]: pgmap v1273: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:21 smithi144 ceph-mon[86635]: pgmap v1273: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:23 smithi144 ceph-mon[86635]: pgmap v1274: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:24 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:39:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:24 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:39:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:24 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:39:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:24 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:39:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:23 smithi049 ceph-mon[112142]: pgmap v1274: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:39:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:39:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:39:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:23 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:39:26.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:26 smithi144 ceph-mon[86635]: pgmap v1275: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:26 smithi049 ceph-mon[112142]: pgmap v1275: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:28 smithi144 ceph-mon[86635]: pgmap v1276: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:28 smithi049 ceph-mon[112142]: pgmap v1276: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:30 smithi144 ceph-mon[86635]: pgmap v1277: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:30 smithi049 ceph-mon[112142]: pgmap v1277: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:32 smithi144 ceph-mon[86635]: pgmap v1278: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:32 smithi049 ceph-mon[112142]: pgmap v1278: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:34 smithi144 ceph-mon[86635]: pgmap v1279: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:34 smithi049 ceph-mon[112142]: pgmap v1279: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:36 smithi144 ceph-mon[86635]: pgmap v1280: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:39:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:36 smithi049 ceph-mon[112142]: pgmap v1280: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:39:38.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:38 smithi144 ceph-mon[86635]: pgmap v1281: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:38 smithi049 ceph-mon[112142]: pgmap v1281: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:40 smithi144 ceph-mon[86635]: pgmap v1282: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:40 smithi049 ceph-mon[112142]: pgmap v1282: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:42 smithi144 ceph-mon[86635]: pgmap v1283: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:42 smithi049 ceph-mon[112142]: pgmap v1283: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:44 smithi144 ceph-mon[86635]: pgmap v1284: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:44 smithi049 ceph-mon[112142]: pgmap v1284: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:46 smithi144 ceph-mon[86635]: pgmap v1285: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:46 smithi049 ceph-mon[112142]: pgmap v1285: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:48.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:48 smithi144 ceph-mon[86635]: pgmap v1286: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:48.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:48 smithi049 ceph-mon[112142]: pgmap v1286: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:50 smithi144 ceph-mon[86635]: pgmap v1287: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:50.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:50 smithi049 ceph-mon[112142]: pgmap v1287: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:39:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:39:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:39:52.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:52 smithi144 ceph-mon[86635]: pgmap v1288: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:52 smithi049 ceph-mon[112142]: pgmap v1288: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:54 smithi144 ceph-mon[86635]: pgmap v1289: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:54 smithi049 ceph-mon[112142]: pgmap v1289: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:56 smithi144 ceph-mon[86635]: pgmap v1290: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:56 smithi049 ceph-mon[112142]: pgmap v1290: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:39:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:39:58 smithi144 ceph-mon[86635]: pgmap v1291: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:39:58.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:39:58 smithi049 ceph-mon[112142]: pgmap v1291: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:00 smithi144 ceph-mon[86635]: pgmap v1292: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:00 smithi144 ceph-mon[86635]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T16:40:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:00 smithi144 ceph-mon[86635]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:40:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:00 smithi144 ceph-mon[86635]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:40:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:00 smithi049 ceph-mon[112142]: pgmap v1292: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:00 smithi049 ceph-mon[112142]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-15T16:40:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:00 smithi049 ceph-mon[112142]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-15T16:40:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:00 smithi049 ceph-mon[112142]: fs cephfs has deprecated feature inline_data enabled. 2024-05-15T16:40:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:02 smithi144 ceph-mon[86635]: pgmap v1293: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:02 smithi049 ceph-mon[112142]: pgmap v1293: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:04.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:04 smithi144 ceph-mon[86635]: pgmap v1294: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:04 smithi049 ceph-mon[112142]: pgmap v1294: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:06 smithi144 ceph-mon[86635]: pgmap v1295: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:40:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:06 smithi049 ceph-mon[112142]: pgmap v1295: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:40:08.204 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:08 smithi049 ceph-mon[112142]: pgmap v1296: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:08 smithi144 ceph-mon[86635]: pgmap v1296: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:10 smithi144 ceph-mon[86635]: pgmap v1297: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:10 smithi049 ceph-mon[112142]: pgmap v1297: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:12.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:12 smithi144 ceph-mon[86635]: pgmap v1298: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:12.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:12 smithi049 ceph-mon[112142]: pgmap v1298: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:14.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:14 smithi144 ceph-mon[86635]: pgmap v1299: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:14.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:14 smithi049 ceph-mon[112142]: pgmap v1299: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:16.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:16 smithi144 ceph-mon[86635]: pgmap v1300: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:16.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:16 smithi049 ceph-mon[112142]: pgmap v1300: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:18.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:18 smithi144 ceph-mon[86635]: pgmap v1301: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:18.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:18 smithi049 ceph-mon[112142]: pgmap v1301: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:20.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:20 smithi144 ceph-mon[86635]: pgmap v1302: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:20.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:20 smithi049 ceph-mon[112142]: pgmap v1302: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:21.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:21 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:40:21.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:21 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:40:22.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:22 smithi144 ceph-mon[86635]: pgmap v1303: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:22.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:22 smithi049 ceph-mon[112142]: pgmap v1303: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:24 smithi144 ceph-mon[86635]: pgmap v1304: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:24.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:24 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:40:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:24 smithi049 ceph-mon[112142]: pgmap v1304: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:24.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:24 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T16:40:25.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:25 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:40:25.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:25 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:40:25.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:25 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:40:25.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T16:40:25.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T16:40:25.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:25 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' 2024-05-15T16:40:26.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:26 smithi144 ceph-mon[86635]: pgmap v1305: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:26.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:26 smithi049 ceph-mon[112142]: pgmap v1305: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:28.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:28 smithi144 ceph-mon[86635]: pgmap v1306: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:28.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:28 smithi049 ceph-mon[112142]: pgmap v1306: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:30.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:30 smithi144 ceph-mon[86635]: pgmap v1307: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:30.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:30 smithi049 ceph-mon[112142]: pgmap v1307: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:32.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:32 smithi144 ceph-mon[86635]: pgmap v1308: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:32.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:32 smithi049 ceph-mon[112142]: pgmap v1308: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:34.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:34 smithi144 ceph-mon[86635]: pgmap v1309: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:34.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:34 smithi049 ceph-mon[112142]: pgmap v1309: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:36 smithi144 ceph-mon[86635]: pgmap v1310: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:36.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:36 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:40:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:36 smithi049 ceph-mon[112142]: pgmap v1310: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:36.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:36 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:40:38.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:38 smithi144 ceph-mon[86635]: pgmap v1311: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:38.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:38 smithi049 ceph-mon[112142]: pgmap v1311: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:40.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:40 smithi144 ceph-mon[86635]: pgmap v1312: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:40.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:40 smithi049 ceph-mon[112142]: pgmap v1312: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:42.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:42 smithi144 ceph-mon[86635]: pgmap v1313: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:42.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:42 smithi049 ceph-mon[112142]: pgmap v1313: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:44.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:44 smithi144 ceph-mon[86635]: pgmap v1314: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:44.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:44 smithi049 ceph-mon[112142]: pgmap v1314: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:46.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:46 smithi144 ceph-mon[86635]: pgmap v1315: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:46.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:46 smithi049 ceph-mon[112142]: pgmap v1315: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:48 smithi144 ceph-mon[86635]: pgmap v1316: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:48.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:48 smithi049 ceph-mon[112142]: pgmap v1316: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:50.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:50 smithi144 ceph-mon[86635]: pgmap v1317: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:50.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:50 smithi049 ceph-mon[112142]: pgmap v1317: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:40:51.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:51 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:40:51.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:51 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:40:52.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:52 smithi144 ceph-mon[86635]: pgmap v1318: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:52.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:52 smithi049 ceph-mon[112142]: pgmap v1318: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:54.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:54 smithi144 ceph-mon[86635]: pgmap v1319: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:54.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:54 smithi049 ceph-mon[112142]: pgmap v1319: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:56.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:56 smithi144 ceph-mon[86635]: pgmap v1320: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:56.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:56 smithi049 ceph-mon[112142]: pgmap v1320: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:40:58.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:40:58 smithi144 ceph-mon[86635]: pgmap v1321: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:40:58.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:40:58 smithi049 ceph-mon[112142]: pgmap v1321: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:41:00.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:00 smithi144 ceph-mon[86635]: pgmap v1322: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:41:00.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:00 smithi049 ceph-mon[112142]: pgmap v1322: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:41:02.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:02 smithi144 ceph-mon[86635]: pgmap v1323: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:41:02.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:02 smithi049 ceph-mon[112142]: pgmap v1323: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:41:04.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:04 smithi144 ceph-mon[86635]: pgmap v1324: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:41:04.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:04 smithi049 ceph-mon[112142]: pgmap v1324: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:41:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:06 smithi144 ceph-mon[86635]: pgmap v1325: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:41:06.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:06 smithi144 ceph-mon[86635]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:41:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:06 smithi049 ceph-mon[112142]: pgmap v1325: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:41:06.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:06 smithi049 ceph-mon[112142]: from='mgr.34104 172.21.15.49:0/2490268460' entity='mgr.smithi049.vuhzgf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-15T16:41:08.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:08 smithi144 ceph-mon[86635]: pgmap v1326: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:41:08.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:08 smithi049 ceph-mon[112142]: pgmap v1326: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-05-15T16:41:10.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:10 smithi144 ceph-mon[86635]: pgmap v1327: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:41:10.454 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:10 smithi049 ceph-mon[112142]: pgmap v1327: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-15T16:41:12.439 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-05-15T16:41:12.440 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-15T16:41:12.442 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-15T16:41:12.452 INFO:tasks.cephadm:Teardown begin 2024-05-15T16:41:12.452 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-15T16:41:12.453 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T16:41:12.455 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:12 smithi049 ceph-mon[112142]: pgmap v1328: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:41:12.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:12 smithi144 ceph-mon[86635]: pgmap v1328: 65 pgs: 65 active+clean; 254 MiB data, 974 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-05-15T16:41:12.488 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T16:41:12.521 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-15T16:41:12.522 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-15T16:41:12.546 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-15T16:41:12.580 INFO:tasks.cephadm:Stopping all daemons... 2024-05-15T16:41:12.580 INFO:tasks.cephadm.mon.smithi049:Stopping mon.smithi049... 2024-05-15T16:41:12.580 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049 2024-05-15T16:41:12.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:12 smithi049 systemd[1]: Stopping Ceph mon.smithi049 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:41:13.364 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:13 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:41:13.037+0000 7f70001fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi049 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:41:13.364 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:13 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049[112138]: 2024-05-15T16:41:13.037+0000 7f70001fa700 -1 mon.smithi049@0(leader) e3 *** Got Signal Terminated *** 2024-05-15T16:41:13.365 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:13 smithi049 podman[168430]: 2024-05-15 16:41:13.088744058 +0000 UTC m=+0.296963883 container died 05295c0417ca4abb31f8485a422428c93418f1a88cd789f5d49e2dc457bfcca8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:41:13.633 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:13 smithi049 podman[168430]: 2024-05-15 16:41:13.512773615 +0000 UTC m=+0.720993433 container cleanup 05295c0417ca4abb31f8485a422428c93418f1a88cd789f5d49e2dc457bfcca8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:41:13.633 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:13 smithi049 bash[168430]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049 2024-05-15T16:41:13.954 INFO:journalctl@ceph.mon.smithi049.smithi049.stdout:May 15 16:41:13 smithi049 podman[168456]: 2024-05-15 16:41:13.731189385 +0000 UTC m=+0.644930434 container remove 05295c0417ca4abb31f8485a422428c93418f1a88cd789f5d49e2dc457bfcca8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi049, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:41:14.002 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi049.service' 2024-05-15T16:41:14.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T16:41:14.048 INFO:tasks.cephadm.mon.smithi049:Stopped mon.smithi049 2024-05-15T16:41:14.048 INFO:tasks.cephadm.mon.smithi144:Stopping mon.smithi144... 2024-05-15T16:41:14.048 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi144 2024-05-15T16:41:14.355 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:14 smithi144 systemd[1]: Stopping Ceph mon.smithi144 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:41:14.610 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:14 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144[86631]: 2024-05-15T16:41:14.354+0000 7f1a189e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi144 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:41:14.610 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:14 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144[86631]: 2024-05-15T16:41:14.355+0000 7f1a189e7700 -1 mon.smithi144@1(peon) e3 *** Got Signal Terminated *** 2024-05-15T16:41:14.922 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:14 smithi144 podman[123360]: 2024-05-15 16:41:14.609876371 +0000 UTC m=+0.355086348 container died 2e8f6c238d313d2e56bceedea00eb0131c9d5d579ece4b290b81cd9d31322de2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:41:15.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:15 smithi144 podman[123360]: 2024-05-15 16:41:15.104458893 +0000 UTC m=+0.849668864 container cleanup 2e8f6c238d313d2e56bceedea00eb0131c9d5d579ece4b290b81cd9d31322de2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-65480-squid-de5210d, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-15T16:41:15.198 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:May 15 16:41:15 smithi144 bash[123360]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-mon-smithi144 2024-05-15T16:41:15.488 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@mon.smithi144.service' 2024-05-15T16:41:15.536 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T16:41:15.536 INFO:tasks.cephadm.mon.smithi144:Stopped mon.smithi144 2024-05-15T16:41:15.536 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-15T16:41:15.537 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.0 2024-05-15T16:41:15.954 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:15 smithi049 systemd[1]: Stopping Ceph osd.0 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:41:16.455 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:15 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[124215]: 2024-05-15T16:41:15.984+0000 7ff43562c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:41:16.455 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:15 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[124215]: 2024-05-15T16:41:15.984+0000 7ff43562c700 -1 osd.0 87 *** Got signal Terminated *** 2024-05-15T16:41:16.455 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:15 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0[124215]: 2024-05-15T16:41:15.984+0000 7ff43562c700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:41:21.454 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:21 smithi049 podman[168564]: 2024-05-15 16:41:21.339464021 +0000 UTC m=+5.598109880 container died 84a251f158d31b88de5e48bb252056fa43b3ae4664e0b583b81095e1b04aca80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T16:41:22.078 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:21 smithi049 podman[168564]: 2024-05-15 16:41:21.868192046 +0000 UTC m=+6.126837909 container cleanup 84a251f158d31b88de5e48bb252056fa43b3ae4664e0b583b81095e1b04aca80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-65480-squid-de5210d, GIT_BRANCH=HEAD, ceph=True) 2024-05-15T16:41:22.078 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:21 smithi049 bash[168564]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0 2024-05-15T16:41:22.454 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:22 smithi049 podman[168579]: 2024-05-15 16:41:22.077953227 +0000 UTC m=+0.730867961 container remove 84a251f158d31b88de5e48bb252056fa43b3ae4664e0b583b81095e1b04aca80 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-15T16:41:22.782 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:22 smithi049 podman[168670]: 2024-05-15 16:41:22.411914743 +0000 UTC m=+0.017869882 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:41:22.782 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:22 smithi049 podman[168670]: 2024-05-15 16:41:22.520830268 +0000 UTC m=+0.126785413 container create 9fe165c8bdfc788db7e50b3d22e393ea792dcde33ca0767435ee23b392ffea21 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-65480-squid-de5210d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True) 2024-05-15T16:41:22.782 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:22 smithi049 podman[168670]: 2024-05-15 16:41:22.707242382 +0000 UTC m=+0.313197525 container init 9fe165c8bdfc788db7e50b3d22e393ea792dcde33ca0767435ee23b392ffea21 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-65480-squid-de5210d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:41:22.782 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:22 smithi049 podman[168670]: 2024-05-15 16:41:22.710850635 +0000 UTC m=+0.316805773 container start 9fe165c8bdfc788db7e50b3d22e393ea792dcde33ca0767435ee23b392ffea21 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True) 2024-05-15T16:41:22.782 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:22 smithi049 podman[168670]: 2024-05-15 16:41:22.781599673 +0000 UTC m=+0.387554799 container attach 9fe165c8bdfc788db7e50b3d22e393ea792dcde33ca0767435ee23b392ffea21 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:41:23.050 INFO:journalctl@ceph.osd.0.smithi049.stdout:May 15 16:41:23 smithi049 podman[168670]: 2024-05-15 16:41:23.027796737 +0000 UTC m=+0.633751872 container died 9fe165c8bdfc788db7e50b3d22e393ea792dcde33ca0767435ee23b392ffea21 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T16:41:23.610 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.0.service' 2024-05-15T16:41:23.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T16:41:23.651 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-15T16:41:23.652 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-15T16:41:23.652 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.1 2024-05-15T16:41:23.954 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:23 smithi049 systemd[1]: Stopping Ceph osd.1 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:41:24.322 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[129752]: 2024-05-15T16:41:24.013+0000 7fc0ca2f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:41:24.322 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[129752]: 2024-05-15T16:41:24.013+0000 7fc0ca2f7700 -1 osd.1 87 *** Got signal Terminated *** 2024-05-15T16:41:24.322 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:24 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1[129752]: 2024-05-15T16:41:24.013+0000 7fc0ca2f7700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:41:29.846 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:29 smithi049 podman[168786]: 2024-05-15 16:41:29.479936836 +0000 UTC m=+5.573131664 container died defc7d001185ba4cd3f24bc8c049e500a168f329b1d9a0da110c93b18a225f3a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, org.label-schema.schema-version=1.0) 2024-05-15T16:41:29.846 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:29 smithi049 podman[168786]: 2024-05-15 16:41:29.845738007 +0000 UTC m=+5.938932886 container cleanup defc7d001185ba4cd3f24bc8c049e500a168f329b1d9a0da110c93b18a225f3a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:41:30.204 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:29 smithi049 bash[168786]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1 2024-05-15T16:41:30.204 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:30 smithi049 podman[168803]: 2024-05-15 16:41:30.056865048 +0000 UTC m=+0.572508847 container remove defc7d001185ba4cd3f24bc8c049e500a168f329b1d9a0da110c93b18a225f3a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514) 2024-05-15T16:41:30.736 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:30 smithi049 podman[168893]: 2024-05-15 16:41:30.384681263 +0000 UTC m=+0.018351371 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:41:30.736 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:30 smithi049 podman[168893]: 2024-05-15 16:41:30.507801459 +0000 UTC m=+0.141471549 container create 1c9588bfd50da9671b099a1ddc295c9d06f4c30853639004e9beb7e3888a4280 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2) 2024-05-15T16:41:30.736 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:30 smithi049 podman[168893]: 2024-05-15 16:41:30.735827228 +0000 UTC m=+0.369497319 container init 1c9588bfd50da9671b099a1ddc295c9d06f4c30853639004e9beb7e3888a4280 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-15T16:41:31.044 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:30 smithi049 podman[168893]: 2024-05-15 16:41:30.739465831 +0000 UTC m=+0.373135928 container start 1c9588bfd50da9671b099a1ddc295c9d06f4c30853639004e9beb7e3888a4280 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-15T16:41:31.044 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:30 smithi049 podman[168893]: 2024-05-15 16:41:30.802379891 +0000 UTC m=+0.436049996 container attach 1c9588bfd50da9671b099a1ddc295c9d06f4c30853639004e9beb7e3888a4280 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-15T16:41:31.455 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:31 smithi049 podman[168893]: 2024-05-15 16:41:31.043966813 +0000 UTC m=+0.677636909 container died 1c9588bfd50da9671b099a1ddc295c9d06f4c30853639004e9beb7e3888a4280 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-15T16:41:31.807 INFO:journalctl@ceph.osd.1.smithi049.stdout:May 15 16:41:31 smithi049 podman[168919]: 2024-05-15 16:41:31.789859031 +0000 UTC m=+0.739355549 container remove 1c9588bfd50da9671b099a1ddc295c9d06f4c30853639004e9beb7e3888a4280 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:41:31.814 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.1.service' 2024-05-15T16:41:31.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T16:41:31.860 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-15T16:41:31.860 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-15T16:41:31.860 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.2 2024-05-15T16:41:32.161 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:31 smithi049 systemd[1]: Stopping Ceph osd.2 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:41:32.454 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:32 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[135444]: 2024-05-15T16:41:32.159+0000 7f6df5760700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:41:32.455 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:32 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[135444]: 2024-05-15T16:41:32.159+0000 7f6df5760700 -1 osd.2 87 *** Got signal Terminated *** 2024-05-15T16:41:32.455 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:32 smithi049 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2[135444]: 2024-05-15T16:41:32.159+0000 7f6df5760700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:41:37.920 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:37 smithi049 podman[168998]: 2024-05-15 16:41:37.637626832 +0000 UTC m=+5.580025443 container died b2e567da235f543dfb801aaa02fa47b7f7c19d215c8b145314cd94097097e797 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-15T16:41:38.204 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:38 smithi049 podman[168998]: 2024-05-15 16:41:38.009715072 +0000 UTC m=+5.952113684 container cleanup b2e567da235f543dfb801aaa02fa47b7f7c19d215c8b145314cd94097097e797 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-15T16:41:38.205 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:38 smithi049 bash[168998]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2 2024-05-15T16:41:38.604 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:38 smithi049 podman[169014]: 2024-05-15 16:41:38.261124567 +0000 UTC m=+0.620927370 container remove b2e567da235f543dfb801aaa02fa47b7f7c19d215c8b145314cd94097097e797 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:41:38.906 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:38 smithi049 podman[169103]: 2024-05-15 16:41:38.603894483 +0000 UTC m=+0.109186204 container create 50fd6464bfbabb6941c1f5cc6864609faa5e73dc2117c59fbca6aac1995c014f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-15T16:41:38.907 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:38 smithi049 podman[169103]: 2024-05-15 16:41:38.512805553 +0000 UTC m=+0.018097250 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:41:38.907 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:38 smithi049 podman[169103]: 2024-05-15 16:41:38.823105553 +0000 UTC m=+0.328397247 container init 50fd6464bfbabb6941c1f5cc6864609faa5e73dc2117c59fbca6aac1995c014f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:41:38.907 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:38 smithi049 podman[169103]: 2024-05-15 16:41:38.82669787 +0000 UTC m=+0.331989547 container start 50fd6464bfbabb6941c1f5cc6864609faa5e73dc2117c59fbca6aac1995c014f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-15T16:41:38.907 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:38 smithi049 podman[169103]: 2024-05-15 16:41:38.906269251 +0000 UTC m=+0.411560930 container attach 50fd6464bfbabb6941c1f5cc6864609faa5e73dc2117c59fbca6aac1995c014f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-15T16:41:39.205 INFO:journalctl@ceph.osd.2.smithi049.stdout:May 15 16:41:39 smithi049 podman[169103]: 2024-05-15 16:41:39.13416709 +0000 UTC m=+0.639458772 container died 50fd6464bfbabb6941c1f5cc6864609faa5e73dc2117c59fbca6aac1995c014f (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:41:39.741 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.2.service' 2024-05-15T16:41:39.785 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T16:41:39.785 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-15T16:41:39.785 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-15T16:41:39.786 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.3 2024-05-15T16:41:40.116 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:39 smithi144 systemd[1]: Stopping Ceph osd.3 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:41:40.448 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:40 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[96020]: 2024-05-15T16:41:40.114+0000 7f670321a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:41:40.448 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:40 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[96020]: 2024-05-15T16:41:40.114+0000 7f670321a700 -1 osd.3 87 *** Got signal Terminated *** 2024-05-15T16:41:40.448 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:40 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[96020]: 2024-05-15T16:41:40.114+0000 7f670321a700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:41:42.698 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:42 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:42.314+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:43.359 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:43 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[96020]: 2024-05-15T16:41:43.023+0000 7f670000f700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:17.520999+0000 front 2024-05-15T16:41:17.521129+0000 (oldest deadline 2024-05-15T16:41:42.821236+0000) 2024-05-15T16:41:43.698 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:43 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:43.358+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:44.323 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[96020]: 2024-05-15T16:41:44.041+0000 7f670000f700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:17.520999+0000 front 2024-05-15T16:41:17.521129+0000 (oldest deadline 2024-05-15T16:41:42.821236+0000) 2024-05-15T16:41:44.698 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:44.321+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:44.698 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:44 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:44.386+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:45.370 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:45 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3[96020]: 2024-05-15T16:41:45.043+0000 7f670000f700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:17.520999+0000 front 2024-05-15T16:41:17.521129+0000 (oldest deadline 2024-05-15T16:41:42.821236+0000) 2024-05-15T16:41:45.698 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:45 smithi144 podman[123481]: 2024-05-15 16:41:45.516034118 +0000 UTC m=+5.523339378 container died 0a1e2d34dad4623c971846a84231a36cae7184032bc14457ccd267f554673873 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-65480-squid-de5210d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-15T16:41:45.699 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:45 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:45.369+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:45.699 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:45 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:45.423+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:46.158 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:45 smithi144 podman[123481]: 2024-05-15 16:41:45.873683348 +0000 UTC m=+5.880988605 container cleanup 0a1e2d34dad4623c971846a84231a36cae7184032bc14457ccd267f554673873 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3, io.buildah.version=1.33.5, RELEASE=wip-65480-squid-de5210d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-15T16:41:46.158 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:45 smithi144 bash[123481]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3 2024-05-15T16:41:46.412 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:46 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:46.334+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:46.412 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:46 smithi144 podman[123512]: 2024-05-15 16:41:46.158207998 +0000 UTC m=+0.637028688 container remove 0a1e2d34dad4623c971846a84231a36cae7184032bc14457ccd267f554673873 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:41:46.677 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:46 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:46.411+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:46.678 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:46 smithi144 podman[123601]: 2024-05-15 16:41:46.403973112 +0000 UTC m=+0.017190350 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:41:46.678 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:46 smithi144 podman[123601]: 2024-05-15 16:41:46.559444046 +0000 UTC m=+0.172661262 container create 649770102cc7805970a99f4560629feec2e026e84df45ca48a0e472f81e85113 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T16:41:46.948 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:46 smithi144 podman[123601]: 2024-05-15 16:41:46.762502631 +0000 UTC m=+0.375719860 container init 649770102cc7805970a99f4560629feec2e026e84df45ca48a0e472f81e85113 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:41:46.948 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:46 smithi144 podman[123601]: 2024-05-15 16:41:46.766019595 +0000 UTC m=+0.379236820 container start 649770102cc7805970a99f4560629feec2e026e84df45ca48a0e472f81e85113 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-65480-squid-de5210d, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-15T16:41:46.948 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:46 smithi144 podman[123601]: 2024-05-15 16:41:46.845329415 +0000 UTC m=+0.458546640 container attach 649770102cc7805970a99f4560629feec2e026e84df45ca48a0e472f81e85113 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-15T16:41:47.335 INFO:journalctl@ceph.osd.3.smithi144.stdout:May 15 16:41:47 smithi144 podman[123601]: 2024-05-15 16:41:47.069325569 +0000 UTC m=+0.682542788 container died 649770102cc7805970a99f4560629feec2e026e84df45ca48a0e472f81e85113 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-3-deactivate, RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-15T16:41:47.666 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:47 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:47.334+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:47.666 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:47 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:47.445+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:47.680 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.3.service' 2024-05-15T16:41:47.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T16:41:47.720 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-15T16:41:47.720 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-15T16:41:47.720 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.4 2024-05-15T16:41:47.948 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:47 smithi144 systemd[1]: Stopping Ceph osd.4 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:41:48.368 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:48 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:48.044+0000 7f5f3038f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:41:48.368 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:48 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:48.044+0000 7f5f3038f700 -1 osd.4 87 *** Got signal Terminated *** 2024-05-15T16:41:48.368 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:48 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:48.044+0000 7f5f3038f700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:41:48.698 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:48 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:48.366+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:48.698 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:48 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:48.440+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:49.448 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:49 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:49.381+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:49.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:49 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:49.457+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:50.698 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:50 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:50.393+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:50.698 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:50 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:50.488+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:51.698 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:51 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:51.415+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:51.698 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:51 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:51.535+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:51.699 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:51 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:51.535+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:41:52.698 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:52 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:52.437+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:19.706380+0000 front 2024-05-15T16:41:19.706362+0000 (oldest deadline 2024-05-15T16:41:41.406182+0000) 2024-05-15T16:41:52.698 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:52 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4[101046]: 2024-05-15T16:41:52.437+0000 7f5f2d184700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.407164+0000 front 2024-05-15T16:41:28.407255+0000 (oldest deadline 2024-05-15T16:41:51.907119+0000) 2024-05-15T16:41:52.699 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:52 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:52.541+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:52.699 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:52 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:52.541+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:41:53.879 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:53 smithi144 podman[123707]: 2024-05-15 16:41:53.587528125 +0000 UTC m=+5.644898138 container died d7e6dd64b07e0add4553557c489b4b3bad2c0aa4927131b5d70cfa7d86c0c26a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-65480-squid-de5210d, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-15T16:41:53.879 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:53 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:53.582+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:53.879 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:53 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:53.582+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:41:54.198 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:53 smithi144 podman[123707]: 2024-05-15 16:41:53.960970188 +0000 UTC m=+6.018340196 container cleanup d7e6dd64b07e0add4553557c489b4b3bad2c0aa4927131b5d70cfa7d86c0c26a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-05-15T16:41:54.199 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:53 smithi144 bash[123707]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4 2024-05-15T16:41:54.566 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:54 smithi144 podman[123724]: 2024-05-15 16:41:54.237096233 +0000 UTC m=+0.641951231 container remove d7e6dd64b07e0add4553557c489b4b3bad2c0aa4927131b5d70cfa7d86c0c26a (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:41:54.825 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:54 smithi144 podman[123809]: 2024-05-15 16:41:54.491223638 +0000 UTC m=+0.017293611 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:41:54.825 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:54 smithi144 podman[123809]: 2024-05-15 16:41:54.604553573 +0000 UTC m=+0.130640680 container create 6547dcf8e6b04d3ad04b08da8624edf6b86590f9ee4efd20f826ce27dc1d3287 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-15T16:41:54.825 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:54 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:54.566+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:54.825 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:54 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:54.566+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:41:55.122 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:54 smithi144 podman[123809]: 2024-05-15 16:41:54.823726794 +0000 UTC m=+0.349796754 container init 6547dcf8e6b04d3ad04b08da8624edf6b86590f9ee4efd20f826ce27dc1d3287 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-15T16:41:55.123 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:54 smithi144 podman[123809]: 2024-05-15 16:41:54.828414225 +0000 UTC m=+0.354484189 container start 6547dcf8e6b04d3ad04b08da8624edf6b86590f9ee4efd20f826ce27dc1d3287 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-15T16:41:55.123 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:54 smithi144 podman[123809]: 2024-05-15 16:41:54.923828419 +0000 UTC m=+0.449898391 container attach 6547dcf8e6b04d3ad04b08da8624edf6b86590f9ee4efd20f826ce27dc1d3287 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-15T16:41:55.419 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 15 16:41:55 smithi144 podman[123809]: 2024-05-15 16:41:55.122664563 +0000 UTC m=+0.648734526 container died 6547dcf8e6b04d3ad04b08da8624edf6b86590f9ee4efd20f826ce27dc1d3287 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T16:41:55.686 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:55 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:55.611+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:55.686 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:55 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:55.611+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:41:55.706 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.4.service' 2024-05-15T16:41:55.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T16:41:55.751 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-15T16:41:55.751 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-15T16:41:55.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.5 2024-05-15T16:41:55.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:55 smithi144 systemd[1]: Stopping Ceph osd.5 for c619ac44-12d1-11ef-bc98-c7b262605968... 2024-05-15T16:41:56.448 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:56 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:56.071+0000 7f35fb1ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T16:41:56.448 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:56 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:56.071+0000 7f35fb1ad700 -1 osd.5 87 *** Got signal Terminated *** 2024-05-15T16:41:56.448 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:56 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:56.071+0000 7f35fb1ad700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T16:41:56.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:56 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:56.584+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:56.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:56 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:56.584+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:41:57.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:57 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:57.612+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:57.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:57 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:57.612+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:41:58.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:58 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:58.587+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:58.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:58 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:58.587+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:41:59.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:59 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:59.570+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:41:59.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:41:59 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:41:59.570+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:42:00.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:00 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:42:00.540+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-05-15T16:41:18.869601+0000 front 2024-05-15T16:41:18.869647+0000 (oldest deadline 2024-05-15T16:41:44.169396+0000) 2024-05-15T16:42:00.948 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:00 smithi144 ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5[106205]: 2024-05-15T16:42:00.540+0000 7f35f7fa2700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.49:6814 osd.1 since back 2024-05-15T16:41:28.770953+0000 front 2024-05-15T16:41:28.770881+0000 (oldest deadline 2024-05-15T16:41:51.070802+0000) 2024-05-15T16:42:01.906 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:01 smithi144 podman[123917]: 2024-05-15 16:42:01.618010017 +0000 UTC m=+5.646023205 container died eec3c5d0c5eed40fa380b745da6feb5f46345be0550bae94610a8fb7ab175388 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-15T16:42:02.198 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:01 smithi144 podman[123917]: 2024-05-15 16:42:01.979871348 +0000 UTC m=+6.007884535 container cleanup eec3c5d0c5eed40fa380b745da6feb5f46345be0550bae94610a8fb7ab175388 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-65480-squid-de5210d) 2024-05-15T16:42:02.198 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:01 smithi144 bash[123917]: ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5 2024-05-15T16:42:02.599 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:02 smithi144 podman[123934]: 2024-05-15 16:42:02.239520152 +0000 UTC m=+0.622242937 container remove eec3c5d0c5eed40fa380b745da6feb5f46345be0550bae94610a8fb7ab175388 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-65480-squid-de5210d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-15T16:42:02.901 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:02 smithi144 podman[124023]: 2024-05-15 16:42:02.598947239 +0000 UTC m=+0.114366741 container create 2b19641fca4aaef804c6f79f4800f044b76864ad0919ffc86bb70a73e1b7e027 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-15T16:42:02.901 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:02 smithi144 podman[124023]: 2024-05-15 16:42:02.501859964 +0000 UTC m=+0.017279463 image pull 48bc18eb22d8284fb8ad68bf46e17b174a8aaf15dd2581bf7f093c432d12db31 quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a 2024-05-15T16:42:02.902 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:02 smithi144 podman[124023]: 2024-05-15 16:42:02.826539847 +0000 UTC m=+0.341959337 container init 2b19641fca4aaef804c6f79f4800f044b76864ad0919ffc86bb70a73e1b7e027 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:42:02.902 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:02 smithi144 podman[124023]: 2024-05-15 16:42:02.830377594 +0000 UTC m=+0.345797076 container start 2b19641fca4aaef804c6f79f4800f044b76864ad0919ffc86bb70a73e1b7e027 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-65480-squid-de5210d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-15T16:42:02.902 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:02 smithi144 podman[124023]: 2024-05-15 16:42:02.901280613 +0000 UTC m=+0.416700086 container attach 2b19641fca4aaef804c6f79f4800f044b76864ad0919ffc86bb70a73e1b7e027 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-65480-squid-de5210d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-05-15T16:42:03.198 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:03 smithi144 podman[124023]: 2024-05-15 16:42:03.12625117 +0000 UTC m=+0.641670656 container died 2b19641fca4aaef804c6f79f4800f044b76864ad0919ffc86bb70a73e1b7e027 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-65480-squid-de5210d, CEPH_POINT_RELEASE=, ceph=True) 2024-05-15T16:42:03.677 INFO:journalctl@ceph.osd.5.smithi144.stdout:May 15 16:42:03 smithi144 podman[124050]: 2024-05-15 16:42:03.662866771 +0000 UTC m=+0.530959988 container remove 2b19641fca4aaef804c6f79f4800f044b76864ad0919ffc86bb70a73e1b7e027 (image=quay.ceph.io/ceph-ci/ceph@sha256:7cad1765acb37925dfdfc4273a170d55bf442c11104759fb310474235081d21a, name=ceph-c619ac44-12d1-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-65480-squid-de5210d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T16:42:03.682 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c619ac44-12d1-11ef-bc98-c7b262605968@osd.5.service' 2024-05-15T16:42:03.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T16:42:03.720 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-15T16:42:03.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c619ac44-12d1-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-15T16:42:03.885 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T16:42:08.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[76016]: fuse finished with error 0 and tester_r 0 2024-05-15T16:42:26.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c619ac44-12d1-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-15T16:42:26.579 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T16:42:39.229 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T16:42:39.259 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T16:42:39.286 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-15T16:42:39.288 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175/remote/smithi049/crash 2024-05-15T16:42:39.289 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash -- . 2024-05-15T16:42:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:tar: /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-15T16:42:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:tar: Error is not recoverable: exiting now 2024-05-15T16:42:39.326 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175/remote/smithi144/crash 2024-05-15T16:42:39.327 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash -- . 2024-05-15T16:42:39.351 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-15T16:42:39.351 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-05-15T16:42:39.352 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-15T16:42:39.353 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-05-15T16:42:39.400 INFO:teuthology.orchestra.run.smithi049.stdout:2024-05-15T16:00:00.000165+0000 mon.smithi049 (mon.0) 298 : cluster 3 [WRN] PG_DEGRADED: Degraded data redundancy: 61/321 objects degraded (19.003%), 20 pgs degraded 2024-05-15T16:42:39.400 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-15T16:42:39.401 INFO:tasks.cephadm:Compressing logs... 2024-05-15T16:42:39.401 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-15T16:42:39.443 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-15T16:42:39.465 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-15T16:42:39.465 INFO:teuthology.orchestra.run.smithi049.stderr:: No such file or directory 2024-05-15T16:42:39.465 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-15T16:42:39.467 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.log 2024-05-15T16:42:39.467 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-15T16:42:39.467 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-15T16:42:39.467 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mgr.smithi049.vuhzgf.log 2024-05-15T16:42:39.470 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-15T16:42:39.470 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mgr.smithi049.vuhzgf.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi049.abbagx.log 2024-05-15T16:42:39.473 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.1.log: 89.4% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-15T16:42:39.475 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-05-15T16:42:39.475 INFO:teuthology.orchestra.run.smithi144.stderr:: No such file or directory 2024-05-15T16:42:39.475 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.audit.log: /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi144.kziuke.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi144.kziuke.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mgr.smithi144.whcjkr.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.cephadm.log: 85.7% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mon.smithi144.log 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mgr.smithi144.whcjkr.log: 89.4% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-15T16:42:39.476 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi144.log 2024-05-15T16:42:39.477 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mon.smithi144.log: 91.4% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-15T16:42:39.477 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-15T16:42:39.477 INFO:teuthology.orchestra.run.smithi049.stderr: 91.3% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-15T16:42:39.479 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-15T16:42:39.479 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mon.smithi049.log 2024-05-15T16:42:39.481 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi144.log: 94.6% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi144.log.gz 2024-05-15T16:42:39.481 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi144.jgodzs.log 2024-05-15T16:42:39.491 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi049.abbagx.log: /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi049.gocadf.log 2024-05-15T16:42:39.491 INFO:teuthology.orchestra.run.smithi049.stderr: 90.5% -- replaced with /var/log/ceph/cephadm.log.gz/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mon.smithi049.log: 2024-05-15T16:42:39.493 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-15T16:42:39.501 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi049.gocadf.log: gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi049.log 2024-05-15T16:42:39.504 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi144.jgodzs.log: 91.1% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-15T16:42:39.515 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.cephadm.log: 85.8% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-15T16:42:39.518 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-15T16:42:39.524 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi049.log: 94.5% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi049.log.gz 2024-05-15T16:42:39.533 INFO:teuthology.orchestra.run.smithi144.stderr: 89.8% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mgr.smithi144.whcjkr.log.gz 2024-05-15T16:42:39.566 INFO:teuthology.orchestra.run.smithi144.stderr: 94.6% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi144.kziuke.log.gz 2024-05-15T16:42:39.586 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-volume.log: 91.2% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-15T16:42:40.047 INFO:teuthology.orchestra.run.smithi144.stderr: 92.6% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mon.smithi144.log.gz 2024-05-15T16:42:40.144 INFO:teuthology.orchestra.run.smithi049.stderr: 89.6% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mgr.smithi049.vuhzgf.log.gz 2024-05-15T16:42:42.099 INFO:teuthology.orchestra.run.smithi049.stderr: 89.9% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mon.smithi049.log.gz 2024-05-15T16:42:44.103 INFO:teuthology.orchestra.run.smithi049.stderr: 93.7% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-15T16:42:45.392 INFO:teuthology.orchestra.run.smithi144.stderr: 94.7% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi144.jgodzs.log.gz 2024-05-15T16:42:45.418 INFO:teuthology.orchestra.run.smithi144.stderr: 93.7% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-15T16:42:45.609 INFO:teuthology.orchestra.run.smithi144.stderr: 93.8% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-15T16:42:46.553 INFO:teuthology.orchestra.run.smithi144.stderr: 93.8% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-15T16:42:46.822 INFO:teuthology.orchestra.run.smithi049.stderr: 93.7% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-15T16:42:47.473 INFO:teuthology.orchestra.run.smithi049.stderr: 93.8% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-15T16:42:53.554 INFO:teuthology.orchestra.run.smithi144.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-05-15T16:42:53.554 INFO:teuthology.orchestra.run.smithi144.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-05-15T16:42:53.556 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-05-15T16:42:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m14.099s 2024-05-15T16:42:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m38.733s 2024-05-15T16:42:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m1.141s 2024-05-15T16:42:54.624 INFO:teuthology.orchestra.run.smithi049.stderr: 94.8% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi049.abbagx.log.gz 2024-05-15T16:42:55.162 INFO:teuthology.orchestra.run.smithi049.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-15T16:44:12.358 INFO:teuthology.orchestra.run.smithi049.stderr: 92.6% -- replaced with /var/log/ceph/c619ac44-12d1-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi049.gocadf.log.gz 2024-05-15T16:44:12.360 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-15T16:44:12.360 INFO:teuthology.orchestra.run.smithi049.stderr:real 1m32.904s 2024-05-15T16:44:12.360 INFO:teuthology.orchestra.run.smithi049.stderr:user 2m22.096s 2024-05-15T16:44:12.360 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m4.015s 2024-05-15T16:44:12.360 INFO:tasks.cephadm:Archiving logs... 2024-05-15T16:44:12.361 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175/remote/smithi049/log 2024-05-15T16:44:12.362 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-15T16:44:31.527 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175/remote/smithi144/log 2024-05-15T16:44:31.528 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-15T16:44:36.673 INFO:tasks.cephadm:Removing cluster... 2024-05-15T16:44:36.673 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c619ac44-12d1-11ef-bc98-c7b262605968 --force 2024-05-15T16:44:36.836 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T16:44:37.270 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c619ac44-12d1-11ef-bc98-c7b262605968 --force 2024-05-15T16:44:37.425 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: c619ac44-12d1-11ef-bc98-c7b262605968 2024-05-15T16:44:37.719 INFO:tasks.cephadm:Removing cephadm ... 2024-05-15T16:44:37.720 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-15T16:44:37.737 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-15T16:44:37.752 INFO:tasks.cephadm:Teardown complete 2024-05-15T16:44:37.753 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-15T16:44:37.764 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-15T16:44:37.765 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-15T16:44:37.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-15T16:44:37.780 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-15T16:44:37.864 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-15T16:44:37.875 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-15T16:44:37.875 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T16:44:37.877 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T16:44:37.890 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-05-15T16:44:37.892 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-05-15T16:44:37.895 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T16:44:37.895 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-05-15T16:44:37.895 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv01.front.sepia.ceph.com 3 7 377 98 +326us[ +293us] +/- 51ms 2024-05-15T16:44:37.895 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 -2348us[-2348us] +/- 59ms 2024-05-15T16:44:37.895 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +2254us[+2254us] +/- 69ms 2024-05-15T16:44:37.895 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T16:44:37.897 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T16:44:37.897 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-05-15T16:44:37.897 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv01.front.sepia.ceph.com 3 7 377 38 +382us[ +324us] +/- 52ms 2024-05-15T16:44:37.897 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 38 -2305us[-2305us] +/- 59ms 2024-05-15T16:44:37.897 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +2263us[+2263us] +/- 69ms 2024-05-15T16:44:37.897 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T16:44:37.897 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-15T16:44:37.908 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-15T16:44:37.908 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-15T16:44:37.919 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-15T16:44:38.021 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-15T16:44:38.095 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T16:44:38.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T16:44:38.126 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T16:44:38.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T16:44:38.155 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-15T16:44:38.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-15T16:44:38.176 INFO:teuthology.task.internal:Duration was 4125.171297 seconds 2024-05-15T16:44:38.176 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-15T16:44:38.187 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-15T16:44:38.187 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-15T16:44:38.189 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-15T16:44:38.227 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T16:44:38.233 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T16:44:38.492 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-15T16:44:38.492 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-05-15T16:44:38.493 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-15T16:44:38.554 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-05-15T16:44:38.554 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-15T16:44:38.582 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-15T16:44:38.582 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-15T16:44:38.597 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-15T16:44:38.648 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-15T16:44:38.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-15T16:44:38.664 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-15T16:44:39.884 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-15T16:44:39.896 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-15T16:44:39.896 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-15T16:44:39.923 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-15T16:44:39.954 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-15T16:44:39.969 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-15T16:44:39.972 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-15T16:44:39.993 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-05-15T16:44:40.022 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-05-15T16:44:40.107 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-15T16:44:40.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T16:44:40.123 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-15T16:44:40.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T16:44:40.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-15T16:44:40.177 INFO:teuthology.task.internal:Transferring archived files... 2024-05-15T16:44:40.178 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175/remote/smithi049 2024-05-15T16:44:40.179 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-15T16:44:40.242 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175/remote/smithi144 2024-05-15T16:44:40.242 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-15T16:44:40.287 INFO:teuthology.task.internal:Removing archive directory... 2024-05-15T16:44:40.287 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-15T16:44:40.290 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-15T16:44:40.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-15T16:44:40.348 INFO:teuthology.task.internal:Not uploading archives. 2024-05-15T16:44:40.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-15T16:44:40.360 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-15T16:44:40.360 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-15T16:44:40.362 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-15T16:44:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: 262165 4 drwxr-xr-x 3 ubuntu ubuntu 4096 May 15 16:44 /home/ubuntu/cephtest 2024-05-15T16:44:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: 655363 4 d--------- 2 ubuntu ubuntu 4096 May 15 15:51 /home/ubuntu/cephtest/mnt.0 2024-05-15T16:44:40.376 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-05-15T16:44:40.377 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-05-15T16:44:40.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T16:44:40.379 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_de5210d58560a4684fdb5b238ef3e6e0285a1de8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi049 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-05-15T16:44:40.379 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-15T16:44:40.393 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-15T16:44:40.405 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-15T16:44:40.405 INFO:teuthology.nuke:Checking targets against current locks 2024-05-15T16:44:40.430 DEBUG:teuthology.nuke:shortname: smithi049 2024-05-15T16:44:40.431 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-15T16:44:40.451 DEBUG:teuthology.nuke:shortname: smithi144 2024-05-15T16:44:40.452 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-15T16:44:40.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 15:25:28.011305', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD1Jby171EFSq6R2aEagIDF938YcGcCIHBPugkkRAnO3W5v+tFD5wjFlceBGa/wwrAlmbHxq8ovHYAIRFzTKwvE='} 2024-05-15T16:44:40.472 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-05-15_15:01:21-orch:cephadm-wip-65480-squid-distro-default-smithi/7707175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 15:25:28.009512', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGihwTLvMGdJFQVEb+CPSk7GxKcvqtZ1DZVuYytq9C2M8LYigFYPnrK7MeJ+fMNeSI5htt1nt9b4/PNaQp7EmEQ='} 2024-05-15T16:44:40.500 INFO:teuthology.orchestra.console.smithi049:Power off 2024-05-15T16:44:40.500 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-15T16:44:40.526 DEBUG:teuthology.orchestra.console.smithi049:power off output: Chassis Power Control: Down/Off 2024-05-15T16:44:40.527 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T16:44:40.541 INFO:teuthology.orchestra.console.smithi144:Power off 2024-05-15T16:44:40.542 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-15T16:44:40.552 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-05-15T16:44:40.566 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-05-15T16:44:40.566 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T16:44:40.588 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-05-15T16:44:44.553 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T16:44:44.589 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T16:44:44.666 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-05-15T16:44:44.699 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-05-15T16:44:48.670 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T16:44:48.700 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T16:44:48.781 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is off 2024-05-15T16:44:48.809 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-05-15T16:44:48.883 INFO:teuthology.orchestra.console.smithi049:Power off completed 2024-05-15T16:44:52.810 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T16:44:52.920 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-05-15T16:44:53.021 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-05-15T16:44:53.125 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 4125.171297073364 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_gabrioux@teuthology status: fail success: false 2024-05-15T16:44:53.126 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-15T16:44:53.210 INFO:teuthology.run:FAIL